Bloom in linear light, and what sRGB averaging costs

Blur an image in a paint program, or downsample a photograph, or add a glow to a bright light, and the result is usually slightly too dark, with grey fringes where there should be bright ones. Nothing in the code is wrong. The arithmetic is being done on numbers that are not the quantity being averaged.

Every number in this article is recomputed from its inputs on each build of this site. The code is the post chain in the demo engine, which renders every frame in linear light and converts once, at the end.

The 128 that should be 188Permalink to “The 128 that should be 188”#

An 8-bit image does not store light. It stores an sRGB-encoded value, roughly a 1/2.2 power of the light, because human vision is more sensitive to differences in dark tones and the encoding spends its 256 codes where they are noticed. The transfer function is

$$\text{linear} = \begin{cases} c/12.92 & c \le 0.04045\\ \left(\dfrac{c + 0.055}{1.055}\right)^{2.4} & \text{otherwise} \end{cases}$$

Take a black pixel and a white pixel — codes 0 and 255 — and average them the obvious way. The answer is code 128, which decodes to 0.214041 of full brightness. The right answer is that half of no light and half of full light is half the light, which is 0.5 linear, and 0.5 linear encodes to code 188 in an 8-bit image.

The naive average is 2.336 times too dark: 1.224 stops, which in photographic terms is more than a full stop of exposure thrown away by an arithmetic mean. It is not a rounding error and it does not average out over an image; it is a systematic bias towards black that appears everywhere two pixels are combined.

Everything that combines pixels does this: a Gaussian blur, a bilinear texture fetch, a mipmap built by box filtering, an alpha composite, an image resize, a motion blur accumulated over subframes, and a bloom pyramid. In a renderer with a post chain the same error is paid a dozen times over.

The framebuffer is unbounded floatsPermalink to “The framebuffer is unbounded floats”#

The engine stores three 32-bit floats per pixel in linear light, unclamped:

f->px = (f32 *)calloc((size_t)w * h * 3, sizeof(f32));

At 1920×1080 that is 23.73 MiB per buffer, against 5.93 MiB for 8-bit RGB. The cost buys two properties that are hard to get any other way.

Values above 1.0 are allowed. A light source at 40 is representable, and that is what makes a bloom look like light rather than like a blurred white patch: the bright pass has something to isolate, and the tone mapper has something to roll off. Clamping at the point of shading — which is what an 8-bit intermediate buffer does — discards exactly the information the rest of the chain needs.

Nothing is encoded until the end. Tone mapping and sRGB encoding happen once, in dm_fb_resolve, after every blur, downsample and composite in the frame. Nothing in the chain has to know about a transfer function, because nothing in the chain sees one.

Extracting what should glowPermalink to “Extracting what should glow”#

Bloom is the light that scatters inside a lens. The first step isolates the part of the image bright enough to scatter visibly, and the naive way — everything above a threshold, nothing below — makes a pixel snap into the bloom the instant it crosses the line, which flickers on a moving image. The engine uses a quadratic soft knee instead:

static f32 knee_weight(f32 x, f32 threshold, f32 knee)
{
    f32 soft = x - threshold + knee;
    soft = dm_clamp(soft, 0.0f, 2.0f * knee);
    soft = soft * soft / (4.0f * knee + 1e-6f);
    f32 contribution = DM_MAX(soft, x - threshold);
    return contribution / DM_MAX(x, 1e-6f);
}

With the shipped threshold of 1.0 and knee of 0.6, the fraction of a pixel that reaches the bloom is:

Below $threshold - knee$ the quadratic is zero and nothing passes; above $threshold + knee$ the linear branch takes over and the knee is no longer in play. The two branches are equal exactly at $x = threshold + knee$, which is what makes the curve continuous rather than merely smooth-looking, and it is worth checking rather than assuming — the site’s audit does.

The extraction also box-filters 2×2 while it reads, so the bright pass halves the resolution and antialiases in the same step. That box filter is only legitimate because the data is linear; the same four-pixel average on encoded data is the 128-instead-of-188 error, applied to the brightest pixels in the frame, where it is most visible.

The pyramidPermalink to “The pyramid”#

The chain is six levels, each half the size of the one above, starting at half resolution:

Every level is blurred with the same kernel, at $\sigma = 1.6$ in its own pixels. Because each level’s pixels are twice as wide as its parent’s, a fixed kernel covers twice the screen distance at every step, and the blurs compound: variances add, so by level 5 the accumulated standard deviation is 118.2 full-resolution pixels and the bloom reaches 354.7 pixels, which is 18.5 per cent of the width of the frame. A single blur that wide would need a kernel of 711 taps.

The pyramid also costs almost nothing to hold. The six levels together are 690,900 pixels, or 0.3332 of one full-resolution buffer — the geometric series $\frac14 + \frac{1}{16} + \dots$ converging to a third — and with a scratch buffer per level for the separable pass, 15.81 MiB next to the 23.73 MiB of the framebuffer itself.

The kernel, and why it is separablePermalink to “The kernel, and why it is separable”#

static int build_gaussian(f32 *kernel, f32 sigma)
{
    int radius = (int)(sigma * 3.0f + 0.5f);
    f32 sum = 0.0f;
    for (int i = -radius; i <= radius; i++) {
        f32 v = expf(-(f32)(i * i) / (2.0f * sigma * sigma));
        kernel[i + radius] = v;
        sum += v;
    }
    for (int i = 0; i < 2 * radius + 1; i++) kernel[i] /= sum;
    return radius;
}

At $\sigma = 1.6$ the radius is 5 and the kernel is 11 taps, with weights 0.2495 at the centre falling to 0.0019 at the edge. Truncating a Gaussian at three standard deviations discards 0.059 per cent of its mass, which is then renormalised away by the division — the reason the kernel is normalised after truncation rather than analytically.

Running it as two one-dimensional passes rather than one two-dimensional pass is the difference between 22 taps per pixel and 121, a factor of 5.5, and it is exact rather than an approximation: a 2D Gaussian is the outer product of two 1D Gaussians, so the separable form computes the same convolution. Across all six levels and both passes the blur costs 45.6 million multiply-adds per frame per colour channel, which is why the levels are small and the kernel is short.

Tone mapping is the last thing that happensPermalink to “Tone mapping is the last thing that happens”#

static inline v3 dm_tonemap_aces(v3 x)
{
    const f32 a = 2.51f, b = 0.03f, c = 2.43f, d = 0.59f, e = 0.14f;
    return v3_sat(v3_div(v3_mul(x, v3_adds(v3_scl(x, a), b)),
                         v3_add(v3_mul(x, v3_adds(v3_scl(x, c), d)), V3s(e))));
}

This is Narkowicz’s rational fit to the ACES filmic curve. What it does to the values that reach it:

Two properties are worth knowing before using it. The curve’s asymptote is 1.0329, above 1, so it does not approach white — it crosses it, at an input of 7.2417, and everything brighter is clipped by the saturation that follows. And middle grey, 0.18 linear, comes out at 0.2669 rather than at 0.18: the curve lifts mid-grey rather than preserving it. Neither property is a defect, but both surprise people who expect a tone curve to be normalised, and both are reasons to set exposure with the curve in mind rather than to tune it until the image “looks right” and then wonder why it moves when the tone mapper is changed.

Dither, and the last 8 bitsPermalink to “Dither, and the last 8 bits”#

The final conversion quantises to 256 codes, and the gradients of a dark scene do not survive that cleanly: in linear light the first code above black is 0.000304, the second is about twice that, and a smooth ramp across a dark sky crosses those boundaries in wide flat bands. Adding a triangular-distribution random value of about one code before rounding trades the banding for noise the eye does not resolve:

f32 d = dm_dither_tpdf(x, y, frame);
o[0] = (u8)(dm_sat(c.x + d) * 255.0f + 0.5f);

It is two hashes per pixel and it is the single most visible quality difference between a naive renderer and a careful one, which is a strange thing to be able to say about adding noise on purpose.

The order, and the errors it preventsPermalink to “The order, and the errors it prevents”#

The chain is: shade in linear, extract in linear, blur in linear, composite in linear, tone map, encode, dither, quantise. Each of the common bugs is a step done in the wrong place:

• Blurring encoded data. The 128-for-188 error, applied to every neighbourhood in the image. Symptom: dark halos around bright edges, and a bloom that greys out instead of glowing.
• Tone mapping before the bloom. The tone mapper compresses everything above 1 into a narrow range, so the bright pass afterwards has nothing to separate: the bloom becomes a blurred copy of the picture.
• Clamping the framebuffer. A light at 40 clamped to 1 blooms the same as a wall at 1. All the structure that makes bright things look bright is above 1.
• Downsampling with a box filter on encoded data. The same error as the first, and the one most often shipped, because mip generation is usually somebody else’s code.
• Dithering before quantisation is the only place it works. Noise added earlier is blurred away by the very steps that were meant to hide the banding.

None of this is expensive. The whole discipline is one decision — hold the framebuffer in unbounded linear floats — plus the discipline of converting exactly once. The rest of the chain then stops having a colour space at all, which is what makes it possible to reason about what a sphere-traced surface should be lit by without also reasoning about what a display will do with the answer, and what lets the same frame be rendered across eight threads without any part of the chain needing to know which thread produced which row.