Epsilon Was Nonzero—Until It Nearly Vanished in the Integer Domain

Series contents · Numerics and Quantization · 阅读中文版

“We add epsilon to prevent division by zero.” Everyone relaxes. Then someone looks at the quantized constant and asks, “What value does it have after quantization?” The room goes quiet.

A small epsilon is protective in a floating-point formula. On a fixed integer grid, small can also mean unrepresentable. Some of normalization’s hardest problems hide in its gentlest-looking constants.

1. Start by Treating Normalization as Groups of M Values

For a group of inputs x, using population variance:

1
2
3

mu = sum(x_i)/M
variance = sum((x_i-mu)^2)/M
y_i = (x_i-mu) / sqrt(variance + epsilon)

InstanceNorm collects spatial statistics separately for each sample and channel. GroupNorm collects them over a group of channels and their spatial elements. When processing each sample separately, or with batch size 1, a channel-first input can be arranged into working shape [G,(C/G)*H,W] and reduced over the last two axes. InstanceNorm is the special case where the group count equals the channel count.

The historical implementation used common processing logic. It checked static shapes, divisibility by the group count, positive finite epsilon, and quantized types, then built mean and variance statistics, epsilon addition, reciprocal square root, centering, and multiplication, followed by optional affine parameters. Its support at that time explicitly restricted some batch and rank conditions. A public description should preserve those boundaries rather than turn a limited integration into universal coverage.

2. Why Each Intermediate Needs Its Own Scale

Inputs, means, centered values, variance, reciprocal standard deviation, and normalized results occupy different numerical ranges.

The centered value x-mu can have a greater magnitude than an individual input relative to its zero point. Variance has squared units. Reciprocal standard deviation grows large near zero variance. The final normalized result may return to a relatively manageable range. Reusing the input scale for all of them is like measuring length, area, and inverse length with the same ruler.

Historical code assigned different quantized types to these intermediates, using wider storage for some. The valuable lesson is not a particular bit width but the responsibility behind type planning: explain each intermediate’s range, where precision is lost, and whether saturation is possible.

A practical review order is to establish the mathematical range, then the storage range, choose a scale, and finally derive computation coefficients. Choosing a scale that “looks sufficient” first often leads to patching the consequences with special cases.

3. Where Does the Variance Bound Come From?

If every input lies in the closed interval [L,U], let D=U-L. Population variance satisfies:

1

variance <= D^2 / 4

Intuitively, pushing values toward the endpoints maximizes spread, and an equal split between the endpoints reaches the bound. Formally, taking expectations of (x-L)(U-x)>=0 bounds variance by (U-mu)(mu-L), whose maximum is D²/4.

The historical implementation used the input storage range and scale to construct this span, then estimated the variance range with that bound. It is conservative and independent of calibration samples. It assumes, however, that real inputs lie within their declared representable range and that the statistic is the corresponding population variance. If the hardware’s statistics mode returns another definition, the compiler and execution unit need a separate contract check.

A conservative range protects against overflow but sacrifices resolution for small variance. Mapping a large range onto finitely many integer steps can flatten distinctions near zero. This is the first tug-of-war between stability and precision.

4. What Changes When Epsilon Must Occupy at Least One Integer Step?

Let the variance scale be s_v and integer epsilon be e:

1
2

e = max(1, round(epsilon / s_v))
epsilon_effective = e * s_v

This prevents epsilon from quantizing to zero, but the computation now uses epsilon_effective. If s_v is much larger than the intended epsilon, the difference may be substantial.

Suppose the desired epsilon is 0.000002 and the variance step is 0.00008. Nearest rounding yields 0. Forcing a minimum integer value of 1 instead uses 0.00008. Protection remains, but scaling in the small-variance region changes.

This is not a story in which “add one” finishes the repair. Effective epsilon belongs in error analysis and test reports, especially for almost-constant inputs. Strict agreement with a framework may require a wider variance type, another scale, a piecewise representation, or higher-precision statistics.

The historical implementation did account for both variance range and epsilon representability, deriving a reciprocal-standard-deviation bound from effective epsilon. The supported claim is that the representability problem was handled explicitly. Meeting a particular model’s error budget still needs numerical experiments.

5. Bounds for Reciprocal Standard Deviation and the Final Result

Because variance is nonnegative:

1
2

1/sqrt(variance + epsilon_effective)
    <= 1/sqrt(epsilon_effective)

This helps choose the reciprocal-square-root output scale. If an approximate implementation can produce negative variance or unexpected rounding, that must be explained earlier; the formula does not automatically guarantee nonnegative integer behavior.

For M values with population variance, in the exact nondegenerate case, each normalized element has absolute value at most sqrt(M-1). To derive it, centered values sum to zero. If one value is a, the other M-1 values sum to -a, and their squared sum is at least a²/(M-1). Population variance is therefore at least a²/(M-1), giving the bound. Positive epsilon only increases the denominator.

For M=1, exact centering gives zero. Handle it separately or choose a conservative range; mechanically assigning a zero scale is invalid. Historical code also used conservative treatment for small reduction lengths.

These are real-arithmetic bounds. A quantized mean may destroy the exact zero sum of centered values, and an approximate reciprocal square root adds error. Mathematical bounds are a starting point for type planning; integer implementations still need room for analysis or validation.

6. Optional Affine Parameters Do Not Make the Final Type Optional

Normalization often ends with gamma*y+beta. All four combinations need handling: both parameters, gamma only, beta only, and neither.

With neither, the internal normalized type may differ from the original output type, so requantization is still required. With gamma alone, multiplication can directly produce the target type. If beta follows, a wider intermediate may avoid premature truncation. Historical tests covered these structural branches separately and checked that requantization remained when affine parameters were absent.

Parameter reads also need length and finiteness checks. Expanding group parameters into channel parameters requires a precise definition of which channels receive each group’s parameter. Reinterpreting raw bytes as floating point is entirely different from reading them according to their actual storage format. Historical canonicalization changes corrected constant reads and parameter-length checks—details sufficient to invalidate a mathematically equivalent rewrite in practice.

7. Aim Regression Tests at Inputs That Barely Change

Mean and variance may read the input repeatedly, while centering and multiplication may produce full-size intermediates. Reciprocal square root generally operates only on per-group statistics. Choose optimization priorities by data volume, not by which function looks most intimidating in the formula. Fusing statistics, reusing reads, and tiling are directions to evaluate.

Normalization’s most troublesome numbers are not always the large ones. Sometimes they are the ones thought too small to cause trouble. Making epsilon’s actual representation and every intermediate’s units and range visible turns debugging back into checkable arithmetic.

8. The Same Variance Can Be Computed by Different Routes

Two common variance formulas are the mean of squared deviations from the mean, and the mean of squares minus the square of the mean. They agree over exact reals. With finite precision, the latter subtracts two nearby large values when inputs have a large common offset and small variation, potentially losing significant precision. The former needs centered data and additional accesses, with potentially different resource costs.

The historical implementation expressed variance through a reduction with a statistics kind. The high-level node name alone does not reveal which algorithm the execution unit uses. A public discussion can identify the contract to verify, but should not invent a particular stable hardware algorithm. IR that expresses variance and an implementation that calculates variance are different layers.

A discriminating teaching test adds one common offset to every input. Ideal normalization, setting aside affine processing and numerical epsilon errors, has the corresponding translation invariance: centered values and variance stay unchanged. Quantized input ranges may change, however, so design the test within one representable range and state whether the scale also changes. Otherwise a failure may come from requantizing the input, not from the variance algorithm.

Now multiply every deviation by a positive factor. If epsilon is negligible relative to variance, normalization should be approximately invariant. If epsilon matters, that scaling invariance is not exact even mathematically. The comparison can expose an excessive effective epsilon: small-variance inputs are suppressed strongly, while larger-variance inputs are affected less. Reference computation must still distinguish the expected effect from an implementation error.

Performance analysis should separate statistics tensors from feature tensors. Means, variances, and reciprocal standard deviations scale with group count; centered values and final outputs scale with total element count. Making reciprocal square root twice as fast need not substantially improve time dominated by full-size intermediate traffic. With many very short groups, statistical and function-approximation overhead may instead become substantial.

The first normalization-optimization diagram should therefore show data sizes and lifetimes, followed by operator dependencies. Locate large tensors that are repeatedly read and written before deciding whether fusion, buffer reuse, or more precise scales are worthwhile.

Back to series contents · Previous · Next

• ← Previous Post
• Next Post →