The Multiplication Is Right; the Scale Is Wrong: An Integer Kernel’s Two Ledgers

Series contents · Numerics and Quantization · 阅读中文版

“The two integers multiply to exactly what the calculator says, but the model is still wrong.” Staring at the multiplication sign rarely helps here. The calculator has checked only one ledger: the integer product. The model keeps another: how large a real value each integer represents.

Quantized multiplication is especially deceptive because every low-level step can be legal while their composition implements the wrong operation.

1. Put All Three Quantization Domains on Paper

Define the inputs and output:

1
2
3

a = s_a * (q_a - z_a)
b = s_b * (q_b - z_b)
y = s_y * (q_y - z_y)

From y=a*b:

1

q_y ≈ round[(s_a*s_b/s_y)*(q_a-z_a)*(q_b-z_b)] + z_y

Multiplication uses the product of both input scales divided by the output scale. Addition instead converts each input to the output domain before adding. A common parameter rule designed for addition cannot be reused without examining its mathematics.

For a teaching example, choose s_a=0.02, s_b=0.05, s_y=0.01, all zero points zero, and input integers 10 and 6. Their real values are 0.2 and 0.3, whose product is 0.06, corresponding to output integer 6. The integer product is 60 and still needs multiplication by 0.1. Incorrectly using only the first input’s scale ratio, 2, gives a completely different result.

A historical patch added independent parameter generation for multiplication, explicitly computing the product of the input scales divided by the output scale and carrying all three zero points. This aligns the mathematical ledger with the register ledger.

2. An Integer multiplier and shift Only Approximate the Coefficient

An execution unit generally does not store an arbitrary real coefficient directly. It uses something like M / 2^r. Let the true coefficient be α and its approximation α̂. Under simplified assumptions of no saturation and a fixed rounding rule:

1

one component of output error ≈ (α̂-α) * (q_a-z_a) * (q_b-z_b)

The same coefficient error may be nearly invisible for small inputs and amplified by the product at extreme inputs. Random tests using small values can look much better than the implementation deserves.

The coefficient representation raises further questions. Is M signed or unsigned? Can r be negative? Is the product wide enough before a left shift? How are negative values shifted right? Does adding a rounding bias before shifting implement the target’s rule for negative values? There is no universal template independent of the execution unit’s contract.

The historical diff establishes changes to coefficient calculation and parameter placement. It does not establish an error bound for every extreme combination. A defensible public description is “the parameters were brought into correspondence with the formula,” followed by the width and rounding conditions still requiring validation.

3. A Scalar Has a Source, Not Just a Small Shape

A scalar constant in the graph may become an immediate, a parameter-register value, an element in local storage, or an addressable buffer. The instruction must know how to read it.

A historical patch propagated scalar-source controls through the kernel, emission interface, and low-level encoding, and added recognition of scalar shapes on either multiplication input. A Boolean introduced only at the top cannot change behavior if it never reaches the encoder. Conversely, an encoder field does not help if callers keep supplying an inappropriate default.

The symptoms often look unlike quantization errors. Reading a stale register as the scalar can make the result depend on the preceding operator. Reading a one-element buffer as a vector can produce a correct first element followed by bad ones. Zero-initialized memory can hide both.

Express the contract as two separate decisions:

1
2

operand_kind = scalar or vector
scalar_source = immediate / parameter / addressed_memory

Do not collapse these into “is it constant?” A constant can be an entire vector, and a runtime value can be scalar. Source and shape are different properties.

4. Why Test Swapping the Multiplication Inputs?

Over the reals, ab=ba. A low-level interface may nevertheless impose different addressing, broadcasting, or type restrictions on its two inputs. Historical changes explicitly added scalar recognition for the left multiplication input, making “only the right input can be scalar” an assumption worth reviewing separately.

Compare tensor * scalar with scalar * tensor, giving the two inputs different scales and zero points. Identical parameters can hide incorrect parameter binding after a swap.

Subtraction is stricter: scalar - tensor and tensor - scalar are not equivalent in the first place. Reusing a binary-kernel interface does not justify extending multiplication’s swapping policy to every operation. Related historical changes also touched scalar-source propagation for addition and subtraction, but do not establish complete support for all their left/right scalar combinations.

5. Squaring Is a Small Operator That Exposes Bad Reuse

The history includes a power operation restricted to exponent 2, using a multiplication-based approach and rejecting other exponents. This avoids introducing a general power function for x², with an explicit, inspectable scope.

Connecting the same input to both multiplication operands does not eliminate numerical parameters:

1

q_square ≈ round[(s_x^2 / s_out)*(q_x-z_x)^2] + z_out

The scale is squared, and the zero point is subtracted first. Squaring q_x alone introduces unwanted cross and constant terms when z_x is nonzero. Input q_x=z_x represents real zero, so its squared output should represent real zero too—a particularly cheap and effective assertion.

Nor does a later scale-construction fix for ordinary multiplication automatically fix every square path that reuses multiplication. Trace the parameter-generation path actually called. The examined diffs cannot replace that end-to-end verification, so square is listed as a linked regression target, not a proven incidental benefit.

6. Diagnose Incorrect Reads before Incorrect Arithmetic

An efficient diagnostic order is to establish scalar sources and addresses first, integer arithmetic second, quantization scales third, and rounding and saturation last. Otherwise, changing shift may appear to fix one incorrectly read scalar, only for another input to break it again.

Prepare a semantic debugging record: both inputs’ shapes, storage types, scales, zero points, broadcast modes, output range, M, and r. Product users need not see internal addresses, but compiler developers learn more from this record than from a long list of machine words.

Scalar broadcasting can reduce parameter traffic, but numerous short loops may increase configuration and branch costs. Implementing square through multiplication reuses infrastructure; it does not prove that it beats a dedicated square instruction on every target. Useful observations would include effective element throughput, parameter-load counts, materialized broadcast size, and intermediate buffers.

The multiplication sign is simple; its surrounding conventions are not. Record “where the value comes from” separately from “what the value represents,” and apparently mysterious numerical failures become two ledgers that can be checked line by line.

7. Separate Failure Modes with Three Input Groups

The first group tests only real zero: set each input integer to its own zero point. The correct product remains real zero, and the output integer should equal the output zero point. Failure directs attention to zero-point placement and sources before coefficient precision. It avoids guessing from a cloud of random errors whether the offset or multiplier is wrong.

The second group uses small values whose centered integers are one and a quantization ratio that is exactly representable. This primarily tests the scale formula and parameter binding. Deliberately give the inputs different scales. When swapping operands, swap their types and values together; the mathematical result should remain unchanged. If it changes, examine left-scalar support, input-type binding, and register selection. Swapping data pointers while leaving types fixed creates a different, already inequivalent experiment.

The third group reaches extrema and rounding boundaries, testing intermediate-product width, fixed-point coefficient error, and saturation. Keeping the groups separate gives failures clearer explanations than one aggregate mean error. Saturated values deserve their own report: two different errors may clamp to the same endpoint and appear to have become smaller.

Two consecutive calls can expose scalar-source problems. First execute an operation with scalar A, then another of the same shape with a different scalar B; then reverse the order. If the second result depends on the first call’s parameters, a missing load, incorrect reuse, or inconsistent source field is a plausible cause. This is a general diagnostic method, not a runtime failure established by the historical diff.

Review default arguments when extending an instruction interface. If a new control defaults to disabled, should every old call site really disable it? A call site compiling successfully establishes signature compatibility, not correct propagation of the new semantics. Listing the intended sources at each call site for a left scalar, right scalar, and two vectors is a simple migration check.

Finally, inspect error handling. Failure to obtain quantization information should propagate clearly through the call chain. If a helper returns an all-zero parameter object and emission continues, compilation may succeed with inexplicable results. This observation is not a repository-wide verdict; it is a reminder to check whether a failed coefficient calculation can be mistaken for valid parameters.

Back to series contents · Previous · Next

• ← Previous Post
• Next Post →