Abs and Sign: One Line of Arithmetic Does Not Make Their Parameter Contracts Interchangeable

Series contents · Operators and Layouts · 阅读中文版

If asked to name two operators that ought to be easy to implement, many people would choose Abs and Sign. One takes an absolute value; the other determines the sign. So someone copies an existing binary operator, supplies the same value to both inputs, and changes the opcode.

It compiles. It produces results. Then the operator that was supposed to be the simplest consumes an entire stretch of debugging time.

The real question is: “If the binary parameter structure looks the same, does every field mean the same thing in a unary instruction?”

Derive Parameters from Quantization

Start with the teaching relationship for uniform quantization:

1
2

x_real = sx * (qx - zx), sx > 0
y_real = sy * (qy - zy)

For Abs, y_real=|x_real|. Temporarily restricting the output zero point to 0 gives the ideal integer relationship:

1

qy ≈ round((sx/sy) * abs(qx-zx))

At least two parameters are immediately visible: the input zero point and the ratio of input to output scales. Substituting binary quantization coefficients for x+x will not do. The historical change constructed parameters specifically for Abs, converting the scale ratio into a fixed-point multiplier and shift and setting the input zero point. We can accurately say that parameter construction became specific to Abs; we cannot infer that the diff resolved every nonzero-output-zero-point case.

Here is a redesigned example: sx=0.25, zx=5, and sy=0.5. Integer input qx=1 represents real value -1; Abs produces 1, and the output integer should be 2. Taking the absolute value of raw integer 1 and interpreting it afterward gives a different value. The error is neglecting the quantization origin, not computing absolute value incorrectly.

Sign makes reuse of that scaling scheme even more tempting—and misleading. Since sx is positive, the sign depends on qx-zx; the magnitude of the scale does not change positivity or negativity. If the instruction directly outputs raw -1, 0, or 1, it primarily needs the input zero point. An ordinary output scale should not reinterpret those integers as tiny fractions. The historical implementation built Sign parameters separately and adjusted model-output metadata when directly storing a Sign result, so that the integers were interpreted with unit scale and zero offset.

“The output integers are right. Why does the comparison tool say they are wrong?” This often points to the output interpretation layer. Raw integer 1 may mean Sign’s positive result, while metadata tells the host to multiply it by 0.125, naturally producing 0.125. Machine output and host interpretation must share one contract. Fixing computation without fixing its wrapper is insufficient.

The limits also deserve candor. Historical detection targeted a particular direct Store(Sign) output pattern. If another operator follows Sign, or the output takes a different wrapping path, that adjustment alone cannot guarantee quantization semantics for the full graph. A more general design might represent the raw sign-value type directly in IR and explicitly convert it when another quantization encoding is needed.

End-to-End Paths and Numerical Tests

The initial unary support connected a substantial chain: operator-library entry points, kernel loading of addresses and parameters, emitter creation of instruction objects, textual and binary instruction representations, and registration with a shared code-generation entry point. The initial commit also integrated multiplication, although its title focused on unary operators. Changes must be read through their files and behavior; a title is not a complete table of contents.

Why were later parameter fixes still necessary? Because end-to-end compilability proves connectivity, not that every field obeys its numerical protocol. A minimal model helps isolate one operator, but equal input and output scales and a zero zero point can hide errors through symmetry. Validating parameters requires deliberately breaking those coincidences.

A suggested Abs numerical matrix includes negative, zero, and positive values; unequal input and output scales; a nonzero input zero point; values near the integer minimum; and output saturation boundaries. The absolute value of the minimum signed integer may not fit in the same signed width. Explicit saturation, widening, or another target convention must determine the outcome, rather than luck with host-language overflow behavior.

For Sign, center the matrix on the quantization origin: zx-1, zx, and zx+1. Check that both output integers and decoded real values satisfy their respective contracts. Changing the input scale while preserving these integer relationships can show that sign detection does not incorrectly depend on scale. Support for special floating-point values needs separate examination of operator semantics and target implementation; integer tests do not establish it.

Addresses, Composite Models, and Costs

Once the algorithm is right, addresses can still be wrong. Another historical change introduced explicit physical-stride loops for Abs: multichannel blocks advanced according to physical alignment, single-channel cases were handled by row, and a compact path within groups was distinguished from other paths. Abs preserves shape, but that does not mean it can view all memory as an unbroken logical vector. Padding makes “same shape” and “compact contiguous storage” visibly different claims.

This suggests a useful localization order. First use a small shape and simple layout to validate numerical parameters. Then hold values fixed and change channel count and width to cross alignment boundaries, checking addresses. Finally return to the full graph and validate metadata and neighboring operators. Change scales, shapes, and postprocessing together, and it becomes difficult to tell which contract failed.

The history also expanded higher-level model cases for depthwise convolution, normalization, parametric activation, and slicing. Scripts checked whether expected primitives appeared after lowering and whether old high-level operators disappeared. Their purpose was to verify that combinations of basic operators could support more complex expressions. A parametric activation, for example, can involve positive and negative branches, subtraction, multiplication, and addition. Locally correct primitives can still fail in combination because broadcast or quantization boundaries disagree.

These scripts mainly provide structural assertions; they do not automatically establish numerical accuracy or device performance. A suggested extension is reference results at intermediate stages: logical values from each primitive, the final combined value, and a stated source for necessary error thresholds. Thresholds should follow quantization-error analysis or product requirements, rather than a conveniently loose number that turns the test green.

Unary arithmetic is often cheap, so parameter loads, loop branches, and data movement can account for a large share of cost. Placing parameters close enough to reuse an existing base address, or using a short load path within a valid immediate range, are possible engineering techniques. The relevant kernels did distinguish near and far parameter addresses. Their actual benefit depends on generated instructions and the execution system.

Parameter Contract Boundaries

Why not share a single “generic quantization parameter table” across all unary operators? A structure can be shared without every field having the same meaning. Abs needs magnitude scaling; Sign primarily needs comparison against the zero point; other functions may need coordinates for a lookup-table input. A common constructor that fills unrelated fields might work initially, then expose hidden dependencies when an instruction mode changes or a field stops being ignored. Documenting which fields each operator consumes is more reviewable than blindly populating them all.

Fixed-point ratios also raise precision questions. Representing a real ratio with an integer multiplier and right shift is not necessarily exact. Very small or large ratios, and ratios near rounding boundaries, exhibit different errors. Code inspection confirms the use of this conversion, but the existence of the function does not establish an error bound. An engineering exercise could calculate representation error for several teaching ratios, propagate it through the maximum input magnitude, and estimate worst-case output deviation. That estimate must still match the target’s rounding and saturation order.

For Abs, subtracting the input zero point before taking absolute value also requires sufficient intermediate width. With narrow stored inputs, the difference may have a wider range than the original storage, and its absolute value may exceed the positive limit. The device’s behavior must be explicit: widen before computing, saturate to the original width, or follow another definition. A reference implemented with same-width host integers can reproduce the very bug it is meant to catch. Use sufficiently wide integers or explicit saturation in an independent reference.

For internal graph uses of Sign, ask another chain of questions: does the output retain raw sign integers? Does the next operator interpret them with unit scale, or decode them using the old quantized type? If re-encoding is needed, on which edge does it occur? Changing metadata only at the final model output may fix host reading but cannot change an operator that already consumed an incorrectly interpreted value inside the graph. That is why this article stresses the scope of the direct-output special case. A stronger design must express the numerical domain to every consumer.

Parameter loading also needs tests of both near and far address branches. A near address may be read as an offset from an existing base; a far address requires loading a complete new location. If tests always place parameter blocks next to each other, they cannot reveal a far-address branch that overwrites a base and then uses it incorrectly. Small operator tests with varying parameter distances separate address-loading faults from arithmetic faults. The comparison concerns generated structure and address relationships; it requires no disclosure of real device memory layouts.

Finally, combined models can test the stability of primitive interfaces, but small unit tests remain valuable. A failed normalization model might involve reduction, reciprocal square root, broadcasting, or constant encoding. A three-point Sign test around its origin points much more directly to comparison and output interpretation. Large models supply realistic combinations; small models supply clear causality. Using both avoids starting again from the first instruction every time a complex graph fails.

Keep this story in mind whenever a review hears “we copied a similar operator.” Put the real-number formula, quantization formula, instruction fields, and output metadata on the same page first. If any one of them cannot be matched to the others, postpone the celebration about how simple the operator is. Even the shortest mathematical expression deserves its own parameter manual.

Series contents · Previous · Next

• ← Previous Post
• Next Post →