Subtracting Zero Can Disappear; Subtracting from Zero Cannot

Series contents · Operators and Layouts · 阅读中文版

“Can we write subtraction the same way as addition?”

Most interfaces do look alike. Both have two inputs and one output; both need data types, address descriptors, and quantization parameters. “Most” is precisely the dangerous word. Addition makes many operand swaps seem harmless. Subtraction is the colleague with an excellent memory: whoever you put first, it remembers all the way to the final result.

Direction and Scalar Recognition

Even if c is a single-element constant, the teaching expressions x-c and c-x cannot become one directionless ScalarOp. For x=[-3,2,7] and c=4, the results are [-7,-2,3] and [7,2,-3]. Test only x=c, and both produce 0; the bug will pass with impeccable manners.

Initial historical subtraction support added the machine-instruction object, text and binary output, emitter interface, kernel wrapper, operator-library entry point, code-generation registration, and a model example. This chain matters because “supporting an operator” is never just adding arithmetic. A missing connection at any layer may surface as an unavailable implementation, empty output, an incorrect source type, or a link failure.

When the importer began recognizing scalars, another detail emerged: a constant need not have rank 0. Shapes [1], [1,1], or more unit dimensions can all contain one element. A historical change recognized single-element weights by checking that total element count was 1 and flattened them to extract the constant value. The value must also truly be a compile-time constant. A runtime single-element input cannot become a constant attribute merely because its shape is small.

A general recognition rule is:

1
2
3
4
5
6
7
8

is_constant_scalar(v):
    return v is compile_time_constant and element_count(v)==1

exactly one input satisfies the predicate:
    choose dynamic input and scalar value
    preserve original subtraction direction
otherwise:
    keep general binary path or use a dedicated constant fold

“Exactly one side” matters. If both are constants, constant folding can handle them; if neither is constant, retain the general computation. Do not force two constants into a pattern with a dynamic input, and do not turn every single-element tensor into an attribute.

Reverse-direction information must survive the entire compilation chain. If import produces SubConst(x,c,reverse=true), lowering must carry the attribute into backend IR, code generation must read it, and the kernel must choose scalar or vector source modes accordingly. Any intermediate layer that forgets to copy it cancels the correct recognition upstream. The verified historical diffs updated these interfaces together, making this a semantic fix across layers.

Zero, Rewriting, and Quantization Boundaries

Zero is the best interview question for directional awareness:

1
2

x - 0 = x
0 - x = -x

A canonicalizer looking only at whether the constant is zero may turn negation into identity. The historical fix added a non-reverse condition to zero-subtraction elimination. It looks like a Boolean check, but it maintains a prerequisite of an algebraic identity.

Direction is not the only prerequisite. If the operation also includes fused activation, saturation, or special output quantization, replacing x-0 directly with its input requires checking output types and additional semantics. Signed floating-point zero, NaN, and similar cases must follow the numerical rules of the particular IR. It specifically addresses erroneous elimination of reverse subtraction in the inspected path.

Someone may suggest rewriting c-x as c+(-1)*x to avoid retaining subtraction direction. Mathematically valid does not necessarily mean economical. The rewrite can add a multiplication, an intermediate tensor, and a quantization boundary, and may move rounding or saturation. If the backend supports reverse scalar subtraction, expressing it directly is generally clearer. Otherwise, decomposition still needs an explicit numerical argument and cost estimate.

When quantizing the scalar, c cannot simply be inserted into integer parameters as a floating-point bit pattern. Under the teaching relation real=s*(q-z), an integer representation of a constant generally requires scaling, offsetting, rounding, and range handling according to the target contract. The historical path called dedicated scalar-quantization logic and converted failures into diagnostics. Whether every operator expresses its constants in the input’s quantization coordinates still depends on that operator’s parameter protocol; one formula cannot settle them all.

Now consider division. A historical importer change sent “tensor divided by a single-element constant” through an existing multiplication-by-reciprocal path, while retaining a reciprocal-style expression for “constant divided by a tensor.” These paths cannot be swapped symmetrically either. With c=2 and x=[1,4], x/c=[0.5,2], whereas c/x=[2,0.5]. Changing positions changes the function, not merely the sign.

Division by zero, reciprocal overflow for tiny constants, and reciprocals quantizing to zero are boundary questions to add. The inspected diff mainly demonstrates import-path selection; it does not establish that all these issues were verified. Test descriptions should distinguish “this checks IR structure” from “this checks numerical error.” Successfully generating a model file is still several layers short of proving correct execution.

Testing and Performance Questions

A focused teaching suite can place the constant on each side, with values 0, positive, and negative; use scalar shapes and several unit dimensions; and distinguish runtime single-element values from true constants. For subtraction, inputs should be greater than, less than, and equal to the constant. For reciprocal rewrites, include noninteger divisors. After checking results, inspect the IR for the correct direction attribute and expected constant path.

The initial Sub model example supplied material spanning import, quantization, memory allocation, and emission. Later changes show why a simple example cannot replace a boundary matrix. A Sub test with two identically shaped inputs is unlikely to exercise a constant on the left, unit-dimension recognition, or zero elimination. A test’s existence and its coverage of a particular semantic issue are different facts.

Performance analysis should be equally concrete. A scalar attribute can avoid storing and loading an entire broadcast constant, but may add parameter loads. Direct reverse subtraction can avoid an explicit negation intermediate, but actual instruction or memory-access savings require inspecting generated artifacts. For very small tensors, launch and parameter costs can outweigh the arithmetic.

Semantic Checks During Canonicalization

Another distinction is easily lost during normalization: a shape containing one element is not necessarily the same information as an IR declaring that “this result is a scalar.” A single-element constant may still participate in broadcasting as a tensor, with result rank determined by the other operand; a true scalar result may follow different interface conventions. Promoting a constant to an attribute must preserve the original result type and required broadcast relationship. Extracting a number is easy. Proving that it participates in the same place after leaving its tensor container is the rewrite’s responsibility.

A teaching graph can track direction by first giving a general Sub node an attribute that reverses the interpretation of its operand list, then placing the constant in different list positions. If a canonicalizer looks only at list order without first restoring semantic left and right, it may reverse direction twice or miss a reversal. The inspected historical rule first used the original direction attribute to select semantic operands, then decided the direction after constant extraction. That order is easier to review than patching in a negated Boolean at the end.

Constant rounding can also complicate algebraic optimization. A tiny nonzero real constant may quantize to zero in an integer encoding. Whether deleting the operation at the real-number level as “approximately zero” matches quantizing to an integer and then computing depends on permitted error and rounding rules. Identity elimination using a fixed tolerance needs a clear rationale; convenience is not a semantic specification. Historical lowering included a near-zero check, and a public discussion should retain that numerical prerequisite.

Commutative addition and multiplication do not imply that their storage objects can be swapped at any stage. A constant on one side may use a narrower type, while a dynamic input uses a different quantization scale, and machine source modes may have asymmetric parameter fields. Decide the new roles first, then generate their types and parameters. The same principle applies to reverse subtraction: a transformation allowed by mathematics must be honored again in the backend protocol.

For localization, split test observations across three points: the constant-specific node after import, the direction attribute after lowering, and the final source modes. If the first is wrong, inspect constant recognition. If the first is right and the second wrong, inspect attribute propagation. If both are right and the final result is wrong, inspect the kernel interface. These layered checks do not replace numerical tests, but greatly narrow the search after failure. They also make a “scalar subtraction is wrong” report easier to turn into a reproducible case.

One performance counterexample is worth keeping. Lowering a large tensor plus a constant to a dedicated scalar path often saves broadcast storage. For a one-element tensor, the ordinary binary path may already be very cheap, while extra normalization and metadata handling mainly add compiler complexity. A compiler need not pursue another “more optimized” path for every tiny case. Clear semantics and stable generated output can be more valuable than another difficult special case.

The lesson is more than the familiar observation that subtraction is special. It is an interface-checking method: follow left and right from the model to the final machine source modes, and follow an algebraic identity through type, quantization, and fusion constraints. If every stage can explain where direction went, subtraction will not quietly turn around halfway through.

Series contents · Previous · Next

• ← Previous Post
• Next Post →