This Reduction Does Nothing—Why Can’t We Delete It?

Series contents · Numerics and Quantization · 阅读中文版

“Summing along an axis of length one returns the input. Delete it and save an instruction.” Mathematically, that sounds solid. In engineering terms, one question remains: does “the input” mean the same real value or the same sequence of integers?

Deleting an operator with apparently no computational work can change the numerical behavior of an entire model. It is like switching a thermometer from Celsius to Fahrenheit while insisting that the displayed number must stay put.

1. Length One Does Not Guarantee the Same Output Representation

Let the input shape be [1,5,7,9], reduced along the first axis. For sum, mean, maximum, and minimum, each group contains one element, so the real value stays unchanged. If dimensions are retained, the logical shape also stays unchanged.

Now give the operation quantized types:

1
2

input:  q_in,  scale = 0.03, zero_point = 0
output: q_out, scale = 0.12, zero_point = 0

Input integer 20 represents the real value 0.6. Keeping integer 20 in the output represents 2.4 instead; the correct output is approximately integer 5. Although the operator leaves the real-valued function unchanged, it promises a conversion of the output representation.

The general relationship is:

1

q_out = clamp(round[(s_in/s_out)*(q_in-z_in)] + z_out)

A sufficient condition for removal is therefore not “the reduction length equals one,” but “the numerical function is the identity, and the complete output type and related semantics permit direct substitution of the input.” The historical fix explicitly required matching input and output types for direct replacement; differing types retained a requantization operation.

2. The Two Faces of Empty axes

Before quantization even enters the discussion, shape processing may set a trap. The first axis of a four-dimensional input has length one, and shape normalization removes it. The original axes were [0]; after remapping, no effective axes remain.

Yet many internal representations also use axes=[] to mean “all axes.” These two empty lists look identical:

1
2

Case A: the user originally requested a reduction over all axes.
Case B: the user requested only a unit axis that later disappeared.

A downstream stage that sees only the list may interpret B as A. An identity mapping that preserves all elements can become a single scalar. Emptiness is not the information; it is the space left after information was lost.

The history first added metadata recording removed axes and moved axis legalization after shape processing. A later change simplified the permutation-and-reduction chain for this special case into forwarding or requantization. The two steps solve different problems: the first preserves semantic provenance, and the second uses it for safe simplification.

3. Why a reshape Cannot Replace Requantization

A reshape reinterprets element indexing and, under appropriate conditions, preserves the element sequence. It is not a general numerical transformation. Assigning a different quantization scale to the result type does not automatically compute new integer values.

Teaching pseudocode should look like this:

1
2
3
4
5
6
7
8

assert normalized_input_shape == normalized_output_shape

if input_type == output_type:
    result = input
else if supported_quantized_conversion(input_type, output_type):
    result = requantize(input, output_type)
else:
    report unsupported conversion

“Supported” needs a definition. How is the integer storage range determined? Is it signed or unsigned? Is the zero point valid? Can the width and intermediate arithmetic represent what is needed? Historical code calculated clamping bounds from the output integer storage type and rejected widths it could not safely handle. This illustrates the value of conservative failure. An early explanation of missing support is easier to diagnose than a result generated using a convenient default range.

Whether a complete type comparison is sufficient also depends on which contracts the IR stores in attributes. If rounding mode, saturation policy, or layout markers live outside the type, they need checking too. One fix does not establish that every such attribute was covered.

4. How Equal-Scale Tests Can Hide the Problem for Years

Suppose both input and output scales are 0.1, with both zero points equal to 0. For many ordinary inputs, requantization and direct forwarding produce the same integers. All-zero data may hide the issue even when scales differ.

These tests should therefore be deliberately less ordinary. Include unequal scales, nonzero values, values near rounding boundaries, and saturation cases. With a nonzero zero point, also test the integer representing real zero; it is usually not integer 0.

A deliberately chosen check is:

1
2
3

s_in = 0.03, s_out = 0.12
q_in = [4, 8, 20, -12]
ideal unsaturated q_out = [1, 2, 5, -3]

These values illustrate scaling but do not exercise rounding. Adding values not divisible by 4 begins testing the actual rounding rule. Expected values must follow the target’s definition; the host language’s default round is not a universal answer.

5. What Is the Performance Benefit of Removing a Useless Reduction?

The historical fix shortened a transformation chain built for a removed unit axis into either the input itself or requantization. Fewer helper nodes are directly observable in the diff.

When input and output types match, forwarding may remove configuration, execution, and temporary buffers, provided downstream consumers permit aliasing and need no extra copy. When requantization is required, elements must still be traversed, read, and written, with traffic generally growing with tensor size. “The reduction length is one, so its cost is zero” is wrong in that case: changing the output representation has a cost of its own.

A performance model can separate the cases:

1
2

Same-type path: primarily check whether materialization and scheduling overhead disappear.
Different-type path: T ≈ startup cost + element count / effective throughput, plus read/write traffic.

There may be a fusion opportunity: can requantization be incorporated into the next operator? Such fusion must preserve the semantics of the original rounding point. Two coefficient multiplications cannot automatically be merged merely because both are visible.

6. A Regression Matrix That Separates the Meanings of “Identity”

Historical tests contain structural assertions for unit-axis reductions disappearing and for requantization remaining when scales differ. This is stronger than checking only “no Reduce remains,” which would reward an incorrect optimization that deleted the numerical conversion too.

The central question applies elsewhere: multiplying by one, adding zero, concatenating a single element, and some copies or type conversions can all carry representation contracts while leaving the mathematical function unchanged. An optimizer must prove that the entire contract can disappear, not just observe that the formula seems inactive.

The next time a review says “isn’t this an identity?”, add one more question: “In which numerical space?” It can save the team a particularly educational debugging round.

7. Turn “Safe to Delete” into a Proof Obligation

A fuller criterion has three layers. First, prove that each output real value equals the corresponding input real value; unit-length reduction addresses this layer. Second, prove a one-to-one correspondence between output and input indices; shape normalization and keep_dims handling address this layer. Third, prove that reusing the input’s storage representation still satisfies the output contract; this is where scales, zero points, widths, and related attributes are checked. Without any one layer, direct replacement is unjustified.

This layered proof also explains why identical types are not sufficient in every setting. An operation may additionally promise copy semantics, an address-space conversion, or observable exception handling. Equal values and types need not preserve those behaviors. The pure tensor reductions discussed here do not assume such external semantics exist, but a general optimization framework should be able to express them rather than place every apparent identity into one pattern.

For requantization, work through another zero-point example. Let the input scale be 0.04 and zero point 7, and the output scale 0.08 and zero point 3. Input integer 11 represents 0.16 and corresponds to output integer 5. Omitting the input zero point scales 11 directly; omitting the output zero point gives 2. Both can look like small discrepancies in an isolated comparison, but their behavior across different inputs helps distinguish a scale error from a zero-point error.

A useful metamorphic test starts with exactly representable real values, encodes them in two quantization domains, and checks that the unit-axis reduction output represents the same real values. It requires neither a fixed original test vector nor all-zero data. When the target has a particular rounding rule, use exactly representable points to validate the main formula first, then test rounding boundaries separately. Otherwise two kinds of error may become one hard-to-explain discrepancy.

Also check structure and values in both directions. For identical types, the reduction should disappear without introducing an unnecessary conversion. For different types, the useless reduction should disappear while the required numerical conversion remains. Counting removed nodes encourages excessive deletion; comparing one numerical example may miss substantial materialization that should have vanished. Optimization tests need to say both what must disappear and what must remain.

These arguments can first be reviewed as proofs at the IR level without hardware. Execution must then confirm that the requantization kernel implements the same rounding and clamping rules before structural correctness can be connected to numerical correctness.

Back to series contents · Previous · Next

• ← Previous Post
• Next Post →