When the Debugger Starts Lying: One Permute, Three Ways to Fix Dump

Series contents · Operators and Layouts · 阅读中文版

“The numbers are wrong.”

“How do you know?”

“Dump told me.”

This is a good moment to pause. Dump can show bytes, but it cannot automatically guarantee that the rules used to interpret them are correct. A mismatched layout label, shape, quantization scale, writeback location, or sampling time can package correct computation as an incorrect result. A debugger does not lie on purpose, but it does bring defaults to work.

Establish the Observed Layout

Suppose a teaching tensor uses HWC order and has shape [2,3,2]. Give every element a readable label:

1

x[h,w,c] = 100*h + 10*w + c

The first few values in contiguous order are 0,1,10,11,20,21. After a Permute to CHW, the interpretation changes. Keep labeling the result HWC, and the comparison tool will look for reference elements at the wrong coordinates. The data is still there; its labels are simply calling everyone by a neighboring colleague’s name.

Three Fixes and Layout Propagation

One verified historical approach recognized only a particular three-dimensional permutation. It looked through one Store and, if it found a Permute in a known direction, changed the label to channel-first; otherwise, it kept the default layout. The change was small and easy to validate against the immediate problem. Its boundary was also clear: this was not general layout inference, and unknown paths still received a default label. Calling it a complete solution to layout tracking would exceed the evidence.

The second approach changed the sampling set, excluding Store and continuing to exclude Permute. This avoided certain ambiguous observation nodes and reduced duplicate output. But having no sample is different from confirming that the result is correct. It may quiet the debugging path while removing the chance to inspect data immediately after a permutation. Choosing where to insert Dump is itself part of observability design.

The third approach was more systematic: it allowed sampling Permute, excluded Store, and attempted to propagate layouts through the IR. It introduced layout anchors, layout-preserving operators, and rules for applying permutations, returning unknown when inference failed. Three directions for the same problem remind us that a Git history is not necessarily a linear novel. An author may explore different tradeoffs on different branches; chronological order does not prove that approach B replaced approach A.

A general way to think about layout inference is to assign a known axis order to a boundary or an operator with explicit semantics, then propagate it along dataflow. For input labels [H,W,C] and permutation [2,0,1], the output labels are:

1
2

output_labels[i] = input_labels[permutation[i]]
result: [C,H,W]

There is no need to hard-code every ordering. There is a need to validate that the permutation really is a permutation: the length must match, entries must be integers, indices must be in range, and none may repeat. If [2,2,0] slips through, array lookup still produces three letters, but that string does not describe a legal axis transformation. String length cannot do the validation.

Why does propagation need anchors? Because knowing that an arbitrary boundary argument has rank 3 may not tell us what its three axes mean. Some operators specify output layouts at particular stages and can provide reliable starting points. A compiler-generated permutation with explicit provenance may also offer a controlled fallback. The more systematic historical approach left uncertain three-dimensional boundaries unknown instead of assuming that every rank-three tensor was an image.

“Unknown” can be more useful than “looks plausible.” It separates fact from guesswork, allowing a comparison tool to inspect only raw bytes, require an additional description, or suspend comparisons across layouts. Quietly replacing unknown with a common layout sends the error across an interface boundary, where it eventually becomes an accusation against some operator’s arithmetic.

Layout-preserving operators also require care. Under certain conventions, an elementwise operation can propagate axis meaning from its feature input. With binary broadcasting, however, the left operand may be a scalar or a smaller tensor; choosing the first operand is not always sufficient. Reshape certainly does not guarantee preservation of axis meanings, even when element counts match. The historical implementation had a restricted operator whitelist and rank checks. Treat it as bounded inference, not a complete replacement for a layout type system.

Inspection also found a useful code-review exercise: one branch contained an additional Reshape check, but the earlier whitelist of layout-preserving operators did not include Reshape. The intended branch may therefore be unreachable. Without runtime evidence, the article cannot announce a demonstrated runtime fault. At minimum, it shows why conditions must be followed through actual control flow, rather than accepted because “there is an if for it.”

Observation Contracts and Tests

The relationship between Store and Dump is subtle too. A Store input may reside in on-chip memory and its output in another memory space. They have the same logical value, but not necessarily the same addresses or layout descriptions. If Dump silently traces back to the Store input when creating a descriptor but still examines the Store output when producing metadata, its data source and interpretation refer to different objects. The more complete historical approach adjusted candidate nodes, data-source handling, and metadata inference together, showing that the issue crossed module boundaries.

An observation node should keep at least four things consistent: which value supplies the bytes it reads, whose shape it uses, which physical layout and strides apply, and which quantization encoding interprets the data. Sampling must also occur after production completes and before the buffer is reused. Correct timing and synchronization require independent verification and cannot be guaranteed by these diffs.

A teaching regression can place an elementwise operation between two inverse permutations:

1

input -> Permute(P) -> Add zero -> Permute(P_inverse) -> output

Check sample counts, shapes, and layout labels at each observation point, then verify values with coordinate labels. Adding zero preserves the result while testing whether layout information propagates through the middle operator. Historical tests included a similar permutation chain and metadata assertions. Suggested additions include unknown boundaries, invalid permutations, broken layout-preserving chains, binary broadcasting, and duplicate sampling.

For performance, more Dumps mean more data movement, storage, and synchronization, and can change scheduling or memory reuse. “The problem disappears when Dump is enabled” is therefore a clue, not immediate proof of a fix. Inspect the effects of metadata-only output separately from actual tensor movement. Sampling every intermediate value helps localization; final performance measurements need an explicitly defined mode.

Coordinate Maps and Unknown Layouts

Layout propagation can also graduate from reasoning about letter strings to a simple composition exercise. Let the first permutation be P and the second Q, using the convention that output axis i takes input axis P[i]. After both, output axis i corresponds to original axis P[Q[i]]. If a comparison tool uses the opposite permutation convention, two systems can record the same array while meaning different things. Tests must specify whether the array means “output takes input” or “input goes to output,” not merely record it.

Unit dimensions make purely numerical tests devious again. Swapping two axes of length one may leave the bytes unchanged while changing layout meaning. When dimensions happen to have equal lengths, even the printed shape may conceal the difference. Teaching labels should combine distinct axis lengths with a value pattern that depends on every axis. Historical test shapes validate particular paths; systematic coverage of every propagation rule also needs deliberate attention to these symmetric and degenerate cases.

Another easily confused question is whether a layout label describes logical axis order or complete physical storage. “HWC” usually expresses axis order; it does not necessarily include row stride, channel padding, within-block interleaving, or bit packing. A comparison tool treating it as a complete physical description may still read a file incorrectly as a compact array. A reliable observation protocol should state whether exported bytes have had physical padding removed or whether consumers must decode them using additional stride information.

Sometimes identical values in two Dumps are themselves suspicious. A Permute and its predecessor might accidentally reference the same buffer before it has been updated. The comparator could then report that “permutation does not change the result,” particularly without raising any concern for all-ones input. Use a coordinate-encoded tensor and calculate a few corresponding positions by hand before and after the operation. Also check which SSA value and memory region each sample references. Observed data and its description should constrain each other, rather than make independent guesses.

How unknown layout is presented also affects debugging efficiency. A reasonable interface can retain a raw-byte summary, shape, and reason for uncertainty, while avoiding numerical comparisons that require a layout assumption. If some axis meanings are known, partial information may be recorded, but missing entries should not be invented merely to produce an attractive string. The historical approach conservatively returned fully unknown. That is more honest than a default; a more detailed explanation trail remains future design space.

Finally, changes to the sampling set belong in test expectations. Excluding Store can reduce sample counts; including Permute can increase them. If a consumer identifies “the nth tensor” only by position, a graph optimization can make that position refer to a different value. Combining stable value identity, readable provenance, and structural information is more robust than relying solely on order. These recommendations do not imply that the historical implementation had a complete tracing system. They explain why a layout fix can pull in an entire observation protocol.

The next time a comparison tool reports “the first incorrect layer,” do not rush to put that layer on trial. Establish the layout, quantization interpretation, and address used to read that “first” result. A trustworthy observer can spare you a great deal of debugging code that was correct all along.

Series contents · Previous · Next

• ← Previous Post
• Next Post →