An Instruction’s Four Identities: Why Code Generation Needs More Than Strings

Series contents · Engineering and Delivery · 阅读中文版

“We are emitting one multiply instruction. Why does it need four layers?”

Someone writes emit("mul", a, b, out) on the review whiteboard. It is short, direct, and almost looks like final assembly. A colleague asks, “Is a a tensor, a register number, a memory address, or a scalar to broadcast?”

A brief silence follows. The string interface looks simple because it hides the questions.

An instruction in a compiler backend often has several identities. It begins as a mathematical operation, becomes a kernel execution step, turns into hardware actions, and finally occupies fixed-width binary fields. Mixing those identities often does not produce an immediate error. It makes lower layers guess what upper layers intended.

Layer one answers “What are we computing?”

For two-dimensional matrix multiplication, the upper-level contract might be:

1
2
3
4

A: [M, K]
B: [K, N]
C: [M, N]
C[m,n] = sum_k A[m,k] * B[k,n]

Transpose conventions, accumulation type, quantization parameters, and any activation need definitions here. The layer does not need to know which configuration register holds an address.

This boundary is useful. A layout change usually leaves the mathematical operation unchanged. A hardware revision that re-encodes an address field should not force changes to the operation layer. Conversely, activation fusion that moves a rounding boundary can change numerical semantics and must not be disguised as a pure encoding adjustment.

“Does not know registers” does not mean “checks nothing.” Dimension mismatches and invalid result types are easier to explain to model users when rejected early and explicitly.

Layer two answers “How is the computation organized?”

The kernel layer chooses tiling, loops, data reuse, and temporary buffers. It might divide the output into small rectangles, load an input portion for each, and accumulate along K.

Two independent kinds of information are particularly easy to mix here:

• Tensor information: shapes, strides, element types, and quantization parameters.
• Execution resources: storage addresses, registers, events, dependencies, and loop labels.

Using one generic integer for both a tensor-axis position and a register number can pass C++ type checking while losing the ability to express the boundary.

Turn the design issue into a concrete question: when a kernel requests “load this slice,” has it specified the logical extent and physical strides? Otherwise, lower layers must infer a shape from an address and byte count. That can occasionally work for contiguous layouts and quickly fails for broadcasting, alignment, and noncontiguous views.

Layer three answers “What actions does the hardware need?”

The emission layer translates an execution step into hardware actions: configure addresses and strides, start transfers, compute, and wait for dependencies. It needs hardware knowledge without redefining matrix multiplication.

A teaching transfer request might contain:

1
2
3
4
5

source: MemoryAddress
destination: MemoryAddress
extent: ElementCount
source_stride: ByteStride
destination_stride: ByteStride

The names seem verbose, but they obstruct specific mistakes: putting an element stride into a byte-stride field, using an address as a register number, or overlooking an element count outside the encoding range.

Strong types do not prove algorithmic correctness. They make some mistakes harder to express and give remaining checks a clear home. The contract should say where address alignment is checked, who reports register exhaustion, and whether an oversized immediate may be expanded.

Layer four answers “How are the bits arranged?”

The final encoding layer owns opcodes, field widths, sign extension, byte order, and reserved bits. Its input should be an explicit instruction structure, not a string list whose meaning still needs interpretation.

One useful choice in the reviewed design was to organize types by encoding format, rather than mechanically creating a class hierarchy for every opcode. Several arithmetic instructions may share register-field positions and differ only in opcode. Memory instructions may require a very different address layout.

Two extremes are common:

1. All instructions use vector<string>, making invalid states easy to construct.
2. Every opcode has a large separate class, scattering shared validation and encoding logic.

Grouping payloads by format and defining the allowed opcode set is a compromise. Its suitability depends on format stability and how much structure opcodes truly share. The number of types is not itself a design-quality metric.

Label displacements must reflect final instruction lengths

Consider a branch:

1
2
3
4

jump done
load_constant large_value
done:
store result

Suppose load_constant needs one machine instruction for some values and two for others. Calculate labels under the assumption that one source line equals one instruction, then expand pseudoinstructions, and the jump can target the wrong location.

A reliable organization usually distinguishes:

1. Semantic operations and pseudoinstructions.
2. The expanded actual-instruction sequence.
3. Label positions computed from actual lengths.
4. Final field checks and encoding.

Some architectures also let branch distance affect instruction length, requiring iterative relaxation or a fixed-length strategy. The historical material does not establish such a mechanism here. The general point is that label resolution must state which stage’s lengths it uses.

That is also why plausible pseudoassembly cannot replace binary validation: the human-readable stage may not yet determine final locations.

Which layer should produce a good error message?

In the review, “Why so many layers?” soon becomes “Who should report this failure?”

Lower layers should still defend against errors missed above, but diagnostics need context. “Field overflow” helps locate an encoding problem; “unsupported tensor stride” better explains a kernel limitation. Collapsing every failure into a final assertion makes distinct problems look identical.

How can we show that layering has not hidden errors deeper?

Each layer needs observations independent of its own implementation:

• Check operation semantics with small reference computations.
• Check kernel slice coverage, boundaries, and memory-access ranges.
• Check emission order, dependencies, and resource lifetimes.
• Check exact encoded bytes for fixed inputs and rejection at boundaries.

These checks complement one another. A reproducible encoder can still encode the wrong mathematics. A numerically correct example may never exercise an immediate limit.

Historical tests progressively strengthened comparisons among pseudoassembly, actual assembly, and binary output. That supports the value of checking consistency across layers. It does not establish that the entire instruction set was fully validated.

A proposed, unexecuted experiment would vary a kernel’s layout, tile size, and immediate range separately, then observe which layers change. A pure field-encoding adjustment that forces mathematical-layer changes may reveal a leaking boundary. A changed tensor shape that triggers no legality check may reveal a boundary that is too permissive.

The four-layer architecture should buy explainability: is the error in the mathematics, the execution organization, the hardware actions, or the encoded bits? With an answer, debugging no longer requires guessing all the way from a model input to its final byte.

Series contents · Previous · Next

• ← Previous Post
• Next Post →