Series contents · Numerics and Quantization · 阅读中文版
“We were extending the dedicated operator last week. Why are we deleting it this week?” In a code review, the question can sound awkward. The useful follow-up is not “who took the wrong turn?” but “which abstraction boundary did we discover was in the wrong place?”
Compiler design often works this way. The first implementation makes the problem visible; the second reveals which layer should own it.
1. The Initial Appeal of a Dedicated Affine Operator
The per-channel affine transform y=a*x+b has clear domain semantics. A dedicated node can express parameter broadcasting, multiplication-addition order, workspace needs, and fusible attributes in one place.
With a suitable execution path, a dedicated kernel can also reuse parameters by channel or row, control loops and strides, and avoid expanding parameters across the feature map too early. High-level analyses can easily recognize an affine transform and consider further fusion with neighboring operations.
Earlier historical implementations did add dedicated configuration, instruction sequences, code-generation registration, and workspace interfaces. One draft used bypass or placeholder numerical parameters; a related version supplied actual quantized multiply-add parameters and more layout handling. These are different historical snapshots. Nearby dates do not prove that every limitation of one was repaired in a straight line by the other.
The price of a dedicated node is having to answer the entire engineering checklist separately: layout, tiling, broadcasting, quantization, resource declarations, diagnostics, registration, and tests. If one remains incomplete, elegant operator semantics can still become the pipeline’s largest collection of special cases.
2. What Does Decomposition into General Multiply-Add Reuse?
Lowering affine computation into multiplication and addition can reuse existing binary-operator machinery: shape legalization, broadcasting, layout conversion, scheduling, memory allocation, and emission. Historical diffs first introduced such a replacement, then removed dedicated operation definitions, kernels, code generation, and resource implementations, along with special cases in layout rules.
This substantially reduces the implementation surface that must be maintained. One fewer operator needs to keep pace with common changes, reducing the chance that a dedicated path is overlooked.
“General” does not automatically mean semantically complete, however. If the affine node carries a fused-activation attribute, decomposition must implement an equivalent activation or explicitly reject it. A later historical lowering checked and rejected unsupported fusion conditions, which is more reliable than silently dropping attributes.
Likewise, a dedicated node may promise one rounding step, while two quantized nodes introduce another. Compact parameters may need full expansion when fed to general operators. Code saved by simplifying the abstraction does not automatically translate into lower runtime cost.
3. Build an Inventory of Semantic Responsibilities
Before deleting a dedicated operation, list every piece of information it carries and assign a destination:
| Original semantics or capability | Home after decomposition | Required check |
|---|---|---|
| Per-channel coefficients | Constant types and broadcast rules | Are the axis and layout explicit? |
| Numerical multiply-add order | Two nodes and an intermediate type | Do rounding or saturation change? |
| Fused activation | Separate activation or explicit rejection | No silently discarded attributes |
| Temporary workspace | General memory planning | Does obsolete workspace remain? |
| Operation provenance | Derived locations and diagnostic relationships | Can errors be traced to the original model? |
| Special shortcuts | canonicalize or lowering | Required type conversions remain |
This inventory is closer to a correct migration than searching for the dedicated class name and deleting every occurrence. Removing references establishes at most that the compiler may still build. Giving each semantic responsibility a home helps establish that functionality has not quietly evaporated.
4. Splitting One IR Node Can Make Failures Easier to Locate
If one dedicated kernel performs multiplication, addition, and broadcasting internally, a numerical failure points to only one node. General operations let the IR distinguish bad constant quantization, an inappropriate multiplication output range, and incorrect bias broadcasting.
Historical changes also adjusted derived locations so multiplication and addition had distinguishable origins. Public articles need not preserve the internal naming scheme, but the principle matters: generated nodes need traceable origins, rather than several nodes sharing a vague “generated somewhere.”
Clear diagnostics do not require the final program to remain decomposed forever. A compiler can use general nodes where semantics should be visible, then fuse them again in target-specific stages. A useful separation exposes semantics to optimization and validation first, and compresses execution form later under justified conditions.
5. Why Document Deletion Matters, but Is Not a Functional Fix
One related historical entry had a title suggesting a normalization update, while its actual diff only removed a temporary review document. It changed no runtime code and added no tests. Writing from the title alone could invent a numerical fix that never occurred.
The removed document recorded limitations, open validation items, and design suggestions. None is an execution result. A note recommending layout tests does not mean those tests passed; deleting the note does not mean every listed problem was closed.
A more reliable writing and archival practice distinguishes implementation, test definitions, and actual execution results. Design notes form a fourth category: intent and questions. They help identify what to ask, but cannot replace the first three.
The drama in a technical article should come from the structure of the problem, not from turning every commit title into a heroic moment. Sometimes the honest sentence is simply: “This entry only removed temporary notes.”
6. How to Judge Whether the Design Change Was Worthwhile
Create at least two comparisons. The first covers functional contracts: layouts, widths, parameter forms, fusion attributes, and invalid inputs in each path. The second is a resource ledger: constant size, peak local memory, intermediate materializations, instruction-configuration counts, and actual runtime.
Proposed regressions include:
- Run the same model through dedicated and decomposed paths; compare the real-value semantics of intermediates and final integer outputs.
- Make the product exceed the range before bias cancels it; check whether extra saturation changes the result.
- Cover compact and materialized broadcasting together, observing parameter volume.
- Check stable, clear diagnostics for unsupported fusion attributes.
- After removing the dedicated implementation, check for stale build registration, interface declarations, and resource estimates.
- Verify support separately on different branches; tests from one branch do not establish another’s behavior.
A performance model should not choose its winner in advance. The dedicated path may avoid intermediates, while the general path may gain better tiling and scheduling. Instruction configuration may dominate small tensors; parameter expansion and extra reads and writes may dominate large ones. Without measurements, these are competing hypotheses.
The history’s value is not to prove the first design wrong and the second right. It shows constraints becoming clearer. When maintenance costs accumulate in one node, decomposition may help. When execution costs accumulate at the resulting boundaries, selective fusion may help again. A good abstraction permits both decisions at different stages.
7. A Fallback Can Help a Design Move Forward
When introducing a dedicated implementation, define a general expression it can fall back to: how existing operations preserve the semantics outside the fast path’s conditions. The dedicated path then optimizes controlled cases, while the general path provides explainable coverage. Neither must immediately absorb every new requirement.
For example, if the dedicated implementation supports only certain layouts and parameter forms, check those constraints explicitly in its matcher and otherwise use general multiply-add. Avoid generating half a configuration only to fail deep inside the kernel on an unknown shape. An early support boundary makes it easier to distinguish unsupported models, missing legalization, and backend bugs.
Fallbacks need tests too. A general path with two rounding steps and a dedicated path with one may agree only within an allowed error bound, not bit for bit. Specify the required equivalence level: exact identity, an absolute error limit, or another mathematically justified bound. Otherwise the fast-path switch can make regression behavior appear intermittently correct.
When deleting a dedicated node, search in the opposite direction as well: which optimizations depended on that node to recognize affine structure? If those opportunities disappear, should new recognition patterns operate on the general graph? Restoring every performance opportunity immediately is not always necessary, but any loss should be visible. Successful linking alone does not complete deletion; the representation’s former analytical role also needs accounting.
Documents deserve equally precise handling. A temporary review draft should not indefinitely serve as a specification, but its unresolved questions may deserve stable design notes, test cases, or recorded conclusions. Deleting a draft reduces stale information; forgetting its unresolved questions does not help.
A useful design retrospective need not portray every change of direction as part of the original plan. “The implementation exposed these constraints, so we moved the boundary” helps future engineers more than editing history into a perfect straight line. Readers need the conditions behind a decision, not a legend without forks.
Back to series contents · Previous · Next
If you like this blog or find it useful for you, you are welcome to comment on it. You are also welcome to share this blog, so that more people can participate in it. All the images used in the blog are my original works or AI works, if you want to take it,don't hesitate. Thank you !