Bruce Lee's Notes

Systems, programming, and engineering practice

Build a Dedicated Operator, Then Remove It: A Design Change Worth Recording

Compare the expressiveness and maintenance cost of a dedicated affine operator and its decomposition.

Series contents · Numerics and Quantization · 阅读中文版 “We were extending the dedicated operator last week. Why are we deleting it this week?” In a code review, the question can sound awkward. The use......

Posted by Bruce Lee on 2026-02-13

Compilers Numerical Computing

Just a Few Channel Parameters—So Why Is Local Memory Full?

Avoid materializing channel parameters across a whole tensor while preserving broadcasting.

Series contents · Numerics and Quantization · 阅读中文版 “The model has one tiny scale vector and one tiny bias vector. How can it run out of memory?” The allocation failure appears to be arguing with c......

Posted by Bruce Lee on 2026-02-09

Compilers Numerical Computing

Once BatchNorm Becomes Multiply-Add, the Numerical Work Begins

Follow fixed statistics through channelwise affine transforms, multipliers, shifts, and biases.

Series contents · Numerics and Quantization · 阅读中文版 “BatchNorm has already been folded into y=ax+b. Surely all that remains is a multiplication and an addition?” Everyone nods. Minutes later, the q......

Posted by Bruce Lee on 2026-02-02

Compilers Numerical Computing

The Multiplication Is Right; the Scale Is Wrong: An Integer Kernel’s Two Ledgers

Track input scales, output scales, and the range of the intermediate product.

Series contents · Numerics and Quantization · 阅读中文版 “The two integers multiply to exactly what the calculator says, but the model is still wrong.” Staring at the multiplication sign rarely helps he......

Posted by Bruce Lee on 2026-01-28

Compilers Numerical Computing

This Reduction Does Nothing—Why Can’t We Delete It?

A mathematical identity can still require a change of quantized representation.

Series contents · Numerics and Quantization · 阅读中文版 “Summing along an axis of length one returns the input. Delete it and save an instruction.” Mathematically, that sounds solid. In engineering ter......

Posted by Bruce Lee on 2026-01-22

Compilers Numerical Computing