I'm Bruce Lee

Subtracting Zero Can Disappear; Subtracting from Zero Cannot

Keep constant folding, commutativity, and operand direction as separate decisions.

Series contents · Operators and Layouts · 阅读中文版 “Can we write subtraction the same way as addition?” Most interfaces do look alike. Both have two inputs and one output; both need data types, addres......

Posted by Bruce Lee on 2026-04-09

Compilers Numerical Computing

The First Row Works, the Second Disappears: Alignment, Strides, and Padded Constants

Make logical widths, physical strides, padded storage, and parameter layouts agree.

Series contents · Operators and Layouts · 阅读中文版 “The first row is perfect. From the second row onward, it looks like a lottery.” That is often a rather helpful error: the result’s structure is alre......

Posted by Bruce Lee on 2026-04-03

Compilers Memory and Concurrency

Broadcasting: Sometimes the Right Address Is the One That Does Not Move

Build per-axis address plans for scalars, lower-rank inputs, and more complex broadcasts.

Series contents · Operators and Layouts · 阅读中文版 “The framework does this in one line. Why does the backend need so many cases?” The mathematical definition of broadcasting is compact: align dimensi......

Posted by Bruce Lee on 2026-03-29

Compilers Memory and Concurrency

When the Debugger Starts Lying: One Permute, Three Ways to Fix Dump

Three alternative branch fixes show why observation paths need layout contracts too.

Series contents · Operators and Layouts · 阅读中文版 “The numbers are wrong.” “How do you know?” “Dump told me.” This is a good moment to pause. Dump can show bytes, but it cannot automatically guarante......

Posted by Bruce Lee on 2026-03-25

Compilers Memory and Concurrency

Why Does Reshape Need Code? The Elements Stay the Same; Their Route Changes

Prove the linear element mapping and distinguish shape changes from physical rearrangement.

Series contents · Operators and Layouts · 阅读中文版 “Not a single element is missing. Why does the compiler say this is unsupported?” Reshape hears this accusation often. The model author sees the same......

Posted by Bruce Lee on 2026-03-18

Compilers Memory and Concurrency

Transposed Convolution Weights: The Shape Is Honest, but Memory Speaks Another Dialect

Derive slices from actual weight storage, not just a logical dimension order.

Series contents · Operators and Layouts · 阅读中文版 The first time this error appears, it is tempting to suspect kernel reversal, stride, or padding: one group works beautifully, but enabling grouping ......

Posted by Bruce Lee on 2026-03-13

Compilers Numerical Computing

Grouped Convolution: Once You Slice the Cake, Slice the Labels Too

Split input and weight groups while carrying their quantization metadata with them.

Series contents · Operators and Layouts · 阅读中文版 “Ordinary convolution already works. Surely grouped convolution just needs one more parameter?” That sentence is about as dangerous as “this is only ......

Posted by Bruce Lee on 2026-03-07

Compilers Numerical Computing

The Importer Got Clever Too Soon: Algebraic Optimization before Quantization Metadata

Understand the interaction between algebraic rewrite order, metadata matching, and constants.

Series contents · Numerics and Quantization · 阅读中文版 “A constant divided by a tensor—wouldn’t a scaled reciprocal be more efficient?” The enthusiastic importer tidies the algebra as soon as the mode......

Posted by Bruce Lee on 2026-03-03

Compilers Numerical Computing

Floating Point Passes, Quantization Fails: A Rewrite Gives the Output the Input’s Identity

Treat the rewritten result type as both a container and a numerical contract.

Series contents · Numerics and Quantization · 阅读中文版 “The floating-point model compiles. The quantized one cannot get through the first stage.” Someone starts suspecting backend integer instructions......

Posted by Bruce Lee on 2026-02-24

Compilers Numerical Computing

Epsilon Was Nonzero—Until It Nearly Vanished in the Integer Domain

Follow epsilon, squares, and reciprocal square roots through normalization range planning.

Series contents · Numerics and Quantization · 阅读中文版 “We add epsilon to prevent division by zero.” Everyone relaxes. Then someone looks at the quantized constant and asks, “What value does it have a......

Posted by Bruce Lee on 2026-02-19

Compilers Numerical Computing

Bruce Lee's Notes

Subtracting Zero Can Disappear; Subtracting from Zero Cannot

Keep constant folding, commutativity, and operand direction as separate decisions.

The First Row Works, the Second Disappears: Alignment, Strides, and Padded Constants

Make logical widths, physical strides, padded storage, and parameter layouts agree.

Broadcasting: Sometimes the Right Address Is the One That Does Not Move

Build per-axis address plans for scalars, lower-rank inputs, and more complex broadcasts.

When the Debugger Starts Lying: One Permute, Three Ways to Fix Dump

Three alternative branch fixes show why observation paths need layout contracts too.

Why Does Reshape Need Code? The Elements Stay the Same; Their Route Changes

Prove the linear element mapping and distinguish shape changes from physical rearrangement.

Transposed Convolution Weights: The Shape Is Honest, but Memory Speaks Another Dialect

Derive slices from actual weight storage, not just a logical dimension order.

Grouped Convolution: Once You Slice the Cake, Slice the Labels Too

Split input and weight groups while carrying their quantization metadata with them.

The Importer Got Clever Too Soon: Algebraic Optimization before Quantization Metadata

Understand the interaction between algebraic rewrite order, metadata matching, and constants.

Floating Point Passes, Quantization Fails: A Rewrite Gives the Output the Input’s Identity

Treat the rewritten result type as both a container and a numerical contract.

Epsilon Was Nonzero—Until It Nearly Vanished in the Integer Domain

Follow epsilon, squares, and reciprocal square roots through normalization range planning.

VISITORS

FEATURED TAGS

ABOUT ME

RECENT POSTS

archives

categories