Why Does Someone Keep Borrowing My “Temporary” Register?

Series contents · Resources and Scheduling · 阅读中文版

“I just need one register to load an address. I’ll give it straight back.”

“Funny. The outer loop counter is borrowing that same one.”

This argument does not happen in a code review chat. It happens in machine code: a loop count suddenly becomes an address, or a configuration load overwrites a scalar that has not yet been used. Each helper looks short in isolation. Together, they resemble a tool room with no checkout register.

The problem is not that programmers cannot remember register names. Names do not express ownership. The string "tmpA" answers “which one?” but not “who owns it right now?”

Ownership Has Two Timelines

When a compiler generates device code, there are two timelines. On the first, the compiler runs on the host: constructing instructions, allocating scratch slots, and calling helpers. On the second, the generated program runs on the device: loops iterate, descriptors are consumed, and asynchronous tasks execute.

RAII manages resource leases on the first timeline. Leaving a C++ scope releases the lease, allowing subsequently generated code to reuse the register. That is correct only if the generated runtime instructions will no longer need its old value.

That qualification matters. Put a counter that is still live at runtime inside a host scope that is too short, and RAII will release the wrong thing, exactly on schedule. Conversely, keeping every scratch handle alive until the entire kernel has been generated is usually conservative, but can create unnecessary register pressure.

Here is a redesigned teaching example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Configuration generation:
    lease a
    load_address(a, table)
    load_descriptor(a)
    release a

Loop generation:
    lease count
    initialize(count, repetitions)
    loop:
        emit_work()
        decrement(count)
        branch_if_nonzero(count, loop)
    release count

The leases do not overlap, so they may reuse the same physical register. Different source variables using the same machine register is perfectly reasonable. A test that insists on seeing a particular scratch register from an older implementation may misclassify legitimate reuse as a regression.

What the Verified Changes Actually Changed

The relevant commits advanced on three fronts. First, immediate loads, address-configuration loads, and parameter loads gained interfaces accepting resource handles, so callers no longer had to extract register-name strings themselves. Next, arithmetic, clipping, sign, reduction, and lookup kernels moved to scoped handles, with branches, decrements, and stride loads joining the same interface. Finally, hard-coded scratch names were progressively replaced in descriptor loading, matrix computation, convolution, input/output transfers, and recurrent-state computation.

One easily missed detail matters: a compound kernel placed its configuration-loading temporaries inside a separate block and allocated its loop counter only after that block ended. Those extra braces were not decoration. They directly determined whether the two stages could reuse resources.

Another interface wrapped the constant-zero register in a distinct type. Fixed architectural registers and recyclable scratch registers are different resource classes; the former should not be “allocated” from the scratch pool. Once types express that distinction, helper signatures themselves become useful review evidence.

Why a Naming Convention Is Not Enough

A rule such as “inner functions use only the second scratch register” soon meets three counterexamples: the inner function calls another helper; the outer loop gains a second counter; or an address-loading path needs an extra register because an immediate no longer fits. Such conventions amount to manually maintaining a global allocation table across functions. The more the code changes, the more expensive that becomes.

Handles make the allocator the resource ledger. However, if low-level interfaces still accept arbitrary strings, a route around that ledger remains. This migration should therefore be understood as progressively expanding explicit ownership, not as proof that the appearance of RAII eliminated every register conflict.

A sound interface policy is to give ordinary kernels handles, retain raw register representations where the encoder needs them, provide a separate entry point for fixed registers, and pass existing handles by reference when helpers borrow them. That avoids quietly allocating new resources during a borrow.

Resource Safety Does Not Mean Generation Can Be Rolled Back

Suppose a kernel has emitted ten instructions when the eleventh encounters an out-of-range field and throws. Local handles are destroyed and the scratch pool recovers. That prevents a resource leak. The first ten instructions may still be in the instruction stream.

Two questions must therefore be answered separately:

1. Are temporary resources returned after failure?
2. Are instructions, labels, dependency counts, and other generation state restored after failure?

RAII can address the first. The second requires transactions, snapshots, or a protocol that discards the entire context on failure. Confusing the two can leave error recovery producing an apparently complete half-product. Error Boundaries examines this boundary further.

Likewise, a scalar configuration register’s lease does not automatically settle the lifetime of an asynchronous completion event. A device queue may still be executing a command. Reusing the associated synchronization resources must follow their own semantics.

How to Discuss Performance Benefits

It is reasonable to infer that shortening host-side handle lifetimes can reduce the number of simultaneously occupied scratch slots and make complex kernels easier to compose. Replacing strings with handles, however, does not by itself imply shorter execution time. Ideally, the generated instruction sequence is almost unchanged. The main gains are correctness, maintainability, and the ability to support more complex compositions.

Register-number changes sometimes affect binary snapshots. First establish what may change: is the encoding valid, is runtime behavior equivalent, and has peak resource usage fallen? If the goal is to preserve existing machine code exactly, either explain why a numbering change is acceptable or deliberately constrain allocation order.

There may also be a compile-time cost. Handle allocation can involve set lookups or dynamic object management. Whether that matters depends on the data structures, kernel granularity, and generation frequency. It requires measurement; the phrase “modern C++” does not make it free.

One More Question: Can the Handle Be Copied?

Once a handle is passed to a helper, ask whether it owns the resource or merely borrows it. If two copyable objects both believe they must return the same slot, the first destructor may make that slot allocatable again while the second object still looks usable. A teaching design would usually make ownership handles move-only, with helpers accepting references to express “I borrow this; I do not release it.”

After a move, the original handle must also enter a clearly empty state. Otherwise, a debug print may still show a register name, encouraging callers to treat it as valid. These are resource-interface review questions. A few kernel migrations alone cannot establish that every move, invalidation, and double-release behavior below them has been fully tested.

A small review exercise can help: an outer scope owns a counter; an inner helper borrows it for a decrement; another helper allocates an independent address temporary; then a failure path exits early. Check who returns what on each path, and draw the generated register-use intervals on one timeline. Looking only at normal return can miss the branch that actually creates a conflict.

There is another trap that looks like a performance optimization: hoist several handles to the outermost function scope to avoid repeated allocation. The number of allocations falls, but the number of simultaneously live resources rises, potentially reaching the limit sooner. Measure peak leases and actual allocation overhead, not just the number of calls to acquire. In resource management, fewer calls do not always mean lower occupancy.

Turn Regression Tests into Contracts

A further experiment could construct parameterized nested kernels, gradually increasing loop depth, configuration-loading paths, and simultaneously live temporaries. Record peak register use, emitted instruction count, and compile time, then use a minimal interpreter to check final counter values. Compare two generation strategies with the same semantics, rather than models containing different numbers of operations.

One useful code-review question remains: when you see auto temp = acquire(), ask not only whether release is automatic, but whether the scope covers the generated program’s final use. Ownership now has a name. Lifetime is the next part of the story that must be made explicit.

Back to series contents · Next

• ← Previous Post
• Next Post →