I Waited for the Event. Why Isn't the Data Ready?

Series contents · Resources and Scheduling · 阅读中文版

“I added a wait. How can it still read gate values that haven’t finished computing?”

“Which event are you waiting for?”

“A ready event.”

“What actual work does it cover?”

That last question often matters more than the ten lines of synchronization code before it. An event’s name expresses an intention; the device executes a protocol. A marker called “data ready” can represent readiness only when it has the right relationship to the commands that produce the data.

A recurrent cell offers two kinds of parallelism

Consider this teaching version of a set of gate computations:

1
2
3
4

a_t = W_x * x_t
b_t = W_h * h_(t-1)
gate_t = activation(a_t + b_t)
h_t = combine(gate_t, ...)

The input projection a_t does not read the newly updated hidden state, so there may be an opportunity to compute it early. The recurrent projection b_t must wait for the previous iteration’s state update to finish. If matrix engine M and vector engine V run asynchronously, a good schedule tries to keep M doing state-independent work while V updates the state.

This suggests an appealing order: issue a_t, wait for the previous hidden state, issue b_t, and finally signal a “gate values ready” event. The order appears to use idle time well, and the mathematical dependency graph still contains all its edges.

But that graph cannot tell us whether a synchronization event means “all preceding work has finished” or tracks only the commands inside an explicit range.

Why an empty dependency range is dangerous

Suppose a teaching device binds an event to the actual commands between begin and end. Then this code:

1
2
3
4
5

issue work_A
issue work_B
begin event_E
end event_E
other_engine waits event_E

does not necessarily wait for work_A and work_B. There is no actual work inside the event’s range. The device may consider it already complete, or it may represent only an empty batch. The wait itself is perfectly legal, yet it promises none of the conditions the reader actually needs.

This protocol is a teaching abstraction, not an explanation of every event API. In some systems, an event placed at the tail of a queue really does cover all preceding commands; others require an explicit dependency range. Engineering work requires reading the target protocol and its implementation. Sharing the name “event” does not imply shared semantics.

The verified commit comment explicitly states that the original empty dependency range could not guarantee that the vector side would observe both gate computations as complete. In the relevant recurrent branch, the fix changes the order: wait for the hidden state, issue the recurrent projection, open the readiness range, place the other input projection inside that range when it needs to run, and then release the event. On the ordinary path, the range now encloses real matrix work.

What opportunity does this change give up?

The earlier arrangement tried to move the projection that does not read hidden state forward, keeping the matrix engine busy during the tail of the vector update. Moving that work later may reduce this overlap.

This is a familiar correctness and performance tradeoff. A plan that appears more parallel has no usable performance if its completion notification does not cover the actual writes. Restoring a provable order first, then looking for legal overlap, establishes a baseline that can be compared meaningfully.

There is also a limit to the historical evidence: the fix is small, and two commits with different titles contain the same file diff. That does not support a story of two independent incidents in which one problem was discovered and another was then solved. A public account should treat them as repeated appearances of the same technical fix in the history unless additional evidence establishes different circumstances.

One fix cannot vouch for every mode

Recurrent kernels commonly have branches for the first step, later steps, fused gates, precomputed input projections, and extra transfers. This change affects one recurrent stage, and issuing the input projection remains conditional.

The supported conclusion is therefore specific: the command order on this path changed, and the accompanying rationale was to make real commands provide the event’s completion guarantee. It does not establish that empty ranges disappeared from every configuration, or that fused and precomputed modes received the same level of verification.

That is why regression coverage should be organized by mode. Testing only the ordinary recurrent path may miss a condition such as “this iteration’s projection was precomputed, so the command that would have occupied the event range is omitted.” A mode switch changes more than the amount of arithmetic. It can change the evidence that makes synchronization valid.

Treat an event as a proof you can inspect

For each synchronization object, a review can answer four questions:

1. Which results does it prove are readable?
2. Which commands actually write those results?
3. Which queue ordering or dependency range connects those commands to the event?
4. Which consumers rely on it before starting?

An event deserves scrutiny if the first answer is broad but the second identifies no concrete commands. If it has multiple waiters, review its consumption count and reuse timing as well, so that an event from one iteration is not mistaken for the next.

In a loop, it helps to give the event a conceptual iteration number, E_t. Even when a physical slot is reused, there should be an explanation for why the slot takes on E_(t+1) only after all consumers of E_t have finished. Checking that registers eventually return to zero cannot, by itself, prove correct semantics across iterations.

Build a new minimal counterexample

Make the previous hidden-state update slow, the input projection fast, and the recurrent projection moderately slow. If the vector engine waits for an event that does not cover the real matrix work, it may read the gate values before the recurrent projection finishes.

A simulator need not perform actual matrix multiplication. It can write version numbers to the output regions: the input projection writes input_version=t, the recurrent projection writes state_version=t, and the gate function checks that both versions equal t when it starts. This exposes the timing error of reading an old result even when the eventual numerical values happen to match.

Next, shorten or lengthen the different engines’ execution times and explore legal queue interleavings. If the error appears only at a particular ratio of execution speeds, that explains how a fixed-latency model might hide the synchronization gap.

Why counting waits can lead you astray

Suppose an optimization reduces five waits to four. That number alone cannot tell us whether it removed redundant synchronization or lost a necessary completion relationship. Conversely, adding a wait can reduce total execution time: if the old program had circular waiting or prolonged blocking, establishing the correct producer-consumer order may let the queues make steadier progress.

A more reliable debugging method is to identify the last writer for every read, then inspect the proof path between them. That path may include producer queue order, event completion, the consumer’s wait, and consumer queue order. The read is protected only if the entire path is valid. A reassuring event name cannot replace a missing segment.

Loops add another question: “Which version?” The gate computation in iteration t may indeed wait for a hidden-state update, yet its numerical dependency is still wrong if it receives a reused notification from an earlier iteration. Recording physical event slots separately from logical event versions often makes logs much easier to interpret.

Without timing observations from a real device, at least list the commands emitted in each mode, mark the work removed by conditional branches, and inspect whether the event range consequently loses its meaning. This does not replace hardware validation, but it can expose obvious empty ranges and mixed-up versions before running on a device.

Performance experiments should follow these semantic checks. Otherwise, a shorter execution time might simply mean that a consumer read an old result early. An incorrect program spending less time waiting is no optimization achievement worth preserving.

Regression matrix and performance observations

For performance, record total latency, matrix-engine idle intervals, vector waiting time, and synchronization command count together. A busier queue does not mean a faster model. It may merely be doing work earlier that will still be followed by a wait.

Future optimization could investigate moving the input projection earlier where legal while supplying a reliable completion guarantee for the recurrent projection, or using explicitly defined queue-tail semantics. These are design directions, not optimizations established by the commit.

“I already waited” is not the end of a synchronization proof. The next sentence must answer: “Why does this wait cover the writes I am about to read?”

Back to series contents · Previous · Next

• ← Previous Post
• Next Post →