Why Does Reshape Need Code? The Elements Stay the Same; Their Route Changes

Series contents · Operators and Layouts · 阅读中文版

“Not a single element is missing. Why does the compiler say this is unsupported?”

Reshape hears this accusation often. The model author sees the same number of elements; the backend may see different physical row strides, different memory-block boundaries, and a movement instruction that can express only particular ways of combining dimensions. Neither side is mistaken. They are simply standing on different floors.

Proving the Linear Element Mapping

Our teaching input is [3,10], and the output is [3,2,5]. In contiguous row-major order, the linear index stays unchanged:

1
2
3

Linear position of input index [a,j] = a*10 + j
Let j = b*5 + c
Linear position of output index [a,b,c] = a*(2*5) + b*5 + c

This example splits the last dimension while preserving the leading 3. Another example, [6,5] -> [2,3,5], splits the first dimension while preserving the trailing 5. Both contain 30 elements, but they change structure along different axes. A backend with limited capabilities may need a different mode for each.

Equal element counts are therefore only the first admission ticket. We must also establish usable shapes, dimension products that cannot overflow, compatible input and output element types, dimensions representable by the target’s shape descriptor, and exactly one expressible transformation. The inspected implementation explicitly rejected runtime shape inputs and an unimplemented grouped code-generation path. Removing those rejections would only postpone the error.

Effective Rank and Supported Shapes

Effective rank is the next easily misunderstood term. It does not simply mean deleting every 1. The historical implementation first stripped matching outer dimensions and matching inner dimensions from the input and output, retaining the part that changed, then handled leading unit dimensions in the remainder. The aim was to isolate the core transformation the instruction actually needed to perform.

For example, an unchanged batch dimension should not disguise a core “split one dimension into two” problem as “three dimensions become four.” But unit dimensions in the middle can participate in defining axis positions; their value of 1 does not make them unconditionally disposable. If a later layout maps dimensions to different physical directions, deleting the wrong 1 can select the wrong instruction axis. Details such as the order of stripping and retaining at least one dimension prevent the shape from being peeled down to nothing.

A small table makes the proposed support range explicit:

This table is much more honest than “supports arbitrary reshape.” A target instruction may cover some of these relations without being able to rearrange any two tensors with equal element counts. For example, [3,10] -> [2,3,5] preserves the count but does not merely split one input dimension while leaving the other side unchanged. A target offering only the native modes above needs further decomposition or an explicit rejection.

The historical changes added effective two-to-three and three-to-two transformations to the classification, with relationship checks for the two dimensional paths. Axis selection also changed from trying just one direction to enumerating candidates and requiring a unique answer. This is a good design signal: mode selection should follow a proof, not a default of “use whichever we encounter first.”

What happens when the answer is ambiguous? A teaching example can include several unit dimensions so that multiple modes hold algebraically. Some implementations might use explicit axis information to disambiguate; others might reject the case. The key behavior in the inspected version was to report an error when candidates were not unique. An attribute with “flatten” in its name does not establish that it forces axis selection. Follow the actual decision path.

Another validation layer is often overlooked: if the graph layer already approves the operation, why must the lower-level operator library approve it again? Because independent tools, tests, or other code-generation entry points may call that library. It cannot assume that every caller is a trusted acquaintance who has passed upper-layer checks. The historical change expanded both rule sets together, preventing a gap where the upper layer gained support while the lower layer still threw an exception. Duplication is not the ideal endpoint—a shared pure function could be considered later—but removing a validation layer does not establish a unified contract.

Views, Movement, and Regression

When input and output share contiguous memory, changing descriptive information may indeed suffice. A device layout with padding or a requirement for physical movement changes that answer. Imagine physical rows aligned to A bytes, with only 5 logical bytes per row. Treating two logical rows as a compact vector of length 10 will read the intervening padding as elements if the shape alone is changed. Reshape’s cost depends on layout compatibility, not on whether its name sounds lightweight.

This leads to useful performance questions: which reshapes are zero-copy views, which require movement, and which force neighboring operators to change layout? A suggested study would record actual bytes moved and temporary-buffer sizes, comparing an isolated reshape with one absorbed into neighboring operators. A seemingly cheap reshape may become a bottleneck by breaking fusion; an explicit copy may instead enable a more efficient downstream layout.

Tests must cover both acceptance and rejection. Accepted cases should distinguish inner splits, outer splits, reverse merges, and compatible unit dimensions. Rejected cases should include unequal element counts, equal totals that fail the mode relations, invalid dimensions, overflow, and missing memory information. Historical tests changed an old “unsupported” case because the added capability made that former negative case valid. Negative tests have a lifecycle too; outdated restrictions should not be used to prove a new implementation wrong.

A useful teaching regression fills each linear position with its own index: 0,1,2,.... After reshaping, restore the sequence through the inverse mapping and check every element. This catches reordering; combining it with actual-stride checks distinguishes logical indexing errors from physical-padding errors. Fill everything with 1, and many errors will politely hide.

A historical change also switched the execution mode of a model-compilation example. That shows a change in the debugging entry point, but proves neither numerical results nor performance of a reshape on a real device. It can be an investigation clue, but it cannot support an invented “final speedup percentage.”

Boundary Conditions and Inverse Checks

Here is another useful pencil-and-paper question: if input and output differ only by a few unit dimensions, why retain their original complete shapes? Because “selecting the core transformation” and “describing the entire tensor to move” are separate responsibilities. The core shape helps choose a mode; the complete descriptor still needs every dimension and address to be correct. Simplifying the complete descriptor to the stripped core shape may move only one core block and omit repetitions in the outer dimensions. The inspected implementation retained these two forms separately.

Boundary checks also include integer arithmetic itself. Before comparing input and output element counts, a program usually multiplies their dimensions. If the products have already overflowed, the comparison can still accept two identical wrong integers. A reliable approach checks, before multiplication, whether the current product exceeds the representable maximum divided by the next dimension. Reject nonpositive dimensions first, so that this division check remains meaningful. The historical implementation explicitly included an element-count overflow check. Local products in axis relationships also need justification from preceding dimension constraints, rather than an assumption that every integer multiplication is safe.

“If we cannot recognize the reshape, why not flatten first and then reshape?” At the high-level logical level, that is often a viable decomposition. But the backend still needs a general flattening movement, enough room for the intermediate buffer, and correct preservation of quantization encodings. The decomposition may turn one unsupported transformation into two unsupported transformations, or add another full copy. Automatic fallback should be an explicit legalization rule with visible costs and failure reasons, not a hidden default branch at the end of mode selection.

Diagnostics can become practical tools as well. “Unsupported reshape” leaves the user guessing shapes repeatedly. “Unequal element counts,” “unrepresentable dimensions,” “unsupported effective transformation,” and “multiple legal axis candidates” each point toward a different response. A teaching recommendation is to include the original shapes, effective shapes, and candidate modes in the error. These aid reproduction without exposing private backend addresses.

Finally, test the reverse direction: for a supported split, construct its corresponding merge and check that the element sequence is restored. That alone does not prove every intermediate layout correct, because two faulty implementations can cancel each other out. Compare with an independent linear-index reference as well. Combining bidirectional checks with an independent reference is more robust than checking only the final shape or only a pair of inverse transformations, and gives the teaching example a real verification role.

The answer is not that reshape is complicated and nothing can be done. Break the broad promise into provable relationships: which dimensions stay fixed, which dimension splits, how linear order is preserved, and how physical addresses advance. Once that proof is clear, the code and its diagnostics can take the same shape.

Series contents · Previous · Next

• ← Previous Post
• Next Post →