Series contents · Engineering and Delivery · 阅读中文版
“The source has not changed. The installation tree has not changed. Why does today’s archive have a different hash from yesterday’s?”
The first suspicion is usually changed file contents. Extract both archives and compare files individually: the contents match. What changed may be file order, timestamps, ownership, or extra time information written by the compression layer.
A release package carries the program and considerable metadata about how it was packed.
Two different meanings of “the same”
Semantic equality means the tools satisfy the same behavioral contract. Byte equality means the complete archives match bit for bit. The second is stricter, but still does not prove that the source build is reproducible.
If installation trees differ, normalizing tar metadata will not turn different programs into the same program. Conversely, identical installed contents can still produce different archive hashes when metadata is not normalized.
Separate the layers:
1 | Source, toolchain, dependencies, build arguments |
Every layer has its own determinism problem. Repairing the bottom layer does not establish end-to-end reproducibility.
A release package has two clocks
The first clock belongs to package contents. To make identical inputs produce identical archives, choose an explicit reference time and normalize member timestamps. The release process might supply that time, or a convention might derive it from a source revision.
The second clock belongs to the archiving event. A package placed into release storage today has an event time of its own. Mechanically preserving source timestamps can make files in the archive directory appear to belong to a different moment.
The reviewed history handled these separately. A small change altered timestamp preservation during archive copying. Later packaging work introduced a common reference time, fixed ownership, sorting, and compression without unnecessary timestamp metadata. These changes do not reduce to “always preserve timestamps” or “always zero timestamps.”
A timestamp should serve the meaning of the object it describes, rather than exist merely because a copy command has another flag.
Why does packaging need a staging directory?
Changing permissions, repairing library links, or deleting files directly in the installed tree makes packaging mutate the development environment. After a failure, the next build may encounter an installation directory that was only partly processed.
Staging has a clearer responsibility: copy the completed installation tree, normalize and validate an isolated temporary copy, then create the archive. Success or failure can clean up the copy without using the development installation as a workbench.
Likewise, writing a temporary archive, checking its compressed structure, and then publishing its final name reduces the chance that readers see a partial file. That sequence should not automatically be advertised as a complete transaction. Cross-filesystem moves, concurrency, and overwrite policy affect its guarantees.
The reviewed script rejected an existing archive name before moving the completed temporary file. This expresses an intent not to overwrite an existing release. A check-then-move sequence does not inherently provide an atomic guarantee under concurrency. Stronger coordination requires a separate design.
Compressible does not mean ready to release
The reviewed release checks covered tool entry points, Python extensions, required directories, permissions, library links, and dynamic dependencies. Archiving added checks for compression integrity, top-level paths, required members, and agreement between changelogs inside and outside the archive.
These checks establish whether the delivery set follows its contract, not whether model calculations are accurate. Each check can seem mundane. Together, they determine whether someone else receives the thing that was actually evaluated.
For example, a changelog edited after testing may describe new functionality while the package still contains the old program. Nothing has to crash for acceptance testing to lose its common reference. Comparing changelog contents catches one class of version-pairing error. A stronger relationship also needs artifact identifiers and test records.
Path checks should be described with the same precision. Requiring members under the intended top-level directory and rejecting parent-directory traversal are useful constraints. They are not a complete extraction sandbox for arbitrary hostile archives. A handful of checks should not become a claim that every possible threat is solved.
Why development dependencies should not automatically become release dependencies
A development environment may contain training frameworks, plotting packages, conversion tools, debuggers, and old experimental libraries. A user running one model-compilation workflow may not need that entire environment.
A separate delivery manifest can define dependencies around actual entry points: what parses the model, what performs conversion, and what loads a native extension. The history included a move from a development manifest to a release manifest, along with version adjustments.
Pinning versions is not proof of compatibility. Python versions, platforms, and underlying ABIs can still change behavior. If development and release manifests deliberately differ, it becomes more important to state which workflow was validated in which environment.
How should reproducibility be measured?
Start by packaging the same installation tree twice with the same release name and reference time, then compare archive hashes. Vary one dimension at a time: member creation order, host ownership, or original file timestamps. Check whether normalization removes the expected differences.
Then move to the harder layer: rebuild from a clean build tree and compare installed artifacts. If they differ, examine debug paths, build identifiers, generated-file ordering, and toolchain inputs separately instead of continuing to tweak tar arguments.
If the default release name includes the current date, fix it before comparing supposedly identical inputs. Otherwise, the inputs themselves have changed and the hash comparison lacks a clear reference.
The performance bill for a release
Copying the installation tree, scanning dependencies, reading archives, and computing checksums all consume time and storage. Repeated scans may be optimized, but saving packaging time is not a good reason to remove the checks most likely to detect a mismatched release.
To measure the costs, record copying, normalization, compression, validation, and archiving separately, with cache state, storage, and compression settings. Without data, a script refactor cannot be credited with higher throughput. More reliable delivery is also not a model-inference speedup.
Two clocks, two kinds of equality, and several completion boundaries may sound like minor file-management details. When someone asks, “Is this the version we just tested?”, they become essential technical evidence.
Series contents · Previous · Next
If you like this blog or find it useful for you, you are welcome to comment on it. You are also welcome to share this blog, so that more people can participate in it. All the images used in the blog are my original works or AI works, if you want to take it,don't hesitate. Thank you !