Series contents · Engineering and Delivery · 阅读中文版
The tool works in the development directory. Unpack its installation bundle somewhere else and convolution still works; several simple operators work too. Then a nonlinear activation reaches Python, which cannot find a module.
“But we packaged the executable, didn’t we?”
An executable is not necessarily the whole compiler. It may embed Python and call a lookup-table parameter generator that lives with the source tree. If the source path was embedded at build time, the program resembles someone who moves house but keeps ordering dinner to the old address.
Why a basic smoke test missed it
--help can show that a process starts. It cannot show that every compilation path can find its runtime resources. Compiling a model containing only addition may never invoke table generation.
Lowering a nonlinear operator often requires converting a mathematical function into approximation parameters. If a Python module performs that work, the failure appears only when fitting is actually needed. The binary itself can load normally. What looks like an operator failure is a missing resource-location contract.
The reviewed change removed a source-tree absolute path from compilation definitions. It searched for the companion module using the executable location and Python search paths instead, while installation and packaging checks made the module a required artifact.
Which starting point can you trust?
Three common resource-search starting points are the working directory, the source directory, and the installation directory.
The caller chooses the working directory. Running a compiler from arbitrary folders is normal, so treating that directory as the installation location is fragile.
The source directory is convenient for developers but should not become an implicit release dependency. A distributed package may have no source tree beside it. If it does, that tree may be another version.
The tool’s own directory more naturally describes an installed layout. If the executable lives in prefix/bin and modules in prefix/python, their relationship survives relocation of the whole package. Development trees may use a different relative arrangement; support those arrangements explicitly instead of searching the entire disk.
The general mechanism can be written as:
1 | Locate the running tool |
This is a teaching model of the mechanism, not a line-by-line reproduction of the implementation.
Who is the main program when Python is embedded?
Another entry point is easy to miss: a Python extension calls the compiler. The process executable may then be the Python interpreter, not the compiler tool inside the bundle.
Deriving every resource path from the main executable can fix native command-line use while leaving Python bindings broken. The reviewed code also considered the host Python search path and checked its directories for the companion module.
Searching those paths does not mean trusting every candidate blindly. Confirm that the target file exists rather than adding many invalid directories. Handle Python API errors and non-string path entries. Adding a path already present has no value.
Search order is behavior too. The priority among explicit environment paths, package resources, and the interpreter environment determines which same-named module wins. The evidence establishes several reasonable location sources. It does not establish that the wrong module version can never be imported. Stronger identity requirements would need separate version or origin checks.
The negative test is the interesting part
The strongest regression is not another run inside the source directory. It deliberately breaks the old assumption.
The reviewed test created a temporary directory, placed the tool and required Python modules into a new relative layout, cleared several source-tree-related environment variables, and compiled a path that actually generates table parameters. It also checked the imported module’s origin so the program could not silently return to the original source tree.
Then it hid the module in the relocated package and ran again, expecting a clear import failure.
That second half matters. If removing the packaged module leaves the test successful, another location is helping. The success is evidence that the test has not isolated the dependency it intended to verify.
The test therefore asks both: “Can the correct resource work at its new location?” and “When it is absent, does the program quietly borrow an old copy?”
The test has boundaries too
The reviewed test retained a configured native-library search path because its target was Python resource relocation. It cannot be described as proving that the entire release package is independent of the development environment.
This is precisely why test scope matters. A focused resource-location test need not solve every ABI, system-library, and runtime-compatibility question at once. Its report must not claim that it does.
Separate the layers:
| Layer | Check |
|---|---|
| Native loading | Required shared libraries for binaries and extensions resolve |
| Python location | Imports come from the target installation rather than the source tree |
| Real functionality | At least one genuine table-generation path executes |
| Missing-resource diagnosis | Absence of the module causes a clear failure |
| Multiple entry points | Command-line and Python-hosted use are covered separately |
The layers are related but not interchangeable. Native loading does not prove that a Python table module exists. Successful table generation does not prove approximation accuracy for every nonlinear function.
Don’t finish the investigation by making PYTHONPATH enormous
Adding every possible directory to an environment variable often makes the immediate failure disappear. It also makes wrong-version imports harder to detect.
When development and installation trees coexist, an edited Python file in the development tree can change the behavior of the release package, while other users do not have that edit. This state is harder to reproduce than a clean error: the program depends on something missing from its delivery manifest.
A clearer policy defines a small set of supported layouts, installs required resources with the package, checks identity or origin as appropriate, and exposes explicit overrides as documented behavior. If the resource cannot be found, report a useful failure instead of falling back indefinitely.
Required-file checks during installation are useful for the same reason. Discovering an omitted module before delivery saves the user from reaching a particular operator before learning that the bundle is incomplete.
How to discuss the performance of relocation support
Path checks and one-time imports may add startup cost, but intuition cannot establish that they are a bottleneck. Fitting, parsing, and total compilation often operate on different time scales. Measure before optimizing.
A proposed experiment would time the first import, repeated calls in one process, table-parameter generation, and total compilation separately. If results are cached, the cache key must include the input domain, quantization parameters, and approximation mode. A shared function name alone does not justify reusing a table.
It repaired the relationship between a resource’s identity and its location. A compiler can move because its dependencies are expressed, not because it happens to remember the developer’s old address.
Series contents · Previous · Next
If you like this blog or find it useful for you, you are welcome to comment on it. You are also welcome to share this blog, so that more people can participate in it. All the images used in the blog are my original works or AI works, if you want to take it,don't hesitate. Thank you !