The Library Is in the Package. Why Can’t the Program Find It?

Series contents · Engineering and Delivery · 阅读中文版

“Just copy the missing shared library into the package.”

Ten minutes later, loading still fails. The directory definitely contains libcompute.so.4.2, and the file size looks reasonable. Someone even gives it a tidier name. The program keeps asking for libcompute.so.4.

The program is not being fussy. Linking and runtime loading may use different names.

A shared library’s three business cards

A common ELF shared-library layout distinguishes the real file, its runtime identity, and the name used by the linker:

1
2
3

libcompute.so       → libcompute.so.4
libcompute.so.4     → libcompute.so.4.2
libcompute.so.4.2   → actual contents

This is an illustrative convention, not the only valid layout. When a developer links with -lcompute, the linker typically finds the library through its linker name. The resulting program may record the library’s declared SONAME as its dependency. At runtime, the loader seeks that identity, which need not be the path originally passed to the linker.

“Copied the real file” and “made the required SONAME resolvable” are different checks. Conversely, the linker-name entry may exist but point to an absolute build-machine path. The archive looks complete until it is unpacked elsewhere and the link becomes dangling.

The reviewed fix added the linked shared library and its runtime name to installation rules and explicitly created the linker name. Release validation also checked that these names resolved to the same file inside the package. It replaced “I can see a .so” with a dependency relationship that could be verified.

Finding an arbitrary system copy is not a repair

Suppose the build used version A, but packaging finds version B somewhere on a search path. Both happen to be called libcompute.so. The loader’s ability to open the file does not prove interface or behavioral compatibility.

Packaging should therefore obtain the dependency location from the target already resolved by the build system whenever possible, rather than repeat an ambiguous search. CMake imported-target properties and generator expressions can identify target files and SONAME files. Real projects still need to account for target types and platforms; an ELF installation rule is not automatically portable to every operating system.

Headers, libraries, and runtime dependencies form a contract together. Checking one invites the three-act surprise: configuration succeeds, linking succeeds, execution fails.

RPATH is not a universal search directory

Once a dependency name is known, the loader must decide where to find it. Absolute paths convenient in a build tree usually do not belong in a release package.

A common relocation strategy places libraries relative to the tool:

1
2
3
4

bundle/
├── bin/compiler-tool
├── lib/libcompute.so.4
└── python/extension.so

The executable and Python extension live in different directories and may require different relative search paths. In ELF, $ORIGIN is a loader-interpreted marker for the relevant object’s directory, not the shell’s working directory. Passing it through several build-script layers also requires preventing premature expansion.

Still, “uses $ORIGIN” does not mean “independent of the system.” Base system libraries, thread runtimes, the Python ABI, and the specific RPATH/RUNPATH search rules may continue to matter. The reviewed release documentation retained external runtime requirements; it did not promise execution on arbitrary Linux systems.

A positive test may be receiving help from the development environment

Running the tool successfully inside its build container is good news. It is also an easy place to acquire false confidence.

Preinstalled libraries, global loader paths, and environment variables can compensate for missing package dependencies. The test then proves that the package plus the development machine works, rather than that the delivery set meets its contract.

A stronger validation plan would:

1. Copy the package to a fresh temporary location, preserving its intended relative structure.
2. Use an explicit environment that avoids automatically searching build directories.
3. Inspect the actual dependency resolution of key executables and extensions.
4. Confirm that libraries required to ship with the package resolve inside it.
5. Remove such a library in a negative test and confirm that validation fails.

These are general recommendations. The reviewed scripts checked unresolved dependencies and whether a specific library resolved inside the package.

Why check where a link actually points?

A symbolic link can exist while pointing outside the package. Two apparently corresponding names can even resolve to different library versions.

Validation should compare normalized paths after resolving links:

1
2

resolve(linker_name) == resolve(soname_name)
resolve(linker_name) belongs_to bundle/lib

Here, belongs_to is a teaching placeholder, not an invitation to use an unchecked string-prefix comparison. Directory boundaries, symbolic links, and normalized paths matter. For example, /tmp/pkg-lib-old is not inside /tmp/pkg-lib merely because their strings share a prefix.

Likewise, a discovered SONAME should conform to an allowed library-name form rather than contain arbitrary path components. These checks are not features added for their own sake. They establish the specific claim that a name denotes this dependency inside this package.

Discuss runtime binding and installation costs separately

Shipping the real library increases distribution size and adds upgrade and licensing-material maintenance. Depending on the host makes the package smaller but increases environment-matching requirements. The right choice depends on the delivery context, not on which directory looks cleaner.

File counts also tell us little about performance. Less ambiguous resolution can improve startup diagnosis, but it does not establish faster model execution. To study loading costs, measure process startup, dynamic relocation, Python import, and the first computation separately. Steady-state execution is usually a different cost model.

Avoid another tempting “optimization”: copying every missing dependency until checks pass. That can introduce incompatible runtimes and turn a clear host requirement into a private system image that is difficult to maintain.

A small table can settle the argument

The next time someone says, “The library is right there,” continue with:

With those answers, a missing-library error becomes less mysterious. It is a chain of names, paths, versions, and runtime conditions. Each link deserves its own check.

Series contents · Previous · Next

• ← Previous Post
• Next Post →