[[!meta title="Automatic Shared Library Dependency Substitution Variables"]] Background ========== Currently, binary packages' control files (`.pkg/control` in source packages) must explicitly list library dependencies. Package maintainers can manually see ELF files' `DT_NEEDED` entries with tools like **readelf** or **ldd** and figure out which binary packages provide each SONAME (usually just dropping `.so` from the middle of the SONAME, e.g. `libz.1` provides `libz.so.1`). Being a manual process, this method is error-prone: it's an extra step that can be forgotten when preparing a new package or updating to a new upstream version, it's possible to miss some ELF files in binary packages, etc. An automated solution is clearly desirable. Debian achieves this with a tool called **dpkg-shlibdeps**, which: 1. Finds all ELF files in each binary package, 2. Finds all libraries against which each ELF file is linked (i.e. `DT_NEEDED`), 3. Finds dependency libraries on the system (resolving SONAMEs to file names), 4. Looks up the packages that provide each dependency library (with `dpkg -S `), and 5. Checks whether each ELF file uses symbols from its library dependencies (and warns of any useless/avoidable dependencies). ProteanOS should have a similar tool. This document describes how this can be implemented. Implementation ============== Changes need to be made in three places in ProteanOS: the Source Package Format specification, opkbuild, and opkhelper. SPF 2.0 ------- SPF 2.0 needs to be amended to specify `tmp/.substvars` files, like the `substvars` file but generated during the build process (while commands from the `build` makefile are run and before opkbuild tools complete the package build) and specific to each binary package. opkbuild -------- opkbuild needs to read the `tmp/.substvars` files and use them when generating binary packages' `control` files. opkhelper --------- opkhelper needs a new tool, which will be called **oh-shlibdeps**, to be run after **oh-installfiles**. This new tool will: 1. Find all ELF files in each binary package (**oh-strip** already does something similar), 2. Run the C library's **ldd** tool and parse its output (e.g. with `sed -n 's/^\t.* => \(.*\) (.*)$/\1/p;'`) to find the file names of all libraries against which each ELF file is linked (a combination of steps 2 and 3 above), and 3. Look up the packages that provide each dependency library (with `opkg search ${filename} | sed 's/ - .*$//;'`). Detecting useless/avoidable dependencies is considered outside the scope of the current proposed design and may be considered in the future. ProteanOS's `opkhelper-3.0` binary package will need to ensure the existence of the **ldd** tool. **ldd** is provided by the `libc-bin` package on the `any-any-glibc` architectures (currently all architectures in ProteanOS). musl's RTLD (a.k.a. dynamic linker) has **ldd** functionality built-in and enabled when executed as `ldd`, so no separate package will be required. So the needed dependencies will depend on the architecture, however `opkhelper-3.0` is `Architecture: all`, which means it can't have architecture-dependent dependencies (which would be solved at build time by opkbuild, not at run time by opkg). Therefore, `opkhelper-3.0` will likely have to depend on `libc-bin | ldd`, which will work on either `any-any-glibc` or `any-any-musl` architectures if the musl library binary package `Provides: ldd`.