dev/shlibdeps: New page

1 files changed, 78 insertions, 0 deletions

diff --git a/dev/shlibdeps.mdwn b/dev/shlibdeps.mdwn
new file mode 100644
index 0000000..e3003ed
--- /dev/null
+++ b/dev/shlibdeps.mdwn
@@ -0,0 +1,78 @@
+[[!meta title="Automatic Shared Library Dependency Substitution Variables"]]
+
+Background
+==========
+
+Currently, binary packages' control files (`<binpkg>.pkg/control` in source
+packages) must explicitly list library dependencies.  Package maintainers can
+manually see ELF files' `DT_NEEDED` entries with tools like **readelf** or
+**ldd** and figure out which binary packages provide each SONAME (usually just
+dropping `.so` from the middle of the SONAME, e.g. `libz.1` provides
+`libz.so.1`).  Being a manual process, this method is error-prone: it's an extra
+step that can be forgotten when preparing a new package or updating to a new
+upstream version, it's possible to miss some ELF files in binary packages, etc.
+An automated solution is clearly desirable.
+
+Debian achieves this with a tool called **dpkg-shlibdeps**, which:
+
+ 1. Finds all ELF files in each binary package,
+ 2. Finds all libraries against which each ELF file is linked (i.e.
+    `DT_NEEDED`),
+ 3. Finds dependency libraries on the system (resolving SONAMEs to file names),
+ 4. Looks up the packages that provide each dependency library (with
+    `dpkg -S <lib-file-name>`), and
+ 5. Checks whether each ELF file uses symbols from its library dependencies (and
+    warns of any useless/avoidable dependencies).
+
+ProteanOS should have a similar tool.  This document describes how this can be
+implemented.
+
+Implementation
+==============
+
+Changes need to be made in three places in ProteanOS: the Source Package Format
+specification, opkbuild, and opkhelper.
+
+SPF 2.0
+-------
+
+SPF 2.0 needs to be amended to specify `tmp/<binpkg>.substvars` files, like the
+`substvars` file but generated during the build process (while commands from the
+`build` makefile are run and before opkbuild tools complete the package
+build) and specific to each binary package.
+
+opkbuild
+--------
+
+opkbuild needs to read the `tmp/<binpkg>.substvars` files and use them when
+generating binary packages' `control` files.
+
+opkhelper
+---------
+
+opkhelper needs a new tool, which will be called **oh-shlibdeps**, to be run
+after **oh-installfiles**.  This new tool will:
+
+ 1. Find all ELF files in each binary package (**oh-strip** already does
+    something similar),
+ 2. Run the C library's **ldd** tool and parse its output (e.g. with
+    `sed -n 's/^\t.* => \(.*\) (.*)$/\1/p;'`) to find the file names of all
+    libraries against which each ELF file is linked (a combination of steps 2
+    and 3 above), and
+ 3. Look up the packages that provide each dependency library (with
+    `opkg search ${filename} | sed 's/ - .*$//;'`).
+
+Detecting useless/avoidable dependencies is considered outside the scope of the
+current proposed design and may be considered in the future.
+
+ProteanOS's `opkhelper-3.0` binary package will need to ensure the existence of
+the **ldd** tool.  **ldd** is provided by the `libc-bin` package on the
+`any-any-glibc` architectures (currently all architectures in ProteanOS).
+musl's RTLD (a.k.a. dynamic linker) has **ldd** functionality built-in and
+enabled when executed as `ldd`, so no separate package will be required.  So the
+needed dependencies will depend on the architecture, however `opkhelper-3.0` is
+`Architecture: all`, which means it can't have architecture-dependent
+dependencies (which would be solved at build time by opkbuild, not at run time
+by opkg).  Therefore, `opkhelper-3.0` will likely have to depend on
+`libc-bin | ldd`, which will work on either `any-any-glibc` or `any-any-musl`
+architectures if the musl library binary package `Provides: ldd`.