path: root/dev/multiarch.mdwn

[[!meta title="Multiarch Design"]]

Earlier design proposals are documented in [[another_page|dev/multiarch/design]]
kept for historical purposes.


Background
==========

ProteanOS is a self-hosting binary distribution that builds its own packages.
It is also an embedded distribution, intended to be suitable for small
resource-limited systems.  It may not be practical to natively build all of
ProteanOS on real hardware for every architecture to be supported.  Thus, it
will be preferable to cross-build all of ProteanOS's packages, making ProteanOS
a highly ambitious cross-built self-hosting binary distribution.

Cross-compiling and cross-assembling self-contained code is a solved problem,
however programs typically use separately built libraries, which must be
installed at build time to be available for compiling and linking.  Sometimes,
these dependency libraries will already be installed for use by the native
system.  So it needs to be possible to "coinstall" different architecture builds
of each library.  Package management and build tools also need to be able to
install dependency packages of the correct architecture; for example, libraries
must be of the *host* architecture (the architecture **for** which a package is
being built), but build utilities must be of the *build* architecture (the
architecture **on** which a package is being built).  Further complicating
dependency resolution, such build utilities often themselves depend on
libraries, which must also be of the build architecture (not the host
architecture like libraries that are direct build-dependencies).  These are the
two main problems in cross-building distribution packages: coinstallability and
dependencies.  The solution is a design known as "multiarch".

ProteanOS's multiarch design is inspired by, but different from, that
[specified][wuc-mas] and [documented][wdo-ma] in Debian and Ubuntu.

ProteanOS's design solves the coinstallability and dependency problems without
*any* modifications to its package manager, opkg-lede.  This is desirable
because opkg-lede is a tool used by many distributions separate from ProteanOS,
and its upstream maintainers (OpenWrt) would likely be unwilling to merge and
maintain invasive patches specific to a different distribution (making the
patches impossible to test upstream).

[wdo-ma]: http://wiki.debian.org/Multiarch
[wuc-mas]: https://wiki.ubuntu.com/MultiarchSpec


Design
======

Coinstallability
----------------

opkg-lede doesn't allow multiple packages to provide the same file path, so, in
order to be coinstallable, library packages must install all of their files into
architecture-qualified locations.  [Debian and Ubuntu][wuc-mas-arch-indep-files]
limit this requirement to only files whose contents differ between
architectures.  This is accomplished by the package manager dpkg requiring all
packages of the same name to be the same version, verifying that shared files
have identical contents, and reference counting shared files.  As explained
above, ProteanOS's multiarch design doesn't entail such modifications to the
package manager.

opkg-lede also doesn't allow installation of multiple packages with the same
name and different architectures.  The only way to solve this without modifying
the package manager is to qualify the names of coinstallable packages with
their host architectures.  This will be done by appending a colon followed by
the host architecture to each coinstallable package name, e.g. `libc.6` becomes
`libc.6:amd64-linux-glibc`.  Package names appear in the file system under
`/var/cache/opkg/archives/` and `/var/lib/opkg/info/`, so using a colon in
package names will prevent the use of restricted file systems such as FAT file
systems as the root file system.

Dependencies
------------

opkg-lede allows packages from different architectures to be installed, but
there is no way to select the architecture to install other than blanket
priority values.  Since libraries of different architectures will often need to
be installed during the same installation transaction, as described above, such
a blunt solution is insufficient.

[Debian and Ubuntu][wuc-mas-control-fields] solve this by declaring the kinds of
dependencies each package is able to satisfy (based on the interfaces it
provides: executables and/or libraries).  Again this requires modifications to
the package manager to affect dependency resolution.  ProteanOS's design is
similar, but implemented in a way such that *dependent* binary packages are
responsible for declaring the architectures of *their dependencies* (which can
be done automatically at build time, so package maintainers can focus on
declaring how their packages satisfy dependencies, as in Debian and Ubuntu).

One way to declare dependency semantics is by introducing a new single-purpose
control field like Debian's and Ubuntu's `Multi-Arch`.  However, ProteanOS will
instead use [a control field that serves multiple purposes:
`Section`][spf-fields-bin].  `Section: lib` packages will be automatically made
coinstallable by opkbuild, by architecture-qualifying their names as described
above.  Library dependencies generated by oh-shlibdeps (with [a
fix][oh-shlibdeps-host-libdirs] to be released) will automatically use the
architecture-qualified names.

ProteanOS systems should have only native architecture feeds for sections
`boot`, `dev`, and `util` but may have multiple feeds for sections `dbg`, `lib`,
and `libdev`.  (Sections `doc`, `locale`, and `share` are
architecture-independent.)

opkg-lede's architecture priorities as described above can be useful in certain
specific ways, through a [wrapper script that adds an `-a`/`--host-architecture`
option][opkg-wrapper] (similar to that of apt-get).  First, since multiple
`Section: dbg` feeds may be downloaded and requiring users to
architecture-qualify such packages would be undesirable, such packages could be
installed instead via `opkg -a amd64-linux-glibc foo-dbg` or even simply `opkg
install foo-dbg` for the native architecture.  Similarly, `Section: libdev`
packages as build dependencies could be automatically installed for the host
architecture by prokit supplying an appropriate `-a` option, allowing
maintainers to continue listing unqualified package names in `Build-Depends`
instead of having to add `:${Host-Arch}` to every `Section: libdev` package in
`Build-Depends`.  Therefore, only `Section: lib` packages need to have
architecture-qualified names (due to the coinstallability issue described
above), and neither users nor maintainers are expected to manually install or
list as dependencies such architecture-qualified packages.

[wuc-mas-arch-indep-files]: https://wiki.ubuntu.com/MultiarchSpec#Architecture-independent_files_in_multiarch_packages
[wuc-mas-control-fields]: https://wiki.ubuntu.com/MultiarchSpec#Binary_package_control_fields
[spf-fields-bin]: http://specs.proteanos.com/spf-2.0/fields.html#fields-bin
[oh-shlibdeps-host-libdirs]: http://git.proteanos.com/opkhelper/opkhelper.git/commit/?id=c214c727f60b070aa502b20dcc1ad7e3ce8579f2
[opkg-wrapper]: http://git.proteanos.com/pkg/opkg-lede.git/tree/opkg


Use Cases
=========

**Run-time dependency on a utility:**  Only the native architecture of the
utility dependency is available in the downloaded feed lists, and the dependency
is not architecture-qualified.  The native architecture will be installed.

**Run-time dependency on a library:**  Multiple architectures of the library
dependency may be available, and the dependency must be qualified with the
native architecture.

**Run-time dependency on a language extension:**  Despite technically being a
shared object file, only the native architecture of the language extension
dependency is available in the downloaded feed lists, and the dependency is not
architecture-qualified.  The language extension dependency in turn depends on a
language interpreter, which is also not architecture-qualified.  The native
architecture of both will be installed.

**Build-time dependency on a utility:**  Only the build architecture of the
utility dependency is available in the downloaded feed lists, and the dependency
is not architecture-qualified.  The build architecture will be installed.

**Build-time dependency on a library:**  Multiple architectures of the library
dependency are available, and the dependency must be qualified with the host
architecture.

**Build-time dependency on a language extension:**  Despite technically being a
shared object file, only the native architecture of the language extension
dependency is available in the downloaded feed lists, and the dependency is not
architecture-qualified.  The language extension dependency in turn depends on a
language interpreter, which is also not architecture-qualified.  The native
architecture of both will be installed.


Implementation
==============

File System Hierarchy
---------------------

As explained above, to be coninstallable, library packages must install all of
their files in architecture-qualified locations.  Debian and Ubuntu install
shared object files in `/usr/lib/<triplet>` or `/lib/<triplet>`, where
`<triplet>` is a [GNU system type][ac-systemtype].  Since multiple ProteanOS
architectures may share the same GNU system type, ProteanOS architecture names
are instead used in such library directories.

ProteanOS's GNU Binutils and GCC packages have been patched and configured to
use multiarch library paths since their initial releases in 2014.  opkhelper has
used multiarch library paths when configuring packages since version 3.0.0-beta1
released in 2012.

[ac-systemtype]: https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/System-Type.html