diff options
Diffstat (limited to 'dev/archive')
| -rw-r--r-- | dev/archive/mirroring.mdwn | 108 | ||||
| -rw-r--r-- | dev/archive/signing.mdwn | 97 |
2 files changed, 120 insertions, 85 deletions
diff --git a/dev/archive/mirroring.mdwn b/dev/archive/mirroring.mdwn index 9d2b38a..deabb91 100644 --- a/dev/archive/mirroring.mdwn +++ b/dev/archive/mirroring.mdwn @@ -1,53 +1,117 @@ [[!meta title="Mirroring the ProteanOS Package Archive"]] -Running a mirror of the ProteanOS package archive contributes to the +Running a public mirror of the ProteanOS package archive contributes to the availability of ProteanOS. -You can keep your mirror private or make it an official mirror. +Organizations deploying many ProteanOS systems are encouraged to make local +mirrors, which they may keep private or make public, to save bandwidth on both +their and ProteanOS's infrastructure. Setting up a Mirror =================== -A mirror takes about 700 MiB of storage space as of this writing and should be +Overview +-------- + +*(Impatient mirror administrators may skip this informational section and read +the instructional sections below.)* + +A mirror takes about 730 MiB of storage space as of this writing and should be expected to grow beyond that. -The following command will list files in the archive: +The curious may view a list of files in the archive with the following command: $ rsync --recursive --exclude .db rsync://files.proteanos.com/proteanos -The following command will mirror the archive: - - $ rsync --recursive --times --delete --exclude .db \ - > rsync://files.proteanos.com/proteanos /path/to/your/mirror/directory/ +See below for commands to copy archive files to a local mirror. -Packages are moved from incoming into the archive four times daily. The +Packages are processed from the incoming queue into the archive four times +daily, so mirrors need not be updated more frequently than that. The [[archive_management_software|dev/pro-archman]] is [configured][arch-conf] to keep unreferenced files in the pool for one day, so mirrors must be updated at least once a day. Otherwise, during the update, some feed index files may -reference deleted files. +reference deleted files. Updating at least twice daily is recommended, in case +the synchronization source is ever temporarily unreachable. + +Serving the archive mirror at `/pub/proteanos` over HTTP and FTP is recommended, +but any path and either protocol may be used. HTTP Strict Transport Security +(HSTS) must not be used except on private mirrors used by ProteanOS systems +known to install the `wolfssl-util` package. The ProteanOS package archive is +already [cryptographically verified][archive-signing] using software smaller +than a TLS implementation. + +Mirrors are classified by their synchronization source as either "primary" or +"secondary" as described below. This design distributes synchronization +bandwidth usage more efficiently across the mirror network. Secondary mirrors +will typically update a few hours after primary mirrors do, but are otherwise +still "first-class citizens". All mirrors regardless of classification are +treated equally by [[dev/prokit]], the installer utility that selects a mirror. + +Please configure private mirrors as secondary mirrors, so as to reserve +ProteanOS project bandwidth for those that contribute bandwidth back to the +project. [arch-conf]: http://files.proteanos.com/pub/proteanos/conf +[archive-signing]:http://lists.proteanos.com/proteanos-dev/2019/04/msg00008.html + +Primary Public Mirror +--------------------- + +Primary mirrors are those that synchronize directly from +<files.proteanos.com> and must serve their contents publicly over HTTP +(without HSTS) and rsync and may also serve over anonymous FTP as defined by +[IETF RFC 1635][rfc1635] with no password requirements. + +Run the following command one to four times (four times preferred) daily to +synchronize your mirror: + + $ rsync --recursive --times --delete --exclude .db \ + > rsync://files.proteanos.com/proteanos /path/to/your/mirror/directory/ + +Secondary Public or Private Mirror +---------------------------------- +Secondary mirrors are those that synchronize from a primary mirror and serve +their contents either publicly or privately. Public mirrors must serve over +HTTP (without HSTS) and may also serve over anonymous FTP as defined by [IETF +RFC 1635][rfc1635] with no password requirements. Private mirrors may serve +over either protocol. -Making an Official Mirror -========================= +Run the following command one to four times (four times preferred) daily to +synchronize your mirror: + + $ rsync --recursive --times --delete --exclude .db \ + > ${src} /path/to/your/mirror/directory/ -Official mirror sites should make their archive mirrors available at -`/pub/proteanos` over HTTP and FTP. Sites should also make their archive -mirrors accessible over rsync, if possible. +Where `${src}` is any primary mirror in the [rsync mirrors list][mirrors-rsync]. +Please try to select a primary mirror that is being used by few or no other +secondary mirrors, to evenly spread the load across the primary mirrors. -Once your mirror is set up, please announce it to the [ProteanOS development -mailing list][proteanos-dev] ([list info][proteanos-dev-info]). Please include -the following information in your announcement: +[rfc1635]: https://tools.ietf.org/html/rfc1635 + + +Announcing a Public Mirror +========================== + +After setting up a public mirror, please announce it to the [ProteanOS +development mailing list][proteanos-dev] ([list info][proteanos-dev-info]). +Please include the following information in your announcement: * Name and e-mail address of site maintainer; * Name and URL of site sponsor, if any; - * Domain name, or IPv4 and/or IPv6 address, of mirror site; - * Available protocols; - * Country of site; - * Frequency of mirroring; and + * URIs of mirror site (HTTP and/or FTP, and rsync for primary mirrors); + * Location (country at a minimum) of site; + * Synchronization frequency (or times); + * Synchronization source, in the case of a secondary mirror; and * Any other comments about the site. +Your mirror will be reviewed and added to the official mirror lists +([HTTP][mirrors-http] and [FTP][mirrors-ftp] used by [[dev/prokit]], as well as +[rsync][mirrors-rsync] referenced above). + [proteanos-dev]: mailto:proteanos-dev@lists.proteanos.com [proteanos-dev-info]: http://lists.proteanos.com/proteanos-dev/ +[mirrors-http]: http://files.proteanos.com/pub/proteanos-mirrors-http +[mirrors-ftp]: http://files.proteanos.com/pub/proteanos-mirrors-ftp +[mirrors-rsync]: http://files.proteanos.com/pub/proteanos-mirrors-rsync diff --git a/dev/archive/signing.mdwn b/dev/archive/signing.mdwn index 0dfd6be..0076ae8 100644 --- a/dev/archive/signing.mdwn +++ b/dev/archive/signing.mdwn @@ -11,6 +11,12 @@ retrieve packages from the archive. Implementation ============== +**TODO:** ProteanOS is implementing [Ed25519][ed25519] signatures, likely +without OpenPGP PKI. The pro-archman and prokit sections below will need some +rethinking. + +**TODO:** Consider [ucert][]. + ProteanOS Archive Manager ------------------------- @@ -39,6 +45,18 @@ key(s) into their keyring. If keys are distributed with prokit, **revocations and key transitions need to be handled somehow**. +New keys can be distributed with new versions of prokit, though this would +require users to upgrade prokit to get new keys. Revocations, being more of a +security risk that can go unnoticed by users, would need to be more actively and +immediately received by users. prokit could perhaps check a key server (over +HKPS) each time before using a key. + +And if prokit needs to check key servers anyway, it could also use them to find +new archive signing keys, as long as at least one "seed" key is distributed with +prokit. prokit should find and use only archive signing keys (by a user ID +specified in the profile) that are signed by a non-revoked previous key (or a +signed chain of keys with the user ID). + A user already has to import a key into their own keyring to verify their prokit download. Maybe it's better to just instruct users to also download the archive signing key(s) into their keyrings. This takes advantage of existing PKI, and @@ -51,66 +69,19 @@ Suggestions welcome. Opkg ---- -The opkg package manager supports verifying package feeds. This feature -requires linking against the [GnuPG Made Easy (GPGME) library][gpgme]. - -ProteanOS packages for GPGME and its dependencies need to be prepared and -uploaded. gpgme depends on libgpg-error (packaged and uploaded) and libassuan. -Its testsuite depends on gnupg. gnupg in turn depends on libgpg-error, -libgcrypt, libassuan, libksba, and npth. libgcrypt depends on libgpg-error. -Below is a DOT-language digraph representation of these dependencies that can be -rendered with Graphviz. - -Once gpgme and its dependencies are packaged in ProteanOS, opkg can be built -with package feed verification. This feature should be optional. The `opkg` -source package should have two builds, with and without verification enabled. -This will probably mean generating new binary packages `opkg-gpg` and -`libopkg.1-gpg` (which will conflict with `opkg` and `libopkg.1` respectively) -and installing data files into binary package directories without the aid of -`oh-installfiles`. Although, option 1 of the [[static_libopkg|pkg/opkg/static]] -plan would simplify this by getting rid of the `libopkg.1` and `libopkg.1-dev` -packages altogether. The `opkg` source package could then build just `opkg`, -`opkg-gpg`, `opkg-dbg`, and `opkg-doc` binary packages. - -The DOT-language dependency graph (which can be rendered as a PNG image with the -command pipeline in the header comment, if saved as `gpgme.dot`): - - /* - * Dependencies of GPGME - * - * Render by running: - * ccomps -x gpgme.dot | dot | gvpack | neato -n2 -Tpng -ogpgme.png - */ - - digraph deps { - graph [fontname="FreeSans"]; - node [fontname="FreeSans"]; - edge [fontname="FreeSans"]; - - subgraph cluster_gpgme { - style = filled; - color = "#0093dd"; - node [style=filled]; - label = "gpgme"; - - "gpgme" [color=red]; - "libgpg-error" [color=green]; - "libassuan" [color=red]; - "gnupg" [color=red]; - "libgcrypt" [color=red]; - "libksba" [color=red]; - "npth" [color=red]; - } - - "gpgme" -> "libgpg-error"; - "gpgme" -> "libassuan"; - "gpgme" -> "gnupg" [color=gray,label="testsuite"]; - "gnupg" -> "libgpg-error"; - "gnupg" -> "libgcrypt"; - "gnupg" -> "libassuan"; - "gnupg" -> "libksba"; - "gnupg" -> "npth"; - "libgcrypt" -> "libgpg-error"; - } - -[gpgme]: https://www.gnupg.org/software/gpgme/index.html +ProteanOS [[now|dev/opkg/future]] uses [opkg-lede][] with [usign][], a +lightweight implementation of the [Ed25519][ed25519] public-key signature +system. + +Archive Keyring +--------------- + +The package archive keyring will need to be maintained in a ProteanOS package. +Key transitions, expiration changes, and revocations will be performed by +revisions to this package. It should possibly be upgraded automatically to +ensure that installed systems always have up-to-date keys. + +[ucert]: https://git.openwrt.org/?p=project/ucert.git;a=blob;f=README.md;hb=HEAD +[opkg-lede]: https://git.openwrt.org/?p=project/opkg-lede.git;a=summary +[usign]: https://git.openwrt.org/?p=project/usign.git;a=summary +[ed25519]: https://ed25519.cr.yp.to/ |