diff options
Diffstat (limited to 'dev/archive/mirroring.mdwn')
| -rw-r--r-- | dev/archive/mirroring.mdwn | 108 |
1 files changed, 86 insertions, 22 deletions
diff --git a/dev/archive/mirroring.mdwn b/dev/archive/mirroring.mdwn index 9d2b38a..deabb91 100644 --- a/dev/archive/mirroring.mdwn +++ b/dev/archive/mirroring.mdwn @@ -1,53 +1,117 @@ [[!meta title="Mirroring the ProteanOS Package Archive"]] -Running a mirror of the ProteanOS package archive contributes to the +Running a public mirror of the ProteanOS package archive contributes to the availability of ProteanOS. -You can keep your mirror private or make it an official mirror. +Organizations deploying many ProteanOS systems are encouraged to make local +mirrors, which they may keep private or make public, to save bandwidth on both +their and ProteanOS's infrastructure. Setting up a Mirror =================== -A mirror takes about 700 MiB of storage space as of this writing and should be +Overview +-------- + +*(Impatient mirror administrators may skip this informational section and read +the instructional sections below.)* + +A mirror takes about 730 MiB of storage space as of this writing and should be expected to grow beyond that. -The following command will list files in the archive: +The curious may view a list of files in the archive with the following command: $ rsync --recursive --exclude .db rsync://files.proteanos.com/proteanos -The following command will mirror the archive: - - $ rsync --recursive --times --delete --exclude .db \ - > rsync://files.proteanos.com/proteanos /path/to/your/mirror/directory/ +See below for commands to copy archive files to a local mirror. -Packages are moved from incoming into the archive four times daily. The +Packages are processed from the incoming queue into the archive four times +daily, so mirrors need not be updated more frequently than that. The [[archive_management_software|dev/pro-archman]] is [configured][arch-conf] to keep unreferenced files in the pool for one day, so mirrors must be updated at least once a day. Otherwise, during the update, some feed index files may -reference deleted files. +reference deleted files. Updating at least twice daily is recommended, in case +the synchronization source is ever temporarily unreachable. + +Serving the archive mirror at `/pub/proteanos` over HTTP and FTP is recommended, +but any path and either protocol may be used. HTTP Strict Transport Security +(HSTS) must not be used except on private mirrors used by ProteanOS systems +known to install the `wolfssl-util` package. The ProteanOS package archive is +already [cryptographically verified][archive-signing] using software smaller +than a TLS implementation. + +Mirrors are classified by their synchronization source as either "primary" or +"secondary" as described below. This design distributes synchronization +bandwidth usage more efficiently across the mirror network. Secondary mirrors +will typically update a few hours after primary mirrors do, but are otherwise +still "first-class citizens". All mirrors regardless of classification are +treated equally by [[dev/prokit]], the installer utility that selects a mirror. + +Please configure private mirrors as secondary mirrors, so as to reserve +ProteanOS project bandwidth for those that contribute bandwidth back to the +project. [arch-conf]: http://files.proteanos.com/pub/proteanos/conf +[archive-signing]:http://lists.proteanos.com/proteanos-dev/2019/04/msg00008.html + +Primary Public Mirror +--------------------- + +Primary mirrors are those that synchronize directly from +<files.proteanos.com> and must serve their contents publicly over HTTP +(without HSTS) and rsync and may also serve over anonymous FTP as defined by +[IETF RFC 1635][rfc1635] with no password requirements. + +Run the following command one to four times (four times preferred) daily to +synchronize your mirror: + + $ rsync --recursive --times --delete --exclude .db \ + > rsync://files.proteanos.com/proteanos /path/to/your/mirror/directory/ + +Secondary Public or Private Mirror +---------------------------------- +Secondary mirrors are those that synchronize from a primary mirror and serve +their contents either publicly or privately. Public mirrors must serve over +HTTP (without HSTS) and may also serve over anonymous FTP as defined by [IETF +RFC 1635][rfc1635] with no password requirements. Private mirrors may serve +over either protocol. -Making an Official Mirror -========================= +Run the following command one to four times (four times preferred) daily to +synchronize your mirror: + + $ rsync --recursive --times --delete --exclude .db \ + > ${src} /path/to/your/mirror/directory/ -Official mirror sites should make their archive mirrors available at -`/pub/proteanos` over HTTP and FTP. Sites should also make their archive -mirrors accessible over rsync, if possible. +Where `${src}` is any primary mirror in the [rsync mirrors list][mirrors-rsync]. +Please try to select a primary mirror that is being used by few or no other +secondary mirrors, to evenly spread the load across the primary mirrors. -Once your mirror is set up, please announce it to the [ProteanOS development -mailing list][proteanos-dev] ([list info][proteanos-dev-info]). Please include -the following information in your announcement: +[rfc1635]: https://tools.ietf.org/html/rfc1635 + + +Announcing a Public Mirror +========================== + +After setting up a public mirror, please announce it to the [ProteanOS +development mailing list][proteanos-dev] ([list info][proteanos-dev-info]). +Please include the following information in your announcement: * Name and e-mail address of site maintainer; * Name and URL of site sponsor, if any; - * Domain name, or IPv4 and/or IPv6 address, of mirror site; - * Available protocols; - * Country of site; - * Frequency of mirroring; and + * URIs of mirror site (HTTP and/or FTP, and rsync for primary mirrors); + * Location (country at a minimum) of site; + * Synchronization frequency (or times); + * Synchronization source, in the case of a secondary mirror; and * Any other comments about the site. +Your mirror will be reviewed and added to the official mirror lists +([HTTP][mirrors-http] and [FTP][mirrors-ftp] used by [[dev/prokit]], as well as +[rsync][mirrors-rsync] referenced above). + [proteanos-dev]: mailto:proteanos-dev@lists.proteanos.com [proteanos-dev-info]: http://lists.proteanos.com/proteanos-dev/ +[mirrors-http]: http://files.proteanos.com/pub/proteanos-mirrors-http +[mirrors-ftp]: http://files.proteanos.com/pub/proteanos-mirrors-ftp +[mirrors-rsync]: http://files.proteanos.com/pub/proteanos-mirrors-rsync |