summaryrefslogtreecommitdiffstats
path: root/dev/archive/mirroring.mdwn
blob: 2f60568aae614d45e490520a13ffd06a1eb37d0d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
[[!meta title="Mirroring the ProteanOS Package Archive"]]

Running a public mirror of the ProteanOS package archive contributes to the
availability of ProteanOS.

Organizations deploying many ProteanOS systems are encouraged to make local
mirrors, which they may keep private or make public, to save bandwidth on both
their and ProteanOS's infrastructure.


Setting up a Mirror
===================

Overview
--------

*(Impatient mirror administrators may skip this informational section and read
the instructional sections below.)*

A mirror takes about 730 MiB of storage space as of this writing and should be
expected to grow beyond that.

The curious may view a list of files in the archive with the following command:

    $ rsync --recursive --exclude .db rsync://files.proteanos.com/proteanos

See below for commands to copy archive files to a local mirror.

Packages are processed from the incoming queue into the archive four times
daily, so mirrors need not be updated more frequently than that.  The
[[archive_management_software|dev/pro-archman]] is [configured][arch-conf] to
keep unreferenced files in the pool for one day, so mirrors must be updated at
least once a day.  Otherwise, during the update, some feed index files may
reference deleted files.

Serving the archive mirror at `/pub/proteanos` over HTTP and FTP is recommended,
but any path and either protocol may be used.

Mirrors are classified by their synchronization source as either "primary" or
"secondary" as described below.  This design distributes synchronization
bandwidth usage more efficiently across the mirror network.  Secondary mirrors
will typically update a few hours after primary mirrors do, but are otherwise
still "first-class citizens".  All mirrors regardless of classification are
treated equally by [[dev/prokit]], the installer utility that selects a mirror.

Please configure private mirrors as secondary mirrors, so as to reserve
ProteanOS project bandwidth for those that contribute bandwidth back to the
project.

[arch-conf]: http://files.proteanos.com/pub/proteanos/conf

Primary Public Mirror
---------------------

Primary mirrors are those that synchronize directly from
<files.proteanos.com> and must serve their contents publicly over HTTP and
rsync and may also serve over anonymous FTP as defined by [IETF RFC
1635][rfc1635] with no password requirements.

Run the following command one to four times daily to synchronize your mirror:

    $ rsync --recursive --times --delete --exclude .db \
    > rsync://files.proteanos.com/proteanos /path/to/your/mirror/directory/

Secondary Public or Private Mirror
----------------------------------

Secondary mirrors are those that synchronize from a primary mirror and serve
their contents either publicly or privately.  Public mirrors must serve over
HTTP and may also serve over anonymous FTP as defined by [IETF RFC
1635][rfc1635] with no password requirements.  Private mirrors may serve over
either protocol.

Run the following command one to four times daily to synchronize your mirror:

    $ rsync --recursive --times --delete --exclude .db \
    > ${src} /path/to/your/mirror/directory/

Where `${src}` is any primary mirror listed in
<http://files.proteanos.com/pub/proteanos-mirrors-rsync>.  Please try to select
a primary mirror that is being used by few or no other secondary mirrors, to
evenly spread the load across the primary mirrors.

[rfc1635]: https://tools.ietf.org/html/rfc1635


Announcing a Public Mirror
==========================

After setting up a public mirror, please announce it to the [ProteanOS
development mailing list][proteanos-dev] ([list info][proteanos-dev-info]).
Please include the following information in your announcement:

  * Name and e-mail address of site maintainer;
  * Name and URL of site sponsor, if any;
  * URIs of mirror site (HTTP and/or FTP, and rsync for primary mirrors);
  * Country of site;
  * Frequency of mirroring;
  * Synchronization source, in the case of a secondary mirror; and
  * Any other comments about the site.

Your mirror will be reviewed and added to the official mirror lists
([HTTP][mirrors-http] and [FTP][mirrors-ftp] used by [[dev/prokit]], as well as
[rsync][mirrors-rsync] referenced above).

[proteanos-dev]: mailto:proteanos-dev@lists.proteanos.com
[proteanos-dev-info]: http://lists.proteanos.com/proteanos-dev/
[mirrors-http]: http://files.proteanos.com/pub/proteanos-mirrors-http
[mirrors-ftp]: http://files.proteanos.com/pub/proteanos-mirrors-ftp
[mirrors-rsync]: http://files.proteanos.com/pub/proteanos-mirrors-rsync