Title: Source Package Format 2.0 Status: DRAFT Date: 2012-08-04 Revises: Source Package Format 1.0 1. Status of This Document =========================== This specification is in `DRAFT` status. It is a work-in-progress and is subject to change. Comments and revisions are welcome. 2. Abstract ============ This document describes version 2.0 of the format for software packages. 3. Background ============== A source package consists of software source code, a build system, and package metadata. From it is built one or more binary packages, which can be installed into an operating system. 4. Changes Since Version 1.0 ============================= Following is a summary of differences between versions 1.0 and 2.0 of this source package format: * The platform configuration file list is now split between two files. `platconf` lists build-time configuration files, and `.pkg/platconf` lists run-time configuration files. - As a result, the platform configuration file lists now have a simpler syntax that does not include the type of configuration file. - It is now easy to determine whether individual binary packages are platform-specific. * Package building can now be configured by a `config` script. - There is now no need for any configuration logic in the `build` makefile. See section 5 for rationale behind this design. - The list of binary packages that can be built from a source package no longer needs to be statically defined. * The source package hierarchy has been clarified to to indicate whether each file is required. - Additionally, certain files are now only required by a certain step in the build process, allowing them to be dynamically generated. * Packages without any sources (such as pure metapackages) are now explicitly allowed. * Maintainer scripts are now explained. * The set of required `build` makefile targets has been changed. - The amount of redundant information in source packages is reduced by not requiring the `build` file to list all binary packages. - The user can choose to build all binary packages, only architecture-independent binary packages, or only architecture-independent binary packages. See section 5 for rationale behind this design. * The source package version is no longer included in `control`. - It is already given in the most recent entry in `changelog`. * The version identifier syntax now encodes information about source archive repacks and non-trunk distribution uploads. - The syntax has been made strict to make parsing simple and deterministic. * The `Source` and `Package` control fields have been removed, as the information they provided is provided elsewhere. - Source package names are now given only in the `changelog` file. - Binary package names are now given only in the names of `.pkg/` directories. * Platform names are no longer specified in the `Architecture` field. - They are now specified in a new `Platform` field. * Wildcard binary architecture strings are now supported. 5. Rationale ============= TODO 6. Scope ========= This specification applies both to source packages and to the package building tools that interact with them. 7. Definitions =============== TODO source package binary package package sources source package directory package building work area, work area package building tool 8. Source Package Directory Structure Summary ============================================== The source package directory hierarchy can be summarized with the following tree: / | No naming requirements are made for this directory. +- .pkg/ | | Required for each binary package after execution of "config". | +- control | | Required. | | Metadata about the binary package. | +- install | | Required. | | A list of patterns to match files to be installed in the binary | | package. | +- platconf | | Optional. | | A list of run-time configuration files. | +- postinst | | Optional. | | A maintainer script to be executed after a binary package is | | unpacked. | +- postrm | | Optional. | | A maintainer script to be executed after a binary package is | | removed. | +- preinst | | Optional. | | A maintainer script to be executed before a binary package is | | unpacked. | \- prerm | Optional. | A maintainer script to be executed before a binary package is | removed. +- build | Required after execution of "config". | A makefile with target rules to build the binary package(s). +- changelog | Required. | A log of changes made to the source package. +- config | Optional. | A shell script to configure the package build. +- control | Required. | Metadata about the source package. +- copyright | Required. | Information about copyrights and licenses in the source package. +- format | Required. | A magic file to identify the source format version. Should simply | contain the string "2.0". +- patches/ | Optional. | Patches to be applied to package sources before building. +- -.tar. | Optional. | Upstream source archive (for non-native packages). +- platconf | Optional. | A list of build-time configuration files. \- src/ Optional. Package sources (for native packages). 9. Package Building Work Area ============================== When building packages, a new directory named `tmp` shall be created in the source package directory. In this directory, the package source code shall be copied or unpacked into a directory called `src` and an installation destination directory named `dest` shall be created. After all expected binary packages have been built, the work area (`tmp` and its children) shall be removed. 10. Descriptions of Files ========================== All of the following files are located under what is called a "source package directory" (`` in the tree in section 8). No naming requirements are made for this directory. See section 21.1 for a recommended naming convention for this directory. 10.1. `.pkg/` ---------------------- After execution of the `config` file, for each binary package there must exist a directory named `.pkg`, where `` is the name of the binary package. Binary package names may only consist of lowercase Latin letters, digits, plus and minus signs, and periods. Names must be at least two characters long and must start with either a letter or a digit. (A regular expression for this is `[a-z0-9][a-z0-9+.-]+`.) This directory contains metadata and scripts for a binary package. 10.2. `.pkg/control` ----------------------------- This file is required. This file contains control fields describing the binary package. See section 15 for the syntax of this file and section 19 for the list of control fields in this file. 10.3. `.pkg/install` ----------------------------- This file is required. This file contains a list of patterns to match files to be installed in the binary package. Each line in this file is a pattern that is expanded to match files as explained in [POSIX.1-2008, Shell & Utilities Volume, Section 2.6.6][posix-xcu-2.6.6]. A file or directory in the destination directory may not be matched by more than one binary package's `install` file. The behavior of package building tools in such a case is undefined. 10.4. `.pkg/platconf` ------------------------------ This file is optional. Its presence indicates that a binary package is platform-specific. This file contains a list of run-time configuration files to be installed in the binary package. Files listed in this file shall not be listed in the binary package's `install` file. See section 14 for the syntax of this file. 10.5. `.pkg/postinst` ------------------------------ This file is optional. This file is a maintainer script to be executed after a binary package is unpacked. See section 20 for more information about maintainer scripts. 10.6. `.pkg/postrm` ---------------------------- This file is optional. This file is a maintainer script to be executed after a binary package is removed. See section 20 for more information about maintainer scripts. 10.7. `.pkg/preinst` ------------------------------ This file is optional. This file is a maintainer script to be executed before a binary package is unpacked. See section 20 for more information about maintainer scripts. 10.8. `.pkg/prerm` ------------------------------ This file is optional. This file is a maintainer script to be executed before a binary package is removed. See section 20 for more information about maintainer scripts. 10.9. `build` -------------- After execution of the `config` file, this file must exist. This file is a makefile that controls the building of binary packages. See section 12 for requirements for this file. 10.10. `changelog` ------------------- This file is required. This file is a log of changes made to the source package. See section 13 for the format of this file. 10.11. `config` ---------------- This file is optional. This file is a script in shell command language that configures the package build. See section 11 for requirements for this file. 10.12. `control` ----------------- This file is required. This file contains control fields describing the source package. See section 15 for the syntax of this file and section 16 for the list of control fields in this file. 10.13. `copyright` ------------------- This file is required. This file contains information about the copyrights in and licenses for the source package. This specification currently makes no requirements for the format of this file. 10.14. `format` ---------------- This file is required. This "magic" file identifies the format version of the source package. For packages in this version of the source package format, this file should simply contain the string `2.0`. 10.15. `patches/` ------------------ This directory is optional. This directory shall contain patches to be applied to package sources before configuring and building any binary packages. See section 15 for the syntax of patch files. 10.16. `-.tar.` -------------------------------------- TODO 10.17. `platconf` ------------------ This file is optional. Its presence indicates that each binary package is platform-specific. This file contains a list of build-time configuration files to be installed into the package building work area. See section 14 for the syntax of this file. 10.18. `src/` -------------- TODO 11. Package Configuration Script Format ======================================== TODO 12. Build File Format ====================== TODO 13. Change Log Format ====================== Changes made to the source package should be explained in the file `changelog`. Each new package revision must be documented with an entry of the following format: package (version) distributions [zero or more blank lines] * change details [zero or more blank lines] * more change details more detailed change details [zero or more blank lines] -- maintainer date `package` is the source package name. Source package names may only consist of lowercase Latin letters, digits, plus and minus signs, and periods. Names must be at least two characters long and must start with either a letter or a digit. (A regular expression for this is `[a-z0-9][a-z0-9+.-]+`.) `version` is the source package version. See section 17 for the syntax of source package version identifiers. `distributions` is a list of distributions into which the package should be installed when uploaded to the package archive. The list consists of one or more distribution names, separated by spaces. `maintainer` is the name and e-mail address of the package maintainer. This field must follow the syntax of the `mailbox` symbol of RFC 5322 section 3.4. `date` is the date of packaging. This field must follow the syntax of the `date-time` symbol of RFC 5322 section 3.3. 14. Platform Configuration File List Format ============================================ Platform-specific configuration files used by the source package at build time shall be listed in the `platconf` file. Platform-specific configuration files used by the binary package(s) at run time shall be listed in the `.pkg/platconf` file. Source packages with a `platconf` file, a `.pkg/platconf` file, or both shall list in their build-time dependencies a package (real or virtual) that provides the listed platform-specific configuration files. Each configuration file must be described with an entry of the following format: source destination `source` is the path to the file, relative to the platform configuration directory -- either `/usr/share/config/PLATFORM/PACKAGE-VERSION` or `/usr/share/config/PLATFORM/PACKAGE`, where `PLATFORM` is the architecture string denoting an application platform, `PACKAGE` is the name of the configurable source package, and `VERSION` is the upstream version of the configurable source package. `destination` is the path (file or directory) to which the file should be copied. For a file used at build time, it is a path relative to the package building work area. For a file used at run time, it is an absolute path in the user's filesystem hierarchy. 15. Patch File Syntax ====================== 16. Control File Format ======================== TODO 17. Source Package Version Identifier Syntax ============================================= In general, the syntax of source package version identifiers is: [+sip][-][+-] `` is the version of the original upstream package or the native package. It may only consist of lowercase Latin letters, digits, periods, and tildes. It must be at least one character long. (A regular expression for this is `[0-9a-z.~]+`.) If the upstream source archive needs to be repacked with certain changes for compliance with the Software Inclusion Policy, the string `+sip` must be appended to ``. `` is a number that should be incremented on each repack while `` remains constant. It should be reset to `1` for the first repack of a new upstream source archive. It must be a string of one or more digits, the first of which must be greater than or equal to `1`. (A regular expression for this is `[1-9][0-9]*`.) `` is the version of the distribution packaging. It is optional and should be omitted for native packages. It should be incremented on each revision of packaging while `` and `` remain constant. It should be reset to `1` for the first revision of packaging for a new upstream package version or source archive repack. It must be a string of one or more digits, the first of which must be greater than or equal to `1`. (A regular expression for this is `[1-9][0-9]*`.) If the package is to be installed into a distribution other than `trunk`, the string `+-` must be appended to the end of the version identifier. `` is the distribution into which the package is to be installed, may only consist of lowercase Latin letters and digits, and must be at least one character long. (A regular expression for this is `[a-z0-9]+`.) `` is a number that should be incremented on each upload to `` while ``, ``, and `` remain constant. It should be reset to `1` for the first upload to `` of a new upstream package version, source archive repack, or distribution packaging revision. It must be a string of one or more digits, the first of which must be greater than or equal to `1`. (A regular expression for this is `[1-9][0-9]*`.) 18. Source Package Metadata ============================ The fields in the source package metadata are: * `Maintainer` (required) The name and e-mail address of the package maintainer. This field must follow the syntax of the `mailbox` symbol in RFC 5322 section 3.4. * `Build-Depends` (optional) A list of packages that must be installed before the package can be built. * `Homepage` (optional) The URL of the Web site for the package. Accessible at this site should be origin source code and documentation and/or information. Though the information in this field is machine-usable, the URL must not be surrounded by angle brackets or any other characters. 19. Binary Package Metadata ============================ The fields in the binary package metadata are: * `Architecture` (required) A list of architectures for which this package may be built. The value may be any one of the following: - The string `all` to specify that all of the provided files are architecture-independent; - The string `any` to specify that at least some of the provided files are architecture-dependent; or - A space-separated list of one or more real or wildcard binary architecture strings to specify that at least some of the provided files are only usable on systems of the named binary architecture(s). A "wildcard" binary architecture string is one in which one or two components (called "wildcard components") is the string `any`. It shall be considered to match any real binary architecture string for which the non-wildcard components are equal. * `Platform` (required) A list of platforms for which this package may be built. The value may be any one of the following: - The string `all` to specify that all of the provided files are platform-independent; - The string `any` to specify that at least some of the provided files are platform-dependent; or - A space-separated list of one or more platform strings to specify that at least some of the provided files are only usable on systems of the named platform(s). * `Essential` (optional) A flag to indicate whether the package is essential for the functioning of a system on which it is installed. If this field is set to `yes`, opkg will refuse to remove the package except when upgrading it. If this field is set to any other value or is omitted, the package may be removed by a user. * `Depends` (optional) A list of packages that must be installed and configured before the package may itself be configured. * `Recommends` (optional) * `Suggests` (optional) * `Pre-Depends` (optional) A list of packages that must be installed before the package may itself be installed. * `Conflicts` (optional) * `Provides` (optional) * `Replaces` (optional) * `Description` (required) A description of the binary package. This is a multiline field. The first line is a short synopsis, and all following lines are an extended description. 20. Maintainer Scripts ======================= 20.1. Maintainer Script Format ------------------------------- TODO 20.2. Calling Standard ----------------------- TODO 21. Packaging Recommendations ============================== 21.1. Source Package Directory Naming -------------------------------------- It is recommended that the name of the source package directory be simply the name of the source package. Including the package version number or any part thereof in the directory name is not recommended, as this number will likely change over time. 21.2. `build` File Generation ------------------------------ For non-trivial packages, it is recommended that the `config` script generates a `build` makefile from some input template, named `build.in` or something similar. This is similar to the way that a `configure` script generated by GNU Autoconf will produce a `Makefile` from `Makefile.in`. With such makefile generation, the `build` file need not contain any conditional logic (e.g. to test for cross building or to enable certain software features). Such logic is supported by certain implementations of `make` including GNU Make. However, it is not specified by POSIX.1 (and therefore cannot be expected to be supported by all implementations). Furthermore, conditional logic goes beyond the original design of the `make` utility. 21.3. Dynamic Binary Package Generation ---------------------------------------- For certain source packages, the set of binary packages to be built depends on the build configuration. For example, the target architecture of a cross compiler may be embedded in the name of the providing binary package. In such cases it is recommended that binary package directories and their files be generated by the `config` script from template directories named `.pkg.in/` or similar. 22. Example Files ================== In the following sections are some example files that might be used for the `opkg` source package. They observe the recommendation made in section 21.2 to make the `config` script generate a `build` makefile from an input `build.in` file. 22.1. Example `config` File ---------------------------- Following is an example `config` file: #! /bin/sh if [ "${OH_BUILD_ARCH_GNU}" = "${OH_HOST_ARCH_GNU}" ]; then ARCH_OPTS= else ARCH_OPTS="--build=${OH_BUILD_ARCH_GNU} --host=${OH_HOST_ARCH_GNU}" fi sed -e "s&@ARCH_OPTS@&${ARCH_OPTS}&" build.in >build It detects whether the user intends to cross build the software and generates appropriate `build` and `host` options for the software's `configure` script. 22.2. Example `build.in` File ------------------------------ Following is an example `build.in` file: #! /usr/bin/make -f PKG_COMMON = libopkg.1 configure: configure.stamp configure.stamp: cd src && ./configure --prefix=/usr @ARCH_OPTS@ \ --disable-static --disable-curl --disable-ssl-curl --disable-gpg touch $@ build: build.stamp build.stamp: configure cd src && make touch $@ install: install.stamp install.stamp: build cd src && make DESTDIR="$${PWD}/../dest" install touch $@ binary-arch: install oh-strip -g oh-installfiles oh-installdocs $(PKG_COMMON) oh-gencontrol oh-buildopk binary-indep: install oh-installfiles oh-installdocs $(PKG_COMMON) oh-gencontrol oh-buildopk binary: binary-arch binary-indep It has the required `binary`, `binary-arch`, and `binary-indep` targets, along with targets to configure, build, and install the packaged software. It makes use of build stamps to record that certain stages of the binary package build process are completed. Additionally, this file relies on the opkhelper utilities to perform tasks related to the production of binary packages. 23. Legal Notice ================= Copyright (C) 2012 Patrick "P. J." McDermott Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. [posix-xcu-2.6.6]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_06