path: root/specs/source-package-format-2.0.txt

    Title: Source Package Format 2.0
    Status: DRAFT
    Date: 2012-08-04
    Revises: Source Package Format 1.0


1.  Status of This Document
===========================

This specification is in `DRAFT` status.  It is a work-in-progress and is
subject to change.  Comments and revisions are welcome.


2.  Abstract
============

This document describes version 2.0 of the format for software packages.


3.  Background
==============

A source package consists of software source code, a build system, and package
metadata.  From it is built one or more binary packages, which can be installed
into an operating system.


4.  Changes Since Version 1.0
=============================

Following is a summary of differences between versions 1.0 and 2.0 of this
source package format:

  * The platform configuration file list is now split between two files.
    `platconf` lists build-time configuration files, and `<binpkg>.pkg/platconf`
    lists run-time configuration files.
    - As a result, the platform configuration file lists now have a simpler
      syntax that does not include the type of configuration file.
    - It is now easy to determine whether individual binary packages are
      platform-specific.
  * Package building can now be configured by a `config` script.
    - There is now no need for any configuration logic in the `build` makefile.
      See section 5 for rationale behind this design.
    - The list of binary packages that can be built from a source package no
      longer needs to be statically defined.
  * The source package hierarchy has been clarified to to indicate whether each
    file is required.
    - Additionally, certain files are now only required by a certain step in the
      build process, allowing them to be dynamically generated.
  * Packages without any sources (such as pure metapackages) are now explicitly
    allowed.
  * Maintainer scripts are now explained.
  * The set of required `build` makefile targets has been changed.
    - The amount of redundant information in source packages is reduced by not
      requiring the `build` file to list all binary packages.
    - The user can choose to build all binary packages, only
      architecture-independent binary packages, or only architecture-independent
      binary packages.  See section 5 for rationale behind this design.
  * The source package version is no longer included in `control`.
    - It is already given in the most recent entry in `changelog`.
  * The version identifier syntax now encodes information about source archive
    repacks and non-trunk distribution uploads.
    - The syntax has been made strict to make parsing simple and deterministic.
  * The `Source` and `Package` control fields have been removed, as the
    information they provided is provided elsewhere.
    - Source package names are now given only in the `changelog` file.
    - Binary package names are now given only in the names of `<binpkg>.pkg/`
      directories.
  * Platform names are no longer specified in the `Architecture` field.
    - They are now specified in a new `Platform` field.
  * Wildcard binary architecture strings are now supported.


5.  Rationale
=============

TODO


6.  Scope
=========

This specification applies both to source packages and to the package building
tools that interact with them.


7.  Definitions
===============

TODO

source package

binary package

package sources

source package directory

package building work area, work area

package building tool


8.  Source Package Directory Structure Summary
==============================================

The source package directory hierarchy can be summarized with the following
tree:

    <source-package-directory>/
     |  No naming requirements are made for this directory.
     +- <binpkg>.pkg/
     |   |  Required for each binary package after execution of "config".
     |   +- control
     |   |      Required.
     |   |      Metadata about the binary package.
     |   +- install
     |   |      Required.
     |   |      A list of patterns to match files to be installed in the binary
     |   |      package.
     |   +- platconf
     |   |      Optional.
     |   |      A list of run-time configuration files.
     |   +- postinst
     |   |      Optional.
     |   |      A maintainer script to be executed after a binary package is
     |   |      unpacked.
     |   +- postrm
     |   |      Optional.
     |   |      A maintainer script to be executed after a binary package is
     |   |      removed.
     |   +- preinst
     |   |      Optional.
     |   |      A maintainer script to be executed before a binary package is
     |   |      unpacked.
     |   \- prerm
     |          Optional.
     |          A maintainer script to be executed before a binary package is
     |          removed.
     +- build
     |      Required after execution of "config".
     |      A makefile with target rules to build the binary package(s).
     +- changelog
     |      Required.
     |      A log of changes made to the source package.
     +- config
     |      Optional.
     |      A shell script to configure the package build.
     +- control
     |      Required.
     |      Metadata about the source package.
     +- copyright
     |      Required.
     |      Information about copyrights and licenses in the source package.
     +- format
     |      Required.
     |      A magic file to identify the source format version.  Should simply
     |      contain the string "2.0".
     +- patches/
     |      Optional.
     |      Patches to be applied to package sources before building.
     +- <pkgname>-<pkgver>.tar.<ext>
     |      Optional.
     |      Upstream source archive (for non-native packages).
     +- platconf
     |      Optional.
     |      A list of build-time configuration files.
     \- src/
            Optional.
            Package sources (for native packages).


9.  Package Building Work Area
==============================

When building packages, a new directory named `tmp` shall be created in the
source package directory.  In this directory, the package source code shall be
copied or unpacked into a directory called `src` and an installation destination
directory named `dest` shall be created.

After all expected binary packages have been built, the work area (`tmp` and its
children) shall be removed.


10.  Descriptions of Files
==========================

All of the following files are located under what is called a "source package
directory" (`<source-package-directory>` in the tree in section 8).  No naming
requirements are made for this directory.  See section 21.1 for a recommended
naming convention for this directory.

10.1.  `<binpkg>.pkg/`
----------------------

After execution of the `config` file, for each binary package there must exist a
directory named `<binpkg>.pkg`, where `<binpkg>` is the name of the binary
package.

Binary package names may only consist of lowercase Latin letters, digits, plus
and minus signs, and periods.  Names must be at least two characters long and
must start with either a letter or a digit.  (A regular expression for this is
`[a-z0-9][a-z0-9+.-]+`.)

This directory contains metadata and scripts for a binary package.

10.2.  `<binpkg>.pkg/control`
-----------------------------

This file is required.

This file contains control fields describing the binary package.  See section 15
for the syntax of this file and section 19 for the list of control fields in
this file.

10.3.  `<binpkg>.pkg/install`
-----------------------------

This file is required.

This file contains a list of patterns to match files to be installed in the
binary package.

Each line in this file is a pattern that is expanded to match files as explained
in [POSIX.1-2008, Shell & Utilities Volume, Section 2.6.6][posix-xcu-2.6.6].

A file or directory in the destination directory may not be matched by more than
one binary package's `install` file.  The behavior of package building tools in
such a case is undefined.

10.4.  `<binpkg>.pkg/platconf`
------------------------------

This file is optional.  Its presence indicates that a binary package is
platform-specific.

This file contains a list of run-time configuration files to be installed in the
binary package.  Files listed in this file shall not be listed in the binary
package's `install` file.

See section 14 for the syntax of this file.

10.5.  `<binpkg>.pkg/postinst`
------------------------------

This file is optional.

This file is a maintainer script to be executed after a binary package is
unpacked.  See section 20 for more information about maintainer scripts.

10.6.  `<binpkg>.pkg/postrm`
----------------------------

This file is optional.

This file is a maintainer script to be executed after a binary package is
removed.  See section 20 for more information about maintainer scripts.

10.7.  `<binpkg>.pkg/preinst`
------------------------------

This file is optional.

This file is a maintainer script to be executed before a binary package is
unpacked.  See section 20 for more information about maintainer scripts.

10.8.  `<binpkg>.pkg/prerm`
------------------------------

This file is optional.

This file is a maintainer script to be executed before a binary package is
removed.  See section 20 for more information about maintainer scripts.

10.9.  `build`
--------------

After execution of the `config` file, this file must exist.

This file is a makefile that controls the building of binary packages.  See
section 12 for requirements for this file.

10.10.  `changelog`
-------------------

This file is required.

This file is a log of changes made to the source package.  See section 13 for
the format of this file.

10.11.  `config`
----------------

This file is optional.

This file is a script in shell command language that configures the package
build.  See section 11 for requirements for this file.

10.12.  `control`
-----------------

This file is required.

This file contains control fields describing the source package.  See section 15
for the syntax of this file and section 16 for the list of control fields in
this file.

10.13.  `copyright`
-------------------

This file is required.

This file contains information about the copyrights in and licenses for the
source package.  This specification currently makes no requirements for the
format of this file.

10.14.  `format`
----------------

This file is required.

This "magic" file identifies the format version of the source package.  For
packages in this version of the source package format, this file should simply
contain the string `2.0`.

10.15.  `patches/`
------------------

This directory is optional.

This directory shall contain patches to be applied to package sources before
configuring and building any binary packages.

See section 15 for the syntax of patch files.

10.16.  `<pkgname>-<pkgver>.tar.<ext>`
--------------------------------------

TODO

10.17.  `platconf`
------------------

This file is optional.  Its presence indicates that each binary package is
platform-specific.

This file contains a list of build-time configuration files to be installed into
the package building work area.

See section 14 for the syntax of this file.

10.18.  `src/`
--------------

TODO


11.  Package Configuration Script Format
========================================

TODO


12.  Build File Format
======================

TODO


13.  Change Log Format
======================

Changes made to the source package should be explained in the file `changelog`.

Each new package revision must be documented with an entry of the following
format:

    package (version) distributions
    [zero or more blank lines]
      * change details
    [zero or more blank lines]
      * more change details
        more detailed change details
    [zero or more blank lines]
     -- maintainer  date

`package` is the source package name.  Source package names may only consist of
lowercase Latin letters, digits, plus and minus signs, and periods.  Names must
be at least two characters long and must start with either a letter or a digit.
(A regular expression for this is `[a-z0-9][a-z0-9+.-]+`.)

`version` is the source package version.  See section 17 for the syntax of
source package version identifiers.

`distributions` is a list of distributions into which the package should be
installed when uploaded to the package archive.  The list consists of one or
more distribution names, separated by spaces.

`maintainer` is the name and e-mail address of the package maintainer.  This
field must follow the syntax of the `mailbox` symbol of RFC 5322 section 3.4.

`date` is the date of packaging.  This field must follow the syntax of the
`date-time` symbol of RFC 5322 section 3.3.


14.  Platform Configuration File List Format
============================================

Platform-specific configuration files used by the source package at build time
shall be listed in the `platconf` file.  Platform-specific configuration files
used by the binary package(s) at run time shall be listed in the
`<binpkg>.pkg/platconf` file.

Source packages with a `platconf` file, a `<binpkg>.pkg/platconf` file, or both
shall list in their build-time dependencies a package (real or virtual) that
provides the listed platform-specific configuration files.

Each configuration file must be described with an entry of the following format:

    source destination

`source` is the path to the file, relative to the platform configuration
directory -- either `/usr/share/config/PLATFORM/PACKAGE-VERSION` or
`/usr/share/config/PLATFORM/PACKAGE`, where `PLATFORM` is the architecture
string denoting an application platform, `PACKAGE` is the name of the
configurable source package, and `VERSION` is the upstream version of the
configurable source package.

`destination` is the path (file or directory) to which the file should be
copied.  For a file used at build time, it is a path relative to the package
building work area.  For a file used at run time, it is an absolute path in the
user's filesystem hierarchy.


15.  Patch File Syntax
======================


16.  Control File Format
========================

TODO


17.  Source Package Version Identifier Syntax
=============================================

In general, the syntax of source package version identifiers is:

    <pkgver>[+sip<siprev>][-<pkgrev>][+<dist>-<distrev>]

`<pkgver>` is the version of the original upstream package or the native
package.  It may only consist of lowercase Latin letters, digits, periods, and
tildes.  It must be at least one character long.  (A regular expression for this
is `[0-9a-z.~]+`.)

If the upstream source archive needs to be repacked with certain changes for
compliance with the Software Inclusion Policy, the string `+sip<siprev>` must be
appended to `<pkgver>`.  `<siprev>` is a number that should be incremented on
each repack while `<pkgver>` remains constant.  It should be reset to `1` for
the first repack of a new upstream source archive.  It must be a string of one
or more digits, the first of which must be greater than or equal to `1`.  (A
regular expression for this is `[1-9][0-9]*`.)

`<pkgrev>` is the version of the distribution packaging.  It is optional and
should be omitted for native packages.  It should be incremented on each
revision of packaging while `<pkgver>` and `<siprev>` remain constant.  It
should be reset to `1` for the first revision of packaging for a new upstream
package version or source archive repack.  It must be a string of one or more
digits, the first of which must be greater than or equal to `1`.  (A regular
expression for this is `[1-9][0-9]*`.)

If the package is to be installed into a distribution other than `trunk`, the
string `+<dist>-<distrev>` must be appended to the end of the version
identifier.  `<dist>` is the distribution into which the package is to be
installed, may only consist of lowercase Latin letters and digits, and must be
at least one character long.  (A regular expression for this is `[a-z0-9]+`.)
`<distrev>` is a number that should be incremented on each upload to `<dist>`
while `<pkgver>`, `<siprev>`, and `<pkgrev>` remain constant.  It should be
reset to `1` for the first upload to `<dist>` of a new upstream package version,
source archive repack, or distribution packaging revision.  It must be a string
of one or more digits, the first of which must be greater than or equal to `1`.
(A regular expression for this is `[1-9][0-9]*`.)

18.  Source Package Metadata
============================

The fields in the source package metadata are:

  * `Maintainer` (required)  

    The name and e-mail address of the package maintainer.  This field must
    follow the syntax of the `mailbox` symbol in RFC 5322 section 3.4.

  * `Build-Depends` (optional)  

    A list of packages that must be installed before the package can be built.

  * `Homepage` (optional)  

    The URL of the Web site for the package.  Accessible at this site should be
    origin source code and documentation and/or information.  Though the
    information in this field is machine-usable, the URL must not be surrounded
    by angle brackets or any other characters.


19.  Binary Package Metadata
============================

The fields in the binary package metadata are:

  * `Architecture` (required)  

    A list of architectures for which this package may be built.  The value may
    be any one of the following:

    - The string `all` to specify that all of the provided files are
      architecture-independent;
    - The string `any` to specify that at least some of the provided files are
      architecture-dependent; or
    - A space-separated list of one or more real or wildcard binary architecture
      strings to specify that at least some of the provided files are only
      usable on systems of the named binary architecture(s).

    A "wildcard" binary architecture string is one in which one or two
    components (called "wildcard components") is the string `any`.  It shall be
    considered to match any real binary architecture string for which the
    non-wildcard components are equal.

  * `Platform` (required)

    A list of platforms for which this package may be built.  The value may be
    any one of the following:

    - The string `all` to specify that all of the provided files are
      platform-independent;
    - The string `any` to specify that at least some of the provided files are
      platform-dependent; or
    - A space-separated list of one or more platform strings to specify that at
      least some of the provided files are only usable on systems of the named
      platform(s).

  * `Essential` (optional)  

    A flag to indicate whether the package is essential for the functioning of a
    system on which it is installed.  If this field is set to `yes`, opkg will
    refuse to remove the package except when upgrading it.  If this field is set
    to any other value or is omitted, the package may be removed by a user.

  * `Depends` (optional)  

    A list of packages that must be installed and configured before the package
    may itself be configured.

  * `Recommends` (optional)  

  * `Suggests` (optional)  

  * `Pre-Depends` (optional)  

    A list of packages that must be installed before the package may itself be
    installed.

  * `Conflicts` (optional)  

  * `Provides` (optional)  

  * `Replaces` (optional)  

  * `Description` (required)  

    A description of the binary package.  This is a multiline field.  The first
    line is a short synopsis, and all following lines are an extended
    description.


20.  Maintainer Scripts
=======================

20.1.  Maintainer Script Format
-------------------------------

TODO

20.2.  Calling Standard
-----------------------

TODO


21.  Packaging Recommendations
==============================

21.1.  Source Package Directory Naming
--------------------------------------

It is recommended that the name of the source package directory be simply the
name of the source package.  Including the package version number or any part
thereof in the directory name is not recommended, as this number will likely
change over time.

21.2.  `build` File Generation
------------------------------

For non-trivial packages, it is recommended that the `config` script generates a
`build` makefile from some input template, named `build.in` or something
similar.  This is similar to the way that a `configure` script generated by GNU
Autoconf will produce a `Makefile` from `Makefile.in`.

With such makefile generation, the `build` file need not contain any conditional
logic (e.g. to test for cross building or to enable certain software features).
Such logic is supported by certain implementations of `make` including GNU Make.
However, it is not specified by POSIX.1 (and therefore cannot be expected to be
supported by all implementations).  Furthermore, conditional logic goes beyond
the original design of the `make` utility.

21.3.  Dynamic Binary Package Generation
----------------------------------------

For certain source packages, the set of binary packages to be built depends on
the build configuration.  For example, the target architecture of a cross
compiler may be embedded in the name of the providing binary package.  In such
cases it is recommended that binary package directories and their files be
generated by the `config` script from template directories named
`<binpkg>.pkg.in/` or similar.


22.  Example Files
==================

In the following sections are some example files that might be used for the
`opkg` source package.  They observe the recommendation made in section 21.2 to
make the `config` script generate a `build` makefile from an input `build.in`
file.

22.1.  Example `config` File
----------------------------

Following is an example `config` file:

    #! /bin/sh
    
    if [ "${OH_BUILD_ARCH_GNU}" = "${OH_HOST_ARCH_GNU}" ]; then
        ARCH_OPTS=
    else
        ARCH_OPTS="--build=${OH_BUILD_ARCH_GNU} --host=${OH_HOST_ARCH_GNU}"
    fi
    
    sed -e "s&@ARCH_OPTS@&${ARCH_OPTS}&" build.in >build

It detects whether the user intends to cross build the software and generates
appropriate `build` and `host` options for the software's `configure` script.

22.2.  Example `build.in` File
------------------------------

Following is an example `build.in` file:

    #! /usr/bin/make -f
    
    PKG_COMMON = libopkg.1
    
    configure: configure.stamp
    configure.stamp:
        cd src && ./configure --prefix=/usr @ARCH_OPTS@ \
            --disable-static --disable-curl --disable-ssl-curl --disable-gpg
        touch $@
    
    build: build.stamp
    build.stamp: configure
        cd src && make
        touch $@
    
    install: install.stamp
    install.stamp: build
        cd src && make DESTDIR="$${PWD}/../dest" install
        touch $@
    
    binary-arch: install
        oh-strip -g
        oh-installfiles
        oh-installdocs $(PKG_COMMON)
        oh-gencontrol
        oh-buildopk
    
    binary-indep: install
        oh-installfiles
        oh-installdocs $(PKG_COMMON)
        oh-gencontrol
        oh-buildopk
    
    binary: binary-arch binary-indep

It has the required `binary`, `binary-arch`, and `binary-indep` targets, along
with targets to configure, build, and install the packaged software.  It makes
use of build stamps to record that certain stages of the binary package build
process are completed.

Additionally, this file relies on the opkhelper utilities to perform tasks
related to the production of binary packages.


23.  Legal Notice
=================

Copyright (C) 2012 Patrick "P. J." McDermott

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


[posix-xcu-2.6.6]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_06