[REP Index] [REP Source]

REP:149
Title:Package Manifest Format Three Specification
Author:Dirk Thomas
Status:Draft
Type:Standards Track
Content-Type:text/x-rst
Created:11-Oct-2017

Outline

  1. Abstract
  2. Motivation
  3. Rationale
  4. Data Representation
  5. Compatibility
  6. Schema
  7. References
  8. Copyright
The current draft highlights significant changes compared to REP-0140 [2]_ in red. Trivial changes appear in blue.

Abstract

This REP specifies the third package.xml format, which is an update to the previous versions specified in REP-0127 [1] and REP-0140 [2].

Motivation

This REP covers three separate topics which are described in the following subsections.

Dependency groups

Several ROS packages aggregate dependencies of a specific type, e.g. message_generation in ROS 1 or rmw_implementation in ROS 2. They explicitly list the dependencies which are being used during the release process. But when building from source they need to be updated to add custom message generators or rmw implementations.

The goal is to allow specifying this kind of group dependency in a way that a from-source build can process the custom packages accordingly to make them available to the package even though the package doesn't list an explicit dependency on the custom package.

ABI version on the package level

When building binary packages on the ROS buildfarm there is no assumption about ABI compatibility. When a new Debian package has been built all downstream packages are being rebuilt too in order to ensure they still work together. This makes e.g. releasing a core package like catkin or roscpp particularly costly. Due to that overhead changes which should be released rather sooner than later, e.g. a documentation fix or simple bugfix in a .cpp file, are being held back to group them with other changes to make the rebuild effort worthwhile.

In order to allow package releases without enforcing a rebuild of all downstream packages a package should be able to declare if such a rebuild is necessary or not.

Dependencies across ROS 1 and ROS 2

In order for a ROS package to work with ROS 1 and ROS 2 from a single source the dependencies listed in the package manifest are a problem. E.g. in ROS 1 a package needs to depend on roscpp where as in ROS 2 it needs to depend on rclcpp. Currently the manifest doesn't support this use case and requires two different code bases (usually two separate branches).

The necessary part outside of the manifest to make a package compatible with ROS 1 as well as ROS 2 is explicitly not part of this document. In general it is possible to conditionally handle those cases in programming languages like CMake, C++, Python, etc.

Rationale

REP-0127 [1] and REP-0140 [1] provide the package.xml design rationale for format one and two, which is not repeated here.

Group dependencies

Packages can already express direct dependencies on other individual packages, but this is not sufficient for cases where a package needs to be built after all instances of a particular kind of package. Instead packages need to be able to declare a dependency on packages without using their name directly.

One alternative is to "reverse" the dependency declaration so that a package states that it "wants" to be a dependency of another package. But that approach does not work in the case where the other package gets forked under a different name since the "reverse" dependency would still only reference the original package name.

Therefore the dependency declaration is being decoupled into two parts:

  • (A) packages declaring a dependency on a "group dependency name", e.g. ros1_bridge can depend on the group "message_packages"
  • (B) packages declaring to be part of a group dependency identified by its name, e.g. sensor_msgs can declare itself part of the "message_packages" group

During a from-source build the build tool can use the information from all package manifests in the workspace to identify group dependencies. It can use this information to process group members before the packages declaring the group dependency as if the dependency was declared explicitly using the existing depend tags.

The group dependency (A) is declared with a group_depend tag. The content of the group dependency tag is the name of the group. Leading and trailing whitespace is being ignored from the name and for consistency it is required to follow the naming rules for packages.

The membership of a group (B) is declared with a member_of_group tag.

ABI version attribute

The early draft of REP 127 ("package.xml format 1") proposed an attribute (abi_version) to identify the package version which is considered "compatible".

Since at the time no consensus could be reached about specifying ABI compatibility in general (for packages as well as libraries) the proposed attribute was removed from the draft (see related discussions [13], [14], [15]).

Since then the number of packages in a ROS distributions has increased (e.g. Indigo has more than 2500 packages) as well as the number of platforms ROS provides binary packages for. Therefore the need to avoid unnecessary rebuilds has increased. It is also desired to be able to encourage more frequent releases if they don't require downstream packages to be rebuilt.

Dependencies across ROS 1 and ROS 2

The differences in dependencies in ROS 1 and ROS 2 could be handled in various ways.

Use the same package names

One option (A) would be to not allow different dependencies depending on the ROS version. That would imply that for the previously described example of roscpp and rclcpp a "dummy" package could be provided to provide the same package names. This could be either a ROS 1 package named rclcpp which simply depends on roscpp or a ROS 2 package named roscpp which depends on rclcpp. That would allow all downstream packages to use a single name as their dependency. While that would be possible it would require dummy packages for every naming difference between ROS 1 and ROS 2.

But option (A) would become even more cumbersome if a package needs to declare a dependency only in one of the ROS versions. It would require a dummy package in both ROS version where as in one of them the dummy would be empty and in the other it would declare a dependency.

Annotate dependencies

Another option (B) is to annotate the dependencies in the manifest if they are specific to a ROS version. The annotation could happen in various ways in the markup. One way is an optional attribute for each dependency tag. Another way would be a new conditional tag under which one or multiple dependency tags would be grouped. In case a dependency doesn't have such an annotation it is being used unconditionally.

In this case various tools will need to be aware of the condition choosing which dependencies should be used:

  • bloom
  • rosdep
  • rosinstall_generator
  • the build tool

The first three tools are all operating on a specific ROS distribution. Assume that a ROS distribution "knows" which ROS version it represents (which would require adding a new field to the distribution file specified in REP 143 [16]) the necessary information should be available to decide the condition.

The build tool does not have access to the ROS distribution metadata. It could either use information provided by an environment variable or fall back to consider all dependencies (independent of their annotation) and work on the assumption that the workspace contains a set of consistent packages and additional dependencies from other ROS versions are not available in the workspace.

Data Representation

The package manifest is an XML file, with restricted syntax.

The only top-level element allowed is <package>. Immediately subordinate to that are several required or optional elements, defined here. No other tags are permitted directly under the <package> element.

<package format="3">

The <package> tag is the unique top-level tag in a package.xml file. All other tags are nested under it.

Attributes

format="NUMBER"

Specifying the package.xml format being used. If not set, format="1" would be assumed, which is not the format described here. For this interface, you must specify format="3". If you are using a different format, please refer to the relevant specification. REP-0127 [1] described format one, REP-0140 [12] format two.

Optional Tags

Meta-information

Optional, but recommended, meta-data include the names of the original authors and links to support documentation.

Dependencies

The dependencies and relations to other packages and system packages have been discussed in [7]. They are described using:

These dependency tags are used with both system packages and ROS packages. For system dependencies specify the rosdep key name, for ROS dependencies use the package name.

The dependency graph must be acyclic. No package may directly or indirectly depend on itself.

Group dependencies and memberships

The group dependencies and membership of groups are described using:

These group dependencies are only applied in from-source builds and are being ignored in the release process.

The dependency graph must be acyclic even when considering group dependencies.

Various

There is a need for additional meta-data in the manifest for other tools that work with packages, like message generators and plugin discovery. Tags for that kind of information are wrapped within this tag:

Some <export> tags used by catkin are defined below. Others are defined by various tools, which must specify their own specific tag structures.

Example

<package format="2">
  <name>my_package</name>
  <version>1.2.3</version>
  <description>
    This is my package's description.
  </description>
  <maintainer email="someone@example.com">Someone</maintainer>

  <license>BSD</license>
  <license>LGPL</license>

  <url type="website">http://wiki.ros.org/my_package</url>
  <url type="repository">http://www.github.com/my_org/my_package</url>
  <url type="bugtracker">http://www.github.com/my_org/my_package/issues</url>
  <author>John Doe</author>
  <author email="jane.doe@example.com">Jane Doe</author>

  <buildtool_depend>catkin</buildtool_depend>
  <build_depend version_gte="1.1" version_lt="2.0">genmsg</build_depend>

  <depend>roscpp</depend>

  <build_depend>libgstreamer0.10-dev</build_depend>
  <build_export_depend>libgstreamer0.10-dev</build_export_depend>
  <exec_depend>libgstreamer0.10-0</exec_depend>

  <test_depend>gtest</test_depend>

  <doc_depend>doxygen</doc_depend>

  <conflict>alternative_implementation</conflict>

  <replace>my_old_package</replace>

  <export>
    ...
  </export>
</package>

<name>

The package name must start with a letter and contain only lowercase alphabetic, numeric or underscore characters [2]. The package name should be unique within the ROS community. It may differ from the folder name into which it is checked out, but that is not recommended.

The following recommended exemptions apply, which are optional for implementations:

  • Dashes may be permitted in package names. This is to support maintaining a consistent dependency name when transitioning back and forth between a system dependency and in-workspace package, since many rosdep keys contain dashes (inherited from the Debian/Ubuntu name).
  • In support of some legacy packages, capital letters may also be accepted in the package name, with a validation warning.

<version>

The version number of the package in the format MAJOR.MINOR.PATCH where each part is numeric only.

Attributes

compatibility="MAJOR.MINOR.PATCH"

Specifying the version up to which the package is compatible with, i.e. always pick the oldest compatible version. If not set, the same value as specified in the version tag is assumed.

"Compatibility" in this context guarantees that downstream packages built against the older version will continue to work with a newer version without the need to be rebuilt. This includes but is not limited to ABI compatibility. Changes in other parts of a package (CMake, Python, etc.) could also require downstream packages to be rebuilt and therefore not qualify as "compatible".

<description>

The description of the package. It can consist of multiple lines and may contain XHTML. But depending on where the description is used XML tags and multiple whitespaces might be stripped.

<maintainer> (multiple, but at least one)

The name of the person maintaining the package. All packages require a maintainer. For orphaned packages see below.

Attributes

email="name@domain.tld" (required)

Email address of the maintainer.

An orphaned package is one with no current maintainer. Orphaned packages should use the following maintainer information to guide volunteers how they can claim maintainership:

Example

<maintainer email="ros-orphaned-packages@googlegroups.com">Unmaintained see http://wiki.ros.org/MaintenanceGuide#Claiming_Maintainership</maintainer>

<license> (multiple, but at least one)

Name of license for this package, e.g. BSD, GPL, LGPL. In order to assist machine readability, only include the license name in this tag. For multiple licenses multiple separate tags must be used. A package will have multiple licenses if different source files have different licenses. Every license occurring in the source files should have a corresponding <license> tag. For any explanatory text about licensing caveats, please use the <description> tag.

Most common open-source licenses are described on the OSI website.

Commonly used license strings:

  • Apache 2.0
  • BSD
  • Boost Software License
  • GPLv2
  • GPLv3
  • LGPLv2.1
  • LGPLv3
  • MIT
  • Mozilla Public License Version 1.1

<url> (multiple)

A Uniform Resource Locator for the package's website, bug tracker or source repository.

It is a good idea to include <url> tags pointing users to these resources. The website is commonly a wiki page on ros.org where users can find and update information about the package.

Attributes

type="TYPE" (optional)

The type should be one of the following identifiers: website (default), bugtracker or repository.

<author> (multiple)

The name of a person who is an author of the package, as acknowledgement of their work and for questions.

Attributes

email="name@domain.tld" (optional)

Email address of author.

Dependency tags

<build_depend> (multiple)

Declares a rosdep key or ROS package name that this package requires at build-time. For system packages, the rosdep key will normally specify the "development" package, which frequently ends in "-dev".

The build and buildtool dependencies are used to determine the build order of multiple packages.

Attributes

All dependencies and relationships may restrict their applicability to particular versions. For each comparison operator an attribute can be used. Two of these attributes can be used together to describe a version range.

version_lt="VERSION" (optional)

The dependency to the package is restricted to versions less than the stated version number.

version_lte="VERSION" (optional)

The dependency to the package is restricted to versions less or equal than the stated version number.

version_eq="VERSION" (optional)

The dependency to the package is restricted to a version equal than the stated version number.

version_gte="VERSION" (optional)

The dependency to the package is restricted to versions greater or equal than the stated version number.

version_gt="VERSION" (optional)

The dependency to the package is restricted to versions greater than the stated version number.

condition="CONDITION_EXPRESSION"

Every dependency can be conditional on a condition expression. If the condition expression evaluate to "true" the dependency is being used and considered as if it doesn't have a condition attribute. If the condition expression evaluate to "false" the dependency is being ignored and considered as if it doesn't exist.

The condition expression can consist of:

  • parenthesis (which must be balanced)
  • logical operators and and or
  • comparison operators == and !=
  • variable names which start with a $ sign and are followed by alphanumerics and underscores
  • literals which can only contain alphanumerics, undercsores and dashes
  • arbitrary whitespaces between these tokens

An expression syntactically correct by the previous definition will be evaluated as follows:

  • All variables are substituted by their values and treated as strings.
  • All literals are also treated as strings.
  • The resulting expression is evaluated as a Python interpreter would evaluate it.

Tools may populate the values for the variables starting with a $ sign in different ways, but typically they are evaluated as environment variables.

As an example, a dependency might only be needed in a ROS 1 environment. Such dependency could be described as follows where the value of $ROS_VERSION is coming from an environment variable:

<depend condition="$ROS_VERSION == 1">roscpp</depend>

<build_export_depend> (multiple)

Declares a rosdep key or ROS package name that this package needs as part of some build interface it exports. For system packages, the rosdep key will normally specify the "development" package, which frequently ends in "-dev".

The <build_export_depend> declares a transitive build dependency. A common example is when one of your dependencies provides a header file included in some header exported by your package. Even if your package does not use that header when building itself, other packages depending on your header will require those transitive dependencies when they are built.

Attributes

The same attributes as for <build_depend> (multiple).

<buildtool_depend> (multiple)

Declares a rosdep key or ROS package name for a tool that is executed during the build process. For cross-compilation, one must distinguish these from normal build dependencies, which may be linked with your package and must be compiled for the target architecture, not the build system. For system packages, the rosdep key will normally specify the "development" package, which frequently ends in "-dev".

Attributes

The same attributes as for <build_depend> (multiple).

<buildtool_export_depend> (multiple)

Declares a rosdep key or ROS package name that this package exports which must be compiled and run on the build system, not the target system. For cross-compilation, one must distinguish these from target build dependencies, which may be linked with your package and must be compiled for the target architecture.

Attributes

The same attributes as for <build_depend> (multiple).

<exec_depend> (multiple)

Declares a rosdep key or ROS package name that this package needs at execution-time. For system packages, the rosdep key will normally not specify the "development" package, so it will generally lack the "-dev" suffix.

The <exec_depend> is needed for packages providing shared libraries, executable commands, Python modules, launch scripts or any other files required for running your package. It is also used by metapackages for grouping packages.

Attributes

The same attributes as for <build_depend> (multiple).

<depend> (multiple)

Declares a rosdep key or ROS package name that this package needs for multiple reasons. A <depend> tag is equivalent to specifying <build_depend>, <build_export_depend> and <exec_depend>, all on the same package or key. The <depend> tag cannot be used in combination with any of the three equivalent tags for the same package or key name.

Attributes

The same attributes as for <build_depend> (multiple).

<doc_depend> (multiple)

Declares a rosdep key or ROS package name that your package needs for building its documentation. A <doc_depend> may reference a package also declared as some other type of dependency.

The current version of the buildsystem does not provide any documentation specific functionality or targets but may do so in the future, similar to how the unit tests are integrated into the configure and make steps. Other infrastructure (like the documentation jobs on the buildfarm) will utilize these additional doc dependencies.

Generated Debian packages are built without the documentation or the documentation dependencies.

Attributes

The same attributes as for <build_depend> (multiple).

<test_depend> (multiple)

Declares a rosdep key or ROS package name that your package needs for running its unit tests. A <test_depend> may reference a package also declared as some other type of dependency.

All tests and their dependencies will be built if the CMake variables CATKIN_ENABLE_TESTING=1 and CATKIN_SKIP_TESTING=0, the default settings. CMakeLists.txt should only define its test targets when CATKIN_ENABLE_TESTING=1 [9].

When building with testing enabled, the <test_depend> packages are available for configuring and building the tests as well as running them. Generated Debian packages are built without the unit tests or their dependencies.

Attributes

The same attributes as for <build_depend> (multiple).

<conflict> (multiple)

Declares a rosdep key or ROS package name with which your package conflicts. This package and the conflicting package should not be installed at the same time. This has no effect on source builds, but maps to Conflicts when creating Debian or RPM packages.

For a detailed explanation how these relationships are used see [4] and [5].

Attributes

The same attributes as for <build_depend> (multiple).

<replace> (multiple)

Declares a rosdep key or ROS package name that your package replaces. This has no effect on source builds, but maps to Replaces when creating Debian packages and Obsoletes for RPM packages.

Attributes

The same attributes as for <build_depend> (multiple).

Group dependency tags

<group_depend> (multiple)

The content is the name of a dependency group on which the package depends. The group name should follow the same rules as the <name> tag.

Attributes

The condition attribute as defined for <build_depend> (multiple).

<member_of_group> (multiple)

The content is the name of a dependency group of which the package is a member.

Attributes

The condition attribute as defined for <build_depend> (multiple).

<export>

This tag serves as a container for additional information various packages and subsystems need to embed. To avoid potential collisions, an export tag should have the same name as the package which is meant to process it. The content of that tag is up to the package to define and use.

Existing rosbuild export tags for tools using pluginlib remain unchanged. For example, a package which implements an rviz plugin might include this:

<export>
  <rviz plugin="${prefix}/plugin_description.xml"/>
</export>

The following are some tags used within an <export> for various package and message generation tasks.

<architecture_independent/>

This empty tag indicates that your package produces no architecture-specific files once built. That information is intended for allowing optimization of packaging.

Specifying <architecture_independent/> is recommended for metapackages and for packages defining only ROS messages and services. Python-only packages are reasonable candidates, too. It is not appropriate for any package which compiles C or C++ code.

Be sure to remove this tag if some subsequent update adds architecture-dependent targets to a formerly independent package.

<build_type>

Various tools use this tag to determine how to handle a package. It was defined in REP-0134 [10], which currently specifies only two valid values:

<build_type>catkin</build_type>
<build_type>cmake</build_type>

If no <build_type> is provided, catkin is assumed.

When the build type is cmake, the package is handled as a non-catkin CMake project. It cannot be included in a normal catkin workspace, but can instead use catkin_make_isolated, which configures and builds a different kind of workspace in which cmake, make, and make install are invoked separately for each package. See REP-0134 for details.

Further build types may eventually be defined, such as: "make", "autotools", "rosbuild", or "custom".

<deprecated>

This tag indicates that your package is deprecated, enabling tools to notify users about that fact. The tag may be empty or may optionally contain an arbitrary text providing user more information about the deprecation:

<export>
  <deprecated>
    This package will be removed in ROS Hydro. Instead, use package
    FOO, which provides similar features with a different API.
  </deprecated>
</export>

<message_generator>

The content defines the identifier for the language bindings generated by this package, i.e. in gencpp this is set to cpp:

<export>
  <message_generator>cpp</message_generator>
</export>

<metapackage/>

This empty tag declares a special kind of catkin package used for grouping other packages. Metapackages only provide execution-time dependencies. They cannot be used for catkin builds and compile nothing themselves. Metapackages may not install any code or other files, although package.xml does get installed automatically. They can depend on other metapackages, but regular catkin packages cannot.

A good use for metapackages is to group the major components of your robot and then provide a comprehensive grouping for your whole system. Package installation tools like apt-get or yum can automatically install all the packages on which a metapackage directly or indirectly depends. Metapackages can also be used to resolve dependencies declared by legacy rosbuild stacks not yet converted to catkin.

Every metapackage must have a CMakeLists.txt containing these commands:

cmake_minimum_required(VERSION 2.8.3)
project(PACKAGE_NAME)
find_package(catkin REQUIRED)
catkin_metapackage()

Because the metapackage CMakeLists.txt contains a catkin macro, its package.xml must declare a buildtool dependency on catkin:

<buildtool_depend>catkin</buildtool_depend>

Additional buildtool, build or test dependencies are not permitted.

Because metapackages only supply execution-time dependencies, they use <exec_depend> to list the packages in their group:

<exec_depend>your_custom_msgs</exec_depend>
<exec_depend>your_server_node</exec_depend>
<exec_depend>your_utils</exec_depend>
<exec_depend>another_metapackage</exec_depend>

Compatibility

Support for format three

Existing tools supporting up to format two will need to be updated to honor the new information provided by format three. If they are not checking that the format is two or lower they will simply ignore the format three specific information when processing a package with a format three manifest.

In order to enable packages to declare different dependencies for ROS 1 and ROS 2 in a single manifest the tools in ROS 1 (catkin_pkg, rosdep, bloom, etc.) should be updated to support format 3.

catkin_prepare_release

As long as a manifest doesn't specify a compatibility version the tools doesn't need to change. For packages which do specify a compatibility version the tool should probably by default remove the attribute and only after confirmation from the user offer to keep it.

ros_buildfarm

The dependencies between packages is directly mapped to upstream / downstream jobs in Jenkins. In order to consider the compatibility attribute the downstream job dependencies must be changed to be conditional which are only triggered when the new package version is not compatible to the previous version.

Backward compatibility

Format one or two packages following REP-0127 [1] or REP-0140 [12] are not affected unless they are updated to declare <package format="3">.

Since format three only adds new functionality and doesn't modify any existing markup a migration to format three only makes sense when the package wants to use any of the new features.

Schema

A schema defining the structure specified in this document is available at [11]. To specify the schema within a manifest you can reference a self contained schema file like this:

<?xml version="1.0"?> <?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?> <package format="3">

References

[1](1, 2, 3, 4, 5) REP-0127 (http://ros.org/reps/rep-0127)
[2](1, 2) ROS naming conventions (http://www.ros.org/wiki/ROS/Patterns/Conventions#Naming_ROS_Resources)
[3]ros-infrastructure/catkin_pkg#43: "add support for depend tag"
[4]Declaring relationships between packages (Debian Policy Manual) (http://www.debian.org/doc/debian-policy/ch-relationships.html)
[5]Advanced RPM Packaging (Fedora Documentation) (http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-advanced-packaging.html)
[6]Buildsystem mailing list discussion: "adding <depend> syntax to package.xml"
[7]Buildsystem mailing list discussion: "Dependency tag types for REP 127"
[8]Buildsystem mailing list discussion: "dev/non-dev packages and required meta information"
[9]Buildsystem mailing list discussion: "REP-0140: internal review"
[10]REP-0134 (http://ros.org/reps/rep-0134)
[11]Schema file (https://github.com/ros-infrastructure/rep/blob/master/xsd/package_format3.xsd)
[12](1, 2) REP-0140 (http://ros.org/reps/rep-0140)
[13]Discussion on REP-0127 (https://groups.google.com/forum/#!topic/ros-sig-buildsystem/_jRvhXFfsVk)
[14]Related topic of versioning ROS libraries (https://groups.google.com/forum/#!topic/ros-sig-buildsystem/Q9BK3MGFY_U)
[15]SO versioning from a package perspective (https://groups.google.com/forum/#!topic/ros-sig-buildsystem/jTB9r3zu580)
[16]REP-0143 (http://ros.org/reps/rep-0143)
[17]ROS_DISTRO environment variable (https://github.com/ros/ros/blob/b202645dc6bea6d4b9ca408dc703c8c7cc8204d9/core/roslib/env-hooks/10.ros.sh.em#L16)