[REP Index] [REP Source]

REP:111
Title:Multiple Package Manager Support for Rosdep
Author:Tully Foote
Status:Final
Type:Standards Track
Content-Type:text/x-rst
Created:28-June-2011
ROS-Version:1.6
Post-History:30-Jun-2011

Abstract

This REP add support to the rosdep YAML format [1] to support multiple package managers on an individual platform. This includes a a new 'source' package manager that replacees the previous mechanisms for performing source-based installs.

Rationale

Previously, the rosdep format has enabled two modes of installation of system dependencies: the platform's preferred package manager, or a custom bash script. Both of these installation mechanisms were limiting.

Using only a single package manager limits integration options on platforms like OS X, where there are multiple mechanisms available for installing system libraries (e.g. MacPorts, distribute/easy_install). For example, for Python libraries, it can be beneficial to use distribute, which works with the default Python installation on OS X 10.6, instead of using the MacPorts Python libraries, which require additional setup by the user.

Similarly, the bash script specification was brittle and provided limited information to rosdep itself. The scripts were required to be idempotent because rosdep was unable to detect whether or not the dependency was already installed.

The goal of this REP is to make rosdep more robust and flexible, as well as enable better support of platforms other than Ubuntu.

Motivation

Existing Implementation

The current implementation of rosdep in ROS Diamondback [2] supports the following structures:

  • Simple

    ROSDEP_NAME:
      OS_NAME1: PACKAGE_ARGUMENTS_A
      OS_NAME2: PACKAGE_ARGUMENTS_A
    
  • Versioned

    ROSDEP_NAME:
      OS_NAME1:
        OS_VERSION1:  PACKAGE_ARGUMENTS_A
        OS_VERSION2:  PACKAGE_ARGUMENTS_A2
    

The names above resolve as follows:

  • ROSDEP_NAME is the name referred to by manifest files. Examples: log4cxx, gtest.
  • OS_NAME is the name of an OS. Examples: ubuntu, fedora, debian, windows, macports.
  • OS_VERSION (''optional'') is the name of specific versions in the OS. Examples: '10.04', squeeze. If no OS_VERSION is specified, the rule is assumed to apply to all versions.
  • PACKAGE_ARGUMENT has two interpretations. If it is a single line, it is expected to be a list of packages to pass to the package manager for the OS. If this field is a multi-line script, it is an idempotent bash script that is executed to install the dependency.

Example:

python:
  ubuntu: python-dev
  fedora: python-devel
  macports: py26-scipy
boost:
  ubuntu:
    '11.04': libboost.42-all-dev
    '10.10': libboost.42-all-dev

Support for Multiple Package Managers

For many Linux-based platforms, there are dominant package managers that are the main mechanism for deploying software, such as deb/apt for Debian-based systems and rpm/yum for RedHat-based systems. However, for non-Linux platforms, as well as for cross-platform language environments (e.g. Python, Java, Ruby), there exist other software deployment mechanisms that are commonly used. There are also platforms like OS X and Windows, where no package manager has a dominant role.

On OS X, rosdep currently uses MacPorts as the default package manager. While MacPorts does provide most of the libraries necessary for integrating ROS on an OS X system, the current implementation limits the flexibility of integration options. One area of integration we wish to improve is Python. The MacPorts Python integration requires a separate Python tree from the default OS X Python interpreter. This has proven to be very brittle for users and a common are of misconfiguration. Now that OS X now deploys with a Python version that is compatible with ROS, we wish to explore using distribute/easy_install for Python libraries instead.

Richer information from scripting interface

The current support for bash scripts requires these scripts to be idempotent: they must detect the presence of the required dependency and install it if it is not present. There is no mechanism for rosdep to determine for itself whether the required dependency is installed, nor is there any mechanism for removing dependencies installed via this mechanism. This creates confusion on the part of the user as there is no mechanism for analyzing whether or not these dependencies are installed correctly. Creating more structure for source-based installations will enable simpler installation scripts, better error reporting, and more advanced features (e.g. uninstall) in the future.

Integration with OS X

OS X dependencies are specified generally as macports and are not versioned to a particular OS X release. OS X releases frequently have significant differences as Apple does not maintain much backwards compatibility.

The current ROS OS X integration suffers from 32-bit/64-bit compatibility problems due to the OS X 10.6 release. The current rosdep specification also does not allow integrators to take advantage of the builtin Python 2.5+ interpeter that was introduced in the OS X 10.5 release.

Another limitation of the OS X integration is that the rosdep syntax did not allow for specification of MacPorts "variants" for individual packages. The rosdep syntax interpreted the package argument value as a list of packages, which was not flexible enough for specifying variants on specific packages.

OS version identifiers

The current OS version identifiers use the release number (e.g. 11.04) instead of the release codename (e.g. lucid). Using the release number has been a frequent source of bugs as YAML interprets 11.04 as a floating point number instead of a string. This requires explicit string escaping when specifying rules (e.g. '11.04').

Specification

The new specification for rosdep introduces a PACKAGE_MANAGER key in the YAML specification, which is a child key of the OS_NAME. rosdep no longer interprets the PACKAGE_MANAGER_ARGUMENTS and instead passes them to the specified PACKAGE_MANAGER handler.

  • Simple
    ROSDEP_NAME:
      OS_NAME1:
        PACKAGE_MANAGER1:
          PACKAGE_ARGUMENTS_A
      OS_NAME2:
        PACKAGE_MANAGER2:
          PACKAGE_ARGUMENTS_A
    
  • Versioned
    ROSDEP_NAME:
      OS_NAME1:
        OS_VERSION1:
          PACKAGE_MANAGER1:
            PACKAGE_ARGUMENTS_A
        OS_VERSION2:
          PACKAGE_MANAGER2:
            PACKAGE_ARGUMENTS_A2
    

The names above resolve as follows:

  • ROSDEP_NAME is the name referred to by manifest files. Examples: log4cxx or gtest.
  • OS_NAME is the name of an OS. Examples: ubuntu, osx, fedora, debian, or windows.
  • OS_VERSION (optional) is the name of specific versions in the OS. Examples: lucid or squeeze. If no OS_VERSION is specified, the rule is assumed to apply to all versions.
  • PACKAGE_MANAGER (optional in ROS Electric, required in ROS Fuerte) is a key to select which package manager to use for this rosdep. Examples: apt, easy_install, macports.
  • PACKAGE_ARGUMENT is free-form YAML that is be passed to the handler for the specified PACKAGE_MANAGER.

Example

For Ubuntu the default package manager is apt. The new syntax is:

rosdep_name:
  ubuntu:
    apt:
      packages: [ debian-package-name, other-debian-package-name]

or versioned as follows:

rosdep_name:
  ubuntu:
    lucid:
      apt:
        packages: [debian-package-name, other-debian-package-name]

OS version identifiers

OS version identifiers will be standardized onto one-word codenames for each distro.

For example:

  • debian: squeeze
  • ubuntu: lucid, maverick, natty, oneiric
  • osx: snow, lion

Changes to OS X

The OS_NAME key for OS X will be changed to osx and will start including OS_VERSION keys that represent the version number of the OS X releases (e.g. lion). The default package is macports, which can be used as a PACKAGE_MANAGER field.

osx:
  lion:
    macports: bzip2

Disambiguation of OS_VERSION and PACKAGE_MANAGER

For backwards compatibility, the PACKAGE_MANAGER is allowed to be optional in the ROS Electric case. As both PACKAGE_MANAGER and OS_VERSION are optional, this creates an ambiguious case where either OS_VERSION or PACKAGE_MANAGER is specified, but not both.

In this ambiguous case, rosdep first interprets the key as a PACKAGE_MANAGER. If this test fails, it will be interpreted as an OS_VERSION. Developers should exercise caution in keeping OS_VERSION and PACKAGE_MANAGER keys globally distinct.

Dependencies

Although not specified in this REP, developers of new package managers for rosdep are encouraged to include a dependency specification in their PACKAGE_ARGUMENTS values. This dependency specification should enable rosdep rules to specify dependencies on other rosdep rules. This, for example, would enable an easy_install package to also depend on a apt-based install. One use case of this is rosinstall, which can be installed via easy_install, but also implicitly requires non-Python libraries for Git, Subversion, Mercurial, and other non-easy_install packages.

This advisory does not apply to package managers that are builtin to a particular platform, such as apt and yum.

Internally, the rosdep library will provide APIs for developers to trigger installation of these dependencies.

New source-based package manager

A new source-based package manager is discussed in the motivation for this REP. This specification of this new source-based package manager will be described in a separate forthcoming REP.

Single package manager per rule

rosdep rules are only allowed to specify a single package manager to fulfill them.

Backwards Compatibility

All current rosdep.yaml files are compatible with the new proposed syntax, though some features will be deprecated and the files should be upgraded during the ROS Electric cycle to maintain compatibility.

Default Package Manager

If no package manager is specified, the previous default package manager will be used.

Defaults:

  • ubuntu: apt
  • debian: apt
  • mint: apt
  • osx: macports
  • freebsd: port

Support for the default package manger will be maintained upto and including ROS Fuerte.

Bash scripts

Multi-line values will still be treated as bash scripts in ROS Electric. Support for this will be removed in ROS Fuerte.

Macports

The macports rules will be supported during ROS Electric.

Ubuntu Codename Usage

For the following codenames backwards compatabilty will be built into the tool to automatically translate rules:

  • '10.04' -> lucid
  • '10.10' -> maverick
  • '11.04' -> natty

It is expected that all future releases will refer to the correct codename, e.g. 'oneiric'.

This will be supported through ROS Fuerte.