Configuring WGE100 Cameras

The various components involved in PR2 camera synchronization expose a large number of options to the user. This section is designed to highlight the options that the user should be using to configure the cameras on the PR2. All the necessary nodes and controllers to work with the WGE100 cameras are brought up automatically when the pr2 robot is started.

It is not recommended that the user adjust parameters that are not listed below as they are likely to leave the system in a non functioning state.

Setting Frame Rates and Projector Modes

Frame rates and projector modes should be set using dynamic_reconfigure on the pr2_camera_synchronizer_node. For example:

rosrun dynamic_reconfigure reconfigure_gui /camera_synchronizer_node

Setting Resolution, Gain, Exposure, Brightness, Companding

(Resolution is currently broken)

Gain, Exposure, Brightness, Companding and Resolution are set using dynamic_reconfigure. For the forearm cameras, the camera node should be configured directly. For the stereo pairs, the wge100_multi_configurator node should be configured; this node controls both cameras of a stereo pair, and will dispatch configuration changes to both underlying camera nodes.

For the wide stereo camera:

rosrun dynamic_reconfigure reconfigure_gui /wide_stereo_cfg

For the narrow stereo camera:

rosrun dynamic_reconfigure reconfigure_gui /narrow_stereo_cfg

For a forearm camera (right in the example):

rosrun dynamic_reconfigure reconfigure_gui /forearm_camera_node

Image Topics

Images from the wide stereo camera and forearm cameras are published in the wide_stereo, forearm_r and forearm_l namespaces, whether they are textured or not.

To facilitate using the narrow stereo camera in AlternatingProjector mode, textured images from the narrow stereo camera are always published in the narrow_stereo_textured namespace, and untextured images are always published in the narrow_stereo namespace.

Hard Reset of the WGE100 Cameras

To reset the cameras, set the camera_reset option to true in the camera_synchronizer_node options. The reset will take about 5-10 seconds.

rosrun dynamic_reconfigure dynparam /camera_synchronizer_node camera_reset true

Note: This will only work if the trigger controllers are working.

Overview

Hardware Components

The following hardware components are involved in producing vision data on the PR2:

• A color wge100 camera on each forearm.

• A wide field-of-view color wge100 stereo camera pair on the head.

• A narrow field-of-view black and white wge100 stereo camera on the head.

• A high resolution prosilica_camera camera on the head.

• A texture projector on the head to project a texture on objects that are being viewed by the stereo cameras.
• Digital outputs on the PR2 motor controller boards that are used to synchronously trigger cameras.

Synchronization Constraints

Synchronization between these pieces of hardware is needed because of system constraints:

• The stereo camera pairs are made up of two independent monocular WGE100 cameras. In order to get synchronized triggering of these cameras, they must be triggered from a common source.
• A single trigger signal is fed to both stereo camera pairs. The triggering protocol allows the stereo cameras to have different exposure phases, but their frame rate must match.
• Cameras must be triggered synchronously with the texture projector. To get a textured image the exposure must coincide with projection. To get an untextured image the exposure must not intersect with projection.
• The texture projector must be fired at at least 40Hz to avoid very unpleasant flickering.
• The Prosilica camera does not have a trigger line, and is usually used with long exposure times. Hence, to get untextured images from the Prosilica camera, the normal texture projector firing must be inhibited during Prosilica camera exposures. An exposure line from the Prosilica camera is connected to an inhibit input on the texture projector controller board to allow this inhibition to occur.

Software Components

The variety of hardware components involved implies that a variety of software components are involved in producing camera images:

• A wge100_camera_node per WGE100 camera.

• A wge100_multi_configurator for each stereo pair, which replicates its parameters to each camera's driver.

• A prosilica_node for the Prosilica camera.

• pr2_etherCAT with suitable controllers running:

  • One MultiTriggerController per WGE100 camera trigger line. I.e., one for each forearm camera, and one for both of the stereo cameras.

  • Two MultiTriggerController for the projector, one to trigger the projector and one to determine whether to activate inhibition by the prosilica camera.

  • One ProjectorController to enable the projector.

• A pr2_camera_syncronizer node in charge of setting the triggers and camera drivers into triggering states that are consistent with the user's desires.

Synchronization Strategy

Projector Waveform

The pr2_camera_synchronizer operates with a standard waveform that repeats after four pulses. The pulses in one period are numbered 1 to 4. The waveform is entirely determined by three parameters:

• projector_rate: The average pulse rate.

• projector_pulse_length: The length of each pulse. The projector driver protection circuitry may in some cases limit the pulse length to be lower than this value.

• projector_pulse_shift: Introduces asymmetry in the pulses. It varies from 0 to 1 with increasing asymmetry.

The timing of the pulses is as follows, where 'T' is the average pulse period:

• Pulses 1 and 3 are spaced by exactly '2T'. Cameras in the WithProjector mode expose during pulses 1 and 3.

• Pulses 2 and 4 are spaced by about one pulse-length more than '2T'. Cameras that are in the AlternatingProjector mode expose during pulse 1, and just before pulse 2.

• The spacing between pulses 1 and 2 is determined by projector_pulse_shift and varies from 'T' to just under one pulse length less than '2T'. Cameras that are in the WithoutProjector have their exposure end just before pulse 2. The start of the exposure depends on the exposure time, but is constrained to happen after the end of pulse 1. Adjusting the projector_pulse_shift parameter allows the exposure time of WithoutProjector cameras to be increased. However, the shift comes at the cost of more unpleasant flickering of the projected texture and potential reduction in length of pulses 1 and 3 if the projector board determines that the inter-pulse spacing has been too short.

Projector Modes

In a running PR2, many cameras may be simultaneously in use by different users. Whether the projector should be on at any given time can't be decided by any one user. Hence, the pr2_camera_synchronizer is organized so that users request the mode that their camera should be in, and the synchronizer will decide on the projector state. The synchronizer can be in one of three modes, depending on which strategy it should use:

• ProjectorOff: The projector is never running. Triggering modes that require the projector will be refused by the synchronizer. The stereo cameras, and each forearm camera can independently select their frame rates. Exposure times are limited only by the frame rate.

• ProjectorOn: The projector is always running. Cameras trigger at multiples of half the projector triggering rate (even multiples for cameras that are alternating between textured and non-textured frames). Exposure times for non-textured images are limited by the time between two projector pulses.

• ProjectorAuto: Everything is configured as if the projector was on, except that the projector is only actually turned on if one of the cameras needs it. This way a change in configuration for one of the cameras that causes the projector to be activated or disactivated do not change the timing of the other cameras.

Trigger Modes

A variety of triggering modes are available for the WGE100 cameras. Because different cameras have different constraints, all triggering modes are not available for all cameras.

• Projector agnostic modes: Cameras in these modes are not syncronous with the projector, which may lead to flashing of the texture.

  • InternalTrigger: The camera generates its own triggering. This mode is applicable only for the forearm cameras, as the two cameras in a stereo pair need to be triggered simultaneously from a common source.

  • IgnoreProjector: The camera is triggered by a motor controller board, but no attempt is made to synchronize the camera's triggering with the projector triggering, so the camera's frame rate is independent of the projector rate. This mode is available on all PR2 WGE100 cameras.

• Projector aware modes: These modes are not available when the projector is in the ProjectorOff mode. They will automatically be switched to IgnoreProjector in that case.

  • WithProjector: The camera is triggered in phase with the projector, resulting in a textured image. The exposure time is set to coincide with the duration of the projector pulse. This mode is available on all PR2 WGE100 cameras.

  • WithoutProjector: The camera is triggered out of phase with the projector, resulting in an untextured image. The maximum exposure time is set to the interval between the projector pulses. This mode is available on all PR2 WGE100 cameras.

  • AlternatingProjector: Textured and untextured frames are alternated. The exposure times are set as in the non-alternating modes. This mode is only available on the narrow stereo camera as it requires special support from the imager chip.

Interaction with the Prosilica Camera

(Not implemented yet.)

To avoid having texture in images taken by the Prosilica camera, the prosilica_projector_disable option can be turned on. In this case, an exposure signal from the Prosilica camera directly inhibits firing of the projector. When the projector is inhibited the WGE100 cameras will continue to be triggered as if the projector was active, so some frames that should be textured may be partially textured or not textured at all.

Implementation Limitations

The current implementation has certain timing limitations that may be removed in future versions.

• Camera and projector periods must be multiples of the 1ms Ethercat interval. In particular rates that are a multiple of 120 Hz cannot currently be achieved exactly, which could lead to flickering when fluorescent lighting is used.
• Resolution of triggering and projecting is limited to 1ms Ethercat intervals. 1ms guard times have been incorporated into all the triggering signals and exposure limits.

Multiple PR2 Robots

The phase of the camera triggering is set absolutely relative to a ROS time of 0. Hence, if multiple robots have their clocks synchronized with sub-millisecond resolution, and their projector rates match, their projectors will fire synchronously. As a result images that are supposed to be textured will see the texture from both robots, and images that are supposed to be untextured should not see texture from either robot.

On the other hand, the Prosilica camera projector inhibition will not work between robots. Therefore, Prosilica images from one robot may see texture projected by the other, even if prosilica_projector_disable is true.

ROS API

pr2_camera_synchronizer

Node to set up triggering of the PR2 forearm cameras, stereo cameras and texture projector. It is configured via dynamic_reconfigure.

Parameters

Dynamically Reconfigurable Parameters

See the dynamic_reconfigure package for details on dynamically reconfigurable parameters.

~projector_rate (double, default: 60.0)

• Projector pulse frequency in Hz. Range: 40.0 to 120.0

~projector_pulse_length (double, default: 0.002)

• Length of the projector pulses in s. At high currents the hardware may limit the pulse length. Range: 0.001 to 0.002

~projector_pulse_shift (double, default: 0.0)

• How far off-center the intermediate projector pulses are. Zero is on-center, one is touching the following pulse. Range: 0.0 to 1.0

~projector_mode (int, default: 2)

• Indicates whether the projector should be off, on when in use or on all the time. Possible values are: ProjectorOff (1): The projector is always off., ProjectorAuto (2): The projector is on if one of the cameras is using it., ProjectorOn (3): The projector is always on.

~prosilica_projector_disable (bool, default: True)

• Indicates if the projector should turn off when the prosilica camera is exposing.

~stereo_rate (double, default: 30.0)

• Indicates the frame rate for both stereo cameras in Hz. (Gets rounded to suitable multiples of projector_rate.) Range: 1.0 to 60.0

~wide_stereo_trig_mode (int, default: 4)

• Indicates the triggering mode of the wide stereo camera. Possible values are: IgnoreProjector (2): The camera's frequency can be set independently of the projector frequency. There is no deterministic phase relation between projector firing and camera triggering., WithProjector (3): The camera always exposes while the projector is on., WithoutProjector (4): The camera always exposes while the projector is off.

~narrow_stereo_trig_mode (int, default: 4)

• Indicates the triggering mode of the narrow stereo camera. Possible values are: IgnoreProjector (2): The camera's frequency can be set independently of the projector frequency. There is no deterministic phase relation between projector firing and camera triggering., WithProjector (3): The camera always exposes while the projector is on., WithoutProjector (4): The camera always exposes while the projector is off., AlternateProjector (5): The camera alternates between frames with and without the projector.

~forearm_r_rate (double, default: 30.0)

• Indicates the frame rate for the right forearm camera in Hz. (Gets rounded to suitable multiples of projector_rate.) Range: 1.0 to 60.0

~forearm_r_trig_mode (int, default: 1)

• Indicates the triggering mode of the right forearm camera. Possible values are: InternalTrigger (1): The camera does not use the trigger input., IgnoreProjector (2): The camera's frequency can be set independently of the projector frequency. There is no deterministic phase relation between projector firing and camera triggering., WithProjector (3): The camera always exposes while the projector is on., WithoutProjector (4): The camera always exposes while the projector is off.

~forearm_l_rate (double, default: 30.0)

• Indicates the frame rate for the left forearm camera in Hz. (Gets rounded to suitable multiples of projector_rate.) Range: 1.0 to 60.0

~forearm_l_trig_mode (int, default: 1)

• Indicates the triggering mode of the left forearm camera. Possible values are: InternalTrigger (1): The camera does not use the trigger input., IgnoreProjector (2): The camera's frequency can be set independently of the projector frequency. There is no deterministic phase relation between projector firing and camera triggering., WithProjector (3): The camera always exposes while the projector is on., WithoutProjector (4): The camera always exposes while the projector is off.

~projector_tweak (double, default: 0.0)

• Adds a time shift in seconds to the projector timing. Useful for debugging but not in normal use. Range: -0.1 to 0.1

~camera_reset (bool, default: False)

• Does a hard reset of all the cameras using a long pulse on the trigger line. This parameter resets itself to false after 3 to 4 seconds.