 Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags. |
How to train and use the BiGG object detector
Description: This tutorial presents a brief description of the steps needed to train and use the BiGG object detectorTutorial Level: INTERMEDIATE
Contents
Training
The detector is trained by presenting it with a series of images of the object of interest (the object to be detected) where the object's segmentation is known. The segmentations must be published on a topic of type rein/MaskArray, each segmentation message synchronized with the corresponding image.
There are various ways of generating image segmentations to use for training, we have experimented with two ways:
- segmenting the point cloud above a table and projecting it into an image
- manually selecting a 3D bounding box around an object and tracking it using a checkerboard near the object
For each of the above cases there is a corresponding launch file inside the bigg_detector package (bigg_training_ptu.launch and bigg_training_cam.launch).
Using a manually selected 3D bounding box
When using a manually selected 3D bounding box, the process is the following:
roslaunch rein select3dobj.launch
This will launch a window showing the image from the camera. When the checkerboard is detected in the image, pressing the 'Enter' key will cause the image to freeze and allow the user to select the 3D wounding box around the object. A second 'Enter' key will confirm the selection and the node will start tracking the 3D bounding box and will also publish the masks representing the segmentation.
The training launch file can be then started:
roslaunch bigg_detector bigg_training_cam.launch
This will launch a simple GUI showing the segmented image. Pressing the 'Start training' button will begin the training process. For training, the object detector must be presented with images of the trained object from multiple different viewpoints. Pressing 'Save Model' will finish training and save the trained model.
Using a segmented point cloud
If a segmented point cloud of the object to be trained is available, it can be projected into the image to obtain the image segmentation, which can be used for training.
roslaunch bigg_detector bigg_training_ptu.launch
Using the object detector
The object detector can be started in detection mode with the following launch file:
roslaunch bigg_detector bigg_detection_cam.launch
