Recently in packages Category

From Piyush at University of Texas

We've just completed a ROSJava based wrapper for the April Fiducial Marker System from the APRIL lab at UMich. The package should be (in principle) compatible with any package that uses the ROS Wrapper around ARToolKit. Hopefully some of you might find it useful.

Stack page: http://www.ros.org/wiki/april

Bug reports: http://code.google.com/p/utexas-ros-pkg/issues/list

Announcement by Filip Muellers to ros-users

Hi all,

I have implemented nice new features for the rxDeveloper 1.3b and need some feedback. I would like to improve this new features.

New features:

• automatic generation of source code from specifications - the specfile-editor allows you to create python and c++ templates. for c++ files the CMakeLists.txt can be modified automatically, too.
• rosdep install and rosmake can be executed graphically for a selected package (component creation tab)
• specfiles can be created by fetching information from running ROS processes (component creation tab)

The rxDeveloper runs on Diamondback, Electric and Fuerte beta.

For more information, sources and tutorials please visit rxdeveloper-ros-pkg/.

Regards,
Filip Muellers

Announcement by Bob Mottram to ros-users

I've written a new stereo camera driver which is intended for use with V4L2 compatible cameras, such as stereo webcams like the Minoru.

https://launchpad.net/stereocamera-v4l2-ros-pkg

This doesn't do any stereo correspondence, but broadcasts images and allows calibration parameters to be stored. Various parameters can be set within the launch file.

Announcement from Arnaud Ramey (University Carlos III of Madrid)

Hello ROS users!

I am happy to announce to release of a image_transport plugin for float images, mainly aimed at broadcasting compressed Kinect depth images. More details follow. I hope this might be useful for other users and would be happy to hear feedback! (that would justify the hours of work to get to that result :) ).

For more information, including installation instructions, please see the ros-users post.

Link: Announcing compressed_rounded_image_transport

Announcement by Chad Rockey (maintainer of laser_drivers to ROS users

Hi ROS Community,

I've been working on a driver that connects the sensors in Android devices to the ROS environment. At this time, it only publishes sensor_msgs/NavSatFix messages, but I will soon introduce sensor_msgs/Imu and sensor_msgs/Image to publish data from accelerometers, gyroscopes, magnetometers, and front/rear cameras.

To get more information and to install, please see the following:

• http://www.ros.org/wiki/androidsensorsdriver/
• https://market.android.com/details?id=org.ros.android.sensors_driver

To file bugs, request features, view source, or contribute UI, translation, or other improvements, please see the Google Code project:

http://code.google.com/p/android-sensors-driver/

I hope everyone finds this useful and I look forward to hearing your feedback and seeing cool uses for Android devices in robotics.

Thanks,
- Chad Rockey

Announcement by Pouyan Ziafati of University of Luxembourg and Utrecht University to ros-users

Dear All,

I am happy to announce a new ROS package (face_recognition) for face recognition in video stream. The package provides an actionlib interface for performing different face recognition functionalities such as adding training images directly from the video stream, re-training (updating the database to include new training images), recognizing faces in the video stream, etc. (For more info see the README file)

The package is accessible from the git repository:

git://github.com/procrob/procrob_functional.git

Best regards,
Pouyan PhD Candidate,
Utrecht University & University of Luxembourg

Announcement from RViz SIG Coordinator Dave Hershberger to ros-users

Rviz is moving from the wxWidgets library to the Qt library as of the ROS Fuerte release. This transition to Qt will improve RViz compatibility on more platforms and better integrate with future GUI tools. For more information on the motivation for these changes, please refer to the RViz and ROS GUI SIGs:

• http://ros.org/wiki/fuerte/Planning/rviz
• http://ros.org/wiki/fuerte/Planning/ROS GUI

This is a fairly big change to the code. The internal plugin API is changed and the Python API will be different (when I get it implemented). The GUI appears and behaves mostly the same as the wx version, most of the differences coming from the differences in style between wx and Qt and not from intentional changes. For a list of major changes, see visualization_experimental/ChangeList.

An early but fairly complete, version of the new code is available in the temporary visualization_experimental stack, which works with ROS Electric. It is available in the ros-electric-visualization-experimental debian package. This version does not have python support implemented yet, but all the built-in display types, tools, and view controllers work. I encourage rviz users to try the new version (called rviz_qt for now) before Fuerte so I can fix bugs and make the transition as smooth as possible.

Please report bugs and feature requests to the same place as usual for rviz, but with "qt" in the keywords field, like so: report a bug, request a feature, list existing tickets.

Thanks,
Dave

Oier Mees on ros-users writes

Hello,

During a summer internship in Tekniker Research Center, I have developed a telepresence and teleoperation application which I have just published. You can teleoperate the robot, in my case a segway rmp 200 with a kinect on top, with the joystick or the gyro of the PS3 controller, besides the usual buttons of the UI. Also, a videoconference is performed with the client PC and if the user clicks inside the rectangle where the kinects video stream is shown, the robot will attempt to reach that destination. For instance, as you can see in the demo video, if the user clicks on a person who is in the middle of a corridor, the robot will calculate the distance thanks to the kinects depth sensor and try to go to destination by using ROS navigation stack.

A coworker is now going to continue developing and maintaining it, so we would be grateful for any kind of feedback or contribution.

You can grab the code at github and see a demo video on YouTube.

Regards,
Oier Mees

Crossposted from willowgarage.com

David Gossow, a recent graduate of the University of Koblenz-Landau and new doctoral student at the Technical University of Munich, visited Willow Garage this spring. David created a new general framework allowing ROS developers to create graphical 3D interfaces to their robot applications, and applied it to building new tools for Human-in-the-Loop robotic manipulation.

David's new framework, called Interactive Markers, allows a ROS application to receive input from a human operator through a compatible client software. It separates the application from the tool used for visualization and user interaction, much like a web application runs independently of the web browser. Interactive Markers offer a wide variety of display and interaction modes, enabling a broad range of new applications within ROS. David also implemented a reference front end for Interactive Markers in rviz, effectively transforming it from a robot visualization tool into an interaction engine. This new front end is a major new feature in the recent ROS Electric release.

David used the Interactive Markers framework to develop new tools for Human-in-the-Loop robotic manipulation. Assistance from a human operator allows a robot to perform complex manipulation tasks even in difficult, unstructured environments. The goal in this framework is to minimize the cognitive effort required on the human side. This can be achieved by taking advantage of the sub-tasks that the robot can perform without assistance. For example, if the operator assists in object recognition, subsequent operations such as grasping and placing can be performed autonomously. When needed, the operator can also get involved at lower levels of task execution, such as specifying grasp poses or even directly operating the robot's gripper. A complete set of tools, based on Interactive Markers, allows an operator to perform all these tasks through the rviz interface.

For more details, see David's presentation below, or check out the interactive_markers and pr2_interactive_manipulation packages on ROS.org.

Announcement from Lorenzo Riano of University of Ulster (uuisrc-ros-pkg) to pr2-users:

We have added a new stack with the packages to run the Rubik's cube solver on the PR2. We will add the documentation shortly. In the meantime you can find it at: https://github.com/uu-isrc-robotics/pr2rubikssolver

Please report any bug-issue-feature request to uu.isrc.robotics@gmail.com

Announcement from Patrick Goebel of Pi Robot to ros-users

I have put together a little ROS package for doing face tracking using the idea described in the first paragraph below. Note that I have not yet added any feature learning as used in the TLD algorithm. Interested parties can check it out on the ROS wiki.

You can also use this package for tracking arbitrary patches of a video by setting the auto_face_tracking parameter to False and selecting the desired region with the mouse. This does not work as well as face tracking since face tracking uses the Haar detector to re-acquire the face if the number of tracked features falls below a prescribed minimum.

Let me know if you find any bugs or can think of improvements.

--patrick

http://www.pirobot.org

Menglong Zhu at Penn has given PR2 a fantastic new skill: the ability to read. Using the literate_pr2 software he wrote, PR2 can drive around and read aloud the signs that it sees. Whether it's writing on a whiteboard, nameplates on a door, or posters advertising events, the ability to recognize text in the real world is an important skill for robots.

Although performing OCR on text is not a new technology, performing it in the real world is much more difficult as text can be written anywhere and first you have to find it. Menglong's code is able to detect areas of text in a camera image and perform text recognition on them separately.

This is another great contribution to ROS from the GRASP Lab. For more information, please see the literate_pr2 page on the ROS wiki.

g2o is available as a package under the vslam stack. g2o is an open-source C++ framework for optimizing graph-based nonlinear error functions. g2o has been designed to be easily extensible to a wide range of problems and a new problem typically can be specified in a few lines of code. The current implementation provides solutions to several variants of SLAM and BA.

A wide range of problems in robotics as well as in computer-vision involve the minimization of a non-linear error function that can be represented as a graph. Typical instances are simultaneous localization and mapping (SLAM) or bundle adjustment (BA). The overall goal in these problems is to find the configuration of parameters or state variables that maximally explain a set of measurements affected by Gaussian noise. g2o is an open-source C++ framework for such nonlinear least squares problems. g2o has been designed to be easily extensible to a wide range of problems and a new problem typically can be specified in a few lines of code. The current implementation provides solutions to several variants of SLAM and BA. g2o offers a performance comparable to implementations of state-of-the-art approaches for the specific problems.

Please see the vslam page for installation instructions.

Links:

• Documentation and library source code
• vslam stack for ROS
• ICRA 2011 conference paper
• Authors: Rainer Kuemmerle, Giorgio Grisetti, Hauke Strasdat, Kurt Konolige, Wolfram Burgard.

Yesterday at Google I/O, developers at Google and Willow Garage announced a new rosjava library that is the first pure-Java implementation of ROS. This new library was developed at Google with the goal of enabling advanced Android apps for robotics.

The library, tools, and hardware that come with Android devices are well-suited for robotics. Smartphones and tablets are sophisticated computation devices with useful sensors and great user-interaction capabilities. Android devices can also be extended with additional sensor and actuators thanks to the Open Accessory and Android @ Home APIs that were announced at Google I/O,

The new rosjava is currently in alpha release mode and is still under active development, so there will be changes to the API moving forward. For early adopters, there are Android tutorials to help you send and receive sensor data to a robot.

This announcement was part of a broader talk on Cloud Robotics, which was given by Ryan Hickman and Damon Kohler of Google, as well Ken Conley and Brian Gerkey of Willow Garage. This talk discusses the many possibilities of harnessing the cloud for robotics applications, from providing capabilities like object recognition and voice services, to reducing the cost of robotics hardware, to enabling the development of user interfaces in the cloud that connect to robots remotely. With the new rosjava library, ROS developers can now take advantage of the Android platform to connect more easily to cloud services.

Announcement from David Lu of Washington University in St. Louis to ros-users

Hey all,

Today I am pleased to announce the release of 2 new stacks and 1 new package as part of the Washington University repository.

First, the Polonius stack. Polonius is a robot control interface designed for running Wizard of Oz style experiments. This work was presented in poster form at HRI2011, and was also used to control a short robot theatre piece we put on last September, the video for which is being presented at the Robots and Art workshop at ICRA next week.

Second, the motion_capture stack. This contains our initial work at incorporating motion capture data into ROS, initially just using the c3d data format. Tools for analyzing this sort of data will be released in the future.

Third, a ROS Wiki documentation tool, roswiki_node. This package contains code to make documentation a bit easier by automatically generating the CS/NodeAPI code used on the wiki automatically using your source code.

Feedback is always welcome, either by emailing me or through our SourceForge project page.

Cheers, and Happy May.

-David Lu!!

Announcement from Felix Endres from the University of Freiburg to ros-users

Dear ROS Users,

we are happy to announce a new release of our entry to the ROS-3D contest.

There have been many changes, we would like to share with the community:

• Improvments w.r.t accuracy and robustness of registration
• Performance improvments w.r.t computation time
• A more convenient user interface with internal 3D visualization
• Many convenience features, e.g., saving to pcd/ply file, node deletion etc

It is available for download.

Quick Installation (see README in svn repo for more detail) for Ubuntu:

$ sudo apt-get install ros-diamondback-desktop-full ros-diamondback-perception-pcl-addons ros-diamondback-openni-kinect meshlab
$ svn co http://alufr-ros-pkg.googlecode.com/svn/trunk/freiburg_tools/hogman_minimal
$ svn co https://svn.openslam.org/data/svn/rgbdslam/trunk rgbdslam
$ roscd rgbdslam && rosmake --rosdep-install rgbdslam

Best regards,
Felix

Guest post from James Bowman of Willow Garage

We recently wanted to hook up an analog gyro (the Analog Devices ADXRS614) to ROS, and decided to use an Arduino to handle the conversion.

The Arduino runs a tiny loop that reads the analog values from its six analog lines,and writes these values to the USB serial connection.  Meanwhile on the robot, a small ROS Python node listens to the USB reports, and publishes them as a ROS topic.

That's all there is to it: the whole thing takes under 40 lines of code.

This is probably the simplest possible way to use an Arduino with ROS.  There are quite a few more sophisticated projects:

• pmad  - controls an Arduino's I/Os using ROS service calls
• avr_bridge - automates generation of ROS-Arduino message transport
• arduino - our own package: an Arduino as an Ethernet-connected ROS node

Announcement from Prof. Dr. Matthias Kranz of TUM

The team of the Distributed Multimodal Information Processing Group of Technische Universität München (TUM) is pleased to announce that we ported rospy to run on Android-based mobile devices.

Python for Android on top of the Scripting Layer for Android (SL4A) serves as basis for our rospy project. We extended the scripting layer, added new support for ctypes and other requirements. Now, rospy, roslib and the std_msgs are working and running on a roscore, directly on your mobile phone. To configure a roscore on a standard computer to cooperate with the roscore on the Android, you simple scan a QR code on the computer's screen to autoconfigure the smartphone. Basic support for OpenCV and the image topics is also included. You are welcome to extend the current state of our work.

You will need a current Version of the Scripting Layer (v3) and the newer Python for Android with the possibility to import custom modules. You can use every Android device able to run the SL4A. In general you be able to run it on every recent Android powered device. You will not harm your phone at all, even no root access is needed to run ROS on your device.

You can find our code, basic documentation and a video in our repository and on ROS.org.

A small video showcasing how to control a ROS-based cognitive intelligent environment via an Android-based smartphone is available here.

Links:

• vmi-ros-pkg
• videos

Announcement from Eric Perko of Case Western to ros-users

Hello folks,

I'm happy to announce a ROS driver for the Neato XV-11 Laser Scanner. ROS has had a driver for the Neato itself for sometime now, but this was only useful for going through the Neato's onboard computer. Well now you can just yank that XV-11 laser scanner out, strap it to your iRobot Create and feed it right on into the rest of ROS! The XV-11 scanner gives 360 pings at 1 degree increments at a rate of 5Hz and is useful from ~6cm to ~5m.

Chad and I have written up a number of tutorials to help people get started using this low-cost scanner including how to remove the laser from the vacuum and wire it to USB, how to get it up and running with our ROS driver and how to interpret the raw bytes if you want to parse the data yourself on a microcontroller.

Let us know if you have any problems/questions/comments.

Eric Perko, Chad Rockey
CWRU Mobile Robotics Lab

Announcement from Patrick Goebel to ros-users

Hello All,

I have released a ROS package for the Serializer microcontroller made by the Robotics Connection. I have been using and developing this package several months along with at least one other ROS user so I hope I have most of the egregious bugs fixed. Please let me know if you run into trouble with either the code or the Wiki page:

• Wiki
• SVN

The Serializer has both analog and digital sensor pins, a PWM servo controller, and a PID controller that works well with the ROS navigation stack.

The underlying driver is written in Python and provides some convenience functions for specific sensors such as the Phidgets temperature, voltage and current sensors, the Sharp GP2D12 IR sensor, and the Ping sonar sensor. It also includes two PID drive commands: Rotate(angle, speed) and TravelDistance(distance, speed) for controlling the drive motors. Most of the Serializer functions have been implemented though a few have not been tested since I don't currently have some of the supported sensors. The functions that have not been tested are:

• step (used with a bipolar stepper motor)
• sweep (also used with a bipolar stepper motor)
• srf04, srf08 and srf10 (used with a Devantech SRF04, SRF08 and SRF10 sonar sensors)
• tpa81 (used with the Devantech TPA81 thermopile sensor)

The driver requires Python 2.6.5 or higher and PySerial 2.3 or higher. It has been tested on Ubuntu Linux 10.04.

• Documentation for the Python Serializer library
• Official Serializer manual

Sincerely,
Patrick Goebel
The Pi Robot Project

Announcement from Adam Stambler of Rutgers to ros-users

Hello Folks,

I am proud to announce a new tool for using Arduinos and AVR processors in ROS projects.  avr_bridge allows the AVR processors to directly publish or subscribe to ROS topics. This allows everything from Arduinos to custom robot boards to be first class ROS components.  This package can be found in Rutgers' new rutgers-ros-pkg.

avr_bridge is meant to simplify the use of an Arduino and avr processors in a ROS based robot by providing a partial ROS implementation in avr c++. In hobbyist robotics, these microcontrollers are often used to read sensors and perform low level motor control. Every time a robot needs to interface with an AVR board, a new communication system is written. Typically they all use a usb-to-serial converter and either a custom binary or text based protocol. AVR bridge replaces these custom protocols with an automatically generated ROS communication stack that allows the AVR processors to directly publish or subscribe to ROS topics.

avr_bridge has already been deployed on Rutger's PIPER robot and in the communications layer for a Sparkfun imu driver. In the next few weeks, it will be deployed on our newest robot, the Rutgers IGVC entry as the communication layer on all of our custom, low level hardware.   By using avr_bridge to communicate with our PCs, we have cut down on redundant code and simplified the driver by allowing the avr_processor to directly publish msgs.  It is our hope that by extending ROS to the 8-bit microcontroller level we will see more open-source hardware that can be quickly integrated into cheap, custom robot platforms.

Cheers,
Adam Stambler
Rutgers University

Daniel Hennes from Maastricht University spent his internship at Willow Garage modeling the dynamics of robotic manipulators using statistical machine learning techniques. He also created a useful visualization utility for ROS/rviz users that enables users to intuitively visualize the joint motor torques of a robot. Please watch the video above for an overview or read the slides below (download pdf) for more technical details. The software is available as open source in the inverse_dynamics and dynamics_markers packages on ROS.org.

Announcement from Mark Moll to robotics-worldwide

Dear colleagues,

The Kavraki Lab is pleased to announce the initial release of the Open Motion Planning Library (OMPL). OMPL is a lightweight, thread-safe, easy to use, and extensible library for sampling-based motion planning. The code is written in C++, includes Python bindings and is released under the BSD license.

Here are some of OMPL's features:

• Implementations of many state-of-the-art sampling-based motion planning algorithms. For purely geometric planning, there are implementations of KPIECE, SBL, RRT, RRT Connect, EST, PRM, Lazy RRT, and others. For planning with differential constraints there are implementations of KPIECE and RRT. Addition of new planners poses very few constraints on the added code.
• A flexible mechanism for constructing arbitrarily complex configuration spaces and control spaces from simpler ones.
• A general method of defining goals: as states, as regions in configuration space, or implicitly.
• Various sampling strategies and an easy way to add other ones.
• Automatic selection of reasonable default parameters. Performance can be improved by tuning parameters, but solutions can be obtained without setting any parameters.
• Support for planning with the Open Dynamics Engine, a popular physics simulator.
• Tools for systematic, large-scale benchmarking.

OMPL is available at http://ompl.kavrakilab.org.

OMPL is also integrated in ROS and will be available as a ROS package (see http://www.ros.org/wiki/ompl/unstable for documentation and download instructions). It will be fully integrated with the next major release of ROS (D-Turtle), which is scheduled for release in early 2011.

On top of the OMPL library, we have developed OMPL.app: a GUI for rigid body motion planning that allows users to load a variety of mesh formats that define a robot and its environment, define start and goal states, and play around with different planners. The OMPL.app code also comes with concrete command-line examples of motion planning in SE(2) and SE(3) (with and without differential constraints), using ODE for physics simulation, PQP for collision checking, and Assimp for reading meshes. OMPL.app is distributed under the Rice University software license (essentially, free for non-commercial use).

The Kavraki lab is fully committed to further developing OMPL for research and educational purposes. Please check out http://ompl.kavrakilab.org/education.html and contact us if you are interested in using OMPL in your class.

This project is supported in part by NSF CCLI grant #0920721 and a generous gift by Willow Garage.

Ed: here's a video from a year and a half ago showing some of Ioan Sucan's work with OMPL and the PR2

Visiting scholar Christian Connette from Fraunhofer IPA has just finished up his projects here at Willow Garage. Christian works on the Care-O-bot 3 robot platform, which shares many ROS libraries in common with the PR2 robot. While he was here at Willow Garage, he worked on implementing an "elastic band" approach (Quinlan and Khatib) for the ROS navigation stack. You can watch the video above to find out more about this work, or checkout the slides below for more technical details (download PDF). The software is available as open source in the eband_local_planner package on ROS.org.

Garratt Gallagher from CSAIL/MIT is at it again. Above, you can see his work on using the new OpenNI-based ROS drivers to get an iRobot Create to follow a person around. This code is based off of the skeleton tracker that comes with the NITE library.

For those of you figuring out how to get the NITE tracking data into ROS, take a look at Garratt's nifun package.

Garratt Gallagher from CSAIL/MIT has followed up his Kinect piano and hand detection hacks with a full "Minority Report" interface. The demo builds on the pcl library to do hand detection. You'll find Garratt's open-source libraries for building your own interface in mit-ros-pkg.

Update: MIT News release with more details

OpenCV 2.2 has been released. Major highlights include:

• Reorganization into several, smaller modules to better separate different OpenCV functionality, as well as experimental vs. stable code.
• A new (alpha) GPU acceleration module, created with the support of NVidia
• Android support by Ethan Rublee.
• New features2d unified framework for keypoint extraction, computing the descriptors and matching them.
• LatentSVM object detector, contributed by Nizhniy Novgorod State University (NNSU) team.
• Gradient boosting trees model has been contributed by NNSU team.
• Experimental Qt backend for highgui by Yannick Verdie. (docs).
• Chamfer matching algorithm has been contributed by Marius Muja, Antonella Cascitelli, Marco Di Stefano and Stefano Fabri. See samples/cpp/chamfer.cpp.
• A lot more of OpenCV 2.x functionality is now covered by Python bindings. These new wrappers require numpy to be installed.
• Over 300 issues have been resolved. Most of the issues (closed and still open) are listed at https://code.ros.org/trac/opencv/report/6.

For more information, please see the complete change log.

Kurt Konolige and Patrick Mihelich have prepared a technical overview of the Kinect calibration provided in the kinect_calibration package for ROS. For those of you wishing to understand the technology behind the PrimeSense sensor, this provides a detailed overview of how depth is calibration -- and how we go about providing the calibration necessary for perception algorithms.

Patrick Mihelich has just finished creating a kinect_calibration package for the ROS kinect stack. This calibration procedure takes advantage of the IR image access we added last week, plus the helpful discovery by Alex Trevor to use a halogen lamp to provide the necessary illumination for the IR image (in lieu of the IR projector).

Tomorrow we hope to do a release of the new stack with this calibration code as well as a proper tutorial. If you're really curious, you can try these barebone instructions.

Garratt Gallagher from CSAIL/MIT has created a fun Kinect hack: Kinect Piano! You hold your hand steady in front of the Kinect and then move your fingers to play individual notes. You can find the code in the piano package in mit-ros-pkg.

This hack also comes with a library to help you create your own -- Garratt wrote a kinect_tools package that uses pcl to implement a hand detector, which he demonstrates below:

Announcement from Associate Professor Chad Jenkins at Brown

Hi ros-users,

Just a quick heads up that brown-ros-pkg now includes ardrone_brown, an AR.Drone driver that supports the front camera.

The following video is a test of ardrone_brown for visually following an AR Tag, using ar_recog (ARToolkit) for tag recognition:

The tag following behavior is produced by the nolan package that is run with only gain modifications from our previous AR following video.

More to come.

-Chad

p.s. Dear Parrot, thanks for releasing Development License v2!

Announcement from Geoff Biggs of AIST on creating a standalone release of PCL

Hi all,

For all of us who plan to spend Thanksgiving working (perhaps we enjoy coding more than turkey, or it could just be that we don't live in the US...), there's a brand new release of PCL. This brings it up to 0.5. (I'm reliably informed that this makes it "half-way decent.")

In conjunction with the 0.5 release, we have just put the finishing touches on a standalone distribution of PCL. This means that those of you who are not using ROS can now use PCL in your applications.

It's still somewhat unrefined, and we need to work with some of the dependencies to make installation easier. Look for many improvements as we head towards 1.0!

Thanks again to all the contributors!

Geoff

• pcl
• Changelist
• Instructions and downloads for the standalone version

The ROS/Kinect integration continues to progress quickly thanks to the efforts of the OpenKinect and ROS community. The initial ROS+Kinect contributors -- Alex Trevor, Ivan Dryanovski, Stéphane Magnenat, and William Morris -- have combined their efforts into a new ROS kinect stack. At Willow Garage, our engineers and researchers have also been working on the stack to improve the driver and integrate it with ROS libraries and tools. The community is now hard at work on solving problems like calibration, which will be important for using the Kinect in robotics. Feel free to sign up for the ros-kinect mailing list to keep up-to-date on the latest efforts.

We thought we'd make a quick video to show some of what's going on at Willow Garage with the Kinect. We've added features like multi-camera support and control of the Kinect motors to the popular libfreenect library. We're also working on making some fun Kinect hacks of our own -- watch until the end of the video to see where we are with those. We look forward to seeing your videos as well.

Hector Martin's libfreenect open-source driver for the Microsoft Kinect has lead to several efforts within the ROS community to create Kinect drivers for ROS. Stéphane Magnenat (ETH Zurich) and Alex Trevor were the to port libfreenect to ROS. The CCNY Robotics Lab has now added their kinect_node package, which adds documentation, depth calibration, and example bag files. It's great to see how their efforts have contributed to each other, as well as to the broader libfreenect community.

The Kinect is obviously an important sensor for robotics. It's a sensor that we can all own at home, instead of having to share in a lab. That has already has enabled so many people to quickly work together on an open-source driver and it will be great to see what the community can build together next.

cross-posted from willowgarage.com

One of the challenges that robots like the PR2 face is knowing how to grasp an object. We have years of experience to help us determine what objects are and how to grasp them. We can tell the difference between a mug, and wine glass, and a bowl, and know that they each should be handled in a different way. For robots, the world is not as certain, but there are approaches they can take that let them interact in an uncertain world.

This summer, Peter Brook from the University of Washington wrote a grasp planning system which lets robots successfully pick up objects, even in cases where they make incorrect guesses about what the object is. This planner uses a probabilistic approach, where the robot uses potentially incomplete or noisy information from its sensors to make multiple guesses about the identity of the object it is looking at. Based on how confident the robot is in each of the possible explanations for the perceived data, it can select the grasps that are most likely to work on the underlying object.

First, the planner builds up a set of representations for the sensed data; some are based on the best guesses provided by ROS recognition algorithms, and some use the raw segmented 3D data. For each representation, it uses a grasp-planning algorithm to generate a list of possible grasps. It then combines the information from all these sources, sorting grasps based on their estimated probability of success across all the representations. For grasp planners running on known object models, it can also use pre-computed grasps that speed up execution time.

This probabilistic planner allows the PR2 robot to cope with uncertainty and reliably grasp a wider range of objects in unstructured environments. It is also integrated into the ROS object manipulation pipeline so that others can experiment and improve upon it. For more information, please see Peter's slides below (download PDF), or checkout the source code in the probabilistic_grasp_planner package on ROS.org.

cross-posted from willowgarage.com

Robot motion planning has traditionally been used to avoid collisions when moving a robot arm. Avoiding collisions is important, but many other desirable criteria are often ignored. For example, motions that minimize energy will let the robot extend its battery life. Smoother trajectories may cause less wear on motors and can be more aesthetically appealing. There may be even more useful criteria, like keeping a glass of water upright when moving it around.

This summer, Mrinal Kalakrishnan from the Computational Learning and Motor Control Lab at USC worked on a new motion planner called STOMP, which stands for "Stochastic Trajectory Optimization for Motion Planning". This planner can plan paths for high-dimensional robotic systems that are collision-free, smooth, and can simultaneously satisfy task constraints, minimize energy consumption, or optimize other arbitrary criteria. STOMP is derived from gradient-free optimization and path integral reinforcement learning techniques (Policy Improvement with Path Integrals, Theodorou et al, 2010).

The accompanying video shows the STOMP planner being used to plan motions for the PR2 arm in simulation and a real-world setup. It shows the ability to plan motions in real-world environments, while optimizing constraints like holding the cans upright at all times. Ultimately, the utility of this motion planner is limited only by the creativity of the system designer, since it can plan trajectories that optimize any arbitrary criteria that may be important to achieve a given task.

For more information, please see Mrinal's slides below (download pdf), or checkout the code in the stomp_motion_planner package on ROS.org. This package builds on the various packages in the policy_learning stack, which was written in collaboration with Peter Pastor. You can also checkout Mrinal's work from last summer on the CHOMP motion planner.

Stéphane Magnenat from the Autonomous System Lab at ETH Zurich has announced a ROS/ASEBA bridge

Dear list,

Thanks to your quick and precise answers, I have programmed a bridge between ASEBA and ROS:

http://github.com/stephanemagnenat/asebaros

This bridge allows to load source code, inspect the network structure, read and write variables, and send and receive events from ROS.

This brings ROS to the following platforms:

• Mobots' marxbot, handbot and smartrob
• e-puck

Kind regards,
Stéphane

cross-posted from willowgarage.com

This summer, Hae Jong Seo, a PhD student from the Multidimensional Signal Processing Research Group at UC Santa Cruz, worked with us on object and action recognition using low-cost web cameras. In order for personal robots to interact with people, it is useful for robots to know where to look, locate and identify objects, and locate and identify human actions. To address these challenges, Hae Jong's implemented a fast and robust object and action detection system using features called locally adaptive regression kernels (LARK).

LARK features have many applications, such as saliency detection. Saliency detection determines which parts of an image are more significant, such as containing objects or people. You can then focus your object detection on the salient regions of the image in order to detect more quickly. Saliency detection can be extended to "space-time" for use with video streams.

LARK features can also be used for generic object and action detection. As you can see in the video, objects such as door knobs, the PR2 robot, and human faces and be detected using LARK. Space-time LARK can also detect human actions, such as waving, sitting down, and getting closer to the camera.

For more information, see the larks package on ROS.org or see Hae Jong's slides below (download PDF). You can also consult Peyman Milanfar's publications for more information on these techniques.

crossposted from willowgarage.com

Bastian Steder, a PhD student from the Autonomous Intelligent Systems Group at the University of Freiburg, Germany, spent the summer at Willow Garage implementing an object recognition system using 3D point cloud data. With 3D sensors becoming cheaper and more widely available, they are a valuable tool for robot perception. 3D data provides extra information to a robot, such as distance and shape, that enables different approaches to identifying objects in the world. Bastian's work focused on using databases of 3D models to identify objects in this 3D sensor data.

The main focus for Bastian's work was on the feature-extraction process for 3D data. One of his contributions was a novel interest keypoint extraction method that operates on range images generated from arbitrary 3D point clouds. This method explicitly considers the borders of the objects identified by transitions from foreground to background. Bastian also developed a new feature descriptor type, called NARF (Normal Aligned Radial Features), that takes the same information into account. Based on these feature matches, Bastian then worked on a process to create a set of potential object poses and added spatial verification steps to assure these observations fit the sensor data.

The full system can identify the existence and poses of arbitrary objects, of which we have a point cloud model in a very efficient manner, using only the geometrical information provided by the 3D sensor. Code for Bastian's work, including object recognition and feature extraction, has been integrated with PCL, which is a general library for 3D geometry processing in development at Willow Garage. To find out more, check out the point_cloud_perception stack on ROS.org. For detailed technical information, you can check out Bastian's presentation slides below (download PDF).

crossposted from willowgarage.com

Adam Harmat from McGill University worked on three projects this summer to make the PR2 more dexterous when manipulating objects: a monitoring system for arm movement, a persistent 3D collision map, and a multi-table manipulation application. All of these projects demonstrated how increased knowledge of its environment is necessary for improving PR2's mobile manipulation capabilities.

The arm-monitoring system uses head-mounted stereo cameras to detect new obstacles. While the arm moves, the PR2 looks at locations that are a few seconds ahead of the arm's current position. Any detected obstacles are added to a collision map, and, if a future collision is anticipated, the arm stops and waits. If the new obstacle doesn't move, the PR2 will attempt to move around it.

The collision map was improved to store information about everything the robot has previously seen. This allows the PR2 to perform tasks that require it to relocate as it maintains knowledge about places it currently cannot see. This new collision map is based on Octomap, an open source package from the University of Freiburg. The octree structure of Octomap is more compact and also enables the storing of probabilistic values.

No one wants a clumsy robot. As a result of these projects, the PR2 is able to maintain more knowledge about its local environment, and is able to keep its arms from bumping into objects. Adam developed a demo application to demonstrate these new capabilities.

In his multi-table manipulation demo, the PR2 continuously finds and moves objects between separate tables. This application is integrated with the ROS navigation stack to determine pickup locations and navigate between tables. Adam's multi-table application demonstrates how planning with the persistent collision map can be integrated with base movement and local task-execution into a complete system.

For more information, you can view Adam's presentation slides below (download as PDF), or checkout the move_arm_head_monitor and the multi_table_detector packages on ROS.org.

Tim Niemueller has announced an actionlib implementation for his roslua client library

Hi ROS users.

We have released another piece of the Lua integration for ROS, this time it's actionlib_lua. It has been developed at Intel Labs Pittsburgh as part of my research stay this year working with Dr. Siddhartha Srinivasa on the Personal Robotics project. You can the source code at http://github.com/timn/actionlib_lua. It requires the most recent version of roslua that you can get from http://github.com/timn/roslua.

It implements most features of actionlib, both client and server side. Additionally it allows for some small optimizations, i.e. you can ignore the feedback and cancellation topics if not required or supported. It interacts well with the original actionlib for C++ and Python and we are using it on HERB.

As always, feedback is welcome,
Tim

Dominick Vanthlenen of kul-ros-pkg announced the release of a driver for NDI's Polaris (R) 3D measurement system along with packages for using the driver with both Orocos and ROS

For all of you having a Polaris (R) 3D measurement system: a ndi_hardware stack has been released! This enables you to use your measurement system on a Linux system and let it publish tf frames.

nick

Jeff Rousseau announced a basic URDF model for the iRobot Create as well as new aptima-ros-pkg code repository

Hi all,

I've put together a basic URDF for the iRobot Create platform. It's available for download through our new svn repo:

svn checkout http://aptima-ros-pkg.googlecode.com/svn/trunk/irobotcreatedescription

(Note: currently it relies on the erratic_gazebo_plugins pkg to implement its diff-drive)

Comments, bug reports and patches are appreciated

I plan to add/fix in the not too distant future:

• a working bumper
• tweak mass/friction params to be more realistic (they're fudged at the moment)
• fix intermittent 'wobble' when transitioning between translation and rotations (friction coeff issue?)

enjoy,
Jeff

The Willow Garage PR2 robots have been out at the PR2 Beta Sites for only a few short months and they have been busy with research projects, developing new software libraries for ROS, and creating Youtube hits. The first PR2 Beta Program Conference call was recently held to highlight this work, and the list of software that they have released as open source is already impressive.

A partial list of this software below so that all the ROS users and researchers can try it out and be involved. You'll find many more libraries in their public code repositories, and there is much more coming soon.

Georgia Tech

• hrl_rfid: Open hardware and software for servoing to RFID tags
• laser_pointer_interface: look at and drive to locations that a user points to with a laser pointer
• Teloperation for dual manipulation using a Phantom Omni and stereo anaglyph display: pr2_omni_teleop, phantom_omni, stereo_anaglyph

KU Leuven

• Orocos RTT 2.0 has been released
• orocos_rtt_ros_toolchain: Automatic Orocos RTT 2.0 typekit and transport creation from ROS .msg files
• Multi-target tracking and detection
• PR2 model for Blender: pr2_blender_model. Working on adding simulation capabilities.

JSK

• EusLisp: now available under a BSD license
• ROS/PR2 integration with EusLisp: roseus, pr2eus, and euscollada
• jsk_ros_tools: includes rostool-alias-generator (e.g. rostopic_robot1) and jsk-rosemacs (support for anything.el)

TUM

• knowrob: tools for knowledge acquisition, representation and reasoning
• CRAM: reasoning and high level control for Steel Bank Common Lisp (cram_pl) and executive that reasons about locations (cram_highlevel)
• prolog_perception: logically interact with perception nodes
• pcl: contributions include pointcloud_registration, pcl_cloud_algos, pcl_cloud_tools, pcl_ias_sample_consensus, pcl_to_octree, mls

Stanford

• Checkout Bot: pickup common drug store objects on table, scan their barcode, and toss into bag
• Find and execute stable grasps for the PR2 gripper: grasping, grasping_executive, grasping_classifier

Berkeley

• towel_folding: version of Towel Folding from pre-PR2 Beta Program that relies on two Canon G10 cameras mounted on chest. Uses optical flow for corner detection.
• LDA-SIFT: recognition for transparent objects
• Utilities:
  • pr2_simple_motions: Classes for easy scripting in Python of PR2 head, arms, grippers, torso, and base
  • visual_feedback: Streamlined image processing for 3d point extraction and capturing images
  • stereo_click: Click a point in any stereo camera feed and the corresponding 3d point is published
  • shape_window: Provides a highgui-based interface for drawing and manipulating 2D shapes.

MIT

• iSAM: Incremental Smooth and Mapping released as LGPL.

USC

• OIT: Overhead interaction toolkit for tracking robots and people using an overhead camera.
• deixis: Deictic gestures, such as pointing

Freiburg

• articulation: (stable) Fit and select appropriate models for observed motion trajectories of articulated objects.
• Contributions to pcl, including range image class and border extraction method

Bosch

• wviz: Web visualization toolkit to support their PR2 Remote Lab. Bosch has already been able to use their Remote Lab to collaborate with Brown University, and Brown University has released a rosjs to access ROS via a web browser.

Penn

• teleop_ar_toolkit: Control PR2 arms and head using fiducials
• minipr2: code for running mini-PR2 constructed out of CKBot modules
• sbpl_arm_planner: search-based motion planning algorithm for robotic manipulators (updated version coming soon).
• pr2_haptic_event_detector: detect contact events using PR2 fingertip pressure sensors

I Heart Robotics has released a zeroconf package for ROS the enables advertising of ROS masters using Zeroconf/Avahi. This provides configuration-less setup to applications like I Heart Robotics's RIND and will also be a useful tool for multi-robot communication.

Sarah Osentoski of Brown's RLAB recently announced a beta version of a ROS to RL-Glue bridge for reinforcement learning

Brown is pleased to announce our beta version of rosglue. rosglue is a bridge between ROS and RL-Glue, a standard reinforcement learning (RL) framework.

rosglue is designed to enable RL researchers and roboticists work together rather than having to reimplement existing methods in both fields. A goal of rosglue is to allow ROS users to use RL algorithms provided by RL researchers and, likewise, to allow RL researchers to more easily use robots running ROS as a learning environment. rosglue allows a robot running ROS to become an RL-Glue environment allowing RL-Glue compatible agents to control the robot. A high level visualization of the framework can be seen here.

rosglue uses a yaml configuration file to specify the topics and services and the learning problem. rosglue automatically subscribes to the topics and services specified in the file. rosglue sends actions selected to the RL-Glue to the robot using the appropriate topic or service. It then creates observations from specified topics for the RL-Glue agent.

rosglue is currently available for download from the brown-ros-pkg repository via:

svn co https://brown-ros-pkg.googlecode.com/svn/trunk/experimental/rlrobot/rosglue rosglue

and preliminary documentation can be found here:

http://code.google.com/p/brown-ros-pkg/wiki/rosglue

Robot Learning and Autonomy @ Brown (RLAB)

The CCNY Robotics Lab, which was recently featured in this CityFlyer blog post, has just announced the release of two packages for laser scan registration.

Dear ROS-Users,

The CCNY Robotics Lab is pleased to announce the release of two packages for laser scan registration. canonical_scan_matcher is a wrapper around Andrea Censi's "Canonical Scan Matcher" [1]. polar_scan_matching is a wrapper around Albert Diosi's "Polar Scan Matching" [2].

Both packages estimate the displacement of a robot by comparing consecutive Laser Scan messages. They can be used without providing any estimate for the displacement of the robot between the scans. In this way, they can serve as an odometric estimate for robots that don't have any other odometric system. Alternatively, a displacement estimate can be provided as input to the scan matchers, in the form of an Imu message or a tf transform, in order to produce better (or faster) scan matching results.

While the two scan matchers use different algorithms and parameters, the ROS wrappers are identical in terms of topics/frames/tf's, making the two packages interchangeable.

Documentation and usage instructions can be found at the respective wiki pages:

• canonical_scan_matcher
• polar_scan_matcher

As usual, we have provided a small demo bag file with laser data and a launch file that can be used to view the packages in action. Each wiki page also has a video of what the output of the demo should look like.

We hope you find the scan matchers useful, and we extend our thanks to the authors of the original implementations.

Ivan Dryanovski
William Morris
The CCNY Robotics Lab

[1] A. Censi, "An ICP variant using a point-to-line metric" Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2008
[2] A. Diosi and L. Kleeman, "Laser Scan Matching in Polar Coordinates with Application to SLAM " Proceedings of 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, August, 2005, Edmonton, Canada

In addition to WowWee drivers and OpenCV tutorials, I Heart Robotics has just released a great Ubuntu panel tool called RIND, which stands for Robot/ROS Status Indicator. You can use it to manage your local roscore as well get information on ROS nodes and topics. Checkout the documentation or read the announcement for more information.

Tim Niemueller has announced a ROS client for Lua

Hello ROS users.

During the last weeks we have developed a Lua-based API to write ROS nodes in the Lua programming language. It allows for communicating with other nodes and participate in the ROS universe. It has been developed at Intel Labs Pittsburgh as part of my research stay this year working with Dr. Siddhartha Srinivasa on the Personal Robotics project.

Some highlights of the implementation:

• Completely written in Lua, no wrappers
• Implements topic and service communication
• Reads message specifications on the fly and generates appropriate data structures at run-time, avoiding offline code generation
• Fully documented API
• only about 2800 lines of code (ohcount)
• Test scripts for all features and simple examples

The implementation benefits from the inherent single-threading in Lua, meaning that everything is processed in a single main loop. This is one of the major factors for its simplicity. No attempts have been made to incoporate Lua add-ons that would provide true multi-threading. The implementation has by far not the same versatility as the Python or C++ API (and we do not aim at that), but it does provide a very simple way to interact with ROS and do this directly without a middle-man from Lua.

The endeavor has been conducted to prepare for porting the Fawkes behavior engine to ROS, which is a framework for developing robot behavior employing Lua as its scripting language.

We would be delighted if the community would take a look at the implementation and provided some feedback. Once it has reached a certain stability and the feature set has been expanded to an acceptable coverage we would like to propose this for inclusion in the experimental package tree.

You can find the source code at http://github.com/timn/roslua. Some documentation on how to get started is provided in the README file.

Regards,
Tim

post by Trevor Jay of Brown to ros-users

Hi ROS-Users!

Recently thanks to the very nice people at Bosch and their remote lab efforts, we were able play around with an actual PR2. We wanted to share the following video of a single ROS node (svn co https://brown-ros-pkg.googlecode.com/svn/trunk/experimental/nolan nolan) running completely unmodified on three very different robots (including the PR2).

Each of the robots is using ar_recog, our ROS-compatible wrapper around ARToolkit to localize ARTags. A simple PID controller then directs them in following. As the Nao does not recognize Twist msgs, we have a simple ten-liner rebroadcasting them as Walk msgs, but even here the the original control node is running unmodified.

Anyone interested in the vision stack can check it out at the brown ros pkg repository.

Thanks everyone in the community who's helping to bring about this level of portability.

_RLAB (Robot Learning and Autonomy @ Brown)

Ivan Dryanovski of the CCNY Robotics Lab has announced a New College Data parser for ROS. The announcement is below.

Dear ROS community,

As you may know, the New College Data is a freely available dataset collected from a robot completing several loops outdoors around the New College campus in Oxford. The data includes odometry, laser scan, and visual information.

We have recently released a ROS parser for the New College Dataset. The parser reads in the .alog data files provided on the NCD website and broadcasts ROS messages in real time. The parser also broadcasts transforms between the robot's base frame and the sensor frames, as defined in the NCD paper.

Details about the package, as well as a demo video of the NCD data being played back in rviz, are available here:

http://www.ros.org/wiki/ncd_parser

We hope you find this useful.

Ivan Dryanovski
CCNY Robotics Lab
The City College of New York

Geoffrey Biggs has released a patch that integrates ROS seamlessly into OpenRTM-aist. OpenRTM-aist users can download a patch that adds in ROS transport.

Although I guess it's not a common thing yet, there have been murmurings for quite a while now here in Japan about a desire to be able to use OpenRTM-aist and ROS together. We would gain the huge range of functional software and the persistent channel-based communications of ROS, and keep the strong life-cycle and execution management of OpenRTM-aist.

So here's a patch for OpenRTM-aist that does exactly that.

This patch adds a new transport type to OpenRTM-aist specifically for communicating across ROS channels. No doubt someone will find the ability to use a persistent channel for communication useful, but the main benefit is that it gives nearly-seamless communications between components written for OpenRTM-aist and nodes written for ROS. Your network of distributed components/nodes no longer has to be in just one framework.

There are no wrappers involved. It's all native communication using the same ROS libraries as you would use in a pure-ROS system - no translation layers means maximum efficiency. You create a port type for the ROS transport, and off you go. If you already know ROS, you'll feel right at home using the ports.

The one caveat is why I say nearly-seamless: we still don't have a unified set of types (also, there are some issues with the typing system in OpenRTM-aist that we're working to sort out). Fortunately, the types issue is a hot topic amongst framework designers at the moment, so I hope we will have solved that problem before too long. :)

I have attached both the patch, for OpenRTM-aist-1.0.0, and a set of examples for each port type (publisher/subscriber/client/server). I hope to get a web page up on the OpenRTM-aist site shortly with a more detailed explanation of usage; for now, the examples and the doxygen comments in the source will point you in the right direction - it's all pretty simple.

Comments, suggestions, and improvements are welcome.

• Patch
• Examples

The ROS community has grown an amazing amount this year. As the Robots Using ROS has illustrated, there are all types of robots using ROS, from mobile manipulators, to autonomous cars, to small humanoids. As the types of robots has increased, so too has the variety of software you can use with ROS, whether it be hardware drivers, libraries like exploration, or even code for research papers. This diversity has allowed all types of developers, including researchers, software engineers, and students, to participate in this growing community.

Today we officially crossed the 1000 ROS package milestone. This is due in no small part to the many new ROS repositories that have come online this year. We are now tracking 25 separate ROS repositories that are providing open source code, including repositories from:

• CU Boulder
• UT Austin/Austin Robot Technology
• Albert-Ludwigs-Universität in Freiburg
• University of Arizona
• Georgia Tech
• Bosch
• University of Maryland
• Care-O-bot
• Brown University
• Technische Universität München
• Stanford AI Lab
• Willow Garage
• Washington University in St. Louis
• I Heart Robotics
• Carnegie Mellon University
• MIT LIS
• WPI
• ModLab at Penn
• USC
• Columbia University
• KU Leuven
• Cornell
• Tully Foote (personal)
• Andrew Harris (personal)

We're excited to see the expansion of such an amazing and vibrant ROS community. Thank you all for taking part.

We're very excited to announce that the mapping library from SRI International's Karto Robotics is now open source with an LGPL license. This mapping library contains a scan matcher, pose graph, loop detection, and occupancy grid construction -- all important building blocks for 2D navigation. When combined with Willow Garage's Sparse Pose Adjustment (SPA) for optimization (in the sba ROS package), it forms a complete stand-alone library for robust 2D mapping.

The Karto mapping library is being hosted on code.ros.org, and we've already integrated it with the ROS navigation stack. The Karto team recently benchmarked various SLAM systems on the RAWSEEDS dataset and found that newest Karto 2.0 with SPA is slightly less precise than Karto 1.1, but Karto 2.0 was more consistent and faster [1]. The maximum error with Karto 2.0 performed as well as a localization-based solution (MCL). A paper describing the SPA technique is due to be published later this year.

We'd like to thank the Karto team for all the hard work that went into making this happen. You can visit kartorobotics.com to find out more about Karto as well as contact them regarding Karto integration services.

[1]: "Comparison of indoor robot localization techniques in the absence of GPS, Vincent, Regis, Limketkai, Benson, and Eriksen, Michael, In Proceedings of SPIE Volume: 7664 Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XV (Proceedings Volume) of Defense, Security, and Sensing Symposium.

Karto SRI/Willow Garage Integration Team: Kurt Konolige, Benson Limketkai, Michael Eriksen, Regis Vincent, Brian Gerkey, Eitan Marder-Eppstein

Above: Grabcut example

OpenCV 2.1 has been released. In addition to many improvements underneath the hood, OpenCV 2.1 adds the Grabcut (C. Rother, V. Kolmogorov, and A. Blake) image segmentation algorithm. The stereo libraries have also been updated with new and improved algorithms, including H. Hirschmuller's semi-global stereo matching algorithm (SGBM).

Mac OS X users will be happy to know that OpenCV has been updated for Snow Leopard. You can now build it as a 64-bit library and highgui has been updated with new Cocoa and QTKit backends (thanks Andre Cohen and Nicolas Butko). Windows users can also build on 64-bit using MSVC 2008 or mingw64.

There are numerous other improvements with this release. We encourage users to check the change list to find out more.

On an administrative note, OpenCV has migrated from SourceForge to code.ros.org to take advantage of faster servers. The ticket tracker has also moved.

The Minoru is an inexpensive stereo webcam that can now be used with ROS. Bob Mottram recently updated the v4l2stereo library so you can now use both the library and the Minoru camera easily with ROS, as the video below demonstrates. The v4l2stereo library also integrates well with OpenCV.

You can find instructions on the Sentience site on how to easily remove the commercial packaging around the Minoru sensor, as well as instructions on how to use it with ROS.

Links:

• v4l2stereo library (GPL v3)
• Minoru/ROS instructions

When talking with people face-to-face, we may experience the "Cocktail Party Effect": even in a crowded, noisy room, we can use our binaural hearing to focus our listening on a single person speaking. With current telepresence technologies, however, we lose this important ability. Thankfully, there are already researchers giving us new tools for effectively bridging these remote distances.

Kyoto University's Professor Hiroshi Okuno and Assistant Professor Toru Takahashi, Honda Research Institute-Japan's Dr. Kazuhiro Nakadai, and four Kyoto University and Tokyo Institute of Technology graduate students spent a week at Willow Garage, integrating HARK with a Texai telepresence robot. HARK stands for Honda Research Institute-Japan (HRI-JP) Audition for Robots with Kyoto University. The robot audition system provides sound source localization, sound source separation, acoustic feature extraction, and automatic speech recognition.

The HARK system integrated well with ROS and our Texai. The Texai was outfitted with a green salad bowl helmet embedded with eight microphones, and there is now a hark package for ROS. Using this setup, their team put together three demos showing off the potential for telepresence technologies.

In the first demo, four people, including one present through a second Texai, talk over each other while the HARK-Texai separates out each voice. The second demo shows that sound is localized and that sound direction and power can be displayed in a radar chart. The final presentation puts these two demos together into a powerful new interface for the remote operator: the Texai pilot can determine where various sounds and voices are coming from, and select which sound to focus on. The HARK system then provides the pilot with the desired audio, cutting out any background noise or additional voices. Even in a crowded room, you can have a one-on-one conversation.

HARK came out of close collaboration between HRI-JP and Kyoto University, and Professor Okuno's passion to make computer/robot audition helpful for the hearing impaired. HARK is provided free and open source for research purposes and can be licensed for commercial applications.

Travis Deyle of Hizook/Healthcare Robotics Lab posted a nice overview of Robotis Dynamixel servos, including code for using the servos with gt-ros-pkg's robotis package. The Healthcare Robotics Lab uses these servos to construct their tilting Hokuyo laser rangefinder. You'll also find them in robots like Prairie Dog for their Crustcrawler arm.

Article

Logging and playback is one of the most critical features when developing software for robotics. Whether you're a hardware engineer recording diagnostic data, a researcher collecting data sets, or a developer testing algorithms, the ability to record data from a robot and play it back is crucial. ROS supports these needs with "Bag files" and tools like rosbag and rxbag.

rosbag can record data from any ROS data source into a bag file, whether it be simple text data or large sensor data, like images. The tool can also record data from programs running on multiple computers. rosbag can play back this data just as it was recorded -- it looks identical. This means that data can be recorded from a physical robot and used to create virtual robots to test software. With the appropriate bag file, you don't even need to own a physical robot.

There are a variety of tools to help you manage your stored data, such as tools for filtering data and updating data formats. There are also tools to manage research workflows for sending bag files to Amazon's Mechanical Turk for dataset labeling. For data visualization, there is the versatile rxbag tool. rxbag can scan through camera data like a movie editor, plot numerical data, and view the raw message data. You can also write plugins to define viewers for your own data types.

You can watch the video to see rosbag and rxbag in action. Both of these tools are part of the ROS Box Turtle release, which you can download from ROS.org.

In addition to core robotics libraries, like navigation, the ROS Box Turtle release also comes with a variety of tools necessary for developing robotics algorithms and applications. One of the most commonly used tools is rviz, a 3-D visualization environment that is part of the ROS Visualization stack.

Whether it's 3D point clouds, camera data, maps, robot poses, or custom visualization markers, rviz can display customizable views of this information. rviz can show you the difference between the physical world and how the robot sees it, and it can also help you create displays that show users what the robot is planning to do.

You can watch the video above for more details about what the ROS rviz tool has to offer, and you can read documentation and download the source code at: ros.org/wiki/rviz.

We're pleased to announce that all of the packages within the Brown Ros Pkg collection have been updated for ROS-1.0.X compatibility. The Brown automated install script has also been updated to reflect these changes as well. This update should fix any problems caused by using the (formerly) 0.9.0 release with an updated roscore.

We encourage anyone interested to visit brown-ros-pkg . We currently have a driver for the iRobot Create, a basic driver for the Aldebaran Nao, and some basic vision packages.

Additions and improvements include:

probe

A ROS webcam capture node new to this release. Probe leverages Gstreamer, making it compatible with almost every Linux camera and video system available. In addition Gstreamer's software video processing can be used to emulate advanced features (e.g. white-balancing) even for cameras that don't have the appropriate v4l hardware support.

teleop_twist_keyboard

A simple keyboard-based teleop interface inspired by teleop_base. It's only dependencies are on the necessary geometry messages.

As always, we're interested in the communities feedback and suggestions.

_Trevor

Andrew Harris just announced support for the Arduino and cmucam3 with ROS packages in his own ajh-ros-pkg repository. The announcement is below.*

Hello, I have released the source code to my ROS packages for communicating with the Arduino and the cmucam3.

pmad (Arduino)

You can get the code from:

https://ajh-ros-pkg.svn.sourceforge.net/svnroot/ajh-ros-pkg/trunk/pmad/

There is a subdirectory in there called "arduino" that contains a sketch.txt file. This is the sketch you must load into the GUI and upload to the Arduino. Once you do that, you can start the PMAD service:

rosrun pmad pmad_service.py

Note that you'll have to have a "tty" ROS parameter set up if the USB does not connect as /dev/ttyUSB0. You might have to set tty to "/dev/ttyUSB1" for example. Once you set up the service, you can toggle the digital pins with:

rosservice call pmad_switch_control 4 0

This for example will set digital pin 4 to LOW. Note the status packet below only returns the state of digital pins 4, ..., 7.

In addition to this, there is a service status that returns the current A/D outputs on analog pins 0, ..., 3, digital pins 4, ..., 7, and a count of the number of commands executed.

If you want to get the status to be a "topic", there is an application

rosrun pmad pmad_status_publisher.py

That will publish the status data once per second. The status message is defined in msg/Status.msg.

BTW pmad stands for power management and distribution, I use solid state relays on the outputs of the digital pins.

cmucam_png (cmucam3)

In terms of the cmucam_png node, it acquires a png file from the cmucam about once every twenty seconds and publishes it as a compressed image. To be honest I haven't used it in a while as I've switched to a firewire USB camera. The image is a PNG file. It takes ~20 seconds per image because they come over a 115,200 bps serial line. They are 352x287 pixels in size.

https://ajh-ros-pkg.svn.sourceforge.net/svnroot/ajh-ros-pkg/trunk/cmucam_png/

Right now for some reason I get compiler errors compiling image_view so I can't try the image_view to make sure the image is getting pushed around. However I have recently tested that I can receive an image from the cmucam and save it to disk as a png by just adding a couple lines to save the received png to disk. I am just having trouble pushing the image around ROS, but it might just be my setup. (It worked in the past ;) )

For the cmucam application, there is also a binary that must be downloaded to the camera. The source, etc, for this is in the directory "png-robin". In order to get this working you should have a working installation of the cmucam3 development environment, and be able to compile the examples in the cc3/projects directory. Once you can do this, and download these examples to the camera, you can make a new directory in that examples directory called png-robin and copy the contents of this directory into that one. The makefile will then work and compile the application. I have also checked in a hex file that you can use if you don't want to compile the application yourself. But you'll still have to download the hex file with lpc21isp.

Hopefully I have made the packages correctly, but these are the first ones I've tried to make. If you have any questions on either of these nodes, of if they don't seem to work, let me know!

thanks, -andrew

crossposted from willowgarage.com

Peter Pastor, a PhD student at USC, spent the past three months developing software that allows the PR2 to learn new motor skills from human demonstration. In particular, the robot learned how to grasp, pour, and place beverage containers after just a single demonstration. Peter focused on tasks like turning a door handle or grasping a cup -- tasks that personal robots like PR2 will perform over and over again. Instead of requiring new trajectory planning each time a common task is encountered, the presented approach enables the robot to build up a library of movements that can be used to execute these common goals.  For this library to be useful, learned movements must be generalizable to new goal poses. In real life, the robot will never face the exact same situation twice. Therefore, the learned movements must be encoded in such a way that they can be adapted to different start and goal positions.

Peter used Dynamic Movement Primitives (DMPs), which allow the robot to encode movement plans. The parameters of these DMPs can be learned efficiently from a single demonstration, allowing a user to teach the PR2 new movements within seconds. Thus, the presented imitation learning set-up allows a user to teach discrete movements, like a grasping, placing, and releasing movement, and then apply these motions to manipulate several objects on a table. This obviates the need to plan a new trajectory every single time a motion is reused. Furthermore, the DMPs allow the robot to complete its task even when the goal is changed on-the-fly.

You can find out more about the open source code for Peter's work here, and check out his presentation slides below (download PDF). For more about Peter's research with DMPs and learning from demonstration, see "Learning and Generalization of Motor Skills by Learning from Demonstration", ICRA 2009.

Update: Documentation is available on ros.org at http://www.ros.org/wiki/alufr-ros-pkg

I'm happy to announce v0.1, the first proper release of Freiburg's Nao Stack located at: http://code.google.com/p/alufr-ros-pkg/

You can check out the trunk (svn) from:
http://alufr-ros-pkg.googlecode.com/svn/trunk/

or download the stack package from:
http://code.google.com/p/alufr-ros-pkg/downloads/list

Changes are:

• improved torso odometry
• nao_ctrl now also transmits the state of Nao's onboard IMU
• new nao_description package makes Nao's complete joint state, transformations and visualization available through robot_state_publisher
• launch files for convenient start up
  

Some basic instructions are available at http://code.google.com/p/alufr-ros-pkg/, but I might also move them to ros.org if that's the more appropriate place. I would be happy to hear if all is working for you (or not), or how to make things more ROS-compliant.

Best regards, Armin

crossposted from willowgarage.com

Ethan Dreyfuss, who recently received a master's degree from Stanford University, is continuing his work here on autonomous person-following and dataset collection and annotation. The former project provides a useful building block for a wide variety of tasks. Consider a robot that helps you carry groceries. This robot is vastly more useful if it can carry your bags to the house without requiring teleoperation; the robot can simply track you and follow behind. At a high level, person-following comprises two principal tasks: person tracking and navigation.

The approach developed by Ethan and Caroline Pantofaru fuses a face detector with two weak person trackers: one for legs, and one for 3D blobs at person-height. None of these approaches is individually effective enough to provide robust tracking, but their strengths are complementary. The face detector is effective when the person is close to, and directly facing the robot. While the leg tracker provides high accuracy when multiple people are present, it is often confused by non-human obstacles and can therefore not work reliably from afar. Conversely, the height-based blob tracker can effectively track from further away, yet it is easily confused by groups of people. By combining techniques, Ethan and Caroline were able to develop a more robust person-tracking tool.

Once the robot can track a designated person, the information is passed on to the navigation stack. This same navigation software was used to complete Milestone 2, with some improvements made to help deal more quickly and robustly with dynamically-moving obstacles such as people.

In addition to the person-following project, Ethan is contributing to the collection and labeling of a large dataset of people in an indoor office environment. One of the major drivers of computer vision research is the availability of high-quality labeled data. The bulk of existing person datasets exclude indoor environments, and instead focus on outdoor pedestrians. Indoor environments present numerous challenges for person detection, including poor lighting and environmental clutter. By automating as much as possible, the process of both collecting (using the robot) and labeling (using Amazon's Mechanical Turk and Alex Sorokin's CV Web Annotation Toolkit), Ethan's team will be able to provide a large, compelling dataset to encourage other researchers to tackle these challenging problems.

Ethan also picked up a number of side projects including rapid neighborhood computation on point clouds, and implementing a package that uses the open-source video codec Theora to allow low-bandwidth video streaming within ROS.

Fast on the heels of the Brown Nao driver, Armin Hornung of Albert-Ludwigs-Unversität Freiburg has announced joy-package compatibility and torso odometry additions for the Nao driver -- as well as the alufr-ros-pkg repository. Armin's announcement is below.

Based on the recently announced Nao driver by Brown University, there is now a regular joystick teleoperation node available. It operates Nao using messages from the "joy" topic, so it should work with any gamepad or joystick in ROS. In addition, the control node running on Nao returns some basic torso odometry estimate. The code is available at http://code.google.com/p/alufr-ros-pkg/ to checkout via SVN. A README with more details can be found in the "nao" stack there.

I'm open for feedback and suggestions!

-- Armin

Brown University has been hard at work on a developing an Aldebaran Nao driver for ROS and recently announced an open source (GPL) version to the community. The text of Trevor Jay's announcement is below.

Brown University is pleased to release a ROS driver for the Aldebaran Nao. The driver makes available: head control, text-to-speech, basic navigation, and (most interestingly) the Nao's forehead camera. Sample clients are part of the download, including a WiiMote teleop client.

Here are two videos of the driver in action (one from the robot's perspective):

The driver is available at the brown-ros-pkg download page.

If you have any problems or just find the driver useful, please let us know! We will add features as our work and the community need them.

_Trevor

crossposted from willowgarage.com

Nate Koenig of the Interaction Lab at USC is continuing his work here at Willow Garage after a busy summer. Nate carried out an empirical study investigating the use of people as teachers for robots, while also researching learning by demonstration with PR2.

Participants in Nate's study used learning by demonstration to teach PR2 how to solve the Towers of Hanoi puzzle. As the name implies, learning by demonstration relies on a human teacher to provide a robot "student" with demonstrations of a complex task. In this case, the robot uses the state of the puzzle (i.e., location of red disk compared to blue and green), along with the teacher's command, to learn the demonstrated task. Volunteers used a web-based teaching tool to guide PR2 through the three-disk puzzle board. In one condition, teachers were able to directly see PR2, while in the other condition teachers viewed the robot's actions through a small video feed on the web tool. This manipulation allowed Nate to study if robot visibility affects teaching strategies and outcomes. Based on participants' commands and other observations from the environment, the robot learned how to solve the puzzle on its own.

Results from this study indicate that teachers perform better when visually separated from the robot. Performing "better" means that participants made fewer unnecessary or repetitive moves when teaching the robot. While this may seem counter-intuitive, the teachers who could see the robot were easily distracted from the task and seemed to build inaccurate mental models of PR2's capabilities.

In addition to this experiment, Nate worked on a number of smaller projects including developing the first video streaming ROS node and creating a web-based graphical interface for interacting with PR2. You can find many of these contributions in the hanoi package for ROS. Nate is also creator and lead developer for Gazebo, a popular open-source 3D robot simulator. Gazebo is heavily used at Willow Garage to simulate the actions of PR2, and Nate is providing us with numerous improvements.

On the Willow Garage Blog, you can find out more about the Texas Robot, which is a telepresence robot built out of leftover PR2 parts and off-the-shelf components. One of the interesting aspects of the Texas robot is that it is running the PR2 software components as-is: motor controllers, tele-operation, visualization, etc... The only changes required were modifying the robot model (urdf) and writing new roslaunch files. You can find these changes in the texas package on ROS.org. Any other changes have been improvements that have been useful for both the Texas and PR2 robots, such as updates to the teleoperation controls.

crossposted from willowgarage.com

This summer, Jason Wolfe (UC Berkeley) worked on hierarchical task planning for the PR2. Planning is important for performing tasks efficiently. If you're gathering ingredients for the Triple Chocolate Volcanic Explosion Soufflé you're fiendishly craving, you may fetch both the eggs and the butter from the fridge before heading to the pantry for the sugar and chocolate. If you weren't planning ahead, you might go get the eggs, drop them off on the counter, go back to the fridge to get the butter, unload again, continue to the pantry for the chocolate, and so on. You'll still make a soufflé, but your plan will be far from optimal.

Jason's work on hierarchical planning helped the PR2 plan out its tasks in a more logical, premeditated manner. The package takes into consideration high-level decisions, like the order in which to fetch the ingredients, down to low-level decisions like where to position the base of the robot and what angle to grasp an object from. In the video, the PR2 is told to move several bottles between two tables. Instead of manipulating each bottle independently of the others, the robot plans ahead and places the bottle being manipulated near the next bottle to grab. That way, the robot can simply put down the current item and, without driving to the next goal, reach over and pick up its next bottle. Jason's code can be found in the hierarchical_planning package. You may also be interested in Jason's and Bhaskara Marthi's RSS 2009 presentation on Angelic Hierarchical Planning.

In the process of working on this package, Jason spent time developing and debugging lower-level component actions, such as moving the base to a particular location. Interactive scripting languages are ideal for this purpose. Jason developed an experimental Java client library for ROS (rosjava), which he then used to create a ROS interface to Clojure (rosclj), a Lisp-like language built on Java. Using this interface, he then developed a large library of scripted actions which can be used for quickly specifying and tele-operating complex sequences. These can be found in the clj_pr2 package.

crossposted from willowgarage.com

Ben Cohen of University of Pennsylvania has returned to the GRASP Lab after his summer internship here at Willow Garage. At Penn, Ben researches search-based methods for path planning for robotic manipulators. During his time here, he worked on two motion planners: one for door opening and one for manipulation. When compared to the door planner used in Milestone 2, the new door planner uses SBPL (Search-Based Planning Library) to give the PR2 two new capabilities. First, it allows the robot to not only push doors open, but also pull. Second, the door planner allows the robot to open doors, regardless of hinge position -- left or right side of the door. These two novel capabilities allow for robust, more universal door opening.

Additionally, Ben's work on a manipulation planner involved integrating SBPL into the move_arm ROS package, which integrates a variety of motion planners. Ben tested the SBPL planner on the PR2's arms, and added the supporting software needed to perform collision checking. With collision checking in place, SBPL can more readily handle cluttered, complex environments.

Here are Ben's end-of-summer presentation slides discussing his planning work (Download PDF from ROS.org):

crossposted from willowgarage.com

This summer, Alex Teichman of Stanford University worked on an image descriptor library, and used this library to develop a new method for people to interact directly with the PR2. Using a keyboard or a joystick is great for directly controlling a robot, but what if an autonomous robot wanders into a group of people -- how can they affect its behavior?

Alex's approach allows the PR2 to "talk to the hand," as many of us cruelly experienced in middle school. In the video, Alex demonstrates that the PR2 can be made to stop and go by simply holding a hand up to the stereo camera. To accomplish this, Alex used a machine-learning algorithm called Boosting along with image descriptors, such as the color of local regions and object edges. He was able to train his algorithm on a data set that was labelled using the Amazon Mechanical Turk library that Alex Sorokin developed. This Mechanical Turk library harnesses the power of paid volunteers on the Internet to perform tasks, like identifying hands in images so that algorithms like Boosting can be trained.

Hand detection is part of a larger effort that Alex Teichman has been working on, developing a library with a common interface to image descriptors. This library, descriptors_2d, enables ROS developers to use easily use descriptors like SURF, HOG, Daisy, and Superpixel Color Histogram.

You can learn more about the hand detection techniques and image descriptors in Alex's final summer presentation (download PDF from ROS.org).

crossposted from willowgarage.com

Matt Piccoli (University of Pennsylvania) and Jürgen Sturm (University of Freiburg) did a lot more than break eggs and learn about how things move. They also tested the limits of PR2's fingertip pressure sensors with a seemingly simple task: identifying, without looking, if a juice bottle is open or closed, and full or empty.

Tactile information is invaluable when determining properties of objects that are visually inaccessible. In this vein, Matt and Jürgen developed a tactile perception strategy that can be used to detect the internal state of liquid containers. By measuring a bottle's reaction to a force applied by a gripper, their system can recognize whether the bottle is full or empty, and open or closed. The system learned this information from a set of training experiments carried out on different types of bottles and soda cans. Knowing whether a bottle is open or closed can help a robot determine the level of care required when manipulating the object.

You can find the code for Matt and Jürgen's work in the pr2_gripper_controller package on ROS.org.

crossposted from willowgarage.com

Marius Muja of University of British Columbia began his internship in the middle of Milestone 2 excitement. For several weeks, he worked on two important perception components: detecting outlets from far away, and detecting door handles.

Thereafter, Marius focused on tabletop object detection and wrote the tabletop_objects package. Determining the exact position and orientation of an object, as well as its identity, is very important if a robot is grasping objects, and especially crucial if the object in question is fragile. Tabletop_objects uses a two-stage approach. In the bottom-up stage, initial estimations of possible object locations are made, and in the top-down stage, 3D models are fit into the estimated locations. After fitting the correct 3D model, the object's identity, position and orientation can be determined with high confidence. This approach can even distinguish between similar-looking drinking glasses. Marius worked with Ioan Sucan to integrate tabletop_objects and motion planning (move_arm), and together, they were able to successfully detect, grasp and manipulate fragile glass objects.

In addition to his work with tabletop_objects, Marius integrated FLANN (Fast Library for Nearest Neighbors) into OpenCV, and developed a phone-based teleoperation mode for PR2 based on Asterisk, an open source PBX.

Here are Marius's end-of-summer slides, where you can find more details about his work.

Marius Muja: Tabletop Object Detection (Download PDF from ROS.org)

crossposted from willowgarage.com

This summer, Dan Munoz of Carnegie Mellon University worked on helping the PR2 understand its environment using its 3-D sensors. Improving 3-D perception is important because it can help the PR2 with many tasks such as localization and object grasping. At CMU, Dan and collaborators are developing techniques to improve 3-D perception for an unmanned vehicle in outdoor natural and urban environments. These techniques first take in a cloud of 3-D points, usually collected from a laser scanner, and a label associated with each point. These labels identify such objects as buildings, tree trunks, plants, power lines, and the ground. Then, various local and more global features that describe the local shape and distribution of each object are extracted for each point and region of points. These labeled examples are then used to train an advanced machine learning tool that reasons the best way to combine the local and global features that describe each object. In new environments, this feature extraction process is repeated and given to the machine learning tool to determine what objects are present in the novel scene.

While at Willow Garage, Dan integrated this learning framework into ROS. As shown in the video, Dan experimented with helping the PR2 perceive objects on the room-sized scale, such as tables and chairs, as well as objects at the table-top-sized scale, including mugs and staplers. During the Intern Challenge, Dan also applied this same framework to distinguish between the three different types of bottles being served: Odwalla, Naked, and water. Dan developed the descriptors_3d package, the library used to compute various 3-D features for a point or region of points from a stereo camera or laser scanner. Additionally, he developed the functional_m3n package (Functional Max-Margin Markov Networks), the advanced machine learning tool that learns how to combine low-level and high-level feature information for each object.

Below are Dan's final presentation slides.

Daniel Munoz: Indoor Object Detection on Scribd (Download PDF from ROS.org)

cross posted from willowgarage.com

Mrinal Kalakrishnan, one of three motion planning interns here at Willow Garage, is finishing up his summer project and returning to the University of Southern California. Mrinal has been working on a smooth motion planning and control pipeline for the PR2, introducing a new approach to object manipulation The key component of this work was the implementation of CHOMP (Covariant Hamiltonian Optimization and Motion Planning), a motion planner developed at CMU and Intel Research. You can find this implementation in the chomp_motion_planner package for ROS.

Mrinal chose to implement this motion planner on the PR2 because CHOMP's method of planning away from obstacles produces very smooth, natural-looking movements. You can see in the video that the PR2's arm trajectory is rather fluid and avoids unusual or awkward joint angles. The animation shows the arm optimizing the trajectory away from the bookshelf, while maintaining a smooth motion plan. Mrinal's work with CHOMP allowed for informative comparisons to be made with the two other motion planners being researched and implemented here, ompl and sbpl_arm_planner. All three motion planners use the same interface, making switches between the three systems very simple.

In addition to his work with CHOMP, Mrinal wrote the distance_field package for ROS which performs 3-D obstacle inflation to generate a cost-map for arm planners. He also wrote spline_smoother, a library of algorithms which can convert a set of waypoints, as typically generated by motion planners, into a smooth spline trajectory suitable for execution on a robot.

Below is Mrinal's end-of-summer presentation, where you can find additional details about his work here at Willow Garage.

Mrinal Kalakrishnan: CHOMP Motion Planner on Scribd (Download PDF from ROS.org)

crossposted from willowgarage.com

Min Sun is returning to University of Michigan at Ann Arbor, where he does computer vision research with particular interest in 3-D object recognition. During his summer here at Willow Garage, he focused on recognizing table-top objects like mice, mugs, and staplers in an office environment. Min is the the primary creator of rf_detector (Random Forest), which recognizes objects and their poses. The detector uses the stereo camera along with the texture light projector to collect images and the corresponding dense stereo point clouds. From there, rf_detector predicts the object type (i.e. mouse, mug, stapler) and its location and orientation. This information can be crucial to have before attempting object manipulation, as many object types, such as mugs, require careful handling.

In the future, Min will be looking for ways to scale up this approach to a wider range of object classes. Min continues to look for other features and model representations that make object recognition more robust.

Min also wrote the geometric_blur package, which calculates geometric blur descriptors.

Here are the slides from Min's final internship presentation describing his work on rf_detector and the detection pipeline.

Min Sun: 3D Object Detection on Scribd (Download PDF from ROS.org)

crossposted from willowgarage.com

Ioan Sucan is headed back to Rice University after his third stay here at Willow Garage. Ioan is a motion planning researcher and is the author of Open Motion Planning Library (OMPL), a library of sampling-based motion planning algorithms. These algorithms are important for the PR2 because they enable the arm to grasp and manipulate objects, while simultaneously avoiding collisions with people and other still or moving objects.

This past winter, Ioan and Radu Rusu used OMPL to do dynamic collision avoidance. This summer, Ioan was able to make many improvements to OMPL so that the PR2 can grasp and manipulate objects in cluttered indoor environments. In the video, you can see how the PR2 is able to grasp objects while moving its arm through complex obstacle courses. Data from the tilting laser scanner is used to construct and update a 3D-representation of the environment, allowing the arm to avoid even moving obstacles.

Ioan also improved the robot_self_filter and the collision_map. When the PR2's arm moves in front of the sensors, two problems occur: the arm looks like an obstacle in the environment, and the arm blocks the robot's view of the environment behind the arm. The robot_self_filter, in combination with the collision_map, allows the PR2 to disregard its arm as an obstacle, and "remember" objects behind the arm.

Here are the slides from Ioan's end-of-summer presentation describing his work on OMPL and other projects.

Ioan Sucan: Motion Planning for the PR2 Arm on Scribd (Download PDF from ROS.org)

crossposted from willowgarage.com

Alex Sorokin of University of Illinois, Urbana-Champaign has been hacking on many projects this summer, extending work that he started at Willow Garage last year. Alex is the author of the CV Web Annotation Toolkit, which is an open source tool that helps researchers in computer perception collect and classify data, with the help of workers on Amazon's Mechanical Turk. If you're a researcher collecting a data set, you can use this toolkit to submit images to Mechanical Turk and pay people to label them for you. For example, you can instruct Mechanical Turk users to draw boxes around all of the people in an image, or draw polygons around people's hands. These manually-labeled datasets allow researchers to test their own algorithms and compare them against what a person sees. If you're worried about the quality of results that random Turkers might give you, you can easily let an additional set of users grade the results that you get back, and increase the likelihood of accurate responses.

In addition to helping our researchers collect data sets, this toolkit is also very useful for our robots. For just a couple of dollars, you can easily submit images to Mechanical Turk and have users teach the robot where your refrigerator and dishwasher are, what your cups look like, or where your power outlets are. We're a long way from having robots that can identify objects in an environment as well as humans can, but toolkits like Alex's help us to bridge that gap by allowing humans to lend their abilities to robots.

In addition to his work with Mechanical Turk, Alex came up with many great extensions to ROS. These include bagserver, which enables random-access into ROS bag files, and bag_image_view, which lets you visually scan through images in a bag file and play them back.

Below is Alex's end-of-summer presentation, which discusses his various contributions in greater detail.

Alex Sorokin: CV/Mechanical Turk Presentation (Download PDF from ros.org)

University of Freiburg researcher Jürgen Sturm is just finishing his second internship here at Willow Garage. In addition to helping out with our egg-breaking efforts, he's been using the stereo cameras of PR2 to detect planar objects like doors, drawers, books, and boxes. More importantly, he has been tracking the movement of these objects to learn articulation models, i.e. how these objects move. Does the door open to the left or the right? Does the drawer slide in or out? Where will the handle of the drawer be when it is fully open? This is the sort of information that is critical for enabling robots to operate in our own environments.

We've posted a video of Jürgen discussing his findings as well as slides from a presentation that he gave at Willow Garage. You can also download his code from the planar_objects ROS package.

Planar Objects and Articulation Models on Scribd (Download PDF from ros.org)

Awhile back we showed a demo video using an iPod Touch to drive around of a PR2. It was a fun experiment, but it relies on a proof-of-concept that's not quite ready for primetime.

Srećko Jurić-Kavelj of the University of Zagreb showed us that there's more than one way to get data from an iPod Touch into ROS. Instead of cross-compiling ROS onto the iPhone, which is still very difficult, he used the open-source accelerometer-simulator project on Google Code to receive the accelerometer on another computer. He was then able to easily adapt a sample Python script to broadcast that data as a rospy node.

You can see the results here as he drives around a Pioneer 3DX:

Integrating ROS with other robot software frameworks, such as OpenRAVE, Player, and Euslisp, has been important to us as it allows developers to leverage the strengths of each. No framework can be the best at everything, so it's important to allow developers to choose the tools they need. This video shows the integration between ROS and the Orocos Real Time Toolkit (RTT), which was done by Orocos developer Ruben Smits during a month-long visit to Willow Garage. Ruben became a vital member of our Milestone 2 team and still managed to have time to build this great, seamless integration.

While ROS has good support for distributing components over the network, RTT offers hard realtime communication between components. This demo combines the best of both worlds: the realtime low level control is handled by a set of RTT components that directly control the the PR2 robot hardware, but the communication between the controllers is visible to the whole ROS network.

The integration makes the RTT components appear as ROS components, without breaking the realtime communication between the RTT components. This allows RTT developers to use tools developed for ROS, such as rviz and rxplot, to visualize the communication between controllers.

-- Wim Meeusen

Josh Faust and Rob Wheeler recently got ROS working on the iPod Touch/iPhone and put together a quick demo that uses an iPod Touch as a joystick for the PR2 robot. The iPod Touch and iPhone combine a high-quality display with different modes of interaction that make it appealing for robotics interfaces, and they are a platform to test cross-compilation of ROS. The difficult challenge in getting ROS running on the iPod Touch was solving the cross-compilation issues. Once they had those figured out, they were able to add about twenty lines of code to the standard iPod Touch accelerator demo to translate the accelerator input into commands to drive the PR2.

This is still a proof of concept, but we hope in the coming months to make it a stable platform for ROS development. We've put up a ROSPod wiki page so you can track our efforts and contribute your own.