Recently in packages Category

Sarah Osentoski of Brown's RLAB recently announced a beta version of a ROS to RL-Glue bridge for reinforcement learning

Brown is pleased to announce our beta version of rosglue. rosglue is a bridge between ROS and RL-Glue, a standard reinforcement learning (RL) framework.

rosglue is designed to enable RL researchers and roboticists work together rather than having to reimplement existing methods in both fields. A goal of rosglue is to allow ROS users to use RL algorithms provided by RL researchers and, likewise, to allow RL researchers to more easily use robots running ROS as a learning environment. rosglue allows a robot running ROS to become an RL-Glue environment allowing RL-Glue compatible agents to control the robot. A high level visualization of the framework can be seen here.

rosglue uses a yaml configuration file to specify the topics and services and the learning problem. rosglue automatically subscribes to the topics and services specified in the file. rosglue sends actions selected to the RL-Glue to the robot using the appropriate topic or service. It then creates observations from specified topics for the RL-Glue agent.

rosglue is currently available for download from the brown-ros-pkg repository via:

svn co https://brown-ros-pkg.googlecode.com/svn/trunk/experimental/rlrobot/rosglue rosglue

and preliminary documentation can be found here:

http://code.google.com/p/brown-ros-pkg/wiki/rosglue

Robot Learning and Autonomy @ Brown (RLAB)

The CCNY Robotics Lab, which was recently featured in this CityFlyer blog post, has just announced the release of two packages for laser scan registration.

Dear ROS-Users,

The CCNY Robotics Lab is pleased to announce the release of two packages for laser scan registration. canonical_scan_matcher is a wrapper around Andrea Censi's "Canonical Scan Matcher" [1]. polar_scan_matching is a wrapper around Albert Diosi's "Polar Scan Matching" [2].

Both packages estimate the displacement of a robot by comparing consecutive Laser Scan messages. They can be used without providing any estimate for the displacement of the robot between the scans. In this way, they can serve as an odometric estimate for robots that don't have any other odometric system. Alternatively, a displacement estimate can be provided as input to the scan matchers, in the form of an Imu message or a tf transform, in order to produce better (or faster) scan matching results.

While the two scan matchers use different algorithms and parameters, the ROS wrappers are identical in terms of topics/frames/tf's, making the two packages interchangeable.

Documentation and usage instructions can be found at the respective wiki pages:

• canonical_scan_matcher
• polar_scan_matcher

As usual, we have provided a small demo bag file with laser data and a launch file that can be used to view the packages in action. Each wiki page also has a video of what the output of the demo should look like.

We hope you find the scan matchers useful, and we extend our thanks to the authors of the original implementations.

Ivan Dryanovski
William Morris
The CCNY Robotics Lab

[1] A. Censi, "An ICP variant using a point-to-line metric" Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2008
[2] A. Diosi and L. Kleeman, "Laser Scan Matching in Polar Coordinates with Application to SLAM " Proceedings of 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, August, 2005, Edmonton, Canada

In addition to WowWee drivers and OpenCV tutorials, I Heart Robotics has just released a great Ubuntu panel tool called RIND, which stands for Robot/ROS Status Indicator. You can use it to manage your local roscore as well get information on ROS nodes and topics. Checkout the documentation or read the announcement for more information.

Tim Niemueller has announced a ROS client for Lua

Hello ROS users.

During the last weeks we have developed a Lua-based API to write ROS nodes in the Lua programming language. It allows for communicating with other nodes and participate in the ROS universe. It has been developed at Intel Labs Pittsburgh as part of my research stay this year working with Dr. Siddhartha Srinivasa on the Personal Robotics project.

Some highlights of the implementation:

• Completely written in Lua, no wrappers
• Implements topic and service communication
• Reads message specifications on the fly and generates appropriate data structures at run-time, avoiding offline code generation
• Fully documented API
• only about 2800 lines of code (ohcount)
• Test scripts for all features and simple examples

The implementation benefits from the inherent single-threading in Lua, meaning that everything is processed in a single main loop. This is one of the major factors for its simplicity. No attempts have been made to incoporate Lua add-ons that would provide true multi-threading. The implementation has by far not the same versatility as the Python or C++ API (and we do not aim at that), but it does provide a very simple way to interact with ROS and do this directly without a middle-man from Lua.

The endeavor has been conducted to prepare for porting the Fawkes behavior engine to ROS, which is a framework for developing robot behavior employing Lua as its scripting language.

We would be delighted if the community would take a look at the implementation and provided some feedback. Once it has reached a certain stability and the feature set has been expanded to an acceptable coverage we would like to propose this for inclusion in the experimental package tree.

You can find the source code at http://github.com/timn/roslua. Some documentation on how to get started is provided in the README file.

Regards,
Tim

post by Trevor Jay of Brown to ros-users

Hi ROS-Users!

Recently thanks to the very nice people at Bosch and their remote lab efforts, we were able play around with an actual PR2. We wanted to share the following video of a single ROS node (svn co https://brown-ros-pkg.googlecode.com/svn/trunk/experimental/nolan nolan) running completely unmodified on three very different robots (including the PR2).

Each of the robots is using ar_recog, our ROS-compatible wrapper around ARToolkit to localize ARTags. A simple PID controller then directs them in following. As the Nao does not recognize Twist msgs, we have a simple ten-liner rebroadcasting them as Walk msgs, but even here the the original control node is running unmodified.

Anyone interested in the vision stack can check it out at the brown ros pkg repository.

Thanks everyone in the community who's helping to bring about this level of portability.

_RLAB (Robot Learning and Autonomy @ Brown)

Ivan Dryanovski of the CCNY Robotics Lab has announced a New College Data parser for ROS. The announcement is below.

Dear ROS community,

As you may know, the New College Data is a freely available dataset collected from a robot completing several loops outdoors around the New College campus in Oxford. The data includes odometry, laser scan, and visual information.

We have recently released a ROS parser for the New College Dataset. The parser reads in the .alog data files provided on the NCD website and broadcasts ROS messages in real time. The parser also broadcasts transforms between the robot's base frame and the sensor frames, as defined in the NCD paper.

Details about the package, as well as a demo video of the NCD data being played back in rviz, are available here:

http://www.ros.org/wiki/ncd_parser

We hope you find this useful.

Ivan Dryanovski
CCNY Robotics Lab
The City College of New York

Geoffrey Biggs has released a patch that integrates ROS seamlessly into OpenRTM-aist. OpenRTM-aist users can download a patch that adds in ROS transport.

Although I guess it's not a common thing yet, there have been murmurings for quite a while now here in Japan about a desire to be able to use OpenRTM-aist and ROS together. We would gain the huge range of functional software and the persistent channel-based communications of ROS, and keep the strong life-cycle and execution management of OpenRTM-aist.

So here's a patch for OpenRTM-aist that does exactly that.

This patch adds a new transport type to OpenRTM-aist specifically for communicating across ROS channels. No doubt someone will find the ability to use a persistent channel for communication useful, but the main benefit is that it gives nearly-seamless communications between components written for OpenRTM-aist and nodes written for ROS. Your network of distributed components/nodes no longer has to be in just one framework.

There are no wrappers involved. It's all native communication using the same ROS libraries as you would use in a pure-ROS system - no translation layers means maximum efficiency. You create a port type for the ROS transport, and off you go. If you already know ROS, you'll feel right at home using the ports.

The one caveat is why I say nearly-seamless: we still don't have a unified set of types (also, there are some issues with the typing system in OpenRTM-aist that we're working to sort out). Fortunately, the types issue is a hot topic amongst framework designers at the moment, so I hope we will have solved that problem before too long. :)

I have attached both the patch, for OpenRTM-aist-1.0.0, and a set of examples for each port type (publisher/subscriber/client/server). I hope to get a web page up on the OpenRTM-aist site shortly with a more detailed explanation of usage; for now, the examples and the doxygen comments in the source will point you in the right direction - it's all pretty simple.

Comments, suggestions, and improvements are welcome.

• Patch
• Examples

The ROS community has grown an amazing amount this year. As the Robots Using ROS has illustrated, there are all types of robots using ROS, from mobile manipulators, to autonomous cars, to small humanoids. As the types of robots has increased, so too has the variety of software you can use with ROS, whether it be hardware drivers, libraries like exploration, or even code for research papers. This diversity has allowed all types of developers, including researchers, software engineers, and students, to participate in this growing community.

Today we officially crossed the 1000 ROS package milestone. This is due in no small part to the many new ROS repositories that have come online this year. We are now tracking 25 separate ROS repositories that are providing open source code, including repositories from:

• CU Boulder
• UT Austin/Austin Robot Technology
• Albert-Ludwigs-Universität in Freiburg
• University of Arizona
• Georgia Tech
• Bosch
• University of Maryland
• Care-O-bot
• Brown University
• Technische Universität München
• Stanford AI Lab
• Willow Garage
• Washington University in St. Louis
• I Heart Robotics
• Carnegie Mellon University
• MIT LIS
• WPI
• ModLab at Penn
• USC
• Columbia University
• KU Leuven
• Cornell
• Tully Foote (personal)
• Andrew Harris (personal)

We're excited to see the expansion of such an amazing and vibrant ROS community. Thank you all for taking part.

We're very excited to announce that the mapping library from SRI International's Karto Robotics is now open source with an LGPL license. This mapping library contains a scan matcher, pose graph, loop detection, and occupancy grid construction -- all important building blocks for 2D navigation. When combined with Willow Garage's Sparse Pose Adjustment (SPA) for optimization (in the sba ROS package), it forms a complete stand-alone library for robust 2D mapping.

The Karto mapping library is being hosted on code.ros.org, and we've already integrated it with the ROS navigation stack. The Karto team recently benchmarked various SLAM systems on the RAWSEEDS dataset and found that newest Karto 2.0 with SPA is slightly less precise than Karto 1.1, but Karto 2.0 was more consistent and faster [1]. The maximum error with Karto 2.0 performed as well as a localization-based solution (MCL). A paper describing the SPA technique is due to be published later this year.

We'd like to thank the Karto team for all the hard work that went into making this happen. You can visit kartorobotics.com to find out more about Karto as well as contact them regarding Karto integration services.

[1]: "Comparison of indoor robot localization techniques in the absence of GPS, Vincent, Regis, Limketkai, Benson, and Eriksen, Michael, In Proceedings of SPIE Volume: 7664 Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XV (Proceedings Volume) of Defense, Security, and Sensing Symposium.

Karto SRI/Willow Garage Integration Team: Kurt Konolige, Benson Limketkai, Michael Eriksen, Regis Vincent, Brian Gerkey, Eitan Marder-Eppstein

Above: Grabcut example

OpenCV 2.1 has been released. In addition to many improvements underneath the hood, OpenCV 2.1 adds the Grabcut (C. Rother, V. Kolmogorov, and A. Blake) image segmentation algorithm. The stereo libraries have also been updated with new and improved algorithms, including H. Hirschmuller's semi-global stereo matching algorithm (SGBM).

Mac OS X users will be happy to know that OpenCV has been updated for Snow Leopard. You can now build it as a 64-bit library and highgui has been updated with new Cocoa and QTKit backends (thanks Andre Cohen and Nicolas Butko). Windows users can also build on 64-bit using MSVC 2008 or mingw64.

There are numerous other improvements with this release. We encourage users to check the change list to find out more.

On an administrative note, OpenCV has migrated from SourceForge to code.ros.org to take advantage of faster servers. The ticket tracker has also moved.

The Minoru is an inexpensive stereo webcam that can now be used with ROS. Bob Mottram recently updated the v4l2stereo library so you can now use both the library and the Minoru camera easily with ROS, as the video below demonstrates. The v4l2stereo library also integrates well with OpenCV.

You can find instructions on the Sentience site on how to easily remove the commercial packaging around the Minoru sensor, as well as instructions on how to use it with ROS.

Links:

• v4l2stereo library (GPL v3)
• Minoru/ROS instructions

When talking with people face-to-face, we may experience the "Cocktail Party Effect": even in a crowded, noisy room, we can use our binaural hearing to focus our listening on a single person speaking. With current telepresence technologies, however, we lose this important ability. Thankfully, there are already researchers giving us new tools for effectively bridging these remote distances.

Kyoto University's Professor Hiroshi Okuno and Assistant Professor Toru Takahashi, Honda Research Institute-Japan's Dr. Kazuhiro Nakadai, and four Kyoto University and Tokyo Institute of Technology graduate students spent a week at Willow Garage, integrating HARK with a Texai telepresence robot. HARK stands for Honda Research Institute-Japan (HRI-JP) Audition for Robots with Kyoto University. The robot audition system provides sound source localization, sound source separation, acoustic feature extraction, and automatic speech recognition.

The HARK system integrated well with ROS and our Texai. The Texai was outfitted with a green salad bowl helmet embedded with eight microphones, and there is now a hark package for ROS. Using this setup, their team put together three demos showing off the potential for telepresence technologies.

In the first demo, four people, including one present through a second Texai, talk over each other while the HARK-Texai separates out each voice. The second demo shows that sound is localized and that sound direction and power can be displayed in a radar chart. The final presentation puts these two demos together into a powerful new interface for the remote operator: the Texai pilot can determine where various sounds and voices are coming from, and select which sound to focus on. The HARK system then provides the pilot with the desired audio, cutting out any background noise or additional voices. Even in a crowded room, you can have a one-on-one conversation.

HARK came out of close collaboration between HRI-JP and Kyoto University, and Professor Okuno's passion to make computer/robot audition helpful for the hearing impaired. HARK is provided free and open source for research purposes and can be licensed for commercial applications.

Travis Deyle of Hizook/Healthcare Robotics Lab posted a nice overview of Robotis Dynamixel servos, including code for using the servos with gt-ros-pkg's robotis package. The Healthcare Robotics Lab uses these servos to construct their tilting Hokuyo laser rangefinder. You'll also find them in robots like Prairie Dog for their Crustcrawler arm.

Article

Logging and playback is one of the most critical features when developing software for robotics. Whether you're a hardware engineer recording diagnostic data, a researcher collecting data sets, or a developer testing algorithms, the ability to record data from a robot and play it back is crucial. ROS supports these needs with "Bag files" and tools like rosbag and rxbag.

rosbag can record data from any ROS data source into a bag file, whether it be simple text data or large sensor data, like images. The tool can also record data from programs running on multiple computers. rosbag can play back this data just as it was recorded -- it looks identical. This means that data can be recorded from a physical robot and used to create virtual robots to test software. With the appropriate bag file, you don't even need to own a physical robot.

There are a variety of tools to help you manage your stored data, such as tools for filtering data and updating data formats. There are also tools to manage research workflows for sending bag files to Amazon's Mechanical Turk for dataset labeling. For data visualization, there is the versatile rxbag tool. rxbag can scan through camera data like a movie editor, plot numerical data, and view the raw message data. You can also write plugins to define viewers for your own data types.

You can watch the video to see rosbag and rxbag in action. Both of these tools are part of the ROS Box Turtle release, which you can download from ROS.org.

In addition to core robotics libraries, like navigation, the ROS Box Turtle release also comes with a variety of tools necessary for developing robotics algorithms and applications. One of the most commonly used tools is rviz, a 3-D visualization environment that is part of the ROS Visualization stack.

Whether it's 3D point clouds, camera data, maps, robot poses, or custom visualization markers, rviz can display customizable views of this information. rviz can show you the difference between the physical world and how the robot sees it, and it can also help you create displays that show users what the robot is planning to do.

You can watch the video above for more details about what the ROS rviz tool has to offer, and you can read documentation and download the source code at: ros.org/wiki/rviz.

We're pleased to announce that all of the packages within the Brown Ros Pkg collection have been updated for ROS-1.0.X compatibility. The Brown automated install script has also been updated to reflect these changes as well. This update should fix any problems caused by using the (formerly) 0.9.0 release with an updated roscore.

We encourage anyone interested to visit brown-ros-pkg . We currently have a driver for the iRobot Create, a basic driver for the Aldebaran Nao, and some basic vision packages.

Additions and improvements include:

probe

A ROS webcam capture node new to this release. Probe leverages Gstreamer, making it compatible with almost every Linux camera and video system available. In addition Gstreamer's software video processing can be used to emulate advanced features (e.g. white-balancing) even for cameras that don't have the appropriate v4l hardware support.

teleop_twist_keyboard

A simple keyboard-based teleop interface inspired by teleop_base. It's only dependencies are on the necessary geometry messages.

As always, we're interested in the communities feedback and suggestions.

_Trevor

Andrew Harris just announced support for the Arduino and cmucam3 with ROS packages in his own ajh-ros-pkg repository. The announcement is below.*

Hello, I have released the source code to my ROS packages for communicating with the Arduino and the cmucam3.

pmad (Arduino)

You can get the code from:

https://ajh-ros-pkg.svn.sourceforge.net/svnroot/ajh-ros-pkg/trunk/pmad/

There is a subdirectory in there called "arduino" that contains a sketch.txt file. This is the sketch you must load into the GUI and upload to the Arduino. Once you do that, you can start the PMAD service:

rosrun pmad pmad_service.py

Note that you'll have to have a "tty" ROS parameter set up if the USB does not connect as /dev/ttyUSB0. You might have to set tty to "/dev/ttyUSB1" for example. Once you set up the service, you can toggle the digital pins with:

rosservice call pmad_switch_control 4 0

This for example will set digital pin 4 to LOW. Note the status packet below only returns the state of digital pins 4, ..., 7.

In addition to this, there is a service status that returns the current A/D outputs on analog pins 0, ..., 3, digital pins 4, ..., 7, and a count of the number of commands executed.

If you want to get the status to be a "topic", there is an application

rosrun pmad pmad_status_publisher.py

That will publish the status data once per second. The status message is defined in msg/Status.msg.

BTW pmad stands for power management and distribution, I use solid state relays on the outputs of the digital pins.

cmucam_png (cmucam3)

In terms of the cmucam_png node, it acquires a png file from the cmucam about once every twenty seconds and publishes it as a compressed image. To be honest I haven't used it in a while as I've switched to a firewire USB camera. The image is a PNG file. It takes ~20 seconds per image because they come over a 115,200 bps serial line. They are 352x287 pixels in size.

https://ajh-ros-pkg.svn.sourceforge.net/svnroot/ajh-ros-pkg/trunk/cmucam_png/

Right now for some reason I get compiler errors compiling image_view so I can't try the image_view to make sure the image is getting pushed around. However I have recently tested that I can receive an image from the cmucam and save it to disk as a png by just adding a couple lines to save the received png to disk. I am just having trouble pushing the image around ROS, but it might just be my setup. (It worked in the past ;) )

For the cmucam application, there is also a binary that must be downloaded to the camera. The source, etc, for this is in the directory "png-robin". In order to get this working you should have a working installation of the cmucam3 development environment, and be able to compile the examples in the cc3/projects directory. Once you can do this, and download these examples to the camera, you can make a new directory in that examples directory called png-robin and copy the contents of this directory into that one. The makefile will then work and compile the application. I have also checked in a hex file that you can use if you don't want to compile the application yourself. But you'll still have to download the hex file with lpc21isp.

Hopefully I have made the packages correctly, but these are the first ones I've tried to make. If you have any questions on either of these nodes, of if they don't seem to work, let me know!

thanks, -andrew

crossposted from willowgarage.com

Peter Pastor, a PhD student at USC, spent the past three months developing software that allows the PR2 to learn new motor skills from human demonstration. In particular, the robot learned how to grasp, pour, and place beverage containers after just a single demonstration. Peter focused on tasks like turning a door handle or grasping a cup -- tasks that personal robots like PR2 will perform over and over again. Instead of requiring new trajectory planning each time a common task is encountered, the presented approach enables the robot to build up a library of movements that can be used to execute these common goals.  For this library to be useful, learned movements must be generalizable to new goal poses. In real life, the robot will never face the exact same situation twice. Therefore, the learned movements must be encoded in such a way that they can be adapted to different start and goal positions.

Peter used Dynamic Movement Primitives (DMPs), which allow the robot to encode movement plans. The parameters of these DMPs can be learned efficiently from a single demonstration, allowing a user to teach the PR2 new movements within seconds. Thus, the presented imitation learning set-up allows a user to teach discrete movements, like a grasping, placing, and releasing movement, and then apply these motions to manipulate several objects on a table. This obviates the need to plan a new trajectory every single time a motion is reused. Furthermore, the DMPs allow the robot to complete its task even when the goal is changed on-the-fly.

You can find out more about the open source code for Peter's work here, and check out his presentation slides below (download PDF). For more about Peter's research with DMPs and learning from demonstration, see "Learning and Generalization of Motor Skills by Learning from Demonstration", ICRA 2009.

Update: Documentation is available on ros.org at http://www.ros.org/wiki/alufr-ros-pkg

I'm happy to announce v0.1, the first proper release of Freiburg's Nao Stack located at: http://code.google.com/p/alufr-ros-pkg/

You can check out the trunk (svn) from:
http://alufr-ros-pkg.googlecode.com/svn/trunk/

or download the stack package from:
http://code.google.com/p/alufr-ros-pkg/downloads/list

Changes are:

• improved torso odometry
• nao_ctrl now also transmits the state of Nao's onboard IMU
• new nao_description package makes Nao's complete joint state, transformations and visualization available through robot_state_publisher
• launch files for convenient start up
  

Some basic instructions are available at http://code.google.com/p/alufr-ros-pkg/, but I might also move them to ros.org if that's the more appropriate place. I would be happy to hear if all is working for you (or not), or how to make things more ROS-compliant.

Best regards, Armin

crossposted from willowgarage.com

Ethan Dreyfuss, who recently received a master's degree from Stanford University, is continuing his work here on autonomous person-following and dataset collection and annotation. The former project provides a useful building block for a wide variety of tasks. Consider a robot that helps you carry groceries. This robot is vastly more useful if it can carry your bags to the house without requiring teleoperation; the robot can simply track you and follow behind. At a high level, person-following comprises two principal tasks: person tracking and navigation.

The approach developed by Ethan and Caroline Pantofaru fuses a face detector with two weak person trackers: one for legs, and one for 3D blobs at person-height. None of these approaches is individually effective enough to provide robust tracking, but their strengths are complementary. The face detector is effective when the person is close to, and directly facing the robot. While the leg tracker provides high accuracy when multiple people are present, it is often confused by non-human obstacles and can therefore not work reliably from afar. Conversely, the height-based blob tracker can effectively track from further away, yet it is easily confused by groups of people. By combining techniques, Ethan and Caroline were able to develop a more robust person-tracking tool.

Once the robot can track a designated person, the information is passed on to the navigation stack. This same navigation software was used to complete Milestone 2, with some improvements made to help deal more quickly and robustly with dynamically-moving obstacles such as people.

In addition to the person-following project, Ethan is contributing to the collection and labeling of a large dataset of people in an indoor office environment. One of the major drivers of computer vision research is the availability of high-quality labeled data. The bulk of existing person datasets exclude indoor environments, and instead focus on outdoor pedestrians. Indoor environments present numerous challenges for person detection, including poor lighting and environmental clutter. By automating as much as possible, the process of both collecting (using the robot) and labeling (using Amazon's Mechanical Turk and Alex Sorokin's CV Web Annotation Toolkit), Ethan's team will be able to provide a large, compelling dataset to encourage other researchers to tackle these challenging problems.

Ethan also picked up a number of side projects including rapid neighborhood computation on point clouds, and implementing a package that uses the open-source video codec Theora to allow low-bandwidth video streaming within ROS.

Fast on the heels of the Brown Nao driver, Armin Hornung of Albert-Ludwigs-Unversität Freiburg has announced joy-package compatibility and torso odometry additions for the Nao driver -- as well as the alufr-ros-pkg repository. Armin's announcement is below.

Based on the recently announced Nao driver by Brown University, there is now a regular joystick teleoperation node available. It operates Nao using messages from the "joy" topic, so it should work with any gamepad or joystick in ROS. In addition, the control node running on Nao returns some basic torso odometry estimate. The code is available at http://code.google.com/p/alufr-ros-pkg/ to checkout via SVN. A README with more details can be found in the "nao" stack there.

I'm open for feedback and suggestions!

-- Armin

Brown University has been hard at work on a developing an Aldebaran Nao driver for ROS and recently announced an open source (GPL) version to the community. The text of Trevor Jay's announcement is below.

Brown University is pleased to release a ROS driver for the Aldebaran Nao. The driver makes available: head control, text-to-speech, basic navigation, and (most interestingly) the Nao's forehead camera. Sample clients are part of the download, including a WiiMote teleop client.

Here are two videos of the driver in action (one from the robot's perspective):

The driver is available at the brown-ros-pkg download page.

If you have any problems or just find the driver useful, please let us know! We will add features as our work and the community need them.

_Trevor

crossposted from willowgarage.com

Nate Koenig of the Interaction Lab at USC is continuing his work here at Willow Garage after a busy summer. Nate carried out an empirical study investigating the use of people as teachers for robots, while also researching learning by demonstration with PR2.

Participants in Nate's study used learning by demonstration to teach PR2 how to solve the Towers of Hanoi puzzle. As the name implies, learning by demonstration relies on a human teacher to provide a robot "student" with demonstrations of a complex task. In this case, the robot uses the state of the puzzle (i.e., location of red disk compared to blue and green), along with the teacher's command, to learn the demonstrated task. Volunteers used a web-based teaching tool to guide PR2 through the three-disk puzzle board. In one condition, teachers were able to directly see PR2, while in the other condition teachers viewed the robot's actions through a small video feed on the web tool. This manipulation allowed Nate to study if robot visibility affects teaching strategies and outcomes. Based on participants' commands and other observations from the environment, the robot learned how to solve the puzzle on its own.

Results from this study indicate that teachers perform better when visually separated from the robot. Performing "better" means that participants made fewer unnecessary or repetitive moves when teaching the robot. While this may seem counter-intuitive, the teachers who could see the robot were easily distracted from the task and seemed to build inaccurate mental models of PR2's capabilities.

In addition to this experiment, Nate worked on a number of smaller projects including developing the first video streaming ROS node and creating a web-based graphical interface for interacting with PR2. You can find many of these contributions in the hanoi package for ROS. Nate is also creator and lead developer for Gazebo, a popular open-source 3D robot simulator. Gazebo is heavily used at Willow Garage to simulate the actions of PR2, and Nate is providing us with numerous improvements.

On the Willow Garage Blog, you can find out more about the Texas Robot, which is a telepresence robot built out of leftover PR2 parts and off-the-shelf components. One of the interesting aspects of the Texas robot is that it is running the PR2 software components as-is: motor controllers, tele-operation, visualization, etc... The only changes required were modifying the robot model (urdf) and writing new roslaunch files. You can find these changes in the texas package on ROS.org. Any other changes have been improvements that have been useful for both the Texas and PR2 robots, such as updates to the teleoperation controls.

crossposted from willowgarage.com

This summer, Jason Wolfe (UC Berkeley) worked on hierarchical task planning for the PR2. Planning is important for performing tasks efficiently. If you're gathering ingredients for the Triple Chocolate Volcanic Explosion Soufflé you're fiendishly craving, you may fetch both the eggs and the butter from the fridge before heading to the pantry for the sugar and chocolate. If you weren't planning ahead, you might go get the eggs, drop them off on the counter, go back to the fridge to get the butter, unload again, continue to the pantry for the chocolate, and so on. You'll still make a soufflé, but your plan will be far from optimal.

Jason's work on hierarchical planning helped the PR2 plan out its tasks in a more logical, premeditated manner. The package takes into consideration high-level decisions, like the order in which to fetch the ingredients, down to low-level decisions like where to position the base of the robot and what angle to grasp an object from. In the video, the PR2 is told to move several bottles between two tables. Instead of manipulating each bottle independently of the others, the robot plans ahead and places the bottle being manipulated near the next bottle to grab. That way, the robot can simply put down the current item and, without driving to the next goal, reach over and pick up its next bottle. Jason's code can be found in the hierarchical_planning package. You may also be interested in Jason's and Bhaskara Marthi's RSS 2009 presentation on Angelic Hierarchical Planning.

In the process of working on this package, Jason spent time developing and debugging lower-level component actions, such as moving the base to a particular location. Interactive scripting languages are ideal for this purpose. Jason developed an experimental Java client library for ROS (rosjava), which he then used to create a ROS interface to Clojure (rosclj), a Lisp-like language built on Java. Using this interface, he then developed a large library of scripted actions which can be used for quickly specifying and tele-operating complex sequences. These can be found in the clj_pr2 package.

crossposted from willowgarage.com

Ben Cohen of University of Pennsylvania has returned to the GRASP Lab after his summer internship here at Willow Garage. At Penn, Ben researches search-based methods for path planning for robotic manipulators. During his time here, he worked on two motion planners: one for door opening and one for manipulation. When compared to the door planner used in Milestone 2, the new door planner uses SBPL (Search-Based Planning Library) to give the PR2 two new capabilities. First, it allows the robot to not only push doors open, but also pull. Second, the door planner allows the robot to open doors, regardless of hinge position -- left or right side of the door. These two novel capabilities allow for robust, more universal door opening.

Additionally, Ben's work on a manipulation planner involved integrating SBPL into the move_arm ROS package, which integrates a variety of motion planners. Ben tested the SBPL planner on the PR2's arms, and added the supporting software needed to perform collision checking. With collision checking in place, SBPL can more readily handle cluttered, complex environments.

Here are Ben's end-of-summer presentation slides discussing his planning work (Download PDF from ROS.org):

crossposted from willowgarage.com

This summer, Alex Teichman of Stanford University worked on an image descriptor library, and used this library to develop a new method for people to interact directly with the PR2. Using a keyboard or a joystick is great for directly controlling a robot, but what if an autonomous robot wanders into a group of people -- how can they affect its behavior?

Alex's approach allows the PR2 to "talk to the hand," as many of us cruelly experienced in middle school. In the video, Alex demonstrates that the PR2 can be made to stop and go by simply holding a hand up to the stereo camera. To accomplish this, Alex used a machine-learning algorithm called Boosting along with image descriptors, such as the color of local regions and object edges. He was able to train his algorithm on a data set that was labelled using the Amazon Mechanical Turk library that Alex Sorokin developed. This Mechanical Turk library harnesses the power of paid volunteers on the Internet to perform tasks, like identifying hands in images so that algorithms like Boosting can be trained.

Hand detection is part of a larger effort that Alex Teichman has been working on, developing a library with a common interface to image descriptors. This library, descriptors_2d, enables ROS developers to use easily use descriptors like SURF, HOG, Daisy, and Superpixel Color Histogram.

You can learn more about the hand detection techniques and image descriptors in Alex's final summer presentation (download PDF from ROS.org).

crossposted from willowgarage.com

Matt Piccoli (University of Pennsylvania) and Jürgen Sturm (University of Freiburg) did a lot more than break eggs and learn about how things move. They also tested the limits of PR2's fingertip pressure sensors with a seemingly simple task: identifying, without looking, if a juice bottle is open or closed, and full or empty.

Tactile information is invaluable when determining properties of objects that are visually inaccessible. In this vein, Matt and Jürgen developed a tactile perception strategy that can be used to detect the internal state of liquid containers. By measuring a bottle's reaction to a force applied by a gripper, their system can recognize whether the bottle is full or empty, and open or closed. The system learned this information from a set of training experiments carried out on different types of bottles and soda cans. Knowing whether a bottle is open or closed can help a robot determine the level of care required when manipulating the object.

You can find the code for Matt and Jürgen's work in the pr2_gripper_controller package on ROS.org.

crossposted from willowgarage.com

Marius Muja of University of British Columbia began his internship in the middle of Milestone 2 excitement. For several weeks, he worked on two important perception components: detecting outlets from far away, and detecting door handles.

Thereafter, Marius focused on tabletop object detection and wrote the tabletop_objects package. Determining the exact position and orientation of an object, as well as its identity, is very important if a robot is grasping objects, and especially crucial if the object in question is fragile. Tabletop_objects uses a two-stage approach. In the bottom-up stage, initial estimations of possible object locations are made, and in the top-down stage, 3D models are fit into the estimated locations. After fitting the correct 3D model, the object's identity, position and orientation can be determined with high confidence. This approach can even distinguish between similar-looking drinking glasses. Marius worked with Ioan Sucan to integrate tabletop_objects and motion planning (move_arm), and together, they were able to successfully detect, grasp and manipulate fragile glass objects.

In addition to his work with tabletop_objects, Marius integrated FLANN (Fast Library for Nearest Neighbors) into OpenCV, and developed a phone-based teleoperation mode for PR2 based on Asterisk, an open source PBX.

Here are Marius's end-of-summer slides, where you can find more details about his work.

Marius Muja: Tabletop Object Detection (Download PDF from ROS.org)

crossposted from willowgarage.com

This summer, Dan Munoz of Carnegie Mellon University worked on helping the PR2 understand its environment using its 3-D sensors. Improving 3-D perception is important because it can help the PR2 with many tasks such as localization and object grasping. At CMU, Dan and collaborators are developing techniques to improve 3-D perception for an unmanned vehicle in outdoor natural and urban environments. These techniques first take in a cloud of 3-D points, usually collected from a laser scanner, and a label associated with each point. These labels identify such objects as buildings, tree trunks, plants, power lines, and the ground. Then, various local and more global features that describe the local shape and distribution of each object are extracted for each point and region of points. These labeled examples are then used to train an advanced machine learning tool that reasons the best way to combine the local and global features that describe each object. In new environments, this feature extraction process is repeated and given to the machine learning tool to determine what objects are present in the novel scene.

While at Willow Garage, Dan integrated this learning framework into ROS. As shown in the video, Dan experimented with helping the PR2 perceive objects on the room-sized scale, such as tables and chairs, as well as objects at the table-top-sized scale, including mugs and staplers. During the Intern Challenge, Dan also applied this same framework to distinguish between the three different types of bottles being served: Odwalla, Naked, and water. Dan developed the descriptors_3d package, the library used to compute various 3-D features for a point or region of points from a stereo camera or laser scanner. Additionally, he developed the functional_m3n package (Functional Max-Margin Markov Networks), the advanced machine learning tool that learns how to combine low-level and high-level feature information for each object.

Below are Dan's final presentation slides.

Daniel Munoz: Indoor Object Detection on Scribd (Download PDF from ROS.org)

cross posted from willowgarage.com

Mrinal Kalakrishnan, one of three motion planning interns here at Willow Garage, is finishing up his summer project and returning to the University of Southern California. Mrinal has been working on a smooth motion planning and control pipeline for the PR2, introducing a new approach to object manipulation The key component of this work was the implementation of CHOMP (Covariant Hamiltonian Optimization and Motion Planning), a motion planner developed at CMU and Intel Research. You can find this implementation in the chomp_motion_planner package for ROS.

Mrinal chose to implement this motion planner on the PR2 because CHOMP's method of planning away from obstacles produces very smooth, natural-looking movements. You can see in the video that the PR2's arm trajectory is rather fluid and avoids unusual or awkward joint angles. The animation shows the arm optimizing the trajectory away from the bookshelf, while maintaining a smooth motion plan. Mrinal's work with CHOMP allowed for informative comparisons to be made with the two other motion planners being researched and implemented here, ompl and sbpl_arm_planner. All three motion planners use the same interface, making switches between the three systems very simple.

In addition to his work with CHOMP, Mrinal wrote the distance_field package for ROS which performs 3-D obstacle inflation to generate a cost-map for arm planners. He also wrote spline_smoother, a library of algorithms which can convert a set of waypoints, as typically generated by motion planners, into a smooth spline trajectory suitable for execution on a robot.

Below is Mrinal's end-of-summer presentation, where you can find additional details about his work here at Willow Garage.

Mrinal Kalakrishnan: CHOMP Motion Planner on Scribd (Download PDF from ROS.org)

crossposted from willowgarage.com

Min Sun is returning to University of Michigan at Ann Arbor, where he does computer vision research with particular interest in 3-D object recognition. During his summer here at Willow Garage, he focused on recognizing table-top objects like mice, mugs, and staplers in an office environment. Min is the the primary creator of rf_detector (Random Forest), which recognizes objects and their poses. The detector uses the stereo camera along with the texture light projector to collect images and the corresponding dense stereo point clouds. From there, rf_detector predicts the object type (i.e. mouse, mug, stapler) and its location and orientation. This information can be crucial to have before attempting object manipulation, as many object types, such as mugs, require careful handling.

In the future, Min will be looking for ways to scale up this approach to a wider range of object classes. Min continues to look for other features and model representations that make object recognition more robust.

Min also wrote the geometric_blur package, which calculates geometric blur descriptors.

Here are the slides from Min's final internship presentation describing his work on rf_detector and the detection pipeline.

Min Sun: 3D Object Detection on Scribd (Download PDF from ROS.org)

crossposted from willowgarage.com

Ioan Sucan is headed back to Rice University after his third stay here at Willow Garage. Ioan is a motion planning researcher and is the author of Open Motion Planning Library (OMPL), a library of sampling-based motion planning algorithms. These algorithms are important for the PR2 because they enable the arm to grasp and manipulate objects, while simultaneously avoiding collisions with people and other still or moving objects.

This past winter, Ioan and Radu Rusu used OMPL to do dynamic collision avoidance. This summer, Ioan was able to make many improvements to OMPL so that the PR2 can grasp and manipulate objects in cluttered indoor environments. In the video, you can see how the PR2 is able to grasp objects while moving its arm through complex obstacle courses. Data from the tilting laser scanner is used to construct and update a 3D-representation of the environment, allowing the arm to avoid even moving obstacles.

Ioan also improved the robot_self_filter and the collision_map. When the PR2's arm moves in front of the sensors, two problems occur: the arm looks like an obstacle in the environment, and the arm blocks the robot's view of the environment behind the arm. The robot_self_filter, in combination with the collision_map, allows the PR2 to disregard its arm as an obstacle, and "remember" objects behind the arm.

Here are the slides from Ioan's end-of-summer presentation describing his work on OMPL and other projects.

Ioan Sucan: Motion Planning for the PR2 Arm on Scribd (Download PDF from ROS.org)

crossposted from willowgarage.com

Alex Sorokin of University of Illinois, Urbana-Champaign has been hacking on many projects this summer, extending work that he started at Willow Garage last year. Alex is the author of the CV Web Annotation Toolkit, which is an open source tool that helps researchers in computer perception collect and classify data, with the help of workers on Amazon's Mechanical Turk. If you're a researcher collecting a data set, you can use this toolkit to submit images to Mechanical Turk and pay people to label them for you. For example, you can instruct Mechanical Turk users to draw boxes around all of the people in an image, or draw polygons around people's hands. These manually-labeled datasets allow researchers to test their own algorithms and compare them against what a person sees. If you're worried about the quality of results that random Turkers might give you, you can easily let an additional set of users grade the results that you get back, and increase the likelihood of accurate responses.

In addition to helping our researchers collect data sets, this toolkit is also very useful for our robots. For just a couple of dollars, you can easily submit images to Mechanical Turk and have users teach the robot where your refrigerator and dishwasher are, what your cups look like, or where your power outlets are. We're a long way from having robots that can identify objects in an environment as well as humans can, but toolkits like Alex's help us to bridge that gap by allowing humans to lend their abilities to robots.

In addition to his work with Mechanical Turk, Alex came up with many great extensions to ROS. These include bagserver, which enables random-access into ROS bag files, and bag_image_view, which lets you visually scan through images in a bag file and play them back.

Below is Alex's end-of-summer presentation, which discusses his various contributions in greater detail.

Alex Sorokin: CV/Mechanical Turk Presentation (Download PDF from ros.org)

University of Freiburg researcher Jürgen Sturm is just finishing his second internship here at Willow Garage. In addition to helping out with our egg-breaking efforts, he's been using the stereo cameras of PR2 to detect planar objects like doors, drawers, books, and boxes. More importantly, he has been tracking the movement of these objects to learn articulation models, i.e. how these objects move. Does the door open to the left or the right? Does the drawer slide in or out? Where will the handle of the drawer be when it is fully open? This is the sort of information that is critical for enabling robots to operate in our own environments.

We've posted a video of Jürgen discussing his findings as well as slides from a presentation that he gave at Willow Garage. You can also download his code from the planar_objects ROS package.

Planar Objects and Articulation Models on Scribd (Download PDF from ros.org)

Awhile back we showed a demo video using an iPod Touch to drive around of a PR2. It was a fun experiment, but it relies on a proof-of-concept that's not quite ready for primetime.

Srećko Jurić-Kavelj of the University of Zagreb showed us that there's more than one way to get data from an iPod Touch into ROS. Instead of cross-compiling ROS onto the iPhone, which is still very difficult, he used the open-source accelerometer-simulator project on Google Code to receive the accelerometer on another computer. He was then able to easily adapt a sample Python script to broadcast that data as a rospy node.

You can see the results here as he drives around a Pioneer 3DX:

Integrating ROS with other robot software frameworks, such as OpenRAVE, Player, and Euslisp, has been important to us as it allows developers to leverage the strengths of each. No framework can be the best at everything, so it's important to allow developers to choose the tools they need. This video shows the integration between ROS and the Orocos Real Time Toolkit (RTT), which was done by Orocos developer Ruben Smits during a month-long visit to Willow Garage. Ruben became a vital member of our Milestone 2 team and still managed to have time to build this great, seamless integration.

While ROS has good support for distributing components over the network, RTT offers hard realtime communication between components. This demo combines the best of both worlds: the realtime low level control is handled by a set of RTT components that directly control the the PR2 robot hardware, but the communication between the controllers is visible to the whole ROS network.

The integration makes the RTT components appear as ROS components, without breaking the realtime communication between the RTT components. This allows RTT developers to use tools developed for ROS, such as rviz and rxplot, to visualize the communication between controllers.

-- Wim Meeusen

Josh Faust and Rob Wheeler recently got ROS working on the iPod Touch/iPhone and put together a quick demo that uses an iPod Touch as a joystick for the PR2 robot. The iPod Touch and iPhone combine a high-quality display with different modes of interaction that make it appealing for robotics interfaces, and they are a platform to test cross-compilation of ROS. The difficult challenge in getting ROS running on the iPod Touch was solving the cross-compilation issues. Once they had those figured out, they were able to add about twenty lines of code to the standard iPod Touch accelerator demo to translate the accelerator input into commands to drive the PR2.

This is still a proof of concept, but we hope in the coming months to make it a stable platform for ROS development. We've put up a ROSPod wiki page so you can track our efforts and contribute your own.