Hi, this is José Rodríguez, a master’s graduate from the UPC and support member of Reem@IRI team!. In this blog I will briefly explain the project I developed for my master’s thesis which consisted in providing REEM with door opening skills, allowing it to identify and traverse doors within domestic or office-like scenarios. This project represents a small contribution to the efforts put together by IRI, PAL Robotics and AESS to provide REEM with the most advanced capabilities among service robots!
In general, the developed system relies on the use of an RGBD camera for environment perception and in REEM’s built in modules for navigation and environment interaction. Data is then processed in two main parts:
-A recognition module where 2D and 3D data from the camera is accessed at different stages to detect a door on a frame-by-frame basis.
-A logics module that coordinates the results from the recognition module with a sequence of actions required to deal with doors at different aperture conditions, including some recovery behaviors in case one of the actions fails to be accomplished.
For the recognition module the first step is to find a door with the appropriate size for the robot to go through. In order to do this, the image is divided into regions by means of color segmentation an each one of the resulting regions is scanned to check its actual dimensions at different image levels both horizontally and vertically. The obtained limits are also used to infer other geometrical properties of the door such as aspect ratio and perspective deformation, which are considered later to select the best match from a number of possible candidates.
Once a candidate has been selected, a second detection stage provided with the expected handle location and size performs a final likelihood comparison and outputs the handle coordinates if the handle was found, or continues with the next candidate if the search was unsuccessful.
A similar region-scan principle is applied to detect open doors, with the difference that instead of color regions, depth discontinuities are used to infer object’s boundaries, and the handle detection stage is replaced by a door panel detection stage that estimates the opening side of the door based on the angle differences found at both sides of each candidate.
In both cases, the final candidate represents a 3D region of interest from which the orientation can be robustly estimated by model fitting methods.
With the information obtained from the recognition stage, the robot is able to know how to approach the door and decide which arm to use for the opening task. It is at this point that REEM’s navigation system is requested to approach to the door and the inverse kinematics modules receive a specific motion request based on the door status. During tests, the proposed set of actions is successfully employed to detect and open a door in a fully simulated environment provided by PAL Robotics.
Further testing and improvements for reliable operation will be carried out with the actual REEM platform and the final skill will be available to see in the upcoming Robocup events. I am sure the team will have an outstanding performance at the competition!
Finally I would like to thank all of the members of Reem@IRI for their great support and enthusiasm during this project, which was the true factor that made any accomplishment possible.
Cheers,
José
For more information on new skills check out also the previous post from @David Martinez on surface cleaning!