The mobile manipulator TIAGo robot and Project NeuTouch
For tasks such as object manipulation in robotics, tactile sensation is important for performance, and Deep Reinforcement Learning (DRL) also allows robots to learn the complex behavior that may be needed. PAL Robotics’ PhD student Luca Lach recently worked on research that enabled the TIAGo service robot to learn new manipulation tasks within a simulated reinforcement learning environment, with tactile sensors that resemble a real sensorised gripper.
Reinforcement learning is an area of Machine Learning, in which artificial intelligence (AI) faces an environment where there is a reward for taking a particular action. The idea of reinforcement learning is that there are many possible outputs or actions. The model learns and decides the best output or action based on the reward. With this type of approach, the aim is for the AI to try to learn behaviour rather than working with data sets.
Furthermore, for AI to learn more about a particular task via reinforcement learning, simulation environments are helpful. Simulation environments provide the ideal environment for the AI to learn new tasks, before transferring the tasks to a real environment, known as sim to real transfer. This is particularly useful if the behavior being learned is complex.
We had the chance to talk to Luca, PhD student at PAL Robotics, and part of the NeuTouch project, to hear more about the research. NeuTouch, which PAL Robotics is a partner in, is training 15 PhD students on cutting-edge touch-related research including artificial systems design and robotics.
Tactile sensation for robot manipulation in structured and unstructured environments – including homes
In the research paper ‘TIAGo RL: Simulated Reinforcement Learning Environments with Tactile Data for Mobile Robots’ written together with Francesco Ferro (PAL Robotics) and Robert Haschke (Bielefeld University), Luca Lach created a simulation environment for PAL Robotics’ robot TIAGo in order to learn the behaviour needed for more advanced manipulation tasks.
Luca told us, “tactile sensation is very important for humans, and we use it to understand objects and regulate force needed to grasp them. We are also able to detect things like object slippage.”
“In robotic manipulation, tactile sensors became popular, however equally there are many sensors without tactile sensation (a sense of touch). These types of sensors work well in structured environments, however, in unstructured environments such as homes, this brings more of a challenge.”
Creating a simulation environment for TIAGo robot to learn the behaviour needed for new manipulation tasks
Luca continued, “tactile sensors in mobile robots mean the robot behaviour becomes more complex. In recent years, Deep Reinforcement Learning (DRL) has been shown to be a powerful tool for robots to learn complex behaviours. This paper is about research we worked on within a tactile mobile robotics environment for reinforcement learning.”
“This is the first version of a simulation environment we have created for the (PAL Robotics’) TIAGo robot. When working with manipulation tasks, it’s very useful to work in an environment where you’ll get feedback – tactile feedback whether to grasp harder or whether there is enough force for example.”
“The difference in this simulation, is that non-sensorised fingers are replaced by load cell sensors, that provide tactile feedback (contact force).”
“Our simulation environments modeled a grasping scenario where the robot is located next to the table with an object. Here we spent some time making the simulation and modelled PAL Robotics’ TIAGo platform with sensorised end effectors that are also available on the real robot.”
Training AI in simulation environments before sim to real transfer on a robot
Luca went on to explain, “Reinforcement learning is data-hungry, reinforcement learning needs many iterations and is also time-consuming on a real robot. The goal is to train Artificial Intelligence (AI) in simulation environments, with the future aim of running this on the real robot (TIAGo in this case), in sim to real transfer.”
“In this research, an agent has shown the ability to learn to reach and maintain a certain goal force in our environments. In future works, we plan to investigate further force control agents in simulation with research transfer learning capabilities.”
Developments on TIAGo robot as a result of this research in simulated Reinforcement Learning Environments
Luca told us, “in the scope of this project, we have sensorised end effectors integrated on TIAGo, that we have also successfully simulated. Using these sensors, we are able to realise more robust manipulation behaviour.” You can find the current software package on Reinforcement Learning Environments for TIAGo on Github here.
Finally, Luca told us, “additionally, we are developing a second gripper that has a sensor matrix instead of a single sensor. These matrices have the advantage of providing more information about an object’s shape and texture.”
Thank you to Luca Lach for taking the time to talk with us about this research.
You can read the full research paper ‘TIAGo RL: Simulated Reinforcement Learning Environments with Tactile Data for Mobile Robots‘.
To learn more about PAL Robotics and TIAGo, visit our website. For any questions, don’t hesitate to contact us.
This research was compiled as part of the EU-Funded project NeuTouch. NeuTouch aims at improving artificial tactile systems, by training researchers that study how human and animal tactile systems work, and developing a new type of technology that is based on the same principles. The aim is for this technology to be used for building robots that can help humans in daily tasks and for artificial limbs that can give the user the sensation of real touch. The project includes the TIAGo robot. Read more about NeuTouch in our previous post about Project NeuTouch and the development of tactile sensation.
TIAGo the mobile manipulator robot combines perception, navigation, manipulation & Human-Robot Interaction skills out of the box. TIAGo robot has a modular design and is customizable based on our customers’ needs. The end effector can be a gripper or a humanoid hand and they can be quickly exchanged for performing various manipulation tasks. Find out more about the different end-effectors currently available here.