Project CANOPIES: TIAGo++ starts work in an Italian vineyard

PAL Robotics is part of the EU project CANOPIES: with the aim that farm workers work together with teams of robots to perform harvesting and pruning in table-grape vineyards. Recently tests have been held in an agricultural cooperative named “Cooperativa Agricola Corsira”, a community of roughly 20 small producers in Aprilia, Lazio, Italy, with an adapted version of PAL Robotics’ TIAGo ++ robot.  PAL Robotics is manufacturing a new upper-body dual arm agronomic system and adapted agronomic end-effectors for the CANOPIES prototype, and recently we integrated the components for the first time for testing. 

CANOPIES is the first attempt to introduce a collaborative prototype in precision agriculture for permanent crops. The overall goal of the EU research project CANOPIES project is to develop a Human-Robot prototype in crop farming (Agri-Food) that addresses the challenges of Human-Robot Interaction and Human-Robot Collaboration. 

The challenge here is that using robotics for harvesting and pruning requires the development of complex robotic processes of perception, communication, shared planning in agreement, prediction of human intentions, interaction, and action.

Customization of the TIAGo ++ robot upper-body design for work in the vineyard 

PAL Robotics’ main role in this project is to provide the upper body of the agricultural robot demonstrator, formed from the TIAGo++ dual-armed robot, which has been placed on top of the Alitrack base, provided by The Sapienza University of Rome. 

For TIAGo++ to be able to perform motions such as harvesting and pruning in the vineyard, alterations to the original TIAGo++ robot design were done in order to increase the agricultural robots’ capabilities. The alterations include having a torque sensor in each joint of the arm, adding two DOFs (Degrees of Freedom) in the base of the torso for rotation and height, and improving the IP rating. 

The agronomically adapted dual arm design is also accompanied by custom CANOPIES gripper designs that will allow the execution of the harvesting and pruning tasks. In addition, these alterations will allow CANOPIES to operate the robot in compliant control, making it more suitable for human workers to share their workspace with the robot. 

This adapted agricultural robot design for CANOPIES will prevent exerting any strong force against obstacles or people, which is also very useful to other fields of application, such as healthcare, retail, and agile manufacturing.

The CANOPIES arm updates include these:

  • New kinematics
  • Compliant arm
  • Torque-sensing in every joint
  • High torque density of the new modules
  • Brakes system in the joints for safety
  • EtherCAT communication for high bandwidth control
  • Velocity control interface
  • Cables for cameras on the end-effector.

EU project CANOPIES

CANOPIES project pilots including testing the agricultural robot performance in darkness and sunlight 

Recent testing and data collection in the project pilot vineyard in Lazio was focused on robot integration and validation on the field. The field validations enable further testing of the CANOPIES robot prototype in complex and unstructured environments, including in the presence of human workers. 

The goal of the field validation has been to collect data for navigation and to aid the perception of the adapted robot. With the tests, we have been able to observe the robot’s behaviour  – checking how the capabilities of the agricultural robot are to perceive and respond to grapes. 

In addition, future work will be done to improve the agricultural robot’s perceptual algorithms used to extend the robotic platform performances outdoors where sunlight, shadows, and partial occlusions can make it difficult to work well. The study of the use of extra devices like a light to work during the night or in dark conditions, a wrist camera for fine manipulation, and so on will also be conducted, to improve the execution performances for precise agriculture tasks.

Finally, further work will also involve the integration of the perception system with the manipulation and navigation systems to automate grapes harvesting. Later together with Human-Robot Interaction and Human-Robot Collaboration, the aim will be to automate the whole harvesting process.

CANOPIES objectives including table-grape vineyard case-study implementation

Objectives for Human-Robot Interaction

  • Objective 1.1 – Safe Human Robot Interaction in shared outdoor workspaces both with and without physical interaction.
  • Objective 1.2 – Predicting future positions of human torso and arms in motion.
  • Objective 1.3 – Efficient communication between robots and humans.
  • Objective 1.4 – Enable both parties to be fully aware of each other intentions.

Objectives for Human-Robot and Multi-Robot Collaboration

  • Objective 2.1 – Human-like bimanual robot manipulation leading to intuitive HRC.
  • Objective 2.2 – Successful robot programming using learning by demonstration.
  • Objective 2.3 – Learning from unexpected events with human-in-the-loop.
  • Objective 2.4 –Multi-robot coordination for collaborative tasks.

Objectives for PA Integrated System

  • Objective 3.1 – Collaborative robots capable of carrying out the required tasks safely.
  • Objective 3.2 – Simulated farming environments for algorithm prototyping.
  • Objective 3.3 – Table-Grape vineyard case-study implementation.

Project CANOPIES made the following video on grape and peduncle detection:

Project partners in the CANOPIES project 

The project partners in the EU-funded research project CANOPIES are:

  • Universita Roma Tre 
  • KTH Royal Institute of Technology
  • Sapienza Università di Roma 
  • Universitat Politècnica de Catalunya 
  • Le Unicitta/ UNICLAM 
  • Danish Technological Institute
  • PaleBlue
  • Agrimessina
  • RSA
  • PAL Robotics 

PAL Robotics’ TIAGo++ robot: the robot that adapts to your research needs and not the other way around

PAL Robotics TIAGo ++ robot, an adapted version of which is taking part in this project combines perception, navigation, manipulation & Human-Robot Interaction skills out of the box. The robot is well suited for applications in light industry, specifically for its manipulation capabilities and modular design, as demonstrated by the numerous European Projects that choose the robot as a research platform. Read more about TIAGo robot and its application in EU projects SeCoIIA and Cyberfactory No. 1 in our blog The mobile manipular TIAGo robot and the factory of the future.

Find out more about the project CANOPIES by visiting collaborative projects on our website. If you would like to talk more about collaborative projects and ways to work together, do not hesitate to get in touch with us.