A Smart Workcell for Human-Robot Cooperative
Assembly of Carbon Fiber Parts
Stefano Ghidoni, Matteo Terreran, Daniele Evangelista

Christian Eitzinger, Sebastian Zambal

Dept. of Information Engineering
University of Padova
Padova, Italy{stefano.ghidoni,matteo.terreran,daniele.evangelista}@unipd.it

Profactor GmbHSteyr, Austria{christian.eitzinger,sebastian.zambal}@profactor.at

Enrico Villagrossi, Nicola Pedrocchi
STIIMA-CNR
CNR
Milan, Italy{enrico.villagrossi,
nicola.pedrocchi}@stiima.cnr.it

Nicola Castaman

Marcin Malecha

Zentrum für Leichtbauproduktionstechnologie
IT+Robotics srl
Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)
Padova, Italy
Augsburg, Germany
nicola.castaman@it-robotics.itmarcin.malecha@dlr.de

Abstract—The production of carbon fiber parts is currently
mainly performed by human operators, due to the complexityof the process. When large parts are involved, more operators
are needed. The DrapeBot project aims at developing a humanrobot cooperative system capable of assisting one single operator
working on large parts. This requires environment perception inthe workcell, intelligent robot task and motion planning, and theunderstanding of the production process. This paper outlines thetrack proposed to create such an intelligent cooperative workcell.
Index Terms—Human-robot cooperation, cooperative production, behavior recognition, body pose estimation, robot task andmotion planning.

I. I NTRODUCTION
Draping is one of the most complex operations in carbon
fiber manufacturing. It is carried out by transporting the carbon
fiber fabric onto the preform and adapting its shape. This
process is usually performed by human operators. When it
comes to large components, usually two or more highly skilled
operators are needed—one usually controlling the process, andthe other(s) acting as a support. This is, at the same time,expensive and unneeded, because one single skilled operator
could manage the draping process, while the task of the additional operators could be carried out by an automatic systemcapable of following the indications provided, as exemplified
in Figure 1. This means to better exploit the skills of the humanoperator who acts as the leader of the draping process, guiding
the whole process and assessing its quality, while, at the sametime, employing robots for executing the more trivial but heavyduties.
A. Human-Robot Interaction for Carbon Fiber Parts Manufacturing
The European project DrapeBot (www.drapebot.eu) has thegoal of developing a novel human-robot interaction technology
This research has received funding from the European Union’s Horizon
2020 research and innovation program under grant agreement No. 101006732.

Fig. 1. A human operator cooperates with a robot for carbon fiber transportation.

capable of supporting one single human operator handlinglarge carbon fiber parts. This is based on cutting edge AItools, both at the perception and the action stage. In suchcontext, a thorough understanding of the scene is needed toproperly feed the AI systems that are in charge of determining
the robot action and motion in real-time. Also, understanding
the human intention is crucial to analyze whether the humanrobot interaction is ergonomically correct and safe.
B. The Smart Workcell
The workcell developed in DrapeBot is smart because it isaware of the processes that take place inside the workspace,thanks to the sensors and the artificial intelligence modulesthat equip the cell. Understanding human behavior starts fromperceiving the human body posture, involving Action Recognition [1], people re-identification [2] and human-computerinteraction—a typical output of a body pose estimation system

Fig. 2. Example of body pose estimation system, a key element of the smart
workcell.

is shown in Figure 2. In recent years, several efforts havebeen done to obtain fast and reliable human motion estimatesin real-time. Several works, like [3], focused specifically on
employing image processing techniques to gain knowledge
about the human pose in real-time. In some cases, filteringhas been also applied to provide smoother results [4].
An intelligent reaction from the robot, however, depends
not only on the perception of human behavior but also on thecurrent status of the workcell. For example, similar gestures
may have different meanings depending on the current statusof the workcell (e.g. the presence or absence of fabric on
the mould). The most flexible approach to understand theenvironment is semantic segmentation. This method is capableof classifying elements of the scene and it commonly runs
on 2D images. Nowadays, however, Deep Neural Networks(DNNs) and Convolutional Neural Networks (CNNs) areboosting semantic segmentation of 3D images.

The semantic environment perception equipping the workcell will be a key element for detecting in detail the current
status of the workcell, because it can provide a high-leveldescription of the components of the environment, enabling
automatic evaluation of the current status of the production
process, possible misalignments, and possible risks when
workers are in proximity of dangerous moving elements.
The DrapeBot project will enhance the state of the art
and the industrial practice by deploying an integrated layered
framework where the Task and Motion Planning (T&MP)
and the low-level control will be jointly designed to workin full synergy. The T&MP will manage human and context
information to do both: reason on the user model and interpret
several sensorial information. Sensor data will come from therobotic system and the human perception system, that will
include an AI-based human action and intention understanding module. This will enable optimization of task sharingbetween humans and robots and selecting the processing areas
to minimize human interference. The low-level control will
exploit the estimated human intention control dealing withthe compensation of the model inaccuracies, and allowing
a continuous adaptation on the best task and motion plancomputed by the T&MP module.
Considering the T&MP layer, the core advancement willconsist in the deployment of modules designed for efficient online computation of a sub-plan after performing each action.
Thus, exploiting at each iteration updated sensory readings, it
implicitly takes into account unexpected changes in the estimated state due to previous sensors and actuators failures, orenvironment re-configurations. This framework will integrate ahuman-aware motion planner that will generate a collision-freetrajectory provided in real-time leveraging on a data-driven
approach (e.g, a Deep Reinforcement Learning framework) toimprove the planning effectiveness.

II. W ORKCELL E NVIRONMENT U NDERSTANDING

III. C ONCLUSIONS

Focusing on the perception stage, DrapeBot aims at enhancing the state of the art and the industrial practice by combining
visual sensors with inertial sensors to obtain a fast and accurate
perception of how human workers move in the workcell.
This will be accompanied by a semantic interpretation of theenvironment. In this way, the system will ensure a smooth andreliable interaction with the human(s), that will not be blocked
nor slowed down due to temporary perception failures caused,
for example, by an occlusion preventing the vision system
to perceive the human operator. Such high-end perception
systems will be employed to provide a thorough understandingof the working environment and the body pose of the person(s)present in the working area of the robot. A probabilistic modelfor human body pose estimation will be derived, capable oftaking advantage of both visual and inertial sensors, enhancing
the detection when both are available but capable of working
when only one sensory system is working. An importantelement of the project will be the definition of the sensing
accuracy needed to meet the safety requirements of the robotic
work cell.

In this paper, the design of a smart workcell for cooperative
processing of large carbon fiber parts was presented. Theworkcell can actively support the human operators in theproduction process thanks to accurate and redundant sensory
information that are integrated with the robot task and motioncontroller.
R EFERENCES[1] F. Han, X. Yang, C. Reardon, Y. Zhang, and H. Zhang, “SimultaneousFeature and Body-Part Learning for real-time robot awareness of human behaviors”, 2017 IEEE International Conference on Robotics andAutomation (ICRA), pp. 2621-2628.
[2] S. Ghidoni, and M. Munaro, “A multi-viewpoint feature-based reidentification system driven by skeleton keypoints”, Robotics and Autonomous Systems, 90, 45-54.
[3] Z. Cao, T. Simon, S.E. Wei, and T. Sheikh, “Realtime multi-person 2D
pose estimation using part affinity fields”, in Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pp. 72917299.
[4] A. Malaguti, M. Carraro, M. Guidolin, L. Tagliapietra, E. Menegatti,
and S. Ghidoni, “Real-time tracking-by-detection of human motion inRGB-D camera networks”, in 2019 IEEE International Conference onSystems, Man and Cybernetics (SMC), pp. 3198-3204.