A Smart Workcell for Cooperative Assembly ofCarbon Fiber Parts Guided by Human Actions
Matteo Terreran∗ , Stefano Ghidoni∗ , Emanuele Menegatti∗ , Enrico Villagrossi† , Nicola Pedrocchi† ,Nicola Castaman‡ , Alberto Gottardi‡ , Christian Eitzinger§ ,Luca Vescovi¶ , Giuseppe Salemi¶ , Matteo Casubolo¶ , Marcin Malecha∥∗ Department of Information Engineering, University of Padova, Italy

Email: [matteo.terreran, stefano.ghidoni, emanuele.menegatti]@unipd.it† STIIMA-CNR, Milan, Italy - Email: [enrico.villagrossi, nicola.pedrocchi]@stiima.cnr.it‡ IT+Robotics srl, Padova, Italy - Email: [nicola.castaman, alberto.gottardi]@it-robotics.it§ Profactor GmbH, Steyr, Austria - Email: christian.eitzinger@profactor.at¶ Dallara Automobili S.p.A, Varano de’ Melegari - Parma, Italy - Email: [l.vescovi, g.salemi, m.casubolo]@dallara.it∥ Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Augsburg, Germany - Email: marcin.malecha@dlr.de

Abstract—The production of carbon fiber parts is a complex
process mainly performed by human operators. For small parts,
high dexterity and skilled operators are needed while for large
parts many people are often needed to transport the material
without damaging it. The DrapeBot project aims at developing ahuman-robot cooperative system capable of assisting an operator working on carbon fiber parts. This requires environment
perception in the workcell, intelligent robot task and motion
planning, and the understanding of the production process. In
particular, the project focuses on realizing a safe and effective
collaboration where the operator can interact with the robot in
an intuitive manner by means of gestures. This paper outlines
the track proposed to create such an intelligent workcell.
Index Terms—Human-robot cooperation, cooperative production, behaviour recognition, body pose estimation, human action
recognition.

I. I NTRODUCTION
Draping is one of the most complex operations in carbon
fiber manufacturing. It is carried out by transporting the carbon
fiber fabric onto the preform and adapting its shape. This
process is usually performed by human operators. When it
comes to large components, usually two or more highly skilled
operators are needed—one usually controlling the process, andthe other(s) acting as a support. This is, at the same time,
expensive and unneeded, because one single skilled operator
could manage the draping process, while the task of the additional operators could be carried out by an automatic systemcapable of following the indications provided, as exemplified
in Figure 1. This means to better exploit the skills of the humanoperator who acts as the leader of the draping process, guiding
the whole process and assessing its quality, while, at the sametime, employing robots for executing the more trivial but heavyduties.
The European project DrapeBot (www.drapebot.eu) has thegoal of developing a novel human-robot interaction technology
capable of supporting one single human operator handlingThis research has received funding from the European Union’s Horizon
2020 research and innovation program under grant agreement No. 101006732.

Fig. 1. An example of the draping process of an aerospace carbon fiber parts.
The human operator cooperates with two humans mimicking a robot holding
the material.

large carbon fiber parts. Among the various use cases envisioned in DrapeBot there are the automotive and aerospacesectors. In the former case, the project addresses the production of complex components of an automotive sport car, likethe front hood or the side panels with dimensions up to 2-3
meters. In the latter case, the project addresses the productionof parts of the structural framework of an aircraft fuselage,
mostly skin stiffening elements such as spars and ribs. The
typical size of a rib is 5 meters and consists of up to 20 layersof material.
The DrapeBot project proposes the development of two
robotic workcells shown in Figure 2, specialized for different carbon fiber parts production processes. A medium size
workcell is designed for supporting the human operator in
the production of small and medium carbon fiber parts suchas components in the automotive industry (e.g., car’s hood);such workcell has a footprint of approximately 3 × 5 meters insize and equipped with an industrial robot arm which assists
the human partner during material transportation phases, or byplacing directly and precisely small patches of material leaving

Fig. 2. The two workcells developed in the DrapeBot project. On the left, a workcell for the production of small and medium parts such as in automotiveindustry; on the right a workcell for large carbon fiber parts production from the aerospace industry.

to the operator to adapt the material in high curvature points.
For production processes involving large carbon fiber parts
instead, a large size workcell (i.e., with a footprint of 6.5×8.5m) equipped with two industrial robot is proposed; the two
manipulators are moved in coordination to assist the humanoperator during the material transportation to prevent large
patches from being deformed or ruined by too high tensions.
Both workcells are based on cutting edge AI tools, both at
the perception and the action stage. In such context, a thoroughunderstanding of the scene is needed to properly feed the AI
systems that are in charge of determining the robot action andmotion in real-time. Also, understanding the human intention
is crucial to analyze whether the human-robot interaction isergonomically correct and safe.
II. W ORKCELL P ERCEPTION S YSTEM
The workcells developed in DrapeBot are smart because
they are aware of the processes that take place inside theworkspace, thanks to: i) prior knowledge of the production
processes being performed; ii) a complex sensory systemcapable of observing the workcell under different viewpoints,
using different sensor technologies and with a certain degreeof redundancy, and iii) AI modules capable of interpreting thesensory data to extract high-level information, with a special
focus on human activities.
A. Human Perception
Regarding human perception, DrapeBot aims at enhancing
the state of the art and the industrial practice by combining
visual sensors with inertial sensors to obtain a fast and accurate
perception of how human workers move in the workcell.
Different perception modules are developed to understandhuman activities at multiple levels, namely: pose estimation,
human parsing, human action recognition.
A probabilistic model for human body pose estimation isderived, capable of taking advantage of both visual and inertial
sensors, enhancing the detection when both are available

but capable of working when only one sensory system isworking. Pose estimation provides a schematic representationof the human body made of links and joints, as depicted
in Figure 3. The human parsing module [2] outputs a 3D
human representation derived from body parts segmentation,
which combines high-level semantic information (i.e., humanbody parts) and volume information. Human parsing andpose estimation outputs represent complementary information
which are then combined together to obtain a complete andaccurate estimation of the human movements and the workcell
space taken by the human body, that is crucial to ensure safety.
Finally, the human perception system includes an AI-based
human action and intention recognition module trained torecognize human behaviors from a predefined set of actions
which can occur in a human-robot collaboration task. The
list of actions includes common collaborative actions such astransporting, picking an object, requiring an object or gestures
to give commands to the robot like stop, move left/right, asshown in Figure 3. Moreover, depending on the use case, theset of actions can be extended to include specific actions forthe processing under consideration; considering the automotive
case, the actions considered are: i) adapt the carbon fiber
patch to a desired shape; ii) press the patch in some key
position to fine-tune the shape adaptation in the previous step;iii) remove a film in order to expose an adhesive layer –this is an action that is required to attach the current patch
to the next one. Such actions are very specific for carbon
fiber production (they are key production steps); however, theactions performed by the human operators are rather genericand limited to pressing, holding and pulling a carbon fiber
patch. This makes it possible to tackle complex tasks related
to carbon fiber production using an action recognition systemtrained to recognize rather generic actions. The gap between ageneral action (like “pull an object”) and the special meaningthat such action has in the specific production process relies in:a) knowledge on the production environment and the objects
involved in the actions; b) a priori knowledge of the production

Fig. 3. An example of possible action during a human-robot collaboration task. From left to right: walking, picking an object, requiring an object and gestures
to give commands to the robot like stop.

process being performed, which is available to the system.
Actions and intentions are computed based on the probabilistic model for the human body pose, which provides
a deeper understanding about what the human operator iscurrently doing or want to achieve. For example, during acollaborative transport task, human intentions describe themotion direction and the force applied by the operator to movethe carbon fiber patch.
B. Environment Understanding
The semantic environment perception module is a key
element for detecting in detail the current status of the workcell, because it can provide a high-level description of thecomponents of the environment, enabling automatic evaluationof the current status of the production process, possible misalignments, and possible risks when workers are in proximityof dangerous moving elements. Indeed, an intelligent reaction
from the robot depends not only on the perception of humanbehavior but also on the current status of the workcell. Moreover, similar gestures may have different meanings dependingon the current status of the workcell (e.g. the presence orabsence of fabric on the mould). The most flexible approachto understand the environment is semantic segmentation. Thismethod is capable of classifying elements of the scene andit is commonly based on Deep Neural Networks (DNNs) andConvolutional Neural Networks (CNNs).
Thanks to semantic segmentation it is possible to monitor
the current status of the workcell and to recognize particularconditions of the material handled during the collaboration,such as if the material is too tight or too loose. Moreover,the semantic segmentation of the environment is helpful toimprove the understanding of the human actions, for example
which objects the operator is using in a given moment.

sequence of human actions: such module is responsible forcreating a continuously updated plan that serves as a guideline
for the robot workflow, composed of a sequence of robot
actions to achieve a given task. Moreover, human intentions
are also taken into account by the task planner to dynamically
adapt the robot behaviour to the human needs during thecollaboration. The robot low-level control will exploit theestimated human intentions to better assist the human partner
during collaborative transportation tasks, dealing with thecompensation of the inaccuracies (e.g., human unexpectedmovement), and allowing a continuous adaptation of the bestrobot motion trajectory. (e.g., during material transportation, if
the human worker slows down the robot adapts its movements
not to pull the material too much).
During collaboration the human operator can trigger specificactions by the robot through the use of gestures (e.g., inspecta specific location on a carbon fiber part). In such a case,the gesture is recognized by the human perception systemprompting the motion planner module to generate a collisionfree trajectory to move the robot in the requested position.
IV. C ONCLUSIONSIn this paper, the design of a smart workcell for cooperative
processing of large carbon fiber parts was presented. The
workcell can actively support the human operators in theproduction process thanks to accurate and redundant sensory
information that are integrated with the robot task and motion
controller. Special emphasis is placed on achieving effective
and intuitive collaboration for the human operator through gesture recognition and intention estimation; by knowing humanintention is indeed possible to program the robot actions suchthat they are perceived the more natural as possible by thehuman partner.

III. H UMAN -ROBOT C OLLABORATIONR EFERENCES

The human perception system provides a complete understanding of the human behaviour, involving the humanpose and the recognized human actions and intentions. This
high-level information can be used to optimize task sharing
between humans and robots and to select the processing areas
to minimize human interference. In particular, a task plannermodule coordinates the robot activities based on the current[1] Z. Cao, T. Simon, S.E. Wei, and T. Sheikh, “Realtime multi-person 2D
pose estimation using part affinity fields”, in Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pp. 72917299, 2017.
[2] M. Terreran, L. Barcellona, D. Evangelista and S. Ghidoni, ”Multi-view
Human Parsing for Human-Robot Collaboration,” 2021 20th International Conference on Advanced Robotics (ICAR), 2021, pp. 905-912,
doi: 10.1109/ICAR53236.2021.9659456.