# Using programming by demonstration tools for DEMO maintenance: a framework proposal with an experimental validation

Olivier David, Victor Dhédin, Julie Dumora, Vincent Padois, Adib Rebbouh, Ferdinando Milella

► To cite this version:

Olivier David, Victor Dhédin, Julie Dumora, Vincent Padois, Adib Rebbouh, et al. Using programming by demonstration tools for DEMO maintenance: a framework proposal with an experimental validation. Fusion Engineering and Design, 2025, 215, pp.114935. 10.1016/j.fusengdes.2025.114935. hal-04990347

HAL Id: hal-04990347

https://inria.hal.science/hal-04990347v1

Submitted on 13 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Using programming by demonstration tools for DEMO maintenance: a framework proposal with an
experimental validation

O.David*<fnref n="1" />, V. Dhédin<fnref n="1" />+, J. Dumora<fnref n="1" />, V. Padois<fnref n="2" />, A. Rebbouh<fnref n="1" />, F. Milella<fnref n="3" />

Abstract

The level of radiation expected in the DEMO fusion reactor facility will make it impossible to send
operators during maintenance phases of the reactor as well as of a large number of ancillary components.
During each scheduled shutdown phase, specifically designed robotized machines will replace
consumable components in the course of complex and carefully studied procedures. However, accurate
predictions for a large volume of tasks such as inspection or replacement of faulty components, are very
difficult. These tasks will therefore require more versatile tools (e.g. manipulator with carrier). The
resulting sum of potential errors (manufacturing, positioning...) makes it complex to carry out these
tasks in a fully automated mode. Man-in-the-loop teleoperation techniques are therefore being
considered, as they provide fault tolerance and adaptability. Previous studies have shown that the
generation of operator assistance by model-based control tools does not only provides comfort when
executing tasks, but also significantly improves their execution quality and allows hybrid modes
switching between man in the loop, automated or semi-automated control modes. However, design of
such models remains complex. Inspired by "programming by demonstration" (PbD) tools used in
robotics, the feasibility of a framework creating the model directly during the execution phase is
investigated, thanks to measurements taken locally. The model is built-up on-site, today with help of
probes and in the future with 3D scanners. Then, the model is used to generate operator assistance. After
reviewing the literature on the subject, we detail the particularities of our approach focusing on the
interface level between a tool and its target and we explain why it is relevant in contexts such as fusion
facilities. Then an experimental validation in the form of a pipe inspection with a camera mounted on a
manipulator arm is presented.

Keywords:

Remote Handling, Teleoperation, Programming by Demonstration, Virtual Guides, Collaborative Robot, Maintenance, DEMO

1 Introduction and context

Due to the high level of radiation expected during the long term shut down of the DEMO
facility, a large majority of maintenance tasks are expected to be carried out with robots. So far, the
works reported in the literature has focused mainly on the organization and zoning of buildings [1],

<footnote n="1">¹ Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France</footnote>

<footnote n="2">² Inria, Auctus, 200 Avenue de la Vieille Tour, Talence, 33405, France</footnote>

<footnote n="3">³ RACE United Kingdom Atomic Energy Authority, Culham Science Centre, Abingdon, OX14 3DB, UK</footnote>

<footnote>+ Centrale-Supelec student in training period at CEA List at the time of that study, now PhD student in robotics at Technical University Munich</footnote>

<footnote>*Corresponding author: Olivier David: odavid@cea.fr</footnote>
maintenance schemes of the main components [2] [3] and on the hot cell work [4]. This has led to comparisons of installation concepts in which the emphasis is on safety constraints and limiting the spread of contamination, or in which flows are organized in such a way as to ensure that maintenance operations are carried out as quickly as possible [1]. Maintenance of the main In-Vessel critical components and consumables such as blankets and divertor [2] [3] [5] are designing equipment involving automated process. Much research effort has been put on these operations as they involve the handling of large and heavy components that are fundamental for the realisation of DEMO.

With an expected lower level of complexity, the maintenance strategy in the ex-vessel part of the facility has so far received less attention. However, the wide variety of situations and tasks could make the maintenance tasks in this area difficult: operating, repairing or replacing valves, performing visual inspection, installing welding equipment... Current thinking is more towards the use of a general-purpose machine that could be used in a large number of specific cases. Unlike in robotic industry where operations take place in front of fixed workstation, maintenance operations in a fusion facility require the robotic equipment to travel to the component location where it will perform a particular and often complex task. There will be no preliminary adjustments, no repetition of the task on site before its execution. The task has to be completed with the required level of satisfaction at the first trial. Although time matters, the time taken to achieve this is of secondary importance compared to the success of the task. Where industrial robotics, due to the large number of tests before the task execution, can rely on repeatability, DEMO's robots face a new situation each time after deployment and will need a good accuracy if one wants to perform the operations fully automatically. The unpredictable nature of the task makes the use of offline programmed machines virtually impossible. A traditional mean to reduce the complexity and increase robustness of the maintenance task is to introduce compliance and adaptability into the task. Compliance is usually achieved by adding mechanical guiding features either onto the components or onto the tools. Adaptability is ensured by adding an operator in the control loop, like JET did with success for many years [6]. However, compliance and adaptability can also be part of a more global control software strategy: providing operators with assistance functions.

Rather than working throughout all maintenance operations in a single reference frame, which is the robot's reference frame, the introduction of Cartesian controllers in place of joint controllers enabled maintenance works to be carried out in the task's reference frame thus reducing dependencies to a specific robot architecture. Working in the task's space greatly simplified the design of assistance such as virtual mechanism [7]. When a connection to a virtual mechanism is active, the operator's workspace is reduced to that of a mechanism with a limited number of degrees of freedom meaning operator's movements are restricted to those needed to accomplish the task (e.g. moving along a line). The nuclear industry already successfully implemented this strategy on real maintenance operations in the La Hague facility [8]. As an evolution of this work, some studies proposed to enhance the traditional "master-slave" units used in teleoperation with features taken out of a virtual physics model connected in real-time at high speed to the master unit [9]. As a result, force feedback to the operator can be provided from either the real or the virtual world. The benefits of such a strategy have been demonstrated on simple examples (e.g. dye penetrant deposit simulation, visual inspection) during studies for ITER [10] where real improvements were seen in the quality of task execution and a significant reduction of operators' cognitive load.

One difficulty pointed out in the literature for model based robotics applications [11], [12] is to provide answer to the challenge of high-level reasoning (e.g. cut a pipe) in a low-level world (controlling the robot means getting information from sensors and driving each joint to a new
position at the required speed until the target is reached). As explained by [11], the problem is to
find a good balance between these two parts of the challenge. In fact, most of the available works
are addressing one or the other side of the problem. An additional challenge is to define the right
level of contribution between the human operator and that of the robot: how to make a distinction
between what belongs to the robot's adaptability to a task from what belongs to assistance to the
operator.

Inspired by the literature on PbD, this work proposes a framework that enables a simple physics model (as such defined in [9]) to be created on the fly from live measurements (e.g. with probing, laser-scanner...), therefore removing time and effort to build the virtual model. As the model is created online, it is more likely to be faithful to reality. Similar to [13], it is chosen to reason not at a high-level (that of the model's objects) but at the more median level of the interfaces between objects (e.g. the eye of a camera, the outer surface of a pipe). The model is therefore made up of model objects named actors, each with a list of interfaces. Each interface has its own role and use. It can be linked to a complementary interface in the scene. Interaction between two compatible interfaces is achieved by creating on the fly a virtual mechanism between the two interfaces that respects the physical and geometric properties of the objects. The creation of such assistance enables operators to carry out their tasks more easily. In that way, the operator can focus on the objective of the task rather than on the secondary constraints to be monitored to ensure correct execution (e.g. maintaining a normal orientation to a surface during a smear test).

In the context of the maintenance of DEMO's Ex-Vessel, the aim of all the maintenance tasks process
is to get the plant up and running again safely, reliably and quickly. The robot is a tool for achieving
this, and robot programming must not be an obstacle to carrying out maintenance operations. The
supervision of maintenance tasks should not be carried out by an expert in robot programming, but
by a specialist in tokamaks and their technology. In this context, PbD tools make sense as they aim to
be a proper Human Machine Interface which simplifies the execution of tasks by operators from the
Human Factors perspective.

After a brief state of the art of the strategies used in PbD tools, the framework developed during this
study is described. Finally, an application example illustrating a visual inspection of a DN80 pipe is
presented.

## 2 Programming by demonstration in the literature

PbD tools belong to a wider group of tools designed to assist robot programming incorporating manufacturers' programming interface but also alternative strategies based on block programming languages. While user interfaces offered by robotics manufacturers today are increasingly ergonomic and user-friendly, they still remain complex for non-professional people. Strategies proposing to use block programming language [14], [15], [16] always produce results syntactically correct. But their major disadvantage is that they require some prior knowledge of programming concepts, such as loops and conditional statements. It also requires the user to understand some basic robot-related concepts, such as actuators and sensors [17]. Therefore, the whole concept still remains complicated for non-expert users.

Another robot programming assistance facet of the literature investigates learning by demonstration techniques. Although the frontier is not always very clear between learning and programming by demonstration, a lot of the literature addresses the problem from the point of view of: "how to learn a task that has to be repeated later". Several works are proposing relevant state of art review on the
subject [18] [19]. For the time being, we have deliberately set aside this part of the literature,
because we believe it does not yet provide a relevant response to the Fusion industry context.

PbD in robotics is usually described in the literature as an approach that enables a robot to be
programmed by observing and imitating the movements and actions of a human user. Rather than
explicitly specifying the details of each step in the process, the user shows the robot how to
accomplish the task by performing it himself. [20] and [21] categorize the various methods of
programming by demonstration into three parts:

* Kinesthetic teaching: the operator directly drives the robot hands-on.

* Teleoperation: demonstration from a remote location.

* Passive observation (e.g. [22]).

In kinesthetic teaching, the user physically interacts with the robot and access to the robot is
mandatory to perform the desired task. The state of the robot during this interaction is recorded by
its on-board sensors, providing the data necessary to reproduce the trajectory. This method is
effective for manipulator arms (including light industrial robots) thanks to its intuitive approach and
the fact that it does not require any robotic or programming skills. In addition, kinesthetic teaching
does not require external sensors, interfaces or additional inputs. However, this method has its
limitations, in particular the quality of the demonstrations depends on the dexterity of the operator,
and even with experts, the data obtained by this method often require smoothing or other post-
processing techniques.

In demonstration by teleoperation the robot is controlled by an external device: haptic interface,
joystick, graphic interface, etc... Teleoperation does not require the operator to be physically present
close to the robot. It also allows the introduction of new tools such as virtual reality in which the
operator drives a digital machine in a digital environment or augmented reality where the operator
has access to real-time video feedback, which can be enhanced with additional information [23].
According to [24], virtual reality makes it easier to program an industrial robotic arm when the tasks
to be performed are complex and require to faithfully reproduce the operator's movements. One
limitation of teleoperation teaching is that it requires additional facilities and hardware linked to the
chosen remote control interface. In some cases, operator training is also required.

In the third approach, the robot learns through passive observation of the operator. The user
performs the task using his or her own body, sometimes instrumented with additional sensors to
facilitate tracking. The robot never moves during the learning phase, acting solely as a passive
observer [19].

A pioneering work presented in [22] consists in first determining, then replaying a sequence of
actions from a video where the operator manipulates blocks. Authors in [25] propose a method in
which the operator demonstrates a sequence of actions to be performed by virtually manipulating
blocks of the real scene using augmented reality tools. The blocks have simple shapes and high-
contrast colours. The authors explain that this method reduces programming time.

One of the advantages of such methods is that they are very easy to perform, and demonstration can
be done by inexperienced users. The operator has to concentrate on completing the task. This makes
the method more intuitive. These methods are known as "Task level programming". The major
disadvantage of passive observation programming is that it is necessary to match the actions of the
operators performing the demonstration to those performed by the robot. This can very quickly
become a very complicated task. Similar to teleoperation teaching, this method also requires
additional hardware and external sensors.

As a conclusion, PbD focuses on the intention of the operation and the completion of the task itself. The robot's trajectory is taken into account but only implicitly. Operators record points of interest, positions for grasping objects, and so on, themselves by demonstration and limited interactions with an interface (e.g. push buttons, browse and select elements in lists). PbD usually requires no prior programming experience, and the operator only needs to interact with a user interface or tool for which the learning curve seems fairly limited. PbD therefore seems fairly intuitive and quick to implement for novice users.

# 3 Proposed Framework

## 3.1 Concepts

As with many PbD tools, the proposed approach is multi-level (see Figure 1):

* Scene level: top level of the model. The scene level describes the world and incorporates actors elements.

* Actor level: describes for each actor the list of what we called actor's interfaces. Interfaces are local positions of interest with local properties. Interfaces are defined with respect to the actor frame.

* Interface level: each interface defines local interaction capabilities and behaviour such as the mechanism. Interface mechanisms define local workspace restriction capabilities (e.g. rotate around interface axis X).

The whole model consists in a world (also known as scene) in which objects called actors will evolve (see Figure 1). Actors are considered as scene elements or objects with interaction capabilities. Actors have a position in the world frame, and a set of properties and possible behaviours all provided with respect to their actor frame. Properties can be global (e.g. actor's weight) or local (e.g. the clamping location). According to the context, roles are affected to actors. Actors roles are either effectors or targets. An effector is a manipulated object. It is mobile and can be attached to a tool or a manipulator. A target is the object to interact with, the object to transform, the object to move. The role of an actor is updated according to the context (e.g. a wrench is a target for a tool changer and becomes an effector for a nut). The execution of a task is the result of the creation of a connection between an effector and a target through a particular interaction.

As explained above, the difficulty often lies in transcribing a high-level information (e.g., “catch” “object”) into low-level instructions to be followed by a set of actuators. The formulation of the task description is deliberately placed not at the level of the actor itself, but at the level of sub-elements called interfaces. Interfaces are specific locations, usually a 2D restriction of the actor with local properties (frame, dimensions, geometry...). Each actor owns a list of interfaces. The actor’s role is forwarded to the interface. Therefore, there are target interfaces and effector interfaces.
<figure><img src="image_2.png" /><figcaption>Figure 1 Model breakdown</figcaption></figure>

As the positions of actors are updated into the physics engine, a periodic task considered as a scene management tool observes the scene and reacts to each new position of all actors. Its role is to identify and sort possible matches between active effectors and active targets.

Activation/deactivation of interfaces is the result of transition events (e.g. grasping/releasing an object, a moving object becomes static...).

Two interfaces can match if they comply with rules such as:

* their role are opposite (one effector, one target)

* their action type (grasping, inserting, placing, inspecting...) is the same

* other rules involving specific geometric properties (prismatic, axisymmetric ...)

* ...

The number and type of rules can be updated according to the context.

The possible interfaces matches are then sorted. Sorting interfaces pairs is today the result of a distance criteria (in both translation and rotation). But one could also imagine to introduce a weight criteria based on the type of action or the local size of two matching interfaces.

On operator demand or following a trigger event, the configuration of the operator assistance between the effector interface and its target starts. When the configuration is over, the assistance helps the operator to bring the effector at the target position.
To do so, a combination of three local mechanisms connected in series is created. Each mechanism is composed of a fixed part called base and a mobile part called hook. Movements of hooks are limited to the degrees of freedom provided by each mechanism. A mechanical chain is then created with appropriate welding of bases and hooks of all three mechanisms. The chain is composed of (see Figure 1):

* A local interface mechanism on the effector side: described with respect to the effector's frame (e.g. a rotation around the normal of one of the effector's surfaces). Its base is welded to the effector frame. Its hook is welded to the coming generalized mechanism.

* A local interface mechanism on the target side. Described with respect to the target's frame (e.g. two translations along the axis of one plane of the target). Its base is welded to the target frame. Its hook is welded to the following generalized mechanism.

* A generalized linear mechanism links the hook of the mechanism on the effector side to the hook of the mechanism on the target side. The configuration of this mechanism is such that movements of the operator are constrained along the line joining the effector position and the target current position (linear interpolation). While the operator moves along the line orientations are constrained such as starting with the local orientation of the effector the operator ends up with the orientation of the target (use of SLERP for Spherical Linear interpolation for the rotations). This can be considered as an optimal path proposed to the operator to move from one position to another (can also be seen on Figure 5a).

The resulting mechanism helps the operator to perform a higher quality gesture. If required or preferred, the operator can adjust the orientation of the effector during the travel phase thanks to the local interface mechanism of the effector. When the final position is reached the mechanism of the target interface only allows moving along (or rotating around) local preferential directions of the target (e.g. axis of a cylinder).

## 3.2 Architecture

### 3.2.1 Framework elements:

Figure 2 describes the organization of the whole framework. A central entity (the whole programming by demonstration framework) plays the role of a gateway between specialized libraries. This central entity holds the world instance of the model. It uses the actor library in order to add actors to the model. The actor library provides convenient functions to access and perform calculations on all actor's properties. An actor can be generic or belong to a more specialized class (e.g. plane, cylinder and box). Once created, the actor mesh representation is transferred to the 3D Graphical User Interface (GUI). The GUI displays each actor at its current position in real-time. The GUI is based on the VTK library (Visual Tool Kit library). The operator can create actors by interacting with the panels of the Man Machine Interface (MMI) or with direct interaction with the robot. For this second usage, the "Primitives" library is used. The "Primitives" library gives access to functions able to create planes, cylinders or boxes starting from a collection of points measured in reality. After running a calibration routine, the operator can collect points from real objects that will be used to create actors at the right place within the model. The data collection process is related to the robot itself meaning that if the operator uses a master arm during the acquisition process, there will be no changes in the result.

### 3.2.2 Creation of interfaces:

The actor library is connected to an interface creation toolbox. Interfaces can either be predefined for known actors' types (boxes, cylinder, plane) or can be created thanks to entries in the Man Machine Interface (MMI) or using a segmentation tool. Thanks to an analysis of the mesh data of the actor, the segmentation tool proposes a set of surfaces (mostly planes or axisymmetric) of interest.
The operator can choose and keep the interfaces making sense for his or her operation. Although the principle of surfaces recognition/extraction from a given mesh was prototyped, its thorough integration in the framework is still pending.

According to each interface characteristics, the function of the interface creator is to fill or define properties such as position, mechanism, role, etc.

One can save and restore scenes and actors for a future use or update the model by taking new measurements.

### 3.2.3 Connection to simulation or real robots, operations:

Once the definition of the model is finished, it is possible to use the robot Application Programming Interface (API) to discuss with the robot controllers (Cortex framework) or with the physics simulation (Score framework) that are both CEA List developments. It is also possible to connect to the ROS (Robot Operating System) open source robot controller and middleware. As this was not the main objective of the project, the list of available function calls is for the moment very limited. It can easily be enriched with new functions as the core functions as well as example functions have been implemented both in the server and client sides. All communication between clients and servers are based onto the RPClib library.

This work concentrated onto the capability to create a physics simulation online. This includes its definition, settings and updates. Thanks to the real-time connection between the real robot controller and the simulation (see Figure 2), any interaction with the model will be reproduced by the real machine and vice versa. When a virtual mechanism reduces the workspace of the virtual robot, the operator's movements interacting with the real robot are also restricted to the same workspace.

Both the real and virtual robots operate in force control mode. Thanks to the bilateral coupling between the real robot and its virtual avatar, the operator can feel real forces as they appear or virtual forces from the model as they are calculated.
<figure><img src="image_3.png" /><figcaption>Figure 2 Scene Creation Framework</figcaption></figure>
# 4 Experimental validation

All the developments have been designed to be able to validate the principles developed in the previous sections on a simple test case. Steps of the generic process include the following operations:

* Create simple actors based on real-world measurements

* Create a scene including these actors

* Define the interfaces associated with the actors in the scene

* Transfer actors to a physical simulation connected to a real robot

* Use the physics simulation tools to perform a representative task

The experimental work therefore focused on a demonstration illustrating all these stages.

A simple, credible task is considered: inspecting a DN80 pipe fixed to a Thorlabs board using a camera carried by a M0617 manipulator from Doosan (see Figure 3). The M0617 belongs to the cooperative robots series from Doosan. It has a payload of 6kg and a maximal reach of 1.7m. Although it is feasible to drive the robot with a 6D master arm, the experiment is performed with the operator interacting with the robot hands on. To take advantage of the existing tools enabling coupling between a real robot and a physics simulation, the robot controller implemented into the M0617 runs under CEA List's Cortex framework.

<figure><img src="image_4.png" /><figcaption>Figure 3 Mock-up installation</figcaption></figure>

The operator needs first to connect the scene creation framework to the robot. The robot is either calibrated (a routine is available) or a calibration file is loaded in order to measure (or read) the position of the effector tip used as touch probe. Within the generic sequence of the framework, after this calibration step, the operator has the capability to create the scene actors:
* With direct contact onto the real elements and the help of the “Primitives” library.

* With a mixed mode using direct contact and manual information (e.g. real diameter and length of the pipe).

* With manual definition only. In that case, calibration routines may also be used in order to set the actor at its correct location.

Once the actor is created, it is inserted into the scene:

* It is possible to define/modify at this step the actor interfaces.

* When more than one actor is available, it is possible to define relationships between actors (e.g. the pipe is on top of the board). Relationships can also be used since the creation of actors in order to position an actor with respect to another.

* If a physics simulation is available, we can use it in order to either simulate or carry out the real operation. It is also possible to use the scene in order to operate the real robot and its interactions into the real environment.

Applied to this use case, the steps become those of Figure 4 in which after calibration of the manipulator, the operator loads an empty simulation (Figure 4a). Then, thanks to the primitive library, after measuring and recording points from the plane (Figure 4b and 4c) and the pipe (Figure 4d) the operator asks for creation and insertion of these two “scene actors” at the right place into the simulation (Figure 4e).

<figure><img src="image_5.png" /><figcaption>Figure 4 Scene creation steps</figcaption></figure>
In a second stage, the actors' interfaces are created and parameterized via the physics simulation's API. Two interfaces were created:

* One effector interface at the location of the camera eye. Orientation complies with the camera frame.

* One target interface 10 cm away from the pipe surface (point A of Figure 5a). Orientation is such that when the camera interface matches the pipe interface, the up vector of the camera is parallel to the pipe axis and the camera direction is collinear to a radius of the pipe.

In order to perform a visual inspection of the pipe, extra mobility to the interfaces must be added. In this particular case, the effector interface (camera eye) does not require any additional degree of freedom (dof). On the target interface an extra prismatic *dof* along the cylinder's axis and an extra hinge *dof* around the same axis (point B of Figure 5a) are added. A rigid link is then created between A and B.

<figure><img src="image_6.png" /><figcaption>Figure 5 Interfaces and their associated mechanism</figcaption></figure>

Following an operator's request, a virtual kinematic chain is created between the effector and target interfaces. This kinematic chain is the serial combination of:

* the mechanism of the effector interface (not existing in this application),
* an interpolation path starting at the current location and orientation of the effector interface (the camera eye) and ending at the location and orientation of the target interface (point A),

* the extra two *dof* added to the target interface.

Operators' movements are therefore constrained such that moving along the virtual mechanism will finally make the camera eye joining the position of point A after a travel phase. Usually during this phase the mechanism of the target interface is locked but this is not mandatory. When unlocked, the operator can slide along and rotate around the pipe.

An accompanying video, available with this article, describes all the tests and allows to follow their progress. For the sake of convenience, the robot, used in collaborative mode, is moved by the operator, who holds it by the wrist. If the robot had been manipulated with a master arm, a similar result would have been obtained.

Figure 6 illustrates this experiment. Figure 6(a) and (b) show the camera pointing toward the centre of the pipe for two different joint configurations. Similarly, Figure 6(c) and (d) show the entire scene for comparable instants. Figure 6(e) shows a view from the on-board camera. The demarcation caused by the tape stuck on the pipe is clearly visible.
<figure><img src="image_7.png" /><figcaption>Figure 6 Visual inspection of DN80 Pipe</figcaption></figure>

This framework makes it easy to interact with all the objects in the scene with very limited adjustments and several combinations of mechanisms are available. Thus, in a second trial available on video, it is shown that the operator also has the capability to create connections between the camera tip and the Thorlabs board. In that case, a second target interface onto the plane of the board just needs to be defined. Orientation is such that when the camera interface matches the board interface, the camera direction is horizontal and parallel to one side of the board. Then, in a similar manner to the one used for the pipe, a mechanism with two prismatic *dof* along the directions of the plane is added. When the connection is established, the movements will be such that the wrist remains orthogonal to the table and the camera moves in a horizontal plane, maintaining a constant orientation.
## 5 Perspective and future work

Although this is clearly not an issue considering the current architecture, it would be interesting to repeat the same demonstration in teleoperation mode with an operator interacting with a master arm.

Possible extensions to this work also include work on a segmentation tool that automatically defines the interfaces of each object in the model, leaving the operator free to modify, validate or delete them.

Other actor creation workflow could be integrated. For example, we can imagine tools extracting planes or cylinders from a point cloud collected by a laser scanner.

Work on the interface analyzer also needs to be continued, with a view to making it as generic as possible. As a reminder, the aim of this tool is to define which interface pair to activate according to the current context of the scene. It will need to be given the means to analyze possible interactions between objects not only on the basis of proximity criteria, but also, for example, on the basis of criteria such as the geometric family (prismatic, axisymmetric) to which the object belongs, or even on the basis of size criteria.

While interaction with the operator remains central to this work, there is also the question of partially automating certain functions. Experience shows that it can sometimes be complex for operators to manipulate the 6 degrees of freedom of a controlling arm to achieve perfect gestures. As soon as operators can rely on a virtual guide, it becomes possible to restrict the number of degrees of freedom of the interface with operators, and thus make their work easier.

During approach phases or highly repetitive phases, and as long as they remain under the supervision of an operator, it seems possible to switch to automatic modes once preparatory work carried out by the operator has enabled the task to be initialized correctly. It would therefore be interesting to study hybrid combinations of robotic and teleoperation modes.

## 6 Conclusions

Using simple application examples, this work illustrates the feasibility of the execution of complex movements in teleoperation or comanipulation with help of a model that can be built directly on the fly during the execution phase. Thanks to this model, it is possible to define assistance for the operator's gestures, with the aim of simplifying the task and improving its quality of execution.

By establishing relationships between objects in a model at the level of their interfaces rather than at the level of the object itself, the authors of this study believe that their work can be generalized more easily, and will thus be less dependent on the primary destination of each object.

No prior knowledge of robotics programming is required to use such tool. Therefore, the operator is likely to be a person with a background knowledge centered onto fusion facility maintenance rather than a roboticist.

In this first phase, a significant integration work was carried out in order to prove the efficacy and benefits of the method on a simple case. For this reason, a number of functions were hard-coded.
Work over the next years will focus on the generalization of functions and on opportunities for validation on more complex cases.

# 7 Acknowledgement

This work has been carried out within the framework of the EUROfusion Consortium, funded by the European Union via the Euratom Research and Training Programme (Grant Agreement No 101052200 — EUROfusion). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union

# 8 References

[1] C. Gliss, C. Bachmann, S. Ciattaglia, B. Drumm, M. Gracia Camacho, I. Moscato, T. Mull et I. Palermo, «Integrated design of tokamak building concepts including ex-vessel maintenance,» vol. 177, 2022.

[2] H. Utoh, K. Tobita, Y. Someya et H. Takase, «Conceptual study of vertical sector transport maintenance for DEMO,» *Fusion Engineering and Design*, vol. 87, pp. 1409-1413, 2012.

[3] C. Li, H. Wu, H. Eskelinen, M. Siuko et A. Loving, «Design and analysis of robot for the maintenance of divertor in DEMO fusion reactor,» *Fusion Engineering and Design*, vol. 146, pp. 2092-2095, 2019.

[4] J. Thomas, A. Loving, C. Bachmann et J. Harman, «DEMO Hot cell and ex-vessel remote maintenance,» *Fusion Engineering and Design*, vol. 88, pp. 2123-2127, 2013.

[5] O. Crofts, A. Loving, D. Iglesias, M. Coleman, M. Siuko, M. Mittwollen, V. Queral, A. Vale et E. Villedieu, «Overview of progress on the European DEMO remote maintenance strategy,» *Fusion Engineering and Design*, vol. 109 111, pp. 1392-1398, 2016.

[6] A. C. Rolfe, «A perspective on fusion relevant remote handling techniques,» *Fusion Engineering and Design*, pp. Vol 82 Issues 15-24 1917-1923, 2007.

[7] L. D. Joly et C. Candriot, «Imposing motion constraints to a force reflecting telerobot through real-time simulation of a virtual mechanism,» chez *IEEE International Conference on Robotics and Automation*, Nagoya, 1995.

[8] F. Geffard, P. Garrec, G. Piolain, M.-A. Brudieu, J.-F. Thro, A. Coudray et E. Lelann, «TAO2000 V2 computer-assisted force feedback telemanipulators used as maintenance and production tools at the AREVA NC-La Hague fuel recycling plant,» *Journal of field robotics*, vol. 29, pp. 161-174, 2012.

[9] O. David, F. X. Russotto, M. Da Silva Simoes et Y. Measson, «Collision avoidance, virtual guides and advanced supervisory control teleoperation techniques for high-tech construction :
framework design,» *Automation in Construction*, pp. 63-72, 2014.

[10] O. David, S. Sarkar, N. Kammerer, C. Nantermoz, F. Mayran de Chamisso, B. Meden, J. J. P. Friconneau et J. P. Martins, «Digital Assistances in remote operations for ITER Test Blanket System replacement: an experimental validation,» *Fusion Engineering and Design*, vol. 188, 2023.

[11] F. W. Heger, Assembly Planning in Constrained Environments: Building Structures with Multiple Mobile Robots, Pittsburgh: PhD in Robotics from Carnegie Mellon University, 2010.

[12] G. Konidaris, L. J. P. Kaelbling et T. Lozano-Perez, «From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning,» *Journal of Artificial Intelligence Research*, pp. 215-289, 2018.

[13] N. Somani, G. A., M. Rickert, A. Perzylo et A. Knoll, «Constraint-Based Task Programming with CAD Semantics : From Intuitive Specification to Real-Time Control,» chez *International Conference on Intelligent Robots and Systems (IROS)*, 2015.

[14] R. Bischoff, A. Kazi et M. Seyfarth, «The MORPHA Style Guide for Icon-Based Programming,»
chez *Workshop on Robot and Human Interactive Communication*, Berlin, 2002.

[15] G. F. Rossano, C. Martinez, M. Hedelind, S. Murphy et T. A. Fuhlbrigge, «Easy Robot Programming Concepts: An Industrial Perspective,» chez *IEEE International Conference on Automation Science and engineering*, Madison, 2013.

[16] N. Ritschel, V. Kovalenko, R. Holmes, R. Garcia et D. C. Shepherd, «Comparing Block-Based Programming Models for Two-Armed Robots,» *IEEE Transactions on Software Engineering*, vol. 48, pp. 1630-1643, 2022.

[17] F. A. Bravo, M. González et E. González, «A review of intuitive robot programming environments for educational purposes,» chez *IEEE 3rd Colombian Conference on Automatic Control (CCAC)*, Cartagena, 2017.

[18] A. Billard, S. Calinon, R. Dillmann et S. Schaal, «Robot programming by demonstration,» chez *Springer Handbook of Robotics*, 2008, pp. 1371-1394.

[19] H. Ravichandar, A. J. S. Polydoros, S. Chernova et A. Billard, «Recent Advances in Robot Learning from Demonstration,» *Annual Review of Control, Robotics, and Autonomous Systems*, pp. Vol.3:297-330, May 2020.

[20] A. Correia et L. A. Alexandre, «A survey of demonstration learning,» 2023.

[21] A. Acharya, R. Dubev et R. Alqasemi, «Programming By Demonstration using A Learning Based Approach: A Mini Review,» chez *36th Florida Conference on Recent Advances in Robotics*.

[22] Y. Kuniyoshi, M. Inaba et H. Inoue, «Learning by watching: extracting reusable task knowledge from visual observation of human performance,» *IEEE Transactions on Robotics and Automation*, vol. 10, pp. 799-822, 1994.

[23] S. K. Ong, A. Yew, N. K. Thangaivel et A. Nee, «Augmented reality-assisted robot programming system for industrial applications,» *Robotics and Computer Integrated Manufacturing*, vol. 61, 2020.
[24] A. Burghardt, D. Szybicki, P. Gierlak, K. Kurc, P. Pietruś et R. Cygan, «Programming of Industrial Robots Using Virtual Reality and Digital Twins,» *Applied sciences*, vol. 10, 2020.

[25] J. Lambrecht, M. Kleinsorge, M. Rosenstrauch et J. Krüger, «Spatial Programming for Industrial Robots through Task Demonstration,» *International Journal of Advanced Robotic Systems*, vol. 10, 2013.