# Digital assistances in remote operations for ITER Test Blanket System replacement: an experimental validation

Olivier David, Soumik Sarkar, Nolwenn Kammerer, Coline Nantermoz, Fabrice Mayran de Chamisso, Boris Meden, Jean-Pierre Friconneau, Jean-Pierre Martins

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Olivier David, Soumik Sarkar, Nolwenn Kammerer, Coline Nantermoz, Fabrice Mayran de Chamisso, et al.. Digital assistances in remote operations for ITER Test Blanket System replacement: an experimental validation. Fusion Engineering and Design, 2023, 188, pp.113425. 10.1016/j.fusengdes.2023.113425. cea-04089133

HAL Id: cea-04089133

https://cea.hal.science/cea-04089133

Submitted on 19 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Digital assistances in remote operations for ITER test blanket system replacement: An experimental validation

Olivier David<sup>a, *</sup>, Soumik Sarkar<sup>a, 1</sup>, Nolwenn Kammerer<sup>a</sup>, Coline Nantermoz<sup>a</sup>, Fabrice Mayran de Chamisso<sup>a</sup>, Boris Meden<sup>a</sup>, Jean-Pierre Friconneau<sup>b</sup>, Jean-Pierre Martins<sup>c</sup>

## ARTICLE INFO

### Keywords:

Remote handling

Teleoperation

Virtual guides

Collaborative robot

Maintenance

ITER

## ABSTRACT

Removal/reinstallation of all Test Blanket System (TBS) equipment present in the Port Cell is required during the ITER Long Term Shutdown. TBS shall be designed so that occupational radiation exposure can be As Low As Reasonably Achievable (ALARA) over the life of the plant to follow the ITER Policy. The expected level of radiation in this area still allows performing maintenance tasks hands-on. However, the cumulated dose could be significant for operators. Classical Dose Reduction Measure (DRM) is to deploy Remote Handling systems. Consequently, use of robotized equipment, remotely operated means or collaborative robotics, have been investigated. Taking advantage of new digital technologies such as digital assistances, is expected to help operators during complex remote operations under limited vision conditions. Experiments were performed on a set of three TBS maintenance representative tasks: remote visual inspection of DN80 pipe, dye penetrant testing operation on pipe and dexterity test. A panel of remote handling equipment operators of different skill level was selected and involved onto these tasks. The results prove without ambiguity that for all operators the quality of the task execution is significantly improved when using digital assistances

## 1. Introduction

Test Blanket Modules (TBM) [1] are part of the ITER experimental equipment that will need replacement during the ITER Long Term Shutdown (LTS). The replacement of a TBM requires removal/reinstallation of all Test Blanket System (TBS) equipment present in the Port Cell, including those in the Port Interspace called Pipe Forest (PF). TBS shall be designed so that occupational radiation exposure can be As Low As Reasonably Achievable (ALARA) over the life of the plant to follow the ITER Policy. The expected level of radiation in this area still allows performing maintenance tasks hands-on. However, the cumulated dose could be significant for operators and potential contamination would require usage of ventilated suits. Classical Dose Reduction Measure (DRM) for the operation under radiation exposure conditions is to deploy Remote Handling systems. However, with progress in robotics science, a single robotized equipment can be designed such as it: can operate fully autonomously, is remotely operated or even share a task and its workspace in collaboration with a human operator. Preliminary

work demonstrated that a single equipment can sweep a flat surface autonomously (full autonomous mode) while it has been previously programmed by demonstration by a human operator either directly at the workplace (collaborative robotics) or from a remote location with help of a master arm (remote handling). A work scheme where the robot and the human share the current maintenance task can therefore become interesting. In this scheme, the operator keeps the part of reflection and the know-how related to the task and will help for example to set up and secure the tooling. The robot will take care of the painful tasks (repetitive movements, carrying loads) and those that are time consuming while the operator returns to a zone where the radiation level is lower.

Industrial robotics today addresses problems for large or medium series where the most important criteria are the execution speed and the accuracy of an operation. In the industrial world, it is acceptable to take time in order to adjust machining parameters at a robot workstation until the part is manufactured with the requested accuracy. Each workstation is made of one or more robots, each robot doing a rather

<footnote><sup>a</sup> Université Paris-Saclay, CEA, List, Palaiseau F-91120, France</footnote>

<footnote><sup>b</sup> Commissariat à L'énergie Atomique et Aux Énergies Alternatives (CEA), Gif-Sur-Yvette F-91190, France</footnote>

<footnote><sup>c</sup> ITER Organization Route de Vinon-sur-Verdon, CS 90 046, St. Paul Lez Durance 13067 CEDEX, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: odavid@cea.fr (O. David).</footnote>

<footnote n="1">1 Master student from Ecole Centrale de Nantes in training period at CEA List at the time of that study.</footnote>
simple subtask of a more complex operation. But at the end the main aim is to gain in productivity, compared to the same service that would be carried out by human operators. In the context of ITER, the use of robots is above all a constraint linked to the radiological risk: either it is impossible for operators to do the work, or there is a desire to reduce operator exposure. This is a fundamental difference. In ITER, the robotic equipment first needs to reach the maintenance area, where it will need to complete a complex task in only one trial. Rather than being fast and very precise, ITER robotic equipment will need adaptability and to follow fault tolerant strategies in order to take into account all variabilities and uncertainties of the environment. In order to improve their adaptability, robots or cobots can today take advantage of digital assistances (also called virtual guides or virtual fixtures [2-5]) when the controller integrated this capability at the design phase. Virtual guides setting is done on the fly according to the execution context. In a kind of semi-automatic mode and under human supervision one can reduce the complexity of a movement to a simpler workspace adapted to the task in progress (e.g. moving along a flat surface, rotating around a pipe). Thanks to a user interface, the operator controls movements of the end-effector. The user interface can be as simple as a push button, or as complex as a 6D force feedback master arm. The main objective of such assistances is to relieve operators of complex movements of secondary importance and to allow them focusing onto the real objective of the task.

An experimental validation was carried out on a set of TBM replacement representative tasks, in order to verify the benefits of such digital assistances during complex remote operations under limited vision conditions. The current paper presents the results of three remote handling tasks carried out by a panel of remote handling equipment operators: remote visual inspection of DN80 pipe, dye penetrant testing operation on pipe and dexterity test. The operator panel involves persons of different skill levels that were tested onto the three tasks. After a short presentation of the tasks and tools involved during the trials, we will present and discuss the results, which prove without ambiguity that for all operators the quality of the task execution is significantly improved when using digital assistances

As said above hands-on intervention is still possible in the TBS area. Under these conditions and in the present context, there is no need to worry about the hardening of the electronics embedded in the robotic systems as these equipment are supposed to replace the operators when performing the tasks. This will of course not be the case throughout the whole ITER installation. However, the authors' intention is to have

demonstrated in this paper the usefulness of a set of generic software tools. It is therefore quite reasonable to think that deployment of these same tools on rad-hardened equipment will be possible without great difficulty.

## 2. Test facility and tools

### 2.1. Test facility

The test facility (see Fig. 1) is composed of a setup made of a Stäubli TX90 XL industrial manipulator located in front of a modular test bench. This TX90 is not a cobot and requires to operate into a cell. The test bench is mounted onto a lifting table with yaw and roll capabilities. Configuration of the test bench is adapted according to the experiment in progress. By setting the lifting table at a random position with respect to the manipulator, we simulate the relative positioning uncertainty of these two equipment. A laser scanner used for calibration purpose is placed in the robotic cell. Its exact position is not important. It only needs to be installed so that its working range encompasses both the manipulator and the sample to be tested. GoPro cameras are installed inside the cell and provide the operator with visual feedback on screens. The operator stands outside the cell and uses a Virtuose 6D master arm to move the manipulator. During the trials, a curtain can be drawn to hide the test bench. A screen also displays a virtual model of the scene that is updated by a real time physics engine (see description below). A computer is used to launch the interaction toolbox. The toolbox is used: to initiate and follow the calibration process, to enable/disable real-time collision monitoring of the manipulator as it moves through its environment and finally to set up and start virtual guides that will help the operator perform the task in a safe and reliable manner.

### 2.2. Robot controller

Industrial robot controllers are mainly designed to provide solutions in the very frozen world of mass production. They are inefficient to deal with: small series, processes with high variability or uncertainty, tasks needing gesture expertise or quick reconfiguration. The robot controller used during the trials was developed within the CEA-List Cortex framework an evolution of the TAO2000 controller [6] developed for force feedback applications. Rather than focusing mainly onto tele-operation applications, the Cortex framework focuses onto genericity, modularity and ergonomics of the tools with a special care onto safety.

<figure><img src="image_5.png" /><figcaption>Fig. 1. Test facility.</figcaption></figure>
Controllers created with Cortex have the capability to switch continuously between autonomous, semi-automatic or teleoperation operating modes. A task can therefore be composed of a mix between autonomous and human supervised operations requiring adaptability to the context. This is a real advantage in ITER like context. Cortex can also be used for robotics applications. Genrobot, a similar framework is today developed by Fusion for Energy.

## 2.3. Supervisory control

The high-level commands used during the tests are programmed, generated and managed in the Score framework (see [7] for detailed explanations). Score performs a real-time bilateral coupling of the robot used for the tests with a virtual robot evolving in a model of the scene represented in the XDE physical engine (see Fig. 2). It is thus possible to apply real efforts in the simulation and virtual efforts in the direction of real machines. In Score, interactions between objects are managed according to the context. Depending on the robot used, the tooling it carries and its environment, Score will propose to the operator the set of interactions he can perform between a tooling and the current target of this tool. The final choice to interact is up to the operator by pressing one of the buttons on the interface after parameterization. Similar strategies are used as for example in [8] where constraints are set up between objects for programming tasks thanks to CAD semantics. However, the advantage of Score relies on its real time interactivity.

## 2.4. Man machine interface for virtual guides

In Score framework, the interactions between the operator and the objects of the model or the environment are essentially generated with the help of virtual guides. A virtual guide can be seen as the local restriction of the mobility of a robot (or of the tool it carries) to the movements allowed by the mechanism to which it is connected. As described in Fig. 3, the end effector of the robot (or the tool) is attached thanks to a virtual spring damper to a hook. Mobility of the hook is constrained to movements allowed along the line S between two curvilinear abscissa boundaries $Smin$ and $Smax$. The space dimension of the line S is not limited to 1 degree of freedom, it is adapted according to the expected local movement. Virtual guides can therefore be considered as passive tools to help the operator to make the right gesture at the right place. In the form of a client connected to Score, the operator can use a

<figure><img src="image_6.png" /><figcaption>Fig. 2. SCORE Supervisory Control System.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 3. Virtual Guides.</figcaption></figure>

toolbox of virtual assistances to, for example: grasp/release an object, turn around an object, slide on a surface or move along a line...

At the operator's request, the guide can be set either in the tool's reference frame or in the target's. For example, he can decide to block all the rotations of his tool and to reduce his movements to a translation along the axis of his bolt runner in the case of a bolting operation. Similarly, he can also decide to grab an object, leaving for example the vertical rotation free in order to turn around it.

## 2.5. Calibration

Calibration aims at updating the position of the modelled geometrical elements with the reality. The accuracy of the assistances that will be generated directly depends on the accuracy of the calibration process. In order to represent the position error between the robot and the test fixture, each experiment starts with a random position between these two elements. Calibration is therefore an essential phase of our experiments. Several calibration techniques can be used: touch probes, 2D cameras, point cloud measurements... During these trials we used a Photoneo 3D laser scanner to create a 3D point cloud of the scene. Once the laser scanner has flashed the scene, the calibration process is divided in two steps. First, identify the position of the robot with respect to the frame of the laser scanner. Second, find the position of the scene objects with respect to the same frame. Then, it is possible to compute the relative position between one object and the robot. Initial position of the laser scanner does not need to be known with accuracy. It only needs to remain the same during the two steps of the calibration process. Positions of the objects of the model within the point cloud are estimated with probabilistic algorithms (see [9–11]). Thanks to the meshes from each object of the model, the algorithm identifies the best subdivision of the point cloud where the object should be located. Best estimates of the 6D pose (position + orientation) are proposed for each element. On the left picture of Fig. 4, one can see on a 2D output image, the proposed position for the segment 2 of the Stäubli manipulator (green color indicates good matching, blue and red are respectively for medium and low quality) and on the right picture the proposed solutions for objects of the model. Once position of the objects relative to manipulator are known, position of the objects are updated in the model considering the position of the manipulator is fixed and never moves.

## 2.6. Video feedback

A set of GoPro Hero7 cameras were used to film the experiments from different viewpoints. The viewpoints were fixed. The videos were
<figure><img src="image_8.png" /><figcaption>Fig. 4. Example of calibration output images.</figcaption></figure>

streamed in real time using the phone and tablet application provided by GoPro. The tablet screen was connected to a PC. Finally, the PC streamed the video flux onto the 42inches high definition screen monitor of Fig. 1. The distance between, the operator and the screen was approximately 2 m. Depending on the trial, 2 to 3 viewpoints were selected and displayed simultaneously onto the screen for the entire duration of the trial. It was not possible to switch between viewpoints during the trial. For prototyping, this organization is simple. It is set up with a wireless network or simple cable connectors. However, it has one major drawback. Indeed, the latency of the video stream is greater than one second (may rise up to two seconds, essentially coming from GoPro app). This is disturbing for the operator who sees an image that is systematically delayed.

The resolution of each view could be set from the app. However we had to stay close to 360p (640 × 360) to balance the latency and quality. This is a low value compared to what could be obtained in a more industrial way with wired cameras.

# 3. Description of the experiments

## 3.1. Operational context

The design of the TBS is still in progress. At this time it is not possible to establish an exact list of maintenance operations that will have to be carried out. Tasks of this study were therefore defined according to the specificities of the TBM context and respond to needs identified as generic and representative. The following specificities can be mentioned:

* The robotic tooling has to move to the location of the intervention and therefore the positioning uncertainty with respect to the object to be maintained is potentially high.
* The environment is constrained.

The operations expected in this area may be related to safety: tritium-monitoring, sampling, decontamination. They can be more specific to the context as well: bolting/unbolting of flanges, disconnection/ connection of TBM piping. Some tasks (e.g. welding a pipe) add a multitude of additional tasks that must also be implemented: cleaning of junctions, inert gas, preheating, control.

For this reason, we do not demonstrate in this paper the feasibility of a particular task and rather aim at opening the minds of the designers on the type of task it is possible to realize with a manipulator in the TBM context. These experiments therefore want to demonstrate the relevance and the genericity of a set of coherent tools to assist an operator during

the maintenance, either by means of a robot during the execution of tasks (in an autonomous way or in teleoperation) or by means of a cobot for the tasks shared between the man and the machine.

## 3.2. Operator panel

The objective of the tests was to verify that it is possible to provide simple assistance to an operator during the execution of a task in tele-operation or cobotics. The aim of the assistance is to help the operator to accomplish a complex technical gesture while leaving him the supervision of the task. The operator can thus focus on the real objective of the task (e.g. inspecting a pipe) and not on the secondary tasks: not colliding with the environment, maintaining a constant orientation of a tooling... The benefit brought by these assistances was evaluated by subjecting a panel of eight operators to the same set of tasks. The panel was constituted by trying to have a fairly homogeneous distribution between beginner, intermediate and experienced operators who each had to perform a dexterity test, the visual inspection of a pipe as well as the simulation of a dye-penetrant deposit. At the end of each test, the operators must fill out a questionnaire that allows them to assess their feelings about the task they have just completed. The questions are adapted from the NASA TLX standards aiming to evaluate the workload perceived by the operator in order to verify the efficiency of a task. In our case, the operator is asked to evaluate his/her perception of the work performed on the basis of questions such as:

* Does he/she feel that he/she has succeeded in the proposed task?
* Was it physically demanding?
* Was it mentally demanding?
* Was it frustrating?
* What is your opinion about the duration of the task?
* ...

## 3.3. Dexterity test

This task is intended to assess the dexterity that can be expected of an operator in a teleoperation task. It is not specifically representative of the TBS. It is, however, sufficiently generic that it can be considered relevant to the entire facility. In this task, the operator must present a tool with a 40 mm inner diameter ring effector (see Fig. 5) in front of a curved pipe. The operator is then asked to progress along the curved part of the pipe until it is rotated 180° and then to descend along a vertical section of the pipe. A buzzer rings each time the operator hits the pipe with his tool. The operator's task is to minimize the number of hits,
<figure><img src="image_9.png" /><figcaption>Fig. 5. Dexterity test setup and parametering.</figcaption></figure>

while traveling along the pipe. The total number of hits are recorded. If an operator makes more than fifty hits during the first 180° rotation, the test is stopped. If the operator reaches the point E, the test is conducted backward to the end regardless of the number of final hits (this explains results in which operators have more than 50 hits). Each operator is asked to perform the task with and without assistances but also with direct view on the mock up and when the curtain is drawn. A comparison of the number of hits with and without assistances should provide information on the usefulness of the digital assistance.

Virtual Guide Parametering

<figure><img src="image_10.png" /></figure>

The digital assistance is set up as shown in the right diagram of Fig. 5. Thanks to the former calibration step, all positions and geometric information are extracted from Score's digital model with accuracy. When the user asks for the creation of the virtual guide, the current 6D pose of the ring effector's center is represented as point A. The virtual guide is created between A and the "safe" intermediate position and orientation A'. This guide is a spherical linear interpolation (SLERP) for the three rotations and a linear interpolation for two translations. The operator has the capability to move freely along this guide back and forward. In

<figure><img src="image_11.png" /><figcaption>Fig. 6. Camera inspection setup and parametering.</figcaption></figure>
A' collision monitoring is deactivated and a new linear virtual guide along Y axis (in the ring effector local frame) is set up in order to reach point B. When point B is reached, the new virtual guide set up is a 1D rotation around the Z axis and centered in point C. Stops of the guide are such that a rotation of 180° is achievable. As a result, the ring effector placed in B will follow the circular curved shape of the pipe until the operator reaches point D. At point D, a linear virtual guide along the Y-axis is started toward point E.

### 3.4. Pipe inspection

As lot of maintenance operation in the TBS area are expected to be made on pipes, this task is intended to simulate the visual inspection of a DN80 pipe. A GoPro camera is mounted onto the wrist of the Stäubli TX90 robot (see Fig. 6). The operator is first asked to reach a starting point located approximately 10 cm above the pipe surface, then to start rotating around the pipe. The objectives of the operator are to maintain the initial distance constant, provide good quality and stable images for the inspection and to achieve a 360° rotation if possible. Each operator is asked to perform the task with and without assistances but also with direct view on the mock up and when the curtain is drawn. In order to help the operator to locate himself, marks were made every 45° on the pipe in the inspected area.

The digital assistance is set up as shown in the right diagram of Fig. 6. When the user asks for the creation of the virtual guide, the current 6D pose of the camera eye is located in A. An approach virtual guide similar to that of the previous experiment (SLERP + translation) is created between point A and the position B. When point B is reached, the new virtual guide set up in B is a 1D rotation around the pipe axis and centered in point C. As a result, the camera eye located in B will remain normal to the pipe and at the same distance of its surface while the operator moves the master arm.

What is evaluated during these tests is the operator's ability to consistently provide a high quality image.

### 3.5. Dye penetrant simulation

The goal of this trial is to simulate a non-destructive dye penetrant test to detect potential cracks in welded joints. In the real task, a liquid penetrant is sprayed on a welded joint, preferably at 30 cm from the surface (good practice recommendation). The surface area swept by the manipulator is therefore larger than for the inspection task. Due to the space constraints of the interface 2A area, performing such an operation as requested by the norm is likely to become difficult. Addressing this task or at least a similar task therefore becomes useful for identifying difficulties. On our test bench, the TX90 effector is equipped with a snow spray bottle (for visual effect, see Fig. 7). On demand, a second operator presses on a button to start spraying (remote control with servomotor). The tested operator is asked to keep: a constant speed during the movements (uniformity of the film), a constant orientation and distance of the nozzle with respect to the pipe (spraying toward the pipe). Two different operations were carried out: spraying around the pipe (circular movement) and spraying along a generatrix of the pipe. As for the two other tasks, operators had to carry out the tests with and without assistances. However, due to the complexity of the task and in order to avoid damage to the machine by novice operators, some of them were only asked to carry out operations with direct viewing onto the mock-up.

Configuration of the virtual guide is similar to that of the visual inspection with the spray nozzle replacing the camera eye. We only updated to 30 cm the 10 cm distance and added a linear degree of freedom along the pipe's axis. The resulting authorized area when connected to the guide is a cylinder located at a 30 cm distance from the pipe surface. With smart adjustments of the stops of the rotation and/or the linear degree of freedom, we can constrain the operator's movements onto a circle or onto a line.

We evaluated the capability of the operator to maintain the correct distance and orientation and move at a constant speed. In fact, we want to know if the operator can follow multiple instructions at a time.

<figure><img src="image_12.png" /><figcaption>Fig. 7. Dye penetrant testing setup configuration.</figcaption></figure>
# 4. Results

An accompanying video of the three tests is provided with the digital version of this article.

## 4.1. General comments on all tests

These tests were carried out with a set of 8 operators in which there are 3 different levels of qualification. Within a same level, the differences between operators can also be very significant. The dispersion of the results is often such that it is difficult to represent on the same graph a quantitative result which would relate to all the operators. When quantitative results have been presented, we have generally chosen to represent only the results within the population of advanced operators. It is indeed this type of operator profile that will later be responsible for carrying out the actual operations. We felt that it was above all for these profiles that the interest of the tools should be demonstrated. Moreover, the main aim of these tests was above all to compare two operating modes: the first based on traditional teleoperation techniques, the second using a digital model to generate operator assistance. We therefore tried primarily to assess whether there was an improvement for each operator in terms of quality, comfort and mental workload rather than seeking to evaluate individual performance. In addition, during these trials, a six degree of freedom force feedback master arm was used. This is arguably the most effective and efficient tool for traditional teleoperation. As we will see in the discussion following these results, in a framework where the operation is supervised and the operator is assisted by models, the master arm is probably not always the optimal interface. It is therefore premature at this stage to draw conclusions on the basis of the duration of an operation or the appearance of the result obtained, and we have deemed it preferable to look at the progress between the "with assistance" and "without assistance" situations

## 4.2. Dexterity test results

Only four of our eight operators were able to complete all four variants of the dexterity tests. Without any surprise, all three operators at the advanced level are among them and the last one is at the intermediate level. The reasons of failure is clearly the lack of experience (number of touches exceed maximum allowed) or the lack of confidence. The test without any assistance and with curtain drawn is clearly the most difficult for everybody. However, beginner operators were also unable to complete the dexterity test without assistance for both viewing configuration. All operators reported being frustrated and needing extreme concentration in order to complete the tests without any assistance. In assisted mode, all operators reported a decrease of the mental load even if for two of them the level remained medium to high.

Qualification of the operator has a high influence on the time spent by operators to travel from the entry point B up to final point E and come back. In addition, some beginner and intermediate operator were not able to complete the task without assistance. Therefore, dispersion of the results is high. However, if we look at the experienced operators only, the results are those of Table 1. The sample is too small to draw conclusions with high confidence. However, it seems that the viewing conditions do not really influence the time spent by operators to complete the task. On the other hand, using assistance clearly decrease the

task duration for all operators by a factor of 2 compared to the same situation without assistance.

The most interesting result is related to the quality of the execution. Indeed, none of the operators was able to accomplish the task in unassisted mode without touching the pipe several dozen times over the entire course. And even more, as we can see in Table 1, the number of hits in unassisted mode is 20 times higher than in assisted mode for the most skilled operators. Looking at the video footage of the tests, we see a great improvement of the management of tooling orientations when digital assistance is on. We also note that all the operators, whatever their skill ability, were able to pass the test with the help of digital assistance.

Even in assisted mode, advanced operators still hit the pipe during the task execution. One of the primary causes of these remaining hits is most likely the error that is made during calibration. Indeed, the guide relies on the position provided by the model which is directly affected by the uncertainty of the calibration process. Furthermore, the assistance helps the operator to follow a preferred trajectory but it does not impose the position. The operator is left free to deviate from this position if he deems it necessary for the needs of the operation. This function is essential because it guarantees adaptability to the task. Doing without this function would be to launch an automatic procedure. The disadvantage is that sometimes the operator deviates from the proposed path in the wrong way resulting in undesired hits. We have noticed that there are more touches during assisted modes when the slave gets closer to the singularities (manipulation with the master arm is then more complicated), but also when the operator disengages and then re-engages the master arm to find a more ergonomic working position. Furthermore, when the operator hits the pipe, it is sometimes complicated to know exactly where and in what direction the touch is made, due to the effort of the guide and the latency of the video feedback. Correction of the bad movement can therefore be slow to be effective and cause several touches to be made almost simultaneously.

## 4.3. Camera inspection test results

For all the inspections performed with the help of the virtual guides, it is easy to divide the test into: a first approach phase in which the operator travels a variable distance from point A to point B (see Fig. 6) and a second inspection phase during which the operator turns around the pipe. During the unassisted tests, the start of this second phase is left to the operator's judgment. Moreover, as the operators do not all start from the same point, comparisons on this phase of the test are not very meaningful. Since the beginner operators had never handled a robot with a master arm before this trial, they were not asked to perform the operations when the curtain was drawn. The level of intermediate operators produced heterogeneous results, and due to lack of confidence, one operator was unwilling to perform the operations when not in direct vision. The results in Table 2 therefore compare the execution times of the advanced operators only when virtual assistance is on. The last line provides the duration of the whole operation (approach + inspection) without assistance when operator had direct viewing onto the mock-up. When the curtain was drawn and the assistance were not applied, none of the advanced operators was able to provide a complete inspection. Since the difficulties were not encountered at the same location, the results are difficult to compare and are therefore not represented in this

<table><caption>**Table 1**
Time (in seconds) spent and number of hits for an advanced operator to travel from point B to point E back and forward.</caption><thead><tr><th rowspan="2">Assistance</th><th rowspan="2">Viewing</th><th>Operator 1</th><th>Operator. 2</th><th>Operator. 3</th><th>Mean</th></tr><tr><th>Time (s)</th><th>Time (s)</th><th>Time (s)</th><th>Time (s)</th></tr></thead><tbody><tr><td>No</td><td>Direct</td><td>204</td><td>221</td><td>173</td><td>199</td></tr><tr><td>No</td><td>Curtain</td><td>308</td><td>120</td><td>217</td><td>215</td></tr><tr><td>Yes</td><td>Direct</td><td>100</td><td>75</td><td>88</td><td>87</td></tr><tr><td>Yes</td><td>Curtain</td><td>86</td><td>81</td><td>99</td><td>89</td></tr></tbody></table>
<table><caption>**Table 2**
Comparison of time (in seconds) spent by advanced operators to turn around the pipe.</caption><thead><tr><th>Assistance</th><th>Viewing</th><th>Operator 1</th><th>Operator. 2</th><th>Operator. 3</th><th>Mean</th></tr></thead><tbody><tr><td>Yes</td><td>Direct</td><td>100</td><td>75</td><td>93</td><td>90</td></tr><tr><td>Yes</td><td>Curtain</td><td>105</td><td>70</td><td>109</td><td>95</td></tr><tr><td colspan="6">Duration below is for total inspection time (no approach phase)</td></tr><tr><td>No</td><td>Direct</td><td>94</td><td>95</td><td>96</td><td>95</td></tr></tbody></table>

table.

As we can notice in the assisted tests, the inspection time depends more on the operator than on the viewing conditions, probably because the understanding of the instruction "to provide a good quality image" is interpreted differently from one operator to another. Fig. 8 provides a set of on-board camera views for a novice operator, an intermediate operator, and an experienced operator. These views are the result of the unassisted inspections. They were extracted during the inspection phase. That is to say, at the moment when the operator considers that he respects the distance instruction and that the view he provides is exploitable. We can clearly see that the higher the skill level of the operator, the better the point of view he provides. In Fig. 9, the camera views are taken from the performances of the same operators but this time they could each benefit from the assistance functions and concentrate on the actual inspection task. It can be seen that the result is not only more homogeneous among the operators but that the quality of the images provided is much better than what the advanced operator could provide without assistance (real picture, no zooming was active). With the help of the virtual guide, the operators were not only able to position themselves better at a distance from the pipe, but also to keep this

distance constant during the inspection phase which prove the added value of the guide.

### 4.4. Dye penetrant simulation test results

Regardless of the operator, the limits of the robot are quickly reached when simulating the deposition of the penetrant film on the circumference of the pipe. Maintaining the 30 cm distance between the pipe and the nozzle while keeping the orientation of the nozzle on a normal to the pipe surface makes the robot work on a large percentage of its working space. The relative position of the robot with respect to the pipe becomes a critical parameter. Moreover, the complexity of the movements regularly leads the operators to find themselves in singular joint configurations that are always tricky to overcome. Although the help provided by the virtual guides is a real added value, no operator was able to correctly rotate around the pipe at constant speed during these tests. For this reason, no operator has been able to deposit a film of constant thickness on the pipe. The procedure therefore requires significant improvements. All operators complained about the excessive mental load required to perform the operations and felt exhausted by the task. The contribution of the virtual guides on this type of operation is easier to prove on the simulation of the deposit along a generatrix of the pipe. As for the camera inspection, the analysis of the videos allows to show without ambiguity that maintaining the 30 cm distance is greatly facilitated by the use of the assistances. However, the most spectacular result is the preservation of the orientation of the nozzle with respect to the normal of the pipe outer surface. On the left side of Fig. 10, it can be seen that operators of all skill levels are unable to correctly assess the depth of the tool relative to the pipe. They also experienced difficulties

<figure><img src="image_13.png" /><figcaption>* Total Time for Unassisted Modes cannot be separated into inspection and auxiliary movements, as all operators start from a random position in space and perform approach and reorientation as per their convenience.</figcaption></figure>

<caption>**Fig. 8.** Camera inspection without assistance. Operator output.</caption>
<figure><img src="image_14.png" /><figcaption>Fig. 9. Camera inspection with assistance. Operator output.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 10. Dye penetrant simulation without assistance (left) with assistance (right). Operator output.</figcaption></figure>
in adjusting the orientation of the nozzle relative to the normal to the pipe surface. It may be difficult for an external observer to catch the real direction of the nozzle. Another option is then to pay a careful attention to the orientation of all manipulator's joints from the base up to the effector on all front views of Fig. 10. We can see during the unassisted mode, that for a similar Cartesian position the articular configuration differs from one operator to another (e.g. axis 4 orientation for beginner and intermediate operator) while during the same trials with virtual guides, all operators maintained a consistent orientation for all joints. And this even in situations where the operator had no direct viewing onto the pipe (curtain was drawn). As a matter of fact, and even if they had direct viewing onto the experiment (task should be easier), none of them was able to spray toward the pipe when they were not assisted. The actual position of the line was 15° ahead of the vertical plane which is perfectly reflected on the Fig. 10 by the direction of the spray.

For the moment, these tests do not allow to demonstrate that we are able to perform easily a dye penetrant testing task with such tools. On the other hand, these tests clearly show an improvement in the operator's gestures as well as in the reduction of the mental load between the situation with and the situation without assistance. Two major difficulties for the operator were noted. These difficulties have a direct impact on the result. On one hand, the management of joint singularities can be complex with a master arm with 6 degrees of freedom, and on the other hand, the respect of several instructions at the same time (maintain a constant speed, stay at 30 cm, maintain a correct orientation, etc...) is very difficult. As it will be discussed in the following paragraph, semi-automatic modes in which the operator interface is simplified by reducing the number of degrees of freedom could be a serious answer to consider.

## 5. Discussion

All users agreed that the feedback from camera views were of poor quality. The image resolution was poor and the latency was high (see 2.6). As a matter of fact, users therefore focused on the 3D view of the scene and, when the test allowed it, on the direct view. In experiments with direct vision and without assistance, experienced operators clearly have an asset for the success of the task. Indeed, experienced operators are better able to locate themselves in the task environment, have a better coordination of the movements between master and slave arms and thus manage to accomplish the requested task more easily and without fear. The difference is less obvious during blind operations. The poor quality of the video feedback forced all operators to concentrate on the 3D model of the scene. The 3D model gives operators the impression that they will be able to succeed at the task. The confidence and expectations placed on the model are high.

In direct vision tests without assistances, some operators admitted to having a false feeling of quality during their task execution. This was particularly true when the operator was asked to follow the pipe generatrix in a straight line over a certain distance (dye penetrant simulation). The operator focuses on the direct view without looking at the ancillary views, misjudges depth perception, and moves along a line that is not centered on the pipe in the belief that he is performing his job correctly. This prove that dealing with more than one parameter at a time during a task should be avoided whenever possible.

For all the users, it is clear that the task is much easier when performed with assistance than without. The difference is not on the duration of the task but on the feeling of having succeeded in the task with a mental load that remains acceptable. These observations are true for the direct vision trials and especially for the blind trials. For all operators, the assistance is easy to use and quite instinctive. This is even more obvious on tasks such as the dexterity test. It gives the impression that one can be more precise by focusing on small movements.

One of the important things to remember is that regardless of the skill level of the operator, the quality of execution in the guided modes has always been acceptable

To work properly, an automatic procedure must either repeat a previously learned trajectory, close the control loop with sensors or use a compliant effector. This requires tuning and/or specific tool or sensor for each task. As explained above, in ITER the first trial must be successful and storage area for a large number of tool is hard to provide when space is constrained. Consequently the number of tools will be limited. In ITER it will be difficult to repeat pre recorded sequence. The reasons are:

* We operate in a changing environment.
* There are uncertainties on this environment.
* The robot has its own positioning error.
* There are also underestimated behavior such as flexibilities of the robotic or maintained equipment (e.g. flexibility of pipes).

For all these reasons, we believe that automatic modes will very hardly provide the requested adaptability and human supervision during operations is therefore mandatory. However, in such a situation it could be interesting to investigate the interest of operating with semi-automatic modes. Indeed, some operators would like to be able to control the guide in speed and not in position with the six degrees of freedom master arm (that is sometimes a tricky interface to handle). This feature is particularly requested in our tests by intermediate and advanced operators who feel that inspection or penetrant testing tasks would be better executed, using simpler interface such as a button or a joystick with direct control of the speed to travel back and forward along the guide. Small local adjustments to the task would be made through a second interface. The operator relies on the assistance to provide a best estimate of the trajectory, while the adaptability to the task is achieved by the operator adjusting the position with the second interface while maintaining control over the progress along the guide. Indeed, this seems to be enough to carry out the task under human supervision and with the same level of accuracy in many situations where force feedback is not an essential requirement.

For many operators the visualization of the 3D view is more relevant than the real camera feedback because it could be oriented and zoomed during the task execution. This is perhaps a false problem linked to the experimental conditions (video feedback were live streams from GoPro with significant latency). Indeed, it should be possible to use real pan-tilt cameras equipped with a zoom and to change also the real points of view. The only real advantage of the model is that it has a workspace without any limits, meaning not only the viewing directions can be updated but also the viewpoints. However, the model will never be representative of the exact conditions and will only reflect what was inserted into the model. Any unexpected event or situation cannot be reflected by the model. A mix of real views and model views is definitely needed.

The vast majority of the operators would have liked to be able to create additional 3D views representing the scene from different angles and would have liked to be able to define their own viewpoint.

It is noticed that when the viewpoint is updated and the orientation changes, it becomes more complicated for the operator to reorient himself, and to resynchronize the movements of the master arm with the view. This problem is probably related to the observation/manipulation frame in which the master arm operates. When the manipulation frame is no longer identical to the proposed view, coordination of the movements then becomes more complex for operators and their mental load increases. The point of view of the 3D camera is known at all times. It is therefore possible to add a feature changing the observation point when the view is updated.

In complement, an extra feature giving the capability to simulate in a virtual view what is currently viewed by the real camera would be very much appreciated for comparison and to secure the task execution.

## 6. Conclusion

Three tests were performed: visual inspection of a DN80 pipe,
simulation of a dye penetrant testing operation and a dexterity test. In order to consolidate the method and the approach, each test was carried out by a panel of 8 operators with different profiles ranging from complete beginners in remote operation to advanced users.

With the help of a set of integrated tools, both hardware and software, we were able to demonstrate during these tests that it is possible to perform relatively complex operations with a rather remarkable quality of result thanks to virtual guides assistance (set up on the fly and according to the context) constraining movements of the operator to a local simpler mechanism.

These tests have also enabled us to validate the principle of each of the key components used for these tests: real-time control of a robot in remote operation, real-time interaction of the robot with its digital avatar, non-contact calibration method of the robot in its environment, generation of operator assistance that can be parameterized and adapted according to the current context.

The software suite used is well perceived by the operators and above all it provides them with a comfort in the execution of their tasks which undeniably reinforces their confidence in their ability to carry out their mission. The benefit of operator assistance on task execution duration remains questionable. However, it is clearly evident that the quality of the result is greatly improved, even for experienced operators. It should also be noted that the frustration of the operator is extremely reduced by the contribution of these aids.

Based on an adaptive strategy, the interactive adjustment of the assistances and of some parameters offers more freedom to the operator in the execution of his task. The proposed tools thus allow to address situations with a certain variability which depends on the analysis of the operator but also on the progress of the task. This is a notable advantage when working in an environment that is changing or that cannot be known with the sub-millimeter precision that would be required if the entire operations were to be carried out automatically. Working by mixing data from the robot controllers with data from the real-time models also provides the operator with additional functionalities that have been appreciated by the users. The operation of the tools is very satisfactory even if it is obvious that their robustness still needs to be qualified and evaluated during other tests or with dedicated campaigns.

The aim of this work was to propose a set of tools and a method to help address a particular maintenance issue. Detailed design is still subject to changes. It is therefore very difficult to define precisely the tasks to be carried out. This work therefore focused on producing a set of generic tools that will need to be adapted more specifically for each task when they are better known.

## Disclaimer

The views and opinions expressed herein do not necessarily reflect those of the ITER Organization.

## Declaration of Competing Interest

Fabrice Mayran de Chamisso and Boris Meden have patent

#WO2020/201392, 2019 Method, computer program and system for identifying an object instance in a three-dimensional scene licensed to Tridimeo. Fabrice Mayran de Chamisso and Boris Meden have patent # WO2020/065177, 2018 Method, computer program and system for object detection and location in a three-dimensional scene licensed to Tridimeo. Fabrice Mayran de Chamisso and Boris Meden have patent #Brevet EP 4 012 651 A1, 2020 Procédés, programme d'ordinateur et système pour l'identification, la détection et la localisation d'une instance d'objet dans une scène tridimensionnelle licensed to Tridimeo.

## Data availability

Data will be made available on request.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.fusengdes.2023.113425.

## References

[1] L.M. Giancarli, X. Bravo, C. Seungyon, M. Ferrari, T. Hayashi, B.Y. Kim, A. Leal-Pereira, J.P. Martins, M. Merola, R. Pascal, I. Schneiderova, Q. Sheng, A. Sircar, Y. Strebkov, J. van der Laan, A. Ying, Overview of recent ITER TBM program activities, Fusion Eng. Des. 158 (2020), 111674 n.

[2] L.B. Rosenberg, Virtual fixtures: perceptual tools for telerobotic manipulation, in: Proceedings of the IEEE Virtual Reality Annual International Symposium, 1993.

[3] L.D. Joly, C. Andriot, Imposing motion constraints to a force reflecting telerobot through real-time simulation of a virtual mechanism, in: Proceedings of the IEEE International Conference on Robotics and Automation, 1995.

[4] S.S. Restrepo, G. Raiola, P. Chevalier, X. Lamy et D. Sidobre, Iterative virtual guides programming for human-robot comanipulation, in: Proceedings of the IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Munich, 2017.

[5] S.S. Restrepo, Intuitive, Iterative and Assisted Virtual Guides Programming For Human-Robot Comanipulation, Université Toulouse 3 Paul Sabatier (UT3 Paul Sabatier), 2018. PHD Thesis from.

[6] F. Geffard, P. Garrec, G. Piolain, M.A. Brudieu, J.F. Thro, A. Coudray, E. Lelann, TAO2000 V2 computer-assisted force feedback telemanipulators used as maintenance and production tools at the AREVA NC-La Hague fuel recycling plant, J. Field Robot. 29 (2012) 161-174.

[7] O. David, F.X. Russotto, M. Da Silva Simoes, Y. Measson, Collision avoidance, virtual guides and advanced supervisory control teleoperation techniques for high-tech construction: framework design, Autom. Constr. 44 (2014) 63-72.

[8] N. Somani, A. Gaschler, R. Markus, A. Perzylo et A. Knoll, Constraint-based task programming with CAD semantics: from intuitive specification to real time control, in: Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 2015.

[9] F. Mayran de Chamisso, M. Tamaazousti, B. Meden, Method, computer program and system for identifying an object instance in a three-dimensional scene, Patent number WO2020/201392 (2019) 2019.

[10] Mayran de Chamisso F., Tamaazousti et B. Meden M., Method, computer program and system for object detection and location in a three-dimensional scene. Patent number WO2020/065177, 2018.

[11] Mayran de Chamisso F., Tamaazousti et B. Meden M., Procédés, programme d'ordinateur et système pour l'identification, la détection et la localisation d'une instance d'objet dans une scène tridimensionnelle. Patent number EP 4 012 651 A1, 2020.