# Stiffness regulation co-pilot in bilateral teleimpedance control: a preliminary user study

Pedro Gomez Hernandez, Jonas Mariager Jakobsen, Claudio Pacchierotti, Francesco Chinello, Cheng Fang

► To cite this version:

Pedro Gomez Hernandez, Jonas Mariager Jakobsen, Claudio Pacchierotti, Francesco Chinello, Cheng Fang. Stiffness regulation co-pilot in bilateral teleimpedance control: a preliminary user study. ICRA 2025 - IEEE International Conference on Robotics and Automation., May 2025, Atlanta (Georgia), United States. pp.5622-5628, 10.1109/ICRA55743.2025.11127810. hal-04981046

HAL Id: hal-04981046

https://hal.science/hal-04981046v1

Submitted on 6 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Stiffness Regulation Co-pilot in Bilateral Teleimpedance Control:
A Preliminary User Study

Pedro Gomez Hernandez<fnref n="1" />, Jonas Mariager Jakobsen<fnref n="2" />, Claudio Pacchierotti<fnref n="3" />, Francesco Chinello<fnref n="1" />, Cheng Fang<fnref n="2" />*

**Abstract**— Variable stiffness of a remote robot is crucial for a teleoperation system to deal with challenging tasks. External stiffness command interfaces have emerged as a promising solution to regulating the remote robot stiffness because of the benefits of their accuracy, ergonomics, and avoidance of the “coupling effect” that usually exists in muscle activity-based stiffness interfaces. However, the use of an external stiffness command interface requires good coordination between two limbs of an operator, which take care of the teleoperation task and the stiffness regulation task, respectively, at the same time, which is demanding for novice operators in dynamic situations necessitating agile and timely stiffness adjustments. In this paper, a new concept of Stiffness Regulation Co-pilot was proposed to facilitate the use of these interfaces. A co-pilot is a virtual agent that consists of a Stiffness Regulation Policy, which infers a reasonable stiffness regulation action from the task performance, and a feedback modality, which conveys the suggested stiffness regulation action to the operator. A preliminary user study was conducted to evaluate the efficacy of the co-pilot and the effect of different modalities of the co-pilot. The results showed that the cutaneous feedback or combined with another modality can potentially improve the task performance of the system and reduce the cognitive load of the operator compared to a teleoperation system without using the co-pilot.

## I. INTRODUCTION

Teleoperation enables human operators to control robots in inaccessible or hazardous environments such as offshore wind turbines, nuclear plants, and deep sea or space exploration sites [1]. These remote-controlled robots can perform complex tasks beyond the capabilities of fully autonomous systems. Bilateral teleoperation systems enhance performance by incorporating force feedback, allowing operators to sense contact forces through haptic devices [2]. Haptic feedback is crucial when interacting with unknown objects in unstructured environments [3], where high stiffness is required for accurate trajectory tracking or exerting large forces, while low stiffness is necessary when manipulating delicate objects to prevent damage. When operators can adjust the remote robot's impedance in real-time, the system evolves into a bilateral teleimpedance system [4], offering greater flexibility and control in diverse operational scenarios.

Proper impedance regulation is crucial for optimal task performance. In variable impedance control, two main approa-

ches exist [5]: demonstration-driven imitation learning and reward-driven reinforcement learning in simulations with sim- to-real transfer These methods, primarily used for autonomous robots, often require physical interaction or prior simulations [5]. However, teleimpedance systems necessitate a different approach: an impedance interface allowing real-time operator control without direct robot contact [6].

Human arm impedance regulation serves as inspiration for designing teleimpedance system interfaces. While traditional perturbation-based methods can estimate human arm impe- dance [7], their online application in teleimpedance control may interfere with task performance. Alternative approaches leverage the relationship between muscle co-activation, arm posture, and endpoint stiffness [8]. These methods use muscle activity sensors to capture muscle co-activation and motion capture systems to obtain arm posture, estimating the commanded stiffness for the remote robot [9]–[13]. However, these approaches face challenges due to complex estimation models [14] and potential sensor artifacts. Grip force-based interfaces offer a simpler alternative, exploiting the correlation between grip force and endpoint impedance [15], [16]. These systems use grip force sensors on haptic devices to regulate remote robot stiffness [17], [18]. Despite their simplicity, they present limitations such as operator fatigue during prolonged use and the “coupling effect,” where involuntary reflexes from force feedback affect commanded stiffness [19].

To address these issues, external stiffness command inter- faces have emerged as promising solutions [6]. Examples include touchscreen tablets [20] and circular rotating discs [21]. These interfaces, operated by the secondary hand or foot, allow manipulation of a stiffness ellipsoid or ellipse<fnref n="1" /> in terms of size, shape, and orientation, while the dominant hand controls the master haptic device. This approach offers a more robust and accurate method for stiffness commanding in bilateral teleimpedance control, potentially overcoming the limitations of muscle activity and grip force-based interfaces.

Regulating stiffness using a secondary limb while per- forming a primary task with the dominant arm presents significant challenges. This approach demands a deep under- standing of how stiffness regulation affects task performance and requires excellent inter-limb coordination. For novice operators, these requirements can be particularly demanding, especially in dynamic situations necessitating efficient and

<footnote n="1">¹Business Development and Technology, Aarhus University, Denmark.</footnote>

<footnote n="2">²SDU Robotics, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Denmark.</footnote>

<footnote n="3">³CNRS, The University of Rennes, INRIA, IRISA – Rennes, France.</footnote>

<footnote>This work is supported by the Innovation Fund Denmark Grand Solutions project, SENSIBLE (No. 2081-00031B). (*Corresponding author: Cheng Fang, e-mail: chfa@mmmi.sdu.dk)</footnote>

<footnote n="1">¹A stiffness matrix is symmetric positive definite, visualized as an ellipsoid (3D) or ellipse (2D) [22].</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1: Schematic diagram of a bilateral teleimpedance control system with a stiffness regulation co-pilot. The human operator sends motion commands to a remote robot by operating a master robot device with one hand and regulates the impedance commands of the remote robot by the tracked movement of the other hand. The stiffness regulation co-pilot receives the control and feedback signals as input and outputs the stiffness regulation instructions through different feedback modalities.</figcaption></figure>

timely stiffness adjustments. Consequently, this method may lead to degraded task performance and increased cognitive load.

To address these challenges, we propose the novel concept of a Stiffness Regulation Co-pilot (SRC). The SRC functions as a virtual agent, guiding the operator in stiffness regulation. It comprises two key components: a Stiffness Regulation Policy (SRP) and a feedback modality. The SRP generates real-time stiffness regulation instructions based on ongoing task performance. These instructions are then conveyed to the operator through the feedback modality, enabling the operator to adjust the stiffness accordingly. This research makes two primary contributions:

* we introduce the concept of a stiffness regulation co-pilot, which, to the best of our knowledge, is the first of its kind in facilitating stiffness regulation in this context.

* we present human subjects experiments that evaluates and compares the effects of various SRC feedback modalities on task performance and cognitive load.

Through these contributions, we aim to enhance the efficacy and user-friendliness of bilateral teleimpedance systems, potentially broadening their applicability and improving operator performance in complex teleoperation tasks.

## II. BILATERAL TELEIMPEDANCE CONTROL

As shown in Fig. 1, a classical bilateral robotic teleoperation system consists of a local robot and a remote robot that exchange information with each other through a communication medium, e.g., the Internet [23]. A position-force architecture is often employed to control such a teleoperation system [1], where the master robot is controlled by a force controller and the remote robot is controlled by a Cartesian impedance or admittance controller [24]–[27]. The remote robot controller tries to render the remote robot end-effector to behave like a virtual mass-spring-damper system which can be described below:

$$M(\ddot{x}_d - \dot{x}) + D(\dot{x}_d - \dot{x}) + K(x_d - x) = f_{ext}, \quad (1)$$

where $M, D$, and $K$ are the desired mass, damping, and stiffness matrices, respectively, and $x_d$ and $x$ denote the desired and measured position and orientation of the end-effector of the remote robot, respectively. The desired motion of the remote robot is commanded by the operator operating a handle attached to the end-effector of the master robot with one hand. The handle is also used to let the operator perceive the measured interaction force (usually by a force-torque sensor on the remote robot) between the remote robot and the environment, i.e., $f_{ext}$ in (1), implemented by the force controller of the master robot.

Although the desired impedance includes $M, D$, and $K$, in most cases, the mass $M$ can be omitted in the controller for better simplicity in practice and the stiffness has a dominant influence on the task performance, for instance, when the robot does not have highly dynamic motions [25], [28], [29], and the damping $D$ can be designed according to the commanded stiffness to stabilize the system. Therefore, unless specially stated, we refer to stiffness regulation when mentioning impedance regulation in the following. Moreover, even though a symmetric positive definite stiffness matrix $M$ can be characterized by an ellipsoid, we only focus on the stiffness size regulation in this pilot study. When the stiffness is actively regulated by the operator through an interface, [9], [10], [17], [18], [20], [21], [30], according to real-time feedback signals from the remote site, the bilateral teleoperation system is turned into a bilateral teleimpedance control system.

## III. STIFFNESS REGULATION CO-PILOT

When one hand operates the master robot to send motion commands and receive force feedback, using another limb, such as the other hand [20] or a foot [21], to command stiffness through an external interface can mitigate the "coupling effect" [19] and improve stiffness regulation accuracy. To streamline coordination between the master robot and external stiffness control, we propose the Stiffness Regulation Co-pilot (SRC). The SRC is a virtual agent designed to enhance task performance and reduce cognitive load during frequent,
complex stiffness adjustments, particularly for operators with limited robotics knowledge.

As shown in Fig. 1, the SRC has two components: a Stiffness Regulation Policy (SRP) and a feedback modality. The SRP processes command and feedback signals to suggest stiffness adjustments, which are conveyed to the operator via cutaneous, visual, or auditory feedback. For example, a wearable device on the operator's hand can provide cutaneous cues, instructing them to move their hand up or down to adjust stiffness<fnref n="2" />. The SRP may use criteria like task performance and system stability [31], [32] to guide stiffness regulation.

Importantly, the SRP does not enforce precise tracking of a reference stiffness profile; instead, it prompts the operator when to increase or decrease stiffness. The operator retains control over the extent and rate of adjustment. This design choice is motivated by two factors: first, precise stiffness profiles are often unnecessary, as capturing general trends suffices for task success [33]; second, stiffness regulation is a secondary task that supports, but does not dominate, the primary task. Automating stiffness adjustments based on operator performance could lead to "mutual adaptation" issues, confusing the operator and degrading performance [34]. Therefore, the operator maintains authority over stiffness control, adjusting at their own pace based on visual feedback and perceived task performance.

## IV. EXPERIMENTS

Three experiments were conducted in a preliminary user study to validate the effectiveness of the SRC, and the experiments were designed based on a real-world teleoperation task, i.e., teleoperating a robot to perform the polishing task on a wind turbine blade edge. A polishing tool needs to be controlled by an operator to track a curved edge as much as possible and maintain a contact force as constant as possible under the disturbance of wind force. A model of the curved edge is unknown to the operator and the robot (an autonomous robot with a motion controller cannot work). A simple SRP of the SRC was used in the experiments:

$$SRP_{\text{in-contact}} = \begin{cases} \text{stiffness decrease} & \text{if } std_f > f_{th}, \\ \text{no stiffness change} & \text{otherwise} \end{cases} \qquad (2)$$

$$SRP_{\text{no-contact}} = \begin{cases} \text{stiffness increase} & \text{if } \|x_d - x\| > d_{th}, \\ \text{no stiffness change} & \text{otherwise} \end{cases} \qquad (3)$$

where std_f is a running standard deviation of the contact force measured by a force sensor on the remote robot, and x_d and x denote the desired position of the tool commanded from the local robot and the actual position measured by the remote robot, respectively. f_th and d_th are predefined thresholds of the force and the distance. SRP_in-contact and SRP_no-contact are the SRP output when there is a contact and no contact between the tool and the edge, respectively. The

contact was detected using the non-restarting cumulative sum (CUSUM) method [35], [36].

### A. Objectives

1) **Experiment 1:** The goal is to find a feedback modality of the SRC that can efficiently instruct an operator to move their secondary hand for stiffness regulation. Six common modalities were examined: Cutaneous (C), Visual (V), Auditory (A), Cutaneous-Visual (CV), Cutaneous-Auditory (CA), and Visual-Auditory (VA) modalities.

2) **Experiment 2:** To eliminate the effects of other influencing factors, a simplified teleoperation scenario can be used to compare the six modalities with respect to situations with No Feedback modality (NF) and No Stiffness regulation (NS) to find a feedback modality of the SRC that can outperform a teleimpedance system without use of the SRC (NF) and a teleoperation system with a constant stiffness of medium level (NS) in terms of task performance and cognitive load.

3) **Experiment 3:** The aim is to use a plausible teleoperation scenario to compare several good-performing modalities selected from Experiment 2 against NF to validate the efficacy of the SRC.

### B. Designs

1) **Experiment 1:** The experiment setup is shown in Fig. 2a, where participants interacted with a virtual environment using a Haply Inverse 3 haptic device, with their non-dominant hand (force feedback not enabled) while wearing a WEART TouchDIVER device on the index, middle finger, and thumb. Participants wore a noise-canceling headset with background white noise to minimize external auditory distractions. The task for the participant was to control a green ball using the Haply device to reach a series of invisible targets on a vertical white line on the screen. The six modalities were used to help guide the target-reaching task. Specifically, to present cutaneous cues, simultaneous pressure on the index and middle fingers indicates upward hand movement, while pressure on the thumb indicates downward movement. For the visual cues, an up (green) or down (red) arrow next to the vertical line indicated upward or downward hand movement. A high- or low-pitch sound was used to instruct an upward or downward movement as an auditory feedback.

2) **Experiment 2:** The setup shown in Fig. 2b was designed to emulate a minimal bilateral teleimpedance control system to focus on the evaluation of the expected benefits of the SRC with different modalities. The user used a Geomagic Touch haptic device, analogous to a local robot device in a teleoperation system, to teleoperate a white ball shown on the screen for a line follower task with the dominant hand (force feedback enabled). In this scenario, the users could perceive the contact force between the ball and a long, straight grey plate simulating a wind turbine blade edge and the disturbance force simulating the wind effect. An invisible disturbance force field was designed to surround the plate. The task was to ask the user to control the ball to move along and maintain a target 1.5N contact force with the plate as much as possible. The actual contact force was visualized in

<footnote n="2">²Future designs could control other parameters, such as stiffness ellipsoid shape and orientation, using finger movements.</footnote>
<figure><img src="image_3.png" /><figcaption>**Fig. 2:** Experimental scenarios. (a) Participants were asked to sit and wear cutaneous devices at the non-dominant hand, while using a grounded haptic device (no force feedback enabled) to reach targets along a vertical line on the screen, guided by feedback modalities perceived. (b) Participants used a grounded haptic device with their dominant hand (right hand in (b)) to perform a line-follower task with presence of force feedback. The non-dominant hand (left hand in (b)) used a grounded haptic device as a tracker (no force feedback), to let the user regulate the stiffness guided by feedback cues provided to the user. (c) Users were asked to use a VR headset to visualize a scenario, with a simulated wind turbine blade edge polishing task, in the presence of disturbances. The force feedback was provided to their dominant hand by a grounded force-controlled robot; the non-dominant hand was used to regulate the stiffness, guided by the modalities.</figcaption></figure>

a graph on the screen in real-time. The six modalities of the SRC were used the same way as in Experiment 1, and the upward or downward movement of the non-dominant hand will increase or decrease the stiffness in this experiment. The stiffness regulation instruction was provided by the output of the SRP in (2) and (3). No feedback modality for stiffness regulation and constant stiffness control were used as the benchmark for the SRC to compare against.

3) *Experiment 3:* A real teleoperation setup shown in Fig. 2c was used to validate the effectiveness of the SRC with good-performing modalities selected from Experiment 2. The user used their dominant hand to operate a real Franka Emika Panda robot as the local robot device to teleoperate a virtual remote Panda for a simulated blade edge polishing task. The non-dominant hand movement was tracked by a HTC VIVE tracker instead of the Haply device. The two robots and the task were visualized through a HTC VIVE Pro headset worn by the user. Gazebo was used for the physics-based simulation while Unity was used for visualizing the simulation results.

More details of all the experiments can be found in the accompanying video.

### C. Protocols

Different groups of subjects participated in the three experiments, their participation was voluntary. All subjects gave written informed consent before participation. The experiments were approved by the Research Ethics Committees of Aarhus University and University of Southern Denmark. In each of the experiments, there was a familiarization session before an experiment session in which the setup and the environment were introduced to the subjects and different modalities were demonstrated to them to ensure they could notice the modality cues and distinguish between the pair of two cues (for upward and downward hand movements) of each of the modalities.

1) *Experiment 1:* Eight subjects participated in Experiment 1. Each participant was asked to reach the same 18 target

positions with the guidance of each of the 6 modalities, starting from a central point for each target. The user independently pressed the Space key on a keyboard once they believed one target was reached. The 18 targets were divided into 3 groups, and the total travel distance for each group of 6 targets was designed to be constant.

2) *Experiment 2:* Ten male and four female participants between 24 and 43 years old (M = 29.3, SD = 4.7) participated in Experiment 2. In the familiarization session, in addition to getting familiarized with the six modalities, NF and NS were also demonstrated. Importantly, the subjects were well informed of the motivation for the stiffness regulation: decreasing the stiffness to increase the force production resolution and stabilize the contact force, while increasing the stiffness to regain the motion control of the ball resisting the disturbance when the contact is lost. In the experiment session, three different disturbance force fields were used to diminish the learning effects, and each was repeated three times. With each of the nine fields, one line-follower task trial was done with each of the eight conditions (NF, C, V, A, CV, CA, VA, and NS) resulting in 72 trials in total per subject. The data recording of a trial started after a green ball at one end of the plate was touched by the teleoperated ball and ended after a red ball at the other end was touched. At the beginning of each trial, the subject was asked to put their secondary hand in the same place to ensure the same initial stiffness level.

3) *Experiment 3:* Five male participants between 23 and 29 years old (M = 25.0, SD = 2.1) participated in Experiment 3. Despite a different setup, as described in section IV-B.3 the procedure is similar to that of Experiment 2. In this case only one disturbance field was used and it was repeated five times.

### D. Evaluation criteria

1) *Experiment 1:* The completion time for one group of six targets was used as the primary task performance criterion, while the average of the positioning errors (when
the Space key was pressed) relative to the group of targets was a secondary criterion because, as mentioned before, capturing the trends of a good stiffness profile may suffices for task success. So, the stiffness regulation responsiveness (reflected by the completion time) is more important than the accuracy (reflected by the error).

2) *Experiments 2 and 3:* Two objective task performance criteria were used in Experiments 2 and 3. One was the root mean square error (RMSE) of the contact force when the ball was in contact and the other one was contact recovery efficiency, which was calculated by averaging the speeds of the ball when the contact was lost. The former was used to evaluate how well the contact force was controlled relative to a target level and the latter was used to assess how quickly the user returned to the path after losing the contact. Since the width of the plate was designed thin enough, we assumed the path-tracking accuracy was good enough as long as the contact was maintained. Additionally, the NASA-TLX questionnaire was used to evaluate the subjective mental and physical workload in Experiment 2 [37] [38].

## E. Statistical analysis

To examine the effects of the independent variable (feedback modality) on the dependent variables, i.e., the completion time and the positioning error in Experiment 1, and the contact force error and the contact recovery efficiency in Experiments 2 and 3, one-way repeated measures ANOVA was chosen to compare these effects between different modalities because it accounts for the subjects being subject to all conditions. $p \le 0.05$ is considered a significant difference. To test the normality of the data, the Shapiro-Wilk normality test was utilized, and a logarithmic transformation for normalization was used before proceeding with the analysis.

After identifying significant main effects in the one-way repeated measures ANOVA, Post-hoc analyses were performed using the pairwise t-test with Bonferroni correction to control the Type I error rate across multiple comparisons.

# V. RESULTS

## A. Experiment 1

a) *Task completion time:* one-way repeated measures ANOVA revealed a significant effect of feedback modality ($F(5,126)=4.157, p=0.002$) on the completion time. The modality V and its combined modalities (CV and VA) consistently resulted in lower completion times as illustrated in Fig. 3, while modality C had the longest completion time with the largest variance. Post-hoc tests revealed significant differences between V, CV, and VA compared to C, with VA showing the strongest difference ($p < 0.001$).

b) *Positioning error:* Although the positioning error to the target was measured, no significant effect of feedback modality was found ($F(5,124) = 1.019, p = 0.405$).

## B. Experiment 2

a) *Contact force error:* the one-way repeated measures ANOVA did not show a significant difference in the contact force error across modalities ($F(7, 91) = 1.591, p = 0.148$).

<figure><img src="image_4.png" /><figcaption>**Fig. 3:** Completion time results (seconds) of the 6 feedback modalities in Experiment 1.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4:** Results of the contact force error (RMSE) (a) and the contact recovery efficiency (b) of the 8 conditions in Experiment 2.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 5:** NASA-TLX mental and physical demand results of the 8 modalities in Experiment 2.</figcaption></figure>

A more descriptive analysis showed that C, and its combined modalities, CA and CV had relatively lower force errors compared to the other modalities shown in Fig. 4a.

b) *Contact recovery efficiency:* From a more descriptive analysis comparing the different feedback modalities in terms of the contact recovery efficiency in Fig. 4b, it showed that C, CV, and VA tended to have slightly better contract recovery efficiency compared to the other modalities. However, a one-way repeated measures ANOVA test did not reveal any significant difference across the modalities for the contact recovery efficiency, ($F(7, 91) = 0.909, p=0.503$).

c) *Subjective results:* The NASA-TLX questionnaire results revealed that CV and CA got relatively lower mental and physical demand scores compared to other feedback modalities, shown in Fig. 5.
<figure><img src="image_7.png" /><figcaption>**Fig. 6:** Results of the contact force error (RMSE) (a) and the contact recovery efficiency (b) of the selected modalities in Experiment 3.</figcaption></figure>

### C. Experiment 3

Based on the results in Experiment 2 presented in Section V-B, good-performing modalities, C, CA, CV, and VA, were selected to compare against NF in Experiment 3.

a) Contact force error: As shown in Fig. 6a, a descriptive interpretation showed quite an even distribution of the force error across the modalities. However, modality C presented low force error with very small variation compared to the other modalities. These results manifested that C was the most effective modality to control and stabilize the contact force that was consistent with one of the design goals of the SRP shown in (2). The one-way ANOVA test did not reveal any significant effect of the modality on the force error ($F(1,21) = 0.658, p = 0.426$).

b) Contact recovery efficiency: All the selected modalities seemed to outperform the NF in terms of the contact recovery efficiency as shown in Fig. 6b. The ANOVA test did not reveal any significant results ($F(1,21)=1.451, p=0.242$).

## VI. DISCUSSION AND LIMITATIONS

Experiment 1 examined the effect of different modalities on the subject's secondary arm movement responsiveness (evaluated by the completion time) to the stiffness regulation instructions (generated based on a group of targets), the results manifested that visual or visual-related (CV and VA) modalities outperformed the other modalities. However, when the primary teleoperation task with the dominant hand was added in Experiment 2, it seemed that better responsiveness to the stiffness regulation instructions couldn't produce better task performance. It was hypothesized that the visual cues were a bit distracting for the primary teleoperation tasks that required a significant amount of visual attention for the subject, e.g., the path and force tracking through the visual feedback on the screen. The responsiveness therefore deteriorated (reflected by the increased cognitive load shown in Fig. 5) which degraded the task performance.

Despite the limited impact of the modality on the task performance, interestingly, the trends in the results of Experiment 2 showed that the worst-performing cutaneous feedback modality in Experiment 1 became better in the task performance, especially for the contact force tracking shown in Fig. 4a, where the baseline conditions, NF (no feedback) and NS (no stiffness regulation), present with a

large range of values indicating an unstable contact force tracking. Importantly, subjective NASA-TLX workload assessment indicated that subjects consistently found less mentally demanding than other modalities including NF and NS when the cutaneous feedback was paired with the other modalities (CV and CA). These observations were validated again in a more realistic simulated teleoperation task in Experiment 3 where the cutaneous feedback received better contact recovery efficiency compared to the benchmark situation NF (Fig. 6b) and the least variance of the contact force error (Fig. 6a), which signified that the cutaneous feedback was the most effective modality to stabilize the contact force. This may be attributed to the convenient use of the same control point, i.e., the fingertips, for receiving the stiffness regulation cues and sending the stiffness regulation commands. This consistency probably brought reduced cognitive load which can facilitate tasks requiring sustained attention, e.g., a complex dual-arm teleimpedance task, in favor of operator endurance and potentially better long-term performance.

One limitation in this work is the relatively small sample size, particularly in Experiment 3 ($N=5$). Despite the small sample size potentially reducing the statistical power and the generalization of the results, the repeated-measures design mitigates variability [39], [40]. However, a confounder, such as participant experience, may still influence the results.

## VII. CONCLUSIONS AND FUTURE WORKS

This study introduced a new concept of Stiffness Regulation Co-pilot which consists of a Stiffness Regulation Policy and a feedback modality. It was proposed to facilitate using an external stiffness command interface in a bilateral teleimpedance control system. A preliminary user study designed based on a real-world teleoperation task, i.e., a wind turbine blade polishing task, was carried out to evaluate the efficacy of the proposed SRC. The results of the comparative experiments with different modalities of the SRC manifested that the cutaneous feedback seemed to be a more suitable modality choice that can potentially better maintain the contact between the tool and the target object under the influence of disturbance and improve contact force control compared to the system without using the SRC. In addition, combined with another feedback modality, the cognitive load of the operator can be likely reduced.

In this work, the stiffness size was regulated by the human arm movement. To further reduce the cognitive load, it is worthwhile to test other stiffness command interfaces like a joystick or buttons. Also, the modality in this paper was only used to instruct the regulation of the stiffness size, how to use different modalities to guide the control of the other aspects of a stiffness ellipsoid can be explored in the future. In addition, only different modalities were discussed in this paper, the effect of different SRPs on the task performance can also be investigated, for instance, through fuzzy control or learning-based methods. Furthermore, shared control/autonomy between the operator and the SRC over the stiffness regulation can be studied to improve the task performance but avoid the effect of mutual adaptation.
REFERENCES

[1] G. Niemeyer, C. Preusche, S. Stramigioli, and D. Lee, *Telerobotics*. Cham: Springer International Publishing, 2016, pp. 1085–1108.

[2] Y. Yokokohji and T. Yoshikawa, "Bilateral control of master-slave manipulators for ideal kinesthetic coupling-formulation and experiment," *IEEE transactions on robotics and automation*, vol. 10, no. 5, pp. 605–620, 1994.

[3] C. Pacchierotti and D. Prattichizzo, "Cutaneous/tactile haptic feedback in robotic teleoperation: Motivation, survey, and perspectives," *IEEE Transactions on Robotics*, 2023.

[4] M. Laghi, A. Ajoudani, M. G. Catalano, and A. Bicchi, "Unifying bilateral teleoperation and tele-impedance for enhanced user experience," *The International Journal of Robotics Research*, vol. 39, no. 4, pp. 514–539, 2020.

[5] F. J. Abu-Dakka and M. Saveriano, "Variable impedance control and learning—a review," *Frontiers in Robotics and AI*, vol. 7, p. 590681, 2020.

[6] L. Peternel and A. Ajoudani, "After a decade of teleimpedance: A survey," *IEEE Transactions on Human-Machine Systems*, vol. 53, no. 2, pp. 401–416, 2022.

[7] E. Burdet, R. Osu, D. Franklin, T. Yoshioka, T. Milner, and M. Kawato, "A method for measuring endpoint stiffness during multi-joint arm movements," *Journal of biomechanics*, vol. 33, no. 12, pp. 1705–1709, 2000.

[8] T. E. Milner, "Contribution of geometry and joint stiffness to mechanical stability of the human arm," *Experimental brain research*, vol. 143, no. 4, pp. 515–519, 2002.

[9] A. Ajoudani, N. G. Tsagarakis, and A. Bicchi, "Tele-impedance: Teleoperation with impedance regulation using a body-machine interface," *The International Journal of Robotics Research*, pp. 1642–1655, 2012.

[10] C. Yang, C. Zeng, C. Fang, W. He, and Z. Li, "A dmpbs-based framework for robot learning and generalization of humanlike variable impedance skills," *IEEE/ASME Transactions on Mechatronics*, vol. 23, no. 3, pp. 1193–1203, 2018.

[11] C. Fang, A. Ajoudani, A. Bicchi, and N. G. Tsagarakis, "A real-time identification and tracking method for the musculoskeletal model of human arm," in *2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)*. IEEE, 2018, pp. 3472–3479.

[12] C. Fang, G. Rigano, N. Kashiri, A. Ajoudani, J. Lee, and N. Tsagarakis, "Online joint stiffness transfer from human arm to anthropomorphic arm," in *2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)*. IEEE, 2018, pp. 1457–1464.

[13] X. Chen, C. Yang, C. Fang, and Z. Li, "Impedance matching strategy for physical human robot interaction control," in *2017 13th IEEE Conference on Automation Science and Engineering (CASE)*. IEEE, 2017, pp. 138–144.

[14] C. Fang, L. Peternel, A. Seth, M. Sartori, K. Mombaur, and E. Yoshida, "Human modeling in physical human-robot interaction: A brief survey," *IEEE Robotics and Automation Letters*, 2023.

[15] H. Nakamura, D. Abbink, and M. Mulder, "Is grip strength related to neuromuscular admittance during steering wheel control?" in *2011 IEEE International Conference on Systems, Man, and Cybernetics*. IEEE, 2011, pp. 1658–1663.

[16] H. Höppner, D. Lakatos, H. Urbanek, C. Castellini, and P. van der Smagt, "The grasp perturbator: Calibrating human grasp stiffness during a graded force task," in *2011 IEEE International Conference on Robotics and Automation*. IEEE, 2011, pp. 3312–3316.

[17] D. S. Walker, R. P. Wilson, and G. Niemeyer, "User-controlled variable impedance teleoperation," in *2010 IEEE International Conference on Robotics and Automation*. IEEE, 2010, pp. 5352–5357.

[18] L. Peternel, T. Petrič, E. Oztop, and J. Babič, "Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach," *Autonomous robots*, vol. 36, pp. 123–136, 2014.

[19] L. M. Doornebosch, D. A. Abbink, and L. Peternel, "Analysis of coupling effect in human-commanded stiffness during bilateral tele-impedance," *IEEE Transactions on Robotics*, vol. 37, no. 4, pp. 1282–1297, 2021.

[20] L. Peternel, N. Beckers, and D. A. Abbink, "Independently commanding size, shape and orientation of robot endpoint stiffness in tele-impedance by virtual ellipsoid interface," in *2021 20th International Conference on Advanced Robotics (ICAR)*. IEEE, 2021, pp. 99–106.

[21] S. Klevering, W. Mugge, D. A. Abbink, and L. Peternel, "Foot-operated tele-impedance interface for robot manipulation tasks in interaction with unpredictable environments," in *2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2022, pp. 3497–3504.

[22] C. Fang, A. Ajoudani, A. Bicchi, and N. G. Tsagarakis, "Online model based estimation of complete joint stiffness of human arm," *IEEE Robotics and Automation Letters*, vol. 3, no. 1, pp. 84–91, 2017.

[23] P. M. Kebria, H. Abdi, M. M. Dalvand, A. Khosravi, and S. Nahavandi, "Control methods for internet-based teleoperation systems: A review," *IEEE Transactions on Human-Machine Systems*, vol. 49, no. 1, pp. 32–46, 2018.

[24] N. Hogan, "Impedance control: An approach to manipulation," in *1984 American control conference*. IEEE, 1984, pp. 304–313.

[25] A. Albu-Schaffer, C. Ott, U. Frese, and G. Hirzinger, "Cartesian impedance control of redundant robots: Recent results with the dlr-light-weight-arms," in *2003 IEEE International conference on robotics and automation (Cat. No. 03CH37422)*, vol. 3. IEEE, 2003, pp. 3704–3709.

[26] C. Ott, R. Mukherjee, and Y. Nakamura, "Unified impedance and admittance control," in *2010 IEEE international conference on robotics and automation*. IEEE, 2010, pp. 554–561.

[27] L. Peternel, C. Fang, M. Laghi, A. Bicchi, N. Tsagarakis, and A. Ajoudani, "Human arm posture optimisation in bilateral teleoperation through interface reconfiguration," in *2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)*. IEEE, 2020, pp. 1102–1108.

[28] C. Zhou, C. Fang, X. Wang, Z. Li, and N. Tsagarakis, "A generic optimization-based framework for reactive collision avoidance in bipedal locomotion," in *2016 IEEE International Conference on Automation Science and Engineering (CASE)*. IEEE, 2016, pp. 1026–1033.

[29] C. Fang, X. Ding, C. Zhou, and N. Tsagarakis, "A2ml: A general human-inspired motion language for anthropomorphic arms based on movement primitives," *Robotics and Autonomous Systems*, vol. 111, pp. 145–161, 2019.

[30] A. Ajoudani, C. Fang, N. Tsagarakis, and A. Bicchi, "Reduced-complexity representation of the human arm active endpoint stiffness for supervisory control of remote manipulation," *The International Journal of Robotics Research*, vol. 37, no. 1, pp. 155–167, 2018.

[31] F. Ferraguti, N. Preda, A. Manurung, M. Bonfe, O. Lambercy, R. Gassert, R. Muradore, P. Fiorini, and C. Secchi, "An energy tank-based interactive control architecture for autonomous and teleoperated robotic surgery," *IEEE Transactions on Robotics*, vol. 31, no. 5, pp. 1073–1088, 2015.

[32] K. Kronander and A. Billard, "Stability considerations for variable impedance control," *IEEE Transactions on Robotics*, vol. 32, no. 5, pp. 1298–1305, 2016.

[33] E. Burdet, D. W. Franklin, and T. E. Milner, *Human robotics: neuromechanics and motor control*. MIT press, 2013.

[34] S. Ikemoto, H. B. Amor, T. Minato, B. Jung, and H. Ishiguro, "Physical human-robot interaction: Mutual learning and adaptation," *IEEE robotics & automation magazine*, vol. 19, no. 4, pp. 24–35, 2012.

[35] A. Gandy and F. D.-H. Lau, "Non-restarting cumulative sum charts and control of the false discovery rate," *Biometrika*, vol. 100, no. 1, pp. 261–268, 2013.

[36] I. Iturrate, A. Kramberger, and C. Sloth, "Quick setup of force-controlled industrial gluing tasks using learning from demonstration," *Frontiers in Robotics and AI*, vol. 8, p. 767878, 2021.

[37] A. Cao, K. K. Chintamani, A. K. Pandya, and R. D. Ellis, "Nasa tlx: Software for assessing subjective mental workload," *Behavior research methods*, vol. 41, pp. 113–117, 2009.

[38] J. M. Noyes and D. P. Bruneau, "A self-analysis of the nasa-tlx workload measure," *Ergonomics*, vol. 50, no. 4, pp. 514–519, 2007.

[39] G. Hoffman and X. Zhao, "A primer for conducting experiments in human-robot interaction," *ACM Transactions on Human-Robot Interaction (THRI)*, vol. 10, no. 1, pp. 1–31, 2020.

[40] F. Strale Jr, "Partitioning for enhanced statistical power and noise reduction: Comparing one-way and repeated measures analysis of variance (anova)." *Cureus*, vol. 16, no. 12, 2024.