Online Transfer and Adaptation of Tactile Skill: A Teleoperation Framework
Authors: Xiao Chen
Tianle Ni
Kübra Karacan
Hamid Sadeghian
Sami Haddadin
Venue: CoRL 2024 (2024)
Keywords: Learning from Demonstration
Online Adaptation
Tactile Skill
Teleoperation
Autonomy Allocation
TL;DR: A tele-teaching framework enables robot online learning and adapting tactile skill from a remote demonstration.

Abstract: This paper presents a teleoperation framework designed for online learning and adaptation of tactile skills, which provides an intuitive interface without need for physical access to execution robot. The proposed tele-teaching approach utilizes periodical Dynamical Movement Primitives (DMP) and Recursive Least Square (RLS) for generating tactile skills. An autonomy allocation strategy, guided by the learning confidence and operator intention, ensures a smooth transition between human demonstration to autonomous robot operation. Our experimental results with two 7 Degree of Freedom (DoF) Franka Panda robot demonstrates that the tele-teaching framework facilitates online motion and force learning and adaptation within a few iterations.

Online Transfer and Adaptation of Tactile Skill: A Teleoperation Framework
Xiao Chen," Tianle Ni;
Kübra Karacan,
Hamid Sadeghian, Sami Haddadin
Munich Institute of Robotics and Machine Intelligence, Technical University of Munich
xiaoyu. cwwn@tum.......m....... de, tianle iil.uum.i@teeeee
Abstract: This paper presents a teleoperation framework designed for online learning and adaptation of tactile skills, which provides an intuitive interface with- out the need for physical access to an execution robot. The proposed tele-teaching approach utilizes periodical Dynamical Movement Primitives (DMP) and Recur- sive Least Square (RLS) for generating tactile skills. An autonomy allocation strategy, guided by learning confidence and operator intention, ensures smooth transition from human demonstration to autonomous robot operation. Our experi- mental results with two 7 Degree of Freedom (DoF) Franka Panda robots demon- strate that the tele-teaching framework facilitates online motion and force learning and adaptation within few iterations.
Learning from Demonstration, Online Adaptation, Tactile Skill, tele- operation, Autonomy Allocation
Keywords:
Introduction
As robots become increasingly integrated into daily activities such as supporting household chores and elderly care [1], the complexity of their tasks continues to grow. Learning from Demonstration (L.fD) [2] has emerged as an efficient method directly transfer expert skills to the robot, thereby reducing the time required for self-learning [3]. Several approaches, including first-order Dynamical Systems [4], Gaussian Mixture Model [5], and Dynamical Movement Primitives (DMP) [6, 7, 8] have been employed to encode the motion skills. However, motion skills alone are insufficient for complex tasks involving rich contact with the environment. Therefore, the incorporation of force skills is also essential. In [9], a modulation term added to DMP to enable environment interaction. force profile is encoded in attractor trajectory though admittance relationship in [10]. An explicit force profile learned in the format of DMP separately from motion [11]. Robots gain the tactile skill to handle complex tasks by combining motion and force profiles.
A recent review on Lff [12] categorizes the demonstration method into three types: 1) Kinesthetic teaching, 2) Teleoperation, and 3) Passive observation. The passive observation method faces chal- lenges when humans cannot access the robot because replicating the robot's environment com- plicated. Kinesthetic teaching refers mode in which the robot is physically guided through task by human operators through direct interaction. Although the kinesthetic teaching provides user-friendly interface, distinguishing between human guiding and interaction forces with the en- vironment involves complex algorithms. Consequently, force profiles must be taught separately through teleoperation in [11]. LID via teleoperation enables the experts to transfer skills to a robot that is not easily accessible. By utilizing the tactile sensors of the operator's 's device [13], the external force in kinesthetic teaching now divided into the operator-applied force and the environment's interaction force, simplifying the process learning tactile skills.
A typical teleoperation system consists of remote station where robot executes the tasks and human operator station with an input interface to control the remote robot. These input interfaces vary from simple Mouse- Keyboard (MK) configuration [14] to intricate Avatar systems showcased
Equal contribution.
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Table 1: Summary of related work
Motion
Force learning
Simultaneous learning
Online adaptation
Remote
Method
learning
teaching
IPFS-KTHI[M]
X
PAPM[25]
X
FACT-DS[26]
X
X
RASHIL-RISMI27]
X
X
Ours
in the ANA XPRIZE Competition [15]. commonly used imput interface is haptic devices, allowing experts to teach haptic guidance references [16] contact-rich tasks [10, 17]. However, due the morphological differences between the haptic device and the execution robot, demonstrations using the haptic device are not very intuitive [18].
In this paper, we design telcoperation framework that consists two identical robot arms. One located in the workspace to execute the tasks after learning them; the other one interacts with the human operator demonstrate the tactile skill and allow the human operator to feel the remote environment. Our framework with identical robot arms merges the intuitiveness of kinesthetic teaching with the capability for force learning address the problem mentioned earlier in telcoper- ation. The goal is to learn and adapt periodic tactile skills, encompassing motion and force profiles, from remote human demonstrations without interrupting skill execution. Compared to the traditional Leader-Follower telcoperation architecture, the leading role transits between the two robots accord- ing to the designed autonomy level allocation. During the teaching phase, the human side takes the lead, and once teaching is complete, the robot in the workspace resumes the leading role and exe- cutes the tactile skill autonomously. This role-switching mechanism is known as shared autonomy [19]. The autonomy level allocation relies metries such as human intention[20), human ability [21, 22], or confidence in the learned skills [23]. In our work, autonomy allocation depends on the user's intention and confidence in skill learning. Another benefit of autonomy allocation is that our framework allows humans to intervene and adjust the learned skill online. Typically, adapting to new skills occurs offline and needs an additional operation to reset the robot [24].
The contributions of this work follows:
A tele-teaching framework that enables manipulator simultaneously learn force and motion profile, thereby acquiring tactile skill from remote demonstration.
A mechanism for online learning and adaptation of tactile skills without interruption through the proposed framework.
The autonomy allocation based operator intention and skill learning confidence provides intuitive interaction between operator and robot.
2
Related Work
The main differences between our work and other methods are listed in Table The IPFS-KTHI method[]]] I] teaches motion and force profiles separately. Motion profiles are taught through Kines- thetic teaching, and the force profile is taught through a haptic device, which could lead a lack of synchronization between motion and force. The RASHIL-RTSM approach[27] integrates a human sensorimotor system into the robot control loop through a teleoperation setup for remote teaching. Inspired by this framework, we use a Teacher Robot instead of the human sensorimotor system for remote teaching. The method most closely related to our periodic motion learning module is PAPM[[5], which focuses only on tactile skills that involve constant contact force rather than learn- ing force profiles through demonstration. Additionally, we expand the autonomy allocation strategy from PAPM, initially designed for motion adaptation multaneous motion and force adaptation within our teleoperation framework. For learning about contact forces. FACT-DS [26] is the closest related work. It aims develop compensation model that corrects force errors caused by non-flat surfaces. Unlike FACT-DS, where force profiles
single robot manipulator, to facilitate si-

---

Human Operator
Environment
S ha
eatts
Teacher Robot
Student Robot
Communication Channel
TA
f Ri
f Fan St
Teacher Controller
Student Controller
cas,er dea
f frafa
Tactile Skill Learning
Autonomy Allocation
Zrafi
& Adaptation
freto
Figure I: Architecture block diagram for tele-teaching of tactile skill.
position-dependent to adjust for these irregularities, in work, force profiles are time-dependent, which allows for learning the forces shown by the user.
The primary innovation our framework, distinguishing it from existing methods, is the simulta- neous learning and adaptation of motion and force profiles-essetiieee essential for tactile skills. This feature leverages the capabilities of our teleoperation framework, which distinguish human guiding forces from external forces exerted on the robot without additional force-torque sensor.
3 Methodology
3.1 Robot control in tele-teaching
The tele-teaching system consists of identical robots to enhance the intuitiveness. The Teacher Robot (TR) is positioned the human operator side, allowing humans to demonstrate tactile skills while feeling the tactile feedback of the remote environment. Conversely, the Student Robot (SR) is located remotely to execute the learned tactile skill. The control architecture described below illustrates how the Teacher Robot (TR) and Student Robot (SR) interact in terms of motion and forces to implement this tele-teaching system.
The sensory data of the robot systems including position æ,, x., measured external force fi. fezt,s and the calculated autonomy level yj are exchanged through the communication channel and utilized within the controller (Figure 1). The dynamics of gravity-compensated TR is written as,
M e..e(a)a + Ce,(li, q.) = rfu, + f
where Ma, Raxn and the C..t gax= are the inertia and Coriolis/centrifuaal matrices of the 7R represented in task space with dimensions, respectively. H, R is the TR end-effector position represented in the task space. JN. E I2" is the force applied by/to the human operator during the interaction with TR
IR" is the control command generated by the 7R controller, and 1 E [0, 1] is the autonomy level that acts as an activation parameter to the controller. A PD+ controller is employed on TR to track the position of SR and provide the haptic feedback,
U K E + D
where Ke E R.x" and D, E IR= represent the stiffness and the damping matrices, respectively. R. Ed,t 2, denotes the trajectory following error between the TR and the desired trajectory Ed E R", which is transmitted from SR
On the SR side, similar dynamics with gravity compensation are written as,
Mcs c.s (q,) + Ca.s(g.. 4,) = u, + fext.e
(3)
where M... _ and the Co.s E guxn are the inertia and Coriolis/centriiuaal matrices of the SR represented in task space, respectively. u, E _ denote the control command SR
U Mi,, (1 17) Ut,
(4)
3

---

where the Uufic,s is the unified foree/impedance control [28] to follow the reference tactile skill:
thu fic,s Umn. + f.at
(5)
The control command um,s is the impedance controller for tracking motion profile,
Um M Me + Ce,s@resss K D.ē,
(6)
where ã, = 2refie is the tracking error and K, E R=x* and D, E R=xx are the impedance matrices. Moreover, us,s is a PTD controller for force tracking with feed-forward term,
ffa frefe kip. ki f | kil
where frffs ext.a represents the tracking error for force. The passivity of the controller be ensured by designing energy tank[29], which provides "energy budget" and releases energy for the execution of non non-passive control actions by modifying the control command.
The control force uts for the SR is designed to track the TR through both position and force chan- nels, i.e.,
Ut,s r., ( - v.)) + (i,a( - it.) + fd,:.-
Notably, for simplicity, only the translational dimensions considered for the task space coordi- nate, i.e., n 3. The orientation is constrained by high impedance, and the null space of the robot is controlled to maintain specific joint configuration with best-effort in compliant manner while ensuring task execution [30].
3.2 Tactile skill generation
The generated reference tactile skill comprises both motion and force profiles. For the reference motion profile, periodic DMP formalism is adopted to generate the reference motion [7]:
Tref. = S22 (32 (x, Øreffs) Brefis Ym
Ω
where Or and B% are constants, along with the anchor point By R². Note that the force profile not encoded by the DMP in Eq. due irreducible noise in the foree reference signal fezt,s from the sensor. Derivatives of the force signal, fext,s and fext,ss will be too noisy to effectively encoded with DMPs. To ensure synchronization across different dimensions, the basic frequency of the generated tactile skill is determined as the minimum frequency value among all the dimensions.
Ω = min [w.m 42.m Wn.m
(10)
The change rate the canonical phase is the learned frequency, i.e., Q. Additionally, Ym is linear combination of N Radial Basis Functions (RBFs) as following.
Ym (8) = En W Wi(m)ss))
(11)
Σ (b)
(s) exp (h(cos(s ci) - 1)) i= 1,2,
(12)
where, represents the center of RBF, h denotes the width, and is the amplitude modulation parameter which is set to 1. Each vector Wi.m E IR" of the weights matrix w. E B is updated with the Recursive Least Square (RLS) algorithm [31]:
Wi. (t + 1) = wa,m (t) + (o.diaag(o,,,l)) + 1))
(13)
where J.m. R the vector inverse covariance associated the weights Wam Each element of Fim is updated
1
Pyim 11)
Di.m P (t)m(() Are. + P...mmett
(14)

---

Agg is the forgetting factor affecting the convergence speed. The initial value of Ps (0) is set to to avoid getting stuck at zero.
The goal of RLS is to minimize the error C,,im which is defined as,
Cr,i.m (t) = (1 ()) Wa,m).
(15)
Notably, the term (1 - 7) ensures the error is set to zero when the SR executes autonomously, thus preventing updates of the weight matrix. The target defined differently for motion and force. The motion profile generated using DMP, and the target is defined as
St Ω
7
(3
x.) Ω
(16)
Note that the subscript -m employed in the previous equations denotes their applicability to motion profile generation. For the force profile, due to the high noise of the first and second-order derivative of the measured force signal, the measured external force is directly assigned as the target value,
Td,f
S Sext,s.
(17)
Substituting the subscript -m with -y in the above equations (excluding equation (9)), such that they are valid for force profile generation, the reference force profile is given as fref, Yf-
3.3 Autonomy allocation
Autonomy allocation enables seamless transitions between human intervention and robot autonomy by assessing two factors: whether a skill has been successfully learned, and whether the user intends end the current demonstration
new one. If the system determines that the skill has been learned and new demonstrations are intended, it transitions to robot autonomy. Conversely, if the skill learning is incomplete or new demonstration is indicated, the system reverts to human intervention mode. The autonomy level [0, 1] on the SR side indicates whether the human or the robot is leading during the learning phase. At r7 = O, the TR complies with the human's motion, and the SR is compliant to follow the motion of TR. Throughout the learning and adaptation process, gradually smoothly changes the role of the robot system from compliantly following the human to following the generated reference tactile skill. After the skill learning and adaptation, the 7) increases 1, indicating the learning process ends and the robot system becomes autonomous until the next human intervention. This autonomy allocation also indicates the role changing between leader and follower in teleoperation system, as disscussed in Appendix 6.2.1
The rate of change of the autonomy level is defined as,
max 0}, 0
1)
Yr
0 << < 1 11 1,
(18)
min min (Tr: 0},
with
7/ ( + (1 - In - 1a), -e
(19)
where In is a value that increases during human intervention for learning a new skill and given by,
I, (1 (1
[1
(20)
with appropriate constant Å1. This way, by applying external force from the human demonstrator, the level of autonomy decreases to comply with the human. Moreover,
1) em )
1)
4
I,- =
+
+
(21)
A2
A
X4
5

---

Ja. a,
Jiefs
fett
Motion Learning & Aduptation
Force Learning Adaptation
Adaptive Frequency Oscillator
Adaptive Frequency Oscillator
Ω
f
Recursive Least Square
Recursive Least Square
Dynamic Movement Primitive
Figure 2: Block diagram of the Tactile Skill Learning & Adaptation. The motion and force profiles follow the same procedure. The force profile does utilize the DMP due the noise associated with its derivative.
which indicates the confidence of the SR in learning the tactile skill. Here, the em denotes the error vector, which will be introduced in the next section. During the learning phase, and as the motion tracking enror ã, and foree tracking error fs gradually goes below certain threshold X and Aa. the value of 7, decreases. This way, the confidence in learning increases, and thus, the autonomy level increases. To prevent getting stuck at zero, a small parameter e is introduced in (19), while acts as scaling parameter. tuning guidance is provided in the Appendix 6.1.1
3.4 Adaptive frequeney oscillators
Adaptive frequency oscillators tasked with identifying the basic frequency Ω of demonstrated periodic skill in real time. During the learning and adaptation of periodic tactile skill, the motion and the force undergo the same procedure, as illustrated in Figure 2. A generalized state p represents both motion and force state. In the following equations, the subscript and the state Pm = x., To adapt the frequency oscillators for the force profile, we need to replace the subscript with - and go through the following formula again, and the state will be p, = Sext,s-
stands for motion profile,
Following the same approach proposed in [32], the frequency oscillator is adapted by,
ê. = wm (1 7) Em sin(Om))
(22)
w. = -(1 (1 7)) (um E. sin(f..))
(23)
Em diag(pm - pm))
(24)
where 8m E R is the vector of the corresponding phase, Wm R is the vector of frequencies, from which the basic frequency Ω is acquired Equation (10) (the basic frequency Ω is the mini- mum), Mm E R is the coupling strength and Em R=x* is diagonal matrix considering the error between the input signal Pm and the estimation of the signal pm. The initial values of wm and o.. are set to small value to avoid being stuck at zero.
Note that the bigger the coupling strength v, the faster the adaptive frequency oscillator converges [31], but the more unstable when w is closer zero. The learning rate pm also impacts the con- vergence speed and algorithmic stability. A detailed sensitivity analysis of coupling strength and learning rate and the tuning strategy is provided in Appendix 6.1.2.
The ith element of the estimation Pm is computed as
M Σ (ar.k.m cos(kef,,m)) Br.k. sin(kk),,m) i= 1, n,..,,,
Pi.m
1,2,....
(25)
K=
where M denotes the number of Fourier components. The amplitudes 0ii,k, Bar.. updated as follows.
(1 11) Pm cos(kOs.m) ei,m
=
[1 7) pm sin sin(k)r..) Ci.m.
(26)
As the frequency the denominator the DMP expression in Eq. and 16, thus we set lower bound the frequency in the experiment for safety.

---

Figure 3: Experimental tele-teaching setup consists of two identical 7DOF robot arms. One rohot executes factile skill, and the other robot used to demonstrate or modify the skill over teleoperation framework.
Courdinate
Pratiin
1M0
143
160
Y Coorditate
60
120
Z. Force
120
Basic frequency
<) Automation level
Time: 3] 13-80s
D
gi 12-57s
16-11.3
15-1480
B1
0L
1L
Vio)
2.4rade
1.5cwbk
Figure 4: Experimental results of the proposed tele-teaching framework. A human operator demonstrates and modifies different periodic tactile skills to the SR through TR. a) and b) SR's end-effector position æ. and DMP reference trajectory Fref.s: e) The contact force of SR fezt.s and reference force fref.s. d) Basic frequency of motion Was wy and of force w, and Autonomy level n7 to j) The final encoded DMP trajectories on the XY plane for every learning cycle. The results show that the SR can quickly learn and adapt the tactile skill.
Experiments and Results
4.1 Experiment setup
The experimental setup consists of two 7 Degrees of Freedom (DoF) Franka Panda robots [33]. as depicted in Figure 3. A sponge is fixed to the end-effector of the SR to interact with the table surface. The operator interacts with TR directly by grasping the robot end-effector. The data between 7R and SR is transmitted through User Datagram Protocol (UDP) with negligible communication delay. The user demonstrates tactile skills on the TR, including motion on the XY plane and force in the Z direction. A series of different tactile skills is performed continuously illustrate the simultancous motion and foree learning and adaptation capability of our tele-teachhn teaching framework. The values of the parameters used in the experiment can be found in Appendix 6.1.
7

---

4.2 Experiment procedure and results
The experiment encompasses the demonstration of five periodic tactile skills denoted as Si, , with planar XY motion and contact force in the direction. Specifically, the skills contain,
S clockwise circular motion with large changing contact force,
S2: a counter-clockwisc circular motion with small changing contact force,
Sy: a linear motion with forceful push and gentle pull.
S1i an oo shape along the Y axis with reduced interaction force
S5: an shape along the X axis with increased interaction force
The outcomes of the demonstrated tasks are depicted in Figure 4. We start the first demonstration of the S₁ at around t 16s. After five iterations (around t 26s) repetition, the reference tactile skill converges to the user's demonstration, which suggests higher confidence in learning- Meanwhile, the basic frequency of the motion and force converge at the same value, around 2.4rad/s. The convergence of the signal shape and frequency increases the autonomy level. The operator can feel that the robot becomes less compliant, indicating the acquisition of the new skill. After two more iterations at t 33s, SR gains full autonomy and continuously reproduce the learned tactile skill. At around & = 418, the operator initiates modifications to the previous skill, increasing the external force fn., on TR, and the tracking errors make 1 drop quickly. Thus, returns to the human demonstration mode and starts a new learning cycle. This process repeats for skills S2. S5. respectively."
Overall, it takes around six iterations (16s) to learn skill S₁ and Sn with minimal frequency at around 2.4rad/ss, three iterations (6s) for Sy with 1.0rad/s, s, and six iterations (23s) for S₃ and S3 with 1.5rad/s and 1.3rad/s. Notably, the frequency of 1 1.Orad/s in S3 does not represent the actual frequency of the linear skill but rather the predefined minimal allowable frequency to prevent significant changes in DMP acceleration, as implicitly depicted in Eq. 16.
In summary, the proposed tele-teaching framework realizes online learning and adaptation of dif- ferent tactile skills in only few demonstration shots. The successful learning of the above five tactile skills, which differ spatially and temporally, verifies the proposed approach's 's effectiveness in simultancous motion and force learning, online adaptation and remote teaching listed in Table 1. Furthermore, the autonomy allocation module obviates the need for robot resets between tactile skills, enabling seamless transitions between robot autonomy and human demonstration. This mod- ule empowers human operators take over the control as needed, augmenting safety and reliability.
5 Conclusion and Future Work
This work develops a tele-teaching framework to enable real-time learning and adaptation of tactile skills from remote demonstration. Tactile skills demonstrated through the Teacher Robot are seam- lessly transferred online to the Student Robot located remotely. Our approach distinctively integrates motion and force into single cohesive tactile skill. The tele-teaching framework demonstrates re- markable speed and efficiency in adapting to new tactile skills. The adaptation occurs smoothly within just a few iterations. The autonomy allocation is based on the learning confidence and the operator's intentions. This strategy not only ensures smooth transitions between human demonstra- tion and autonomous robot execution but also empowers the user to dictate the cessation or initiation of teaching phases.
We currently focus on simple periodic tactile skills with only translational motions and forces. Since the motion component is represented by Dynamic Movement Primitives (DMPs), the limitations as- sociated with periodic DMPs are also applicable to our framework. Future works include exploring complex tactile skills that involve changes in orientation, potential stability issues led by communi- cation delays, and expanding into deeper human- collaboration such as rehabilitation tasks.
'The experiment video is available https:

---

Acknowledgments
We gratefully acknowledge the funding the Lighthouse Initiative Geriatronics by LongLeif GaPa gGmbH (Project Y). The authors acknowledge the financial support by the Federal Ministry of Edu- cation and Research of Germany (BMBF) in the programme of "Souvernn Digital. Vernetzt." Joint project 6G-life, project identification number 16KISK002. This work was supported by the Euro- pean Union's 's Horizon 2020 research and innovation programme as part of the project ReconCycle under grant no. 871352.
References
[1] M. Tröbinger, C. Jähne, Z. Qu, J. Elsner, A. Reindl, S. Getz, T. Goll, B. Loinger, T. Loibl, C. Kugler, C. Calafell, M. Sabaghian, T. Ende, D. Wahrmann, S. Parusel, S. Haddadin, and
S. Haddadin. Introducing garmi service robotics platform support the elderly at home: Design philosophy, system overview and first results. JEEE Robotics and Automation Letters, 6(3)-5857-5864, 2021. doi: 1011009/.2222 20012022....
[2] S. Schaal. Learning from demonstration. Advances in neural information processing systems, 1996.
[3] DelPreto, J. I. Lipton, L. Sanneman, A. J. Fay, C. Fourie, Choi, and Rus. Helping robots learn: A human-robot master-apprentice model using demonstrations via virtual reality teleoperation. In 2020 IEEE International Conference Roboties and Automation (ICRA), pages 10226-10233, 2020. doi:10. 100.10999CRA444452200000220
[4] S. M. Khansari-Zaat and A. Billard. Learning stable nonlinear dynamical systems with gaus- sian mixture models. IEEE Transactions on Robotics, 27(5):943-957, 2011.
[5] A. Pervez and D. Lee. Learning task porkammttterizee dynamic movement primitives using mix- ture of gmms. Intelligent Service Robotics, 11(1):61-78, 2018.
[6] A. Ijspeert, Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal. Dynamical movement primitives: Learning attractor models formotor behaviors. Neural Computation, 25(2):328- 373, 2013. ISSN 08997667. 1oi:10.1162//EBB
00393.
[7] A. J. Ijspeert, J. Nakanishi, and S. Schaal. Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RS, International Conference on In- telligent Robots and Systems (IROS2002), pages 958-963, 2002.
[8] M. Saveriano, F. J. Abu-Dakka, A. Kramberger, and L. Peternel. Dynamic movement primi- tives robotics: tutorial survey. The International Journal of Robotics Research, 42(13): 1133-1184, 2023.
[9] A. Gams, B. Nemec, A. J. ljspeert, and A. Ude. Coupling movement primitives: Interaction with the environment and bimanual tasks. IEEE Transactions Robotics, 30(4):816-830, 2014. doi: 109/TRO.2014.230477
[10] Y. Michel, Z. Li, and D. Lee. A dearning-based shared control approach for contact tasks. IEEE Robotics and Automation Letters, 8(12):8002-8009 2023. 10. W09/LRA..023.3322....
[11] Kormushev. S. Calinon, and D. G. Caldwell. Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Advanced Robotics, 25(5):581-603, 2011.
[12] H. Ravichandar, S. Polydoros, S. Chernova, and A. Billard. Recent advances in robot learning from demonstration. Annual review of control, robotics, and autonomous systems, 3: 297-330, 2020.

---

[13] S. Haddadin, L. Johannsmeier, and F. Díaz Ledezma. Tactile robots as a central embodiment of the tactile internet. Proceedings of the IEEE, volume 107, pages 471-487, Feb. 2019.
[14] A. Moortgat-Piece Pick, P. So. M. J. Sack, E. G. Cunningham, B. P. Hughes, A. Adamezyk, A. Sarabakha, L. Takayama, and S. Haddadin. A-rift: Visual substitution of force feedback for a zero-cost interface in telemanipulation. In 2022 IEEE/RSJ International Conference on In- telligent Robots and Systems (IROS), pages 3926-3933, 2022. doi: 109//000476111.2222.. 1981365.
[15] M. Schwarz, C. Lenz, A. Rochow, M. Schreiber, and S. Behnke. Nimbro avatar: Interactive immersive telepresence with force-feedback telemanipulation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5312-5319, 2021. doi:10.1 109/ IROS51168.2021 2021.66691.....111
[16] C. J. Pérez-del Pulgar, J. Smisek, V. F. Muñoz, and A. Schiele. Using learning from demon- stration to generate real-time guidance for haptic shared control. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 003205-0032I0 2016. doi:
10.1109/SMC.2016.744477
[17] B. Güleçyüzz V. von Büren, X. Xu, and E. Steinbach. Netfd: Network-aware learning from demonstration for in-contact skills via teleoperation. IEEE Robotics and Automation Letters, 8(1 (11):9955-7002 2023. doi:10.1 T09..RA..023.333.33333
[18] A. Pervez, A. Ali, J. Ryu, and D. Lee. Novel learning from demonstration approach for repetitive teleoperation tasks. In 2017 IEEE World Haptics Conference (WHC), pages 60-65, 2017. doi: 10. 1109///C.201...99997777
[19] M. Selvaggio, M. Cognetti, S. Nikolaidis, S. Ivaldi, and B. Siciliano. Autonomy in physical human-robot interaction: A brief survey. IEEE Robotics and Automation Letters, 6(4):7989- 7996, 2021.
[20] Y. Li, K. P. Tee, R. Yan, W. L. Chan, and Y. Wu. A framework of human-robot coordination based on game theory and policy iteration. IEEE Transactions on Robotics, 140-))4:108411 2016. doi: 10. 2099/RRO.001.25973333
[21] L. Pezeshki, H. Sadeghian, M. Keshmiri, X. Chen, and S. Haddadin. Cooperative assist-as- needed control for robotic rehabilitation: A two-player game approach. IEEE Robotics and Automation Letters, 855):2852-2859, 2023. doi:10.1
[22] X. Chen, H. Sadeghian, Y. Li, and S. Haddadin. Tele-rehabilitation of upper extremity with garmi robot: Concept and preliminary results. IFAC-Puper P 56(2)3346333333)3555 2023.
[23] M. Zeestraten, I. Havoutis, and Calinon. Programming by demonstration for shared control with application in teleoperation. IEEE Robotics and Aulomation Letters, 3()): 1848-1855, 2018.
[24] M. Khoramshahi and A. Billard. A dynamical system approach task-adaptation in physical human-robot interaction. Autonomous Robots, 43(4):927-946. Apr. 2019. ISSN 1573-7527. 11111101000/11111401888----88811
URL https: htttps:/ddei.org/1....
$10514-00 --144-066 8 0 -7
[25] Dimeas and Z. Doulgeri. Progressive automation of periodic tasks on planar surfaces of unknown pose with hybrid force/position control. In 2020 IEFE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5246-5252, 2020. 1oi: 109.11000//00/5544.. 2020.9341374.
[26] W. Amanhoud, M. Khoramshahi, M. Bonnesoeur, and A. Billard. Force adaptation in con- tact tasks with dynamical systems. In 2020 IEEE International Conference Robotics and Automation (ICRA), pages 6841-6847, 2020. 100200..1000111.4444445:20000000
10

---

[27] L. Peternel, T. Petrič, and J. Babič. Robotic assembly solution by numan-in-the--oee teaching method based on real-time stiffness modulation. Autonomous Robots, 42:1-17, 2018.
[28] C. Schindlbeck and S. Haddadin. Unified Passivity-Based Cartesian Force Impedance Con- trol for Rigid and Flexible Joint Robots via Task-Energy Tanks. 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 440-447, 2015. doi:10.1109/ICR 2015.7139036.
[29] E. Shahriari, L. Johannsmeier, and S. Haddadin. Valve-based virtual energy tanks: A frame- work to simultaneously passify controls and embed control objectives. In 2018 Annual Amer- ican Control Conference (ACC), pages
IEEE, 2018.
[30] H. Sadeghian, L. Villani, M. Keshmiri, and Siciliano. Task-space control of robot manipu- lators with null-space compliance. IEEE Transactions on Robotics, 30(2):993-506, 2013.
[31] A. Gams, A. Ijspeert, S. Schaal, and J. Lenarcic. On-line learning and modulation of periodic movements with nonlinear dynamical systems. Autonomous Robots, 27:3-23, 07 2009. doi: 10.10077/110514-00-999133
[32] K. Karacan, H. Sadeghian, R. Kirschner, and S. Haddadin. Passivity-based skill motion learning in stiffness-adaptive unified force-impedance control. In 2022 IEEE/RSJ Interna- lional Conference Intelligent Robots and Systems (IROS), pages 9604-9611, 2022. doi:
10. 00110099006476122222299991
[33] Haddadin, S. Parusel, Johannsmeier, S. Golz, Gabl, Walch, M. Sabaghian, C. Jähne, L. Hausperger, and S. Haddadin. The franka emika robot: A reference platform for robotics rescarch and education. IEEE Robotics Automation Magazine, (9(2))46-64, 2022. doi: 10. 1109/MRA.2021.3133888
[34] J. Ryu, Artigas, and C. Preusche. A passive bilateral control scheme for teleoperator with time-varying communication delay. Mechatronics, 20(7):812-223, 2010. ISSN 0957- 4158. Special Issue on Design and Control Methodologies in Telerobotics.
[35] X. Chen, H. Sadeghian, L.. Chen, M. Tröhinger, A. Swirkir, A. Naceri, and Haddadin. A passivity-basel approach on relocating high-frequency robot controller to the edge cloud. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5242-52-2 5248.. IEEE, 2023.
[36] M. Franken, S. Stramigioli, S. Misra, C. Secchi, and A. Macchelli, Bilateral telemanipulation with time delays: A two-layer approach combining passivity and transparency. IEEE transac- tions on robotics, 27(4)7741-756, 2011.
[37] D. Heck, A. Saccon, R. Beerens, and H. Nijmeijer. Direct force-reflecting two-layer approach for passive bilateral teleoperation with time delays. IEEE Transactions on Robotics, 34(1): 194-206, 2018.
Appendix
6.1 Guidance for parameter tuning
In the table 2, we list all the parameters and their values and range used during the experiment. Note that we dynamically adjust the parameters within specific range. the table 3, we explain the effects of the key parameters for every module in system, including autonomy allocation, adaptive frequency oscillators(AFO), and recursive least squares(RLS).
11

---

Table Parameter values and ranges used during the experiment.
Param
Value
Param
Value
Param
Value
Param
Value
X1
25
22
0.04
A3
0.06
A
5
Pm
[10,100]
Vj
2
Mm
[0.02,0.8]
[1,10]
50
M
1
X5g
[0.99, 0.9996]
P
0.8
Cxr
20
5
0.01
h
N²
Table 3: Summary of the key parameters
Module
Parameter
Feature
Autonomy Allocation
Sensitivity to new demo
22,3,4
Tolerance for learning error
P
Smoothness of the transitions
Coupling strength:
Vm,
convergence speed and stability Learning rate:
Adaptive Frequency Oscillators
Pm..f
convergence speed and stability
Number of Fouriers:
M
complexity of the skill
Number of RBFs:
Recursive Least Square
N
complexity of the skill
Forgetting factor: how sensitive to new coming data
Xfg
6.1.1 Autonomy allocation
The autonomy allocation module in our framework acts as a mechanism that is both aware of learn- ing confidence and responsive to user intentions, employing heuristic methods. The assessment of whether a skill has been successfully learned depends on various factors, including learning er- rors related motion, force, and frequency, as well as the external force applied the Teaching Robot, which indicates user intentions, such initiating new demonstration. Parameters within the autonomy allocation module configured to set thresholds based these criteria.
Given that variations in sensor and robot hardware can affect performance, it is essential to ad- just these thresholds to align with the system's specific characteristics. To assist in this calibration process, we provide guidance on effectively tuning the parameters involved in autonomy allocation.
A is force threshold, affecting how sensitive stration force on the Teacher Robot is larger than λ₁ the Teacher Robot will gain the leading role. indicating readiness to receive a new demonstration.
demonstration intention. When the demon-
Other 22,3,4 indicates how tolerant the learning error in frequency(\z), motion()a), and force(Aq) As human demonstration always contains noises, large 22,3,4 means more tolerance/less sensitivity to learning error. Too small A2,3,4 cause longer learning iterations, and too large 22,3,4 cause insufficient learning of the tactile skills.
affects how smooth the transition between robot autonomy to human intervention mode. Larger leads to a more smooth transition. A common choice set around 1.
6.1.2 Adaptive Frequency Oscillators
Coupling strength Vn.ff and learning rate fm.f the critical parameters for adaptive frequency oscillatorsAAFO), They are related to the convergence speed, algorithm stability, and accuracy of the AFO. Regarding the number of fouriers M, we find that M = 1 is already enough for the periodic skills demonstrated this paper.
A sensitivity analysis of these two parameters provides insights into how changes in coupling strength and learning rate impact the learning process of the motion's frequency. Note that the analysis also applies to learning the force's 'ss frequency. Figure. 5 illustrates the effect of different set-
12

---

X Coeedhote
by Freginey, uyeenilyyyy
Ereqeenyy 37 seaalilly tol de.
Figure 5: Comparison between various coupling strength the and learning rate jom values within single skill context. a) The target periodic demonstration. b) The estimated frequency the target demonstration with various values of Vm. and fixed jmm- c) The estimated frequency with various values of fm and a fixed Mn- d) The estimation of frequency through error-driven dynamic coupling strength and learning rate adjustments, where Vm and jmm gradually decrease from 100 and 2 to I and 0.02, respectively, as the error diminishes.
tings for coupling strength Um and learning rate jmm on learning performance, with the target motion recorded from actual demonstrations of circular motions the robot. The analysis of the coupling strength in the Figure. 5b) reveals that higher coupling strength leads to quicker convergence but introduces larger initial oscillations. The analysis of the learning rate in the Figure. 5c) shows that a higher learning rate accelerates convergence but will cause instability when above a certain threshold. In summary, there is a trade-off between achieving rapid convergence and minimizing oscillations when tuning the coupling strength and learning rate.
Building on the insights from our sensitivity analysis, we propose an error-driven dynamic adjust- ment for the parameters f and pm. Note that here we use the error between input signal and the estimation of the signal in AFO. This strategy is designed to ensure rapid convergence when errors are large and to mitigate oscillations when errors are small. Specifically, both the coupling strength and learning rate are configured to decay progressively as the error decreases. The implementation of this dynamic adjustment is demonstrated Figure. 5d). The results indicate that, compared to scenar- ios with fixed parameters, this adaptive method maintains swift convergence while simultaneously minimizing oscillations, thus optimizing the learning process for adaptive frequency oscillators.
To show the sensitivity of coupling strength and learning rate in AFO when learning different skills. We test the performance of frequency learning with varying sets of parameters across the five peri- odic tactile skills demonstrated in the experiment section. As shown in the Figure. 6a), the frequency of motions in the y-dimension changes after every new demonstration. Figure 6d) reflects the robust- ness of our dynamic coupling strength and learning rate adjustment under different skills. Compared to the fixed setting shown in Figure 6b) and c), we observe faster convergence when adapting to new skill and fewer oscillations(more accuracy) for the steady-state of learned frequency.
6.1.3 Recursive Least Square
The forgetting factor the Recursive Least Squares (RLS) algorithm plays a crucial role in deter- mining how much historical data influences current model updates. It dictates the size of "sliding window" of historical data considered in the regression. For periodic skills, good choice of the forgetting factor is intricately linked to the periods of the demonstrated skill. This relationship spec-
13

---

33 Coeediintr
Frngenecy scaetiiity lest
Propecccy somolnitt
Erequeacy
5
Figure 6: Comparison between various coupling strength V/n and learning rate Jom values across five different periodic skills. a) The target periodic demonstration. b) The estimated frequency with various values Vn and a fixed fmm c) The estimated frequency with various values of pm and fixed Vn- d) The estimation of frequency through error-driven dynamic coupling strength and learning rate adjustments, where Pmn and gradually decrease as the етrоr diminishes.
ifies the number of demonstration cycles considered relevant for updating the learning model. An approximate relationship exists between the forgetting factor and the time window size over which historical data remains significant within the RLS framework. This relationship can be expressed mathematically, as shown in Equation 27. A larger forgetting factor expands the time window over which historical data is considered valid, thereby incorporating more past data in the regression.
1 A
1 Ω
k
(27)
where 1-Aça is the estimated time window. &s decides how many rounds demonstrations are con- sidered. If is set to four, the learned skill is approximately an average of four last demonstrations.
The RLS adapts slowly to recent changes, with high Azg. Conversely, a low A makes RLS sensitive to new observations. To clearly show how the forgetting factor affects the learning proce- dure, we compare three sets of constant forgetting factors(k 0.5, 2, 8) under three nearly identical demonstrations of a circular movement the x-y plane. is challenging to maintain identical force profiles, omitted the force in this experiment. As shown in Figure 7a) and b), small forget- ting factor(wher when h: = 0.5) helps quick convergence at the beginning of a new demonstration, trivial demonstration divergence will lead to shape distortionnbetteeen 21s and 27s). As the robot becomes less compliant with the user during the autonomy transition, it is hard for the user to demonstrate identical motions. Therefore, a larger forgetting factor is necessary during the transition make less sensitive to recent data. However, as shown in Figure 7e) and f). larger forgetting factor(when k 8) leads to less sensitivity recent changes, resulting in longer convergence time.
In conclusion, the optimal forgetting factor setting is small value corresponding to k = 0.5 when the error big, and a large value corresponds to, e.g.. 1 = 8 when the error is small. The set of forgetting factors follows an error-driven dynamic adjustment, increasing for rapid convergence during significant errors and decreasing for stable autonomy transitions during minor errors. Note that here we use the error between the input signal and the learned signal in RLS. Regarding the number of RBFs N, we find that the common setting of N ranging from 30 to 50 fits the skills demonstrated in this paper.
14

---

x Condinstely -05/)
6.0
b Casediandel 0.57)
157 4
Cordinnte),
0.44
4) Y Cardinitch - 27)
e C 0T)
1077 ML
Cuedlantehy 87)
Automation
25
Figure 7: Comparison between different Asg-
6.2 Teleoperation system
6.2.1 Role allocation between TR and SR
The autonomy level y dictates role allocation between the human and the robot system, resulting in the role change of the 7R and SR When 7) = 0, the SR completely follows the TR by both position and force channel. However, when the learning process finishes with v 1, the TR transitions into a follower role, synchronizing its motion with the SR Traditional fealer-follower structure does not include the autonomy level, and SR only follows the TR by force. We realize that using both position and force channels from 7R to SR is necessary in the tele-teaching framework. the force channel were used, minor position deviations would exist between TR and SR during the learning process(r=0) because the two robot arms not entirely identical in practice. As the au- tonomy level rises, the leading role transits from TR to SR. TR would follow SR gradually more in the motion channel, and the minor position deviations would be compensated. However, the human operator at 7R 's side would feel that the SR learns a different motion. Therefore, the operator tends to naturally resist the compensation and keep the original demonstration, making the autonomy level 7 fall again. The minor position deviations lead to unsmooth leading role transitions from TR to SR resulting in unsuccessful tactile skill transfer. Furthermore, L.D via teleoperation presents certain drawbacks. Reference [18] shows that demonstrating a peg-in-hole task through teleoperation using haptic device is more difficult for the operator than kinesthetic teaching. Users tend to demon- strate trajectory with higher variance. However, the method proposed in this work offers potential improvements. Another concern might be the stability issue with sub-optimal communication qual- ity, leading to delays and packet loss. This problem can be alleviated by passivity-based methods like the Time Domain Passivity Approach (TDPA) [34, 35] or energy- based methods [36, 37].
only
15


Reviews:
[{"review_id": "16cJVwTG90", "reviewer": "Reviewer_4fok", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents an innovative teleoperation framework for the online learning and adaptation of tactile skills, eliminating the need for physical access to the execution robot. The proposed tele-teaching approach employs periodical Dynamical Movement Primitives (DMP) and Recursive Least Square (RLS) for generating tactile skills. The framework incorporates an autonomy allocation strategy guided by learning confidence and operator intention to ensure a smooth transition from human demonstration to autonomous robot operation. Experimental results using two 7 Degree of Freedom (DoF) Franka Panda robots demonstrate the framework's capability for online motion and force learning and adaptation within just a few iterations.", "summary_of_recommendation": "This paper presents a high-quality, innovative teleoperation framework for the online transfer and adaptation of tactile skills. The integration of Dynamical Movement Primitives (DMP) and Recursive Least Square (RLS) for tactile skill generation, coupled with a dynamic autonomy allocation strategy, demonstrates notable originality and significance. Empirical validation using advanced robotics hardware shows promising results in efficient skill adaptation. However, the framework's performance on more complex tasks, long-term stability, and deeper levels of human-robot collaboration need further exploration. Addressing these issues will further solidify the paper's contributions, making it a compelling addition to the field.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "1. Can the authors provide more details or preliminary results on how the framework performs with more complex tactile tasks? What are the anticipated challenges and how do they plan to address them?\n2. The framework relies on remote operation. How does it handle potential issues arising from noise and latency in the communication channels? Are there any mitigations in place to address these challenges?\n3. The success of DMPs and RLS often depends on the tuning of parameters. How sensitive is the framework to parameter settings, and what guidelines do the authors provide for optimal tuning?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "isvZMzDAy6", "reviewer": "Reviewer_32qn", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors present a framework for transferring skills from a human teacher to a robot. Hardware-wise, the setup consists of two panda arms. The human operator controls one of those arms while the student arm follows the arm reference force and motion profile generated by the operator. While doing this, a periodic DMP is trained in the background trying to mimic the motion/force profile. Over time, the DMP learns to mimic those profiles and the autonomy factor of the student arm increases, meaning the arm starts to perform the task on its own without human input. When the operator wants to change the motion, he/she can move the teacher arm again, leading to a decrease of the autonomy level of the student arm. Hence, the robot can interactively learn a new skill over time.", "summary_of_recommendation": "I think the approach is nice and according to the video and the evaluation it seems to work pretty well. I still think the paper could be made a lot stronger by adding some explanations and maybe improving the experiments section.", "technical_quality": 4, "clarity_of_presentation": 2, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- Please correct \"dimenstions\" in video caption\n\n- \"Passivity can be ensured by energy tank\": Could you add 1-2 sentences about this instead of just citing the paper?\n\n- You mention that you do not encode rotation but your task-space is limited to the translational components? What modifications would be needed to go to 6D?\n\n- It seems like the decision about which dimensions control force or motion is made beforehand and cannot be learned online. Is that true? What modifications would be necessary to support this?\n\n- In line 166 you mention \"without communication delay\" which is of course not true. Maybe say \"with minimal communication delay\"?\n\n- Why is there no comparison with a baseline method or with a state of the art method? If such a comparison is not feasible please elaborate on why you think that's the case.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c057p3HEgl", "reviewer": "Reviewer_encR", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces an DMP-based framework for learning and adapting tactile-aware manipulation skills. The authors show that the system is able to automatically imitate a teacher robot, and it can then gradually switch from imitation to automation with some heuristics.", "summary_of_recommendation": "This paper presents an interesting control framework where a student robot imitates a teacher robot and can automatically switch to autonomy based on some heuristics. While the learned skills look interesting, I think a few very important aspects are missing, such as details on assumptions and a sensitivity analysis. Also, I do not think the paper’s topic aligns with the main focus of CoRL.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 1, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "Please see more details in the review above.\n\n(1) What are the underlying assumptions of the proposed method?\n\n(2) How sensitive are the hyperparameters when it comes to learning a different skill?\n\n(3) What are the limitations of the learnable skills?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors propose a new method for teleoperation based on DMP. The method is new and the results are good. However, there are many confusing parts about the writing and we suggest the authors respond to the questions posted by the reviewers and improve the clarification of the paper.

-----
The authors addressed some questions posted by the reviewers. While the reviewers agree that the paper is of good quality and makes a good contribution, there is a concern about the weak correlation between the paper's method and CoRL.
Decision: Accept