Online Transfer and Adaptation of Tactile Skill: A Teleoperation Framework

Authors: xiao  chen
tianle  ni
k  übra karacan
hamid  sadeghian
sami  haddadin
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 01:59:38

Abstract: This paper presents a teleoperation framework designed for online learning and adaptation of tactile skills, which provides an intuitive interface without need for physical access to execution robot. The proposed tele-teaching approach utilizes periodical Dynamical Movement Primitives (DMP) and Recursive Least Square (RLS) for generating tactile skills. An autonomy allocation strategy, guided by the learning confidence and operator intention, ensures a smooth transition between human demonstration to autonomous robot operation. Our experimental results with two 7 Degree of Freedom (DoF) Franka Panda robot demonstrates that the tele-teaching framework facilitates online motion and force learning and adaptation within a few iterations.

Main paper: 
 introduction: 
 As robots become increasingly integrated into daily activities such as supporting household chores and elderly care [1] , the complexity of their tasks continues to grow. Learning from Demonstration (LfD) [2] has emerged as an efficient method to directly transfer expert skills to the robot, thereby reducing the time required for self-learning [3] . Several approaches, including first-order Dynamical Systems [4] , Gaussian Mixture Model [5] , and Dynamical Movement Primitives (DMP) [6, 7, 8] have been employed to encode the motion skills. However, motion skills alone are insufficient for complex tasks involving rich contact with the environment. Therefore, the incorporation of force skills is also essential. In [9] , a modulation term is added to DMP to enable environment interaction. A force profile is encoded in attractor trajectory though admittance relationship in [10] . An explicit force profile is learned in the format of DMP separately from motion [11] . Robots gain the tactile skill to handle complex tasks by combining motion and force profiles.A recent review on LfD [12] categorizes the demonstration method into three types: 1) Kinesthetic teaching, 2) Teleoperation, and 3) Passive observation. The passive observation method faces challenges when humans cannot access the robot because replicating the robot's environment is complicated. Kinesthetic teaching refers to a mode in which the robot is physically guided through a task by human operators through direct interaction. Although the kinesthetic teaching provides a user-friendly interface, distinguishing between human guiding and interaction forces with the environment involves complex algorithms. Consequently, force profiles must be taught separately through teleoperation in [11] . LfD via teleoperation enables the experts to transfer skills to a robot that is not easily accessible. By utilizing the tactile sensors of the operator's device [13] , the external force in kinesthetic teaching is now divided into the operator-applied force and the environment's interaction force, simplifying the process of learning tactile skills. A typical teleoperation system consists of a remote station where a robot executes the tasks and a human operator station with an input interface to control the remote robot. These input interfaces vary from a simple Mouse-Keyboard (MK) configuration [14] to intricate Avatar systems showcased [11] ✓ ✓ × × × PAPM [25] ✓ × × ✓ × FACT-DS [26] × ✓ × ✓ × RASHIL-RTSM [27] ✓× × × ✓ Ours ✓ ✓ ✓ ✓ ✓in the ANA XPRIZE Competition [15] . A commonly used input interface is haptic devices, allowing experts to teach haptic guidance references [16] or contact-rich tasks [10, 17] . However, due to the morphological differences between the haptic device and the execution robot, demonstrations using the haptic device are not very intuitive [18] .In this paper, we design a teleoperation framework that consists of two identical robot arms. One is located in the workspace to execute the tasks after learning them; the other one interacts with the human operator to demonstrate the tactile skill and allow the human operator to feel the remote environment. Our framework with two identical robot arms merges the intuitiveness of kinesthetic teaching with the capability for force learning to address the problem mentioned earlier in teleoperation. The goal is to learn and adapt periodic tactile skills, encompassing motion and force profiles, from remote human demonstrations without interrupting skill execution. Compared to the traditional Leader-Follower teleoperation architecture, the leading role transits between the two robots according to the designed autonomy level allocation. During the teaching phase, the human side takes the lead, and once teaching is complete, the robot in the workspace resumes the leading role and executes the tactile skill autonomously. This role-switching mechanism is known as shared autonomy [19] . The autonomy level allocation relies on metrics such as human intention [20] , human ability [21, 22] , or confidence in the learned skills [23] . In our work, autonomy allocation depends on the user's intention and confidence in skill learning. Another benefit of autonomy allocation is that our framework allows humans to intervene and adjust the learned skill online. Typically, adapting to new skills occurs offline and needs an additional operation to reset the robot [24] .The contributions of this work are as follows:• A tele-teaching framework that enables a manipulator to simultaneously learn force and motion profile, thereby acquiring tactile skill from a remote demonstration.• A mechanism for online learning and adaptation of tactile skills without interruption through the proposed framework.• The autonomy allocation based on operator intention and skill learning confidence provides intuitive interaction between operator and robot. 
related work: 
 The main differences between our work and other methods are listed in Table 1 . The IPFS-KTHI method [11] teaches motion and force profiles separately. Motion profiles are taught through Kinesthetic teaching, and the force profile is taught through a haptic device, which could lead to a lack of synchronization between motion and force. The RASHIL-RTSM approach [27] integrates a human sensorimotor system into the robot control loop through a teleoperation setup for remote teaching.Inspired by this framework, we use a Teacher Robot instead of the human sensorimotor system for remote teaching. The method most closely related to our periodic motion learning module is PAPM [25] , which focuses only on tactile skills that involve constant contact force rather than learning force profiles through demonstration. Additionally, we expand the autonomy allocation strategy from PAPM, initially designed for motion adaptation in a single robot manipulator, to facilitate simultaneous motion and force adaptation within our teleoperation framework. For learning about contact forces, FACT-DS [26] position-dependent to adjust for these irregularities, in our work, force profiles are time-dependent, which allows for learning the forces shown by the user.The primary innovation of our framework, distinguishing it from existing methods, is the simultaneous learning and adaptation of motion and force profiles-essential for tactile skills. This feature leverages the capabilities of our teleoperation framework, which can distinguish human guiding forces from external forces exerted on the robot without an additional force-torque sensor.3 Methodology 
robot control in tele-teaching: 
 The tele-teaching system consists of two identical robots to enhance the intuitiveness. The Teacher Robot (TR) is positioned at the human operator side, allowing humans to demonstrate tactile skills while feeling the tactile feedback of the remote environment. Conversely, the Student Robot (SR) is located remotely to execute the learned tactile skill. The control architecture described below illustrates how the Teacher Robot (TR) and Student Robot (SR) interact in terms of motion and forces to implement this tele-teaching system.The sensory data of the robot systems including position x t , x s , measured external force f h,t , f ext,s , and the calculated autonomy level η are exchanged through the communication channel and utilized within the controller (Figure 1 ). The dynamics of a gravity-compensated TR is written as,EQUATIONwhere M c,t ∈ R n×n and the C c,t ∈ R n×n are the inertia and Coriolis/centrifugal matrices of the TR represented in task space with n dimensions, respectively. x t ∈ R n is the TR end-effector position represented in the task space. f h,t ∈ R n is the force applied by/to the human operator during the interaction with TR . u t ∈ R n is the control command generated by the TR controller, and η ∈ [0, 1] is the autonomy level that acts as an activation parameter to the controller. A P D+ controller is employed on TR to track the position of SR and provide the haptic feedback,EQUATIONwhere K t ∈ R n×n and D t ∈ R n×n represent the stiffness and the damping matrices, respectively. xt = x d,t -x t denotes the trajectory following error between the TR and the desired trajectory x d,t ∈ R n , which is transmitted from SR .On the SR side, similar dynamics with gravity compensation are written as,EQUATIONwhere M c,s ∈ R n×n and the C c,s ∈ R n×n are the inertia and Coriolis/centrifugal matrices of the SR represented in task space, respectively. u s ∈ R n denote the control command to SR :EQUATIONwhere the u uf ic,s is the unified force/impedance control [28] to follow the reference tactile skill:EQUATIONThe control command u m,s is the impedance controller for tracking motion profile,EQUATIONwhere xs = x ref,s -x s is the tracking error and K s ∈ R n×n and D s ∈ R n×n are the impedance matrices. Moreover, u f,s is a P ID controller for force tracking with feed-forward term,EQUATIONwhere f s = f ref,s -f ext,s represents the tracking error for force. The passivity of the controller can be ensured by designing an energy tank [29] , which provides an "energy budget" and releases energy for the execution of non-passive control actions by modifying the control command.The control force u t,s for the SR is designed to track the TR through both position and force channels, i.e.,EQUATIONNotably, for simplicity, only the translational dimensions are considered for the task space coordinate, i.e., n = 3. The orientation is constrained by high impedance, and the null space of the robot is controlled to maintain a specific joint configuration with best-effort in a compliant manner while ensuring task execution [30] .The generated reference tactile skill comprises both motion and force profiles. For the reference motion profile, a periodic DMP formalism is adopted to generate the reference motion [7] :EQUATIONwhere α x and β x are constants, along with the anchor point x g ∈ R n . Note that the force profile is not encoded by the DMP in Eq. 9 due to irreducible noise in the force reference signal f ext,s from the sensor. Derivatives of the force signal, ḟext,s and fext,s , will be too noisy to be effectively encoded with DMPs. To ensure synchronization across different dimensions, the basic frequency of the generated tactile skill is determined as the minimum frequency value among all the dimensions.EQUATIONThe change rate of the canonical phase is the learned frequency, i.e., ṡ = Ω. Additionally, γ m is a linear combination of N Radial Basis Functions (RBFs) as following,EQUATIONEQUATIONwhere, c i represents the center of RBF, h denotes the width, and r is the amplitude modulation parameter which is set to 1. Each vector w i,m ∈ R n of the weights matrix w m ∈ R n×N is updated with the Recursive Least Square (RLS) algorithm [31] :EQUATIONwhere σ i,m ∈ R n is the vector of inverse covariance associated to the weights w i,m . Each element of σ i,m is updated asEQUATIONλ f g is the forgetting factor affecting the convergence speed. The initial value of P j,i,m (0) is set to 1 to avoid getting stuck at zero.The goal of RLS is to minimize the error e r,i,m which is defined as,EQUATIONNotably, the term (1 -η) ensures the error is set to zero when the SR executes autonomously, thus preventing updates of the weight matrix. The target is defined differently for motion and force. The motion profile is generated using DMP, and the target is defined asEQUATIONNote that the subscriptm employed in the previous equations denotes their applicability to motion profile generation. For the force profile, due to the high noise of the first and second-order derivative of the measured force signal, the measured external force is directly assigned as the target value,EQUATIONSubstituting the subscriptm withf in the above equations (excluding equation ( 9)), such that they are valid for force profile generation, the reference force profile is given as f ref,s = γ f .Autonomy allocation enables seamless transitions between human intervention and robot autonomy by assessing two factors: whether a skill has been successfully learned, and whether the user intends to end the current demonstration or start a new one. If the system determines that the skill has been learned and no new demonstrations are intended, it transitions to robot autonomy. Conversely, if the skill learning is incomplete or a new demonstration is indicated, the system reverts to human intervention mode. The autonomy level η ∈ [0, 1] on the SR side indicates whether the human or the robot is leading during the learning phase. At η = 0, the TR complies with the human's motion, and the SR is compliant to follow the motion of TR . Throughout the learning and adaptation process, η gradually smoothly changes the role of the robot system from compliantly following the human to following the generated reference tactile skill. After the skill learning and adaptation, the η increases to 1, indicating the learning process ends and the robot system becomes autonomous until the next human intervention. This autonomy allocation also indicates the role changing between leader and follower in teleoperation system, as disscussed in Appendix 6.2.1The rate of change of the autonomy level is defined as,EQUATIONwithEQUATIONwhere I h is a value that increases during human intervention for learning a new skill and given by,EQUATIONwith appropriate constant λ 1 . This way, by applying external force from the human demonstrator, the level of autonomy decreases to comply with the human. Moreover, which indicates the confidence of the SR in learning the tactile skill. Here, the ∥ e m ∥ denotes the error vector, which will be introduced in the next section. During the learning phase, and as the motion tracking error xs and force tracking error fs gradually goes below a certain threshold λ 3 and λ 4 , the value of I s decreases. This way, the confidence in learning increases, and thus, the autonomy level increases. To prevent getting stuck at zero, a small parameter ϵ is introduced in ( 19), while ρ acts as a scaling parameter. A tuning guidance is provided in the Appendix 6.1.1EQUATIONAdaptive frequency oscillators are tasked with identifying the basic frequency Ω of a demonstrated periodic skill in real time. During the learning and adaptation of a periodic tactile skill, the motion and the force undergo the same procedure, as illustrated in Figure 2 . A generalized state p represents both motion and force state. In the following equations, the subscriptm stands for motion profile, and the state p m = x s . To adapt the frequency oscillators for the force profile, we need to replace the subscript withf and go through the following formula again, and the state will be p f = f ext,s .Following the same approach proposed in [32] , the frequency oscillator is adapted by,EQUATIONEQUATIONEQUATIONwhere θ m ∈ R n is the vector of the corresponding phase, ω m ∈ R n is the vector of frequencies, from which the basic frequency Ω is acquired as Equation (10) (the basic frequency Ω is the minimum), ν m ∈ R is the coupling strength and E m ∈ R n×n is a diagonal matrix considering the error between the input signal p m and the estimation of the signal pm . The initial values of ω m and θ m are set to a small value to avoid being stuck at zero. Note that the bigger the coupling strength ν, the faster the adaptive frequency oscillator converges [31] , but the more unstable when ω is closer to zero. The learning rate µ m also impacts the convergence speed and algorithmic stability. A detailed sensitivity analysis of coupling strength and learning rate and the tuning strategy is provided in Appendix 6.1.2.The ith element of the estimation pm is computed asEQUATIONwhere M denotes the number of Fourier components. The amplitudesEQUATIONAs the frequency Ω is the denominator in the DMP expression in Eq. 9 and 16, thus we set a lower bound to the frequency in the experiment for safety. 4 Experiments and Results 
experiment setup: 
 The experimental setup consists of two 7 Degrees of Freedom (DoF) Franka Panda robots [33] , as depicted in Figure 3 . A sponge is fixed to the end-effector of the SR to interact with the table surface. The operator interacts with TR directly by grasping the robot end-effector. The data between TR and SR is transmitted through User Datagram Protocol (UDP) with negligible communication delay.The user demonstrates tactile skills on the TR, including motion on the XY plane and force in the Z direction. A series of different tactile skills is performed continuously to illustrate the simultaneous motion and force learning and adaptation capability of our tele-teaching framework. The values of the parameters used in the experiment can be found in Appendix 6.1.The experiment encompasses the demonstration of five periodic tactile skills denoted as S 1 , • • • , S 5 with planar XY motion and contact force in the Z direction. Specifically, the skills contain,• S 1 : a clockwise circular motion with a large changing contact force,• S 2 : a counter-clockwise circular motion with a small changing contact force,• S 3 : a linear motion with forceful push and gentle pull,• S 4 : an ∞ shape along the Y axis with reduced interaction force• S 5 : an ∞ shape along the X axis with increased interaction forceThe outcomes of the demonstrated tasks are depicted in Figure 4 . We start the first demonstration of the S 1 at around t = 16s. After five iterations (around t = 26s) of repetition, the reference tactile skill converges to the user's demonstration, which suggests higher confidence in learning. Meanwhile, the basic frequency of the motion and force converge at the same value, around Ω = 2.4rad/s. The convergence of the signal shape and frequency increases the autonomy level. The operator can feel that the robot becomes less compliant, indicating the acquisition of the new skill.After two more iterations at t = 33s, SR gains full autonomy and continuously reproduce the learned tactile skill. At around t = 41s, the operator initiates modifications to the previous skill, increasing the external force f h,t on TR, and the tracking errors make η drop quickly. Thus, it returns to the human demonstration mode and starts a new learning cycle. This process repeats for skills S 2 , • • • , S 5 , respectively. 2Overall, it takes around six iterations (16s) to learn skill S 1 and S 2 with a minimal frequency at around 2.4rad/s, three iterations (6s) for S 3 with 1.0rad/s, and six iterations (23s) for S 4 and S 5 with 1.5rad/s and 1.3rad/s. Notably, the frequency of 1.0rad/s in S 3 does not represent the actual frequency of the linear skill but rather the predefined minimal allowable frequency to prevent significant changes in DMP acceleration, as implicitly depicted in Eq. 16.In summary, the proposed tele-teaching framework realizes online learning and adaptation of different tactile skills in only a few demonstration shots. The successful learning of the above five tactile skills, which differ spatially and temporally, verifies the proposed approach's effectiveness in simultaneous motion and force learning, online adaptation and remote teaching listed in Table 1 . Furthermore, the autonomy allocation module obviates the need for robot resets between tactile skills, enabling seamless transitions between robot autonomy and human demonstration. This module empowers human operators to take over the control as needed, augmenting safety and reliability. 
conclusion and future work: 
 This work develops a tele-teaching framework to enable real-time learning and adaptation of tactile skills from remote demonstration. Tactile skills demonstrated through the Teacher Robot are seamlessly transferred online to the Student Robot located remotely. Our approach distinctively integrates motion and force into a single cohesive tactile skill.The tele-teaching framework demonstrates remarkable speed and efficiency in adapting to new tactile skills. The adaptation occurs smoothly within just a few iterations. The autonomy allocation is based on the learning confidence and the operator's intentions. This strategy not only ensures smooth transitions between human demonstration and autonomous robot execution but also empowers the user to dictate the cessation or initiation of teaching phases.We currently focus on simple periodic tactile skills with only translational motions and forces. Since the motion component is represented by Dynamic Movement Primitives (DMPs), the limitations associated with periodic DMPs are also applicable to our framework. Future works include exploring complex tactile skills that involve changes in orientation, potential stability issues led by communication delays, and expanding into deeper human-robot collaboration such as rehabilitation tasks. Forgetting factor: how sensitive to new coming data 
autonomy allocation: 
 The autonomy allocation module in our framework acts as a mechanism that is both aware of learning confidence and responsive to user intentions, employing heuristic methods. The assessment of whether a skill has been successfully learned depends on various factors, including learning errors related to motion, force, and frequency, as well as the external force applied to the Teaching Robot, which indicates user intentions, such as initiating a new demonstration. Parameters within the autonomy allocation module are configured to set thresholds based on these criteria.Given that variations in sensor and robot hardware can affect performance, it is essential to adjust these thresholds to align with the system's specific characteristics. To assist in this calibration process, we provide guidance on effectively tuning the parameters involved in autonomy allocation.λ 1 is a force threshold, affecting how sensitive to new demonstration intention. When the demonstration force on the Teacher Robot is larger than λ 1 , the Teacher Robot will gain the leading role. indicating readiness to receive a new demonstration.Other λ 2,3,4 indicates how tolerant to the learning error in frequency(λ 2 ), motion(λ 3 ), and force(λ 4 ).As human demonstration always contains noises, large λ 2,3,4 means more tolerance/less sensitivity to learning error. Too small λ 2,3,4 can cause longer learning iterations, and too large λ 2,3,4 can cause insufficient learning of the tactile skills.ρ affects how smooth the transition between robot autonomy to human intervention mode. Larger ρ leads to a more smooth transition. A common choice is set around 1.Coupling strength ν m,f and learning rate µ m,f are the critical parameters for adaptive frequency oscillators(AFO). They are related to the convergence speed, algorithm stability, and accuracy of the AFO. Regarding the number of fouriers M , we find that M = 1 is already enough for the periodic skills demonstrated in this paper.A sensitivity analysis of these two parameters provides insights into how changes in coupling strength and learning rate impact the learning process of the motion's frequency. Note that the analysis also applies to learning the force's frequency. Figure . 5 illustrates the effect of different set- tings for coupling strength ν m and learning rate µ m on learning performance, with the target motion recorded from actual demonstrations of circular motions on the robot. The analysis of the coupling strength in the Figure . 5b) reveals that a higher coupling strength leads to quicker convergence but introduces larger initial oscillations. The analysis of the learning rate in the Figure . 5c ) shows that a higher learning rate accelerates convergence but will cause instability when above a certain threshold. In summary, there is a trade-off between achieving rapid convergence and minimizing oscillations when tuning the coupling strength and learning rate.Building on the insights from our sensitivity analysis, we propose an error-driven dynamic adjustment for the parameters ν m,f and µ m,f . Note that here we use the error between input signal and the estimation of the signal in AFO. This strategy is designed to ensure rapid convergence when errors are large and to mitigate oscillations when errors are small. Specifically, both the coupling strength and learning rate are configured to decay progressively as the error decreases. The implementation of this dynamic adjustment is demonstrated Figure . 5d ). The results indicate that, compared to scenarios with fixed parameters, this adaptive method maintains swift convergence while simultaneously minimizing oscillations, thus optimizing the learning process for adaptive frequency oscillators.To show the sensitivity of coupling strength and learning rate in AFO when learning different skills.We test the performance of frequency learning with varying sets of parameters across the five periodic tactile skills demonstrated in the experiment section. As shown in the Figure . 6a), the frequency of motions in the y-dimension changes after every new demonstration. Figure 6d ) reflects the robustness of our dynamic coupling strength and learning rate adjustment under different skills. Compared to the fixed setting shown in Figure 6b ) and c), we observe faster convergence when adapting to a new skill and fewer oscillations(more accuracy) for the steady-state of learned frequency.The forgetting factor in the Recursive Least Squares (RLS) algorithm plays a crucial role in determining how much historical data influences current model updates. It dictates the size of a "sliding window" of historical data considered in the regression. For periodic skills, a good choice of the forgetting factor is intricately linked to the periods of the demonstrated skill. This relationship spec- ifies the number of demonstration cycles considered relevant for updating the learning model. An approximate relationship exists between the forgetting factor and the time window size over which historical data remains significant within the RLS framework. This relationship can be expressed mathematically, as shown in Equation 27. A larger forgetting factor expands the time window over which historical data is considered valid, thereby incorporating more past data in the regression.EQUATIONwhere 1 1-λ f g is the estimated time window. k decides how many rounds of demonstrations are considered. If k is set to four, the learned skill is approximately an average of four last demonstrations.The RLS adapts slowly to recent changes, with a high λ f g . Conversely, a low λ f g makes RLS sensitive to new observations. To clearly show how the forgetting factor affects the learning procedure, we compare three sets of constant forgetting factors(k = 0.5, 2, 8) under three nearly identical demonstrations of a circular movement on the x-y plane. It is challenging to maintain identical force profiles, so we omitted the force in this experiment. As shown in Figure 7a ) and b), a small forgetting factor(when k = 0.5) helps quick convergence at the beginning of a new demonstration, trivial demonstration divergence will lead to shape distortion(between 21s and 27s). As the robot becomes less compliant with the user during the autonomy transition, it is hard for the user to demonstrate identical motions. Therefore, a larger forgetting factor is necessary during the transition to make it less sensitive to recent data. However, as shown in Figure 7e ) and f), a larger forgetting factor(when k = 8) leads to less sensitivity to recent changes, resulting in a longer convergence time.In conclusion, the optimal forgetting factor setting is a small value corresponding to k = 0.5 when the error is big, and a large value corresponds to, e.g., k = 2 ∼ 8 when the error is small. The set of forgetting factors follows an error-driven dynamic adjustment, increasing for rapid convergence during significant errors and decreasing for stable autonomy transitions during minor errors. Note that here we use the error between the input signal and the learned signal in RLS. Regarding the number of RBFs N , we find that the common setting of N ranging from 30 to 50 fits to the skills demonstrated in this paper. The autonomy level η dictates role allocation between the human and the robot system, resulting in the role change of the TR and SR . When η = 0, the SR completely follows the TR by both position and force channel. However, when the learning process finishes with η = 1, the TR transitions into a follower role, synchronizing its motion with the SR . Traditional leader-follower structure does not include the autonomy level, and SR only follows the TR by force. We realize that using both position and force channels from TR to SR is necessary in the tele-teaching framework. If only the force channel were used, minor position deviations would exist between TR and SR during the learning process(η=0) because the two robot arms are not entirely identical in practice. As the autonomy level rises, the leading role transits from TR to SR . TR would follow SR gradually more in the motion channel, and the minor position deviations would be compensated. However, the human operator at TR 's side would feel that the SR learns a different motion. Therefore, the operator tends to naturally resist the compensation and keep the original demonstration, making the autonomy level η fall again. The minor position deviations lead to unsmooth leading role transitions from TR to SR , resulting in unsuccessful tactile skill transfer. Furthermore, LfD via teleoperation presents certain drawbacks. Reference [18] shows that demonstrating a peg-in-hole task through teleoperation using a haptic device is more difficult for the operator than kinesthetic teaching. Users tend to demonstrate trajectory with higher variance. However, the method proposed in this work offers potential improvements. Another concern might be the stability issue with sub-optimal communication quality, leading to delays and packet loss. This problem can be alleviated by passivity-based methods like the Time Domain Passivity Approach (TDPA) [34, 35] or energy-based methods [36, 37] . 
 
 Appendix: The experiment video is available at https://youtu.be/LYnUJ0cYJgs 


Decision: The authors propose a new method for teleoperation based on DMP. The method is new and the results are good. However, there are many confusing parts about the writing and we suggest the authors respond to the questions posted by the reviewers and improve the clarification of the paper.

-----
The authors addressed some questions posted by the reviewers. While the reviewers agree that the paper is of good quality and makes a good contribution, there is a concern about the weak correlation between the paper's method and CoRL.

Reviews: review: title: title: Evaluating the Teleoperation Framework for Online Transfer and Adaptation of Tactile Skills
paper_summary: paper_summary: The paper presents an innovative teleoperation framework for the online learning and adaptation of tactile skills, eliminating the need for physical access to the execution robot. The proposed tele-teaching approach employs periodical Dynamical Movement Primitives (DMP) and Recursive Least Square (RLS) for generating tactile skills. The framework incorporates an autonomy allocation strategy guided by learning confidence and operator intention to ensure a smooth transition from human demonstration to autonomous robot operation. Experimental results using two 7 Degree of Freedom (DoF) Franka Panda robots demonstrate the framework's capability for online motion and force learning and adaptation within just a few iterations.
main_review: main_review: ### Quality
The quality of the paper is commendable, with a thorough explanation of the teleoperation framework and its components.

### Clarity
The paper is clear and concise, effectively conveying complex concepts in an accessible manner. 

### Strengths:
1. The integration of DMP and RLS for tactile skill generation is a novel approach, providing a robust solution for tele-teaching.
2. The strategy based on learning confidence and operator intention ensures smooth transitions and effective human-robot collaboration.

### Weaknesses
1.  The paper briefly mentions future work on more complex tactile skills. The current experiments seem to focus on relatively simpler skills, and the framework's performance on more complex tasks is not demonstrated.
2.  While the framework shows promise in transitioning between human and robot control, deeper levels of human-robot collaboration are not explored in this paper. Further exploration could provide a more comprehensive understanding of the framework's capabilities.
questions: questions: 1. Can the authors provide more details or preliminary results on how the framework performs with more complex tactile tasks? What are the anticipated challenges and how do they plan to address them?
2. The framework relies on remote operation. How does it handle potential issues arising from noise and latency in the communication channels? Are there any mitigations in place to address these challenges?
3. The success of DMPs and RLS often depends on the tuning of parameters. How sensitive is the framework to parameter settings, and what guidelines do the authors provide for optimal tuning?
limitations: limitations: 2
review_summary: review_summary: This paper presents a high-quality, innovative teleoperation framework for the online transfer and adaptation of tactile skills. The integration of Dynamical Movement Primitives (DMP) and Recursive Least Square (RLS) for tactile skill generation, coupled with a dynamic autonomy allocation strategy, demonstrates notable originality and significance. Empirical validation using advanced robotics hardware shows promising results in efficient skill adaptation. However, the framework's performance on more complex tasks, long-term stability, and deeper levels of human-robot collaboration need further exploration. Addressing these issues will further solidify the paper's contributions, making it a compelling addition to the field.
title: title: Valuable contribution but the paper needs to be improved
paper_summary: paper_summary: The authors present a framework for transferring skills from a human teacher to a robot. Hardware-wise, the setup consists of two panda arms. The human operator controls one of those arms while the student arm follows the arm reference force and motion profile generated by the operator. While doing this, a periodic DMP is trained in the background trying to mimic the motion/force profile. Over time, the DMP learns to mimic those profiles and the autonomy factor of the student arm increases, meaning the arm starts to perform the task on its own without human input. When the operator wants to change the motion, he/she can move the teacher arm again, leading to a decrease of the autonomy level of the student arm. Hence, the robot can interactively learn a new skill over time.
main_review: main_review: In general, the paper is well written even though I think the writing could still be improved.
The controller framework is nicely explained and contains sufficient detail, but the tactile skill generation section (2.2) and some other parts lack clarity. Here is the summary of my points:

Strengths:
- The control architecture in section 2.1, which is one of the core parts of the paper, is nicely explained
- According to the video, the approach seems to be working well and is a valuable contribution to the scientific community.

Major weaknesses:
- There is no distinct related work section. The introduction summarizes the state of the art but often the authors miss to highlight the differences between their work and the state of the art which makes it difficult to judge the novelty of the presented approach.
- The experiments do not contain any comparison with state of the art or a baseline method.  
- I found the tactile skill generation section (2.1) a bit confusing. The authors present the DMP formulation and in the end of the section they mention that they do not really use DMPs for encoding the force profile. I suggest to mention in the beginning of the
section how motion and force are encoded before going into detail with the equations. The same is true for the beginning of Section 2. I think the reader would appreciate if the authors gave a short explanation of the individual parts (control architecture, frequency oscillator, DMPs, ..)
at the beginning of Section 2 before going into detail about the individual parts.
- The authors give an intuition about tuning the parameters in the discussion section but I wonder why no experiment was made about this so that the reader sees how learning performance is affected by some of the parameters.

Minor things:
- In the video there is a typo "dimenstions" instead of "dimensions"
- There are many equations in the paper which makes it difficult to maintain an overview over the type of variables (e.g., learned, parameter, input, output). Maybe a table with an overview would help here.
questions: questions: - Please correct "dimenstions" in video caption

- "Passivity can be ensured by energy tank": Could you add 1-2 sentences about this instead of just citing the paper?

- You mention that you do not encode rotation but your task-space is limited to the translational components? What modifications would be needed to go to 6D?

- It seems like the decision about which dimensions control force or motion is made beforehand and cannot be learned online. Is that true? What modifications would be necessary to support this?

- In line 166 you mention "without communication delay" which is of course not true. Maybe say "with minimal communication delay"?

- Why is there no comparison with a baseline method or with a state of the art method? If such a comparison is not feasible please elaborate on why you think that's the case.
limitations: limitations: 3
review_summary: review_summary: I think the approach is nice and according to the video and the evaluation it seems to work pretty well. I still think the paper could be made a lot stronger by adding some explanations and maybe improving the experiments section.
title: title: Interesting contact-aware tele-operation system with a few important aspects missing
paper_summary: paper_summary: This paper introduces an DMP-based framework for learning and adapting tactile-aware manipulation skills. The authors show that the system is able to automatically imitate a teacher robot, and it can then gradually switch from imitation to automation with some heuristics.
main_review: main_review: This paper introduces an DMP-based framework for learning and adapting tactile-aware manipulation skills. The authors show that the system is able to automatically imitate a teacher robot, and it can then gradually switch from imitation to automation with some heuristics. The paper is overall well written. However, in my opinion a few important points are unclear:
Assumptions of the system are not clearly presented. E.g. the paper seems to assume that the learned skill is always periodical - what is the permissible range of period?
The paper lacks a sensitivity analysis or failure mode analysis. The system requires quite a few hyperparameters (16 according to Tab. 1), and it is unclear whether those parameters are robust and transferable between skills.
What is the underlying infrastructure for robot control?
Furthermore, in my opinion the paper’s impact is limited in that (1) it only applies to periodic skill learning and (2) the formulation of some important components of the system such as Eqn.20 and Eqn.21 is heavily based on heuristics. Although the paper presents an interesting framework that works well as it seems from the demo video, I do not think its topic aligns well with the main focus of CoRL.

In summary, I do not think this paper is suitable for publication at CoRL. Some more detailed comments:

Introduction: I’m not sure what “kinesthetic teaching” refers to and what is the difference between it and teleoperation. I assume it means scenarios where human operators directly move the robot arms - but it’s still quite ambiguous. Consider adding a short explanation.

Eqn. 2
TR’s control input u_t is calculated based on the positional difference between TR’s and SR’s trajectories. I’m wondering (1) instead of solely relying on delta positions, have the authors tried feeding back the force reading of SR to TR? (2) If positional difference is the only input for u_t, it is a traditional trajectory following system and in my opinion there is no haptic feedback from SR to TR and the text is a bit misleading.

“the null space of the robot is controlled according to the method outlined in [29]” - please provide an overview of the method as it is quite disruptive to scroll  to the end of the paper then scroll back.


Eqn.21
Consider using a different term for C_s, as C has already been used in Eqn.1 for Coriolis terms.

Eqn. 19
I’m also not sure why the confidence term is needed to determine the level of autonomy. If I understand it correctly, lower tracking errors leads to lower C_s, which leads to higher autonomy, which means the student robot relies more on replaying its learned skills rather than following the teacher’s motion - but why shouldn’t the student robot follows the teacher’s motion at all times as long as f_{d,s} is non-zero? Otherwise, wouldn’t the tracking error become larger and larger?

It is interesting to see the learning and adaptation processes of the 5 skills. However, I’m concerned about how sensitive the system is with regard to hyperparameters and whether a set of hyperparameters can adapt to a different skill. A sensitivity analysis, or a failure case analysis, which is actually required by CoRL (https://www.corl.org/contributions/call-for-papers), will be helpful.
questions: questions: Please see more details in the review above.

(1) What are the underlying assumptions of the proposed method?

(2) How sensitive are the hyperparameters when it comes to learning a different skill?

(3) What are the limitations of the learnable skills?
limitations: limitations: 1
review_summary: review_summary: This paper presents an interesting control framework where a student robot imitates a teacher robot and can automatically switch to autonomy based on some heuristics. While the learned skills look interesting, I think a few very important aspects are missing, such as details on assumptions and a sensitivity analysis. Also, I do not think the paper’s topic aligns with the main focus of CoRL.