# A Robot Learning Method with Physiological Interface for Teleoperation Systems

Jing Luo, Chenguang Yang, Hang Su, Chao Liu

► To cite this version:

Jing Luo, Chenguang Yang, Hang Su, Chao Liu. A Robot Learning Method with Physiological Interface for Teleoperation Systems. Applied Sciences, 2019, 9 (10), pp.2099. 10.3390/app9102099. lirmm-02315641

HAL Id: lirmm-02315641

https://hal-lirmm.ccsd.cnrs.fr/lirmm-02315641

Submitted on 14 Oct 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

Article

# A Robot Learning Method with Physiological Interface for Teleoperation Systems

Jing Luo<sup>1,2 ID</sup>, Chenguang Yang<sup>3,* ID</sup>, Hang Su<sup>4 ID</sup> and Chao Liu<sup>2 ID</sup>

Received: 16 December 2018; Accepted: 10 May 2019; Published: 22 May 2019

<figure><img src="image_3.png" /></figure>

check for
updates

**Abstract:** The human operator largely relies on the perception of remote environmental conditions to make timely and correct decisions in a prescribed task when the robot is teleoperated in a remote place. However, due to the unknown and dynamic working environments, the manipulator's performance and efficiency of the human-robot interaction in the tasks may degrade significantly. In this study, a novel method of human-centric interaction, through a physiological interface was presented to capture the information details of the remote operation environments. Simultaneously, in order to relieve workload of the human operator and to improve efficiency of the teleoperation system, an updated regression method was proposed to build up a nonlinear model of demonstration for the prescribed task. Considering that the demonstration data were of various lengths, dynamic time warping algorithm was employed first to synchronize the data over time before proceeding with other steps. The novelty of this method lies in the fact that both the task-specific information and the muscle parameters from the human operator have been taken into account in a single task; therefore, a more natural and safer interaction between the human and the robot could be achieved. The feasibility of the proposed method was demonstrated by experimental results.

**Keywords:** bioinformatics interface; locally weighted regression; human muscle stiffness; learning from demonstration; human-robot interaction

## 1. Introduction

With significant progress in computer science, information science, automation, and artificial intelligence techniques, telerobots have been extensively applied in areas as diverse as telemedicine [1,2], telerehabilitation [3], minimally invasive surgery [4], disaster rescue and relief operation [5], maintenance and exploration in deep sea or out space [6,7], surgeon training [8], and telemanufacture [9], etc. Telerobots provide an alternative interactive way between the human operator and the teleoperation in order to enhance perception and motion ability of the human beings [10,11]. It is the integration of human intelligence and the robot's advantages under the constraint of long distance [12,13]. The performance of teleoperation largely depends on the perception of remote environmental conditions.

During the recent years, many achievements indicated that some related algorithms such as impedance control, virtual fixture, and shared control, could further improve the performance of teleoperation. According to literature [14], a switched-impedance control algorithm was presented

<footnote n="1">1 Key Lab of Autonomous Systems and Networked Control, School of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, China; jingluo.ac@gmail.com</footnote>

<footnote n="2">2 Department of Robotics, LIRMM, University of Montpellier—CNRS, UMR 5506, 161 Rue Ada, 34095 Montpellier, France; liu@lirmm.fr</footnote>

<footnote n="3">3 Bristol Robotics Laboratory, University of the West of England, Bristol BS16 1QY, UK</footnote>

<footnote n="4">4 Department of Electronics, Information and Bioengineering, Politecnico di Milano, 20133 Milan, Italy; hang.su@polimi.it</footnote>

<footnote>* Correspondence: cyang@ieee.org</footnote>
and implemented for teleoperated heart surgery and two switched reference impedance models were designed for the master and slave robots to achieve both the stabilities of motion and force feedback. Mojtaba et al. proposed a nonlinear adaptive impedance controller with two impedance models in Cartesian space for the master and slave robots based on the consideration of dynamic uncertainties. Besides, the parameters of the impedance model were adjusted according to the objectives and requirements of the applications. This proposed adaptive impedance control method has been successfully applied in certain medical applications [3]. Furthermore, according to literature [15], a force reflection framework based on gain force observer, was developed to acquire the force information. By considering internal uncertainties and time-varying delay of the system, a satisfactory performance of the system was guaranteed for the teleoperation system. Moreover, haptic assistance method involving shared control was a choice to enhance the performance of teleoperation [16–18]. In a previous study [16], a shared-based control method was proposed for a teleoperation system and implemented in haptic shared control (HRC) and state shared control (SSC). A decision-making model with impedance control algorithm was presented to improve the tracking capability in partially unstructured environments for a teleoperation system [18]. Virtual fixture-based control algorithms were proposed to improve the performance of robot teleoperation for a cooperative task [19,20]. In our previous study, a neural network (NN) control method with wave variable was proposed to cope with the problems of time delay and dynamics uncertainties for the teleoperation system [21]. According to literature [22], a variable gain control method with tremor attenuation was developed to adapt to the unstructured environments and to reduce the reliance on humans’ operation skill. The above-mentioned algorithms could successfully deal with the problems of model uncertainty. However, they could not guarantee the improvement of efficiency of the prescribed task and release of the work pressure of the human operator.

In order to improve the efficiency of the teleoperation and to reduce the workload of the human operator, many researchers have focused on how to deliver humans' skills to the robots with a short processing time and few trials. The results indicated that robot learning method could significantly improve the efficiency of the teleoperation and reduce the operational pressure for the human operator. Schaal proposed a learning form demonstration (LfD) framework for the learning control method [23]. Lin et al. [24] proposed a remote lead through teaching (RLTT) with a dynamic time warping (DTW) method to learn the task knowledge from human demonstration [24]. Moreover, a hidden semi-Markov model (HSMM) was presented to learn a task model for the purpose of offering assistance for remote operation tasks [25]. A probabilistic method was developed to construct a task model in the assistive teleoperated operation. The performance of the teleoperation was improved by the method and demonstrated by remotely operated vehicle (ROV) tasks [26,27]. In order to simplify the complexity of the system and to cope up with the varying dynamical interaction, Huang et al. [28] developed a hierarchical interactive learning (HIL) algorithm with dynamic movement primitives (DMPs) and locally weighted regression (LWR) to learn the task trajectories for an exoskeleton system. Furthermore, researchers also employed the human-in-the-loop method with shared control and LWR for robot learning online in the non-trivial task [29,30]. Pervez et al. utilized Gaussian mixture model (GMM) with DMP model to encode the demonstrated trajectories in peg-in-hole tasks. The experimental results demonstrated that the developed method could cope up with the problem of teleoperation tasks with large variabilities [31]. Ravankar et al. [32] investigated the path smoothing algorithms for the robot navigation task. A novel human-exoskeleton interface was proposed [33] to demonstrate the performance of cable driven arm exoskeleton (CAREX) with a cable tension planner for smooth motion. The above-mentioned robot learning methods could successfully improve the work efficiency of the teleoperation by learning human skills. However, these algorithms rarely consider the relationship between remote environments and the human operator operational characteristic in the process of human-robot interaction (HRI).

In a specific task with HRI, a human operator can adjust his stiffness of upper limb to deal with external disturbance. For example, the human operator can increase his stiffness or force when the
resistance force of external environment gets larger in a natural way. Inspired by this, we tend to transfer the human stiffness to the robot in a real-time way enhance its interactive abilities. To be specific, electromyography (EMG) signals represent the comprehensive impact of motor unit action potentials (MAUP) of the muscle fiber both in time and in space. The EMG signals have the following three applications: indicator of the muscle activation, representation of force generated by the human muscle, and a viewer of the fatigue for the human muscle. As an indicator of the muscle activation, the surface electromyography (sEMG) signal includes more information such as force and stiffness related to human motion control information. Therefore, we extract the stiffness from the sEMG signals and transfer it to the slave device of the teleoperation system. EMG-based technologies are widely used for the robotics. Meattini et al. [34] proposed an sEMG-based HRI interface with pattern recognition and factorization methods to provide a natural interaction between humans and robots. Furthermore, a dynamic switching method [35] was presented to predict human operator's behavior and to provide the most effective control according to the given context or the specific task in myoelectric training tool (MTT) platform. Antuvan et al. embedded a human as a part of the controller in the system to provide a myoelectric interface in different control tasks. The method was verified through the experiments involving the muscle activation of human upper limb [36]. According to literature [37], an HRI control interface was developed by integration of EMG signals and different functionally effective muscle (FEM) methods for a robotic wheelchair. A position-independent decoding movement intention method based on proportional myoelectric interface was proposed for the purpose of estimation of possibility of various arm positions [38]. It was concluded that the muscle activation could be used as an interaction tool to capture the information details of the remote operation environments. In our previous study [39], a hidden semi-Markov model with Gaussian mixture method was proposed to carry out the repetitive tasks with enhanced intelligence.

Inspired by the robot learning algorithms and EMG-based control method, in this study, a nonlinear regression model with physiological interface was constructed to relieve workload of the human operator to improve efficiency of the teleoperation system, and to capture the information details of the remote operation environments. First, the DTW method was employed to guarantee the demonstration data in the same time scale before proceeding with other steps in demonstration process (In our previous work [39], we did not consider the influence of the length of demonstration data). Secondly, a human-centric interaction method based on muscle activation was developed to collect the operation information and to actively capture the remote environment on-line. Besides, in order to improve the efficiency of the teleoperation system and to reduce human operator's workload in a natural way, the LWR method was proposed to model the teleoperated prescribed task based on the collected task trajectories and the human muscle stiffness. Finally, the feasibility and efficiency of the proposed method was verified by the experimental results.

The remaining part of this study is organized as follows. Section 2 presents equipment used for a MYO armband, Touch X and Baxter robot. In Section 3, the proposed algorithms including physiological interface design, task demonstrations, and learning method are described. Results are presented in Section 4. The conclusion and the future work are presented in Section 5. The main notations are presented in Table A1 in order to facilitate reading.

## 2. Equipment

### 2.1. MYO Armband

Figure 1a shows that the MYO armband was used to detect raw sEMG signal, which consists of eight electrodes and nine-axis inertial measurement units (IMU). The MYO armband was produced by Thalmic Labs, Canada and it can be easily adorned in upper limbs of human with a default frequency of 200 Hz. Figure 1b demonstrates the detection of the sEMG signals using the MYO armband with eight channels.
<figure><img src="image_4.png" /><figcaption>**Figure 1.** Detection of sEMG signal by using the MYO armband [22].</figcaption></figure>

## 2.2. Touch X

The Touch X serves as a master device to control remote slave device (Figure 2a). The Touch X with six degrees of freedom (DoF) was manufactured by 3D Systems, Inc. and it could capture the position (three DoFs) and its orientations (another three DoFs) with force feedback. The Touch X is widely applied in various areas in terms of simulation, virtual assembly, robot control, etc. (The kinematics information of the Touch X with respect to its forward/inverse kinematics and Jacobian matrix have been deeply studied [40,41]). In this study, Touch X was controlled via the Matlab/Simulink [42].

## 2.3. Baxter Robot

Figure 2b exhibits that the Baxter robot (Rethink Robotics. made) was composed of two arms with seven DoFs, a torso and a head with a radar. Each robot arm was designed with eight links and seven rotational joints (The mechanical structure between Baxter and Touch X is different, therefore, a workspace matching method was developed in our previous studies [21,22]).

<figure><img src="image_5.png" /><figcaption>**Figure 2.** The master device (a) and slave device (b) [22].</figcaption></figure>

# 3. Method

Figure 3 shows that the developed scheme consists of the following three modules: physiological interface module, demonstrations and robot learning module, and robot execution module.

* Physiological interface module. Physiological interface module consists of sEMG signal processing unit and sEMG-signals-stiffness translation unit. In this module, raw sEMG signal is collected by the MYO armband. The human stiffness can be obtained through the sEMG-signals-stiffness unit.

* Demonstrations and robot learning module. This module is the main part of the proposed frame. The data processing unit is used to process task trajectories from the remote Baxter robot.
The collected data contains task trajectories and human stiffness. The collected task trajectories and collected stiffness are ready for the demonstration and learning process. DTW unit is used to align the demonstration data in a united time scale. The robot learning model is mainly used to obtain a generative model according to the demonstrated information related to the collected task trajectories and collected muscle stiffness.

* Robot execution module. This module is mainly used to enable the remote Baxter robot to execute the task according to the learned task trajectories and learned muscle stiffness.

<figure><img src="image_6.png" /><figcaption>Figure 3. Schematic illustration of the proposed algorithm.</figcaption></figure>

### 3.1. Physiological Interface Design

In the teleoperation system, the human operator manipulates the robot remotely, and the slave movement follows that of the master synchronously. In general, the interaction information between the human operator and the remote environments can be reflected by variability of the muscle activation of the human operator in the process of manipulation.

In this study, the MYO armband with 200 Hz was used to collect the raw sEMG signal with eight channels [22]. According to the sampled sEMG signal, the sum of sEMG signal can be defined as follows:

$$u = \sum_{i=0}^{7} \hat{u}_i, \quad i = 0, 1, \dots, 7. \qquad (1)$$

where $u$ indicates the amplitude of sEMG signal and $\hat{u}_i, i = 0, 1, \dots, 7$ represents the raw signal.

The obtained sEMG signal is influenced by noise, therefore, a moving average filter is presented as follows:

$$\tilde{u} = \frac{1}{K} \sum_{k=1}^{K} u \qquad (2)$$

where $K$ is the moving window size. $\tilde{u}$ is the filtered sEMG signals through the moving average filter. Then, an envelope of sEMG signals can be obtained as follows:

$$\tilde{u} = \sum_{j}^{j+M} W_{M-j} \tilde{u}_j \qquad (3)$$
where $W_{M-j}$ indicates the weight parameter and M is an empirical parameter in the experiment.

According to literature [43,44], the relationship between the sEMG signals and the indicator of the muscle activation can be presented as follows:

$$ \rho = \frac{e^{\beta} \bar{u} - 1}{e^{\beta} - 1} \qquad (4) $$

where $-3 \le \beta \le 0$ in this study.

In order to achieve friendly and natural interaction between the human operator and the remote robot, muscle stiffness of human upper limbs is taken into consideration. To this end, muscle stiffness can be presented as a linear relationship according to literatures [22,39]

$$ \bar{a} = \{\bar{a}_{max} - \bar{a}_{min}\} \frac{\rho - \rho_{min}}{\rho_{max} - \rho_{min}} + \bar{a}_{min}, \qquad (5) $$

where $\bar{a} \in [\bar{a}_{min}, \bar{a}_{max}]$ indicate the parameters to guarantee stability of the slave. $\rho \in [\rho_{min}, \rho_{max}]$ represent the muscle activation.

## 3.2. Task Demonstrations

In the data collection stage, the demonstration data consist of several observations of the same task. After collecting demonstration data, a DTW method was proposed to synchronize the collected data.

### 3.2.1. Data Constitution

The proposed teleoperation system employed a heterogeneous master-slave structure, therefore we utilized motion of the slave as the data source for demonstrations. Owing to the workspace differences between the master and the slave, a workspace mapping between the master and the slave was carried out to ensure the accuracy of tracking.

The collected demonstrated information contains the position of the slave and human muscle stiffness. In the process of demonstration, the demonstration data can be organized as follows:

$$ O = \{P, \bar{a}\}, \qquad (6) $$

where $P = \{P_x, P_y, P_z\}$ and $\bar{a}$ indicate the collected task trajectories and collected muscle stiffness, respectively.

### 3.2.2. Data Preprocessing

Lengths of the demonstration data differ in time frame, thus it is impossible to train a task model in teleoperation. Therefore, the first step of data preprocessing was to align the demonstration data with different data lengths (if the demonstration data are not aligned in a united time scale, the task model could change both in temporal states and spatial states).

The alignment of the demonstration data can be performed according to the entire obtained demonstration data in comparison with a reference demonstration data. In this study, the DTW method was employed to find the optimal alignment of the demonstration data in the time scale [45]. The introduced DTW method could deal with the problem of the spatial distortion for the demonstration data. By employing the DTW method, the demonstration data can be presented as
$$D_{opt}\{L(x_1), L(x_2)\} = (x_1 - x_2)^2 + \left\{ \begin{array}{ll} 0 & \text{when } L(x_1) = L(x_2) = 1. \\ D\{L(x_1), L(x_2 - 1)\} & \text{when } L(x_1) = 1, L(x_2) > 1. \\ D\{L(x_1 - 1), L(x_2)\} & \text{when } L(x_1) > 1, L(x_2) = 1. \\ \min\{D\{L(x_1), L(x_2 - 1)\}, D\{L(x_1 - 1), L(x_2)\}, \\ D\{L(x_1), L(x_2)\}\} & \text{otherwise,} \end{array} \right. \quad (7)$$

where $L(.)$ indicates the length of trajectory, and $x_1$ and $x_2$ are the two different time lengths of trajectories. $D\{L(x_1), L(x_2)\}$ is the wrap path distance (Figure 4). $D_{opt}\{L(x_1), L(x_2)\}$ is the optimal wrap path distance.

<figure><img src="image_7.png" /><figcaption>Figure 4. The cost matrix $D\{L(x_1), L(x_2)\}$ with minimum wrap path distance.</figcaption></figure>

### 3.3. Learning Algorithm

The demonstration data are high dimensional; therefore, it is difficult to find a function to describe the demonstration data globally. In this study, the locally weighted regression (LWR) algorithm was employed to explore the approximate function locally between the input and the output for the given aligned demonstration data.

According to the demonstration data $T = \{(x(i), y(i)), i = 1, 2, \dots, N\}_{i}^{N}$, the local relationship between the output $y(i)$ and the input $x(i)$ can be obtained as follows:

$$Y = \frac{\sum_{i=1}^{N} w(i)y(i)}{\sum_{i=1}^{N} w(i)}, \quad (8)$$

where $y(i)$ denote the output values of the $i$th local model, $w(i)$ represent the weights of the $y(i)$. In the learning algorithm, input $x_i$ consists of the collected task trajectories $P = \{P_x, P_y, P_z\}$ and the collected stiffness $\bar{a}$. $x_i$ are four-dimensional vectors, $y_i$ are four-dimensional vectors.
Based on the receptive fields of the model, the weight $w_i$ can be computed by Gaussian forms as follows:

$$w(i) = \exp\{-\frac{1}{\theta}(\mathbf{x} - \mathbf{x}(i))^T(\mathbf{x} - \mathbf{x}(i))\}, \quad (9)$$

where $\mathbf{x}$ is the fitting position. $\theta > 0$ is used to adjust the Gaussian weight. $\theta = 2$ is used in this study. Equation (9) concludes that the weight value becomes larger as the input $\mathbf{x}(i)$ is near to the fitting position $\mathbf{x}$.

For the given input data $\mathbf{x}(i)$, the log likelihood for the probability $p(y(i)|\mathbf{x}(i))$ between $y(i)$ and $\mathbf{x}(i)$ can be presented as

$$\begin{aligned} \ell(\phi) &= \log\{\prod_{i=1}^{N} p(y(i)|\mathbf{x}(i))\} \\ &= \sum_{i=1}^{N} \log\left(\frac{w(i)}{2\pi}\right) \frac{1}{2} - \frac{1}{2} \sum_{i=1}^{N} w(i)\{y(i) - \phi x(i)\}^T \{y(i) - \phi x(i)\}, \end{aligned} \quad (10)$$

where $\phi$ is parameter vector of the LWR method.

With

$$p(y(i)|\mathbf{x}(i)) = \left(\frac{w(i)}{2\pi}\right)^{\frac{1}{2}} \exp\left(-\frac{w(i)(y(i) - \phi x(i))^T(y(i) - \phi x(i))}{2}\right). \quad (11)$$

Inspired by literature reports [46,47], the maximum of $\ell(\phi)$ by minimizing the $H(\phi)$ can be obtained as follows:

$$H(\phi) = \frac{1}{2} \sum_{i=1}^{N} w(i) \{y(i) - \phi x(i)\}^T \{y(i) - \phi x(i)\}. \quad (12)$$

On minimizing the $H(\phi)$, we have

$$\phi_{min} = \operatorname{argmin} \left\{ \frac{1}{2} \sum_{i=1}^{N} w(i) \{y(i) - \phi x(i)\}^T \{y(i) - \phi x(i)\} \right\}. \quad (13)$$

By using partial derivatives notion for Equation (13), we obtain

$$\begin{aligned} \frac{\partial \ell(\phi)}{\partial \phi} &= \sum_{i=1}^{N} w(i)(\phi x(i) - y(i))x(i)^T \\ &= \phi \sum_{i=1}^{N} w(i)x(i)x(i)^T - \sum_{i=1}^{N} w(i)y(i)x(i)^T. \end{aligned} \quad (14)$$

To set Equation (14) to be 0, a diagonal weight matrix $W$ can be represented as follows:

$$W = \begin{bmatrix} w(1) & 0 & \dots & 0 \\ 0 & w(2) & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & w(N) \end{bmatrix}. \quad (15)$$

Based on Equations (14) and (15), the nonlinear model can be obtained as

$$\phi = B^T W A (A^T W A)^{-1}, \quad (16)$$

with

$$A = \begin{bmatrix} x(1)^T \\ x(2)^T \\ \vdots \\ x(N)^T \end{bmatrix}, \quad (17)$$
with

$$B = \begin{bmatrix} y(1)^T \\ y(2)^T \\ \vdots \\ y(N)^T \end{bmatrix}. \qquad (18)$$

After the task is learned by using the LWR method, the slave robot was operated according to the learned task trajectories and learned human stiffness.

# 4. Results and Discussion

## 4.1. Experimental Setup

In order to verify the performance of the proposed algorithm, a practical sweep task was performed in this study. Figure 5 demonstrates the construction of the experimental platform as follows:

* Hardware equipment. The experiment hardware equipment consist of the Touch X, the Baxter robot, and the MYO armband with eight channels. The left panel of Figure 5, exhibits that a red garbage bucket and a yellow tapeline (as garbage) are placed on the testbed. A paint brush is installed in the right arm of the Baxter robot.

* Software environment. MATLAB software and Visual Studio 2013 (VS 2013, Microsoft, US) operate on Windows 7 in the master computer. Robot operating system (ROS) runs on the Ubuntu system in the remote computer. The master computer communicates with the remote computer through the User Datagram Protocol (UDP).

<figure><img src="image_8.png" /><figcaption>Figure 5. Experimental setup in this study.</figcaption></figure>

In the sweep task, a paint brush was used as a sweeping tool, and a yellow tapeline as garbage. Touch X was performed by the human operator to telecontrol the Baxter robot to sweep the garbage into the red bucket. By performing the prescribed sweep task, the task trajectories and human stiffness can be learned by using the proposed method. The slave robot was manipulated according to the learned task model.

Figure 6 exhibits the experimental process of the proposed method. Figure 6 (left) shows the demonstration and learning processes. The human operator directly telecontrols the slave. The proposed approach was trained by collected prescribed task trajectories and human stiffness. In the robot execution process, the slave executed the task based on the learned trajectories and learned human stiffness.
<figure><img src="image_9.png" /><figcaption>**Figure 6.** Experiment process of the proposed method.</figcaption></figure>

## 4.2. Results

The human operator performed the sweep task three times. By collecting data in the process of demonstration, the demonstration data are presented in Figure 7a-c. Clearly, the completion time of the sweep task was inconsistent for each demonstration. Accordingly, the demonstrated trajectory curves were mismatched in the time scale for demonstrations.

<figure><img src="image_10.png" /><figcaption>**Figure 7.** Demonstrated observations in process of demonstrations.</figcaption></figure>

The DTW algorithm was employed to normalize the demonstration data in the same time scale. Figure 8a-d, show that the demonstration data retain its original characteristics in the same time domain after the DTW algorithm preprocessing. Compared to the results shown in Figure 7a-c,
the preprocessing results guaranteed the uniformity of the demonstrations and make preparations for the next steps of robot learning task. In Figure 8d, values of demonstrated muscle stiffness in starting point were 0.4060, 0.1340 and 0.0110.

<figure><img src="image_11.png" /><figcaption>(a) Demonstrated data preprocessing by using dynamic time warping (DTW) method in the x-axis.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(c) Demonstrated data preprocessing by using DTW method in the z-axis.</figcaption></figure>

<caption>(d) Demonstrated human muscle stiffness preprocessing by using the DTW method.</caption>

<caption>**Figure 8.** Preprocessing of demonstrated observations by the DTW method.</caption>

A learned task model was calculated by using the LWR algorithm involving the demonstrated task trajectories and human muscle stiffness. The learned Cartesian trajectories are shown in Figure 9a-c. The learned trajectories were bold with green for the convenience of inspection. Figure 9d illustrates the learned human muscle stiffness in the process of demonstrations. Figure 9a-d demonstrate that the model of demonstrated trajectories and human muscle stiffness can be built up by the proposed LWR method. In Figure 9d, the value of the learned muscle stiffness in starting point is 0.1502.

Based on the learned model by using the LWR algorithm, the Baxter robot can execute the sweep task automatically with the learned human muscle stiffness. The sweep task fully indicates the dexterity of human arms in the process of operation. In this experiment, human muscle stiffness was employed based on muscle activation to reflect the interaction information in demonstration phase.

Figures 10I–VI show the process of robot execution for the sweep task. By employing the proposed LWR algorithm with physiological interface, the sweep task was performed successfully within 0–10 s.
<figure><img src="image_13.png" /><figcaption>(a) Learned trajectory by using the locally weighted (b) Learned trajectory by using the LWR method in y-axis. regression (LWR) method in x-axis.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>(c) Learned trajectory by using the LWR method in z-axis.</figcaption></figure>

<caption>(d) Learned human stiffness by using the LWR method.</caption>

<figure><img src="image_15.png" /><figcaption>**Figure 10.** Robot execution by employing the LWR algorithm.</figcaption></figure>

In order to validate the performance of the learning model, we evaluate average value of robot in the process of robot execution in Table 1. It can be seen that the average error for learned trajectories in x-y-z directions and learned stiffness were 0.0130, 0.0018, 0.0346 and 0.0124, respectively.

<table><caption>**Table 1.** Average error of robot execution.</caption><thead><tr><th>Variable</th><th>X</th><th>Y</th><th>Z</th><th>Stiffness</th></tr></thead><tbody><tr><td>Error</td><td>0.0130 (m)</td><td>0.0018 (m)</td><td>0.0346 (m)</td><td>0.0124</td></tr></tbody></table>
### 4.3. Discussion

First of all, the proposed LWR method constructs a nonlinear model for a specific task. Compared to traditional robot learning methods [23,24], the proposed method combines the task trajectories with the muscle stiffness during the demonstration process.

Moreover, the human operator can regulate the muscle stiffness according to the remote environments in the HRI. That is to say, the variability of muscle stiffness represents the operational characteristics of human operator. Therefore, a model to describe the relationship between the prescribed task, task trajectories, and the muscle stiffness are very useful in the learning process. Thus, the learned model can be used to improve the efficiency of the reproduction of the repetitive task. In the proposed method, we transmitted the learned model, i.e., learned trajectories and learned human stiffness, to the controller of the slave robot of the teleoperation system after robot learning. Then, the slave robot can be executed according to the learned model.

Noteworthy, the experimental results in this work are preliminary and the experiments were conducted mainly to show the feasibility and effectiveness of the proposed method. The stiffness of the human operator varies from person to person. Moreover, even for the same person, the stiffness property at different time could be different (time-varying). Therefore, it is hard to find a baseline to compare the results under "with stiffness" and "without stiffness" conditions. Different people may have different stiffness, and thus may show different motion model. Therefore, it is difficult to define fair comparison criteria for different/multiple experiment subjects. There is no problem foreseen to generalize the proposed method to different subjects as the implementation procedures will not change and each step is not affected by the specific human subject. However, surely the obtained specific muscle activation/stiffness pattern(s) would be different from person to person.

This work makes different contribution from our previous work [39,48], wherein a hidden semi-Markov model with Gaussian mixture method was proposed to carry out the repetitive tasks with enhanced intelligence. In this work, a locally weighted regression method with DTW was proposed for a prescribed task. Moreover, in our recent study, it was found that the length of demonstration data could significantly impact the performance of the teleoperation. To handle this problem, we employed the DTW method to deal with the issue related to different lengths. However, in our previous work [39], the influence of the length of demonstration data was not taken into account.

## 5. Conclusions and Future Work

In this study, a robot learning method with physiological interface was proposed for the teleoperated sweep task. In order to capture the information details of the remote operation environments in HRI, a novel method of human-centric interaction with human muscle stiffness was first developed. On the one hand, the sEMG signal of the human operator was used to capture the human's muscle stiffness. On the other hand, the collected muscle stiffness represented the human operator operational characteristic in the process of manipulation. Integration of the DTW method and the LWR method enabled the robots to learn the task trajectories and human muscle stiffness in the same time scale after human demonstrations in teleoperation system. The remote robot can execute the task according to the learned task trajectories and learned stiffness. Finally, the effectiveness of the proposed method was demonstrated by the experimental results.

Undeniably, a lot more systematic explorations are demanded to introduce the vision information and rotation information to the perception system of the teleoperation system which will be pursued in future. Besides, the relationship between shared control and the human stiffness in fatigue status should be comprehensively analyzed. Moreover, the shared control strategy with respect to robot learning method is an interesting research topic in the teleoperation system. The level of autonomy between direct teleoperation and robot-learning-based automation is worth to exploit to explain the cooperative strategies for HRI. Moreover, the smoothness of task trajectories and safe trajectory generation with respect to the muscle stiffness are very important aspects for the complex task. In order to obtain the model with smoothness of task trajectories in-depth exploration of the intelligent
segmentation algorithm to divide the complicated task into certain subtasks which can be approximated explicitly by locally smooth mathematical functions/polynomial will be carried out in future.

**Author Contributions:** J.L. conceived the method, performed the experiments and wrote the paper; C.Y. conceived the method and helped to improve it; H.S. and C.L. further helped to improve it. All authors have read and approved the manuscript.

**Funding:** This work was partially supported by Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/S001913.

**Conflicts of Interest:** The authors declare no conflict of interest.

## Appendix A

In order to facilitate reading, the main notations are presented in Table A1.

<table><caption>Table A1. NOMENCLATURE.</caption><tr><td>û</td><td>Raw sEMG signal.</td></tr><tr><td>u</td><td>Sum of sEMG signal.</td></tr><tr><td>ŭ</td><td>Moving average filter.</td></tr><tr><td>ū</td><td>Envelope of sEMG signal.</td></tr><tr><td>ρ</td><td>Indicator of the muscle activation.</td></tr><tr><td>ā</td><td>Muscle stiffness.</td></tr><tr><td>O</td><td>Demonstration data.</td></tr><tr><td>P</td><td>Collected task trajectories.</td></tr><tr><td>x(i)</td><td>Input with related to LWR.</td></tr><tr><td>y(i)</td><td>Output with related to LWR.</td></tr><tr><td>w(i)</td><td>Weights of the y(i).</td></tr><tr><td>ℓ(φ)</td><td>Log likelihood for the probability with related to LWR.</td></tr><tr><td>H(φ)</td><td>Maximum of ℓ(φ).</td></tr><tr><td>W</td><td>Diagonal weight matrix of y(i).</td></tr><tr><td>φ</td><td>Nonlinear model with related to LWR.</td></tr></table>

## References

1. Kalam, A.A.E.; Ferreira, A.; Kratz, F. Bilateral teleoperation system using qos and secure communication networks for telemedicine applications. *IEEE Syst. J.* **2016**, *10*, 709–720. [CrossRef]

2. Falleni, S.; Filippeschi, A.; Ruffaldi, E.; Avizzano, C.A. Teleoperated multi-modal robotic interface for telemedicine: A case study on remote auscultation. In Proceedings of the 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal, 28 August–1 September 2017; pp. 476–482.

3. Sharifi, M.; Behzadipour, S.; Salarieh, H. Nonlinear bilateral adaptive impedance control with applications in telesurgery and telerehabilitation. *J. Dyn. Syst. Meas. Control.* **2016**, *138*, 111010. [CrossRef]

4. Milstein, A.; Ganel, T.; Berman, S.; Nisky, I. The scaling of the gripper affects the action and perception in teleoperated grasping via a robot-assisted minimally invasive surgery system. *IEEE Trans. Hum. Mach. Syst.* **2018**. [CrossRef]

5. Gregory, J.; Fink, J.; Stump, E.; Twigg, J.; Rogers, J.; Baran, D.; Fung, N.; Young, S. Application of multi-robot systems to disaster-relief scenarios with limited communication. In *Field and Service Robotics*; Springer: Berlin/Heidelberg, Germany, 2016; pp. 639–653.
6. Gancet, J.; Urbina, D.; Letier, P.; Ilzokvitz, M.; Weiss, P.; Gauch, F.; Antonelli, G.; Indiveri, G.; Casalino, G.; Birk, A.; et al. Dexrov: Dexterous undersea inspection and maintenance in presence of communication latencies. *IFAC-PapersOnLine* **2015**, *48*, 218–223. [CrossRef]

7. Artigas, J.; Hirzinger, G. A brief history of dlrs space telerobotics and force feedback teleoperation. *Acta Polytech. Hung.* **2016**, *13*, 239–249.

8. Taylor, R.H.; Menciassi, A.; Fichtinger, G.; Fiorini, P.; Dario, P. Medical robotics and computer-integrated surgery. In *Springer Handbook of Robotics*; Springer: Berlin/Heidelberg, Germany, 2016; pp. 1657–1684.

9. Manupati, V.; Krishnan, M.G.; Varela, M.L.R.; Machado, J. Telefacturing based distributed manufacturing environment for optimal manufacturing service by enhancing the interoperability in the hubs. *J. Eng.* **2017**, *2017*, 9305989. [CrossRef]

10. Bennett, M.; Williams, T.; Thames, D.; Scheutz, M. Differences in interaction patterns and perception for teleoperated and autonomous humanoid robots. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 6589–6594.

11. Ravankar, A.; Ravankar, A.A.; Kobayashi, Y.; Emaru, T. Hitchhiking Robots: A Collaborative Approach for Efficient Multi-Robot Navigation in Indoor Environments. *Sensors* **2017**, *17*, 1878. [CrossRef] [PubMed]

12. Sheridan, T.B. Human-robot interaction: status and challenges. *Hum. Fact.* **2016**, *58*, 525–532. [CrossRef] [PubMed]

13. Ramos, J.; Wang, A.; Kim, S. Robot-human balance state transfer during full-body humanoid teleoperation using divergent component of motion dynamics. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; pp. 1587–1592.

14. Cheng, L.; Tavakoli, M. Switched-impedance control of surgical robots in teleoperated beating-heart surgery. *J. Med. Robot. Res.* **2018**, *2018*, 1841003. [CrossRef]

15. Yuan, Y.; Wang, Y.; Guo, L. Force reflecting control for bilateral teleoperation system under time-varying delays. *IEEE Trans. Ind. Inform.* **2018**, *15*, 1162–1172. [CrossRef]

16. Zeestraten, M.J.; Havouris, I.; Calinon, S. Programming by demonstration for shared control with an application in teleoperation. *IEEE Robot. Autom. Lett.* **2018**, *3*, 1848–1855. [CrossRef]

17. Janabi-Sharifi, F.; Hassanzadeh, I. Experimental analysis of mobile-robot teleoperation via shared impedance control. *IEEE Trans. Syst. Man Cybern. Part (Cybern.)* **2011**, *41*, 591–606. [CrossRef]

18. J. Corredor, J. Sofrony, A. Peer, Decision-making model for adaptive impedance control of teleoperation systems. *IEEE Trans. Haptics* **2017**, *10*, 5–16. [CrossRef]

19. M. Li, M. Ishii, R. H. Taylor, Spatial motion constraints using virtual fixtures generated by anatomy. *IEEE Trans. Robot.* **2007**, *23*, 4–19. [CrossRef]

20. Ni, D.; Song, A.; Li, H. Haptic assisted teleoperation based on virtual fixture and dynamic modelling. *Sens. Mater.* **2017**, *29*, 1367–1381.

21. Yang, C.; Wang, X.; Li, Z.; Li, Y.; Su, C.-Y. Teleoperation control based on combination of wave variable and neural networks. *IEEE Trans. Syst. Man Cybern. Syst.* **2017**, *47*, 2125–2136. [CrossRef]

22. Yang, C.; Luo, J.; Pan, Y.; Liu, Z.; Su, C.-Y. Personalized variable gain control with tremor attenuation for robot teleoperation. *IEEE Trans. Syst. Man Cybern. Syst.* **2018**, *48*, 1759–1770. [CrossRef]

23. Schaal, S. Learning from demonstration. In *Advances in Neural Information Processing Systems*; MIT Press: Cambridge, MA, USA, 1997; pp. 1040–1046.

24. Lin, H.-C.; Tang, T.; Fan, Y.; Zhao, Y.; Tomizuka, M.; Chen, W. Robot learning from human demonstration with remote lead through teaching. In Proceedings of the 2016 European Control Conference (ECC), Alborg, Denmark, 29 June–1 July 2016; pp. 388–394.

25. Tanwani, A.K.; Calinon, S. A generative model for intention recognition and manipulation assistance in teleoperation. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 43–50.

26. Havouris, I.; Calinon, S. Learning assistive teleoperation behaviors from demonstration. In Proceedings of the 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Lausanne, Switzerland, 23–27 October 2016; pp. 258–263.

27. Havouris, I.; Calinon, S. Learning from demonstration for semi-autonomous teleoperation. *Auton. Robot.* **2018**, *43*, 713–726. [CrossRef]
28. Huang, R.; Cheng, H.; Guo, H.; Chen, Q.; Lin, X. Hierarchical interactive learning for a human-powered augmentation lower exoskeleton. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; pp. 257–263.

29. Peternel, L.; Oztop, E.; Babic, J. A shared control method for online human-in-the-loop robot learning based on locally weighted regression. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 3900–3906.

30. Peternel, L.; Petric, T.; Babic, J. Human-in-the-loop approach for teaching robot assembly tasks using impedance control interface. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 25–30 May 2015; pp. 1497–1502.

31. Pervez, A.; Ali, A.; Ryu, J.-H.; Lee, D. Novel learning from demonstration approach for repetitive teleoperation tasks. In Proceedings of the 2017 IEEE World Haptics Conference (WHC), Munich, Germany, 6–9 June 2017; pp. 60–65.

32. Ravankar, A.; Ravankar, A.; Kobayashi, Y.; Hoshino, Y.; Peng, C.C. Path smoothing techniques in robot navigation: State-of-the-art, current and future challenges. *Sensors* **2018**, *18*, 3170. [CrossRef]

33. Mao, Y.; Jin, X.; Dutta, G.G.; Scholz, J.P.; Agrawal, S.K. Human movement training with a cable driven arm exoskeleton (CAREX). *IEEE Trans. Neural Syst. Rehabil. Eng.* **2015**, *23*, 84–92. [CrossRef]

34. Meattini, R.; Benatti, S.; Scarcia, U.; Gregorio, D.D.; Benini, L.; Melchiorri, C. An sEMG-based human-robot interface for robotic hands using machine learning and synergies. *IEEE Trans. Compon. Packag. Manuf. Technol.* **2018**, *8*, 1149–1158. [CrossRef]

35. Pilarski, P.M.; Dawson, M.R.; Degris, T.; Carey, J.P.; Sutton, R.S. Dynamic switching and real-time machine learning for improved human control of assistive biomedical robots. In Proceedings of the 2012 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy, 24–27 June 2012; pp. 296–302.

36. Antuvan, C.W.; Ison, M.; Artemiadis, P. Embedded human control of robots using myoelectric interfaces. *IEEE Trans. Neural Syst. Rehabil. Eng.* **2014**, *22*, 820–827. [CrossRef] [PubMed]

37. Ohkubo, H.; Shimono, T. Motion control of mobile robot by using myoelectric signals based on functionally different effective muscle theory. In Proceedings of the 2013 IEEE International Conference on Mechatronics (ICM), Vicenza, Italy, 27 February–1 March 2013; pp. 786–791.

38. Park, K.-H.; Suk, H.-I.; Lee, S.-W. Position-independent decoding of movement intention for proportional myoelectric interfaces. *IEEE Trans. Neural Syst. Rehabil. Eng.* **2016**, *24*, 928–939. [CrossRef] [PubMed]

39. Yang, C.; Luo, J.; Liu, C.; Li, M.; Dai, S. Haptics Electromyography Perception and Learning Enhanced Intelligence for Teleoperated Robot. *IEEE Trans. Autom. Sci. Eng.* **2018**, *PP*, 1–10. [CrossRef]

40. Sansanayuth, T.; Nilkhamhang, I.; Tungpimolrat, K. Teleoperation with inverse dynamics control for phantom omni haptic device. In Proceedings of the 2012 SICE Annual Conference (SICE), Akita, Japan, 20–23 August 2012; pp. 2121–2126.

41. Silva, A.J.; Ramirez, O.A.D.; Vega, V.P.; Oliver, J.P.O. Phantom omni haptic device: Kinematic and manipulability. In Proceedings of the 2009 Electronics, Robotics and Automotive Mechanics Conference, CERMA’09, Cuernavaca, Morelos, Mexico, 22–25 September 2009; pp. 193–198.

42. Ju, Z.; Yang, C.; Ma, H. Kinematics modelling and experimental verification of baxter robot. In Proceedings of the 2014 33rd Chinese Control Conference (CCC), Nanjing, China, 28–30 July 2014; pp. 8518–8523.

43. Han, J.; Ding, Q.; Xiong, A.; Zhao, X. A state-space EMG model for the estimation of continuous joint movements. *IEEE Trans. Ind. Electron.* **2015**, *62*, 4267–4275. [CrossRef]

44. Lloyd, D.G.; Besier, T.F. An EMG-driven musculoskeletal model to estimate muscle forces and knee joint moments in vivo. *J. Biomech.* **2003**, *36*, 765–776. [CrossRef]

45. Wllmer, M.; Al-Hames, M.; Eyben, F.; Schuller, B.; Rigoll, G. A multidimensional dynamic time warping algorithm for efficient multimodal fusion of asynchronous data streams. *Neurocomputing* **2009**, *73*, 366–380. [CrossRef]

46. Sun, Y.; Qian, H.; Xu, Y. Robot learns chinese calligraphy from demonstrations. In Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), Chicago, IL, USA, 14–18 September 2014; pp. 4408–4413.
47. Wang, J.; Yu, L.C.; Lai, K.R.; Zhang, X. Locally weighted linear regression for cross lingual valence-arousal prediction of affective words. *Neurocomputing* **2016**, *1943*, 271–278. [CrossRef]

48. Luo, J.; Yang, C.; Wang, N.; Wang, M. Enhanced teleoperation performance using hybrid control and virtual fixture. *Int. J. Syst. Sci.* **2019**, *50*, 451–462. [CrossRef]

<figure><img src="image_16.png" /></figure>

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).