# Delayed Bilateral Teleoperation of Wheeled Robots including  a Command Metric

Authors: Franco Penizzotto; Sebastian García; Emanuel Slawiñski; Vicente Mut
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

This paper proposes a control scheme applied to the delayed bilateral teleoperation of wheeled robots
with force feedback, considering the performance of the operator’s command execution. In addition, the
stability of the system is analyzed taking into account the dynamic model of the master as well as the
remote mobile robot under asymmetric and time-varying delays of the communication channel. Besides,
the performance of the teleoperation system, where a human operator drives a 3D simulator of a wheeled
dynamic robot, is evaluated. In addition, we present an experiment where a robot Pioneer is teleoperated,
based on the system architecture proposed.


---

## Body

## 1. Introduction

Robot teleoperation allows the execution of different tasks in remote environments including possibly dangerous and harmful jobs for the human operator [1]. In the teleoperation systems of robots with force feedback, a user completes some task and physically interacts with the environment through a master-slave system. Thus, several control schemes and strategies for the teleoperation of mobile robots have been developed in order to solve tasks such as land surveying in inaccessible or remote sites, transportation and storage of hazardous material, inspection of high-voltage power lines, deactivation of explosive devices, high-risk fire control, pesticide and fertilizer crop spraying and dusting, mining exploration, and various other tasks [2–8]. Additionally, the use of Internet as a communication channel increases the application of the teleoperation systems. However, it is known that the presence of time delay(s) can induce instability or poor performance in a delayed teleoperation system [9–11] as well as a poor transparency [12].Some strategies involve compensation based on a human operator model [13] in which only visual feedback is considered, ordinary structures such as control based on impedance [14], event-based control [15], signals fusion [16], and others based on kinematic models like [17–20]. Finally, [21, 22] consider a dynamic model and analyze the r-passivity of the system, while [23] presents a stability analysis of teleoperation systems using Linear Matrix Inequalities (LMIs) approach based on Lyapunov-Krasovskii functionals.In addition, researchers have considered human factors in the design or evaluation of the teleoperated systems as it is shown in [24], where about 150 papers are described, covering human performance issues and mitigation solutions for teleoperated systems. However [25] emphasizes that human factor (HF) potential is underexploited and [26] expresses that, despite advances in autonomy, there will always be a need for human involvement in vehicle teleoperation. In addition, [27] states that many serious accidents, in which human error has been involved, can be attributed to faulty operator decision making. In this context, an operator can correctly select the actions but executes them poorly by overcorrecting, losing control, or flipping the commands.This work presents a novel method to assist operators of a remote mobile robot in driving task. The method consists of weighting the operator commands by the risk associated to the current command activating also some safety action in the remote robot but only if the operator command increases the risk of collision. A control scheme based on impedance [28] is proposed to avoid obstacles, and the gain is variable according to the risk-based metric of the user commands (which is also presented in this work). The benefits of this scheme were analyzed in a previous work presented by the authors [29], where the channel delay was not considered. Besides, the stability of the system is analyzed considering the dynamics of master and slave robots as well as time-varying delays. Furthermore, the controller is tested from experiences over a virtual environment (Human in the Loop tests), in which a user drives a mobile robot teleoperated in presence of time delay, avoiding obstacles and feeling the force executed by the robot. In addition, an experiment of driving a real robot through a local network is presented.This work is organized as follows. First, in Section2, the statement of the problem is presented, while in Section 3 the control scheme and a method for a dynamic risk-based command metric are exposed. Then, in Section 4, the stability of the system is analyzed. Then, an explanation of the implementation of the method developed is given and experiments are shown in Section 5. Finally, conclusions are exposed (Section 7).


## 2. Preliminary

This paper proposes and analyses teleoperation systems in which a human operator drives a wheeled robot while he perceives the environment near the robot through visual feedback and a force feedback related to the force executed by the mobile robot. In addition, the scheme computes the command performance in order to help the controller, as it is shown in Figure1.Figure 1
Scheme of a human-robot bilateral teleoperation system.The master or local device is represented by the typical model(1)Mqq¨+Cq,q˙q˙+gq=τm+fh,where q(t)∈Rn×1 is the joint position of the master (in this work q(t)=q1tq2tT, n=2); q˙(t)∈Rn×1 is the joint velocity (in this work q˙(t)=q˙1tq˙2tT); M(q)∈Rn×n is the inertia matrix; C(q,q˙)∈Rn×n is the matrix representing centripetal and Coriolis torques; g(q)∈Rn×1 is the gravitational torque; fh∈Rn×1 is the torque caused by the human operator force; and τm∈Rn×1 is the control torque applied to the master.For the case of teleoperation of wheeled robots, the dynamic model of a unicycle-type mobile robot is considered [21]. It has two independently actuated rear wheels and is represented by(2)Dη˙+Qηη=τs+fe,where η=η1η2T is the robot velocity vector with η1 and η2 representing the linear and angular velocity of the mobile robot, fe is the force caused by the elements of the environment on the robot, D=m00i is the inertia matrix, and Q=0-maωmaω0 is the Coriolis matrix, where m is the mass of the robot, i is the rotational inertia, and a is the distance between the mass center and the geometric center. In addition, τ=u1u2 involves a control force u1 and a control torque u2, with u1=(1/rω)(uleft+uright) and u2=(1/rω)(uright-uleft) where rω>0 is the radius of the wheels, c>0 is the half-width of the cart, and uleft and uright are the torques of the left and right rear wheels, respectively. Furthermore, the communication channel adds a forward time delay h1 and a backward time delay h2. Generally, these delays are time-varying and different between them (asymmetric delays).On the other hand, the following ordinary properties, assumptions, and lemmas will be used in this paper.Property 1.
The inertia matricesM(q) and D are symmetric positive definite.Property 2.
The matrixM˙(q)-2C(q,q˙) is skew-symmetric.Assumption 1.
The time delaysh1(t) and h2(t) are bounded. Therefore, there exist positive scalars h-1 and h-2 such that 0≤h1(t)≤h-1 and 0≤h2(t)≤h-2 for all t.Lemma 2 (see [30]).
For the real vector functionsa(·) and b(·) and a time-varying scalar h(t) with 0≤h(t)≤h-, the following inequality holds:(3)-2aT(t)∫t-h(t)tb(ξ)dξ-∫t-h(t)tbT(ξ)Δb(ξ)dξ≤htaTtΔ-1at≤h-taTtΔ-1at,where Δ>0 is a positive definite matrix.


## 3. Control Scheme Based on Human’s Commands

The scheme proposed includes a PD-like velocity control, feedback of the force executed by the mobile robot, and an impedance loop to avoid obstacles depending on the performance of the delayed user’s actions. The control structure is well known in teleoperation systems of manipulator robots [30, 31].Considering the control actions, we have for the local site(4)τm=-km(kgq(t-h1-h2)-η(t-h2))-αmq˙+gand for the remote site(5)τs=ks((kgq(t-h1)-uk)-η)-αsz+Q(η)η,uk=sgnqt-h1kz1-e-khf,where the controller is formed by τm∈Rn×1 (torque applied to the master), τs∈Rn×1 (torque applied to the robot), and uk∈Rn×1 (velocity impedance). The parameters ks and αs are positive constant (diagonal matrices) and they represent the proportional gain and damping added by a remote velocity controller; αm is the damping injected in the master, km represents the scaling of force applied to the master, relative to the real force exerted by the mobile robot, kg converts position to a velocity command compatible with the restrictions about maximum speed of the mobile robot (all of them are also diagonal matrices), f=ftfy is a fictitious force, and z is defined by(6)η˙=z+γz˙with γ→0+. Signal z represents the mobile robot acceleration η˙ at an infinitesimal time instant before t so it is assumed η˙≈z consideringż without discontinuities.In addition,kh∈Rn×1 is a parameter based on the risk-based command metric and kz∈R1×1 is a user impedance tuning parameter.


### 3.1. Impedance Subsystem

In the control loop of the teleoperated robot, an impedance control law is applied based on a fictitious force, which depends on the nearest of the obstacles around the robot, in order to prevent collisions [32]. Despite operator commands, the robot automatically reduces its mobility when some object is near it. Therefore, the reference commands sent by the human could differ from the real actions executed by the controller when the impedance is active.Figure2 shows a graphics representation of the fictitious force vector, generated from the robot-obstacle interaction. d(t) is the distance between the robot and the obstacle, and a minimum dmin⁡ and a maximum dmax⁡ distance are set, in order to restrict the values of the fictitious force vector. It is decomposed in two components that affect the linear and angular speed impedance signal. Equation for computing the vector mentioned is described in [33], where f-(t)=(f-t(t),f-y(t)), being f(t)=a-bd(t), with a and b as positive constant such that a-bdmax⁡=0 and a-bdmin⁡=1. Then, ft=f(t)cos⁡Ω(t) and fy=f(t)sinΩ(t), where Ω(t) is the angle between f-(t) and f-t(t).Figure 2
Fictitious force vector.The impedance signal (uk) is calculated like (5), where speed references will keep the sense and direction of the user commands.Variation ofkh is proportional to the risk-based command metric proposed (Me). That is, kh=k1k2T=k1k2T.


### 3.2. Proposed Definition of the Command Metric

In this section, we propose a performance indexMe to quantify theOperator command execution in front of risk. In order to define Me, the following terms are established.(i) Real Risk (
P
i
(
t
)
;
l
o
w
0
,
…
,
1
h
i
g
h
). It is the level of risk produced by each obstacle i near to the robot, in the current time.(ii) Command Risk (
P
c
i
(
t
)
;
l
o
w
0
,
…
,
1
h
i
g
h
). It is the level of risk that would appear if the robot controller directly applies the user commands (considering uk=0), in the current time.(iii) Graduation of Command’s Performance (
Z
i
(
t
)
;
l
o
w
-
1
,
…
,
1
h
i
g
h
). It is thelevel of improvement or worsening ofReal risk as an effect of theCommand risk, taking into account the ideal commands that are possible to be applied.We differentiate when the effect of the operator’s command (Pci, command risk) potentiallyimproves,preservers, orworsens the level of risk Pi. For example, if the current Pi is valuated in 0.9 and Pci=0.5, it means that the command applied by the user would decrease the real risk.Then, in order to valuate the user skill required for the current situation, the more the high-risk level is, the more strongly should the metric weigh the improvement or worsening. For example, for the same numeric difference betweenPi=0.9 and Pci=0.5 or Pi=0.6 and Pci=0.2 at a time instant tn, Zi should throw a higher value for the first case due to the improvement demanding wiser and quicker commands.Therefore, in order to quantifyZi(t) as a function of Pci and Pi, a surface is proposed and showed in Figure 3. Zi(t) is bounded between −1 and 1 and includes 3 ranges:preservation,improvements, andworsening. Figure 3(b) shows the nonlinear quantification of the Zi(t) signal. Figure 3(a) pictures the contour of the surface. Green line corresponds to Zi=0.Figure 3
Surface for computingZi(t).
(a)
Contour of the surface
(b)
Surface of theP, Pc, and ZNow, we defineMe as “The degree of goodness of the command that the user executes before risk situations.”M
e is computed as theDistance between theIdeal Action and theReal Action, where the Ideal Action (Meid) is the best possible user’s command. That is,(7)Me=Meid-Mere,where (8)Meid=(Z1id,Z2id,…,Zqid)with Ziid calculated like Zi but considering Pc=0 and(9)Mere=(Z1re,Z2re,…,Zqre)with Zire calculated as Zi but taking into account Pc.Besides,i corresponds to the identification subindex for each obstacle and the variable q is the amount of obstacles in the current time tn.Figure4 shows the vectorial model proposed in order to compute the metric definition (Me), considering only two obstacles for simplicity. A value of Me=0 implies a perfect operator decision making in command execution, and Me=q(1+zbp) limits the worst decision for the q present obstacles. zbp is the worst value of the rangePreservation. The upper limit of Me is nonlinear and it is altered by the increment of obstacles in the environment.Figure 4
Quantitative components of the metric proposed.


### 3.3. Implementation of the Metric in This Work

A function calculates the collision probabilityPi based on the robot and obstacle position information, while Pci signal is calculated from the robot’s kinematic model and the direct operator’s commands. Once the calculation of P(t) and Pc(t) is made, Z(t)=fP(t)Pc(t) is computed through a neural network.In order to create many pointsZ to train the net, a four-plane surface is defined as it is shown in Figure 5. The net consists in one hidden layer with 10 neurons, and a linear function output layer. Figure 6 shows the approximation reached with the neural network used. After computing Z for each obstacle, Me is calculated as in (7).Figure 5
Example of calculation ofZi.Figure 6
Test of the neural network.


## 3.1. Impedance Subsystem

In the control loop of the teleoperated robot, an impedance control law is applied based on a fictitious force, which depends on the nearest of the obstacles around the robot, in order to prevent collisions [32]. Despite operator commands, the robot automatically reduces its mobility when some object is near it. Therefore, the reference commands sent by the human could differ from the real actions executed by the controller when the impedance is active.Figure2 shows a graphics representation of the fictitious force vector, generated from the robot-obstacle interaction. d(t) is the distance between the robot and the obstacle, and a minimum dmin⁡ and a maximum dmax⁡ distance are set, in order to restrict the values of the fictitious force vector. It is decomposed in two components that affect the linear and angular speed impedance signal. Equation for computing the vector mentioned is described in [33], where f-(t)=(f-t(t),f-y(t)), being f(t)=a-bd(t), with a and b as positive constant such that a-bdmax⁡=0 and a-bdmin⁡=1. Then, ft=f(t)cos⁡Ω(t) and fy=f(t)sinΩ(t), where Ω(t) is the angle between f-(t) and f-t(t).Figure 2
Fictitious force vector.The impedance signal (uk) is calculated like (5), where speed references will keep the sense and direction of the user commands.Variation ofkh is proportional to the risk-based command metric proposed (Me). That is, kh=k1k2T=k1k2T.


## 3.2. Proposed Definition of the Command Metric

In this section, we propose a performance indexMe to quantify theOperator command execution in front of risk. In order to define Me, the following terms are established.(i) Real Risk (
P
i
(
t
)
;
l
o
w
0
,
…
,
1
h
i
g
h
). It is the level of risk produced by each obstacle i near to the robot, in the current time.(ii) Command Risk (
P
c
i
(
t
)
;
l
o
w
0
,
…
,
1
h
i
g
h
). It is the level of risk that would appear if the robot controller directly applies the user commands (considering uk=0), in the current time.(iii) Graduation of Command’s Performance (
Z
i
(
t
)
;
l
o
w
-
1
,
…
,
1
h
i
g
h
). It is thelevel of improvement or worsening ofReal risk as an effect of theCommand risk, taking into account the ideal commands that are possible to be applied.We differentiate when the effect of the operator’s command (Pci, command risk) potentiallyimproves,preservers, orworsens the level of risk Pi. For example, if the current Pi is valuated in 0.9 and Pci=0.5, it means that the command applied by the user would decrease the real risk.Then, in order to valuate the user skill required for the current situation, the more the high-risk level is, the more strongly should the metric weigh the improvement or worsening. For example, for the same numeric difference betweenPi=0.9 and Pci=0.5 or Pi=0.6 and Pci=0.2 at a time instant tn, Zi should throw a higher value for the first case due to the improvement demanding wiser and quicker commands.Therefore, in order to quantifyZi(t) as a function of Pci and Pi, a surface is proposed and showed in Figure 3. Zi(t) is bounded between −1 and 1 and includes 3 ranges:preservation,improvements, andworsening. Figure 3(b) shows the nonlinear quantification of the Zi(t) signal. Figure 3(a) pictures the contour of the surface. Green line corresponds to Zi=0.Figure 3
Surface for computingZi(t).
(a)
Contour of the surface
(b)
Surface of theP, Pc, and ZNow, we defineMe as “The degree of goodness of the command that the user executes before risk situations.”M
e is computed as theDistance between theIdeal Action and theReal Action, where the Ideal Action (Meid) is the best possible user’s command. That is,(7)Me=Meid-Mere,where (8)Meid=(Z1id,Z2id,…,Zqid)with Ziid calculated like Zi but considering Pc=0 and(9)Mere=(Z1re,Z2re,…,Zqre)with Zire calculated as Zi but taking into account Pc.Besides,i corresponds to the identification subindex for each obstacle and the variable q is the amount of obstacles in the current time tn.Figure4 shows the vectorial model proposed in order to compute the metric definition (Me), considering only two obstacles for simplicity. A value of Me=0 implies a perfect operator decision making in command execution, and Me=q(1+zbp) limits the worst decision for the q present obstacles. zbp is the worst value of the rangePreservation. The upper limit of Me is nonlinear and it is altered by the increment of obstacles in the environment.Figure 4
Quantitative components of the metric proposed.


## 3.3. Implementation of the Metric in This Work

A function calculates the collision probabilityPi based on the robot and obstacle position information, while Pci signal is calculated from the robot’s kinematic model and the direct operator’s commands. Once the calculation of P(t) and Pc(t) is made, Z(t)=fP(t)Pc(t) is computed through a neural network.In order to create many pointsZ to train the net, a four-plane surface is defined as it is shown in Figure 5. The net consists in one hidden layer with 10 neurons, and a linear function output layer. Figure 6 shows the approximation reached with the neural network used. After computing Z for each obstacle, Me is calculated as in (7).Figure 5
Example of calculation ofZi.Figure 6
Test of the neural network.


## 4. System Stability Analysis

It is important to highlight the common states variables that should be analyzed in a teleoperation system. They are(1) the master-slave tracking error, (2) the master velocity (q˙ should be bounded or tend to 0), and (3) the acceleration of the master (q¨ should tend to 0).The stability analysis of the scheme proposed is based on the theory of Lyapunov-Krasovskii. Therefore, a positive definite functionalV=V1+V2+V3+V4>0 is proposed. It is important to remark that there is not an equilibrium point but a Krasovskii-like equilibrium solution that depends on the state x≔[q˙η-kgqz]T in the time interval t-h1-h2,t [34, 35]. The functional is formed by four parts. The first, second, and third subfunctional are proposed in the following manner: (10)V1=12kgkmq˙TMq˙,(11)V2=12η-kgqT(η-kgq),(12)V3=12γzTz.The time derivative ofV1, along master dynamics (1) and taking into account Properties 1 and 2, is the following:(13)V˙1=12kgkmq˙TM˙q˙+kgkmq˙TMq¨,V˙1=12kgkmq˙TM˙q˙+kgkmq˙TMM-1(τm+fh-Cq˙-g),V˙1=kgkmq˙T(τm+fh-g).Now, if the control actionτm  (5) is included in (13), it yields(14)V˙1=kgkmq˙T(τm-g)+kgkmq˙Tfh,V˙1=kgkmq˙T(-km(kgq(t-h1-h2)-η(t-h2))-αmq˙)+kgkmq˙Tfh,V˙1=kgkmq˙T-kgq(t-h1-h2)-η(t-h2)+kgqmmmmmmm-kgq+η-η-αmkgkmq˙Tq˙+kgkmq˙Tfh,V˙1=-αmkgkmq˙Tq˙+kgq˙T(η-kgq˙)+kgkmq˙Tfh-kg2q˙TD∫(t-h1-h2)tq˙(ξ)dξ-kgq˙T∫(t-h2)tη˙(ξ)dξ.Next,V˙2 along the dynamics of wheeled robot (2) is obtained as(15)V˙2=η-kgqT(η˙-kgq˙),V˙2=η-kgqTη˙-kgη-kgqTDq˙,V˙2=ksη-kgqTD-1(kgq(t-h1)-uk-η+kgq-kgq)-kgη-kgqTq˙+η-kgqTD-1fe-αsη-kgqTD-1z,V˙2=-ksη-kgqTD-1(η-kgq)-kgη-kgqTq˙+η-kgqTD-1fe-kskgη-kgqTD-1∫t-h1tq˙(ξ)dξ-αsη-kgqTD-1z-ksη-kgqTD-1uk.Later, from including (6) into the derivative of (12), and considering (2) and (5), V˙3 can be written in the following way:(16)V˙3=γzTz˙=γzTη˙γ-zγ=zTη˙-zTz,V˙3=-αszTD-1z+kszTD-1(kgq(t-h1)-uk-η)-zTD-1fe-zTz,V˙3=-αszTD-1z-kszTD-1(η-kgq)+zTD-1fe-kskgzTD-1∫t-h1tq˙ξdξ-zTfe-zTz-kszTD-1uk.It can be appreciated from (14) to (16) that there are delayed variables that make the stability analysis difficult. For solving the inconvenience of integral terms in the time derivative form of the Lyapunov’s candidate, V4 is proposed (17) as a mathematic resource:(17)V4=∫-h-20∫t+θtzξTWzξdξdθ+∫-h-10∫t+θtq˙ξTXq˙ξdξdθ+∫-h-10∫t+θtq˙ξTYq˙ξdξdθ+kg22∫-h-1-h-20∫t+θtq˙ξTq˙ξdξdθ,where W,X,Y>0 are positive definite matrices.Considering the assumption of Preliminary section,V˙4 can be formed as(18)V˙4=h-2zTWz-∫(t-h-2)tzT(ξ)Wz(ξ)dξ+h-1q˙TXq˙-∫(t-h-1)t‍q˙T(ξ)Xq˙(ξ)dξ+h-1q˙TYq˙-∫(t-h-1)t‍q˙T(ξ)Yq˙(ξ)dξ+kg22(h-1+h-2)q˙Tq˙-kg22∫t-h-1-h-2tq˙Tξq˙ξdξ≤h-2zTWz-∫(t-h2)tzT(ξ)Wz(ξ)dξ+h-1q˙TXq˙-∫(t-h1)tq˙T(ξ)Xq˙(ξ)dξ+h-1q˙TYq˙-∫(t-h1)tq˙T(ξ)Yq˙(ξ)dξ+kg22(h-1+h-2)q˙Tq˙-kg22∫(t-h1-h2)tq˙T(ξ)q˙(ξ)dξ.On the other hand, each term ofV˙1+V˙2+V˙3 that includes a delayed variable can be conveniently joined using Lemma 2  (3) with one term of V˙4. Consider(19)-kg22∫(t-h1-h2)tq˙T(ξ)q˙(ξ)dξ+kg2q˙T∫(t-h1-h2)tq˙(ξ)dξ≤12(h-1+h-2)kg2q˙Tq˙,(20)-∫(t-h1)tq˙T(ξ)Xq˙(ξ)dξ-kgksη-qTD-1∫(t-h1)tq˙(ξ)dξ≤14h-1kg2ks2η-qTD-1X-1D-1(η-qm),(21)-∫(t-h1)tq˙T(ξ)Yq˙(ξ)dξ-kgkszTD-1∫(t-h1)tq˙(ξ)dξ≤14h-1kg2ks2zTD-1Y-1D-1z,(22)-∫(t-h2)tzT(ξ)Wz(ξ)dξ-kgq˙T∫(t-h2)tη˙(ξ)dξ=-∫t-h2tzTξWzξdξ-kgq˙T∫(t-h2)tzξ+γz˙dξ≤14h-2kg2qTW-1q˙+γkgh2σq˙.In (22), the integral function is applied in a closed interval to z˙(t) (assumed without discontinuities); therefore the function ∫t-h2tz˙(ξ)dξ has a maximum real value σ that bounds such function and then ∫t-h2tz˙(ξ)dξ≤h2σ.Next, joining terms from (15) and (16), the following inequality can be deduced:(23)-(αs+ks)η-kgqTD-1z≤14αs+ks2η-kgqTβ(η-kgq)-zTD-1β-1D-1zwith β being a positive definite matrix. Finally, V˙ can be built linking from (14) to (23) in the following way:(24)V˙=V˙1+V˙2+V˙3+V˙4≤q˙T-αmkgkmI+kg2(h-1+h-2)Immmcαmkgkm+14kg2h-2W-1+h-1(X+Y)q˙-zT-αsD-1-I+D-1β-1D-114mmmc+14h-1ks2kg2D-1Y-1D-1+h-2Wz-η-kgqT-ksD-1+14αs+ks2βmmmmcmmc+14h-1ks2kg2D-1X-1D-1(η-kgqm)+zTD-1fe+η-kgqmTD-1fe+kgkmq˙Tfh+ψq˙-ksη-kgqTD-1uk-kszTD-1uk,where I is the identity matrix and ψ=γkgh2σ. The controller can be set to guarantee that the quadratic terms of (24) are negative definite and therefore the signals q˙, (η-kgq), and z will tend to a ball centered in the origin and whose size depends on the maximum magnitude of ψ, fe, fh, and uk.


## 5. Experimental Test and Results

In our test, a human drives a mobile robot as quick as possible (Figure7), from an initial point to a final one, in an environment with multiple fixed and mobile obstacles. Operator drives the robot by using a Novint joystick while he receives the image captured by the camera onboard the robot, in presence of time delays h1 and h2. The accomplishment of the task was evaluated based on some performance metrics proposed in Table 1.Table 1
Table of evaluation metrics computed for experimentE1.
h
1; h2
Case
N  [%]
T
task
σ
T
task
HCP
σ
hcp
J
Low delay
A
1
33
66.88
3.44
0.12
0.012
39.43
B
1
20
139.89
0.18
0.042
0.005
28.91
C
1
16
89.09
2.51
0.069
0.006
20.67
Medium delay
A
2
66
67.13
0.51
0.116
0.014
21.91
B
2
20
145.30
7.32
0.040
0.006
39.05
C
2
33
90.93
2.43
0.071
0.005
28.33
High delay
A
3
83
85.75
3.40
0.079
0.01
56.53
B
3
20
161.15
9.14
0.033
0.003
41.79
C
3
50
94.64
3.45
0.064
0.007
38.47Figure 7
Photograph of the experimentE1. Simulated environment and communication channel.The experimental test consisted in two parts (E1 and E2). In the first one, many participant were asked to drive over a simulated robot (running under Matlab-Simulink), where also a simulated channel time delay was considered. Simulated environment and channel allows performing the same task under similar conditions, which makes possible a comparison between differences in human aspects. The second test was made using a real robot Pioneer through a local network.


### 5.1. ExperimentE1

ForE1 in this work, we simulate three levels of channel time delays:(1)
verylow (h1=h2=0.025),(2)
medium (h1=0.15; h2=0.1*sin(0.5*t)+0.15),(3)
high (h1=0.30; h2=0.1*sin(0.5*t)+0.3, quite big considering the robot dynamic.The parameters used for the mobile robot arem=30 kg, i=2.5 kg, and a=0.05.In the stability analysis of the proposed scheme, parameterskg, ks, αs, km, and αm were considered as scalar due to simplicity, but the analysis and results are also valid for positive definite diagonal matrices. Therefore, for the PD-like controller in the local site, the parameters set are(25)ks=34Nsm007Nmsrad;αs=0.5Nss2000.5Nms2radand, for the controller of the master (local side),(26)km=10Nsm003Nsm;αm=200Nss00130Nsm.Finally, in order to transform the position of the master to a speed reference,kg is established as follows:(27)kg=20.01s0020.01s.The range of the master positions in both axes is from-0.05[m] to 0.05[m]; therefore, the maximum speed linear and angular references are 1[m/s] and 1[rad/s]. The parameter kz is set to 0.8.In order to analyze the control scheme proposed against classical methods based on velocity impedance or other methods that do not incorporate an impedance law, differentkh=k1k2T are used: a constant high value (maximum), null, and kh variable value (from 0 to a maximum, depending on the metric proposed (Me(t)). In Table 1, cases Ai are for null velocity impedance kh=00T, cases Bi for constant velocity impedance kh=11T (classical method based on impedance), and cases Ci for impedance varying according to the metric kh=5Me5MeT.About the fictitious force for the impedance subsystem, the parameters taken aredmin⁡=0.5[m] and dmax⁡=3.5[m]; therefore a=1.16 and b=0.33 (see Section 3.1).Different drivers were taken to perform the required task (age range from 20 to 60, students and workers from the Universidad Nacional de San Juan).Table1 resumes the average values of evaluation index for the system (computed for experiment E1, 10 trials for each case row) in order to test the control system proposed and compare different cases listed above. N represents the percentage of experiments in which the robot crashes against any obstacle. In this case the experiment is stopped. Next, Ttask measures the time to complete the task.Then, HCP is an index abouthigh collision probability. On one hand, the main task is required to be done as quickly as possible (evaluated by Ttask), while, on the other hand, obstacles over the path represent restrictions to the movement of the robot. Hence, this index is about the risk the robot faces. We describe this evaluation metric as(28)HCP=1nob∑i=1nob1Texp⁡∫0Texp⁡Pi3(t)dt,where nob is the quantity of obstacles, Pi is the real risk robot-obstaclei, Texp⁡ is the time of the experiment, and Pi is the collisions probability of the robot against obstacle i. Designers can use any method for computing the collision probability. We use a method based on a linear integration of the velocities (velocity in x and y coordinates of the plane) of the robot and obstacle. The cubic function of the collision probability is chosen, due to the fact that itenhances high values of risk (high collision probabilities attempt mission’s safety) andreduces the low values (not important for evaluating risk). Standard deviation is also computed for each case (σTtask and σhcp).In order to enhance the understanding of the readers, Figure8 introduces a graphic of the index of the table.Figure 8
Graphic from Table1: percentage of collisions, time to complete the task, and index of high collision probability.
(a)
(b)
(c)A short part of one trial of experimentE1 was extracted and the main signals were plotted. Figure 9 shows the trajectories followed by the robot and the obstacles, where the darkness of each point represents the experiment’s time for each sample (more dark is addressed to a time increasing). The separation of the points gives an idea of the speed reached by the robot and obstacles. One obstacle moves around a point and the human must take the short path through the other two fixed obstacles, trying to keep a low risk. The situation is present seven times along the path between the initial and final point, in order to stimulate the performance of the user before risk.Figure 9
Trajectories of the robot (triangles) and obstacles (circles).The torque applied to the master for both speed references (linear and angular) is shown in Figure10.Figure 10
Applied torque to the master.
(a)
(b)The impedance reduces the intensity of the commands applied in some moments (like at timet=9 s), depending on the value of the metric proposed (Figure 13). As a result of this, the robot dynamics and the channel delays, velocities (linear and angular) reached by the robot, are different with respect to the human’s commands references (Figure 11).Figure 11
Human’s commands beforeh1(kgqm(t)) and robot’s output.
(a)
(b)Figure12 shows that q˙ is bound to all time. Figure 13 exposes the real and command collision probabilities of the robot against each obstacle (PciPi), as well as the computation of the command metric proposed. It can be observed for t≈10 that the metric is high because the command tends to worsen the collision probability situation against obstacle 1.Figure 12
q
˙ of the master.
(a)
(b)Figure 13
Real and command collision probabilities of the robot against each obstacle.Additionally, a cost function to minimize was introduced in Table1 in order to summarize the information of the index introduced. For each row i of the table, Ji was computed as(29)Ji=0.3Ni+0.3Ttaski+0.1σTtaski+0.2HCPi+0.1σhcpi,where variables in bold represent the normalized value of the index (e.g., Ni=((Ni-Nmin⁡)/(Nmax⁡-Nmin⁡))100), and the weight gains were chosen according to the relevance of each index.Next, in order to achieve a direct performance comparison between schemes, cost functions of each one (JA1 is the first row of J, corresponding to the cost functional of the scheme A under low delays) can be averaged as in (30)JA=(JA1+JA2+JA3)3=49.29,JB=(JB1+JB2+JB3)3=36.58,JC=(JC1+JC2+JC3)3=29.15.It can be noted that schemeC (which represents the main proposal of this work) is the one that achieves the minimum cost function.


### 5.2. ExperimentE2

In order to test the system in a real robot, a Pioneer was teleoperated through a Local Network, based on the video feedback. The video of the experiment is available in  Supplementary Material available online athttp://dx.doi.org/10.1155/2015/460476.The channel delay was approximately about 300 ms and despite being lower than the one used in the experiments over a simulated robot, the visual feedback is affected by video limitations like field of view and scope and quality and size of the image. The video was transmitted using the free software YawCam (http://www.yawcam.com). In Figure 14, a resume of the local and remote sides is shown. The robot must go from the starting point to the final one, going between the two obstacles present in the environment.Figure 14
Local and remote sides of the experiment using a robot Pioneer (photographs not synchronized).The results obtained are in agreement with the results achieved with the 3D simulator.


## 5.1. ExperimentE1

ForE1 in this work, we simulate three levels of channel time delays:(1)
verylow (h1=h2=0.025),(2)
medium (h1=0.15; h2=0.1*sin(0.5*t)+0.15),(3)
high (h1=0.30; h2=0.1*sin(0.5*t)+0.3, quite big considering the robot dynamic.The parameters used for the mobile robot arem=30 kg, i=2.5 kg, and a=0.05.In the stability analysis of the proposed scheme, parameterskg, ks, αs, km, and αm were considered as scalar due to simplicity, but the analysis and results are also valid for positive definite diagonal matrices. Therefore, for the PD-like controller in the local site, the parameters set are(25)ks=34Nsm007Nmsrad;αs=0.5Nss2000.5Nms2radand, for the controller of the master (local side),(26)km=10Nsm003Nsm;αm=200Nss00130Nsm.Finally, in order to transform the position of the master to a speed reference,kg is established as follows:(27)kg=20.01s0020.01s.The range of the master positions in both axes is from-0.05[m] to 0.05[m]; therefore, the maximum speed linear and angular references are 1[m/s] and 1[rad/s]. The parameter kz is set to 0.8.In order to analyze the control scheme proposed against classical methods based on velocity impedance or other methods that do not incorporate an impedance law, differentkh=k1k2T are used: a constant high value (maximum), null, and kh variable value (from 0 to a maximum, depending on the metric proposed (Me(t)). In Table 1, cases Ai are for null velocity impedance kh=00T, cases Bi for constant velocity impedance kh=11T (classical method based on impedance), and cases Ci for impedance varying according to the metric kh=5Me5MeT.About the fictitious force for the impedance subsystem, the parameters taken aredmin⁡=0.5[m] and dmax⁡=3.5[m]; therefore a=1.16 and b=0.33 (see Section 3.1).Different drivers were taken to perform the required task (age range from 20 to 60, students and workers from the Universidad Nacional de San Juan).Table1 resumes the average values of evaluation index for the system (computed for experiment E1, 10 trials for each case row) in order to test the control system proposed and compare different cases listed above. N represents the percentage of experiments in which the robot crashes against any obstacle. In this case the experiment is stopped. Next, Ttask measures the time to complete the task.Then, HCP is an index abouthigh collision probability. On one hand, the main task is required to be done as quickly as possible (evaluated by Ttask), while, on the other hand, obstacles over the path represent restrictions to the movement of the robot. Hence, this index is about the risk the robot faces. We describe this evaluation metric as(28)HCP=1nob∑i=1nob1Texp⁡∫0Texp⁡Pi3(t)dt,where nob is the quantity of obstacles, Pi is the real risk robot-obstaclei, Texp⁡ is the time of the experiment, and Pi is the collisions probability of the robot against obstacle i. Designers can use any method for computing the collision probability. We use a method based on a linear integration of the velocities (velocity in x and y coordinates of the plane) of the robot and obstacle. The cubic function of the collision probability is chosen, due to the fact that itenhances high values of risk (high collision probabilities attempt mission’s safety) andreduces the low values (not important for evaluating risk). Standard deviation is also computed for each case (σTtask and σhcp).In order to enhance the understanding of the readers, Figure8 introduces a graphic of the index of the table.Figure 8
Graphic from Table1: percentage of collisions, time to complete the task, and index of high collision probability.
(a)
(b)
(c)A short part of one trial of experimentE1 was extracted and the main signals were plotted. Figure 9 shows the trajectories followed by the robot and the obstacles, where the darkness of each point represents the experiment’s time for each sample (more dark is addressed to a time increasing). The separation of the points gives an idea of the speed reached by the robot and obstacles. One obstacle moves around a point and the human must take the short path through the other two fixed obstacles, trying to keep a low risk. The situation is present seven times along the path between the initial and final point, in order to stimulate the performance of the user before risk.Figure 9
Trajectories of the robot (triangles) and obstacles (circles).The torque applied to the master for both speed references (linear and angular) is shown in Figure10.Figure 10
Applied torque to the master.
(a)
(b)The impedance reduces the intensity of the commands applied in some moments (like at timet=9 s), depending on the value of the metric proposed (Figure 13). As a result of this, the robot dynamics and the channel delays, velocities (linear and angular) reached by the robot, are different with respect to the human’s commands references (Figure 11).Figure 11
Human’s commands beforeh1(kgqm(t)) and robot’s output.
(a)
(b)Figure12 shows that q˙ is bound to all time. Figure 13 exposes the real and command collision probabilities of the robot against each obstacle (PciPi), as well as the computation of the command metric proposed. It can be observed for t≈10 that the metric is high because the command tends to worsen the collision probability situation against obstacle 1.Figure 12
q
˙ of the master.
(a)
(b)Figure 13
Real and command collision probabilities of the robot against each obstacle.Additionally, a cost function to minimize was introduced in Table1 in order to summarize the information of the index introduced. For each row i of the table, Ji was computed as(29)Ji=0.3Ni+0.3Ttaski+0.1σTtaski+0.2HCPi+0.1σhcpi,where variables in bold represent the normalized value of the index (e.g., Ni=((Ni-Nmin⁡)/(Nmax⁡-Nmin⁡))100), and the weight gains were chosen according to the relevance of each index.Next, in order to achieve a direct performance comparison between schemes, cost functions of each one (JA1 is the first row of J, corresponding to the cost functional of the scheme A under low delays) can be averaged as in (30)JA=(JA1+JA2+JA3)3=49.29,JB=(JB1+JB2+JB3)3=36.58,JC=(JC1+JC2+JC3)3=29.15.It can be noted that schemeC (which represents the main proposal of this work) is the one that achieves the minimum cost function.


## 5.2. ExperimentE2

In order to test the system in a real robot, a Pioneer was teleoperated through a Local Network, based on the video feedback. The video of the experiment is available in  Supplementary Material available online athttp://dx.doi.org/10.1155/2015/460476.The channel delay was approximately about 300 ms and despite being lower than the one used in the experiments over a simulated robot, the visual feedback is affected by video limitations like field of view and scope and quality and size of the image. The video was transmitted using the free software YawCam (http://www.yawcam.com). In Figure 14, a resume of the local and remote sides is shown. The robot must go from the starting point to the final one, going between the two obstacles present in the environment.Figure 14
Local and remote sides of the experiment using a robot Pioneer (photographs not synchronized).The results obtained are in agreement with the results achieved with the 3D simulator.


## 6. Discussions

### 6.1. Fix Velocity Impedance (CasesBi, kh=[11]T, Classical Method)

When the remote control applies a fixed gain-impedance, despite good or wrong operator commands, the robot moves very slowly when he navigates in the vicinity of the obstacle. The navigation of the robot is always safe for any kind of commands, but it offers bad performance in path and time to complete task for all occasions (about 35% more than the cases with the same time delay and using the metric; seeTtask of Table 1). In this case, theTransparency is unsatisfactory because the user’s command is highly modified by the controller, and the operator perceives that the robot does not respond to his orders.This impedance law does not consider human driver performance. The increment in the time delay does not affect significantly the Ttask (less than 15%). The indexes of collisions probabilities measurements are nice, concluding that a safe robot navigation can be assured by using a fix impedance but with a bad Ttask (time to complete the task), regardless of operator performance. Therefore, good commands are wasted by the local controller.


### 6.2. Null Velocity Impedance (CasesAi, kh=[00]T)

Here, thesafety is low and only depends on the delayed operator’s command performance. When bad commands occur, a high risk situation appears, and collisions could happen. This was demonstrated by the high percentage of collisions as a result of the bad commands. All participants express that they felt a good sensation when commanding the robot which exactly follows the joystick (niceTransparency) for low delays but, when time delay rises, the driving becomes quite difficult.Some good drivers achieve good executions, but this condition is difficult to maintain during a long task, or when secondary demanding tasks appear, which overload the attention or cognition of the driver, and therefore, poor decision and execution could be taken.


### 6.3. Variable Velocity Impedance (CasesCi, kh=[k1k2]T)

In this case, conditions ofsafety driving do not depend strongly on commands performance. If bad commands occur, Me takes high values and the risk situation is bounded. Thus, the collisions are mitigated at a cost of losing transparency.The impedance reduces the received commands only if the potential risk worsens due to the user executions.R is lower with this scheme than in the cases of scheme A, showing a safer navigation. Furthermore, standard deviation of R is approximately the same as the deviation for cases B, indicating a more homogeneous driving despite the performance of the commands executed. In addition, despite the time delay increment, standard deviation of each index of cases C remains similar, which proves that the homogeneous performance in front of delays increases.Time to complete the task depends somewhat on the commands performance, but it is smaller than caseA and closer to the best one reached.


## 6.1. Fix Velocity Impedance (CasesBi, kh=[11]T, Classical Method)

When the remote control applies a fixed gain-impedance, despite good or wrong operator commands, the robot moves very slowly when he navigates in the vicinity of the obstacle. The navigation of the robot is always safe for any kind of commands, but it offers bad performance in path and time to complete task for all occasions (about 35% more than the cases with the same time delay and using the metric; seeTtask of Table 1). In this case, theTransparency is unsatisfactory because the user’s command is highly modified by the controller, and the operator perceives that the robot does not respond to his orders.This impedance law does not consider human driver performance. The increment in the time delay does not affect significantly the Ttask (less than 15%). The indexes of collisions probabilities measurements are nice, concluding that a safe robot navigation can be assured by using a fix impedance but with a bad Ttask (time to complete the task), regardless of operator performance. Therefore, good commands are wasted by the local controller.


## 6.2. Null Velocity Impedance (CasesAi, kh=[00]T)

Here, thesafety is low and only depends on the delayed operator’s command performance. When bad commands occur, a high risk situation appears, and collisions could happen. This was demonstrated by the high percentage of collisions as a result of the bad commands. All participants express that they felt a good sensation when commanding the robot which exactly follows the joystick (niceTransparency) for low delays but, when time delay rises, the driving becomes quite difficult.Some good drivers achieve good executions, but this condition is difficult to maintain during a long task, or when secondary demanding tasks appear, which overload the attention or cognition of the driver, and therefore, poor decision and execution could be taken.


## 6.3. Variable Velocity Impedance (CasesCi, kh=[k1k2]T)

In this case, conditions ofsafety driving do not depend strongly on commands performance. If bad commands occur, Me takes high values and the risk situation is bounded. Thus, the collisions are mitigated at a cost of losing transparency.The impedance reduces the received commands only if the potential risk worsens due to the user executions.R is lower with this scheme than in the cases of scheme A, showing a safer navigation. Furthermore, standard deviation of R is approximately the same as the deviation for cases B, indicating a more homogeneous driving despite the performance of the commands executed. In addition, despite the time delay increment, standard deviation of each index of cases C remains similar, which proves that the homogeneous performance in front of delays increases.Time to complete the task depends somewhat on the commands performance, but it is smaller than caseA and closer to the best one reached.


## 7. Conclusions

The work has presented a bilateral teleoperation controller for wheeled robots under time delays, in which the master controller is a PD-like scheme with gravity compensation and the slave controller is a scheme that incorporates an impedance compensation based on fictitious force and a variable gain, employed for obstacle avoidance. The variable gain is a function of the human operator performance when facing risk situations. The metric gives an online value that qualifies the command performance under risk situations, taking into account multiple obstacles environment.With multiple experiments (Human in the Loop simulations and an experiment with a real robot), it has been proved that the proposed scheme has a better trade-off between safety, transparency, and time to completion, without a hard dependence on the operators expertise and differences in time delays. It also has been demonstrated that the system is stable even when incorporating the variable impedance compensation, based on a Lyapunov analysis, which also gives a guideline to the calibration of the control parameters.



---

*Source: 460476-2015-03-04.xml*