# Complete Bilateral Teleoperation System for a Rotorcraft UAV with Time-Varying Delay

Authors: Lucio R. Salinas; Emanuel Slawiñski; Vicente A. Mut
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

This paper proposes the use of simple proportional
plus damping injection (P+d) controllers for delayed
bilateral teleoperation of a rotorcraft UAV. The proposed
control scheme involves P+d remote and local
controllers, considers master and slave dynamics, and
takes into account asymmetric time-varying delays.
The stability of the proposed teleoperation system is
analyzed using Lyapunov-Krasovskii functionals and
delay-dependent stability criteria are obtained under
linear-matrix-inequalities conditions. The performance
of the teleoperation scheme is tested driving
a virtual nonlinear 6DOF dynamic model of a minihelicopter
in a human-in-the-loop simulation.


---

## Body

## 1. Introduction

Over the last two decades, unmanned aerial vehicles (UAVs) have become an important scientific research topic. The reason for this is based on three main aspects: UAVs avoid the human risk inherent to manned aerial vehicles in many situations (e.g., hostile environment and adverse weather condition); they are generally far less expensive and smaller than their manned counterparts; and they can reach places and/or overfly specific areas that other vehicles cannot.Within UAVs, rotorcraft-based UAVs (RUAVs) have unique flight capabilities such as hover, vertical take-off and landing, and side-slip, which cannot be achieved by conventional fixed-wing aircraft. The hovering mode and flying at low speed make RUAVs ideal for reconnaissance missions and very useful in rescue missions [1]. These singular features called the attention of many academic research groups in the last years. Although there has been much research in the area of RUAVs, the majority of published works have covered the design and implementation of control systems for autonomous navigation only (see [2] and references therein). However, a lot of inspection, surveillance, and monitoring tasks cannot be easily performed with a fully autonomous vehicle; the need for high-level reasoning and specific knowledge inherent in the task make RUAV teleoperation more suitable for these missions [3].In general, UAV teleoperation systems have three elements: a ground station (local site), where a human operator receives information and drives a hand-controller device (master device); a UAV (slave vehicle or device) that receives control signals and flies through an environment (remote site); and a communication channel that links both sites. The master device at the local site and the slave vehicle at the remote site are endowed with local controllers as seen in Figure1. The information provided in the local site is usually visual and the operator lacks other sensory information that a pilot of a manned aircraft would normally perceive, such as sound, vibration, and motion [4]. Additionally, the communication channel adds delays in the range of 100 to 1670 milliseconds (and even more) [5] that can produce instability or poor performance [6, 7].Figure 1
Block diagram of a UAV teleoperation system.In recent years, several authors have developed teleoperation control systems for RUAVs; even so, only a few have focused on delayed bilateral teleoperation schemes. Bilateral teleoperation provides the human operator with a haptic feedback; this signal can complement the visual information to improve thetelepresence and enhance safety; as a drawback, it can destabilize the closed-loop teleoperation system [8, 9].In [10] the influence of different force-feedback settings on a human operator is studied and thewave variables conventional teleoperation technique [11, 12] is used to reduce the negative effects of time delays. In [13] a bilateral teleoperation of multiple mobile slave agents coupled to a single master robot under constant communication delays is achieved using a passifying PD-based controller; nevertheless, it addresses only the master-slave position-position teleoperation and thus it is not suitable for UAVs with unbounded workspace. In [14–17] a bilateral teleoperation of multiple UAVs is performed by a single human operator and the stability is analyzed employing passivity theory. In the first two researches, the passivity of the master side is obtained applyingfeedback r-passivity notion [18]; then, it is claimed that the passivity of the teleoperation system can be easily enforced using any of the techniques developed in conventional teleoperation settings. In the latter two investigations,passive set-position modulation (PSPM) framework [19] is utilized to theoretically guarantee master-passivity/VPs-stability of the closed-loop teleoperation system (where VPs refer to the virtual points followed by each UAV). In [20] the port-based modeling network, in particular, port-Hamiltonian systems and bond graphs, is used in the bilateral teleoperation of aerial robots, where a variable impedance master controller and avirtual slave system allow the design of a passive teleoperation control architecture.Contribution. In bilateral teleoperation of manipulator robots, proportional plus damping injection (P+d) controllers have gained momentum in the last few years [21–23]. Due to its simplicity and robustness, this work proposes the use of P+d controllers for bilateral teleoperation of a RUAV with time-varying delay considering the master device and slave vehicle dynamics. The main contribution of the paper involves combining in a simple way P+d controllers, a trajectory tracking controller, an impedance control based on fictitious force, an elastic coupling, and a cascade PID architecture, to achieve a stable bilateral teleoperation of an underactuated 6DOF minihelicopter in presence of time-varying delay. The stability analysis carried out gives as a result a delay-dependent LMI condition, which can be held if the control parameters are set from such condition, assuring thus a stable performance. In the state of the art in the field of delayed bilateral teleoperation of aerial vehicles, most controllers are based on passivity theory and they generally include a difficult calibration procedure. Different from this, the proposed scheme involves a simple parameters setting where the control blocks (P+d control, trajectory tracking controller, impedance control, etc.) can be set independently in practice.To test the performance of the teleoperation scheme a human-in-the-loop (HITL) simulation is carried out; there, a human operator drives a RUAV through an outdoor environment and has visual and force feedback under a communication channel with asymmetric time-varying delay. Two master devices are used increasing the maneuverability of the RUAV and the visual feedback is only generated by an onboard camera providing a field of view like the one achievable in practice.The rest of the paper is organized as follows. Sections2 and 3 introduce the models of the master device and slave vehicle, respectively. Section 4 presents some mathematical properties and assumptions. The P+d controllers are proposed and explained in Section 5. In Section 6 the stability analysis is outlined. The complete teleoperation system is presented in Section 7. The HITL simulation framework and the simulation results are provided in Section 8. Finally, the conclusions are given in Section 9.


## 2. Master Device

The master device at the local site is composed of two parts: the translational master and the rotational master. The first is a 3-degree-of-freedom (DOF) device with force feedback which controls the translation rates of the UAV and the second is a 1DOF device (a two-pedal configuration) which commands the rotation rate of the UAV (see Figure2).Figure 2
3DOF Novint Falcon translational master device and 1DOF Genius rotational master device.


### 2.1. Translational Master Device

The typical nonlinear dynamic model in Cartesian coordinates is used to represent the 3DOF device:(1)Mxx¨+Cx,x˙x˙+gmx=fm∗+fh,where x∈R3 is the end-effector position, M∈R3×3 is the inertia matrix, C∈R3×3 is the matrix representing centripetal and Coriolis effects, gm∈R3 is the gravitational force, and fm∗∈R3 and fh∈R3 are the control and human operator forces applied to the translational master, respectively.The gravitational force is precompensated by the controllerfm∗; that is, fm∗=fm+gmx. The final model for the translational master is the following:(2)Mxx¨+Cx,x˙x˙=fm+fh.


### 2.2. Rotational Master Device

The 1DOF device is modeled as two forced mass-spring-damper systems:(3)mrq¨+brq˙+krq=τh,where q=qrpqlpT∈R2 is a vector indicating the joint angular displacement of the right and left pedals; mr, br, and kr are the pedals inertia, viscosity, and stiffness, respectively; and τh=τrfτlfT∈R2 is a vector that indicates the human operator’s right and left feet torques.The output signalqp of the rotational master is the difference between the joint angular displacements of the pedals; that is, qp=qrp-qlp.


## 2.1. Translational Master Device

The typical nonlinear dynamic model in Cartesian coordinates is used to represent the 3DOF device:(1)Mxx¨+Cx,x˙x˙+gmx=fm∗+fh,where x∈R3 is the end-effector position, M∈R3×3 is the inertia matrix, C∈R3×3 is the matrix representing centripetal and Coriolis effects, gm∈R3 is the gravitational force, and fm∗∈R3 and fh∈R3 are the control and human operator forces applied to the translational master, respectively.The gravitational force is precompensated by the controllerfm∗; that is, fm∗=fm+gmx. The final model for the translational master is the following:(2)Mxx¨+Cx,x˙x˙=fm+fh.


## 2.2. Rotational Master Device

The 1DOF device is modeled as two forced mass-spring-damper systems:(3)mrq¨+brq˙+krq=τh,where q=qrpqlpT∈R2 is a vector indicating the joint angular displacement of the right and left pedals; mr, br, and kr are the pedals inertia, viscosity, and stiffness, respectively; and τh=τrfτlfT∈R2 is a vector that indicates the human operator’s right and left feet torques.The output signalqp of the rotational master is the difference between the joint angular displacements of the pedals; that is, qp=qrp-qlp.


## 3. Slave Vehicle

RUAVs belong to the class of underactuated mechanical systems, which have fewer control inputs than degrees of freedom. The 6DOF dynamic model of a RUAV is very complex, and it is certainly difficult to obtain the mathematical equations and parameters that follow its physical model [24]. One practical solution to this problem could be to use a more data-driven/model-free design framework of the control scheme (see [25] and references therein). Another solution, applied in this work, is to use a simplified fully actuated 4DOF model in the controller design stage and then couple it to a full RUAV model. In Section 8, the interaction between the 4DOF model and a more realistic underactuated 6DOF RUAV model of a minihelicopter is explained.In Figure3, the reference point o0 on the surface of the earth is the origin of a right handed orthogonal system of axes (o0x0y0z0), o0x0 points to the north, o0y0 points to the east, and o0z0 points vertically “down” along the gravity vector (NED Cartesian system). The kinematics and dynamics of the 4DOF vehicle are equivalent to the ones of a rigid body whose roll and pitch movements are constrained; hence, itsbody axes (oExEyEzE) are tantamount to a Cartesian coordinate system “tied” to the earth by means of gravity vector with the difference being that the origin oE coincides with the center of gravity of the aircraft and oExE axis points forward along the vehicle.Figure 3
Fully actuated 4DOF RUAV.The fully actuated 4DOF dynamic model is the following:(4)Dη˙∗+Qη∗η∗+gs=σs∗+σe,where η∗=vxErvyErvzErωzErT∈R4 is the RUAV velocity vector with vxEr, vyEr, and vzEr representing the forward-backward, lateral, and downward-upward linear velocity, respectively, and ωzEr is the angular velocity as seen in Figure 3. σe∈R4 is the force/torque caused by the interaction between the environment and the RUAV and σs∗∈R4 is the control force/torque applied to the RUAV.The inertia matrixD∈R4×4, the Coriolis matrix Qη∗∈R4×4, and the gravity vector gs∈R4 are the following: (5)D=m0000m0000m0000ir,Q=0-mωzEr00mωzEr00000000000,gs=00-mg0T,where m, ir, and g are positive constants denoting the mass, the rotational inertia, and the gravitational acceleration of the RUAV, respectively.To devise the translational and rotational control actions independently the RUAV dynamic model is decoupled. For this purpose, the control force/torqueσs∗ in (4) is established as follows:(6)σs∗=σs+Qη∗η∗+gs.Including (6) in (4) we obtain Dη˙∗=σs+σe, being the translational slave dynamics(7)mη˙=fs+feand the rotational slave dynamics(8)irη˙r=τs+τe,where η˙∗T=η˙Tη˙r=v˙xErv˙yErv˙zErω˙zEr, σsT=fsTτs, and σeT=feTτe.


## 4. Preliminaries

The following properties, assumptions, and lemma will be used in this paper.Property 1.
The inertia matrixM is a symmetric positive-definite function and there exist positive constants m1 and m2 such that m1I≤M≤m2I.Property 2.
The matrixM˙-2C is skew symmetric.Property 3.
There exists a positive constantkc such that Cx˙≤kc|x˙|.Assumption 1.
The human operator and the environment behave in a nonpassive way and they are represented by the following models:(9)fh=-mhx¨-αhx˙+fh∗(10)fe=-meη˙-αeη+fe∗,where mi, αi, and fi∗ with i=h,e are the inertia, damping, and exogenous force of the human operator and environment, respectively. The exogenous forces are subject to fi∗∞<f¯i∗<+∞, and f¯i∗ is a positive constant.Note. Models (9) and (10) are modified versions of the second-order LTI models successfully used by other researchers [26–28]. In (9) and (10), the stiffness of the human operator and environment is ignored considering the operator does not want to maintain a certain hand position and the vehicle makes no physical contact with the environment. Also, mi and αi in (9) and (10) were taken as scalars but they can be diagonal matrices or, more generally, positive-definite matrices.Assumption 2.
The communication channel is represented by a forward time-varying delayh1t (local site → remote site) and a backward time-varying delay h2t (remote site → local site). These time delays are considered bounded; that is, there exist positive scalars h¯1 and h¯2 such that 0≤h1t≤h¯1 and 0≤h2t≤h¯2 for all t.Lemma 3 (see [22]).
For vector functionsa· and b·, a positive-definite matrix Δ, and a time-varying scalar ht with 0≤ht≤h¯, the following inequality holds: (11)-2aTt∫t-httbξdξ-∫t-httbTξΔbξdξ≤htaTtΔ-1at≤h¯taTtΔ-1at.


## 5. P+d Controllers for Bilateral Teleoperation

The teleoperation system has two controllers (Figure1) that interact with the master device and consequently the human operator on the one hand and with the slave vehicle and its environment on the other hand. Both controllers have a simple P+d structure; thus they possess few parameters that can be quickly calibrated.


### 5.1. Remote Controller

The aim of the remote controller is to synchronize the slave vehicle’s linear and angular velocities (η,ηr) with the position and angular displacement of the master device, respectively. The control actions are the following:(12)fs=ksknxt-h1-η-αsz,(13)τs=ksr⁡knrqpt-h1-ηr-αsr⁡zr,where ks, ksr⁡, αs, and αsr⁡ are positive constant parameters representing the proportional gain and acceleration-dependent damping added by the P+d controllers. kn and knr are positive constant parameters that map master position and angular displacement to velocity commands and z∗∈R4 is an auxiliary vector defined as(14)z∗+γz˙∗=η˙∗,where γ→0+ and z∗T=zTzr. That is, z∗ represents the RUAV acceleration η˙∗ at an infinitesimal time instant before t.The end-effector position vector of the translational master and the angular displacement of the rotational master generate linear and angular velocity setpoints for the remote controller. This relation between master device and slave vehicle helps overcome the master-slave kinematic dissimilarity typical of mobile robot teleoperation; that is, the master device has bounded workspace and the slave vehicle has unbounded workspace.


### 5.2. Local Controller

The objective of the local controller is to provide the human operator with a haptic cue that represents the mismatch between the commanded velocity (specified by the end-effector position vector) and the velocity of the slave. The mismatch gives different kind of information: dynamics of the slave vehicle, external disturbances on the RUAV motion, and quality of velocity tracking in the remote site. The local controller only affects the translational master device as the rotational one is unactuated. The control action is the following:(15)fm=-kmx-kn-1ηt-h2-αmx˙,where km and αm are positive constant parameters representing a relative spring depending on the mismatch between the master reference and the RUAV linear velocity and damping added by the P+d controller, respectively. kn-1 maps slave vehicle’s velocity to master position.


## 5.1. Remote Controller

The aim of the remote controller is to synchronize the slave vehicle’s linear and angular velocities (η,ηr) with the position and angular displacement of the master device, respectively. The control actions are the following:(12)fs=ksknxt-h1-η-αsz,(13)τs=ksr⁡knrqpt-h1-ηr-αsr⁡zr,where ks, ksr⁡, αs, and αsr⁡ are positive constant parameters representing the proportional gain and acceleration-dependent damping added by the P+d controllers. kn and knr are positive constant parameters that map master position and angular displacement to velocity commands and z∗∈R4 is an auxiliary vector defined as(14)z∗+γz˙∗=η˙∗,where γ→0+ and z∗T=zTzr. That is, z∗ represents the RUAV acceleration η˙∗ at an infinitesimal time instant before t.The end-effector position vector of the translational master and the angular displacement of the rotational master generate linear and angular velocity setpoints for the remote controller. This relation between master device and slave vehicle helps overcome the master-slave kinematic dissimilarity typical of mobile robot teleoperation; that is, the master device has bounded workspace and the slave vehicle has unbounded workspace.


## 5.2. Local Controller

The objective of the local controller is to provide the human operator with a haptic cue that represents the mismatch between the commanded velocity (specified by the end-effector position vector) and the velocity of the slave. The mismatch gives different kind of information: dynamics of the slave vehicle, external disturbances on the RUAV motion, and quality of velocity tracking in the remote site. The local controller only affects the translational master device as the rotational one is unactuated. The control action is the following:(15)fm=-kmx-kn-1ηt-h2-αmx˙,where km and αm are positive constant parameters representing a relative spring depending on the mismatch between the master reference and the RUAV linear velocity and damping added by the P+d controller, respectively. kn-1 maps slave vehicle’s velocity to master position.


## 6. Stability of the Bilateral Teleoperation System

The rotational part of the teleoperation system is unilateral; thus the complexity of its control is much reduced; therefore, only the stability of the translational part is analyzed using the theory of Lyapunov-Krasovskii [29]. It is important to remark that there is not an equilibrium point but a Krasovskii-like equilibrium solution that depends on the state ϕT=x˙Tknx-ηTηTzT in the time interval t-h1-h2,t.Theorem 4.
Consider a delayed bilateral teleoperation system where a human operator (9) using a master device (2) drives a RUAV (7) interacting with an environment (10), and control laws (12), (13), and (15) are included. For positive constant parameters km, ks, kn, αm, and αs, slave vehicle mass m, and maximum time delays h¯1 and h¯2, considering Assumptions 1 and 2 and Properties 1, 2, and 3, if there exist positive-definite matrices P and Q such that the following LMI holds,(16)Ψ1,112h¯2kmks00-h¯2P00∗Ψ3,312h¯1kmks∗∗-h¯1Q<0,where (17)Ψ1,1=-αmksknI+h¯1QΨ3,3=-αs+mkmknI+h¯2Pand ∗ represents the transpose of corresponding matrix, then ϕ∈L∞.Proof.
First, a positive-definite functionalV=V1+V2+V3+V4+V5>0 is proposed in order to analyze its evolution along the system trajectories. The first four subfunctionals depend on ϕ and are defined in the following manner:(18)V1=12ksknx˙TM+mhIx˙(19)V2=12kmksknknx-ηTknx-η(20)V3=αe12kmknηTη(21)V4=m+meγ12kmknzTz.
The time derivative ofV1 along the master dynamics (2), taking into account Properties 1 and 2 and considering (9), is the following one: (22)V˙1=12ksknx˙TM˙x˙+ksknx˙TM+mhIx¨=12ksknx˙TM˙-2Cx˙+ksknx˙Tfm+ksknx˙T-αhx˙+fh∗=ksknx˙Tfm+ksknx˙T-αhx˙+fh∗.
Now, if the control actionfm (15) is included in (22), it yields (23)V˙1=ksknx˙T-kmx-kn-1ηt-h2-αmx˙+ksknx˙T-αhx˙+fh∗=-kmksx˙Tknx-ηt-h2+η-η-αm+αhksknx˙Tx˙+ksknx˙Tfh∗=-kmksx˙Tknx-η-kmksx˙T∫t-h2tη˙ξdξ-αm+αhksknx˙Tx˙+ksknx˙Tfh∗.
Next,V˙2 is obtained from (19) considering (14): (24)V˙2=kmksknknx-ηTknx˙-η˙=kmksknx-ηTx˙-kmksknknx-ηTz-γkmksknknx-ηTz˙.
On the other hand,V˙3 is computed in a similar way from (20), taking into account (14):(25)V˙3=αekmknηTη˙=αekmknηTz+γαekmknηTz˙.
Moreover,V˙4 can be written including (12) in (7) and considering (10) and (14), in the following way: (26)V˙4=m+meγkmknzTz˙=m+meγkmknzTη˙γ-zγ=kmknzTfs-αeη+fe∗-m+mekmknzTz=kmknzTksknxt-h1-η-αsz+kmknzT-αeη+fe∗-m+mekmknzTz=kmksknzTknxt-h1-η+knx-knx-αs+m+mekmknzTz-αekmknzTη+kmknzTfe∗=kmksknzTknx-η-kmkszT∫t-h1tx˙ξdξ-αs+m+mekmknzTz-αekmknzTη+kmknzTfe∗.
It is possible to appreciate in (23) and (26) that there are terms like -kmkszT∫t-h1t‍x˙ξdξ which make stability analysis difficult; to solve this, V5 is proposed as follows: (27)V5=∫-h¯20∫t+θtzTξPzξdξdθ+∫-h¯10∫t+θtx˙TξQx˙ξdξdθ,where P and Q are positive-definite matrices.
From (27) and considering Assumption 2, V˙5 can be computed by (28)V˙5=h¯2zTPz-∫t-h¯2tzTξPzξdξ+h¯1x˙TQx˙-∫t-h¯1tx˙TξQx˙ξdξ,V˙5≤h¯2zTPz-∫t-h2tzTξPzξdξ+h¯1x˙TQx˙-∫t-h1tx˙TξQx˙ξdξ.
On the other hand, each term of (23) and (26) that includes a delayed variable can be conveniently joined using Lemma 3 with one term of V˙5. The second term of (23) and the second term of (28) can be linked, considering (14): (29)-kmksx˙T∫t-h2tη˙ξdξ-∫t-h2tzTξPzξdξ=-kmksx˙T∫t-h2tzξdξ-∫t-h2tzTξPzξdξ-γkmksx˙T∫t-h2tz˙ξdξ≤h¯2km24ks-2x˙TP-1x˙-γkmksx˙T∫t-h2tz˙ξdξ.
In the same manner, the second term of (26) and the fourth term of (28) can be combined to establish the following: (30)-kmkszT∫t-h1tx˙ξdξ-∫t-h1tx˙TξQx˙ξdξ≤h¯1km24ks-2zTQ-1z.
Finally,V˙ can be written associating (23)–(26) and (28)–(30), disregarding the terms with the infinitesimal γ and neglecting αh and me, in the following way: (31)V˙=V˙1+V˙2+V˙3+V˙4+V˙5≤x˙T-αmksknI+h¯1Q+h¯2km24ks-2P-1x˙+zT-αs+mkmknI+h¯2P+h¯1km24ks-2Q-1z+ksknx˙Tfh∗+kmkn-1zTfe∗,where I∈R3×3 is the identity matrix.
LMI conditions (16) are equivalent to the following two inequalities:(32)-αmksknI+h¯1Q+h¯2km24ks-2P-1<0-αs+mkmknI+h¯2P+h¯1km24ks-2Q-1<0.
Givenh¯1 and h¯2 maximum delays and slave vehicle mass m, the controllers parameters (km, ks, αm, αs, and kn) can be selected according to LMI conditions (16) to ensure that the two inequalities (32) are satisfied; that is, the first two terms of (31) are negative definite and therefore the variables x˙, knx-η, η, and z are bounded (ϕ∈L∞). The proof is completed.Remark 5.
If the exogenous force of the human operatorfh∗ and environment fe∗ are null (f¯h∗=f¯e∗=0), then the system is stable. For this particular case, Barbalat’s lemma can be used in (31), taking into account Assumptions 1 and 2, Property 3, and the fact that ϕ∈L∞ (and x∈L∞); it is possible to deduce that x¨ and z˙ are bounded; then, V¨ is bounded too. Finally, x˙ and z will tend to zero as t→∞.


## 7. Complete Bilateral Teleoperation System

To test the performance of the teleoperation system a HITL simulation will be executed; there, a human operator will drive a very realistic 6DOF model of a minihelicopter (extracted from [24]) through an obstructed environment. The helicopter model considers nonideal dynamics, for example, flapping, drag, and actuators’ dynamic, and precisely describes the vehicle’s motion for both hovering and low-speed translational flight.The minihelicopter needs to be outside the closed-loop bilateral teleoperation system for the stability analysis of previous section to remain valid; hence, a coupling between the 4DOF and 6DOF RUAVs is proposed. The complete bilateral teleoperation system can be observed in Figure4.Figure 4
Complete bilateral teleoperation system. The slave vehicle block is composed of the fully actuated 4DOF RUAV and the underactuated 6DOF RUAV (minihelicopter) both attached to a coupling controller.The coupling controller has two main functions: allow the 6DOF RUAV to track the trajectory of the 4DOF RUAV and transmit the interactions between the minihelicopter and its environment to the 4DOF RUAV. In addition, obstacle avoidance is incorporated to the minihelicopter. Next, each part of the coupling controller is explained.


### 7.1. Trajectory Tracking Controller

Inspired by [30], the 4DOF RUAV is treated as a moving reference frame r (axes oExEyEzE) whose trajectory is the reference for the helicopter frame h (axes oExEyEzE) attached to the 6DOF as seen in Figure 5. The axes (oExEyEzE) of the frame h are not the actualbody axes of the minihelicopter but 4DOF RUAV-like axes. The kinematics of the helicopter frame h is equal to the kinematics of the reference frame r:(33)x˙iy˙iz˙iψ˙i=cos⁡ψi-sin⁡ψi00sin⁡ψicos⁡ψi0000100001vxEivyEivzEiωzEi,being i=r for the reference frame and i=h for the helicopter frame.Figure 5
Position and orientation of the 4DOF and 6DOF RUAVs along with helicopterh and reference r frames.Considering the RUAVs’ kinematics model (33), the trajectory tracking problem corresponds to the design of a control law that allows the helicopter frame to reach and automatically follow a path described by the reference frame according to the 4DOF RUAV velocities. The controller must generate the flight commands vxEh, vyEh, vzEh, and ωzEh such that(34)Xrt-Xht⟶0,where Xh=xhyhzhψhT and Xr=xryrzrψrT.First, the following errors are defined:(35)e=e1e2e3e4=cos⁡ψhsin⁡ψh00-sin⁡ψhcos⁡ψh0000100001xr-xhyr-yhzr-zhψr-ψh.The statesei are errors (xr-xh), (yr-yh), (zr-zh), and (ψr-ψh) expressed in the helicopter frame h. The time derivative of e is the following:(36)e˙=0ωzEh00-ωzEh00000000000e1e2e3e4+1000010000100001μ1μ2μ3μ4being(37)μ1=-vxEh+vxErcos⁡e4-vyErsin⁡e4μ2=-vyEh+vxErsin⁡e4+vyErcos⁡e4μ3=-vzEh+vzErμ4=-ωzEh+ωzEr.Now, the following Lyapunov candidate is considered:(38)Ve=12e12+e22+e32+e42.The time derivativeV˙e along the system states (35) is given by(39)V˙e=e1e˙1+e2e˙2+e3e˙3+e4e˙4V˙e=e1μ1+ωne2+e2μ2-ωne1+e3μ3+e4μ4V˙e=e1μ1+e2μ2+e3μ3+e4μ4.If the flight commands in (37) are selected as follows(40)vxEh=k1tanh⁡k5e1+vxErcos⁡e4-vyErsin⁡e4vyEh=k2tanh⁡k5e2+vxErsin⁡e4+vyErcos⁡e4vzEh=k3tanh⁡k5e3+vzErωzEh=k4e4+ωzErwith ki>0, and then (37) is replaced in (39), the final expression for V˙e is(41)V˙e=-k1e1tanh⁡k5e1-k2e2tanh⁡k5e2-k3e3tanh⁡k5e3-k4e42,which is a negative definite function. This means that system errors e1,e2,e3,e4 asymptotically converge to zero.


### 7.2. Impedance Controller

To add obstacle avoidance in the coupling controller the following correction to the flight commands (40) is made:(42)vxEch=vxEh-kvxFlonvyEch=vyEh-kvyFlatvzEch=vzEh-kvzFverωzEch=ωzEh.The fictitious force is calculated using a simulated 3D laser sensor which measures distance between the minihelicopter and the obstacles. The force is obtained as follows:(43)FlonFlatFver=klon∑i=1jficos⁡ϕicos⁡φiklat∑i=1jfisin⁡ϕicos⁡φikver∑i=1jfisin⁡φi,Flon, Flat, and Fver being the longitudinal, lateral, and vertical forces, respectively; klon, klat, and kver normalize forces to 0,1 for Flon and -1,1 for Flat and Fver. fi=a-bdi(t) indicates the magnitude of the force for each laser beam (1≤i≤j), with a,b>0 such that a-bdmax⁡=0 and a-bdmin⁡=1(dmin⁡≤di≤dmax⁡). Angles ϕi and φi express the direction of each laser beam in the horizontal and vertical plane, respectively.


### 7.3. Velocity Controller

To let the minihelicopter track the flight commands (42), the velocity controller depicted in Figure 6 is incorporated. It has a velocity transformation matrix that maps the 6DOF actualbody axes kinematics (u,v,w,p,q,r in Figure 5) to the helicopter frame h and a cascade PID architecture to generate the servo signal inputs (see details in [31]).Figure 6
Block diagram of the velocity controller.


### 7.4. Elastic Controller

The interaction between the helicopter and its environment is reflected to the human operator, improving thetelepresence, thanks to the elastic controller; moreover, the position and orientation errors between RUAVs are minimized. To achieve both goals, the exogenous environment force fe∗ and an exogenous environment torque τe∗ acting on the 4DOF RUAV are considered to be the following:(44)fe∗=-kepEp,τe∗=-keoeoEp=e1e2e3T and eo=e4 being the position and orientation errors previously defined in (35). Equation (44) must satisfy -kepEp≤f¯e∗ to fulfill the stability analysis of Section 6.


## 7.1. Trajectory Tracking Controller

Inspired by [30], the 4DOF RUAV is treated as a moving reference frame r (axes oExEyEzE) whose trajectory is the reference for the helicopter frame h (axes oExEyEzE) attached to the 6DOF as seen in Figure 5. The axes (oExEyEzE) of the frame h are not the actualbody axes of the minihelicopter but 4DOF RUAV-like axes. The kinematics of the helicopter frame h is equal to the kinematics of the reference frame r:(33)x˙iy˙iz˙iψ˙i=cos⁡ψi-sin⁡ψi00sin⁡ψicos⁡ψi0000100001vxEivyEivzEiωzEi,being i=r for the reference frame and i=h for the helicopter frame.Figure 5
Position and orientation of the 4DOF and 6DOF RUAVs along with helicopterh and reference r frames.Considering the RUAVs’ kinematics model (33), the trajectory tracking problem corresponds to the design of a control law that allows the helicopter frame to reach and automatically follow a path described by the reference frame according to the 4DOF RUAV velocities. The controller must generate the flight commands vxEh, vyEh, vzEh, and ωzEh such that(34)Xrt-Xht⟶0,where Xh=xhyhzhψhT and Xr=xryrzrψrT.First, the following errors are defined:(35)e=e1e2e3e4=cos⁡ψhsin⁡ψh00-sin⁡ψhcos⁡ψh0000100001xr-xhyr-yhzr-zhψr-ψh.The statesei are errors (xr-xh), (yr-yh), (zr-zh), and (ψr-ψh) expressed in the helicopter frame h. The time derivative of e is the following:(36)e˙=0ωzEh00-ωzEh00000000000e1e2e3e4+1000010000100001μ1μ2μ3μ4being(37)μ1=-vxEh+vxErcos⁡e4-vyErsin⁡e4μ2=-vyEh+vxErsin⁡e4+vyErcos⁡e4μ3=-vzEh+vzErμ4=-ωzEh+ωzEr.Now, the following Lyapunov candidate is considered:(38)Ve=12e12+e22+e32+e42.The time derivativeV˙e along the system states (35) is given by(39)V˙e=e1e˙1+e2e˙2+e3e˙3+e4e˙4V˙e=e1μ1+ωne2+e2μ2-ωne1+e3μ3+e4μ4V˙e=e1μ1+e2μ2+e3μ3+e4μ4.If the flight commands in (37) are selected as follows(40)vxEh=k1tanh⁡k5e1+vxErcos⁡e4-vyErsin⁡e4vyEh=k2tanh⁡k5e2+vxErsin⁡e4+vyErcos⁡e4vzEh=k3tanh⁡k5e3+vzErωzEh=k4e4+ωzErwith ki>0, and then (37) is replaced in (39), the final expression for V˙e is(41)V˙e=-k1e1tanh⁡k5e1-k2e2tanh⁡k5e2-k3e3tanh⁡k5e3-k4e42,which is a negative definite function. This means that system errors e1,e2,e3,e4 asymptotically converge to zero.


## 7.2. Impedance Controller

To add obstacle avoidance in the coupling controller the following correction to the flight commands (40) is made:(42)vxEch=vxEh-kvxFlonvyEch=vyEh-kvyFlatvzEch=vzEh-kvzFverωzEch=ωzEh.The fictitious force is calculated using a simulated 3D laser sensor which measures distance between the minihelicopter and the obstacles. The force is obtained as follows:(43)FlonFlatFver=klon∑i=1jficos⁡ϕicos⁡φiklat∑i=1jfisin⁡ϕicos⁡φikver∑i=1jfisin⁡φi,Flon, Flat, and Fver being the longitudinal, lateral, and vertical forces, respectively; klon, klat, and kver normalize forces to 0,1 for Flon and -1,1 for Flat and Fver. fi=a-bdi(t) indicates the magnitude of the force for each laser beam (1≤i≤j), with a,b>0 such that a-bdmax⁡=0 and a-bdmin⁡=1(dmin⁡≤di≤dmax⁡). Angles ϕi and φi express the direction of each laser beam in the horizontal and vertical plane, respectively.


## 7.3. Velocity Controller

To let the minihelicopter track the flight commands (42), the velocity controller depicted in Figure 6 is incorporated. It has a velocity transformation matrix that maps the 6DOF actualbody axes kinematics (u,v,w,p,q,r in Figure 5) to the helicopter frame h and a cascade PID architecture to generate the servo signal inputs (see details in [31]).Figure 6
Block diagram of the velocity controller.


## 7.4. Elastic Controller

The interaction between the helicopter and its environment is reflected to the human operator, improving thetelepresence, thanks to the elastic controller; moreover, the position and orientation errors between RUAVs are minimized. To achieve both goals, the exogenous environment force fe∗ and an exogenous environment torque τe∗ acting on the 4DOF RUAV are considered to be the following:(44)fe∗=-kepEp,τe∗=-keoeoEp=e1e2e3T and eo=e4 being the position and orientation errors previously defined in (35). Equation (44) must satisfy -kepEp≤f¯e∗ to fulfill the stability analysis of Section 6.


## 8. HITL Simulation

The simulation environment has been developed inMatlab©, where the human operator’s view simulates an onboard camera. The aim of the simulation was to reach eight successive target points (see Figure 7). The calculations of the whole teleoperation system (including RUAVs dynamics) and its connection with the master devices and Matlab© are accomplished by a program made in C++.Figure 7
Human operator visual feedback of the different target points in the remote environment.The teleoperation system’s parameters are the following:αmC1,C2,C3,C4=0,5,20,70Ns/m (obtained using Matlab© LMI toolbox for each of the four different cases of time delay as seen in Table 1), αs=0.65Ns2/m, αsr⁡=0.1Nms2, km=100N/m, ks=10Ns/m, ksr⁡=0.7Nms, knx,y,z=200,100,401/s, knr=0.751/s, k1,2,3,4,5=10,5,2,1/π,0.1, kvx,y,z=10,3.75,1.5, kepx,y,z=20,10,4, and keo=0.5. The 4DOF RUAV has mass m = 7.4 [kg] and rotational inertia ir=0.28kgm2, and the 6DOF RUAV has the minihelicopter’s nominal parameters (which refer to MIT’s X-Cell .60).Table 1
Average values for time to complete the task (TCT) and mean square translational synchronization error (MSE) for different time delays.
Case
Time delays [s]
TCT [s]
MSE
x
[m/s]
MSE
y [m/s]
MSE
z [m/s]
1
h
1
=
h
2
=
0
77.86
3.93
1.4
0.13
2
h
1
=
0.1
+
0.1
sin
⁡
(
1.26
t
)
80.69
6.86
1.98
0.15
h
2
=
0.1
+
0.1
sin
⁡
(
0.63
t
)
3
h
1
=
0.2
+
0.1
sin
⁡
(
0.63
t
)
100.14
7.41
2.43
0.14
h
2
=
0.3
+
0.2
sin
⁡
(
1.26
t
)
4
h
1
=
0.5
+
0.4
sin
⁡
(
1.26
t
)
129.93
6.87
2.4
0.16
h
2
=
0.4
+
0.2
sin
⁡
(
0.63
t
)


### 8.1. Simulation Results

In Table1 the time to complete the task (TCT) and mean square translational synchronization error (MSE) for four different time delays are presented. The values correspond to the average of five simulations for each case of time delay. When time delay increases, the TCT gets bigger, as expected. The behaviour of the MSE is much more complicated: when time delays are zero (case 1), all the MSE is due to the dynamics of the RUAV models and the coupling controller, specially the impedance controller because the obstacle avoidance directly modifies the flight commands; when time delay increases (cases 2, 3, and 4), the MSE gets bigger as the synchronism gets worse; however, when the local controller damping αm rises considerably because of time delay (αm=70 for case 4 so that condition (16) is satisfied), the movement of the master device is slowed down causing the MSE to decrease. The magnitude difference between MSEx, MSEy, and MSEz is promoted by the different velocity mappings kni, and the invariance in MSEz appears as a result of better low level PID control (improved immunity to desynchronization) and the fact that there is only one obstacle that seriously affects the vertical axis (see Figure 8).Figure 8
3D (a) and 2D (b) views of the 6DOF RUAV trajectory for a simulation with case 4 time delays.
(a)
(b)Figures8 and 9 present some results obtained in a case 4 HITL simulation (h¯1=0.9s and h¯2=0.6s). Figure 8 shows 3D and 2D views of the trajectory made by the minihelicopter in the simulation task, every 5 seconds the 6DOF RUAV is drawn. In Figure 9, it can be observed that all the teleoperation signals are bounded, and when the environment exogenous force decreases, the synchronization error and force feedback diminish too. It is necessary to highlight the complexity of the performed task, mainly due to the big asymmetric time-varying delay but also by the number of obstacles and the speed of the minihelicopter in the simulation.Figure 9
Teleoperation system signals, environment exogenous force, and human operator haptic feedback for a simulation with case 4 time delays.


## 8.1. Simulation Results

In Table1 the time to complete the task (TCT) and mean square translational synchronization error (MSE) for four different time delays are presented. The values correspond to the average of five simulations for each case of time delay. When time delay increases, the TCT gets bigger, as expected. The behaviour of the MSE is much more complicated: when time delays are zero (case 1), all the MSE is due to the dynamics of the RUAV models and the coupling controller, specially the impedance controller because the obstacle avoidance directly modifies the flight commands; when time delay increases (cases 2, 3, and 4), the MSE gets bigger as the synchronism gets worse; however, when the local controller damping αm rises considerably because of time delay (αm=70 for case 4 so that condition (16) is satisfied), the movement of the master device is slowed down causing the MSE to decrease. The magnitude difference between MSEx, MSEy, and MSEz is promoted by the different velocity mappings kni, and the invariance in MSEz appears as a result of better low level PID control (improved immunity to desynchronization) and the fact that there is only one obstacle that seriously affects the vertical axis (see Figure 8).Figure 8
3D (a) and 2D (b) views of the 6DOF RUAV trajectory for a simulation with case 4 time delays.
(a)
(b)Figures8 and 9 present some results obtained in a case 4 HITL simulation (h¯1=0.9s and h¯2=0.6s). Figure 8 shows 3D and 2D views of the trajectory made by the minihelicopter in the simulation task, every 5 seconds the 6DOF RUAV is drawn. In Figure 9, it can be observed that all the teleoperation signals are bounded, and when the environment exogenous force decreases, the synchronization error and force feedback diminish too. It is necessary to highlight the complexity of the performed task, mainly due to the big asymmetric time-varying delay but also by the number of obstacles and the speed of the minihelicopter in the simulation.Figure 9
Teleoperation system signals, environment exogenous force, and human operator haptic feedback for a simulation with case 4 time delays.


## 9. Conclusions

In this paper, a bilateral teleoperation scheme for a RUAV has been proposed. The complete teleoperation system incorporates P+d controllers, a trajectory tracking control, an impedance control with obstacle avoidance, an elastic coupling controller, and a cascade PID control architecture. The stability of the system has been analyzed and delay-dependent stability criteria have been obtained under specific LMI conditions. The teleoperation scheme has performed well in the presence of different time-varying delays as shown in the HITL simulation results. The inclusion of simple controllers in the teleoperation system makes their parameters easy to set, allowing accurately calibrating how the human operator perceives the environment and dynamics of the RUAV through force feedback.



---

*Source: 305314-2015-08-11.xml*