# AoI-based switching control for safe haptic teleoperation over a wireless network

Yu Yeh<fnref n="1" />, Vineeth S. Varma<fnref n="2" />, Salah. E. Elayoubi<fnref n="1" />

**Abstract**— In this paper, we propose a switching policy based on the Age of Information (AoI) to enhance safety in haptic teleoperation over a wireless network, which is a key application of interest in upcoming 6G networks. Given the unique challenges of human-in-the-loop scenarios, our focus is on safety-oriented control since imperfect transmissions could introduce safety risks in typical haptic teleoperation scenarios. We introduce an AoI-based Markov Jump Linear System (MJLS) to model the closed-loop system, considering the AoI-based safety switching policy. This policy activates a “safe” controller on the remote side when the AoI exceeds a predefined threshold. While the safe controller does not know the desired human control, it is able to bring the robot or plant to a safe state. For a given AoI-based threshold, we utilize MJLS properties to ensure closed-loop stability. We also introduce and optimize performance criteria that address safety issues as well as transparency for the human controller, i.e., how closely the system can follow the human’s intention. Numerical results demonstrate that our safe control scheme improves performance in 2D reference tracking tasks while ensuring safety and reducing undesirable behavior like overshoots.

## I. INTRODUCTION

Recent technological advancements have enabled the practical realization of haptic teleoperation through tactile networks. Numerous studies affirm that integrating haptic feedback into human-involved tasks can enhance performance and safety [1]. However, challenges in haptic teleoperation arise due to imperfect transmissions, which can distort feedback and user commands, potentially destabilizing the closed-loop system [2]. According to [3], various model-based methods have been proposed to address stability and transparency in bilateral teleoperation, including the Time Domain Passivity Approach (TDPA), which aims to maintain passivity in the closed loop [4]. Inspired by these approaches, we aim to enhance the system modeling by introducing a network model, which has not been considered in previous works.

On the other hand, haptic teleoperation can be seen as a sub-domain of the general framework of networked control systems (NCS). According to [5], NCS research tackles two complementary problems. (i) *Control of network*: configure the communication and network settings to optimize their

compatibility with NCSs and (ii) *Control over network*: emphasizing control strategies across the network to mitigate the impact of unfavorable network conditions, like delays, on the performance of NCSs. As a main specificity of haptic teleoperation is that humans play the role of the controller, we focus on the control as an assistance to the human and model the impact of the network on the quality of the task. A similar idea could be found in haptic shared control [6], which proposes that the human experiences assisting forces on the control interface to improve performance, such as safety.

Safe control is essential in teleoperation, especially when the scenario involves critical applications such as telesurgery [7]. [8] presented an example of safe control in haptic teleoperation, aiming to provide haptic feedback for guiding the user in manipulating a virtual UAV under safety constraints. However, this approach is unsuitable for network-based teleoperation due to potential distortion in commands and haptic feedback from imperfect transmission. As a result, we address safety by placing a safe controller near the plant that switches to a local control when needed. Such a safety-driven switching policy is also proposed in [9]. Inspired by [10], our work diverges by adopting an MJLS framework and considering the Age of Information (AoI) to ensure stable conditions.

### A. Contribution

This paper proposes a switching policy between human manual control and an automatic safe controller in haptic teleoperation. We aim to find an optimal AoI threshold to decide when the safe controller intervenes in the control of the teleoperator, as it has no knowledge of the human model and the user's intention. We fulfill the target as follows:

* We consider a scenario where a remote plant alternates between human control and a safe controller accounting for AoI in the bilateral transmission. This approach aims to enhance the closed-loop dynamics for safe control while maintaining system stability.

* By modeling the stochastic process of the transmission status as a Markov chain, we employ the framework of MJLS to provide conditions for the stability of the closed-loop system under the switching policy.

* We provide a simulation-based method to decide the optimal AoI threshold for safe switching. The numerical results demonstrate that the selected AoI threshold reaches the optimal performance.

<footnote>*This work is supported by TOAST funded by European Union's Horizon Europe research and innovation program (HORIZON-MSCA-2022-DN-01) under the Marie Skłodowska-Curie grant agreement No. 101073465.</footnote>

<footnote n="1">¹Yu Yeh, and S. E. Elayoubi are with Université Paris Saclay; CentraleSupélec; L2S, CNRS, Gif-Sur-Yvette, France yu.yeh@centralesupelec.fr, salaheddine.elayoubi@centralesupelec.fr</footnote>

<footnote n="2">²Vineeth S. Varma is with the Université de Lorraine, CNRS, CRAN, 54000 Nancy, France vineeth.satheeskumar-varma@univ-lorraine.fr</footnote>
## B. Notations

The subscripts *f* and *b* denote forward and backward communication directions. We use [·, ·] to represent an element in a matrix. The main notations used in this work are presented in Table I.

<table><caption>TABLE I
DEFINITIONS OF NOTATIONS</caption><thead><tr><th>Notation</th><th>Definition</th></tr></thead><tbody><tr><td>x<sub>c</sub>, u<sub>l</sub>, y<sub>l</sub></td><td>The states, input, and output of the local controller.</td></tr><tr><td>x<sub>p</sub>, u<sub>r</sub>, y<sub>r</sub></td><td>The states, input, and output of the remote plant.</td></tr><tr><td>h</td><td>Human's desired haptic reference</td></tr><tr><td>ζ</td><td>Control mode</td></tr><tr><td>τ, θ</td><td>Environment state variable; Markov chain state variable</td></tr><tr><td>σ, Δ</td><td>Transmission status; AoI</td></tr></tbody></table>

## II. PROBLEM STATEMENT

### A. Networked haptic teleoperation system

In our haptic teleoperation system, a human and a haptic user interface (HUI) are controllers on the local side that manipulate a plant on the remote side through a wireless channel. A safe controller is located on the remote side to impose a safe behavior, as shown in Figure 1.

We consider the remote plant as a discrete-time linear system. At each time index $t \in \mathbb{Z}_{\ge 0}$, the control mode, denoted as $\zeta(t) \in \{m \mid m = 0, \dots, N_{\zeta} - 1\}$, indicates the current control mode of the plant. In this work, we consider binary cases with $N_{\zeta} = 2$, representing either the human control mode ($m = 0$) or the safe control mode ($m = 1$). The system model is as follows:

$$ 
\begin{aligned}
x_p(t+1) &= A_p^{\zeta(t)} x_p(t) + B_p^{\zeta(t)} u_r(t) \\
y_r(t) &= C_p^{\zeta(t)} x_p(t)
\end{aligned} 
\quad (1) 
$$

where $A_p^{\zeta(t)}, B_p^{\zeta(t)}, C_p^{\zeta(t)}$ are the matrices for state-space representation of the plant, whose values depend on the control mode. The state of the plant is $x_p(t) \in \mathbb{R}^{N_p}$, the command applied to the remote plant is $u_r(t) \in \mathbb{R}^1$, and the haptic feedback received from the plant on the remote side is $y_r(t) \in \mathbb{R}^1$. In this work, the design of the safe control aims to control some of the plant states $x_p(t)$ to zero, such as velocity.

For the local controller, we adopt the following discrete-time linear system model.

$$ 
\begin{aligned}
x_c(t+1) &= A_c^{\zeta(t)} x_c(t) + B_c^{\zeta(t)}(h(t) - y_l(t)) \\
u_l(t) &= C_c^{\zeta(t)} x_c(t) + D_c^{\zeta(t)}(h(t) - y_l(t)),
\end{aligned} 
\quad (2) 
$$

where $A_c^{\zeta(t)}, B_c^{\zeta(t)}, C_c^{\zeta(t)}$ are the matrices for state-space representation of the controller, which includes the dynamic of human motor skill and haptic operator, similar to what is proposed in [11]. Next, $x_c(t) \in \mathbb{R}^{N_c}$ is the state, and $h(t)$ is the human's desired haptic reference, which is only known to the user. In this work, we assume that the $h(t)$ is a piece-wise constant value as it reflects that the operators' sampling rate

<figure><img src="image_1.png" /><figcaption>Fig. 1. Block diagram of the haptic teleoperation system over the network with the safe controller. The human operator is on the local side, while the environment, teleoperator, and safe controller are on the remote side, with the wireless channel in between.</figcaption></figure>

is usually higher than the human reaction time for changing their desired reference. Similar to (1), the dynamic matrix of the controller depends on $\zeta(t)$ reflecting the current control mode.

The commands and haptic feedback information are sent wirelessly to the opposite side, with potential transmission failures in the direction from controller to plant (forward, denoted as *f*) and the direction from plant to controller (backward, denoted as *b*), as shown in Figure 1. To address the packet loss, a common strategy is to hold the last sample (HLS), which utilizes the most recently successfully received data to compensate for any missing commands or haptic feedback. Based on the situation mentioned above, the relationship between commands and feedback information in both transmission directions is defined as

$$ 
\begin{aligned}
u_r(t) &= \begin{cases} u_l(t), & \text{if } \sigma_f(t) = 1 \\ u_r(t-1), & \text{if } \sigma_f(t) = 0 \end{cases} \\
y_l(t) &= \begin{cases} y_r(t), & \text{if } \sigma_b(t) = 1 \\ y_l(t-1), & \text{if } \sigma_b(t) = 0 \end{cases}
\end{aligned} 
\quad (3) 
$$

where $\sigma_f, \sigma_b \in \{0, 1\}$ stand for the transmission status at time $t$, as subscripts 1 and 0 refer to successful and failed transmissions in the forward and backward transmission directions, respectively. The stochastic process describing the network status can be represented by the transition matrices $\Pi_{\sigma_f}$ and $\Pi_{\sigma_b}$, given by

$$ \Pr(\sigma_k(t) = i \mid \sigma_k(t-1) = j) = \Pi_{\sigma_k}(i, j), $$

where

$$ \Pi_{\sigma_k} = \begin{bmatrix} p_k^0 & 1 - p_k^1 \\ 1 - p_k^0 & p_k^1 \end{bmatrix} \quad (4) $$

for $i, j \in \{0, 1\}$, $k \in \{f, b\}$, and $p_k^1, p_k^0 \in [0, 1] \subset \mathbb{R}$ are the probabilities of remaining in the same transmission status.

## B. MJLS

In this section, we introduce the modeling of the networked haptic teleoperation system as an MJLS. Considering the variables mentioned above, we define an environment state variable $\tau(t) := (\sigma_f(t), \sigma_b(t), \zeta(t))$ enabling us to consider the bilateral transmission together, then we concatenate the state variables to get the augmented state $z(t) :=$
$[x_p(t)^T x_c(t)^T u_r(t-1) y_l(t-1)]^T$. The closed loop system is defined as a linear time-varying $\mathcal{G}$

$$ \mathcal{G}: z(t+1) = \mathcal{A}_{\tau(t)}z(t) + \mathcal{B}_{\tau(t)}h(t) \quad (5) $$

whose dynamic depends on the state variable and control model at time $t$, is given by

$$ \begin{aligned} A_{(\tau(t)=(1,1,0))} &:= \begin{bmatrix} A_p^0 - B_p^0 D_c^0 C_p^0 & B_p^0 C_c^0 & 0 & 0 \\ -B_c^0 C_p^0 & A_c^0 & 0 & 0 \\ -D_c^0 C_p^0 & C_c^0 & 0 & 0 \\ C_p^0 & 0 & 0 & 0 \end{bmatrix} \\ A_{(\tau(t)=(1,0,m))} &:= \begin{bmatrix} A_p^m & B_p^m C_c^m & 0 & -B_p^m D_c^m \\ 0 & A_c^m & 0 & -B_c^m \\ 0 & C_c^m & 0 & -D_c^m \\ 0 & 0 & 0 & 1 \end{bmatrix} \\ A_{(\tau(t)=(0,1,m))} &:= \begin{bmatrix} A_p^m & 0 & B_p^m & 0 \\ -B_c^m C_p^m & A_c^m & 0 & 0 \\ 0 & 0 & 1 & 0 \\ C_p^m & 0 & 0 & 0 \end{bmatrix} \\ A_{(\tau(t)=(0,0,m))} &:= \begin{bmatrix} A_p^m & 0 & B_p^m & 0 \\ 0 & A_c^m & 0 & -B_c^m \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \end{aligned} \quad (6) $$

for any $m \in \{0, 1\}$. Also, $B_{\tau(t)}$ is defined as

$$ \begin{aligned} B_{(\tau(t)=(1,1,0))} &:= [ B_p^0 D_c^0 & B_c^0 & D_c^0 & 0 ]^T \\ B_{(\tau(t)=(1,0,m))} &:= [ B_p^m D_c^m & B_c^m & D_c^m & 0 ]^T \\ B_{(\tau(t)=(0,1,m))} = B_{(\tau(t)=(0,0,m))} &:= [ 0 & B_c^m & 0 & 0 ]^T \end{aligned} \quad (7) $$

for any $m \in \{0, 1\}$. Based on the matrix definition in (6) and (7), we derive the transition matrix of the transmission status variable as $\Pi_{\tau} = \Pi_{\sigma_f} \otimes \Pi_{\sigma_b}$ to have the MJLS based on the transmission status. Before introducing the switch policy for $\zeta(t)$ in the next section, we consider that $\zeta(t) = 0$ for pure human control. This will also be a baseline for comparison in future sections.

### C. AoI and switching policy

Except for the ideal transmission case $\tau(t) = (1, 1, 0)$ in (6) and (7), the switching policy, aiming for managing the control mode expressed by $\zeta(t) = 1$ or $\zeta(t) = 0$ at each time $t$, needs to be decided. When there is a burst of packet losses on the wireless network, the HLS technique makes the plant blindly apply the last successfully transmitted command, leading to possible undesired plant behavior. This safety concern scenario motivates us to propose the control policy depending on the Age of Information (AoI) in the bilateral transmission as denoted as $\Delta_f$ and $\Delta_b$ (which we assume is known at the remote end via an acknowledgment protocol). These variables reflect the information freshness of the data packet at each receiver node based on the transmission status and evolve according to:

$$ \Delta_k(t) = \begin{cases} 1, & \text{if } \sigma_k(t) = 1 \\ \Delta_k(t-1) + 1, & \text{if } \sigma_k(t) = 0 \end{cases} \quad (8) $$

for $k \in \{f, b\}$. The stochastic process governing $\Delta_{k=\{f,b\}}$ is modeled as a Markov chain. The target is to find the AoI thresholds for switching to safe control mode in both transmission directions, denoted by $\bar{\Delta}_f$ and $\bar{\Delta}_b$. The switching policy is

$$ \zeta(t) = \begin{cases} 1, & \text{if } \Delta_f > \bar{\Delta}_f \text{ or } \Delta_b > \bar{\Delta}_b \\ 0, & \text{otherwise} \end{cases} \quad (9) $$

The objective is that the resulting system is mean-square stable (MSS), as defined below.

**Definition 1.** Consider a stochastic process $x(t)$, mean-square stability refers to the convergence of $\mathbb{E}[x(t)^T x(t)]$, i.e., $\mathbb{E}[x(t)^T x(t)] < \epsilon$ when $t \to \infty$, where $\epsilon$ is a constant.

In imperfect transmission scenarios, human control performance typically declines, resulting in higher associated costs than in ideal transmission conditions, and safe control with a proper switching policy should improve performance. In Section IV, we show how to decide the evaluation function to evaluate the performance and safety cost in a reference tracking task.

## III. MAIN RESULT

### A. AoI-based MJLS

Based on the AoI-based switching policy in (9), we need to extend the original MJLS to the AoI-based MJLS to proceed with further analysis. By defining the Markov chain state variable $\theta = (\min(\Delta_f, \bar{\Delta}_f), \min(\Delta_b, \bar{\Delta}_b))$, where $\bar{\Delta}_k$ for $k \in \{f, b\}$ is the AoI threshold, we can define the AoI-based MJLS $\mathcal{G}_{\text{AoI}}$ as follows:

$$ \mathcal{G}_{\text{AoI}}: z(t+1) = \mathcal{A}_{\theta(t)}z(t) + \mathcal{B}_{\theta(t)}h(t) \quad (10) $$

As such AoI-based MJLS is an extension of the system in (5), the mapping relationship of the system matrices $(\mathcal{A}_{\theta(t)}, \mathcal{B}_{\theta(t)}) = (\mathcal{A}_{\tau(t)}, \mathcal{B}_{\tau(t)})$ is established by defining the mapping function $\tau = f(\theta)$ as

$$ f(\theta) = \begin{cases} (1, 1, 0), & \text{if } \theta = (1, 1) \\ (1, 0, 0), & \text{if } \theta = (1, 1 < \Delta_b < \bar{\Delta}_b) \\ (0, 1, 0), & \text{if } \theta = (1 < \Delta_f < \bar{\Delta}_f, 1) \\ (0, 0, 0), & \text{if } \theta = \begin{cases} (1 < \Delta_f < \bar{\Delta}_f, 1) \\ 1 < \Delta_b < \bar{\Delta}_b \end{cases} \\ (1, 0, 1), & \text{if } \theta = (1, \bar{\Delta}_b) \\ (0, 1, 1), & \text{if } \theta = (\bar{\Delta}_f, 1) \\ (0, 0, 1), & \text{if } \theta = (\bar{\Delta}_f, \bar{\Delta}_b) \end{cases} \quad (11) $$

Let us denote the corresponding transition matrix of the above MJLS by $\Pi_{\theta}$, represented by

$$ \mathbb{P}(\theta(t+1) = m | \theta(t) = n) = \Pi_{\theta}[m, n]. \quad (12) $$

which is computed by $\Pi_{\theta} = \Pi_{\Delta_f} \otimes \Pi_{\Delta_b}$, where $\Pi_{\Delta_f}, \Pi_{\Delta_b}$ are given by

$$ \Pi_{\Delta_k}[i, j] = \begin{bmatrix} p_k^1 & 1 - p_k^0 & \cdots & 1 - p_k^0 \\ 1 - p_k^1 & 0 & \cdots & 0 \\ 0 & p_k^0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & \cdots & p_k^0 & p_k^0 \end{bmatrix} \quad (13) $$
<figure><img src="image_2.png" /><figcaption>Fig. 2. The partial visualization of AoI-based MJLS. The nodes are categorized into either human manual control mode or safety control mode.</figcaption></figure>

for $i, j \in \{1, \dots, \bar{\Delta}_k\}, k \in \{f, b\}$. The AoI-based MJLS (10) is illustrated in Fig. 2. Note that the size of the AoI-based MJLS depends on the $\bar{\Delta}_k$.

## B. Stability

From the previous section, we see that each selection of $\bar{\Delta}_f$ and $\bar{\Delta}_b$ creates a distinct MJLS, resulting in the different closed-loop dynamics between the local controller and the remote plant. Our objective in this section is to analyze the stability of the closed-loop system in (10) with the given switching policy since an improper policy may lead to unstable and, thus, undesirable behavior.

To present the main condition for stability, we define the operator $\mathcal{T} := \sum_{i=1}^N \Pi(i, j) A_i A_i^T$ and its matrix form $\mathcal{T} = \Pi^T \otimes I_{N^2} \mathrm{diag}(A_1^T A_1, \dots, A_N^T A_N)$. Define $\lambda_1, \dots, \lambda_N$ as the eigenvalues of a matrix $A \in \mathbb{R}^{N \times N}$, and $\rho(A) = \max(|\lambda_1|, \dots, |\lambda_N|)$ its spectral radius.

**Theorem 1.** If $p_k^n \in (0, 1)$ for all $n \in \{0, 1\}, k \in \{f, b\}$, then the AoI-based MJLS (10) is MSS if $\rho(\mathcal{T}) = \rho(T) < 1$.

*Proof:* First, we observe that (10) is a non-homogeneous MJLS due to the external signal $h(t)$. To prove mean-square stability for this system, we use Theorem (3.33) in [12] about stability for non-homogeneous MJLS. To apply this theorem, we first consider the homogeneous MJLS in (10) with $h(t) = 0$

$$G_h: x(t+1) = A_{\theta(t)}x(t)$$

with $A_{\theta(t)} = (A_1, \dots, A_N) \in \mathbb{R}^{N \times N}$. According to Theorem (3.9) in [12], $G_h$ is MSS if $\rho(\mathcal{T}) = \rho(T) < 1$.

For the considered non-homogeneous MJLS, we notice that $h(t)$ is a wide sense stationary sequence, namely $\mathbb{E}[h(t)] = h_0$ and $E(h(k)h(k)^T) = H_0$ are both constants since in this work, we make the assumption that the human's desired haptic reference $h(t)$ is a piece-wise constant. We can also easily verify that $\theta$ is ergodic as we can always return to (1, 1) from any state and reach any state starting from it as long as $p_k^n \in (0, 1)$ for all $n \in \{0, 1\}, k \in \{f, b\}$.

Theorem (3.33) in [12] states that, if $h(t)$ is a wide sense stationary sequence and the Markov chain $\theta$ is ergodic, the MJLS is MSS if the alternative system without the disturbance $h(t)$ is MSS. Thus, if $\rho(\mathcal{T}) = \rho(T) < 1$, we have MSS for (10).

In conclusion, we can apply Theorem 1 to check the stability of AoI-based MJLS through $\rho(\mathcal{T}) = \rho(T) < 1$.

## C. Tuning Policy

To appropriately determine the AoI threshold ($\bar{\Delta}_k \in \{f, b\}$) for the switching policy, we calculate the expected value of the step response of the MJLS, denoted as $\bar{y}(t)$, for the following two cases:

(1) Imperfect transmission: $\zeta(t) = 0, p_f^1, p_b^1 \neq 1$

(2) AoI-based safe control: $\zeta(t)$ from (9), $p_f^1, p_b^1 \neq 1$.

By denoting $C_z$ as the observation vector of state $z(t)$ for the MJLS system in (10), we aim to compute $\bar{y}(t) = \mathbb{E}[C_z z(t)]$. We first define $\omega_{t,i} = \mathbb{E}[z(t)1\{\theta(t) = \theta^i\}]$ and note that $\mathbb{E}[z(t)] = \sum_{i=1}^{N_\theta} \mathbb{E}[z(t)1\{\theta(t) = \theta^i\}] = \sum_{i=1}^{N_\theta} \omega_{t,i}$, where $N_\theta$ is the number of states in AoI-MJLS. By defining $\omega_t = [\omega_{t,1}, \dots, \omega_{t,N_\theta}]$, we adapt the method in [13] to compute the expected values of the step response as

$$\begin{aligned} \omega_{t+1} &= \bar{A}\omega_t + \bar{B} \\ \mathbb{E}[z(t)] &= \bar{C}\omega_t \end{aligned} \quad (14)$$

with $w_0 = \vec{0}$, where $\bar{A}, \bar{B}, \bar{C}$ are defined as

$$\begin{aligned} \bar{A} &= \begin{bmatrix} \Pi_{\theta}[1, 1]A_1 & \cdots & \Pi_{\theta}[1, N_{\theta}]A_{N_{\theta}} \\ \vdots & \ddots & \vdots \\ \Pi_{\theta}[1, N_{\theta}]A_{N_{\theta}} & \cdots & \Pi_{\theta}[N_{\theta}, N_{\theta}]A_{N_{\theta}} \end{bmatrix} \\ \bar{B} &= \left[ \sum_{i=1}^{N_{\theta}} \Pi_{\theta}[1, i] B_i \pi_i \quad \dots \quad \sum_{i=1}^{N_{\theta}} \Pi_{\theta}[N_{\theta}, i] B_i \pi_i \right]^T \\ \bar{C} &= \overbrace{[C_z \quad C_z \dots C_z]}^{N_{\theta}} \end{aligned} \quad (15)$$

The tuning of the switching policy relies on an evaluation cost by the cost function $J$, which reflects the improvement of the proposed method. The details of the cost function are introduced in section IV. A similar scheme for tuning the controller for closed-loop dynamic through analyzing the step response can be seen in [14].

For simplicity in this study, we set the AoI threshold as the same for both transmission directions ($\bar{\Delta}_f = \bar{\Delta}_b = \bar{\Delta}_{opt}$). The tuning procedure is summarized in Algorithm 1.

<caption>**Algorithm 1** Linear search for optimal AoI threshold</caption>

**Initial:** $c_{min} \leftarrow \mathrm{Inf}, \bar{\Delta}_{opt} \leftarrow 1$
**for** $\Delta \in \{1, \dots, \bar{\Delta}\}$ **do**
  Compute $c \leftarrow J(\mathcal{Y}_{(1)}, \mathcal{Y}_{(2)})$
  **if** $c < J_{min}$ and $\rho(T) < 1$ **then**
    $c_{min} \leftarrow c$
    $\bar{\Delta}_{opt} \leftarrow \Delta$
  **end if**
  **end for**
**Output:** Optimal AoI threshold $\bar{\Delta}_{opt}$

# IV. NUMERICAL RESULTS

## A. Task 1: Tuning with step response

This section demonstrates how to decide the optimal AoI threshold toward a closed-loop step response. We consider
the haptic teleoperation system as the user with a one-dimensional operator on the local side and manipulate the virtual plant with teleoperation on the remote side. The plant model was assigned as a spring-damper system with an ideal teleoperation operator designed as (i) a position controller in human manually control mode, and (ii) a velocity controller in safe control mode. Those are given by

$$ 
\begin{aligned}
A_p^0 &= \begin{bmatrix} 0.9982 & 0.0009463 \\ -3.666 & 0.8926 \end{bmatrix} B_p^0 = \begin{bmatrix} 0.001758 \\ 3.516 \end{bmatrix} \\
A_p^1 &= \begin{bmatrix} 0.9999 & 0.0009658 \\ -0.1448 & 0.9317 \end{bmatrix} B_p^1 = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \\
C_p^0 &= C_p^1 = \begin{bmatrix} 15 & 0.05 \end{bmatrix} D_p^0 = D_p^1 = 0
\end{aligned} 
\quad (16) $$

with 1 ms sampling time. The controller models are given by

$$ 
\begin{aligned}
A_c^0 &= \begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ -0.005 & 0 & 1 & 0.0009 \\ -0.9063 & 0 & 0 & 0.8187 \end{bmatrix} B_c^0 = \begin{bmatrix} 1 \\ 0.01 \\ 0.008 \\ 1.5408 \end{bmatrix} \\
A_c^1 &= \begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ -0.005 & 0 & 1 & 0.0006 \\ -0.6321 & 0 & 0 & 0.3679 \end{bmatrix} B_c^1 = \begin{bmatrix} 1 \\ 0.01 \\ 0.006 \\ 1.0746 \end{bmatrix} \\
C_c^0 &= C_c^1 = \begin{bmatrix} 0 & 0 & 1 & 0 \end{bmatrix}
\end{aligned} 
\quad (17) $$

During human manual control ($\zeta(t) = 0$), the plant operator switches to the position controller to conduct the position command from the local user. It switches to the safe control mode ($\zeta(t) = 1$), adopting a velocity controller with a safe controller aiming to achieve zero velocity. The transmission environment is such that the probabilities of successful transmission of the forward and backward are $p_f^1 = p_b^1 = 0.93$ and $p_f^0 = p_b^0 = 0.91$.

To decide the optimal AoI threshold, we focus on two metrics: (i) The suppression of the magnitude's overshoot, and (ii) the speed of the response. The selection of the two metrics is based on the observation that, although the switching policy may improve the overshoot due to imperfect transmission, it may sacrifice the speed of response, resulting in a safe operation but with lower task performance. By denoting $N_s$ as the length of simulation, the trade-off between the two metrics could be converted into the following evaluation function:

$$ J(\bar{y}^{(1)}(t), \bar{y}^{(2)}(t)) = c_0 \left( \frac{\alpha(\bar{y}^{(1)}(t))}{\alpha(\bar{y}^{(2)}(t))} \right) + c_1 \left( \frac{\beta(\bar{y}^{(1)}(t))}{\beta(\bar{y}^{(2)}(t))} - 1 \right) \quad (18) $$

where $c_0, c_1$ are weighting coefficients, and
$$ 
\left\{ 
\begin{array}{l}
\alpha(\bar{y}(t)) = \max_{t \in \{0, \dots, N_s-1\}} \|\bar{y}(t)\|_2 : \mathbb{R}^{N_s} \to \mathbb{R}_{\ge 0} \\
\beta(\bar{y}(t)) = \min\{t \mid |\bar{y}(t) - 1| \le \varepsilon\} : \mathbb{R}^{N_s} \to \mathbb{R}_{\ge 0}
\end{array}
\right. 
$$

Specifically, $\beta$ is a function indicating the first time when the step response reaches the value equal to $1 \pm \varepsilon$, where $\varepsilon$ is a small constant to indicate a range. Figure 3 is an example of visualizing the expectation of step response in different cases. With the given candidates of AoI-threshold

<figure><img src="image_3.png" /><figcaption>Fig. 3. Comparison of expectation of step response from AoI-based MJLS in different cases. The smaller AoI threshold suppresses the magnitude overshoot but sacrifices the response speed.</figcaption></figure>

$\bar{\Delta}_f = \bar{\Delta}_b \in [1, 50]$, we conduct Algorithm 1 to explore the minimum tuning cost with stability check. The search result is shown in Figure 5 as indicating that the feasible choice is $\bar{\Delta}_f = \bar{\Delta}_b = 16$, and the stability check is ensured by $1 - \rho(T) < 10^{-14}$.

## B. Task 2: Application to reference tracking

In this subsection, we consider the task in which the user desires to manipulate a 2-axis platform in a virtual environment with a local operator (with the same degree of freedom) to trace a reference trajectory. The model of the local controller and remote plant in each axis inherits the same definition provided in (16) and (17), and the information among the 2-axis operators is sent through a single bilateral channel with the same parameters in IV-A. The human's desired haptic reference $h(t)$ is modeled as a stochastic process characterizing the human's behavior, specifically reflecting the intention for the haptic reference in a repetitive task, defined as

$$ h(t) = h_i \text{ with } \mathbb{P}(h(t) = h_i | h(t-1) = h_j) = \Pi_h[i, j] \quad (19) $$

for any $i, j \in \{1 \dots N_h\}$, where $\Pi_h$ is the transition matrix of $h(t)$ and $N_h$ is the number of states of human reference.

We inherit the turning result in the previous section and employ the same optimal AoI threshold for the reference task. Similar procedures for tuning the controller based on a simple response and evaluating it on a complex task could be found in [15]. For the evaluation, we consider a task-oriented cost function composed of tracking and safety costs. Tracking cost is related to tracking errors between the trajectory under ideal and non-ideal transmission conditions. About safety cost, we first define the set $X_p$ to encompass all reachable augmented states of the MJLS ($z(t)$), and the critical space $X_c(t) \subset X_p$ that the plant state should avoid accessing. As a result, the evaluation function is defined as follows:

$$ J = \mathbb{E} \left[ \sum_{t=0}^{N_s-1} w_0 \|C_z(z(t) - z^*(t))\|_2 + w_1 \mathbf{1}(z(t) \in X_c(t)) \right] \quad (20) $$

where $z^*(t)$ is the task response in the perfect transmission as ground truth since the human's desired reference could not be actually measured, $N_s$ is length of simulation time,
<figure><img src="image_4.png" /><figcaption>Fig. 4. Comparison of trajectories in different cases. The left illustrates one realization of possible outcomes with the same human's desired reference, and the right is an average trajectory over 1000 simulations.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 5. The evaluation cost v.s. the candidates of AoI threshold. The cost function coefficients are set as $c_0 = 7, c_1 = 1, w_0 = w_1 = 1$. Each evaluation cost value is the average result from the 500-times simulation. Note that the trend of evaluation cost fits the intuition that the higher AoI threshold performs tasks similarly to the same condition with inactive safety control mode.</figcaption></figure>

$C_z = [15 \ 0.05 \ 0\dots0]^6$ and $w_0, w_1 \in \mathbb{R}^+$ are the weighting coefficients. Here, we define the critical region $X_c(t)$ in (20) as the 20% overshoot of the trajectory $C_z z^*(t)$.

The comparison of trajectory results between the different cases is shown in Figure 4. After 1000 simulations, we could derive the average evaluation value in the cases without and with safety control, which are 7.0494 and 3.4862, respectively. This supports the idea that a proper switching policy could improve task performance in a non-ideal transmission environment. The same simulation result could be further visualized by comparing their maximum trajectory boundaries. It clearly shows that without the safety control mode, the trajectory may access the region far from the ideal workspace.

We also compute the evaluation cost versus AoI candidates with and without safe control, as shown in Figure 5. The curves for tasks 1 and 2 exhibit a similar trend, reaching their lowest value at the same optimal AoI threshold and gradually increasing afterward. This supports the idea that we can tune the AoI-based switching policy based on the step response for complex tasks.

## V. CONCLUSION

In this work, we proposed an AoI-based switching policy to provide safe control for haptic teleoperation over an imperfect network. We use an AoI-based MJLS to model a closed-loop system encompassing local and remote environments and provide conditions for closed-loop stability.

The main idea is to switch to a safe control mode on the remote side whenever the AoI exceeds a predefined threshold. For certain simple evaluation functions and tasks, we can find the optimal AoI threshold by computing the expectation step response. In a 2-axis reference tracking scenario, our numerical results demonstrate that the safety control scheme with a proper switching policy effectively improves performance. For future work, we will address the issue of tuning the transmission policy based on goal-oriented communication to improve the overall performance.

## REFERENCES

[1] S. Ullah, P. Richard, S. Otmane, M. Naud, and M. Mallem, “Haptic guides in cooperative virtual environments: Design and human performance evaluation,” in *2010 IEEE Haptics Symposium*. IEEE, 2010, pp. 457–462.

[2] S. Hirche and M. Buss, “Packet loss effects in passive telepresence systems,” in *2004 43rd IEEE Conference on Decision and Control (CDC)* (IEEE Cat. No. 04CH37601), vol. 4. IEEE, 2004, pp. 4010–4015.

[3] S. N. F. Nahri, S. Du, and B. J. Van Wyk, “A review on haptic bilateral teleoperation systems,” *Journal of Intelligent & Robotic Systems*, vol. 104, no. 1, p. 13, 2022.

[4] X. Xu, M. Panzirsch, Q. Liu, and E. Steinbach, “Integrating haptic data reduction with energy reflection-based passivity control for time-delayed teleoperation,” in *2020 IEEE Haptics Symposium (HAPTICS)*. IEEE, 2020, pp. 109–114.

[5] R. A. Gupta and M.-Y. Chow, “Networked control system: Overview and research trends,” *IEEE transactions on industrial electronics*, vol. 57, no. 7, pp. 2527–2535, 2009.

[6] D. A. Abbink, M. Mulder, and E. R. Boer, “Haptic shared control: smoothly shifting control authority?” *Cognition, Technology & Work*, vol. 14, pp. 19–28, 2012.

[7] A. M. Okamura, “Methods for haptic feedback in teleoperated robot-assisted surgery,” *Industrial Robot: An International Journal*, vol. 31, no. 6, pp. 499–508, 2004.

[8] D. Zhang, G. Yang, and R. P. Khurshid, “Haptic teleoperation of uavs through control barrier functions,” *IEEE transactions on haptics*, vol. 13, no. 1, pp. 109–115, 2020.

[9] M. Stefanovic and M. G. Safonov, “Safe adaptive switching control: Stability and convergence,” *IEEE Transactions on Automatic Control*, vol. 53, no. 9, pp. 2012–2021, 2008.

[10] V. S. Varma, R. Postoyan, I.-C. Morărescu, and J. Daafouz, “Stochastic maximum allowable transmission intervals for the stability of linear wireless networked control systems,” in *2017 IEEE 56th Annual Conference on Decision and Control (CDC)*. IEEE, 2017, pp. 6634–6639.

[11] T. Hulin, A. Albu-Schäffer, and G. Hirzinger, “Passivity and stability boundaries for haptic systems with time delay,” *IEEE Transactions on Control Systems Technology*, vol. 22, no. 4, pp. 1297–1309, 2013.

[12] O. L. V. Costa, M. D. Fragoso, and R. P. Marques, *Discrete-time Markov jump linear systems*. Springer Science & Business Media, 2006.

[13] D. J. Antunes and H. Qu, “Frequency-domain analysis of networked control systems modeled by markov jump linear systems,” *IEEE Transactions on Control of Network Systems*, vol. 8, no. 2, pp. 906–916, 2021.

[14] A. Nassirharand, N. Hoq, and H. Tzou, “Design of nonlinear pid controllers using system step response,” *Computer-aided design*, vol. 21, no. 4, pp. 232–238, 1989.

[15] O. Dahuni and J. Pedro, “Pid reference tracking control for nonlinear active vehicle suspension system,” in *International Conference on Industrial Application Engineering*, 2015, pp. 132–139.