# Optimal Control Techniques for Sampled-Data Control Systems with Medical Applications

Bernard Bonnard, Toufik Bakir, Piernicola Bettiol, Loïc Bourdin, Jérémy

Rouot, Sandrine Gayrard

► To cite this version:

Bernard Bonnard, Toufik Bakir, Piernicola Bettiol, Loïc Bourdin, Jérémy Rouot, et al. Optimal Control Techniques for Sampled-Data Control Systems with Medical Applications. PGMO Days 2020, Dec 2020, Paris, France. hal-03021332

HAL Id: hal-03021332

https://inria.hal.science/hal-03021332

Submitted on 24 Nov 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Optimal Control Techniques for Sampled-Data Control Systems with Medical Applications

**PI:** Bernard Bonnard (UBFC, INRIA)

**Participants:** T. Bakir, P. Bettiol, L. Bourdin, J. Rouot, S. Gayrard<fnref n="1" />

PGMO Days December 2020

<footnote n="1">1. Thèse CIFRE : The Mathematical Frame and the Application to functional Muscular Control, 2020–</footnote>
**Control system** of the form

$$ \dot{x}(t) = f(x(t), u(t)), \quad x(0) = x_0, \quad x(T) = x_T $$

and an optimal control of the Mayer type

$$ \min_{u(\cdot)} \varphi(x(T)) $$

with $\varphi: \mathbb{R}^n \to \mathbb{R}$.

* **Permanent Control**: $u: [0, T] \mapsto U$, $u \in \mathcal{U}$ where $\mathcal{U}$ are the absolutely continuous maps valued in $U$.

* **Sampled Data Case**: $u \in \mathcal{U}_{sampled}$: fix $n \in \mathbb{N}$,

* $n$ sampling times

$$ 0 < t_1 < \dots < t_n < T. $$

* $(n+1)$-amplitudes

$$ \eta = (\eta_0, \dots, \eta_n) \in [0, 1]^{n+1} $$

The control is constant over $[t_i, t_{i+1}]$.
In the **permanent case**, the optimal control can be computed using

* Direct scheme : the problem is transformed into a finite dimensional optimization problem using

* discretization scheme for the dynamics

* discretization scheme for the control

* Indirect scheme : the problem can be analyzed using Pontryagin Maximum Principle which leads to Hamiltonian equations

$$ \begin{gathered} \dot{x} = \frac{\partial H}{\partial p}, \quad \dot{p} = -\frac{\partial H}{\partial x} \\ H(x, p, u) = \max_{v \in U} H(x, p, v) \end{gathered} $$

with $H(x, p, v) = p \cdot f(x, v)$, $p$ being the adjoint vector satisfying

$$ p(T) = p_0 \frac{\partial \varphi}{\partial x}(x(T)) \quad (\text{transversality condition}). $$

This necessary optimality condition can be handled using a *shooting method*
Shooting method

<figure><img src="image_1.png" /></figure>
Force-Fatigue muscular model<fnref n="2" />

FES input i. Dirac impulses $\delta$ at times $t = 0, t_1, t_2, \dots, t_N$.

$$i(t) = \sum_{i=0}^{N} R_i \eta_i \delta(t - t_i), \quad \eta_i \in [0, 1]$$

where

$$R_i := \begin{cases} 1, & \text{for } i = 0, \\ 1 + (\bar{R} - 1) \exp\left(-\frac{t_i - t_{i-1}}{\tau_c}\right), & \text{for } i = 1, \dots, N, \end{cases}$$

takes into account the tetanic contraction.

FES signal $E_s$.

$$E_s(t) = \frac{1}{\tau_c} \sum_{i=0}^{N} R_i \eta_i H(t - t_i) \exp\left(-\frac{t - t_i}{\tau_c}\right)$$

H : Heaviside

<footnote n="2">2. based on the Ding et al. / Hill-Huxley model</footnote>
The FES signal drives the evolution of the dynamics :

$$ \dot{C}_N(t) = -\frac{C_N(t)}{\tau_c} + E_s(t; t_i, \eta_i), $$

$$ \dot{F}(t) = -F(t) \boldsymbol{\gamma}(t) + A \boldsymbol{\beta}(t). $$

where the Hill functions are given by

$$ \boldsymbol{\beta}(t) := \frac{C_N(t)}{K_m + C_N(t)}, \quad \text{and} \quad \boldsymbol{\gamma}(t) := \frac{1}{\tau_1 + \tau_2 \boldsymbol{\beta}(t)}. $$

$(A, K_m, \tau_1, \tau_2)$ are the fatigue parameters.
The problem fits in the sampled data control frame with :

◦ $u_0 = \eta_0 e^{-t/\tau_c} / \tau_c$ on $[0, T]$

◦ $u_1 = u_0(t_1) + \eta_1 R_1 e^{-(t-t_1)/\tau_c} / \tau_c$ on $[t_1, T]$

∴

Note that in this form each control splits into

◦ *Head*: restricting to $[t_i, t_{i+1}]$

◦ *Tail*: restricting to $[t_i, T]$

**Optimal control problems considered.**

◦ (**OCP1**) $\max_{t_i, \eta_i} F(T)$

◦ (**OCP2**) $\min_{t_i, \eta_i} \int_0^T |F(t) - F_{ref}|^2 dt$ ($F_{ref}$ : reference force).
# Main theoretical results

<figure><img src="image_2.png" /></figure>

$$c_N(T) = \frac{1}{\tau_c} \sum_{i=0}^{n} e^{-(t-t_i)/\tau_c} (T-t_i) R_i$$

$F(T)$ is a piecewise $C^\infty$ mapping with respect to $\eta_i, t_i$ and the problem fits in a finite dimensional optimization problem.

* The non-fatigue model for a sequence of train, fatigue parameters $P$ satisfy a dynamics of the form

$$\dot{P}(t) = \frac{P(t) - P_{rest}}{\tau_p} + \alpha_p F(t).$$

* First order necessary optimality conditions can be obtained adapting [5] and are described in [2]. They were numerically implemented and compared with a direct optimization scheme in [3].
Numerical results using direct method

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>
**Theorem (see [2])**

If $(\eta_0^*, \eta_1^*, \dots, \eta_N^*, t_1^*, \dots, t_N^*)$ is optimal, then there exists $p$ satisfying the co-state equation and the transversality condition.
Moreover, the necessary conditions are:

(i) the inequality

$$ \left( \int_{t_i^*}^T p_1(s) b(s) \, ds \right) \tilde{\eta}_i \le 0, $$

for all $i = 0, \dots, n$ and all admissible perturbation $\tilde{\eta}_i$ of $\eta_i^*$;

(ii) and the inequality

$$ \mathbf{NC}_i := \left( -p_1(t_i^*) b(t_i^*) G(t_{i-1}^*, t_i^*) \eta_i^* + b(-t_i^*) \eta_i^* \int_{t_i^*}^T p_1(s) b(s) \, ds \right. \\ \left. \qquad + b(-t_i^*) (\bar{R}-1) \eta_{i+1}^* \int_{t_{i+1}^*}^T p_1(s) b(s) \, ds \right) \tilde{t}_i \le 0, $$

for all $i = 1, \dots, n$ and all admissible perturbation $\tilde{t}_i$ of $t_i^*$.
* We need efficient algorithms (real-time application) :

* Explicit expression of
    $(t_1, \dots, t_n, \eta_0, \dots, \eta_n) \to F(T)$

* to apply a direct optimization scheme

* LQ methods

* MPC methods

* Online parameter estimation coupled with optimization methods [1]

* Extension to the non-isometric case with joint angle variable to produce a motion [4]
[1] T. Bakir, B. Bonnard B. and J. Rouot, A case study of optimal input-output system with sampled-data control : Ding et al. force and fatigue muscular control model. *Netw. Hetero. Media*, **14** no.1 (2019), 79–100

[2] T. Bakir, B. Bonnard B., L. Bourdin and J. Rouot, Pontryagin-type conditions for optimal muscular force response to functional electrical stimulations. *J. Optim. Theory Appl.*, **184** (2020), 581–602.

[3] T. Bakir, B. Bonnard B., L. Bourdin and J. Rouot, Direct and Indirect Methods to Optimize the Muscular Force Response to a Pulse Train of Electrical Stimulation, preprint, hal-02053566.

[4] B. Bonnard B. and J. Rouot, Geometric optimal techniques to control the muscular force response to functional electrical stimulation using a non-isometric force-fatigue model. Accepted in *Journal of Geometric Mechanics* (2020)

[5] L. Bourdin and E. Trélat, Optimal sampled-data control, and generalizations on time scales, *Math. Cont. Related Fields*, **6** (2016), 53–94