# Singular arcs in optimal periodic control problems with scalar dynamics and integral input constraint

Thomas Guilmeau, Alain Rapaport

► To cite this version:

Thomas Guilmeau, Alain Rapaport. Singular arcs in optimal periodic control problems with scalar dynamics and integral input constraint. Journal of Optimization Theory and Applications, 2022, 195, pp.548-574. 10.1007/s10957-022-02095-y. hal-03740426

HAL Id: hal-03740426

https://hal.inrae.fr/hal-03740426

Submitted on 29 Jul 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Singular arcs in optimal periodic control problems with scalar dynamics and integral input constraint

Thomas Guilmeau† and Alain Rapaport*

†Université Paris-Saclay, Inria, CentraleSupélec, Centre de Vision Numérique. E-mail: thomas.guilmeau@inria.fr

*MISTEA, Univ. Montpellier, INRAE, Montpellier SupAgro, 2 pl. Viala, 34060 Montpellier, France. E-mail: alain.rapaport@inrae.fr

July 29, 2022

## Abstract

We revisit recent results about optimal periodic control for scalar dynamics with input integral constraint, under lack of convexity and concavity. We show that in this more general framework, the optimal solutions are bang-singular-bang and generalize the bang-bang solutions for the convex case and purely singular for the concave one. We introduce a non-local slope condition to characterize the singular arcs. The results are illustrated on a class of bioprocesses models.

**Keywords.** Optimal control, periodic control, integral input constraint, singular arc.

**2020 Mathematics Subject Classification.** 49J15, 49K15, 34C25, 49N20, 49J30.

## 1 Introduction

Periodic optimal control has received relatively few attention in the literature, apart the well-known $\pi$-criterion [8]. This later one consists in a linear-quadratic approximation about a steady state with constant control, and studying frequencies of a sinusoidal control that could improve the average performance index over a neighboring periodic solution. Extensions to other shapes of periodic controls have also been considered. However, the global optimal periodic optimal control has been very rarely investigated, apart from [17] for the characterization of the value function under quite strong assumptions. The two boundaries condition that periodic solutions have to satisfy might explain the difficulties in extending the usual approaches. Indeed, most of the existing works deal with local necessary conditions ([10, 14]), second order conditions ([9, 23, 15]) or approximations techniques ([13, 2, 6]).

Recently, a class of scalar dynamics with integral input constraint has been investigated [5], and it has been shown that under convexity and monotonicity assumptions, the global optimal periodic solution is bang-bang and therefore improves the averaged criterion over constant controls. For the concave case, it has been shown that constant controls remain the best ones. These results have been in particular motivated by bioprocesses applications, for which a kind of duality has been derived [4].

However, situations for which neither the convex nor the concave conditions are fulfilled have not been yet considered, which is the purpose of the present work. This allows to solve the problem of optimal periodic operations of bioprocesses with growth functions that are neither convex nor concave, which has been an open problem up to now.

The paper is organized as follows. In Section 2, the main results of [5] are recalled, and the setting of the present contribution is specified. Then, in Section 3, we propose and prove a geometric optimality necessary condition in terms of slopes, which is central to our approach. In Section 4, further results on the optimal trajectories under the eventual lack of monotonicity in our assumptions are proved. Section 5 gives then the complete synthesis of the optimal control. Finally, our results are illustrated on a bioprocess model in Section 6, which shows the quantitative benefits of having singular arcs.
## 2 Some preliminaries

Let us consider two functions $f, g: \mathbb{R} \to \mathbb{R}$ of class $C^1$ and the controlled dynamics

$$ \dot{x} = f(x) + ug(x) \quad (1) $$

where $u$ takes values in $[-1, 1]$. Let us assume the following hypotheses:

(H1) There exists $(a, b) \in \mathbb{R}^2$ with $a < b$ such that $g$ is positive on the interval $I := (a, b)$ with

$$ f(a) - g(a) = 0 \quad \text{and} \quad f(b) + g(b) = 0 $$

(H2) One has $f - g < 0$ and $f + g > 0$ on $I$.

One can straightforwardly check that the interval $I$ is invariant by (1) under Hypothesis H1. Hypothesis H2 implies the controllability of (1) on $I$. In the following, we shall consider solutions on the interval $I$ only. For convenience, we define the function

$$ \psi(x) := -\frac{f(x)}{g(x)}, \quad x \in I $$

Hypotheses H1-H2 imply $\psi(I) \subset [-1, 1]$ and then for any $\bar{x} \in I$, the control $\bar{u} := \psi(\bar{x})$, which allows the system to stay at steady-state $\bar{x}$, is admissible i.e. $\bar{u} \in [-1, 1]$. Let us stress that we do not impose $f$ and $g$ to be non null on the boundary of $I$. Therefore $\bar{\psi}(I)$ is not necessary $[-1, 1]$ and $\bar{\psi}$ is not necessarily non decreasing. Examples 1 and 2 below show that $\bar{\psi}$ can be non monotonic on $I$.

Let us fix $\bar{u} \in \psi(I)$ as a nominal constant control, and consider for $T > 0$ solutions $x(\cdot)$ in $I$ that are $T$-periodic with a $T$-periodic control $u$ satisfying the integral constraint

$$ \frac{1}{T} \int_{0}^{T} u(t) dt = \bar{u} \quad (2) $$

We denote by $\mathcal{U}_T$ the set of admissible controls, that is

$$ \mathcal{U}_T := \{u: [0, +\infty) \to [-1, 1] \text{ s.t. } u \text{ is meas., } T\text{-periodic and fulfills (2)}\} \quad (3) $$

Let us now consider a function $\ell: \mathbb{R} \to \mathbb{R}$ of class $C^1$ and associate the criterion

$$ J_T(u) := \min \left\{ \frac{1}{T} \int_0^T \ell(x_u(t)) dt; x_u \text{ is } T\text{-periodic} \right\} \quad (4) $$

to be minimized over controls $u \in \mathcal{U}_T$, where $x_u$ denotes the solutions of (1) in $I$ associated to $u$.

In the former work [5], it has been shown that convexity is playing an important role in the possibility of having $J_T(u)$ lower than the cost with constant control $J_T(\bar{u})$ under the following additional condition on the dynamics.

(H3) The function $\ell: I \to \mathbb{R}$ is increasing and the function $\gamma = \psi \circ \ell^{-1}$ is strictly convex increasing over $\ell(I)$.

Under H3, the exists an unique $\bar{x}$ in $I$ with $\bar{\psi}(\bar{x}) = \bar{u}$ and one has the following result.

**Proposition 1.** If H1 and H3 hold true, any non-constant $T$-periodic solution $x$ of (1) with $x(0) = \bar{x}$ and $u \in \mathcal{U}_T$ satisfies $J_T(u) < J_T(\bar{u})$.

In the opposite way, it has been shown in [5] that concavity prevents improving the cost $J_T(\bar{u})$ with non-constant controls, under the following condition on the dynamics:

(H4) There exists a continuous function $\bar{\psi}$ such that

(i) $\bar{\psi} \ge \psi$ on $I$ with $\bar{\psi}(\bar{x}) = \psi(\bar{x})$,

(ii) the function $\bar{\gamma} = \bar{\psi} \circ \ell^{-1}$ is concave increasing on $\ell(I)$.

Under H4 one has also the uniqueness of $\bar{x}$ in $I$ with $\bar{\psi}(\bar{x}) = \bar{u}$ and the following result holds.
**Proposition 2.** If H1 and H4 hold true, any non-constant T-periodic solution x of (1) with $x(0) = \bar{x}$ and $u \in U_T$ satisfies $J_T(u) > J_T(\bar{u})$.

In the present work, we assume that Hypotheses H1-H2 are satisfied and aim at relaxing Hypothesis (H3) or (H4) by allowing a change of convexity as well as a change of monotonicity of the function $\gamma := \psi \circ \ell^{-1}$ on the interval $I$, while keeping $\ell$ increasing. Note that under Hypotheses H1-H2, there does not necessarily exist an unique $\bar{x}$ such that $\psi(\bar{x}) = \bar{u}$. However, we shall assume that there is an unique stable one, which is guaranteed by the following hypothesis.

(H) The function $\ell$ is increasing on $I$, and for any $\check{u} \in \text{int}(\psi(I))$, there exists an unique $\check{x} \in I$, such that

$$ \begin{cases} \psi(\check{x}) = \check{u}, \\ \text{there exists a neighborhood } \mathcal{V} \subset I \text{ of } \check{x} \text{ such that } (\psi(x) - \psi(\check{x}))(x - \check{x}) > 0, \quad \forall x \in \mathcal{V} \setminus \{\check{x}\} \end{cases} $$

Then, as $\ell$ is increasing, the steady-state $\bar{x}$ that gives the best cost $J_T(\bar{u})$ is clearly the smallest one. Note that Hypothesis H amounts to $\psi$ having at most one change of monotonicity on $I$. Under Hypotheses H1-H2, if $\psi$ is non monotonic on $I$, it must be increasing first and then decreasing. Therefore, we can define values $\bar{x}$ and $\hat{x}$ as follows.

**Definition 1.** Under H1-H2-H, let

$$ \bar{x} := \min\{x \in I, \text{ s.t. } \psi(x) = \bar{u}\} $$

and

$$ \hat{x} := \arg \max_{x \in I} \psi(x) $$

**Remark 1.** Under H1-H2-H, one has necessarily $\bar{x} \le \hat{x}$, with $\bar{x}$ being the only value fulfilling H. Moreover, having $\psi$ non monotonic with at most one change of monotonicity implies that the function f and g are null at a or b.

Let us first give a preliminary result about periodic solutions, in the spirit of the former work [5].

**Lemma 1.** Under Hypotheses H1-H2-H, any T-periodic solution x of (1) in I with $u \in U_T$ fulfills the property

$$ \int_{0}^{T} (\psi(x(t)) - \bar{u}) \, dt = 0 \qquad (5) $$

and any optimal trajectory x takes the value $\bar{x}$.

*Proof.* On the interval $I$, the function $g$ is positive and from equation (1), one can write

$$ \int_{0}^{T} \frac{\dot{x}(t)}{g(x(t))} dt = - \int_{0}^{T} \psi(x(t)) dt + \int_{0}^{T} u(t) dt $$

Consider then the function $t \mapsto y(t) := h(x(t))$ for $t \in [0, T]$, where $h$ is defined as follows.

$$ h(x) := \int_{\bar{x}}^{x} \frac{d\xi}{g(\xi)}, \quad x \in I $$

For any control function $u$ that fulfills the constraint (2), one gets

$$ y(T) - y(0) = - \int_{0}^{T} (\psi(x(t)) - \bar{u}) \, dt $$

where $\bar{u} = \psi(\bar{x})$. Therefore, for any T-periodic solution $x$ in $I$, $y$ is also T-periodic and one obtains property (5).

According to the above, for any T-periodic solution $x$, the map $t \mapsto \psi(x(t)) - \bar{u}$ has to take the value 0 on $[0, T)$. Therefore, there exists $\bar{t} \in [0, T)$ such that $x(\bar{t}) = \bar{x}$ with $\psi(\bar{x}) = \bar{u}$. If $\bar{x} = \bar{x}$, then we have proved that $x$ takes the value $\bar{x}$. If not, one has necessarily $\bar{x} > \bar{x}$ because of Definition 1. Therefore, if the solution $x$ does not take the value $\bar{x}$, one should have $x(t) > \bar{x}$ for any $t \in [0, T)$. The function $\ell$ being increasing on $I$, it comes that

$$ \int_{0}^{T} \ell(x(t)) dt > \ell(\bar{x}) = J_{T}(\bar{u}) $$

which shows that $x$ cannot be optimal. We conclude that one has $\bar{x} = \bar{x}$.
This Lemma allows to look for optimal solutions with $x(0) = x(T) = \bar{x}$ without any loss generality, as we shall do now. Note that when $\bar{x} = \hat{x}$, the single $T$-periodic solution of (1) with the constraint (2) is the constant solution $x = \bar{x}$, and there is no optimization to be made. Therefore, we shall assume in the following that $\bar{u}$ is taken in $\text{int}(\psi(I))$, which implies $\bar{x} < \hat{x}$.

We introduce now the Hypotheses H5a and H5b that generalize Hypotheses H3 or H4, keeping $\ell$ increasing.

(H5a) The function $\ell: I \to \mathbb{R}$ is increasing and there exists $x_c \in (a, b]$ such that the function $\gamma := \psi \circ \ell^{-1}$ is strictly convex over $\ell((a, x_c))$ and strictly concave over $\ell((x_c, b))$.

(H5b) The function $\ell: I \to \mathbb{R}$ is increasing and there exists $x_c \in (a, b]$ such that the function $\gamma := \psi \circ \ell^{-1}$ is strictly concave over $\ell((a, x_c))$ and strictly convex over $\ell((x_c, b))$.

**Remark 2.** When $x_c = b$, Hypotheses H5a and H3 are equivalent, as well as H5b and H4. Indeed, if $x_c > \hat{x}$, then H3 or H4 are also recovered, as it will be seen later in Proposition 5 of Section 4.

We provide now examples that fulfill H5a or H5b.

**Example 1.** Let $a = 0, b = 3$ and functions $f, g$ defined as follows

$$f(x) = -\frac{3}{2}x^3 + x^4 - \frac{1}{6}x^5, \quad g(x) = 3x - x^2$$

Hypotheses H1 and H2 are fulfilled (see Figure 1). We take the identify function for $\ell$. One can straightforwardly check that the function $\psi$ is given by the expression

$$\psi(x) = \frac{x^2}{2} - \frac{x^3}{6}$$

One can then easily check that Hypotheses H5a and H5b are fulfilled (see also Figure 1).

<figure><img src="image_2.png" /><figcaption>Figure 1: Example 1 fulfills Hypothesis H5a.</figcaption></figure>

**Example 2.** Let $a = 0, b = 6$ and functions $f, g$ defined as follows

$$f(x) = \frac{4x^3 - 24x^2}{4 + x + x^2}, \quad g(x) = 6x - x^2$$
Hypotheses H1 and H2 are fulfilled (see Figure 2). We take the identify function for $\ell$. One can straightforwardly check that the function $\psi$ is given by the expression

$$ \psi(x) = \frac{4x}{4 + x + x^2} $$

One can then easily check that Hypotheses H1 and H2 are fulfilled (see also Figure 2).

<figure><img src="image_3.png" /><figcaption>Figure 2: Example 2 fulfills Hypothesis H5b.</figcaption></figure>

## 3 A slope condition

In this section, we derive as a "slope condition" a geometric necessary condition for optimality, which links the switching points of an optimal trajectory through the function $\psi$.

We first reformulate the constraint (2) by considering the augmented dynamics

$$ \left\{ \begin{array}{l} \dot{x} = f(x) + ug(x) \\ \dot{y} = u \end{array} \right. \qquad (6) $$

with the boundary conditions:

$$ (x(0), y(0)) = (\bar{x}, 0) \quad \text{and} \quad (x(T), y(T)) = (\bar{x}, \bar{u}T) \qquad (7) $$

The optimal control problem can then be stated as

$$ \inf_{u \in \mathcal{U}} \int_{0}^{T} \ell(x(t)) dt \quad \text{s.t.} \ (x, y) \text{ satisfies } (6) - (7) \qquad (8) $$

where $\mathcal{U}$ denotes the set of measurable control functions $u$ over $[0, T]$ taking values in $[-1, 1]$. Note that Problem (8) in $\mathbb{R}^2$ admits a solution by classical existence results. Indeed, the set of trajectories that satisfy the boundary conditions (7) is non empty (it contains the steady state $\bar{x}$ with constant control $\bar{u}$), and since the system is affine w.r.t. the control and $\ell$ is continuous, the existence of an optimal control follows by Filippov's existence theorem [11]. We define the Hamiltonian $H: \mathbb{R}^2 \times \mathbb{R}^2 \times \mathbb{R} \times \mathbb{R} \to \mathbb{R}$ as

$$ H = H(x, y, \lambda_x, \lambda_y, \lambda_0, u) = \lambda_0 \ell(x) + \lambda_x f(x) + u(\lambda_x g(x) + \lambda_y) $$
where $\lambda := (\lambda_x, \lambda_y)$ is the adjoint vector. From the Pontryagin Maximum Principle [19], we know that for any optimal control $u \in \mathcal{U}$ and $(x, y)$ the associated solution of (6)-(7), there exists a scalar $\lambda_0 \le 0$ and an absolutely continuous map $\lambda : [0, T] \to \mathbb{R}^2$ solution of the adjoint dynamics

$$ \left\{ \begin{array}{l} \dot{\lambda}_x = -\lambda_0 \ell'(x(t)) - \lambda_x(f'(x(t)) + u(t)g'(x(t))) \\ \dot{\lambda}_y = 0 \end{array} \right. \qquad (9) $$

for a.e. $t \in [0, T]$. Moreover one has $(\lambda_0, \lambda) \neq 0$ and the Hamiltonian condition writes

$$ u(t) \in \arg \max_{v \in [-1,1]} H(x(t), \lambda(t), \lambda_0, v) \quad \text{a.e. } t \in [0, T]. \qquad (10) $$

A solution $x$ satisfying (6)-(7) for a control $u \in \mathcal{U}$ and such that there exists $(\lambda_0, \lambda) \neq 0$ verifying (9)-(10) is called an extremal. Since the dynamics is affine w.r.t. $u$, the switching function

$$ t \mapsto \phi(t) := \lambda_x(t)g(x(t)) + \lambda_y, $$

provides the following property of the optimal control $u$ for almost any $t \in [0, T]$:

$$ \left\{ \begin{array}{ll} \phi(t) > 0 & \Rightarrow u(t) = 1 \\ \phi(t) < 0 & \Rightarrow u(t) = -1 \\ \phi(t) = 0 & \Rightarrow u(t) \in [-1, 1] \end{array} \right. \qquad (11) $$

We recall that a singular arc occurs if $\phi$ vanishes on some time interval $[t_1, t_2]$ with $t_1 < t_2$, and a switching time $t_s \in (0, T)$ is such that an extremal control $u$ is non-constant in any neighborhood of $t_s$ (which implies $\phi(t_s) = 0$). Let us mention that from Hypothesis H2, when $\phi > 0$, resp. $\phi < 0$, then $x$ is increasing, resp. decreasing.

For convenience, we shall define the following numbers.

**Definition 2.** For a solution $x(\cdot)$ of (1) with $x(0) = \bar{x}$, let us denote by $x_m$, respectively $x_M$, the minimum, respectively the maximum, of $x(\cdot)$ on $[0, T]$.

Let us first analyze the case of abnormal extremals i.e. with $\lambda_0 = 0$.

**Proposition 3.** Assume that hypotheses H1-H2- $\bar{H}$ are fulfilled.

* If $\psi$ is increasing on $I$, there is no abnormal extremal.

* If $\psi$ is not increasing on whole $I$, abnormal extremals are non constant with $x_m \le \hat{x} \le x_M$ and switches occur only at $x = x_m$ or $x = x_M$, without singular arc.

*Proof.* If $\lambda_0 = 0$, then $\lambda_x$ cannot vanish from the adjoint equation (9). Otherwise $\lambda_x$ would be null over $[0, T]$ and the switching function would be constant equal to $\lambda_y$. Since $\lambda_y$ cannot be simultaneously equal to 0, $\phi$ would be of constant sign over $[0, T]$ implying that $u = 1$ or $u = -1$ over $[0, T]$. This is a contradiction with the periodicity of $x(\cdot)$ (recall that one has $f + g > 0$ and $f - g < 0$ over $I$). Consequently, $\lambda_x$ has constant non null sign.

Since $\lambda_0 = 0$, one has from the adjoint equations (9)

$$ \dot{\phi}(t) = \lambda_x(t)g(x(t))^2\psi'(x(t)), \quad \text{a.e. } t \in [0, T] \qquad (12) $$

If $\psi$ is increasing on $I$, $\phi$ is monotonic and has at most one switching point, implying that $x$ is either entirely above or entirely below $\bar{x}$. But then the equality (5) of Lemma 1 cannot be verified when $\psi$ is increasing.

Consider now the case when $\psi$ is non increasing on $I$. One has then $\hat{x} \in I$. For any extremal $x$, we know from Lemma 1 that one has $x_m \le \bar{x} \le x_M$. Remind, as already mentioned in Section 2, that one has necessarily $\hat{x} > \bar{x}$. If $\hat{x} > x_M$, then $\psi$ is increasing on $[0, T]$ and we conclude as previously that this is not possible. Therefore an abnormal extremal should verify $x_m \le \hat{x} \le x_M$. Note that the constant solution $\bar{x}$ cannot be an abnormal extremal when $\hat{x} \neq \bar{x}$. The extreme values of $x$ having necessarily to be a switching locus, one should have $\phi(t_m) = \phi(t_M) = 0$ for some $t_m, t_M \in (0, T)$ with $x(t_m) = x_m$ and $x(t_M) = x_M$. Since the Hamiltonian is conserved along extremal trajectories, it comes with $\lambda_0 = 0$

$$ H(t_m) = \lambda_x(t_m)f(x_m) = -\lambda_y\psi(x_m) = H(t_M) = \lambda_x(t_M)f(x_M) = -\lambda_y\psi(x_M) $$

Since $\lambda_x$ is non null and $\phi$ is null lat $t_m$ and $t_M$, $\lambda_y$ cannot be null. We conclude that the equality $\psi(x_m) = \psi(x_M)$ is necessarily satisfied. Finally if a switch occurs at $x_s \notin \{x_m, x_M\}$ one should have $\psi(x_s) = \psi(x_m) = \psi(x_M)$, which is in contradiction with Hypothesis $\bar{H}$. Finally, note from equation (12) that a singular arc on a time interval $[t_1, t_2]$ with $t_1 < t_2$ (i.e. such that $\phi(t) = 0, t \in [t_1, t_2]$) could occur only for $x(t) = \hat{x}, t \in [t_1, t_2]$ but then $\phi(t) = \lambda_x(t)g(x(t)) + \lambda_y$ could not be constant for $t \in [t_1, t_2]$. $\square$
We focus now on regular extremals, i.e. with $\lambda_0 = -1$. We begin by a lemma that characterizes the singular arcs as constant values of $x$.

**Lemma 2.** Assume that hypotheses H1-H2-\bar{H} and H5a or H5b are fulfilled, and let $x$ be a regular extremal. If there exists a time interval $[t_1, t_2]$ with $t_1 < t_2$ such that $\phi$ vanishes on $[t_1, t_2]$, then there exists $\tilde{x} \in I$ such that $x(t) = \tilde{x}$ and $\lambda_y^{-1} = \gamma'(\ell(\tilde{x}))$ for any $t \in [t_1, t_2]$ with $u(t) = \psi(\tilde{x})$ for a.e. $t \in [t_1, t_2]$.

*Proof.* Assume $\phi(t) = 0$ for $t \in [t_1, t_2]$, which yields

$$ \lambda_x(t) = -\frac{\lambda_y}{g(x(t))}, \quad t \in [t_1, t_2] $$

If $\lambda_y = 0$, then $\lambda_x = 0$ on $[t_1, t_2]$ and from (9), it comes that $\lambda_0 = 0$. The extremal cannot be regular. So, one has $\lambda_y \neq 0$.

It comes from the adjoint equations (9) with $\lambda_0 = -1$ that

$$ \dot{\phi}(t) = \ell'(x(t))g(x(t)) + \lambda_x(t)g(x(t))^2\psi'(x(t)), \quad \text{a.e. } t \in [t_1, t_2] $$

As $g > 0$ on $I$ and $\phi = 0$ on $[t_1, t_2]$, one has then

$$ 0 = 1 - \lambda_y \gamma'(\ell(x(t))), \quad \text{a.e. } t \in [t_1, t_2] \Rightarrow 1/\lambda_y = \gamma'(\ell(x(t))), \quad \text{a.e. } t \in [t_1, t_2] $$

Note that under hypothesis H5a or H5b, the function $\gamma'$ is almost everywhere increasing or decreasing and $\ell$ is increasing. Therefore, the function $\gamma' \circ \ell$ cannot be constant on any interval of non null length. The result follows with $\tilde{x}$ solution of $\gamma'(\ell(x)) = 1/\lambda_y$. $\square$

We give now our optimality "slope condition", that shall play an important role in the derivation of an optimal synthesis.

**Proposition 4.** Assume that hypotheses H1-H2-\bar{H} and H5a or H5b are fulfilled. Let $x$ be a non constant regular extremal. Then $\lambda_y \neq 0$ and the switching set $S := \{x(t_s); t_s \in [0, T], \phi(t_s) = 0\}$ is equal to $\{x_m, x_M\}$ or $\{x_m, x_I, x_M\}$ with $x_m < x_I < x_M$, satisfying the so-called slope condition

$$ (x_1, x_2) \in S^2, x_1 \neq x_2 \Rightarrow \frac{\gamma(\ell(x_1)) - \gamma(\ell(x_2))}{\ell(x_1) - \ell(x_2)} = \frac{1}{\lambda_y} \quad (13) $$

Moreover, if $x$ is optimal with a singular arc at $\tilde{x}$, then $S = \{x_m, x_M\}$, $\tilde{x} < \hat{x}$ with

* $\tilde{x} = x_M$ under Hypothesis H5a, satisfying the slope condition

$$ \frac{\gamma(\ell(x_M)) - \gamma(\ell(x_m))}{\ell(x_M) - \ell(x_m)} = \gamma'(\ell(x_M)) \quad (14) $$

* $\tilde{x} = x_m$ under Hypothesis H5b, satisfying the slope condition

$$ \frac{\gamma(\ell(x_M)) - \gamma(\ell(x_m))}{\ell(x_M) - \ell(x_m)} = \gamma'(\ell(x_m)) \quad (15) $$

*Proof.* Consider $t_1, t_2 \in [0, T]$ and distinct $x_1, x_2$ in $I$ such that $x(t_i) = x_i$ and $\phi(t_i) = 0$ for $i = 1, 2$. Since the Hamiltonian $H$ is conserved along any extremal, one has (for a regular extremal)

$$ H = -l(x_i) - \lambda_y \frac{f(x_i)}{g(x_i)} = -\ell(x_i) + \lambda_y \psi(x_i) \quad \text{for } i = 1, 2 $$

which shows that $\lambda_y \neq 0$, using the fact that $l$ is increasing on $I$. Then it comes the following "slope condition"

$$ \frac{1}{\lambda_y} = \frac{\psi(x_1) - \psi(x_2)}{\ell(x_1) - \ell(x_2)} = \frac{\gamma(\ell(x_1)) - \gamma(\ell(x_2))}{\ell(x_1) - \ell(x_2)} $$

As extreme values of $x$ are necessarily switching locus, one has then

$$ \frac{1}{\lambda_y} = \frac{\gamma(\ell(x_M)) - \gamma(\ell(x_m))}{\ell(x_M) - \ell(x_m)} $$
Since $\gamma$ change convexity only once on $l(I)$ under Hypothesis H5a or H5b, there can only exist at most one other value $x_I \in (x_m, x_M)$ such that the slope constraint (13) is satisfied.

Consider now the case when $x$ admits a singular arc $\tilde{x}$ under Hypothesis H5a (the proof for the case with Hypothesis H5b is similar and left to the reader). Let us first show that having $\tilde{x} < x_c$ cannot be optimal. Otherwise, considering any interval $[t_1, t_1 + \delta] \subset [t_1, t_2]$ with $\delta > 0$, the constant solution $x = \tilde{x}$ has be optimal for the periodic optimal problem (4) with period $\delta$, initial condition $x(0) = \tilde{x}$ and input constraint $\frac{1}{\delta} \int_0^\delta u(t)dt = \tilde{u} = \psi(\tilde{x})$. But $\gamma$ being increasing and strictly convex at $\tilde{x}$, any admissible solution on $[0, \delta]$ with $x(0) = \tilde{x}$ lies on a domain where $\gamma$ is increasing and strictly convex, provided that $\delta$ is small. Then, Proposition 1 applies, which proves that $x = \tilde{x}$ cannot be optimal. We have thus $\tilde{x} \ge x_c$. If $\tilde{x} \ne x_M$, then the slope condition (13) gives

$$ \frac{1}{\lambda_y} = \frac{\psi(x_M) - \psi(\tilde{x})}{\ell(x_M) - \ell(\tilde{x})} = \frac{\gamma(\ell(x_M)) - \gamma(\ell(\tilde{x}))}{\ell(x_M) - \ell(\tilde{x})} $$

With $\frac{1}{\lambda_y} = \gamma'(\ell(\tilde{x}))$ given by Lemma 2, it comes the following equality

$$ \gamma'(\ell(\tilde{x})) = \frac{\gamma(\ell(x_M)) - \gamma(\ell(\tilde{x}))}{\ell(x_M) - \ell(\tilde{x})} $$

which contradicts the function $\gamma$ being strictly concave on $(\tilde{x}, x_M)$. We conclude that $\tilde{x}$ has to be equal to $x_M$. Now, let us show that $S = \{x_m, \tilde{x}\}$. Suppose by contradiction that $S = \{x_m, x_I, x_M\}$. Using the same concavity argument as before, it comes that $x_I < x_c$. Consider now the function $\hat{\gamma} : \ell(I) \to \mathbb{R}$ defined as

$$ \hat{\gamma}(\ell(x)) = \begin{cases} \gamma(\ell(x)) & \text{if } x \le x_c \\ \gamma'(\ell(x_c))(\ell(x) - \ell(x_c)) + \gamma(\ell(x_c)) & \text{otherwise} \end{cases} $$

which implies that $\hat{\gamma}$ is convex and above $\gamma$ on $\ell(I)$, with $\hat{\gamma} = \gamma$ on $\ell([a, x_c])$. Now let us consider the point $x_i \in (x_c, \tilde{x})$ such that

$$ \gamma'(\ell(x_c))(\ell(x_i) - \ell(x_c)) + \gamma(\ell(x_c)) = \gamma'(\ell(\tilde{x}))(\ell(x_i) - \ell(\tilde{x})) + \gamma(\ell(\tilde{x})) $$

implying that $x_m < x_I < x_i$.

By convexity of $\hat{\gamma} \circ \ell$, $\gamma(\ell(x_I))$ is strictly under the straight line that passes through $(x_m, \gamma(\ell(x_m)))$ and $(x_i, \gamma(\ell(x_i)))$ which is by construction the line $x \mapsto \gamma'(\ell(\tilde{x}))(\ell(x) - \ell(\tilde{x})) + \gamma(\ell(\tilde{x}))$, hence showing that $x_I$ cannot satisfy the slope condition (13).

Finally, the slope condition (13) with $x_1 = x_M$, $x_2 = x_m$ and $\frac{1}{\lambda_y} = \gamma'(\ell(x_M))$ gives the condition (14). Now, let us show that one has necessarily $x_M < \hat{x}$. If $x_M \ge \hat{x}$, then $\ell(x_M)$ has to be in the concave part of $\gamma$ with $\gamma'(\ell(x_M)) < 0$ (and $\gamma$ is necessarily concave on $(\ell(\hat{x}), \ell(b))$. This implies the inequality

$$ \gamma(\ell(\hat{x})) \le \gamma(\ell(x_M)) + \gamma'(\ell(x_M))(\ell(\hat{x}) - \ell(x_M)) $$

Recall from Lemma 1 that one has $x_m < \bar{x}$ (and $\bar{x} \le \hat{x}$ from Remark 1). As $\gamma$ is increasing on $(\ell(a), \ell(\hat{x}))$ (and $\ell$ is an increasing function), one gets

$$ \gamma(\ell(x_m)) < \gamma(\ell(x_M)) + \gamma'(\ell(x_M))(\ell(x_m) - \ell(x_M)) $$

which contradicts the condition (14). □

The slope condition is a necessary condition for optimality, which states that two cases can exist: either there are two switching points (the maximum and the minimum), and one of them might correspond to a singular arc (i.e. a constant portion of the trajectory), or there are three switching points without any singular arc.

# 4 Restriction to the increasing part of the function $\psi$

In this section, we show that when the function $\psi$ is not increasing on the whole interval $I$, an optimal trajectory remains necessarily in the domain $I \cap \{x < \hat{x}\}$ where the function $\psi$ is increasing (or equivalently that one has $x_M < \hat{x}$). The main idea of the proof of Proposition 5 below is to show that if it is not the case, one can exhibit a piece of the trajectory which is increasing up to $x_M$ and decreasing so that replacing it by a constant state remains admissible and gives a better cost (see Fig. 3).
<figure><img src="image_4.png" /><figcaption>Figure 3: An illustration of the truncation in the proof of Proposition 5: the blue part is replaced by the red one</figcaption></figure>

For convenience, we first introduce the following function, as in [5],

$$ \eta(x) := \frac{1}{f(x) + g(x)} - \frac{1}{f(x) - g(x)}, \quad x \in I $$

which is positive and $C^1$ on $I$ thanks to H2. This function possesses the following property related to bang-bang controls.

**Lemma 3.** Under H1 and H2, consider a piece of trajectory $x(\cdot)$ in $I$ on an interval $[t_0, t_0 + T_0]$ (with $T_0 > 0$) generated by a "bang-bang" control $u(\cdot)$ equal to $u_{BB}^-(\cdot)$ or $u_{BB}^+(\cdot)$ defined as

$$ u_{BB}^{-}(t) = \begin{cases} -1 & t \in [t_0, t'_1) \\ 1 & t \in [t'_1, t'_2) \\ -1 & t \in [t'_2, t_0 + T_0) \end{cases}, \quad u_{BB}^{+}(t) = \begin{cases} 1 & t \in [t_0, t'_1) \\ -1 & t \in [t'_1, t'_2) \\ 1 & t \in [t'_2, t_0 + T_0) \end{cases} $$

for some $t'_1 < t'_2$ in $[t_0, t_0 + T_0]$, that satisfies

$$ x(t_0 + T_0) = x(t_0) \quad \text{and} \quad \frac{1}{T_0} \int_{t_0}^{t_0+T} u(t)dt = \psi(x(t_0)) \qquad (16) $$

Then, one has

$$ \int_{x'_m}^{x'_M} \eta(\xi)d\xi = T_0 \quad \text{and} \quad \int_{x'_m}^{x'_M} \eta(\xi)\psi(\xi)d\xi = \psi(x(t_0))T_0 \qquad (17) $$

where $x'_m = \min_{t \in [t_0, t_0 + T_0]} x(t)$, $x'_M = \max_{t \in [t_0, t_0 + T_0]} x(t)$.

*Proof.* Let us consider $u(\cdot) = u_{BB}^-(\cdot)$ (the case $u(\cdot) = u_{BB}^+(\cdot)$ is analogous). For $t \in [t_0, t'_1) \cup [t'_2, t_0 + T_0)$, one has $\dot{x} = f(x) - g(x) < 0$. Therefore $\xi : t \mapsto x(t)$ defines a diffeomorphism from $[t_0, t'_1]$ to its image, and from $[t'_2, t_0 + T_0]$ to its image as well. Then, one can write

$$ t'_1 - t_0 = - \int_{x(t'_1)}^{x(t_0)} \frac{d\xi}{f(\xi) - g(\xi)}, \quad T_0 + t_0 - t'_2 = - \int_{x(T+t_0)}^{x(t'_2)} \frac{d\xi}{f(\xi) - g(\xi)} $$

Similarly, for $t \in [t'_1, t'_2)$, one has $\dot{x} = f(x) + g(x) > 0$ and can write

$$ t'_2 - t'_1 = \int_{x(t'_1)}^{x(t'_2)} \frac{d\xi}{f(\xi) + g(\xi)} $$

Then $x(t_0 + T_0) = x(t_0)$ gives

$$ T_0 = \int_{x(t'_1)}^{x(t'_2)} \frac{d\xi}{f(\xi) + g(\xi)} - \int_{x(t'_1)}^{x(t'_2)} \frac{d\xi}{f(\xi) - g(\xi)} = \int_{x(t'_1)}^{x(t'_2)} \eta(\xi)d\xi $$
Proceeding with the same decomposition of the interval $[t_0, t_0 + T_0]$, one obtains

$$ \int_{t_0}^{t_0+T} u(t)dt = \int_{x(t'_1)}^{x(t_0)} \frac{d\xi}{f(\xi) - g(\xi)} + \int_{x(t'_1)}^{x(t'_2)} \frac{d\xi}{f(\xi) + g(\xi)} + \int_{x(T+t_0)}^{x(t'_2)} \frac{d\xi}{f(\xi) + g(\xi)} $$

and with conditions (16) the equality

$$ \psi(x(t_0))T_0 = \int_{x(t'_1)}^{x(t'_2)} \left( \frac{1}{f(\xi) + g(\xi)} + \frac{1}{f(\xi) - g(\xi)} \right) d\xi = \int_{x(t'_1)}^{x(t'_2)} \eta(\xi)\psi(\xi)d\xi $$

is fulfilled. Clearly, one has $x'_m = x(t'_1)$ and $x'_M = x(t'_2)$, which give the expressions (17). □

**Proposition 5.** Suppose that hypotheses H1-H2-\bar{H} and H5a or H5b are verified. A non constant optimal trajectory $x(\cdot)$ for Problem (8) verifies $x(t) < \hat{x}$ for any $t \in [0, T]$.

*Proof.* Let $u(\cdot)$ be an optimal solution of Problem (8) and $x(\cdot)$ its associated trajectory. Let us remind the notation $x_m, x_M$ given by Definition 2, that we shall use below. If it is a regular extremal, the slope condition (13) of Proposition 4 gives

$$ \frac{1}{\lambda_y} = \frac{\psi(x_M) - \psi(x_m)}{\ell(x_M) - \ell(x_m)} $$

(since $\psi = \gamma \circ \ell$). Recall that $\ell$ is increasing. Therefore one has $\psi(x_M) \neq \psi(x_m)$. We distinguish two cases.

1. $\psi(x_M) < \psi(x_m)$. One has $\lambda_y < 0$ and a switching at $x_I$ such that $x_m < x_I < x_M$ imposes to have $\psi(x_I) < \psi(x_m)$ and $\psi(x_I) < \psi(x_M)$ from the slope condition (13). Since $\psi$ is increasing on $(x_m, \hat{x})$ and decreasing on $(\hat{x}, x_M)$, it comes that $x_I$ must satisfy $x_I > x_M$, which is a contradiction. Now suppose that $x(\cdot)$ admits a singular arc. The inequality $\psi(x_M) < \psi(x_m)$ yields $\gamma'(\ell(\hat{x})) < 0$ with $\tilde{x} = x_M$ under H5a and $\tilde{x} = x_m$ under H5b. This contradicts $\tilde{x} < \hat{x}$ as it was proved in Proposition 4. Therefore, $x(\cdot)$ has switching only at $x_m$ and $x_M$ and no singular arc.

2. $\psi(x_M) > \psi(x_m)$. Under Hypotheses H1-H2-\bar{H}, we define $x'_m$ as the unique value in $I$ different from $x_M$ such that $\psi(x'_m) = \psi(x_M)$. One has necessarily $x'_m < \hat{x}$. The slope condition (13) allows the existence of intermediate switch at $x_I \in (x_m, x_M)$ but one has $x_I < x'_m$. The case of a singular arc at $\tilde{x} = x_M$ is ruled out as Proposition 4 states that inequality $\tilde{x} < \hat{x}$ is satisfied.

If it is an abnormal extremal, one has $\psi(x_M) = \psi(x_m)$ and from Proposition 3, $x(\cdot)$ has switching only at $x_m$ and $x_M$ and no singular arc, as for case 1. above. Now, we posit

$$ \tilde{x}_m = \begin{cases} x_m & \text{if } \psi(x_M) \le \psi(x_m) \\ x'_m & \text{if } \psi(x_M) > \psi(x_m) \end{cases} $$

(which satisfies $\tilde{x}_m < \hat{x}$) and define the functions

$$ g(\tilde{x}) := \psi(\tilde{x}) \int_{\tilde{x}}^{x_M} \eta(\xi)d\xi, \quad h(\tilde{x}) := \int_{\tilde{x}}^{x_M} \eta(\xi)\psi(\xi)d\xi, \quad \tilde{x} \in [\tilde{x}_m, \hat{x}] $$

Clearly, one has

$$ h(\hat{x}) = \int_{\hat{x}}^{x_M} \eta(\xi)\psi(\xi)d\xi < \psi(\hat{x}) \int_{\hat{x}}^{x_M} \eta(\xi)d\xi = g(\hat{x}) $$

When $\psi(x_M) \le \psi(x_m)$, the trajectory $x(\cdot)$ has switches only at $x_m$ and $x_M$ and no singular arc, according to what has been shown previously. If it has only two switches, one can apply Lemma 3 on the interval $[0, T]$ and write

$$ h(x_m) = \int_{x_m}^{x_M} \eta(\xi)\psi(\xi)d\xi = \psi(\bar{x})T > \psi(x_m)T = g(x_m) $$

In the case where $x(\cdot)$ commutes more than once at $x_m$ or $x_M$ on the interval $[0, T]$, say $n$ times, the trajectory $x(\cdot)$ is $T/n$ periodic and we can apply Lemma 3 on the interval $[0, T/n]$ to obtain the same inequality. When $\psi(x_M) > \psi(x_m)$, one has

$$ h(x'_m) = \int_{x'_m}^{x_M} \eta(\xi)\psi(\xi)d\xi > \psi(x'_m) \int_{x'_m}^{x_M} \eta(\xi)d\xi = g(x_m) $$
Since the functions g and h are continuous, we deduce the existence of a number $x_d \in (\tilde{x}_m, \hat{x})$ such that $g(x_d) = h(x_d)$. From what precedes, $x_d$ is not a switching point of $x(\cdot)$, and by periodicity of the trajectory, $x(\cdot)$ has to pass by $x_d$ alternatively by increasing and by decreasing or vice-versa. We can then define

$$t_d := \inf\{t > 0; x(t) = x_d, x(\cdot) \text{ is increasing at } t\} \quad \text{and} \quad T_d := \inf\{T > 0; x(t_d + T) = x_d\}$$

which are such that $[t_d, t_d+T_d] \subset [0, T]$. On the time interval $(t_d, t_d+T_d)$, $x(\cdot)$ is above $x_d$ and from what precedes, switching occurs only at $x_M$ with no singular arc. Therefore one has

$$T_d = \int_{t_d}^{t_d+T_d} dt = \int_{x_d}^{x_M} \frac{d\xi}{f(\xi) + g(\xi)} + \int_{x_M}^{x_d} \frac{d\xi}{f(\xi) - g(\xi)} = \int_{x_d}^{x_M} \eta(\xi)d\xi = \frac{g(x_d)}{\psi(x_d)}$$

and

$$\int_{t_d}^{t_d+T_d} u(t)dt = \int_{x_d}^{x_M} \frac{d\xi}{f(\xi) + g(\xi)} - \int_{x_M}^{x_d} \frac{d\xi}{f(\xi) - g(\xi)} = \int_{x_d}^{x_M} \eta(\xi)\psi(\xi)d\xi = h(x_d)$$

The equality $g(x_d) = h(x_d)$ gives then

$$\int_{t_d}^{t_d+T_d} u(t)dt = \psi(x_d)T_d$$

Therefore, the control $u^\#(\cdot)$ defined on $[0, T]$ by

$$u^\#(t) = \begin{cases} \psi(x_d) & \text{if } t \in [t_d, t_d + T_d], \\ u(t) & \text{if } t \in [0, T] \setminus [t_d, t_d + T_d] \end{cases}$$

verifies

$$\int_{0}^{T} u^{\#}(t)dt = \int_{0}^{T} u(t)dt$$

and its associated trajectory $x^\#(\cdot)$ on $[0, T]$ is given by

$$x^\#(t) = \begin{cases} x_d & \text{if } t \in [t_d, t_d + T_d], \\ x(t) & \text{if } t \in [0, T] \setminus [t_d, t_d + T_d] \end{cases}$$

which consists in a truncation of the original trajectory $x(\cdot)$ (see Fig. 3). Clearly $u^\#(\cdot)$ is admissible, and its cost satisfies

$$J_T(u^\#) = \frac{1}{T} \left( \int_0^{t_d} \ell(x(t))dt + T_d\ell(x_d) + \int_{t_d+T_d}^T \ell(x(t))dt \right) < \frac{1}{T} \int_0^T \ell(x(t))dt = J_T(u)$$

$\ell$ being increasing and $x(\cdot) > x_d$ on $(t_d, t_d + T_d)$, which contradicts the optimality of $u(\cdot)$. $\square$

From Proposition 3, one obtains immediately the following property.

**Corollary 1.** Under hypotheses H1-H2-\bar{H} and H5a or H5b, an abnormal extremal cannot be optimal.

# 5 Optimal synthesis

Let us recall that the former results in [5] do not cover the case of a change of convexity of the function $\gamma$. However, these results can still be applied when $\bar{x}$ and $T$ are such that for any admissible solution $x(t)$ remains in one of the subsets $(a, x_c)$ or $(x_c, b)$, where $\gamma$ does not change its convexity. Then, either bang-bang, or constant solutions are optimal (depending if $\gamma$ is convex or concave on the subset, as stated in [5]). However, situations for which solutions $x(\cdot)$ can pass from one subset to another one has not been yet treated. We consider here the class of BSB (for "bang-singular-bang") control strategies with at most one singular arc.

**Definition 3.** Let $\tilde{t} \in [0, T]$. For the initial condition $x(0) = \bar{x}$ of system (1), we call BSB controls any time function $u_a(\tilde{t}; \cdot)$ or $u_b(\tilde{t}; \cdot)$ such that

$$u_a(\tilde{t}; t) = \begin{cases} -1 & \text{if } t \in [0, t_1) \\ +1 & \text{if } t \in [t_1, t_2) \\ \psi(x(t_2)) & \text{if } t \in [t_2, t_2 + \tilde{t}) \\ -1 & \text{if } t \in [t_2 + \tilde{t}, T) \end{cases}$$
where switching times $t_1, t_2$ are such that $0 \le t_1 \le t_2 \le T - \tilde{t}$, and

$$ u_b(\tilde{t}; t) = \begin{cases} -1 & \text{if } t \in [0, t_1) \\ \psi(x(t_1)) & \text{if } t \in [t_1, t_1 + \tilde{t}) \\ +1 & \text{if } t \in [t_1 + \tilde{t}, t_2) \\ -1 & \text{if } t \in [t_2, T) \end{cases} $$

where switching times $t_1, t_2$ are such that $0 \le t_1 \le t_2 - \tilde{t} \le T - \tilde{t}$.

Note that $\tilde{t}$ represents the duration of the trajectory spent on the singular arc $\tilde{x}$ (with $\tilde{x} = x(t_2)$ for the control $u_a(\tilde{t}; \cdot)$, and $\tilde{x} = x(t_1)$ for the control $u_b(\tilde{t}; \cdot)$). With the notations introduced in Definition 2, one has $x_m = x(t_2)$, $x_M = \tilde{x}$ with control $u_a(\tilde{t}; \cdot)$, and $x_m = \tilde{x}$, $x_M = x(t_2)$ with control $u_b(\tilde{t}; \cdot)$. Note also that the particular case $\tilde{t} = 0$ corresponds to pure "bang-bang" trajectories (i.e. without singular arc) while $\tilde{t} = T$ corresponds to the constant solution $\bar{x}$ (for these two particular cases, the definitions of $u_a$ and $u_b$ coincide). We show now that for any value of $\tilde{t}$, there exist unique controls $u_a, u_b$ that are admissible and such that the trajectory is periodic with $x_M < \hat{x}$. The following proposition is in the spirit of Proposition 3.2 in [5] but extended to the present context with singular arcs.

**Proposition 6.** Under Hypotheses H1-H2-Î, for any $\tilde{t} \in [0, T]$, there exist unique controls $u_a^*(\tilde{t}; \cdot)$, $u_b^*(\tilde{t}; \cdot)$ such that the solution of (6) satisfies the boundaries conditions (7) and belong to $I \cap \{x < \hat{x}\}$. Moreover, the corresponding $x_m, x_M$ are the unique solutions in $I \cap \{x < \hat{x}\}$ of the equations

$$ \int_{x_m}^{x_M} \eta(\xi) d\xi = T - \tilde{t} \qquad (18) $$

and

$$ \int_{x_m}^{x_M} \eta(\xi) \psi(\xi) d\xi = T\bar{u} - \tilde{t}\psi(\tilde{x}) \qquad (19) $$

with $\tilde{x} = x_M$ for the control $u_a(\tilde{t}; \cdot)$, and $\tilde{x} = x_m$ for the control $u_b(\tilde{t}; \cdot)$.

*Proof.* We consider controls $u_a$ only (the proof for controls $u_b$ is analogous).

On the interval $[0, t_1]$, one has $\dot{x} = f(x) - g(x) < 0$ and thus $\xi : t \mapsto x(t)$ defines a diffeomorphism from $[0, t_1]$ to its image, and similarly on the interval $[t_2 + \tilde{t}, T]$. On the interval $[t_1, t_2]$, one has $\dot{x} = f(x) + g(x) > 0$ and again $\xi : t \mapsto x(t)$ defines again a diffeomorphism from $[t_1, t_2]$ to its image. Then one can write

$$ \int_{0}^{T} dt = \int_{\bar{x}}^{x(t_1)} \frac{d\xi}{f(\xi) - g(\xi)} + \int_{x(t_1)}^{\tilde{x}} \frac{d\xi}{f(\xi) + g(\xi)} + \tilde{t} + \int_{\bar{x}}^{x(T)} \frac{d\xi}{f(\xi) - g(\xi)} $$

where $\tilde{x} = x(t_2)$. The trajectory $x(\cdot)$ is $T$-periodic when $x(T) = \bar{x}$ and one has then $x_m = x(t_1)$, $x_M = \tilde{x}$, which amounts to have

$$ T = \int_{x_m}^{x_M} \left( \frac{1}{f(\xi) + g(\xi)} - \frac{1}{f(\xi) - g(\xi)} \right) d\xi + \tilde{t} = \int_{x_m}^{x_M} \eta(\xi) d\xi + \tilde{t} $$

which is exactly equation (18). In the same way, one can write

$$ \int_{0}^{T} u(t) dt = - \int_{\bar{x}}^{x_m} \frac{d\xi}{f(\xi) - g(\xi)} + \int_{x_m}^{x_M} \frac{d\xi}{f(\xi) + g(\xi)} + \tilde{t}\psi(\tilde{x}) - \int_{x_M}^{\bar{x}} \frac{d\xi}{f(\xi) - g(\xi)} $$

and get for an admissible control $u(\cdot)$

$$ T\bar{u} = \int_{x_m}^{x_M} \left( \frac{1}{f(\xi) + g(\xi)} + \frac{1}{f(\xi) - g(\xi)} \right) d\xi + \tilde{t}\psi(\tilde{x}) = \int_{x_m}^{x_M} \eta(\xi)\psi(\xi)d\xi + \tilde{t}\psi(\tilde{x}) $$

which is exactly equation (19). We show now that for $\tilde{t}$ in $[0, T]$, there exists an unique pair $(x_m, x_M)$ in $I \cap \{x < \bar{x}\}$ that satisfy conditions (18) and (19).

For the particular case $\tilde{t} = T$, condition (18) imposes to have $x_M = x_m = \tilde{x}$ and condition (19) to have $\psi(\tilde{x}) = \bar{u} = \psi(\bar{x})$. However, under Hypothesis H, $\tilde{x} = \bar{x}$ is the only admissible constant solution in $I \cap \{x < \bar{x}\}$. We consider now $\tilde{t} < T$. Let us define the map

$$ \chi : (\xi_{-}, \xi_{+}) \mapsto \chi(\xi_{-}, \xi_{+}) := \int_{\xi_{-}}^{\xi_{+}} \eta(\xi) d\xi, \quad (\xi_{-}, \xi_{+}) \in I^2 $$
Under Hypotheses H1, H2, $\eta$ is a positive map on $I$, and one can easily check that for any $\alpha \in I$, $\chi(\alpha, \cdot)$ is $C^1$ and increasing with $\chi(\alpha, \alpha) = 0$ and $\chi(\alpha, b) = +\infty$. By the Implicit Function Theorem, there exists a unique map $\beta: I \to I$ of class $C^1$, such that $\chi(\alpha, \beta(\alpha)) = T - \tilde{t}$ for any $\alpha \in I$. Moreover, one has $\beta(\alpha) > \alpha$ for any $\alpha \in I$, and one can easily check that $\beta(I) = I$. Note that one obtains also

$$ \beta'(\alpha) = -\frac{\partial_{\xi_{-}}\chi(\alpha, \beta(\alpha))}{\partial_{\xi_{+}}\chi(\alpha, \beta(\alpha))} = \frac{\eta(\alpha)}{\eta(\beta(\alpha))} > 0, \quad \alpha \in I. $$

Let us then consider the map

$$ F(\alpha) := \int_{\alpha}^{\beta(\alpha)} \eta(\xi)\psi(\xi)d\xi - (T\bar{u} - \tilde{t}\psi(\beta(\alpha))), \quad \alpha \in I $$

Condition (18) amounts to have $\beta(x_m) = x_M$ and condition (19) to have $F(x_m) = 0$. Let us write equivalently the function $F$ as follows

$$ F(\alpha) := \int_{\alpha}^{\beta(\alpha)} \eta(\xi)(\psi(\xi) - \psi(\bar{x}))d\xi + \tilde{t}(\psi(\beta(\alpha)) - \psi(\bar{x})), \quad \alpha \in I $$

Under Hypothesis $\bar{H}$, one has $\psi(\xi) > \psi(\bar{x})$ for $\xi \in (\bar{x}, \beta^{-1}(\hat{x}))$ and $\psi(\xi) < \psi(\bar{x})$ for $\xi \in (a, \beta^{-1}(\bar{x}))$. Therefore, one has

$$ F(\alpha) > 0, \alpha \in (\bar{x}, \beta^{-1}(\hat{x})) \quad \text{and} \quad F(\alpha) < 0, \alpha \in (a, \beta^{-1}(\bar{x})) $$

By the Intermediate Value Theorem, we deduce that there exists $\alpha \in (\beta^{-1}(\bar{x}), \bar{x})$ such that $F(\alpha) = 0$. Moreover, one has

$$ \begin{aligned} F'(\alpha) &= \eta(\beta(\alpha))(\psi(\beta(\alpha)) - \psi(\bar{x}))\beta'(\alpha) - \eta(\alpha)(\psi(\alpha) - \psi(\bar{x})) + \tilde{t}\psi'(\beta(\alpha))\beta'(\alpha) \\ &= \eta(\alpha)(\psi(\beta(\alpha)) - \psi(\alpha)) + \tilde{t}\psi'(\beta(\alpha))\beta'(\alpha) > 0, \quad \alpha \in (\beta^{-1}(\bar{x}), \bar{x}) \end{aligned} $$

Therefore, we deduce that there exists an unique $x_m \in I$ such that $F(x_m) = 0$ with $\beta(x_m) < \hat{x}$, and $x_M$ is then uniquely defined as $x_M = \beta(x_m)$. $\square$

**Remark 3.** Under uniqueness of $x_m$, $x_M$ solutions of (18), (19), the controls $u_a^*(\tilde{t}; \cdot)$, $u_b^*(\tilde{t}; \cdot)$ can be expressed as follows

$$ \begin{aligned} u_a^*(\tilde{t}; t) &= \begin{cases} -1 & \text{if } t < t_1 := \inf\{t > 0, x(t) = x_m\} \\ +1 & \text{if } t_1 \le t < t_2 := \inf\{t > t_1, x(t) = x_M\} \\ \psi(x_M) & \text{if } t_2 \le t < t_2 + \tilde{t} \\ -1 & \text{if } t_2 + \tilde{t} \le t < t < T \end{cases} \\ u_b^*(\tilde{t}; t) &= \begin{cases} -1 & \text{if } t < t_1 := \inf\{t > 0, x(t) = x_m\} \\ \psi(x_m) & \text{if } t_1 \le t < t_1 + \tilde{t} \\ +1 & \text{if } t_1 + \tilde{t} \le t < t_2 := \inf\{t > t_1 + \tilde{t}, x(t) = x_M\} \\ -1 & \text{if } t_2 \le t < t < T \end{cases} \end{aligned} $$

We state now our main result, which says that optimal trajectories are of the BSB type.

**Theorem 1.** Under Hypotheses H1-H2-$\bar{H}$, and H5a, resp. H5b, there exists $\tilde{t} \in [0, T]$ such that the control $u_a^*(\tilde{t}; \cdot)$, resp. $u_b^*(\tilde{t}; \cdot)$ given by Proposition 6 is optimal for Problem (8) and the associated trajectory satisfies the slope condition (14), resp. (15) when $\tilde{t} \in (0, T)$.

*Proof.* Let $u(\cdot)$ be an optimal solution for Problem (8) and $x(\cdot)$ the corresponding trajectory. If $x_M \le x_c$ or $x_m \ge x_c$, then $x(\cdot)$ remains in a domain of $I$ where $\gamma$ does not change its concavity. One can then apply the former results of [5] that state that the optimal trajectory on $[0, T]$ is either constant, or "bang-bang" with a single switch at $x_m$ and at $x_M$. This amounts to claim that $u_a(\tilde{t}; \cdot)$ or $u_b(\tilde{t}; \cdot)$ is optimal with $\tilde{t} = 0$ or $\tilde{t} = T$. Let us now consider cases for which $x_M > x_c$ and $x_m < x_c$.

Assume that Hypothesis H5a is fulfilled (the proof under Hypothesis H5b is similar, where $u_a$ is replaced by $u_b$, and is left to the reader). If $x(\cdot)$ does not have a singular arc, let us show that $x(\cdot)$ cannot switch more than
once at $x_M$ on the interval $[0, T]$. Otherwise, there exist $t_1^M < t_2^M$ in $(0, T)$ such that $x(t_1^M) = x(t_2^M) = x_M$ with $x(t) < x_M$ for any $t \in (t_1^M, t_2^M)$. From Proposition 4, we know that there exists $t_I \in (t_1^M, t_2^M)$ such that one has

$$u(t) = \begin{cases} -1, & \text{a.e. } t \in [t_1^M, t_I) \\ 1, & \text{a.e. } t \in [t_I, t_2^M] \end{cases}$$

One can consider four numbers $t_i$ in $(0, T)$, $i \in \{1, \dots, 4\}$, such that

$$(i) \quad t_1 < t_1^M < t_2 < t_3 < t_2^M < t_4$$

$$(ii) \quad u(t) = 1 \text{ for a.e. } t \in (t_1, t_1^M); u(t) = -1 \text{ for a.e. } t \in (t_2^M, t_4)$$

$$(iii) \quad x(t_i) = x_b, i \in \{1, 2, 3, 4\}, \text{ with } x_b > x_c$$

Then, one has $x(t) > x_b$ for $t \in (t_1, t_2) \cup (t_3, t_4)$. Now, we swap the pieces of the trajectory $x(\cdot)$ on $(t_2, t_3)$ and $(t_3, t_4)$, defining a new trajectory $x_\theta(t) = x(\theta(t))$, $t \in [0, T]$, where

$$\theta(t) = \begin{cases} t & \text{if } t < t_2 \\ t + (t_3 - t_2) & \text{if } t_2 \le t < t_2 + t_4 - t_3 \\ t - (t_4 - t_3) & \text{if } t_2 + t_4 - t_3 \le t < t_4 \\ t & \text{if } t_4 \le t \le T \end{cases}$$

so that the successive two peaks are now in the domain $\{x > x_b\}$ (see Figure 4). One can straightforwardly check that $x_\theta(\cdot)$ is another admissible solution that satisfies the constraints (7) with the same cost than $x(\cdot)$. Take $\tilde{x} \in (x_b, x_M)$ and let $\tilde{t} = \inf\{t > t_1, x_\theta(t) > \tilde{x}\}, \tilde{T}(\tilde{x}) = \inf\{t > t_2^M - (t_3 - t_2), x_\theta(t) < \tilde{x}\} - \tilde{t}$. On the interval $[\tilde{t}, \tilde{t} + \tilde{T}(\tilde{x})]$, the trajectory $x_\theta(\cdot)$ is bang-bang with three switches (increasing up to $x_M$, decreasing down to $x_b$, increasing again up to $x_M$ and finally decreasing down to $\tilde{x}$), and one can write

$$\tilde{T}(\tilde{x}) = \int_{\tilde{x}}^{x_M} \frac{d\xi}{f(\xi) + g(\xi)} - \int_{x_M}^{x_b} \frac{d\xi}{f(\xi) - g(\xi)} + \int_{x_b}^{x_M} \frac{d\xi}{f(\xi) + g(\xi)} - \int_{x_M}^{\tilde{x}} \frac{d\xi}{f(\xi) - g(\xi)} \\ = \int_{\tilde{x}}^{x_M} \eta(\xi)d\xi + \int_{x_b}^{x_M} \eta(\xi)d\xi$$

In a similar way, one obtains

$$\tilde{U}(\tilde{x}) := \int_{0}^{\tilde{T}(\tilde{x})} u(\theta(\tau - \tilde{t}))d\tau = \int_{\tilde{x}}^{x_M} \eta(\xi)\psi(\xi)d\xi + \int_{x_b}^{x_M} \eta(\xi)\psi(\xi)d\xi$$

As $\eta$ is positive and $\psi$ is increasing on $[x_M, x_M]$ (by Proposition 5), one gets

$$\tilde{U}(x_b) > \tilde{T}(x_b)\psi(x_b) \quad \text{and} \quad \tilde{U}(x^M) < \tilde{T}(x_M)\psi(x_M)$$

Therefore, by the Intermediate Value Theorem, we can choose $\tilde{x} \in (x_b, x_M)$ such that $\tilde{U}(\tilde{x}) = \tilde{T}(\tilde{x})\psi(\tilde{x})$, which amounts to have the average $\tilde{u}$ of the control $u \circ \theta(\cdot)$ on the interval $[\tilde{t}, \tilde{t} + \tilde{T}(\tilde{x})]$ equal to $\psi(\tilde{x})$. On this interval, we can consider the optimal control problem (8) with $T = \tilde{T}(\tilde{x})$, $\tilde{u} = \tilde{u}$ and $\tilde{x} = \tilde{x}$. As $\gamma$ is concave increasing on $[x_b, x_M]$, we can use the results of [5] that claim that a non-constant trajectory cannot minimize the average of $l \circ x_\theta(\cdot)$ on this interval, leading to a contradiction with the optimality of $x_\theta(\cdot)$, and thus of $x(\cdot)$.

In a similar way, one can prove that $x(\cdot)$ cannot switch more than once at $x_M$. Indeed, if it is the case one can consider an analogous construction of an optimal trajectory with a piece in the domain where $\gamma$ is convex, and where it switches twice at $x_M$, contradicting the former results of [5] for the convex case.

Thus, if $x(\cdot)$ has no singular arc, it has exactly one switch at $x_M$ and one switch at $x_M$, and is synthesized by the control $u_a(0; \cdot)$, which is uniquely defined according to Proposition 6.

Finally, if $x(\cdot)$ possesses a singular arc at a certain $\tilde{x}$, we know from Proposition 4 that $x_M$ and $x_m$ are the only values of $x(\cdot)$ for which switches occur, and that $\tilde{x} = x_M$. As $x(\cdot)$ cannot have more than one switch at $x_M$ we deduce that the set $S = \{t \in [0, T], x(t) = x_M\}$ is connected. Therefore, $x(\cdot)$ has a unique singular arc of length $\tilde{t} = |S| > 0$ and is synthesized by a control with a BSB structure, such as $u_a(\tilde{t}; \cdot)$ which is uniquely defined according to Proposition 6. $\square$
<figure><img src="image_5.png" /><figcaption>Figure 4: Illustration of the swap of piece of trajectory in the proof of Theorem 1: the trajectory $x(\cdot)$ on the left is replaced by the trajectory $x_\theta(\cdot)$ on the right</figcaption></figure>

**Remark 4.** In practice, one has simply to look for the best value $J_T(u_a^*(\tilde{t}; \cdot))$ resp. $J_T(u_b^*(\tilde{t}; \cdot))$ among $\tilde{t} \in [0, T]$ such that the slope condition is verified when $\tilde{t}$ is not 0 or T, as illustrated in Section 6. Accordingly to Proposition 6, one can equivalently look for values $\tilde{x}$ of the singular arc that give the best value of the criterion. For this purpose, one can first determine the subset $\mathcal{X}_M$, resp. $\mathcal{X}_m$, of values $x_M \in (x_c, \hat{x})$ such that the slope condition (14) is fulfilled for some $x_m < x_c$ (with a numerical tolerance), resp. of values $x_m < x_c$ such that the slope condition (15) is fulfilled for some $x_M \in (x_c, \hat{x})$. Then, one has simply to test the performances of the BSB strategy with $\tilde{x}$ in $\mathcal{X}_M$, resp. $\mathcal{X}_m$.

# 6 Illustration on a bioprocess model

In the past decades, periodic operations of biological or chemical processes have been investigated to enhance their performances [3, 1, 22, 20]. Several contributions have identified situations for which a periodic solution improves an objective function, such as the productivity, compared to its value at steady state [24, 1, 18, 12], or not [21]. In the recent work [4], an application for the piloting of waste water bioprocesses has been investigated. It has been shown that depending on the characteristics of the growth function of the micro-organisms, a non constant periodic flow rate could provide a lower average concentration of pollutant at the output of the process, compared to a constant flow rate treating the same quantity of contaminated water on a given period of time. However, when the growth function is neither convex nor concave, such as the Hill growth function (see below), a bang-bang periodic control is optimal when the nominal steady-state is in a region of local convexity of the growth function and when the period is small enough. For larger periods, this control strategy could lead the state of the dynamics to a region of concavity of the growth rate, and the criterion could be even worse than for a constant control. For these cases, a repetition of bang-bang over the period has been proposed as a sub-optimal strategy. We are now in position to show that this strategy is indeed not optimal and that the optimal bang-singular-bang does a much better job.

We recall the chemostat model, traditionally used in waste-water treatment modeling (see, e.g., [16]))

$$ \left\{ \begin{array}{l} \dot{s} = -\mu(s)b + D(s_{in} - s) \\ \dot{b} = \mu(s)b - Db \end{array} \right. \qquad (20) $$

where $s$ and $b$ denote the concentrations of pollutant and biomass. The control variable is the dilution rate $D$ (taking values in $[D_-, D_+]$ with $0 \le D_- < D_+$) and $s_{in} > 0$ is the input pollutant concentration. The function $\mu(\cdot)$ is the specific growth rate of the microbial population (the conversion rate of the bio-reaction has been kept equal to 1 without loss of generality, by a choice of the unit of $b$). One can straightforwardly see that non-trivial equilibria of (20) for constant $D = \bar{D} \in [D_-, D_+]$ are of the form

$$ (s, b) = (\bar{s}, s_{in} - \bar{s}) \quad \text{with} \quad \bar{D} = \mu(\bar{s}) $$

Given $T > 0$ and $\bar{D} \in (D_-, D_+)$, the optimal control problem considered in [4] is

$$ \inf_{D(\cdot)} \left\{ \frac{1}{T} \int_{0}^{T} s(t) dt \quad \text{s.t.} \ s(0) = s(T); \frac{1}{T} \int_{0}^{T} D(t) dt = \bar{D} \right\} \qquad (21) $$
Note that the system (20) can be reduced to a one-dimensional control dynamics. Indeed, for any periodic solution $(s(\cdot), b(\cdot))$ one has $\dot{z} = -Dz$ where $z = s_{in} - s - b$. Since $z(\cdot)$ is also periodic, one has necessarily $z(t) = 0$ for any $t$ so that (20) becomes

$$ \dot{s} = (D(t) - \mu(s))(s_{in} - s) \qquad (22) $$

The function $\mu$ is assumed to be $C^1$ increasing with $\mu(0) = 0$. The nominal dilution rate $\bar{D}$ is chosen in $(0, \mu(s_{in}))$, so that there exists an unique steady-state $\bar{s} \in (0, s_{in})$ for constant control $D = \bar{D}$, as the unique solution of $\mu(\bar{s}) = \bar{D}$, which is globally asymptotically stable on the domain $(0, s_{in})$.

Let us show that this problem falls exactly in the framework of Section 2. We take for $I$ the largest open interval containing $\bar{s}$ that is invariant for (22) with controls in $[D_{-}, D_{+}]$ and posit

$$ u := \alpha D + \beta \in [-1, 1] \quad \text{with} \quad \alpha := \frac{2}{D_{+} - D_{-}}, \quad \beta := -\frac{D_{+} + D_{-}}{D_{+} - D_{-}} $$

On the interval $I$, we consider the functions

$$ f(s) := (-\mu(s) - \beta/\alpha)(s_{in} - s), \quad g(s) := (s_{in} - s)/\alpha $$

and for the criterion (21) one takes $\ell(s) = s$. Let us check that Hypotheses H1-H2 are fulfilled. First of all, the bounds $a, b$ of $I$ are such that $\mu(a) = D^{-}$ and $\mu(b) = D^{+}$ or $b = s_{in}$. Then, one has

$$ f(a) = g(a) = \frac{D_{+} - D_{-}}{2}(s_{in} - a), \quad f(b) = -g(b) = -\frac{D_{+} - D_{-}}{2}(s_{in} - b) $$

and

$$ f(s) - g(s) = (D_{-} - \mu(s))(s_{in} - s) < 0, \quad f(s) + g(s) = (D_{+} - \mu(s))(s_{in} - s) > 0, \quad s \in (a, b) $$

From the expression of $f$ and $g$, we get

$$ \psi(s) = \alpha\mu(s) - \beta $$

which is increasing. Hypothesis $\bar{H}$ is thus fulfilled.

Here, we have considered for the function $\mu$ the Hill function (as in [4]) which is a monotonic growth function given by the expression

$$ \mu(s) := \frac{\mu_{max}s^n}{K_s^n + s^n}, \quad (n > 1) $$

This function is convex for $s$ lower than

$$ s_c = K_s \left( \frac{n-1}{n+1} \right)^{\frac{1}{n}} $$

and concave for $s$ above this value (see Figure 5). Hypothesis H5a is thus fulfilled.

<figure><img src="image_6.png" /><figcaption>Figure 5: Graph of the Hill function for $n = 2$, $K_s = \sqrt{3}$ and $\mu_{max} = 2$</figcaption></figure>

For the simulations, we have considered the following values of the parameters of the Hill function: $n = 2$, $K_s = \sqrt{3}$ and $\mu_{max} = 2$, and for the operating conditions $s_{in} = 4$, $D_{-} = 0$, $D_{+} = 1.2\mu(s_{in})$ with $T = 20$. For different values of $\bar{D}$, we have computed the cost of the periodic solution for the three control laws
<table><caption>Table 1</caption><thead><tr><th>D̄</th><th>cost of D = D̄</th><th>cost of BB</th><th>cost of optimal BSB</th><th>optimal t̃</th><th>optimal s̃</th></tr></thead><tbody><tr><td>0.214</td><td>0.6</td><td>0.462</td><td>0.428</td><td>3.57</td><td>1.710</td></tr><tr><td>0.352</td><td>0.8</td><td>0.756</td><td>0.663</td><td>6.38</td><td>1.704</td></tr><tr><td>0.500</td><td>1.0</td><td>1.086</td><td>0.915</td><td>0.46</td><td>1.695</td></tr><tr><td>0.649</td><td>1.2</td><td>1.425</td><td>1.166</td><td>12.7</td><td>1.676</td></tr></tbody></table>

1. constant control,

2. bang-bang control,

3. optimal bang-singular-bang control, for which the optimal value of $\tilde{t} \in (0, T)$ has been determined numerically, for which the value $\tilde{s}$ of the singular arc satisfies the slope condition (14) (see Figure 7)

Computations have been performed with the programming language Julia [7]. Results are summed up in Table 1, and Figure 6 depicts the corresponding trajectories. On Figure 7, one can also verify that the slope condition is

<figure><img src="image_7.png" /><figcaption>Figure 6: Examples of trajectories for the three control laws</figcaption></figure>

verified for the optimal BSB trajectory (given by the optimal $\tilde{t}$), in agreement with Proposition 4.

## 7 Conclusion

In this work, we have revisited a class of optimal periodic control problems linear with respect to the control, relaxing the convexity or concavity hypothesis on the dynamics. Based on the adjoint equations of the Maximum Principle, we have introduced several non-local techniques (slope condition, trajectory truncation, swap of pieces of trajectory...) to show that the optimal solution admits a single singular arc. This result generalizes former ones in the sense that a singular arc of null length gives a pure bang-bang solution which is optimal in the convex case, while a singular arc of full length is a constant solution that is optimal in the concave case. An illustration on a biological model shows the gains of the optimal strategy over bang-bang or steady-state solutions. More generally, this result shows the interest of "bang-singular-bang" periodic controls, compared to patterns considered in other approaches such as the $\pi$-criterion.

## Acknowledgments

Most of this work has been achieved while the first author was in internship from ENSTA, Paris at MISTEA Lab, Montpellier, France. Since Dec. 2020, the first author's research is funded by the European Research Council Starting Grant MAJORIS ERC-2019-STG-850925.

## References

[1] E. Abulesz and G. Lyberatos. Periodic optimization of continuous microbial growth processes. *Biotechnology and Bioengineering*, 29:1059–1065, 1987.
<figure><img src="image_8.png" /><figcaption>Figure 7: Graphs of the cost function $J_T(u_a^*(\tilde{t}, \cdot))$ as a function of $\tilde{t}$, for the four cases considered in Table 1. The portions of the curves in red correspond to BSB trajectories for which the slope condition (14) is verified (with a numerical tolerance of 5%). Dashed lines represent the costs of the constant controls.</figcaption></figure>

[2] E.M. Abulesz and G. Lyberatos. Periodic impulse forcing of nonlinear systems: a new method. *International Journal of Control*, 48:469–480, 1988.

[3] J.E. Bailey. Periodic operation of chemical reactors: a review. *Chemical Engineering Communications*, 1(3):111–124, 1974.

[4] T. Bayen, A. Rapaport, and F.-Z. Tani. Improvement of performances of the chemostat used for continuous biological water treatment with periodic controls. *Automatica*, 121(109199), 2020.

[5] T. Bayen, A. Rapaport, and F.-Z. Tani. Optimal periodic control for scalar dynamics under integral constraint on the input. *Math. Control Relat. Fields*, 10(3):547–571, 2020.

[6] A.O. Belyakov and V.M. Veliov. Constant versus periodic fishing: age structured optimal control approach. *Math. Model. Nat. Phenom.*, 9:20–37, 2014.

[7] J. Bezanson, A. Edelman, S. Karpinski, and Viral B. Shah. Julia: A fresh approach to numerical computing. *SIAM Review*, 2017.

[8] S. Bittanti, G. Fronza, and G. Guardabassi. Periodic control: A frequency domain approach. *IEEE Transactions on Automatic Control*, 18:33–38, 1973.

[9] S. Bittanti, A. Locatelli, and Maffezzoni C. Second-variation methods in periodic optimization. *J. Optimization Theory and Appl.*, 14:31–49, 1974.

[10] S. Bittanti, A. Locatelli, and S. Rinaldi. Status of periodic optimization of dynamical systems. *J. Optimization Theory and Appl.*, 14:1–20, 1974.

[11] L. Cesari. *Optimization – Theory and Applications. Problems with ordinary differential equations*. Springer, 1983.
[12] G. D'Avino, S. Crescitelli, P.L. Maffettone, and M. Grosso. On the choice of the optimal periodic operation for a continuous fermentation process. *Biotechnology Progress*, 26:1580-1589, 2010.

[13] R. T. Evans, J. L. Speyer, and C. H. Chuang. Solution of a periodic optimal control problem by asymptotic series. *J. Optimization Theory and Appl.*, 52:343-364, 1987.

[14] E. G. Gilbert. Optimal periodic control: A general theory of necessary conditions. *SIAM J. Control Optim.*, 15:717-746, 1977.

[15] E. G. Gilbert, Q. Wang, and J.L. Speyer. Necessary and sufficient conditions for local optimality of a periodic process. *SIAM J. Control Optim.*, 28:482-497, 1990.

[16] J. Harmand, C. Lobry, A. Rapaport, and T. Sari. *The Chemostat: mathematical theory of microorganisms cultures*. ISTE Wiley, 2017.

[17] C. Maffezzoni. Hamilton-jacobi theory for periodic control problems. *J. Optimization Theory and Appl.*, 14:21-29, 1974.

[18] S. Parulekar. Analysis of forced periodic operations of continuous bioprocesses: single input variations. *Chemical Engineering Science*, 3(55):2481-2502, 1998.

[19] L.S. Pontryagin, V.G. Boltyanskiy, R.V. Gamkrelidze, and E.F. Mishchenko. Mathematical theory of optimal processes. *The Macmillan Company*, 1964.

[20] L. Ruan and X.D. Chen. Comparison of several periodic operations of a continuous fermentation process. *Biotechnology Progress*, 12(2):286-288, 1996.

[21] M. Sadeghi, M.A. Al-Radhawi, M. Margaliot, and E.D. Sontag. No switching policy is optimal for a positive linear system with a bottleneck entrance. *IEEE Control Systems Letters*, 3(4):889-894, 2019.

[22] P.L. Silveston and R.R. Hudgins. *Periodic Operation of Chemical Reactors*. Butterworth-Heinemann, 2013.

[23] J. L. Speyer and R. T Evans. A second variation theory for optimal periodic processes. *IEEE Transactions on Automatic Control*, 29:138-148, 1984.

[24] N. Watanabe, K. Onogi, and M. Matsubara. Periodic control of continuous stirred tank reactors-I: The pi criterion and its applications to isothermal cases. *Chemical Engineering Science*, 36(5):809-818, 1981.