# Necessary optimality condition for the minimal time crisis relaxing transverse condition via regularization

Térence Bayen, Kenza Boumaza, Alain Rapaport

► To cite this version:

Térence Bayen, Kenza Boumaza, Alain Rapaport. Necessary optimality condition for the minimal time crisis relaxing transverse condition via regularization. ESAIM: Control, Optimisation and Calculus of Variations, 2021, 27, 10.1051/cocv/2021102. hal-03445347

HAL Id: hal-03445347

https://hal.inrae.fr/hal-03445347

Submitted on 23 Nov 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Necessary optimality condition for the minimal time crisis relaxing
transverse condition via regularization

Térence Bayen*, Kenza Boumaza, Alain Rapaport

November 23, 2021

Abstract

We derive necessary optimality conditions for the time of crisis problem under a more general hypothesis
than the usual one encountered in the hybrid setting, which requires that any optimal solution should cross
the boundary of the constraint set transversely. Doing so, we apply the Pontryagin Maximum Principle to a
sequence of regular optimal control problems whose integral cost approximates the time of crisis. Optimality
conditions are derived by passing to the limit in the Hamiltonian system (without the use of the hybrid
maximum principle). This convergence result essentially relies on the boundedness of the sequence of adjoint
vectors in $L^\infty$. Our main contribution is to relate this property to the boundedness in $L^1$ of a suitable sequence
which allows to avoid the use of the transverse hypothesis on optimal paths. An example with non-transverse
trajectories for which necessary conditions are derived highlights the use of this new condition.

**Key-words.** Time crisis, Necessary optimality conditions, Hybrid Maximum Principle, Regularization, Transverse crossing time.

MSC2020 Classification. 34H05, 49K15, 49J45, 34A38.

# 1 Introduction

This paper proposes a novel approach to derive optimality conditions for the so-called *time of crisis problem* [7] as
well as (new) sufficient conditions ensuring the well-posedness of this method. Such conditions will slightly differ
of those available in the literature that involve the behavior of an (a priori unknown) optimal path.

Originally, the time of crisis problem was introduced in [15], and it consists in minimizing the total time spent
by a solution of a control system outside a given constraint set. It is of particular interest whenever it is not
possible to maintain the system in such a set. In that case, alternative strategies consist in finding a control policy
such that the associated solution spends the minimum of time outside the constraint set. The time of crisis arises
in the context of viability theory [2, 3], see, e.g., a case study in ecology in [9], and more generally whenever one
is unable to maintain a controlled dynamics within a prescribed constraint set over a time window.

From a theoretical point of view, the formulation of the time of crisis involves a discontinuous function w.r.t. the
state, namely the indicator function of the constraint set. The integrand is then equal to 0 or 1 depending if the
state of the system lies inside or outside the constrained set (hence, it can be viewed as a particular instance of
a hybrid optimal control problem). In particular, the *Pontryagin Maximum Principle* (PMP), see [19], cannot be
applied straightforwardly to derive necessary conditions. Various approaches have been proposed in the literature
to study this issue and we now wish to give an overview of the available methods in order to highlight the differences
with our approach.

In the first paper about the time of crisis (see [15]), the optimal control problem was tackled via a dynamic
programming approach only. The question of necessary optimality conditions has been investigated more recently
in [7] using the so-called *hybrid maximum principle* (HMP) which is an extension of the PMP adapted to hybrid
systems (see [16, 17, 10]). In [7], the authors provide necessary conditions by a direct application of this principle
that requires a so-called *transverse hypothesis* on optimal trajectories which is as follows: any optimal solution
crosses the boundary of the constraint set transversely. As in [17], this hypothesis is crucial for the derivation
of necessary conditions (in particular, for an accurate definition of the jump of the covector at a crossing time).

<footnote>*Avignon Université, Laboratoire de Mathématiques d'Avignon (EA 2151), F-84018 Avignon, France.
terence.bayen@univ-avignon.fr</footnote>

<footnote>†UMR MISTEA, Univ. Montpellier, INRAE, Institut Agro, F-34060 Montpellier, France. **kenza.boumaza@inrae.fr**</footnote>

<footnote>‡UMR MISTEA, Univ. Montpellier, INRAE, Institut Agro, F-34060 Montpellier, France. **alain.rapaport@inrae.fr**</footnote>
Thanks to this hypothesis, it is also shown in [7, 8] that extremals of a regularized<fnref n="1" /> optimal control problem converge, up to a sub-sequence, to an extremal of the time of crisis problem. The methodology is in line with [17] using properties of variation vectors. Note that first and second order conditions have also been derived in [6] using the PMP on a reformulation of the time of crisis problem obtained via an augmentation technique (in the spirit of [12, 13, 14]). Let us also point out [1, 21] in which an approximation technique is introduced in order to obtain necessary conditions. The study of convergence of regularized extremals relies on a similar transversality assumption on optimal paths as in the framework of the HMP. In addition, the approximated optimal control problem involves the (unknown) optimal solution. It is worth noticing that this approach is of interest for obtaining optimality conditions via the use of the PMP on a smooth problem, however, it is not usable from a numerical point of view since the sequence of approximated problems itself involves the optimal solution.

Let us now describe more into details the content of this paper. As we have recalled, the time of crisis can be viewed as an application of the HMP on a discontinuous problem w.r.t. the state. The HMP available in the literature is a powerful tool, but its application requires an optimal solution to satisfy a transverse assumption related to the boundary of the constraint set (see [17]). In practice, this condition is hardly possible to check if one does not know in advance the optimal solution or estimates it with enough accuracy. That is why, we can wonder whether or not it is possible to derive optimality conditions for discontinuous integrands w.r.t. the state without the use of this hypothesis. Doing so, we introduce a sequence of regular optimal control problems whose integral cost approximates the time of crisis (this is made possible using mollifiers). Our contribution is twofold:

* We propose in the context of the time of crisis problem a sufficient condition for the derivation of necessary optimality conditions (see our Theorem 4.1). This condition relies on the data of the problem and on a sequence of approximated solutions, that can be easily checked.

* Necessary conditions for the time of crisis are derived under this condition which also covers the transverse case (recovering optimality conditions that can be alternatively obtained using the HMP in this case).

It is worth noticing that our alternative condition (see Theorem 4.1) does not involve the knowledge of the velocity of the optimal solution at a crossing time. As a byproduct of this approach (and in contrast with [1, 21] for instance), the sequence of approximated optimal control problems allows to approach an optimal solution of the original problem with a solution associated with a regular optimal control problem, that can be solved numerically with existing efficient methods.

The paper is organized as follows. In Section 2, we introduce the time of crisis problem and the regularization scheme, and we also apply the PMP on the regularized optimal control problem. In section 3, we study properties of a sequence of approximated solutions (namely, the integral of the approximated solution computed along the mollifier). This sequence will be crucial to introduce an auxiliary hypothesis in Section 5 that will allow us to derive optimality conditions for the time of crisis problem. Section 4 provides optimality conditions for the time of crisis under the hypothesis that the suitable sequence is bounded (and so, without a transverse hypothesis which is the novelty here). A sufficient condition involving this sequence is presented in Section 5. The last section is devoted to a complete study of an illustrative example that highlights the convergence of the sequence of adjoint vectors to a discontinuous covector having a jump at each crossing time. This phenomenon is not only depicted in a transverse situation, but also when a trajectory hits the boundary of the constrained set tangentially (and leaves the set non-tangentially).

# 2 Definitions and regularization of the time crisis problem

## 2.1 Notations and main hypotheses

Throughout the paper, $m, n \ge 1$ are integers and $T > 0$. Let us introduce the following notations:

* For $x, y \in \mathbb{R}^n$, $|x|$ and $x \cdot y$, denote the euclidean norm of $x$ and the scalar product between $x$ and $y$. If $A$ is a square matrix of dimension $n$, $\|A\|$ stands for the norm of $A$ and its transpose is written $A^T$.

* Given a mapping $f: \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}^n$, we denote respectively by $D_x f(x, u)$, $D_u f(x, u)$ the Jacobian matrix of $f$ w.r.t. variables $x$ and $u$ at some point $(x, u) \in \mathbb{R}^n \times \mathbb{R}^m$. The notations $\nabla \varphi(x)$, $\partial_{x_i} \varphi(x)$, $D_{xx} \varphi(x)$ denote the gradient, the partial derivative w.r.t. $x_i$, and the Hessian of a function $\varphi: \mathbb{R}^n \to \mathbb{R}$ at some point $x \in \mathbb{R}^n$.

<footnote n="1">¹In [7, 9], the regularization technique is based on the Moreau-Yosida approximation of the indicator function, provided the constrained set to be convex.</footnote>
• Given two integers $k, p \ge 1$ and a function $w : [0, T] \to \mathbb{R}^k$, the notation $\|w\|_{L^p(I; \mathbb{R}^k)}$ will stand for the $L^p$-norm of $w$ over some time interval $I \subset [0, T]$.

• If $g \in L^\infty([0, T]; \mathbb{R}^s)$, $s \in \mathbb{N}^*$, we denote by $g(t^\pm)$ the right and left limits (when it exists) of $g$ at point $t$. In the same way, we shall denote by $\dot{g}(t^\pm)$ the right and left derivative of a scalar function $g$ (when it exists).

In the sequel, we consider two non-empty subsets $U$ and $K$ of $\mathbb{R}^m$ and $\mathbb{R}^n$ respectively, as well as two mappings $\varphi : \mathbb{R}^n \to \mathbb{R}$ and $f : \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}^n$ (the dynamics). Throughout the paper, we suppose that the following hypotheses are satisfied:

(H1) The set $U$ is compact and $f$ is of class $C^1$ with linear growth, i.e., there is $c \ge 0$ such that for every $(x, u) \in \mathbb{R}^n \times U$, one has $|f(x, u)| \le c(|x| + 1)$.

(H2) For every $(x, p) \in \mathbb{R}^{2n}$, the set

$$ \bigcup_{u \in U} \left[ \begin{array}{l} f(x, u) \\ -D_x f(x, u)^\top p \end{array} \right] $$

is a non-empty compact<fnref n="2" /> convex subset of $\mathbb{R}^{2n}$.

(H3) We suppose that $\varphi$ is of class $C^2$, that the set $K$ is with non-empty interior and is the 0-sub-level set of $\varphi$:

$$ K = \{x \in \mathbb{R}^n; \varphi(x) \le 0\}. $$

(H4) For every $x \in \partial K$ (the boundary of $K$), one has $\varphi(x) = 0$ and $\nabla \varphi(x) \ne 0$.

Note that (H2) is fulfilled whenever the dynamics $f$ is affine w.r.t. to the control $u$ and $U$ is convex.

## 2.2 The time of crisis

Throughout the paper, we consider the admissible control set

$$ \mathcal{U} := \{u : [0, T] \to U; u \in L^\infty([0, T]; U)\}. $$

Given an initial condition $x_0 \in \mathbb{R}^n$, the minimal time crisis problem<fnref n="3" /> (over $[0, T]$) is defined as

$$ \inf_{u \in \mathcal{U}} \int_{0}^{T} \mathbb{1}_{K^c}(x_u(t)) \, dt, \qquad (2.1) $$

where $\mathbb{1}_{K^c}$ denotes the characteristic function of the complement of $K$, i.e., $\mathbb{1}_{K^c}(x) = 1$ if $x \notin K$, $\mathbb{1}_{K^c}(x) = 0$ if $x \in K$, and $x_u(\cdot)$ is the unique (global) solution to the Cauchy problem

$$ \left\{ \begin{array}{l} \dot{x}(t) = f(x(t), u(t)) \quad \text{a.e. } t \in [0, T], \\ x(0) = x_0. \end{array} \right. \qquad (2.2) $$

Under the previous assumptions, the existence of an optimal control for (2.1) follows easily from the upper semi-continuity of $\mathbb{1}_{K^c}$ (see, e.g., [7, Proposition 2.1] for more details). An important feature for studying optimality conditions of (2.1) is the notion of crossing time that we recall below.

**Definition 2.1.** Given a solution $x(\cdot)$ of (2.2), let us define the absolutely function $\rho$ as:

$$ \rho(t) := \varphi(x(t)), \quad t \in [0, T]. \qquad (2.3) $$

(i) A crossing time from $K$ to $K^c$, resp. from $K^c$ to $K$, is a time $\tau \in (0, T)$ for which there is $\eta > 0$ with $[\tau - \eta, \tau + \eta] \subset [0, T]$ such that $x(t) \in K$, resp. $x(t) \in K^c$, for $t \in [\tau - \eta, \tau]$ and $x(t) \in K^c$, resp. $x(t) \in K$, for $t \in (\tau, \tau + \eta]$. We shall say that a crossing time is "outward" if $x$ crosses $\partial K$ from $K$ to $K^c$, and "inward" if it crosses from $K^c$ to $K$.

(ii) A crossing time $\tau$ is transverse if moreover the function $\rho$ admits non-null left and right limits, i.e.,

$$ \dot{\rho}(\tau^{\pm}) = \lim_{t \to \tau^{\pm}} \nabla \varphi(x(t)) \cdot \dot{x}(t) \neq 0, \qquad (2.4) $$

(negative for an outward crossing time, positive for an inward crossing time.)

<footnote n="2">²The compactness property actually follows from the continuity of $f$ and $D_x f$ and the compactness of $U$.</footnote>

<footnote n="3">³To shorten, we also write time of crisis or time crisis problem.</footnote>
**Remark 2.1.** Definition 2.1 (i) is equivalent to say that $\tau$ is an isolated root of $\rho$ such that the map $t \mapsto \rho(t)(t-\tau)$ is locally of constant sign (positive from from K to K$^c$, negative from from K$^c$ to K).

Throughout the paper, we suppose that the following assumption holds true:

**Assumption 2.1.** Any optimal solution $x^*$ of (2.1) has a finite number $r \ge 1$ of (alternated) crossing times $(\tau_i)_{1 \le i \le r}$ such that

$$0 < \tau_1 < \tau_2 < \cdots < \tau_r < T.$$

In particular, we will not consider optimal trajectories that may cross the boundary of K an infinite number of times over $[0, T]$ (such as chattering [23]).

## 2.3 Regularization scheme

The approach developed in the present paper is an approximation procedure of Problem (2.1) with a sequence of regular problems that can be solved with standard optimality conditions (such as the PMP) or existing numerical tools. It will allow us to recover the conditions obtained for instance in [7] using the HMP [17]. As mentioned in the introduction, other authors considered regularization of problems similar to (2.1), but requiring an a priori knowledge of an optimal control [1, 21], which therefore cannot be used as a practical approximation, in contrast with our approach. In addition, we shall see that the sufficient condition for the derivation of optimality conditions that we obtain in Section 5 does not involve the assumption that each crossing time of an optimal solution is transverse, as it is required in the HMP. Instead, this condition relies only on the boundedness in $L^1$ of a suitable sequence related to the regularized problem, that can be tested numerically.

Let us now introduce a regularized scheme associated with (2.1). Doing so, we consider approximations of the Heaviside step function$^4$, denoted here G, by a sequence of functions that fulfill the following properties.

(H5) For any $n \in \mathbb{N}$, the map $G_n: \mathbb{R} \to [0, 1]$ is $C^2$ and non decreasing. The sequence $(G_n)_n$ is such that

$$\forall \sigma \in \mathbb{R}, \quad \lim_{n \to +\infty} G_n(\sigma) = G(\sigma).$$

Moreover, there exists two sequences of numbers $(a_n)_n \in \mathbb{R}_-$ and $(b_n)_n \in \mathbb{R}_+$ resp. increasing and decreasing and of limit 0 when $n \to +\infty$ such that:

$$\forall n \in \mathbb{N}, \quad \begin{cases} G_n(\sigma) = 0 & \text{if } \sigma \le a_n, \\ G_n(\sigma) = 1 & \text{if } \sigma \ge b_n. \end{cases}$$

In view of its definition, each function $G_n$ is Lipschitz continuous, and its Lipschitz constant $L_n$ is such that $L_n \to +\infty$ whenever $n \to +\infty$. Note also that the sequence $(h_n)_n$ defined as

$$h_n := G'_n,$$

is a $C^1$-mollifier, i.e., for every $n \in \mathbb{N}$, the support of $h_n$ is contained in $[a_n, b_n]$ and one has $\int_{\mathbb{R}} h_n(\sigma)d\sigma = 1$ for all $n \in \mathbb{N}$. In addition, one has $\sup_{\sigma \in \mathbb{R}} |h'_n(\sigma)| \to +\infty$ whenever $n \to +\infty$. A typical example of such a sequence of functions is given by:

$$G_n(\sigma) := \begin{cases} 0 & \text{if } \sigma \le -\frac{1}{2n}, \\ 3\left(n\sigma + \frac{1}{2}\right)^2 - 2\left(n\sigma + \frac{1}{2}\right)^3 & \text{if } \sigma \in \left(-\frac{1}{2n}, \frac{1}{2n}\right), \\ 1 & \text{if } \sigma \ge \frac{1}{2n}, \end{cases} \quad (2.5)$$

(see Fig. 1). In the sequel, we consider the following sequence of optimal control problems

$$\inf_{u \in \mathcal{U}} \int_{0}^{T} G_n(\varphi(x_u(t))) dt, \quad (2.6)$$

where $x_u(\cdot)$ is the unique solution to (2.2). The existence of an optimal control of (2.6) is straightforward using Filippov's existence Theorem [22] under our assumptions. Hereafter, we denote by $(x_n, u_n)$ an optimal pair of (2.6). Following [5], we can show that, up to a sub-sequence, the sequence $x_n$ converges strongly-weakly$^5$ to an optimal pair $(x^*, u^*)$ of (2.1). Let us stress that in general, $(u_n)$ may not converge point-wise to $u^*$. However, more can be said about this point whenever the system is affine w.r.t. the control and under additional assumptions (such as the absence of singular arcs), see e.g. [8, Remark 4.3] or [20].

<footnote>$^4$We define G as the function such that $G(\sigma) = 0$, resp. $G(\sigma) = 1$ whenever $\sigma \le 0$, resp. $\sigma > 0$.</footnote>

<footnote>$^5$This means that $(x_n)_n$ uniformly converges to $x^*$ over $[0, T]$ and that $(\dot{x}_n)_n$ weakly converges to $\dot{x}^*$ in $L^2([0, T]; \mathbb{R}^n)$.</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 1: Graphs of functions $G_n$ given by (2.5) for $n \in \{1, ..., 10\}$ (Fig. left) and of their derivatives $h_n$ (Fig. right).</figcaption></figure>

## 2.4 Optimality conditions for the regularized problem

We are now in a position to apply the Pontryagin Maximum Principle on Problem (2.6). Let $H_n : \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}$ be the Hamiltonian<fnref n="6" /> associated with (2.6) defined as:

$$H_n(x, p, u) = p \cdot f(x, u) - G_n(\varphi(x)).$$

Since $(x_n, u_n)$ is optimal for (2.6), there is an absolutely continuous map $p_n : [0, T] \to \mathbb{R}^n$ satisfying the adjoint equation $\dot{p}_n = -\frac{\partial H_n}{\partial x}$ almost everywhere and $p_n(T) = 0$, that is,

$$\left| \begin{array}{l} \dot{p}_n(t) = -D_x f(x_n(t), u_n(t))^{\top} p_n(t) + h_n(\varphi(x_n(t))) \nabla \varphi(x_n(t)) \quad \text{a.e. } t \in [0, T], \\ p_n(T) = 0, \end{array} \right. \qquad (2.7)$$

as well as the Hamiltonian condition which can be written

$$\forall u \in U, \quad p_n(t) \cdot f(x_n(t), u) \le p_n(t) \cdot f(x_n(t), u_n(t)) \quad \text{a.e. } t \in [0, T]. \qquad (2.8)$$

A triple $(x_n, p_n, u_n)$ satisfying (2.2)-(2.7)-(2.8) is called an extremal (recall that only normal extremals occur here as there is no terminal condition). Let us observe that the problem is autonomous, therefore, the Hamiltonian is conserved along any extremal. For every $n \in \mathbb{N}$, there is $\tilde{H}_n \in \mathbb{R}$ such that:

$$\tilde{H}_n = H_n(x_n(t), p_n(t), u_n(t)) = p_n(t) \cdot f(x_n(t), u_n(t)) - G_n(\varphi(x_n(t))) \quad \text{a.e. } t \in [0, T].$$

**Remark 2.2.** Since $(x_n)_n$ strongly-weakly converges to $x^*$ which has $r$ crossing times, it can be observed that $t \mapsto h_n(\varphi(x_n(t)))$ takes arbitrarily large values in (2.7). Hence, we can expect the sequence $(\dot{p}_n)_n$ to be unbounded in $L^\infty([0, T]; \mathbb{R}^n)$. We shall see that whenever every crossing time of $x^*$ is transverse, then $(p_n)_n$ is indeed bounded in $L^\infty([0, T]; \mathbb{R}^n)$ even though this is not the case for $(\dot{p}_n)_n$. This is actually the main difficulty for studying the behavior of (2.7) and for passing to the limit when $n \to +\infty$.

The boundedness of the sequence $(p_n)_n$ is related to the behavior of the sequence $(I_n)_n$ defined as

$$I_n := \int_0^T h_n(\varphi(x_n(t))) \, dt. \qquad (2.9)$$

As this integral will play a crucial role in the establishment of optimality conditions, passing to the limit into (2.7) when $n \to +\infty$, we devote the next session to the analysis of its properties.

## 3 Properties of the sequence of integrals $(I_n)_n$

We start by proving that $(I_n)_n$ is bounded if and only if $(p_n)_n$ is bounded in $L^\infty([0, T]; \mathbb{R}^n)$. Recall that the limiting path $x^*$ has a finite number of crossing times $(\tau_i)_{1 \le i \le r}$ (recall Assumption 2.1). Set

$$\delta := \min_{0 \le i \le r} (\tau_{i+1} - \tau_i),$$

<footnote n="6">⁶Since no terminal constraint appear in (2.6), one can directly take $p^0 = -1$ for the multiplier associated to the objective function $G_n$, i.e., only normal extremals occur.</footnote>
with the convention $\tau_0 := 0$, $\tau_{r+1} := T$, and define the subsets

$$ \mathcal{I}_{\eta} := \bigcup_{1 \le i \le r} [\tau_i - \eta, \tau_i + \eta] \quad ; \quad \mathcal{J}_{\eta} := [0, T] \setminus \mathcal{I}_{\eta}. $$

The following property will be used at several places.

**Property 3.1.** For all $\eta \in (0, \delta)$, there is $N \in \mathbb{N}$ such that for all $n \ge N$ and $t \in \mathcal{J}_{\eta}$, one has $h_n(\varphi(x_n(t))) = 0$.

*Proof.* Take $\eta \in (0, \delta)$. Since $(x_n)_n$ uniformly converges to $x^*$ and as $\varphi(x^*(t)) = 0$ if and only if $t \in \{\tau_1, \dots, \tau_r\}$, there are $N_1 \in \mathbb{N}$ and $\gamma > 0$ such that

$$ \forall n \ge N_1, \forall t \in \mathcal{J}_{\eta}, \quad |\varphi(x_n(t))| \ge \gamma. $$

Now, recall that both sequences $(a_n)_n$, $(b_n)_n$ defining the support of $h_n$ converge to zero, whence the result. $\square$

Next, we have the following equivalence between the boundedness of $(p_n)_n$ and $(I_n)_n$.

**Proposition 3.1.** The sequence $(p_n)_n$ is bounded in $L^\infty([0, T]; \mathbb{R}^n)$ if and only if $(I_n)_n$ is bounded in $\mathbb{R}_+$.

*Proof.* Since $p_n(T) = 0$ for every $n \in \mathbb{N}$, the boundedness of $(p_n)_n$ easily follows from the boundedness of $(I_n)_n$ using Gronwall's Lemma and the fact that $(x_n)_n$ and $(u_n)_n$ are bounded in $L^\infty([0, T]; \mathbb{R}^n)$ and $L^\infty([0, T]; \mathbb{R}^m)$ respectively.

Let us now assume that $(p_n)_n$ is bounded in $L^\infty([0, T]; \mathbb{R}^n)$. By (2.7) one has

$$ \int_0^T h_n(\varphi(x_n(t)))|\nabla\varphi(x_n(t))|^2 dt = \int_0^T \dot{p}_n(t) \cdot \nabla\varphi(x_n(t)) dt + \int_0^T D_x f(x_n(t), u_n(t))^{\top} p_n(t) \cdot \nabla\varphi(x_n(t)) dt. \quad (3.1) $$

From (H4) and using the uniform convergence of $(x_n)_n$ to $x^*$, there are $\eta \in (0, \delta)$, $N_1 \in \mathbb{N}$, and $\gamma' > 0$ such that

$$ \forall t \in \mathcal{I}_{\eta}, \forall n \ge N_1, \quad |\nabla\varphi(x_n(t))|^2 \ge \gamma'. $$

From property 3.1, there is $N \in \mathbb{N}$ such that for every $n \ge N$ and every $t \in \mathcal{J}_{\eta}$, one has $h_n(\varphi(x_n(t))) = 0$. It follows that for every $n \ge N_2 := \max(N, N_1)$, one has:

$$ \int_0^T h_n(\varphi(x_n(t)))|\nabla\varphi(x_n(t))|^2 dt \ge \gamma' \int_{\mathcal{I}_{\eta}} h_n(\varphi(x_n(t))) dt. \quad (3.2) $$

Now, from the hypotheses on $f$ and $\varphi$, there is $C \ge 0$ such that:

$$ \forall n \in \mathbb{N}, \forall t \in [0, T], \quad |D_x f(x_n(t), u_n(t))^{\top} p_n(t) \cdot \nabla\varphi(x_n(t))| \le C. $$

By an integration by parts, we obtain using the terminal condition $p_n(T) = 0$ for every $n \in \mathbb{N}$:

$$ \int_0^T \dot{p}_n(t) \cdot \nabla\varphi(x_n(t)) dt = -p_n(0)\nabla\varphi(x_0) - \int_0^T p_n(t) \cdot D_{xx}\varphi(x_n(t))\dot{x}_n(t) dt. $$

As the sequence $(x_n)_n$, $(\dot{x}_n)_n$, and $(p_n)_n$ are uniformly bounded, we deduce that there is $C' \ge 0$ such that for every $n \in \mathbb{N}$, one has

$$ \left| \int_0^T \dot{p}_n(t) \cdot \nabla\varphi(x_n(t)) dt \right| \le |p_n(0)| |\nabla\varphi(x_0)| + \int_0^T \|D_{xx}\varphi(x_n(t))\| |p_n(t)| |\dot{x}_n(t)| dt \le C' \quad (3.3) $$

Combining (3.1)-(3.2)-(3.3) then implies

$$ \forall n \ge N_2, \quad \gamma' \int_{\mathcal{I}_{\eta}} h_n(\varphi(x_n(t))) dt \le C T + C'. $$

We have thus proved that the sequence $(\int_{\mathcal{I}_n} h_n(\varphi(x_n(t))) dt)_n$ is bounded. Since $h_n(\varphi(x_n(t))) = 0$ for every $t \in \mathcal{J}_{\eta}$ and every $n \ge N$, the result follows. $\square$
This proposition will be used later on to obtain optimality conditions for (2.1) by letting $n \to +\infty$ in (2.7) and by reasoning by contradiction supposing that $(p_n)_n$ is unbounded in $L^\infty([0, T]; \mathbb{R}^n)$. We now aim at studying convergence properties of the sequence $(I_n)_n$. For $1 \le i \le r$ and $n \in \mathbb{N}$, define the integrals

$$ \tilde{I}_{i,n}(\eta) := \int_{\tau_i - \eta}^{\tau_i + \eta} h_n(\varphi(x_n(t))) \, dt, \quad \eta \in (0, \delta). $$

**Lemma 3.1.** Suppose that $(I_n)_n$ is bounded. Then, for any $i \in \{1, ..., r\}$, there exists $\ell_i \in \mathbb{R}_+$ such that for every $\eta \in (0, \delta]$, $\tilde{I}_{i,n}(\eta) \to \ell_i$ whenever $n \to +\infty$.

*Proof.* Fix $i \in \{1, ..., r\}$. Since $(I_n)_n$ is bounded so is $(\tilde{I}_{i,n}(\eta))_n$ for $\eta \in (0, \delta]$, hence, we may assume that there exists $\ell_i \in \mathbb{R}_+$ such that (up to a sub-sequence), $\tilde{I}_{i,n}(\delta) \to \ell_i$ when $n \to +\infty$. We can then write

$$ \tilde{I}_{i,n}(\delta) - \tilde{I}_{i,n}(\eta) = \int_{\tau_i - \delta}^{\tau_i - \eta} h_n(\varphi(x_n(t))) \, dt + \int_{\tau_i + \eta}^{\tau_i + \delta} h_n(\varphi(x_n(t))) \, dt. $$

Let then

$$ \gamma_{\eta} := \min_{t \in [\tau_i - \delta, \tau_i - \eta] \cup [\tau_i + \eta, \tau_i + \delta]} |\varphi(x^*(t))| > 0. $$

Recall that $(x_n)_n$ uniformly converges to $x^*$ when $n \to +\infty$. Thus, there exists $N \in \mathbb{N}$ such that:

$$ \forall n \ge N, \forall t \in [\tau_i - \delta, \tau_i - \eta] \cup [\tau_i + \eta, \tau_i + \delta], \quad |\varphi(x_n(t))| \ge \frac{\gamma_{\eta}}{2}. $$

Now, both sequence $(a_n)_n$ and $(b_n)_n$ converge to zero, hence there exists $N' \ge N$ such that

$$ \forall n \ge N', [a_n, b_n] \subset [-\frac{\gamma_{\eta}}{2}, \frac{\gamma_{\eta}}{2}]. $$

Because the support of $h_n$ is contained in $[a_n, b_n]$, we conclude that

$$ \forall n \ge N', \tilde{I}_{i,n}(\delta) = \tilde{I}_{i,n}(\eta). $$

This proves that $\tilde{I}_{i,n}(\eta) \to \ell_i$ when $n \to +\infty$. Since $\eta \in (0, \delta]$ is arbitrary, the result follows. $\square$

Thanks to this lemma, we can now show the following result which provides a kernel-type property<fnref n="7" /> of $(I_n)$ and that will be crucial hereafter to study the convergence of $(p_n)_n$.

**Proposition 3.2.** If $g : \mathbb{R}^n \to \mathbb{R}^n$ is a function of class $C^1$ and $(I_n)_n$ is bounded, then:

$$ \forall \varepsilon > 0, \exists \eta_i \in (0, \delta], \exists N \in \mathbb{N}, \forall n \ge N, \quad \left| \int_{\tau_i - \eta_i}^{\tau_i + \eta_i} h_n(\varphi(x_n(t)))g(x_n(t)) \, dt - \ell_i g(x^*(\tau_i)) \right| \le \varepsilon. \quad (3.4) $$

In addition, $\eta_i$ goes to zero as $\varepsilon \downarrow 0$.

*Proof.* For $\eta \in (0, \delta]$, one can write:

$$ \int_{\tau_i - \eta}^{\tau_i + \eta} h_n(\varphi(x_n(t)))g(x_n(t)) \, dt - \ell_i g(x^*(\tau_i)) = \underbrace{\int_{\tau_i - \eta}^{\tau_i + \eta} h_n(\varphi(x_n(t))) [g(x_n(t)) - g(x^*(t))] \, dt}_{\Lambda_n^1(\eta)} + \underbrace{\int_{\tau_i - \eta}^{\tau_i + \eta} h_n(\varphi(x_n(t))) [g(x^*(t)) - g(x^*(\tau_i))] \, dt}_{\Lambda_n^2(\eta)} + (\tilde{I}_{i,n}(\eta) - \ell_i)g(x^*(\tau_i)). $$

Let then $\varepsilon > 0$ be fixed and set $M := \sup_n I_n$. By continuity of $x^*$ at $t = \tau_i$, there exists $\eta_i > 0$ such that

$$ |g(x^*(t)) - g(x^*(\tau_i))| \le \frac{\varepsilon}{3M}, \quad t \in [\tau_i - \eta_i, \tau_i + \eta_i]. \quad (3.5) $$

<footnote n="7">⁷We refer to a classical property which asserts that given a sequence of mollifier $(f_n)_n$ defined over $[0, 1]$ and a continuous function $g : [0, 1] \to \mathbb{R}$, then $\int_0^1 f_n(t)g(t) \, dt \to g(0)$ when $n$ goes to infinity.</footnote>
Without any loss of generality, one can choose $\eta_i$ such that it goes to zero as $\varepsilon$ tends to 0 because $t \mapsto (g \circ x^*)(t)$ is Lipschitz continuous over $[0, T]$ (since $g$ is of class $C^1$ and $\dot{x}^*$ is bounded in $L^\infty([0, T]; \mathbb{R}^n)$).

Since for every $\eta \in (0, \eta_i]$ one has

$$ \forall n \in \mathbb{N}, |\Lambda_n^2(\eta)| \le \int_{\tau_i - \eta}^{\tau_i + \eta} h_n(\varphi(x_n(t)))|g(x^*(t)) - g(x^*(\tau_i))| dt \le \int_{\tau_i - \eta_i}^{\tau_i + \eta_i} h_n(\varphi(x_n(t)))|g(x^*(t)) - g(x^*(\tau_i))| dt, $$

we deduce that

$$ \forall n \in \mathbb{N}, \forall \eta \in (0, \eta_i], |\Lambda_n^2(\eta)| \le \frac{\varepsilon}{3}. \quad (3.6) $$

Now, the sequence $(x_n)_n$ uniformly converges to $x^*$ over $[0, T]$. Hence, there exists $N' \in \mathbb{N}$ such that for all $n \ge N'$, one has $|g(x_n(t)) - g(x^*(t))| \le \frac{\varepsilon}{3M}$ for every $t \in [0, T]$. This gives us the following property:

$$ \exists N' \in \mathbb{N}, \forall n \ge N', \forall \eta \in (0, \delta], |\Lambda_n^1(\eta)| \le \frac{\varepsilon}{3}. \quad (3.7) $$

The last step is to apply Lemma 3.1 for $\eta = \eta_i$ which provides the next property:

$$ \exists N'' \in \mathbb{N}, \forall n \ge N'', |(\tilde{I}_{i,n}(\eta_i) - \ell_i)g(x^*(\tau_i))| \le \frac{\varepsilon}{3}. \quad (3.8) $$

Let us set $N := \max(N', N'')$. Combining (3.6), (3.7) with $\eta = \eta_i$, and (3.8) then gives (3.4). $\square$

Let us underline that for an arbitrary sequence $(x_n)_n$ satisfying (2.2), which (up to a sub-sequence) converges strongly-weakly to a solution $\bar{x}$ of (2.2), then, the associated sequence of integrals $(I_n)_n$ is not necessarily bounded even if the limiting trajectory $\bar{x}$ has a transverse crossing time (see example below).

**Example 3.1.** Consider the scalar dynamics

$$ \dot{x}(t) = u(t) \quad \text{a.e. } t \in [0, 2], $$

where $u(t) \in [0, 1]$, together with the set $K := \mathbb{R}_-$ (associated with $\varphi(x) := x$). As nominal path, we consider $\bar{x}(t) := t - 1$, $t \in [0, 2]$. Observe that the function $\bar{\rho}(t) = \varphi(\bar{x}(t))$ is differentiable with $\bar{\rho}'(t) = 1 > 0$ for every $t \in [0, 2]$, thus $\tau = 1$ is a transverse crossing time. For this example, we suppose for convenience that the mollifier $(h_n)_n$ is such that $a_n = 0$ and $b_n > 0$ for every $n \in \mathbb{N}$. Let us denote by $c_n \in (a_n, b_n)$ the unique point at which $h_n$ achieves its maximum (one has $h_n(c_n) \to +\infty$ whenever $n \to +\infty$). Next, we consider the sequence of absolutely continuous function $(x_n)_n$ defined as

$$ \left\{ \begin{array}{ll} x_n(t) = \frac{1+c_n}{1-\xi_n}t - 1 & t \in [0, 1-\xi_n], \\ x_n(t) = c_n & t \in [1-\xi_n, 1+\xi'_n], \\ x_n(t) = t - 1 + \zeta_n & t \in [1+\xi'_n, 2], \end{array} \right. \quad (3.9) $$

where $(\zeta_n)_n$ converges to 0 when $n \to +\infty$, $(\xi_n)_n$ and $(\xi'_n)_n$ are with values in $(0, 1)$, and:

$$ \forall n \in \mathbb{N}, \quad \xi'_n + \zeta_n = c_n. \quad (3.10) $$

Equality (3.10) guarantees the continuity of $x_n$ at $t = 1 + \xi'_n$ for all $n \in \mathbb{N}$. Sequences $(\xi_n)_n$ and $(\xi'_n)_n$ will be made more precise hereafter and (3.10) allows to uniquely define the sequence $(\zeta_n)_n$.

Suppose now that $(\xi_n)_n$ and $(\xi'_n)_n$ are chosen such that $(\xi_n + \xi'_n)h_n(c_n) \to +\infty$ whenever $n \to +\infty$. Then, we can easily see that $I_n \to +\infty$. Indeed, $I_n$ can be written $I_n = I_n^1 + I_n^2 + I_n^3$ with

$$ I_n^1 := \int_0^{1-\xi_n} h_n(\varphi(x_n(t))) dt; \quad I_n^2 := \int_{1-\xi_n}^{1+\xi'_n} h_n(\varphi(x_n(t))) dt; \quad I_n^3 := \int_{1+\xi'_n}^2 h_n(\varphi(x_n(t))) dt. $$

Recall that $\varphi(x) = x$, thus by changing the integration variable $t$ into $s = \frac{1+c_n}{1-\xi_n}t - 1$, resp. $s := t - 1 + \zeta_n$ in $I_n^1$, resp. in $I_n^3$, we obtain that for every $n \in \mathbb{N}$, one has $I_n^1 \le \frac{1-\xi_n}{1+c_n}$ and $I_n^3 \le 1$. We deduce that $(I_n^3)_n$ is bounded and so is $(I_n^1)_n$ because $(c_n)_n$ and $(\xi_n)_n$ converge to zero. Now, for all $n \in \mathbb{N}$, we have

$$ I_n^2 = h_n(c_n)(\xi_n + \xi'_n), $$

which shows that $I_n \to +\infty$. In addition, we easily check that the sequence $(x_n)_n$ strongly-weakly converges to $\bar{x}$. First, one has

$$ \sup_{t \in [0,2]} |x_n(t) - \bar{x}(t)| \le \max(c_n + \xi_n, \zeta_n) \to 0, $$

whenever $n \to +\infty$. Second, one can verify that $(\dot{x}_n)_n$ converges a.e. to $\dot{\bar{x}}$ (actually for every $t \in [0, 2]\setminus\{1\}$) which is enough to ensure the weak convergence of $(\dot{x}_n)_n$ to $\dot{x}$ in $L^2([0, 2]; \mathbb{R})$.
Nevertheless, we shall see later on that this phenomenon does not occur whenever $(x_n)_n$ is a minimizing sequence obtained from (2.6). This is due to the application of Pontryagin's Principle that provides additional properties on the extremal $(x_n, p_n, u_n)$ preventing $(I_n)_n$ to blow up whenever every crossing time of the optimal path $x^*$ is transverse.

# 4 Optimality conditions for the time crisis problem

In this section, we give necessary optimality conditions without the HMP, i.e., by passing to the limit into the state-adjoint system satisfied by the extremal $(x_n, p_n, u_n)$. Let us start by giving a definition of a covector associated with Problem (2.1). Doing so, we define the Hamiltonian $H: \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}$ associated with (2.1) as

$$H(x, p, u) = p \cdot f(x, u) - 1_{K^c}(x).$$

**Definition 4.1.** Given a solution $(x^*, u^*)$ of (2.1) with $r$ crossing times, we say that a piecewise absolutely continuous function $p: [0, T] \to \mathbb{R}^n$ is a covector associated to $x^*$ if $p$ is absolutely continuous on $[0, T]\setminus\{\tau_1, \dots, \tau_r\}$ and satisfies the following conditions.

* The function $p$ fulfills the backward adjoint equation:

$$\left| \begin{array}{l} \dot{p}(t) = -D_x f(x^*(t), u^*(t))^{\top} p(t) \quad \text{a.e. } t \in [0, T], \\ p(T) = 0. \end{array} \right. \qquad (4.1)$$

* The Hamiltonian condition is fulfilled almost everywhere:

$$\forall u \in U, \ p(t) \cdot f(x^*(t), u) \le p(t) \cdot f(x^*(t), u^*(t)) \quad \text{a.e. } t \in [0, T]. \qquad (4.2)$$

* At every crossing time, $p$ admits left and right limits, i.e.,

$$\forall i \in \{1, \dots, r\}, \ \exists p(\tau_i^{\pm}) := \lim_{t \to \tau_i^{\pm}} p(t). \qquad (4.3)$$

* There exist $r$ positive numbers $\ell_1, \dots, \ell_r$ such that:

$$\forall i \in \{1, \dots, r\}, \quad p(\tau_i^{+}) - p(\tau_i^{-}) = \ell_i \nabla \varphi(x^*(\tau_i)). \qquad (4.4)$$

* The Hamiltonian is constant almost everywhere over $[0, T]$, i.e., there is $\tilde{H} \in \mathbb{R}$ such that

$$\tilde{H} = H(x(t), p(t), u(t)) = \max_{u \in U} H(x(t), p(t), u) = -1_{K^c}(x^*(T)) \quad \text{a.e. } t \in [0, T]. \qquad (4.5)$$

We call extremal of (2.1) any triple $(x^*, p, u^*)$ satisfying (2.2), (4.1)-(4.2), (4.4) and (4.5).

**Remark 4.1.** Equality (4.4) amounts to say that $p$ has a jump at $t = \tau_i$ in the normal cone to the set $K$ (meaning here that the jump is in the direction of $\nabla\varphi(x(\tau_i))$, being assumed that $K$ has a smooth boundary).

To establish optimality conditions for (2.1), we start by proving the convergence of $(p_n)_n$. Doing so, we proceed step by step.

**Lemma 4.1.** Suppose that $(p_n)_n$ is uniformly bounded in $L^\infty([0, T]; \mathbb{R}^n)$ and let $1 \le i \le r$, $\eta \in (0, \delta]$. Then, up to a sub-sequence, the pair $(x_n, p_n)_n$ strongly-weakly converges over $[\tau_i + \eta, \tau_{i+1} - \eta]$ to a solution of

$$\left| \begin{array}{l} \dot{x}^*(t) = f(x^*(t), u^*(t)) \\ \dot{p}(t) = -D_x f(x^*(t), u^*(t))^{\top} p(t) \end{array} \right. \quad \text{a.e. } t \in [\tau_i + \eta, \tau_{i+1} - \eta].$$

*Proof.* In view of Property 3.1, for $n$ large enough, the pair $(x_n, p_n)$ satisfies the system

$$\left| \begin{array}{l} \dot{x}_n(t) = f(x_n(t), u_n(t)) \\ \dot{p}_n(t) = -D_x f(x_n(t), u_n(t))^{\top} p_n(t) \end{array} \right. \quad \text{a.e. } t \in [\tau_i + \eta, \tau_{i+1} - \eta].$$

Let $t_0 \in [\tau_i + \eta, \tau_{i+1} - \eta]$. Since $(p_n)_n$ is bounded, we may assume that $(p_n(t_0))_n$ converges (up to a sub-sequence). Because $(x_n)_n$ uniformly converges to $x^*$, we deduce that $x_n(t_0) \to x^*(t_0)$. Using (H1)-(H2), the result of compactness of solutions of a control system (see, e.g., [11, Theorem 1.11]) yields the result. □
Next, we show that $(p_n)_n$ strongly-weakly converges over $[0, T]\setminus\{\tau_1, ..., \tau_r\}$.

**Lemma 4.2.** Suppose that $(p_n)_n$ is uniformly bounded in $L^\infty([0, T]; \mathbb{R}^n)$. Then, there exists a function $p: [0, T] \to \mathbb{R}^n$ absolutely continuous on $[0, T]\setminus\{\tau_1, ..., \tau_r\}$ satisfying

$$ \dot{p}(t) = -D_x f(x^*(t), u^*(t))^{\top} p(t) \quad \text{a.e. } t \in [0, T], \qquad (4.6) $$

such that for every $\eta \in (0, \delta)$, $(p_n)_n$ strongly-weakly converges to $p$ over $\mathcal{J}_\eta$.

*Proof.* Let $\eta \in (0, \delta)$. By using the previous lemma, for every $1 \le i \le r$, we obtain the existence of an absolutely continuous function $p_\eta$ defined over $\mathcal{J}_\eta$ and satisfying (4.6) over $\mathcal{J}_\eta$. We now argue that $p_\eta$ does not depend on $\eta$ by considering a sequence of positive numbers $(\eta_k)_k$ such that $\eta_k \downarrow 0$ which allows us to define $p_{\eta_k}$ for every $k \in \mathbb{N}$, as previously (over $\mathcal{J}_{\eta_k}$). We then obtain $p_{\eta_{k+1}}(t) = p_{\eta_k}(t)$ for every $t \in \mathcal{J}_{\eta_k}$ because $\mathcal{J}_{\eta_k} \subset \mathcal{J}_{\eta_{k+1}}$ for every $k \in \mathbb{N}$. This shows that we can then define a function $p: [0, T]\setminus\{\tau_1, ..., \tau_r\} \to \mathbb{R}^n$ without any ambiguity by the equality $p = p_\eta$ on every set $\mathcal{J}_\eta$, $\eta \in (0, \delta]$. By construction, $p$ does not depend on $\eta$, which is as wanted. $\square$

Let us now address the question of the constancy of the Hamiltonian along $(x^*, p, u^*)$ and the Hamiltonian condition (4.2).

**Lemma 4.3.** Suppose that $(p_n)_n$ is uniformly bounded in $L^\infty([0, T]; \mathbb{R}^n)$. Then, the function $p$ satisfies the Hamiltonian condition (4.2) and (4.5).

*Proof.* Recall that $x^*$ has $r$ crossing times $\tau_i$, $i = 1, ..., r$. Let $0 \le i \le r$ and $t_0 \in (\tau_i, \tau_{i+1})$ be a Lebesgue point of the measurable function $t \mapsto p(t) \cdot \dot{x}^*(t)$. From (2.8), we obtain for any $u \in U$ and $\nu > 0$ (small enough):

$$ \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p_n(t) \cdot f(x_n(t), u) \, dt \le \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p_n(t) \cdot \dot{x}_n(t) \, dt. $$

Now, $(p_n)_n$ and $(x_n)_n$ uniformly converge over $[t_0, t_0 + \nu]$ to $p$ and $x^*$ respectively. In addition, $(\dot{x}_n)_n$ weakly converges to $\dot{x}^*$ over $[t_0, t_0 + \nu]$. It follows that

$$ \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p(t) \cdot f(x^*(t), u) \, dt \le \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p(t) \cdot \dot{x}^*(t) \, dt. $$

Letting $\nu \downarrow 0$ then gives

$$ p(t_0) \cdot f(x(t_0), u) \le p(t_0) \cdot \dot{x}^*(t_0). $$

Since $u \in U$ is arbitrary and almost every point of $[0, T]$ is a Lebesgue point of $t \mapsto p(t) \cdot \dot{x}^*(t)$, we obtain (4.2).

We proceed similarly to show the constancy of $H$ over time, i.e., for showing (4.5). Since the Hamiltonian $H_n$ is autonomous for any $n \in \mathbb{N}$, one has

$$ \tilde{H}_n := \max_{u \in U} H_n(x_n(t), p_n(t), u) = -G_n(\varphi(x_n(T))) \quad \text{a.e. } t \in [0, T], $$

and as $G_n(\varphi(x_n(T))) \in [0, 1]$, $(\tilde{H}_n)_n$ converges, up to a sub-sequence, to some constant $\tilde{H} \in [-1, 0]$. Let $i \in \{0, ..., r\}$ and again, let $t_0 \in (\tau_i, \tau_{i+1})$ be a Lebesgue point of $t \mapsto p(t) \cdot \dot{x}^*(t)$. For $\nu > 0$ small enough, one has:

$$ \tilde{H}_n = \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p_n(t) \cdot \dot{x}_n(t) \, dt - \frac{1}{\nu} \int_{t_0}^{t_0+\nu} G_n(\varphi(x_n(t))) \, dt. $$

By letting $n \to +\infty$, we see that

$$ \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p_n(t) \cdot \dot{x}_n(t) \, dt \to \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p(t) \cdot \dot{x}^*(t) \, dt. $$

By the uniform convergence of $(x_n)_n$ over $[t_0, t_0 + \nu] \subset (\tau_i, \tau_{i+1})$, we also have

$$ \frac{1}{\nu} \int_{t_0}^{t_0+\nu} G_n(\varphi(x_n(t))) \, dt \to \frac{1}{\nu} \int_{t_0}^{t_0+\nu} 1_{K^c}(x^*(t)) \, dt, $$

when $n \to +\infty$ (this follows using the dominated convergence Theorem). We can then conclude that

$$ \tilde{H} = \frac{1}{\nu} \int_{t_0}^{t_0+\nu} p(t) \cdot \dot{x}^*(t) \, dt - \frac{1}{\nu} \int_{t_0}^{t_0+\nu} 1_{K^c}(x^*(t)) \, dt. $$
Now, letting $\nu \downarrow 0$ (recall that $t_0 \in (\tau_i, \tau_{i+1})$ for $i = 1, \dots, r$) gives

$$ \tilde{H} = p(t_0) \cdot \dot{x}(t_0) - \mathbb{1}_{K^c}(x(t_0)). $$

Since almost every point $t_0 \in [0, T]$ is a Lebesgue point of the map $t \mapsto p(t) \cdot \dot{x}^*(t)$, one has then

$$ H(x^*(t), p(t), u^*(t)) = \tilde{H} \quad \text{a.e. } t \in [0, T]. $$

By the Hamiltonian condition (4.2) and the continuity property of the map $t \mapsto \max_{u \in U} H(x^*(t), p(t), u)$ over $[0, T] \setminus \{\tau_1, \dots, \tau_r\}$, (4.5) is thus fulfilled (recall that $p(T) = 0$ and thus $\tilde{H} = -\mathbb{1}_{K^c}(x(T))$). $\square$

Thanks to the previous lemma, we can now give the main result of this section, namely that $(x^*, p, u^*)$ is an extremal of Problem (2.1).

**Theorem 4.1.** Suppose that the sequence of integrals $(I_n)_n$ is bounded or that the sequence $(p_n)_n$ is bounded in $L^\infty([0, T]; \mathbb{R}^n)$. Then, there exists a non-null covector $p : [0, T] \to \mathbb{R}^n$ associated with $x^*$ in the sense of Definition 4.1. In addition, the sequence $(p_n)_n$ strongly-weakly converges to $p$ over every compact subset of $[0, T] \setminus \{\tau_1, \dots, \tau_r\}$.

*Proof.* Recall from Proposition 3.1 that having the sequence of integrals $(I_n)_n$ bounded or the sequence $(p_n)_n$ uniformly bounded in $L^\infty([0, T]; \mathbb{R}^n)$ are equivalent. The existence of a function $p : [0, T] \setminus \{\tau_1, \dots, \tau_r\} \to \mathbb{R}^n$ satisfying (4.1)-(4.2)-(4.5) follows from the previous lemma. Thanks to Lemma 4.2, we also obtain the desired strong-weak convergence of $(p_n)_n$ on every subset $\mathcal{J}_\eta$ (with $0 < \eta < \delta$) of $[0, T]$, and thus on every compact subset of $[0, T] \setminus \{\tau_1, \dots, \tau_r\}$. Note that one has $p(T) = 0$ because $p_n(T) = 0$ for all $n \in \mathbb{N}$ and $p(T) = \lim_{n \to +\infty} p_n(T)$ (recall that $x^*(T) \notin \partial K$ since $\tau_r < T$ is the last crossing time).

Let us now show (4.3). Since $(p_n)_n$ is uniformly bounded in $L^\infty([0, T]; \mathbb{R}^n)$, there is $R \ge 0$ such that for every $n \in \mathbb{N}$ one has $\|p_n\|_{L^\infty([0,T]; \mathbb{R}^n)} \le R$. Since for every $t \in [0, T] \setminus \{\tau_1, \dots, \tau_r\}$, one has $p(t) = \lim_{n \to +\infty} p_n(t)$, we deduce that $\|p\|_{L^\infty([0,T]; \mathbb{R}^n)} \le R$. Now, fix $1 \le i \le r$ and observe that

$$ \dot{p}(t) = -D_x f(x^*(t), u^*(t))^{\top} p(t) \quad \text{a.e. } t \in (\tau_i, \tau_{i+1}). $$

Given $t_1, t_2 \in (\tau_i, \tau_{i+1})$, we can thus write:

$$ |p(t_2) - p(t_1)| = \left| \int_{t_1}^{t_2} -D_x f(x^*(t), u^*(t))^{\top} p(t) \, dt \right| \le A |t_1 - t_2|, $$

where $A := R \times \sup_{t \in [0,T]} |D_x f(x^*(t), u^*(t))^{\top}|$. This inequality shows that $p(\cdot)$ satisfies the Cauchy criterion at $t = \tau_i^+$ which proves that the right limit $p(\tau_i^+)$ exists. Similarly, one obtains the existence of a left limit $p(\tau_i^-)$. We can repeat this argumentation at every crossing time $\tau_i$ which gives (4.3).

Let us now prove the jump formula (4.4). Doing so, consider a sequence $(\varepsilon_k)_k$ such that $\varepsilon_k \downarrow 0$ and let us apply Proposition 3.2 with $\nabla\varphi$ in place of $g$ ($\varphi$ being of class $C^2$). For every $k \in \mathbb{N}$, there exist $\eta_k \in (0, \delta]$ and $N_k \in \mathbb{N}$ such that for every $n \ge N_k$, one has

$$ \left| \int_{\tau_i - \eta_k}^{\tau_i + \eta_k} h_n(\varphi(x_n(t))) \nabla \varphi(x_n(t)) \, dt - \ell_i \nabla \varphi(x^*(\tau_i)) \right| \le \varepsilon_k. $$

Notice that from Proposition 3.2, one has $\eta_k \to 0$ when $k \to +\infty$. Integrating (2.7) over $[\tau_i - \eta_k, \tau_i + \eta_k]$ yields

$$ \forall n \in \mathbb{N}, p_n(\tau_i + \eta_k) - p_n(\tau_i - \eta_k) = \int_{\tau_i - \eta_k}^{\tau_i + \eta_k} -D_x f(x_n(t), u_n(t))^{\top} p_n(t) \, dt + \int_{\tau_i - \eta_k}^{\tau_i + \eta_k} h_n(\varphi(x_n(t))) \nabla \varphi(x_n(t)) \, dt. $$

Now, $t \mapsto D_x f(x_n(t), u_n(t)) p_n(t)$ is uniformly bounded over $[0, T]$ (say by a constant $B \ge 0$). It follows that

$$ \forall n \ge N_k, \quad |p_n(\tau_i + \eta_k) - p_n(\tau_i - \eta_k) - \ell_i \nabla \varphi(x^*(\tau_i))| \le 2B\eta_k + \varepsilon_k. $$

First, we let $n$ go to infinity ($k$ being fixed) which gives:

$$ \forall k \in \mathbb{N}, \quad |p(\tau_i + \eta_k) - p(\tau_i - \eta_k) - \ell_i \nabla \varphi(x^*(\tau_i))| \le 2B\eta_k + \varepsilon_k. $$

Now, we let $k \to +\infty$ observing that $p(\tau_i \pm \eta_k) \to p(\tau_i^{\pm})$ and we obtain

$$ p(\tau_i^+) - p(\tau_i^-) = \ell_i \nabla \varphi(x^*(\tau_i)), $$
which is the desired property.

The last step is to show that for every $1 \le i \le r$, one has $\ell_i \ne 0$. Consider the map

$$h(t) := \max_{u \in U} H(x^*(t), p(t), u) \quad t \in [0, T] \setminus \{\tau_1, \dots, \tau_r\},$$

which is continuous on each time interval $(\tau_{i-1}, \tau_i)$. As $p$ admits left and right limits at each $\tau_i$, so is $h$. Consider $i \in \{1, \dots, r\}$ such that $x^*$ crosses $\partial K$ from $K$ to $K^c$ at $t = \tau_i$. One has then

$$h(\tau_i^-) = \max_{u \in U} p(\tau_i^-) \cdot f(x^*(t), u), \quad h(\tau_i^+) = \max_{u \in U} p(\tau_i^+) \cdot f(x^*(t), u) - 1.$$

If $\ell_i = 0$, one has $p(\tau_i^+) = p(\tau_i^-)$ and thus $h(\tau_i^+) - h(\tau_i^-) = -1$, which contradicts property (4.5). Similarly, if $x^*$ crosses $\partial K$ from $K^c$ to $K$ at $\tau_i$ with $\ell_i = 0$, one gets $h(\tau_i^+) - h(\tau_i^-) = 1$ and again a contradiction with (4.5). $\square$

Let us stress that this result does not involve Assumption (H') (to be found below in Section 5) which is about the transversality of $x^*$. Using the constancy of the Hamiltonian along an extremal, the jump formula can be also written as follows (see also [17]).

**Corollary 4.1.** Assume that the sequence of integrals $(I_n)_n$ is bounded. If $\dot{x}^*$ admits left and right derivative at a crossing time $\tau_i$, $i \in \{1, \dots, r\}$ such that $\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) \ne 0$ or $\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) \ne 0$, then the jump of the covector $p$ at $\tau_i$ can be written as:

$$p(\tau_i^+) = p(\tau_i^-) + \frac{\delta_i + p(\tau_i^-) \cdot (\dot{x}^*(\tau_i^-) - \dot{x}^*(\tau_i^+))}{\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+)} \nabla\varphi(x^*(\tau_i)), \quad \text{if } \nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) \ne 0, \quad (4.7)$$

or

$$p(\tau_i^-) = p(\tau_i^+) - \frac{\delta_i + p(\tau_i^+) \cdot (\dot{x}^*(\tau_i^-) - \dot{x}^*(\tau_i^+))}{\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-)} \nabla\varphi(x^*(\tau_i)), \quad \text{if } \nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) \ne 0, \quad (4.8)$$

where $\delta_i = +1$ resp. $\delta_i = -1$ if the crossing time $\tau_i$ is outward, resp. inward.

**Proof.** Let us write the conservation of the Hamiltonian in a right and left neighborhood of $\tau_i$. If $\tau_i$ is an outward crossing time, one has then:

$$p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) - 1 = p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+), \quad (4.9)$$

and $p(\tau_i^+) = p(\tau_i^-) + \ell_i \nabla\varphi(x^*(\tau_i))$. If $\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) \ne 0$, replacing $p(\tau_i^+)$ by this last expression in equation (4.9) raises

$$\ell_i = \frac{1 + p(\tau_i^-) \cdot (\dot{x}^*(\tau_i^-) - \dot{x}^*(\tau_i^+))}{\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+)}.$$

Similarly, if $\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) \ne 0$, replacing $p(\tau_i^-)$ by $p(\tau_i^+) - \ell_i \nabla\varphi(x^*(\tau_i))$ in equation (4.9) gives the equivalent expression

$$\ell_i = \frac{1 + p(\tau_i^+) \cdot (\dot{x}^*(\tau_i^-) - \dot{x}^*(\tau_i^+))}{\nabla\varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-)}.$$

If $\tau_i$ is an inward crossing time, one can easily check that this amounts to replace 1 by $-1$ in (4.9). $\square$

**Remark 4.2.** When every crossing time $\tau_i$ of the optimal solution $x^*$ is transverse, one can use the expression (4.8) to determine explicitly the jumps of $p$, recursively from the terminal time $T$ in a univocal way. Therefore, for any choice of sequences $(a_n)_n$, $(b_n)_n$ such that $(I_n)_n$ is bounded and $(x_n)_n$ converges to $x^*$ (up to a sub-sequence), one obtains the same numbers $\ell_i$. If not, the jumps are determined only implicitly from conditions (4.7) and (4.8).

# 5 Sufficient conditions for the boundedness of the sequence $(I_n)_n$

The aim of this section is to give sufficient conditions on the system that ensure the boundedness of $(I_n)_n$. In that case, necessary optimality conditions for an optimal path are guaranteed by Theorem 4.1. Given an optimal solution $(x^*, u^*)$ of Problem 2.1, let us introduce the following hypothesis (in the spirit of the Hybrid Maximum Principle that requires also a transverse assumption [17]).

(H') Every crossing time of $x^*$ is transverse.
This hypothesis excludes the cases where the optimal solution $x^*$ hits the boundary of $K$ tangentially, i.e.,

$$ \lim_{t \to \tau^+} \nabla \varphi(x(t)) \cdot \dot{x}(t) = 0 \quad \text{or} \quad \lim_{t \to \tau^-} \nabla \varphi(x(t)) \cdot \dot{x}(t) = 0, \qquad (5.1) $$

at a crossing time $\tau$. Actually, several cases could appear depending if both scalar products are zero in (5.1) or only one. As far as we know, the derivation of necessary conditions in this case has been very little considered in the literature (except in [4]) and remains a thorough open question. We shall see in Section 6 an example of optimal paths that possess non-transverse crossing times.

## 5.1 The transverse case

We start by the following result which covers the case where every crossing times of the optimal solution $x^*$ is transverse.

**Proposition 5.1.** *Under Hypothesis (H'), the sequence $(I_n)_n$ is bounded.*

*Proof.* Suppose by contradiction that $(I_n)_n$ is unbounded. Extracting a sub-sequence if necessary, we may assume that $I_n \to +\infty$ whenever $n \to +\infty$. Observe that the function $q_n := \frac{p_n}{I_n}$ satisfies the differential system

$$ \left| \begin{array}{l} \dot{q}_n(t) \\ q_n(T) \end{array} \right. = -D_x f(x_n(t), u_n(t))^{\top} q_n(t) + \tilde{h}_n(\varphi(x_n(t))) \nabla \varphi(x_n(t)) \quad \text{a.e. } t \in [0, T], $$

where $\tilde{h}_n(\sigma) := \frac{h_n(\sigma)}{I_n}$, $\sigma \in \mathbb{R}$. By this change of variable, one has obviously

$$ \forall n \in \mathbb{N}, \quad \int_0^T \tilde{h}_n(\varphi(x_n(t))) \, dt = 1, \qquad (5.2) $$

so, Proposition 3.1 implies that the sequence $(q_n)_n$ is uniformly bounded in $L^\infty([0, T]; \mathbb{R}^n)$. One can then repeat the same argumentation than in the proof of Theorem 4.1 on the sequence $(q_n)_n$, excepted the last point about the value of the Hamiltonian. Indeed, as the covector $p_n$ has been renormalized, we have no longer the value of the Hamiltonian equal to $-G_n(\varphi(x_n(T)))$. However, we obtain that there exists a piecewise absolutely continuous function $q: [0, T] \setminus \{\tau_1, \dots, \tau_r\} \to \mathbb{R}^n$ satisfying the following properties:

* Up to a sub-sequence, $(q_n)_n$ converges to $q$ on every compact set of $[0, T] \setminus \{\tau_1, \dots, \tau_r\}$.

* The function $q$ satisfies

$$ \left| \begin{array}{l} \dot{q}(t) \\ q(T) \end{array} \right. = -D_x f(x^*(t), u^*(t))^{\top} q(t) \quad \text{a.e. } t \in [0, T], $$

* The Hamiltonian condition is fulfilled almost everywhere:

$$ \forall u \in U, \quad q(t) \cdot f(x^*(t), u) \le q(t) \cdot f(x^*(t), u^*(t)) \quad \text{a.e. } t \in [0, T]. \qquad (5.3) $$

* For every $1 \le i \le r$, $q$ admits a limit at $\tau_i^{\pm}$, i.e., there exists $\lim_{t \to \tau_i^{\pm}} q(t)$.

* At every crossing time $\tau_i$, $1 \le i \le r$, there exists $\tilde{\ell}_i \in \mathbb{R}$ such that

$$ q(\tau_i^+) - q(\tau_i^-) = \tilde{\ell}_i \nabla \varphi(x^*(\tau_i)). \qquad (5.4) $$

Here, we can no longer guarantee that each scalar $\tilde{\ell}_i$ is non null. Nevertheless, Proposition 3.2 implies that for every $1 \le i \le r$ and every $\eta \in (0, \delta]$ one has

$$ \tilde{\ell}_i = \lim_{n \to +\infty} \int_{\tau_i - \eta}^{\tau_i + \eta} \tilde{h}_n(\varphi(x_n(t))) \, dt. \qquad (5.5) $$

Observe that

$$ \forall n \in \mathbb{N}, \quad \frac{\tilde{H}_n}{I_n} = q_n(t) \cdot f(x_n(t), u_n(t)) - \frac{G_n(\varphi(x_n(t)))}{I_n} \quad \text{a.e. } t \in [0, T]. $$
For every $t \in [0, T]$, one has $\frac{\tilde{H}_n}{I_n} \to 0$ and $-\frac{G_n(\varphi(x_n(t)))}{I_n} \to 0$ when $n \to \infty$ because $\tilde{H}_n \in [-1, 0]$ and $G_n(\varphi(x_n(t))) \in [0, 1]$ for every $n \in \mathbb{N}$ and every $t \in [0, T]$. We then obtain that the covector $q$ satisfies:

$$q(t) \cdot f(x^*(t), u^*(t)) = 0 \quad \text{a.e. } t \in [0, T], \qquad (5.6)$$

by considering Lebesgue points of the map $t \mapsto q(t) \cdot f(x^*(t), u^*(t))$ and repeating exactly the same argumentation as in the proof of Lemma 4.3.

We claim now that $q \neq 0$, i.e., $q(\cdot)$ is non-null over $[0, T]$. To show this property, it is enough to prove that there exists $1 \le i \le r$ such that $\tilde{\ell}_i \neq 0$. Suppose then by contradiction that for every $1 \le i \le r$, one has $\tilde{\ell}_i = 0$. By using Lemma 3.1 with $\tilde{h}_n$ in place of $h_n$, we obtain that

$$\forall i \in \{1, \dots, r\}, \forall \eta \in (0, \delta], \lim_{n \to +\infty} \int_{\tau_i - \eta}^{\tau_i + \eta} \tilde{h}_n(\varphi(x_n(t))) dt = 0, \qquad (5.7)$$

where $\eta \in (0, \delta]$ is fixed. From Property 3.1 one also has

$$\lim_{n \to +\infty} \int_{J_{\eta}} h_n(\varphi(x_n(t))) dt = \lim_{n \to +\infty} \int_{J_{\eta}} \tilde{h}_n(\varphi(x_n(t))) dt = 0. \qquad (5.8)$$

Combining (5.7) and (5.8) gives us a contradiction with (5.2), thus we have obtained that there is $1 \le i \le r$ such that $\tilde{\ell}_i \neq 0$.

To conclude the proof of the proposition, we will finally exhibit a contradiction involving optimality conditions (5.4) and (5.6) satisfied by the covector $q$. Fix $1 \le i \le r$ such that $\tilde{\ell}_i \neq 0$. First, using (5.3)-(5.6) at $t = \tau_i^-$, one has

$$q(\tau_i^-) \cdot \dot{x}^*(\tau_i^+) \le q(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) = 0.$$

Using (5.4), we get

$$q(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) - q(\tau_i^-) \cdot \dot{x}^*(\tau_i^+) = \tilde{\ell}_i \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+).$$

It follows that

$$\tilde{\ell}_i \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) = -q(\tau_i^-) \cdot \dot{x}^*(\tau_i^+) \ge 0$$

Because $\tilde{\ell}_i \neq 0$ and $\nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) \neq 0$, we deduce that

$$\tilde{\ell}_i \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) > 0. \qquad (5.9)$$

We now proceed with the same reasoning using (5.3)-(5.6) at $t = \tau_i^+$. We obtain

$$q(\tau_i^+) \cdot \dot{x}^*(\tau_i^-) \le q(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) = 0.$$

Again, thanks to (5.4), one has

$$q(\tau_i^+) \cdot \dot{x}^*(\tau_i^-) - q(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) = \tilde{\ell}_i \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-).$$

It follows that

$$\tilde{\ell}_i \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) = q(\tau_i^+) \cdot \dot{x}^*(\tau_i^-) \le 0.$$

Using again that $\tilde{\ell}_i \neq 0$ and $\nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) \neq 0$, we deduce that

$$\tilde{\ell}_i \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) < 0. \qquad (5.10)$$

We claim that (5.9)-(5.10) is a contradiction. Indeed, because $\tilde{\ell}_i$ is non-zero, (5.9)-(5.10) imply that the scalar products $\nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-)$ and $\nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+)$ are of opposite sign. Because at $t = \tau_i$, the trajectory crosses the boundary of $K$, we necessarily have that

$$\nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) > 0 \quad \text{and} \quad \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) > 0,$$

if $\tau_i$ is an outward crossing time, or

$$\nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^-) < 0 \quad \text{and} \quad \nabla \varphi(x^*(\tau_i)) \cdot \dot{x}^*(\tau_i^+) < 0,$$

if it is inward. This contradiction completes the proof of the proposition and shows that $(I_n)_n$ is necessarily a bounded sequence. $\square$

As a consequence, we recover the HMP of the literature in the transverse case, as stated below.

**Corollary 5.1.** Under Hypothesis (H'), there exists a non-null covector $p: [0, T] \to \mathbb{R}^n$ associated with $x^*$ in the sense of Definition 4.1.

*Proof.* Under Hypothesis (H'), the sequence $(I_n)_n$ is bounded, thanks to the previous proposition, and so is $(p_n)_n$ in $L^\infty([0, T]; \mathbb{R}^n)$ from Proposition 3.1. The result follows from Theorem 4.1. $\square$
## 5.2 A sufficient condition on the approximating sequence

Note that condition (H') is expressed on the optimal solution $x^*$, which is usually not known a priori. Instead, we give now conditions on the subsequence $(x_n)_n$, and not on the limiting solution $x^*$, that ensure the boundedness of the sequence of integrals $(I_n)_n$. For $n \in \mathbb{N}$, define the absolutely continuous function $\rho_n$ as:

$$ \rho_n(t) := \varphi(x_n(t)) \quad t \in [0, T]. $$

We know that for each $n \in \mathbb{N}$, $x_n$ is differentiable almost everywhere over $[0, T]$ and thus $\rho_n$ as well, so that:

$$ \dot{\rho}_n(t) = \nabla\varphi(x_n(t))\dot{x}_n(t) \quad \text{a.e. } t \in [0, T]. $$

In addition, $(\rho_n)_n$ is uniformly bounded in $L^\infty([0, T]; \mathbb{R})$ thanks to (H1) and (H3). Therefore, for $i \in \{1, ..., r\}$ and $n \in \mathbb{N}$, we can define:

$$ l_{i,n}^{+} := \limsup_{h \to 0} \ \underset{t \in [\tau_i - h, \tau_i + h]}{\text{ess sup}} \ \dot{\rho}_n(t) \quad ; \quad l_{i,n}^{-} := \liminf_{h \to 0} \ \underset{t \in [\tau_i - h, \tau_i + h]}{\text{ess inf}} \ \dot{\rho}_n(t). $$

**Remark 5.1.** In many optimal control problems, optimal solutions $x_n$ are piecewise $C^1$, and thus the function $\rho_n$ admits left and right derivatives at any $t \in (0, T)$. Then, the definition of the numbers $l_{i,n}^{\pm}$ simply involves the maximum and minimum of $\dot{\rho}_n(\tau_i^{\pm})$.

**Proposition 5.2.** If for every $1 \le i \le r$, one has

$$ \begin{aligned} & \liminf_{n \to +\infty} l_{i,n}^{-} > 0 \quad \text{if } \tau_i \text{ is an outward crossing time, or} \\ & \limsup_{n \to +\infty} l_{i,n}^{+} < 0 \quad \text{if } \tau_i \text{ is an inward crossing time,} \end{aligned} \quad (5.11) $$

then the sequence $(I_n)_n$ is bounded.

*Proof.* For $i = 1, ..., r$, set

$$ l_i = \liminf_{n \to +\infty} l_{i,n}^{-}, \quad \bar{l}_i = \limsup_{n \to +\infty} l_{i,n}^{+}, $$

and define the numbers $\eta_i^-$, $\eta_i^+$ ($i = 1, ..., r$) as

$$ \eta_i^+ = \eta_{i+1}^- := \frac{1}{2}(\tau_{i+1} - \tau_i) \quad 1 \le i \le r-1, $$

together with $\eta_1^- := \tau_1$ and $\eta_r^+ := T - \tau_r$. Also, for $i \in \{1, ..., r\}$, let us define the integrals:

$$ \tilde{I}_{i,n}^{-}(\eta) := \int_{\tau_i - \eta}^{\tau_i} h_n(\varphi(x_n(t))) \, dt \quad ; \quad \tilde{I}_{i,n}^{+}(\eta) := \int_{\tau_i}^{\tau_i + \eta} h_n(\varphi(x_n(t))) \, dt. $$

One has clearly

$$ I_n = \sum_{i=1}^{r} \left[ \tilde{I}_{i,n}^{-}(\eta_i^{-}) + \tilde{I}_{i,n}^{+}(\eta_i^{+}) \right]. $$

We show now that for any $i \in \{1, ..., r\}$ such that $\tau_i$ is an outward crossing time, the sequence $(\tilde{I}_{i,n}^{-}(\eta_i^{-}))_n$ is bounded. Observe first that one has:

$$ \begin{aligned} l_i^{-} \tilde{I}_{i,n}^{-}(\eta_i^{-}) &= \int_{\tau_i - \eta_i^{-}}^{\tau_i} h_n(\rho_n(t))(l_i - \dot{\rho}_n(t)) dt + \int_{\tau_i - \eta_i^{-}}^{\tau_i} h_n(\rho_n(t))\dot{\rho}_n(t) dt \\ &= \int_{\tau_i - \eta_i^{-}}^{\tau_i} h_n(\rho_n(t))(l_i - \dot{\rho}_n(t)) dt + G_n(\varphi(x_n(\tau_i))) - G_n(\varphi(x_n(\tau_i - \eta_i^{-}))). \end{aligned} \quad (5.12) $$

Let $\varepsilon \in (0, l_i)$. We claim that there exist $\zeta_i \in (0, \eta_i^{-})$ and $N > 0$ such that

$$ \forall n \ge N, \quad l_i < \dot{\rho}_n(t) + \varepsilon \quad \text{a.e. } t \in [\tau_i - \zeta_i, \tau_i]. \quad (5.13) $$

Indeed, by definition of $l_i$, there exists $N \in \mathbb{N}$ such that for every $n \ge N$ one has

$$ l_i \le l_{i,n}^{-} + \frac{\varepsilon}{2}. $$
We also have for every $h > 0$

$$ \forall n \in \mathbb{N}, \quad \underset{s \in [\tau_i - h, \tau_i + h]}{\operatorname{ess\,inf}} \dot{\rho}_n(s) \le \dot{\rho}_n(t) \quad \text{a.e. } t \in [\tau_i - h, \tau_i + h]. $$

By definition of $l_{i,n}^-$, we deduce that there exists $\zeta_i \in (0, \eta_i^-)$ such that

$$ l_{i,n}^- \le \underset{s \in [\tau_i - \zeta_i, \tau_i + \zeta_i]}{\operatorname{ess\,inf}} \dot{\rho}_n(s) + \frac{\varepsilon}{2}, $$

and thus, we obtain (5.13). One can then write

$$ \begin{aligned} \int_{\tau_i - \eta_i^-}^{\tau_i} h_n(\rho_n(t))(\underline{l}_i - \dot{\rho}_n(t)) dt &< \varepsilon \int_{\tau_i - \zeta_i}^{\tau_i} h_n(\rho_n(t)) dt + \int_{\tau_i - \eta_i^-}^{\tau_i - \zeta_i} h_n(\rho_n(t))(\underline{l}_i - \dot{\rho}_n(t)) dt \tag{5.14} \\ &< \varepsilon \tilde{I}_{i,n}^-(\eta_i^-) + \int_{\tau_i - \eta_i^-}^{\tau_i - \zeta_i} h_n(\rho_n(t))(\underline{l}_i - \dot{\rho}_n(t)) dt. \end{aligned} $$

Observe now that the scalar $\mu$ defined as

$$ \mu := \min_{t \in [\tau_i - \eta_i^-, \tau_i - \zeta_i]} \varphi(x^*(t)), $$

is such that $\mu < 0$ since $x^*(t)$ belongs to $K$ for $t \in [\tau_i - \eta_i^-, \tau_i - \zeta_i]$. From the uniform convergence of the sequence $(x_n)_n$ to $x^*$ over $[0, T]$ and since $\varphi$ is continuous, there exists $N' \ge N$ such that one has

$$ \varphi(x_n(t)) = \rho_n(t) < -\mu/2, $$

for any $n > N'$ and every $t \in [\tau_i - \eta_i^-, \tau_i - \zeta_i]$. From the convergence of the sequence $(a_n)_n$ to 0 (recall (H5)), there exists $\bar{N} \ge N'$ such that one has $a_n > -\mu/2$ for any $n > \bar{N}$. Since the support of $h_n$ is contained in $[a_n, b_n]$, we deduce that

$$ \forall n > \bar{N}, \forall t \in [\tau_i - \eta_i^-, \tau_i - \zeta_i], h_n(\rho_n(t)) = 0. \tag{5.15} $$

Finally, from (5.12), (5.14) and (5.15), one obtains

$$ \forall n > \bar{N}, \quad \underline{l}_i \tilde{I}_{i,n}^-(\eta_i^-) < \varepsilon \tilde{I}_{i,n}^-(\eta_i^-) + 1. $$

This shows that the sequence $(\tilde{I}_{i,n}^-(\eta_i^-))_n$ is bounded. For $i \in \{1, ..., r\}$ such that $\tau_i$ is an inward crossing time, we proceed in the same way to show that the sequence $(\tilde{I}_{i,n}^-(\eta_i^-))_n$ is bounded, considering the number $\bar{l}_i < 0$. The proof of boundedness of the sequences $(\tilde{I}_{i,n}^+(\eta_i^+))_n$ is analogous. $\square$

From a practical point of view, $x^*$ and its crossing times $\tau_i$ are known only approximately. Whenever $\rho_n$ admits left and right derivatives, condition (5.11) is merely guaranteed whenever $(\dot{\rho}_n(t^+))_n$ is bounded from below by a positive number, or bounded from above by a negative number, locally at each crossing time $\tau_i$, $1 \le i \le r$.

## 5.3 A reciprocal property

We have seen previously that under (H'), the sequence $(I_n)_n$ is bounded as well as under condition (5.11) which is a sufficient condition (expressed) on the approximated sequence $(x_n)_n$. Thanks to these conditions, we obtained optimality conditions for an optimal solution $x^*$ (under the assumption that it has a finite number of crossing times). We now would like to address the converse question, namely, what can be said about $x^*$ whenever the sequence $(I_n)_n$ is bounded? In that case, we can prove the following result.

**Proposition 5.3.** Suppose that the sequence $(I_n)_n$ is bounded and let $\tau_i$ be a crossing time of $x^*$ such that $\dot{x}^*(\tau_i^+)$ exist. Then, if $\tau_i$ is an outward, resp. inward, crossing time, one has $\dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x(\tau_i)) \ne 0$, resp. $\dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x(\tau_i)) \ne 0$.

*Proof.* Since $(I_n)_n$ is bounded, there exists a (non-null) covector $p$ as in Definition 4.1 (see Theorem 4.1). Consider now an outward crossing time $\tau_i$. The conditions satisfied by the extremal $(x^*, p, u^*)$ imply a jump of $p$ at $t = \tau_i$ and the constancy of the Hamiltonian as in Definition 4.1. These conditions becomes

$$ \begin{cases} p(\tau_i^+) = p(\tau_i^-) + \ell_i \nabla\varphi(x^*(\tau_i)), \\ p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) = p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) - 1. \end{cases} $$
The Hamiltonian condition at $t = \tau_i$ also gives us the following inequalities:

$$ \begin{aligned} & \left| p(\tau_i^-) \cdot f(x^*(\tau_i), u) \le p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-), \right. \\ & \left. p(\tau_i^+) \cdot f(x^*(\tau_i), u) \le p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+), \right. \end{aligned} $$

for every $u \in U$. Suppose by contradiction that $\dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) = 0$. It follows that:

$$ p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) = p(\tau_i^-) \cdot \dot{x}^*(\tau_i^+) $$

Using the Hamiltonian condition, we obtain:

$$ p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) = p(\tau_i^-) \cdot \dot{x}^*(\tau_i^+) \le p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) = p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) - 1, $$

which is a contradiction. It follows that $\dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) \ne 0$ as was to be proved.

In the case where $\tau_i$ is an inward crossing time, we proceed in the same way supposing by contradiction that $\dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) = 0$. In that case, the constancy of the Hamiltonian gives us:

$$ p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) - 1 = p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+). $$

By a similar computation, we obtain using that $\dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) = 0$:

$$ p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) = p(\tau_i^+) \cdot \dot{x}^*(\tau_i^-) \le p(\tau_i^+) \cdot \dot{x}^*(\tau_i^+) = p(\tau_i^-) \cdot \dot{x}^*(\tau_i^-) - 1. $$

This is a contradiction, which ends the proof.

<figure><img src="image_3.png" /></figure>

This proposition shows that if $(I_n)_n$ is bounded, then, at every crossing time $\tau_i$ of $x^*$ for which $\dot{x}(\tau_i^\pm)$ exists, the trajectory is always transverse “at the exterior” of $K$, i.e., we can consider the following cases:

* case 1: if $\tau_i$ is an outward crossing time, then $\dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) > 0$;

* case 2: if $\tau_i$ is an inward crossing time, then $\dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) < 0$.

In both cases, we can only say that $\dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) \ge 0$ (case 1) or $\dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) \le 0$ (case 2). If this scalar product is null, we shall say that the crossing time is one-sided transverse. In other words, if $\tau_i$ is one-sided transverse, then, one has:

$$ \dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) = 0 \quad \text{and} \quad \dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) > 0 \qquad (5.16) $$

in case 1 and

$$ \dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) > 0 \quad \text{and} \quad \dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) = 0, \qquad (5.17) $$

in case 2. Case 1 will be illustrated in Section 6. The previous proposition also allows us to find an interesting property about the sequence $(p_n)_n$ whenever there is $i \in \{1, ..., r\}$ such that $\dot{x}^*(\tau_i^\pm)$ exist and

$$ \dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) = \dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) = 0. \qquad (5.18) $$

In that case, we shall say that the crossing time $\tau_i$ is two-sided non transverse (i.e., the trajectory $x^*$ enters and leaves $K$ at $\tau_i$ tangentially).

**Corollary 5.2.** Suppose that there exists $i \in \{1, ..., r\}$ such that $\tau_i$ is a two-sided non transverse crossing time of $x^*$. Then, the sequence $(p_n)_n$ is not bounded in $L^\infty([0, T] ; \mathbb{R}^n)$.

*Proof.* Suppose by contradiction that $(p_n)_n$ is bounded in $L^\infty([0, T] ; \mathbb{R}^n)$. Proposition 3.1 then implies that the sequence $(I_n)_n$ is bounded. By Proposition 5.3, we obtain that either $\dot{x}^*(\tau_i^+) \cdot \nabla\varphi(x^*(\tau_i)) \ne 0$ or $\dot{x}^*(\tau_i^-) \cdot \nabla\varphi(x^*(\tau_i)) \ne 0$ depending if $\tau_i$ is an inward or an outward crossing time. This contradicts (5.18) which ends the proof. □

<figure><img src="image_4.png" /></figure>

**Remark 5.2.** When any optimal solution $x^*$ fulfills Hypothesis (H'), then the sequence $(I_n)_n$ is bounded for any choice of the sequences $(a_n)_n$, $(b_n)_n$, accordingly to Proposition 5.1. If not, one cannot guarantee a priori that the sequence of $(I_n)_n$ is bounded. However, we show in the example of section 6 that for the one-sided transverse case, a right choice of $(a_n)_n$, $(b_n)_n$ allows to obtain the boundedness of $(I_n)_n$ and thus the necessary optimality conditions.
# 6 Example of an optimal path with a non-transverse crossing time

In this section, we provide an example for which an optimal solution of the time of crisis is non transverse. But still, for this solution, we can write optimality conditions using Theorem 4.1 even though the optimal path possesses a non-transverse crossing time.

We consider the time crisis problem over a finite interval $[0, T]$

$$ \min_{u(\cdot)} \int_{0}^{T} \mathbb{1}_{K^c}(x(t)) \, dt, \qquad (6.1) $$

for the controlled dynamics $x(\cdot)$ in the plane

$$ \left\{ \begin{array}{ll} \dot{x}_1 = 2 - u, & x_1(0) = 0, \\ \dot{x}_2 = 4 \cos^2\left(\frac{\pi}{2}x_1\right) + 4u \sin^2\left(\frac{\pi}{2}x_1\right), & x_2(0) = 0, \end{array} \right. \quad u(\cdot) \in [0, 1], \qquad (6.2) $$

and the set

$$ K := \{x \in \mathbb{R}^2; x_2 \in (-\infty, 1] \cup [5, +\infty)\}, $$

that can be written as $K = \{x \in \mathbb{R}^2; \varphi(x_2) \le 0\}$ where

$$ \varphi(y) := -(y-1)(y-5), \quad y \in \mathbb{R}. $$

Let us define the myopic$^8$ feedback strategy as

$$ \psi^*(x) := \begin{cases} 0, & x \in K \\ 1, & x \notin K \end{cases} \qquad (6.3) $$

One can straightforwardly check that the solution $x^*$ generated by the myopic strategy is given by the following expression

$$ x_1^*(t) = \begin{cases} 2t & t \in [0, \tau_1^*], \\ t + \frac{1}{2} & t \in [\tau_1^*, \tau_2^*], \\ 2t - 1 & t \ge \tau_2^* \end{cases}, \quad x_2^*(t) = \begin{cases} 2t + \frac{1}{\pi} \sin(2\pi t) & t \in [0, \tau_1^*], \\ 4t - 1 & t \in [\tau_1^*, \tau_2^*], \\ 2t + \frac{1}{\pi} \sin(\pi(2t-1)) + 2 & t \ge \tau_2^* \end{cases}, \qquad (6.4) $$

with $\tau_1^* = \frac{1}{2}$ and $\tau_2^* = \frac{3}{2}$. Note that $x^*(t) \in K^c$ for $t \in (\tau_1^*, \tau_2^*)$ and that $\tau_2^* - \tau_1^* = 1$ (see Fig. 2). Also, we set

$$ u^*(t) := \psi(x^*(t)) \quad \text{a.e. } t \in [0, T]. \qquad (6.5) $$

We shall next prove that $u^*$ is optimal for any $T > 0$. Before proving this fact, we give a useful property related to the controlled dynamics (6.2).

**Lemma 6.1.** For any fixed $t_f > 0$, the control $u(\cdot) = 0$ is optimal for the auxiliary optimal control problem

$$ \min_{u(\cdot)} x_2(t_f), $$

where $x(\cdot)$ denotes the unique solution of (6.2) associated with an admissible control $u(\cdot)$.

*Proof.* We are in a position to apply the PMP on this Mayer optimal control problem. The Hamiltonian associated with this problem is

$$ H_{aux}(x, p, u) := p_1(2-u) + 4p_2 \left( \cos^2\left(\frac{\pi}{2}x_1\right) + u \sin^2\left(\frac{\pi}{2}x_1\right) \right), $$

and the adjoint equations are

$$ \begin{cases} \dot{p}_1 = -\partial_{x_1} H_{aux} = 2p_2\pi \sin(\pi x_1)(1-u) & p_1(t_f) = 0, \\ \dot{p}_2 = -\partial_{x_2} H_{aux} = 0 & p_2(t_f) = -1. \end{cases} $$

Notice that $x_1$ is always increasing. The switching function is (since $p_2$ is constant):

$$ \phi(t) := -p_1(t) - 4 \sin^2 \left( \frac{\pi}{2} x_1(t) \right), $$

<footnote>$^8$The terminology “myopic” means that the control strategy only considers the set $K$ and its complementary and does not involve the dynamics.</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 2: Trajectory $x^*$ provided by the myopic strategy starting at (0,0) (the set K is depicted in gray). The first crossing time is non-transverse (it is one-sided transverse of case 1, see (5.16)) whereas the second one is transverse.</figcaption></figure>

and a straightforward computation gives

$$ \dot{\phi}(t) = -2\pi \sin(\pi x_1(t)). $$

Since $x_1$ is non constant with bounded derivative, we deduce that no singular arc occurs along an optimal path and that the number of switching times is at most finite. In addition, if $u = 1$ a.e. on some time interval $[t_f - \delta, t_f]$ was optimal, one should have $p_1 = 0$ and thus $\phi < 0$ a.e. on this interval, which is a contradiction with the maximization of the Hamiltonian w.r.t. $u$. Thus, integrating backward $\dot{x}_1$, $\dot{p}_1$ from $t_f$ with the control $u = 0$ gives $x_1(t) = 2(t - t_f) + x_1(t_f)$, $p_1(t) = \cos(\pi x_1(t)) - \cos(\pi x_1(t_f))$. We can conclude that $\phi$ satisfies

$$ \phi(t) = \cos(\pi x_1(t)) + \cos(\pi x_1(t_f)) - 2, \quad t \in [0, t_f]. $$

Thus, $\phi(t)$ is negative for a.e. $t \le t_f$ whence the result.

□

Let then $\tilde{x}$ be the unique solution of (6.2) with $u = 0$. One can straightforwardly check that

$$ \tilde{x}_1(t) = 2t, \quad \tilde{x}_2(t) = 2t + \frac{1}{\pi} \sin(2\pi t), $$

for $t \ge 0$ and that one has $\tilde{x}_2(\frac{5}{2}) = 5$. Lemma 6.1 also implies that any admissible path of (6.2) satisfies

$$ \forall t \in [0, T], \quad x_2(t) \ge \tilde{x}_2(t) \qquad (6.6) $$

(to show this property it is enough to argue by contradiction and to use Lemma 6.1). We now have the following optimality result about (6.1).

**Proposition 6.1.** For any fixed $T > 0$, an optimal control of (6.1) is given by (6.3).

*Proof.* Observe that any admissible solution $x$ of (6.2) is such that $x_2(\cdot)$ is increasing. Suppose first that $T \le \tau_1^*$. Then the time crisis of $x^*$ over $[0, T]$ (that is $\int_0^T 1_{K^c}(x^*(t))dt$) is null and $x^*$ is thus optimal. Suppose now that $T > \tau_1^*$. Then, the time crisis of $x^*$ over $[0, T]$ is $\min(1, T - \tau_1^*)$. Take now an admissible solution $x$ of (6.2). From (6.6), we deduce that there exists $\tau_1 \le \tau_1^*$ such that $x_2(\tau_1) = 1$. Moreover, one has $\dot{x}_2(t) \le 4$ for a.e. $t$, which implies $x_2(\tau_1 + 1) \le 5$. Therefore the time crisis of $x$ over $[0, T]$ is $\min(1, T - \tau_1)$, and as $\tau_1 \le \tau_1^*$, we conclude that $\min(1, T - \tau_1) \ge \min(1, T - \tau_1^*)$ implying the result. □

Thanks to this result, we deduce the following properties.

* For $T > \frac{5}{2}$, any admissible solution $x$ satisfies $x_2(T) \ge \tilde{x}_2(T) > \tilde{x}_2(\frac{5}{2}) = 5$, and thus has to cross the boundary of $K$ exactly two times at $\tau_1 < \tau_2$ such that $x_2(\tau_1) = 1$, $x_2(\tau_2) = 5$.
* From (6.4), the crossing time $\tau_1^*$ of the optimal solution $x^*$ is non transverse because one has $\dot{x}_2(\tau_1^{*-}) = 0$ (and $\dot{x}_2(\tau_1^{*+}) = 4$), while the crossing time $\tau_2^*$ is transverse with $\dot{x}_2(\tau_2^{*-}) = \dot{x}_2(\tau_2^{*+}) = 4$.

Therefore, Corollary 5.1 does not apply, but we shall next see that Theorem 4.1 can be applied. We then need to verify that the sequence $(I_n)_n$ is bounded. Doing so, we consider the regularized problem $\mathcal{P}_n$ for the following sequence of functions $(G_n)_n$ that fulfills Hypothesis H5 (see Fig. 3):

$$G_n(\sigma) := \begin{cases} 0 & \text{if } \sigma \le 0, \\ 3(n\sigma)^2 - 2(n\sigma)^3 & \text{if } 0 < \sigma < \frac{1}{n}, \\ 1 & \text{if } \sigma \ge \frac{1}{n}. \end{cases}$$

Take $T > 0$ and let $x(\cdot)$ be an admissible solution of (6.2). From the expression of $G_n$, one has for every $n \in \mathbb{N}^*$:

<figure><img src="image_6.png" /><figcaption>Figure 3: Graphs of the functions $G_n$ for $n \in \{1, ..., 10\}$ (Fig. left) and of their derivatives (Fig. right)</figcaption></figure>

$$J_n(x) := \int_0^T G_n(\varphi(x_2(t))) dt = \int_{\tau_1}^{\tau_2} G_n(\varphi(x_2(t))) dt.$$

Since the map $t \mapsto x_2(t)$ is increasing and $\dot{x}_2(t) \le 4$ for a.e. $t \ge 0$, one obtains the inequality

$$J_n(x(\cdot)) \ge \frac{1}{4} \int_1^5 G_n(\varphi(x_2)) dx_2,$$

with equality whenever $x = x^*$. We thus conclude that $x^*$ is also optimal for the regularized problem over $[0, T]$ (with $T > \frac{5}{2}$) for every $n \in \mathbb{N}^*$. We now continue the study of this example showing that Theorem 4.1 can be applied. The next step consists in showing that the sequence of integrals

$$I_n := \int_0^T G'_n(\varphi(x_2^*(t))) dt$$

is bounded. Observe that one has

$$I_n = \int_{\tau_1^*}^{\tau_2^*} G'_n(\varphi(x_2^*(t))) dt = \int_{\frac{1}{2}}^{\frac{3}{2}} G'_n(-(4t-2)(4t-6)) dt.$$

We now proceed to a change of variable in the integral $I_n$. Consider the function $t \mapsto z(t) = -(4t-2)(4t-6)$ which is increasing from 0 to 4 for $t \in [\frac{1}{2}, 1]$ with $z'(t) = 8\sqrt{4-z(t)}$, and decreasing from 4 to 0 for $t \in [1, \frac{3}{2}]$ with $z'(t) = -8\sqrt{4-z(t)}$. One can then write

$$I_n = \int_0^4 \frac{G'_n(z)}{8\sqrt{4-z}} dz + \int_4^0 \frac{G'_n(z)}{-8\sqrt{4-z}} dz = \frac{1}{4} \int_0^4 \frac{G'_n(z)}{\sqrt{4-z}} dz.$$

Note that for any $n \in \mathbb{N}^*$, one has $G'_n(z) = 0$ for $z \ge 1$ which gives

$$\int_0^4 \frac{G'_n(z)}{\sqrt{4-z}} dz = \int_0^1 \frac{G'_n(z)}{\sqrt{4-z}} dz \le \frac{1}{\sqrt{3}} \int_0^1 G'_n(z) dz = \frac{G_n(1) - G_n(0)}{\sqrt{3}} = \frac{1}{\sqrt{3}}.$$
We deduce that the sequence $(I_n)_n$ is bounded as wanted, so Theorem 4.1 does apply for $x^*$ even if $\tau_1^*$ is non-transverse. Let us next verify the existence of a covector associated with $(x^*, u^*)$ as in Definition 4.1 (recall that its existence is provided by Theorem 4.1). Doing so, recall that $u^*$ is given by (6.5), that $x^*$ has two crossing times $\tau_1^*, \tau_2^*$ such that $0 < \tau_1^* < \tau_2^* < T$. Also, let $H$ be the Hamiltonian associated with (6.1):

$$H(x, p, u) := p_1(2-u) + 4p_2 \left( \cos^2\left(\frac{\pi}{2}x_1\right) + u \sin^2\left(\frac{\pi}{2}x_1\right) \right) - 1_{K^c}(x) \quad (6.7)$$

The adjoint system writes

$$\begin{cases} \dot{p}_1 = -\partial_{x_1} H = 2p_2\pi \sin(\pi x_1^*(t))(1 - u^*(t)) & p_1(T) = 0, \\ \dot{p}_2 = -\partial_{x_2} H = 0 & p_2(T) = 0, \end{cases} \quad (6.8)$$

and jumps of $p$ at crossing times $\tau_i^*$ must satisfy

$$\left\{ \begin{aligned} p_1(\tau_i^{*+}) - p_1(\tau_i^{*-}) &= 0, \\ p_2(\tau_i^{*+}) - p_2(\tau_i^{*-}) &= l_i \varphi'(x_2(\tau_i^*)) \quad \text{with } l_i > 0. \end{aligned} \right. \quad (6.9)$$

From (6.8), we first get $p = 0$ over $(\tau_2^*, T]$. Note that the Hamiltonian (6.7) is equal to 0 on this interval. At the crossing time $\tau_2^*$, the adjoint $p_2$ jumps from $p_2(\tau_2^{*-}) > 0$ to $p_2(\tau_2^{*+}) = 0$ accordingly to (6.9). Over the interval $(\tau_1^*, \tau_2^*)$, $p_2$ is constant equal to $p_2(\tau_2^{*-})$ and $p_1 = 0$ because $u^* = 1$ on this interval. The value $p_2(\tau_2^{*-}) = \frac{1}{4}$ is obtained from the conservation of the Hamiltonian (6.7) which has also to be null on $(\tau_1^*, \tau_2^*)$.

At the crossing time $\tau_1^*$, $p_2$ jumps from $p_2(\tau_1^{*-}) < p_2(\tau_1^{*+})$ to $p_2(\tau_1^{*+}) = \frac{1}{4}$ accordingly to (6.9). Then, on the interval $[0, \tau_1^*)$, $p = 0$ is a solution to the adjoint system that satisfying additionally the jump condition and the conservation of the Hamiltonian.

We can then conclude that the covector $p$ is given over $[0, T]$ by the expression

$$p_1(t) = 0, \ t \in [0, T] \quad ; \quad p_2(t) = \begin{cases} 0, & t \in [0, \tau_1^*), \\ \frac{1}{4}, & t \in (\tau_1^*, \tau_2^*), \\ 0, & t \in (\tau_1^*, T]. \end{cases} \quad (6.10)$$

Finally, one can straightforwardly check that the control $u^*$ verifies the maximization of the Hamiltonian (6.7) almost everywhere over $[0, T]$. Consequently, we have checked that the triple $(x^*, p, u^*)$ satisfies the necessary optimality conditions given by Theorem 4.1.

We conclude this section by giving an alternative approach showing that the sequence $(p_n)_n$ of covectors associated with the regularized problem $\mathcal{P}_n$ (whose optimal solution is $x^*$) is bounded. Doing so, fix $n \in \mathbb{N}$. The Hamiltonian associated with $\mathcal{P}_n$ becomes

$$H_n(x, p_n, u) := p_{1,n}(2-u) + 4p_{2,n} \left( \cos^2\left(\frac{\pi}{2}x_1\right) + u \sin^2\left(\frac{\pi}{2}x_1\right) \right) - G_n(\varphi(x_2)), \quad (6.11)$$

and $p_n$ is solution to the adjoint system

$$\left\{ \begin{aligned} \dot{p}_{n,1} &= -\partial_{x_1} H_n = 2p_{n,2}\pi \sin(\pi x_1^*(t))(1 - u^*(t)) & p_{n,1}(T) = 0, \\ \dot{p}_{n,2} &= -\partial_{x_2} H_n = G'_n(\varphi(x_2^*(t)))\varphi'(x_2^*(t)) & p_{n,2}(T) = 0. \end{aligned} \right. \quad (6.12)$$

Over the interval $[\tau_2^*, T]$, one has $G_n(\varphi(x_2^*)) = 0$ and thus $p_n \equiv 0$ over this interval. Over the interval $(\tau_1^*, \tau_2^*)$, one has $u^* = 1$ and we deduce that $p_{n,1} = 0$ over this interval. One also has $\dot{x}_2^* = 4$ and thus

$$p_{n,2}(t) = - \int_{t}^{\tau_2^*} G'_n(\varphi(x_2(s)))\varphi'(x_2(s)) \, ds = -\frac{1}{4} G_n(\varphi(x_2^*(t))) \quad t \in (\tau_1^*, \tau_2^*).$$

Finally, over the interval $[0, \tau_1^*]$, one has again $G_n(\varphi(x_2^*)) = 0$ and $p_{n,2}$ is thus constant over this interval equal to $p_{n,2}(\tau_2^*) = 0$. We deduce that $p_n \equiv 0$ over $[0, \tau_1^*]$. We can then conclude that $p_n$ is given by:

$$p_{n,1}(t) = 0 \quad t \in [0, T] \quad ; \quad p_{n,2}(t) = \begin{cases} 0 & t \in [0, \tau_1^*), \\ \frac{1}{4} G_n(\varphi(4t-1)) & t \in (\tau_1^*, \tau_2^*), \\ 0 & t \in (\tau_1^*, T]. \end{cases}$$

A simple consequence of this computation is that $(p_n)_n$ is bounded in $L^\infty([0, T] ; \mathbb{R}^2)$ and converges pointwise to the piecewise constant function $p$ given in (6.10) (see Fig. 4).
<figure><img src="image_7.png" /><figcaption>Figure 4: Illustration of the convergence of $(p_n)$ to the piecewise constant $p$.</figcaption></figure>

# 7 Conclusion

In this work, we have developed an approach based on a sequence of approximate optimal control problems. The proof of boundedness of the sequence of covectors that we have proposed here presents some analogy with materials of the work [17]. However, we have treated the question of boundedness via a suitable sequence of integrals $(I_n)_n$, and without the use of the Hybrid Maximum Principle (that we obtain indeed as a byproduct of this approach). We also did not need to introduce needle-like perturbations and variation vectors to derive necessary optimality conditions. The main feature is that we obtain necessary optimality conditions for the time crisis problem under a more general condition than requiring transverse crossing times for the optimal trajectory. This condition is related to the sequence of integrals $(I_n)_n$ and it does not involve the optimal path $x^*$ neither its velocity. Besides, it allows to deal with non-transverse optimal paths at the boundary of the constraint set as we saw in Section 6.

Our methodology presents several interests from a numerical point of view since we introduced a regularized optimal control problem in place of a discontinuous problem (which is more delicate to handle numerically). In addition, we introduced an auxiliary condition to guarantee necessary conditions for an optimal solution $x^*$. This condition involves an approximated sequence which can be tested numerically and not a solution $x^*$ to the initial optimal control problem (that is unknown a priori). Once this condition is checked (i.e., the boundedness of the sequence $(I_n)_n$), one can then write the necessary optimality conditions which provides an exact characterization of the limit $x^*$.

Some interesting issues that are out of the scope of this paper could be the matter of future works. In particular, one would like to better grasp the link between the boundedness of the sequence $(I_n)_n$ and the behavior of $x^*$ at a crossing time $\tau$ (i.e., to find a characterization of the boundedness of $(I_n)_n$ in terms of geometrical properties of the optimal path $x^*$ at the crossing times), and to deal with optimal paths that could cross the constrained set tangentially on both sides. The methodology that has been developed in this paper could also be used to obtain an extension of necessary optimality conditions for more general hybrid problems whenever an optimal path has a so-called one-sided transverse crossing time, i.e., is tangent to the constrained set only on one side.

# Acknowledgments

The authors are grateful to the AVERROES program for the funding of the PhD grant of Kenza Boumaza. They would also like to thank Anas Bouali and Loïc Bourdin for fruitful exchanges about the boundedness of the sequences of covectors $(p_n)_n$.
References

[1] S.M. Aseev, A.I. Smirnov, *Necessary first-order conditions for optimal crossing of a given region*, Comput. Math. Model., vol. 18, 4, pp. 397–419, 2007.

[2] J.-P. AUBIN, *Viability Theory*, Systems & Control: Foundations & Applications. Birkhäuser Boston, 1991.

[3] J.-P. AUBIN, A.M. BAYEN, P. SAINT-PIERRE, *Viability Theory, New Directions*, Second Editions, Springer, Heidelberg, 2011.

[4] G. BARLES, A. BRIANI, E. TRÉLAT, *Value function for regional problems via dynamic programming and Pontryagin maximum principle*, Math. Control Relat. Fields, vol. 8, 3&4, pp. 509–533, 2018.

[5] T. BAYEN, K. BOUMAZA, A. RAPAPORT, *Penalty function method for the minimal time crisis problem*, ESAIM Proc. Surveys, EDP Sci., vol. 71, pp. 21–32, 2021.

[6] T. BAYEN, L. PFEIFFER, *Second-order analysis for the time crisis problem*, J. Convex Anal., vol. 27, 1, pp. 139–163, 2020.

[7] T. BAYEN, A. RAPAPORT, *About Moreau-Yosida regularization of the minimal time crisis problem*, J. Convex Anal. 23, 1, pp. 263–290, 2016.

[8] T. BAYEN, A. RAPAPORT, *About the minimal time crisis problem*, ESAIM Proc. Surveys, EDP Sci., vol. 57, pp. 1–11, 2017.

[9] T. BAYEN, A. RAPAPORT, *Minimal time crisis versus minimum time to reach a viability kernel: a case study in the prey-predator model*, Optim. Control Appl. Meth., vol. 40, 2, pp. 330–350, 2019.

[10] F.H. CLARKE, *Functional Analysis, Calculus of Variation, Optimal control*, Graduate Texts in Mathematics, 264, Springer, London, 2013.

[11] F.H. CLARKE, YU.S. LEDYAEV, R.J. STERN, P.R. WOLENSKI, *Nonsmooth Analysis and Control Theory*, Graduate Texts in Mathematics, Springer, 1998.

[12] A.V. DMITRUK, *The hybrid maximum principle is a consequence of Pontryagin maximum principle*, Systems Control Lett. 57, 11, pp. 964–970, 2008.

[13] A.V. DMITRUK AND A. M. KAGANOVICH, *Maximum principle for optimal control problems with intermediate constraints*, Comput. Math. Model., vol. 22, 2, pp. 180–215, 2011.

[14] A.V. DMITRUK AND A. M. KAGANOVICH, *Quadratic order conditions for an extended weak minimum in optimal control problems with intermediate and mixed constraints*, Discrete Contin. Dyn. Syst. 29, 2, pp. 523–545, 2011.

[15] L. DOYEN, P. SAINT-PIERRE, *Scale of viability and minimal time of crisis*, Set-Valued Var. Anal., 5, pp. 227–246, 1997.

[16] M. GARAVELLO, B. PICCOLI, *Hybrid necessary principle*, SIAM J. Control Optim., vol. 43, 5, pp. 1867–1887, 2005.

[17] T. HABERKORN, E. TRÉLAT, *Convergence results for smooth regularizations of hybrid nonlinear optimal control problems*, SIAM J. Control Optim., vol. 49, 4, pp. 1498–1522, 2011.

[18] Team Commands, Inria Saclay, BOCOP: an open source toolbox for optimal control. http://bocop.org

[19] L.S. PONTRYAGIN, V.G. BOLTYANSKIY, R.V. GAMKRELIDZE, E.F. MISHCHENKO, *The Mathematical Theory of Optimal Processes*, The Macmillan Co., New York, 1964.

[20] C. SILVA, E. TRÉLAT, *Asymptotic approach on conjugate points for minimal time bang-bang controls*, Systems Control Lett., vol. 59, 11, pp. 720–733, 2010.

[21] A. SMIRNOV, *Necessary optimality conditions for a class of optimal control problems with discontinuous integrand*, Proc. Steklov Inst. Math., vol. 262, 1, pp. 213–230, 2008.
[22] R. VINTER, *Optimal Control*, Systems and Control: Foundations and Applications, Birkhäuser, Boston, 2000.

[23] M. I. ZELIKIN, V. F. BORISOV, *Theory of Chattering Control*, Systems & Control: Foundations & Applications, Birkhäuser, 1994.