# New Necessary Conditions for Optimal Control Problems in Discontinuous Dynamic Systems

Ekaterina Kostina, Olga Kostyukova, Werner Schmidt

► To cite this version:

Ekaterina Kostina, Olga Kostyukova, Werner Schmidt. New Necessary Conditions for Optimal Control Problems in Discontinuous Dynamic Systems. 25th System Modeling and Optimization (CSMO), Sep 2011, Berlin, Germany. pp.122-135, 10.1007/978-3-642-36062-6_13 . hal-01347530

HAL Id: hal-01347530

https://inria.hal.science/hal-01347530

Submitted on 21 Jul 2016

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# New Necessary Conditions for Optimal Control Problems in Discontinuous Dynamic Systems

Ekaterina Kostina<fnref n="1" />, Olga Kostyukova<fnref n="2" />, and Werner Schmidt<fnref n="3" />

**Abstract.** In the paper we derive new necessary optimality conditions for optimal control of differential equations systems with discontinuous right hand side. The main attention is paid to a situation when an optimal trajectory slides on the discontinuity surface. The new conditions, derived in the paper, are essential and do not follow from any known necessary conditions for such systems.

**Keywords:** Optimal control problem, maximum principle, discontinuous dynamic system

## 1 Introduction

Optimal control of systems of differential equations with discontinuous right hand side are widely used to describe numerous applications in natural sciences and engineering, where, e.g., there is a necessity to model dynamics with different scales or even with jumps. Such models appear e.g. in economics, mechanics (e.g. optimal control of mechanical systems with Coulomb friction), chemical processes, electrical and radio engineering, aerodynamics, automatic control theory, and theory of hybrid systems [3,5,8,10,19].

There is a row of papers devoted to the numerical solution of such optimal control problems, see [18] and references therein. However, due to the complexity of these optimal control problems there are few papers devoted to their theoretical studies.

The aim of this paper is to present new necessary optimality conditions for optimal control of differential equations systems with discontinuous right hand side. It is assumed that the discontinuity of the function, which describes the dynamic system, appears at some surface defined in the state space. This surface is called the discontinuity or switching surface. Most papers on the topic consider the situation when an optimal trajectory crosses the surface [2,4,17]. In this

<footnote n="1">¹ Department of Mathematics and Computer Science, University of Marburg, Hans-Meerwein-Str., 35032 Marburg, Germany</footnote>

<footnote n="2">² Institute of Mathematics, Belarus Academy of Sciences, Surganov Str. 11, 220072 Minsk, Belarus</footnote>

<footnote n="3">³ Institute of Mathematics and Computer Science, University of Greifswald, Rathenaustr. 47, 17487 Greifswald, Germany</footnote>
case the optimality conditions are a slight modification of the maximum principle of Pontryagin. However, the more interesting and practically non-studied case is the situation when an optimal trajectory slides on the surface. There are only very few papers [2,12,15,16], which deal with this case, however, the maximum conditions presented there are weaker than the conditions presented in this paper.

Dynamic systems with discontinuous right hand side may be presented as differential inclusions. Necessary optimality conditions in optimal control problems for differential inclusions have been intensively studied, see e.g. [6,14,20] and references therein. However, the results obtained there are based on the assumption that the differential inclusion is Lipschitz continuous or possesses a modified one-sided Lipschitz property, which is not the case for the differential inclusion describing the dynamic system of this paper.

## 2 Problem Statement and Assumptions

Let us consider the following optimal control problem with discontinuous right hand side:

$$ \begin{array}{ll} \min_{u(\cdot)} & f_0(x(t^*)), \\ \dot{x} = & \begin{cases} \bar{f}^{-}(x, u) & \text{if } S(x(t)) < 0, \\ \bar{f}^{+}(x, u) & \text{if } S(x(t)) > 0, \end{cases} \\ x(t_*) = x_0, h(x(t^*)) = 0, & |u(t)| \le 1, t \in T = [t_*, t^*]. \end{array} \quad (1) $$

Here $\bar{f}^{\pm}(x, u) = f^{\pm}(x) + b(x)u$, $x = x(t) \in \mathbb{R}^n$ denotes the state n-vector, $u = u(t) \in \mathbb{R}$ is a scalar control, $S(x) := d^T x - \gamma = 0$ is a surface of discontinuity (switching surface), the functions $f_0(x) \in \mathbb{R}$, $f^{\pm}(x), b(x) \in \mathbb{R}^n$, $h(x) \in \mathbb{R}^{s*}$ are given sufficiently smooth scalar and vector functions, $d, x_0$ are given vectors, $\gamma, t_*, t^*$ are given numbers, $s_* = \dim(h)$ is the dimension of the vector-function $h = h(x)$.

In the general case, even for a fixed control $u(\cdot) = (u(t), t \in T)$ and a given initial value $x(t_*) = x_0$, the system (1) may not have a classical solution, since the system is not defined at the switching surface. Therefore, we redefine a solution of the system (1) at the switching surface following Filippov [8]. Then the problem may be reformulated as

$$ \begin{array}{ll} \min_{u(\cdot), \alpha(\cdot)} & f_0(x(t^*)), \\ \dot{x} = & \begin{cases} \bar{f}^{-}(x, u), & \text{if } d^T x(t) < \gamma, \\ \bar{f}^{+}(x, u), & \text{if } d^T x(t) > \gamma, \\ F(x, u, \alpha), & \text{if } d^T x(t) = \gamma, \end{cases} \end{array} \quad (2) $$
$$ 
\begin{array}{l}
x(t_*) = x_0, \quad h(x(t^*)) = 0, \\
|u(t)| \le 1, \quad 0 \le \alpha(t) \le 1, \quad t \in T = [t_*, t^*].
\end{array}
$$

where $F(x, u, \alpha) := \alpha \bar{f}^{-}(x, u) + (1 - \alpha) \bar{f}^{+}(x, u) = f^{+}(x) + a(x)\alpha + b(x)u$, $a(x) := f^{-}(x) - f^{+}(x)$. In this problem the control is $u(t)$, $\alpha(t)$, $t \in T$.

Using classical optimal control theory [13], we may conclude that if the problem (2) is feasible, it has an optimal solution in the class of measurable functions $u(t)$, $\alpha(t)$, $t \in T$.

Let $u^*(\cdot) = (u^*(t), t \in T)$, $\alpha^*(\cdot) = (\alpha^*(t), t \in T)$, $x^*(\cdot) = (x^*(t), t \in T)$ be an optimal control and the corresponding trajectory of the system (2). Here and in what follows, we formally suppose that

$$ 
\begin{array}{l}
\alpha^*(t) = 1 \text{ if } d^T x^*(t) < \gamma, \quad \alpha^*(t) = 0 \text{ if } d^T x^*(t) > \gamma, \\
0 \le \alpha^*(t) \le 1 \text{ if } d^T x^*(t) = \gamma, \quad t \in T.
\end{array}
$$

For the formulation and the proof of the maximum principle we need several assumptions.

**Assumption 1** The functions $u^*(\cdot) = (u^*(t), t \in T)$, $\alpha^*(\cdot) = (\alpha^*(t), t \in T)$ are piecewise continuous and piecewise smooth.

We denote $T_s = \{t \in T : d^T x^*(t) = \gamma\}$. In the general case this set contains points, which correspond to crossing the discontinuity surface by a trajectory, and segments, which correspond to the case when the trajectory lies on the surface. As it was mentioned before, the case when the trajectory crosses the discontinuity surface is well-studied. The aim of this paper is to study the case when the trajectory may lie on the discontinuity surface. For this purpose we need the following assumption.

**Assumption 2** The set $T_s$ consists of a finite number of segments $[\tau_k, \tau^k]$, $k = 1, ..., p$,

$$ t_* < \tau_1 < \tau^1 < \tau_2 < \tau^2 < \dots < \tau_p < \tau^p < t^*, $$

and the following inequalities hold,

$$ d^T \dot{x}^*(\tau_k - 0) \neq 0, \quad d^T \dot{x}^*(\tau^k + 0) \neq 0, \quad k = 1, \dots, p. \quad (3) $$

Here and in what follows, $z(\bar{t} - 0)$ and $z(\bar{t} + 0)$ for a given function $z(t)$, $t \in T$, are defined by $z(\bar{t} + 0) := \lim_{t \to \bar{t}, t \ge \bar{t}} z(t)$, $z(\bar{t} - 0) := \lim_{t \to \bar{t}, t \le \bar{t}} z(t)$.

We denote

$$ T_s^0 = \{t \in T_s : \alpha^*(t) = 0\}, \quad T_s^1 = \{t \in T_s : \alpha^*(t) = 1\}, \quad T_s^* = T_s \setminus (T_s^0 \cup T_s^1). $$
**Assumption 3** The “active” set $T_s^0 \cup T_s^1$ does not contain isolated points and the following relations hold true for an optimal control:

$$ \exists \epsilon_0 > 0, \quad |u^*(t)| \le 1 - \epsilon_0, \ t \in T_s^0 \cup T_s^1, \\ |d^T b(x^*(t))| \ge \epsilon_0, \ t \in T_s^0 \cup T_s^1; \quad |d^T a(x^*(t))| \ge \epsilon_0, \ t \in T_s^*, \\ \text{rank} \frac{\partial h(x^*(t^*))}{\partial x} = s_* $$

### 3 Necessary Optimality Conditions

The main goal of this section is to formulate and prove new necessary optimality conditions in the form of the maximum principle. For this purpose we will need some auxiliary results.

Without loss of generality, we will suppose for simplicity that the following relations are satisfied

$$ d^T x^*(t) > \gamma, t \in [t_*, \tau_1), d^T x^*(t) = \gamma, t \in [\tau_1, \tau^1], d^T x^*(t) < \gamma, t \in (\tau^1, t^*], \quad (4) \\ \alpha^*(t) = 1, |u^*(t)| < 1, t \in (\tau_1, \tau_0), 0 < \alpha^*(t) < 1, t \in (\tau_0, \tau^1). $$

Therefore, for the case under consideration we have

$$ p = 1, \ T_s = [\tau_1, \tau^1], \ T_\alpha^+ = (\tau_1, \tau_0), \ T_\alpha^0 = \emptyset, \ T_\alpha^* = (\tau_0, \tau^1), \\ T^+ = [t_*, \tau_1), \ T^- = (\tau^1, t^*]. $$

We introduce the set of parameters

$$ \mu = (y_0, y, \lambda_1, \lambda^1), \qquad (5) $$

where $y_0 \ge 0$, $\lambda_1$, $\lambda^1$ are scalars, $y \in \mathbb{R}^{s^*}$, and denote by $\psi(t|\mu)$, $t \in T$, a solution of the system

$$ \begin{aligned} \dot{\psi}^T(t) &= -\psi^T(t) \frac{\partial \bar{f}^+(x^*(t), u^*(t))}{\partial x}, \quad t \in [t_*, \tau_1); \\ \dot{\psi}^T(t) &= -\psi^T(t) \frac{\partial \bar{f}^-(x^*(t), u^*(t))}{\partial x}, \quad t \in [\tau^1, t^*]; \\ \dot{\psi}^T(t) &= -\psi^T(t) \left( \frac{\partial F(x^*(t), u^*(t), \alpha^*(t))}{\partial x} - q_1^*(t) d^T \right), \quad t \in [\tau_1, \tau_0), \\ \dot{\psi}^T(t) &= -\psi^T(t) \left( \frac{\partial F(x^*(t), u^*(t), \alpha^*(t))}{\partial x} - q_2^*(t) d^T \right), \quad t \in [\tau_0, \tau^1), \quad (6) \\ \psi(t^*) &= -y_0 \frac{\partial f_0(x^*(t^*))}{\partial x} - \frac{\partial h^T(x^*(t^*))}{\partial x} y, \\ \psi(\tau_1 - 0) &= \psi(\tau_1 + 0) + d\lambda_1, \quad \psi(\tau^1 - 0) = \psi(\tau^1 + 0) + d\lambda^1. \end{aligned} $$
Further we pick up any $m \in \mathbb{N}$, $m \ge 2$, and consider a set of points $\bar{t}_1, \bar{t}_2, \dots, \bar{t}_{2m}$, such that

$$ \tau_1 = \bar{t}_1 < \bar{t}_2 < \dots < \bar{t}_{2m-1} = \tau_0 < \bar{t}_{2m} = \tau^1. \quad (7) $$

This set of points satisfies the following lemma:

**Lemma 1.** Let $u^*(\cdot), \alpha^*(\cdot), x^*(\cdot)$ be an optimal control and the trajectory in the problem (2), for which Assumptions 1-3 are fulfilled. For any choice of the points (7) there exists a vector of parameters $\mu$, $\|\mu\| = 1$, (5), such that along the corresponding solution $\psi(t) = \psi(t|\mu)$, $t \in T$, of the system (6) the following relations hold true,

$$ \psi^T(t)b(x^*(t))u^*(t) = \max_{|u| \le 1} \psi^T(t)b(x^*(t))u, \text{ a.e. } t \in T; \quad (8) $$

$$ \psi^T(t)a(x^*(t))\alpha^*(t) = \max_{0 \le \alpha \le 1} \psi^T(t)a(x^*(t))\alpha, \quad (9) $$

$$ \text{a.e. } t \in [\bar{t}_{2i-1}, \bar{t}_{2i}], \quad i = 1, 2, \dots, m, $$

$$ \psi^T(t)q_1^*(t) \ge 0, \quad \text{a.e. } t \in [\bar{t}_{2i}, \bar{t}_{2i+1}], \quad i = 1, 2, \dots, m-1. \quad (10) $$

The proof of the lemma can be found in [11] and its main idea is sketched here. For a fixed $m \ge 2$ we consider a set of parameters $\theta = (t_1, t_2, \dots, t_{2m})$ and formulate the optimal control problem of a hybrid system:

$$ \begin{array}{ll} \displaystyle \min_{u(\cdot), \alpha(\cdot), \theta} & f_0(x(t^*)), \\ \displaystyle \dot{x} = \bar{f}^+(x, u), \quad d^T x(t) \ge \gamma, \quad t \in [t_*, t_1[ \\ \displaystyle \dot{x} = F(x, u, \alpha), \quad d^T x(t) = \gamma, \quad t \in [t_{2i-1}, t_{2i}], \\ \displaystyle \dot{x} = \bar{f}^-(x, u), \quad d^T x(t) \le \gamma, \quad t \in [t_{2i}, t_{2i+1}], \quad i = 1, \dots, m, \quad (11) \\ \displaystyle x(t_*) = x_0, \quad h(x(t_*)) = 0, \\ \displaystyle t_* = t_0 \le t_1 \le \dots \le t_{2m} \le t_{2m+1} = t^*, \\ \displaystyle |u(t)| \le 1, \quad t \in T; \quad 0 \le \alpha(t) \le 1, \quad t \in \bigcup_{i=1}^m [t_{2i-1}, t_{2i}]. \end{array} $$

Let us note that in the problem (11), the decision variables are the control $u(\cdot), \alpha(\cdot)$ and a vector $\theta$.

With the notations

$$ \begin{align} z_i(\tau) &= x(t_{i-1} + \tau(t_i - t_{i-1})), \quad i = 1, 2, \dots, 2m + 1, \notag \\ v_i(\tau) &= u(t_{i-1} + \tau(t_i - t_{i-1})), \quad i = 1, 2, \dots, 2m + 1, \tag{12} \\ \beta_i(\tau) &= \alpha(t_{i-1} + \tau(t_i - t_{i-1})), \quad i = 2, 4, \dots, 2m, \notag \end{align} $$

we form the extended state vector

$$ \begin{gathered} Z(\tau) = (z_i(\tau), i = 1, \dots, 2m + 1; t_i(\tau), i = 1, \dots, 2m) \in \mathbb{R}^{n \times (2m+1)+2m}, \quad (13) \\ \tau \in [0, 1], \end{gathered} $$
and the extended control vector

$$V(\tau) = (v_i(\tau), i = 1, \dots, 2m + 1, \beta_i(\tau), i = 2, 4, \dots, 2m) \in \mathbb{R}^{3m+1}, \quad (14)$$
$$\tau \in [0, 1].$$

Using the introduced notations we may rewrite the problem (11) as follows,

$$\begin{array}{ll}
\min_{V(\cdot)} & f_0(z_{2m+1}(1)), \\
\dot{Z}(\tau) = & \mathcal{F}(Z(\tau), V(\tau)), \Phi(Z(0), Z(1)) = 0, \quad (15) \\
& G_1^T Z(\tau) \ge \gamma, G_{2i}^T Z(\tau) = \gamma, G_{2i+1}^T Z(\tau) \le \gamma, i = 1, 2, \dots, m; \\
& |v_i(\tau)| \le 1, i = 1, \dots, 2m+1, \\
& 0 \le \beta_i(\tau) \le 1, i = 2, 4, \dots, 2m, \\
& t_i(\tau) \le t_{i+1}(\tau), i = 0, \dots, 2m, \tau \in [0, 1]. \quad (16)
\end{array}$$

Here $V(\tau)$ is the control vector (14), $Z(\tau)$ is the state vector (13),

$$\begin{array}{l}
\mathcal{F}^T(Z, V) = ((t_1 - t_0)\bar{f}^+(z_1, v_1), \\
\qquad (t_{2i} - t_{2i-1})F(z_{2i}, v_{2i}, \beta_{2i}), (t_{2i+1} - t_{2i})\bar{f}^-(z_{2i+1}, v_{2i+1}), \\
\qquad i = 1, \dots, m, \underbrace{0, \dots, 0}_{2m}), \\
\\
\Phi(Z(0), Z(1)) = \begin{pmatrix} z_1(0) - x_0 \\ z_i(1) - z_{i+1}(0), \quad i = 1, \dots, 2m \\ h(z_{2m+1}(1)) \\ d^T z_i(1) - \gamma, \quad i = 1, \dots, 2m \end{pmatrix}, \\
\\
G_i^T = \underbrace{\mathbb{O}^T, \dots, \mathbb{O}^T}_{i-1}, \underbrace{d^T, \dots, d^T}_{2m+1-i}, \underbrace{\mathbb{O}^T, \dots, \mathbb{O}^T, 0, \dots, 0}_{2m}, \quad i = 1, \dots, 2m+1, \\
\\
\mathbb{O} \in \mathbb{R}^n \text{ is a vector of zeros,} \\
\\
t_0(\tau) \equiv \bar{t}_0 = t_*, \quad t_{2m+1}(\tau) \equiv \bar{t}_{2m+1} = t^*.
\end{array}$$

Now consider the set of points $\bar{t}_1, \bar{t}_2, \dots, \bar{t}_{2m}$ satisfying (7) and denote by $Z^*(\tau), V^*(\tau), \tau \in [0, 1]$, the functions (12) - (14), constructed using this set, the optimal control $u^*(\cdot), \alpha^*(\cdot)$ and the trajectory $x^*(\cdot)$ of the problem (2).

Since the control $u^*(\cdot), \alpha^*(\cdot)$ and the trajectory $x^*(\cdot)$ are optimal in the problem (2), it is obvious that $V^*(\tau), Z^*(\tau), \tau \in [0, 1]$, are an optimal control and the corresponding trajectory of the problem (15), (16). By the assumptions (see (3) and (4)) we have

$$\begin{array}{l}
G_1^T Z^*(\tau) > \gamma, \tau \in [0, 1), G_1^T Z^*(1) = \gamma, G_1^T \dot{Z}^*(1-0) \neq 0, \\
G_{2m+1}^T Z^*(\tau) < \gamma, \tau \in (0, 1], G_{2m+1}^T Z^*(0) = \gamma, G_{2m+1}^T \dot{Z}^*(+0) \neq 0.
\end{array}$$
Hence, $V^*(\tau)$, $Z^*(\tau)$, $\tau \in [0, 1]$, is also a strong local extremal in the problem that results from the problem (15), (16) by removing the state constraints $G_1^T Z(\tau) \ge \gamma$, $G_{2m+1}^T Z(\tau) \le \gamma$, $\tau \in [0, 1]$, and the constraints (16), namely in the problem

$$ 
\begin{array}{l}
\displaystyle \min_{V(\cdot)} f_0(z_{2m+1}(1)), \\
\displaystyle \dot{Z}(\tau) = \mathcal{F}(Z(\tau), V(\tau)), \Phi(Z(0), Z(1)) = 0, \hfill (17) \\
\displaystyle G_i^T Z(\tau) \le \gamma, i = 3, 5, \dots, 2m-1; G_i^T Z(\tau) = \gamma, i = 2, 4, \dots, 2m; \\
\displaystyle |v_i(\tau)| \le 1, i = 1, \dots, 2m+1, 0 \le \beta_i(\tau) \le 1, i = 2, 4, \dots, 2m; \\
\displaystyle \tau \in [0, 1],
\end{array}
$$

with the control $V(\tau)$ (see (14)) and the state vector $Z(\tau)$ (see (13)).

The problem (17) is an optimal control problem with inequality and equality state constraints and boundary constraints $\Phi(Z(0), Z(1)) = 0$. Due to Assumptions 1-3 and the specific structure of this problem, regularity conditions (see [1]) are satisfied for the control $V^*(\tau)$ and the trajectory $Z^*(\tau)$, $\tau \in [0, 1]$. Thus we can apply Theorems 4.1 and 12.1 from [1] and results from [7], according to which the control $V^*(\tau)$ and the trajectory $Z^*(\tau)$, $\tau \in [0, 1]$, satisfy certain relations. Analyzing these relations and taking into account the structure of the vectors $V^*(\tau)$ and $Z^*(\tau)$ allow us to get the assertions of the lemma. ◇

Let us note that it follows from Lemma 1 that the continuity of the function $\psi(t|\mu)$, $t \in (\tau_1, \tau^1)$, the relations (9), (10) and the assumption $\alpha^*(t) = 1, t \in (\tau_1, \tau_0)$, imply the inequalities

$$ \psi^T(\bar{t}_i|\mu)a(x^*(\bar{t}_i)) \ge 0, \psi^T(\bar{t}_i|\mu)q_1^*(\bar{t}_i) \ge 0, i = 2, 3, \dots, 2m-1, \quad (18) $$

for each point set $\bar{t}_1, \bar{t}_2, \dots, \bar{t}_{2m}$ satisfying (7).

Now we are ready to formulate and prove new necessary optimality conditions for problem (2) in the form of the maximum principle.

**Theorem 1.** Let $u^*(\cdot), \alpha^*(\cdot), x^*(\cdot)$ be an optimal control and the corresponding trajectory of the problem (2), which satisfy Assumptions 1-3. Then there exist numbers $\lambda_k, \lambda^k, k = 1, \dots, p, y_0 \ge 0$, and a vector $y \in \mathbb{R}^{s^*}$, not all trivial,

$$ \sum_{k=1}^{p} (|\lambda_k| + |\lambda^k|) + y_0 + ||y|| > 0, $$
such that along a solution of the adjoint system

$$
\begin{align*}
\dot{\psi}^T(t) &= \begin{cases}
-\psi^T(t) \frac{\partial(f^\pm(x^*(t))+b(x^*(t))u^*(t))}{\partial x}, & t \in \{t \in T : \pm(d^T x^*(t) - \gamma) > 0\}, \\
-\psi^T(t) \left( \frac{\partial F(x^*(t),u^*(t),\alpha^*(t))}{\partial x} - q_1^*(t) d^T \right), & t \in T_s^0 \cup T_s^1, \\
-\psi^T(t) \left( \frac{\partial F(x^*(t),u^*(t),\alpha^*(t))}{\partial x} - q_2^*(t) d^T \right), & t \in T_s^*,
\end{cases} \\
\psi(t^*) &= -y_0 \frac{\partial f_0(x^*(t^*))}{\partial x} - \frac{\partial h^T(x^*(t^*))}{\partial x} y, \\
\psi(\tau_k - 0) &= \psi(\tau_k + 0) + d\lambda_k, \\
\psi(\tau^k - 0) &= \psi(\tau^k + 0) + d\lambda^k, \quad k = 1, \dots, p,
\end{align*}
$$

the following relations hold true:

$$
\begin{align}
\psi^T(t)b(x^*(t))u^*(t) &= \max_{|u| \le 1} \psi^T(t)b(x^*(t))u, \quad \text{a.e. } t \in T; \notag \\
\psi^T(t)a(x^*(t))\alpha^*(t) &= \max_{0 \le \alpha \le 1} \psi^T(t)a(x^*(t))\alpha, \quad \text{a.e. } t \in T_s; \tag{19}
\end{align}
$$

$$
\begin{align}
\psi^T(t-0)\dot{x}^*(t-0) &= \psi^T(t+0)\dot{x}^*(t+0), \quad t = \tau_k, t = \tau^k, k = 1, \dots, p, \notag \\
\psi^T(t)q_1^*(t) \le 0, \quad t \in \text{int } T_s^0, \quad \psi^T(t)q_1^*(t) \ge 0, \quad t \in \text{int } T_s^1. \tag{20}
\end{align}
$$

Here

$$
\begin{align*}
q_i^*(t) &:= q_i(x^*(t), u^*(t), \alpha^*(t)); \quad i = 1, 2; \\
q_1(x, u, \alpha) &:= \left( \frac{\partial F(x, u, \alpha)}{\partial x} b(x) - \frac{\partial b(x)}{\partial x} F(x, u, \alpha) \right) / d^T b(x), \\
q_2(x, u, \alpha) &:= \left( \frac{\partial F(x, u, \alpha)}{\partial x} a(x) - \frac{\partial a(x)}{\partial x} F(x, u, \alpha) \right) / d^T a(x).
\end{align*}
$$

**Proof.** Again we will suppose for simplicity that relations (4) are satisfied. For an arbitrary $m \in \mathbb{N}$, $m \ge 2$, we consider the set of points

$$
t_i^{(m)} = \tau_1 + (i-1) \frac{(\tau_0 - \tau_1)}{2m-2}, \quad i = 1, \dots, 2m-1, \quad t_{2m}^{(m)} = \tau^1. \quad (21)
$$

For any $m \ge 2$ the set (21) satisfies relations (7). Hence Lemma 1 and the relation (18) imply that for a set (21) there exists a vector

$$
\mu(m) = (y_0(m), y(m), \lambda_1(m), \lambda^1(m)), \quad ||\mu(m)|| = 1,
$$
such that the following relations hold true,

$$
\begin{align}
\psi^T(t|\mu(m))b(x^*(t))u^*(t) &= \max_{|u|\le 1} \psi^T(t|\mu(m))b(x^*(t))u, \text{ a.e. } t \in T, \nonumber \\
\psi^T(t|\mu(m))a(x^*(t))\alpha^*(t) &= \max_{0\le \alpha \le 1} \psi^T(t|\mu(m))a(x^*(t))\alpha \tag{22} \\
\text{a.e. } t \in [\tau_0, \tau^1], \nonumber \\
\psi^T(t-0|\mu(m))\dot{x}^*(t-0) &= \psi^T(t+0|\mu(m))\dot{x}^*(t+0), \quad t = \tau_1, t = \tau^1; \nonumber
\end{align}
$$

$$
\begin{equation}
\begin{gathered}
\psi^T(t_i^{(m)} | \mu(m)) a(x^*(t_i^{(m)})) \ge 0, \quad \psi^T(t_i^{(m)} | \mu(m)) q_1^*(t_i^{(m)}) \ge 0, \quad (23) \\
i = 2, 3, \dots, 2m-1.
\end{gathered}
\end{equation}
$$

Consider the sequence of the vectors $\mu(m)$, $m = 2, 3, \dots$. Since $||\mu(m)|| = 1$, $m = 2, 3, \dots$, there exists a converging subsequence. Without loss of generality, we assume that the sequence $\mu(m)$, $m = 2, 3, \dots$ converges itself,

$$
\mu^* = \lim_{m \to \infty} \mu(m).
$$

Obviously, ||μ*|| = 1.

It follows from (21) that for any point $t \in [\tau_1, \tau_0]$ there exists a sequence of indices

$$
i(m) = i(m|t) \in \{2, 3, \dots, 2m-1\}, m = 2, 3, 4, \dots,
$$

such that

$$
t_{i(m)}^{(m)} \to t \text{ as } m \to \infty.
$$

By construction (see (23)),

$$
\begin{equation}
\begin{gathered}
\psi^T(t_{i(m)}^{(m)}|\mu(m))a(x^*(t_{i(m)}^{(m)})) \ge 0, \quad \psi^T(t_{i(m)}^{(m)}|\mu(m))q_1^*(t_{i(m)}^{(m)}) \ge 0, \quad (24) \\
m = 2, 3, 4, \dots
\end{gathered}
\end{equation}
$$

For $m \to \infty$ in the last inequalities we get

$$
\psi^T(t|\mu^*)a(x^*(t)) \ge 0, \quad \psi^T(t|\mu^*)q_1^*(t) \ge 0, \quad t \in [\tau_1, \tau_0]. \quad (25)
$$

Similarly, for $m \to \infty$ in (22) we obtain

$$
\begin{align}
\psi^T(t|\mu^*)b(x^*(t))u^*(t) &= \max_{|u|\le 1} \psi^T(t|\mu^*)b(x^*(t))u, \text{ a.e. } t \in T; \notag \\
\psi^T(t|\mu^*)a(x^*(t))\alpha^*(t) &= \max_{0\le\alpha\le 1} \psi^T(t|\mu^*)a(x^*(t))\alpha \quad \text{a.e. } t \in [\tau_0, \tau^1]; \tag{26} \\
\psi^T(t-0|\mu^*)\dot{x}^*(t-0) &= \psi^T(t+0|\mu^*)\dot{x}^*(t+0), \quad t = \tau_1, t = \tau^1. \notag
\end{align}
$$

The relations (25) and (26) are nothing but the assertions of Theorem 1 for the
considered structure of the solution of the problem (2) (see the assumption (4)).
Analogously we may prove the Theorem for other types of solution structure. ◦
# 4 Discussion of the New Necessary Optimality Conditions

Because of the problem complexity, there are only few papers, e.g. [2,12,16,15], where necessary optimality conditions for optimal control problems with discontinuous dynamics are presented. The theorem formulated in this paper contains new crucial conditions (20) and (19).

Let us briefly discuss differences between our maximum principle and maximum principles known from the literature.

The necessary conditions in [2] are proved under the very strong assumption $T_s^0 = T_s^1 = \emptyset$, which is not needed in our theorem.

As in [12], the optimality conditions derived here are based on the so-called direct approach (see [9]). However, the necessary conditions in [12] contain the conditions (20) but not the condition (19).

The optimality conditions in [2,16] are formulated based on an so-called indirect approach (see [9]), which a priori leads to weaker results compared with the direct approach.

In [16], conditions (19), (20) are not considered. However, the conditions (19), (20) are essential and they are not a consequence of any other conditions mentioned in [12], [16].

The maximum principle in [15] is weaker than the Theorem 1. Indeed, one can easily show that for the problems of the form (2) the maximum principle from [15] is satisfied trivially for any feasible control. Furthermore, one can construct examples (one of them is presented below) where a non-optimal control satisfies the conditions of maximum principle from [15], but not the conditions of Theorem 1.

To finish the discussion of the new necessary conditions, we would like to stress the importance of the conditions (19).

Let us consider an optimal control problem with state constraints in the form

$$ \begin{array}{l} \min f_0(x(t^*)), \\ \dot{x} = f^{-}(x) + b(x)u, \quad x(t_*) = x_0, \quad h(x(t^*)) = 0, \\ d^T x(t) \le \gamma, \quad |u(t)| \le 1, \quad t \in T = [t_*, t^*]. \end{array} $$

Suppose that this problem has an optimal control $u^*(t), t \in T$, and the corresponding trajectory $x^*(t), t \in T$, such that $\text{mes } T_s > 0, T_s := \{t \in T : d^T x^*(t) = \gamma\}$. Then the control $u^*(t), t \in T$, and the function $\alpha^*(t) = 1, t \in T$, are feasible in the original problem (2) for any function $f^+(x)$ and satisfy the necessary optimality conditions from [12], [16]. However, one can easily construct functions $f^+(x)$ (and corresponding optimal control problems (2)), which together with the functions $u^*(t), \alpha^*(t) = 1, t \in T$, violate the maximum condition (19). Hence, the control $u^*(t), \alpha^*(t), t \in T$, is not optimal in (2) according to Theorem 1.
# 5 Illustrative examples

In order to demonstrate differences of our maximum principle from maximum principles known from the literature we have constructed several examples. The aim of the examples is to show that the necessary conditions presented in this paper are stronger than known necessary conditions. Namely, we want to show that the new conditions (20) and (19) may be violated for controls that satisfy other necessary optimality conditions known from the literature, and hence our maximum principle guarantees that such controls are not optimal. Furthermore, we want to show that the new conditions are essential and does not follow from other known optimality conditions.

**Example 1.** Consider the optimal control problem depending on a parame-
ter $c$,

$$
\begin{array}{ll}
\mathbf{P}(c): & \min x_1(t^*) - 2.5x_2(t^*), \\
& x(t_*) = x_0, x_3(t^*) = 1, \\
& |u(t)| \le 1, 0 \le \alpha(t) \le 1, t \in [t_*, t^*], \\
& \left.
\begin{array}{l}
\dot{x}_1 = x_2, \\
\dot{x}_2 = x_3 + 5, \\
\dot{x}_3 = u + 1/2,
\end{array}
\right\} \text{if } x_3 < 0, \quad
\left.\egin{array}{l}
\dot{x}_1 = x_2 + c, \\
\dot{x}_2 = x_3, \\
\dot{x}_3 = u,
\end{array}\right\} \text{if } x_3 > 0, \\
\\
& \left.
\begin{array}{l}
\dot{x}_1 = x_2 + c(1-\alpha), \\
\dot{x}_2 = x_3 + 5\alpha, \\
\dot{x}_3 = u + 1/2\alpha,
\end{array}
\right\} \text{if } x_3 = 0, \\
\\
\text{with } x_0^T = (19/32, -37/16, -3/4), t_* = -0.5, t^* = 2
\end{array}
$$

and the control $u^*(\cdot)$, $\alpha^*(\cdot)$:

$$
u^*(t) = \begin{cases} 1, & t \in [-0.5, 0], \\ -0.5, & t \in [0, 1], \\ 1, & t \in [1, 2], \end{cases} \qquad \alpha^*(t) = \begin{cases} 1, & t \in [-0.5, 0], \\ 1, & t \in [0, 1], \\ 0, & t \in [1, 2]. \end{cases}
$$

For the control $u^*(\cdot)$, $\alpha^*(\cdot)$ we have $T_s = [0, 1]$, $T_s^1 = [0, 1]$, $T_s^0 = \emptyset$, $T_s^* = \emptyset$.

If we choose $c = c^0 = 0$, then in the problem $P(c^0)$ the control $u^*(\cdot)$, $\alpha^*(\cdot)$ is feasible, locally optimal and satisfies all necessary optimality conditions from Theorem 1.

If we choose $c = c^* = -5.5$, then in the problem $P(c^*)$ the control $u^*(\cdot)$, $\alpha^*(\cdot)$ is feasible and not optimal but it satisfies all necessary conditions from [12,16], and it satisfies all necessary conditions from Theorem 1 except for the condition (19). Hence, according to [12,16], this control may be locally optimal in the problem $P(c^*)$. On the other hand, following Theorem 1, it cannot be locally optimal in the problem $P(c^*)$.
**Example 2.** Consider the optimal control problem

$$
\begin{array}{l}
\min 2x_1(t^*) - 2x_2(t^*), \\
x(t_*) = x_0, \quad x_3(t^*) = -1, \quad |u(t)| \le 1, \quad 0 \le \alpha(t) \le 1, \quad t \in [t_* = -0.5, t^* = 2], \\
\left.
\begin{array}{l}
\dot{x}_1 = x_2, \\
\dot{x}_2 = x_3, \\
\dot{x}_3 = u,
\end{array}
\right\}
\quad \text{if } x_3 < 0, \quad
\left.
\begin{array}{l}
\dot{x}_1 = x_2 + 1, \\
\dot{x}_2 = x_3, \\
\dot{x}_3 = u,
\end{array}
\right\}
\quad \text{if } x_3 > 0,
\end{array}
\tag{27}
$$

$$
\left.
\begin{array}{l}
\dot{x}_1 = x_2 + (1 - \alpha), \\
\dot{x}_2 = x_3, \\
\dot{x}_3 = u,
\end{array}
\right\}
\quad \text{if } x_3 = 0,
$$

with $x_0^T = (-23/48, -1/8, 1/2)$,

and the control $u^*(\cdot)$, $\alpha^*(\cdot)$

$$
u^*(t) = -1, \alpha^*(t) = 0; t \in [-0.5, 0],
$$

$$
u^*(t) = 0, \alpha^*(t) = 1; t \in [0, 1],
$$

$$
u^*(t) = -1, \alpha^*(t) = 1; t \in [1, 2].
$$

For the control $u^*(\cdot)$, $\alpha^*(\cdot)$ we have $T_s = [0, 1]$, $T_s^1 = [0, 1]$, $T_s^0 = T_s^* = \emptyset$.

The control $u^*(\cdot)$, $\alpha^*(\cdot)$ satisfies all necessary optimality conditions from [16], satisfies all necessary optimality conditions from Theorem 1, except for the condition (20), and is not locally optimal.

Note that results from [15] can not be applied to this example because Assumption 3) from [15] (namely the condition $d^T(\bar{f}^-(x, u^*(t)) - \bar{f}^+(x, u^*(t))) > 0$, $\forall x \in S_0(t)$, $t \in T_s$) is not satisfied.

**Example 3.** Consider the following problem,

$$
\begin{array}{ll}
\mathbf{P}^* : & \min c^T x(t^*), \\
& x(t_*) = x_0, \quad d^T x(t^*) = 1, \quad |u(t)| \le 1, \quad 0 \le \alpha(t) \le 1, \quad t \in [t_*, t^*], \\
& \dot{x} = Ax + bu + g^{-}, \quad \text{if } d^T x < 0, \\
& \dot{x} = Ax + bu + g^{+}, \quad \text{if } d^T x > 0, \\
& \dot{x} = Ax + bu + \alpha g^{-} + (1 - \alpha) g^{+}, \quad \text{if } d^T x = 0,
\end{array}
$$

with $x \in \mathbb{R}^n$, $n = 4$, $t_* = -0.5$, $t^* = 2$,

$$
c = \begin{pmatrix} -\frac{32}{7} \\ \frac{48}{7} \\ -5 \\ 0 \end{pmatrix}, \quad d = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 1 \end{pmatrix},
$$

$$
g^+ = \begin{pmatrix} -2 \\ 1 \\ 0 \\ 0 \end{pmatrix}, g^- = \begin{pmatrix} 0 \\ 0 \\ 5 \\ 1/2 \end{pmatrix}, \quad b = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 1 \end{pmatrix}, \quad A = \begin{pmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{pmatrix}.
$$
The vector $x_0$ is uniquely defined by the condition that the trajectory $x(t), t \in [-0.5, 0]$, of the system $\dot{x} = Ax + b + g^-$, $x(-0.5) = x_0$, should satisfy the equality $x(0) = 0 \in \mathbb{R}^n$. Hence $d^T x_0 = -3/4$.
Consider the control $u^*(t), \alpha^*(t), t \in [-0.5, 2]$,

$$ 
\begin{align} 
u^*(t) &= 1, \quad \alpha^*(t) = 1, \quad t \in [-0.5, 0], \tag{28} \\ 
u^*(t) &= -1/2, \quad \alpha^*(t) = 1, \quad t \in [0, 1], \nonumber \\ 
u^*(t) &= 1, \quad \alpha^*(t) = 0, \quad t \in [1, 2], \nonumber 
\end{align} 
$$

and the corresponding trajectory $x^*(t) = (x_1^*(t), x_2^*(t), x_3^*(t), x_4^*(t)), t \in [t_*, t^*]$, with the function $d^T x^*(t) = x_4^*(t)$ satisfying

$$ x_4^*(t) < 0, t \in [-0.5, 0); x_4^*(t) = 0, t \in [0, 1]; x_4^*(t) > 0, t \in (1, 2]. $$

For the control (28) we have $T_s = [0, 1]$, $T_s^1 = [0, 1]$, $T_s^0 = \emptyset$.

We can show that the control (28) is not locally optimal in the problem $P_*$, satisfies all assumptions and all necessary optimality conditions from [15] and satisfies all conditions from Theorem 1 except for the condition (20).

## 6 Conclusions

We have presented a new maximum principle for optimal control problems in discontinuous systems, which takes into account a situation when a solution of the dynamic system lies on the switching surface. We have shown that the new maximum principle is stronger than known optimality conditions and contains new conditions which are essential and do not follow from other known optimality conditions.

## References

1. Arutyunov, A. V.: Optimality Conditions: Abnormal And Degenerate Problems, Mathematics And Its Applications. Vol. 526. Kluwer (2000)

2. Ashchepkov, L. T.: Optimal Control of Discontinuous Systems. Nauka, Sibirsk. Otdel., Novosibirsk (1987)

3. Bock, H. G., Longman, R. W.: Optimal Control of Velocity Profiles for Minimization of Energy Consumption in the New York Subway System. In: Proc. 2nd IFAC Workshop on Control Applications of Nonlinear Programming and Optimization, pp. 34–43. IFAC (1980)

4. Boltyanskii, V.: The Maximum Principle for Systems with Discontinuous Right-Hand Side. In: Proceeding of the 6th IASTED International Conference “Intelligent Systems and Control”, pp. 384 – 387 (2004)

5. Branicky, M. S.: Introduction to Hybrid Systems. In: Handbook of Networked and Embedded Control Systems. pp. 91 – 116. Birkhäuser (2005)
6. Donchev, T., Farkhi, E., Mordukhovich, B. S.: Discrete Approximations, Relaxation, and Optimization of One-Sided Lipschitzian Differential Inclusions in Hilbert Spaces. J. Differential Equations 243, pp. 301–328 (2007)

7. Dubovitskii, A. Ya., Dubovitskii, V. A.: Criterion of Existence of a Substantive Maximum Principle in a State Constraint Problem. Differential Equations 31(10), pp. 1611-1616 (1995)

8. Filippov, A. F.: Differential Equations with Discontinuous Right-Hand Sides. Kluwer (1988)

9. Hartl, R., Sethi, S., Vickson, R.; A Survey of the Maximum Principles for Optimal Control Problems with State Constraints. SIAM Review 37(2), pp. 181-218 (1995)

10. Kirches, Ch., Sager, S., Bock, H. G., Schlöder, J. P.: Time-Optimal Control of Autotobile Test Drives with Gear Shifts. Optimal Control Applications and Methods 31(2), pp. 137-153 (2010)

11. Kostina, E. A., Kostyukova, O. I., Schmidt W.: New Maximum Principle for Optimal Control Problems in Discontinuous Dynamic Systems: Proof and Examples. Technical Report, University of Marburg (2012), available online at http://www.uni-marburg.de/fb12/forschung/berichte/berichtemathe/pdfbfm/bfm12-03.pdf

12. Kugushev, E. I.: Necessary Optimality Conditions for Systems Described by Equations with Discontinuous Right-Hand Side. Vestn. Mosk. Gos. Univ., Ser. math. mechanics. 2, pp. 83-90 (1974)

13. Lee, E. B., Markus, L.: Foundations of Optimal Control Theory. Wiley (1967)

14. Mordukhovich, B. S.: Variational Analysis and Generalized Differentiation, II: Applications. Fundamental Principles of Mathematical Sciences, Vol. 331. Springer, Heidelberg (2006)

15. Morozov, S. F., Sumin, M. I.: Optimal Control of Sliding Modes of Discontinuous Dynamical Systems. Izv. Vyssh. Uchebn. Zaved. Mat. 392(1), pp. 53-61 (1990)

16. Oberle, H. J., Rosendahl, R.: Numerical Computation of a Singular-State Subarc in an Economic Optimal Control Model. Optimal Control Applications and Methods, 27(4), pp. 211-235 (2006)

17. Pontryagin, L. S., Boltyanskij, V. G., Gamkrelidze, R. V., Mishchenko, E. F.: Selected Works, Vol. 4, The Mathematical Theory of Optimal Processes. In: Gamkrelidze R. V. (ed.) Classics of Soviet Mathematics, Gordon and Breach Science Publishers, New York (1986) (Translation from Russian edition, Nauka, Moscow, 1961)

18. Stewart, D. E., Anitescu, M.: Optimal Control of Systems with Discontinuous Differential Equations. Numerische Mathematik, 114(4), pp. 653-695 (2010)

19. Utkin, V.: Sliding Modes in Control and Optimization. Springer, Heidelberg (1992)

20. Vinter, R. B.: Optimal Control. Birkhäuser (2000)