# Robust Planar Pursuit of Uncertain Evader: Differential Games Approach

Valery Y. Glizer and Vladimir Turetsky

January 25, 2024
To the memory of our senior colleague Professor Josef Shinar
<table><tr><td>3.5.3</td><td>Solution of the SGBC</td><td>51</td></tr><tr><td>3.5.4</td><td>Solution of the VGBC</td><td>73</td></tr><tr><td>3.6</td><td>Proofs of Theorems 3.4.7-3.4.10 and 3.5.13-3.5.15</td><td>74</td></tr><tr><td>3.6.1</td><td>Proof of Theorem 3.4.7</td><td>74</td></tr><tr><td>3.6.2</td><td>Proof of Theorem 3.4.9</td><td>79</td></tr><tr><td>3.6.3</td><td>Proof of Theorem 3.4.10</td><td>79</td></tr><tr><td>3.6.4</td><td>Proof of Theorem 3.5.13</td><td>79</td></tr><tr><td>3.6.5</td><td>Proof of Theorem 3.5.15</td><td>83</td></tr><tr><td>3.7</td><td>Concluding Remarks and Literature Review</td><td>84</td></tr></table>

<table><tr><td>4</td><td>Scalar Zero-Sum Linear-Quadratic Differential Game with Cheap Controls</td><td>89</td></tr><tr><td>4.1</td><td>Introduction</td><td>89</td></tr><tr><td>4.2</td><td>Game Formulation</td><td>90</td></tr><tr><td>4.3</td><td>Main Definitions</td><td>90</td></tr><tr><td>4.4</td><td>Feedback Saddle-Point Solution of the LQGCC Uniform in the Cheap Control Parameter</td><td>92</td></tr><tr><td>4.5</td><td>Concluding Remarks and Literature Review</td><td>94</td></tr></table>

<table><tr><td>5</td><td>Differential Games Based Robust Capture Pursuit</td><td>97</td></tr><tr><td>5.1</td><td>Introduction</td><td>97</td></tr><tr><td>5.2</td><td>Planar Pursuit-Evasion Game with First-Order Dynamics and Bounded Controls of Players</td><td>98</td></tr><tr><td>5.3</td><td>Robust Capture Controls and Capture Zones</td><td>102</td></tr><tr><td>5.3.1</td><td>Main Assertions</td><td>103</td></tr><tr><td>5.3.2</td><td>Construction of the Function L(t, z) Based on Linear-Quadratic Differential Game with Cheap Controls</td><td>107</td></tr><tr><td>5.4</td><td>Numerical Examples: Pursuit-Evasion Engagement of Two Flying Vehicles</td><td>110</td></tr><tr><td>5.4.1</td><td>Example 1: Two Changes of Sign of the Determining Function</td><td>110</td></tr><tr><td>5.4.2</td><td>Example 2: One Change of Sign of the Determining Function</td><td>118</td></tr><tr><td>5.4.3</td><td>Evaluation of the Robust Capture Controls in Numerical Simulations</td><td>124</td></tr><tr><td>5.5</td><td>Proofs of Theorems 5.3.1-5.3.5, Lemmas 5.3.3-5.3.10 and Proposition 5.3.12</td><td>133</td></tr><tr><td>5.5.1</td><td>Proof of Theorem 5.3.1</td><td>133</td></tr><tr><td>5.5.2</td><td>Proof of Lemma 5.3.3</td><td>136</td></tr><tr><td>5.5.3</td><td>Proof of Theorem 5.3.4</td><td>136</td></tr><tr><td>5.5.4</td><td>Proof of Theorem 5.3.5</td><td>138</td></tr></table>
CONTENTS

5

<table><tr><td>5.5.5 Proof of Lemma 5.3.10</td><td>140</td></tr><tr><td>5.5.6 Proof of Proposition 5.3.12</td><td>141</td></tr><tr><td>5.6 Concluding Remarks and Literature Review</td><td>141</td></tr></table>
# Chapter 1

# Introduction

The problem of an optimal pursuit of a maneuvering evader has a consid-
erable interest in theory and different applications. This problem arises
in such real-life areas as: interception of a maneuvering target (see e.g.
[2, 3, 7, 8, 14, 15, 16, 17, 22, 23, 24, 25, 27, 28, 31, 32, 37]); defender-
attacker-target problem (see e.g. [4, 18, 19, 21, 35]); navigation in presence
of outer disturbances (see e.g. [5, 38]); airplane landing under wind shear
conditions (see e.g. [20]); controllability of a system with uncertainty (see
e.g. [10, 12, 30]); tracking in presence of disturbances (see e.g. [34, 36]).

The aforementioned optimal pursuit problem can be formulated in two
different versions. If the evader's behaviour is predictable, we can (based on
this prediction) formulate an optimal control problem for the pursuer (see e.g.
[6, 13] and references therein). Without such an information, we should de-
sign an admissible pursuer's control which provides an acceptable outcome
of the pursuit against any admissible evader's behaviour. In this case, we
should formulate the optimal pursuit problem as a robust pursuit one. The
best outcome of the robust pursuit is the so called "capture", i.e., the situa-
tion where a properly defined distance between the pursuer and the evader
(the so called "miss distance") becomes zero in a prescribed time against any
admissible evader's behaviour. To reach such an outcome, we can use one of
the following approaches. First approach consists in the formulation of the
robust pursuit problem as a dynamic/differential zero-sum game where the
miss distance is the cost function. In this game, the pursuer is the minimizing
player (the minimizer), while the evader is the maximizing player (the maxi-
mizer). If the minimizer's optimal feedback control provides zero upper value
of the game from some set of the initial game's positions then this control
is the required robust control guaranteeing the capture from any of these
positions against any admissible evader's behaviour. The aforementioned
minimizer's control and the corresponding set of the initial game's positions
are called a "robust capture control" and a "capture zone/set", respectively. This approach can be found for instance in [16, 24, 26]. The advantage of the first approach is that it directly yields the robust capture control and its capture zone which is the maximal possible one. However, this control is as a rule of a bang-bang structure leading to sliding modes and producing a chattering phenomena which is the drawback of this approach. The second approach consists in a preliminary design of a pursuer's feedback control with desirable properties like continuity/smoothness/linearity with respect to the state variable. Then, subject to proper additional conditions imposed on this control, its capture zone is constructed. This approach can be found for instance in [10, 12, 28, 32, 33, 37]. The advantage of the second approach is the continuity/smoothness (or even linearity) of the designed robust capture control which does not lead to sliding modes and chattering. However, the capture zone of this control is smaller than the one in the first approach which is the drawback of this approach. The third approach is a combination of the first and the second approaches. Namely, this approach consists in a design of a pursuer's feedback control, continuous/smooth/linear with respect to the state variable, capture zone of which coincides with the one obtained in the first approach. Thus, such a designed robust capture control does not lead to sliding modes and chattering, while has the maximal possible capture zone. This approach can be found for instance in [11, 29, 31].

In the present book, we consider the robust pursuit problem in a plane. Planar geometry of a pursuit-evasion is natural if the engagement takes place on ground/sea (see e.g. [17]). If a pursuit-evasion takes place either in the atmosphere or in the space, the 3D geometry is natural for such an engagement. However, due to proper conditions (see e.g. [1, 17, 26, 37]), the 3D-engagement can be decoupled into two planar engagements in perpendicular planes which can be analyzed independently of each other.

To analyze and solve the robust planar pursuit problem, we use in this book the aforementioned first and third approaches which yield several different robust capture controls in an explicit analytical form. Then, these controls are compared with each other in numerical simulations.

The book is the extension and generalization of the authors' results published in the journal and conference papers [9, 28, 29, 30, 31, 32, 37].

The book consists of the introduction chapter (the present one) and four body chapters (Chapters 2-5). In Chapter 2, the nonlinear mathematical model of the planar pursuit-evasion engagement is constructed. Linearization of this model in a small neighbourhood of the initial collision course of the pursuer and the evader is carried out. The notion of robust capture pursuit and corresponding definitions are introduced. In Chapter 3, the finite-horizon two-person zero-sum differential game with linear dynamics
and hard-bounded controls of the players is considered. Saddle-point solutions (open-loop and feedback ones) to this game are obtained. In Chapter 4, the finite-horizon two-person zero-sum linear quadratic differential game with scalar dynamics and cheap controls of both players is considered. Condition for the existence of the feedback saddle-point solution to this game, uniformly valid with respect to the arbitrarily small cost of the players' controls, is established. The solution itself is derived. In Chapter 5, based on the results of Chapters 3-4, three different robust capture controls in the pursuit-evasion engagement, introduced in Chapter 2, are designed and their capture zones are constructed. Real-life examples of pursuit-evasion engagement of two flying vehicles, illustrating the theoretical results, are presented.

For the sake of better readability, each chapter is organized to be self-contained as much as possible. Moreover, each of Chapters 2-5 contains an introduction, a list of notations used in the chapter, concluding remarks with a brief literature review on the topic of the chapter and a corresponding bibliography. Technically complicated proofs of theorems/lemmas/propositions are placed into separate sections/subsections. Due to such an organization of the books chapters, each chapter (excepting Chapter 5) can be studied independently of the others. Chapter 5 uses the results of the previous body chapters with the corresponding references.

The book can be useful for researchers and engineers, working in such areas as Applied Mathematics, Aerospace Engineering, Control Engineering and Mechanical Engineering. Also, the book can be helpful for graduate students in these areas. Moreover, due to the detailed and systematic presentation of the theoretical material accompanied by illustrative examples, different parts of the book can be used in various graduate university's courses in the aforementioned areas.
Bibliography

[1] Adler, F.P. Missile guidance by three-dimensional proportional navigation. J. Appl. Phys. **27**, 500-507 (1956).

[2] Ben-Asher, J., Yaesh, I.: Advances in Missile Guidance Theory, Vol. 180 in Progress in Astronautics and Aeronautics, AIAA, Inc., Reston, VA, USA (1998)

[3] Bryson, A.E., Ho, Y.C.: Applied Optimal Control, Hemisphere, New York, NY, USA (1975)

[4] Casbeer, D.W., Garcia, E., Pachter, M.: The target differential game with two defenders. Journal of Intelligent & Robotic Systems, **89**, 87-106 (2017)

[5] Clements, J.C.: Minimum-time turn trajectories to fly-to points. Optim. Control Appl. Methods **11**, 39-50 (1990)

[6] Glizer, V.Y.: Optimal planar interception with fixed end conditions: closed-form solution. J. Optim. Theory Appl. **88**, 503-539 (1996)

[7] Glizer, V.Y.: Homicidal chauffeur game with target set in the shape of a circular angular sector: conditions for existence of a closed barrier. J. Optim. Theory Appl. **101**, 581-598 (1999)

[8] Glizer, V.Y., Shinar, J.: On the structure of a class of time-optimal trajectories. Optimal Control Appl. Methods **14**, 271-279 (1993)

[9] Glizer, V.Y., Turetsky, V.: Complete solution of a differential game with linear dynamics and bounded controls. Appl. Math. Res. Express. AMRX **2008**, Article ID abm012 (2008)

[10] Glizer, V.Y., Turetsky, V.: Robust Controllability of Linear Systems, Nova Science Publishers, New York, NY, USA (2012)
[11] Glizer, V.Y., Turetsky, V., Fridman, L., Shinar, J.: History-dependent modified sliding mode interception strategies with maximal capture zone. J. Franklin Inst. 349, 638–657 (2012)

[12] Glizer, V.Y., Turetsky, V., Shinar, J.: Robust controllability sets of linear and saturated linear strategies with discontinuous gains. In: Proceedings of the 6th International Conference on Integrated Modeling and Analysis in Applied Control and Automation, pp. 11–20, Vienna, Austria (2012)

[13] Guelman, M., Shinar, J.: Optimal guidance law in the plane. J. Guid. Control Dyn. 7, 471–476 (1984)

[14] Gutman, S.: On optimal guidance for homing missiles. J. Guid. Control 3, 296-300 (1979)

[15] Gutman, S.: Applied Min-Max Approach to Missile Guidance and Control, Series Progress in Astronautics and Aeronautics, Vol. 209, AIAA, Inc., Reston, VA (2006)

[16] Gutman, S., Leitmann G.: Optimal strategies in the neighbourhood of a collision course. AIAA J. 14, 1210-1212 (1976)

[17] Isaacs, R.: Differential Games, John Wiley, New York, NY, USA (1965)

[18] Li, D., Cruz, J.B.: Defending an asset: a linear quadratic game approach. IEEE Trans. Aerosp. Electron. Syst. 47, 1026-1044 (2011)

[19] Lipman, Y., Shinar, J.: A linear pursuit-evasion game with a state constraint for a highly maneuverable evader. In: G.J. Olsder (ed.) New Trends in Dynamic Games and Applications. Annals of the International Society of Dynamic Games, vol. 3, 143-164. Birkhauser, Boston, MA (1995)

[20] Patsko, V.: Switching surfaces in linear differential games with fixed termination time. J. Appl. Math. Mech. 68, 583-595 (2004)

[21] Perelman, A., Shima, T., Rusnak, I.: Cooperative differential games strategies for active aircraft protection from a homing missile. J. Guid. Control Dyn. 34, 761-773 (2011)

[22] Shima, T.: Intercept-angle guidance. J. Guid. Control Dyn. 34, 484–492 (2011)
[23] Shima, T., Golan, O.M.: Head pursuit guidance. J. Guid. Control Dyn. **30**, 1437–1444 (2007)

[24] Shima, T., Shinar, J.: Time-varying linear pursuit-evasion game models with bounded controls. J. Guid. Control Dyn. **25**, 425–432 (2002)

[25] Shinar, J.: Solution techniques for realistic pursuit-evasion games. In: C.T. Leonides (ed.) Advances in Control and Dynamic Systems, vol. 17, 63-124. Academic Press, New York, NY, USA (1981)

[26] Shinar, J., Glizer, V.Y., Turetsky, V.: The effect of pursuer dynamics on the value of linear pursuit-evasion games with bounded controls. In: V. Křivan and G. Zaccour (eds.) Advances in Dynamic Games. Annals of the International Society of Dynamic Games, vol. **13**, 313–350. Springer, Switzerland (2013)

[27] Shinar, J., Turetsky, V., Glizer, V.Y., Ianovsky, E.: Solvability of linear-quadratic differential games associated with pursuit-evasion problems. Int. Game Theory Rev. **10**, 481-515 (2008)

[28] Turetsky, V.: Upper bounds of the pursuer control based on a linear-quadratic differential game. J. Optim. Theory Appl. **121**, 163-191 (2004)

[29] Turetsky, V., Glizer, V.Y.: Saturated pursuer control based on a linear-quadratic differential game. In: Proceedings of the 44th Israel Annual Conference on Aerospace Sciences, Tel-Aviv - Haifa, Israel (2004)

[30] Turetsky, V., Glizer, V.Y.: Robust state-feedback controllability of linear systems to a hyperplane in a class of bounded controls. J. Optim. Theory Appl. **123**, 639-667 (2004)

[31] Turetsky, V., Glizer, V.Y.: Continuous feedback control strategy with maximal capture zone in a class of pursuit games. Int. Game Theory Rev. **7**, 1–24 (2005)

[32] Turetsky, V., Glizer, V.Y.: Robust solution of a time-variable interception problem: a cheap control approach. Int. Game Theory Rev. **9**, 637-655 (2007)

[33] Turetsky, V., Glizer, V.Y.: Feasibility sets of nonlinear strategies in scalarizable robust transfer problem. In: Proceedings of the 10th IASTED International Conference on Intelligent Systems and Control, pp. 434-439, Cambridge, MA, USA (2007)
[34] Turetsky, V., Glizer, V.Y.: Robust trajectory tracking problem: cheap control approach. Proceedings of the 5th International Conference on Integrated Modeling and Analysis in Applied Control and Automation, pp. 26–34, Rome, Italy (2011)

[35] Turetsky, V., Glizer, V.Y.: Open-loop solution of a defender-attacker-target game: penalty function approach. J. Control Decis. **6**, 166-190 (2019)

[36] Turetsky, V., Glizer, V.Y., Shinar, J.: Robust trajectory tracking: differential game/cheap control approach. Internat. J. Systems Sci. **45**, 2260–2274 (2014)

[37] Turetsky, V., Shinar, J.: Missile guidance laws based on pursuit-evasion game formulations. Automatica J. IFAC **39**, 607-618 (2003)

[38] Zermelo, E.: Über das navinationsproblem bei ruhender oder veränderlich windverteilung. Z. Angew. Math. Mech. **11**, 114–124 (1931)
# Chapter 2

# Mathematical Model of Planar Pursuit-Evasion Engagement

## 2.1 Introduction

In this chapter, we derive the mathematical model of pursuit-evasion engagement between two participants. We call these participants the pursuer and the evader. The derivation of the aforementioned mathematical model is based on the assumptions presented in Section 2.2 and Subsection 2.5.1. Subject to the assumptions of Section 2.2, the nonlinear kinematics model of the engagement is constructed and its linearization is carried out. Then, due to the assumption of Subsection 2.5.1, the complete mathematical model is obtained. The latter consists of the linear kinematics and the first-order control dynamics models. Based on the complete mathematical model of the pursuit-evasion engagement, in Section 2.6 the notion of the robust capture pursuit and corresponding definitions are introduced.

The following main notations are applied in the chapter.

1. $E^n$ is the $n$-dimensional real Euclidean space, thus $E^1$ is the set of all real numbers.

2. $O(\alpha)$, where $\alpha > 0$ is a small value, denotes a positive value of the same order of smallness as $\alpha$, i.e., $\lim_{\alpha \to +0} O(\alpha)/\alpha = c$ and $c$ is a positive number.

3. $\Theta_1 \times \Theta_2$, where each of $\Theta_k$, ($k = 1, 2$) is some set either of numbers, or of vectors, denotes the direct product of these sets, i.e., the set of all pairs $(\theta_1, \theta_2)$ with any entries $\theta_1 \in \Theta_1$ and $\theta_2 \in \Theta_2$.
## 2.2 Basic Assumptions of the Engagement Kinematics

The mathematical model of the engagement is constructed subject to the following assumptions:

(A1) the engagement is between two participants called the pursuer and the evader;

(A2) the engagement takes place in a plane;

(A3) the pursuer and the evader are considered as points (point-masses) in this plane;

(A4) the origin of the right-handed orthogonal Cartesian coordinate system in the aforementioned plane coincides with the pursuer's initial position, the X-axis coincides with the pursuer's initial line-of-sight and it is directed from the pursuer's initial position to the evader's initial position;

(A5) the velocity magnitudes (speeds) of both participants are known continuously differentiable functions of time, not becoming zero during the engagement;

(A6) the trajectories of both participants are close to their initial collision course trajectories.

## 2.3 Initial Collision Course Trajectories of the Pursuer and the Evader

The geometry of the initial collision course is presented in Fig. 2.1.

In this figure, the points $P_0$ and $E_0$ represent the initial positions of the pursuer and the evader, respectively. The initial velocity vectors of the engagement participants are denoted by $\vec{V}_{P0}$ and $\vec{V}_{E0}$, i.e., $\vec{V}_{P0} = \vec{V}_P(t_0)$, $\vec{V}_{E0} = \vec{V}_E(t_0)$, ($t_0$ is an initial time instant of the engagement). Here and in the sequel, $\vec{V}_P(t)$ and $\vec{V}_E(t)$ are current velocity vectors of the pursuer and the evader, while the magnitudes of these vectors (the speeds of the pursuer and the evader) are denoted by $V_P(t)$ and $V_E(t)$, respectively. By $\phi_{P0}$ and $\phi_{E0}$, we denote the initial aspect angles of the pursuer and the evader, i.e., the angles between the corresponding velocity vectors ($\vec{V}_{P0}$ and $\vec{V}_{E0}$) and the X-axis. The vector $\vec{V}_{PC}$ shows the direction of the pursuer's motion in the initial collision course, and $\phi_{PC}$ is the angle between this vector and the X-axis. The collision point of the initial collision course is denoted by $C$, while $X_C$ and $Y_C$ are the respective coordinates of this point. By $R_0 > 0$, we denote the initial range between the pursuer and the evader, i.e., $R_0 = |E_0P_0|$.
2.3. INITIAL COLLISION COURSE OF THE PURSUER AND THE EVADER17

<figure><img src="image_1.png" /><figcaption>Figure 2.1: Geometry of the initial collision course</figcaption></figure>

Due to Fig. 2.1, the trajectories of the pursuer and the evader in the initial collision course are the segments of the straight lines $P_0C$ and $E_0C$, respectively. The motions of the pursuer and the evader along these trajectories are in the directions of $\vec{V}_{PC}$ and $\vec{V}_{E0}$, respectively, but with the speeds $V_P(t)$ and $V_E(t)$.

It is important to note that the trajectories of the pursuer and the evader in the initial collision course are, in general, virtual (not real). However in the next section, based on these trajectories, the mathematical model of the real kinematics of the engagement between the pursuer and the evader is derived. Also, it should be noted that the pursuer, being aware of the evader's angle $\phi_{E0} \in [0, 2\pi)$ and assuming the evader's motion in the direction of $\vec{V}_{E0}$, chooses its own angle $\phi_{PC}$ (and, therefore, the direction of $\vec{V}_{PC}$) in the initial collision course preferably such that the collision will happen as soon as possible. However, another choice of the angle $\phi_{PC}$, providing the collision, also can be acceptable from the practical viewpoint. As it was aforementioned, the engagement starts at the time instant $t_0$. Let $t_C > t_0$ be the collision time-instant of the initial collision course, i.e., the time instant when the pursuer and the evader, moving along their initial collision course trajectories, reach simultaneously the collision point C.

Thus, using Fig. 2.1 and the aforementioned notations and notion of the initial collision course, we directly obtain the following equations:

$$X_C = \int_{t_0}^{t_C} V_P(t)dt \cos(\phi_{PC}), \qquad (2.1)$$

and

$$X_C + \int_{t_0}^{t_C} V_E(t)dt \cos(\pi - \phi_{E0}) = R_0, \qquad (2.2)$$
or, using (2.1) and the equality $\cos(\pi - \phi_{E0}) = -\cos(\phi_{E0})$, we have after a
simple rearrangement

$$
\int_{t_0}^{t_C} V_P(t) dt \cos(\phi_{PC}) = R_0 + \int_{t_0}^{t_C} V_E(t) dt \cos(\phi_{E0}). \quad (2.3)
$$

The equation (2.3) is a nonlinear algebraic equation with respect to two
unknowns, the angle $\phi_{PC}$ and the time instant $t_C$. To obtain values of these
unknowns, we need one more equation with respect to $\phi_{PC}$ and $t_C$. This
equation is

$$
Y_C = \int_{t_0}^{t_C} V_P(t) dt \sin(\phi_{PC}) = \int_{t_0}^{t_C} V_E(t) dt \sin(\pi - \phi_{E0}), \quad (2.4)
$$

or, taking into account that $\sin(\pi - \phi_{E0}) = \sin(\phi_{E0})$,

$$
\int_{t_0}^{t_C} V_P(t) dt \sin(\phi_{PC}) = \int_{t_0}^{t_C} V_E(t) dt \sin(\phi_{E0}). \quad (2.5)
$$

The equations (2.3) and (2.5) constitute the system of two nonlinear al-
gebraic equations with respect to two unknowns, $\phi_{PC}$ and $t_C$. Let us solve
this system. Raising both parts of these equations in the square power,
adding separately the left-hand sides and the right-hand sides of the result-
ing equations, and taking into account that $\cos^2(\phi_{PC}) + \sin^2(\phi_{PC}) = 1$ and
$\cos^2(\phi_{E0}) + \sin^2(\phi_{E0}) = 1$, we obtain

$$
\left(\int_{t_0}^{t_C} V_P(t) dt\right)^2 = R_0^2 + 2R_0 \int_{t_0}^{t_C} V_E(t) dt \cos(\phi_{E0}) + \left(\int_{t_0}^{t_C} V_E(t) dt\right)^2, \quad (2.6)
$$

or, taking into account that $V_P(t) > 0$, $t \ge t_0$,

$$
\int_{t_0}^{t_C} V_P(t) dt = \left[ R_0^2 + 2R_0 \int_{t_0}^{t_C} V_E(t) dt \cos(\phi_{E0}) + \left( \int_{t_0}^{t_C} V_E(t) dt \right)^2 \right]^{1/2} . \quad (2.7)
$$

The equation (2.7) is a nonlinear algebraic equation with respect to $t_C$.
We assume that this equation has at least one positive solution, and let
$t_C = t_{C,min} > t_0$ be the smallest positive solution of (2.7).

Substituting $t_C = t_{C,min}$ into (2.3) and (2.5), and resolving the resulting
equations with respect to $\cos(\phi_{PC})$ and $\sin(\phi_{PC})$, respectively, we obtain

$$
\cos(\phi_{PC}) = \frac{R_0 + \int_{t_0}^{t_{C,min}} V_E(t) dt \cos(\phi_{E0})}{\int_{t_0}^{t_{C,min}} V_P(t) dt}, \quad (2.8)
$$

$$
\sin(\phi_{PC}) = \frac{\int_{t_0}^{t_{C,min}} V_E(t) dt \sin(\phi_{E0})}{\int_{t_0}^{t_{C,min}} V_P(t) dt}. \quad (2.9)
$$
## 2.4. MATHEMATICAL MODEL OF KINEMATICS OF THE ENGAGEMENT BETWEEN THE PURSUER AND THE EVADER

**Remark 1.** Note that, subject to the inequalities

$$ \left|R_0 + \int_{t_0}^{t_{C,min}} V_E(t)dt \cos(\phi_{E0})\right| \le \int_{t_0}^{t_{C,min}} V_P(t)dt \quad (2.10) $$

and

$$ \left|\int_{t_0}^{t_{C,min}} V_E(t)dt \sin(\phi_{E0})\right| \le \int_{t_0}^{t_{C,min}} V_P(t)dt, \quad (2.11) $$

the expressions (2.8) and (2.9) for $\cos(\phi_{PC})$ and $\sin(\phi_{PC})$ are feasible. Moreover, these expressions determine the unique angle $\phi_{PC}$ satisfying the inequality

$$ 0 \le \phi_{PC} < 2\pi. \quad (2.12) $$

Thus, we have solved the system (2.3), (2.5) with respect to $\phi_{PC}$ and $t_C$. The solution $(\phi_{PC}, t_{C,min})$ of this system, along with the initial evader's angle $\phi_{E0}$, uniquely determines the trajectories of the pursuer and the evader in the initial collision course.

**Remark 2.** It should be noted that the system (2.3), (2.5) has been derived and its solution has been obtained in the case of the initial collision course presented in Fig. 2.1. Other cases of the initial collision course can be treated similarly.

## 2.4 Mathematical Model of Kinematics of the Engagement Between the Pursuer and the Evader Along the Initial Collision Course

First, we derive the mathematical model of the kinematics separately for each of the engagement participants. We start with the pursuer's kinematics. Due to the assumption (A6) (see Section 2.2), the current pursuer's position $P(t) = (x_P(t), y_P(t))$, $t \in [t_0, t_C]$ is close to the corresponding point $P_C(t) = (x_{PC}(t), y_{PC}(t))$, $t \in [t_0, t_C]$ on its initial collision course trajectory. More precisely, we assume that the distance between these two points is negligible, i.e., $P(t) \approx P_C(t)$. Therefore, in the construction of the mathematical model of the pursuer's kinematics, we set that $P(t) = P_C(t)$ (see Fig. 2.2).

In this figure, $\alpha_P(t) = \phi_P(t) - \phi_{PC}$, while $\phi_P(t)$, $t \in [t_0, t_C]$ is the current pursuer's aspect angle, i.e., the angle between the current pursuer's velocity vector $\vec{v}_P(t)$ and the X-axis. By $\vec{a}_P(t)$, $t \in [t_0, t_C]$, we denote the pursuer's current lateral acceleration vector. This vector is perpendicular to the current pursuer's velocity vector. By $a_P(t)$, we denote the magnitude of the
<figure><img src="image_2.png" /><figcaption>Figure 2.2: Geometry of the engagement along the initial collision course</figcaption></figure>

vector $\vec{a}_P(t)$ taken with the sign "+" if $\vec{a}_P(t)$ rotates the vector $\vec{V}_P(t)$ counterclockwise, and taken with the sign "-" if $\vec{a}_P(t)$ rotates the vector $\vec{V}_P(t)$ clockwise. Using Fig. 2.2 and the aforementioned notations, we directly obtain the following equations:

$$ \frac{dx_P(t)}{dt} = V_P(t) \cos (\phi_{PC} + \alpha_P(t)), \quad t \in [t_0, t_C], \qquad (2.13) $$

$$ \frac{dy_P(t)}{dt} = V_P(t) \sin (\phi_{PC} + \alpha_P(t)), \quad t \in [t_0, t_C], \qquad (2.14) $$

$$ \frac{d\alpha_P(t)}{dt} = \frac{a_P(t)}{V_P(t)}, \quad t \in [t_0, t_C]. \qquad (2.15) $$

Differentiating the equation (2.14) with respect to $t$ and using the equation (2.15), we obtain

$$ \frac{d^2 y_P(t)}{dt^2} = \frac{dV_P(t)}{dt} \sin (\phi_{PC} + \alpha_P(t)) + a_P(t) \cos (\phi_{PC} + \alpha_P(t)), \quad t \in [t_0, t_C]. \qquad (2.16) $$

Thus, the equations (2.13)-(2.16) describe the pursuers kinematics along its initial collision course trajectory.

Proceed to the derivation of the evader's kinematics (see Fig. 2.2). In this figure, $E(t) = (x_E(t), y_E(t))$, $t \in [t_0, t_C]$ is the current evader's position, $\alpha_E(t) = \phi_E(t) - \phi_{E0}$, where $\phi_E(t)$, $t \in [t_0, t_C]$ is the angle between the current evader's velocity vector $\vec{V}_E(t)$ and the X-axis. By $\vec{a}_E(t)$, $t \in [t_0, t_C]$, we denote the evader's current lateral acceleration vector. This vector is perpendicular to the current evader's velocity vector. By $a_E(t)$, we denote
the magnitude of the vector $\vec{a}_E(t)$ taken with the sign "+" if $\vec{a}_E(t)$ rotates the vector $\vec{V}_E(t)$ counterclockwise, and taken with the sign "-" if $\vec{a}_E(t)$ rotates the vector $\vec{V}_E(t)$ clockwise. Similarly to the derivation of the pursuer's kinematics, we are based on the assumption that $E(t) = E_C(t)$, $t \in [t_0, t_C]$, where $E_C(t) = (x_{EC}(t), y_{EC}(t))$, $t \in [t_0, t_C]$ is the current evader position on its initial collision course trajectory. Now, using these notations, we obtain (similarly to the equations (2.13)-(2.16)) the following equations for the evader's kinematics along its initial collision course trajectory:

$$ \frac{dx_E(t)}{dt} = V_E(t) \cos(\phi_{E0} + \alpha_E(t)), \quad t \in [t_0, t_C], \qquad (2.17) $$

$$ \frac{dy_E(t)}{dt} = V_E(t) \sin(\phi_{E0} + \alpha_E(t)), \quad t \in [t_0, t_C], \qquad (2.18) $$

$$ \frac{d\alpha_E(t)}{dt} = \frac{a_E(t)}{V_E(t)}, \quad t \in [t_0, t_C], \qquad (2.19) $$

$$ \frac{d^2 y_E(t)}{dt^2} = \frac{dV_E(t)}{dt} \sin(\phi_{E0} + \alpha_E(t)) + a_E(t) \cos(\phi_{E0} + \alpha_E(t)), \quad t \in [t_0, t_C]. \qquad (2.20) $$

The sets of the equations (2.13)-(2.16) and (2.17)-(2.20) are sets of nonlinear differential equations, because of nonlinear dependence of their right-hand sides on $\alpha_P(t)$ and $\alpha_E(t)$, respectively. However, based on the assumption (A6) (see Section 2.2), these systems can be linearized. Namely, due to this assumption, the absolute values of the angles $\alpha_P(t)$ and $\alpha_E(t)$, $t \in [t_0, t_C]$ are small, yielding

$$ \begin{aligned} \cos(\phi_{PC} + \alpha_P(t)) &\approx \cos(\phi_{PC}), & t \in [t_0, t_C], \\ \sin(\phi_{PC} + \alpha_P(t)) &\approx \sin(\phi_{PC}) + \cos(\phi_{PC})\alpha_P(t), & t \in [t_0, t_C], \end{aligned} \qquad (2.21) $$

$$ \begin{aligned} \cos(\phi_{E0} + \alpha_E(t)) &\approx \cos(\phi_{E0}), & t \in [t_0, t_C], \\ \sin(\phi_{E0} + \alpha_E(t)) &\approx \sin(\phi_{E0}) + \cos(\phi_{E0})\alpha_E(t), & t \in [t_0, t_C]. \end{aligned} \qquad (2.22) $$

**Remark 3.** It should be noted that the accuracies of the approximations in (2.21) for the functions $\cos$ and $\sin$ are $O(|\alpha_P(t) \sin(\phi_{PC} + \alpha_P(t)/2)|)$ and $O(|\alpha_P^2(t) \sin(\phi_{PC} + \varphi(\alpha_P(t)))|)$, $t \in [t_0, t_C]$. The accuracies of the approximations in (2.22) for $\cos$ and $\sin$ are $O(|\alpha_E(t) \sin(\phi_{E0} + \alpha_E(t)/2)|)$ and $O(|\alpha_E^2(t) \sin(\phi_{E0} + \varphi(\alpha_E(t)))|)$, $t \in [t_0, t_C]$. The function $\varphi(\alpha_i(t))$, ($i = P, E$), appearing in the accuracies, is the angle satisfying the equalities

$$ \cos(\varphi(\alpha_i(t))) = \frac{1}{\sqrt{1 + \alpha_i^2(t)/9}}, \quad \sin(\varphi(\alpha_i(t))) = \frac{\alpha_i(t)/3}{\sqrt{1 + \alpha_i^2(t)/9}}, \quad t \in [t_0, t_C] $$
and the inequality $0 \le \varphi(\alpha_i(t)) < 2\pi$, $i = P, E, t \in [t_0, t_C]$.

Using the approximations (2.21) and (2.22), we can linearize the sets of the equations (2.13)-(2.16) and (2.17)-(2.20) as follows:

$$ \frac{dx_P(t)}{dt} = V_P(t) \cos(\phi_{PC}), \quad t \in [t_0, t_C], \qquad (2.23) $$

$$ \frac{dy_P(t)}{dt} = V_P(t)(\sin(\phi_{PC}) + \cos(\phi_{PC})\alpha_P(t)), \quad t \in [t_0, t_C], \qquad (2.24) $$

$$ \frac{d\alpha_P(t)}{dt} = \frac{a_P(t)}{V_P(t)}, \quad t \in [t_0, t_C], \qquad (2.25) $$

$$ \frac{d^2 y_P(t)}{dt^2} = \frac{dV_P(t)}{dt}(\sin(\phi_{PC}) + \cos(\phi_{PC})\alpha_P(t)) + a_P(t) \cos(\phi_{PC}), \quad t \in [t_0, t_C], \qquad (2.26) $$

and

$$ \frac{dx_E(t)}{dt} = V_E(t) \cos(\phi_{E0}), \quad t \in [t_0, t_C], \qquad (2.27) $$

$$ \frac{dy_E(t)}{dt} = V_E(t)(\sin(\phi_{E0}) + \cos(\phi_{E0})\alpha_E(t)), \quad t \in [t_0, t_C], \qquad (2.28) $$

$$ \frac{d\alpha_E(t)}{dt} = \frac{a_E(t)}{V_E(t)}, \quad t \in [t_0, t_C], \qquad (2.29) $$

$$ \frac{d^2 y_E(t)}{dt^2} = \frac{dV_E(t)}{dt}(\sin(\phi_{E0}) + \cos(\phi_{E0})\alpha_E(t)) + a_E(t) \cos(\phi_{E0}), \quad t \in [t_0, t_C]. \qquad (2.30) $$

Now, based on the pursuer's and evader's linearized kinematics models (2.23)-(2.26) and (2.27)-(2.30), let us derive the kinematics model of the engagement, i.e., the relative kinematics of the engagement's participants. Let us start with the relative motions of the pursuer and the evader with respect to the X-axis. Taking into account that $x_P(t_0) = 0$ and integrating the differential equation (2.23) from $t = t_0$ to $t = t_C$, we directly obtain

$$ x_P(t_C) = \int_{t_0}^{t_C} V_P(t) dt \cos(\phi_{PC}). \qquad (2.31) $$

Similarly, taking into account that $x_E(t_0) = R_0$ and integrating the differential equation (2.27) from $t = t_0$ to $t = t_C$, we directly have

$$ x_E(t_C) = R_0 + \int_{t_0}^{t_C} V_E(t) dt \cos(\phi_{E0}). \qquad (2.32) $$

The equations (2.31) and (2.32), along with the equation (2.3), yield the equality

$$ x_P(t_C) = x_E(t_C), \qquad (2.33) $$
meaning that the X-coordinates of the pursuer and the evader meet each other at the time instant $t = t_C$. This circumstance allows us to choose the final time-instance of the engagement $t_f$ as $t_C$, i.e.,

$$t_f = t_C. \tag{2.34}$$

Proceed with the relative motion of the pursuer and the evader with respect to the Y-axis. To derive the model of this motion, let us introduce into the consideration the following new state variables:

$$\begin{aligned} x_1(t) &= y_E(t) - y_P(t), & x_2(t) &= \frac{dy_E(t)}{dt} - \frac{dy_P(t)}{dt}, & x_3(t) &= a_P(t), \\ x_4(t) &= a_E(t), & x_5(t) &= \alpha_P(t), & x_6(t) &= \alpha_E(t). \end{aligned} \tag{2.35}$$

These state variables are the ones of the relative motion of the pursuer and the evader with respect to the Y-axis. Using the equations (2.24)-(2.26), (2.28)-(2.30) and (2.34), we derive the equations of this motion in the form

$$\frac{dx_1(t)}{dt} = x_2(t), \quad t \in [t_0, t_f], \tag{2.36}$$

$$\begin{aligned} \frac{dx_2(t)}{dt} = & -\cos(\phi_{PC})x_3(t) + \cos(\phi_{E0})x_4(t) \\ & -(dV_P(t)/dt)\cos(\phi_{PC})x_5(t) + (dV_E(t)/dt)\cos(\phi_{E0})x_6(t) \\ & -(dV_P(t)/dt)\sin(\phi_{PC}) + (dV_E(t)/dt)\sin(\phi_{E0}), \quad t \in [t_0, t_f], \end{aligned} \tag{2.37}$$

$$\frac{dx_5(t)}{dt} = \frac{x_3(t)}{V_P(t)}, \quad t \in [t_0, t_f], \tag{2.38}$$

$$\frac{dx_6(t)}{dt} = \frac{x_4(t)}{V_E(t)}, \quad t \in [t_0, t_f]. \tag{2.39}$$

Let us derive the initial conditions for the system (2.36)-(2.39). Taking into account the assumption (A4) (see Section 2.2), as well as the equality $\phi_E(t_0) = \phi_{E0}$ and the equations (2.24) and (2.28), we obtain

$$\begin{aligned} x_1(t_0) = 0, & \quad x_2(t_0) = x_{20}(\phi_{PC}, \phi_{E0}) \triangleq V_E(t_0) \sin(\phi_{E0}) \\ & -V_P(t_0)(\sin(\phi_{PC}) + \cos(\phi_{PC})\alpha_{P0}(\phi_{PC})), \quad x_5(t_0) = \alpha_{P0}(\phi_{PC}), \quad x_6(t_0) = 0, \end{aligned} \tag{2.40}$$

where

$$\alpha_{P0}(\phi_{PC}) = \phi_P(t_0) - \phi_{PC} = \phi_{P0} - \phi_{PC}. \tag{2.41}$$
**Remark 4.** Let us note that the system (2.36)-(2.39) contains six state variables $x_j(t)$, ($j = 1, ..., 6$), while only four differential equations for the state variables $x_1(t)$, $x_2(t)$, $x_5(t)$ and $x_6(t)$. The variables $x_3(t)$ and $x_4(t)$ and their differential equations will be discussed in Section 2.5.

**Remark 5.** As it was aforementioned, the linear relative motion of the pursuer and the evader with respect to the X-axis yields the unavoidable coincidence of their X-coordinates at $t = t_f$. However, such a coincidence is not a case for the linear relative motion of the pursuer and the evader with respect to the Y-axis. Therefore, the participants of the engagement focus only on their relative motion with respect to the Y-axis, however, with opposite aims. The pursuer aims to decrease the value $|x_1(t_f)|$ as much as possible, while the evader aims to increase this value as much as possible. We call the value $|x_1(t_f)|$ the miss distance in the engagement (or, briefly, the miss distance). The best outcome of the engagement from the pursuer's viewpoint is zero miss distance. This outcome has a considerable interest in theory and applications.

## 2.5 Complete Mathematical Model of the Relative Motion Between the Pursuer and the Evader

To complete the kinematics model of the engagement between the pursuer and the evader, given by the equations (2.36)-(2.41), we should determine the status of the variables $x_3(t)$ and $x_4(t)$, $t \in [t_0, t_f]$. Namely, it should be decided whether the pursuer (the evader) directly disposes of the variable $x_3(t)$ (the variable $x_4(t)$), or the corresponding variable is subject to some differential equation. If the pursuer (the evader) can directly change the variable $x_3(t)$ (the variable $x_4(t)$), then this variable can be considered as a control variable of this engagement's participant. In such a case, the dynamics of the control is called *ideal* and the corresponding engagement's participant also is called *ideal*. i.e., the *ideal pursuer* (the *ideal evader*). However, the case of the ideal control dynamics is rather theoretical. The engagement where only one participant, the pursuer (the evader), uses the ideal control dynamics can be considered as the worst case engagement for its opponent, the evader (the pursuer).
## 2.5.1 First-Order Control Dynamics of the Pursuer and the Evader

Much more realistic type of the control dynamics of the pursuer (the evader) is the one described by a linear differential equation with respect to the variable $x_3(t)$ (the variable $x_4(t)$). This equation can be of the first-order, the second-order and so on. The corresponding control dynamics is called the first-order dynamics, the second-order dynamics and so on. In this section and in the subsequent chapters of the book, we focus on the case of the first-order control dynamics for both engagement's participants, which is of a considerable interest in the theory of pursuit-evasion problems and its numerous applications.

Thus, the differential equations of the first-order control dynamics for the pursuer and the evader are, respectively,

$$ \frac{dx_3(t)}{dt} = \frac{a_{P,c}(t) - x_3(t)}{\tau_P}, \quad t \in [t_0, t_f], \quad x_3(t_0) = x_{30} \triangleq a_P(t_0), \quad (2.42) $$

and

$$ \frac{dx_4(t)}{dt} = \frac{a_{E,c}(t) - x_4(t)}{\tau_E}, \quad t \in [t_0, t_f], \quad x_4(t_0) = x_{40} \triangleq a_E(t_0), \quad (2.43) $$

where $a_{P,c}(t)$ and $a_{E,c}(t)$, $t \in [t_0, t_f]$ are the lateral acceleration commands (controls) of the pursuer and the evader, respectively; $\tau_P$ and $\tau_E$ are the time constants of the pursuer's and the evader's control dynamics.

It should be noted that the lateral acceleration commands are bounded as follows:

$$ |a_{P,c}(t)| \le a_{P,c}^{\max}(t), \quad |a_{E,c}(t)| \le a_{E,c}^{\max}(t), \quad t \in [t_0, t_f], \quad (2.44) $$

where $a_{P,c}^{\max}(t)$ and $a_{E,c}^{\max}(t)$ are positive continuous functions known to both participants of the engagement. An important case from the viewpoints of theoretical analysis and practical implementation is the one where the bound of the pursuer's (the evader's) acceleration command is constant, i.e., $a_{P,c}^{\max}(t) \equiv \text{const} = a_{P,c}^{\max}(a_{E,c}^{\max}(t) \equiv \text{const} = a_{E,c}^{\max})$, $t \in [t_0, t_f]$.

## 2.5.2 Complete Linear Model of the Relative Motion Between the Pursuer and the Evader

Let us introduce into the consideration the following matrix and vectors:

$$ A(t, \phi_{PC}, \phi_{E0}) = $$
$$
\begin{pmatrix}
0 & 1 & 0 & 0 & 0 & 0 \\
0 & 0 & -\cos(\phi_{PC}) & \cos(\phi_{E0}) & -\frac{dV_P(t)}{dt}\cos(\phi_{PC}) & \frac{dV_E(t)}{dt}\cos(\phi_{E0}) \\
0 & 0 & -\frac{1}{\tau_P} & 0 & 0 & 0 \\
0 & 0 & 0 & -\frac{1}{\tau_E} & 0 & 0 \\
0 & 0 & \frac{1}{V_P(t)} & 0 & 0 & 0 \\
0 & 0 & 0 & \frac{1}{V_E(t)} & 0 & 0
\end{pmatrix},
\tag{2.45}
$$

$$
x(t) = \begin{pmatrix} x_1(t) \\ x_2(t) \\ x_3(t) \\ x_4(t) \\ x_5(t) \\ x_6(t) \end{pmatrix}, \quad b = \begin{pmatrix} 0 \\ 0 \\ \frac{1}{\tau_P} \\ 0 \\ 0 \\ 0 \end{pmatrix}, \quad c = \begin{pmatrix} 0 \\ 0 \\ 0 \\ \frac{1}{\tau_E} \\ 0 \\ 0 \end{pmatrix}, \quad (2.46)
$$

$$
f(t, \phi_{PC}, \phi_{E0}) = \begin{pmatrix} 0 \\ f_2(t, \phi_{PC}, \phi_{E0}) \\ 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}, \quad x_0(\phi_{PC}, \phi_{E0}) = \begin{pmatrix} 0 \\ x_{20}(\phi_{PC}, \phi_{E0}) \\ x_{30} \\ x_{40} \\ \alpha_{P0}(\phi_{PC}) \\ 0 \end{pmatrix}, \tag{2.47}
$$

where

$$
f_2(t, \phi_{PC}, \phi_{E0}) = -(dV_P(t)/dt) \sin(\phi_{PC}) + (dV_E(t)/dt) \sin(\phi_{E0}), \quad t \in [t_0, t_f], \tag{2.48}
$$

$x_{20}$, $x_{30}$, $x_{40}$ and $\alpha_{P0}(\phi_{PC})$ are given in the equations (2.40), (2.42), (2.43) and (2.41), respectively.

Using the equations ((2.36)-(2.41),(2.42),(2.43) and (2.45)-(2.47), we can
represent the complete linear mathematical model of the relative motion
between the pursuer and the evader in the following vector form:

$$
\begin{equation}
\begin{split}
\frac{dx(t)}{dt} ={}& A(t, \phi_{PC}, \phi_{E0})x(t) + ba_{P,c}(t) + ca_{E,c}(t) \\
& + f(t, \phi_{PC}, \phi_{E0}), \quad t \in [t_0, t_f], \quad x(t_0) = x_0(\phi_{PC}, \phi_{E0}).
\end{split}
\tag{2.49}
\end{equation}
$$
This system is subject to the inequalities (2.44).

Let us introduce new controls $u(t)$ and $v(t)$ instead of $a_{P,c}(t)$ and $a_{E,c}(t)$, respectively, in the system (2.49), namely,

$$u(t) = \frac{a_{P,c}(t)}{a_{P,c}^{\max}(t)}, \quad v(t) = \frac{a_{E,c}(t)}{a_{E,c}^{\max}(t)}, \quad t \in [t_0, t_f]. \qquad (2.50)$$

Using the equation (2.50), the differential system (2.49) and the control constraints (2.44) are transformed equivalently as follows:

$$\frac{dx(t)}{dt} = A(t, \phi_{PC}, \phi_{E0})x(t) + b_u(t)u(t) + c_v(t)v(t) + f(t, \phi_{PC}, \phi_{E0}), \quad t \in [t_0, t_f], \quad x(t_0) = x_0(\phi_{PC}, \phi_{E0}), \qquad (2.51)$$

and

$$|u(t)| \le 1, \quad |v(t)| \le 1, \quad t \in [t_0, t_f]. \qquad (2.52)$$

In the system (2.51), the vector-valued coefficients $b_u(t)$ and $c_v(t)$ have the form

$$b_u(t) = \begin{pmatrix} 0 \\ 0 \\ \frac{a_{P,c}^{\max}(t)}{\tau_P} \\ 0 \\ 0 \\ 0 \end{pmatrix}, \quad c_v(t) = \begin{pmatrix} 0 \\ 0 \\ 0 \\ \frac{a_{E,c}^{\max}(t)}{\tau_E} \\ 0 \\ 0 \end{pmatrix}, \quad t \in [t_0, t_f]. \qquad (2.53)$$

## 2.6 Robust Capture Pursuit: Main Definitions

Due to Remark 5, the aim of the pursuer in the engagement (2.51),(2.52) is to decrease the miss distance $|x_1(t_f)|$ by a proper choice of the control $u$. The best result of this aim, having considerable theoretical meaning and practical importance, is to achieve zero miss distance, i.e.,

$$x_1(t_f) = 0. \qquad (2.54)$$

Such a result of the engagement is called the *capture*.

It is extremely important for the pursuer that the capture would be achieved against any possible behaviour of the evader, i.e., robustly with respect to any possible evader's control $v$ which is unknown to the pursuer.
Such a behaviour of the pursuer is called the *robust pursuit with capture* (or briefly, the *robust capture pursuit*).

It is clear that not any pursuer's control $u$ and not any starting engagement position $(\bar{t}, \bar{x}) \in [t_0, t_f) \times E^6$ is suitable for the robust pursuit. Based on this observation, we present the following definitions.

**Definition 2.6.1.** *Evader's control $v(t) : [t_0, t_f] \to E^1$ is called admissible if it satisfied the constraint in (2.52).*

**Definition 2.6.2.** *Pursuer's control $u = u(t, x) : [t_0, t_f] \times E^6 \to E^1$ is called admissible if, for any point $(\bar{t}, \bar{x}) \in [t_0, t_f) \times E^6$ and any admissible evader's control $v(t)$, the following conditions are fulfilled: (i) the differential equation in (2.51) with $u(t, x)$ instead of $u(t)$ has the unique solution $x(t) = x(t; \bar{t}, \bar{x}, v(\cdot))$, $t \in [\bar{t}, t_f)$, satisfying the initial condition $x(\bar{t}) = \bar{x}$; (ii) there exists the finite limit $x(t_f) \stackrel{\Delta}{=} \lim_{t \to t_f - 0} x(t)$.*

**Remark 6.** It should be noted that we understand a solution (a continuous one) of the initial-value problem, mentioned in Definition 2.6.2, in the sense of a particular case of the Krasovskii's constructive motion [4], i.e., as a limit of a convergent sequence of piecewise-linear Eiler's functions, associated with this initial-value problem. In this particular case, in contrast with [4], we do not replace the given initial state's position with a convergent sequence of initial positions, as well as we do not replace the evader's control with a convergent sequence of such controls.

**Definition 2.6.3.** Let $(\bar{t}, \bar{x})$ be any given point of the set $[t_0, t_f) \times E^6$. An admissible pursuer's control $u = u(t, x)$ is called a robust capture control from the point $(\bar{t}, \bar{x})$ if, for all admissible evader's controls $v(t)$, the following conditions are fulfilled: (a) the first component $x_1(t)$ of the solution $x(t) = x(t; \bar{t}, \bar{x}, v(\cdot))$, mentioned in Definition 2.6.2, becomes zero at $t = t_f$, i.e., $x_1(t_f) = 0$; (b) the time realization $u(t) \stackrel{\Delta}{=} u(t, x(t))$ of the control $u(t, x)$ along the solution $x(t)$ satisfies the inequality $|u(t)| \le 1$, $t \in [\bar{t}, t_f]$.

**Definition 2.6.4.** Let $u = u(t, x)$ be any given admissible pursuer's control. The set $C_x(u(t, x))$ of all points $(\bar{t}, \bar{x}) \in [t_0, t_f) \times E^6$, from which $u(t, x)$ is the robust capture control, is called a capture zone of this pursuer's control. If there exist a point $\bar{x} \in E^6$ such that $(t_0, \bar{x}) \in C_x(u(t, x))$, then the capture zone $C_x(u(t, x))$ is called t-complete.
## 2.7 Concluding Remarks and Literature Review

In this chapter, the mathematical model of the planar pursuit-evasion engagement between two participants, the pursuer and the evader, was derived. This derivation is due to the assumptions formulated in Section 2.2. In Section 2.3, the initial collision course between the pursuer and the evader was analyzed. Using the assumption that the pursuer is aware of the evader's aspect angle in this course, the equations for obtaining the collision time-instant and the pursuer's aspect angle in the initial collision course were derived. In Section 2.4, first, the nonlinear mathematical models of the pursuer's and evader's kinematics in a sufficiently small neighbourhood of the initial collision course (see the assumption (A6) in Section 2.2) were derived. Then, using the smallness of this neighbourhood, the linearization of the nonlinear models was carried out, the errors of this linearization were established and the linear model of the relative motion kinematics between the pursuer and the evader was obtained. The complete linear mathematical model of the relative motion between the pursuer and the evader, including the equations of their first-order control dynamics, was derived in Section 2.5. Note that the aforementioned nonlinear model of Section 2.4 is an extension of the model, considered in the work [3], to the case where the speeds of the pursuer and the evader are functions of time. The linear model of Section 2.5 is an extension of the models, considered in the works [1, 7, 9], to the case where the pursuer and the evader have any aspect angles in the initial collision course. In the models of [1, 7, 9], the evader's aspect angle in the initial collision course equals $\pi$, while the pursuer's one equals 0, i.e., the models of these works describe only the *head-on* scenario of the pursuit-evasion engagement. The models of Sections 2.4 and 2.5 include not only the head-on engagement scenario, but also, for instance, the *head-pursuit* and *tail-chase* engagement scenarios which were considered in the works [5, 6] for the case of constant speeds of the pursuer and the evader.

In Section 2.6, the notion of the *robust capture pursuit* was introduced. Main definitions connected with this notion, such as the definitions of the admissible pursuer's and evader's controls, the definition of the pursuer's *robust capture control* from a given point in the space (*time*, *state variable*) and the definition of the *capture zone*, were formulated. To the best of our knowledge, the notions of the pursuer's *robust capture control/strategy* and the corresponding *capture zone/set* were introduced for the first time in the literature in the works [8, 9, 10]. Similar (to the *capture control/strategy*) notion, called the *perfect pursuit/intercept*, was introduced in [5, 6]. In the
frame of the controllability issue for systems with unknown disturbances,
the notions of the *robust transferring strategy* and the corresponding *robust
controllability set* were introduced in the literature (see [2] and references
therein). These notions are close in a proper sense to the ones of the robust
capture control/strategy and the capture zone/set. The definitions, formu-
lated in Section 2.6, are proper modifications of the definitions of [8, 9].

In subsequent chapters, various types of robust capture controls are de-
signed and corresponding capture zones are constructed.
Bibliography

[1] Glizer, V.Y.: Controllability of Singularly Perturbed Linear Time Delay Systems, Birkhäuser, Switzerland (2021)

[2] Glizer, V.Y., Turetsky, V.: Robust Controllability of Linear Systems, Nova Science Publishers, New York, NY, USA (2012)

[3] Glizer, V.Y., Turetsky, V., Fridman, L., Shinar, J.: History-dependent modified sliding mode interception strategies with maximal capture zone. J. Franklin Inst. **349**, 638–657 (2012)

[4] Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems, Springer, New York, NY, USA (2011)

[5] Shima, T.: Intercept-angle guidance. J. Guid. Control Dyn. **34**, 484–492 (2011)

[6] Shima, T., Golan, O.M.: Head pursuit guidance. J. Guid. Control Dyn. **30**, 1437–1444 (2007)

[7] Shima, T., Shinar, J.: Time-varying linear pursuit-evasion game models with bounded controls. J. Guid. Control Dyn. **25**, 425–432 (2002)

[8] Turetsky, V., Glizer, V.Y.: Continuous feedback control strategy with maximal capture zone in a class of pursuit games. Int. Game Theory Rev. **7**, 1–24 (2005)

[9] Turetsky, V., Glizer, V.Y.: Robust solution of a time-variable interception problem: a cheap control approach. Int. Game Theory Rev. **9**, 637–655 (2007)

[10] Turetsky, V., Shinar, J.: Missile guidance laws based on pursuit-evasion game formulations. Automatica J. IFAC **39**, 607–618 (2003)
# Chapter 3

## Saddle-Point Solution of a
Zero-Sum Linear Dynamics
Differential Game with
Bounded Controls

### 3.1 Introduction

In this chapter, we consider one class of two-person finite-duration zero-sum differential games. The differential equation of the game is an $n$-dimensional linear non-homogeneous one with time-dependent coefficients and non-homogeneity. Thus, the state variable of the game is an $n$-dimensional vector, while the players' controls are scalars. The cost function in this game is the distance between the terminal value of the state variable and a given $(n-1)$-dimensional linear manifold in the $n$-dimensional linear space, which is to be minimized by one player (the minimizer) and to be maximized by the other player (the maximizer). The information pattern in this game is the following. Both players know perfectly all the game's data and the current game's position (*time*, *state variable*). We look for two types of the saddle-point solution to this game, the open-loop and closed-loop (state-feedback) ones. The solution of the game is essentially based on its terminal projection transformation to an auxiliary scalar zero-sum differential game and the rigorous analysis of the latter.

The following main notations and notions are applied in the chapter.

1. $E^n$ is the $n$-dimensional real Euclidean space, thus $E^1$ is the set of all real numbers.

2. $\| \cdot \|$ denotes the norm in $E^n$.
3. The superscript T denotes the transposition of a vector $x \in E^n$, $(x^T)$.

4. $\Theta_1 \times \Theta_2$, where each of $\Theta_k$, ($k = 1, 2$) is some set either of numbers, or of vectors, or of functions, denotes the direct product of these sets, i.e., the set of all pairs $(\theta_1, \theta_2)$ with any entries $\theta_1 \in \Theta_1$ and $\theta_2 \in \Theta_2$.

5. $I_n$ denotes the identity matrix of the dimension $n$.

6. By $\partial(\mathcal{D})$, where $\mathcal{D} \in E^n$, we denote the boundary of the set $\mathcal{D}$.

7. By $\mathrm{clo}(\mathcal{D})$, where $\mathcal{D} \in E^n$, we denote the closure of the set $\mathcal{D}$, i.e., $\mathrm{clo}(\mathcal{D}) = \mathcal{D} \cup \partial(\mathcal{D})$.

8. We call a function $\varphi(x) : E^n \to E^1$ piece-wise continuous in $E^n$ if: (a) this function is bounded in $E^n$; (b) there exists a finite number of nonintersecting open sets $\mathcal{D}_i$, ($i = 1, ..., K$), such that $\bigcup_{i=1}^K \mathrm{clo}(\mathcal{D}_i) = E^n$ and $\varphi(x)$ is continuous in each set $\mathcal{D}_i$, ($i = 1, ..., K$).

9. Let $\Theta_1$ and $\Theta_2$ be some sets such that $\Theta_2 \subseteq \Theta_1$. Then, $\Theta_1 \setminus \Theta_2$ denotes the set of all entries of $\Theta_1$ which do not belong to $\Theta_2$.

## 3.2 Original Game Formulation

Consider the differential game with the dynamics described by the following system:

$$ \frac{dx(t)}{dt} = A(t)x(t) + b(t)u(t) + c(t)v(t) + f(t), \quad t \in [t_0, t_f], \quad x(t_0) = x_0, \quad (3.1) $$

where $x(t) \in E^n$ is a state vector; $u(t)$ and $v(t)$ are scalar controls of the players; $t_0$ and $t_f$, ($t_0 < t_f$) are initial and terminal time instants; the matrix-valued function $A(t)$ and the vector-valued functions $b(t)$, $c(t)$, $f(t)$ are given and assumed continuous for $t \in [t_0, t_f]$; $x_0 \in E^n$ is a given initial state vector.

The controls of the players are assumed to be measurable functions in the interval $[t_0, t_f]$, which satisfy the constraints

$$ |u(t)| \le 1, \quad |v(t)| \le 1, \quad t \in [t_0, t_f]. \qquad (3.2) $$

The set of all such functions is denoted by $\mathcal{C}$, i.e., $u(\cdot) \in \mathcal{C}$, $v(\cdot) \in \mathcal{C}$.

Consider the following hyperplane in the space $E^n$:

$$ \mathcal{D} = \{x \in E^n : d^T x + d_0 = 0\}, $$

where $d \in E^n$ is a given non-zero vector, $d_0$ is a given scalar value.

The player with the control $u(t)$ (the minimizer) aims to minimize the distance $\mathrm{Dist}(\mathcal{D}, x(t_f))$ between the terminal state vector $x(t_f)$ and the hyperplane $\mathcal{D}$, while the player with the control $v(t)$ (the maximizer) tends to
maximize Dist($\mathcal{D}$, $x(t_f)$). This distance is given as follows:

$$ \mathrm{Dist}(\mathcal{D}, x(t_f)) = |d^T x(t_f) + d_0| / \|d\| $$

Since $\|d\|$ is a positive constant, the cost function, evaluating the performance of the minimizer and the maximizer, can be chosen as:

$$ J_x = J_x(u(\cdot), v(\cdot)) = |d^T x(t_f) + d_0|. \quad (3.3) $$

Below, a feasibility of this formally defined cost function is justified.

As the information pattern in this game, we assume that both players are aware of all the game's data, presenting in (3.1)-(3.3), and of the current game's position $(t, x(t))$.

**Remark 7.** The system (3.1), the control constraints (3.2) and the cost function (3.3), along with the aims of the players and the information pattern, constitute a zero-sum differential game. We call this differential game the Vector-valued Game with Bounded Controls (VGBC).

Below, we present two types of the VGBC solution: (a) the open-loop solution (the controls of the players are functions of the time only) and (b) the feedback solution (the controls of the players are functions of a current game position $(t, x) \in [t_0, t_f] \times E^n$).

## 3.3 Transformation of the Original Game

By the $n \times n$-matrix $\Phi(t)$, $t \in [t_0, t_f]$, we denote the unique solution of the terminal-value problem

$$ \frac{d\Phi(t)}{dt} = -\Phi(t)A(t), \quad t \in [t_0, t_f], \quad \Phi(t_f) = I_n. \quad (3.4) $$

Using the matrix-valued function $\Phi(t)$, we construct the scalar function of two variables

$$ z(t, x) = d^T \left( \Phi(t)x + \int_t^{t_f} \Phi(s)f(s)ds \right) + d_0, \quad (t, x) \in [t_0, t_f] \times E^n. \quad (3.5) $$

Now, let us make the following transformation of the state variable in the system (3.1):

$$ z(t) \stackrel{\Delta}{=} z(t, x(t)) = d^T \left( \Phi(t)x(t) + \int_t^{t_f} \Phi(s)f(s)ds \right) + d_0, \quad t \in [t_0, t_f]. \quad (3.6) $$
**Proposition 3.3.1.** Let, for any given $u(\cdot) \in C$ and $v(\cdot) \in C$, the vector-valued function $x(t)$, $t \in [t_0, t_f]$ be the absolutely continuous solution of the system (3.1). Then the scalar function $z(t)$, given by the equation (3.6), is the absolutely continuous solution of the initial-value problem

$$ \frac{dz(t)}{dt} = h_u(t)u(t) + h_v(t)v(t), \quad t \in [t_0, t_f], \quad z(t_0) = z_0, \quad (3.7) $$

where

$$ h_u(t) = d^T \Phi(t)b(t), \quad h_v(t) = d^T \Phi(t)c(t), \quad t \in [t_0, t_f], \quad (3.8) $$

$$ z_0 = d^T \left( \Phi(t_0)x_0 + \int_{t_0}^{t_f} \Phi(s)f(s)ds \right) + d_0. \quad (3.9) $$

Moreover, the following equality is valid:

$$ z(t_f) = d^T x(t_f) + d_0. \quad (3.10) $$

*Proof.* Differentiating $z(t)$, given by the equation (3.6), we obtain

$$ \frac{dz(t)}{dt} = d^T \left( \frac{d\Phi(t)}{dt}x(t) + \Phi(t)\frac{dx(t)}{dt} - \Phi(t)f(t) \right), \quad t \in [t_0, t_f]. \quad (3.11) $$

Now, using the differential equation for $x(t)$ in (3.1) and the differential equation for $\Phi(t)$ in (3.4), we can rewrite the expression (3.11) for $dz(t)/dt$ as follows:

$$ \frac{dz(t)}{dt} = d^T \left( -\Phi(t)A(t)x(t) + \Phi(t)A(t)x(t) + \Phi(t)b(t)u(t) + \Phi(t)c(t)v(t) + \Phi(t)f(t) - \Phi(t)f(t) \right) = d^T \Phi(t)b(t)u(t) + d^T \Phi(t)c(t)v(t), \quad t \in [t_0, t_f]. $$

The latter, along with the equation (3.8), directly yields that the scalar function $z(t)$, $t \in [t_0, t_f]$ satisfies the differential equation in (3.7). The initial condition for this equation in (3.7) with $z_0$, given by (3.9), is obtained by immediate substitution of $t = t_0$ into (3.6) and use of the initial condition for $x(t)$ (see the equation (3.1)). Thus, the first statement of the proposition is proven. The second statement is directly obtained by substitution of $t = t_f$ into (3.6) and use of the terminal condition for $\Phi(t)$ (see the equation (3.4)). □

Due to the equality (3.10), the cost function (3.3) can be rewritten in the terms of $z(t)$ as:

$$ J_z = J_z(u(\cdot), v(\cdot)) = |z(t_f)|. \quad (3.12) $$
**Remark 8.** The scalar differential equation with the initial condition (3.7), the control constraints (3.2) and the cost function (3.12) constitute a scalar (reduced) differential game with bounded controls (SGBC). In this game (like in the VGBC), the player with the control $u(t)$ (the minimizer) tends to minimize the cost function (3.12), while the player with the control $v(t)$ (the maximizer) aims to maximize this cost function. Thus, similarly to the VGBC, the SGBC is a zero-sum game. However, being scalar, the SGBC is much simpler than the VGBC.

# 3.4 Open-Loop Solution

## 3.4.1 Main Definitions

First of all, let us note the following. For any given players' controls $u(\cdot) \in C$ and $v(\cdot) \in C$, the system (3.1) has the unique absolutely continuous solution $x(t)$. Therefore, there exists a finite limit $\lim_{t \to t_f - 0} x(t)$, and

$$x(t_f) \triangleq \lim_{t \to t_f - 0} x(t).$$

Hence, the cost function (3.3) is feasible.

**Definition 3.4.1.** For a fixed initial state $x_0$, the control $u^*(\cdot) \in C$ is called the **optimal open-loop minimizer control** in the VGBC, if for any $u(\cdot) \in C$,

$$\sup_{v(\cdot) \in C} J_x(u^*(\cdot), v(\cdot)) \le \sup_{v(\cdot) \in C} J_x(u(\cdot), v(\cdot)). \quad (3.13)$$

The value

$$J_{xu} = \sup_{v(\cdot) \in C} J_x(u^*(\cdot), v(\cdot)), \quad (3.14)$$

is called the **upper value** in **open-loop controls** of the VGBC.

**Definition 3.4.2.** For a fixed initial state $x_0$, the control $v^*(\cdot) \in C$ is called the **optimal open-loop maximizer control** in the VGBC, if for any $v(\cdot) \in C$,

$$\inf_{u(\cdot) \in C} J_x(u(\cdot), v^*(\cdot)) \ge \inf_{u(\cdot) \in C} J_x(u(\cdot), v(\cdot)). \quad (3.15)$$

The value

$$J_{xl} = \inf_{u(\cdot) \in C} J_x(u(\cdot), v^*(\cdot)), \quad (3.16)$$

is called the **lower value** in **open-loop controls** of the VGBC.
It is well known that (see e.g. [13]):

$$J_{xu} \geq J_{xl}.$$

**Definition 3.4.3.** If

$$J_{xu} = J_{xl} \triangleq J_x^*, \qquad (3.17)$$

then $J_x^*$ is called the **value in open-loop controls** of the VGBC and the pair $\{u^*(\cdot), v^*(\cdot)\}$ is called the **saddle point in open-loop controls** of the VGBC. The trajectory $x^*(t)$ of the system (3.1), generated by the pair of controls $\{u^*(t), v^*(t)\}$, $t \in [t_0, t_f]$, is called the **saddle-point trajectory in open-loop controls** of the VGBC.

By the well-known Saddle Point Inequality [1], for any $u(\cdot) \in \mathcal{C}$ and $v(\cdot) \in \mathcal{C}$,

$$J_x(u^*(\cdot), v(\cdot)) \leq J_x^* = J_x(u^*(\cdot), v^*(\cdot)) \leq J_x(u(\cdot), v^*(\cdot)).$$

**Remark 9.** Note that the SGBC can be considered as a particular case of the VGBC. Therefore, the definitions of the optimal open-loop minimizer/maximizer control, the upper/lower value in open-loop controls, the game value in open-loop controls, the saddle point in open-loop controls and the saddle-point trajectory in open-loop controls for the SGBC are quite similar to the corresponding definitions for the VGBC.

## 3.4.2 Equivalence of the VGBC and the SGBC

**Lemma 3.4.4.** If $u^*(\cdot) \in \mathcal{C}$ ($v^*(\cdot) \in \mathcal{C}$) is an optimal open-loop minimizer (maximizer) control in the VGBC, then it is an optimal open-loop minimizer (maximizer) control in the SGBC, and vice versa.

*Proof.* We prove the lemma for the minimizer control. For the maximizer control, the proof is similar.

Let for any pair $(u(\cdot), v(\cdot)) \in \mathcal{C} \times \mathcal{C}$, the vector-valued function $x(t)$ and the scalar function $z(t)$ be the unique solutions of (3.1) and (3.7), respectively. Then, due to Proposition 3.3.1,

$$J_x(u(\cdot), v(\cdot)) = J_z(u(\cdot), v(\cdot)). \qquad (3.18)$$

Let $u^*(\cdot) \in \mathcal{C}$ be an optimal open-loop minimizer control in the VGBC, i.e. (3.13) is valid for all $u(\cdot) \in \mathcal{C}$. The latter, along with (3.18), means that $u^*(\cdot) \in \mathcal{C}$ is an optimal open-loop minimizer control in the SGBC.
Now, let $u^*(\cdot) \in \mathcal{C}$ be an optimal open-loop minimizer control in the SGBC, i.e. for any $u(\cdot) \in \mathcal{C}$,

$$ \sup_{v(\cdot) \in \mathcal{C}} J_z(u^*(\cdot), v(\cdot)) \le \sup_{v(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)). \quad (3.19) $$

Assume that $u^*(\cdot)$ is not optimal in the VGBC. This means that there exists $u^{**}(\cdot) \in \mathcal{C}$ such that

$$ \sup_{v(\cdot) \in \mathcal{C}} J_x(u^*(\cdot), v(\cdot)) > \sup_{v(\cdot) \in \mathcal{C}} J_x(u^{**}(\cdot), v(\cdot)). \quad (3.20) $$

Hence, due to (3.18), the same inequality holds for $J_z(u(\cdot), v(\cdot))$, which contradicts the aforementioned assumption on the optimality of $u^*(\cdot)$ in the SGBC. This contradiction proves that $u^*(\cdot)$ is optimal in the VGBC. Thus the lemma is proven. □

The following two assertions are direct consequences of Lemma 3.4.4, Definitions 3.4.1-3.4.3 and Remark 9.

**Corollary 3.4.5.** The upper values, as well as the lower values, of the VGBC and the SGBC coincide with each other, i.e., $J_{xu} = J_{zu}$ and $J_{xl} = J_{zl}$.

**Corollary 3.4.6.** If the pair of the players' controls $(u^*(\cdot), v^*(\cdot)) \in \mathcal{C} \times \mathcal{C}$ is a saddle point in the VGBC, then it is a saddle point in the SGBC, and vice versa. Moreover, the values of these games are equal to each other: $J_x^* = J_z^*$.

### 3.4.3 Solution of the SGBC

Here and in what follows of this chapter we assume:

**A1.** The functions $h_u(t)$ and $h_v(t)$, given by the equation (3.8), have no more than a finite number of distinct zeros in the interval $[t_0, t_f]$.

The following theorem establishes the condition for the existence of the SGBC saddle point in open-loop controls.

**Theorem 3.4.7.** Let

$$ |z_0| \ge \int_{t_0}^{t_f} |h_u(t)| dt. \quad (3.21) $$

Then the saddle point $(u^*(t), v^*(t))$ (in the open-loop controls) of the SGBC exists and its components have the form

$$ u^*(t) = -(\operatorname{sign}(z_0))(\operatorname{sign}(h_u(t))), \quad \forall t \in [t_0, t_f] : h_u(t) \ne 0, \quad (3.22) $$

$$ v^*(t) = (\operatorname{sign}(z_0))(\operatorname{sign}(h_v(t))), \quad \forall t \in [t_0, t_f] : h_v(t) \ne 0. \quad (3.23) $$
The value in open-loop controls of the SGBC is

$$J_z^* = |z_0| - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt. \quad (3.24)$$

**Remark 10.** In the equations (3.22) and (3.23), the control functions $u^*(t)$ and $v^*(t)$ are defined for all $t \in [t_0, t_f]$ such that $h_u(t) \neq 0$ and $h_v(t) \neq 0$. For any $\bar{t}_u \in [t_0, t_f]$ ($\bar{t}_v \in [t_0, t_f]$), for which $h_u(\bar{t}_u) = 0$ ($h_v(\bar{t}_v) = 0$), we define the value sign($h_u(\bar{t}_u)$) (sign($h_v(\bar{t}_v)$)) as follows: $\text{sign}(h_u(\bar{t}_u)) = \bar{h}_u$ ($\text{sign}(h_v(\bar{t}_v)) = \bar{h}_v$). Here, $\bar{h}_u$ ($\bar{h}_v$) is any prechosen real number satisfying the inequality $|\bar{h}_u| \le 1$ ($|\bar{h}_v| \le 1$). Due to such a definition of $\text{sign}(h_u(\bar{t}_u))$ ($\text{sign}(h_v(\bar{t}_v))$), the control function $u^*(t)$ ($v^*(t)$) is defined in the entire interval $[t_0, t_f]$.

The proof of the theorem is presented in Subsection 3.6.1.

**Corollary 3.4.8.** If the inequality (3.21) is satisfied, then the saddle-point trajectory in open-loop controls of the SGBC $z^*(t)$ preserves the sign, i.e.,

$$\text{sign}(z^*(t)) = \text{sign}(z_0), \quad t \in [t_0, t_f]. \quad (3.25)$$

*Proof.* Substitution of $u(t) = u^*(t)$ and $v(t) = v^*(t)$, $t \in [t_0, t_f]$ (see the equations (3.22) and (3.23)) into (3.7) yields after a routine rearrangement the following initial-value problem for $z^*(t)$:

$$\frac{dz^*(t)}{dt} = -\text{sign}(z_0)(|h_u(t)| - |h_v(t)|), \quad t \in [t_0, t_f], \quad z(t_0) = z_0. \quad (3.26)$$

The solution of (3.26) can be represented as:

$$z^*(t) = \text{sign}(z_0) \left( |z_0| - \int_{t_0}^{t} |h_u(s)|ds + \int_{t_0}^{t} |h_v(s)|ds \right), \quad t \in [t_0, t_f]. \quad (3.27)$$

Since the inequality (3.21) is satisfied, then the following inequality is valid:

$$|z_0| \ge \int_{t_0}^{t} |h_u(t)|dt \quad \forall t \in [t_0, t_f].$$

The latter, along with the assumption A1, implies the inequality

$$|z_0| - \int_{t_0}^{t} |h_u(s)|ds + \int_{t_0}^{t} |h_v(s)|ds > 0 \quad \forall t \in [t_0, t_f].$$

This inequality, along with the equation (3.27), directly yields the equality (3.25). Thus, the corollary is proven. □
Now, consider the case

$$|z_0| < \int_{t_0}^{t_f} |h_u(t)| dt. \qquad (3.28)$$

The following two theorems are proven in Subsections 3.6.2 and 3.6.3.

**Theorem 3.4.9.** In the case of (3.28), the optimal open-loop minimizer control $u^*(\cdot)$ of the SGBC is any function from the set C satisfying the integral equation

$$z_0 + \int_{t_0}^{t_f} h_u(t) u^*(t) dt = 0. \qquad (3.29)$$

The upper value in open-loop controls of the SGBC is

$$J_{zu} = \int_{t_0}^{t_f} |h_v(t)| dt. \qquad (3.30)$$

**Theorem 3.4.10.** Let (3.28) be valid. If

$$|z_0| - \int_{t_0}^{t_f} |h_u(t)| dt + \int_{t_0}^{t_f} |h_v(t)| dt \le 0, \qquad (3.31)$$

then the optimal open-loop maximizer control $v^*(\cdot)$ of the SGBC is arbitrary function from the set C. The lower value in open-loop controls of the SGBC is

$$J_{zl} = 0. \qquad (3.32)$$

If

$$|z_0| - \int_{t_0}^{t_f} |h_u(t)| dt + \int_{t_0}^{t_f} |h_v(t)| dt > 0, \qquad (3.33)$$

then the optimal open-loop maximizer control $v^*(\cdot)$ of the SGBC is

$$v^*(t) = \begin{cases} \operatorname{sign}(z_0)\operatorname{sign}(h_v(t)), & z_0 \ne 0, \\ \pm\operatorname{sign}(h_v(t)), & z_0 = 0. \end{cases} \qquad (3.34)$$

The lower value in open-loop controls of the SGBC is

$$J_{zl} = |z_0| - \int_{t_0}^{t_f} |h_u(t)| dt + \int_{t_0}^{t_f} |h_v(t)| dt. \qquad (3.35)$$

Based on Theorems 3.4.9 and 3.4.10, we obtain the following corollary.

**Corollary 3.4.11.** In the case of (3.28), the SGBC has no saddle point.
*Proof.* The statement of the corollary directly follows from the comparison of the expression (3.30) for $J_{zu}$, and the expressions (3.32) and (3.35) for $J_{zl}$. Indeed, in the case (3.31), we have

$$J_{zu} = \int_{t_0}^{t_f} |h_v(t)|dt > 0 = J_{zl}.$$

In the case (3.33), due to (3.28), we obtain

$$J_{zu} = \int_{t_0}^{t_f} |h_v(t)|dt > \int_{t_0}^{t_f} |h_v(t)|dt + \left(|z_0| - \int_{t_0}^{t_f} |h_u(t)|dt\right) = J_{zl}.$$

Thus in both cases, we obtain the inequality

$$J_{zu} > J_{zl}.$$

This inequality, along with Definition 3.4.3 and Remark 9, directly yields the statement of the corollary. □

Summarizing the statements of Theorems 3.4.7-3.4.10 and Corollary 3.4.11, we obtain the following assertion.

**Theorem 3.4.12.** The SGBC has a saddle point in the open-loop controls if and only if the condition (3.21) is valid.

Thus, we have obtained the complete open-loop solution of the SGBC. Due to Lemma 3.4.4 and Corollaries 3.4.5 - 3.4.6, the obtained solution is also the complete open-loop solution of the VGBC.

## 3.5 Feedback Solution

In this section, in contrast with Section 3.4, we assume that the controls of the minimizer and the maximizer in the VGBC are functions not only of the time $t$, but also of the current state $x = x(t)$, i.e. the players use the *feedback* controls $u(t, x)$, $v(t, x)$.

### 3.5.1 Main definitions

Let $\mathcal{G}^x$ be the set of all scalar functions $g(t, x)$, $(t, x) \in [t_0, t_f] \times E^n$, measurable with respect to $t \in [t_0, t_f]$ for any $x \in E^n$, piece-wise continuous with respect to $x \in E^n$ for any $t \in [t_0, t_f]$, and satisfying the following inequality:

$$|g(t, x)| \le 1, \quad \forall (t, x) \in [t_0, t_f] \times E^n.$$
Consider a pair of functions $(u(t, x), v(t, x)) \in \mathcal{G}^x \times \mathcal{G}^x$. Let us find out whether such a pair can be used as a pair of the players' feedback controls in the VGBC. For this purpose, we substitute $u(t, x)$ and $v(t, x)$ into the system (3.1) instead of $u(t)$ and $v(t)$, respectively. This substitution yields the system

$$ \frac{dx(t)}{dt} = A(t)x(t)+b(t)u(t,x(t))+c(t)v(t,x(t))+f(t), \quad t \in [t_0, t_f], \quad x(t_0) = x_0. \tag{3.36} $$

**Remark 11.** Since at least one of the functions, $u(t, x)$ or $v(t, x)$, may be discontinuous with respect to $x \in E^n$ for some $t \in [t_0, t_f]$, the classical Caratheodory notion of a differential equation's solution may not be applicable to (3.36). To overcome this obstacle, we use the results of [14] (Section 6). Due to these results, the solution of (3.36) is understood in the sense of the Krasovskii's constructive motion. Namely, the solution (the continuous one) of (3.36) is defined as a limit of a convergent sequence of piecewise-linear Eiler's vector-valued functions, associated with this system. Due to the results of [14], such a sequence exists for any initial position $(t_0, x_0)$ and any pair of functions $(u(t, x), v(t, x)) \in \mathcal{G}^x \times \mathcal{G}^x$ yielding the existence of the aforementioned solution to (3.36). However this solution is not in general unique, because of possible existence of several convergent sequences of piecewise-linear Eiler's vector-valued functions associated with (3.36) and having different limits. The non-uniqueness of solution to the system (3.36) can yield the non-uniqueness of the value of the VGBC cost function $J_x(u(\cdot), v(\cdot))$ (see the equation (3.3)), which is unfeasible. Taking into account all the aforementioned in this remark, we make the following definition.

**Definition 3.5.1.** A pair of functions $(u(t, x), v(t, x)) \in \mathcal{G}^x \times \mathcal{G}^x$ is called an **admissible pair of players' feedback controls** (or briefly, an **admissible pair**) in the VGBC if, for any point $(\bar{t}, \bar{x}) \in [t_0, t_f) \times E^n$, all the solutions $x(t) = x(t; u(\cdot), v(\cdot), \bar{t}, \bar{x})$ of the differential equation in (3.36) with the initial condition $x(\bar{t}) = \bar{x}$ yield the same value $|d^T x(t_f; u(\cdot), v(\cdot), \bar{t}, \bar{x}) + d_0|$. The set of all the admissible pairs of players' feedback controls in the VGBC is denoted as $\mathcal{F}^x$.

**Remark 12.** Due to Definition 3.5.1 and the equation (3.3), any pair of the players' feedback controls $(u(t, x), v(t, x)) \in \mathcal{F}^x$ generates a single value of the cost function in the VGBC. Therefore, the cost function of the VGBC and the entire game are feasible in the set $\mathcal{F}^x$.

For a given $u(t, x) \in \mathcal{G}^x$, consider the set

$$ \mathcal{H}_v^x(u(t, x)) \triangleq \{v(t, x) \in \mathcal{G}^x : (u(t, x), v(t, x)) \in \mathcal{F}^x\}. $$
Let

$$ \mathcal{K}_u^x \triangleq \{u(t, x) \in \mathcal{G}^x : \mathcal{H}_v^x(u(t, x)) \neq \emptyset\}. $$

**Definition 3.5.2.** The minimizer's feedback control $u^0(\cdot) \in \mathcal{K}_u^x$ is called **optimal** in the VGBC, if for any starting game's position $(\bar{t}, \bar{x}) \in [t_0, t_f) \times E^n$ and for any $u(\cdot) \in \mathcal{K}_u^x$, the following inequality is valid:

$$ \sup_{v(\cdot) \in \mathcal{H}_v^x(u^0(\cdot))} J_x(u^0(\cdot), v(\cdot)) \le \sup_{v(\cdot) \in \mathcal{H}_v^x(u(\cdot))} J_x(u(\cdot), v(\cdot)). \quad (3.37) $$

The value

$$ J_{xu} = J_{xu}(\bar{t}, \bar{x}) = \sup_{v(\cdot) \in \mathcal{H}_v^x(u^0(\cdot))} J_x(u^0(\cdot), v(\cdot)) \quad (3.38) $$

is called the **upper value in feedback controls** of the VGBC.

Similarly, for a given $v(t, x) \in \mathcal{G}^x$, consider the set

$$ \mathcal{H}_u^x(v(t, x)) \triangleq \{u(t, x) \in \mathcal{G}^x : (u(t, x), v(t, x)) \in \mathcal{F}^x\}. $$

Let

$$ \mathcal{K}_v^x \triangleq \{v(t, x) \in \mathcal{G}^x : \mathcal{H}_u^x(v(t, x)) \neq \emptyset\}. $$

**Definition 3.5.3.** The maximizer's feedback control $v^0(\cdot) \in \mathcal{K}_v^x$ is called **optimal** in the VGBC, if for any starting game's position $(\bar{t}, \bar{x}) \in [t_0, t_f) \times E^n$ and for any $v(\cdot) \in \mathcal{K}_v^x$, the following inequality is valid:

$$ \inf_{u(\cdot) \in \mathcal{H}_u^x(v^0(\cdot))} J_x(u(\cdot), v^0(\cdot)) \ge \inf_{u(\cdot) \in \mathcal{H}_u^x(v(\cdot))} J_x(u(\cdot), v(\cdot)). \quad (3.39) $$

The value

$$ J_{xl} = J_{xl}(\bar{t}, \bar{x}) = \inf_{u(\cdot) \in \mathcal{H}_u^x(v^0(\cdot))} J_x(u(\cdot), v^0(\cdot)) \quad (3.40) $$

is called the **lower value in feedback controls** of the VGBC.

**Definition 3.5.4.** If $(u^0(\cdot), v^0(\cdot)) \in \mathcal{F}^x$ and

$$ J_{xu}(\bar{t}, \bar{x}) = J_{xl}(\bar{t}, \bar{x}) \triangleq J_x^0(\bar{t}, \bar{x}) \quad \forall (\bar{t}, \bar{x}) \in [t_0, t_f) \times E^n, \quad (3.41) $$

then $J_x^0(\bar{t}, \bar{x})$ is called the **value in feedback controls** of the VGBC and the pair $(u^0(\cdot), v^0(\cdot))$ is called the **saddle point in feedback controls** of the VGBC. Any solution $x^0(t) = x^0(t; \bar{t}, \bar{x})$, $t \in [\bar{t}, t_f]$ of the differential equation in (3.36) for $(u(t, x), v(t, x)) = (u^0(t, x), v^0(t, x))$, satisfying the initial condition $x^0(\bar{t}) = \bar{x}$, is called a **saddle-point trajectory in feedback controls** of the VGBC.
The following relations are a direct consequence of Definitions 3.5.2 - 3.5.4:

$$ 
\begin{aligned}
J_x(u^0(\cdot), v(\cdot)) &\le J_x^0(\bar{t}, \bar{x}) = J_x(u^0(\cdot), v^0(\cdot)) \le J_x(u(\cdot), v^0(\cdot)) \\
\forall v(\cdot) \in \mathcal{H}_v^x(u^0(\cdot)), \quad u(\cdot) \in \mathcal{H}_u^x(v^0(\cdot)), \quad (\bar{t}, \bar{x}) \in [t_0, t_f) \times E^n.
\end{aligned} 
\quad (3.42) $$

**Remark 13.** As it was aforementioned, the SGBC can be considered as a particular case of the VGBC. Therefore, the definitions of the admissible pair of players' feedback controls, the optimal feedback minimizer/maximizer control, the upper/lower value in feedback controls, the game value in feedback controls, the saddle point in feedback controls and the saddle-point trajectory in feedback controls for the SGBC are similar to the corresponding definitions for the VGBC. However, for the sake of the book's reading convenience, below we present these definitions for the SGBC.

Let $Q^z$ be the set of all scalar functions $q(t, z)$, $(t, z) \in [t_0, t_f] \times E^1$, measurable with respect to $t \in [t_0, t_f]$ for any $z \in E^1$, piece-wise continuous with respect to $z \in E^1$ for any $t \in [t_0, t_f]$, and satisfying the following inequality:

$$ |q(t, z)| \le 1, \quad \forall (t, z) \in [t_0, t_f] \times E^1. $$

Consider a pair of functions $(u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) \in Q^z \times Q^z$. Similarly to the pair of functions $(u(t, x), v(t, x)) \in \mathcal{G}^x \times \mathcal{G}^x$ in the VGBC, we are going to find out whether $(u_{\text{sg}}(t, z), v_{\text{sg}}(t, z))$ can be used as a pair of the players' feedback controls in the SGBC. For this purpose, we substitute $u_{\text{sg}}(t, z)$ and $v_{\text{sg}}(t, z)$ into the system (3.7) instead of $u(t)$ and $v(t)$, respectively. Thus, we obtain the system

$$ \frac{dz(t)}{dt} = h_u(t)u_{\text{sg}}(t, z(t)) + h_v(t)v_{\text{sg}}(t, z(t)), \quad t \in [t_0, t_f], \quad z(t_0) = z_0. \quad (3.43) $$

**Remark 14.** Like the solution of the system (3.36) (see Remark 11), we understand the solution (the continuous one) of the system (3.43) in the sense of the Krasovskii's constructive motion [14], i.e., as a limit of a convergent sequence of piecewise-linear Eiler's functions, associated with this system. Such a sequence exists for any initial position $(t_0, z_0)$ and any pair of functions $(u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) \in Q^z \times Q^z$ yielding the existence of the aforementioned solution to (3.43). However, similarly to the solution of (3.36), the solution of (3.43) is not in general unique. This non-uniqueness can yield the non-uniqueness of the value of the SGBC cost function $J_z(u_{\text{sg}}(\cdot), v_{\text{sg}}(\cdot))$ (see the equation (3.12)), which is unfeasible. Taking into account all the aforementioned in this remark, we make the following definition.
**Definition 3.5.5.** A pair of functions $(u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) \in \mathcal{Q}^z \times \mathcal{Q}^z$ is called an admissible pair of players' feedback controls (or briefly, an admissible pair) in the SGBC if, for any point $(\bar{t}, \bar{z}) \in [t_0, t_f) \times E^1$, all the solutions $z(t) = z(t; u_{\text{sg}}(\cdot), v_{\text{sg}}(\cdot), \bar{t}, \bar{z})$ of the differential equation in (3.43) with the initial condition $z(\bar{t}) = \bar{z}$ yield the same value $|z(t_f; u_{\text{sg}}(\cdot), v_{\text{sg}}(\cdot), \bar{t}, \bar{z})|$. The set of all the admissible pairs of players' feedback controls in the SGBC is denoted as $\mathcal{P}^z$.

**Remark 15.** Due to Definition 3.5.5 and the equation (3.12), any pair of the players' feedback controls $(u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) \in \mathcal{P}^z$ generates a single value of the cost function in the SGBC. Therefore, the cost function of the SGBC and the entire game are feasible in the set $\mathcal{P}^z$.

For a given $u_{\text{sg}}(t, z) \in \mathcal{Q}^z$, consider the set

$$ \mathcal{L}_v^z(u_{\text{sg}}(t, z)) \triangleq \{v_{\text{sg}}(t, z) \in \mathcal{Q}^z : (u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) \in \mathcal{P}^z\}. \quad (3.44) $$

Let

$$ \mathcal{M}_u^z \triangleq \{u_{\text{sg}}(t, z) \in \mathcal{Q}^z : \mathcal{L}_v^z(u_{\text{sg}}(t, z)) \neq \emptyset\}. \quad (3.45) $$

**Definition 3.5.6.** The minimizer's feedback control $u_{\text{sg}}^0(\cdot) \in \mathcal{M}_u^z$ is called optimal in the SGBC, if for any starting game's position $(\bar{t}, \bar{z}) \in [t_0, t_f) \times E^1$ and for any $u_{\text{sg}}(\cdot) \in \mathcal{M}_u^z$, the following inequality is valid:

$$ \sup_{v_{\text{sg}}(\cdot) \in \mathcal{L}_v^z(u_{\text{sg}}^0(\cdot))} J_z(u_{\text{sg}}^0(\cdot), v_{\text{sg}}(\cdot)) \le \sup_{v_{\text{sg}}(\cdot) \in \mathcal{L}_v^z(u_{\text{sg}}(\cdot))} J_z(u_{\text{sg}}(\cdot), v_{\text{sg}}(\cdot)). \quad (3.46) $$

The value

$$ J_{zu} = J_{zu}(\bar{t}, \bar{z}) = \sup_{v_{\text{sg}}(\cdot) \in \mathcal{L}_v^z(u_{\text{sg}}^0(\cdot))} J_z(u_{\text{sg}}^0(\cdot), v_{\text{sg}}(\cdot)) \quad (3.47) $$

is called the upper value in feedback controls of the SGBC.

Similarly, for a given $v_{\text{sg}}(t, z) \in \mathcal{Q}^z$, consider the set

$$ \mathcal{L}_u^z(v_{\text{sg}}(t, z)) \triangleq \{u_{\text{sg}}(t, z) \in \mathcal{Q}^z : (u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) \in \mathcal{P}^z\}. \quad (3.48) $$

Let

$$ \mathcal{M}_v^z \triangleq \{v_{\text{sg}}(t, z) \in \mathcal{Q}^z : \mathcal{L}_u^z(v_{\text{sg}}(t, z)) \neq \emptyset\}. \quad (3.49) $$

**Definition 3.5.7.** The maximizer's feedback control $v_{\text{sg}}^0(\cdot) \in \mathcal{M}_v^z$ is called optimal in the SGBC, if for any starting game's position $(\bar{t}, \bar{z}) \in [t_0, t_f) \times E^1$ and for any $v_{\text{sg}}(\cdot) \in \mathcal{M}_v^z$, the following inequality is valid:

$$ \inf_{u_{\text{sg}}(\cdot) \in \mathcal{L}_u^z(v_{\text{sg}}^0(\cdot))} J_z(u_{\text{sg}}(\cdot), v_{\text{sg}}^0(\cdot)) \ge \inf_{u_{\text{sg}}(\cdot) \in \mathcal{L}_u^z(v_{\text{sg}}(\cdot))} J_z(u_{\text{sg}}(\cdot), v_{\text{sg}}(\cdot)). \quad (3.50) $$
The value

$$J_{zl} = J_{zl}(\bar{t}, \bar{z}) = \inf_{u_{\text{sg}}(\cdot) \in \mathcal{L}_u^z(v_{\text{sg}}^0(\cdot))} J_z(u_{\text{sg}}(\cdot), v_{\text{sg}}^0(\cdot)) \quad (3.51)$$

is called the lower value in feedback controls of the SGBC.

**Definition 3.5.8.** If $(u_{\text{sg}}^0(\cdot), v_{\text{sg}}^0(\cdot)) \in \mathcal{P}^z$ and

$$J_{zu}(\bar{t}, \bar{z}) = J_{zl}(\bar{t}, \bar{z}) \triangleq J_z^0(\bar{t}, \bar{z}) \quad \forall (\bar{t}, \bar{z}) \in [t_0, t_f) \times E^1, \quad (3.52)$$

then $J_z^0(\bar{t}, \bar{z})$ is called the **value in feedback controls** of the SGBC and the pair $(u_{\text{sg}}^0(\cdot), v_{\text{sg}}^0(\cdot))$ is called the **saddle point in feedback controls** of the SGBC. Any solution $z^0(t) = z^0(t; \bar{t}, \bar{z})$, $t \in [\bar{t}, t_f]$ of the differential equation in (3.43) for $(u_{\text{sg}}(t, z), v_{\text{sg}}(t, z)) = (u_{\text{sg}}^0(t, z), v_{\text{sg}}^0(t, z))$, satisfying the initial condition $z^0(\bar{t}) = \bar{z}$, is called a **saddle-point trajectory in feedback controls** of the SGBC.

From Definitions 3.5.6 - 3.5.8, we directly have the following relations:

$$\begin{gathered} J_z(u_{\text{sg}}^0(\cdot), v_{\text{sg}}^0(\cdot)) \le J_z^0(\bar{t}, \bar{z}) = J_z(u_{\text{sg}}^0(\cdot), v_{\text{sg}}^0(\cdot)) \le J_z(u_{\text{sg}}(\cdot), v_{\text{sg}}^0(\cdot)) \\ \forall v_{\text{sg}}(\cdot) \in \mathcal{L}_v^z(u_{\text{sg}}^0(\cdot)), \ u_{\text{sg}}(\cdot) \in \mathcal{L}_u^z(v_{\text{sg}}^0(\cdot)), \ (\bar{t}, \bar{z}) \in [t_0, t_f) \times E^1. \end{gathered} \quad (3.53)$$

## 3.5.2 Connection between the VGBC and the SGBC

In this subsection, we present preliminary arguments explaining that, similarly to the open-loop solution, the feedback solution of the VGBC (the feedback optimal players' controls and the saddle point in feedback controls) can be obtained based on the feedback solution of the RGBC.

Consider the vector-valued function of two variables

$$y(t, x) = \Phi(t)x + \int_t^{t_f} \Phi(s)f(s)ds, \quad (t, x) \in [t_0, t_f] \times E^n, \quad (3.54)$$

where the matrix-valued function $\Phi(t)$ is the unique solution of the terminal-value problem (3.4).

Now, let us make the following formal state transformation in the original system (3.1):

$$y(t) \triangleq y(t, x(t)) = \Phi(t)x(t) + \int_t^{t_f} \Phi(s)f(s)ds, \quad t \in [t_0, t_f]. \quad (3.55)$$

This transformation yields the system

$$\frac{dy(t)}{dt} = \Phi(t)b(t)u(t) + \Phi(t)c(t)v(t), \quad t \in [t_0, t_f], \quad y(t_0) = y_0, \quad (3.56)$$
where

$$y_0 = \Phi(t_0)x_0 + \int_{t_0}^{t_f} \Phi(s)f(s)ds.$$

Moreover, the transformation (3.55) yields the equality

$$y(t_f) = x(t_f), \qquad (3.57)$$

which means that this transformation converts the original cost function (3.3) into the following one:

$$J_y = J_y(u(\cdot), v(\cdot)) = |d^T y(t_f) + d_0|. \qquad (3.58)$$

Thus, the transformation (3.55) formally converts the VGBC into the auxiliary differential game with bounded controls (AGBC) consisting of the dynamic system (3.56), the control constraints (3.2) and the cost function (3.58) minimized by the player with the control $u(\cdot)$ (the minimizer) and maximized by the player with the control $v(\cdot)$ (the maximizer).

**Remark 16.** Since the matrix $\Phi(t)$ is invertible for all $t \in [t_0, t_f]$, the transformation (3.55) also is invertible, i.e.,

$$x(t) \triangleq x(t, y(t)) = \Phi^{-1}(t) \left(y(t) - \int_{t}^{t_f} \Phi(s)f(s)ds\right), \quad t \in [t_0, t_f]. \qquad (3.59)$$

This means that the transformation (3.59) formally converts the AGBC into the VGBC.

**Remark 17.** The AGBC can be considered as a particular case of the VGBC. Therefore, the definitions of the admissible pair of players' feedback controls, the optimal feedback minimizer/maximizer control, the upper/lower value in feedback controls, the game value in feedback controls, the saddle point in feedback controls and the saddle-point trajectory in feedback controls for the AGBC are quite similar to the corresponding definitions for the VGBC. We denote the set of all admissible pairs of players' feedback controls in the AGBC as $\mathcal{F}^y$.

Remarks 11, 16, 17 and Definitions 3.5.1-3.5.3 directly yield the following assertion.

**Proposition 3.5.9.** If $(u(t, x), v(t, x)) \in \mathcal{F}^x$ then $(u_{\text{ag}}(t, y), v_{\text{ag}}(t, y)) = (u(t, x(t, y)), v(t, x(t, y))) \in \mathcal{F}^y$, where $x(t, y)$ is given in (3.59). Moreover, if $u^0(t, x)$ ($v^0(t, x)$) is the optimal feedback control of the minimizer (the maximizer) in the VGBC, then $u_{\text{ag}}^0(t, y) = u^0(t, x(t, y))$ ($v_{\text{ag}}^0(t, y) =$
$v^0(t, x(t, y)))$ is the optimal feedback control of the minimizer (the maximizer) in the AGBC. Vice versa: if $(u_{ag}(t, y), v_{ag}(t, y)) \in \mathcal{F}^y$ then $(u(t, x), v(t, x)) = (u_{ag}(t, y(t, x)), v_{ag}(t, y(t, x))) \in \mathcal{F}^x$, where $y(t, x)$ is given by (3.54). Moreover, if $u_{ag}^0(t, y) (v_{ag}^0(t, y))$ is the optimal feedback control of the minimizer (the maximizer) in the AGBC, then $u^0(t, x) = u_{ag}^0(t, y(t, x)) (v^0(t, x) = v_{ag}^0(t, y(t, x)))$ is the optimal feedback control of the minimizer (the maximizer) in the VGBC.

Let $(\bar{t}, \bar{x}) \in [t_0, t_f] \times E^n$ be a given point. Using the equation (3.54), we define

$$ \bar{y} \triangleq \Phi(\bar{t})\bar{x} + \int_{\bar{t}}^{t_f} \Phi(s)f(s)ds. \qquad (3.60) $$

**Corollary 3.5.10.** The upper values $J_{xu}(\bar{t}, \bar{x})$ and $J_{yu}(\bar{t}, \bar{y})$ of the VGBC and the AGBC, as well as the lower values $J_{xl}(\bar{t}, \bar{x})$ and $J_{yl}(\bar{t}, \bar{y})$ of these games, coincide with each other, i.e., $J_{xu}(\bar{t}, \bar{x}) = J_{yu}(\bar{t}, \bar{y})$ and $J_{xl}(\bar{t}, \bar{x}) = J_{yl}(\bar{t}, \bar{y})$. Moreover, if the pair $(u^0(t, x), v^0(t, x))$ constitutes the saddle point in feedback controls of the VGBC then the pair $(u^0(t, x(t, y)), v^0(t, x(t, y)))$ constitutes the saddle point in feedback controls of the AGBC and vice versa, if the pair $(u_{ag}^0(t, y), v_{ag}^0(t, y))$ constitutes the saddle point in feedback controls of the AGBC then the pair $(u_{ag}^0(t, y(t, x)), v_{ag}^0(t, y(t, x)))$ constitutes the saddle point in feedback controls of the VGBC. In this case, the values in feedback controls of the VGBC and AGBC coincide with each other, i.e., $J_x^0(\bar{t}, \bar{x}) = J_y^0(\bar{t}, \bar{y})$.

*Proof.* The statements of the corollary directly follow from Proposition 3.5.9, Definitions 3.5.2-3.5.4, Remark 17 and the equations (3.3), (3.57), (3.58), (3.60). □

<figure><img src="image_3.png" /></figure>

**Remark 18.** Due to Proposition 3.5.9 and Corollary 3.5.10, the functions $y(t, x)$ and $x(t, y)$ yield the one-to-one mapping between the sets $\mathcal{F}^x$ and $\mathcal{F}^y$ of all admissible pairs of the players' feedback controls in the VGBC and AGBC. Moreover, the functions $y(t, x)$ and $x(t, y)$ yield the one-to-one mapping between the players' optimal feedback controls and the saddle points in feedback controls of these games. Thus, in the sense of this mapping, the VGBC and AGBC are equivalent.

Now, we are going to analyze a connection between the AGBC and the SGBC.

Let, $(u_{sg}(t, z), v_{sg}(t, z)) \in \mathcal{P}^z$ be any given admissible pair of players' feedback controls in the SGBC. Using the inclusion $(u_{sg}(t, z), v_{sg}(t, z)) \in \mathcal{P}^z$ and the equations (3.5), (3.10), (3.54), (3.57), as well as Definitions 3.5.1, 3.5.5 and Remark 17, we directly have that $(u_{ag}(t, y), v_{ag}(t, y)) = (u_{sg}(t, d^T y +
$d_0), v_{sg}(t, d^Ty + d_0))$ is an admissible pair of players' feedback controls in the AGBC, i.e., $(u_{ag}(t, y), v_{ag}(t, y)) \in \mathcal{F}^y$.

Furthermore, if $y(t) = y(t; u_{ag}(t, y), v_{ag}(t, y), t_0, y_0)$ is a solution of the system

$$ \frac{dy(t)}{dt} = \Phi(t)b(t)u_{ag}(t, y(t)) + \Phi(t)c(t)v_{ag}(t, y(t)), \quad t \in [t_0, t_f], \quad y(t_0) = y_0, $$

then $z(t) = z(t; u_{sg}(t, z), v_{sg}(t, z), t_0, z_0) = d^Ty(t) + d_0$ is a solution of the system (3.43).

The target hyperplane in the AGBC is similar to that in the VGBC, i.e., it is the following:

$$ \mathcal{D}_y = \{y \in E^n : d^Ty + d_0 = 0\}. \qquad (3.61) $$

For a given $t \in [t_0, t_f]$, we can construct the hyperplane, containing $y(t)$ and parallel to (3.61)

$$ \mathcal{Y}(t) = \{y \in E^n : d^T(y - y(t)) = 0\}. \qquad (3.62) $$

The distance between the hyperplanes (3.61) and (3.62) is

$$ l(t) = \frac{|d^Ty(t) + d_0|}{\|d\|}, $$

or

$$ l(t) = \frac{|z(t)|}{\|d\|}. $$

Note that for all $t \in [t_0, t_f]$, the distance between the hyperplanes $\mathcal{Y}(t)$ and $\mathcal{D}_y$ coincides with the distance between the hyperplane $\mathcal{D}_y$ and the point $y(t)$, regardless of its position on $\mathcal{Y}(t)$. Thus, the cost function (3.58) in the AGBC (the distance between $y(t_f)$ and the hyperplane $\mathcal{D}_y$) can be replaced by $l(t_f)$ (the distance between the hyperplanes $\mathcal{Y}(t_f)$ and $\mathcal{D}_y$), regardless of the position $y(t_f)$ on $\mathcal{Y}(t_f)$. Therefore, instead of the motion of the point $y(t)$, we can consider the motion of the hyperplane $\mathcal{Y}(t)$. Since the orientation of $\mathcal{Y}(t)$ is determined by the vector $d$, its motion is fully described by the behavior of the scalar function $d^Ty(t) + d_0 = z(t)$, indicating the current position of $\mathcal{Y}(t)$ with respect to $\mathcal{D}_y$. Based on these arguments, we can replace a search for the players' optimal feedback controls and the saddle point in the AGBC (and, therefore, in the VGBC) by another search which explores the players' optimal feedback controls and the saddle point in the SGBC.
### 3.5.3 Solution of the SGBC

In this subsection, we construct the saddle point of the SGBC in the feedback controls. This construction is based on the saddle point (3.22)-(3.23) in the open-loop controls. Due to (3.25), the pair (3.22)-(3.23) can be rewritten as:

$$u^*(t) = -(\mathrm{sign}(z(t_f)))(\mathrm{sign}(h_u(t))), \quad (3.63)$$

$$v^*(t) = (\mathrm{sign}(z(t_f)))(\mathrm{sign}(h_v(t))). \quad (3.64)$$

It is important to note that the pair (3.63)-(3.64) constitutes the saddle point in the open-loop controls, if and only if the initial position of the game satisfies the condition (3.21). Nevertheless, in the present subsection, based on this pair of controls, we design the saddle point of the SGBC in the feedback controls for any initial game's position.

The construction of the saddle point of the SGBC in the feedback controls consists of three stages.

#### Stage I: Decomposition of the game space.

Substituting (3.63) and (3.64) into (3.7), we obtain the system

$$\frac{dz(t)}{dt} = R(t)\mathrm{sign}(z(t_f)), \quad t \in [t_0, t_f], \quad z(t_0) = z_0, \quad (3.65)$$

where

$$R(t) = |h_v(t)| - |h_u(t)|, \quad t \in [t_0, t_f]. \quad (3.66)$$

In what follows, we call $R(t)$ the *determining function*.

For any given $z(t_f) \neq 0$, let us integrate the differential equation in (3.65) from $t = t_f$ in the backward time. This integration yields the following trajectory:

$$\begin{aligned} \tilde{z}(t) &= \tilde{z}(t; u^*(\cdot), v^*(\cdot), z(t_f)) = z(t_f) + \int_{t_f}^t R(\sigma) d\sigma \mathrm{sign}(z(t_f)) \\ &= \left( |z(t_f)| + \int_{t_f}^t R(\sigma) d\sigma \right) \mathrm{sign}(z(t_f)), \quad t \in [\tilde{t}, t_f] \subseteq [t_0, t_f], \quad (3.67) \end{aligned}$$

where the time instant $\tilde{t}$ is defined as follows. If the function $|z(t_f)| + \int_{t_f}^t R(\sigma) d\sigma$ does not changes its sign in the entire interval $[t_0, t_f]$, then $\tilde{t} = t_0$. If $|z(t_f)| + \int_{t_f}^t R(\sigma) d\sigma$ changes its sign in this interval, then $\tilde{t}$ is the largest (the first from the right) point in the interval $(t_0, t_f)$ of such a change.

Varying $z(t_f) \neq 0$, we obtain the family of all the trajectories $\tilde{z}(t)$. Note that this family is symmetric with respect to the $t$-axis. This means that, for any $z_f \neq 0$, the trajectories with $z(t_f) = z_f$ and $z(t_f) = -z_f$ are symmetric
to each other with respect to the *t*-axis. All these trajectories cover some set $\mathcal{R}_r$ of the $(t, z)$-plane, which is called in the sequel the *regular region*.

Let $(\bar{t}, \bar{z})$, $(\bar{z} \neq 0)$ be any given point from $\mathcal{R}_r$. Then, there exists the unique value $z_f \neq 0$ and, therefore, the unique trajectory $\tilde{z}(t; u^*(\cdot), v^*(\cdot), z_f)$ such that $\tilde{z}(\bar{t}; u^*(\cdot), v^*(\cdot), z_f) = \bar{z}$. For this $(\bar{t}, \bar{z})$, we define $u_{\text{sg}}^0(\bar{t}, \bar{z}) = u^*(\bar{t})$, $v_{\text{sg}}^0(\bar{t}, \bar{z}) = v^*(\bar{t})$. If the point $(\bar{t}, \bar{z})$ lies on the trajectory emanated from $z_f > 0$, then $u^*(\cdot), v^*(\cdot)$, generating this trajectory, are given by (3.63)-(3.64) with $\text{sign}(z(t_f)) = 1$. For $z_f < 0$, such controls are given by (3.63)-(3.64) with $\text{sign}(z(t_f)) = -1$. Varying the position $(\bar{t}, \bar{z}) \in \mathcal{R}_r$, $(\bar{z} \neq 0)$, we construct the feedback controls $u_{\text{sg}}^0(t, z)$ and $v_{\text{sg}}^0(t, z)$ uniquely defined in this set of the $(t, z)$-plane. These feedback controls generate the trajectories completely filling the set $\mathcal{R}_r$, $(z \neq 0)$ of the $(t, z)$-plane. Therefore, we can consider the pair $(u_{\text{sg}}^0(t, z), v_{\text{sg}}^0(t, z))$ as a candidate saddle-point in feedback controls of the SGBC in this set. Note that the points of $\mathcal{R}_r$ with $z=0$ constitute a set of zero Lebesgue measure. For these points, the candidate saddle-point in feedback controls of the SGBC can be defined in some different way. This issue is clarified below.

If $\mathcal{R}_r$ coincides with the entire strip $S_{[t_0, t_f]} = \{(t, z) : t \in [t_0, t_f], z \in E^1\}$, then the aforementioned feedback controls $u_{\text{sg}}^0(t, z)$ and $v_{\text{sg}}^0(t, z)$ are defined almost everywhere in $S_{[t_0, t_f]}$ excepting its subset of zero Lebesgue measure where $z=0$. In what follows, the strip $S_{[t_0, t_f]}$ is called the space of the SGBC.

If $S_{[t_0, t_f]} \setminus \mathcal{R}_r \triangleq \mathcal{R}_s \neq \emptyset$, we should also define $u_{\text{sg}}^0(t, z)$ and $v_{\text{sg}}^0(t, z)$ for $(t, z) \in \mathcal{R}_s$ (in what follows, the set $\mathcal{R}_s$ is called the *singular region*). Note that in this case, the upper and lower boundaries of $\mathcal{R}_r$ are formed by two curves $z = z^+(t) \ge 0$ and $z = z^-(t) = -z^+(t)$, symmetrical with respect to the *t*-axis. These curves can be of two types. The curves of the first type are arcs of the "extreme" trajectories $\tilde{z}(t)$ and, by definition, they belong to $\mathcal{R}_r$. The curves of the second type are obtained as limits of the trajectories (3.67) for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively. For the game's positions on these curves, we define $u_{\text{sg}}^0(\cdot)$ and $v_{\text{sg}}^0(\cdot)$ as follows. If $(\bar{t}, \bar{z})$ lies on the curve $z = z^+(t)$, then $u_{\text{sg}}^0(\bar{t}, \bar{z}) = -\text{sign}(h_u(\bar{t}))$, $v_{\text{sg}}^0(\bar{t}, \bar{z}) = \text{sign}(h_v(\bar{t}))$. For $(\bar{t}, \bar{z})$ belonging to the curve $z = z^-(t)$, $u_{\text{sg}}^0(\bar{t}, \bar{z}) = \text{sign}(h_u(\bar{t}))$, $v_{\text{sg}}^0(\bar{t}, \bar{z}) = -\text{sign}(h_v(\bar{t}))$. Thus, $u_{\text{sg}}^0(t, z)$ and $v_{\text{sg}}^0(t, z)$ are uniquely defined on these curves. This argument allows the inclusion of all the points of the second type curves $z = z^+(t)$ and $z = z^-(t)$ into the regular region $\mathcal{R}_r$. In what follows, the curve $z = z^+(t)$ is called the **positive (upper) boundary** of the regular region $\mathcal{R}_r$, while the curve $z = z^-(t) = -z^+(t)$ is called the **negative (lower) boundary** of this region. As to the values of $u_{\text{sg}}^0(t, z)$, $v_{\text{sg}}^0(t, z)$ in the singular region $\mathcal{R}_s$, it will be shown that they can be chosen arbitrarily subject to Definition 3.5.5.
Since the regular region $\mathcal{R}_r$ consists of the trajectories (3.67), its structure
is completely defined by the determining function $R(t)$.

Below, different cases of the behavior of the determining function and
the respective structure of $\mathcal{R}_r$ are analyzed. Based on this structure, the
feedback controls $u_{\text{sg}}^0(\cdot)$ and $v_{\text{sg}}^0(\cdot)$ are formally derived.

**Stage II: Formal construction of the saddle point.**

We assume that the determining function $R(t)$ has a finite number of
distinct zeros in the interval $(t_0, t_f)$. Moreover, it changes its sign $N_R$ times
in $(t_0, t_f)$. Let $t_{s1} < t_{s2} < \dots < t_{sN_R}$ ($t_{s1} > t_0, t_{sN_R} < t_f$) are all zeros of $R(t)$
where its sign changes.

Consider the following cases of the behavior of the function $R(t)$.

Case 0. $N_R = 0$.

Subcase 0.1. $R(t) \ge 0$ for all $t \in (t_0, t_f)$.

If $z(t_f) > 0$, the trajectory $\tilde{z}(t)$, given by (3.67), slopes down monotonically from the point $(t_f, z(t_f))$ to the $t$-axis while $t$ varies from $t_f$ backwardly to $\tilde{t}$. Similarly, if $z(t_f) < 0$, such a trajectory slopes up monotonically from the point $(t_f, z(t_f))$ to the $t$-axis. In this subcase, $\mathcal{R}_r = S_{[t_0, t_f]}$, which is illustrated by Fig. 3.1. In this figure, the family of all the trajectories $\tilde{z}(t)$, $t \in [\tilde{t}, t_f]$ is depicted for $h_u(t) = 4 - 4t$, $h_v(t) = t^2$, $t_0 = 1$, $t_f = 6$.

<figure><img src="image_4.png" /><figcaption>Figure 3.1: Family of the trajectories $\tilde{z}(t)$ in Subcase 0.1</figcaption></figure>

It is seen that this family completely covers the strip $S_{[t_0,t_f]}$, i.e., $\mathcal{R}_r = S_{[t_0,t_f]}$ and $\mathcal{R}_s = \emptyset$. Any point $(\bar{t}, \bar{z}) \in \mathcal{R}_r$, for which $\bar{z} \neq 0$, lies on the unique trajectory (3.67). Moreover, the sign of $\bar{z}$ coincides with the sign of $z(t_f)$. This observation, along with the equations (3.63)-(3.64), allows the following definition of the candidate saddle point in feedback controls of the SGBC for
$t \in [t_0, t_f], z \neq 0$: $(u_{\text{sg}}^0(t, z), v_{\text{sg}}^0(t, z))$, where

$$u_{\text{sg}}^{0}(t, z) = -\text{sign}(z)\text{sign}(h_{u}(t)), \qquad (3.68)$$

$$v_{\text{sg}}^{0}(t, z) = \text{sign}(z)\text{sign}(h_{v}(t)). \qquad (3.69)$$

Proceed to the formal construction of the candidate saddle point in feedback controls for the points $(t, 0)$, $t \in [t_0, t_f]$. For these points, we define the components of the candidate saddle point as follows:

$$u_{\text{sg}}^{0}(t, 0) = \alpha_{u}(t)\text{sign}(h_{u}(t)), \qquad (3.70)$$

$$v_{\text{sg}}^{0}(t, 0) = \alpha_{v}(t)\text{sign}(h_{v}(t)), \qquad (3.71)$$

where $\alpha_u(t)$, $\alpha_v(t)$ are any functions given in the interval $t \in [t_0, t_f]$ and satisfying the following conditions:

$$|\alpha_u(t)| \le 1, \quad |\alpha_v(t)| \le 1, \quad t \in [t_0, t_f], \qquad (3.72)$$

the function $\alpha_u(t)|h_u(t)| + \alpha_v(t)|h_v(t)|$ has no more than a finite number of distinct zeros in the interval $(t_0, t_f)$.

**Remark 19.** Note that the set of all pairs $(\alpha_u(t), \alpha_v(t))$, satisfying the aforementioned conditions, is not empty. For instance, the pairs $(\alpha_u(t), \alpha_v(t)) \equiv (-1, 1)$ and $(\alpha_u(t), \alpha_v(t)) \equiv (1, -1)$ satisfy these conditions.

The equations (3.68)-(3.69) and (3.70)-(3.71) yield the following components of the candidate saddle point in feedback controls of the SGBC in the entire game space $S_{[t_0, t_f]}$:

$$u_{\text{sg}}^{0}(t, z) = \begin{cases} -\text{sign}(z)\text{sign}(h_{u}(t)), & t \in [t_0, t_f], \quad z \neq 0, \\ \alpha_{u}(t)\text{sign}(h_{u}(t)), & t \in [t_0, t_f], \quad z = 0, \end{cases} \qquad (3.73)$$

$$v_{\text{sg}}^{0}(t, z) = \begin{cases} \text{sign}(z)\text{sign}(h_{v}(t)), & t \in [t_0, t_f], \quad z \neq 0, \\ \alpha_{v}(t)\text{sign}(h_{v}(t)), & t \in [t_0, t_f], \quad z = 0. \end{cases} \qquad (3.74)$$

**Remark 20.** Note that the feedback controls $u_{\text{sg}}^0(t, z)$ and $v_{\text{sg}}^0(t, z)$, given by the equations (3.73) and (3.74), respectively, belong to set $\mathbb{Q}^z$. Furthermore, for any given point $(\bar{t}, \bar{z}) \in S_{[t_0, t_f]}$, the pair of these controls generates the unique trajectory $\bar{z}(t)$, $t \in [\bar{t}, t_f]$ of the differential equation in (3.7) $(u(t) = u_{\text{sg}}^0(t, z), v(t) = v_{\text{sg}}^0(t, z))$ subject to the initial condition $\bar{z}(\bar{t}) = \bar{z}$. Thus, due to Definition 3.5.5, the pair of the controls $(u_{\text{sg}}^0(t, z), v_{\text{sg}}^0(t, z))$, $(t, z) \in S_{[t_0, t_f]}$ is admissible in the SGBC. Moreover, $\bar{z}(t) = \tilde{z}(t)$, $t \in [\bar{t}, t_f]$, where $\tilde{z}(t)$ is given by the equation (3.67) with $z(t_f) = \bar{z}(t_f)$, $\tilde{t} = \bar{t}$. The
aforementioned trajectory $\bar{z}(t)$, $t \in [\bar{t}, t_f]$ is understood in the sense of a particular case of the Krasovskii's constructive motion [14], i.e., as a limit of a convergent sequence of piecewise-linear Euler's functions, associated with the corresponding initial-value problem. In this particular case, in contrast with [14], we do not replace the given initial state's position with a convergent sequence of initial positions.

Thus, the components of the candidate saddle point in feedback controls of the SGBC are formally designed in this subcase.

**Subcase 0.2.** $R(t) \le 0$ for all $t \in (t_0, t_f)$.

In this subcase, in contrast with Subcase 0.1, the trajectory $\tilde{z}(t)$, given by (3.67), slopes up monotonically for $z(t_f) > 0$ and slopes down monotonically for $z(t_f) < 0$. The family of such trajectories is depicted in Fig. 3.2 for $h_u(t) = -2t^2 - 5$, $h_v(t) = t^2 + 6t - 4$, $t_0 = 1$, $t_f = 6$.

<figure><img src="image_5.png" /><figcaption>Figure 3.2: Family of the trajectories $\tilde{z}(t)$ in Subcase 0.2</figcaption></figure>

In this figure, the curves $z = z^+(t)$ and $z = z^-(t)$, $t \in [t_0, t_f] = [1, 6]$ are the limits of the trajectories $\tilde{z}(t)$ for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively:

$$z^+(t) = \int_{t_f}^{t} R(\sigma) d\sigma = - \int_{6}^{t} (\sigma - 3)^2 d\sigma = - \frac{1}{3}(t - 3)^3 + 9, \quad t \in [t_0, t_f] = [1, 6], \qquad (3.75)$$

$$z^-(t) = -z^+(t), \quad t \in [t_0, t_f] = [1, 6]. \qquad (3.76)$$

Note that the pair (3.75)-(3.76) constitutes the Krasovskii's constructive motion of the differential equation in (3.7) generated by the feedback controls (3.73)-(3.74) from the point $(t_f, 0)$ in the backward time. Also, it should be noted that in this subcase, $\tilde{t} = t_0$ for any $z(t_f) \in E^1$.
It is seen that the family of the trajectories $\tilde{z}(t)$, $t \in [t_0, t_f]$ covers the part of $S_{[t_0, t_f]}$ over the curve $z = z^+(t)$ and under the curve $z = z^-(t)$ (including both curves), while between these curves, there are no trajectories $\tilde{z}(t)$, $t \in [t_0, t_f]$. Thus,

$$ \mathcal{R}_r = \{(t, z) : z \ge z^+(t) \text{ or } z \le z^-(t), t \in [t_0, t_f]\}, \quad (3.77) $$

$$ \mathcal{R}_s = \{(t, z) : z^-(t) < z < z^+(t), t \in [t_0, t_f]\}. \quad (3.78) $$

Based on the results of the previous stage, we can conclude that the components of the candidate saddle point in feedback controls are defined in the regular region $\mathcal{R}_r$ by (3.68)-(3.69), i.e., like it is in Subcase 0.1.

Proceed to the construction of the components of the candidate saddle point in feedback controls in the singular region $\mathcal{R}_s$. For any trajectory $z(t)$ of the differential equation in (3.7), starting from any point of $\mathcal{R}_s$ in the forward time and generated by an admissible pair of feedback controls $(u_{\text{sg}}(\cdot), v_{\text{sg}}(\cdot)) \in \mathcal{P}^z$, there are only two possibilities of its behavior. First, it can remain in $\mathcal{R}_s$ till $t = t_f$. This possibility yields $z(t_f) = 0$ and, consequently, the zero game outcome. Second, it can achieve the positive or negative boundary of $\mathcal{R}_r$ ($z = z^+(t)$ or $z = z^-(t)$) at some $t = \bar{t} < t_f$. For the sake of the definiteness, assume that the trajectory achieves the positive boundary $z = z^+(t)$, which yields (see the equation (3.75)) $z^+(\bar{t}) = \int_{t_f}^{\bar{t}} R(\sigma)d\sigma$. If, from the point $(\bar{t}, z^+(\bar{t}))$, the game is governed by the pair of controls (3.68)-(3.69), then, due to (3.75), the aforementioned trajectory coincides with the arc $z = z^+(t)$, $t \in [\bar{t}, t_f]$. This possibility also yields $z(t_f) = 0$ and the zero game outcome. Therefore, the components of the candidate saddle point in feedback controls in $\mathcal{R}_s$ can be chosen arbitrarily, subject to Definition 3.5.5.

**Case 1.** $N_R = 1$.

**Subcase 1.1** $R(t) \ge 0$ for all $t \in (t_{s1}, t_f)$.

In this subcase, there are two types of the trajectories $\tilde{z}(t)$, given by (3.67). These trajectories are depicted in Fig. 3.3 for $h_u(t) = 14t^2 + 100$, $h_v(t) = -t^3 - 65t$, $t_0 = 3$, $t_f = 6$, $t_{s1} = 4$.

Each trajectory $\tilde{z}(t)$ of the first type exists in the proper interval $[\tilde{t}, t_f]$, where $\tilde{t} > t_{s1}$. These trajectories behave similarly to Subcase 0.1, i.e., they slope down monotonically for $z(t_f) > 0$ and slope up monotonically for $z(t_f) < 0$, while $t$ varies from $t_f$ backwardly. The trajectories of the second type exist in the entire interval $[t_0, t_f]$. These trajectories for $z(t_f) > 0$, slope down monotonically on $(t_{s1}, t_f]$ and slope up monotonically on $[t_0, t_{s1})$, while $t$ varies backwardly. For $z(t_f) < 0$, the behaviour of these trajectories is opposite. In Fig. 3.3, the curves $z = z^+(t)$ and $z = z^-(t)$, $t \in [t_0, t_{s1}] = [1, 4]$ are the arcs of the trajectories $\tilde{z}(t)$, $t \in [t_0, t_f]$, tangent to the $t$-axis at the
<figure><img src="image_6.png" /><figcaption>Figure 3.3: Family of the trajectories $\tilde{z}(t)$ in Subcase 1.1</figcaption></figure>

point $(t_{s1}, 0) = (4, 0)$, namely:

$$ 
\begin{aligned}
z^{+}(t) &= \int_{t_{s1}}^{t} R(\sigma) d\sigma = \int_{4}^{t} (\sigma^{3} - 14\sigma^{2} + 65\sigma - 100) d\sigma \\
&= 0.25t^{4} - \frac{14}{3}t^{3} + 32.5t^{2} - 100t + 114.6667, \quad t \in [t_{0}, t_{s1}] = [1, 4],
\end{aligned}
\quad (3.79) $$

$$ z^{-}(t) = -z^{+}(t), \quad t \in [t_{0}, t_{s1}] = [1, 4]. $$

The aforementioned trajectories of both types cover the regular region $\mathcal{R}_r$, given as:

$$ \mathcal{R}_r = \mathcal{R}_{r1} \cup \mathcal{R}_{r2}, \quad (3.80) $$

where

$$ \mathcal{R}_{r1} = \mathcal{S}_{[t_{s1}, t_f]} = \{(t, z) : t \in [t_{s1}, t_f], z \in E^1\}, \quad (3.81) $$

$$ \mathcal{R}_{r2} = \{(t, z) : z \ge z^{+}(t) \text{ or } z \le z^{-}(t), t \in [t_0, t_{s1}]\}. \quad (3.82) $$

Thus, $\mathcal{R}_r \subset \mathcal{S}_{[t_0, t_f]}$. The latter means the existence of the nonempty singular region $\mathcal{R}_s$, given as follows:

$$ \mathcal{R}_s = \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in [t_0, t_{s1}]\}. \quad (3.83) $$

**Remark 21.** In this subcase, the strip $S_{[t_0, t_f]}$ can be decomposed into two strips $S_{[t_{s1}, t_f]}$ and $S_{[t_0, t_{s1}]}$ such that $S_{[t_0, t_f]} = S_{[t_{s1}, t_f]} \cup S_{[t_0, t_{s1}]}$ and $S_{[t_{s1}, t_f]} \cap S_{[t_0, t_{s1}]} = \{(t, z) : t = t_{s1}, z \in E^1\}$. The function $R(t)$ is non-negative in the interval $(t_{s1}, t_f)$ and we are in the conditions of Subcase 0.1 where $t_0$ is replaced with $t_{s1}$. Therefore, in the strip $S_{[t_{s1}, t_f]}$ the family of the trajectories $\tilde{z}(t)$ is constructed due to Subcase 0.1 (see Fig. 3.4 a)). Note that the end points of
these trajectories fill the entire line $t = t_{s1}$. Furthermore, the function $R(t)$ is non-positive in the interval $(t_0, t_{s1})$ and we are in the conditions of Subcase 0.2 where $t_f$ is replaced with $t_{s1}$. Therefore, in the strip $S_{[t_0, t_{s1}]}$ the family of the trajectories $\tilde{z}(t)$ is constructed due to Subcase 0.2 (see Fig. 3.4 b)). By joining Figs. 3.4 a) and 3.4 b) along the line $t = t_{s1}$, we obtain the family of the trajectories $\tilde{z}(t)$ in the entire strip $S_{[t_0, t_f]}$ (compare with Fig. 3.3).

<figure><img src="image_7.png" /><figcaption>Figure 3.4: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Subcase 1.1</figcaption></figure>

Due to Remark 21, for $(t, z) \in S_{[t_{s1}, t_f]}$, the components of the candidate saddle point in feedback controls are constructed like in Subcase 0.1, while for $(t, z) \in S_{[t_0, t_{s1}]}$, such components are constructed like in Subcase 0.2. Sure, that in the set $\{(t, z) : t = t_{s1}, z \in E^1\}$, both constructions yield the same result. Thus, we have the following. For $(t, z) \in \mathcal{R}_r$, the aforementioned components are given by the equations (3.73)-(3.74). For $(t, z) \in \mathcal{R}_s$, the components of the candidate saddle point in feedback controls can be chosen arbitrarily, subject to Definition 3.5.5.

**Subcase 1.2.** $R(t) \le 0$ for all $t \in (t_{s1}, t_f)$.

In this subcase, for $z(t_f) > 0$, the trajectories $\tilde{z}(t)$, given by (3.67), slope up monotonically in the interval $[t_{s1}, t_f]$ and slope down monotonically in the interval $[t_0, t_{s1}]$, while $t$ varies backwardly. The family of these trajectories is either of the form, shown in Fig. 3.5 ($h_u(t) = t, h_v(t) \equiv 4, t_0 = 1, t_f = 6, t_{s1} = 4$), or of the form, shown in Fig. 3.6 ($h_u(t) = t, h_v(t) \equiv 2, t_0 = 1, t_f = 6, t_{s1} = 2$). In both figures, the curves $z = z^+(t)$ and $z = z^-(t)$ are the limits of the trajectories $\tilde{z}(t)$ for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively. In Fig. 3.5, it is shown the case (Subcase 1.2.1) where there exists the unique time instant $t_{in} \in [t_0, t_{s1})$ satisfying the equation

$$ \int_{t_f}^{t_{in}} R(\sigma) d\sigma = 0. \qquad (3.84) $$

For the data of Fig. 3.5, $t_{in} = 2$.
<figure><img src="image_8.png" /><figcaption>Figure 3.5: Family of the trajectories $\tilde{z}(t)$ in Subcase 1.2.1</figcaption></figure>

The case of nonexistence of such $t_{in}$ (Subcase 1.2.2) is shown in Fig. 3.6. Let $\hat{t} = t_{in}$ in Subcase 1.2.1, and $\hat{t} = t_0$ in Subcase 1.2.2. Then

$$z^{+}(t) = \int_{t_f}^{t} R(\sigma)d\sigma, \quad t \in [\hat{t}, t_f], \qquad (3.85)$$

and $z^{-}(t) = -z^{+}(t)$, $t \in [\hat{t}, t_f]$.

Based on Figs. 3.5 and 3.6, we can conclude the following. In Subcase 1.2.1, the regular region $\mathcal{R}_r$ is given as:

$$\mathcal{R}_r = S_{[t_0, t_{in}]} \cup \{(t, z) : z \geq z^{+}(t) \text{ or } z \leq z^{-}(t), t \in [t_{in}, t_f]\}, \qquad (3.86)$$

while in Subcase 1.2.2, the regular region $\mathcal{R}_r$ is given by (3.77).

The singular region $\mathcal{R}_s$ is given as:

$$\mathcal{R}_s = \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in (\hat{t}, t_f)\}. \qquad (3.87)$$

**Remark 22.** Similarly to Subcase 1.1, the family of the trajectories $\tilde{z}(t)$ can be constructed by a proper decomposition of the strip $S_{[t_0, t_f]}$. Namely, the function $R(t)$ becomes non-positive in the interval $(t_{s1}, t_f)$ and we are in the conditions of Subcase 0.2 where $t_0$ is replaced with $t_{s1}$. Therefore, in the strip $S_{[t_{s1}, t_f]}$ the family of the trajectories $\tilde{z}(t)$ is constructed due to Subcase 0.2 (see Fig. 3.7 a) and 3.8 a)). Furthermore, the function $R(t)$ becomes non-negative in the interval $(t_0, t_{s1})$, i.e., we fall into the conditions of Subcase 0.1. However, to provide a proper matching of the family of the trajectories $\tilde{z}(t)$ in the strip $S_{[t_{s1}, t_f]}$ with such a family in the strip $S_{[t_0, t_{s1}]}$, we should use Subcase 0.1 in the strip $S_{[t_0, t_{s1}]}$ in a reduced form, i.e., only for the right-hand
<figure><img src="image_9.png" /><figcaption>Figure 3.6: Family of the trajectories $\tilde{z}(t)$ in Subcase 1.2.2</figcaption></figure>

initial positions $z(t)|_{t=t_{s1}}$ lying in the two half-lines $\{t = t_{s1}, z \ge z^+(t_{s1})\}$ and $\{t = t_{s1}, z \le z^-(t_{s1})\}$. Such a structure of the set of the right-hand initial positions yields the singular region $\mathcal{R}_s$ (see Fig. 3.7 b) and 3.8 b)), although the "pure" Subcase 0.1 lacks a singular region.

<figure><img src="image_10.png" /><figcaption>Figure 3.7: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Subcase 1.2.1</figcaption></figure>

Based on Remark 22, we can conclude the following. For $(t, z) \in S_{[t_0, t_{s1}]}$, the components of the candidate saddle point in feedback controls are constructed like in Subcase 0.1, while for $(t, z) \in S_{[t_{s1}, t_f]}$, such components are constructed like in Subcase 0.2. In the set $\{(t, z) : t = t_{s1}, z \ge z^+(t_{s1}) \text{ or } z \le z^-(t_{s1})\}$, both constructions yield the same result. Therefore, if $(t, z) \in \mathcal{R}_r$, the aforementioned components are given by the equations (3.73)-(3.74). For $(t, z) \in \mathcal{R}_s$, the components of the candidate saddle point in feedback controls can be chosen arbitrarily, subject to Definition 3.5.5.

**Case 2.** $N_R = 2$.

**Subcase 2.1.** $R(t) \ge 0$ for all $t \in (t_{s2}, t_f)$.
<figure><img src="image_11.png" /><figcaption>Figure 3.8: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Subcase 1.2.2</figcaption></figure>

In this subcase, similarly to Subcase 1.2, the family of the trajectories $\tilde{z}(t)$, given by (3.67), can be of two types depending on the existence of the solution $t_{in} \in [t_0, t_{s1})$ to the equation

$$ \int_{t_{s2}}^{t_{in}} R(\sigma)d\sigma = 0. $$

If such a solution $t_{in}$ exists (Subcase 2.1.1), the family of the trajectories $\tilde{z}(t)$ has the form shown in Fig. 3.9 ($h_u(t) \equiv 2$, $h_v(t) = t^2 - 6.5t + 12$, $t_0 = 1$, $t_f = 6$, $t_{s1} = 2.5$, $t_{s2} = 4$).

<figure><img src="image_12.png" /><figcaption>Figure 3.9: Family of the trajectories $\tilde{z}(t)$ in Subcase 2.1.1</figcaption></figure>

For the data of Fig. 3.9, $t_{in} = 1.75$.

If the aforementioned $t_{in}$ does not exist (Subcase 2.1.2), the family of the trajectories $\tilde{z}(t)$ has the form shown in Fig. 3.10 ($h_u(t) \equiv 14$, $h_v(t) = t^2 - 5.5t + 20$, $t_0 = 1$, $t_f = 6$, $t_{s1} = 1.5$, $t_{s2} = 4$).

The curves $z = z^+(t)$ and $z = z^-(t)$ are the arcs of the trajectories $\tilde{z}(t)$, tangent to the $t$-axis at the time instant $t_{s2}$. The curve $z = z^+(t)$ is given by
<figure><img src="image_13.png" /><figcaption>Figure 3.10: Family of the trajectories $\tilde{z}(t)$ in Subcase 2.1.2</figcaption></figure>

the equation (3.85) where $\hat{t} = t_{in}$ in Subcase 2.1.1 and $\hat{t} = t_0$ in Subcase 2.1.2, while $t_f$ is replaced with $t_{s2}$. The curve $z^{-}(t)$ is given as $z^{-}(t) = -z^{+}(t)$.

Using Figs. 3.9 and 3.10, we obtain the decomposition of the SGBC space into the regular and singular regions. Namely, in Subcase 2.1.1, the regular region $\mathcal{R}_r$ is given as:

$$ \mathcal{R}_r = \mathcal{S}_{[t_0, t_{in}]} \cup \{(t, z) : z \ge z^+(t) \text{ or } z \le z^-(t), t \in [t_{in}, t_{s2}]\} \cup \mathcal{S}_{[t_{s2}, t_f]}, \quad (3.88) $$

while in Subcase 2.1.2, the regular region $\mathcal{R}_r$ is given as:

$$ \mathcal{R}_r = \{(t, z) : z \ge z^+(t) \text{ or } z \le z^-(t), t \in [t_0, t_{s2}]\} \cup \mathcal{S}_{[t_{s2}, t_f]}. \quad (3.89) $$

The singular region $\mathcal{R}_s$ is given as:

$$ \mathcal{R}_s = \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in (\hat{t}, t_{s2})\}, \quad (3.90) $$

where $\hat{t} = t_{in}$ in Subcase 2.1.1 and $\hat{t} = t_0$ in Subcase 2.1.2.

**Remark 23.** In Subcase 2.1, the family of the trajectories $\tilde{z}(t)$ can be also constructed similarly to Case 1 by the decomposition of the strip $S_{[t_0, t_f]}$. This decomposition is presented in Figs. 3.11 and 3.12. It is seen that in the strip $S_{[t_{s1}, t_f]}$, we have Subcase 1.1 where $t_0$ is replaced with $t_{s1}$ (see Figs. 3.11 a) and 3.12 a)). In the strip $S_{[t_0, t_{s1}]}$, we have Subcase 0.1 where $t_f$ is replaced with $t_{s1}$ and the right-hand initial positions $z(t)|_{t=t_{s1}}$ lie in the two half-lines $\{t = t_{s1}, z \ge z^+(t_{s1})\}$ and $\{t = t_{s1}, z \le z^-(t_{s1})\}$ (see Figs. 3.11 b) and 3.12 b)). Joining the two parts of Fig. 3.11, as well as the two parts of Fig. 3.12, along the line $t = t_{s1}$ yields the family of the trajectories $\tilde{z}(t)$ in the entire strip $S_{[t_0, t_f]}$ for Subcases 2.1.1 and 2.1.2, respectively, (compare with Figs. 3.9 and 3.10).
<figure><img src="image_14.png" /><figcaption>Figure 3.11: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Subcase 2.1.1</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Figure 3.12: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Subcase 2.1.2</figcaption></figure>

Using Remark 23, we can construct the components of the candidate saddle point in feedback controls as follows. For $(t, z) \in S_{[t_0, t_{s1}]}$, these components are constructed as in Subcase 0.1 with replacement $t_f$ by $t_{s1}$. For $(t, z) \in S_{[t_{s1}, t_f]}$ these components are constructed as in Subcase 1.1 with replacement $t_0$ by $t_{s1}$. In the set $\{(t, z) : t = t_{s1}, z \ge z^+(t_{s1}) \text{ or } z \le z^-(t_{s1})\}$, both constructions yield the same result. Thus, for $(t, z) \in \mathcal{R}_r$, the aforementioned components are given by the equations (3.73)-(3.74). For $(t, z) \in \mathcal{R}_s$, the components of the candidate saddle point in feedback controls can be chosen arbitrarily, subject to Definition 3.5.5.

**Subcase 2.2** $R(t) \le 0$ for all $t \in (t_{s2}, t_f)$.

In this subcase, the family of the trajectories $\tilde{z}(t)$, given by (3.67), can be of three types depending of existence and placement of the solution $t_{in} \in [t_0, t_{s2})$ to the equation (3.84). If such a solution $t_{in}$ exists and belongs to the interval $(t_{s1}, t_{s2})$ (Subcase 2.2.1), the family of the trajectories $\tilde{z}(t)$ has the form shown in Fig. 3.13 ($h_u(t) = t^2 + 7.5$, $h_v(t) = 6.5t$, $t_0 = 0$, $t_f = 6$, $t_{s1} = 1.5$, $t_{s2} = 5$). For these data, $t_{in} = 3.75$. The curves $z_1^+(t)$ and $z_2^+(t)$ are described by the equations

$$z_1^+(t) = \int_{t_{s1}}^t R(\sigma) d\sigma = \int_{1.5}^t (-\sigma^2 + 6.5\sigma - 7.5) d\sigma$$
$$
\begin{align}
&= -\frac{t^3}{3} + 3.25t^2 - 7.5t + 5.0625, \quad t \in [t_0, t_{s1}] = [1, 1.5]; \nonumber \\
& z_2^+(t) = \int_{t_f}^t R(\sigma)d\sigma = \int_6^t (-\sigma^2 + 6.5\sigma - 7.5)d\sigma \nonumber \\
&= -\frac{t^3}{3} + 3.25t^2 - 7.5t, \quad t \in [t_{in}, t_f] = [3.75, 6], \tag{3.91}
\end{align}
$$

and $z_1^-(t) = -z_1^+(t)$, $t \in [t_0, t_{s1}] = [1, 1.5]$; $z_2^-(t) = -z_2^+(t)$, $t \in [t_{in}, t_f] = [3.75, 6]$.

Let us denote

$$
z^{+}(t) \triangleq \begin{cases} z_{1}^{+}(t), & t \in [t_{0}, t_{s1}], \\ z_{2}^{+}(t), & t \in [t_{in}, t_{f}]. \end{cases} \tag{3.92}
$$

<figure><img src="image_16.png" /><figcaption>Figure 3.13: Family of the trajectories $\tilde{z}(t)$ in Subcase 2.2.1</figcaption></figure>

Due to Fig. 3.13, the regular region $\mathcal{R}_r$ in Subcase 2.2.1 is described as
follows:

$$
\begin{align}
& \mathcal{R}_r = \mathcal{R}_{r1} \cup \mathcal{R}_{r2} \cup \mathcal{R}_{r3}, \nonumber \\
& \mathcal{R}_{r1} = \{(t, z) : z \ge z_1^+(t) \text{ or } z \le z_1^-(t), t \in [t_0, t_{s1}]\}, \quad \mathcal{R}_{r2} = \mathcal{S}_{[t_{s1}, t_{in}]}, \nonumber \\
& \mathcal{R}_{r3} = \{(t, z) : z \ge z_2^+(t) \text{ or } z \le z_2^-(t), t \in [t_{in}, t_f]\}. \tag{3.93}
\end{align}
$$

Note that each of the positive and negative boundaries of the regular
region $\mathcal{R}_r$ consists of the curves of two types. Namely, the curves $z = z_1^+(t)$
and $z = z_1^-(t)$ are the arcs of the trajectories $\tilde{z}(t)$, tangent to the $t$-axis at
the time instant $t_{s1}$. The curves $z = z_2^+(t)$ and $z = z_2^-(t)$ are the limits
of the trajectories $\tilde{z}(t)$ for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively. Due to the equations (3.92) and (3.93), the positive and negative boundaries of the regular region $\mathcal{R}_r$ can be described by the equations $z = z^+(t)$ and $z = -z^+(t)$, respectively, where $t \in [t_0, t_{s1}] \cup [t_{in}, t_f]$.

The singular region $\mathcal{R}_s = \mathcal{S}_{[t_0, t_f]} \setminus \mathcal{R}_r$ is a non-connected set in the $(t, z)$-plane described as:

$$ \begin{aligned} \mathcal{R}_s &= \mathcal{R}_{s1} \cup \mathcal{R}_{s2}, \\ \mathcal{R}_{s1} &= \{(t, z) : z_1^-(t) < z < z_1^+(t), t \in [t_0, t_{s1}]\}, \\ \mathcal{R}_{s2} &= \{(t, z) : z_2^-(t) < z < z_2^+(t), t \in (t_{in}, t_f]\}. \end{aligned} \tag{3.94} $$

Moreover,

$$ \mathrm{clo}(\mathcal{R}_{s1}) \cap \mathrm{clo}(\mathcal{R}_{s2}) = \emptyset. \tag{3.95} $$

If the solution $t_{in}$ of the equation (3.84) exists and coincides with $t_{s1}$ ($t_{in} = t_{s1}$, Subcase 2.2.2), the family of the trajectories $\tilde{z}(t)$ has the form shown in Fig. 3.14 ($h_u(t) = t^2 + 6.75$, $h_v(t) = 6t$, $t_0 = 0$, $t_f = 6$, $t_{s1} = 1.5$, $t_{s2} = 4.5$). The curves $z = z^+(t)$ and $z = z^-(t)$ are the arcs of the trajectories $\tilde{z}(t)$, tangent to the $t$-axis at the time instant $t = t_{in} = t_{s1}$. The curve $z = z^+(t)$ is given by the equation (3.85) where $\hat{t} = t_0$. The curve $z^-(t)$ is given as $z^-(t) = -z^+(t)$. Note that the curves $z = z^+(t)$ and $z = z^-(t)$ are the limits of the trajectories $\tilde{z}(t)$ for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively.

<figure><img src="image_17.png" /><figcaption>Figure 3.14: Family of the trajectories $\tilde{z}(t)$ in Subcase 2.2.2</figcaption></figure>

Based on Fig. 3.14, we obtain the following formula for the regular region $\mathcal{R}_r$ in Subcase 2.2.2:

$$ \mathcal{R}_r = \{(t, z) : z \ge z^+(t) \text{ or } z \le z^-(t), t \in [t_0, t_f]\}. \tag{3.96} $$
Like in Subcase 2.2.1, the singular region $\mathcal{R}_s$ in Subcase 2.2.2 also is a non-connected set in the $(t, z)$-plane. It is described as:

$$ \begin{align} \mathcal{R}_s &= \mathcal{R}_{s1} \cup \mathcal{R}_{s2}, \nonumber \\ \mathcal{R}_{s1} &= \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in [t_0, t_{s1})\}, \nonumber \\ \mathcal{R}_{s2} &= \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in (t_{s1}, t_f)\}. \tag{3.97} \end{align} $$

However, in contrast with Subcase 2.2.1, the intersection of the closures of $\mathcal{R}_{s1}$ and $\mathcal{R}_{s2}$ is nonempty, i.e., $\text{clo}(\mathcal{R}_{s1}) \cap \text{clo}(\mathcal{R}_{s2}) = \{t_{s1}, 0\} \neq \emptyset$.

If the solution $t_{in}$ of the equation (3.84) does not exist (Subcase 2.2.3), the family of the trajectories $\tilde{z}(t)$ has the form shown in Fig. 3.15 ($h_u(t) = t^2+6$, $h_v(t) = 5.5t$, $t_0 = 0$, $t_f = 6$, $t_{s1} = 1.5$, $t_{s2} = 4$). The curves $z = z^{+}(t)$ and $z = z^{-}(t)$ are the limits of the trajectories $\tilde{z}(t)$ for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively, but in contrast with Subcase 2.2.2, they are not tangent to the $t$-axis. The curve $z = z^{+}(t)$ is given by the equation (3.85) with $\hat{t} = t_0$, while the curve $z^{-}(t)$ is given as $z^{-}(t) = -z^{+}(t)$.

Using Fig. 3.15, we obtain that the analytical description of the regular region in Subcase 2.2.3 is the same as in Subcase 2.2.2, i.e., it is (3.96). However, opposite to Subcases 2.2.1 and 2.2.2, the singular region $\mathcal{R}_s$ in Subcase 2.2.3 is simply connected, and it is described as:

$$ \mathcal{R}_s = \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in [t_0, t_f)\}. \tag{3.98} $$

<figure><img src="image_18.png" /><figcaption>Figure 3.15: Family of the trajectories $\tilde{z}(t)$ in Subcase 2.2.3</figcaption></figure>
**Remark 24.** Figs. 3.13 - 3.15 clearly show that in Subcase 2.2 the family of the trajectories $\tilde{z}(t)$ can be constructed (similarly to Subcase 2.1) by the decomposition of the strip $S_{[t_0,t_f]}$ into two strips $S_{[t_0,t_{s_1}]}$ and $S_{[t_{s_1},t_f]}$. Then in the strip $S_{[t_0,t_{s_1}]}$, this family is constructed by Subcase 0.2, while in the strip $S_{[t_{s_1},t_f]}$, it is constructed by Subcase 1.2. More precisely, in the strip $S_{[t_{s_1},t_f]}$, the family of the trajectories $\tilde{z}(t)$ is constructed as follows: in Subcases 2.2.1 and 2.2.2 - due to Subcase 1.2.1 (see Fig. 3.5) where $t_0$ is replaced with $t_{s_1}$; in Subcase 2.2.3, - due to Subcase 1.2.2 (see Fig. 3.6) with the same replacement. In the strip $S_{[t_0,t_{s_1}]}$, the pure Subcase 0.2 (see Fig. 3.2) with $t_f = t_{s_1}$ is realized for Subcases 2.2.1 and 2.2.2. However, for Subcase 2.2.3, the right-hand initial positions $z(t_f = t_{s_1})$ of Subcase 0.2 are chosen from the two half-lines $\{t = t_{s_1}, z \ge z^+(t_{s_1})\}$ and $\{t = t_{s_1}, z \le z^-(t_{s_1})\}$.

Using Remark 24, we obtain that in Subcase 2.2 the components of the candidate saddle point in feedback controls are constructed as follows. For $(t, z) \in S_{[t_{s_1},t_f]}$, these components constructed like in Subcase 1.2, while for $(t, z) \in S_{[t_0,t_{s_1}]}$, they are constructed like in Subcase 0.2. Note, that in the set $\{(t, z) : t = t_{s_1}, z \ge z^+(t_{s_1}) \text{ or } z \le z^-(t_{s_1})\}$, both constructions yield the same result.

**Case k.** $N_R = k$ ($k \ge 1$).

**Subcase k.1.** $R(t) \ge 0$ for all $t \in (t_{sk}, t_f)$.

Based on Cases 0, 1, 2 and Remarks 21 - 24, the family of the trajectories $\tilde{z}(t)$, given in the equation (3.67), can be constructed by the decomposition of the strip $S_{[t_0,t_f]}$ into two strips $S_{[t_0,t_{s_1}]}$ and $S_{[t_{s_1},t_f]}$. In the strip $S_{[t_{s_1},t_f]}$, the family of the trajectories $\tilde{z}(t)$ is constructed by Subcase $(k-1).1$. Let $\mathcal{E}_1$ be the set of all end points $(t_{s_1}, \tilde{z}(t_{s_1}))$ of the trajectories of this family. Note that $\mathcal{E}_1$ can be either the entire line $t = t_{s_1}$, or two non-connected with each other half-lines, belonging to the line $t = t_{s_1}$ and symmetrical with respect to the $t$-axis. In the strip $S_{[t_0,t_{s_1}]}$, the family of the trajectories $\tilde{z}(t)$ is constructed by Subcase 0.1 (for even $k$) and Subcase 0.2 (for odd $k$), with the initial right-hand positions in $\mathcal{E}_1$.

**Subcase k.2.** $R(t) \le 0$ for all $t \in (t_{sk}, t_f)$.

In the strip $S_{[t_{s_1},t_f]}$, the family of the trajectories $\tilde{z}(t)$ is constructed by Subcase $(k-1).2$. Let $\mathcal{E}_2$ be the set of all end points $(t_{s_1}, \tilde{z}(t_{s_1}))$ of the trajectories of this family. The set $\mathcal{E}_2$ has the similar structure as the set $\mathcal{E}_1$ in Subcase k.1. In the strip $S_{[t_0,t_{s_1}]}$, the family of the trajectories $\tilde{z}(t)$ is constructed by Subcase 0.2 (for even $k$) and Subcase 0.1 (for odd $k$), with the initial right-hand positions in $\mathcal{E}_2$.

The components of the candidate saddle point in feedback strategies in Subcases k.1 and k.2 are derived in accordance with the above described construction of the family of the trajectories $\tilde{z}(t)$ (like it is done in Cases 0,
1, 2).

We illustrate Case *k* by the following examples.

**Example 3.5.11.** Let $h_u(t) = 95t^2 + 280$, $h_v(t) = 10t^3 + 290t$ in the interval $[t_0, t_f] = [1.5, 4.5]$. Due to the equation (3.66), the determining function $R(t)$ in this example is $R(t) = 10t^3 - 95t^2 + 290t - 280$. In Fig. 3.16, the graph of this function is shown. It is seen that in the interval $[1.5, 4.5]$, the function $R(t)$ changes its sign three times, i.e., $N_R = 3$ and $t_{s1} = 2$, $t_{s2} = 3.5$, $t_{s3} = 4$. Moreover, $R(t) > 0$ for $t \in (t_{s3}, t_f) = (4, 4.5)$. Thus, we are in Subcase 3.1.

<figure><img src="image_19.png" /><figcaption>Figure 3.16: Graph of $R(t)$ in Example 3.5.11</figcaption></figure>

In Fig. 3.17 a) the construction of the family of the trajectories $\tilde{z}(t)$ in the strip $S_{[t_{s1},t_f]} = S_{[2,4.5]}$ by Subcase 2.1 is presented (compare with Fig. 3.9). Here, $\mathcal{E}_1 = \{(t,z) : t = t_{s1} = 2, z \in E^1\}$. In Fig. 3.17 b) the construction of the family of the trajectories $\tilde{z}(t)$ in the strip $S_{[t_0,t_{s1}]} = S_{[1.5,2]}$ by the "pure" Subcase 0.2, is shown (compare with Fig. 3.2).

<figure><img src="image_20.png" /><figcaption>Figure 3.17: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Example 3.5.11 (Subcase 3.1)</figcaption></figure>
In Fig. 3.18, the family of the trajectories $\tilde{z}(t)$ in the entire strip $S_{[t_0, t_f]} = S_{[1.5,4.5]}$, obtained by the joining Figs. 3.17 a) and b) along the line $t = t_{s1} = 2$, is depicted. In all these figures, the curves $z = z_1^+(t)$ and $z = z_1^-(t)$ are arcs of the trajectories $\tilde{z}(t)$ tangent to the t-axis at the time instant $t_{s1}$, while $z = z_2^+(t)$, $z = z_2^-(t)$ are arcs of the trajectories $\tilde{z}(t)$ tangent to the t-axis at the time instant $t_{s3}$. These curves are given as:

$$z_1^+(t) = \int_{t_{s1}}^t R(\sigma)d\sigma = \int_2^t (10\sigma^3 - 95\sigma^2 + 290\sigma - 280)d\sigma \\ = 2.5t^4 - 31.6667t^3 + 145t^2 - 280t + 193.3333, \quad t \in [t_0, t_{s1}] = [1.5, 2],$$

$$z_2^+(t) = \int_{t_{s3}}^t R(\sigma)d\sigma = \int_4^t (10\sigma^3 - 95\sigma^2 + 290\sigma - 280)d\sigma \\ = 2.5t^4 - 31.6667t^3 + 145t^2 - 280t + 186.6667, \quad t \in [t_{in}, t_{s3}] = [3.22, 4],$$

$$z_1^-(t) = -z_1^+(t), \quad t \in [t_0, t_{s1}] = [1.5, 2]; \\ z_2^-(t) = -z_2^+(t), \quad t \in [t_{in}, t_{s3}] = [3.22, 4],$$

where $t_{in} \in (t_{s1}, t_{s2})$ is the solution of the equation

$$\int_{t_{s3}}^{t_{in}} R(\sigma)d\sigma = 0.$$

From Fig. 3.18, we see that the regular region $\mathcal{R}_r$, which is filled by the trajectories $\tilde{z}(t)$, can be presented as:

$$\mathcal{R}_r = \mathcal{R}_{r1} \cup \mathcal{R}_{r2} \cup \mathcal{R}_{r3} \cup \mathcal{R}_{r4}, \\ \mathcal{R}_{r1} = \{(t, z) : z \ge z_1^+(t) \text{ or } z \le z_1^-(t), t \in [t_0, t_{s1}]\}, \quad \mathcal{R}_{r2} = \mathcal{S}_{[t_{s1}, t_{in}]} \\ \mathcal{R}_{r3} = \{(t, z) : z \ge z_2^+(t) \text{ or } z \le z_2^-(t), t \in [t_{in}, t_{s3}]\}, \quad \mathcal{R}_{r4} = \mathcal{S}_{[t_{s3}, t_f]}.$$

The positive and negative boundaries of the regular region $\mathcal{R}_r$ are described by the equations $z = z^+(t)$ and $z = z^-(t) = -z^+(t)$, respectively, $t \in [t_0, t_{s1}] \cup [t_{in}, t_{s3}]$, where

$$z^+(t) = \begin{cases} z_1^+(t), & t \in [t_0, t_{s1}], \\ z_2^+(t), & t \in [t_{in}, t_{s3}]. \end{cases}$$

The singular region $\mathcal{R}_s = S_{[t_0, t_f]} \setminus \mathcal{R}_r$, which is not filled by the trajectories $\tilde{z}(t)$, consists of two disconnected sets $\mathcal{R}_{s1}$ and $\mathcal{R}_{s2}$, i.e., $\mathcal{R}_s = \mathcal{R}_{s1} \cup \mathcal{R}_{s2}$, where

$$\mathcal{R}_{s1} = \{(t, z) : z_1^-(t) < z < z_1^+(t), t \in [t_0, t_{s1}]\}, \\ \mathcal{R}_{s2} = \{(t, z) : z_2^-(t) < z < z_2^+(t), t \in (t_{in}, t_{s3}]\}.$$
<figure><img src="image_21.png" /><figcaption>Figure 3.18: Family of the trajectories $\tilde{z}(t)$ in Example 3.5.11 (Subcase 3.1)</figcaption></figure>

Based on this structure of the family of the trajectories $\tilde{z}(t)$ and on the
corresponding decomposition of the game space $S_{[t_0, t_f]}$ into the regular $\mathcal{R}_r$
and singular $\mathcal{R}_s$ regions, the components of the candidate saddle point in
feedback controls are designed as follows. For $(t, z) \in \mathcal{R}_r$, these components
are given by the equations (3.73)-(3.74). For $(t, z) \in \mathcal{R}_s$, the components
of the candidate saddle point in feedback controls can be chosen arbitrarily
subject to Definition 3.5.5.

**Example 3.5.12.** In this example, the functions $h_u(t)$ and $h_v(t)$ are the following: $h_u(t) = t^3 + 30.25t$, $h_v(t) = 10t^2 + 26.25$ in the interval $[t_0, t_f] = [1, 6]$. In Fig. 3.19, the graph of the determining function $R(t) = -t^3 + 10t^2 - 30.25t + 26.25$ is shown. The number of its sign changes ($N_R = 3$) and the respective zeros of $R(t)$ are $t_{s1} = 1.5$, $t_{s2} = 3.5$, $t_{s3} = 5$. It is seen that, in contrast with Example 3.5.11, $R(t) < 0$ for $t \in (t_{s3}, t_f)$. Thus, we are in Subcase 3.2.

In Fig. 3.20 a) the construction of the family of the trajectories $\tilde{z}(t)$ in the strip $S_{[t_{s1}, t_f]} = S_{[1.5, 6]}$ by Subcase 2.2 is presented (compare with Fig. 3.15). Here, $\mathcal{E}_2 = \{(t, z) : t = 1.5, z \in [6.3281, +\infty) \cup (-\infty, -6.3281]\}$.

In Fig. 3.20 b) the construction of the family of the trajectories $\tilde{z}(t)$ in the strip $S_{[t_0, t_{s1}]} = S_{[1, 1.5]}$ by Subcase 0.2 with the initial positions in $\mathcal{E}_2$, is shown.

In Fig. 3.21, the family of the trajectories $\tilde{z}(t)$ in the entire strip $S_{[t_0, t_f]} = S_{[1, 6]}$, obtained by the joining of Figs. 3.20 a) and b) along the line $t = t_{s1} = 1.5$, is depicted. In Fig. 3.21, the curves $z = z^+(t)$ and $z = z^-(t)$ are the limits of the trajectories $\tilde{z}(t)$ for $z(t_f) \to +0$ and $z(t_f) \to -0$, respectively. In Figs. 3.20 a) and b), the corresponding segments of these limits are presented.
<figure><img src="image_22.png" /><figcaption>Figure 3.19: Graph of $R(t)$ in Example 3.5.12</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>Figure 3.20: Decomposition of the family of the trajectories $\tilde{z}(t)$ in Example 3.5.12 (Subcase 3.2)</figcaption></figure>

The curves $z = z^{+}(t)$ and $z = z^{-}(t)$ are described by the following equations for $t \in [t_0, t_f] = [1, 6]$:

$$z^{+}(t) = -0.25t^{4} + 3.3333t^{3} - 15.125t^{2} + 26.25t - 9, \quad z^{-}(t) = -z^{+}(t).$$

From Fig. 3.21, we see that the regular $\mathcal{R}_r$ and singular $\mathcal{R}_s$ regions are described as: $\mathcal{R}_r = \{(t, z) : z \ge z^{+}(t) \text{ or } z \le z^{-}(t), t \in [t_0, t_f]\}$, $\mathcal{R}_s = \{(t, z) : z^{-}(t) < z < z^{+}(t), t \in [t_0, t_f]\}$. Note that in contrast with Example 3.5.11, in the present example the singular region is a simply connected set.

Using this structure of the family of the trajectories $\tilde{z}(t)$ and the corresponding decomposition of the game space $S_{[t_0, t_f]}$ into the regular $\mathcal{R}_r$ and singular $\mathcal{R}_s$ regions, the components of the candidate saddle point in feedback controls are designed as follows. For $(t, z) \in \mathcal{R}_r$, these components are given by the equations (3.68)-(3.69). For $(t, z) \in \mathcal{R}_s$, these components can be chosen arbitrarily subject to Definition 3.5.5.

**Stage III: Justification of the formally constructed saddle point.**
<figure><img src="image_24.png" /><figcaption>Figure 3.21: Family of the trajectories $\tilde{z}(t)$ in Example 3.5.12 (Subcase 3.2)</figcaption></figure>

**Theorem 3.5.13.** The SGBC has the saddle point in the feedback controls $(u_{\text{sg}}^0(t, z), v_{\text{sg}}^0(t, z))$, $(t, z) \in S_{[t_0, t_f]}$. For $(t, z) \in \mathcal{R}_r$, the components $u_{\text{sg}}^0(\cdot)$ and $v_{\text{sg}}^0(\cdot)$ of this saddle point are given by (3.73) and (3.74), respectively. For $(t, z) \in \mathcal{R}_s$, the pair of the controls $(u_{\text{sg}}^0(t, z), v_{\text{sg}}^0(t, z))$ is arbitrary admissible.

Saddle-point trajectory in feedback controls of the SGBC, starting at any position $(\bar{t}, \bar{z}) \in \mathcal{R}_r$, remains in $\mathcal{R}_r$ till the end of the game, i.e., $z^0(t; \bar{t}, \bar{z}) \in \mathcal{R}_r$, $t \in [\bar{t}, t_f]$.

For any given starting position $(\bar{t}, \bar{z}) \in S_{[t_0, t_f]}$ of the SGBC, its value in feedback controls is

$$J_z^0 = J_z^0(\bar{t}, \bar{z}) = \begin{cases} |\bar{z}| + \int_{\bar{t}}^{t_f} R(\sigma) d\sigma, & \text{if } (\bar{t}, \bar{z}) \in \mathcal{R}_r, \\ z^+(\bar{t}) + \int_{\bar{t}}^{t_f} R(\sigma) d\sigma, & \text{if } (\bar{t}, \bar{z}) \in \mathcal{R}_s, \end{cases} \quad (3.99)$$

where $R(t), t \in [t_0, t_f]$ is the determining function; $z = z^+(t), t \in \mathcal{T}$ is the positive boundary of the regular region $\mathcal{R}_r$ and $\mathcal{T} \subseteq [t_0, t_f]$ is its existence set.

The proof of the theorem is presented in Subsection 3.6.4.

**Corollary 3.5.14.** Let $S_s$ be any connected subset of the singular region $\mathcal{R}_s \neq \emptyset$ of the SGBC. Let $(\bar{t}_1, \bar{z}_1)$ and $(\bar{t}_2, \bar{z}_2)$ be any points from $S_s$. Then,

$$J_z^0(\bar{t}_1, \bar{z}_1) = J_z^0(\bar{t}_2, \bar{z}_2). \quad (3.100)$$

*Proof.* First of all let us note that, due to the equation (3.99), the statement of the corollary becomes trivial if $\bar{t}_1 = \bar{t}_2$. Therefore, in the sequel of the
proof, we assume that $\bar{t}_1 \neq \bar{t}_2$. Furthermore, for the sake of definiteness, we assume that $\bar{t}_1 < \bar{t}_2$.

Since $S_s$ is a connected part of $\mathcal{R}_s$ and $(\bar{t}_i, \bar{z}_i) \in S_s$, $(i = 1, 2)$ then, due to the results of Stages I and II, the positive boundary $z = z^+(t)$ of the regular region $\mathcal{R}_r$ exists in the entire interval $[\bar{t}_1, \bar{t}_2]$. Moreover, in this interval the function $z^+(t)$ can be represented as:

$$z^+(t) = z_f + \int_{t_f}^{t} R(\sigma) d\sigma, \quad t \in [\bar{t}_1, \bar{t}_2], \qquad (3.101)$$

where $z_f \ge 0$ is a proper number.

Now, using the equations (3.99) and (3.101), we obtain

$$\begin{aligned} J_z^0(\bar{t}_i, \bar{z}_i) &= z^+(\bar{t}_i) + \int_{\bar{t}_i}^{t_f} R(\sigma) d\sigma \\ &= z_f + \int_{t_f}^{\bar{t}_i} R(\sigma) d\sigma + \int_{\bar{t}_i}^{t_f} R(\sigma) d\sigma = z_f, \quad i = 1, 2, \end{aligned}$$

which implies the validity of the equality (3.100).

Thus, the corollary is proven. □

**Remark 25.** Using the results of the work [12], as well as Theorem 3.5.13 and Corollary 3.5.14, we directly obtain that any connected parts of the positive and negative boundaries of the regular region $\mathcal{R}_r$ are semi-permeable curves.

### 3.5.4 Solution of the VGBC

Based on the feedback controls $u_{\text{sg}}^0(t, z)$ and $v_{\text{sg}}^0(t, z)$ of the SGBC (see the equations (3.73) and (3.74)), we construct the following feedback controls in the VGBC:

$$u^0(t, x) = u_{\text{sg}}^0(t, z(t, x)), \quad (t, x) \in [t_0, t_f] \times E^n, \qquad (3.102)$$

$$v^0(t, x) = v_{\text{sg}}^0(t, z(t, x)), \quad (t, x) \in [t_0, t_f] \times E^n, \qquad (3.103)$$

where the scalar function $z(t, x)$ is given by (3.5).

**Theorem 3.5.15.** The VGBC has the saddle point in the feedback controls $(u^0(t, x), v^0(t, x)), (t, x) \in [t_0, t_f] \times E^n$. For all $(t, x)$ such that $(t, z(t, x)) \in \mathcal{R}_r$, the components $u^0(\cdot)$ and $v^0(\cdot)$ of this saddle point are given by (3.102) and (3.103), respectively. For all $(t, x)$ such that $(t, z(t, x)) \in \mathcal{R}_s$, the pair of the controls $(u^0(\cdot), v^0(\cdot))$ is arbitrary admissible. For any given starting position $(\bar{t}, \bar{x})$ of the VGBC such that $(\bar{t}, z(\bar{t}, \bar{x})) \in \mathcal{R}_r$, its saddle-point trajectory
in feedback controls $x^0(t) = x^0(t; \bar{t}, \bar{x})$ satisfies the inclusion $(t, z(t, x^0(t))) \in \mathcal{R}_r, t \in [\bar{t}, t_f]$.

For any given starting position $(\bar{t}, \bar{x}) \in [t_0, t_f] \times E^n$ of the VGBC, its value in feedback controls is

$$ J_x^0 = J_x^0(\bar{t}, \bar{x}) = \begin{cases} |z(\bar{t}, \bar{x})| + \int_{\bar{t}}^{t_f} R(\sigma) d\sigma, & \text{if } (\bar{t}, z(\bar{t}, \bar{x})) \in \mathcal{R}_r, \\ z^+(\bar{t}) + \int_{\bar{t}}^{t_f} R(\sigma) d\sigma, & \text{if } (\bar{t}, z(\bar{t}, \bar{x})) \in \mathcal{R}_s, \end{cases} \quad (3.104) $$

where $\mathcal{R}_r$ and $\mathcal{R}_s$ are the regular and singular regions of the SGBC; the functions $R(t)$ and $z^+(t)$ are the same as in Theorem 3.5.13.

The proof of the theorem is presented in Subsection 3.6.5.

The following assertion is a direct consequence of Theorem 3.5.15. Using the state transformation (3.6) in the VGBC, this assertion is proven similarly to Corollary 3.5.14.

**Corollary 3.5.16.** Let $S_s$ be any connected subset of the singular region $\mathcal{R}_s \neq \emptyset$ of the SGBC. Let $(\bar{t}_1, \bar{x}_1)$ and $(\bar{t}_2, \bar{x}_2)$ be any points from $[t_0, t_f] \times E^n$ such that $(\bar{t}_1, z(\bar{t}_1, \bar{x}_1)) \in S_s$ and $(\bar{t}_2, z(\bar{t}_2, \bar{x}_2)) \in S_s$. Then,

$$ J_x^0(\bar{t}_1, \bar{x}_1) = J_x^0(\bar{t}_2, \bar{x}_2). $$

**Remark 26.** Theorems 3.5.13, 3.5.15 and Corollaries 3.5.14, 3.5.16 imply the equivalence of the VGBC and the SGBC with respect to the saddle point and the game value, despite that the scalarizing transformation (3.6) is not a bijection.

## 3.6 Proofs of Theorems 3.4.7-3.4.10 and 3.5.13-3.5.15

### 3.6.1 Proof of Theorem 3.4.7

Let us start with the case

$$ z_0 \geq \int_{t_0}^{t_f} |h_1(t)| dt, \quad (3.105) $$

meaning that $\operatorname{sign}(z_0) = 1$.

First, we are going to show that the controls $u^*(t) = -\operatorname{sign}(h_u(t))$ and $v^*(t) = \operatorname{sign}(h_v(t))$ satisfy the definitions of the optimality in the SGBC. For
this purpose, due to Remark 9 and Definitions 3.4.1, 3.4.2, we should prove
the validity of the inequalities

$$
\sup_{v(\cdot) \in \mathcal{C}} J_z(u^*(\cdot), v(\cdot)) \le \sup_{v(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) \quad \forall u(\cdot) \in \mathcal{C}, \qquad (3.106)
$$

and

$$
\inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v^*(\cdot)) \ge \inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) \quad \forall v(\cdot) \in \mathcal{C}. \qquad (3.107)
$$

Begin with $u^*(t) = -\text{sign}(h_u(t))$. Solving the initial value problem (3.7) with $u = u^*(t)$ and any $v(\cdot) \in \mathcal{C}$, we obtain

$$
J_z(u^*(\cdot), v(\cdot)) = \left| z_0 - \int_{t_0}^{t_f} |h_u(t)| dt + \int_{t_0}^{t_f} h_v(t) v(t) dt \right|. \quad (3.108)
$$

Due to (3.105), the supremum of (3.108) with respect to $v(\cdot) \in \mathcal{C}$ is attained
at $v(t) = v^*(t)$, and

$$
\sup_{v(\cdot) \in \mathcal{C}} J_z(u^*(\cdot), v(\cdot)) = z_0 - \int_{t_0}^{t_f} |h_u(t)| dt + \int_{t_0}^{t_f} |h_v(t)| dt. \quad (3.109)
$$

Now, for any $u(\cdot) \in \mathcal{C}$ and $v(\cdot) \in \mathcal{C}$, we have

$$
J_z(u(\cdot), v(\cdot)) = \left| z_0 + \int_{t_0}^{t_f} h_u(t)u(t)dt + \int_{t_0}^{t_f} h_v(t)v(t)dt \right|. \quad (3.110)
$$

Note that since $u(\cdot) \in \mathcal{C}$, the following inequality is satisfied:

$$
z_0 + \int_{t_0}^{t_f} h_u(t)u(t)dt \geq z_0 - \int_{t_0}^{t_f} |h_u(t)|dt \geq 0. \quad (3.111)
$$

Hence, the supremum of the functional (3.110) with respect to $v(\cdot) \in \mathcal{C}$ also
is attained at $v(t) = v^*(t)$, and

$$
\sup_{v(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) = z_0 + \int_{t_0}^{t_f} h_u(t)u(t)dt + \int_{t_0}^{t_f} |h_v(t)|dt. \quad (3.112)
$$

By virtue of the inequality (3.111), the equations (3.109) and (3.112)
yield the inequality (3.106), meaning that $u^*(\cdot)$ is the optimal open-loop
minimizer's control of the SGBC in the case (3.105). Moreover, in this case,
the upper value of the SGBC is

$$
J_{zu} = J_z(u^*(\cdot), v^*(\cdot)) = z_0 - \int_{t_0}^{t_f} |h_u(t)| dt + \int_{t_0}^{t_f} |h_v(t)| dt. \quad (3.113)
$$
Proceed to the analysis of the control $v^*(t) = \text{sign}(h_v(t))$. Similarly to (3.108), we obtain

$$J_z(u(\cdot), v^*(\cdot)) = \left|z_0 + \int_{t_0}^{t_f} h_u(t)u(t)dt + \int_{t_0}^{t_f} |h_v(t)|dt\right|. \quad (3.114)$$

Due to (3.111), the infinum of the functional (3.114) with respect to $u(\cdot) \in \mathcal{C}$ is attained at $u(t) = u^*(t)$, and

$$\inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) = z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt. \quad (3.115)$$

To calculate $\inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot))$ for arbitrary $v(\cdot) \in \mathcal{C}$, we introduce the following subclass $\mathcal{C}_u \subset \mathcal{C}$:

$$\mathcal{C}_u = \{u(\cdot) \in \mathcal{C} : u(t) = \kappa(\text{sign}(h_u(t))), t \in [t_0, t_f], \kappa \in [-1, 1]\}. \quad (3.116)$$

Now, we replace the minimization of $J_z(u(\cdot), v(\cdot))$ in the class $\mathcal{C}$ by its minimization in the subclass $\mathcal{C}_u$. Let arbitrary $v(\cdot) \in \mathcal{C}$ be fixed. Then for $u(\cdot) \in \mathcal{C}_u$, the functional $J_z(u(\cdot), v(\cdot))$ becomes a function of $\kappa$:

$$J_z(u(\cdot), v(\cdot)) = |a + b\kappa|, \quad (3.117)$$

where

$$a = z_0 + \int_{t_0}^{t_f} h_v(t)v(t)dt, \quad b = \int_{t_0}^{t_f} |h_1(t)|dt > 0. \quad (3.118)$$

Thus, the minimization of $J_z(u(\cdot), v(\cdot))$ by $u(\cdot) \in \mathcal{C}_u$ is transformed to its minimization by $\kappa \in [-1, 1]$. For the latter, three cases can be distinguished (see Fig. 3.22 for the graphical illustration).

(i) If $|a/b| \le 1$, then the infinum of (3.117) with respect to $\kappa \in [-1, 1]$ is attained at $\kappa^* = -a/b$, and

$$\inf_{u(\cdot) \in \mathcal{C}_u} J_z(u(\cdot), v(\cdot)) = 0. \quad (3.119)$$

(ii) If $a/b < -1$, then the infinum of (3.117) with respect to $\kappa \in [-1, 1]$ is attained at $\kappa^* = 1$, and

$$\inf_{u(\cdot) \in \mathcal{C}_u} J_z(u(\cdot), v(\cdot)) = |a + b| = -(a + b). \quad (3.120)$$

(iii) If $a/b > 1$, then the infinum of (3.117) with respect to $\kappa \in [-1, 1]$ is attained at $\kappa^* = -1$, and
<figure><img src="image_25.png" /><figcaption>Figure 3.22: Minimization of $J_z$ over $C_u$</figcaption></figure>

$$ \inf_{u(\cdot) \in C_u} J_z(u(\cdot), v(\cdot)) = |a - b| = a - b. \qquad (3.121) $$

Since $J_z(u(\cdot), v(\cdot)) \ge 0$ for all $u(\cdot), v(\cdot) \in C$, the equality (3.119) implies that in the case (i),

$$ \inf_{u(\cdot) \in C} J_z(u(\cdot), v(\cdot)) = \inf_{u(\cdot) \in C_u} J_z(u(\cdot), v(\cdot)) = 0. \qquad (3.122) $$

In the cases (ii) and (iii), using the inequality

$$ \left| \int_{t_0}^{t_f} h_u(t)u(t)dt \right| \le \int_{t_0}^{t_f} |h_u(t)|dt, \qquad (3.123) $$

we directly obtain

$$ \inf_{u(\cdot) \in C} J_z(u(\cdot), v(\cdot)) = \inf_{u(\cdot) \in C_u} J_z(u(\cdot), v(\cdot)) = \begin{cases} -(a+b), & a < -b, \\ a-b, & a > b. \end{cases} \qquad (3.124) $$

To complete the proof of the optimality of $v^*(\cdot) = \text{sign}(h_v(t))$, we should show that the inequality (3.107) is valid in all three cases (i) - (iii). In the case (i), due to (3.105) and (3.122), this inequality is obvious. In two other cases, (ii) and (iii), it is a direct consequence of (3.105) and the inequality

$$ \left| \int_{t_0}^{t_f} h_v(t)v(t)dt \right| \le \int_{t_0}^{t_f} |h_v(t)|dt. \qquad (3.125) $$
Namely, using (3.125), we have in the case (ii)

$$
\begin{align}
& \inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v^*(\cdot)) - \inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) \nonumber \\
& = \left( z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt \right) \nonumber \\
& - \left( -z_0 - \int_{t_0}^{t_f} |h_u(t)|dt - \int_{t_0}^{t_f} h_v(t)v(t)dt \right) \nonumber \\
& = 2z_0 + \int_{t_0}^{t_f} |h_v(t)|dt + \int_{t_0}^{t_f} h_v(t)v(t)dt \ge 0. \tag{3.126}
\end{align}
$$

In the case (iii), using (3.125), we obtain

$$
\begin{align}
& \inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v^*(\cdot)) - \inf_{u(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) \nonumber \\
& = \left( z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt \right) \nonumber \\
& - \left( z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} h_v(t)v(t)dt \right) \nonumber \\
& = \int_{t_0}^{t_f} |h_v(t)|dt - \int_{t_0}^{t_f} h_v(t)v(t)dt \ge 0. \tag{3.127}
\end{align}
$$

Thus, $v^*(t) = \text{sign}(h_v(t))$ is the optimal open-loop maximizer's control of the SGBC in the case (3.105). Moreover, in this case, the lower value of the SGBC is

$$
J_{zl} = J_z(u^*(\cdot), v^*(\cdot)) = z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt. \quad (3.128)
$$

Now we show that in the case (3.105), the pair $(u^*(\cdot), v^*(\cdot)) = (-\text{sign}(h_u(t)), \text{sign}(h_v(t)))$ is a saddle point (in the open-loop controls) of the SGBC, i.e., that $J_{zu} = J_{zl}$. This equality is a direct consequence of the equations (3.113) and (3.128). Moreover, the game value is

$$
J_z^* = J_{zu} = J_{zl} = J_z(u^*(\cdot), v^*(\cdot)) = z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt. \quad (3.129)
$$

The case

$$
z_0 \le - \int_{t_0}^{t_f} |h_u(t)| dt, \qquad (3.130)
$$

is treated similarly. In this case, the pair $(u^*(\cdot), v^*(\cdot)) =$
(sign($h_u(t)$), -sign($h_v(t)$)) is the saddle point (in the open-loop controls) of
the SGBC and the game value is

$$
J_z^* = -z_0 - \int_{t_0}^{t_f} |h_u(t)|dt + \int_{t_0}^{t_f} |h_v(t)|dt. \quad (3.131)
$$

Finally, the equations (3.129) and (3.131) yield (3.24), which completes the
proof of the theorem.
### 3.6.2 Proof of Theorem 3.4.9

First of all, note that, due to the condition (3.28), there exist an infinite set of controls $u^*(\cdot) \in \mathcal{C}$ satisfying the integral equation (3.29). Let $u^*(\cdot)$ be any such a control. Then

$$ \sup_{v(\cdot) \in \mathcal{C}} J_z(u^*(\cdot), v(\cdot)) = \int_{t_0}^{t_f} |h_v(t)| dt. \qquad (3.132) $$

For an arbitrary $u(\cdot) \in \mathcal{C}$, we have

$$ \sup_{v(\cdot) \in \mathcal{C}} J_z(u(\cdot), v(\cdot)) = \left| z_0 + \int_{t_0}^{t_f} h_1(t)u(t)dt \right| + \int_{t_0}^{t_f} |h_2(t)|dt. \qquad (3.133) $$

In (3.132) and (3.133), the supremum is attained at $v(t) = \text{sign}(h_v(t))$.

Comparing the equations (3.132) and (3.133) yields the inequality (3.106), proving the optimality of $u^*(\cdot)$. The latter, along with (3.14), leads to (3.30). Thus, the theorem is proven.

### 3.6.3 Proof of Theorem 3.4.10

First, consider the case (3.31). Similarly to the case (i) in the proof of Theorem 3.4.7 (see Subsection 3.6.1), we obtain that for any $v(\cdot) \in \mathcal{C}$, the equation (3.122) is valid, and the infinum is attained at $u(t) = -(a/b)\text{sign}(h_u(t))$, where $a$ and $b$ are given by (3.118). The latter proves that in the case (3.31), any maximizer's control $v(\cdot) \in \mathcal{C}$ is optimal and (3.32) is valid.

In the case (3.33), the optimality of $v^*(\cdot)$, given by (3.34), is shown by the same arguments as the optimality of $v^*(t) = \text{sign}(h_v(t))$ in the proof of Theorem 3.4.7. Moreover, the expression (3.35) for the lower value of the SGBC in the case (3.33) is obtained in the same way as (3.128). This completes the proof of the theorem.

### 3.6.4 Proof of Theorem 3.5.13

For the sake of transparency, the proof is carried out in the Subcase 1.1 of the decomposition of the game's space $S_{[t_0, t_f]}$ ($N_R = 1$, $R(t) \ge 0$ for $t \in (t_{s1}, t_f)$, see Fig. 3.3). We start the proof with establishing the optimality of the minimizer's feedback control $u_{\text{sg}}^0(t, z)$, $(t, z) \in S_{[t_0, t_f]}$ (see the equation (3.73)). Remember that the optimality of $u_{\text{sg}}^0(t, z)$ is determined in Definition 3.5.6.

#### 1. Optimality of $u_{\text{sg}}^0(\cdot)$.

##### Case I. The initial game position $(\bar{t}, \bar{z}) \in \mathcal{R}_r$.

For the sake of definiteness, assume that $\bar{z} \ge 0$. Begin with the left-hand side of (3.46).
### Subcase I.1.

$$ \bar{z} \geq \int_{t_{s1}}^{\bar{t}} R(\sigma) d\sigma, \quad \bar{z} \neq 0. \qquad (3.134) $$

The set $\mathcal{L}_v^z(u_{sg}^0(\cdot))$ (see the equation (3.44)) can be partitioned into two subsets such that $\mathcal{L}_v^z(u_{sg}^0(\cdot)) = \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot)) \cup \mathcal{L}_{v,2}^z(u_{sg}^0(\cdot))$. The subset $\mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))$ consists of all feedback controls $v(\cdot)$ such that any trajectory $z = z(t)$, $t \in [\bar{t}, t_f]$ of the differential equation in (3.7), starting at $(\bar{t}, \bar{z})$ and generated by the pair $(u^0(\cdot), v(\cdot))$, does not intersect the curve $z = \int_{t_{s1}}^t R(\sigma)d\sigma$, $t \in [t_0, t_f]$, meaning that $z(t) > 0$, $t \in [\bar{t}, t_f]$; $\mathcal{L}_{v,2}^z(u_{sg}^0(\cdot)) = \mathcal{L}_v^z(u_{sg}^0(\cdot)) \setminus \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))$.

Let $\mathcal{L}_{v,1}^z(u_{sg}^0(\cdot)) \neq \emptyset$ and $v(\cdot) \in \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))$. Then, for any aforementioned trajectory $z(t)$,

$$ J_z(u^0(\cdot), v(\cdot)) = z(t_f) > 0. \qquad (3.135) $$

Let $z^0(t)$, $t \in [\bar{t}, t_f]$ be the trajectory of the differential equation in (3.7), generated from the same initial position $(\bar{t}, \bar{z})$ by the pair of the feedback controls $(u^0(\cdot), v^0(\cdot))$, where $v^0(\cdot)$ is given by (3.74). Due to Remark 20, this trajectory is unique. Thus, we have

$$ J_z(u^0(\cdot), v^0(\cdot)) = z^0(t_f) = \bar{z} - \int_{\bar{t}}^{t_f} |h_u(t)| dt + \int_{\bar{t}}^{t_f} |h_v(t)| dt > 0. \quad (3.136) $$

Since $v(\cdot) \in \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))$, then $|v(t, z)| \le 1$, $(t, z) \in S_{[t_0, t_f]}$. Therefore, $|h_v(t)v(t, z)| \le h_v(t)v_0(t, z)$ for all $(t, z) \in [\bar{t}, t_f] \times \{z > 0\}$. The latter, along with the Krasovskii's constructive motion [14], yields the inequality $z^0(t_f) \ge z(t_f)$ valid for all $v(\cdot) \in \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))$. Hence, using (3.66) and (3.135)-(3.136), we obtain

$$ \sup_{v(\cdot) \in \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))} J_z(u^0(\cdot), v(\cdot)) = J_z(u^0(\cdot), v^0(\cdot)) = \bar{z} + \int_{\bar{t}}^{t_f} R(t) dt. \quad (3.137) $$

Now, let $\mathcal{L}_{v,2}^z(u_{sg}^0(\cdot)) \neq \emptyset$ and $v(\cdot) \in \mathcal{L}_{v,2}^z(u_{sg}^0(\cdot))$. Thus, there exists at least one trajectory $z = z(t)$, $t \in [\bar{t}, t_f]$ of the differential equation in (3.7), starting at the point $(\bar{t}, \bar{z})$ and generated by the pair $(u^0(\cdot), v(\cdot))$, which intersects the curve $z = \int_{t_{s1}}^t R(\sigma)d\sigma$, $t \in [t_0, t_f]$. Let $\mathcal{Z}(u^0(\cdot), v(\cdot))$ be the set of all trajectories $z = z(t)$, $[\bar{t}, t_f]$ of the differential equation in (3.7), starting at $(\bar{t}, \bar{z})$ and generated by the pair $(u^0(\cdot), v(\cdot))$. We partition this set into two subsets such that $\mathcal{Z}(u^0(\cdot), v(\cdot)) = \mathcal{Z}_1(u^0(\cdot), v(\cdot)) \cup \mathcal{Z}_2(u^0(\cdot), v(\cdot))$. The subset $\mathcal{Z}_1(u^0(\cdot), v(\cdot))$ consists of all the trajectories, not intersecting the curve $z = \int_{t_{s1}}^t R(\sigma)d\sigma$, while the subset $\mathcal{Z}_2(u^0(\cdot), v(\cdot))$ consists of all the trajectories, intersecting this curve. Since $\mathcal{L}_{v,2}^z(u_{sg}^0(\cdot)) \neq \emptyset$, then $\mathcal{Z}_2(u^0(\cdot), v(\cdot)) \neq \emptyset$, while the subset $\mathcal{Z}_1(u^0(\cdot), v(\cdot))$ can be either nonempty or empty.
First, we assume that $\mathcal{Z}_1(u^0(\cdot), v(\cdot))$ is nonempty. For any trajectory $z(\cdot) \in \mathcal{Z}_1(u^0(\cdot), v(\cdot))$, the equation (3.135) holds. Thus, due to (3.137), for any $v(\cdot) \in \mathcal{L}_{v,2}^z(u_{sg}^0(\cdot))$ and $z(\cdot) \in \mathcal{Z}_1(u^0(\cdot), v(\cdot))$,

$$J_z(u^0(\cdot), v(\cdot)) \leq \sup_{v(\cdot) \in \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))} J_z(u^0(\cdot), v(\cdot)). \quad (3.138)$$

Moreover, due to the equation (3.12), Definition 3.5.5 and Remark 15, the inequality (3.138) is valid for any $v(\cdot) \in \mathcal{L}_{v,2}^z(u_{sg}^0(\cdot))$ and $z(\cdot) \in \mathcal{Z}(u^0(\cdot), v(\cdot))$.

Proceed with the assumption that $\mathcal{Z}_1(u^0(\cdot), v(\cdot))$ is empty. Thus, any $z(\cdot) \in \mathcal{Z}_2(u^0(\cdot), v(\cdot))$. If the first intersection of this trajectory with the curve $z = \int_{t_{s1}}^t R(\sigma)d\sigma$ occurs for $\tilde{t} < t_{s1}$ (i.e., $\bar{t} < t_{s1}$ and the trajectory penetrates into the singular region $\mathcal{R}_s$), then, due to the inequalities

$$h_v(t)v(t, z) - |h_u(t)|\mathrm{sign}(z) \leq R(t), \quad \text{if } z = \int_{t_{s1}}^t R(\sigma)d\sigma, \quad t \in [t_0, t_f], \\ h_v(t)v(t, -z) - |h_u(t)|\mathrm{sign}(z) \geq -R(t), \quad \text{if } z = \int_t^{t_{s1}} R(\sigma)d\sigma, \quad t \in [t_0, t_f], \qquad (3.139)$$

the trajectory cannot leave $\mathcal{R}_s$ before $t = t_{s1}$, and, moreover, it remains in the closed domain $AB^+B^-$ for $t \in [t_{s1}, t_f]$ (see Fig. 3.23). Similarly, if the first intersection of the trajectory with the curve $z = \int_{t_{s1}}^t R(\sigma)d\sigma$ occurs for $\tilde{t} \geq t_{s1}$, then, due to the inequalities in (3.139), this trajectory remains in the closed domain $AB^+B^-$ for $t \in (\tilde{t}, t_f]$. Thus, for $v(\cdot) \in \mathcal{L}_{v,2}^z(u_{sg}^0(\cdot))$, $z(\cdot) \in \mathcal{Z}_2(u^0(\cdot), v(\cdot))$,

$$J_z(u^0(\cdot), v(\cdot)) \leq \int_{t_{s1}}^{t_f} R(t)dt. \qquad (3.140)$$

Since $\bar{z} > 0$, then, due to (3.137), (3.138) and (3.140),

$$\sup_{v(\cdot) \in \mathcal{L}_{v,2}^z(u_{sg}^0(\cdot))} J_z(u^0(\cdot), v(\cdot)) \leq \sup_{v(\cdot) \in \mathcal{L}_{v,1}^z(u_{sg}^0(\cdot))} J_z(u^0(\cdot), v(\cdot)), $$

and, therefore,

$$\sup_{v(\cdot) \in \mathcal{L}_v^z(u_{sg}^0(\cdot))} J_z(u^0(\cdot), v(\cdot)) = J_z(u^0(\cdot), v^0(\cdot)) = \bar{z} + \int_{\bar{t}}^{t_f} R(t)dt. \quad (3.141)$$

**Subcase I.2.**

$$\bar{t} \geq t_{s1}, \quad 0 \leq \bar{z} \leq \int_{t_{s1}}^{\bar{t}} R(\sigma)d\sigma. \qquad (3.142)$$

This subcase is treated similarly to Subcase I.1, yielding (3.141).
<figure><img src="image_26.png" /><figcaption>Figure 3.23: Geometrical illustration of the proof of Theorem 3.5.13</figcaption></figure>

Proceed to the right-hand side of (3.46). In both subcases (see (3.134) and (3.142)), for any trajectory $z = z(t)$, $t \in [\bar{t}, t_f]$ of the differential equation in (3.7), starting at the point $(\bar{t}, \bar{z})$ and generated by the pair of controls $(u(\cdot), v^0(\cdot))$, $(u(\cdot) \in \mathcal{L}_u^z(v_{\text{sg}}^0(\cdot)))$, we obtain

$$J_z(u(\cdot), v^0(\cdot)) = z(t_f) \geq \bar{z} + \int_{\bar{t}}^{t_f} R(t)dt, \quad (3.143)$$

which is a direct consequence of the inequality

$$h_u(t)u(t, z) + |h_2(t)|\text{sign}(z) \geq R(t)\text{sign}(z), \quad \forall z > 0.$$

The equation (3.141) and the inequality (3.143) yield immediately the validity of the inequality (3.46) for $(\bar{t}, \bar{z}) \in \mathcal{R}_r$, $\bar{z} \geq 0$. The case $\bar{z} < 0$ is treated similarly.

**Case II. The initial game position $(\bar{t}, \bar{z}) \in \mathcal{R}_s$.** This means that $\bar{t} \in [t_0, t_{s1})$.

By the same arguments as in Subcase I.1, in the present case we have

$$\sup_{v(\cdot) \in \mathcal{L}_v^z(u_{\text{sg}}^0(\cdot))} J_z(u^0(\cdot), v(\cdot)) = \int_{t_{s1}}^{t_f} R(t)dt. \quad (3.144)$$

Moreover, as in Case I, the following inequality is valid:

$$J_z(u(\cdot), v^0(\cdot)) = |z(t_f)| \geq \int_{t_{s1}}^{t_f} R(t)dt \quad \forall u(\cdot) \in \mathcal{L}_u^z(v_{\text{sg}}^0(\cdot)). \quad (3.145)$$

The equality (3.144) and the inequality (3.145) yield (3.46) for any point $(\bar{t}, \bar{z}) \in \mathcal{R}_s$. This completes the proof of the optimality of the minimizer's
feedback control $u^0(\cdot)$. Moreover, using the equation for the positive boundary of $\mathcal{R}_r$ (see the equation (3.79)) and the equations (3.141),(3.144), we directly obtain that the upper value in the feedback strategies of the SGBC coincides with the right-hand side expression in (3.99).

Proceed with the proof of the optimality of the maximizer's feedback control $v_{\text{sg}}^0(t, z)$, $(t, z) \in \mathcal{S}_{[t_0, t_f]}$ (see the equation (3.74)). Remember that the optimality of $v_{\text{sg}}^0(t, z)$ is determined in Definition 3.5.7.

## 2. Optimality of $v_{\text{sg}}^0(\cdot)$.

The proof of the validity of the inequality (3.50) is symmetric to the proof of the inequality (3.46). Moreover, the lower value in the feedback strategies of the SGBC coincides with the right-hand side expression in (3.99).

Thus, in Subcase 1.1 of the game's space decomposition ($N_R = 1$, $R(t) \ge 0$ for $t \in (t_{s1}, t_f)$, see Fig. 3.3), $u^0(\cdot)$ and $v^0(\cdot)$ are optimal feedback controls of the minimizer and maximizer, respectively, in the SGBC; the pair $(u^0(\cdot), v^0(\cdot))$ is the saddle point and the game value is given by (3.99). The other cases and subcases of the game's space decomposition are treated similarly. This completes the proof of the theorem.

## 3.6.5 Proof of Theorem 3.5.15

Like the proof of Theorem 3.5.13, the proof of the present theorem also is carried out for Subcase 1.1 of the SGBC space decomposition ($N_R = 1$, $R(t) \ge 0$ for $t \in (t_{s1}, t_f)$, see Fig. 3.3). Let $(u(t, x), v(t, x)) \in \mathcal{F}^x$ be an admissible pair of feedback controls in the VGBC. Then for any initial position $(\bar{t}, \bar{x}) \in [t_0, t_f] \times E^n$, the system (3.1) has a solution $x(t)$ for $t \in [\bar{t}, t_f]$. Thus, the pair of functions $(x(t), z(t, x(t)))$, where $z(t, x)$ is given by (3.5), is the solution of the system, consisting of the equation (3.1) and the scalar equation

$$ \frac{dz(t)}{dt} = h_u(t)u(t,x) + h_v(t)v(t,x), \quad (3.146) $$

with the initial condition

$$ z(t)|_{t=\bar{t}} = z(\bar{t}, \bar{x}). $$

Note that $|z(t_f)| = J_x(u(\cdot), v(\cdot))$.

The equation (3.146) looks like (3.7) with a single difference. Namely, in (3.7) the feedback controls depend on $(t, z)$, while in (3.146) they depend on $(t, x)$. Nevertheless, for (3.146), the following inequalities, similar to the inequalities (3.139) and (3.144), are valid:

$$ \begin{gathered} -|h_u(t)|\text{sign}(z(t,x)) + h_v(t)v(t,x) \le R(t) \\ \forall (t,x) \in [t_0, t_f] \times E^n \text{ and } z(t,x) = \int_{t_{s1}}^t R(\sigma)d\sigma, \end{gathered} $$
$$-|h_u(t)|\mathrm{sign}(z(t,x)) + h_v(t)v(t,x) \geq -R(t)$$

$$\forall (t, x) \in [t_0, t_f] \times E^n \text{ and } z(t, x) = -\int_{t_{s1}}^{t} R(\sigma)d\sigma,$$

$$h_u(t)u(t,x) + |h_v(t)|\mathrm{sign}(z(t,x)) \geq R(t)\mathrm{sign}(z(t,x)),$$

$$\forall (t, x) \in [t_0, t_f] \times E^n \text{ and } z(t, x) > 0.$$

Using these inequalities, the rest of the proof is carried out quite similarly to the proof of Theorem 3.5.13 after replacing the trajectory of the differential equation (3.7) by the trajectory of the differential equation (3.146).

## 3.7 Concluding Remarks and Literature Review

In this chapter, the two-person zero-sum differential game with the dynamics, described by the n-dimensional linear time-varying non-homogeneous differential system, was considered. The duration of this game is prescribed. The controls of the players are scalar, and these controls are subject to the hard (geometric) constraints. The cost function of the game is the distance between the terminal value of the system's state and a given target hyperplane in $E^n$. Two types of the saddle-point solution to this game were obtained. Namely, in Section 3.4, the saddle-point solution was derived in the proper class of pairs of open-loop players' controls. In Section 3.5, based on the form of this solution, the saddle-point solution in the properly defined class of pairs of players' feedback controls was derived. The derivation of both types of the saddle-point solution is based on the terminal projection transformation of the original differential game to an auxiliary scalar differential game made in Section 3.3 (see Proposition 3.3.1). This transformation takes into account the non-homogeneity of the linear dynamics and the target hyperplane of the original game. Applications of different types of the terminal projection transformation for analysis and solution of various differential games and control problems with linear non-homogeneous/homogeneous dynamics can be found for instance in [2, 3, 4, 5, 6, 7, 9, 11, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23] and references therein. In Subsection 3.4.2, the equivalence of the original game and the scalar game with respect to the open-loop saddle-point solution was established (see Lemma 3.4.4). The necessary and sufficient condition for the existence of this solution was obtained in Subsection 3.4.3 (see Theorem 3.4.12). The solution itself and the game value were derived in Theorem 3.4.7. As it was above mentioned, the feedback saddle-point solution was obtained in Section 3.5. The derivation of this solution is considerably based on
the feedback saddle-point solution of the auxiliary scalar game. This solution consists of the decomposition of the scalar game's space into two regions, the regular and singular ones, and of the saddle-point components of the players in each of these regions. The game space decomposition is based on the behavior of the *determining function* $R(t)$ defined in Subsection 3.5.3. Namely, this decomposition depends: (a) on the number of distinct zeros of $R(t)$ in the open game's time interval where this function changes its sign; (b) on the sign of $R(t)$ in the open time interval between the last zero and the final time instant. It is important to note that this decomposition was carried out for any number of the aforementioned zeros of the determining function. In the regular region the saddle-point components of the players have a bang-bang structure. The value of the game depends on its initial position (*time*, *state variable*) in the regular region. In the singular region, consisting of a finite number of nonintersecting parts, the saddle-point components of the players are arbitrary admissible. The value of the game, starting in each part of the singular region, is constant, but this constant can, in general, change from one part to another. Theorem 3.5.13 and Corollary 3.5.14 summarize the derivation of the feedback saddle-point solution to the scalar differential game. Using these results and the terminal projection transformation of the original differential game, its feedback saddle-point solution was obtained in Subsection 3.5.4 (see Theorem 3.5.15 and Corollary 3.5.16).

In Section 3.6, rigorous proofs of Theorems 3.4.7-3.4.10 and 3.5.13-3.5.15 were done.

Note that various particular cases of the differential game of this chapter, much simpler than this game, were considered in [7, 10, 15, 23]. The differential game, similar to the game in this chapter, was considered for the first time in the literature in the work [4]. However, in the present chapter, the feedback saddle-point solution of this game is derived subject to the new (more precise and more detailed) definitions of the admissible pair of the players' state feedback controls, of the players' optimal controls and of the saddle-point solution (see Remarks 11 - 12, Definitions 3.5.1 - 3.5.4, Remarks 14 - 15 and Definitions 3.5.5 - 3.5.8). In this chapter, the assertions on the feedback saddle-point solutions of the original and auxiliary scalar games are formulated in the new (more compact and more clear) form (see Theorems 3.5.13 and 3.5.15). Moreover, the new assertions (Corollaries 3.5.14 and 3.5.16) and the new illustrative examples (see Figs. 3.1 - 3.21) are presented in this chapter.
Bibliography

[1] Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. SIAM Books, Philadelphia, PA, USA (1999)

[2] Bryson, A.E., Ho, Y.C.: Applied Optimal Control, Hemisphere, New York, NY, USA (1975)

[3] Glizer, V.Y.: Asymptotic analysis and open-loop solution of one class of partial cheap control zero-sum differential games with state and control delays. Commun. Optim. Theory **2022**, Article ID 16 (2022)

[4] Glizer, V.Y., Turetsky, V.: Complete solution of a differential game with linear dynamics and bounded controls. Appl. Math. Res. Express. AMRX **2008**, Article ID abm012 (2008)

[5] Glizer, V.Y., Turetsky, V.: Robust Controllability of Linear Systems, Nova Science Publishers, New York, NY, USA (2012)

[6] Glizer, V.Y., Turetsky, V.: Increasing pursuer capturability by using hybrid dynamics. Int. J. Appl. Math. Comput. Sci. **25**, 77–92 (2015)

[7] Glizer, V.Y., Turetsky, V., Fridman, L., Shinar, J.: History-dependent modified sliding mode interception strategies with maximal capture zone. J. Franklin Inst. **349**, 638–657 (2012)

[8] Gutman, S.: On optimal guidance for homing missiles. J. Guid. Control **3**, 296-300 (1979)

[9] Gutman, S.: Applied Min-Max Approach to Missile Guidance and Control, Series Progress in Astronautics and Aeronautics, Vol. 209, AIAA, Inc., Reston, VA (2006)

[10] Gutman, S., Leitmann G.: Optimal strategies in the neighbourhood of a collision course. AIAA J. **14**, 1210-1212 (1976)
[11] Halanay, A. : Differential games with delay, SIAM J. Control **6**, 579–593 (1968)

[12] Isaacs, R.: Differential Games, John Wiley, New York, NY, USA (1965)

[13] Karlin, S.: Mathematical Methods and Theory in Games, Programming and Economics, Addison-Wesley, Reading, MA, USA (1959)

[14] Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems, Springer, New York, NY, USA (2011)

[15] Shima, T., Shinar, J.: Time-varying linear pursuit-evasion game models with bounded controls. J. Guid. Control Dyn. **25**, 425–432 (2002)

[16] Shinar, J.: Solution techniques for realistic pursuit-evasion games. In C.T. Leonides (ed.) Advances in Control and Dynamic Systems, vol. 17, 63-124. Academic Press, New York, NY, USA (1981)

[17] Shinar, J., Glizer, V.Y., Turetsky, V.: Capture zone of linear strategies in the interception problems with variable structure dynamics. J. Franklin Inst. **351**, 2378–2395 (2014)

[18] Turetsky, V.: Upper bounds of the pursuer control based on a linear-quadratic differential game. J. Optim. Theory Appl. **121**, 163-191 (2004)

[19] Turetsky, V., Glizer, V.Y.: Robust state-feedback controllability of linear systems to a hyperplane in a class of bounded controls. J. Optim. Theory Appl. **123**, 639-667 (2004)

[20] Turetsky, V., Glizer, V.Y.: Continuous feedback control strategy with maximal capture zone in a class of pursuit games. Int. Game Theory Rev. **7**, 1–24 (2005)

[21] Turetsky, V., Glizer, V.Y.: Robust solution of a time-variable interception problem: a cheap control approach. Int. Game Theory Rev. **9**, 637-655 (2007)

[22] Turetsky, V., Glizer, V.Y.: Cheap control in a non-scalarizable linear-quadratic pursuit-evasion game: asymptotic analysis. Axioms **11**, Paper No. 214 (2022)

[23] Turetsky, V., Shinar, J.: Missile guidance laws based on pursuit-evasion game formulations. Automatica J. IFAC **39**, 607-618 (2003)
# Chapter 4

## Scalar Zero-Sum
## Linear-Quadratic Differential
## Game with Cheap Controls

### 4.1 Introduction

In this chapter one class of two-person zero-sum linear-quadratic differential games is considered. The game is finite-horizon. The dynamics of the game is described by a scalar differential equation, right-hand side of which is independent of the state variable. The players' scalar controls are not subject to any constraints. The cost functional of this game is the sum of the cost of the terminal state value and the integral costs of the players' controls. The important feature of the game is that the costs of the controls of both players are much smaller than the cost of the terminal state value. This smallness is due to a small multiplier $\varepsilon > 0$ for the squares of the players' controls in the integral part of cost functional. Thus, the considered game is a cheap control differential game. We look for the state-feedback saddle-point solution of this game uniformly valid for all arbitrarily small values $\varepsilon > 0$. Necessary and sufficient conditions for the existence of such a solution are derived in this chapter. Based on these conditions, the explicit-form solution is obtained.

The following main notations are applied in the chapter.

1. $E^1$ means the particular case of $E^n$ which denotes the $n$-dimensional real Euclidean space. Thus $E^1$ is the set of all real numbers.

2. $\Theta_1 \times \Theta_2$, where $\Theta_1$ and $\Theta_2$ are some sets of real numbers, denotes the direct product of these sets, i.e., the set of all pairs $(\theta_1, \theta_2)$ with any entries $\theta_1 \in \Theta_1$ and $\theta_2 \in \Theta_2$.

3. $L^2[t_1, t_2]$ is the linear space of real functions, square-integrable in the finite
interval $[t_1, t_2]$.

## 4.2 Game Formulation

Consider the differential game with the dynamics described by the following equation:

$$ \frac{dz(t)}{dt} = h_u(t)u(t) + h_v(t)v(t), \quad t \in [t_0, t_f], \quad z(t_0) = z_0, \qquad (4.1) $$

where $z(t)$ is a scalar state variable; $u(t)$ and $v(t)$ are scalar controls of the players; $t_0$ and $t_f$, ($t_0 < t_f$) are initial and terminal time instants; the scalar functions $h_u(t)$, $h_v(t)$ are given and assumed continuous for $t \in [t_0, t_f]$; $z_0$ is a given initial value of the state variable.

The cost functional of the considered game has the form

$$ J(u(\cdot), v(\cdot)) = (z(t_f))^2 + \varepsilon \int_{t_0}^{t_f} \left[ \lambda(u(t))^2 - (v(t))^2 \right] dt, \qquad (4.2) $$

where $\lambda > 0$ is some given number; $\varepsilon > 0$ is a small parameter (a cheap control parameter).

The objective of the player with the control $u(t)$ (the minimizer) is to minimize the cost functional (4.2), while the player with the control $v(t)$ (the maximizer) aims to maximize this cost functional. As the information pattern in this game, we assume that both players are aware of all the game's data, presenting in (4.1)-(4.2), and of the current game's position $(t, z(t))$.

In what follows, we call the differential game, consisting of the equation (4.1), the cost functional (4.2), the aforementioned objectives of the minimizer and the maximizer and the information pattern, the Linear-Quadratic Game with Cheap Controls (LQCC).

## 4.3 Main Definitions

Consider the set $\Omega$ of all functions $\omega(t, z) : [0, t_f] \times E^1 \to E^1$, which are measurable w.r.t. $t \in [0, t_f]$ for any fixed $z \in E^1$ and satisfy the local Lipschitz condition w.r.t. $z \in E^1$ uniformly in $t \in [0, t_f]$.

Based on the results of the book [5], we introduce into the consideration the following definitions.

**Definition 4.3.1.** Let UV be the set of all pairs of functions $(u(t, z), v(t, z))$, $(t, z) \in [0, t_f] \times E^1$, such that the following conditions are valid: (i) $u(t, z) \in \Omega$, $v(t, z) \in \Omega$; (ii) the initial-value problem (4.1) for $u(t) = u(t, z)$, $v(t) =$
$v(t, z)$ and any $z_0 \in E^1$ has the unique absolutely continuous solution $z_{uv}(t; z_0)$ in the entire interval $[t_0, t_f]$; (iii) $u(t, z_{uv}(t; z_0)) \in L^2[t_0, t_f]$; (iv) $v(t, z_{uv}(t; z_0)) \in L^2[t_0, t_f]$. The pair $(u(t, z), v(t, z))$ is called an **admissible pair of the players' feedback controls** ( or, briefly, an **admissible pair**) in the LQGCC, and UV is called the set of all such pairs in this game.

For a given $u(t, z) \in \Omega$, consider the set $\Upsilon_v(u(t, z)) \triangleq \{v(t, z) \in \Omega : (u(t, z), v(t, z)) \in UV\}$. Let $\Sigma_u \triangleq \{u(t, z) \in \Omega : \Upsilon_v(u(t, z)) \neq \emptyset\}$.

**Definition 4.3.2.** The minimizer's feedback control $u^*(\cdot) \in \Sigma_u$ is called **optimal** in the LQGCC, if for any $z_0 \in E^1$ and for any $u(\cdot) \in \Sigma_u$, the following inequality is valid:

$$ \sup_{v(\cdot) \in \Upsilon_v(u^*(\cdot))} J(u^*(\cdot), v(\cdot)) \leq \sup_{v(\cdot) \in \Upsilon_v(u(\cdot))} J(u(\cdot), v(\cdot)). \quad (4.3) $$

The value

$$ J_u = J_u(z_0) = \sup_{v(\cdot) \in \Upsilon_v(u^*(\cdot))} J(u^*(\cdot), v(\cdot)) \quad (4.4) $$

is called the **upper value** of the LQGCC.

Similarly, for a given $v(t, z) \in \Omega$, consider the set $\Upsilon_u(v(t, z)) \triangleq \{u(t, z) \in \Omega : (u(t, z), v(t, z)) \in UV\}$. Let $\Sigma_v \triangleq \{v(t, z) \in \Omega : \Upsilon_u(v(t, z)) \neq \emptyset\}$.

**Definition 4.3.3.** The maximizer's feedback control $v^*(\cdot) \in \Sigma_v$ is called **optimal** in the LQGCC, if for any $z_0 \in E^1$ and for any $v(\cdot) \in \Sigma_v$, the following inequality is valid:

$$ \inf_{u(\cdot) \in \Upsilon_u(v^*(\cdot))} J(u(\cdot), v^*(\cdot)) \geq \inf_{u(\cdot) \in \Upsilon_u(v(\cdot))} J(u(\cdot), v(\cdot)). \quad (4.5) $$

The value

$$ J_1 = J_1(z_0) = \inf_{u(\cdot) \in \Upsilon_u(v^*(\cdot))} J(u(\cdot), v^*(\cdot)) \quad (4.6) $$

is called the **lower value** of the LQGCC.

**Definition 4.3.4.** If $(u^*(\cdot), v^*(\cdot)) \in UV$ and

$$ J_u(z_0) = J_1(z_0) \triangleq J^*(z_0) \quad \forall z_0 \in E^1, \quad (4.7) $$

then $J^*(z_0)$ is called the **value** of the LQGCC and the pair $(u^*(\cdot), v^*(\cdot))$ is called the **saddle point** of the LQGCC. Solution $z^*(t)$ of the initial-value problem (4.1) for $u(t) = u^*(t, z)$, $v(t) = v^*(t, z)$, $(t, z) \in [t_0, t_f] \times E^1$, is called the **saddle-point trajectory** of the LQGCC.
## 4.4 Feedback Saddle-Point Solution of the LQGCC Uniform in the Cheap Control Parameter

Consider the following terminal-value problem for the function $K(t)$:

$$ \frac{dK(t)}{dt} = \frac{(h_u(t))^2/\lambda - (h_v(t))^2}{\varepsilon} K^2(t), \quad t \in [t_0, t_f], \quad K(t_f) = 1. \quad (4.8) $$

Based on the results of [1, 2, 3, 5], we directly have the following assertion.

**Proposition 4.4.1.** If, for some given $\lambda > 0$ and $\varepsilon > 0$, the terminal-value problem (4.8) has the solution $K(t, \lambda, \varepsilon)$ in the entire interval $[t_0, t_f]$, then the LQGCC has the saddle point $(u^*(t, z, \lambda, \varepsilon), v^*(t, z, \lambda, \varepsilon))$, $(t, z) \in [t_0, t_f] \times E^1$, where

$$ u^*(t, z, \lambda, \varepsilon) = -\frac{h_u(t)K(t, \lambda, \varepsilon)}{\varepsilon\lambda}z, \quad v^*(t, z, \lambda, \varepsilon) = \frac{h_v(t)K(t, \lambda, \varepsilon)}{\varepsilon}z. \quad (4.9) $$

Formal solution of the problem (4.8) yields the expression

$$ K(t, \lambda, \varepsilon) = \left[ 1 + \frac{1}{\varepsilon} \int_{t}^{t_f} \left( \frac{(h_u(\sigma))^2}{\lambda} - (h_v(\sigma))^2 \right) d\sigma \right]^{-1}, \quad t \in [t_0, t_f]. \quad (4.10) $$

**Lemma 4.4.2.** The solution $K(t, \lambda, \varepsilon)$ of the terminal-value problem (4.8) exists in the entire interval $[t_0, t_f]$ for all arbitrarily small values $\varepsilon > 0$ if and only if the following inequality is satisfied:

$$ \int_{t}^{t_f} \left( \frac{(h_u(\sigma))^2}{\lambda} - (h_v(\sigma))^2 \right) d\sigma \geq 0, \quad t \in [t_0, t_f]. \quad (4.11) $$

*Proof.* First of all let us observe the following. For the existence of $K(t, \lambda, \varepsilon)$ in the entire interval $[t_0, t_f]$ it is necessary and sufficient that the expression in the square brackets of (4.10) does not become zero at any point of this interval. Since this expression is a continuous function of $t \in [t_0, t_f]$ and it becomes $1 > 0$ at $t = t_f$, then the aforementioned necessary and sufficient condition with respect to the existence of $K(t, \lambda, \varepsilon)$, $t \in [t_0, t_f]$ is transformed into the validity of the inequality

$$ 1 + \frac{1}{\varepsilon} \int_{t}^{t_f} \left( \frac{(h_u(\sigma))^2}{\lambda} - (h_v(\sigma))^2 \right) d\sigma > 0, \quad t \in [t_0, t_f], $$

or the validity of the equivalent inequality

$$ \int_{t}^{t_f} \left( \frac{(h_u(\sigma))^2}{\lambda} - (h_v(\sigma))^2 \right) d\sigma > -\varepsilon, \quad t \in [t_0, t_f]. $$

This inequality is valid for all arbitrarily small values $\varepsilon > 0$ if and only if the inequality (4.11) is satisfied. Thus, the lemma is proven. $\square$
Let us denote

$$H_{uv}(t) \triangleq \frac{\int_{t}^{t_f} (h_u(\sigma))^2 d\sigma}{\int_{t}^{t_f} (h_v(\sigma))^2 d\sigma}, \quad t \in [t_0, t_f]. \qquad (4.12)$$

Taking into account that the integrals $\int_{t}^{t_f} (h_u(\sigma))^2 d\sigma$ and $\int_{t}^{t_f} (h_v(\sigma))^2 d\sigma$ are positive for all $t \in [t_0, t_f)$, we directly have the following two assertions.

**Lemma 4.4.3.** Let the following inequality is satisfied:

$$\lambda_{\inf} \triangleq \inf_{t \in [t_0, t_f)} H_{uv}(t) > 0. \qquad (4.13)$$

Then, the inequality (4.11) is valid if and only if the positive number $\lambda$ satisfies the inequality

$$\lambda \le \lambda_{\inf}. \qquad (4.14)$$

Moreover, the inequality (4.11) becomes a strict one for all $t \in [t_0, t_f)$ if and only if the positive number $\lambda$ satisfies the strict version of the inequality (4.14).

**Corollary 4.4.4.** Let $H_{uv}(t)$ be an increasing function in the interval $[t_0, t_f)$. Then, for $\lambda = \lambda_{\inf} = H_{uv}(t_0)$, the inequality (4.11) is a strict one for all $t \in (t_0, t_f)$, while for $t = t_0$ and $t = t_f$, this inequality becomes an equality.

Substitution of the expression for $K(t, \lambda, \varepsilon)$ (see the equation (4.10)) into the expressions for $u^*(t, z, \lambda, \varepsilon)$ and $v^*(t, z, \lambda, \varepsilon)$ (see the equation (4.9)) yields after a routine algebra the following expressions for these controls:

$$u^*(t, z, \lambda, \varepsilon) = -\frac{h_u(t)}{\varepsilon\lambda + \int_{t}^{t_f} [(h_u(\sigma))^2 - \lambda(h_v(\sigma))^2]d\sigma} z, \quad t \in [t_0, t_f], \qquad (4.15)$$

$$v^*(t, z, \lambda, \varepsilon) = \frac{\lambda h_v(t)}{\varepsilon\lambda + \int_{t}^{t_f} [(h_u(\sigma))^2 - \lambda(h_v(\sigma))^2]d\sigma} z, \quad t \in [t_0, t_f]. \qquad (4.16)$$

Due to Lemmas 4.4.2 and 4.4.3, these expressions are feasible for all arbitrarily small $\varepsilon > 0$ and all $t \in [t_0, t_f]$, if and only if the positive number $\lambda$ satisfies the inequality (4.14). This observation and Proposition 4.4.1 yield immediately the following assertion.

**Theorem 4.4.5.** Controls $u^*(t, z, \lambda, \varepsilon)$ and $v^*(t, z, \lambda, \varepsilon)$, given by the equations (4.15) and (4.16), constitute the saddle point $(u^*(t, z, \lambda, \varepsilon), v^*(t, z, \lambda, \varepsilon))$, $(t, z) \in [t_0, t_f] \times E^1$ of the LQGCC for all arbitrarily small $\varepsilon > 0$, if and only if the positive number $\lambda$ satisfies the inequality (4.14).
## 4.5 Concluding Remarks and Literature Review

In this chapter, the scalar finite-horizon two-person zero-sum linear-quadratic differential game was considered. The cost functional of this game is the sum of the square of the terminal state value and the integral of the weighted sum of the squares of the players’ controls. The coefficient for the square of the control of the minimizing player (the minimizer) is some given positive number $\lambda$ multiplying by a small positive parameter $\varepsilon$. The coefficient for the square of the control of the maximizing player (the maximizer) is $(-\varepsilon)$. Thus, due to the presence of the small parameter $\varepsilon$ in the coefficients for the squares of the players’ controls, the costs of both controls are small (cheap). Based on the results of the book [5], the definitions of the admissible pair of the players’ feedback controls, of the minimizer’s and maximizer’s optimal feedback controls and of the saddle point were introduced in Section 4.3. Using these definitions and the well known solvability conditions of finite-horizon two-person linear-quadratic differential games (see e.g. [1, 2, 3, 5]), the obtaining the feedback saddle-point solution of the considered game was reduced to the obtaining the solution of the terminal-value problem for the Riccati differential equation (see Proposition 4.4.1 in Section 4.4). Due to the absence of the quadratic state cost in the integral part of the game’s cost functional, this Riccati equation becomes the Bernoulli equation (see e.g. [4]). The latter, along with the terminal condition, yields the explicit-form solution. In Lemmas 4.4.2 - 4.4.3 and Corollary 4.4.4, $\varepsilon$-independent necessary and sufficient conditions, providing the feasibility of this solution in the entire time-interval of the game’s duration for all arbitrarily small values $\varepsilon > 0$, are obtained. Theorem 4.4.5, presents $\varepsilon$-independent necessary and sufficient condition, providing the existence of the feedback saddle-point solution to the considered zero-sum linear-quadratic game for all arbitrarily small values $\varepsilon > 0$. Note that Lemmas 4.4.2 - 4.4.3 and Theorem 4.4.5 are extensions of the results of [6, 7, 8, 9, 10] to the cases of the general coefficients for the players’ controls in the dynamics of the game and of the small (cheap) costs of the controls of both players. Corollary 4.4.4 is a new assertion, has not been published before.
Bibliography

[1] Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. SIAM Books, Philadelphia, PA, USA (1999)

[2] Ben-Asher, J., Yaesh, I.: Advances in Missile Guidance Theory, Vol. 180 in Progress in Astronautics and Aeronautics, AIAA, Inc., Reston, VA, USA (1998)

[3] Bryson, A.E., Ho, Y.C.: Applied Optimal Control, Hemisphere, New York, NY, USA (1975)

[4] Elsgolts, L.: Differential Equations and the Calculus of Variations, University Press of the Pacific, Honolulu, HI, USA (2003)

[5] Glizer, V.Y., Kelis, O.: Singular Linear-Quadratic Zero-Sum Differential Games and $H_\infty$ Control Problems: Regularization Approach, Birkhäuser, Switzerland (2022)

[6] Turetsky, V.: Upper bounds of the pursuer control based on a linear-quadratic differential game. J. Optim. Theory Appl. **121**, 163-191 (2004)

[7] Turetsky, V., Glizer, V.Y.: Robust state-feedback controllability of linear systems to a hyperplane in a class of bounded controls. J. Optim. Theory Appl. **123**, 639-667 (2004)

[8] Turetsky, V., Glizer, V.Y.: Continuous feedback control strategy with maximal capture zone in a class of pursuit games. Int. Game Theory Rev. **7**, 1-24 (2005)

[9] Turetsky, V., Glizer, V.Y.: Robust solution of a time-variable interception problem: a cheap control approach. Int. Game Theory Rev. **9**, 637-655 (2007)

[10] Turetsky, V., Shinar, J.: Missile guidance laws based on pursuit-evasion game formulations. Automatica J. IFAC **39**, 607-618 (2003)
# Chapter 5

## Differential Games Based
Robust Capture Pursuit

### 5.1 Introduction

In this chapter, several types of robust capture controls are designed and their capture zones are constructed for the planar engagement between two participants, the pursuer and the evader. The mathematical model of this engagement has been derived in Chapter 2. Obtaining the robust capture controls and the corresponding capture zones is based on the notion of the *robust capture pursuit* (see Section 2.6), and it uses the analysis and the saddle-point solution in feedback controls of the zero-sum differential game with linear dynamics and bounded controls (see Chapter 3). This use means the following. First, by application of the results of Chapter 3, the 6-dimensional planar pursuit-evasion game of the aforementioned engagement and the corresponding scalar pursuit-evasion game are solved, including the obtaining the games' spaces decompositions and optimal feedback controls of the pursuer and the evader. Then, applying these results and the results of Chapter 4, the robust capture controls are designed and their capture zones are constructed for this engagement. Illustrative examples of the pursuit-evasion engagement between two flying vehicles are presented.

The following main notations are applied in the chapter.

1. $E^n$ is the $n$-dimensional real Euclidean space, thus $E^1$ is the set of all real numbers.

2. The superscript $T$ denotes the transposition of a vector $x$, $(x^T)$.

3. By $\text{col}(c_1, c_2, ..., c_n)$, where $c_i$, $(i = 1, ..., n)$ are real numbers, we denote the $n$-dimensional column vector with the upper entry $c_1$, the next entry $c_2$ and so on. The lower entry of this vector is $c_n$.
4. $\Theta_1 \times \Theta_2$, where each of $\Theta_k$, ($k = 1, 2$) is some set either of numbers, or of vectors, denotes the direct product of these sets, i.e., the set of all pairs $(\theta_1, \theta_2)$ with any entries $\theta_1 \in \Theta_1$ and $\theta_2 \in \Theta_2$.

5. By $\partial(\mathcal{D})$, where $\mathcal{D} \in E^n$, we denote the boundary of the set $\mathcal{D}$.

6. By $\mathrm{clo}(\mathcal{D})$, where $\mathcal{D} \in E^n$, we denote the closure of the set $\mathcal{D}$, i.e., $\mathrm{clo}(\mathcal{D}) = \mathcal{D} \cup \partial(\mathcal{D})$.

7. Let $\mathcal{D}_1$ and $\mathcal{D}_2$ be some sets in $E^n$ such that $\mathcal{D}_2 \subseteq \mathcal{D}_1$. Then, $\mathcal{D}_1 \setminus \mathcal{D}_2$ denotes the set of all entries of $\mathcal{D}_1$ which do not belong to $\mathcal{D}_2$.

8. By $\mathrm{int}(\mathcal{D})$, where $\mathcal{D} \in E^n$, we denote the interior of the set $\mathcal{D}$, i.e., $\mathrm{int}(\mathcal{D}) = \mathrm{clo}(\mathcal{D}) \setminus \partial(\mathcal{D})$.

9. Let $g(w)$ be a scalar function of $w \in W \subseteq E^n$. By $\mathrm{sat}(g(w))$ we denote the saturation of the function $g(w)$, i.e.,

$$ \mathrm{sat}(g(w)) = \begin{cases} g(w), & \text{if } |g(w)| \le 1, \\ 1, & \text{if } g(w) > 1, \\ -1, & \text{if } g(w) < -1. \end{cases} $$

## 5.2 Planar Pursuit-Evasion Game with First-Order Dynamics and Bounded Controls of Players

Consider the differential game consisting of the differential system (2.51), the control constraints (2.52) and the cost function

$$ J_x = J_x(u(\cdot), v(\cdot)) = |d^T x(t_f)|, \quad d = \mathrm{col}(1, 0, 0, 0, 0, 0). \quad (5.1) $$

Remember that in (2.51), the state vector $x(t)$ belongs to the Euclidean space $E^6$, i.e., $x(t) = \mathrm{col}(x_1(t), x_2(t), x_3(t), x_4(t), x_5(t), x_6(t))$; the controls $u(t)$ and $v(t)$ of the players (the pursuer and the evader) are scalar; the matrix of the coefficients $A(t, \phi_{PC}, \phi_{E0})$ for the state vector $x(t)$ is given by the equation (2.45); the vectors of the coefficients $b_u(t)$ and $c_v(t)$ for the controls $u(t)$ and $v(t)$, respectively, are given by the equation (2.53); the vector-valued function $f(t, \phi_{PC}, \phi_{E0})$ is given in the equations (2.47)-(2.48); the initial state vector $x_0(\phi_{PC}, \phi_{E0})$ is given in (2.47).

The pursuer disposes of the control $u(\cdot)$, while the evader disposes of the control $v(\cdot)$. The pursuer aims to minimize the cost function (5.1) by a proper choice of the control $u(\cdot)$, while the evader aims to maximize this cost function by a proper choice of the control $v(\cdot)$. Similarly to the differential game in Chapter 3, we assume the following information pattern in the considered pursuit-evasion game. Namely, we assume that both players are aware of all
the games data, presenting in (2.51),(2.52),(5.1), and of the current game's position $(t, x(t))$.

**Remark 27.** Due to the results of Subsection 2.4 (see the equations (2.33)-(2.35) and Remark 5), the cost function (5.1) represents the distance between the pursuer and the evader at the final time-instance $t_f$ of the engagement, i.e., the miss distance. Therefore, the objective of the pursuer is to minimize the miss distance, while the evader tries to maximize this distance. Thus, the game (2.51),(2.52),(5.1), along with the objectives of the pursuer and the evader and the aforementioned information pattern, is a particular case of the zero-sum differential game (3.1),(3.2),(3.3) solved in Chapter 3. In the sequel, we call the game (2.51),(2.52),(5.1) as the Pursuit-Evasion Game with Bounded Controls (PEGBC). Using the results of Sections 3.2, 3.3 and 3.5, we look for the solution (the saddle point and the value in feedback controls) of this game.

First, we should transform the PEGBC to a scalar differential game similar to the game (3.7),(3.2),(3.12). For this purpose, due to the results of Section 3.3 (see the equations (3.4)-(3.6)), we use the solution of the terminal-value problem

$$ \frac{d\Phi(t)}{dt} = -\Phi(t)A(t, \phi_{PC}, \phi_{E0}), \quad t \in [t_0, t_f], \quad \Phi(t_f) = I_6. \quad (5.2) $$

It should be noted the following. Due to the equations (3.8),(3.9) and the form of the vector $d$ in the cost function (5.1), we need to obtain only the first row $\varphi_1(t) = (\varphi_{11}(t), \varphi_{12}(t), \varphi_{13}(t), \varphi_{14}(t), \varphi_{15}(t), \varphi_{16}(t))$ of the solution $\Phi(t)$ to (5.2). Using (2.45), we obtain that the entries of this row satisfy the following terminal-value problem in the interval $[t_0, t_f]$:

$$ \begin{aligned} \frac{d\varphi_{11}(t)}{dt} &= 0, & \varphi_{11}(t_f) &= 1, \\ \frac{d\varphi_{12}(t)}{dt} &= -\varphi_{11}(t), & \varphi_{12}(t_f) &= 0, \\ \frac{d\varphi_{13}(t)}{dt} &= \cos(\phi_{PC})\varphi_{12}(t) + \frac{1}{\tau_P}\varphi_{13}(t) - \frac{1}{V_P(t)}\varphi_{15}(t), & \varphi_{13}(t_f) &= 0, \\ \frac{d\varphi_{14}(t)}{dt} &= -\cos(\phi_{E0})\varphi_{12}(t) + \frac{1}{\tau_E}\varphi_{14}(t) - \frac{1}{V_E(t)}\varphi_{16}(t), & \varphi_{14}(t_f) &= 0, \\ \frac{d\varphi_{15}(t)}{dt} &= \frac{dV_P(t)}{dt}\cos(\phi_{PC})\varphi_{12}(t), & \varphi_{15}(t_f) &= 0, \\ \frac{d\varphi_{16}(t)}{dt} &= -\frac{dV_E(t)}{dt}\cos(\phi_{E0})\varphi_{12}(t), & \varphi_{16}(t_f) &= 0. \end{aligned} \quad (5.3) $$
Solving the subsystem of the system (5.3), consisting of its two first and two last equations, we obtain

$$
\begin{align}
\varphi_{11}(t) &= 1, \quad \varphi_{12}(t) = t_f - t, \quad t \in [t_0, t_f], \nonumber \\
\varphi_{15}(t) &= \cos(\phi_{PC}) \left( V_P(t)(t_f - t) + \int_{t_f}^t V_P(\sigma) d\sigma \right), \quad t \in [t_0, t_f], \nonumber \\
\varphi_{16}(t) &= -\cos(\phi_{E0}) \left( V_E(t)(t_f - t) + \int_{t_f}^t V_E(\sigma) d\sigma \right), \quad t \in [t_0, t_f]. \tag{5.4}
\end{align}
$$

Furthermore, substitution of the expressions for $\varphi_{12}(t)$ and $\varphi_{15}(t)$ into the third equation of the system (5.3), as well as substitution of the expressions for $\varphi_{12}(t)$ and $\varphi_{16}(t)$ into the fourth equation of this system, yield the following two separate terminal-value problems for obtaining $\varphi_{13}(t)$ and $\varphi_{14}(t)$:

$$
\frac{d\varphi_{13}(t)}{dt} = \frac{1}{\tau_P}\varphi_{13}(t) - \frac{1}{V_P(t)}\cos(\phi_{PC}) \int_{t_f}^{t} V_P(\sigma) d\sigma, \quad t \in [t_0, t_f], \quad \varphi_{13}(t_f) = 0,
$$

$$
\frac{d\varphi_{14}(t)}{dt} = \frac{1}{\tau_E}\varphi_{14}(t) + \frac{1}{V_E(t)}\cos(\phi_{E0}) \int_{t_f}^{t} V_E(\sigma)d\sigma, \quad t \in [t_0, t_f], \quad \varphi_{14}(t_f) = 0.
$$

Solving these problems, we obtain

$$
\begin{align}
\varphi_{13}(t) &= -\cos(\phi_{PC}) \int_{t_f}^{t} \left[ \exp\left(\frac{t-\xi}{\tau_P}\right) \frac{1}{V_P(\xi)} \int_{t_f}^{\xi} V_P(\sigma) d\sigma \right] d\xi, \quad t \in [t_0, t_f], \nonumber \\
\varphi_{14}(t) &= \cos(\phi_{E0}) \int_{t_f}^{t} \left[ \exp\left(\frac{t-\xi}{\tau_E}\right) \frac{1}{V_E(\xi)} \int_{t_f}^{\xi} V_E(\sigma) d\sigma \right] d\xi, \quad t \in [t_0, t_f]. \tag{5.5}
\end{align}
$$

Due to the results of Section 3.3, let us consider the function

$$
\begin{align}
z(t, x) &= d^T \left( \Phi(t)x + \int_t^{t_f} \Phi(s) f(s, \phi_{PC}, \phi_{E0}) ds \right) \nonumber \\
&= \varphi_1(t)x + \int_t^{t_f} \varphi_1(s) f(s, \phi_{PC}, \phi_{E0}) ds, \quad (t, x) \in [t_0, t_f] \times E^6, \tag{5.6}
\end{align}
$$

where the vector $d$ appears in the cost function (5.1), the matrix-valued function $\Phi(t)$ is the solution of the terminal-value problem (5.2), $\varphi_1(t)$ is the first row of $\Phi(t)$, the vector-valued function $f(t, \phi_{PC}, \phi_{E0})$ appears in the right-hand side of the differential equation in (2.51).
Furthermore, based on the results of Section 3.3 (see the equations (3.6)-(3.8)) and using the function (5.6), we can transform the PEGBC to the Scalar Pursuit-Evasion Game with Bounded Controls (SPEGBC) consisting of the initial-value problem

$$ \frac{dz(t)}{dt} = h_u(t, \phi_{PC})u(t) + h_v(t, \phi_{E0})v(t), \quad t \in [t_0, t_f], \quad z(t_0) = z_0, \quad (5.7) $$

the control constraints (2.52) and the cost function

$$ J_z = J_z(u(\cdot), v(\cdot)) = |z(t_f)|, \quad (5.8) $$

where

$$ \begin{align} h_u(t, \phi_{PC}) &= \varphi_1(t)b_u(t) = \frac{a_{P,c}^{\max}(t)}{\tau_P}\varphi_{13}(t) \nonumber \\ &= -\frac{a_{P,c}^{\max}(t)}{\tau_P}\cos(\phi_{PC}) \int_{t_f}^{t} \left[ \exp\left(\frac{t-\xi}{\tau_P}\right) \frac{1}{V_P(\xi)} \int_{t_f}^{\xi} V_P(\sigma) d\sigma \right] d\xi, \quad t \in [t_0, t_f], \nonumber \\ h_v(t, \phi_{E0}) &= \varphi_1(t)c_v(t) = \frac{a_{E,c}^{\max}(t)}{\tau_E}\varphi_{14}(t) \nonumber \\ &= \frac{a_{E,c}^{\max}(t)}{\tau_E}\cos(\phi_{E0}) \int_{t_f}^{t} \left[ \exp\left(\frac{t-\xi}{\tau_E}\right) \frac{1}{V_E(\xi)} \int_{t_f}^{\xi} V_E(\sigma) d\sigma \right] d\xi, \quad t \in [t_0, t_f]. \tag{5.9} \end{align} $$

**Remark 28.** In what follows, we assume that $\cos(\phi_{PC}) \neq 0$ and $\cos(\phi_{E0}) \neq 0$. This assumption, along with the equation (5.9), means that $h_u(t, \phi_{PC}) \neq 0$ and $h_v(t, \phi_{E0}) \neq 0$ for all $t \in [t_0, t_f]$.

Using the equation (3.9) and the equations (2.40), (2.47), (2.48), (5.4), (5.5), we obtain after a routine algebra the following expression for $z_0$:

$$ \begin{align} z_0 = z_0(\phi_{PC}, \phi_{E0}) &= \varphi_1(t_0)x_0(\phi_{PC}, \phi_{E0}) + \int_{t_0}^{t_f} \varphi_1(s)f(s, \phi_{PC}, \phi_{E0})ds \nonumber \\ &= -(\sin(\phi_{PC}) + \cos(\phi_{PC})) \int_{t_0}^{t_f} V_P(t)dt + \sin(\phi_{E0}) \int_{t_0}^{t_f} V_E(t)dt \nonumber \\ &\quad + \varphi_{13}(t_0)x_{30} + \varphi_{14}(t_0)x_{40}. \tag{5.10} \end{align} $$

Due to the results of Section 3.5, the derivation of the feedback solution (the saddle point and the value in feedback controls) to the PEGBC is based on the obtaining such a solution to the SPEGBC. The feedback solution of the latter is based on its game space $S_{[t_0, t_f]}$ decomposition into the regular $\mathcal{R}_r$