# Convexity of the Set of Fixed Points Generated by Some Control Systems

Authors: Vadim Azhmyakov
Journal: Journal of Applied Mathematics
 (2009)

---

## Abstract

We deal with an application of the fixed point theorem for nonexpansive mappings to a class of control systems. We study closed-loop and open-loop controllable dynamical systems governed by ordinary differential equations (ODEs) and establish convexity of the set of trajectories. Solutions to the above ODEs are considered as fixed points of the associated system-operator. If convexity of the set of trajectories is established, this can be used to estimate and approximate the reachable set of dynamical systems under consideration. The estimations/approximations of the above type are important in various engineering applications as, for example, the verification of safety properties.


---

## Body

## 1. Introduction and Motivation

This paper addresses an application of the well-known fixed point theorem fornonexpansive mappings in Hilbert spaces (see, e.g., [1, 2]) to a class of dynamical systems. The main aim of our contribution is to characterize the set of solutions (trajectories) of the dynamical systems under consideration and to establish the convexity property of this set. First, let us consider a nonlinear closed-loop system given by(1.1)ẋ(t)=f(x(t),u(x(t))),a.e.on[0,tf],x(0)=x0,
where f:ℝn×ℝm→ℝn is Lipschitz continuous in both components. Let U be a compact and convex subset of ℝm and consider measurable feedback control functions u:ℝn→U.Assume that for every such feedback control functionu(·), there exists a solution xu(·) of (1.1), for uniqueness conditions and for some constructive existence conditions for systems (1.1) we refer to [3, 4]. Given an initial value x0∈ℝn and such a feedback control function u(·), the solution of (1.1) is an absolutely continuous function. Let 𝕎n1,1(0,tf) be the Sobolev space of all absolutely continuous ℝn-valued functions y(·) such that the derivative ẏ(·) exists almost everywhere and belongs to the Lebesgue space 𝕃n1(0,tf) of all measurable functions y:[0,tf]→ℝn with(1.2)∥y(·)∥𝕃n1(0,tf)=∫0tf∥y(t)∥dt<∞.
Recall that 𝕎n1,1(0,tf) equipped with the norm ∥·∥𝕎n1,1(0,tf), defined by(1.3)∥w(·)∥𝕎n1,1(0,tf):=∥w(·)∥𝕃n1(0,tf)+∥ẇ(·)∥𝕃n1(0,tf)
for w(·)∈𝕎n1,1(0,tf) is a Banach space. Moreover, 𝕎n1,1(0,tf) is the completion of the space of all continuously differentiable ℝn-valued functions ℂn1(0,tf) with respect to the norm ∥·∥𝕃n1(0,tf) (see, e.g., [5, 6]). The initial value problem (1.1) can also be considered as a problem in the space 𝕎n1,1(0,tf).Thereachable set𝒦̃(t,x0) at time t is the set of states of (1.1) which can be reached at time t, when starting at x0 at time t=0, using all possible controls (see, e.g., [7]). That is, 𝒦̃(t,x0):={xu(t)∣u(·)∈𝕃m1(ℝn),u(x)∈U}, where 𝕃m1(ℝn) denotes the Lebesgue space of all measurable functions u:ℝn→ℝm. We now formulate our standing hypothesis.(H1)
The reachable sets𝒦̃(t,x0) are contained in an open bounded superset Ω⊂ℝn for t∈[0,tf].This is for example the case ifΩ is a positively invariant set for the system (1.1). Recall that a set in the state space is said to be positively invariant for a given dynamical system if any trajectory initiated in this set remains inside the set at all future time instants. Besides, for a dynamical system (1.1) with bounded right-hand side, the reachable set 𝒦̃(t,x0) is trivially bounded.Givenl∈ℝ+, we introduce the space 𝒰lf of admissible feedback control functions u:Ω→U as the space of all Lipschitz functions with Lipschitz constants lu≤l on Ω. Under the above-mentioned boundedness assumption for the reachable set, we can now consider the reachable set 𝒦(t,x0) of (1.1) with respect to 𝒰lf given by 𝒦(t,x0):={xu(t)∣u(·)∈𝒰lf}. For the given control system (1.1), we address the task of formulating sufficient conditions for the convexity of the reachable set 𝒦(t,x0) for every t∈[0,tf]. Note that the convexity of the reachable set or the existence of convex approximations for the 1 reachable set bear a close relation to a computational method for determining positively invariant sets, namely, the ellipsoidal technique (see [8, 9]). In this paper, we also derive conditions for the set of trajectories of (1.1) on [0,tf] , that is,(1.4)𝒯(x0):={xu(·):[0,tf]→Ω∣u(·)∈𝒰lf},
to be convex. The main convexity result for system (1.1) is based on an abstract fixed point theorem for nonexpansive mappings in Hilbert spaces (see, e.g., [1, 2]). For some abstract convexity results for nonlinear mappings we refer to [10], for some applications to optimization and optimal control to [10, 11]. For an analysis of reachable sets of dynamical systems in an abstract or hybrid setting, see also [8].While the main topic of our paper is the estimation of reachable sets for closed-loop systems of type (1.1), we also consider open-loop control systems:(1.5)ẋ(t)=g(x(t),u(t)),a.e.on[0,tf],x(0)=x0,
where g is a Lipschitz continuous function (in both components) and where u(t) belongs to U for t∈[0,tf]. Let 𝒰:={u(·)∈𝕃m2(0,tf)∣u(t)∈U} be the space of admissible control signals for system (1.5). Here, 𝕃m2(0,tf) denotes the Lebesgue space of all square-integrable functions u:[0,tf]→ℝm with the corresponding norm. It is assumed that for every admissible time-dependent control u(·)∈𝒰 system (1.5) has a unique solution xu(·)∈𝕎n1,1(0,tf). As for the closed-loop system (1.1), we will obtain estimates for the reachable sets of (1.5) provided the right-hand sides are bounded.The paper is organized as follows. In Section2, we provide the necessary definitions and mathematical results. Section 3 contains the convexity result for the sets of trajectories and for reachable sets of the closed-loop control system (1.1). Section 4 discusses overapproximation of reachable sets for some classes of closed-loop and open-loop control systems with bounded right-hand sides. We also use some techniques from optimal control theory to obtain general approximations of convex reachable sets under consideration. In Section 5, we discuss a possible application of our convexity criterion to optimal control problems with constraints. Section 6 summarizes the paper.


## 2. Preliminary Results

We first provide some relevant definitions and facts. Let𝒳 and 𝒴 be two Banach spaces with 𝒳⊂𝒴. We say that the space 𝒳 is compactly embedded in 𝒴 and write 𝒳↪c𝒴, if ∥v∥𝒴≤c∥v∥𝒳 for all v∈𝒳 and each bounded sequence in 𝒳 has a convergent subsequence in 𝒴. We recall a special case of the Sobolev Embedding Theorem (cf. [5, 6]) in Proposition 2.1 and list some interpolation properties of Lebesgue spaces (cf. [6]) in Proposition 2.2.Proposition 2.1.
It holds that𝕎n1,1(0,tf)↪c𝕃n2(0,tf).Proposition 2.2.
If1≤p≤q≤∞, then 𝕃nq(Ω)⊂𝕃np(Ω) and
(2.1)∥v∥𝕃np(Ω)≤meas(Ω)1/p-1/q∥v∥𝕃nq(Ω),∀v∈𝕃nq(Ω),
where 1/∞ is understood to be 0. In particular one has 𝕃n2(0,tf)⊂𝕃n1(0,tf) and, for all functions y(·)∈𝕃n2(0,tf),
(2.2)∥y(·)∥𝕃n1(0,tf)≤tf∥y(·)∥𝕃n2(0,tf).We now consider the concept of anonexpansive mapping in Hilbert spaces and present a fundamental fixed point theorem for such mappings in Proposition 2.3 (cf. [1, 2, 12]). Let C be a subset of a Hilbert space ℋ with norm ∥·∥ℋ. A mapping T:C→ℋ is said to be nonexpansive if(2.3)∥T(h1)-T(h2)∥ℋ≤∥h1-h2∥ℋ
holds true for all h1,h2∈C.Proposition 2.3.
LetC be a nonempty, closed, and convex subset of a  Hilbert space ℋ and let T be a nonexpansive mapping of C into itself. Then the set F(T) of fixed points of T is nonempty, closed, and convex.Now, we return to the given control system (1.1) for which we introduce the system operator(2.4)P:𝒰lf×𝕎n1,1(0,tf)→𝒰lf×𝕎n1,1(0,tf)
defined by the following formula:(2.5)P(u(·),x(·))(t):=(u(x(t))x0+∫0tf(x(τ),u(x(τ)))dτ).
Using the Sobolev Embedding Theorem (as stated in Proposition 2.1), we extend P to the operator:(2.6)P̃:𝒰lf×𝕃n2(0,tf)→𝒰lf×𝕃n2(0,tf)
with P̃ still being given by the right-hand side of formula (2.5).We now consider the set of admissible feedback controls𝒰lf, which is contained in ℂm(Ω), as a subset of the space 𝕃m2(Ω). The following result specifies properties of the set 𝒰lf.Lemma 2.4.
The set𝒰lf is a closed convex subset of the  Hilbert space 𝕃m2(Ω).Proof.
The set𝒜U of all continuous functions u(·) with range in U and the set 𝒜l of all Lipschitz continuous functions with Lipschitz constants lu≤l are both convex subsets of ℂm(Ω). Therefore, the intersection 𝒰lf:=𝒜U⋂𝒜l is also convex.
Because ofℂm(Ω)⊂𝕃m2(Ω), the set 𝒜l is a closed set in the sense of the supnorm. Hence 𝒜l is also closed in the sense of the norm of the space 𝕃m∞(Ω). Using Proposition 2.2, we deduce that this set is a closed subset of the space 𝕃m2(Ω). Now let us consider a sequence {uk(·)} of functions from 𝒰lf such that
(2.7)limk→∞∥uk(·)-û(·)∥𝕃m2(Ω)=0,
where û(·)∈𝕃m2(Ω). Then there exists a subsequence {us(·)} of {uk(·)} satisfying lims→∞∥us(x)-û(x)∥ℝm=0 for almost all x∈Ω (see [13]). The assumption û(·)∉𝒰lf implies the existence of a set ℐ⊆Ω of positive measure with û(x)∉U for all x∈ℐ. On the other hand, we have for a fixed x∈ℐ:(2.8)lims→∞∥us(x)-û(x)∥ℝm=0.
Since U is a compact set, û(x) belongs to U for the considered x∈ℐ, contradicting our assumption. Thus, we obtain û(·)∈𝒰lf showing that the set 𝒰lf is closed. The proof is finished.By Lemma2.4, the set 𝒰lf×𝕃n2(0,tf) is a closed convex subset of the Hilbert space ℋ:=𝕃m2(Ω)×𝕃n2(0,tf). Note that the scalar product and the norm in this space can be introduced as follows:(2.9)〈(u1(·),x1(·)),(u2(·),x2(·))〉ℋ:=〈u1(·),u2(·)〉𝕃m2(Ω)+〈x1(·),x2(·)〉𝕃n2(0,tf),∥(u(·),x(·))∥ℋ:=∥u(·)∥𝕃m2(Ω)2+∥x(·)∥𝕃n2(0,tf)2.
Using the triangle inequality and the Schwarz inequality for the Hilbert spaces 𝕃m2(Ω) and 𝕃n2(0,tf), one can verify the standard properties of the introduced scalar product and norm.


## 3. Convexity Criteria for Reachable Sets of Closed-Loop Systems

We next state and prove our main result concerning the operatorP̃ from (2.6) and (2.5) under our standing hypothesis (H1). It will be the basis for formulating sufficient conditions for the convexity of the set 𝒯(x0) of trajectories and of the reachable set 𝒦(t,x0).Theorem 3.1.
Assume thatf satisfies the  Lipschitz condition:
(3.1)∥f(x1,u1)-f(x2,u2)∥≤l1∥x1-x2∥+l2∥u1-u2∥∀x1,x2∈Ω,∀u1,u2∈U,
where tfl2+(l1+l2l)2≤1. Then the operator P̃ of the corresponding system is nonexpansive, and the set F(P̃) of fixed points of P̃ is nonempty closed and convex.Proof.
We claim thatP̃ is a nonexpansive mapping. To see this, consider
(3.2)∥P̃(u1(·),y1(·))-P̃(u2(·),y2(·))∥ℋ2=∥u1(y1(·))-u2(y2(·))∥𝕃m2(Ω)2+∥∫0·δ(τ)dτ∥𝕃n2(0,tf)2
with δ(τ):=f(y1(τ),u1(y1(τ)))-f(y2(τ),u2(y2(τ))) where control functions u1(·),u2(·) are from 𝕃m2(Ω), and y1(·),y2(·) are elements of 𝕃n2(0,tf). Consider the second term of the right-hand side of (3.2). We obtain
(3.3)∥∫0·δ(τ)dτ∥𝕃n2(0,tf)2=∫0tf∥∫0tδ(τ)dτ∥ℝn2dt≤∫0tf(∫0t∥δ(τ)∥ℝndτ)2dt≤∫0tf(∫0tf∥δ(τ)∥ℝndτ)2dt=tf(∫0tf∥δ(τ)∥ℝndτ)2.
From (3.3) and from the Lipschitz condition for the function f it follows that
(3.4)∥∫0·δ(τ)dτ∥𝕃n2(0,tf)2≤tf(l1+l2l)2(∫0tf∥y1(τ)-y2(τ)∥ℝndτ)2=tf(l1+l2l)2∥y1(·)-y2(·)∥𝕃n1(0,tf)2.
By Proposition 2.2, we have the following estimation:
(3.5)∥y1(·)-y2(·)∥𝕃n1(0,tf)2≤tf∥y1(·)-y2(·)∥𝕃n2(0,tf)2.
This fact and inequality (3.6) both imply
(3.6)∥∫0·δ(τ)dτ∥𝕃n2(0,tf)2≤tf2(l1+l2l)2∥y1(·)-y2(·)∥𝕃n2(0,tf)2.
Since tfl2+(l1+l2l)2≤1, we finally deduce from Proposition 2.2, and formulas (3.2)–(3.6):
(3.7)∥P̃(u1(·),y1(·))-P̃(u2(·),y2(·))∥ℋ≤[∥u1(·)-u2(·)∥𝕃m2(Ω)2+∥y1(·)-y2(·)∥𝕃n2(0,tf)2]1/2=∥(u1(·),y1(·))-(u2(·),y2(·))∥ℋ.
Thus, the introduced operator P̃ is a nonexpansive operator in the Hilbert space ℋ. By Lemma 2.4, 𝒰lf×𝕃n2(0,tf) is a nonempty closed and convex subset of ℋ. Finally, from Proposition 2.3, it follows that F(P̃) is a nonempty closed and convex set. The proof is completed.Note that Theorem3.1 establishes the convexity property of the set of fixed points for the extended system operator P̃ on 𝒰lf×𝕃n2(0,tf) (see (2.6) and (2.5)). As a consequence of this result, we also can formulate the corresponding theorem for the operator P on 𝒰lf×𝕎n1,1(0,tf) (see (2.4) and (2.5)).Theorem 3.2.
Under the assumption of  Theorem3.1, the set F(P) is convex. Moreover, the set of trajectories 𝒯(x0)={xu(·)∣u(·)∈𝒰lf} for the initial value problem (1.1) on [0,tf] is also convex.Proof.
Sincef is a Lipschitz continuous function, the initial value problem (1.1) has a solution and the set F(P) of fixed points of the operator P is nonempty. By Proposition 2.1, we have 𝕎n1,1(0,tf)⊂𝕃n2(0,tf). Hence,
(3.8)F(P)=𝕎n1,1(0,tf)⋂F(P̃).
Since 𝕎n1,1(0,tf) is a convex subset of 𝕃n2(0,tf), the set F(P) is also convex.
In fact,F(P) is a subset of the product-space 𝒰lf×𝕎n1,1(0,tf) and the structure of the operator P defines the structure of the set F(P)=𝒰lf×𝒯(x0). Since F(P) and 𝒰lf are convex, we obtain the convexity of the set 𝒯(x0).We now deal with the reachable set𝒦(t,x0) for the closed-loop system (1.1). Our next result is an immediate consequence of the convexity criterion just presented in Theorem 3.2.Theorem 3.3.
Under the assumption of  Theorem3.1, the reachable set 𝒦(t,x0) for the initial value problem  (1.1) is convex for every t∈[0,tf].Proof.
Theorem3.2 states the convexity of the set 𝒯(x0). It means that for
(3.9)xu1(·),xu2(·)∈𝒯(x0)
with u1(·),u2(·)∈𝒰lf and for fixed λ∈(0,1), there exists an admissible control u3(·)∈𝒰lf such that xu3(·)=λxu1(·)+(1-λ)xu2(·)∈𝒯(x0). On the other hand, at a time-instant t∈[0,tf], we have xu1(t),xu2(t)∈𝒦(t,x0). Hence,
(3.10)xu3(t)=λxu1(t)+(1-λ)xu2(t)∈𝒦(t,x0).
This shows that the reachable set 𝒦(t,x0) is convex for every t∈[0,tf].Remark 3.4.
Note that under the conditions of  Theorem3.1, the reachable set K(t,x0) from  Theorem 3.3 is closed for every t∈[0,tf]. This fact also follows from  Proposition 2.3 and Theorem 3.1. Moreover, the set of trajectories 𝒯(x0) of (1.1) is a closed subset of the space ℂ(0,tf) [14].We now present two illustrative examples of control systems (1.1) satisfying the Lipschitz conditions from the main Theorem 3.1.Example 3.5.
Let us consider anf with every component fj being a convex function fi:∈U×ℝn→ℝ. In case mj≤fj(x,u)≤Mj,j=1,…,n holds for all (u,x) in the ball ℬ(0,2Δ) of radius 2Δ around 0, every fj is  Lipschitzian on ℬ(0,Δ) with
(3.11)|fj(x1,u1)-fj(x2,u2)|≤Mj-mjΔ
for all (u1,x1),(u2,x2)∈ℬ(0,Δ)  (cf. [15]) implying that
(3.12)ν:=maxj=1,…,n(Mj-mj)Δ|f(x1,u1)-f(x2,u2)|≤ν(∥u1-u2∥+∥x1-x2∥).
Therefore, the condition tfl2+(l1+l2l)2≤1 from Theorem 3.1 can be written as follows:
(3.13)tfl2+ν2(1+l)2≤1,
where ℬ(0,Δ) is taken for Ω. Note that Mj and mj may depend on Δ too.Example 3.6.
Consider the following two-dimensional control system:(3.14)ẋ1(t)=x2(t),ẋ2(t)=-sinx1(t)+u(x(t)),x1(0)=x2(0)=0,
where x:=(x1,x2)T and t∈[0,0.5]. It is easy to see that l1=l2=1. The condition tfl2+(l1+l2l)2≤1 from Theorem 3.1 implies 2l2+2l-3≤0 and
(3.15)0<l≤7-12.
We see that under this condition the reachable set 𝒦(t,0ℝ2) of the presented system is convex for every t∈[0,0.5].


## 4. Overapproximations of Reachable Sets

In this section we will discuss a special class of closed-loop and open-loop systems (1.1) and (1.5), namely, systems which satisfy the following condition:(4.1)f(x,u),g(x,u)∈K,∀(x,u)∈ℝn×U,
where K is a closed convex subset of ℝn containing 0. The right-hand sides f and g of (1.1) and (1.5) are assumed to be continuous in both components. Let us first formulate the following auxiliary abstract result.Lemma 4.1.
Let𝒳 be a separable Banach space and {𝒮,Σ,μ} be a measurable space with a probability measure μ. Let C⊂𝒳 be closed and convex. If the mapping q:𝒮→C is a μ-measurable function, then
(4.2)∫q(τ)μ(dτ)∈C.Proof.
Assumea:=∫q(τ)μ(dτ)∉C and let
(4.3)ℬ(a,R):={ξ∈𝒳∣∥ξ-a∥𝒳<R}
be the ball around a with radius R. Evidently, there is a radius R such that we have C∩ℬ(a,R)=∅. Using a Separating Theorem from convex analysis (cf. [13, 16]), we obtain a nontrivial ℒ∈𝒳* with ℒξ≤ℒψ for all ξ∈ℬ(a,R),ψ∈C. By 𝒳*, we have denoted the (topological) dual space to 𝒳. Thereby, we have the inequality
(4.4)sup∥ξ∥𝒳≤1(ℒ(a+Rξ))=ℒa+R|ℒ|≤ℒ(q(τ)),τ∈𝒮,
and—by integration with respect to μ—we have also the corresponding inequality
(4.5)ℒa+R|ℒ|≤∫ℒ(q(τ))μ(dτ).
Because of
(4.6)∫ℒ(q(τ))μ(dτ)=ℒ∫q(τ)μ(dτ)=ℒa,
(4.5) leads to ℒa+R|ℒ|≤ℒ contradicting the fact that ℒ is nontrivial. Therefore,
(4.7)a=∫q(τ)μ(dτ)
belongs to C.Returning to control systems of type (1.1) or (1.5) satisfying condition (4.1), we introduce the following Lebesgue probability measure μ=τ/t on the interval [0,t]. Then, we apply Lemma 4.1 to our control systems and compute the state of system (1.1) (or the state of system (1.5)) at time t∈[0,tf] as(4.8)x(t)=x0+∫0tf(x(τ),u(x(τ)))ds=x0+t∫0tf(x(τ),u(x(τ)))μ(dτ)∈x0+tK.
For the open-loop system (1.5), we have the analogous result(4.9)x(t)=x0+∫0tg(x(τ),u(τ))dτ∈x0+tK.
This means that the reachable sets of systems (1.1) and (1.5) with initial value x0 belong to the closed convex set Ω:=x0+tfK. Since x0 belongs to Ω, we have the set Ω as a positively invariant set for the corresponding control system. In particular, this set Ω contains the reachable set of the considered dynamical system.We now describe an abstract approach for estimating convex reachable sets. Our main idea is as follows: under the assumption of convexity for the reachable set of a given closed-loop control system, we formulate an auxiliary feedback optimal control problem with a linear cost functional. A solution of this problem makes it possible to construct a tangent hyperplane (supporting hyperplane) to the reachable set under consideration. Considering a sufficiently “rich” set of these hyperplanes and their intersections, one can approximate the reachable set with arbitrary accuracy.Let𝒦(t̂,x0),t̂∈[0,tf] be a bounded closed and convex reachable set for (1.1). Following the idea sketched above, let us consider the auxiliary optimal feedback control problem(4.10)minimize〈c,x(t̂)〉subjectto(1.1),
where c is a fixed vector from ℝn, u(·)∈𝒰lf, and 〈·,·〉 denotes the scalar product in ℝn. Note that (4.10) is formulated as a minimizing problem with respect to aterminal linear cost functional. Linearity of this cost functional and the above properties of the reachable set 𝒦(t̂,x0) to the time t̂∈[0,tf] imply the existence of an optimal solution (u*(·),x*(·)) for (4.10) (see [16]), where u*(·)∈𝒰lf. Note that (4.10) can be reformulated as the following convex linear problem in ℝn:(4.11)minimize〈c,z〉subjecttoz∈𝒦(t̂,x0).
Therefore, x*(t̂)∈∂𝒦(t̂,x0), where ∂𝒦(t̂,x0) is the boundary (the set of all extremal points) of the convex set 𝒦(t̂,x0) (see, e.g., [15, 16]).We now recall the Rademacher Theorem (see, e.g., [17]), which states that a function which is Lipschitz on an open subset of ℝn is differentiable almost everywhere (in the sense of a Lebesgue measure) on that subset. Since Ω⊂ℝn is an open set, the function u*(·) is differentiable almost everywhere on Ω. The set of points at which the optimal control u*(·) fails to be differentiable is denoted Ω0. Evidently meas(Ω0)=0. Let Ω*:=Ω∖Ω0. We now formulate our next hypotheses.(H2)
The right-hand sidef(·,·) of (1.1) is a differentiable function (in both components) such that the partial derivatives fx(·,·),fu(·,·) are integrable functions on Ω×U.(H3)
It hold thatx*(t)∈Ω* for all t∈[0,t̂] and the derivative (u*)′ of u*(·) is locally integrable on Ω*.Clearly, the optimal control problem (4.10) is equivalent to the following minimization problem:(4.12)minimize(c,[x0+∫0t̂f(x(t),u(x(t)))dt])
for u(·)∈𝒰lf. Since the right-hand side of the differential equation from (1.1) is supposed to be differentiable in both components, the cost functional in (4.12) is Fréchet differentiable (see, e.g., [18]). Assume (H2)-(H3) and formulate the necessary optimality condition for (u*(·),x*(·)) to be an optimal solution of (4.12):(4.13)D(c,x*(t̂))|y(·)=(c,∫0t̂[fx(x*(t),u*(x*(t)))+fu(x*(t),u*(x*(t)))(u*)′(x*(t))]y(t)dt)=0∀y(·)∈𝕎n1,1(0,t̂),
where D(c,x*(t̂))∣y(·) is the Fréchet derivative of the cost functional from (4.12) at x*(·). Note that under the above assumptions (H2)-(H3), the integrand in (4.13) is a locally integrable function. Moreover, (4.13) holds for all functions y(·) from the space 𝕎n1,1(0,t̂). Therefore, the expression in (4.13) is also equal to zero for all functions y(·) from ℂ0∞(0,t̂), where(4.14)ℂ0∞(0,t̂):={ξ(·)∈ℂ∞(0,t̂)|supp{ξ(·)}isacompactsubsetof(0,t̂)}
and supp{ξ(·)}:={t∈(0,t̂)∣ξ(t)≠0}. By the Generalized Variational Lemma (see e.g., [6, Lemma 7.1.2]), we deduce from (4.13) that(4.15)[fx(x*(t),u*(x*(t)))+fu(x*(t),u*(x*(t)))(u*)′(x*(t))]c=0a.e.on(0,t̂).
The nonlinear equation (4.15) with a given vector c∈ℝn provides a basis for solving optimal control problem (4.10).Consider now an interior pointζ of the convex hull conv{Ω̅} and a family {zs} of elements zs∈∂conv{Ω̅}, s=1,…,S, for a sufficiently large number S∈ℕ such that {zs} approximate the boundary ∂conv{Ω̅} of convΩ̅. By Ω,̅ we denote here the closure of Ω. If we solve the family of problems (4.10) with cs:=zs-ζ, s=1,…,S, we obtain the corresponding optimal state vectors xs*(t̂). As established above, xs*(t̂)∈∂𝒦(t̂,x0). Therefore, we can write the equation of the approximating tangent hyperplane Ts to the reachable set at xs*(t̂) in the form(4.16)Ts:={x∈ℝn|cs,x-xs*(t̂)=0}.
If we examine all hyperplanes Ts,s=1,…,S and their intersections, we can constract a convex polyhedron which contains the reachable set 𝒦(t̂,x0). In principle, the proposed idea guarantees an overapproximation for a convex reachable set of a control system (1.1). However, it is necessary to stress that complexity of this approximation grows rapidly if we increase the number S∈ℕ. Finally, note that the same idea can also be used for the overapproximations of reachable sets for open-loop control systems. We refer to [10] for details.


## 5. An Application to Optimal Control Problems with Constraints

Let us now apply the main convexity result of Theorem3.2 to the following constrained optimal feedback control problem:(5.1)minimizeJ(x(·))subjectto(1.1),x(·)∈A,
where J is a bounded, convex, and lower semicontinuous objective functional (see [16]) and A a nonempty, bounded, closed, and convex subset of 𝕃n2(0,tf). The given control system (1.1) is supposed to satisfy the conditions of Theorem 3.1. We consider the optimal control problem (5.1) on the Hilbert space 𝕃n2(0,tf) with feedback controls from 𝒰lf. Note that the class of feedback optimal control problems of type (5.1) is quite general [3]. For example, the objective functional J could be given by(5.2)J(x(·))=∫0tfx2(t)dt,
and the abstract restriction x(·)∈A could arise from a system of the following inequalities hs(x(t))≤0for all  t∈[0,tf] with convex functions hs:ℝn→ℝ, where s=1,…,S. It is clear that an optimal control problem does not always have a solution. The question of existence of an optimal feedback solution is generally a delicate one (cf. [3]).Let𝒯(x0)⋂A be nonempty. Evidently, problem (5.1) can be rewritten as an optimization problem over the set 𝒞:=𝒯(x0)⋂A of admissible trajectories as follows:(5.3)minimizeJ(x(·))subjecttox(·)∈𝒞.
Here, the state (1.1) is included into the constraints x(·)∈𝒞. We claim that (5.3) is a standard convex optimization problem on a bounded closed convex subset of a Hilbert space (see, e.g., [16]). To see this, we note that the set of solutions:(5.4)𝒯(x0)={xu(·)|u(·)∈𝒰lf}
is a closed subset of the space ℂ(0,tf) (see [14]). Therefore, this set is also closed in the sense of the norm of 𝕃n2(0,tf). Moreover, 𝒯(x0) is convex by Theorem 3.2. The intersection 𝒞 of the two closed convex sets A and 𝒯(x0) is again closed and convex set in the Hilbert space 𝕃n2(0,tf). Since A is bounded, the set 𝒞 is also bounded. Using the well-known existence results from convex optimization theory (cf. [16]), one can establish the following existence result for optimal control problem (5.1).Theorem 5.1.
Under the conditions of  Theorem3.2, the optimal control problem (5.3) with a bounded convex and lower semicontinuous objective functional J and bounded closed convex set A has at least one optimal solution
(5.5)(uopt(·),xopt(·))∈𝒰lf×𝕃n2(0,tf)
provided that  𝒞=𝒯(x0)⋂A is nonempty.Since𝕎n1,1(0,tf)⊂𝕃n2(0,tf) and 𝕎n1,1(0,tf) is convex, the following intersection 𝕎n1,1(0,tf)⋂𝒞 is also a bounded closed and convex subset of 𝕃n2(0,tf). Therefore, we also obtain the corresponding existence result for problem (5.1) considered on the space 𝕎n1,1(0,tf). Let {xk(·)}⊂𝕎n1,1(0,tf) be a minimizing sequence for (5.3) defined on the space 𝕎n1,1(0,tf), that is,(5.6)limk→∞J(xk(·))=min𝕎n1,1(0,tf)⋂𝒞J(x(·)).
It is well known that a minimizing sequence does not always converge to an optimal solution. The question of creating a convergent minimizing sequence is a central question in the numerical analysis of optimization algorithms (see, e.g., [11, 19]). By Proposition 2.1, each bounded sequence in 𝕎n1,1(0,tf) has a convergent subsequence in 𝕃n2(0,tf). Since {xk(·)} is bounded, we have(5.7)liml→∞∥xkl(·)-Argmin𝕎n1,1(0,tf)⋂𝒞J(x(·))∥𝕃n2(0,tf)=0
for a subsequence {xkl(·)} of {xk(·)}. Thus, by Theorem 5.1, we deduce the existence of an 𝕃n2-convergent minimizing sequence {xkl(·)} for the optimal control problem (5.3).


## 6. Conclusion

In this paper, we proposed a new convexity criterion for reachable sets for a class of closed-loop control systems. This sufficient condition is based on a general convexity result for solution sets of the corresponding nonlinear dynamical systems. Convexity of the set of trajectories makes it also possible to study some constrained feedback optimal control problems. For some families of closed-loop and open-loop control systems, we construct an overestimation of the examined reachable set, that is, we provide sets that contain the reachable sets of the dynamical system under consideration.



---

*Source: 291849-2009-10-21.xml*