# Indefinite LQ Optimal Control with Terminal State Constraint for Discrete-Time Uncertain Systems

Authors: Yuefen Chen; Minghai Yang
Journal: Journal of Control Science and Engineering
 (2016)

---

## Abstract

Uncertainty theory is a branch of mathematics for modeling human uncertainty based on the normality, duality, subadditivity, and product axioms. This paper studies a discrete-time LQ optimal control with terminal state constraint, whereas the weighting matrices in the cost function are indefinite and the system states are disturbed by uncertain noises. We first transform the uncertain LQ problem into an equivalent deterministic LQ problem. Then, the main result given in this paper is the necessary condition for the constrained indefinite LQ optimal control problem by means of the Lagrangian multiplier method. Moreover, in order to guarantee the well-posedness of the indefinite LQ problem and the existence of an optimal control, a sufficient condition is presented in the paper. Finally, a numerical example is presented at the end of the paper.


---

## Body

## 1. Introduction

The linear quadratic (LQ) optimal control problem has been pioneered by Kalman [1] for deterministic systems, which is extended to stochastic systems by Wonham [2], and has rapid development in both theory and application [3]. Usually, it is an assumption that the control weighting matrix in the cost is strictly definite. For stochastic LQ optimal control, it is first revealed in [4] that even if the state and control weighting matrices are indefinite the corresponding problem may be still well-posed, which evoked a series of subsequent researches in continuous time [5] and in discrete-time [6]. In fact, some constraints are of considerable importance in many physical systems; the system state and control input are always subject to various constraints, so the constrained stochastic LQ issue has a concrete application background. For that reason, some researchers discussed stochastic LQ optimal problems with indefinite control weights and constraints [7, 8].As is well known, these stochastic optimal control problems have been well studied by probability theory which is based on a large number of sample sizes. Sometimes, no samples are available to estimate the probability distribution. For such situation, we have to invite some domain experts to evaluate the belief degree that each event will occur. In order to rationally deal with belief degrees, uncertainty theory was established by Liu [9] in 2007 and refined by Liu [10] in 2010. Nowadays, uncertainty theory has become a new branch of mathematics for modeling indeterminate phenomena, which has been well developed and applied in a wide variety of real problems: option pricing problem [11], facility location problem [12], inventory problem [13], assignment problem [14], and production control problem [15].Based on the uncertainty theory, Zhu [16] proposed an uncertain optimal control model in 2010 and gave an equation of optimality as a counterpart of Hamilton-Jacobi-Bellman equation. After that, some uncertain optimal control problems have been solved. As such, Sheng and Zhu [17] investigated an optimistic value model of uncertain optimal control problem; Yan and Zhu [18] established an uncertain optimal control model for switched systems. Inspired by the preceding work, we will tackle an indefinite LQ optimal control with terminal state constraint for discrete-time uncertain systems, which is a constrained uncertain optimal control problem. The rest of the paper is organized as follows. Section 2 collects some preliminary results. In Section 3, an indefinite LQ optimal control with terminal state constraint is discussed. We present a general expression for the optimal control set in Section 4. A numerical example is applied in Section 5 to demonstrate the effectiveness of the model. We conclude the paper in Section 6.For convenience, throughout the paper, we adopt the following notations:R
n is the real n-dimensional Euclidean space; R
m
×
n is the set of all m
×
n matrices; M
τ is the transpose of matrix M; and t
r
(
M
) is the trace of a square matrix M. Moreover, M
>
0 (resp., M
≥
0) means that M
=
M
τ and M is positive (resp., positive semidefinite) definite.


## 2. Some Preliminaries

In this section, we introduce some useful definitions about uncertainty theory and Moore-Penrose pseudoinverse of a matrix.LetΓ be a nonempty set, and let L be a σ-algebra over Γ. Each element Λ in L is called an event. Anuncertain measure was defined by Liu [9] via the following three axioms.Axiom 1 (normality axiom).
M
{
Γ
}
=
1 for the universal set Γ.Axiom 2 (duality axiom).
M
{
Λ
}
+
M
{
Λ
c
}
=
1 for any event Λ.Axiom 3 (subadditivity axiom).
For every countable sequence of eventsΛ
1
,
Λ
2
,
…, we have (1)M⋃i=1∞Λi≤∑i=1∞MΛi.The triplet(
Γ
,
L
,
M
) is called anuncertainty space. Furthermore, Liu [19] defined a product uncertain measure by the product axiom.Axiom 4 (product axiom).
Let(
Γ
k
,
L
k
,
M
k
) be uncertainty spaces for k
=
1,2
,
…. Then, the product uncertain measure M on the product σ-algebra satisfies(2)M∏k=1∞Λk=⋀k=1∞MkΛk,where Λ
k are arbitrarily chosen events from L
k for k
=
1,2
,
…, respectively.Anuncertain variable is defined by Liu [9] as a function ξ from an uncertainty space (
Γ
,
L
,
M
) to the set of real numbers such that {
ξ
∈
B
} is an event for any Borel set B. In addition, anuncertainty distribution of ξ is defined as(3)Φx=Mγ∈Γ∣ξγ≤x,for any real number x.Independence is an important concept in uncertainty theory. The uncertain variablesξ
1
,
ξ
2
,
…
,
ξ
m are said to beindependent (Liu [19]) if(4)M⋂i=1mξi∈Bi=min1≤i≤mMξi∈Bifor any Borel sets B
1
,
B
2
,
…
,
B
n of real numbers.An uncertain variableξ is calledlinear (Liu [9]) if it has a linear uncertainty distribution (5)Φx=0,ifx≤a,x-ab-a,ifa≤x≤b,1,ifx≥bdenoted by L
(
a
,
b
), where a and b are real numbers with a
<
b.Letξ be an uncertain variable. Then, theexpected value (Liu [9]) of ξ is defined by (6)Eξ=∫0+∞Mξ≥rdr-∫-∞0Mξ≤rdrprovided that at least one of the two integrals is finite.Remark 1.
For numbersa and b, E
[
a
ξ
+
b
η
]
=
a
E
[
ξ
]
+
b
E
[
η
] if ξ and η are independent uncertain variables. Generally speaking, the expected value operator is not necessarily linear if the independence is not assumed.Remark 2.
Let(7)ξ=ξ11ξ12⋯ξ1qξ21ξ22⋯ξ2q⋯⋯⋯⋯ξp1ξp2⋯ξpq,where ξ
i
j are uncertain variables for i
=
1,2
,
…
,
p,  j
=
1,2
,
…
,
q. The expected value of ξ is provided by (8)Eξ=Eξ11Eξ12⋯Eξ1qEξ21Eξ22⋯Eξ2q⋯⋯⋯⋯Eξp1Eξp2⋯Eξpq.Lemma 3 (Penrose [20]).
Let a matrixM
∈
R
m
×
n be given. Then, there exists a unique matrix M
+
∈
R
n
×
m such that (9)MM+M=M,M+MM+=M+,MM+τ=MM+,M+Mτ=M+M.The matrix M
+ is called the Moore-Penrose pseudoinverse of M.Lemma 4 (Penrose [20]).
Let matrices L, M, and N be given with appropriate sizes. Then, the matrix equationL
X
M
=
N has a solution X if and only if L
L
+
N
M
M
+
=
N
. Moreover, any solution to L
X
M
=
N is represented by X
=
L
+
N
M
+
+
Y
-
L
+
L
Y
M
M
+, where Y is a matrix with an appropriate size.


## 3. Indefinite LQ Optimal Control with Constraints

### 3.1. Problem Statement

Consider the following indefinite LQ optimal control with terminal state constraint for discrete-time uncertain systems:(10)inf⁡uk0≤k≤N-1Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxNsubjecttoxk+1=Akxk+Bkuk+λkAkxk+Bkukξk,k=0,1,…,N-1,x0=x0ExNτxN=c,where 0
≤
λ
k
≤
1, state x
k
∈
R
n, control input u
k
∈
R
m, k
=
0,1
,
…
,
N
-
1, and x
0
∈
R
n is a given crisp vector. Denote u
=
(
u
0
,
u
1
,
…
,
u
N
-
1
). Moreover, Q
0
,
Q
1
,
…
,
Q
N and R
0
,
R
1
,
…
,
R
N
-
1 are real symmetric matrices with appropriate dimensions. In addition, c
≥
0 is a constant; the coefficients A
0
,
A
1
,
…
,
A
N
-
1 and B
0
,
B
1
,
…
,
B
N
-
1 are crisp matrices having appropriate dimensions determined from context. Besides, the noises ξ
0
,
ξ
1
,
…
,
ξ
N
-
1 are independent linear uncertain variables L
(
-
1,1
) with the distribution (11)Φx=0,ifx≤-1,x+12,if-1≤x≤1,1,ifx≥1.In this paper, the weighting matrices in the objective functional are not required to be definite. Therefore, problem (10) is an indefinite LQ optimal control problem. Next, we give the following definitions.Definition 5.
The indefinite LQ problem (10) is called well-posed if (12)Vx0=inf⁡uk0≤k≤N-1Jx0,u>-∞,∀x0∈Rn.Definition 6.
A well-posed problem is called solvable, if, forx
0
∈
R
n, there is a control sequence (u
0
∗
,
u
1
∗
,
…
,
u
N
-
1
∗) that achieves V
(
x
0
). In this case, the control (u
0
∗
,
u
1
∗
,
…
,
u
N
-
1
∗) is called an optimal control sequence.


### 3.2. An Equivalent Problem

Next, we transform the uncertain LQ optimal control problem (10) into an equivalent deterministic LQ optimal control problem which is subject to a matrix difference equation constraint.LetX
k
=
E
[
x
k
x
k
τ
]. Since state x
k
∈
R
n, x
k
x
k
τ is n
×
n matrix whose elements are uncertain variables, and X
k is a symmetric crisp matrix (
k
=
0,1
,
…
,
N
). Denote K
=
(
K
0
,
K
1
,
…
,
K
N
-
1
), where K
i are matrices for i
=
0,1
,
…
,
N
-
1.Theorem 7.
If the indefinite LQ problem (10) is solvable by a feedback control (13)uk=Kkxk,where K
k are constant crisp matrices, then it is equivalent to the following deterministic optimal control problem:(14)minKk0≤k≤N-1JX0,K=∑k=0N-1trQk+KkτRkKkXk+trQNXNsubjecttoXk+1=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ,X0=x0x0τ,trXN=c,for k
=
0,1
,
…
,
N
-
1.Proof.
Assume that the indefinite LQ problem (10) is solvable by a feedback control (15)uk=Kkxk,for k
=
0,1
,
…
,
N
-
1. Let X
k
=
E
[
x
k
x
k
τ
] for k
=
0,1
,
…
,
N. Then, we have(16)Xk+1=Exk+1xk+1τ=EAk+BkKk+λkAk+BkKkξkxkxkτAkτ+KkτBkτ+λkAkτ+KkτBkτξk=AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ+EUkξk+Vkξk2,where (17)Uk=2λkAkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτVk=λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ.Then, we obtain that λ
k
U
k
=
2
V
k. Because ξ
k and ξ
k
2 are not independent, we know that(18)EUkξk+Vkξk2≠UkEξk+VkEξk2. We will deal with (18) as follows.(i)
IfV
k
=
0, we obtain(19)EUkξk+Vkξk2=EUkξk=UkEξk=0.
(ii)
IfV
k
≠
0, we know that λ
k
≠
0 and 2
/
λ
k
≥
2. According to Example  2 in [21], we have(20)EUkξk+Vkξk2=E2λkVkξk+Vkξk2=VkE2λkξk+ξk2=13Vk.
Therefore, we have(21)EUkξk+Vkξk2=13Vk. Substituting (21) into (16) produces the following state matrix:(22)Xk+1=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ.
The associated cost function reduces to(23)minKk0≤k≤N-1JX0,K=minKk0≤k≤N-1∑k=0N-1trQk+KkτRkKkXk+trQNXN,and the constraint E
[
x
N
τ
x
N
]
=
c becomes t
r
X
N
=
c
.Remark 8.
Obviously, if problem (10) has a linear feedback optimal control solution u
k
∗
=
K
k
∗
x
k
(
k
=
0,1
,
…
,
N
-
1
), then K
k
∗
(
k
=
0,1
,
…
,
N
-
1
) is the optimal solution of problem (14).


### 3.3. A Necessary Condition for State Feedback Control

In this subsection, a necessary condition for the optimal linear state feedback control with deterministic gains to the indefinite LQ problem (10) is obtained by applying the deterministic matrix maximum principle [22].Theorem 9.
If the indefinite LQ problem (10) is solvable by a feedback control(24)uk=Kkxk,where K
k are constant crisp matrices, then there exist symmetric matrices H
k and a nonnegative γ
∈
R
1 solving the following constrained difference equation:(25)Hk=Qk+1+13λk2AkτHk+1Ak-MkτLk+Mk,LkLk+Mk-Mk=0,Lk=Rk+1+13λk2BkτHk+1Bk≥0,Mk=1+13λk2BkτHk+1Ak,HN=QN+γI,for k
=
0,1
,
…
,
N
-
1. Moreover,(26)Kk=-Lk+Mk+Yk-Lk+LkYkwith Y
k
∈
R
m
×
n, k
=
0,1
,
…
,
N
-
1, being any given crisp matrices.Proof.
Assume that the indefinite LQ problem (10) is solvable by (27)uk=Kkxk,where the matrices K
k
(
k
=
0,1
,
…
,
N
-
1
) are viewed as the control to be determined. It is obvious that K
k is also the optimal solution of problem (14) which is deterministic LQ optimal control problem. Hence, we can apply the matrix Lagrangian multiplier method to solve problem (14).
Let matricesH
k
+
1
(
k
=
0,1
,
…
,
N
-
1
) be the Lagrange multipliers of h
k
+
1
(
X
k
,
K
k
)
(
k
=
0,1
,
…
,
N
-
1
), and let γ
∈
R
1 be the Lagrange multiplier of g
(
X
N
)
=
0. Then, the Lagrange function is formed as(28)L=JX0,K+∑k=0N-1trHk+1hk+1Xk,Kk+γgXN,where (29)JX0,K=∑k=0N-1trQk+KkτRkKkXk+trQNXNhk+1Xk,Kk=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ-Xk+1,gXN=trXN-c.According to the first-order necessary conditions for optimality [22], we have(30)∂L∂Kk=0k=0,1,…,N-1,(31)Hk=∂L∂Xkk=0,1,…,N-1,(32)HN=QN+γI.
Based on the partial rule of gradient matrices [22], (30) can be transformed into(33)Rk+1+13λk2BkτHk+1BkKk+1+13λk2BkτHk+1Ak=0. Let(34)Lk=Rk+1+13λk2BkτHk+1Bk,Mk=1+13λk2BkτHk+1Ak. Then, (33) can be rewritten as L
k
K
k
+
M
k
=
0. Applying Lemma 4, we have L
k
L
k
+
M
k
=
M
k, and(35)Kk=-Lk+Mk+Yk-Lk+LkYk,Yk∈Rm×n. For (31), according to (36)Hk=∂L∂Xkk=0,1,…,N-1,we have(37)Hk=Qk+1+13λk2AkτHk+1Ak+KkτRk+1+13λk2BkτHk+1BkKk+1+13λk2AkτHk+1BkKk+1+13λk2KkτBkτHk+1Ak. Substituting (35) into (37), we obtain(38)Hk=Qk+1+13λk2AkτHk+1Ak-MkτLk+Mk. Consider the objective functional (39)Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxN=∑k=0N-1ExkτQkxk+ukτRkuk+Exk+1τHk+1xk+1-ExkτHkxk+ExNτQNxN-ExNτHNxN+x0τH0x0=∑k=0N-1trQk+KkτRkKkXk+trHk+1Xk+1-trHkXk+trQN-HNXN+x0τH0x0.Since X
k
+
1
=
(
1
+
1
/
3
λ
k
2
)
(
A
k
X
k
A
k
τ
+
A
k
X
k
K
k
τ
B
k
τ
+
B
k
K
k
X
k
A
k
τ
+
B
k
K
k
X
k
K
k
τ
B
k
τ
), the objective functional can be rewritten as (40)JX0,K=∑k=0N-1trQk+KkτRkKk+1+13λk2AkτHk+1Ak+KkτBkτHk+1Ak+AkτHk+1BkKk+KkτBkτHk+1BkKk-HkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trQk+1+13λk2AkτHk+1Ak-Hk+1+13λk2KkτBkτHk+1Ak+1+13λk2AkτHk+1BkKk+KkτRk+1+13λk2BkτHk+1BkKkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trMkτLk+Mk+KkτMk+MkτKk+KkτLkKkXk+trQN-HNXN+x0τH0x0.By applying (32) and Lemma 3, a completion of square implies(41)JX0,K=∑k=0N-1trKk+Lk+MkτLkKk+Lk+MkXk-cγ+x0τH0x0.
We assert thatL
k (k
=
0,1
,
…
,
N
-
1) must satisfy(42)Lk=Rk+1+13λk2BkτHk+1Bk≥0. If it is not so, there is an L
p for p
∈
{
0,1
,
…
,
N
-
1
} with a negative eigenvalue λ. Denote the unitary eigenvector with respect to λ as v
λ (i.e., v
λ
τ
v
λ
=
1 and L
p
v
λ
=
λ
v
λ). Let δ
≠
0 be an arbitrary scalar and construct a control sequence u
~
=
(
u
~
1
,
u
~
2
,
…
,
u
~
N
-
1
) as follows:(43)u~k=-Lk+Mkxk,k≠p,δλ-1/2vλ-Lk+Mkxk,k=p.The associated cost functional becomes (44)Jx0,u~=∑k=0N-1trK~k+Lk+MkτLkK~k+Lk+MkXk-cγ+x0τH0x0=∑k=0N-1Eu~k+Lk+MkxkτLku~k+Lk+Mkxk-cγ+x0τH0x0=δλ1/2vλτLpδλ1/2vλ-cγ+x0τH0x0=-δ2-cγ+x0τH0x0.Let δ
→
∞. Then, J
(
x
0
,
u
~
)
→
-
∞, which contradicts the well-posedness of problem (10).


### 3.4. Special Cases

We have obtained thatL
k
≥
0 in the constrained difference equation (25) of Theorem 9. The following corollaries are special cases of the above result if we have L
k
>
0 and L
k
=
0.Corollary 10.
The indefinite LQ problem (10) is uniquely solvable if and only if L
k
>
0 for k
=
0,1
,
…
,
N
-
1. Moreover, the unique optimal control is given by (45)uk=-Lk-1Mkxk,k=0,1,…,N-1.Proof.
By using Theorem9, we immediately obtain the corollary.Corollary 11.
IfL
k
=
0 for k
=
0,1
,
…
,
N
-
1, then any admissible control of the indefinite LQ problem (10) is optimal and the constrained difference equation (25) reduces to the following linear system:(46)Hk=Qk+1+13λk2AkτHk+1Ak,Rk+1+13λk2BkτHk+1Bk=0,BkτHk+1Ak=0,HN=QN+γI,for k
=
0,1
,
…
,
N
-
1
.Proof.
LettingL
k
=
0 in (25), it is easy to obtain the linear system (46). Letting L
k
=
0 in (41), (41) is simplified as(47)Jx0,u=-cγ+x0τH0x0,which implies that V
(
x
0
)
=
-
c
γ
+
x
0
τ
H
0
x
0 for any admissible control. Then, any admissible control of the indefinite LQ problem (10) is optimal.


### 3.5. Well-Posedness of the Indefinite LQ Problem

In the following, it is shown that the solvability of the constrained difference equation (25) is sufficient for the well-posedness of the indefinite LQ problem and the existence of an optimal control. Moreover, any optimal control can be represented explicitly as a linear state feedback by the solution of (25).Theorem 12.
The indefinite LQ problem (10) is well-posed if there exist symmetric matrices H
k and γ
∈
R
1 satisfying the constrained difference equation (25). Moreover, the optimal control is given by(48)uk=-Rk+1+13λk2BkτHk+1Bk+1+13λk2BkτHk+1Akxk,k=0,1,…,N-1.Furthermore, the optimal cost of the indefinite LQ problem (10) is(49)Vx0=x0τH0x0-cγ.Proof.
LetH
k and γ
∈
R
1 satisfy (25). Then,(50)Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxN=∑k=0N-1ExkτQkxk+ukτRkuk+Exk+1τHk+1xk+1-ExkτHkxk+ExNτQNxN-ExNτHNxN+x0τH0x0=∑k=0N-1trQk+KkτRkKkXk+trHk+1Xk+1-trHkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trQk+1+13λk2AkτHk+1Ak-Hk+1+13λk2KkτBkτHk+1Ak+1+13λk2AkτHk+1BkKk+KkτRk+1+13λk2BkτHk+1BkKkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trMkτLk+Mk+KkτMk+MkτKk+KkτLkKkXk+trQN-HNXN+x0τH0x0.By applying Lemma 3, a completion of square implies(51)JX0,K=∑k=0N-1trKk+Lk+MkτLkKk+Lk+MkXk+trQN-HNXN+x0τH0x0.Since L
k
≥
0, from (51), we can easily deduce that the cost function of problem (10) is bounded from below by(52)Vx0=trQN-HNXN+x0τH0x0>-∞,∀x0∈Rn. Hence, the indefinite LQ problem (10) is well-posed. It is clear that it is solvable by the feedback control(53)uk=-Kkxk=-Lk+Mkxk,k=0,1,…,N-1.
Furthermore, by usingt
r
X
N
=
c and H
N
=
Q
N
+
γ
I which we have obtained in Theorems 7 and 9, (52) indicates that the optimal value of problem (10) equals(54)Vx0=x0τH0x0-cγ.


## 3.1. Problem Statement

Consider the following indefinite LQ optimal control with terminal state constraint for discrete-time uncertain systems:(10)inf⁡uk0≤k≤N-1Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxNsubjecttoxk+1=Akxk+Bkuk+λkAkxk+Bkukξk,k=0,1,…,N-1,x0=x0ExNτxN=c,where 0
≤
λ
k
≤
1, state x
k
∈
R
n, control input u
k
∈
R
m, k
=
0,1
,
…
,
N
-
1, and x
0
∈
R
n is a given crisp vector. Denote u
=
(
u
0
,
u
1
,
…
,
u
N
-
1
). Moreover, Q
0
,
Q
1
,
…
,
Q
N and R
0
,
R
1
,
…
,
R
N
-
1 are real symmetric matrices with appropriate dimensions. In addition, c
≥
0 is a constant; the coefficients A
0
,
A
1
,
…
,
A
N
-
1 and B
0
,
B
1
,
…
,
B
N
-
1 are crisp matrices having appropriate dimensions determined from context. Besides, the noises ξ
0
,
ξ
1
,
…
,
ξ
N
-
1 are independent linear uncertain variables L
(
-
1,1
) with the distribution (11)Φx=0,ifx≤-1,x+12,if-1≤x≤1,1,ifx≥1.In this paper, the weighting matrices in the objective functional are not required to be definite. Therefore, problem (10) is an indefinite LQ optimal control problem. Next, we give the following definitions.Definition 5.
The indefinite LQ problem (10) is called well-posed if (12)Vx0=inf⁡uk0≤k≤N-1Jx0,u>-∞,∀x0∈Rn.Definition 6.
A well-posed problem is called solvable, if, forx
0
∈
R
n, there is a control sequence (u
0
∗
,
u
1
∗
,
…
,
u
N
-
1
∗) that achieves V
(
x
0
). In this case, the control (u
0
∗
,
u
1
∗
,
…
,
u
N
-
1
∗) is called an optimal control sequence.


## 3.2. An Equivalent Problem

Next, we transform the uncertain LQ optimal control problem (10) into an equivalent deterministic LQ optimal control problem which is subject to a matrix difference equation constraint.LetX
k
=
E
[
x
k
x
k
τ
]. Since state x
k
∈
R
n, x
k
x
k
τ is n
×
n matrix whose elements are uncertain variables, and X
k is a symmetric crisp matrix (
k
=
0,1
,
…
,
N
). Denote K
=
(
K
0
,
K
1
,
…
,
K
N
-
1
), where K
i are matrices for i
=
0,1
,
…
,
N
-
1.Theorem 7.
If the indefinite LQ problem (10) is solvable by a feedback control (13)uk=Kkxk,where K
k are constant crisp matrices, then it is equivalent to the following deterministic optimal control problem:(14)minKk0≤k≤N-1JX0,K=∑k=0N-1trQk+KkτRkKkXk+trQNXNsubjecttoXk+1=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ,X0=x0x0τ,trXN=c,for k
=
0,1
,
…
,
N
-
1.Proof.
Assume that the indefinite LQ problem (10) is solvable by a feedback control (15)uk=Kkxk,for k
=
0,1
,
…
,
N
-
1. Let X
k
=
E
[
x
k
x
k
τ
] for k
=
0,1
,
…
,
N. Then, we have(16)Xk+1=Exk+1xk+1τ=EAk+BkKk+λkAk+BkKkξkxkxkτAkτ+KkτBkτ+λkAkτ+KkτBkτξk=AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ+EUkξk+Vkξk2,where (17)Uk=2λkAkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτVk=λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ.Then, we obtain that λ
k
U
k
=
2
V
k. Because ξ
k and ξ
k
2 are not independent, we know that(18)EUkξk+Vkξk2≠UkEξk+VkEξk2. We will deal with (18) as follows.(i)
IfV
k
=
0, we obtain(19)EUkξk+Vkξk2=EUkξk=UkEξk=0.
(ii)
IfV
k
≠
0, we know that λ
k
≠
0 and 2
/
λ
k
≥
2. According to Example  2 in [21], we have(20)EUkξk+Vkξk2=E2λkVkξk+Vkξk2=VkE2λkξk+ξk2=13Vk.
Therefore, we have(21)EUkξk+Vkξk2=13Vk. Substituting (21) into (16) produces the following state matrix:(22)Xk+1=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ.
The associated cost function reduces to(23)minKk0≤k≤N-1JX0,K=minKk0≤k≤N-1∑k=0N-1trQk+KkτRkKkXk+trQNXN,and the constraint E
[
x
N
τ
x
N
]
=
c becomes t
r
X
N
=
c
.Remark 8.
Obviously, if problem (10) has a linear feedback optimal control solution u
k
∗
=
K
k
∗
x
k
(
k
=
0,1
,
…
,
N
-
1
), then K
k
∗
(
k
=
0,1
,
…
,
N
-
1
) is the optimal solution of problem (14).


## 3.3. A Necessary Condition for State Feedback Control

In this subsection, a necessary condition for the optimal linear state feedback control with deterministic gains to the indefinite LQ problem (10) is obtained by applying the deterministic matrix maximum principle [22].Theorem 9.
If the indefinite LQ problem (10) is solvable by a feedback control(24)uk=Kkxk,where K
k are constant crisp matrices, then there exist symmetric matrices H
k and a nonnegative γ
∈
R
1 solving the following constrained difference equation:(25)Hk=Qk+1+13λk2AkτHk+1Ak-MkτLk+Mk,LkLk+Mk-Mk=0,Lk=Rk+1+13λk2BkτHk+1Bk≥0,Mk=1+13λk2BkτHk+1Ak,HN=QN+γI,for k
=
0,1
,
…
,
N
-
1. Moreover,(26)Kk=-Lk+Mk+Yk-Lk+LkYkwith Y
k
∈
R
m
×
n, k
=
0,1
,
…
,
N
-
1, being any given crisp matrices.Proof.
Assume that the indefinite LQ problem (10) is solvable by (27)uk=Kkxk,where the matrices K
k
(
k
=
0,1
,
…
,
N
-
1
) are viewed as the control to be determined. It is obvious that K
k is also the optimal solution of problem (14) which is deterministic LQ optimal control problem. Hence, we can apply the matrix Lagrangian multiplier method to solve problem (14).
Let matricesH
k
+
1
(
k
=
0,1
,
…
,
N
-
1
) be the Lagrange multipliers of h
k
+
1
(
X
k
,
K
k
)
(
k
=
0,1
,
…
,
N
-
1
), and let γ
∈
R
1 be the Lagrange multiplier of g
(
X
N
)
=
0. Then, the Lagrange function is formed as(28)L=JX0,K+∑k=0N-1trHk+1hk+1Xk,Kk+γgXN,where (29)JX0,K=∑k=0N-1trQk+KkτRkKkXk+trQNXNhk+1Xk,Kk=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ-Xk+1,gXN=trXN-c.According to the first-order necessary conditions for optimality [22], we have(30)∂L∂Kk=0k=0,1,…,N-1,(31)Hk=∂L∂Xkk=0,1,…,N-1,(32)HN=QN+γI.
Based on the partial rule of gradient matrices [22], (30) can be transformed into(33)Rk+1+13λk2BkτHk+1BkKk+1+13λk2BkτHk+1Ak=0. Let(34)Lk=Rk+1+13λk2BkτHk+1Bk,Mk=1+13λk2BkτHk+1Ak. Then, (33) can be rewritten as L
k
K
k
+
M
k
=
0. Applying Lemma 4, we have L
k
L
k
+
M
k
=
M
k, and(35)Kk=-Lk+Mk+Yk-Lk+LkYk,Yk∈Rm×n. For (31), according to (36)Hk=∂L∂Xkk=0,1,…,N-1,we have(37)Hk=Qk+1+13λk2AkτHk+1Ak+KkτRk+1+13λk2BkτHk+1BkKk+1+13λk2AkτHk+1BkKk+1+13λk2KkτBkτHk+1Ak. Substituting (35) into (37), we obtain(38)Hk=Qk+1+13λk2AkτHk+1Ak-MkτLk+Mk. Consider the objective functional (39)Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxN=∑k=0N-1ExkτQkxk+ukτRkuk+Exk+1τHk+1xk+1-ExkτHkxk+ExNτQNxN-ExNτHNxN+x0τH0x0=∑k=0N-1trQk+KkτRkKkXk+trHk+1Xk+1-trHkXk+trQN-HNXN+x0τH0x0.Since X
k
+
1
=
(
1
+
1
/
3
λ
k
2
)
(
A
k
X
k
A
k
τ
+
A
k
X
k
K
k
τ
B
k
τ
+
B
k
K
k
X
k
A
k
τ
+
B
k
K
k
X
k
K
k
τ
B
k
τ
), the objective functional can be rewritten as (40)JX0,K=∑k=0N-1trQk+KkτRkKk+1+13λk2AkτHk+1Ak+KkτBkτHk+1Ak+AkτHk+1BkKk+KkτBkτHk+1BkKk-HkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trQk+1+13λk2AkτHk+1Ak-Hk+1+13λk2KkτBkτHk+1Ak+1+13λk2AkτHk+1BkKk+KkτRk+1+13λk2BkτHk+1BkKkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trMkτLk+Mk+KkτMk+MkτKk+KkτLkKkXk+trQN-HNXN+x0τH0x0.By applying (32) and Lemma 3, a completion of square implies(41)JX0,K=∑k=0N-1trKk+Lk+MkτLkKk+Lk+MkXk-cγ+x0τH0x0.
We assert thatL
k (k
=
0,1
,
…
,
N
-
1) must satisfy(42)Lk=Rk+1+13λk2BkτHk+1Bk≥0. If it is not so, there is an L
p for p
∈
{
0,1
,
…
,
N
-
1
} with a negative eigenvalue λ. Denote the unitary eigenvector with respect to λ as v
λ (i.e., v
λ
τ
v
λ
=
1 and L
p
v
λ
=
λ
v
λ). Let δ
≠
0 be an arbitrary scalar and construct a control sequence u
~
=
(
u
~
1
,
u
~
2
,
…
,
u
~
N
-
1
) as follows:(43)u~k=-Lk+Mkxk,k≠p,δλ-1/2vλ-Lk+Mkxk,k=p.The associated cost functional becomes (44)Jx0,u~=∑k=0N-1trK~k+Lk+MkτLkK~k+Lk+MkXk-cγ+x0τH0x0=∑k=0N-1Eu~k+Lk+MkxkτLku~k+Lk+Mkxk-cγ+x0τH0x0=δλ1/2vλτLpδλ1/2vλ-cγ+x0τH0x0=-δ2-cγ+x0τH0x0.Let δ
→
∞. Then, J
(
x
0
,
u
~
)
→
-
∞, which contradicts the well-posedness of problem (10).


## 3.4. Special Cases

We have obtained thatL
k
≥
0 in the constrained difference equation (25) of Theorem 9. The following corollaries are special cases of the above result if we have L
k
>
0 and L
k
=
0.Corollary 10.
The indefinite LQ problem (10) is uniquely solvable if and only if L
k
>
0 for k
=
0,1
,
…
,
N
-
1. Moreover, the unique optimal control is given by (45)uk=-Lk-1Mkxk,k=0,1,…,N-1.Proof.
By using Theorem9, we immediately obtain the corollary.Corollary 11.
IfL
k
=
0 for k
=
0,1
,
…
,
N
-
1, then any admissible control of the indefinite LQ problem (10) is optimal and the constrained difference equation (25) reduces to the following linear system:(46)Hk=Qk+1+13λk2AkτHk+1Ak,Rk+1+13λk2BkτHk+1Bk=0,BkτHk+1Ak=0,HN=QN+γI,for k
=
0,1
,
…
,
N
-
1
.Proof.
LettingL
k
=
0 in (25), it is easy to obtain the linear system (46). Letting L
k
=
0 in (41), (41) is simplified as(47)Jx0,u=-cγ+x0τH0x0,which implies that V
(
x
0
)
=
-
c
γ
+
x
0
τ
H
0
x
0 for any admissible control. Then, any admissible control of the indefinite LQ problem (10) is optimal.


## 3.5. Well-Posedness of the Indefinite LQ Problem

In the following, it is shown that the solvability of the constrained difference equation (25) is sufficient for the well-posedness of the indefinite LQ problem and the existence of an optimal control. Moreover, any optimal control can be represented explicitly as a linear state feedback by the solution of (25).Theorem 12.
The indefinite LQ problem (10) is well-posed if there exist symmetric matrices H
k and γ
∈
R
1 satisfying the constrained difference equation (25). Moreover, the optimal control is given by(48)uk=-Rk+1+13λk2BkτHk+1Bk+1+13λk2BkτHk+1Akxk,k=0,1,…,N-1.Furthermore, the optimal cost of the indefinite LQ problem (10) is(49)Vx0=x0τH0x0-cγ.Proof.
LetH
k and γ
∈
R
1 satisfy (25). Then,(50)Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxN=∑k=0N-1ExkτQkxk+ukτRkuk+Exk+1τHk+1xk+1-ExkτHkxk+ExNτQNxN-ExNτHNxN+x0τH0x0=∑k=0N-1trQk+KkτRkKkXk+trHk+1Xk+1-trHkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trQk+1+13λk2AkτHk+1Ak-Hk+1+13λk2KkτBkτHk+1Ak+1+13λk2AkτHk+1BkKk+KkτRk+1+13λk2BkτHk+1BkKkXk+trQN-HNXN+x0τH0x0=∑k=0N-1trMkτLk+Mk+KkτMk+MkτKk+KkτLkKkXk+trQN-HNXN+x0τH0x0.By applying Lemma 3, a completion of square implies(51)JX0,K=∑k=0N-1trKk+Lk+MkτLkKk+Lk+MkXk+trQN-HNXN+x0τH0x0.Since L
k
≥
0, from (51), we can easily deduce that the cost function of problem (10) is bounded from below by(52)Vx0=trQN-HNXN+x0τH0x0>-∞,∀x0∈Rn. Hence, the indefinite LQ problem (10) is well-posed. It is clear that it is solvable by the feedback control(53)uk=-Kkxk=-Lk+Mkxk,k=0,1,…,N-1.
Furthermore, by usingt
r
X
N
=
c and H
N
=
Q
N
+
γ
I which we have obtained in Theorems 7 and 9, (52) indicates that the optimal value of problem (10) equals(54)Vx0=x0τH0x0-cγ.


## 4. General Expression for the Optimal Control Set

In this part, we will present a general expression for the optimal control set based on the solution to (25).Theorem 13.
Assume thatH
k
(
k
=
0,1
,
…
,
N
-
1
) and γ
≥
0
∈
R
1 solves the constrained difference equation (25). A sufficient and necessary condition that u
k is in the set of all optimal feedback controls for indefinite LQ problem (10) is that(55)uk=-Lk+Mk+Yk-Lk+LkYkxk+Zk-Lk+MkZk,k=0,1,…,N-1,where Y
k
∈
R
m
×
n and Z
k
∈
R
m are arbitrary variables with appropriate size.Proof.
Sufficiency. According to the same calculation as in Theorem 9, we have(56)Jx0,u=∑k=0N-1ExkτQkxk+ukτRkuk+ExNτQNxN=∑k=0N-1trQk+1+13λk2AkτHk+1Ak-Hk+1+13λk2KkτBkτHk+1Ak+1+13λk2AkτHk+1BkKk+KkτRk+1+13λk2BkτHk+1BkKkXk-cγ+x0τH0x0=∑k=0N-1ExkτMkτLk+Mk+KkτMk+MkτKk+KkτLkKkxk-cγ+x0τH0x0=∑k=0N-1ExkτMkτLk+Mkxk+2xkτMkτuk+ukτLkuk-cγ+x0τH0x0.By denoting T
k
1
=
-
(
Y
k
-
L
k
+
L
k
Y
k
) and T
k
2
=
-
(
Z
k
-
L
k
+
L
k
Z
k
), we obtain(57)LkTk1=0,LkTk2=0.According to (56) and (57), we obtain (58)Jx0,u=∑k=0N-1Euk+Lk+Mk+Tk1xk+Tk2τLkuk+Lk+Mk+Tk1xk+Tk2-cγ+x0τH0x0.As L
k
≥
0, we know that the control u
k
=
-
[
(
L
k
+
M
k
+
T
k
1
)
x
k
+
T
k
2
] minimizes J
(
x
0
,
u
) with the optimal value -
c
γ
+
x
0
τ
H
0
x
0 for k
=
0,1
,
…
,
N
-
1.
Necessity. If any control sequence u
~
=
(
u
~
1
,
u
~
2
,
…
,
u
~
N
-
1
) which minimizes the cost function J
(
x
0
,
u
), then we have(59)Jx0,u~=∑k=0N-1Eu~k+Lk+MkxkτLku~k+Lk+Mkxk-cγ+x0τH0x0,for k
=
0,1
,
…
,
N
-
1
. The above equality implies that (60)∑k=0N-1Eu~k+Lk+MkxkτLku~k+Lk+Mkxk=0,k=0,1,…,N-1.Since L
k
≥
0, we get the following equivalent condition: (61)Lku~k+Lk+Mkxk=0,k=0,1,…,N-1.We see that u
~
k solves the following equation:(62)Lku~k+LkLk+Mkxk=0,k=0,1,…,N-1.By using Lemma 3 with L
=
L
k, M
=
I, N
=
-
L
k
L
k
+
M
k
x
k, it is easy to verify that (63)LL+NMM+=N.Then, we obtain the solution of  (62) with (64)u~k=-Lk+Mkxk+Zk-Lk+LkZk,Zk∈Rm,k=0,1,…,N-1.As in (35), the optimal control can be represented by (65)uk=-Lk+Mk+Yk-Lk+LkYkxk+Zk-Lk+MkZk,k=0,1,…,N-1.


## 5. Numerical Example

In this section, application of Theorem9 to solve constraint optimal control problem is illustrated. We present a two-dimensional indefinite LQ problem with terminal state constraint for discrete-time uncertain systems. A set of specific parameters of the coefficients are given as follows: (66)x0=01,c=2.0408,N=2,A0=1011,A1=2001,B0=21,B1=10,λ0=-32,λ1=32.The state weights and the control weights are as follows:(67)Q0=-1001,Q1=-100-1,Q2=0000,R0=-1,R1=-2.Note that, in this example, the state weight Q
0 is negative semidefinite, Q
1 is negative definite, and Q
2 is positive semidefinite and the control weights R
0 and R
1 are negative definite.In order to find the optimal controls and optimal cost value of this example, we have to solve the following equations:(68)Hk=Qk+1+13λk2AkτHk+1Ak-MkτLk+Mk,LkLk+Mk-Mk=0,Lk=Rk+1+13λk2BkτHk+1Bk≥0,Mk=1+13λk2BkτHk+1Ak,k=0,1,H2=Q2+γI,Xk+1=1+13λk2AkXkAkτ+AkXkKkτBkτ+BkKkXkAkτ+BkKkXkKkτBkτ,k=0,1,X0=x0x0τ.Firstly, we have(69)X0=x0x0τ=0001.Then, we get γ
=
2 by solving (68), and we obtain(70)H2=Q2+γI=γ1001=2002.Secondly, by applying Theorem9, we obtain the optimal feedback control and optimal cost value as follows.Fork
=
1, we obtain(71)L1=R1+1+13λ12B1τH2B1=0.5>0,M1=1+13λ12B1τH2A1=5,0,H1=Q1+1+13λ12A1τH2A1-M1τL1+M1=-41001.5.The optimal feedback control is u
1
=
K
1
x
1, where(72)K1=-L1+M1=-10,0.Fork
=
0, we obtain(73)L0=R0+1+13λ02B0τH1B0=0.875>0,M0=1+13λ02B0τH1A0=1.875,1.875,H0=Q0+1+13λ02A0τH1A0-M0τL0+M0=-54.3929-2.1429-2.1429-1.1429.The optimal feedback control is u
0
=
K
0
x
0, where(74)K0=-L0+M0=-2.1429,-2.1429.Finally, the optimal cost value is(75)Vx0=x0τH0x0-cγ=-5.2245.


## 6. Conclusion

We have considered the indefinite LQ optimal control with terminal state constraint involving state and control dependent uncertain noises. We first transform the uncertain LQ optimal control problem into a deterministic LQ optimal control problem. By means of the matrix maximum principle, we have presented a necessary condition for the existence of optimal linear state feedback control. Besides, we have proved the well-posedness of the indefinite LQ constraint problem by applying the technique of completing squares. For further work, we will consider discrete-time indefinite LQ optimal control model with inequality constraint.



---

*Source: 7241390-2016-02-22.xml*