# Stochastic Linear Quadratic Optimal Control with Indefinite Control Weights and Constraint for Discrete-Time Systems

Authors: Xikui Liu; Guiling Li; Yan Li
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

The Karush-Kuhn-Tucker (KKT) theorem is used to study stochastic linear quadratic optimal control with terminal constraint for discrete-time systems, allowing the control weighting matrices in the cost to be indefinite. A generalized difference Riccati equation is derived, which is different from those without constraint case. It is proved that the well-posedness and the attainability of stochastic linear quadratic optimal control problem are equivalent. Moreover, an optimal control can be denoted by the solution of the generalized difference Riccati equation.


---

## Body

## 1. Introduction

The linear quadratic (LQ) optimal control problem has been pioneered by Kalman [1] for deterministic systems; it is an assumption that the control weighting matrix in the cost is strictly definite. The definite LQ control problem has been investigated extensively by many researchers [2, 3]. The optimal control for the definite LQ problem has a feedback given by the solution of the Riccati equation. The extension of deterministic LQ problem to stochastic case has been playing an important role in engineering design and applications; see monographs [4–7]. Stochastic LQ control problem for the Itô systems is initiated by Wonham [4], while the nonlinear regulator problem is discussed in [8] and has caused a sequence of works [9–11]. Some of the works on this subject reveal that, for stochastic Itô systems, even if the state and control weighting matrices Q and R are indefinite, the corresponding stochastic LQ problem may be still well posed, which is first found in [12].For the discrete-time LQ control problems with control and/or state dependent noises, there have been some works in literature [13, 14]. It is worth noting that the state weight matrix is nonnegative and the control weight matrix is positive definite in both papers. However, the control weighting matrix is not required to be positive definite, or even negative [15–18]. In addition, most previous researchers mainly study indefinite stochastic LQ problems without constraints. In fact, some constraints are of considerable importance in many physical systems. The finite time indefinite stochastic LQ control with linear terminal state constraint is discussed in [19] and has been extended in [20–22]. It is generally known that, for the system components are perturbed by an additive Gaussian white noise, the LQ problem is called linear quadratic Gaussian problem. As said in [15], many real systems are not only subject to Gaussian white noise, but also subject to non-Gaussian noise.In this paper, different from [20–22], we discuss a stochastic optimal control of discrete-time systems which are subject to non-Gaussian noises. We concentrate our attention on the finite horizon indefinite stochastic LQ control with terminal inequality constraint. Such constraints are often seen in H
∞ filtering problems [23, 24]. The existence of optimal linear state feedback control in terms of KKT theorem will be shown. We present the fact that the solvability of the GDRE, the well-posedness, and the attainability of the LQ problem are all equivalent. The outline of this paper is organized as follows. In Section 2, we give some definitions and preliminaries. Section 3 contains our main theorems. A necessary condition for the existence of optimal linear state feedback control is derived. Moreover, it is shown that the solvability of the GDRE, the well-posedness, and the attainability of the LQ problem are all equivalent. In Section 4, we give the structure of the optimal control. Section 5 concludes the paper.For convenience, we adopt the following notations in this note.M
′: is the transpose of a matrix M; tr
⁡
(
M
) is the trace of a square matrix M; M
>
0
(
M
≥
0
): M is positive definite (positive semidefinite) symmetric matrix; E
[
x
] represents the mathematical expectation of a random variable x; R
k is the k-dimensional Euclidean space with the usual 2-norm ·; R
m
×
nis the vector space of all m
×
n matrices with entries in R; M
† is the Moore-Penrose pseudoinverse of a matrix M; I is the identity matrix with appropriate dimension; N
t
:
0,1
,
2
,
…
,
t
-
1.


## 2. Preliminaries

Consider the discrete-time stochastic systemΣ
*
:
(1)
x
k
+
1
=
A
(
k
)
+
ω
x
(
k
)
C
(
k
)
x
(
k
)
+
B
k
+
ω
u
k
D
k
u
(
k
)
+
ω
(
k
)
,
k
∈
N
T
,
a
i
1
x
1
T
+
a
i
2
x
2
T
+
⋯
+
a
i
n
x
n
T
≤
ξ
i
,
where x
(
0
)
=
x
0
∈
R
n is the given initial state and x
(
k
)
∈
R
n and u
(
k
)
∈
R
m are, respectively, the system state and controlled input. A
(
k
), C
(
k
)
∈
R
n, B
(
k
), and D
(
k
)
∈
R
n
×
m are matrix-valued functions with appropriate dimensions.The noisesω
x
(
k
)
,
ω
u
(
k
)
,
ω
(
k
) are defined on a complete probability space Ω
,
F
,
μ. Without loss of generality, we assume that ω
x
(
k
)
,
ω
u
(
k
) are scalar random variables. The initial state is assumed to be independent of ω
x
(
k
)
,
ω
u
(
k
)
,
ω
(
k
) and satisfies E
ω
x
k
=
E
ω
u
k
=
E
ω
k
=
0,  E
ω
x
k
2
=
E
ω
u
k
2
=
1, E
ω
k
ω
′
k
=
V
k, E
ω
x
k
ω
u
k
=
ρ
x
u
(
k
), and E
ω
k
ω
x
k
=
E
ω
k
ω
u
k
=
0.We denoteF
k the σ-algebra generated by (
ω
x
(
k
)
,
ω
u
(
k
)
,
ω
(
k
)
); that is, F
k
=
{
σ
(
ω
x
(
k
)
,
ω
u
(
k
)
,
ω
(
k
)
:
k
∈
N
T
)
}. u
(
·
) belongs to the admissible control set U
ad
=
{
u
(
k
)
∈
R
m
:
E
[
∑
k
=
0
T
-
1
u
(
k
)
2
]
<
+
∞
}. ξ
i is F
T measurable square integrable stochastic process; namely, E
[
ξ
i
]
<
+
∞. Let N
r
×
n
=
(
a
i
j
)
r
×
n
,
ξ
=
ξ
1
,
ξ
2
,
…
,
ξ
r
′; then the constraint in (1) can be denoted by N
x
(
T
)
≤
ξ, where a
i
j is constant and N has row full rank.We consider the following cost function correlated with the systemΣ
*
:
(2)
J
x
0
,
u
0
,
…
,
u
T
-
1
=
∑
k
=
0
T
-
1
‍
E
x
′
k
Q
k
x
k
+
u
′
k
R
k
u
k
+
x
′
T
S
x
T
,
where Q
(
k
)
,
S, and R
(
k
) are symmetric matrices with appropriate dimension, which are possibly indefinite. We define
(3)
V
(
x
0
)
=
inf
⁡
u
(
0
)
,
…
,
u
(
T
-
1
)
J
(
x
0
,
u
(
0
)
,
…
,
u
(
T
-
1
)
)
.In the sequel, we study the LQ problem for the systems (1)–(3), that is to say, finding a control to minimize J
(
x
0
,
u
(
0
)
,
…
,
u
(
T
-
1
)
). Firstly, we state some useful definitions and lemmas that are essential to the discussions of our main results.Definition 1.
IfV
(
x
0
)
>
-
∞ for any x
0, systems (1)–(3) are called well posed.Definition 2.
If there exists an admissible control(
u
*
(
0
)
,
…
,
u
*
(
T
-
1
)
) such that
(4)
V
x
0
=
J
x
0
,
u
*
0
,
…
,
u
*
T
-
1
,
then systems (1)–(3) are said to be attainable and (
u
*
(
0
)
,
…
,
u
*
(
T
-
1
)
) is called an optimal control.If a linear feedback control is optimal for the LQ problem (1)–(3), then it must be also optimal linear feedback control of the following form:
(5)
u
k
=
K
k
x
k
,
k
∈
N
T
,
where K
(
k
) is matrix-valued function.MP (mathematical programming)(6)
min
⁡
f
(
x
)
s
.
t
.
g
x
≤
0
s
.
t
.
h
x
=
0
,
where g
(
x
)
=
(
g
1
(
x
)
,
…
,
g
p
(
x
)
)
,
h
(
x
)
=
(
h
1
(
x
)
,
…
,
h
q
(
x
)
).Definition 3 (regularity condition see [25]).
LetI
*
=
{
i
∣
g
i
(
x
*
)
=
0
}. If the gradient vectors ∇
g
i
(
x
*
), i
∈
I
*, and ∇
h
j
(
x
*
), j
=
1
,
…
,
q, are linearly independent, this linear independence is called a regularity condition (or constraint qualification).Definition 4 (regular point see [25]).
LetI
*
=
{
i
∣
g
i
(
x
*
)
=
0
}. Then x
* is called a regular point of the constraints if the gradient vectors ∇
g
i
(
x
*
), ∇
h
j
(
x
*
), i
∈
I
*, j
=
1
,
…
,
q, are linearly independent.Lemma 5 (KKT theorem see [25]).
In MP above, suppose that the objective functionf and the constraint functions g
=
g
1
,
…
,
g
p
′, h
=
h
1
,
…
,
h
q
′ are continuously differentiable at a point x
*. If x
* is a local minimum that satisfies some regularity conditions, then there exist a vector λ
≥
0 in R
m and a vector μ in R
n, called KKT multipliers, such that
(7)
∇
x
L
x
*
,
λ
*
,
μ
*
=
0
,
λ
′
g
x
*
=
0
complementary slackness condition
,
where the Lagrangian function L
(
x
,
λ
,
μ
)
=
f
(
x
)
+
λ
′
g
+
μ
′
h
(
x
).Lemma 6 (see [26]).
Let a matrixM
∈
R
m
×
n be given, matrix M
†
∈
R
n
×
m which is called the Moore-Penrose pseudoinverse of M, such that
(8)
M
M
†
M
=
M
,
M
†
M
M
†
=
M
†
,
M
M
†
′
=
M
M
†
,
M
†
M
′
=
M
†
M
.Lemma 7 (see [26]).
Let a symmetric matrixM be given. Then
(9)
M
′
†
=
M
†
′
,
M
M
†
=
M
†
M
,
M
≥
0
iff
M
†
≥
0
.Lemma 8 ((extended Schur's lemma) see [27]).
Let matricesM
=
M
′, N, and R
=
R
′ be given with appropriate sizes. Then the following conditions are equivalent:(i)
M
-
N
R
†
N
′
≥
0
,
R
≥
0, and N
(
I
-
R
R
†
)
=
0;
(ii)
M
N
N
′
R
≥
0;
(iii)
R
N
′
N
M
≥
0.Lemma 9 (see [15]).
Let matricesL, M, and N be given. Then the matrix equation L
X
M
=
N has a solution X if and only if L
L
†
N
M
M
†
=
N. X is given by X
=
L
†
N
M
†
+
Y
-
L
†
L
Y
M
M
†, where Y is a matrix with an appropriate dimension.Lemma 10 (see [15]).
Let matricesF
=
F
′, H, and G
=
G
′ be given with appropriate sizes. Consider the following quadratic form:
(10)
q
x
,
u
=
E
x
′
t
F
t
x
t
+
2
x
′
t
H
t
u
t
+
u
′
t
G
t
u
t
,
where x and u are random variables defined on a probability space (
Ω
,
B
,
P
). Then the following conditions are equivalent: (i)
inf
⁡
u
t
⁡
q
(
x
,
u
)
>
-
∞ for any random variable x;
(ii)
there exists a symmetric matrixW
=
W
′ such that inf
⁡
u
t
q
(
x
,
u
)
=
E
[
x
′
W
x
] for any random variable x;
(iii)
G
≥
O and H
I
-
G
G
†
=
0;
(iv)
G
≥
O and Ker
⁡
G
⊆
Ker
⁡
H;
(v)
there exists a symmetric matrixT
=
T
′ such that F
-
T
H
H
′
G
≥
0.
Moreover, if any of the above condition holds, then (ii) is satisfied byW
=
F
-
H
G
†
H
′. In addition, W
≥
T for any T satisfying (v). Finally, for any random variable x, the random variable u
*
=
-
G
†
H
′
x is optimal with the following optimal value:
(11)
q
x
,
u
*
=
E
x
′
F
-
H
G
†
H
′
x
.


## 3. Well-Posedness and Attainability under State Feedback Control

In this section, we transform the LQ problem into an equivalent deterministic optimization problem. By means of the KKT theorem, we present a generalized difference Riccati equation (GDRE) without any positiveness constraint. Then, it is shown that the well-posedness and the attainability are equivalent to the solvability of GDRE.Theorem 11.
If the LQ optimal control problem (1)–(3) is attainable by u
(
k
)
=
K
(
k
)
x
(
k
) and the regular point (
u
*
(
k
)
,
x
*
(
k
)
) is a locally optimal solution of problem (1)–(3), then the following generalized difference Riccati equation (GDRE) has solutions (
P
(
k
)
,
μ
) with 0
≤
μ
∈
R
1
,
k
∈
N
T:
(12)
P
k
=
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
+
Q
k
-
H
′
k
G
†
k
H
k
,
H
(
k
)
=
B
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
ρ
x
u
k
D
′
k
P
k
+
1
C
k
,
G
k
=
R
k
+
B
′
k
P
k
+
1
B
k
+
D
′
k
P
k
+
1
D
k
≥
0
,
k
∈
N
T
,
μ
=
tr
⁡
[
P
(
T
)
-
S
]
tr
⁡
(
N
′
N
)
,
μ
[
tr
⁡
(
X
T
N
′
N
)
-
tr
⁡
(
M
)
]
=
0
,
M
=
E
[
ξ
ξ
′
]
.
In addition,(13)
u
*
k
=
-
G
†
k
H
k
+
Y
k
-
G
†
k
G
k
Y
k
x
k
,
Y
k
∈
R
m
×
n
,
k
∈
N
T
,
V
(
x
0
)
=
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
x
0
′
P
(
0
)
x
0
-
μ
tr
⁡
(
M
)
.Proof .
LetX
(
k
)
=
E
x
k
x
′
(
k
) and u
(
k
)
=
K
(
k
)
x
(
k
) for any k
∈
N
T; it can be shown that the LQ problem (1)–(3) can be rewritten as the following optimization problem:
(14)
min
⁡
K
(
0
)
,
…
,
K
(
T
-
1
)
∑
k
=
0
T
-
1
‍
tr
⁡
[
(
Q
(
k
)
+
K
′
(
k
)
R
(
k
)
K
(
k
)
)
X
(
k
)
]
+
tr
⁡
S
X
T
s
.
t
.
X
k
+
1
=
A
k
X
k
A
′
k
+
C
k
X
k
C
′
k
+
B
k
K
k
X
k
K
′
k
B
′
k
X
k
×
K
′
k
D
′
k
+
A
k
X
k
K
′
k
B
′
k
+
B
k
K
k
X
k
A
′
k
+
ρ
x
u
k
D
k
K
k
X
k
C
′
k
+
ρ
x
u
k
C
k
X
k
K
′
k
D
′
k
+
V
k
,
X
0
=
E
x
0
x
′
0
=
x
0
x
0
′
,
tr
⁡
X
T
N
′
N
≤
tr
⁡
M
,
M
=
E
ξ
ξ
′
.
Obviously, the problem (14) is a MP problem indicated as
(15)
min
⁡
f
[
X
(
k
)
,
K
(
k
)
]
,
k
∈
N
T
,
s
.
t
.
h
X
k
,
K
k
=
0
,
k
∈
N
T
,
s
.
t
.
g
[
X
(
T
)
]
≤
0
,
where
(16)
f
[
X
(
k
)
,
K
(
k
)
]
=
∑
k
=
0
T
-
1
‍
tr
⁡
Q
k
+
K
′
k
R
k
K
k
X
k
+
tr
⁡
S
X
T
,
h
[
X
(
k
)
,
K
(
k
)
]
=
A
k
X
k
A
′
(
k
)
+
C
k
X
k
C
′
(
k
)
+
B
k
K
k
X
k
K
′
k
B
′
k
X
k
K
′
k
D
′
(
k
)
+
A
k
X
k
K
′
k
B
′
(
k
)
+
B
k
K
k
X
k
A
′
(
k
)
+
ρ
x
u
k
D
k
K
k
X
k
C
′
(
k
)
+
ρ
x
u
k
C
k
X
k
K
′
k
D
′
k
+
V
k
,
g
[
X
(
T
)
]
=
tr
⁡
[
X
T
N
′
N
]
-
M
.
According to KKT theorem, the Lagrangian function is defined as(17)
L
(
X
(
k
)
,
K
(
k
)
,
P
(
k
+
1
)
,
μ
)
=
f
[
X
(
k
)
,
K
(
k
)
]
+
∑
k
=
0
T
-
1
‍
tr
⁡
{
P
(
k
+
1
)
h
[
X
(
k
)
,
K
(
k
)
]
}
+
μ
g
X
T
,
where μ
∈
R
1 and the matrices P
(
0
)
,
P
(
1
)
,
…
,
P
(
T
) are Lagrangian multipliers.
Moreover, the following result,(18)
∂
L
∂
X
(
k
)
=
∂
L
∂
K
(
k
)
=
∂
L
∂
X
(
T
)
=
0
,
is obvious.
By calculating, we conclude thatP
(
k
) and λ satisfy the equations of the form
(19)
R
k
+
B
′
k
P
k
+
1
B
k
+
D
′
k
P
k
+
1
D
k
K
(
k
)
=
-
B
′
(
k
)
P
(
k
+
1
)
A
(
k
)
-
ρ
x
u
k
D
′
(
k
)
P
(
k
+
1
)
C
(
k
)
,
(20)
P
k
=
C
′
(
k
)
P
(
k
)
C
(
k
)
+
K
′
(
k
)
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
+
1
)
D
(
k
)
K
(
k
)
+
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
ρ
x
u
k
D
′
(
k
)
P
(
k
+
1
)
C
(
k
)
+
Q
k
+
A
′
k
P
k
+
1
A
k
,
(21)
P
T
=
S
+
μ
N
′
N
.
From Lemma9, (19) has a solution K
(
k
) if and only if G
†
(
k
)
G
(
k
)
H
(
k
)
=
H
(
k
) and K
(
k
)
=
-
G
†
(
k
)
H
(
k
)
+
Y
(
k
)
-
G
†
(
k
)
G
(
k
)
Y
(
k
), where
(22)
G
(
k
)
=
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
+
1
)
D
(
k
)
,
H
k
=
B
′
k
P
k
+
1
A
k
+
ρ
x
u
k
D
′
k
P
k
+
1
C
k
,
Y
k
∈
R
m
×
n
.
We substitute the above gains into (21); then the corresponding equations are formed as
(23)
P
k
=
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
+
Q
k
-
H
′
k
G
†
k
H
k
,
H
(
k
)
=
B
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
ρ
x
u
k
D
′
k
P
k
+
1
C
k
,
G
k
=
R
k
+
B
′
k
P
k
+
1
B
k
+
D
′
k
P
k
+
1
D
k
≥
0
,
k
∈
N
T
,
μ
=
tr
⁡
[
P
(
T
)
-
S
]
tr
⁡
(
N
′
N
)
.
The only thing to note is that we can assumeP
(
k
) is symmetric. Otherwise, we take P
~
(
k
)
=
(
P
′
k
+
P
k
)
/
2.
Now we add the equality(24)
∑
k
=
0
T
-
1
‍
E
x
′
k
+
1
P
k
+
1
x
k
+
1
-
x
′
k
P
k
x
k
=
E
x
′
T
P
T
x
T
-
x
0
′
P
0
x
0
to (2) and use (23); then we have
(25)
J
x
0
,
u
0
,
…
,
u
T
-
1
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
k
R
k
u
k
+
x
′
k
+
1
P
k
+
1
x
k
+
1
-
x
′
(
k
)
P
(
k
)
x
(
k
)
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
-
P
(
k
)
+
A
′
k
P
k
+
1
A
k
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
x
(
k
)
+
2
x
′
(
k
)
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
ρ
x
u
k
C
′
(
k
)
P
(
k
+
1
)
D
(
k
)
u
(
k
)
+
u
′
(
k
)
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
)
D
(
k
)
u
(
k
)
+
∑
k
=
0
T
-
1
‍
E
ω
′
k
P
k
+
1
ω
k
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
.
By completion of square, we obtain(26)
J
(
x
0
,
u
(
0
)
,
…
,
u
(
T
-
1
)
)
=
∑
k
=
0
T
-
1
‍
E
u
k
+
G
†
k
H
k
x
k
′
G
(
k
)
×
u
(
k
)
+
G
†
(
k
)
H
(
k
)
x
(
k
)
u
k
+
G
†
k
H
k
x
k
′
+
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
.
Here, we must prove thatG
(
k
)
≥
0
,
k
∈
N
T. Let us assume that there exists a G
(
l
) with a negative eigenvalue λ. Let v
λ be the unitary eigenvector about λ; it implies that v
λ
′
v
λ
=
1 and G
(
l
)
v
λ
=
λ
v
λ. For any δ
≠
0, let us suppose that a control sequence is given by
(27)
u
~
(
k
)
=
-
G
k
†
H
(
k
)
x
(
k
)
,
k
≠
l
,
δ
λ
-
1
/
2
v
λ
-
G
†
(
k
)
H
(
k
)
x
(
k
)
,
k
=
l
.
The corresponding cost is
(28)
J
(
x
0
,
u
~
(
0
)
,
…
,
u
~
(
T
-
1
)
)
=
∑
k
=
0
T
-
1
‍
E
u
k
+
G
†
k
H
k
x
k
′
G
(
k
)
×
u
k
+
G
†
k
H
k
x
k

u
k
+
G
†
k
H
k
x
k
′
+
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
E
[
x
′
(
T
)
(
S
-
P
(
T
)
)
x
(
T
)
]
+
x
0
′
P
(
0
)
x
0
=
δ
λ
-
1
/
2
v
λ
′
G
l
δ
λ
-
1
/
2
v
λ
+
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
=
-
δ
2
+
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
.
Letting δ
→
∞, it yields J
(
x
0
,
u
~
(
0
)
,
…
,
u
~
(
T
-
1
)
)
→
-
∞, which is in contradiction with the attainability of the LQ problem (1)–(3).
From the above discussion and (21), it can be seen that the optimal value V
(
x
0
) is given by
(29)
V
x
0
=
J
x
0
,
u
*
0
,
…
,
u
*
T
-
1
=
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
x
0
′
P
(
0
)
x
0
-
μ
tr
⁡
(
M
)
.
This proof is complete.The following corollary shows that whenG
(
k
)
>
0 in GDRE (12), then P
(
k
), μ, and u
*
(
k
) are all unique.Corollary 12.
If the LQ optimal control problem (1)–(3) is attainable by u
(
k
)
=
K
(
k
)
x
(
k
) and the regular point (
u
*
(
k
)
,
x
*
(
k
)
) is a locally optimal solution of problem (1)–(3), then the following GDRE has unique solutions (
P
(
k
)
,
μ
) with 0
≤
μ
∈
R
1
,
k
∈
N
T:
(30)
P
k
=
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
+
Q
k
-
H
′
k
G
-
1
k
H
k
,
H
(
k
)
=
B
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
ρ
x
u
k
D
′
k
P
k
+
1
C
k
,
G
k
=
R
k
+
B
′
k
P
k
+
1
B
k
+
D
′
k
P
k
+
1
D
k
>
0
,
k
∈
N
T
,
μ
=
tr
⁡
[
P
(
T
)
-
S
]
tr
⁡
N
′
N
,
μ
tr
⁡
X
T
N
′
N
-
tr
⁡
M
=
0
,
N
=
E
ξ
ξ
′
.In addition,(31)
u
*
k
=
-
G
-
1
k
H
k
x
k
,
k
∈
N
T
,
V
(
x
0
)
=
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
x
0
′
P
0
x
0
-
μ
tr
⁡
M
.The following result is useful in the sequel, which gives an equivalent connection between the solvability of the GDRE and the well-posedness of the LQ problem.Theorem 13.
The LQ problem (1)–(3) is well posed; then there exist solutions (
P
(
k
)
,
μ
) to the GDRE (12). Conversely, if the GDRE (12) has solutions (
P
(
k
)
,
μ
), then the LQ problem (1)–(3) is well posed. Moreover, the optimal cost satisfies
(32)
V
x
0
=
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
x
0
′
P
(
0
)
x
0
-
μ
tr
⁡
(
M
)
.Proof.
Necessity part: consider the following cost froml to T
:
(33)
W
l
x
l
=
inf
⁡
u
0
,
…
,
u
T
-
1
∑
k
=
l
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
(
k
)
R
(
k
)
u
(
k
)
+
x
′
(
T
)
S
x
(
T
)
.
According to the optimal principle, if W
l
(
x
(
l
)
) is finite, so is W
m
(
x
(
m
)
) for any l
≤
m. As W
0
(
x
0
) is finite, we can infer that W
l
(
x
(
l
)
) is finite for any 0
≤
l
≤
T
-
1.
Letl
=
T
-
1 and P
(
T
)
=
S
+
μ
N
′
N. By (1) and (33), it follows that
(34)
W
T
-
1
(
x
(
T
-
1
)
)
-
tr
⁡
[
V
(
T
-
1
)
P
(
T
)
]
-
E
[
x
′
T
S
-
P
T
x
(
T
)
]
=
inf
⁡
u
T
-
1
E
x
′
(
T
-
1
)
Q
(
T
-
1
)
x
(
T
-
1
)
+
u
′
(
T
-
1
)
R
(
T
-
1
)
u
(
T
-
1
)
+
x
′
(
T
)
S
x
(
T
)
-
tr
⁡
V
T
-
1
P
T
-
E
x
′
T
S
-
P
T
x
T
=
inf
⁡
u
T
-
1
E
x
′
T
-
1
×
Q
T
-
1
+
A
′
T
-
1
P
T
A
T
-
1
+
C
′
(
T
-
1
)
P
(
T
)
C
(
T
-
1
)
x
(
T
-
1
)
+
2
x
′
(
T
-
1
)
A
′
T
-
1
P
T
B
T
-
1
+
ρ
x
u
T
-
1
C
′
×
(
T
-
1
)
P
(
T
)
D
(
T
-
1
)
A
′
T
-
1
P
T
B
T
-
1
×
u
(
T
-
1
)
+
u
′
(
T
-
1
)
D
′
T
-
1
P
T
D
T
-
1
R
T
-
1
+
B
′
(
T
-
1
)
P
(
T
)
B
(
T
-
1
)
+
D
′
T
-
1
P
T
D
T
-
1
×
u
T
-
1
x
′
T
-
1
.
Applying Lemma 10 to the above quadratic form, there exists a symmetric matrix P
(
T
-
1
) such that
(35)
W
T
-
1
(
x
(
T
-
1
)
)
-
tr
⁡
[
V
(
T
-
1
)
P
(
T
)
]
-
E
[
x
′
(
T
)
(
S
-
P
(
T
)
)
x
(
T
)
]
=
E
[
x
′
(
T
-
1
)
P
(
T
-
1
)
x
(
T
-
1
)
]
,
P
(
T
-
1
)
=
Q
(
T
-
1
)
+
A
′
(
T
-
1
)
P
(
T
)
A
(
T
-
1
)
+
C
′
(
T
-
1
)
P
(
T
)
C
(
T
-
1
)
-
H
′
T
-
1
G
†
T
-
1
H
T
-
1
,
G
(
T
-
1
)
=
R
(
T
-
1
)
+
B
′
(
T
-
1
)
P
(
T
)
B
(
T
-
1
)
+
D
′
(
T
-
1
)
P
(
T
)
D
(
T
-
1
)
≥
0
,
H
(
T
-
1
)
=
B
′
(
T
-
1
)
P
(
T
)
A
(
T
-
1
)
+
ρ
x
u
T
-
1
D
′
(
T
-
1
)
P
(
T
)
C
(
T
-
1
)
.
It is obvious that the above are GDRE (12) for k
=
T
-
1.
Hence, assume that GDRE (12) admits a pair of solutions (
P
(
k
)
,
μ
)
,
(
k
=
l
,
…
,
T
-
1
) with
(36)
W
l
(
x
(
l
)
)
-
tr
⁡
[
V
(
l
)
P
(
l
+
1
)
]
-
E
[
x
′
(
T
)
(
S
-
P
(
T
)
)
x
(
T
)
]
=
E
[
x
′
(
l
)
P
(
l
)
x
(
l
)
]
.
From (33), we have
(37)
W
l
-
1
x
l
-
1
-
∑
k
=
l
-
1
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
-
E
x
′
T
S
-
P
T
x
T
=
inf
⁡
u
l
-
1
E
x
l
x
′
l
-
1
Q
l
-
1
x
l
-
1
+
u
′
(
l
-
1
)
R
(
l
-
1
)
u
(
l
-
1
)
+
V
l
x
l
-
∑
k
=
l
-
1
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
-
E
x
′
T
S
-
P
T
x
T
=
inf
⁡
u
l
-
1
E
x
′
(
l
-
1
)
Q
(
l
-
1
)
x
(
l
-
1
)
+
u
′
(
l
-
1
)
R
(
l
-
1
)
u
(
l
-
1
)
+
x
′
(
l
)
P
(
l
)
x
(
l
)
-
tr
⁡
[
V
(
l
-
1
)
P
(
l
+
1
)
]
-
E
x
′
T
S
-
P
T
x
T
=
inf
⁡
u
l
-
1
E
x
′
(
l
-
1
)
C
l
-
1
+
C
′
l
-
1
P
l
-
1
C
l
-
1
Q
(
l
-
1
)
+
A
′
l
-
1
P
l
-
1
A
l
-
1
+
C
′
l
-
1
P
l
-
1
C
l
-
1
×
x
l
-
1
+
2
x
′
l
-
1
A
′
l
-
1
P
l
B
l
-
1
+
ρ
x
u
l
-
1
C
′
l
-
1
×
Q
l
D
l
-
1
A
′
l
-
1
×
u
(
l
-
1
)
+
u
′
(
l
-
1
)
R
l
-
1
+
B
′
l
-
1
P
l
B
l
-
1
+
D
′
l
-
1
P
l
D
l
-
1
×
u
l
-
1
x
′
(
l
-
1
)
C
l
-
1
Q
(
l
-
1
)
.
By Lemma 10, it is straightforward that the finiteness of V
l
-
1
x
(
l
-
1
) is equivalent to the following:
(38)
P
l
-
1
=
A
′
l
-
1
P
l
A
l
-
1
+
C
′
l
-
1
P
l
C
l
-
1
+
Q
l
-
1
-
H
′
l
-
1
G
†
l
-
1
H
l
-
1
,
H
l
-
1
=
B
′
(
l
-
1
)
P
(
l
)
A
(
l
-
1
)
+
ρ
x
u
l
-
1
D
′
l
-
1
P
l
C
l
-
1
,
G
l
-
1
=
R
(
l
-
1
)
+
B
′
(
l
-
1
)
P
(
l
)
B
(
l
-
1
)
+
D
′
(
l
-
1
)
P
(
l
)
D
(
l
-
1
)
≥
0
,
k
∈
N
T
,
G
(
l
-
1
)
G
†
(
l
-
1
)
H
(
l
-
1
)
-
H
(
l
-
1
)
=
0
.
Moreover, W
l
-
1
(
x
(
l
-
1
)
)
=
∑
k
=
l
-
1
T
-
1
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
[
x
′
(
T
)
(
S
-
P
(
T
)
)
x
(
T
)
]
+
x
′
(
l
)
P
(
l
)
x
(
l
).
Sufficiency part: let(39)
L
(
P
(
k
+
1
)
)
=
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
-
P
k
+
C
′
k
P
k
+
1
C
k
+
Q
k
,
M
(
P
(
k
+
1
)
)
=
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
ρ
x
u
k
C
′
k
P
k
+
1
D
k
,
N
(
P
(
k
+
1
)
)
=
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
+
1
)
D
(
k
)
.
Assume P
(
1
)
,
…
,
P
(
T
) satisfy
(40)
L
P
k
+
1
M
P
k
+
1
M
′
P
k
+
1
N
P
k
+
1
≥
0
,
for k
=
0,1
,
…
,
T
-
1 and P
(
T
)
≤
S
+
μ
N
′
N.
As in the preceding,(41)
J
x
0
,
u
0
,
…
,
u
T
-
1
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
k
R
k
u
k
+
x
′
(
T
)
S
x
(
T
)
=
∑
k
=
0
T
-
1
E
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
(
k
)
R
(
k
)
u
(
k
)
‍
+
x
′
k
+
1
P
k
+
1
x
k
+
1
-
x
′
(
k
)
P
(
k
)
x
(
k
)
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
-
P
(
k
)
+
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
x
(
k
)
+
2
x
′
(
k
)
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
ρ
x
u
k
C
′
(
k
)
P
(
k
+
1
)
D
(
k
)
u
(
k
)
+
u
′
(
k
)
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
)
D
(
k
)
u
(
k
)
+
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
[
x
′
(
T
)
(
S
-
P
(
T
)
)
x
(
T
)
+
x
0
′
P
(
0
)
x
(
0
)
]
.
By Lemma8, we get that
(42)
J
x
0
,
u
0
,
…
,
u
T
-
1
≥
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
x
0
′
P
(
0
)
x
(
0
)
-
μ
tr
⁡
M
.
In other words,
(43)
V
(
x
0
)
=
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
x
0
′
P
(
0
)
x
(
0
)
-
μ
tr
⁡
M
,
which implies that the LQ problem (1)–(3) is well posed.
We are now equipped to present the main result in this section.Theorem 14.
The following assertions are equivalent.(i)
The LQ problem (1)–(3) is attainable.
(ii)
The LQ problem (1) – (3) is well posed.
(iii)
The GDRE (12) is solvable.In addition, the feedback control law is achieved by
(44)
u
k
=
-
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
k
P
k
+
1
D
k
†
×
B
′
k
P
k
+
1
A
k
+
ρ
x
u
k
D
′
(
k
)
P
×
(
k
+
1
)
C
(
k
)
B
′
k
P
k
+
1
A
k
x
(
k
)
,
where P
(
0
)
,
…
,
P
(
T
) are solutions to the GDRE (12) and k
∈
N
T.Proof.
By Theorem13, it is easy to have that (ii) is equivalent to (iii). Our objective is to show that (i) is equivalent to (iii). From Theorem 11, we only need to show (iii) ⇒ (i).
Suppose the GDRE (12) admits a pair of solutions (
P
(
1
)
,
…
,
P
(
T
)
,
μ
). By the same way as Theorem 11, the following can be proved:
(45)
J
x
0
,
u
0
,
…
,
u
T
-
1
=
∑
k
=
0
T
-
1
‍
E
x
′
k
Q
k
x
k
+
u
′
k
R
k
u
k
+
x
′
(
T
)
S
x
(
T
)
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
(
k
)
R
(
k
)
u
(
k
)
+
x
′
(
k
+
1
)
P
(
k
+
1
)
x
(
k
+
1
)
-
x
′
(
k
)
P
(
k
)
x
(
k
)
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
Q
(
k
)
-
P
(
k
)
+
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
x
(
k
)
+
2
x
′
(
k
)
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
ρ
x
u
k
C
′
(
k
)
P
×
(
k
+
1
)
D
(
k
)
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
u
(
k
)
+
u
′
(
k
)
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
)
D
(
k
)
u
(
k
)
+
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
=
∑
k
=
0
T
-
1
‍
E
u
k
+
G
†
k
H
k
x
k
′
G
(
k
)
×
u
(
k
)
+
G
†
(
k
)
H
(
k
)
x
(
k
)
u
k
+
G
†
k
H
k
x
k
′
+
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
.
So, the optimal value V
(
x
0
)
=
∑
k
=
0
T
-
1
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
x
0
′
P
(
0
)
x
0
-
μ
tr
⁡
(
M
) and the feedback u
(
k
)
=
-
G
†
(
k
)
H
(
k
)
x
(
k
).


## 4. Relation between Optimal Synthesis and GDRE

In this section, we first attempt to verify that any optimal control can be denoted by virtue of the solution of the GDRE (12) with two degrees of freedom and the optimal cost is given.Theorem 15.
Assume that the GDRE (12) admits a solution. Then the optimal control satisfies the following:
(46)
u
Y
(
k
)
,
Z
(
k
)
(
k
)
=
-
G
†
k
H
k
+
Y
k
-
G
†
k
G
k
Y
k
x
k
+
Z
(
k
)
-
G
†
(
k
)
G
(
k
)
Z
(
k
)
,
where Y
(
k
)
,
k
∈
N
T are arbitrary random variables defined on the probability space Ω
,
F
,
μ. And the optimal cost value is given by
(47)
V
(
x
0
)
=
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
x
0
′
P
(
0
)
x
0
-
μ
tr
⁡
(
M
)
,
where P
(
0
)
,
…
,
P
(
T
-
1
) solve the GDRE (12).Proof.
Suppose the GDRE (12) admits solutions P
(
0
)
,
…
,
P
(
T
-
1
). As the preceding calculation, we have
(48)
J
(
x
0
,
u
(
0
)
,
…
,
u
(
T
-
1
)
)
=
∑
k
=
0
T
-
1
‍
E
+
u
′
(
k
)
R
(
k
)
u
(
k
)
+
u
′
(
k
)
R
(
k
)
u
(
k
)
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
(
k
)
R
(
k
)
u
(
k
)
+
x
′
(
T
)
S
x
(
T
)
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
Q
(
k
)
x
(
k
)
+
u
′
(
k
)
R
(
k
)
u
(
k
)
+
x
′
(
k
+
1
)
P
(
k
+
1
)
x
(
k
+
1
)
-
x
′
(
k
)
P
(
k
)
x
(
k
)
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
=
∑
k
=
0
T
-
1
‍
E
x
′
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
Q
(
k
)
-
P
(
k
)
+
A
′
(
k
)
P
(
k
+
1
)
A
(
k
)
+
C
′
(
k
)
P
(
k
+
1
)
C
(
k
)
x
(
k
)
+
2
x
′
(
k
)
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
ρ
x
u
k
C
′
(
k
)
P
×
(
k
+
1
)
D
(
k
)
A
′
(
k
)
P
(
k
+
1
)
B
(
k
)
u
(
k
)
+
u
′
(
k
)
+
D
′
(
k
)
P
(
k
)
D
(
k
)
+
D
′
(
k
)
P
(
k
)
D
(
k
)
R
(
k
)
+
B
′
(
k
)
P
(
k
+
1
)
B
(
k
)
+
D
′
(
k
)
P
(
k
)
D
(
k
)
u
(
k
)
+
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
0
x
0
=
∑
k
=
0
T
-
1
‍
E
x
′
k
H
′
(
k
)
G
†
(
k
)
H
(
k
)
x
(
k
)
+
2
x
′
k
H
′
(
k
)
u
(
k
)
+
u
′
(
k
)
G
(
k
)
u
(
k
)
+
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
.
LetF
1
(
k
)
=
G
†
(
k
)
G
(
k
)
Y
(
k
)
-
Y
(
k
) and F
2
(
k
)
=
G
†
(
k
)
G
(
k
)
Z
(
k
)
-
Z
(
k
); then
(49)
G
(
k
)
F
1
(
k
)
=
G
(
k
)
F
2
(
k
)
=
0
.
So,J
(
x
0
,
u
(
0
)
,
…
,
u
(
T
-
1
)
) can be rewritten as
(50)
J
(
x
0
,
u
(
0
)
,
…
,
u
(
T
-
1
)
)
=
∑
k
=
0
T
-
1
‍
E
u
(
k
)
+
G
†
(
k
)
H
(
k
)
+
F
1
k
x
(
k
)
+
F
2
k
u
(
k
)
+
G
†
(
k
)
H
(
k
)
+
F
1
k
′
G
(
k
)
×
u
(
k
)
+
G
†
k
H
k
+
F
1
k
x
k
+
F
2
(
k
)
u
(
k
)
+
G
†
(
k
)
H
(
k
)
+
F
1
k
+
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
.
Because ofG
(
k
)
≥
0, we immediately obtain that
(51)
V
(
x
0
)
=
∑
k
=
0
T
-
1
‍
tr
⁡
V
k
P
k
+
1
+
x
0
′
P
(
0
)
x
0
-
μ
tr
⁡
(
M
)
,
and the control u
(
k
)
=
-
(
G
†
(
k
)
H
(
k
)
+
F
1
(
k
)
)
x
(
k
)
+
F
2
(
k
).
Now, we are interested in arbitrary control sequence(
u
~
(
0
)
,
…
,
u
~
(
T
-
1
)
) which minimizes the cost function J. So we deduce that
(52)
J
(
x
0
,
u
~
(
0
)
,
…
,
u
~
(
T
-
1
)
)
=
∑
k
=
0
T
-
1
‍
E
u
~
k
+
G
†
k
H
k
x
k
′
G
(
k
)
×
u
~
k
+
G
†
k
H
k
x
k
u
~
k
+
G
†
k
H
k
x
k
′
+
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
=
∑
k
=
0
T
-
1
‍
tr
⁡
[
V
(
k
)
P
(
k
+
1
)
]
+
E
x
′
T
S
-
P
T
x
T
+
x
0
′
P
(
0
)
x
0
.
Thus there must be
(53)
E
u
~
k
+
G
†
k
H
k
x
k
′
G
(
k
)
×
u
~
k
+
G
†
k
H
k
x
k
u
~
k
+
G
†
k
H
k
x
k
′
=
0
.
By G
(
k
)
≥
0, the above is equivalent to the following:
(54)
G
(
k
)
u
~
(
k
)
+
G
†
(
k
)
H
(
k
)
x
(
k
)
=
0
.
From Lemma10, it follows that
(55)
u
~
(
k
)
=
-
G
†
(
k
)
H
(
k
)
x
(
k
)
+
Z
(
k
)
-
G
†
G
(
k
)
Z
(
k
)
.
The following numerical example illustrates the effectiveness of our theoretical results.Example 16.
The coefficients of the systems (1)-(2) are as follows:
(56)
A
0
=
1
0
0
2
,
B
0
=
2
1
,
C
(
0
)
=
-
1
0
0
1
,
D
(
0
)
=
1
-
1
,
V
(
0
)
=
2
1
1
1
,
ρ
x
u
(
0
)
=
-
2
,
x
0
=
1
2
3
4
,
Q
0
=
-
1
1
,
S
=
1
0
0
1
,
R
(
0
)
=
-
13
,
a
11
=
a
22
=
1
,
a
12
=
a
21
=
0
,
ξ
1
=
ξ
2
=
1
.
We solve the corresponding GDRE (12) and calculate the optimal cost value:
(57)
P
1
=
2
0
0
2
,
H
0
=
8
8
,
G
0
=
1
,
P
0
=
-
65
-
64
-
64
-
52
,
μ
=
1
,
V
(
x
0
)
=
17
.


## 5. Conclusion

This paper mainly studies linear quadratic optimal control with inequality constraint for discrete-time indefinite stochastic systems. With the aid of the KKT theorem, we present a necessary and sufficient condition under which the problem is well posed and a state feedback control can be derived. Moreover, it is shown that the solvability of the GDRE, the well-posedness, and the attainability of the LQ problem are equivalent to each other. Finally, we give a structure of all optimal controls. To some degree, the previous results on stochastic LQ control without constraint can be regarded as corollaries of the main theorems of this paper.



---

*Source: 476545-2015-01-22.xml*