1

ME233 Advanced Control II
Lecture 10
Infinite-horizon LQRPART I(ME232 Class Notes pp. 135-137)

2

LTI Optimal regulators (review)• State space description of a discrete time LTI• Find optimal control• That drives the state to the origin

Finite Horizon LQ optimal regulator(review)

LTI system:

We want to find the optimal control sequence:

which minimizes the cost functional:

3

4

LQ Cost Functional (review)•total number of steps—“horizon”•penalizes the final statedeviation from the origin•

penalizes the transient state deviationfrom the origin and the control effort

symmetric

5

Finite-horizon LQR solution (review)

Where P(k) is computed backwards in time using thediscrete Riccati difference equation :6Properties of Matrix P(k) (review)
P(k) satisfies:
1)
2)(symmetric)(positive semi-definite)

7

Example – Double Integrator
Double integrator with ZOH and sampling time T =1:
u(k)

ZOH

u(t)

1
sv(t)

1
Tx(t)

s

position
velocity

Tx(k)
v(k)

8

Example – Double Integrator

LQR cost:

Choose:only penalize
position x1
and control u9Example – Double Integrator (DI)Compute P(k) for an arbitrary

Computing backwards:and N.

10Example – DI Finite Horizon Case 1• N = 10 , R = 10,
98P22(k)P(1,1), P(1,2),P(2,2)

7
6
54P12(k)3P11(k)2
100246k

810

11Example – DI Finite Horizon Case 2• N = 30 , R = 10,
9P(1,1), P(1,2),P(2,2)

8P22(k)7
6
54P12(k)3P11(k)2
10051015k20

2530

12Example – DI Finite Horizon Case 3• N = 30 , R = 10,
9P(1,1), P(1,2),P(2,2)

8P22(k)7
6
5
4P12(k)3P11(k)

2
100

5

10

15

k20

25

30

13

Example – DI Finite Horizon
Observation:
In all cases, regardless of the choice ofwhen the horizon, N, is sufficiently large
the backwards computation of the Riccati Eq.
always converges to the same solution:

14

Infinite-Horizon LQ regulator
LTI system:

LQR that minimizes the cost:• We now consider the limiting behavior when

15

Infinite Horizon (IH) LQ regulator
Consider the limiting behavior when
LTI system:

Optimal control:

Riccati equation:

16

Infinite Horizon LQ regulator question 1
Consider the limiting behavior when
1) When does there exist a BOUNDED limiting solution

to the Riccati Eq.

for all choices of?

17

Infinite Horizon LQ regulator question 2
Consider the limiting behavior when
2) When does there exist a UNIQUE limiting solution

to the Riccati Eq.

regardless of the choice of?

18

Infinite Horizon LQ regulator question 3
Consider the limiting behavior when
3) When does the limiting solution

to the Riccati Eq.
yield an asymptotically stable closed loop system?
is Schur(all eigenvalues inside unit circle)

19

LQ regulator Cost

Define the square root of, i.e.

Define the matrices C and D such that20

LQ regulator Cost

• Define the fictitious output p(k) such that

21

Infinite Horizon LQ optimal regulator
LTI system:

Find optimal control which minimizes the cost functional:

22

Stabilizability Assumption
We are only interested in the case where theclosed-loop dynamics are asymptotically stable
If (A,B) is not stabilizable, then there does notexist a control scheme that results is
asymptotically stable closed-loop dynamics
For the infinite horizon optimal LQR
problem, we always assume that (A,B) isstabilizable

23

Theorem 1 : Existence of a bounded P∞
Letbe stabilizable(uncontrollable modes are asymptotically stable)

Then, for, as
the “backwards” solution of the Riccati Eq.

converges to a BOUNDED limiting solutionthat satisfies the algebraic Riccati equation (DARE):

24

Theorem 1 : Notes•

Theorem-1 only guarantees the existence ofa bounded solution
to the algebraic
Riccati Equation•The solution may not be unique, i.e. different
final conditions
may result indifferent limiting solutions P∞ or may noteven yield a limiting solution!

Theorem 2 : Existence and uniqueness of apositive definite asymptotic stabilizing solution
If (A,B) is stabilizable and the state-spacerealization C(zI - A)-1B + D has no transmission
zeros, then

1) There exists a unique, boundedsolution
to the DARE

2) The closed-loop plant
is asymptotically stable25

26

Theorem 3 : Existence of a stabilizing solution
If (A,B) is stabilizable and the state-spacerealization C(zI - A)-1B + D has no transmission
zeros satisfying, then

1) There exists a unique, boundedsolution
to the DARE

2) The closed-loop plant
is asymptotically stable

27

Theorem 4 : A different approach
The discrete algebraic Riccati equation (DARE) has asolution for whichis Schur if and only if(A,B) is stabilizable and the state-space realization
has no transmission zeros on the unit circle.
Moreover,is the optimal control
policy that achieves asymptotic stability

28

Special case: S=0
It turns out that the transmission zeros ofcorrespond to the unobservable modes of(This will be assigned as a homework problem)

In Theorems 2 and 3, the transmission zeros
condition becomes an observability/detectability
condition

Theorem 2 : Existence and uniqueness of a positive
definite asymptotic stabilizing solution, S = 0
If (A,B) is stabilizable and (C,A) is observable, then

1) There exists a unique, bounded
solutionto the DARE

2) The closed-loop plant
is asymptotically stable

29

30

Theorem 3 : Existence of a stabilizing solution, S = 0
If (A,B) is stabilizable and (C,A) is detectable, then1) There exists a unique, boundedsolution
to the DARE2) The closed-loop plant
is asymptotically stable

31

Theorem 4 : A different approach, S = 0
The discrete algebraic Riccati equation (DARE) has asolution for whichis Schur if and only if(A,B) is stabilizable and (C,A) has no unobservable
modes on the unit circle.
Moreover,is the optimal control
policy that achieves asymptotic stability

32

Notes, S=0
When (A,B) stabilizable and (C,A) observable ordetectable, the infinite-horizon cost (N →∞)becomes•

The closed-loop plant is asymptotically stable•Solution of the DARE is unique, independent of P(N)33

Explanation: why is stabilizability needed
not stabilizable
there are unstable uncontrollable modes

there might be some initial conditions such thatsince the optimal cost is given by

34

Explanation: why is detectability is needed, S=0
not detectable
there are unstable unobservable modes

these modes do not affect the optimal cost

no need to stabilize these modes

35

Explanation: why is observability needed
The DARE can be written in the Joseph stabilized
form:(closed-loop matrix)
define36

Explanation: why is observability needed
Joseph stabilized ARE:

Looks like a Lyapunov equation and, in fact, itis the Lyapunov equation for the observability
Grammian of the pair37

Explanation: why is observability needed
Joseph stabilized ARE:

It can be shown that:observable

observablestabilizable

andasymptoticallystable (Schur)38

Example – Double Integrator
LQR

39

Example – Double Integrator
Penalize position in the infinite horizon cost functional:

Observable

Controllable

40

Example - Steady State Solution• The steady state solution of the DARE:• Use matlab function dare

• Get steady state answer:

41

Example - Infinite Horizon LQ Regulator• The control law is given by:

Answer • Closed-loop poles are the eigenvalues of• Use matlab command:>> abs(eig(Ac))
ans =is Schur

0.6736
0.6736

42Summary• Convergence of LQR as horizon––

stabilizable
detectable• Infinite-horizon LQR• Unique, positive definite solution of algebraic
Riccati equation• Closed-loop system is asymptotically stableAdditional Material (you are notresponsible for this)• Solutions of Infinite Horizon LQR using theHamiltonian Matrix– (see ME232 class notes by M. Tomizuka)• Strong and stabilizing solutions of the discrete time
algebraic Riccati equation (DARE)• Some additional results on the convergence of theasymptotic convergence of the discrete time Riccati
equation (DRE)

43

44

Infinite Horizon LQ optimal regulator
Consider the nth order discrete time LTI system:

We want to find the optimal control which minimizes thecost functional :Assume:•is controllable or asymptotically stabilizable•

is observable or asymptotically detectable

45

Infinite Horizon LQR Solution:

Discrete time Algebraic Riccati (DARE) equation:

46

Solution of the DARE
DARE:

1) Assume that A is nonsingular and define the 2n x 2n
Backwards Hamiltonian matrix:2) Compute its first n eigenvalues ():47

Solution of the DARE
• The first n eigenvalues of H are the eigenvalues ofwhereand are all inside the unit circle,(I.e. asymptotically stable)• The remaining eigenvalues of H satisfy:

48

Solution of the DARE
3) For each unstable eigenvalue of H
(outside the unit circle), compute its associated eigenvector :4)

Define the n x n matrices:49

Solution of the ARE5) Finally, P is computed as follows:•Matlab command dare: (Discrete ARE)

50

Strong Solution of the DARE
A solutionof the DARE

is said to be a strong solution
if the corresponding closed loop matrix Ac

has all its eigenvalues on or inside the unit circle.

51

Stabilizing Solution of the DAREA strong solutionof the DARE

is said to be stabilizing
if the corresponding closed loop matrix Ac

is Schur, i.e. it has all its eigenvalues inside theunit circle.

52

Theorem – Solutions to the DARE
Provided that (A,B) is stabilizable, theni. the strong solution of the DARE exists andis unique.
ii. if (C,A) is detectable, the strong solution is
the only nonnegative definite solution of theDARE.
iii. if (C,A) is has no unobservable modes on
the unit circle, then the strong solution
coincides with the stabilizing solution.
iv. if (C,A) has an unobservable mode on theunit circle, then there is no stabilizing
solution.

53

Theorem – Solution to the DARE
Provided that (A,B) is stabilizable, thenv. if (C,A) has an unobservable mode insideor on the unit circle, then the strong solution
is not positive definite.
vi. if (C,A) has an unobservable mode outside
the unit circle, then as well as the thestrong solution, there is at least one
nonnegative definite solution of the DARE
S. W. Chan, G.C. Goodwin and K.S. Sin, “Convergenceproperties of the Riccati difference equation in optimal filteringof nonstabilizable systems, “IEEE Trans. of Automatic Control
AC-29 (1984) pp 110-118.

54

Theorems - convergence of the DRE
Consider the “backwards” solution of the discrete time
Riccati Equation1)
i.
ii.

Subject to(A,B) is stabilizable and (C,A) is detectable,

then, asP(k) converges exponentially to aunique stabilizing solution P∞ of the DARE

55

Theorems - convergence of the DRE
Consider the “backwards” solution of the discrete time
Riccati Equation2)
i.
ii.
iii.

Subject to(A,B) is stabilizable(C,A) is has no unobservable modes on the unit circle

then, asP(k) converges exponentially to aunique stabilizing solution P∞ of the DARE

56

Theorems - convergence of the DRE
Consider the “backwards” solution of the discrete timeRiccati Equation
3)
i.
ii.

Subject to(A,B) is controllable
orthen, asP(k) converges to a unique strong
solution P∞ of the DARE

S. W. Chan, G.C. Goodwin and K.S. Sin, “Convergenceproperties of the Riccati difference equation in optimal filteringof nonstabilizable systems, “IEEE Trans. of Automatic Control
AC-29 (1984) pp 110-118.