1ME 233 Advanced Control II
Lecture 11
Kalman Filters Stationary Properties
andLQR-KF Duality(ME233 Class Notes pp.KF1-KF6)

2

Summary• Stationary Kalman filters (KF):– KF algebraic Riccati equation– Convergence properties• Kalman filter / LQR duality• KF return difference equality– Reciprocal root locus– Guaranteed robustness margins

3

Stochastic State Estimation
Linear system contaminated by noise:

WBw
U

V+
B

(zI-A ) -1

X+CY

Two random disturbances:• Input noise w(k) - contaminates the state x(k)• Measurement noise v(k) - contaminates theoutput y(k)4Stochastic state modelState estimation of LTI system:

Where:•known control input•

Gaussian, uncorrelated, zero mean, input noise•

Gaussian, uncorrelated, zero mean, meas. noise•

Gaussian5

Assumptions (review)•

Initial conditions:•

Noise properties:

Zero-mean
Gaussian
uncorrelated
noises6

Kalman Filter Solution V-1 (review)
A-posteriori state observer structure:

7

Kalman Filter Solution V-1 (review)• A-posteriori estimator as output-1BX

YC(zI-A )UC

B++-oz-1

AX+oY~YoF(k)+-+ X

8Kalman Filter Solution V-2 (review)
A-priori state observer structure:9Kalman Filter Solution V-2 (review)• Same structure as deterministic a-prioriobserver

X-1B(zI-A )U

B--1(zI-A )C

 +L+Y-XCY10

Kalman Filter State Space (review)

Kalman Filter (KF) Properties(review)
The KF a-priori output error (a-priori output residual)

is often called the innovation
it contains only the “new information” in y(k)Moreover,i.e.

is an uncorrelated RVS

11

12

KF as an innovations filter (review)
For the figure on the next slide, we will assume without
loss of generality that the control input is zero, i.e.
• Plant:• Kalman filter V-2:

13

KF as an innovations filter (review)
plantW(z)+C  (z) Bw

Uncorrelated
noise input

Kalman filter
Y(z) +-o^
Y (z)Correlated
noise output~oY (z)C  (z) L

Uncorrelated
noise output

Kalman Filter (KF) Properties(review)• The KF is a linear time varying estimator.
• The KF is the optimal state estimator whenthe input and measurement noises areGaussian.
• The KF is still the optimal linear stateestimator even when the input andmeasurement noises are not Gaussian.
• The KF covariance Riccati equation isiterated in a forward manner, rather than in a
backwards manner as in the LQR.

14

15

Steady State Kalman Filter•

Assume now that we want to estimate thestate under zero-mean, stationary input andoutput Gaussian white noise, I.e.

WSSGaussian
Noise16A priori estimation error dynamics

Proof:

Subtracting equations gives

17

Steady state Kalman filter, question 11) When does there exist a BOUNDED limiting solution

to the Riccati Eq.

for each choice of?

18

Steady state Kalman filter, question 22) When does there exist a UNIQUE limiting solution

to the Riccati Eq.

regardless of the choice of?

19

Steady state Kalman filter, question 33) When does the limiting solution

to the Riccati Eq.
yield asymptotically stable estimation error dynamics?
is Schur(all eigenvalues inside unit circle)

20

Detectability Assumption
We are only interested in the case where theestimation error dynamics are asymptoticallystableIf (C,A) is not detectable, then there does notexist a estimator that results is asymptoticallystable estimation error dynamics
For the stationary Kalman filter, we always
assume that (C,A) is detectable

21

Theorem 1 : Existence of a bounded M∞
Let

be detectable(unobservable modes are asymptotically stable)

Then, for, as
the solution of the Riccati Eq.

converges to a BOUNDED limiting solution M∞
that satisfies the algebraic Riccati equation (DARE):

22

Theorem 1 : Notes•Theorem 1 only guarantees the existence ofa bounded solution M∞ to the algebraic
Riccati Equation•The solution may not be unique.

•Different initial conditions
may result in different limiting solutions M∞
or may yield no limiting solution at all!

Theorem 2 : Existence and uniqueness of a positive
definite asymptotic stabilizing solution
If (C,A) is detectable and (A,BwW 1/2) is controllable

1) There exists a unique, boundedsolution
to the DARE

2) The estimation error dynamics are
asymptotically stable23

24

Theorem 3 : Existence of a stabilizing solution
If (C,A) is detectable and (A,BwW 1/2) is stabilizable

1) There exists a unique, boundedsolution
to the DARE2) The estimation error dynamics are
asymptotically stable

25

Theorem 4: A different approach
The discrete algebraic Riccati equation (DARE) has asolution for whichis Schurif and only if

is detectable and the matrix pairhas no uncontrollable modes on the unit circle.

26

Kalman Filter Solution V-1
A-posteriori state observer structure:

is Schur

27

Kalman Filter Solution V-2
A-priori state observer structure:

is Schur

28

Kalman Filter State Spaceis Schur

29

Kalman Filter (KF) Properties
The KF a-priori output error (a-priori output residual)

is often called the innovation
it contains only the “new information” in y(k)Moreover,i.e.

is white

30

KF as an innovations filter
For the figure on the next slide, we will assume without
loss of generality that the control input is zero, i.e.
• Plant:• Kalman filter V-2:

31

KF as an innovations (whitening) filter
plant

W(z)+C  (z) Bw

Whitenoise input

Kalman filter
Y(z) +-o^
Y (z)Colorednoise output~oY (z)C  (z) L

Whitenoise output

32

Output Y(k) is colored noise• Plant:

W(z)Bw(zI - A)

-1+X(z)CY(z)33

Output Y(k) is colored noise• Plant:W(z)+C  (z) Bw

Y(z)

34

KF as an innovations filter• A-priori KF:~oY (z)Y(z) +-o^
X (z)o^Y (z)C-1(zI - A)L35

KF as an innovations filter• A-priori KF:~oY (z)Y(z) +-o^Y (z)C  (z) L36

KF as an innovations filter• Plant• A-priori KF:

W(z)+C  (z) Bw~oY (z)Y(z) +-o^Y (z)C  (z) L37

Y(k) Power spectrum
Power spectrum ofW(z)C  (z) Bw

Yw(z) +Y(z)w(k) and v(k) are uncorrelated!

38

Y(k) Power spectrum
Power spectrum ofW(z)C  (z) Bw

Yw(z) +Y(z)yw(k) and v(k) are also uncorrelated!

39

Y(k) Power spectrum
Power spectrum ofW(z)C  (z) Bw

Yw(z) +Y(z)40

Y(k) Power spectrum
Power spectrum ofW(z)C  (z) Bw

Yw(z) +Y(z)v(k) is white noise41

Y(k) Power spectrum

G(k)

42

Y(k) Power spectrum
Power spectrum ofW(z)C  (z) Bw

Yw(z) +w(k) is white noise

Y(z)

43

Y(k) Power spectrum
Power spectrum ofW(z)C  (z) Bw

Yw(z) +Y(z)44

KF as an innovations filter
Power spectrum ofY(z) +-o^Y (z)C  (z) L~oY (z)45

KF as an innovations filter
Combining two results:

W(z)+C  (z) Bw~oY (z)Y(z) +-ando^
Y (z)C  (z) L

46

KF as an innovations filter
Combining two results:W(z)+C  (z) Bw~oY (z)Y(z) +-o^Y (z)C  (z) L47

KF as an innovations filter
Recall what Theorem part 3) says about the a-priori
output error (the innovation sequence)is also white noise!!

48

KF as an innovations filter
Recall what Theorem part 3) says about the a-priori
output error (the innovation sequence)

49

KF as a innovations (whitening) filter•plantW(z)+C  (z) Bw

Kalman filter
Y(z) +-o^Y (z)C  (z) L

whitening filter~oY (z)50

KF as an innovations filter
W(z)therefore,+C  (z) Bw~oY (z)Y(z) +-o^Y (z)C  (z) L

51

KF return difference equality

52

Kalman Filter & LQR Duality
Recall Steady state LQR:

53

Note:
We need to distinguish between:• LQR: state cost weight• KF: output matrix

54

Kalman Filter & LQR Duality
Infinite-horizon LQR Closed-loop dynamics:

55

Kalman Filter & LQR Duality
Steady State KF Estimation error dynamics

56

Kalman Filter & LQR Duality
Let’s compare the DAREs:
LQR

KF

57

Kalman Filter & LQR Duality
Let’s compare the AREs:
LQR

KF

58

Kalman Filter & LQR Duality
Let’s compare the AREs:
LQR

KF

59

Kalman Filter & LQR Duality
Let’s compare the AREs:
LQR

KF

60

Kalman Filter & LQR Duality
Let’s compare the AREs:
LQR

KF

61

Kalman Filter & LQR Duality
Let’s compare the Feedback gains:
LQR

KF

62

Kalman Filter & LQR Duality
Let’s compare the Feedback gains:
LQR

KF

63

Kalman Filter & LQR Duality
Comparing ARE’s and feedback gains, we obtain the following
dualityduality
LQR

KF

PM

A

AT

B

CT

R

V

CQT

B’w = BwW 1/2

K

LT(A-BK)(A-LC)T64

Kalman Filter & LQR Duality
duality
LQR

KF

P

M

A

AT

B

CT

R

V

CQT

B’w = BwW 1/2

K

LT(A-BK)(A-LC)T65

Kalman Filter & LQR Duality
duality
LQR

KF

P

M

AAT

B

CT

R

V

CQT

B’w = BwW 1/2

KLT(A-BK)(A-LC)T66

Kalman Filter & LQR Duality• It is possible to use duality to prove
theorems 1-4 for stationary Kalman filtersfrom the corresponding theorems from theinfinite horizon LQR• The following slides give an outline of how todo this• The main idea is to design an infinite horizon
LQR for a fictitious system

67

Theorems 1-4 proof methodology• Consider the LQR problem:• Solution:

68

Theorems 1-4 proof methodology• The solution of the Riccati equationis• Use LQR convergence results foras
to yield convergence results foras

69

Theorems 1-4 proof methodology• Other key ideas in proofs– (AT,CT) stabilizable iff (C,A) detectable– Unobservable modes of ((BwW1/2)T, AT) are
the uncontrollable modes of (A, BwW1/2)– AT-CTLT is Schur iff A-LC is Schur

70

Steady State LQR
Theorem 1):
If the pair [A, B] is controllable (or stabilizable), thesolution of the DRE

withconverges, as, to a constant that satisfies

71

Steady State LQR
Theorem 2:
If the pair [ A, Cq ] is observable (or detectable)
Then [A,B] is controllable (or stabilizable) if and only if:
1) The solution ofwithConverges to a unique stationary solution P, which satisfies

72

Steady State LQ
Theorem 2: (continuation)2) P is positive definite (semi-definite)3) The close loop matrix
is Schur

73

Kalman Filter & LQR Duality
LQR

KF

PM

AAT

B

CT

R

V

CQT

B’w = BwW 1/2

K

LT(A-BK)(A-LC)TKF

LQR
duality

KF

74

Kalman Filter & LQR Duality
LQR

KF

PM

A

AT

B

CT

R

V

CQT

B’w = BwW 1/2

K

LT(A-BK)(A-LC)TKF

LQR
duality

KF

75

Steady State Kalman Filter
Theorem 1:If the pair [A,C] is observable (or detectable):the solution ofwithConverges to a stationary solution, M, which satisfies

76

Steady State Kalman Filter
Theorem 2:
If the pair [ A, B’w ] is controllable (or stabilizable), where

Then [A,C] is observable (or detectable) if and only if:
1) The solution ofConverges to a unique stationary solution M, which satisfies

77

Steady State Kalman Filter
Theorem 2: (continuation)2) M is positive definite (semi-definite)3) The close loop matrix
is Schur

78

Steady State Kalman Filter
Theorem 3:
Under stationary noise and the conditions intheorems 1) and 2),

The observer a-priori residual (innovations)is white

79

KF as an innovations filter
We will assume, without loss of generality that the control
input is zero, I.e.

•Plant:80

KF as an innovations (whitening) filter•plantW(z)+C  (z) Bw

White noise
input

Kalman filter
Y(z) +-o^
Y (z)Colored noise
output~oY (z)C  (z) L

White noise
output81

KF as a innovations (whitening) filter•plantW(z)+C  (z) Bw

White noise
input

Kalman filterY(z) +-o^
Y (z)Colored noiseoutput~oY (z)C  (z) LWhite noiseoutput

Return difference equality for LQR(review)
Y (z)CQ+U(z)
B-Open loop transfer function:(zI - A)

-1

X(z)

Q

K

TF from U(z) to YQ(z):82Return difference equality for LQR(review)
Substituting,into

We obtain,

83

84

Kalman Filter & LQR Duality

LQR

KF

LQR

KF

PM

RV

A

AT

CQT

B’w = BwW 1/2

B

CTKLT

85

From,

KF return difference equality

we perform transpose operations and notice that:

This gives the desired result:

86

Kalman filter closed-loop eigenvalues• A-priori KF (for u(k) = 0)~oY (z)Y(z) +-o^Y (z)C  (z) L87Kalman filter closed-loop eigenvalues•KF closed-loop eigenvalues

88

Kalman filter return difference~oY (z)Y(z) +-o^Y (z)C  (z) L

Return difference:

89

Kalman filter return difference• Similar to the LQR case, we have that• KF closed-loop eigenvalues• KF open-loop eigenvalues

90

KF return difference equality
W(z)+C  (z) Bw~oY (z)Y(z) +-o^Y (z)C  (z) L

91

KF return difference equality (SISO)
Assume that both,W(z)+C  (z) Bw~oY (z)Y(z) +-o^Y (z)C  (z) L

92

KF root locus for SISO Systems
W(z)+C  (z) Bw~oY (z)Y(z) +-o^Y (z)C  (z) L

c.l. poleso.l. poleso.l. zeros
o.l. poles

93

KF root locus for SISO Systems

input noise intensitymeasurement noise intensity

94

KF Loop phase margins (SISO)~oY (z)Y(z) +-o^Y (z)C  (z) L

Utilizing LQR-KF duality,Therefore, a lower bound to the phase marginofis:

95

KF Loop gain margins (SISO)~oY (z)Y(z) +-o^Y (z)C  (z) L

Estimator was designed for
Estimator is guaranteed to remain asymptotically stable for96

Summary• Stationary Kalman filters (KF):– KF algebraic Riccati equation– Convergence properties• Kalman filter / LQR duality• KF return difference equality– Reciprocal root locus– Guaranteed robustness margins