Rayleigh-Ritz theorem

Rayleigh-Ritz theorem

Let

A

∈

𝐂

n

×

n

be a

Hermitian matrix

. Then its

eigenvectors

are the critical points (vectors) of the ”Rayleigh quotient”,
which is the real function

R

:

ℂ

n

\

{

𝟎

}

→

ℝ

R

⁢

(

𝐱

)

=

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

,

∥

𝐱

∥

≠

0

and its

eigenvalues

are its values at such critical points.

As a

consequence

, we have:

λ

m

⁢

a

⁢

x

=

max

∥

𝐱

∥

≠

0

⁡

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

and

λ

m

⁢

i

⁢

n

=

min

∥

𝐱

∥

≠

0

⁡

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

Proof:

First of all, let’s observe that for a hermitian matrix, the number

𝐱

H

⁢

A

⁢

𝐱

is a real one (actually,

<

𝐱

,

A

𝐱

≥

𝐱

H

A

𝐱

=

(

A

H

𝐱

)

H

𝐱

=

(

A

𝐱

)

H

𝐱

=

<

A

𝐱

,

𝐱

≥

<

𝐱

,

A

𝐱

>

∗

,
whence

<

𝐱

,

A

⁢

𝐱

≥

𝐱

H

⁢

A

⁢

𝐱

is real), so that the
Rayleigh quotient is real as well.

Let’s now compute the critical points

𝐱

¯

of the
Rayleigh quotient, i.e. let’s solve the equations system

d

⁢

R

⁢

(

𝐱

¯

)

d

⁢

𝐱

=

𝟎

T

. Let’s write

𝐱

=

𝐱

(

R

)

+

j

⁢

𝐱

(

I

)

,

𝐱

(

R

)

and

𝐱

(

I

)

being
respectively the real and imaginary part of

𝐱

. We have:

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

=

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

R

)

+

j

⁢

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

I

)

so that we must have:

d

⁢

R

⁢

(

𝐱

¯

)

d

⁢

𝐱

(

R

)

=

d

⁢

R

⁢

(

𝐱

¯

)

d

⁢

𝐱

(

I

)

=

𝟎

T

Using

derivatives

rules, we obtain:

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

R

)

=

d

d

⁢

𝐱

(

R

)

⁢

(

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

)

=

d

⁢

(

𝐱

H

⁢

A

⁢

𝐱

)

d

⁢

𝐱

(

R

)

⁢

𝐱

H

⁢

𝐱

-

𝐱

H

⁢

A

⁢

𝐱

⁢

d

⁢

(

𝐱

H

⁢

𝐱

)

d

⁢

𝐱

(

R

)

(

𝐱

H

⁢

𝐱

)

2

=

d

⁢

(

𝐱

H

⁢

A

⁢

𝐱

)

d

⁢

𝐱

(

R

)

-

R

⁢

(

𝐱

)

⁢

d

⁢

(

𝐱

H

⁢

𝐱

)

d

⁢

𝐱

(

R

)

𝐱

H

⁢

𝐱

⁢

.

Applying matrix calculus rules, we find:

d

⁢

(

𝐱

H

⁢

A

⁢

𝐱

)

d

⁢

𝐱

(

R

)

=

𝐱

H

⁢

A

⁢

d

⁢

𝐱

d

⁢

𝐱

(

R

)

+

𝐱

T

⁢

A

T

⁢

d

⁢

𝐱

∗

d

⁢

𝐱

(

R

)

=

𝐱

H

⁢

A

+

𝐱

T

⁢

A

T

=

𝐱

H

⁢

A

+

(

𝐱

H

⁢

A

H

)

∗

=

and since

A

=

A

H

,

=

𝐱

H

⁢

A

+

(

𝐱

H

⁢

A

)

∗

=

2

⁢

ℜ

⁡

(

𝐱

H

⁢

A

)

⁢

.

In a

similar

way, we get:

d

⁢

(

𝐱

H

⁢

𝐱

)

d

⁢

𝐱

(

R

)

=

2

⁢

ℜ

⁡

(

𝐱

H

)

⁢

.

Substituting, we obtain:

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

R

)

=

2

⁢

ℜ

⁡

(

𝐱

H

⁢

A

)

-

R

⁢

(

𝐱

)

⁢

ℜ

⁡

(

𝐱

H

)

𝐱

H

⁢

𝐱

and, after a

transposition

, equating to the null

column vector

,

𝟎

=

(

ℜ

⁡

(

𝐱

¯

H

⁢

A

)

-

R

⁢

(

𝐱

¯

)

⁢

ℜ

⁡

(

𝐱

¯

H

)

)

T

=

=

ℜ

⁡

(

A

T

⁢

𝐱

¯

∗

)

-

R

⁢

(

𝐱

¯

)

⁢

ℜ

⁡

(

𝐱

¯

∗

)

=

ℜ

⁡

(

(

A

H

⁢

𝐱

¯

)

∗

)

-

R

⁢

(

𝐱

¯

)

⁢

ℜ

⁡

(

𝐱

¯

∗

)

=

=

ℜ

⁡

(

(

A

⁢

𝐱

¯

)

∗

)

-

R

⁢

(

𝐱

¯

)

⁢

ℜ

⁡

(

𝐱

¯

∗

)

=

ℜ

⁡

(

A

⁢

𝐱

¯

)

-

R

⁢

(

𝐱

¯

)

⁢

ℜ

⁡

(

𝐱

¯

)

and, since

R

⁢

(

𝐱

)

is real,

ℜ

⁡

(

A

⁢

𝐱

¯

-

R

⁢

(

𝐱

¯

)

⁢

𝐱

¯

)

=

𝟎

Let’s then evaluate

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

I

)

:

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

I

)

=

d

d

⁢

𝐱

(

I

)

⁢

(

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

)

=

d

⁢

(

𝐱

H

⁢

A

⁢

𝐱

)

d

⁢

𝐱

(

I

)

⁢

𝐱

H

⁢

𝐱

-

𝐱

H

⁢

A

⁢

𝐱

⁢

d

⁢

(

𝐱

H

⁢

𝐱

)

d

⁢

𝐱

(

I

)

(

𝐱

H

⁢

𝐱

)

2

=

d

⁢

(

𝐱

H

⁢

A

⁢

𝐱

)

d

⁢

𝐱

(

I

)

-

R

⁢

(

𝐱

)

⁢

d

⁢

(

𝐱

H

⁢

𝐱

)

d

⁢

𝐱

(

I

)

𝐱

H

⁢

𝐱

⁢

.

Applying again matrix calculus rules, we find:

d

⁢

(

𝐱

H

⁢

A

⁢

𝐱

)

d

⁢

𝐱

(

I

)

=

𝐱

H

⁢

A

⁢

d

⁢

𝐱

d

⁢

𝐱

(

I

)

+

𝐱

T

⁢

A

T

⁢

d

⁢

𝐱

∗

d

⁢

𝐱

(

I

)

=

j

⁢

𝐱

H

⁢

A

-

j

⁢

𝐱

T

⁢

A

T

=

j

⁢

(

𝐱

H

⁢

A

-

(

𝐱

H

⁢

A

H

)

∗

)

=

and since

A

=

A

H

,

=

j

⁢

(

𝐱

H

⁢

A

-

(

𝐱

H

⁢

A

)

∗

)

=

j

⁢

(

2

⁢

j

⁢

ℑ

⁡

(

𝐱

H

⁢

A

)

)

=

-

2

⁢

ℑ

⁡

(

𝐱

H

⁢

A

)

⁢

.

In a similar way, we get:

d

⁢

(

𝐱

H

⁢

𝐱

)

d

⁢

𝐱

(

I

)

=

j

⁢

𝐱

H

-

j

⁢

𝐱

T

=

j

⁢

(

𝐱

H

-

(

𝐱

H

)

∗

)

=

j

⁢

(

2

⁢

j

⁢

ℑ

⁡

(

𝐱

H

)

)

=

-

2

⁢

ℑ

⁡

(

𝐱

H

)

⁢

.

Substituting, we obtain:

d

⁢

R

⁢

(

𝐱

)

d

⁢

𝐱

(

I

)

=

-

2

⁢

ℑ

⁡

(

𝐱

H

⁢

A

)

-

R

⁢

(

𝐱

)

⁢

ℑ

⁡

(

𝐱

H

)

𝐱

H

⁢

𝐱

and, after a transposition, equating to the null column vector,

𝟎

=

(

ℑ

⁡

(

𝐱

¯

H

⁢

A

)

-

R

⁢

(

𝐱

¯

)

⁢

ℑ

⁡

(

𝐱

¯

H

)

)

T

=

=

ℑ

⁡

(

A

T

⁢

𝐱

¯

∗

)

-

R

⁢

(

𝐱

¯

)

⁢

ℑ

⁡

(

𝐱

¯

∗

)

=

ℑ

⁡

(

(

A

H

⁢

𝐱

¯

)

∗

)

-

R

⁢

(

𝐱

¯

)

⁢

ℑ

⁡

(

𝐱

¯

∗

)

=

=

ℑ

⁡

(

(

A

⁢

𝐱

¯

)

∗

)

-

R

⁢

(

𝐱

¯

)

⁢

ℑ

⁡

(

𝐱

¯

∗

)

=

-

ℑ

⁡

(

A

⁢

𝐱

¯

)

+

R

⁢

(

𝐱

¯

)

⁢

ℑ

⁡

(

𝐱

¯

)

and, since

R

⁢

(

𝐱

)

is real,

ℑ

⁡

(

A

⁢

𝐱

¯

-

R

⁢

(

𝐱

¯

)

⁢

𝐱

¯

)

=

𝟎

In

conclusion

, we have that a

stationary

vector

𝐱

¯

for
the Rayleigh quotient

satisfies

the complex eigenvalue equation

A

⁢

𝐱

¯

-

R

⁢

(

𝐱

¯

)

⁢

𝐱

¯

=

𝟎

whence the thesis.

□

Remarks:

1) The two

relations

λ

max

=

max

∥

𝐱

∥

≠

0

⁡

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

and

λ

min

=

min

∥

𝐱

∥

≠

𝟎

⁡

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

can also be obtained in a simpler way.
By Schur’s canonical form

theorem

, any normal (and hence any hermitian)
matrix is

unitarily diagonalizable

, i.e. a

unitary matrix

U

exists such
that

A

=

U

⁢

Λ

⁢

U

H

with

Λ

=

d

⁢

i

⁢

a

⁢

g

⁢

(

λ

1

,

λ

2

,

⋯

,

λ

n

)

. So, since all eigenvalues of a hermitian matrix are
real, it’s possible to write:

𝐱

H

⁢

A

⁢

𝐱

=

𝐱

U

H

⁢

Λ

⁢

U

H

⁢

𝐱

=

(

U

H

⁢

𝐱

)

H

⁢

Λ

⁢

(

U

H

⁢

𝐱

)

=

𝐲

H

⁢

Λ

⁢

𝐲

=

∑

i

=

1

n

λ

i

⁢

|

y

i

|

2

≤

λ

max

⁢

∑

i

=

1

n

|

y

i

|

2

=

=

λ

max

⁢

𝐲

H

⁢

𝐲

=

λ

max

⁢

(

U

H

⁢

𝐱

)

H

⁢

(

U

H

⁢

𝐱

)

=

λ

max

⁢

(

𝐱

H

⁢

U

⁢

U

H

⁢

𝐱

)

=

λ

max

⁢

(

𝐱

H

⁢

𝐱

)

whence

λ

max

≥

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

But, having defined

A

⁢

𝐯

M

=

λ

max

⁢

𝐯

M

, we
have:

𝐯

M

H

⁢

A

⁢

𝐯

M

𝐯

M

H

⁢

𝐯

M

=

𝐯

M

H

⁢

λ

max

⁢

𝐯

M

𝐯

M

H

⁢

𝐯

M

=

λ

max

so that

λ

max

=

max

∥

𝐱

∥

≠

𝟎

⁡

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

In a much similar way, we obtain

λ

min

=

min

∥

𝐱

∥

≠

𝟎

⁡

𝐱

H

⁢

A

⁢

𝐱

𝐱

H

⁢

𝐱

2) The above relations yield the following noteworthing bounds for the

diagonal

entries of a hermitian matrix:

λ

min

≤

a

i

⁢

i

≤

λ

max

In fact, having defined

𝐞

i

=

[

0

,

0

,

…

,

0

,

⏞

i

-

1

⁢

1

,

0

,

…

,

0

]

T

and observing that

a

i

⁢

i

=

𝐞

i

H

⁢

A

⁢

𝐞

i

𝐞

i

H

⁢

𝐞

i

=

R

⁢

(

𝐞

i

)

, we have:

λ

min

=

min

∥

𝐱

∥

≠

𝟎

⁡

R

⁢

(

𝐱

)

≤

R

⁢

(

𝐞

i

)

≤

max

∥

𝐱

∥

≠

𝟎

⁡

R

⁢

(

𝐱

)

=

λ

max

⁢

.

Title

Rayleigh-Ritz theorem

Canonical name

RayleighRitzTheorem

Date of creation

2013-03-22 15:37:18

Last modified on

2013-03-22 15:37:18

Owner

gufotta (12050)

Last modified by

gufotta (12050)

Numerical id

8

Author

gufotta (12050)

Entry type

Theorem

Classification

msc 15A18

Generated on Fri Feb 9 18:26:25 2018 by

LaTeXML