best approximation in inner product spaces

best approximation in inner product spaces

The study of best

approximations

in

inner product spaces

has a very elegant treatment with profound

consequences

. Most of the theory of

Hilbert spaces

depends on this study and several

approximation problems

are better understood using this techniques and results.

For example: least square fitting, linear regression, approximation of functions by

polynomials

, among many other problems, can be seen as particular cases of the general study of best approximation in inner product spaces.

Some of the above problems are going to be discussed later in this entry.

1

Existence and Uniqueness

Our fundamental result on the

existence

and uniqueness of best approximations is the following (we postpone its proof to this attached entry (

http://planetmath.org/ProofOfExistenceAndUniquenessOfBestApproximations

)):

Theorem

-

Let

X

be an inner product space and

A

⊆

X

a

complete

(

http://planetmath.org/Complete

), convex and non-empty subset. Then for every

x

∈

X

there exists a unique best approximation (

http://planetmath.org/BestApproximation

) of

x

in

A

, i.e. there exists a unique

element

a

0

∈

A

such that

∥

x

-

a

0

∥

=

d

⁢

(

x

,

A

)

=

inf

a

∈

A

⁡

∥

x

-

a

∥

.

2

Geometric Interpretation

The following result gives a very geometric

interpretation

of the best approximation when

A

is a

subspace

of

X

. We also postpone its proof to an attached entry.

Theorem -

Let

X

be an inner product space,

A

⊆

X

a subspace and

x

∈

X

. The following statements are

equivalent

:

•

a

0

∈

A

is the best approximation of

x

in

A

.

•

a

0

∈

A

and

x

-

a

0

⟂

A

.

Thus, the best approximation of

x

in a subspace

A

is just the

orthogonal projection

of

x

in

A

.

3

Calculation of Best Approximations

When the

A

is a complete subspace of

X

, the best approximation can be ”calculated” explicitly. Recall that, in this case,

A

becomes an Hilbert space (since it is complete) and therefore it has an

orthonormal basis

.

Again, we postpone the proof of the next result to an attached entry.

Theorem -

Let

X

be an inner product space and

A

⊆

X

a complete subspace. Let

(

e

i

)

i

∈

J

be an orthonormal basis for

A

. Then for every

x

∈

X

the best approximation

a

0

∈

A

of

x

in

A

is given by

a

0

=

∑

i

∈

J

⟨

x

,

e

i

⟩

⁢

e

i

.

One can also write the best approximation in of any other basis (not necessarily an orthonormal one). For simplicity we present here how that can be done when

A

is a finite dimensional subspace of

X

.

Theorem -

Let

X

be an inner product space and

A

⊆

X

a finite dimensional subspace. Let

v

1

,

…

,

v

n

be a basis for

A

. Then for every

x

∈

X

the best approximation

a

0

∈

A

of

x

in

A

is given by

a

0

=

∑

i

=

1

n

a

0

i

⁢

v

i

where the coefficients

a

0

i

are the solutions of the system of equations

(

⟨

v

1

,

v

1

⟩

⋯

⟨

v

1

,

v

n

⟩

⋮

⋱

⋮

⟨

v

n

,

v

1

⟩

⋯

⟨

v

n

,

v

n

⟩

)

⁢

(

a

0

1

⋮

a

0

n

)

=

(

⟨

x

,

v

1

⟩

⋮

⟨

x

,

v

n

⟩

)

.

R

⁢

e

⁢

m

⁢

a

⁢

r

⁢

k

-

The above matrix is a

symmetric

positive definite

(

http://planetmath.org/PositiveDefinite

) matrix, which implies that the system has a unique solution as expected.

4

Applications

There are several applications of the above results. We explore two of them in the following.

4.0.1

- Approximation of functions by polynomials :

Suppose we want to find a polynomial of degree

≤

n

that approximates in the best possible way a given

function

f

. We are in fact trying to find a point in the subspace of polynomials of degree

≤

n

that is closest to

f

, i.e. we are trying to find the best approximation of

f

in that subspace.

For example, let

f

∈

L

2

⁢

(

[

0

,

1

]

)

. Consider the basis

v

k

⁢

(

t

)

=

t

k

,

0

≤

k

≤

n

,

of the subspace of polynomials of degree

≤

n

.

The best approximation of

f

by these polynomials is the function

a

0

⁢

(

t

)

=

a

0

1

+

a

0

1

⁢

t

+

…

+

a

0

n

⁢

t

n

, where the coefficients

a

0

1

,

…

,

a

0

n

are the solutions of the system

(

1

⋯

1

n

+

1

⋮

⋱

⋮

1

n

+

1

⋯

1

2

⁢

n

+

1

)

⁢

(

a

0

1

⋮

a

0

n

)

=

(

∫

0

1

f

⁢

(

t

)

⁢

𝑑

t

⋮

∫

0

1

t

n

⁢

f

⁢

(

t

)

⁢

𝑑

t

)

.

R

⁢

e

⁢

m

⁢

a

⁢

r

⁢

k

-

Instead of polynomials we could approximate

f

by any other of functions using the same procedure.

4.0.2

- Best Fitting Lines :

Suppose we want to find the line that best fits some given points

(

t

1

,

y

1

)

,

…

,

(

t

n

,

y

n

)

, i.e. the affine function

a

0

⁢

(

t

)

=

α

⁢

t

+

β

that minimizes

∑

k

=

1

n

|

a

0

⁢

(

t

k

)

-

y

k

|

2

.

We are then led to consider the

inner product

⟨

f

,

g

⟩

=

∑

k

=

1

n

f

⁢

(

t

k

)

⁢

g

⁢

(

t

k

)

in the

space of functions

h

:

{

t

1

,

…

,

t

k

}

⟶

ℝ

.

With this setting we are then looking for the best approximation of the function

f

⁢

(

t

k

)

=

y

k

on the subspace of affine functions.

A base for the subspace of affine functions is given by the functions

v

1

⁢

(

t

)

=

1

and

v

2

⁢

(

t

)

=

t

.

The best approximation of

f

on this space is the function

a

0

⁢

(

t

)

=

β

+

α

⁢

t

, where the coefficients

β

,

α

are the solutions of the system

(

n

∑

k

=

1

n

t

k

∑

k

=

1

n

t

k

∑

k

=

1

n

t

k

2

)

⁢

(

β

α

)

=

(

∑

k

=

1

n

y

k

∑

k

=

1

n

y

k

⁢

t

k

)

.

Thus, the function

a

0

⁢

(

t

)

=

β

+

α

⁢

t

obtained by the above procedure provides the line that best fits the data

(

t

1

,

y

1

)

,

…

,

(

t

n

,

y

n

)

.

Title

best approximation in inner product spaces

Canonical name

BestApproximationInInnerProductSpaces

Date of creation

2013-03-22 17:32:16

Last modified on

2013-03-22 17:32:16

Owner

asteroid (17536)

Last modified by

asteroid (17536)

Numerical id

12

Author

asteroid (17536)

Entry type

Feature

Classification

msc 41A65

Classification

msc 46C05

Classification

msc 46N10

Classification

msc 49J27

Classification

msc 41A52

Classification

msc 41A50

Defines

approximation by polynomials

Defines

best fitting lines

Generated on Fri Feb 9 21:28:08 2018 by

LaTeXML