Newton’s method

Newton’s method

Let

𝐟

be a

differentiable function

from

ℝ

n

to

ℝ

n

.
Newton’s method consists of starting at an

𝐚

𝟎

for the equation

𝐟

⁢

(

𝐱

)

=

𝟎

. Then the

function

is

linearized

at

𝐚

𝟎

by replacing the increment

𝐟

⁢

(

𝐱

)

-

𝐟

⁢

(

𝐚

𝟎

)

by a

linear function

of the increment

[

𝐃

⁢

(

𝐚

𝟎

)

]

⁢

(

𝐱

-

𝐚

𝟎

)

.

Now we can solve the

linear equation

𝐟

⁢

(

𝐚

𝟎

)

+

[

𝐃

⁢

(

𝐚

𝟎

)

]

⁢

(

𝐱

-

𝐚

𝟎

)

=

𝟎

.
Since this is a system of

n

linear equations in

n

unknowns,

[

𝐃

⁢

(

𝐚

𝟎

)

]

⁢

(

𝐱

-

𝐚

𝟎

)

=

-

𝐟

⁢

(

𝐚

𝟎

)

can be likened to the general linear system

A

⁢

𝐱

=

𝐛

.

Therefore, if

[

𝐃

⁢

(

𝐚

𝟎

)

]

is

invertible

, then

𝐱

=

𝐚

𝟎

-

[

𝐃

⁢

(

𝐚

𝟎

)

]

-

1

⁢

𝐟

⁢

(

𝐚

𝟎

)

.
By renaming

𝐱

to

𝐚

𝟏

, you can reiterate Newton’s method to get an

𝐚

𝟐

. Thus, Newton’s method states

𝐚

n

+

1

=

𝐚

n

-

[

𝐃𝐟

⁢

(

𝐚

n

)

]

-

1

⁢

𝐟

⁢

(

𝐚

n

)

Thus we get a series of

𝐚

’s that we hope will

converge

to

𝐱

such that

𝐟

⁢

(

𝐱

)

=

𝟎

. When we solve an equation of the form

𝐟

⁢

(

𝐱

)

=

𝟎

, we call the solution a

root

of the equation. Thus, Newton’s method is used to find roots of nonlinear equations.

Unfortunately, Newton’s method does not always converge. There are tests for

neighborhoods

of

𝐚

𝟎

’s where Newton’s method will converge however. One such test is Kantorovitch’s

theorem

, which combines what is needed into a concise mathematical equation.

Corollary 1: Newton’s Method in one

dimension

- The above equation is simplified in one dimension to the well-used

a

1

=

a

0

-

f

⁢

(

a

0

)

f

′

⁢

(

a

0

)

This intuitively cute equation is pretty much

the

equation of first year

calculus

. :)

Corollary 2: Finding a square root

- So now that you know the equation,
you need to know how to

use

it, as it is an

algorithm

. The construction of the primary equation, of course is the important part. Let’s see how you do it if you want to find a square root of a number

b

.

We want to find a number

x

(

x

for unknown), such that

x

2

=

b

. You might think “why not find
a number such that

x

=

b

?” Well, the problem with that approach is that we don’t have a value for

b

, so we’d be right back where we started. However, squaring both sides of the equation to get

x

2

=

b

lets us work with the number we

do

know,

b

.) Back to

x

2

=

b

. With some manipulation, we see this means that

x

2

-

b

=

0

! Thus we have our

f

⁢

(

x

)

=

0

scenario

.

We can see that

f

′

⁢

(

x

)

=

2

⁢

x

thus,

f

′

⁢

(

a

0

)

=

2

⁢

a

0

and

f

⁢

(

a

0

)

=

a

0

2

-

b

. Now we have all we need to carry out Newton’s method. By renaming

x

to be

a

1

, we have

a

1

=

a

0

-

1

2

⁢

a

0

⁢

(

a

0

2

-

b

)

=

1

2

⁢

(

a

0

+

b

a

0

)

.

The equation on the far right is also known as the

divide and

average

method, for those who have not learned the full Newton’s method, and just want a fast way to find square roots. Let’s see how this works out to find the square root of a number like 2:

Let

x

2

=

2

x

2

-

2

=

0

=

f

⁢

(

x

)

Thus, by Newton’s method,…

a

1

=

a

0

-

a

0

2

-

2

2

⁢

a

0

All we did was plug in the

expressions

f

⁢

(

a

0

)

and

f

′

⁢

(

a

0

)

where Newton’s method asks for them. Now we have to pick an

a

0

. Hmm, since

1

<

2

<

4

1

<

2

<

2

let’s pick a reasonable number between 1 and 2 like 1.5

a

1

=

1.5

-

1.5

2

-

2

2

⁢

(

1.5

)

a

1

=

1.41

⁢

6

¯

Looks like our guess was too high. Let’s see what the next

iteration

says

a

2

=

1.41

⁢

6

¯

-

1.41

⁢

6

¯

2

-

2

2

⁢

(

1.41

⁢

6

¯

)

a

2

=

1.414215686

⁢

…

getting better =) You can use your

calculator

to find that

2

=

1.414213562

⁢

…

Not bad for only two iterations! Of course, the more you iterate, the more

decimal places

your

a

n

will be accurate to. By the way, this is also how your calculator/

computer

finds square roots!

Geometric

Interpretation

:

Consider an arbitrary function

f

:

ℝ

→

ℝ

such as

f

⁢

(

x

)

=

x

2

-

b

. Say you wanted to find a root of this function. You know that in the neighborhood of

x

=

a

0

, there is a root (Maybe you used Kantorovitch’s theorem or tested and saw that the function changed signs in this neighborhood). We want to use our knowledge

a

0

to find an

a

1

that is a better approximation to

x

0

(in this case, closer to it on the x-axis).

So we know that

x

0

≤

a

1

≤

a

0

or in another case

a

0

≤

a

1

≤

x

0

. What is an efficient algorithm to bridge the gap between

a

0

and

x

0

? Let’s look at a

tangent line

to the graph.

Note that the line intercepts the x-axis between

a

0

and

x

0

, which is exactly what we want. The slope of this tangent line is

f

′

⁢

(

a

0

)

by

definition

of the

derivative

at

a

0

, and we know one point on the line is

(

a

1

,

0

)

, since that is the x-intercept. That is all we need to find the

formula

of the line, and solve for

a

1

.

y

-

y

1

=

m

⁢

(

x

-

x

1

)

f

⁢

(

a

0

)

-

0

=

f

′

⁢

(

a

0

)

⁢

(

a

0

-

a

1

)

Substituting

f

⁢

(

a

0

)

f

′

⁢

(

a

0

)

=

a

0

-

a

1

-

a

1

=

-

a

0

+

f

⁢

(

a

0

)

f

′

⁢

(

a

0

)

Aesthetic change. Flipped the equation around.

a

1

=

a

0

-

f

⁢

(

a

0

)

f

′

⁢

(

a

0

)

Newton’s method!

Title

Newton’s method

Canonical

name

NewtonsMethod

Date of creation

2013-03-22 11:58:05

Last modified on

2013-03-22 11:58:05

Owner

alozano (2414)

Last modified by

alozano (2414)

Numerical id

28

Author

alozano (2414)

Entry type

Algorithm

Classification

msc 49M15

Synonym

Newton-Raphson method

Related topic

LipschitzCondition

Related topic

KantorovitchsTheorem

Related topic

Derivative

Related topic

Superconvergence

Related topic

FixedPointsOfNormalFunctions

Generated on Fri Feb 9 21:48:54 2018 by

LaTeXML