gradient

gradient

Summary.

The

gradient

is a first-order

differential operator

that maps
scalar functions to

vector fields

. It is a

generalization

of the ordinary

derivative

, and as such conveys information about the rate of change
of a

function

relative to small variations in the independent
variables. The gradient of a function

f

is customarily denoted by

∇

⁡

f

or by

grad

⁡

f

.

Contents:

1

Definition: Euclidean space

2

Geometric and physical interpretations

3

Definition: Riemannian geometry

4

Duality with differential one-forms

5

Differential identities

6

The

∇

symbolism

1

Definition: Euclidean space

Let

f

:

ℝ

n

→

ℝ

be

continuously differentiable

.
The

gradient

of

f

, denoted by

∇

⁡

f

,
is defined
by the property:

D

𝐯

⁡

f

=

∇

⁡

f

⋅

𝐯

for all vectors

𝐯

∈

ℝ

n

.

(1)

The middle dot is the

dot product

,
and

D

𝐯

is the

directional derivative

with respect to

𝐯

.

If

x

1

,

…

,

x

n

are

Euclidean

coordinates

,
corresponding to the

orthonormal basis

𝐞

1

,

…

,

𝐞

n

,
then

∇

⁡

f

=

∑

i

=

1

n

∂

⁡

f

∂

⁡

x

i

⁢

𝐞

i

.

(2)

The

formula

(

2

) is sometimes given as the definition of

∇

⁡

f

.
We prefer to define

∇

⁡

f

by the coordinate-free formula (

1

) instead,
because then the geometric

interpretations

(see below) become obvious,
and (

1

) also indicates how we would go about
calculating the gradient in other

curvilinear coordinate

systems.
Formula (

1

) also makes it clear that the gradient
is a

physical vector

, depending only on the

inner product

structure

of

ℝ

n

,
and not on the specific

coordinate system

used to

calculate

it.

There is the issue of whether the

∇

⁡

f

as defined by (

1

)
exists; but this is proved easily enough, by substituting the concrete expression (

2

)
and seeing that it

satisfies

(

1

).

The gradient can be considered to be a
vector-valued differential operator, written as

∇

=

∑

i

=

1

n

𝐞

i

⁢

∂

∂

⁡

x

i

,

or, in the

context

of Euclidean 3-space, as

∇

=

𝐢

⁢

∂

∂

⁡

x

+

𝐣

⁢

∂

∂

⁡

y

+

𝐤

⁢

∂

∂

⁡

z

,

where

𝐢

,

𝐣

,

𝐤

are the

unit vectors

lying along the positive

direction

of the

x

,

y

,

z

axes, respectively.

2

Geometric and physical interpretations

(a)

The direction of the vector

∇

⁡

f

is the direction of the greatest positive change, or increase, in

f

.
The magnitude of

∇

⁡

f

is the magnitude of this increase.
This follows immediately from (

1

):

D

𝐯

⁡

f

=

∇

⁡

f

⋅

𝐯

=

∥

∇

⁡

f

∥

⁢

∥

𝐯

∥

⁢

cos

⁡

θ

,

where

θ

is the angle between

∇

⁡

f

and

𝐯

.
So among all unit directions

𝐯

of change, if

𝐯

is

perpendicular

to

∇

⁡

f

then the change

D

𝐯

⁡

f

is zero; if

𝐯

is

parallel

to

∇

⁡

f

then the change is maximized.

Similarly,

-

∇

⁡

f

is the direction of the greatest negative change, or decrease, in

f

.

(b)

If

M

is the hypersurface in

ℝ

n

defined by

M

=

{

p

∈

ℝ

n

:

f

⁢

(

p

)

=

0

,

D

⁡

f

⁢

(

p

)

≠

0

}

,

then

∇

⁡

f

⁢

(

p

)

is the normal to the hypersurface

M

at the point

p

. For

ker

⁡

D

⁡

f

⁢

(

p

)

is the

tangent space

T

p

⁢

M

to

M

at

p

, that is,

D

𝐯

⁡

f

⁢

(

p

)

=

0

for all

𝐯

∈

T

p

⁢

M

,
and by definition (

1

),

∇

⁡

f

⁢

(

p

)

must be perpendicular to all

𝐯

∈

T

p

⁢

M

.

Note that

D

⁡

f

≠

0

is

equivalent

to

∇

⁡

f

≠

0

. Consequently,

∇

⁡

f

also gives
an

orientation

to the hypersurface

M

.

For example, if

f

⁢

(

𝐱

)

=

∥

𝐱

∥

-

1

for

𝐱

∈

ℝ

n

,

M

is the

(

n

-

1

)

-dimensional sphere of unit radius, embedded in

ℝ

n

.
Its normal,

∇

⁡

f

⁢

(

𝐱

)

=

𝐱

/

∥

𝐱

∥

, as one would expect, points outward radially.

(c)

As a simple case of (b), consider the

surface

z

=

f

⁢

(

x

,

y

)

in

ℝ

3

, with

Cartesian coordinates

(

x

,

y

,

z

)

.
Think of this surface as describing a hill, with height

z

.
Then the direction of the gradient vector

∇

⁡

f

is the direction of steepest ascent of the hill, while its
magnitude

∥

∇

⁡

f

∥

=

(

∂

⁡

f

∂

⁡

x

)

2

+

(

∂

⁡

f

∂

⁡

y

)

2

is the slope or steepness in that direction.

If a ball is placed on the hill at a point

(

x

,

y

,

z

)

,
theoretically it should roll down the hill in the direction of the gradient
vector

-

∇

⁡

f

⁢

(

x

,

y

)

. This may be easily derived by considering the mechanical forces
on the ball. The direction of

-

∇

⁡

f

⁢

(

x

,

y

)

is, in fact, the

projection

to the xy-plane of an outward

normal vector

to the hill at

(

x

,

y

,

z

)

;
the normal vector is involved because the movement of the ball
arises from the normal force from the hill.

(d)

Suppose the surface

z

=

f

⁢

(

x

,

y

)

in (c) describes a bowl instead of a hill,
and we place a marble at any point

(

x

,

y

,

z

)

on this bowl.
We would expect the marble to roll down to a

local minimum

point of

f

⁢

(

x

,

y

)

.
Since the marble should roll down in the direction of

-

∇

⁡

f

,
we might hope that we can

find

local minima
of a

given function

f

by following the path mapped out
by the gradients

-

∇

⁡

f

. Formally, this method of
finding local extrema (with some modifications) is called

gradient descent

.

(e)

If

U

is the potential function corresponding to a

conservative

physical force, then

𝐅

=

-

∇

⁡

U

is the corresponding force field.

Consequently, the

gradient theorem

,

∫

γ

𝐅

⋅

d

𝐬

=

-

∫

γ

∇

U

⋅

d

𝐬

=

-

U

(

γ

(

b

)

)

+

U

(

γ

(

a

)

)

,

γ

:

[

a

,

b

]

→

ℝ

3

simply gives the formula for the change in the potential energy

U

when an object “does work” along a path

γ

in a conservative force field

𝐅

.

3

Definition: Riemannian geometry

It is obvious how (

1

)
can be generalized to the setting of

Riemannian manifolds

:
the dot product of

ℝ

n

must be replaced
by the

Riemannian metric

, and the analogue
of

D

𝐯

⁡

f

is the directional derivative

𝐯

⁢

[

f

]

, for

tangent vectors

𝐯

on the Riemannian manifold.
Thus for a smooth scalar-valued function

f

on a Riemannian manifold,

𝐗

=

grad

⁡

f

⇔

d

⁢

f

p

⁢

(

𝐯

)

=

𝐯

⁢

[

f

]

=

⟨

𝐗

,

𝐯

⟩

p

.

(3)

We can calculate

𝐗

explicitly as follows.
If

x

i

are

local coordinates

on the

manifold

(not necessarily orthonormal),
set

𝐗

=

X

i

⁢

∂

∂

⁡

x

i

(the

Einstein summation convention

is being used).
Let

g

i

⁢

j

and

g

i

⁢

j

be the covariant and contravariant

metric tensors

, respectively.
Then from (

3

),

∂

⁡

f

∂

⁡

x

j

=

⟨

X

i

⁢

∂

∂

⁡

x

i

,

∂

∂

⁡

x

j

⟩

=

X

i

⁢

⟨

∂

∂

⁡

x

i

,

∂

∂

⁡

x

j

⟩

=

g

i

⁢

j

⁢

X

i

,

and taking

inverses

,

X

i

=

g

i

⁢

j

⁢

∂

⁡

f

∂

⁡

x

j

.

(4)

4

Duality with differential one-forms

Definitions (

1

) and (

3

)
exhibit

∇

⁡

f

as the vector field
dual to the

differential form

d

⁢

f

.
The

isomorphism

is given by applying the inner product or Riemannian metric.
This isomorphism is, of course, linear;
in particular it leads to the identity

∇

⁡

f

=

∂

⁡

f

∂

⁡

x

i

⁢

∇

⁡

x

i

,

(5)

which is the dual to the standard formula of

differential

one-forms

:

d

⁢

f

=

∂

⁡

f

∂

⁡

x

i

⁢

d

⁢

x

i

.

Using (

3

) and (

4

),
we have

∇

⁡

x

i

=

g

i

⁢

j

⁢

∂

∂

⁡

x

j

,

∂

∂

⁡

x

i

=

g

i

⁢

j

⁢

∇

⁡

x

j

.

(6)

So the isomorphism between vector fields and one-forms
is expressed by changing
the

∇

’s in (

6

) to

d

’s, and vice versa. That is,

d

⁢

x

i

↔

g

i

⁢

j

⁢

∂

∂

⁡

x

j

,

∂

∂

⁡

x

i

↔

g

i

⁢

j

⁢

d

⁢

x

j

.

(7)

It is commonly said that this isomorphism is expressed by
“raising and lowering the indices of a

tensor field

,
using

contractions

with

g

i

⁢

j

and

g

i

⁢

j

”.

Notice that when

x

i

are orthonormal coordinates
on

ℝ

n

,

equation

(

5

)
reduces to equation (

2

), because

g

i

⁢

j

=

𝐞

i

⋅

𝐞

j

=

δ

i

⁢

j

(

Kronecker delta

).

The formulae presented in this

section

are useful in the Euclidean setting as
well, for deriving the formulae for the gradient in various curvilinear coordinate systems (

http://planetmath.org/GradientInCurvilinearCoordinates

).

5

Differential identities

Several properties of the one-dimensional derivative generalize to a
multi-dimensional setting

∇

⁡

(

a

⁢

f

+

b

⁢

g

)

=

a

⁢

∇

⁡

f

+

b

⁢

∇

⁡

g

Linearity

∇

⁡

(

f

⁢

g

)

=

f

⁢

∇

⁡

g

+

g

⁢

∇

⁡

f

Product rule

∇

⁡

(

f

∘

ϕ

)

⁡

(

p

)

=

D

⁡

ϕ

⁢

(

p

)

*

⁢

∇

⁡

f

⁢

(

ϕ

⁢

(

p

)

)

Chain rule

∇

⁡

(

h

∘

f

)

⁡

(

p

)

=

h

′

⁢

(

f

⁢

(

p

)

)

⁢

∇

⁡

f

⁢

(

p

)

Another Chain rule

The function

h

is

h

:

ℝ

→

ℝ

.
The notation

(

D

⁡

ϕ

)

*

denotes the

transpose

of the

Jacobian matrix

,
in Euclidean coordinates, of

ϕ

:

ℝ

m

→

ℝ

n

.
In the abstract setting,

(

D

⁡

ϕ

)

*

is the

adjoint

to the

tangent map

D

⁡

ϕ

between the

tangent bundles

of two Riemannian manifolds.

These identities can be proved directly from the definition,
but the first three are really just the duals
of the following well-known identities for differential forms:

d

⁢

(

a

⁢

f

+

b

⁢

g

)

=

a

⁢

d

⁢

f

+

b

⁢

d

⁢

g

d

⁢

(

f

⁢

g

)

=

f

⁢

d

⁢

g

+

g

⁢

d

⁢

f

d

⁢

(

ϕ

*

⁢

f

)

=

ϕ

*

⁢

d

⁢

f

and so may be derived by changing the

d

’s here
to

∇

’s! (Though the third identity may take a bit of thought.)

The following identity

curl

⁡

grad

⁡

f

=

∇

×

∇

⁡

f

=

0

,

is a special case of the differential forms identity

d

2

=

0

.

Conversely

, if

curl

⁡

g

=

0

on a simply connected domain, then there
is

f

such that

g

=

grad

⁡

f

. See laminar field for details.

6

The

∇

symbolism

(This discussion does not really belong here, but should be moved
to the nabla entry.)

Using the

∇

formalism

,
the

divergence

operator

can be expressed as

∇

⋅

, the curl operator as

∇

×

, and the
Laplacian operator as

∇

2

. To wit, for a given vector field

𝐀

=

A

x

⁢

𝐢

+

A

y

⁢

𝐣

+

A

z

⁢

𝐤

,

and a given function

f

we have

∇

⋅

𝐀

=

∂

⁡

A

x

∂

⁡

x

+

∂

⁡

A

y

∂

⁡

y

+

∂

⁡

A

z

∂

⁡

z

∇

×

𝐀

=

(

∂

⁡

A

z

∂

⁡

y

-

∂

⁡

A

y

∂

⁡

z

)

⁢

𝐢

+

(

∂

⁡

A

x

∂

⁡

z

-

∂

⁡

A

z

∂

⁡

x

)

⁢

𝐣

+

(

∂

⁡

A

y

∂

⁡

x

-

∂

⁡

A

x

∂

⁡

y

)

⁢

𝐤

∇

2

⁡

f

=

∂

2

⁡

f

∂

⁡

x

2

+

∂

2

⁡

f

∂

⁡

y

2

+

∂

2

⁡

f

∂

⁡

z

2

.

References

1

Michael Spivak.

A Comprehensive

Introduction

to

Differential Geometry

,
Volume I. Publish or Perish, 1979.

Title

gradient

Canonical name

Gradient

Date of creation

2013-03-22 11:59:20

Last modified on

2013-03-22 11:59:20

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

20

Author

CWoo (3771)

Entry type

Definition

Classification

msc 58A10

Classification

msc 26B12

Classification

msc 26B10

Related topic

Derivative2

Related topic

RiemannianMetric

Related topic

GradientInCurvilinearCoordinates

Related topic

NablaNabla

Related topic

DifferentialForms

Related topic

FirstOrderOperatorsInRiemannianGeometry

Related topic

VectorField

Related topic

HessianMatrix

Related topic

DerivativeNotation

Related topic

JacobianMatrix

Related topic

PartialDerivative

Related topic

TiltCurve

Defines

gradient

Generated on Sat Feb 10 11:50:05 2018 by

LaTeXML