one-pass algorithm to compute sample variance

one-pass algorithm to compute sample variance

In many situations it is desirable to

calculate

,
in one

iteration

, the

sample variance

of many numbers, and without
having to have every number available
in

computer

memory before beginning processing.

Let

x

1

,

x

2

,

…

denote the data.
The naïve formula for calculating the sample
variance in one pass,

v

=

1

n

-

1

⁢

∑

i

=

1

n

(

x

i

-

x

¯

)

2

=

(

∑

i

=

1

n

x

i

2

)

-

n

⁢

x

¯

2

n

-

1

,

x

¯

=

1

n

⁢

∑

i

=

1

n

x

i

,

suffers from computational round-off error.
If the mean

x

¯

is large in

absolute value

,
and

∑

i

=

1

n

x

i

2

is close to

n

⁢

x

¯

2

,
then the

subtraction

at the end will tend to lose

significant digits

on the result.
Also, in rare cases, the sum of squares

∑

i

=

1

n

x

i

2

can overflow on a computer.

A better alternative, though requiring more work per iteration,
is to calculate the

running

sample mean

and

variance

instead,
and update these as each datum is processed.
Here we give the derivation of the one-pass algorithm — which
involves nothing more than simple

algebraic

manipulations.

Define the running

arithmetic mean

and the sum of squared

residuals

:

a

n

=

1

n

⁢

∑

i

=

1

n

x

i

,

s

n

=

∑

i

=

1

n

(

x

i

-

a

n

)

2

.

We want to express

a

n

+

1

and

s

n

+

1

in terms of the old values

a

n

and

s

n

.

For convenience, let

δ

=

x

n

+

1

-

a

n

and

γ

=

a

n

+

1

-

a

n

.
Then we have

a

n

+

1

=

n

⁢

a

n

+

x

n

+

1

n

+

1

=

(

n

+

1

)

⁢

a

n

+

x

n

+

1

-

a

n

n

+

1

=

a

n

+

δ

n

+

1

.

For the variance calculation,
we have

s

n

+

1

=

∑

i

=

1

n

(

(

x

i

-

a

n

)

-

γ

)

2

+

(

x

n

+

1

-

a

n

+

1

)

2

=

∑

i

=

1

n

(

x

i

-

a

n

)

2

-

2

⁢

γ

⁢

∑

i

=

1

n

(

x

i

-

a

n

)

+

∑

i

=

1

n

γ

2

+

(

x

n

+

1

-

a

n

+

1

)

2

=

s

n

+

0

+

n

⁢

γ

2

+

(

x

n

+

1

-

a

n

+

1

)

2

.

Now observe:

γ

=

δ

n

+

1

,

x

n

+

1

-

a

n

+

1

=

δ

-

γ

=

(

n

+

1

)

⁢

γ

-

γ

=

n

⁢

γ

;

hence we obtain:

s

n

+

1

=

s

n

+

n

⁢

γ

2

+

n

2

⁢

γ

2

=

s

n

+

n

⁢

(

n

+

1

)

⁢

γ

2

=

s

n

+

n

⁢

γ

⁢

δ

=

s

n

+

(

x

n

+

1

-

a

n

+

1

)

⁢

δ

.

Note that the number to be added to

s

n

is never

negative

,
so no cancellation error will occur from this procedure.
(However, there can still be computational round-off error if

s

n

+

1

-

s

n

happens to be very small compared to

s

n

.)

The

recurrence relation

for the
sample covariance of two lists of numbers

x

1

,

x

2

,

…

and

y

1

,

y

2

,

…

can be derived similarly.
If

a

n

and

b

n

denote the arithmetic means
of first

n

numbers of each of the two lists respectively,
then the sum of adjusted products

c

n

=

∑

i

=

1

n

(

x

i

-

a

n

)

⁢

(

y

i

-

b

n

)

can be incrementally updated by

c

n

+

1

=

c

n

+

(

y

n

+

1

-

b

n

+

1

)

⁢

(

x

n

+

1

-

a

n

)

=

c

n

+

(

x

n

+

1

-

a

n

+

1

)

⁢

(

y

n

+

1

-

b

n

)

.

References

1

B. P. Welford. “Note on a Method for Calculating Corrected Sums of Squares and Products”.

Technometrics

, Vol. 4, No. 3 (Aug., 1962), p. 419-420.

2

“

http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance

Algorithms for calculating variance”.

Wikipedia, The Free Encyclopedia

. Accessed 25 February 2007.

Title

one-pass algorithm to compute sample variance

Canonical

name

OnepassAlgorithmToComputeSampleVariance

Date of creation

2013-03-22 16:45:19

Last modified on

2013-03-22 16:45:19

Owner

stevecheng (10074)

Last modified by

stevecheng (10074)

Numerical id

9

Author

stevecheng (10074)

Entry type

Algorithm

Classification

msc 68W01

Classification

msc 65-00

Classification

msc 62-00

Generated on Sat Feb 10 12:50:37 2018 by

LaTeXML