proof of Kolmogorov’s strong law for IID random variables

proof of Kolmogorov’s strong law for IID random variables

Kolmogorov’s strong law for square integrable

random variables

states that if

X

1

,

X

2

,

…

is a

sequence

of

independent

random variables with

∑

n

Var

⁡

[

X

n

]

/

n

2

<

∞

then

n

-

1

⁢

∑

k

=

1

n

(

X

k

-

𝔼

⁢

[

X

k

]

)

converges

to zero with

probability

one as

n

→

∞

(see

martingale

proof of Kolmogorov’s strong law for square integrable variables).
We show that the following version of the strong law for IID random variables follows from this.

Theorem

(Kolmogorov).

Let

X

1

,

X

2

,

…

be independent and identically distributed random variables with

E

⁢

[

|

X

n

|

]

<

∞

. Then,

n

-

1

⁢

∑

k

=

1

n

(

X

k

-

E

⁢

[

X

k

]

)

→

0

as

n

→

∞

, with probability one.

Note that here, the random variables

X

n

are not necessarily square integrable. Let us set

X

~

n

=

X

n

-

𝔼

⁢

[

X

n

]

, so that

X

~

n

are IID random variables with

𝔼

⁢

[

X

~

n

]

=

0

. Then, set

Y

n

=

{

X

~

n

,

if

⁢

|

X

~

n

|

<

n

,

0

,

otherwise

.

Using the fact that

X

n

has the same

distribution

as

X

1

gives

∑

n

𝔼

⁢

[

Y

n

2

]

/

n

2

=

∑

n

𝔼

⁢

[

1

{

|

X

~

n

|

<

n

}

⁢

n

-

2

⁢

X

~

n

2

]

=

∑

n

𝔼

⁢

[

1

{

|

X

~

1

|

<

n

}

⁢

n

-

2

⁢

X

~

1

2

]

=

𝔼

⁢

[

∑

n

1

{

|

X

~

1

|

<

n

}

⁢

n

-

2

⁢

X

~

1

2

]

.

(1)

Letting

N

be the smallest

integer

greater than

|

X

~

1

|

,

∑

n

1

{

|

X

~

1

|

<

n

}

⁢

n

-

2

≤

∑

n

=

N

∞

4

4

⁢

n

2

-

1

=

∑

n

=

N

∞

(

2

2

⁢

n

-

1

-

2

2

⁢

n

+

1

)

=

2

2

⁢

N

-

1

≤

2

N

≤

2

|

X

~

1

|

.

So, putting this into equation (

1

),

∑

n

Var

⁡

[

Y

n

]

/

n

2

≤

∑

n

𝔼

⁢

[

Y

n

2

]

/

n

2

≤

𝔼

⁢

[

2

⁢

|

X

~

1

|

]

<

∞

.

Therefore,

Y

n

satisfies

the required properties to apply the strong law for square integrable random variables,

n

-

1

⁢

∑

k

=

1

n

(

Y

k

-

𝔼

⁢

[

Y

k

]

)

→

0

(2)

as

n

→

∞

, with probability one. Also,

𝔼

⁢

[

Y

n

]

=

𝔼

⁢

[

Y

n

-

X

~

n

]

=

-

𝔼

⁢

[

1

{

|

X

~

n

|

≥

n

}

⁢

X

~

n

]

=

-

𝔼

⁢

[

1

{

|

X

~

1

|

≥

n

}

⁢

X

~

1

]

converges to

0

as

n

→

∞

(by the

dominated convergence theorem

). So, the

𝔼

⁢

[

Y

k

]

terms in (

2

) vanish in the limit, giving

n

-

1

⁢

∑

k

=

1

n

Y

k

→

0

(3)

as

n

→

∞

with probability one.

We finally note that

𝔼

⁢

[

∑

n

1

{

X

~

n

≠

Y

n

}

]

=

𝔼

⁢

[

∑

n

1

{

|

X

~

1

|

≥

n

}

]

≤

𝔼

⁢

[

|

X

~

1

|

]

<

∞

,

so

∑

n

1

{

X

~

n

≠

Y

n

}

<

∞

, and

X

~

n

=

Y

n

for large

n

(with probability one). So,

Y

k

can be replaced by

X

~

k

in (

3

), giving the result.

References

1

David Williams,

Probability with martingales

,
Cambridge Mathematical Textbooks, Cambridge University Press, 1991.

2

Olav Kallenberg,

Foundations

of modern probability

, Second edition. Probability and its Applications. Springer-Verlag, 2002.

Title

proof of Kolmogorov’s strong law for IID random variables

Canonical name

ProofOfKolmogorovsStrongLawForIIDRandomVariables

Date of creation

2013-03-22 18:33:57

Last modified on

2013-03-22 18:33:57

Owner

gel (22282)

Last modified by

gel (22282)

Numerical id

7

Author

gel (22282)

Entry type

Proof

Classification

msc 60F15

Related topic

KolmogorovsStrongLawOfLargeNumbers

Related topic

MartingaleProofOfKolmogorovsStrongLawForSquareIntegrableVariables

Generated on Sat Feb 10 11:57:06 2018 by

LaTeXML