proof of Bernstein inequalities

proof of Bernstein inequalities

1) By Chernoff-Cramèr bound (

http://planetmath.org/ChernoffCramerBound

), we have:

Pr

⁡

{

∑

i

=

1

n

(

X

i

-

E

⁢

[

X

i

]

)

>

ε

}

≤

exp

⁡

[

-

sup

t

>

0

⁡

(

t

⁢

ε

-

ψ

⁢

(

t

)

)

]

where

ψ

⁢

(

t

)

=

∑

i

=

1

n

(

ln

⁡

E

⁢

[

e

t

⁢

X

i

]

-

t

⁢

E

⁢

[

X

i

]

)

Since

ln

⁡

x

≤

x

-

1

∀

x

≥

0

,

ψ

⁢

(

t

)

=

∑

i

=

1

n

(

ln

⁡

E

⁢

[

e

t

⁢

X

i

]

-

t

⁢

E

⁢

[

X

i

]

)

≤

∑

i

=

1

n

E

⁢

[

e

t

⁢

X

i

]

-

t

⁢

E

⁢

[

X

i

]

-

1

=

∑

i

=

1

n

E

⁢

[

1

+

t

⁢

X

i

+

1

2

⁢

t

2

⁢

X

i

2

+

∑

k

=

3

+

∞

t

k

⁢

X

i

k

k

!

]

-

t

⁢

E

⁢

[

X

i

]

-

1

=

∑

i

=

1

n

(

1

2

⁢

t

2

⁢

E

⁢

[

X

i

2

]

+

∑

k

=

3

+

∞

t

k

⁢

E

⁢

[

X

i

k

]

k

!

)

=

1

2

⁢

t

2

⁢

∑

i

=

1

n

E

⁢

[

X

i

2

]

+

∑

k

=

3

+

∞

t

k

⁢

∑

i

=

1

n

E

⁢

[

X

i

k

]

k

!

≤

1

2

⁢

t

2

⁢

∑

i

=

1

n

E

⁢

[

X

i

2

]

+

∑

k

=

3

+

∞

t

k

⁢

∑

i

=

1

n

E

⁢

[

|

X

i

|

k

]

k

!

,

and, keeping in mind hypotheses a) and b),

ψ

⁢

(

t

)

≤

1

2

⁢

t

2

⁢

v

2

+

∑

k

=

3

+

∞

t

k

2

⁢

v

2

⁢

c

k

-

2

=

1

2

⁢

t

2

⁢

v

2

+

1

2

⁢

t

3

⁢

v

2

⁢

c

⁢

∑

k

=

0

+

∞

(

t

⁢

c

)

k

Now, if

t

⁢

c

<

1

, we obtain

ψ

⁢

(

t

)

≤

1

2

⁢

t

2

⁢

v

2

⁢

(

1

+

t

⁢

c

1

-

t

⁢

c

)

=

v

2

⁢

t

2

2

⁢

(

1

-

t

⁢

c

)

whence

sup

t

>

0

⁡

(

t

⁢

ε

-

ψ

⁢

(

t

)

)

≥

sup

0

<

t

<

1

c

⁡

(

t

⁢

ε

-

v

2

⁢

t

2

2

⁢

(

1

-

t

⁢

c

)

)

By

elementary

calculus

, we obtain the value of

t

that maximizes the

expression

in brackets (out of the two roots of the second degree polynomial equation, we choose the one which is

<

1

c

):

t

o

⁢

p

⁢

t

=

v

2

+

2

⁢

c

⁢

ε

-

v

2

⁢

1

+

2

⁢

c

⁢

ε

v

2

c

⁢

(

v

2

+

2

⁢

c

⁢

ε

)

=

1

c

⁢

(

1

-

1

1

+

2

⁢

c

⁢

ε

v

2

)

which, once plugged into the bounds, yields

Pr

⁡

{

∑

i

=

1

n

(

X

i

-

E

⁢

[

X

i

]

)

>

ε

}

≤

exp

⁡

[

-

v

2

c

2

⁢

(

1

+

c

⁢

ε

v

2

-

1

+

2

⁢

c

⁢

ε

v

2

)

]

Observing that

1

+

x

≤

1

+

1

2

⁢

x

, one gets:

t

o

⁢

p

⁢

t

=

1

c

⁢

(

1

-

1

1

+

2

⁢

c

⁢

ε

v

2

)

≤

1

c

⁢

(

1

-

1

1

+

c

⁢

ε

v

2

)

=

ε

v

2

+

c

⁢

ε

=

t

′

<

1

c

Plugging

t

′

in the bound expression, the sub-optimal yet more
easily manageable

formula

is obtained:

Pr

⁡

{

∑

i

=

1

n

(

X

i

-

E

⁢

[

X

i

]

)

>

ε

}

≤

exp

⁡

(

-

ε

2

2

⁢

(

v

2

+

c

⁢

ε

)

)

which is obviously a worse bound than the preceeding one, since

t

′

≠

t

o

⁢

p

⁢

t

. One can also verify the

consistency

of this

inequality

directly proving that, for any

x

≥

0

,

1

+

x

-

1

+

2

⁢

x

≥

x

2

2

⁢

(

1

+

x

)

(see here (

http://planetmath.org/ASimpleMethodForComparingRealFunctions

) for an easy way, which can be used with

x

0

=

0

)

2) To prove this more specialized statement let’s recall that the condition

Pr

⁡

{

|

X

i

|

≤

M

}

=

1

⁢

⁢

∀

i

implies that, for all

i

,

E

⁢

[

|

X

i

|

k

]

≤

M

k

⁢

⁢

∀

k

≥

0

(See here (

http://planetmath.org/RelationBetweenAlmostSurelyAbsolutelyBoundedRandomVariablesAndTheirAbsoluteMoments

) for a proof.)

Now, it’s enough to verify that the condition

E

⁢

[

|

X

i

|

k

]

≤

M

k

imply both conditions a) and b) in part 1).

Indeed, part a) is

obvious

, while for part b) one happens to have:

E

⁢

[

|

X

i

|

k

]

≤

E

⁢

[

X

i

2

]

⁢

M

k

-

2

(see here (

http://planetmath.org/AbsoluteMomentsBoundingNecessaryAndSufficientCondition

) for a proof).

So

∑

i

=

1

n

E

⁢

[

|

X

i

|

k

]

≤

∑

i

=

1

n

E

⁢

[

X

i

2

]

⁢

M

k

-

2

=

v

2

⁢

M

k

-

2

Let’s find a value for

c

such that

v

2

⁢

M

k

-

2

≤

k

!

2

⁢

v

2

⁢

c

k

-

2

,
thus satisfying part b) of the hypotheses.

After simplifying, we have to study the inequality

k

!

⁢

c

k

-

2

≥

2

⋅

M

k

-

2

for any

k

≥

3

. Let’s proceed by

induction

. For

k

=

3

, we have

6

⁢

c

≥

2

⁢

M

which suggests

c

=

M

3

. Let’s now verify if this position is

consistent

with the

inductive hypothesis

:

(

k

+

1

)

!

=

(

k

+

1

)

⁢

k

!

≥

(

k

+

1

)

⋅

2

⋅

3

k

-

2

≥

3

⋅

2

⋅

3

k

-

2

=

2

⋅

3

(

k

+

1

)

-

2

which confirms the

validity

of the choice

c

=

M

3

, which has to be
plugged into the former bound to obtain the new one.

[to be continued…]

Title

proof of Bernstein inequalities

Canonical

name

ProofOfBernsteinInequalities

Date of creation

2013-03-22 16:09:32

Last modified on

2013-03-22 16:09:32

Owner

Andrea Ambrosio (7332)

Last modified by

Andrea Ambrosio (7332)

Numerical id

32

Author

Andrea Ambrosio (7332)

Entry type

Proof

Classification

msc 60E15

Generated on Sat Feb 10 11:56:28 2018 by

LaTeXML