proof of Bennett inequality

proof of Bennett inequality

By Chernoff-Cramèr

inequality

(

http://planetmath.org/ChernoffCramerBound

), we have:

Pr

⁡

{

∑

i

=

1

n

(

X

i

-

E

⁢

[

X

i

]

)

>

ε

}

≤

exp

⁡

[

-

sup

t

≥

0

⁡

(

t

⁢

ε

-

ψ

⁢

(

t

)

)

]

where

ψ

⁢

(

t

)

=

∑

i

=

1

n

(

ln

⁡

E

⁢

[

e

t

⁢

X

i

]

-

t

⁢

E

⁢

[

X

i

]

)

.

Keeping in mind that the condition

Pr

⁡

{

|

X

i

|

≤

M

}

=

1

⁢

⁢

∀

i

implies that, for all

i

,

E

⁢

[

|

X

i

|

k

]

≤

M

k

⁢

⁢

∀

k

≥

0

(see here (

http://planetmath.org/RelationBetweenAlmostSurelyAbsolutelyBoundedRandomVariablesAndTheirAbsoluteMoments

) for a proof) and since

ln

⁡

x

≤

x

-

1

∀

x

>

0

, and

E

⁢

[

|

X

|

k

]

≤

M

k

⁢

⟹

⁢

E

⁢

[

|

X

|

k

]

≤

E

⁢

[

X

2

]

⁢

M

k

-

2

⁢

⁢

∀

k

≥

2

,

k

∈

N

(see here (

http://planetmath.org/AbsoluteMomentsBoundingNecessaryAndSufficientCondition

) for a proof), one has:

ψ

⁢

(

t

)

=

∑

i

=

1

n

(

ln

⁡

E

⁢

[

e

t

⁢

X

i

]

-

t

⁢

E

⁢

[

X

i

]

)

≤

∑

i

=

1

n

E

⁢

[

e

t

⁢

X

i

]

-

t

⁢

E

⁢

[

X

i

]

-

1

=

∑

i

=

1

n

E

⁢

[

∑

k

=

0

∞

(

t

⁢

X

i

)

k

k

!

]

-

t

⁢

E

⁢

[

X

i

]

-

1

=

∑

i

=

1

n

(

∑

k

=

0

∞

t

k

⁢

E

⁢

[

X

i

k

]

k

!

)

-

t

⁢

E

⁢

[

X

i

]

-

1

=

∑

i

=

1

n

(

∑

k

=

2

∞

t

k

⁢

E

⁢

[

X

i

k

]

k

!

)

≤

∑

i

=

1

n

(

∑

k

=

2

∞

t

k

⁢

E

⁢

[

|

X

i

|

k

]

k

!

)

≤

∑

i

=

1

n

(

∑

k

=

2

∞

t

k

⁢

E

⁢

[

X

i

2

]

⁢

M

k

-

2

k

!

)

=

∑

k

=

2

∞

t

k

⁢

M

k

-

2

⁢

∑

i

=

1

n

E

⁢

[

X

i

2

]

k

!

=

v

2

M

2

⁢

∑

k

=

2

∞

(

t

⁢

M

)

k

k

!

=

v

2

M

2

⁢

[

exp

⁡

(

t

⁢

M

)

-

t

⁢

M

-

1

]

One can now write

sup

t

≥

0

⁡

(

t

⁢

ε

-

ψ

⁢

(

t

)

)

≥

sup

t

≥

0

⁡

(

t

⁢

ε

-

v

2

M

2

⁢

(

e

t

⁢

M

-

t

⁢

M

-

1

)

)

=

sup

t

>

0

⁡

[

v

2

M

2

⁢

(

M

2

⁢

ε

v

2

⁢

t

-

(

e

t

⁢

M

-

t

⁢

M

-

1

)

)

]

.

By elementary

calculus

, we obtain the value of

t

that maximizes the
expression in round brackets:

t

o

⁢

p

⁢

t

=

1

M

⁢

ln

⁡

(

1

+

M

⁢

ε

v

2

)

which, once plugged into the bound, yields

Pr

⁡

{

∑

i

=

1

n

(

X

i

-

E

⁢

[

X

i

]

)

>

ε

}

≤

exp

⁡

[

-

v

2

M

2

⁢

(

(

1

+

M

⁢

ε

v

2

)

⁢

ln

⁡

(

1

+

M

⁢

ε

v

2

)

-

M

⁢

ε

v

2

)

]

.

Observing that

(

1

+

x

)

⁢

ln

⁡

(

1

+

x

)

-

x

≥

x

2

⁢

ln

⁡

(

1

+

x

)

∀

x

≥

0

(see here (

http://planetmath.org/ASimpleMethodForComparingRealFunctions

)), one gets the
sub-optimal yet more easily manageable formula:

Pr

⁡

{

∑

i

=

1

n

(

X

i

-

E

⁢

[

X

i

]

)

>

ε

}

≤

exp

⁡

[

-

ε

2

⁢

M

⁢

ln

⁡

(

1

+

ε

⁢

M

v

2

)

]

.

Title

proof of Bennett inequality

Canonical name

ProofOfBennettInequality

Date of creation

2013-03-22 16:12:27

Last modified on

2013-03-22 16:12:27

Owner

Andrea Ambrosio (7332)

Last modified by

Andrea Ambrosio (7332)

Numerical id

22

Author

Andrea Ambrosio (7332)

Entry type

Proof

Classification

msc 60E15

Generated on Sat Feb 10 11:56:24 2018 by

LaTeXML