1

Bounds on Variance for Unimodal Distributions
Hye Won Chung, Brian M. Sadler, and Alfred O. Hero

Abstract
We show a direct relationship between the variance and the differential entropy for subclasses of symmetric and asymmetric
unimodal distributions by providing an upper bound on variance in terms of entropy power. Combining this bound with the wellknown entropy power lower bound on variance, we prove that the variance of the appropriate subclasses of unimodal distributions
can be bounded below and above by the scaled entropy power. As differential entropy decreases, the variance is sandwiched
between two exponentially decreasing functions in the differential entropy. This establishes that for the subclasses of unimodal
distributions, the differential entropy can be used as a surrogate for concentration of the distribution.

arXiv:1510.08341v4 [] 2 Sep 2017

Index Terms
Differential entropy, variance bounds, unimodal distributions, Lipschitz continuity.

I. I NTRODUCTION
In this paper, we establish a direct relationship between variance and differential entropy of subclasses of symmetric and
asymmetric unimodal distributions over R. The variance of a random variable X having a distribution with density function
p(x) with mean m ∈ R is denoted by
Z
∞

Var(X) =

−∞

(x − m)2 p(x)dx,

and the differential entropy of X with density function p(x) by
Z ∞
h(p) = −
p(x) log p(x)dx.

(1)

(2)

−∞

Our main contribution in this paper is the observation that for appropriate subclasses of unimodal distributions, the variance
of p(x) can be bounded as
e2h(p)
ce2h(p)
≤ Var(X) ≤
(3)
2πe
2πe
for some constant c ≥ 1. Thus, for such unimodal densities, as h(p) → −∞, the variance is guaranteed to converge to 0.
The lower bound in (3) on variance is a well-known result and holds for all probability densities. This is established by the
estimation counterpart to Fano’s inequality (Theorem 8.6.6 in [2])
e2h(p)
≤ Var(X),
(4)
2πe
where equality is achieved for Gaussian distributions. This inequality shows that for general distributions, variance can approach
0 only if its differential entropy converges to −∞.
The question is then whether there exists a generally applicable upper bound on variance in terms of differential entropy
for all probability densities. The answer is negative and we can easily construct a counterexample. Consider the distribution

 


1
1
,
x ∈ −t − 2
, −t ∪ t, t + 2
,
p(x) =
(5)
0,
otherwise.
t
The differential entropy of this distribution is h(p) = log 1 , which is independent of t, but the variance is Var(X) = t2 + 2
+
1
,
which
increases
without
bound
in
t.
Thus
in
general
there
does
not
exist
an
upper
bound
on
variance
that
is
monotone
122
in differential entropy, and, even if the differential entropy of a distribution goes to −∞, the variance of this distribution can
be strictly larger than a positive constant.
However, for certain distributions, including Gaussian and uniform, there does exist a monotonic relationship between
variance and differential entropy. For a Gaussian distribution with mean m and variance σ 2 , denoted N (m, σ 2 ), the entropy
power, defined as e2h(p) , is proportional to the variance as

σ2 =

e2h(p)
.
2πe

(6)

Hye Won Chung (hwchung@kaist.ac.kr) is with the School of Electrical Engineering at KAIST in South Korea. Alfred O. Hero (hero@eecs.umich.edu) is
with the EECS department at the University of Michigan. Brian M. Sadler (brian.m.sadler6.civ@mail.mil) is with the US Army Research Laboratory. This
research was supported in part by ARO grants W911NF-11-1-0391 and W911NF-15-1-0479.

2

1
For a uniform distribution p(x) = unif(m − 2
,m +
log(1/). Thus, for uniform distributions, we have

1
2 ),

the variance is equal to 1/(122 ) and the differential entropy is

Var (X) =

e2h(p)
.
12

(7)

Therefore, for these cases, the variance is proportional to the entropy power e2h(p) .
Finding such a monotonic relationship between variance and differential entropy for a larger class of distributions than the
Gaussian or uniform is of broad interest with many applications in signal processing, machine learning, information theory,
probability, and statistics. For instance, in target localization or state estimation problems, differential entropy has been often
used as a natural objective for developing sensor selection or querying strategies to minimize estimation errors [3], [4], [5].
For example, in [5] an entropy-based sensor selection heuristic for target localization was proposed, which selects the most
informative sensor that would yield on average the greatest or nearly the greatest reduction in the entropy of the target location
distribution. The effectiveness of this heuristic was evaluated using simulations with Gaussian sensing models. However, there
are few theoretical guarantees for the performance of entropy-based policies for the non-Gaussian case, where a monotonic
relationship between variance and entropy is unavailable or unknown. Therefore, it is of great interest to find bounds of the
form (3) for a more general set of densities than the Gaussian in order to provide theoretical guarantees for diverse entropy-based
tasks such as waveform design or sensor selection.
In probability and statistics, there have been studies to find the most general classes of distributions that resemble Gaussian
distributions, for which several functional inequalities hold. In [6], [7], [8], an important observation was made that several
functional inequalities that hold for Gaussian, such as Poincare and logarithmic Sobolev inequalities as well as reverse entropy
power inequalities, also hold for a random variable X whose density function p(x) is log concave, i.e.,
p(αx + (1 − α)y) ≥ p(x)α p(y)1−α

(8)

for each x, y ∈ R and each 0 ≤ α ≤ 1. In particular, in [9], it was shown that for log-concave distributions the variance of
p(x) can be bounded in terms of the entropy power, e2h(p) , as
e2h(p)
≤ Var(X) ≤ c0 e2h(p) ,
(9)
2πe
for some positive constant c0 < 2. By using [10], [11], a tighter upper bound, which makes c0 as small as 1/2, can be shown.
In recent work [12], it was also shown that the entropy and p-th absolute moment of a symmetric log-concave random variable
are comparable. The fact that log-concave densities resemble Gaussian has led to several applications of log-concave densities
in inference and modeling [13], [14], [15].
In this paper, we establish the complementary result that resemblance to Gaussian can be extended to subclasses of unimodal
distributions that even includes some heavy-tailed distributions that are not log-concave. More precisely, we provide bounds of
the form (3) on the variance of subclasses of unimodal densities. These subclasses include symmetric unimodal linear mixture
densities of the form
n
n
X
X
p(x) =
αi pi (x) for αi > 0,
αi = 1,
(10)
i=1

i=1
−βi |x−m|θi

with any βi > 0, θi > 0, or uniform
that is a mixture of exponentially
decreasing distributions, pi (x) ∝ e

distributions, pi (x) = unif m − 21i , m + 21i with any i > 0, for i = 1, . . . , n. Also, we establish variance bounds for more
general (not necessarily linear mixture or symmetric) unimodal densities p(x) with bounded support [b − sl , b + sr ] having the
unique mode at x = b and satisfying Lipschitz continuity with constant cs > 0, i.e.,
|p(x + y) − p(x)| ≤ cs |y|

(11)

for any x, y ∈ [b − sl , b + sr ]. There exist many unimodal distributions considered in this paper that are not log-concave, e.g.,
the class of generalized Gaussian densities
θ
1
e−β|x−m| , for β > 0,
(12)
Z(θ, β)
R∞
θ
with order 0 < θ < 1 where Z is a normalizing constant: Z(θ, β) = −∞ e−β|x−m| dx. Note that these include some
heavy-tailed distributions.
Unimodal distributions and Gaussian mixtures have been widely studied and used in probability theory, statistics, signal
processing and machine learning, and in particular, for estimation and testing [16], [17], [18], [19], [20], [21]. Thus our
extended entropy upper bound on variance may have broad applicability. For example, in Bayesian sequential optimal design
of experiments [22], [23], [4], successive entropy minimization is often proposed as a way to progressively concentrate the
posterior distribution. When combined with the results of [9] our results provide additional justification for such approaches
when the posterior is either log-concave or included in subclasses of unimodal distributions that will be discussed in this paper.

p(x) =

3

The remainder of this paper is organized as follows. Section II provides precise statements of our main results, including
bounds on the variance of symmetric unimodal mixture densities (Theorem 1, Corollary 1 and Corollary 2), and bounds on
the variance of Lipschitz-continuous unimodal densities with bounded support (Theorem 2 and Theorem 3). In Section III,
we prove the upper bounds on variance of symmetric unimodal mixture densities and discuss the tightness of the bounds. To
prove these variance upper bounds, we assume that the ratio between the maximum and minimum variances of the mixture
components is bounded. When this assumption is violated, the variance of symmetric unimodal densities are not necessarily
dominated by a monotonic function of entropy power. We demonstrate this by providing a counterexample in Section IV. In
Section V, we prove the upper bounds on variance of Lipschitz-continuous unimodal densities with bounded support. More
technical aspects of these proofs are provided in the appendices. We conclude in Section VI.
Notation: We use the O(·) and Θ(·) notations to describe the asymptotics of real sequences {an } and {bn }: an = O(bn )
implies that an ≤ M bn for some positive real number M for all n ≥ n0 ; an = Θ(bn ) implies that an ≤ M bn and an ≥ M 0 bn
for some positive real numbers M and M 0 for all n ≥ n00 .
II. S TATEMENT OF M AIN R ESULTS
A. Bounds on Variance of Symmetric Unimodal Mixture Densities
We first consider symmetric unimodal linear mixture densities of the form
p(x) =

n
X

αi pi (x) for αi > 0,

n
X

i=1

αi = 1,

(13)

i=1
θi

that is a mixture of exponentially decreasing distributions, pi (x) ∝ e−βi |x−m| for any βi > 0, θi > 0, i = 1, . . . , n.
Theorem 1 establishes an upper bound on the variance of a linear mixture of exponentially decreasing densities under the
assumption that the ratio between the maximum and minimum variances of the mixture components pi (x) is bounded.
Theorem 1: Let p(x) be a symmetric unimodal linear mixture density of the form (13) with the mixture component pi (x) =
1
−βi |x−m|θi
where Zi (θi , βi ) is a normalizing constant and θi , βi > 0. Let σi2 denote the variance of pi (x), and
Zi (θi ,βi ) e
2
2
assume that then ratio
o of component variances σi /σj is bounded for all i 6= j by some positive constant, i.e., let r :=

maxi,j∈{1,...,n}

σi2
σj2

. Then the variance of the density p(x) satisfies

e2h(p)
≤ Var(X) ≤ B(θθ , r)e2h(p) .
2πe
Here
B(θθ , r) = M (r) ·

n 
Y
i=1

1
A(θi )

for θ = (θ1 , . . . , θn )T , where

αi
,

(15)

3

A(θ) = 4θ−2
with the Gamma function Γ(t) :=

(14)

R∞
0

(Γ (1/θ)) 2/θ
e ,
Γ (3/θ)

(16)

xt−1 e−x dx defined for t > 0, and,
1

M (r) :=

(r − 1)r r−1
,
e log r

r ≥ 1.

(17)

This theorem is proved in Section III-B. Equality in the variance lower bound in (14) is achieved if and only if p(x) is
a Gaussian distribution. The equality in the variance upper bound is met when all pi (x)’s are the same distribution, i.e.,
p(x) = pi (x) for ∀i. Therefore, the upper and lower bounds become equivalent when all pi (x)’s are the same Gaussian
distribution. For a mixture distribution of the form (13), possibly with pi (x) 6= pj (x) for some pairs of (i, j), the constant
scale B(θθ , r) in (15)
or equal to 1/2πe since the geometric mean of 1/A(θi ) ≥ 1/(2πe), ∀θi , with weights
Qn is greater than
α
{αi }, is given by i=1 (1/A(θi )) i ≥ 1/(2πe), and M (r) ≥ 1 with limr→1 M (r) = 1.
In Fig. 1, we compare the constant factor B(θθ , r) in the upper bound for three different mixture distributions. The constant
in the lower bound, 1/2πe, is also plotted as the dotted line. When n = 1, or equivalently, when all pi (x)’s are the same
distribution with θ = θi , ∀i, the factor B(θθ , r) = 1/A(θ). For this case, the equality in the variance upper bound (14) is
achieved. The curve 1/A(θ) is plotted as the solid line in Fig. 1. The minimum of 1/A(θ) is achieved at θ = 2 (for Gaussian
distributions) with the value 1/(2πe) ≈ 0.0585. As θ decreases below 2, 1/A(θ) increases, and it diverges as θ → 0. On the
other hand, as θ increases above 2, 1/A(θ) increases and converges to 1/12 ≈ 0.0833 as θ → ∞. These properties are proven
in Lemma 1. We compare this n = 1 case with those of two different mixture distributions when n = 2 and the mixture weights
α1 = α2 = 0.5. Specifically, consider the case when a Gaussian distribution with θ1 = 2 is mixed with another distribution
with θ2 = θ. If the variances of the two distributions are the same, i.e., r = 1, the resulting B(θθ , r) is the geometric mean

4

Fig. 1. A plot of 1/A(θ) in (16) vs. θ for n = 1 (solid line) and plots of B(θθ , r) in (15) for n = 2 with mixture weights α1 = α2 = 0.5 and decaying
orders θ1 = 2 and θ2 = θ when the ratio between variances r = 1 (dashed line) and when r = 10 (dash-dot line). The lower bound 1/2πe is also shown
(dotted line).

of 1/A(2) = 1/(2πe) and 1/A(θ) ≥ 1/(2πe), and thus B(θθ , r) is smaller than 1/A(θ) for all θ 6= 2 and equal to 1/A(θ) at
θ = 2, as shown by the dashed line. On the other hand, when the ratio between the variances of two mixture components is
r = 10, B(θθ , r) is increased by a factor of M (10) ≈ 1.86 compared to the case when r = 1, as shown by the dash-dot curve.
We summarize the properties of 1/A(θ) below.
Lemma 1: The function 1/A(θ) with A(θ) in (16) is decreasing on [0, 2] and increasing on [2, +∞). The minimum of 1/A(θ)
occurs at θ = 2 with the value 1/A(2) = 1/(2πe). As θ → 0, limθ→0 1/A(θ) = ∞, and as θ → ∞, limθ→∞ 1/A(θ) = 1/12.
Proof: Appendix A.
By using this lemma, we can further simplify the upper bound on variance of the mixture distribution when all θi ’s of the mixture
θi
component pi (x) = Zi (θ1i ,βi ) e−βi |x−m| , i = 1, . . . , n, are lower bounded by some positive constant. For example, when θi ≥ 1
Qn
αi
for all i, the maximum ratio between
and the lower bound is equal to 2πe
≤ 1/12.
i=1 (1/A(θi ))
12 M (r) since
Qn the upper bound
αi
When θi ≥ 1/2 for all i, since i=1 (1/A(θi )) ≤ (2e4 )/15, the maximum ratio between the upper bound and the lower bound
5
becomes 4πe
15 M (r). Note that when θi ≥ 1/2, ∀i, some of the mixture components pi (x) can be heavy-tailed distributions,
but the variance of p(x) can still be bounded above by a monotone function P
of the entropy power e2h(p)P
.
n
n
Corollary 1: Consider symmetric unimodal densities of the form p(x) = i=1 αi pi (x) for αi > 0, i=1 αi = 1, where
1
−βi |x−m|θi
with a normalizing constant Zi (θi , βi ) and βi > 0. When the order θi ≥ 1 for all i, the variance
pi (x) = Zi (θi ,βi ) e
is bounded as
e2h(p)
M (r) 2h(p)
≤ Var(X) ≤
e
.
(18)
2πe
12
When θi ≥ 1/2 for all i,
e2h(p)
2e4 M (r) 2h(p)
≤ Var(X) ≤
e
,
(19)
2πe
15
for M (r) in (17).
We next show that Theorem 1 implies a similar variance upper bound for a linear mixture of uniform distributions or
a mixture of exponentially decreasing distributions and uniform distributions. Let p be the uniform density on the interval
n
1
1
, m + 2
] with mean m ∈ R and  > 0. When we define a sequence of densities pn (x) = e−βn |x−m| /Zn with
[m − 2
n
βn = (2) and the normalizing constant Zn , the sequence of densities {pn } is almost everywhere convergent to p and
is dominated by some integrable function g in the sense that |pn (x)| ≤ g(x). By the dominated convergence theorem, the
variance and the entropy power of pn converge to the variance and the entropy power of p, respectively. For the sequence of
densities {pn }, the exponentially-decreasing order θ(= n) of the density goes to +∞ as n → ∞. In Lemma 1, we showed
that limθ→∞ 1/A(θ) = 1/12. By using this fact and Theorem 1, we can establish the bound on variance of a linear mixture
of uniform distributions.
Pn
Corollary 2:
form p(x) = i=1 αi pi (x) where
 Assume that p(x) is a bounded-support symmetric unimodal density ofn the
o
2
pi (x) = unif m − 21i , m + 21i for i > 0. Also, assume that r := maxi,j∈{1,...,n} i2 is bounded. Then the variance is
j

5

bounded as

e2h(p)
M (r) 2h(p)
≤ Var(X) ≤
e
,
2πe
12

(20)

with M (r) in (17).
The equality in the upper bound on variance is achievable when i = j for ∀i 6= j, i.e., when r = 1, since limr→1 M (r) = 1.
r
The ratio between the upper and the lower bound, which is proportional to M (r), increases in r and as r → ∞, M (r) ∼ e log
r,
which diverges.
Remark 1: The upper bounds
(18), (19) and (20) on variance of linear mixture densities are scaling-invariant.
For a
P(14),
Pn
n
linear mixture density p(x) = i=1 αi pi (x), consider a scaled version of the density pγ (x) = γp(γx) = i=1 αi (γpi (γx))
for some γ > 0. In order to show that the upper bounds (14), (18), (19) are scaling-invariant, it is enough to show that the
ratio r between the maximum and minimum variances of the mixture components γpi (γx) does not change in γ. This is true
since the variances of all the mixture components γpi (γx) are scaled by the same constant 1/γ 2 .
Remark 2: Note that in both Theorem 1 and Corollary 2, we assumedthe boundedness of the ratio r between themaximum
and minimum variances of the mixture components, i.e., maxi,j∈{1,...,n} σi2 /σj2 in Theorem 1 and maxi,j∈{1,...,n} 2i /2j in
Corollary 2 are bounded. In Section IV, by constructing a counterexample, we show that when the ratio between the maximum
and minimum variances of the mixture components is not bounded, the variance of a symmetric unimodal mixture density is
not necessarily dominated by a monotonic function of the entropy power.
θi

Remark 3: In Theorem 1, we considered a linear mixture of only exponentially decreasing distributions, pi (x) ∝ e−βi |x−m|
for any βi >0, θi > 0, i = 1, .. . , n, and in Corollary 2, we considered a linear mixture of only uniform distributions
pi (x) = unif − 21i + m, 21i + m for any i > 0, i = 1, . . . , n. But a similar upper bound holds for any linear mixture of
both exponentially decreasing distributions and uniform distributions, since the uniform distributions are the limit of particular
exponential distributions.
B. Bounds on Variance of Lipschitz-Continuous Unimodal Densities with Bounded Support
We next consider Lipschitz-continuous unimodal densities with bounded support. We first focus our discussion on Lipschitzcontinuous symmetric unimodal densities and establish a variance upper bound in terms of entropy power. We then generalize
this result for asymmetric unimodal densities.
With sufficiently large number of mixture components, any symmetric unimodal density can be arbitrarily closely approximated as a linear mixture of the form (13). This was established for the special case of uniform densities pi (x) by Feller
in [24]. We generalize this result as follows. Let p(x) be a symmetric unimodal density with bounded support [m − s, m + s].
Suppose that p(x) satisfies the Lipschitz-continuity condition with constant cs > 0, i.e.,
|p(x + y) − p(x)| ≤ cs |y|

(21)

for any x, y ∈ [m − s, m + s]. For such a p(x), we construct a linear mixture density p̄n (x) that approximates p(x) such that
the difference between entropy powers of p(x) and p̄n (x) and variances of p(x) and p̄n (x) can be bounded as
e2h(p̄n ) ≤ e2h(p) (1 + c1 n−1 log n),
Z m+s
(x − m)2 (p(x) − pn (x))dx ≤ c2 n−1

(22)

m−s

for some constants c1 , c2 > 0. We provide a construction of such a mixture density p̄n (x) and prove these bounds in Lemma 4
of Section V-A.
We also establish an upper bound on the variance of such a linear mixture density p̄n (x), denoted Var(p̄n ), such that
2


cs s2 ecs s 2h(p̄n )
Var(p̄n ) ≤
e
1 + c3 n−1
(23)
24
for some constant c3 > 0 in Lemma 5 of Section V-A.
By combining (22) with (23) and letting n → ∞, we establish an upper bound on variance of any Lipschitz-continuous
symmetric unimodal densities with bounded support, in terms of the entropy power.
Theorem 2: For any symmetric unimodal density p(x) with bounded support [m − s, m + s], when p(x) satisfies the Lipschitz
condition in (21) with constant cs > 0, the variance Var(X) can be bounded above and below by a constant scaling of entropy
power as
2
cs s2 ecs s 2h(p)
e2h(p)
≤ Var(X) ≤
e
.
(24)
2πe
24

6

triangle with
width: 2
2
height: cs s
area: cs s2

triangle with
width: 2s
height: cs s
area: cs s2

scaling by 1/s

max slope

cs s2

max slope

cs

p1 (x)
ps (x)

s

0

s

1

0

1

Fig. 2. When a symmetric unimodal density ps (x) with bounded support [−s, +s] satisfies the Lipschitz condition (21) with Lipschitz constant cs , then
the density ps (x) lies below a triangle that is symmetric about x = 0 with base 2s and with slope cs . The area of this triangle cs s2 is a parameter that
determines the variance upper bound in (24). For a density p1 (x) = sps (sx), the area of the associated triangle is still cs s2 since the Lipschitz constant for
p1 is cs s2 . Therefore, the parameter cs s2 , which appears in the variance upper bound in (24), is scaling invariant for any p ≡ ps with a chosen s.

Remark 4: The tightness of the upper bound in (24) depends on the parameter cs s2 where cs is the Lipschitz constant and
s is a half of the support size. From the assumption of Lipschitz continuity in (21), the maximum slope of p(x) is cs over
the support [m − s, m + s]. Consider a triangle that is symmetric about x = m with base 2s and with slope on each side cs ,
whose area is cs s2 . When the density p(x) is Lipshitz continuous with the maximum slope less than or equal to cs , then p(x)
lies below this symmetric triangle of area cs s2 . Since the area under p(x) is 1, we can see that cs s2 ≥ 1. The area of such a
symmetric triangle determines the tightness of the upper bound in (24).
Remark 5: We also show that the upper bound in (24), or more precisely the parameter cs s2 , is scaling-invariant. Suppose
that ps is a symmetric unimodal probability density with bounded support [−s, s]. We introduce a density p1 defined by
p1 (x) = sps (sx).

(25)

It is easy to see that p1 is a symmetric unimodal probability density with bounded support [−1, 1]. If ps is Lipschitz continuous
with constant cs as in (21) for any x, y, then
|p1 (x + y) − p1 (x)| = |sps (sx + sy) − sps (sx)| ≤ cs s2 |y|.

(26)

Thus, p1 is Lipschitz continuous with constant c1 = cs s2 . Since it is natural that the Lipschitz constant cs is proportional to
s−2 , the parameter cs s2 is independent of the scaling parameter s. This fact is illustrated in Fig. 2. Therefore, we can write
the upper bound in (24) for any p ≡ ps with a chosen s as
e2h(p)
c1 ec1 2h(p)
≤ Var(X) ≤
e
,
2πe
24
with a constant c1 = cs s2 > 0, which is scaling-invariant.

(27)

We next generalize this variance upper bound to the case of Lipschitz-continuous asymmetric unimodal densities. Consider
a unimodal density p(x) with bounded support [b − sl , b + sr ] with sl , sr > 0. Define s :=
max{sl , sr }. Assume that the
R +∞
unique mode of this density occurs at x = b. Let m denote the mean of this density, m := −∞ xp(x)dx. Suppose that p(x)
is Lipschitz continuous, i.e.,
|p(x + y) − p(x)| ≤ cs |y|
(28)
for any x, y ∈ [b − sl , b + sr ]. Then, the variance of this density p(x) can be bounded above by a constant scaling of entropy
power minus (m − b)2 , where the constant is a function of the Lipschitz constant cs and the maximum size s := max{sl , sr }
of one-sided support from the mode x = b.
Theorem 3: For any unimodal density p(x) over bounded support [b − sl , b + sr ] with mode x = b and mean m, when p(x)
satisfies the Lipschitz condition in (28), the variance Var(X) is bounded below and above in terms of the entropy power as

2
cs s2 ecs s M 128(cs s2 )4 2h(p)
e2h(p)
(29)
≤Var(X) ≤
e
− (m − b)2
2πe
6
with s := max{sl , sr } and M (r) in (17).
This theorem is proved in Section V-B.

7

Remark 6: In Section V-B, we derive an upper bound on the variance of asymmetric unimodal density p(x) in terms of a
parameter rv ≥ 1 defined for a density p(x) and of {βl , βr }, which are the areas under the density p(x) in the left and right
sides of the mode x = b, respectively, i.e.,
Z +∞
Z b
p(x)dx = 1 − βl .
(30)
p(x)dx,
βr :=
βl :=
b

−∞

The resulting upper bound on the variance Var(X) is
2

Var(X) ≤

cs s2 ecs s 4M (rv ) 2h(p)
e
− (m − b)2 .
24
e2HB (βl )

(31)

4
where HB (βl ) = −βl ln βl − βr ln βr . The upper bound in (29) is obtained by showing that rv ≤ 128 cs s2 and that M (r)
in (17) is an increasing function in r > 1 and by using the fact that HB (βl ) ≥ 0 for any βl > 0.
When the density p(x) is symmetric with mode x = b, then the mean m equals the mode b and the parameter rv = 1 and
the entropy HB (βl ) = ln 2 since βl = βr = 1/2. For this case, the upper bound (31) is specialized to (24), which we derived
for symmetric unimodal densities.
Remark 7: We show that the parameter cs s2 in the variance upper bound (29) is scaling-invariant due to the similar reasoning
as in Remark 5. Suppose that ps is a Lipschitz-continuous asymmetric unimodal density over bounded support [b − sl , b + sr ]
with Lipschitz constant cs . Define s := max{sl , sr }. Consider a density p1 defined by
p1 (x) = sps (s(x − b) + b).

(32)

It can be easily checked that p1 is a Lipschitz-continuous probability density over bounded support [b − sl /s, b + sr /s] with
mode x = b and Lipschitz constant c1 := cs s2 . For this density p1 , the maximum size of one-sided support from the mode
x = b equals 1 = max{sl /s, sr /s}. Therefore, for any density p ≡ ps with a chosen s, the variance of X ∼ p can be bounded
above by
c1 ec1 M (128(c1 )4 ) 2h(p)
e
− (m − b)2
(33)
Var(X) ≤
6
with a constant c1 = cs s2 > 0, which is scaling-invariant. Notice, however, that the term (m − b)2 is proportional to s2 .
III. U PPER B OUND ON THE VARIANCE OF S YMMETRIC U NIMODAL M IXTURE D ENSITIES
In Section III-A, we consider generalized Gaussian distributions and show that for this class of distributions, variance and
entropy power have an exact monotonic relationship. By using this result, we provide a proof of Theorem 1 in Section III-B.
A. Generalized Gaussian Distribution
The unimodality and symmetry of the Gaussian distribution motivate the definition of a larger class of distributions for which
variance has a monotonic relationship to its entropy power. Consider a symmetric unimodal class of generalized Gaussian
densities
θ
1
e−β|x−m| , for β, θ > 0
(34)
p(x) =
Z(θ, β)
R∞
θ
where Z is a normalizing constant: Z(θ, β) = −∞ e−β|x−m| dx. The normalizing constant Z and the variance Var(X) of
this distribution can be written in terms of β and θ as follows.
θ
1
Lemma 2: For distributions of the form p(x) = Z(θ,β)
e−β|x−m| for β, θ > 0, the normalizing constant Z and the variance
Var(X) can be expressed as
 
1
1
,
(35)
Z(θ, β) =2β − θ θ−1 Γ
θ

3
2 Γ
θ
Var(X) =β − θ
,
(36)
Γ θ1
R∞
where Γ denotes the Gamma function Γ(t) := 0 xt−1 e−x dx for t > 0.
Proof: Appendix B
We show that for the class of generalized Gaussian distributions in (34) with a fixed order parameter θ the variance has an
exact monotonic relationship to the entropy power. Note that this set of symmetric unimodal distributions include heavy-tailed
distributions when the order parameter θ is between 0 < θ < 1. Moreover, this set of distributions is not log-concave for
0 < θ < 1 so the results of [9] do not apply.

8

Lemma 3: For symmetric unimodal distributions of the form p(x) =
proportional to the entropy power
1 2h(p)
Var(X) =
e
,
A(θ)

1
−β|x−m|θ
Z(θ,β) e

for β, θ > 0, the variance is
(37)

with A(θ) in (16).
Proof: Appendix C
Important properties of 1/A(θ) are stated in Lemma 1 of Section II-A. By using these properties of 1/A(θ), when θ is
known to be larger than some positive constant, we can bound the variance of the generalized Gaussian distribution p(x) =
1
−β|x−m|θ
with β, θ > 0 in terms of a constant scaling of entropy power, ce2h(p) , with c independent of θ.
Z(θ,β) e
Corollary 3: For symmetric unimodal densities of the form p(x) =

1
−β|x−m|θ
Z(θ,β) e

with β > 0, when θ ≥ 1

Var(X) ≤

1 2h(p)
e
.
12

(38)

Var(X) ≤

2e4 2h(p)
e
.
15

(39)

When θ ≥ 1/2

B. Proof of Theorem 1
We next consider a generalization of Lemma 3 to a much broader class of symmetric unimodal distributions. Let us consider
θi
a mixture distribution p(x) composed of a finite number of exponentially decreasing distributions, pi (x) = Zi (θ1i ,βi ) e−βi |x−m|
with order θi > 0 and βi > 0 for i = 1, · · · , n, with mixture weights αi , i.e.,


n
X
1
−βi |x−m|θi
p(x) =
αi
e
(40)
Zi (θi , βi )
i=1
Pn
where αi > 0 and i=1 αi = 1. All the mixture components pi (x) have the same mean m to ensure unimodality of p(x).
−1/θi −1
Note that for each mixture component pi (x), the normalizing constant is Zi (θi , βi ) = 2βi
θi Γ (1/θi ) and the variance is
−2/θi Γ(3/θi )
2
σi := βi
,
as
shown
in
Proposition
2.
The
variance
of
the
symmetric
unimodal
mixture
density (40) is
Γ(1/θi )
Z ∞
n
X
Var(X) =
(x − m)2 p(x)dx =
αi σi2 .
(41)
−∞

i=1

For a linear mixture density p(x) of the form (40) we obtain an upper bound on the variance in terms of the entropy power,
as stated in Theorem 1. Here we present the proof.
Using the concavity of the differential entropy h(p) in distribution p(x),
h(p) ≥

n
X

αi h(pi ),

(42)

i=1

and thus
e2h(p) ≥e
As shown in Lemma 3, for pi (x) =

Pn

i=1

2αi h(pi )

n 
Y

=

e2h(pi )

αi

.

(43)

i=1

1
−βi |x−m|θi
,
Zi (θi ,βi ) e

σi2 =
and thus


e

2h(p)

≥

n
Y

1
· e2h(pi ) ,
A(θi )

αj 

A(θj )

j=1

·

n
Y

(44)

α i
σi2

!
.

(45)

i=1

By using the reverse power mean inequality shown in [25] (English version: p.79 in [26]), a lower bound on the geometric
mean of {σi2 } with orders {αi } in terms of the arithmetic mean of {σi2 } with orders {αi } is given by
n
X
i=1

αi σi2 ≤ M (r)

where M (r) is defined in (17) and r := maxi,j∈{1,...,n}

n

σi2
σj2

o
.

n
Y

i=1

σi2

αi

(46)

9

Pn
Pn
Since the variance of the mixture distribution p(x) = i=1 αi pi (x) is Var(X) = i=1 αi σi2 , by combining (45) and (46),
we obtain
n
n
X
Y
α i
Var(X) =
αi σi2 ≤ M (r)
σi2
≤ M (r)

i=1
n 
Y
i=1

i=1

1
A(θi )

αi !
e

2h(p)

(47)
.

IV. M IXTURE D ENSITIES WITH U NBOUNDED VARIANCE R ATIO
In both Theorem 1 and Corollary 2, we assumed
boundedness of the ratio r between the maximum
and minimum variances of


the mixture components, i.e., maxi,j∈{1,...,n} σi2 /σj2 in Theorem 1 and maxi,j∈{1,...,n} 2i /2j in Corollary 2 are bounded.
Here we show by a counterexample that when this assumption is violated, the variance of a symmetric unimodal mixture
density is not necessarily dominated by a monotonic function of the entropy power. The following example illustrates this
point. Consider a symmetric unimodal distribution composed of two uniform densities of the form


2
X
1 1
(48)
p(x) =
αi · unif − ,
2i 2i
i=1
P2
where αi > 0 and i=1 αi = 1 for 1 > 2 > 0. The variance of this distribution is equal to


1
1
1
Var(X) =
α1 2 + α2 2 ,
(49)
12
1
2
and the differential entropy of this distribution is
1
(α1 1 + α2 2 ) log(α1 1 + α2 2 )
1


1
1
−
−
α2 2 log(α2 2 ).
2
1

h(p) = −

(50)

When 1 /2 → ∞, the limit of the differential entropy becomes
lim

1 /2 →∞

h(p) = −α1 log 1 − α2 log 2 + HB (α1 )

where HB (α1 ) = −α1 log α1 − (1 − α1 ) log(1 − α1 ). Then, the entropy power becomes
 α1  α2
1
1
2h(p)
2HB (α1 )
lim e
=e
.
2
1
22
1 /2 →∞

(51)

(52)

In order to find an
 upper bound
 on the variance in (49) with theentropy
α1  power
α2 in (52), we need an upper bound on the
1
arithmetic mean α1 12 + α2 12 in terms of the geometric mean 12
. However, if 2 → 0 for a fixed 1 , since
22
1
2
1



(2α )
the arithmetic mean increases on the order of Θ 1/22 while the geometric mean on the order of Θ 1/2 2 , for α2 < 1
the variance increases much faster than e2h(p) so that it cannot be bounded above by any constant scaling of entropy power.
On the other hand, if 1 → ∞ for a fixed 2 , the arithmetic mean, which is proportional to the variance, is approximately


(2α )
α2 /22 , which is a constant, but the geometric mean, which is proportional to e2h(p) , goes to 0 on the order of Θ 1/1 1
for α1 < 1. Therefore, for both cases satisfying 1 /2 → ∞, the variance of p(x) cannot be bounded above by a constant
scaling of e2h(p) . This example shows that when the ratio between variances of mixture components is unbounded, there does
not necessarily exist an upper bound on the variance of a symmetric unimodal distribution that is a constant scaling of entropy
power.
V. U PPER B OUND ON VARIANCE OF L IPSCHITZ -C ONTINUOUS U NIMODAL D ENSITY WITH B OUNDED S UPPORT
In Section V-A, we first consider Lipschitz-continuous symmetric unimodal densities and provide a proof of Theorem 2. In
Section V-B, we generalize this result to Lipschitz-continuous asymmetric unimodal densities and provide a proof of Theorem 3.

10

cdf

pdf

F (x)

1

F̄n (x)
p(x)

p̄n (x)

s

s

0

x

s

s

0

x

Fig. 3. The cumulative distribution function (cdf) F (x) (solid line in the left figure) is approximated by the piecewise linear cdf F̄n (x) (dashed line in the
left figure) such that F̄n (x) = F (x) at discrete points x = kh for k ∈ {−n, . . . , n} where nk = s. The corresponding probability density functions (pdf)
p(x) (solid line) and p̄n (x) (dashed line) are shown in the right figure. The density p̄n (x) is a linear mixture of uniform densities.

A. Proof of Theorem 2: Lipschitz-Continuous Symmetric Unimodal Densities
In this section, we show that the variance of any Lipschitz-continuous symmetric unimodal density p(x) with bounded
support can be bounded above by a scaled entropy power of the form ce2h(p) for some constant c > 0. We emphasize that here
p(x) is not restricted to a mixture density. To prove the bound, we construct a linear mixture density p̄n (x) of uniform densities
that approximates p(x). The difference in variances and the difference in entropy powers of p(x) and of p̄n (x) are shown to
be monotonically decreasing as the number of mixture components, n, increases. Moreover, we establish an upper bound on
variance of X ∼ p̄n (x) in terms of the entropy power of p̄n (x). By combining this result with the fact that p̄n (x) approximates
p(x) with monotonically decreasing differences both in variance and in entropy power, we obtain a bound (24) on variance of
the Lipschitz-continuous symmetric unimodal density p(x) in terms of entropy power e2h(p) , as stated in Theorem 2.
Let us first construct a mixture density p̄n (x) that approximates p(x). Assume that p(x) is a Lipschitz-continuous symmetric
unimodal density with bounded support [−s, s], and denote its cumulative distribution function (cdf) as F (x). Here for simplicity
we consider the density with mean 0, but this result can be easily generalized to any symmetric unimodal density with an
arbitrary mean m. Let F̄n (x) be a cdf having the same value as F (x) at discrete points x = kh for k ∈ {−n, . . . , n} where
nh = s and n is an integer. Let F̄n (x) be linear in each interval [kh, (k + 1)h) for all k ∈ {−n, . . . , n − 1}. The distribution
F (x) is symmetric and unimodal if and only if F̄n (x) is symmetric and unimodal for any h > 0. Denote the density of the
distribution F̄n (x) by p̄n (x). Since F̄n (x) is a piecewise linear distribution, its density p̄n (x) can be written as a step function,
which is a linear mixture of uniform distributions with mean 0, i.e.,
n
X
αt · unif(−th, th).
(53)
p̄n (x) =
t=1

Fig. 3 illustrates the piecewise linear cdf F̄n (x) that approximates F (x) and the corresponding probability density functions
(pdfs) p̄n (x) and p(x).
From the definition of F̄n (x), F (x) = F̄n (x) at x = kh for k ∈ {−n, . . . , n}, and this implies that
Z (k+1)h
Z (k+1)h
(54)
p(x)dx =
p̄n (x)dx.
kh

kh

R (k+1)h
Since p̄n (x) is constant in each interval [kh, (k + 1)h), the value of p̄n (x) in this interval equals h1 kh
p(x)dx. From the
R
1 (k+1)h
fact that p̄n (x) is equal to the average value h kh
p(x)dx in each interval [kh, (k + 1)h) for k ∈ {−n, . . . , n − 1} and
that p(x) is monotone in each interval, we know that there exists at least one x ∈ [kh, (k + 1)h) for each k ∈ {−n, . . . , n − 1}
such that p̄n (x) = p(x).
From the representation of p̄n (x) in (53), for any x ∈ [kh, (k + 1)h) with k ∈ {−n, . . . , n − 1},
Z
n
X
1
1 (k+1)h
p̄n (x) =
αt
=
p(x)dx.
(55)
2th
h kh
t=|k|

These relations determine αt for t = 1, . . . , n − 1 and αn as
Z −(t−1)h
Z
αt =2t
p(x)dx −
−th

Z

−th

!
p(x)dx ,

−(t+1)h

−th

=2t
−(t+1)h

Z

(p(x + h) − p(x)) dx,
!

−s+h

αn =2n

p(x)dx .
−s

(56)

11

Since
Pn p(x) increases monotonically in [−s, 0], we can check that αi > 0 for all i ∈ {1, . . . , n}. From the definition of p̄n (x),
i=1 αi = 1.
We show that both the difference in variances and the difference in entropy powers of p(x) and of p̄n (x) decrease
monotonically in n as n goes to infinity.
Lemma 4: When a symmetric unimodal density p(x) with bounded support [−s, s] is Lipschitz continuous, p̄n (x) defined
in (53) approximates p(x) with bounded variance difference and bounded entropy power difference, which are monotonically
decreasing in n, such that
e2h(p̄n ) ≤ e2h(p) (1 + c1 n−1 log n),
Z s
Z s
x2 pn (x)dx ≤ c2 n−1 ,
x2 p(x)dx −

(57)

−s

−s

for some constants c1 , c2 > 0.
Proof: Appendix D.
We also prove that for the linear mixture density p̄n (x) in (53) with αt ’s in (56), the variance, denoted Var(p̄n ), can be
upper bounded in a scaled entropy power in the form of ce2h(p̄n ) for some constant c > 0.
Lemma 5: For a linear mixture density p̄n (x) in (53) with αt ’s in (56), the variance can be upper bounded as
2


cs s2 ecs s 2h(p̄n )
Var(p̄n ) ≤
e
1 + c3 n−1
24

(58)

for some constant c3 > 0.
Proof: Appendix E
By combining Lemma 4 and Lemma 5, we can prove that for any Lipschitz-continuous symmetric unimodal density p(x)
with bounded support, the variance can be bounded above by a constant scaling of entropy power:
2

Var(X) ≤

cs s2 ecs s 2h(p)
e
(1 + cn−1 log n),
24

for some constant c > 0. Now letting n → ∞,

(59)

2

cs s2 ecs s 2h(p)
e
.
24
When combined with the lower bound on variance (4), this proves Theorem 2.
Var(X) ≤

(60)

B. Proof of Theorem 3: Lipschitz-Continuous Asymmetric Unimodal Densities
In this section, we establish a variance upper bound for Lipschitz-continuous asymmetric unimodal distributions by using
the result of the previous section. RConsider a Lipschitz-continuous unimodal density p(x) over bounded support [b − sl , b + sr ]
+∞
with mode x = b and mean m = −∞ xp(x)dx. Let {βl , βr } denote the areas under the density p(x) in the left and right sides
of the mode x = b, respectively, as defined in (30). Assume that the density p(x) satisfies the Lipschitz-continuity condition
with constant cs > 0, i.e.,
|p(x + y) − p(x)| ≤ cs |y|
(61)
for any x, y ∈ [b − sl , b + sr ].
By using p(x), we define two symmetric unimodal densities pl (x)

1

 2βl p(x),
1
pl (x) = 2βl p(−x + 2b),


0,

1

 2βr p(−x + 2b),
pr (x) = 2β1r p(x),


0,

and pr (x) with the unique mode x = b such that
x ∈ [b − sl , b],
x ∈ (b, b + sl ],
otherwise,
x ∈ [b − sr , b]
x ∈ (b, b + sr ],
otherwise.

(62)

Since both pl and pr are Lipschitz-continuous symmetric unimodal densities with Lipschitz constant cs > 0 and support
[b − sl , b + sl ] and [b − sr , b + sr ], respectively, by using Theorem 2, the variances of X ∼ pl and of X ∼ pr are bounded
above in terms of their respective entropy powers as
2

cs s2 ecs sl 2h(pl )
Var(pl ) ≤ l
e
,
24
2
cs s2 ecs sr 2h(pr )
Var(pr ) ≤ r
e
.
24

(63)

12

Moreover, the variance Var(X) of the original asymmetric unimodal density p(x) can be represented as

and the entropy power eh(p) of p(x) as

Var(X) = βl Var(pl ) + βr Var(pr ) − (m − b)2 ,
e2h(p) =

(64)

1  2h(pl ) βl  2h(pr ) βr 2HB (βl )
e
e
e
4

(65)

where HB (βl ) = −βl ln βl − βr ln βr .
From the reverse power mean inequality [25] (English version: p.79 in [26]), we have
β

βl Var(pl ) + βr Var(pr ) ≤ M (rv ) (Var(pl )) l (Var(pr ))

βr

(66)

for rv := max{Var(pl )/Var(pr ), Var(pr )/Var(pl )} with M (r) in (17). By applying this inequality to (64), we obtain
β

βr

Var(X) ≤ M (rv ) (Var(pl )) l (Var(pr ))

− (m − b)2 .

(67)

Moreover, by using the variance upper bounds in (63) and the fact that βl + βr = 1, we obtain
2

Var(X) ≤

cs s2 ecs s
M (rv )e2βl h(pl ) e2βr h(pr ) − (m − b)2
24

(68)

for s := max{sl , sr }. Finally, by using (65), the variance can be bounded above in terms of e2h(p) as
2

cs s2 ecs s 4M (rv ) 2h(p)
e
− (m − b)2 .
(69)
24
e2HB (βl )
In Lemma 6, we show that rv ≤ 128(cs s2 )4 . Furthermore, notice that M (r) is an increasing function of r for r > 1, which
can be easily check from that
1
r r−1 −1 (r − log r − 1)(−r + r log r + 1)
0
> 0,
(70)
M (r) =
e(r − 1) log2 r
Var(X) ≤

where we used that log r < r − 1 < r log r for r > 1. By using these results and the fact that HB (βl ) ≥ 0, the variance upper
bound in (69) can be further simplified as

4 
2
cs s2 ecs s M 128 cs s2
Var(X) ≤
e2h(p) − (m − b)2 .
(71)
6
This completes the proof of Theorem 3.
Lemma 6: Consider two Lipschitz-continuous symmetric unimodal densities pl and pr in (62), defined in terms of a Lipschitzcontinuous unimodal density p with bounded support [b − sl , b + sr ] having the unique mode at x = b. Let s = max{sl , sr },
and assume that the density p satisfies the Lipschitz-continuity condition (61) with constant cs . Then, the parameter rv :=
max{Var(pl )/Var(pr ), Var(pr )/Var(pl )} is bounded as
rv ≤ 128(cs s2 )4 .

(72)

Proof: Appendix F.
VI. C ONCLUSION AND F UTURE D IRECTIONS
In this paper, we established upper bounds on the variance of subclasses of unimodal distributions in terms of the entropy
power. We first considered symmetric unimodal mixture densities of exponentially decreasing distributions and uniform
distributions. The tightness of the upper bounds on variance depends on the ratio between the maximum and minimum
variances of the mixture components. By constructing a counterexample, we showed that when the ratio between variances of
the mixture components is unbounded, there may not necessarily exist an upper bound on the variance of symmetric unimodal
mixture densities that is a constant scaling of entropy power. We also showed that the variance of any Lipschitz-continuous
unimodal density with a bounded support can be upper bounded in terms of the entropy power. All the upper bounds on the
variance of unimodal densities presented in this paper are scaling-invariant.
In signal processing, adaptive sensing, and machine learning, information theoretic surrogates such as Kullback-Leibler
divergence, entropy, and Fisher information have been widely adopted in place of task-specific cost functions such as mean
squared error or probability of classification error. Since such task-specific cost functions are often intractable, informationtheoretic surrogates are used as natural objectives for developing waveforms or sensor selection strategies to collect and/or
filter information. The results reported in this paper can be used to justify the use of differential entropy as a surrogate for the
mean squared error in such applications with a provable performance guarantee, when the posterior distribution is Lipschitzcontinuous and unimodal with bounded support or when it can be approximated by a linear mixture of exponentially decreasing
densities and uniform densities.

13

One of the interesting future research directions is to extend this work to a multivariate setting. Concentration inequalities for
functions of a large number of random variables have been an active research topic with diverse applications in high-dimensional
statistics, machine learning and information theory. Recent progress in concentration inequalities and their applications to
information theory can be found in an excellent monograph [27]. Among many interesting aspects, we are particularly interested
in finding conditions on the multivariate distribution to derive a concentration inequality in terms of entropy power. When the
random variables composing a k-dimensional random vector X = [X1 , X2 , . . . , Xk ] are mutually independent and the marginal
distribution pi of each random variable Xi are unimodal and satisfying conditions discussed in this paper, the variance Var(Xi )
2h(pi )
of each random variable Xi is upper bounded
for some constant ci > 0 and the determinant
of the
Qk by Var(Xi ) ≤ ci e
Qk
k × k covariance matrix Σ equals |Σ| = i=1 Var(Xi ). Moreover, the entropy power e2h(X) equals e2h(X) = i=1 e2h(pi ) .
By using these facts, we can easily extend our work to the multivariate setting and show that
|Σ| ≤ ce2h(X)

(73)

for some positive constant c > 0. Therefore, for this case of the multivariate distribution with independent random variables
each satisfying the Lipschitz-continuity and unimodality, as the entropy h(X) of the random vector X tends to −∞, the
determinant of the covariance matrix, which is the product of the variances of the marginal distributions, converges to 0 and
the random vector concentrates around its expectation E[X]. One might want to find a similar concentration result for more
general multivariate distributions by bounding the determinant of the covariance matrix. But it is important to note that the
determinant of the covariance matrix does not always capture the concentration of the k-dimensional distribution when the
random variables are not mutually independent. For example, when X1 = X2 = · · · = Xk , regardless of how large the variance
of Xi is, the determinant of the covariance matrix equals 0. Therefore, an interesting open question is to find a proper metric
which captures the concentration of a multivariate distribution and to derive a concentration inequality for the metric in terms
of entropy power for an appropriate class of distributions.
A PPENDIX A
P ROOF OF L EMMA 1
In Lemma 1, we show that the function 1/A(θ) is decreasing on [0, 2] and increasing on [2, +∞) where
3

A(θ) = 4θ−2

(Γ (1/θ)) 2/θ
e .
Γ (3/θ)

(74)

The minimum of 1/A(θ) occurs at θ = 2 with the value 1/A(2) = 1/(2πe). As θ → 0, limθ→0 1/A(0) = ∞, and as θ → +∞,
limθ→+∞ 1/A(θ) = 1/12. To prove this lemma, we first show that A(θ) is unimodal with the maximum reached at 2, i.e.,
increasing on [0, 2] and decreasing on [2, +∞). The derivative of A(θ) with respect to θ equals
A0 (θ)
2

(75)
(Γ(1/θ)) 2/θ 
Γ(1/θ)Γ0 (3/θ)(3/θ2 )
e
− 3Γ0 (1/θ)(1/θ2 ) − Γ(1/θ)(2/θ2 )
− 2θ−1 Γ(1/θ) +
Γ (3/θ)
Γ(3/θ)
R ∞ z−1 −x
where Γ(z) is the Gamma function Γ(z) := 0 x e dx with z > 0 and Γ0 (z) is the derivative of Γ(z), which equals
Γ0 (z) = Γ(z)ψ0 (z) where ψ0 (z) is the polygamma function
"
#
∞ 
X
1
1
1
+γ+
−
ψ0 (z) = −
(76)
z
n+z
n
n=1

= 4θ−2

with the Euler-Mascheroni constant γ. By using Γ0 (z) = Γ(z)ψ0 (z), A0 (θ) can be written as



  
 
3
(Γ(1/θ)) 2/θ
2
1
3
3
1
A0 (θ) = 4θ−2
e
−
1+
+ 2 ψ0
− ψ0
,
Γ (3/θ)
θ
θ
θ
θ
θ

(77)

and by using (76), A0 (θ) can be further simplified as
A0 (θ) = 4θ−4


  
 
3
3
1
(Γ(1/θ)) 2/θ
e
−2 + 3 φ
−φ
Γ (3/θ)
θ
θ

where
φ(x) =

∞ 
X
1
n=1

which converges absolutely on (0, ∞).

n

−

1
n+x

(78)


,

(79)

14

3

2/θ
Since 4θ−4 (Γ(1/θ))
> 0 for θ > 0, the sign of A0 (θ) is determined by (−2 + 3 (φ (3/θ) − φ (1/θ))). When we define
Γ(3/θ) e
f (θ) := φ (3/θ) − φ (1/θ), we can show that f (θ) is a strictly decreasing function of θ. In particular,
2
f (θ) > f (2) =
for θ < 2,
(80)
3
and
2
f (θ) < f (2) =
for θ > 2.
(81)
3
For ease of notation, we let a = 1/θ and show that f (θ) = f (1/a) = φ(3a) − φ(a) is a strictly increasing function of a. Note
that
∞
X
1
.
(82)
φ0 (x) =
(n + x)2
n=1

It suffices to show that (φ(3a) − φ(a))0 = 3φ0 (3a) − φ0 (a) > 0. This can be easily checked from that
∞
X
1
0
φ (3a) =
(n
+
3a)2
n=1
∞ 

X
1
1
1
+
+
=
(3m − 2 + 3a)2
(3m − 1 + 3a)2
(3m + 3a)2
m=1


∞
1 X
1
1
1
+
+
=
9 m=1 (m + a − 23 )2
(m + a)2
(m + a − 31 )2


∞
1
1
1 X
1
+
+
>
9 m=1 (m + a)2
(m + a)2
(m + a)2

(83)

∞
1
1 X
=
3 m=1 (m + a)2

φ0 (a)
.
3
This proves (80) and (81), which imply that A0 (θ) > 0 for 0 < θ < 2, A0 (θ) = 0 at θ = 2, and A0 (θ) < 0 for θ > 2. Thus,
A(θ) is increasing on [0, 2] and decreasing on [2, +∞).
+ 1) ∼
√ We next
z prove the statement that as θ → 0, limθ→0 1/A(0) = ∞. By using Stirling’s formula, as z → ∞, Γ(z
2πz ze . q
Using this approximation and the propertyqthat Γ(z + 1) = zΓ(z), it can be shown that as θ → 0, θ1 Γ θ1 =

1


3
1 θ
3 θ
Γ θ1 + 1 ∼ 2π θ1 θe
and θ3 Γ θ3 = Γ θ3 + 1 ∼ 2π θ3 θe
. Therefore, as θ ↓ 0, i.e., 1/θ → ∞,
√
1
A(θ) ∼ 8 3πθ−1 e θ (2−3 log 3) ,
(84)
=

which tends to 0 since (2 − 3 log 3) < 0. Therefore, 1/A(θ) diverges as θ → 0.
We next show that as θ → ∞, limθ→∞ 1/A(θ) = 1/12. From the Gamma function property that zΓ(z) = Γ(z + 1) and
θ3
Γ(1) = 1, it follows that limz→0 zΓ(z) = 1, i.e, Γ(z) ∼ z1 as z → 0. Thus, when θ → ∞, A(θ) ∼ 4θ−2 θ/3
exp (2/θ) and
limθ→∞ 1/A(θ) = 1/12.
A PPENDIX B
P ROOF OF L EMMA 2
R∞
R∞
θ
θ
The normalizing constant Z is Z(θ, β) = −∞ e−β|x−m| dx = 2 0 e−βx dx. Let βxθ = y. Then, βθxθ−1 dx = dy and
θ−1
xθ−1 = (y/β) θ . Thus, Z can be written in terms of y as
 
Z ∞
1
− θ1 −1
− θ1 −1 −1+ θ1 −y
e dy = 2β θ Γ
.
(85)
Z(θ, β) =2
β θ y
θ
0
In a similar way,
Z ∞
θ
1
Var(X) =
(x − m)2 e−β|x−m| dx
Z(θ, β) −∞
Z ∞
θ
2
=
x2 e−βx dx
Z(θ, β) 0
Z ∞
(86)
3
2
− θ3 −1
=
β θ
y −1+ θ e−y dy
Z(θ, β)
0

 
3
3
2
3
− θ −1
− θ2 Γ θ 
=
β θ Γ
=β
.
Z(θ, β)
θ
Γ θ1

15

A PPENDIX C
P ROOF OF L EMMA 3
For p(x) =

1
−β|x−m|θ
Z(θ,β) e

where β, θ > 0, the differential entropy can be directly calculated as follows.

Z ∞ −β|x−m|θ 
e
1
θ
h(p) = −
log
− β|x − m| dx
Z(θ, β)
−∞ Z(θ, β)
Z ∞
θ
2β
xθ e−βx dx.
= log Z(θ, β) +
Z(θ, β) 0

(87)

1

Let βxθ = y. Then, βθxθ−1 dx = dy and x = (y/β) θ . The differential entropy becomes
Z ∞
1
2β
β −1 θ−1 (y/β) θ e−y dy
h(p) = log Z(θ, β) +
Z(θ, β) 0
Z
1
2β − θ θ−1 ∞ 1 −y
y θ e dy
= log Z(θ, β) +
Z(θ, β) 0


1
1
2β − θ θ−1
Γ 1+
.
= log Z(θ, β) +
Z(θ, β)
θ

(88)

By using the normalizing constant Z in (85) and the variance in (86) as well as the property that Γ(1 + z) = zΓ(z) for z > 0,
the entropy power can be written in terms of the variance as follows
2

2

e2h(p) =4β − θ θ−2 (Γ (1/θ)) e2/θ
=A(θ) · Var(X)
where

(89)

3

A(θ) = 4θ−2

(Γ (1/θ)) 2/θ
e .
Γ (3/θ)

(90)

A PPENDIX D
P ROOF OF L EMMA 4
Suppose that p(·) satisfies the Lipschitz condition with constant cs > 0, i.e.,
|p(x + y) − p(x)| ≤ cs |y|

(91)

for any x, y. From the Lipschitz continuity of p(x) and the fact that there exists x ∈ [kh, (k + 1)h) such that p̄n (x) = p(x)
for every k ∈ {−n, . . . , n − 1}, the difference between p(x) and p̄n (x) can be bounded as
|p(x) − p̄n (x)| ≤ cs h

(92)

for any x ∈ [kh, (k + 1)h). Let Mk := maxx∈[kh,(k+1)h) p(x), then
|p(x) − p̄n (x)| ≤ min{Mk , cs h}
for every x ∈ [kh, (k + 1)h), since p(x), p̄n (x) ≥ 0 for any x ∈ R.
The difference in variances of p(x) and of p̄n (x) can be bounded as
Z s
Z s
2
x p(x)dx −
x2 p̄n (x)dx
−s
−s
Z s
≤
x2 |p(x) − p̄n (x)|dx
−s
Z s
=2
x2 |p(x) − p̄n (x)|dx
=2

0
n−1
X Z (k+1)h
kh

x2 |p(x) − p̄n (x)|dx

k=0
n−1
X Z (k+1)h

≤ 2cs h

k=0

kh

where h = s/n. The last inequality in (94) is from (92).

x2 dx =

2cs s4 1
2
cs hs3 =
3
3 n

(93)

(94)

16

Consider next the difference in entropies of p(x) and of p̄n (x), denoted by h(p) and h(p̄n ), respectively. Define f (y) =
−y log y for y > 0 with f (0) = 0. We approximate f (p(x)) for any x ∈ [kh, (k + 1)h) by a Taylor expansion at p̄n (x) =
R
1 (k+1)h
p(x)dx,
h kh
df (y)
(p(x) − p̄n (x)) + O((p(x) − p̄n (x))2 /p̄n (x))
dy y=p̄n (x)
= − p̄n (x) log p̄n (x) − (1 + log p̄n (x))(p(x) − p̄n (x)) + O((p(x) − p̄n (x))2 /p̄n (x)).

f (p(x)) =f (p̄n (x)) +

(95)

Thus,
|f (p(x)) − f (p̄n (x))| ≤|1 + log p̄n (x)| · |p(x) − p̄n (x)| + O((p(x) − p̄n (x))2 /p̄n (x)).

(96)

To further bound the right hand side of this inequality, we next find an upper bound on |1+log p̄n (x)| for x ∈ [kh, (k+1)h). Note
R (k+1)h
that p̄n (x) for x ∈ [kh, (k + 1)h) is equal to the area of p(x) over [kh, (k + 1)h) divided by h, i.e., p̄n (x) = h1 kh
p(x)dx.
Since p(x) is a unimodal distribution, in each sub-interval [kh, (k + 1)h), p(x) is either monotonically increasing (when
−n ≤ k ≤ −1) or monotonically decreasing (when 0 ≤ k ≤ n − 1). Let us consider the case when −n ≤ k ≤ −1. For
this case Mk = maxx∈[kh,(k+1)h) p(x) = p((k + 1)h). From the Lipschitz continuity of p(x), the maximum slope of p(x) is
smaller than cs . Therefore, for x ∈ [kh, (k + 1)h) with −n ≤ k ≤ −1 we can bound p(x) as
p(x) ≥ max{cs (x − kh) + Mk − cs h, 0}.

(97)

The area under p(x) in the interval [kh, (k + 1)h) is thus lower bounded by the area under the curve max{cs (x − kh) + Mk −
cs h, 0} over x ∈ [kh, (k + 1)h), i.e.,
Z
1 (k+1)h
p(x)dx
p̄n (x) =
h kh
(

(98)
Mk − cs2h ≥ cs2h , when Mk ≥ cs h,
≥ 1 M2
k
when Mk < cs h.
h 2cs ,
R (k+1)h
ss
When Mk ≥ cs h, from (98) and h = s/n, it can be shown that c2n
≤ p̄n (x) = h1 kh
p(x)dx ≤ h1 = ns , which implies
|1 + log p̄n (x)| ≤ c log n for some constant c > 0. Combining this bound with |p(x) − p̄n (x)| ≤ cns s from (93), we obtain
|1 + log p̄n (x)| · |p(x) − p̄n (x)| ≤

c0 log n
n

(99)

for some constant c0 > 0.
When Mk < cs h, from (93) it can be shown that |p(x) − p̄n (x)| ≤ Mk < cns s , and from (98) it can be shown that
R
Mk2 n
1 (k+1)h
p(x)dx ≤ h1 = ns . Since |Mk log Mk2 | ≤ 2cns s log cns s for Mk ≤ cns s ≤ e−1 with a sufficiently large
2cs s ≤ p̄n (x) = h kh
n,
c00 log n
|1 + log p̄n (x)| · |p(x) − p̄n (x)| ≤
(100)
n
for some constant c00 > 0. Therefore, by using (99) and (100), we can bound |f (p(x)) − f (p̄n (x))| in (96) as
c log n
+ O(n−1 )
(101)
n
for some constant c > 0. Even though we proved this bound for x ≤ 0, i.e, for x ∈ [kh, (k + 1)h) with −n ≤ k ≤ −1, due to
symmetry of p(x) about x = 0, this bound also holds for any x ≥ 0. Therefore, the difference in entropies of p(x) and p̄n (x)
can be bounded as
|h(p) − h(p̄n )|
Z s
=
(−p(x) log p(x) + p̄n (x) log p̄n (x)) dx
|f (p(x)) − f (p̄n (x))| ≤

=
≤
≤

−s
n−1
X

Z

(k+1)h

k=−n kh
n−1
X Z (k+1)h
k=−n
n−1
X
k=−n

kh

h

(f (p(x)) − f (p̄n (x)))dx
|f (p(x) − f (p̄n (x))|dx

c log n
+ O(n−1 )
n

= 2ch log n + O(n−1 ) =

2cs log n
+ O(n−1 ).
n

(102)

17

This bound then implies that
e2h(p̄n ) ≤ e2h(p) (1 + c2 n−1 log n)

(103)

for some constant c2 > 0.
A PPENDIX E
P ROOF OF L EMMA 5
Suppose that p(x) satisfies the Lipschitz condition with constant cs > 0, i.e.,
|p(x + y) − p(x)| ≤ cs |y|

(104)

for any x, y. From (56), for t = 1, . . . , n,
Z

−th

αt = 2t
−(t+1)h
2

(p(x + h) − p(x)) dx

(105)

≤ 2cs th ,
Pn
where h = s/n. We also have the relation t=1 αt = 1.
For the linear mixture density p̄n (x) in (53),
p̄n (x) =

n
X
t=1

the variance of X ∼ p̄n (x) is
Var(p̄n ) =

αt · unif(−th, th),

 2
n
n
1X
s2 X
t
αt (th)2 =
αt
3 t=1
3 t=1
n

(106)

n
2cs s4 X t3
cs s4
≤
=
+ O(n−1 ),
4
3 t=1 n
6

and the entropy of X ∼ p̄n (x) is
h(p̄n ) ≥

n
X

αt log(2th) = log(2s) +

t=1

n
X

αt log

t=1

≥ log(2s) +

n
X

2cs th2 log

t=1

t
n

t
n

n
X
t
t
log
= log(2s) + 2cs s2
2
n
n
t=1

= log(2s) −

(107)

cs s2
+ O(n−1 ),
2

where the last equality holds since
Z 1
n
X
t
t
1
log
=
x log x dx + O(n−1 ) = − + O(n−1 ).
2
n
n
4
0
t=1
From (107), the entropy power e2h(p̄n ) is bounded below as
e2h(p̄n ) ≥
for some constant c03 > 0.
From (106) and (108), we obtain

(108)

2

Var(p̄n ) ≤
for some constant c3 > 0.


4s2
1 − c03 n−1
2
c
s
es


cs s2 ecs s 2h(p̄n )
e
1 + c3 n−1
24

(109)

18

A PPENDIX F
P ROOF OF L EMMA 6
In Lemma 6, we show that the parameter rv = max{Var(pl )/Var(pr ), Var(pr )/Var(pl )} for Lipschitz-continuous symmetric
unimodal densities pl and pr in (62), defined as a function of a Lipschitz-continuous unimodal density p(x) over bounded
support [b − sl , b + sr ] with the unique mode at x = b, is bounded above as
rv ≤ 128(cs s2 )4

(110)

where s = max{sl , sr } and cs is the Lipschitz constant of p(x) such that
|p(x) − p(x + y)| ≤ cs |y|.

(111)

From the definition of the two densities pl and pr in (62), the variances Var(pl ) and Var(pr ) of the respective densities are
Z
1 b
Var(pl ) =
(x − b)2 p(x)dx,
βl b−sl
(112)
Z b+sr
1
2
Var(pr ) =
(x − b) p(x)dx,
βr b
where {βl , βr } denote the areas under the density p(x) in the left and right of the mode x = b, respectively, as defined in (30)
To find an upper bound on rv = max{Var(pl )/Var(pr ), Var(pr )/Var(pl )}, we first find an upper bound on rβ := max{βl /βr , βr /βl }.
Assume without loss of generality that βl ≤ 1/2 ≤ βr . Let t = p(b). Then, for any x < b, we have from the Lipschitz condition
|p(b) − p(x)| ≤ cs |b − x| that
p(x) ≥ t − cs |b − x|.
(113)
Integrating the above inequality, we find that
Z

b

βl =
−∞

p(x)dx ≥

Z

b

p(x)dx =
b−t/cs

t2
.
2cs

(114)

It can be also shown that sr t ≥ βr from the unimodality condition. Hence,
rβ =

2βr cs
2cs s2r
βr
≤
≤
≤ 4cs s2r ≤ 4cs s2 .
2
βl
t
βr

(115)

We next find an upper bound on
( Rb
ri := max

(x − b)2 p(x)dx

b−sl
R b+sr
(x
b

− b)2 p(x)dx

R b+sr
,

(x − b)2 p(x)dx

b
Rb
(x
b−sl

− b)2 p(x)dx

)
.

(116)

Since p(x) has the unique mode at x = b, when we let t = p(b), p(x) satisfies
t − cs |b − x| ≤ p(x) ≤ t
for any x ∈ [b − sl , b + sr ]. By using this bound, we can show that
Z b
t4
ts3
≤
(x − b)2 p(x)dx ≤ l ,
3
12cs
3
b−sl
Z b+sr
4
t
ts3r
2
≤
(x
−
b)
p(x)dx
≤
.
12c3s
3
b

(117)

(118)

Therefore, the parameter ri in (116) is bounded above as
4(cs s)3
(119)
t3
for s = max{sl , sr }. Since p(x) has the unique mode at x = b with the value t = p(b), t is bounded below as t ≥ 1/(2s).
Hence,
ri ≤ 32(cs s2 )3 .
(120)
ri ≤

By using (115) and (120) and from the definition of rv , it can be shown that
rv ≤ rβ ri ≤ 128(cs s2 )4 .

(121)

19

R EFERENCES
[1] H. W. Chung, B. M. Sadler, and A. O. Hero, “Bounds on variance for symmetric unimodal distributions,” in 2015 53rd Annual Allerton Conference on
Communication, Control, and Computing (Allerton). IEEE, 2015, pp. 1235–1240.
[2] T. M. Cover and J. A. Thomas, Elements of Information Theory. John Wiley & Sons, 2006.
[3] B. Jedynak, P. I. Frazier, R. Sznitman et al., “Twenty questions with noise: Bayes optimal policies for entropy loss,” Journal of Applied Probability,
vol. 49, no. 1, pp. 114–136, 2012.
[4] T. Tsiligkaridis, B. M. Sadler, and A. O. Hero, “Collaborative 20 questions for target localization,” IEEE Transactions on Information Theory, vol. 60,
no. 4, pp. 2233–2252, 2014.
[5] H. Wang, K. Yao, G. Pottie, and D. Estrin, “Entropy-based sensor selection heuristic for target localization,” in Proceedings of the 3rd International
Symposium on Information Processing in Sensor Networks. Berkeley, California, USA: ACM, April 2004, pp. 36–45.
[6] D. Bakry, F. Barthe, P. Cattiaux, and A. Guillin, “A simple proof of the Poincaré inequality for a large class of probability measures including the
log-concave case,” Electron. Commun. Probab, vol. 13, pp. 60–66, 2008.
[7] S. G. Bobkov et al., “Isoperimetric and analytic inequalities for log-concave probability measures,” The Annals of Probability, vol. 27, no. 4, pp.
1903–1921, 1999.
[8] M. Madiman, J. Melbourne, and P. Xu, “Forward and reverse entropy power inequalities in convex geometry,” in Convexity and Concentration. Springer,
2017, pp. 427–485.
[9] S. Bobkov and M. Madiman, “The entropy per coordinate of a random vector is highly constrained under convexity conditions,” IEEE Transactions on
Information Theory, vol. 57, no. 8, pp. 4940–4954, 2011.
[10] D. Hensley, “Slicing convex bodies–bounds for slice area in terms of the body’s covariance,” Proceedings of the American Mathematical Society, vol. 79,
no. 4, pp. 619–625, 1980.
[11] S. Webb, “Central slices of the regular simplex,” Geometriae Dedicata, vol. 61, no. 1, pp. 19–28, 1996.
[12] A. Marsiglietti and V. Kostina, “A lower bound on the differential entropy for log-concave random variables with applications to rate-distortion theory,”
in IEEE International Symposium on Information Theory (ISIT), 2017. IEEE, 2017, pp. 46–50.
[13] G. Walther, “Inference and modeling with log-concave distributions,” Statistical Science, pp. 319–327, 2009.
[14] M. Bagnoli and T. Bergstrom, “Log-concave probability and its applications,” Economic Theory, vol. 26, no. 2, pp. 445–469, 2005.
[15] A. Saumard and J. A. Wellner, “Log-concavity and strong log-concavity: a review,” Statistics Surveys, vol. 8, p. 45, 2014.
[16] B. P. Rao, “Estimation of a unimodal density,” Sankhyā: The Indian Journal of Statistics, Series A, pp. 23–36, 1969.
[17] E. J. Wegman, “Maximum likelihood estimation of a unimodal density, II,” The Annals of Mathematical Statistics, pp. 2169–2174, 1970.
[18] P. Groeneboom, “Estimating a monotone density,” Department of Mathematical Statistics, no. R 8403, pp. 1–14, 1984.
[19] L. Birge, “Estimation of unimodal densities without smoothness assumptions,” The Annals of Statistics, pp. 970–981, 1997.
[20] J. A. Hartigan and P. Hartigan, “The dip test of unimodality,” The Annals of Statistics, pp. 70–84, 1985.
[21] A. Eskenazis, P. Nayar, and T. Tkocz, “Gaussian mixtures: entropy and geometric inequalities,” arXiv preprint arXiv:1611.04921, 2016.
[22] M. H. DeGroot, “Uncertainty, information, and sequential experiments,” The Annals of Mathematical Statistics, pp. 404–419, 1962.
[23] D. Geman and B. Jedynak, “An active testing model for tracking roads in satellite images,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 18, no. 1, pp. 1–14, 1996.
[24] W. Feller, An introduction to probability theory and its applications. John Wiley & Sons, 2008, vol. 2.
[25] W. Specht, “Zur theorie der elementaren mittel,” Mathematische Zeitschrift, vol. 74, no. 1, pp. 91–98, 1960.
[26] D. S. Mitrinovic and P. M. Vasic, Analytic inequalities. Springer, 1970, vol. 1.
[27] M. Raginsky and I. Sason, Concentration of Measure Inequalities in Information Theory, Communications, and Coding. Now Publishers Inc., 2014.

