# Generalized Information for theγ-Order Normal Distribution

Authors: Thomas L. Toulias
Journal: Journal of Probability and Statistics
 (2015)

---

## Abstract

This paper investigates a generalization of Fisher’s entropy type information measure under the multivariateγ-order normal distribution, related to his
measure, as well as its corresponding Shannon entropy. Certain boundaries of this information measure are also proved and discussed.


---

## Body

## 1. Introduction

The Poincaré inequality is one of the most well-known results of the theory of the Sobolev spaces; that is, we can obtain bounds on a functionf belonging to the Sobolev space H
1
(
R
p
,
μ
)
≔
f
∈
L
2
(
R
p
,
μ
)
:
E
μ
(
f
)
<
∞ using the bounds on the derivatives, while the domain is still important. The energy E
μ
f of a local μ-integrable function f with ∇
f
∈
L
2
(
R
p
,
μ
) is defined as(1)
E
μ
f
≔
E
μ
∇
f
2
.The corresponding Poincaré constant, c
P, can be easily evaluated when the domain is convex. It holds that (2)
V
a
r
μ
f
≤
c
P
E
μ
f
,where E
μ
(
f
) and V
a
r
μ
f are, respectively, the expected value and the variance of f, corresponding to the probability measure μ; that is, E
μ
f
=
∫
‍
f
d
μ and V
a
r
μ
f
=
E
μ
[
(
f
-
E
μ
f
)
2
]
=
E
μ
f
2
-
(
E
μ
f
)
2. Under some regularity conditions for the measure μ, there exists a constant c
P
∈
(
0
,
+
∞
) such that the Poincaré inequality as in (2) can be written as (3)
V
a
r
μ
f
≤
c
P
∫
R
p
∇
f
2
d
μ
,with f being a differentiable function having compact support. That is, we need to evaluate bounds for the variance and therefore for the information, either the parametric or the entropy type.The Logarithmic Sobolev Inequalities (LSI) attempt to estimate the lower-order derivatives of a given function in terms of higher-order derivatives. The well-known LSI was introduced in 1938; see [1, 2] for details. The introductory and well-known Sobolev Inequality is (4)
∫
R
p
f
x
2
p
/
p
-
2
d
x
p
-
2
/
2
p
≤
c
S
∫
R
p
∇
f
x
2
d
x
1
/
2
,or, using the norm notation, (5)
f
q
≤
c
S
∇
f
2
,
q
=
2
p
p
-
2
.The constant c
S is known as Sobolev constant. Since then, various attempts were made to generalize (4). The first optimal Sobolev Inequality was of the form (6)
∫
R
p
f
x
n
p
/
p
-
n
d
x
p
-
n
/
n
p
≤
C
p
,
n
∫
R
p
∇
f
x
n
d
x
1
/
n
,with n
∈
[
1
,
p
).The usual normal distribution, also known as Gaussian, plays an important role to all statistical problems as do the information measures related to it. New entropy type information measures were introduced in [3], generalizing the known Fisher’s entropy type information measure, while an exponential-power generalization of the usual normal distribution was introduced and studied in [3–5]. This generalized normal, calledthe
γ
-order normal distribution, defined in Section 2, emerges as an extremal function for the Logarithmic Sobolev Inequality.In particular, following [3], the Gross Logarithm Inequality with respect to the Gaussian weight [1] is of the form(7)
∫
R
p
g
2
log
⁡
g
2
d
m
≤
1
π
∫
R
p
∇
g
2
d
m
,where g
2
≔
∫
R
p
‍
g
(
x
)
2
d
x
=
1 is the norm in L
2
(
R
p
,
d
m
) with d
m
≔
e
x
p
{
-
π
|
x
|
2
}
d
x. Inequality (7) is equivalent to the (Euclidean) LSI: (8)
∫
R
p
u
2
log
⁡
u
2
d
x
≤
p
2
log
⁡
2
π
p
e
∫
R
p
∇
u
2
d
x
,for any function u
∈
H
1
/
2
(
R
p
) with u
2
=
1, and H
1
/
2 being the Sobolev space; see [3] for details. This inequality is optimal, in the sense that (9)
2
π
p
e
=
inf
⁡
∫
R
p
∇
u
2
d
x
exp
⁡
2
/
n
∫
R
p
u
2
log
⁡
u
2
d
x
:
u
2
=
1
;see [6]. Extremal functions for inequality (8) are precisely the Gaussian distribution (10)
u
x
=
π
σ
2
-
p
/
4
exp
⁡
-
x
-
μ
σ
2
,with σ
>
0 and μ
∈
R
p; see [7, 8] for details. Now, consider the extension by del Pino et al. in [9] for the LSI as in (8). For any u
∈
H
1
/
2
(
R
p
) with u
γ
=
1, the γ-LSI holds; that is,(11)
∫
R
p
u
γ
log
⁡
u
d
x
≤
p
γ
2
log
⁡
K
γ
∫
R
p
∇
u
γ
d
x
,with the optimal constant K
γ being equal to (12)
K
γ
=
γ
p
γ
-
1
e
γ
-
1
π
-
γ
/
2
Γ
p
/
2
+
1
Γ
p
γ
-
1
/
γ
+
1
γ
/
p
,where Γ
(
·
) is the usual gamma function. Inequality (11) is optimal and the equality holds when (13)
u
x
=
u
x
;
μ
,
Σ
,
γ
≔
C
γ
p
Σ
exp
⁡
-
γ
-
1
γ
Q
θ
x
γ
/
2
γ
-
1
,
x
∈
R
p
,with normalizing factor(14)
C
γ
p
Σ
≔
γ
-
1
/
γ
p
γ
-
1
/
γ
π
p
/
2
det
⁡
Σ
Γ
p
/
2
+
1
Γ
p
γ
-
1
/
γ
+
1
=
max
⁡
uand p-quadratic form Q
θ
(
x
)
≔
(
x
-
μ
)
Σ
-
1
(
x
-
μ
)
T, x
∈
R
p. The function ϕ
(
γ
)
=
u
1
/
γ
(
x
) with Σ
=
(
σ
2
/
α
)
2
(
γ
-
1
)
/
γ
I
p corresponds to the extremal function for the LSI due to [9]. The essential result is that the defined u
(
x
) can be considered as a probability density function (p.d.f.) of an r.v. X and works as an extremal function to a generalized form of the Logarithmic Sobolev Inequality; see Section 2.Various attempts were made in the past to generalize the normal distribution; see [10, 11]. The privilege of the introduced family of the γ-order normal distribution is that there is a theoretical insight due to the generalization of Fisher’s information and the use of the LSI and not just a technical extension as in [11].This is why there is an interest to have at least inequalities among various statistical-analytical information measures concerning the Gaussian as well as theγ-order normal distribution: to be able to compare the “information” we can obtain for an experiment, which usually is assumed to follow the Gaussian distribution (see Section 3).In principle, the information measures are divided into three main categories: parametric (a typical example Fisher’s information), nonparametric (with Shannon information measure to be the most well known), and entropy type (see [12]) which are adopted in this paper. The introduced new entropy type measure of information J
α
(
X
) is a function of the density f of the p-variate random variable X (see [3]) defined as(15)
J
α
X
≔
E
∇
log
⁡
f
X
α
=
∫
R
p
f
x
∇
l
o
g
f
x
α
d
x
,where · is the usual L
2
(
R
p
) norm. Notice that J
2
=
J, with J being the known Fisher entropy type information measure.Moreover, the known entropy powerN
(
X
), defined through the Shannon entropy H
(
X
), has been extended to (16)
N
α
X
≔
ν
α
exp
⁡
α
p
H
X
,where(17)
ν
α
≔
α
-
1
e
π
-
α
/
2
Γ
p
/
2
+
1
Γ
p
α
-
1
/
α
+
1
α
/
p
,
α
>
1
;see [3] for details. Notice that ν
2
=
(
2
π
e
)
-
1 and N
2
=
N, where N is the known Shannon entropy power for the normal distribution. It is interesting that [3] (18)
J
α
X
N
α
X
≥
p
,which extends the well-known Information Inequality, obtained for α
=
2.The so-called Information Inequality is generalized due to the introduced information measures [3]. The Generalized Information Inequality (GII) is given by (19)
2
π
e
p
V
a
r
X
1
/
2
ν
α
p
J
α
X
1
/
α
≥
1
.When α
=
2 we have V
a
r
(
X
)
J
2
(
X
)
≥
p
2, and, therefore, the Cramer-Rao inequality ([12], Theorem  11.10.1) holds. The lower boundary B
α
p for the introduced generalized information J
α
(
X
) is (20)
J
α
X
≥
B
α
p
≔
p
ν
α
2
π
e
p
V
a
r
X
-
α
/
2
.We are also interested to have at least inequalities among various statistical-analytical information measures concerning the Gaussian as well as theγ-order normal distribution: to be able to compare the “information” we can obtain for an experiment, which usually is assumed to follow the Gaussian distribution (see Section 3).


## 2. Theγ-Order Normal Distribution and the Generalized Fisher Information

LetH denote the Shannon (or differential) entropy. For any multivariate random variable X with zero mean and covariance matrix Σ, it holds that(21)
H
X
≤
1
2
log
⁡
2
π
e
p
det
⁡
Σ
,while the equality in (21) holds if and only if X is a normally distributed variable; that is, X
~
N
p
(
μ
,
Σ
); see [12]. Moreover, the normal distribution, according to Information Measures Theory, is adopted for the noise, acting additively to the input variable when an input-output time discrete channel is formed. Therefore, the Gaussian distribution needs a special treatment evaluating the LSI.Kitsos and Tavoularis [3, 4] introduced and studied the multivariate and spherically contoured γ-order normal distribution, denoted with N
γ
p
(
μ
,
Σ
) with Σ
=
σ
2
I
p. See also [5, 13] for further reading. Recall the definition of the N
γ
p family of distributions.Definition 1.
Thep-dimensional random variable X follows the γ-order normal distribution N
γ
p
(
μ
,
Σ
) with location vector μ
∈
R
p
×
1, shape parameter γ
∈
R
∖
[
0,1
], and positive definite scale matrix Σ
∈
R
p
×
p, when the density function f
X is of the form (22)
f
X
x
;
μ
,
Σ
,
γ
≔
C
γ
p
det
⁡
Σ
exp
⁡
-
γ
-
1
γ
Q
θ
x
γ
/
2
γ
-
1
,
x
∈
R
p
×
1, where Q
θ, θ
≔
(
μ
,
Σ
), is the quadratic form Q
θ
(
x
)
≔
(
x
-
μ
)
T
Σ
-
1
(
x
-
μ
), x
∈
R
p
×
1. The normalizing factor C
γ
p is defined by (23)
C
γ
p
≔
π
-
p
/
2
Γ
p
/
2
+
1
Γ
p
γ
-
1
/
γ
+
1
γ
-
1
γ
p
γ
-
1
/
γ
.From the p.d.f.f
X as above, notice that the location vector of X is essentially the mean vector of X; that is, μ
=
E
(
X
). Moreover, for the shape parameter value γ
=
2, N
2
p
(
μ
,
Σ
) is reduced to the well-known multivariate normal distribution. We also comment that the function ϕ
(
α
)
=
f
α
(
μ
,
Σ
)
1
/
α, with Σ
=
(
σ
2
/
α
)
2
(
α
-
1
)
/
α
I
p, corresponds to an extremal function for inequality extending LSI due to [9]. The essential result is that the defined γ-order normal distribution works as an extremal function to a generalized form of the Logarithmic Sobolev Inequality.Various attempts to generalize the usual normal distribution are known. The introduced univariateγ-order normal N
γ
(
μ
,
σ
2
) coincides with the existent generalized normal distribution introduced in [11], with density function (24)
f
x
=
f
x
;
μ
,
a
,
b
≔
b
2
a
Γ
1
/
b
exp
⁡
-
x
-
μ
a
b
,
x
∈
R
,when a
≔
σ
[
γ
/
(
γ
-
1
)
]
(
γ
-
1
)
/
γ and b
≔
γ
/
(
γ
-
1
). The multivariate case of the γ-order normal N
γ
p
(
μ
,
Σ
) coincides with the existent multivariate power exponential distribution P
E
p
(
μ
,
Σ
′
,
b
), as introduced in [10], where Σ
′
≔
2
2
(
γ
-
1
)
/
γ
Σ and b
≔
1
/
2
γ
/
(
γ
-
1
). See also [14]. These exponential-power generalizations as above are technically obtained (involving an extra parameter b). On the contrary, the form of the γ-order normal N
γ is obtained as an extremal of a generalized LSI, and therefore N
γ is having a strong mathematical background. Moreover, the family of N
γ distributions acts as the usual normal distribution for the Information Inequality by obtaining the equality. Recall that J
(
X
)
N
(
X
)
=
p for X
~
N
p
(
μ
,
Σ
). In fact, the generalized form of the Information Inequality, as in (18), is reduced into equality for the γ-order normally distributed random variable, as it holds that J
α
(
X
)
N
α
(
X
)
=
p for X
~
N
γ
p
(
μ
,
Σ
); see [3] for details.Denote withE
θ the area included by the p-ellipsoid defined by the quadratic form Q
θ; that is, E
θ
≔
{
x
∈
R
p
×
1
:
Q
θ
(
x
)
≤
1
}. The family of N
γ
p
(
μ
,
Σ
), that is, the family of the elliptically contoured γ-order normals, provides a smooth bridging between the multivariate (and elliptically countered) uniform, normal, Laplace, and the degenerate Dirac distributions, that is, between the r.v. U
~
U
p
(
μ
,
Σ
), Z
~
N
p
(
μ
,
Σ
), L
~
L
p
(
μ
,
Σ
), and D
~
D
p
(
μ
) (with pole at the point μ), with probability density functions (25)
f
U
x
;
μ
,
Σ
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
,
x
∈
E
θ
,
0
,
x
∉
E
θ
,
(26)
f
Z
x
;
μ
,
Σ
=
1
2
π
p
/
2
det
⁡
Σ
exp
⁡
-
1
2
Q
x
,
(27)
f
L
x
;
μ
,
Σ
=
Γ
p
/
2
+
1
p
!
π
p
/
2
det
⁡
Σ
exp
⁡
-
Q
x
,
(28)
f
D
x
;
μ
=
+
∞
,
x
=
μ
,
0
,
x
∈
R
p
×
1
∖
μ
,
x
∈
R
p
×
1, respectively. That is, the N
γ
p family of distributions generalizes not only the usual normal but also two other very significant distributions, as the uniform and the Laplace distributions.Theorem 2.
The multivariateγ-order normal distribution, N
γ
p
(
μ
,
Σ
), for order values of γ
=
1,2
,
±
∞, coincides with (29)
N
γ
p
μ
,
Σ
=
D
p
μ
,
f
o
r
γ
=
0
,
p
=
1,2
,
0
,
f
o
r
γ
=
0
,
p
≥
3
,
U
p
μ
,
Σ
,
f
o
r
γ
=
1
,
N
p
μ
,
Σ
,
f
o
r
γ
=
2
,
L
p
μ
,
Σ
,
f
o
r
γ
=
±
∞
.Proof.
The p.d.f. definition (22) of N
γ
p
(
μ
,
Σ
) is depending on the real-valued shape parameter γ defined outside the closed interval [
0,1
]. We let X
γ
~
N
p
(
μ
,
Σ
) and denote g
≔
γ
-
1
/
γ. We distinguish the following cases: (i)
The uniform caseγ
=
1. From (22) with x
∈
E
θ, that is, E
θ or Q
θ
(
x
)
≤
1, we get (30)
l
i
m
γ
→
1
+
f
X
γ
x
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
l
i
m
g
→
0
+
exp
⁡
-
g
Q
θ
x
-
1
/
2
g
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
·
1
·
e
0
,
while, forx
∈
R
p
∖
E
θ, that is, Q
θ
(
x
)
>
1, we obtain (31)
l
i
m
γ
→
1
+
f
X
γ
x
;
μ
,
Σ
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
·
1
·
0
=
0
,
due to the fact thatg
x
1
/
g
→
+
∞ as g
→
0
+ for all x
∈
R
+
∗
≔
R
+
∗
∖
0. Therefore, from (25), the third branch of (29) holds true as f
X
1
≔
l
i
m
γ
→
1
+
f
X
γ, or N
1
p
(
μ
,
Σ
)
≔
l
i
m
γ
→
1
+
N
γ
p
(
μ
,
Σ
)
=
U
p
(
μ
,
Σ
) through (25). That is, the multivariate first-ordered normal distribution coincides with the elliptically contoured uniform distribution.
(ii)
The Gaussian caseγ
=
2. It is clear that N
2
p
(
μ
,
Σ
)
=
N
p
(
μ
,
Σ
), as f
X
2 coincides with the multivariate (and elliptically contoured) normal density function as in (26). That is, the multivariate second-ordered normal distribution coincides with the usual elliptically contoured normal distribution.
(iii)
The Laplace caseγ
=
±
∞. For the limiting case of g
(
=
γ
-
1
/
γ
)
→
1 (as γ
→
±
∞), we derive that N
±
∞
p
(
μ
,
Σ
)
≔
l
i
m
γ
→
±
∞
N
γ
p
(
μ
,
Σ
)
=
L
p
(
μ
,
Σ
) as f
X
±
∞
≔
l
i
m
γ
→
±
∞
f
X
γ clearly coincides with the multivariate (and elliptically contoured) Laplace density as in (27). That is, the multivariate infinite-ordered normal distribution coincides with the elliptically contoured Laplace distribution.
(iv)
The degenerate Dirac caseγ
=
0. Firstly, we assume that x
=
μ; that is, Q
θ
(
x
)
=
0, and hence, from definition (22), (32)
f
X
γ
μ
=
g
p
g
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
Γ
p
g
+
1
.
Given that(33)
f
X
0
μ
≔
l
i
m
γ
→
0
-
f
X
γ
μ
=
l
i
m
g
=
γ
-
1
/
γ
→
+
∞
f
X
γ
μ
=
l
i
m
k
=
p
g
→
∞
f
X
γ
μ
,
where[
x
] is the integer value of x
∈
R, we obtain (34)
f
X
0
μ
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
l
i
m
k
→
∞
k
k
p
k
k
!
.
Utilizing now Stirling’s asymptotic formula,k
!
≈
2
π
k
(
k
/
e
)
k as k
→
∞, (34) implies(35)
f
X
0
μ
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
l
i
m
k
→
∞
1
2
π
k
p
/
e
k
,
and, thus, forp
≥
3
>
e, (35) implies f
X
0
(
μ
)
=
0, while for p
=
1 or p
=
2 it implies f
X
0
(
μ
)
=
+
∞.
Assuming now thatx
≠
μ and using (34), we have(36)
f
X
0
x
=
l
i
m
γ
→
0
-
f
X
γ
μ
l
i
m
g
→
+
∞
exp
⁡
-
g
Q
θ
x
1
/
2
g
,
and hence, forp
≥
3
>
e, (36) implies that f
X
0
(
x
)
=
0 (due to the fact that g
x
1
/
g
→
0 for all x
∈
R
+
∗ as g
→
+
∞) while, for p
=
1 or p
=
2, applying (35) into (36) we obtain (37)
f
X
0
x
=
Γ
p
/
2
+
1
π
p
/
2
det
⁡
Σ
l
i
m
k
→
∞
e
x
p
1
-
1
/
p
Q
θ
x
p
/
2
k
p
k
2
π
k
.
That is,f
X
0
=
0. Therefore, it is clear that N
0
p
(
μ
,
Σ
)
≔
l
i
m
γ
→
0
-
N
γ
p
(
μ
,
Σ
)
=
D
p
(
μ
) for p
=
1,2, as p.d.f. f
X
0 coincides with the multivariate Dirac density f
D given in (25). Therefore, the univariate and bivariate zero-ordered normal distributions with mean at μ
∈
R coincide with the (univariate and bivariate) degenerate Dirac distributions with poles at the point μ. Moreover, the p-variate, p
≥
3, zero-ordered normal has a degenerated (zero) density function.
Considering the above cases of (i), (iii), and (iv) we can now extend the defining values of the shape parameterγ of the N
γ distribution to include the limiting values of γ
=
0,1
,
±
∞, respectively; that is, γ can now be defined outside the real open interval (
0,1
). Eventually, the uniform, normal, Laplace, and also the degenerate distributions, as the Dirac or the flat one, can be considered as members of the N
γ
p family of distributions.The following proposition calculates the Shannon entropy for theγ-order normally distributed random variable. Recall that the well-known Shannon (or differential) entropy measure is defined by [12] (38)
H
X
≔
E
log
⁡
f
X
=
∫
R
p
f
x
log
⁡
f
x
d
x
,where the usual minus sign is omitted, for a p-variate continuous random variable X with density function f.Proposition 3.
The Shannon entropy of a random variableX
γ
~
N
γ
p
(
μ
,
Σ
) is of the form (39)
H
X
γ
=
p
γ
-
1
γ
-
log
⁡
C
γ
p
det
⁡
Σ
.Proof.
Consider the p.d.f.f
X
γ as in (22). From the definition (38) we have that the Shannon entropy of X is (40)
H
X
γ
=
-
log
⁡
C
γ
p
Σ
+
C
γ
p
Σ
γ
-
1
γ
∫
R
p
Q
θ
x
γ
/
2
γ
-
1
exp
⁡
-
γ
-
1
γ
Q
θ
x
γ
/
2
γ
-
1
d
x
,where C
γ
p
(
Σ
) denotes the normalized coefficient C
γ
p
·
(
d
e
t
Σ
)
-
1
/
2 of the generalized Gaussian kernel e
x
p
{
-
(
1
/
g
)
Q
θ
g
/
2
(
x
)
} with g
≔
γ
/
(
γ
-
1
) of the p.d.f f
X
γ. Applying the linear transformation z
=
(
x
-
μ
)
T
Σ
-
1
/
2 with d
x
=
d
(
x
-
μ
)
=
d
e
t
Σ
d
z, the H
(
X
γ
) above is reduced to (41)
H
X
γ
=
-
log
⁡
C
γ
p
Σ
+
C
γ
p
γ
-
1
γ
∫
R
p
z
γ
/
γ
-
1
exp
⁡
-
γ
-
1
γ
z
γ
/
γ
-
1
d
z
,where I
p denotes the p
×
p identity matrix. Switching to hyperspherical coordinates, we get (42)
H
X
γ
=
-
log
⁡
C
γ
p
Σ
+
C
γ
p
γ
-
1
γ
ω
p
-
1
∫
R
+
r
γ
/
γ
-
1
exp
⁡
-
γ
-
1
γ
r
γ
/
γ
-
1
r
p
-
1
d
r
,where ω
p
-
1
≔
2
π
p
/
2
/
Γ
(
p
/
2
) is the volume of the (
p
-
1
)-sphere. Applying the variable change d
u
≔
d
(
γ
-
1
/
γ
r
γ
/
(
γ
-
1
)
)
=
r
1
/
(
γ
-
1
)
d
r we obtain successively(43)
H
X
γ
=
L
+
C
γ
p
ω
p
-
1
∫
R
+
u
e
-
u
r
p
-
1
γ
-
1
-
1
/
γ
-
1
d
u
=
L
+
C
γ
p
I
p
ω
p
-
1
∫
R
+
u
e
-
u
ρ
γ
/
γ
-
1
p
-
1
γ
-
1
-
1
/
γ
d
u
=
L
+
C
γ
p
I
p
ω
p
-
1
γ
γ
-
1
p
γ
-
1
/
γ
-
1
∫
R
+
u
p
γ
-
1
/
γ
e
-
u
d
u
=
L
+
p
γ
-
1
γ
Γ
p
γ
-
1
γ
C
γ
p
ω
p
-
1
,where L
≔
-
l
o
g
C
γ
p
(
Σ
). Finally, by substitution of the volume ω
p
-
1 and the normalizing factors C
γ
p
(
Σ
) and C
γ
p, relation (39) is obtained.Applying Theorem2 to the above Proposition 3 the following holds.Corollary 4.
The usual Shannon entropy for the multivariate (and elliptically countered) uniform, normal, and Laplace distributed random variables is given by(44)
H
X
=
log
⁡
π
p
/
2
det
⁡
Σ
Γ
p
/
2
+
1
,
f
o
r
U
p
μ
,
Σ
,
1
2
log
⁡
2
π
e
p
det
⁡
Σ
,
f
o
r
N
p
μ
,
Σ
,
p
+
log
⁡
p
!
π
p
/
2
det
⁡
Σ
Γ
p
/
2
+
1
,
f
o
r
L
p
μ
,
Σ
,
+
∞
,
f
o
r
X
~
N
0
p
μ
,
Σ
.For the univariate case p
=
1, we are reduced to (45)
H
X
=
log
⁡
2
σ
,
f
o
r
X
~
U
μ
-
σ
,
μ
-
σ
,
log
⁡
2
π
e
σ
,
f
o
r
X
~
N
μ
,
σ
2
,
1
+
log
⁡
2
σ
,
f
o
r
X
~
L
μ
,
σ
,
+
∞
,
f
o
r
X
~
N
0
p
μ
,
σ
2
,where U
(
μ
-
σ
,
μ
-
σ
), N
(
μ
,
σ
2
), and L
(
μ
,
σ
) are the usual notations for the univariate uniform, normal, and Laplace distributions, respectively.Proof.
For the normal case ofγ
=
2 and the Laplace case of γ
=
±
∞, or γ
-
1
/
γ
=
1 (in limit), the second and the third branch of (44) follows easily from (39).
For the uniform case ofγ
=
1 we obtain that C
1
p
≔
l
i
m
γ
→
1
+
C
γ
p
=
π
-
p
/
2
Γ
(
p
/
2
+
1
) and hence the first branch of (44) holds. Moreover, the corresponding univariate case p
=
1 yields H
(
X
)
=
l
o
g
2
σ for X
~
N
1
1
(
μ
,
σ
2
) or, through (25), X
~
U
(
a
,
b
) which is the known (continuous) uniform distribution where a
=
μ
-
σ and b
=
μ
+
σ. Thus, H
(
X
)
=
l
o
g
(
b
-
a
) using the usual notation of U
(
a
,
b
).
Consider now the degenerate case ofγ
=
0. We can write (39) in the form (46)
H
X
γ
=
log
⁡
π
p
/
2
det
⁡
Σ
Γ
p
/
2
+
1
·
Γ
p
g
+
1
g
/
e
p
g
,where X
γ
~
N
γ
p
(
μ
,
Σ
) and g
=
γ
-
1
/
γ. We then have (47)
l
i
m
γ
→
0
-
H
X
γ
=
log
⁡
π
p
/
2
det
⁡
Σ
Γ
p
/
2
+
1
l
i
m
k
≔
p
g
→
∞
p
k
k
!
k
/
e
k
,and using Stirling’s asymptotic formula, k
!
≈
2
π
k
(
k
/
e
)
k as k
→
∞, (47) yields (48)
l
i
m
γ
→
0
-
H
X
γ
=
log
⁡
2
π
det
⁡
Σ
π
p
/
2
l
i
m
k
→
∞
p
k
k
Γ
p
/
2
+
1
=
+
∞
,which proves the corollary.The generalized Fisher information measure has been calculated for the spherically contouredN
γ
p random variable [15]. We extend this with the following theorem.Theorem 5.
The generalized Fisher informationJ
α of an r.v. X
γ
~
N
γ
p
(
μ
,
Σ
), where Σ is a real matrix that consisted of orthogonal vectors (columns) with the same norm, is given by (49)
J
α
X
γ
=
γ
/
γ
-
1
α
/
γ
Γ
α
+
p
γ
-
1
/
γ
det
⁡
Σ
α
/
2
p
Γ
p
γ
-
1
/
γ
.Proof.
From (15) we have(50)
J
α
X
γ
=
∫
R
p
∇
log
⁡
f
X
γ
x
α
f
X
γ
x
d
x
=
∫
R
p
α
f
X
γ
x
α
f
X
γ
1
-
α
x
d
x
=
α
α
∫
R
p
∇
f
X
γ
1
/
α
x
α
d
x
,while, from the definition of the density function f
X
γ as in (22), we have(51)
J
α
X
γ
=
α
α
C
γ
p
Σ
-
1
/
2
∫
R
p
∇
exp
⁡
-
γ
-
1
α
γ
Q
θ
x
γ
/
2
γ
-
1
α
d
x
=
m
Σ
∫
R
p
exp
⁡
-
γ
-
1
γ
Q
θ
γ
/
2
γ
-
1
x
∇
Q
θ
γ
/
2
γ
-
1
x
α
d
x
,where m
(
Σ
)
≔
(
γ
-
1
/
γ
)
α
C
γ
p
|
Σ
|
-
1
/
2 and Σ
≔
d
e
t
Σ. From the assumption there exist a positive real λ
∈
R
+ such that Σ
=
λ
Σ
⊥, where Σ
⊥ is an orthogonal p
×
p matrix with Σ
⊥
=
1, and thus Σ
=
|
Σ
|
1
/
p
Σ
⊥. For the gradient of the quadratic form Q
θ it holds that ∇
Q
θ
(
x
)
=
∇
{
(
x
-
μ
)
T
Σ
-
1
(
x
-
μ
)
}
=
2
Σ
-
1
(
x
-
μ
) while, from the fact that Σ
⊥ is an orthogonal matrix, we have Σ
-
1
(
x
-
μ
)
=
|
Σ
|
-
1
/
p
x
-
μ. Therefore, (51) can be written as (52)
J
α
X
γ
=
C
γ
p
Σ
-
α
/
p
-
1
/
2
∫
R
p
exp
⁡
-
γ
-
1
γ
Q
θ
γ
/
2
γ
-
1
x
Q
θ
α
γ
/
2
γ
-
1
-
α
x
x
-
μ
α
d
x
.Applying the linear transformation z
=
Σ
-
1
/
2
(
x
-
μ
) into the above integral, we get d
x
=
d
(
x
-
μ
)
=
|
Σ
|
d
z; the quadratic form Q
θ is reduced to (53)
Q
θ
x
=
x
-
μ
T
Σ
-
1
x
-
μ
=
Σ
-
1
/
2
x
-
μ
T
Σ
-
1
/
2
x
-
μ
=
z
2
.Hence,(54)
J
α
X
γ
=
C
γ
p
Σ
-
α
/
2
p
∫
R
p
z
α
/
γ
-
1
exp
⁡
-
γ
-
1
γ
z
γ
/
γ
-
1
d
z
.Switching to hyperspherical coordinates with radius r, we get (55)
J
α
X
γ
=
C
γ
p
ω
p
-
1
Σ
α
/
2
p
∫
0
+
∞
r
α
/
γ
-
1
exp
⁡
-
γ
-
1
γ
r
γ
/
γ
-
1
r
p
-
1
d
r
,where ω
p
-
1
=
2
π
p
/
2
/
Γ
(
p
/
2
) is the volume of the (
p
-
1
)-sphere S
p
-
1, and hence (56)
J
α
X
γ
=
2
π
p
/
2
Γ
π
/
2
C
γ
p
Σ
-
α
/
2
p
∫
0
+
∞
r
α
+
p
-
1
γ
-
1
/
γ
-
1
exp
⁡
-
γ
-
1
γ
r
γ
/
γ
-
1
d
r
.From the fact that d
(
γ
-
1
/
γ
r
γ
/
γ
-
1
)
=
r
1
/
γ
-
1
d
r and the definition of the gamma function, we obtain successively(57)
J
α
X
γ
=
2
π
p
/
2
Γ
π
/
2
C
γ
p
Σ
-
α
/
2
p
∫
0
+
∞
r
α
+
p
-
1
γ
-
1
/
γ
-
1
-
1
/
γ
-
1
exp
⁡
-
γ
-
1
γ
r
γ
/
γ
-
1
d
γ
-
1
γ
r
γ
/
γ
-
1
=
2
π
p
/
2
Γ
π
/
2
C
γ
p
Σ
-
α
/
2
p
∫
0
+
∞
r
α
+
p
γ
-
γ
-
p
/
γ
-
1
exp
⁡
-
γ
-
1
γ
r
γ
/
γ
-
1
d
γ
-
1
γ
r
γ
/
γ
-
1
=
2
π
p
/
2
Γ
π
/
2
γ
γ
-
1
α
-
γ
+
p
γ
-
1
/
γ
C
γ
p
Σ
-
α
/
2
p
∫
0
+
∞
γ
-
1
γ
r
γ
/
γ
-
1
α
-
γ
+
p
γ
-
1
/
γ
exp
⁡
-
γ
-
1
γ
r
γ
/
γ
-
1
d
γ
-
1
γ
r
γ
/
γ
-
1
=
2
π
p
/
2
Γ
π
/
2
γ
γ
-
1
α
-
γ
+
p
γ
-
1
/
γ
C
γ
p
Σ
-
α
/
2
p
Γ
α
+
p
γ
-
1
γ
,and, finally, applying the normalizing factor C
γ
p as in (23), we derive (58) and the theorem has been proved.Therefore, for the spherically contoured case, the following holds.Theorem 6.
The generalized Fisher informationJ
α of a spherically contoured r.v. X
γ
~
N
γ
p
(
μ
,
σ
2
I
p
) is given by (58)
J
α
X
γ
=
γ
/
γ
-
1
α
/
γ
Γ
α
+
p
γ
-
1
/
γ
σ
α
Γ
p
γ
-
1
/
γ
.


## 3. Bounds for the Generalized Fisher Information

For the defined generalized Fisher information measure and theγ-order normal distribution, it is clear that the values of J
α
(
X
γ
) as in (50) depend on the two parameters α and γ. In this section we will investigate certain bounds for the J
α
(
X
γ
) under these parameters. For dimension p
≥
1, we obtain boundaries for parameter α which are greater or lower than the shape parameter γ under some restrictions (see Proposition 7) while for dimension p
≥
2, the restrictions are removed; see Corollary 8.In the following proposition we provide some inequalities for the generalized Fisher entropy type information measureJ
α for the γ-order normally distributed r.v. X
γ with positive order γ, that is, for X
γ
~
N
γ
>
1
p
(
μ
,
Σ
). We denote with z
m
i
n
≈
1.4628 the point of minimum for the positive gamma function, Γ
(
z
), z
>
0; that is, m
i
n
{
Γ
(
z
)
}
z
>
0
=
Γ
(
z
m
i
n
).Proposition 7.
The generalized Fisher information measuresJ
α with X
γ
~
N
γ
p
(
μ
,
Σ
) where the vectors of the orthogonal scale matrix Σ
∈
R
⊥
p
×
p are adopting the same norm satisfy the inequalities (59)
J
α
X
γ
>
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
>
γ
,
=
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
=
γ
,
<
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
g
p
γ
<
α
<
γ
,for values of γ
>
p
/
p
+
1
-
z
m
i
n
≈
2
p
/
2
p
-
1 where g
p
(
γ
)
≔
γ
(
z
m
i
n
-
p
)
+
p
≈
γ
/
2
(
3
-
2
p
)
+
p.Proof.
For the proof of the first branch of (59) we assume that α
>
γ; that is, α
/
γ
>
1. Then, we have α
+
p
(
γ
-
1
)
/
γ
>
1
+
p
γ
-
1
/
γ. This implies (60)
Γ
α
+
p
γ
-
1
γ
>
Γ
1
+
p
γ
-
1
γ
=
p
γ
-
1
γ
Γ
p
γ
-
1
γ
,when 1
+
p
γ
-
1
/
γ
≥
z
m
i
n. That is, if the inequality z
=
1
+
p
γ
-
1
/
γ
≥
z
m
i
n holds, then Γ
(
z
)
≥
Γ
(
z
m
i
n
), as the gamma function is an increasing function for z
≥
z
m
i
n. Inequality 1
+
p
γ
-
1
/
γ
≥
x
m
i
n is equivalent to γ
≥
p
/
p
+
1
-
z
m
i
n
≈
p
/
p
-
0.4628
>
1. As a result, (60) holds indeed, provided that γ
≥
p
/
p
+
1
-
z
m
i
n, and thus (61)
Γ
α
+
p
γ
-
1
/
γ
Γ
p
γ
-
1
/
γ
>
p
γ
-
1
γ
.Our main assumption of α
/
γ
>
1 together with the fact that γ
/
γ
-
1
>
1 for all (defined) parameter values γ
∈
R
-
[
0,1
] leads us to (
γ
/
γ
-
1
)
α
/
γ
>
γ
/
γ
-
1. Then, inequality (61) provides(62)
γ
γ
-
1
α
/
γ
Γ
α
+
p
γ
-
1
/
γ
Γ
p
γ
-
1
/
γ
>
γ
γ
-
1
p
γ
-
1
γ
=
p
,and, through (58), we derive that J
α
(
X
γ
)
>
p
(
d
e
t
Σ
)
-
α
/
(
2
p
) for α
>
γ
≥
γ
p, with γ
p
≔
p
/
p
+
1
-
z
m
i
n; that is, the first branch of (59) holds. The order of inequalities, α
>
γ
≥
γ
p
>
1, is valid, as γ
p
>
1 is valid. This is true, because z
m
i
n
>
1 implies p
+
1
-
z
m
i
n
<
p; that is, γ
p
=
p
/
p
+
1
-
z
m
i
n
>
1. The values of γ
p are decreasing and 1
<
γ
p
≤
γ
1
≈
1.8615
<
2 for all p
≥
1. Moreover, γ
p
=
p
/
p
+
1
-
z
m
i
n
≈
p
/
p
-
0.4628
<
p
/
p
-
1
/
2
=
2
p
/
2
p
-
1.
For the proof of the third branch of (59) we assume now that α
<
γ; that is, α
/
γ
<
1. We then have α
+
p
(
γ
-
1
)
/
γ
<
1
+
p
γ
-
1
/
γ and hence (63)
Γ
α
+
p
γ
-
1
γ
<
Γ
1
+
p
γ
-
1
γ
=
p
γ
-
1
γ
Γ
p
γ
-
1
γ
,provided that z
m
i
n
≤
α
/
γ
+
p
γ
-
1
/
γ. That is, if the inequality z
m
i
n
≤
α
/
γ
+
p
γ
-
1
/
γ
=
z holds, then Γ
(
z
m
i
n
)
≤
Γ
(
z
), as the gamma function is an increasing function for z
≥
z
m
i
n. Inequality z
m
i
n
≤
α
/
γ
+
p
γ
-
1
/
γ is equivalent to α
≥
γ
(
z
m
i
n
-
p
)
+
p. As a result, (63) holds indeed, for orders γ such that γ
(
z
m
i
n
-
p
)
≤
α
-
p, and so (64)
Γ
α
+
p
γ
-
1
/
γ
Γ
p
γ
-
1
/
γ
<
p
γ
-
1
γ
.The assumption α
/
γ
<
1, together with the fact that γ
/
γ
-
1
>
1 for all defined order values γ, leads to (
γ
/
γ
-
1
)
α
/
γ
<
γ
/
γ
-
1. Then, inequality (64) provides (65)
γ
γ
-
1
α
/
γ
Γ
α
+
p
γ
-
1
/
γ
Γ
p
γ
-
1
/
γ
<
γ
γ
-
1
p
γ
-
1
γ
=
p
,and, using (58), we derive that J
α
(
X
γ
)
<
p
(
d
e
t
Σ
)
-
α
/
(
2
p
) for γ
(
z
m
i
n
-
p
)
+
p
≤
α
<
γ; that is, the third branch of (59) holds. These inequalities have a valid order when γ
(
z
m
i
n
-
p
)
+
p
<
γ is valid, that is, when γ
>
γ
p
=
p
/
p
+
1
-
z
m
i
n is assumed. Therefore, g
p
≤
α
<
γ, where g
p
≔
γ
(
z
m
i
n
-
p
)
+
p
≈
γ
/
2
(
3
-
2
p
)
+
p as z
m
i
n
=
1.4628
≈
3
/
2.
Finally, for the case whereα
=
γ, from (58) we easily get J
α
(
X
γ
)
=
p
(
d
e
t
Σ
)
-
α
/
(
2
p
); that is, the middle branch of (59) holds. In this case, the restriction γ
>
γ
p is not binding.Notice that, as the quantityp
(
d
e
t
Σ
)
-
α
/
(
2
p
) with α
=
2 is the known Fisher information measure J
=
J
α
=
2 with respect to the multivariate normal distribution, Proposition 7 shows that the generalized Fisher information J
α for the N
γ
p family of distributions is greater than this quantity when α
>
γ and lower when g
p
<
α
<
γ.Also notice that as the dimensionp of the involved variable X
γ increases then γ
p
→
1; for example, γ
6
≈
12
/
11
≈
1.09. Moreover, g
p
<
1 as p rises. Therefore, Proposition 7 holds without, practically, the restrictions of γ
>
γ
p and g
p
<
α, for large enough values of p.Corollary 8.
The generalized Fisher entropy type information measureJ
α of a random variable X
γ
~
N
γ
p
(
μ
,
Σ
), with Σ
∈
R
⊥
p
×
p as in Proposition 7, satisfies the following inequalities for shape parameter γ
≥
2 and dimension p
≥
2: (66)
J
α
X
γ
>
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
>
γ
,
=
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
=
γ
,
<
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
<
γ
.Proof.
Applying Proposition7 for p
≥
2, we get g
p
=
γ
(
z
m
i
n
-
p
)
+
p
<
1, because if γ
(
z
m
i
n
-
p
)
+
p
>
1 then γ
<
p
-
1
/
p
-
z
m
i
n
<
p
/
p
+
1
-
z
m
i
n
=
γ
p
<
2 (as 1
<
γ
p
<
4
/
3 holds for p
≥
2), which is not valid due to the assumption that γ
≥
2. Moreover, γ
≥
2
>
4
/
3
>
γ
p, and therefore, from Proposition 7, Corollary 8 indeed holds.Due to (29) and to the above Corollary 8 we have the following result for the multivariate Laplace, in contrast with the multivariate normal distribution.Corollary 9.
The generalized Fisher information measureJ
α of a random variable X following the p-variate, p
≥
2, elliptically contoured Laplace distribution L
(
μ
,
Σ
), Σ
∈
R
⊥
p
×
p as in Proposition 7, is always lower than p
(
d
e
t
Σ
)
-
α
/
(
2
p
) for all the parameter values of α
>
1; that is, (67)
J
α
X
<
p
det
⁡
Σ
-
α
/
2
p
,
α
>
1
.For the multivariate normal distribution case, that is, X
~
N
p
(
μ
,
Σ
) with p
≥
2, we have (68)
J
α
X
>
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
>
2
,
<
p
det
⁡
Σ
-
α
/
2
p
,
f
o
r
α
<
2
,while J
2
(
X
) is reduced to the known Fisher information for the multivariate N
p
(
μ
,
Σ
); that is, J
2
(
X
)
=
p
(
d
e
t
Σ
)
-
α
/
(
2
p
).Proof.
The normal distribution case is straightforward from Corollary8. For the Laplace case, as N
∞
p
(
μ
,
Σ
)
=
L
p
(
μ
,
Σ
) from (29), then J
α
(
X
∞
)
<
p
(
d
e
t
Σ
)
-
α
/
(
2
p
) for α
<
∞; that is, the inequality holds for all values of α, and the corollary has been proved.


## 4. Discussion

In this paper we considered the generalized form of the multivariate normal distribution, namely, theγ-order normal distribution, or N
γ
p. This generalization is obtained as an extremal of the LSI corresponding to a power generalization of the entropy type Fisher information.The Shannon entropy of the introducedN
γ
p distribution was evaluated (including the specific cases of the multivariate elliptically contoured uniform and Laplace distributions, resulting from N
γ
p), while the generalized entropy type information measure J
α, which extends the known entropy type Fisher information J, was also evaluated; see Theorem 6. In Proposition 7 and in the following Corollaries 8 and 9, we obtained boundaries for the generalized information measure J
α.



---

*Source: 385285-2015-09-30.xml*