ON GENERALIZED ENTROPY MEASURES AND PATHWAYS
A.M. MATHAI
Department of Mathematics and Statistics, McGill University, Montreal,
Canada H3A 2K6, and
Centre for Mathematical Sciences, Pala Campus, Arunapuram P.O., Pala-686
574, Kerala, India
H.J. HAUBOLD
Office for Outer Space Affairs, United Nations, Vienna International Centre,
P.O. Box 500, A-1400 Vienna, Austria and
Centre for Mathematical Sciences, Pala Campus, Arunapuram P.O., Pala-686
574, Kerala, India
Abstract. Product probability property, known in the literature as statistical
independence, is examined first. Then generalized entropies are introduced, all
of which give generalizations to Shannon entropy. It is shown that the nature
of the recursivity postulate automatically determines the logarithmic functional
form for Shannon entropy. Due to the logarithmic nature, Shannon entropy
naturally gives rise to additivity, when applied to situations having product
probability property. It is argued that the natural process is non-additivity,
important, for example, in statistical mechanics (Tsallis 2004, Cohen 2005),
even in product probability property situations and additivity can hold due to
the involvement of a recursivity postulate leading to a logarithmic function.
Generalized entropies are introduced and some of their properties are exam-
ined. Situations are examined where a generalized entropy of order α leads to
pathway models, exponential and power law behavior and related differential
equations. Connection of this entropy to Kerridge’s measure of “inaccuracy” is
also explored.
1. Introduction
Mathai and Rathie (1975) consider various generalizations of Shannon en-
tropy (Shannon, 1948), called entropies of order α, and give various properties,
including additivity property, and characterization theorems. Recently, Mathai
and Haubold (2006, 2006a) explored a generalized entropy of order α, which
is connected to a measure of uncertainty in a probability scheme, Kerridge’s
(Kerridge, 1961) concept of inaccuracy in a scheme, and pathway models that
are considered in this paper.
As defined in Mathai and Haubold (2006, 2006a) the entropy Mk,α(P ) is a
non-additive entropy and his measure M∗k,α(P ) is an additive entropy. It is also
shown that maximization of the continuous analogue of Mk,α(P ), denoted by
Mα(f), gives rise to various functional forms for f , depending upon the types
of constraints on f .
http://arxiv.org/abs/0704.0326v2
Occasionally, emphasis is placed on the fact that Shannon entropy satisfies
the additivity property, leading to extensivity. It will be shown that when
the product probability property (PPP) holds then a logarithmic function can
give a sum and a logarithmic function enters into Shannon entropy due to the
assumption introduced through a certain type of recursivity postulate. The
concept of statistical independence will be examined in Section 1 to illustrate
that simply because of PPP one need not expect additivity to hold or that
one should not expect this PPP should lead to extensivity. The types of non-
extensivity, associated with a number of generalized entropies, are pointed out
even when PPP holds. The nature of non-extensivity that can be expected
from a multivariate distribution, when PPP holds or when there is statistical
independence of the random variables, is illustrated by taking a trivariate case.
Maximum entropy principle is examined in Section 2. It is shown that
optimization of measures of entropies, in the continuous populations, under
selected constraints, leads to various types of models. It is shown that the
generalized entropy of order α is a convenient one to obtain various probability
models.
Section 3 examines the types of differential equations satisfied by the various
special cases of the pathway model.
1.1. Product probability property (PPP) or statistical independence
of events
Let P (A) denote the probability of the event A. If the definition P (A∩B) =
P (A)P (B) is taken as the definition of independence of the events A and B then
any event A ∈ S, and S the sure event are independent. But A is contained in S
and then the definition of independence becomes inconsistent with the common
man’s vision of independence. Even if the trivial cases of the sure event S and
the impossible event φ are deleted, still this definition becomes a resultant of
some properties of positive numbers. Consider a sample space of n distinct
elementary events. If symmetry in the outcomes is assumed then we will assign
equal probabilities 1
each to the elementary events. Let C = A ∩B. If A and
B are independent then P (C) = P (A)P (B). Let
P (A) =
, P (B) =
, P (C) =
⇒ nz = xy, x, y, z = 1, 2, ..., n− 1, z < x, y (1)
deleting S and φ. There is no solution for x, y, z for a large number of n, for
example, n = 3, 5, 7. This means that there are no independent events in such
cases and it sounds strange from a common man’s point of view.
The term “independence” of events is a misnomer. This property should
have been called product probability property or PPP of events. There is no
reason to expect the information or entropy in a joint distribution to be the sum
of the information contents of the marginal distributions when the PPP holds
for the distributions, that is when the joint density or probability function is
a product of the marginal densities or probability functions. We may expect a
term due to the product probability to enter into the expression for the entropy
in the joint distribution in such cases. But if the information or entropy is
defined in terms of a logarithm, then naturally, logarithm of a product being
the sum of logarithms, we can expect a sum coming in such situations. This is
not due to independence or due to the PPP of the densities but due to the fact
that a functional involving logarithm is taken thereby a product has become
a sum. Hence not too much importance should be put on whether or not the
entropy on the joint distribution becomes sum of the entropies on marginal
distributions or additivity property when PPP holds.
1.2. How is logarithm coming in Shannon’s entropy?
Several characterization theorems for Shannon entropy and its various gen-
eralizations are given in Mathai and Rathie (1975. Modified and refined versions
of Shannon’s own postulates are given as postulates for the first theorem charac-
terizing Shannon entropy in Mathai and Rathie (1975). Apart from continuity,
symmetry, zero-indifference and normalization postulates the main postulate
in the theorem is a recursivity postulate, which in essence says that when the
PPP holds then the entropy will be a weighted sum of the entropies, thus in
effect, assuming a logarithmic functional form. The crucial postulate is stated
here. Consider a multinomial population P = (p1, ..., pm), pi > 0, i = 1, ...,m,
p1 + ... + pm = 1, that is, pi = P (Ai), i = 1, ...,m, A1 ∪ ... ∪ Am = S,
Ai ∩ Aj = φ, i 6= j. If any pi can take a zero value also then zero-indifferent
postulate, namely that the entropy remains the same when an impossible event
is incorporated into the scheme, is to be added. Let Hn(p1, ..., pn) denote the
entropy to be defined. Then the crucial recursivity postulate says that
Hn(p1, ..., pm−1, pmq1, .., pmqn−m+1)
= Hm(p1, ..., pm) + pmHn−m+1(q1, ..., qn−m+1) (2)
i=1 pi = 1,
∑n−m+1
i=1 qi = 1. This says that if the m-th event Am is par-
titioned into independent events P (Am ∩ Bj) = P (Am)P (Bj) = pmqj , j =
1, ..., n − m + 1 so that pm = pmq1 + ... + pmqn−m+1 then the entropy Hn(·)
becomes a weighted sum. Naturally, the result will be a logarithmic function
for the measure of entropy.
There are several modifications to this crucial recursivity postulate. One
suggested by Tverberg is that n−m+ 1 = 2 and q1 = q, q2 = 1− q, 0 < q < 1
and H2(q, 1 − q) is assumed to be Lebesgue integrable in 0 ≤ q ≤ 1. Again
a characterization of Shannon entropy is obtained. In all the characterization
theorems for Shannon entropy this recursivity property enters in one form or the
other as a postulate, which in effect implies a logarithmic form for the entropy
measure. Shannon entropy Sk has the following form:
Sk = −A
pi ln pi, pi > 0, i = 1, ..., k, p1 + ...+ pk = 1, (3)
where A is a constant. If any pi is assumed to be zero then 0 ln 0 is to be
interpreted as zero. Since the constant A is present, logarithm can be taken to
any base. Usually the logarithm is taken to the base 2 for ready application to
binary systems. We will take logarithm to the base e.
1.3. Generalization of Shannon entropy
Consider again a multinomial population P = (p1, ..., pk), pi > 0, i =
1, ..., k, p1 + ... + pk = 1. The following are some of the generalizations of
Shannon entropy Sk.
Rk,α(P ) =
i=1 p
, α 6= 1, α > 0, (4)
(Rényi entropy of order α of 1961)
Hk,α(P ) =
i=1 p
i − 1
21−α − 1
, α 6= 1, α > 0 (5)
(Havrda-Charvát entropy of order α of 1967)
Tk,α(P ) =
i=1 p
i − 1
, α 6= 1, α > 0 (6)
(Tsallis entropy of 1988)
Mk,α(P ) =
i=1 p
i − 1
, α 6= 1, −∞ < α < 2 (7)
(entropic form of order α)
M∗k,α(P ) =
i=1 p
, α 6= 1, −∞ < α < 2, (8)
(additive entropic form of order α).
When α → 1 all the entropies of order α described above in (4) to (7) go to
Shannon entropy Sk.
Rk,α(P ) = lim
Hk,α(P ) = lim
Tk,α(P ) = lim
Mk,α(P ) = lim
M∗k,α(P ) = Sk.
Hence all the above measures are called generalized entropies of order α.
Let us examine to see what happens to the above entropies in the case of a
joint distribution. Let pij > 0, i = 1, ...,m, j = 1, ..., n such that
j=1 pij =
1. This is a bivariate situation of a discrete distribution. Then the entropy in
the joint distribution, for example,
Mm,n,α(P,Q) =
j=1 p
ij − 1
. (10)
If the PPP holds and if pij = piqj , p1 + ... + pm = 1, q1 + ... + qn = 1,
pi > 0, i = 1, ...,m, qj > 0, j = 1, ..., n and if P = (p1, ..., pm), Q = (q1, ..., qn)
(α− 1)Mm,α (P ) Mn,α(Q) =
i − 1
j − 1
j + 1
= Mm,n,α(P,Q) −Mm,α(P )−Mn,α(Q).
Therefore
Mm,n,α(P,Q) = Mm,α(P ) +Mn,α(Q) + (α− 1)Mm,α(P )Mn,α(Q). (11)
If any one of the above mentioned generalized entropies in (4) to (8) is written
as Fm,n,α(P,Q) then we have the relation
Fm,n,α(P,Q) = Fm,α(P ) + Fn,α(Q) + a(α)Fm,α(P )Fn,α(Q). (12)
where
a(α) = 0 (Rényi entropy Rk,α(P ))
= 21−α − 1 (Havrda-Charvát entropy Hk,α(P ))
= 1− α (Tsallis entropy Tk,α(P ))
= α− 1 (entropic form of order α, i.e., Mk,α(P ))
= 0 (additive entropic form of order α, i.e., M∗k,α(P )). (13)
When a(α) = 0 the entropy is called additive and when a(α) 6= 0 the entropy
is called non-additive. As can be expected, when a logarithmic function is
involved, as in the cases of Sk(P ), Rk,α(P ),M
k,α(P ), the entropy is additive
and a(α) = 0.
1.4. Extensions to higher dimensional joint distributions
Consider a trivariate population or a trivariate discrete distribution pijk >
0, i = 1, ...,m, j = 1, ..., n, k = 1, ..., r such that
k=1 pijk = 1. If
the PPP holds mutually, that is, pair-wise as well as jointly, which then will
imply that
pijk = piqjsk,
pi = 1,
qj = 1,
sk = 1,
P = (p1, ..., pm), Q = (q1, ..., qn), S = (s1, ..., sr).
Then proceeding as before, we have for any of the measures described above in
(4) to (8), calling it F (·),
Fm,n,r,α(P,Q, S) = Fm,α(P ) + Fn,α(Q) + Fr,α(S) + a(α)[Fm,α(P )Fn,α(Q)
+Fm,α(P )Fr,α(S) + Fn,α(Q)Fr,α(S)]
+[a(α)]2Fm,α(P )Fn,α(Q)Fr,α(S) (14)
where a(α) is the same as in (13). The same procedure can be extended to any
multivariable situation. If a(α) = 0 we may call the entropy additive and if
a(α) 6= 0 then the entropy is non-additive.
1.5. Crucial recursivity postulate
Consider the multinomial population P = (p1, ..., pk), pi > 0, i = 1, ..., k, p1+
... + pk = 1. Let the entropy measure to be determined through appropriate
postulates be denoted by Hk(P ) = Hk(p1, ..., pk). For k = 2 let
f(x) = H2(x, 1− x), 0 ≤ x ≤ 1 or x ∈ [0, 1]. (15)
If another parameter α is to be involved in H2(x, 1−x) then we will denote f(x)
by fα(x). From (5) to (7) it can be seen that the generalized entropies of order
α of Havrda-Charvát (1967), Tsallis (1988, 2004) and Shannon (1948) entropy
satisfy the functional equation
fα(x) + bα(x)fα
= fα(y) + bα(x)f
for x, y ∈ [0, ) with x+ y ∈ [0, 1], with the boundary condition
fα(0) = fα(1) (17)
where
bα(x) = 1− x (Shannon entropy Sk(P ))
= (1− x)α (Harvda-Charvát entropy Hk,α(P ))
= (1− x)α (Tsallis entropy Tk,α(P ))
= (1− x)2−α (entropic form of order α, i.e., Mk,α(P )). (18)
Observe that the normalizing constant at x = 1
is equal to 1 for Hk,α(P ) and it
is different for other entropies. Thus equations (6),(7),(8), with the appropriate
normalizing constants fα(
), can give characterization theorems for the various
entropy measures. The form of bα(x) is coming from the crucial recursivity
postulate, assumed as a desirable property for the measures.
1.6. Continuous analogues
In the continuous case let f(x) be the density function of a real random
variable x. Then the various entropy measures, corresponding to the ones in (4)
to (8) are the following:
Rα(f) =
[f(x)]αdx
, α 6= 1, α > 0 (19)
(Rényi entropy of order α)
Hα(f) =
21−α − 1
[f(x)]αdx− 1
, α 6= 1, α > 0 (20)
(Havrda-Charvát entropy of order α)
Tα(f) =
[f(x)]αdx− 1
, α 6= 1, α > 0, (21)
(Tsallis entropy of order α)
Mα(f) =
[f(x)]2−αdx− 1
, α 6= 1, α < 2 (22)
(entropic form of order α)
M∗α(f) =
[f(x)]2−αdx
, α 6= 1, α < 2 (23)
(additive entropic form of order α).
As expected, Shannon entropy in this case is given by
S(f) = −A
f(x) ln f(x)dx (24)
where A is a constant.
Note that when PPP (product probability property) or statistical indepen-
dence holds then in the continuous case also we have the property in (12) and
(14) and then non-additivity holds for the measures analogous to the ones in
(3),(5),(6),(7) with a(α) remaining the same. Since the steps are parallel a
separate derivation is not given here.
2. Maximum Entropy Principle
If we have a multinomial population P = (p1, ..., pk), pi > 0, i = 1, ..., k, p1+
...+ pk = 1 or the scheme P (Ai) = pi, A1 ∪ ... ∪ Ak = S, P (S) = 1, Ai ∩ Aj =
φ, i 6= j then we know that the maximum uncertainty in the scheme or the
minimum information from the scheme is obtained when we cannot give any
preference to the occurrence of any particular event or when the events are
equally likely or when p1 = p2 = ... = pk =
. In this case, Shannon entropy
becomes,
Sk(P ) = Sk(
, ...,
) = −A
= A ln k (25)
and this is the maximum uncertainty or maximum Shannon entropy in this
scheme. If the arbitrary functional f is to be fixed by maximizing the entropy
then in (19) to (21) we have to optimize
[f(x)]αdx for fixed α, over all
functional f , subject to the condition
f(x)dx = 1 and f(x) ≥ 0 for all x.
For applying calculus of variation procedure we consider the functional
U = [f(x)]α − λ[f(x)]
where λ is a Lagrangian multiplier. Then the Euler equation is the following:
= 0 ⇒ αfα−1 − λ = 0 ⇒ f =
= constant. (26)
Hence f is the uniform density in this case, analogous to the equally likely
situation in the multinomial case. If the first moment E(x) =
xf(x)dx
is assumed to be a given quantity for all functional f then U will become the
following for (19) to (21).
U = [f(x)]α − λ1[f(x)]− λ2xf(x)
and the Euler equation leads to the power law. That is,
= 0 ⇒ αfα−1 − λ1 − λ2x = 0 ⇒ f = c1
. (27)
By selecting c1, λ1, λ2 appropriately we can create a density out of (27). For
α > 1 and λ2
> 0 the right side in (27) increases exponentially. If α = q > 1 and
= q − 1 then we have Tsallis’ q-exponential function from the right side of
(27). If α > 1 and λ2
= −(α−1) then (27) can produce a density in the category
of a type-1 beta. From (27) it is seen that the form of the entropies of Havrda-
CharvátHk,α(P ) and Tsallis Tk,α(P ) need special attention to produce densities
(Ferri et al. 2005). However, Tsallis has considered a different constraint on
E(x). If the density f(x) is replaced by its escort density, namely, µ[f(x)]α
where µ−1 =
[f(x)]αdx and if the expected value of x in this escort density
is assumed to be fixed for all functional f then the U of (26) becomes
U = fα − λ1f + µλ2xf
= 0 ⇒ αfα−1[1 + µλ2x] = λ1 ⇒ f =
(1+λ3x)
f = λ∗1[1 + λ3x]
where λ3 is a constant and λ
1 is the normalizing constant. If λ3 is taken as
λ3 = α− 1 then
f = λ∗1[1 + (α− 1)x]
α−1 . (28)
Then (28) for α > 1 is Tsallis statistics (Tsallis 2004, Cohen 2005). Then for
α < 1 also by writing α − 1 = −(1 − α) one gets the case of Tsallis statistics
for α < 1 (Ferri et al. 2005). These modifications and the consideration of
escort distribution are not necessary if we take the generalized entropy of order
α. Thus if we consider Mα(f) and if we assume that the first moment in f(x)
itself is fixed for all functional f then the Euler equation gives
(2− α)f1−α − λ1 + λ2x = 0 ⇒ f = λ̄
and for λ2
= 1− α we have Tsallis statistics (Tsallis 2004, Cohen 2005)
f = λ̄[1− (1− α)x]
1−α (29)
coming directly, where λ̄ is the normalizing constant.
Let us start with Mα(f) of (20) under the assumptions that f(x) ≥ 0 for all
f(x)dx = 1,
xδf(x)dx is fixed for all functional f and for a specified
δ > 0, f(a) is the same for all functional f , f(b) is the same for all functional
f , for some limits a and b, then the Euler equation becomes
(2 − α)f1−α − λ1 − λ2x
δ = 0 ⇒ f = c1[1 + c
1−α . (30)
If c∗1 is written as −s(1− α), s > 0 then we have, writing f1 for f ,
f1 = c1[1− s(1 − α)x
1−α , δ > 0, α < 1, 0 ≤ x ≤
[s(1− α)]
where 1 − s(1 − α)xδ > 0. For α < 1 or −∞ < α < 1 the right side of (31)
remains as a generalized type-1 beta model with the corresponding normalizing
constant c1. For α > 1, writing 1 − α = −(α − 1) the model in (31) goes to a
generalized type-2 beta form, namely,
f2 = c2[1 + s(α− 1)x
α−1 . (32)
When α → 1 in (31) or in (32) we have an extended or stretched exponential
form,
f3 = c3e
. (33)
If c∗1 in (30) is taken as positive then (30) for α < 1, α > 1, α → 1 will be
increasing exponentially. Hence all possible forms are available from (30). The
model in (31) is a special case of the distributional pathway model and for a
discussion of the matrix-variate pathway model see Mathai (2005). Special cases
of (31) and (32) for δ = 1 are Tsallis statistics (Gell-Mann and Tsallis, 2004;
Ferri et al. 2005).
Instead of optimizing Mα(f) of (22) under the conditions that f(x) ≥ 0
for all x,
f(x)dx = 1 and
xδf(x)dx is fixed, let us optimize under the
following conditions: f(x) ≥ 0 for all x,
f(x)dx < ∞ and the following two
moment-like expressions are fixed quantities for all functional f ,
x(γ−1)(1−α)f(x)dx = fixed ,
x(γ−1)(1−α)+δf(x)dx = fixed.
Then the Euler equation becomes
(2− α)f1−α −λ1x
(γ−1)(1−α)
− λ2x
(γ−1)(1−α)+δ = 0 ⇒
f = c xγ−1[1 + c∗xδ]
and for c∗ = −s(1 − α), s > 0, we have the distributional pathway model for
the real scalar case, namely
f(x) = c xγ−1[1− s(1− α)xδ ]
1−α , δ > 0, s > 0 (34)
where c is the normalizing constant. For α < 1, (34) gives a generalized type-1
beta form, for α > 1 it gives a generalized type-2 beta form and for α → 1
we have a generalized gamma form. For α > 1, (34) gives the superstatistics
of Beck (2006) and Beck and Cohen (2003). For γ = 1, δ = 1, (34) gives
Tsallis statistics (Tsallis 2004, Cohen 2005). Densities appearing in a number
of physical problems are seen to be special cases of (34), a discussion of which
may be seen from Mathai and Haubold (2006a). For example, (34) for δ =
2, γ = 3, α → 1, x > 0 is the Maxwell-Boltzmann density; for δ = 2, γ = 1, α →
1,−∞ < x < ∞ is the Gaussian density; for γ = δ, α → 1 is the Weibull density.
For γ = 1, δ = 2, 1 < q < 3 we have the Wigner function W (p) giving the atomic
moment distribution in the framework of Fokker-Planck equation, see Douglas,
Bergamini, and Renzoni (2006) where
W (p) = z−1q [1− β(1 − q)p
1−q , 1 < q < 3. (35)
Before closing this section we may observe one more property for Mα(f). As
an expected value
Mα(f) =
E[f(x)]1−α − 1
. (36)
But Kerridge’s (Kerridge, 1961) measure of “inaccuracy” in assigning q(x) for
the true density f(x), in the generalized form is
Hα(f : q) =
(21−α − 1)
E[q(x)]α−1 − 1
, (37)
which is also connected to the measure of directed divergence between q(x) and
f(x). In (37) the normalizing constant is 21−α−1, the same factor appearing in
Havrda-Charvt́ entropy. With different normalizing constants, as seen before,
(36) and (37) have the same forms as an expected value with q(x) replaced
by f(x) in (36). Hence Mα(f) can also be looked upon as a type of directed
divergence or “inaccuracy” measure.
3. Differential Equations
The functional part in (34), for a more general exponent, namely
g(x) =
= xγ−1[1− s(1 − α)xδ]
1−α , α 6= 1, δ > 0, β > 0, s > 0 (38)
is seen to satisfy the following differential equation for γ 6= 1 which defines the
differential pathway.
g(x) = (γ − 1)xγ−1[1− s(1− α)xδ]
−sβδxδ+γ−1[1− s(1− α)xδ]
(1−α)
. (39)
Then for δ =
(γ−1)(α−1)
, γ 6= 1, α > 1 we have
g(x) = (γ − 1)g(x)− sβδ[g(x)]1−
(1−α)
β (40)
= (γ − 1)g(x)− sδ[g(x)]α (41)
for β = 1, γ 6= 1, δ = (γ − 1)(α− 1), α > 1.
For γ = 1, δ = 1 in (38) we have
g(x) = −s[g(x)]η, η = 1−
(1 − α)
= −s[g(x)]α for β = 1. (43)
Here (43) is the power law coming from Tsallis statistics (Gell-Mann and Tsallis,
2004).
Acknowledgement The authors would like to thank the Department of Science
and Technology, Government of India, New Delhi, for the financial assistance for
this work under project No. SR/S4/MS:287/05 which enabled this collaboration
possible.
4. References
Beck, C. (2006). Stretched exponentials from superstatistics. Physica A, 365,
96-101.
Beck, C. and Cohen, E.G.D. (2003). Superstatistics. Physica A, 322, 267-275.
Cohen, E.G.D. (2005). Boltzmann and Einstein: Statistics and dynamics - An
unsolved problem. Pramana, 64, 635-643.
Douglas, P., Bergamini, S., and Renzoni, F. (2006). Tunable Tsallis distribution
in dissipative optical lattices. Physical Review Letters, 96, 110601-1-4.
Ferri, G.L., Martinez, S., and Plastino, A. (2005). Equivalence of the four
versions of Tsallis’s statistics. Journal of Statistical Mechanics: Theory and
Experiment, PO4009.
Gell-Mann, M. and Tsallis, C. (Eds.) (2004). Nonextensive Statistical Mechan-
ics: Interdisciplinary Applications. Oxford University Press, Oxford.
Havrda, J. and Charvát, F. (1967). Quantification method of classification pro-
cedures: Concept of structural α-entropy. Kybernetika, 3, 30-35.
Kerridge, D.F. (1961). Inaccuracy and inference. Journal of the Royal Statisti-
cal Society Series B, 23, 184-194.
Mathai, A.M. (2005). A pathway to matrix-variate gamma and normal densi-
ties. Linear Algebra and Its Applications, 396, 317-328.
Mathai, A.M. and Haubold, H.J. (2006). Pathway model, Tsallis statistics, su-
perstatistics and a generalized measure of entropy. Physica A , 375), 110-122.
Mathai,A.M. and Haubold, H.J. (2006a). On generalized distributions and path-
ways. arXiv:cond-mat/0609526v2.
Mathai, A.M. and Rathie, P.N. (1975). Basic Concepts in Information Theory
and Statistics: Axiomatic Foundations and Applications, Wiley Halstead, New
York and Wiley Eastern, New Delhi.
Rényi, A. (1961). On measure of entropy and information. Proceedings of the
Fourth Berkeley Symposium on Mathematical Statistics and Probability, 1960,
University of California Press, 1961, Vol. 1, 547-561.
Shannon, C.E. (1948). A mathematical theory of communication. Bell System
Technical Journal, 27, 379-423, 547-561.
Tsallis, C. (1988). Possible generalization of Boltzmann-Gibbs statistics. Jour-
nal of Statistical Physics, 52, 479-487.
Tsallis, C. (2004). What should a statistical mechanics satisfy to reflect nature?,
Physica D, 193, 3-34.
http://arxiv.org/abs/cond-mat/0609526

  Product probability property, known in the literature as statistical
independence, is examined first. Then generalized entropies are introduced, all
of which give generalizations to Shannon entropy. It is shown that the nature
of the recursivity postulate automatically determines the logarithmic
functional form for Shannon entropy. Due to the logarithmic nature, Shannon
entropy naturally gives rise to additivity, when applied to situations having
product probability property. It is argued that the natural process is
non-additivity, important, for example, in statistical mechanics, even in
product probability property situations and additivity can hold due to the
involvement of a recursivity postulate leading to a logarithmic function.
Generalizations, including Mathai's generalized entropy are introduced and some
of the properties are examined. Situations are examined where Mathai's entropy
leads to pathway models, exponential and power law behavior and related
differential equations. Connection of Mathai's entropy to Kerridge's measure of
"inaccuracy" is also explored.

Introduction
Mathai and Rathie (1975) consider various generalizations of Shannon en-
tropy (Shannon, 1948), called entropies of order α, and give various properties,
including additivity property, and characterization theorems. Recently, Mathai
and Haubold (2006, 2006a) explored a generalized entropy of order α, which
is connected to a measure of uncertainty in a probability scheme, Kerridge’s
(Kerridge, 1961) concept of inaccuracy in a scheme, and pathway models that
are considered in this paper.
As defined in Mathai and Haubold (2006, 2006a) the entropy Mk,α(P ) is a
non-additive entropy and his measure M∗k,α(P ) is an additive entropy. It is also
shown that maximization of the continuous analogue of Mk,α(P ), denoted by
Mα(f), gives rise to various functional forms for f , depending upon the types
of constraints on f .
http://arxiv.org/abs/0704.0326v2
Occasionally, emphasis is placed on the fact that Shannon entropy satisfies
the additivity property, leading to extensivity. It will be shown that when
the product probability property (PPP) holds then a logarithmic function can
give a sum and a logarithmic function enters into Shannon entropy due to the
assumption introduced through a certain type of recursivity postulate. The
concept of statistical independence will be examined in Section 1 to illustrate
that simply because of PPP one need not expect additivity to hold or that
one should not expect this PPP should lead to extensivity. The types of non-
extensivity, associated with a number of generalized entropies, are pointed out
even when PPP holds. The nature of non-extensivity that can be expected
from a multivariate distribution, when PPP holds or when there is statistical
independence of the random variables, is illustrated by taking a trivariate case.
Maximum entropy principle is examined in Section 2. It is shown that
optimization of measures of entropies, in the continuous populations, under
selected constraints, leads to various types of models. It is shown that the
generalized entropy of order α is a convenient one to obtain various probability
models.
Section 3 examines the types of differential equations satisfied by the various
special cases of the pathway model.
1.1. Product probability property (PPP) or statistical independence
of events
Let P (A) denote the probability of the event A. If the definition P (A∩B) =
P (A)P (B) is taken as the definition of independence of the events A and B then
any event A ∈ S, and S the sure event are independent. But A is contained in S
and then the definition of independence becomes inconsistent with the common
man’s vision of independence. Even if the trivial cases of the sure event S and
the impossible event φ are deleted, still this definition becomes a resultant of
some properties of positive numbers. Consider a sample space of n distinct
elementary events. If symmetry in the outcomes is assumed then we will assign
equal probabilities 1
each to the elementary events. Let C = A ∩B. If A and
B are independent then P (C) = P (A)P (B). Let
P (A) =
, P (B) =
, P (C) =
⇒ nz = xy, x, y, z = 1, 2, ..., n− 1, z < x, y (1)
deleting S and φ. There is no solution for x, y, z for a large number of n, for
example, n = 3, 5, 7. This means that there are no independent events in such
cases and it sounds strange from a common man’s point of view.
The term “independence” of events is a misnomer. This property should
have been called product probability property or PPP of events. There is no
reason to expect the information or entropy in a joint distribution to be the sum
of the information contents of the marginal distributions when the PPP holds
for the distributions, that is when the joint density or probability function is
a product of the marginal densities or probability functions. We may expect a
term due to the product probability to enter into the expression for the entropy
in the joint distribution in such cases. But if the information or entropy is
defined in terms of a logarithm, then naturally, logarithm of a product being
the sum of logarithms, we can expect a sum coming in such situations. This is
not due to independence or due to the PPP of the densities but due to the fact
that a functional involving logarithm is taken thereby a product has become
a sum. Hence not too much importance should be put on whether or not the
entropy on the joint distribution becomes sum of the entropies on marginal
distributions or additivity property when PPP holds.
1.2. How is logarithm coming in Shannon’s entropy?
Several characterization theorems for Shannon entropy and its various gen-
eralizations are given in Mathai and Rathie (1975. Modified and refined versions
of Shannon’s own postulates are given as postulates for the first theorem charac-
terizing Shannon entropy in Mathai and Rathie (1975). Apart from continuity,
symmetry, zero-indifference and normalization postulates the main postulate
in the theorem is a recursivity postulate, which in essence says that when the
PPP holds then the entropy will be a weighted sum of the entropies, thus in
effect, assuming a logarithmic functional form. The crucial postulate is stated
here. Consider a multinomial population P = (p1, ..., pm), pi > 0, i = 1, ...,m,
p1 + ... + pm = 1, that is, pi = P (Ai), i = 1, ...,m, A1 ∪ ... ∪ Am = S,
Ai ∩ Aj = φ, i 6= j. If any pi can take a zero value also then zero-indifferent
postulate, namely that the entropy remains the same when an impossible event
is incorporated into the scheme, is to be added. Let Hn(p1, ..., pn) denote the
entropy to be defined. Then the crucial recursivity postulate says that
Hn(p1, ..., pm−1, pmq1, .., pmqn−m+1)
= Hm(p1, ..., pm) + pmHn−m+1(q1, ..., qn−m+1) (2)
i=1 pi = 1,
∑n−m+1
i=1 qi = 1. This says that if the m-th event Am is par-
titioned into independent events P (Am ∩ Bj) = P (Am)P (Bj) = pmqj , j =
1, ..., n − m + 1 so that pm = pmq1 + ... + pmqn−m+1 then the entropy Hn(·)
becomes a weighted sum. Naturally, the result will be a logarithmic function
for the measure of entropy.
There are several modifications to this crucial recursivity postulate. One
suggested by Tverberg is that n−m+ 1 = 2 and q1 = q, q2 = 1− q, 0 < q < 1
and H2(q, 1 − q) is assumed to be Lebesgue integrable in 0 ≤ q ≤ 1. Again
a characterization of Shannon entropy is obtained. In all the characterization
theorems for Shannon entropy this recursivity property enters in one form or the
other as a postulate, which in effect implies a logarithmic form for the entropy
measure. Shannon entropy Sk has the following form:
Sk = −A
pi ln pi, pi > 0, i = 1, ..., k, p1 + ...+ pk = 1, (3)
where A is a constant. If any pi is assumed to be zero then 0 ln 0 is to be
interpreted as zero. Since the constant A is present, logarithm can be taken to
any base. Usually the logarithm is taken to the base 2 for ready application to
binary systems. We will take logarithm to the base e.
1.3. Generalization of Shannon entropy
Consider again a multinomial population P = (p1, ..., pk), pi > 0, i =
1, ..., k, p1 + ... + pk = 1. The following are some of the generalizations of
Shannon entropy Sk.
Rk,α(P ) =
i=1 p
, α 6= 1, α > 0, (4)
(Rényi entropy of order α of 1961)
Hk,α(P ) =
i=1 p
i − 1
21−α − 1
, α 6= 1, α > 0 (5)
(Havrda-Charvát entropy of order α of 1967)
Tk,α(P ) =
i=1 p
i − 1
, α 6= 1, α > 0 (6)
(Tsallis entropy of 1988)
Mk,α(P ) =
i=1 p
i − 1
, α 6= 1, −∞ < α < 2 (7)
(entropic form of order α)
M∗k,α(P ) =
i=1 p
, α 6= 1, −∞ < α < 2, (8)
(additive entropic form of order α).
When α → 1 all the entropies of order α described above in (4) to (7) go to
Shannon entropy Sk.
Rk,α(P ) = lim
Hk,α(P ) = lim
Tk,α(P ) = lim
Mk,α(P ) = lim
M∗k,α(P ) = Sk.
Hence all the above measures are called generalized entropies of order α.
Let us examine to see what happens to the above entropies in the case of a
joint distribution. Let pij > 0, i = 1, ...,m, j = 1, ..., n such that
j=1 pij =
1. This is a bivariate situation of a discrete distribution. Then the entropy in
the joint distribution, for example,
Mm,n,α(P,Q) =
j=1 p
ij − 1
. (10)
If the PPP holds and if pij = piqj , p1 + ... + pm = 1, q1 + ... + qn = 1,
pi > 0, i = 1, ...,m, qj > 0, j = 1, ..., n and if P = (p1, ..., pm), Q = (q1, ..., qn)
(α− 1)Mm,α (P ) Mn,α(Q) =
i − 1
j − 1
j + 1
= Mm,n,α(P,Q) −Mm,α(P )−Mn,α(Q).
Therefore
Mm,n,α(P,Q) = Mm,α(P ) +Mn,α(Q) + (α− 1)Mm,α(P )Mn,α(Q). (11)
If any one of the above mentioned generalized entropies in (4) to (8) is written
as Fm,n,α(P,Q) then we have the relation
Fm,n,α(P,Q) = Fm,α(P ) + Fn,α(Q) + a(α)Fm,α(P )Fn,α(Q). (12)
where
a(α) = 0 (Rényi entropy Rk,α(P ))
= 21−α − 1 (Havrda-Charvát entropy Hk,α(P ))
= 1− α (Tsallis entropy Tk,α(P ))
= α− 1 (entropic form of order α, i.e., Mk,α(P ))
= 0 (additive entropic form of order α, i.e., M∗k,α(P )). (13)
When a(α) = 0 the entropy is called additive and when a(α) 6= 0 the entropy
is called non-additive. As can be expected, when a logarithmic function is
involved, as in the cases of Sk(P ), Rk,α(P ),M
k,α(P ), the entropy is additive
and a(α) = 0.
1.4. Extensions to higher dimensional joint distributions
Consider a trivariate population or a trivariate discrete distribution pijk >
0, i = 1, ...,m, j = 1, ..., n, k = 1, ..., r such that
k=1 pijk = 1. If
the PPP holds mutually, that is, pair-wise as well as jointly, which then will
imply that
pijk = piqjsk,
pi = 1,
qj = 1,
sk = 1,
P = (p1, ..., pm), Q = (q1, ..., qn), S = (s1, ..., sr).
Then proceeding as before, we have for any of the measures described above in
(4) to (8), calling it F (·),
Fm,n,r,α(P,Q, S) = Fm,α(P ) + Fn,α(Q) + Fr,α(S) + a(α)[Fm,α(P )Fn,α(Q)
+Fm,α(P )Fr,α(S) + Fn,α(Q)Fr,α(S)]
+[a(α)]2Fm,α(P )Fn,α(Q)Fr,α(S) (14)
where a(α) is the same as in (13). The same procedure can be extended to any
multivariable situation. If a(α) = 0 we may call the entropy additive and if
a(α) 6= 0 then the entropy is non-additive.
1.5. Crucial recursivity postulate
Consider the multinomial population P = (p1, ..., pk), pi > 0, i = 1, ..., k, p1+
... + pk = 1. Let the entropy measure to be determined through appropriate
postulates be denoted by Hk(P ) = Hk(p1, ..., pk). For k = 2 let
f(x) = H2(x, 1− x), 0 ≤ x ≤ 1 or x ∈ [0, 1]. (15)
If another parameter α is to be involved in H2(x, 1−x) then we will denote f(x)
by fα(x). From (5) to (7) it can be seen that the generalized entropies of order
α of Havrda-Charvát (1967), Tsallis (1988, 2004) and Shannon (1948) entropy
satisfy the functional equation
fα(x) + bα(x)fα
= fα(y) + bα(x)f
for x, y ∈ [0, ) with x+ y ∈ [0, 1], with the boundary condition
fα(0) = fα(1) (17)
where
bα(x) = 1− x (Shannon entropy Sk(P ))
= (1− x)α (Harvda-Charvát entropy Hk,α(P ))
= (1− x)α (Tsallis entropy Tk,α(P ))
= (1− x)2−α (entropic form of order α, i.e., Mk,α(P )). (18)
Observe that the normalizing constant at x = 1
is equal to 1 for Hk,α(P ) and it
is different for other entropies. Thus equations (6),(7),(8), with the appropriate
normalizing constants fα(
), can give characterization theorems for the various
entropy measures. The form of bα(x) is coming from the crucial recursivity
postulate, assumed as a desirable property for the measures.
1.6. Continuous analogues
In the continuous case let f(x) be the density function of a real random
variable x. Then the various entropy measures, corresponding to the ones in (4)
to (8) are the following:
Rα(f) =
[f(x)]αdx
, α 6= 1, α > 0 (19)
(Rényi entropy of order α)
Hα(f) =
21−α − 1
[f(x)]αdx− 1
, α 6= 1, α > 0 (20)
(Havrda-Charvát entropy of order α)
Tα(f) =
[f(x)]αdx− 1
, α 6= 1, α > 0, (21)
(Tsallis entropy of order α)
Mα(f) =
[f(x)]2−αdx− 1
, α 6= 1, α < 2 (22)
(entropic form of order α)
M∗α(f) =
[f(x)]2−αdx
, α 6= 1, α < 2 (23)
(additive entropic form of order α).
As expected, Shannon entropy in this case is given by
S(f) = −A
f(x) ln f(x)dx (24)
where A is a constant.
Note that when PPP (product probability property) or statistical indepen-
dence holds then in the continuous case also we have the property in (12) and
(14) and then non-additivity holds for the measures analogous to the ones in
(3),(5),(6),(7) with a(α) remaining the same. Since the steps are parallel a
separate derivation is not given here.
2. Maximum Entropy Principle
If we have a multinomial population P = (p1, ..., pk), pi > 0, i = 1, ..., k, p1+
...+ pk = 1 or the scheme P (Ai) = pi, A1 ∪ ... ∪ Ak = S, P (S) = 1, Ai ∩ Aj =
φ, i 6= j then we know that the maximum uncertainty in the scheme or the
minimum information from the scheme is obtained when we cannot give any
preference to the occurrence of any particular event or when the events are
equally likely or when p1 = p2 = ... = pk =
. In this case, Shannon entropy
becomes,
Sk(P ) = Sk(
, ...,
) = −A
= A ln k (25)
and this is the maximum uncertainty or maximum Shannon entropy in this
scheme. If the arbitrary functional f is to be fixed by maximizing the entropy
then in (19) to (21) we have to optimize
[f(x)]αdx for fixed α, over all
functional f , subject to the condition
f(x)dx = 1 and f(x) ≥ 0 for all x.
For applying calculus of variation procedure we consider the functional
U = [f(x)]α − λ[f(x)]
where λ is a Lagrangian multiplier. Then the Euler equation is the following:
= 0 ⇒ αfα−1 − λ = 0 ⇒ f =
= constant. (26)
Hence f is the uniform density in this case, analogous to the equally likely
situation in the multinomial case. If the first moment E(x) =
xf(x)dx
is assumed to be a given quantity for all functional f then U will become the
following for (19) to (21).
U = [f(x)]α − λ1[f(x)]− λ2xf(x)
and the Euler equation leads to the power law. That is,
= 0 ⇒ αfα−1 − λ1 − λ2x = 0 ⇒ f = c1
. (27)
By selecting c1, λ1, λ2 appropriately we can create a density out of (27). For
α > 1 and λ2
> 0 the right side in (27) increases exponentially. If α = q > 1 and
= q − 1 then we have Tsallis’ q-exponential function from the right side of
(27). If α > 1 and λ2
= −(α−1) then (27) can produce a density in the category
of a type-1 beta. From (27) it is seen that the form of the entropies of Havrda-
CharvátHk,α(P ) and Tsallis Tk,α(P ) need special attention to produce densities
(Ferri et al. 2005). However, Tsallis has considered a different constraint on
E(x). If the density f(x) is replaced by its escort density, namely, µ[f(x)]α
where µ−1 =
[f(x)]αdx and if the expected value of x in this escort density
is assumed to be fixed for all functional f then the U of (26) becomes
U = fα − λ1f + µλ2xf
= 0 ⇒ αfα−1[1 + µλ2x] = λ1 ⇒ f =
(1+λ3x)
f = λ∗1[1 + λ3x]
where λ3 is a constant and λ
1 is the normalizing constant. If λ3 is taken as
λ3 = α− 1 then
f = λ∗1[1 + (α− 1)x]
α−1 . (28)
Then (28) for α > 1 is Tsallis statistics (Tsallis 2004, Cohen 2005). Then for
α < 1 also by writing α − 1 = −(1 − α) one gets the case of Tsallis statistics
for α < 1 (Ferri et al. 2005). These modifications and the consideration of
escort distribution are not necessary if we take the generalized entropy of order
α. Thus if we consider Mα(f) and if we assume that the first moment in f(x)
itself is fixed for all functional f then the Euler equation gives
(2− α)f1−α − λ1 + λ2x = 0 ⇒ f = λ̄
and for λ2
= 1− α we have Tsallis statistics (Tsallis 2004, Cohen 2005)
f = λ̄[1− (1− α)x]
1−α (29)
coming directly, where λ̄ is the normalizing constant.
Let us start with Mα(f) of (20) under the assumptions that f(x) ≥ 0 for all
f(x)dx = 1,
xδf(x)dx is fixed for all functional f and for a specified
δ > 0, f(a) is the same for all functional f , f(b) is the same for all functional
f , for some limits a and b, then the Euler equation becomes
(2 − α)f1−α − λ1 − λ2x
δ = 0 ⇒ f = c1[1 + c
1−α . (30)
If c∗1 is written as −s(1− α), s > 0 then we have, writing f1 for f ,
f1 = c1[1− s(1 − α)x
1−α , δ > 0, α < 1, 0 ≤ x ≤
[s(1− α)]
where 1 − s(1 − α)xδ > 0. For α < 1 or −∞ < α < 1 the right side of (31)
remains as a generalized type-1 beta model with the corresponding normalizing
constant c1. For α > 1, writing 1 − α = −(α − 1) the model in (31) goes to a
generalized type-2 beta form, namely,
f2 = c2[1 + s(α− 1)x
α−1 . (32)
When α → 1 in (31) or in (32) we have an extended or stretched exponential
form,
f3 = c3e
. (33)
If c∗1 in (30) is taken as positive then (30) for α < 1, α > 1, α → 1 will be
increasing exponentially. Hence all possible forms are available from (30). The
model in (31) is a special case of the distributional pathway model and for a
discussion of the matrix-variate pathway model see Mathai (2005). Special cases
of (31) and (32) for δ = 1 are Tsallis statistics (Gell-Mann and Tsallis, 2004;
Ferri et al. 2005).
Instead of optimizing Mα(f) of (22) under the conditions that f(x) ≥ 0
for all x,
f(x)dx = 1 and
xδf(x)dx is fixed, let us optimize under the
following conditions: f(x) ≥ 0 for all x,
f(x)dx < ∞ and the following two
moment-like expressions are fixed quantities for all functional f ,
x(γ−1)(1−α)f(x)dx = fixed ,
x(γ−1)(1−α)+δf(x)dx = fixed.
Then the Euler equation becomes
(2− α)f1−α −λ1x
(γ−1)(1−α)
− λ2x
(γ−1)(1−α)+δ = 0 ⇒
f = c xγ−1[1 + c∗xδ]
and for c∗ = −s(1 − α), s > 0, we have the distributional pathway model for
the real scalar case, namely
f(x) = c xγ−1[1− s(1− α)xδ ]
1−α , δ > 0, s > 0 (34)
where c is the normalizing constant. For α < 1, (34) gives a generalized type-1
beta form, for α > 1 it gives a generalized type-2 beta form and for α → 1
we have a generalized gamma form. For α > 1, (34) gives the superstatistics
of Beck (2006) and Beck and Cohen (2003). For γ = 1, δ = 1, (34) gives
Tsallis statistics (Tsallis 2004, Cohen 2005). Densities appearing in a number
of physical problems are seen to be special cases of (34), a discussion of which
may be seen from Mathai and Haubold (2006a). For example, (34) for δ =
2, γ = 3, α → 1, x > 0 is the Maxwell-Boltzmann density; for δ = 2, γ = 1, α →
1,−∞ < x < ∞ is the Gaussian density; for γ = δ, α → 1 is the Weibull density.
For γ = 1, δ = 2, 1 < q < 3 we have the Wigner function W (p) giving the atomic
moment distribution in the framework of Fokker-Planck equation, see Douglas,
Bergamini, and Renzoni (2006) where
W (p) = z−1q [1− β(1 − q)p
1−q , 1 < q < 3. (35)
Before closing this section we may observe one more property for Mα(f). As
an expected value
Mα(f) =
E[f(x)]1−α − 1
. (36)
But Kerridge’s (Kerridge, 1961) measure of “inaccuracy” in assigning q(x) for
the true density f(x), in the generalized form is
Hα(f : q) =
(21−α − 1)
E[q(x)]α−1 − 1
, (37)
which is also connected to the measure of directed divergence between q(x) and
f(x). In (37) the normalizing constant is 21−α−1, the same factor appearing in
Havrda-Charvt́ entropy. With different normalizing constants, as seen before,
(36) and (37) have the same forms as an expected value with q(x) replaced
by f(x) in (36). Hence Mα(f) can also be looked upon as a type of directed
divergence or “inaccuracy” measure.
3. Differential Equations
The functional part in (34), for a more general exponent, namely
g(x) =
= xγ−1[1− s(1 − α)xδ]
1−α , α 6= 1, δ > 0, β > 0, s > 0 (38)
is seen to satisfy the following differential equation for γ 6= 1 which defines the
differential pathway.
g(x) = (γ − 1)xγ−1[1− s(1− α)xδ]
−sβδxδ+γ−1[1− s(1− α)xδ]
(1−α)
. (39)
Then for δ =
(γ−1)(α−1)
, γ 6= 1, α > 1 we have
g(x) = (γ − 1)g(x)− sβδ[g(x)]1−
(1−α)
β (40)
= (γ − 1)g(x)− sδ[g(x)]α (41)
for β = 1, γ 6= 1, δ = (γ − 1)(α− 1), α > 1.
For γ = 1, δ = 1 in (38) we have
g(x) = −s[g(x)]η, η = 1−
(1 − α)
= −s[g(x)]α for β = 1. (43)
Here (43) is the power law coming from Tsallis statistics (Gell-Mann and Tsallis,
2004).
Acknowledgement The authors would like to thank the Department of Science
and Technology, Government of India, New Delhi, for the financial assistance for
this work under project No. SR/S4/MS:287/05 which enabled this collaboration
possible.
4. References
Beck, C. (2006). Stretched exponentials from superstatistics. Physica A, 365,
96-101.
Beck, C. and Cohen, E.G.D. (2003). Superstatistics. Physica A, 322, 267-275.
Cohen, E.G.D. (2005). Boltzmann and Einstein: Statistics and dynamics - An
unsolved problem. Pramana, 64, 635-643.
Douglas, P., Bergamini, S., and Renzoni, F. (2006). Tunable Tsallis distribution
in dissipative optical lattices. Physical Review Letters, 96, 110601-1-4.
Ferri, G.L., Martinez, S., and Plastino, A. (2005). Equivalence of the four
versions of Tsallis’s statistics. Journal of Statistical Mechanics: Theory and
Experiment, PO4009.
Gell-Mann, M. and Tsallis, C. (Eds.) (2004). Nonextensive Statistical Mechan-
ics: Interdisciplinary Applications. Oxford University Press, Oxford.
Havrda, J. and Charvát, F. (1967). Quantification method of classification pro-
cedures: Concept of structural α-entropy. Kybernetika, 3, 30-35.
Kerridge, D.F. (1961). Inaccuracy and inference. Journal of the Royal Statisti-
cal Society Series B, 23, 184-194.
Mathai, A.M. (2005). A pathway to matrix-variate gamma and normal densi-
ties. Linear Algebra and Its Applications, 396, 317-328.
Mathai, A.M. and Haubold, H.J. (2006). Pathway model, Tsallis statistics, su-
perstatistics and a generalized measure of entropy. Physica A , 375), 110-122.
Mathai,A.M. and Haubold, H.J. (2006a). On generalized distributions and path-
ways. arXiv:cond-mat/0609526v2.
Mathai, A.M. and Rathie, P.N. (1975). Basic Concepts in Information Theory
and Statistics: Axiomatic Foundations and Applications, Wiley Halstead, New
York and Wiley Eastern, New Delhi.
Rényi, A. (1961). On measure of entropy and information. Proceedings of the
Fourth Berkeley Symposium on Mathematical Statistics and Probability, 1960,
University of California Press, 1961, Vol. 1, 547-561.
Shannon, C.E. (1948). A mathematical theory of communication. Bell System
Technical Journal, 27, 379-423, 547-561.
Tsallis, C. (1988). Possible generalization of Boltzmann-Gibbs statistics. Jour-
nal of Statistical Physics, 52, 479-487.
Tsallis, C. (2004). What should a statistical mechanics satisfy to reflect nature?,
Physica D, 193, 3-34.
http://arxiv.org/abs/cond-mat/0609526
