# Characterizations of Generalized Entropy Functions by Functional Equations

Authors: Shigeru Furuichi
Journal: Advances in Mathematical Physics
 (2011)

---

## Abstract

We will show that a two-parameter extended entropy function is characterized by
a functional equation. As a corollary of this result, we obtain that Tsallis entropy function is
characterized by a functional equation, which is a different form that used by Suyari and Tsukada, 2009, that is, in a proposition
2.1 in the present paper. We give an interpretation of the functional equation in our main
theorem.


---

## Body

## 1. Introduction

Recently, generalized entropies have been studied from the mathematical point of view. The typical generalizations of Shannon entropy [1] are Rényi entropy [2] and Tsallis entropy [3]. The recent comprehensive book [4] and the review [5] support to understand the Tsallis statistics for the readers. Rényi entropy and Tsallis entropy are defined by (1.1)Rq(X)=11-qlog⁡∑j=1npjq,(q≠1,q>0),Sq(X)=∑j=1npjq-pj1-q,(q≠1,q>0),for a given information sourceX={x1,…,xn} with the probability pj≡Pr(X=xj). Both entropies recover Shannon entropy (1.2)S1(X)≡-∑j=1npjlog⁡pj,
in the limit q→1. The uniqueness theorem for Tsallis entropy was firstly given in [6] and improved in [7].Throughout this paper, we call a parametric extended entropy, such as Rényi entropy and Tsallis entropy, a generalized entropy. If we taken=2 in (1.2), we have the so-called binary entropy sb(x)=-xlogx-(1-x)log(1-x). Also we take n=1 in (1.2), and we have the Shannon's entropy function f(x)=-xlogx. In this paper, we treat the entropy function with two parameters. We note that we can produce the relative entropic function -yf(x/y)=x(logx-logy)by the use of the Shannon's entropy function f(x).We note that Rényi entropy has the additivity(1.3)Rq(X×Y)=Rq(X)+Rq(Y),
but Tsallis entropy has the nonadditivity (1.4)Sq(X×Y)=Sq(X)+Sq(Y)+(1-q)Sq(X)Sq(Y),
where X×Y means that X and Y are independent random variables. Therefore, we have a definitive difference for these entropies although we have the simple relation between them (1.5)exp⁡(Rq(X))=exp⁡q(Sq(X)),(q≠1),
where q-exponential function expq(x)≡{1+(1-q)x}1/(1-q) is defined if 1+(1-q)x≥0. Note that we have expq(Sq(X))=(∑j=1npjq)1/(1-q)>0.Tsallis entropy is rewritten by(1.6)Sq(X)=-∑j=1npjqln⁡qpj,
where q-logarithmic function (which is an inverse function of expq(·)) is defined by (1.7)ln⁡qx≡x1-q-11-q,(q≠1),
which converges to logx in the limit q→1.Since Shannon entropy can be regarded as the expectation value for each value-logpj, we may consider that Tsallis entropy can be regarded as the q-expectation value for each value -lnqpj, as an analogy to the Shannon entropy, where q-expectation value Eq is defined by (1.8)Eq(X)≡∑j=1npjqxj.However, theq-expectation value Eq lacks the fundamental property such as E(1)=1, so that it was considered to be inadequate to adopt as a generalized definition of the usual expectation value. Then the normalized q-expectation value was introduced (1.9)Eq(nor)(X)≡∑j=1npjqxj∑i=1npiq,
and by using this, the normalized Tsallis entropy was defined by (1.10)Sq(nor)(X)≡Sq(X)∑j=1npjq=-∑j=1npjqln⁡qpj∑i=1npiq,(q≠1).We easily find that we have the following nonadditivity relation for the normalized Tsallis entropy:(1.11)Sq(nor)(X×Y)=Sq(nor)(X)+Sq(nor)(Y)+(q-1)Sq(nor)(X)Sq(nor)(Y).As for the details on the mathematical properties of the normalized Tsallis entropy, see [8], for example. See also [9] for the role of Tsallis entropy and the normalized Tsallis entropy in statistical physics. The difference between two non-additivity relations (1.4) and (1.11) is the signature of the coefficient 1-q in the third term of the right-hand sides.We note that Tsallis entropy is also rewritten by(1.12)Sq(X)=∑j=1npjln⁡q1pj,
so that we may regard it as the expectation value such as Sq(X)=E1[lnq1/pj],where E1 means the usual expectation value E1[X]=∑j=1npjxj. However, if we adopt this formulation in the definition of Tsallis conditional entropy, we do not have an important property such as a chain rule (see [10] for details). Therefore, we often adopt the formulation using the q-expectation value.As a further generalization, a two-parameter extended entropy(1.13)Sκ,r(X)≡-∑j=1npjln⁡(κ,r)(pj)
was recently introduced in [11, 12] and systematically studied with the generalized exponential function and the generalized logarithmic function lnκ,r(x)≡xr((xκ-x-κ)/2κ). In the present paper, we treat a two-parameter extended entropy defined in the following form: (1.14)Sα,β(X)≡∑j=1npjα-pjββ-α,(α,β∈R,α≠β),for two positive numbersα and β. This form can be obtained by putting α=1+r-κ and β=1+r+κ in (1.13), and it coincides with the two-parameter extended entropy studied in [13]. In addition, the two-parameter extended entropy (1.14) was axiomatically characterized in [14]. Furthermore, a two-parameter extended relative entropy was also axiomatically characterized in [15].In the paper [16], a characterization of Tsallis entropy function was proven by using the functional equation. In the present paper, we will show that the two-parameter extended entropy function (1.15)fα,β(x)=xα-xββ-α(α,β∈R,α≠β)
can be characterized by the simple functional equation.


## 2. A Review of the Characterization of Tsallis Entropy Function by the Functional Equation

The following proposition was originally given in [16] by the simple and elegant proof. Here, we give the alternative proof along to the proof given in [17].Proposition 2.1 (see [16]).
If the differentiable nonnegative functionfq with positive parameter q∈ℝ satisfies the following functional equation:
(2.1)fq(xy)+fq((1-x)y)-fq(y)=(fq(x)+fq(1-x))yq,(0<x<1,0<y≤1),
then the function fq is uniquely given by
(2.2)fq(x)=-cqxqln⁡qx,
where cq is a nonnegative constant depending only on the parameter q.Proof.
If we puty=1 in (2.1), then we have fq(1)=0. From here, we assume that  y≠1. We also put gq(t)≡fq(t)/t then we have
(2.3)xgq(xy)+(1-x)gq((1-x)y)-gq(y)=(xgq(x)+(1-x)gq(1-x))yq-1.
Putting x=1/2 in (2.3), we have
(2.4)gq(y2)=gq(12)yq-1+gq(y).
Substituting  y/2 into y, we have
(2.5)gq(y22)=gq(12)(yq-1+(y2)q-1)+gq(y).
By repeating similar substitutions, we have
(2.6)gq(y2N)=gq(12)yq-1(1+(12)q-1+(12)2(q-1)+⋯+(12)(N-1)(q-1))+gq(y)=gq(12)yq-1(2N(1-q)-121-q-1)+gq(y).
Then, we have
(2.7)lim⁡N→∞gq(y/2N)2N=0,
due to q>0. Differentiating (2.3) by y, we have
(2.8)x2gq(xy)+(1-x)2gq((1-x)y)-gq′(y)=(q-1)(xgq(x)+(1-x)gq(1-x))yq-2.
Putting y=1 in the above equation, we have
(2.9)x2gq′(x)+(1-x)2gq′(1-x)+(1-q)(xgq(x)+(1-x)gq(1-x))=-cq,
where cq=-gq′(1).
By integrating (2.3) from 2-N to 1 with respect to y and performing the conversion of the variables, we have
(2.10)∫2-Nxxgq(t)dt+∫2-N(1-x)1-xgq(t)dt-∫2-N1gq(t)dt=(xgq(x)+(1-x)gq(1-x))1-2-qNq.
By differentiating the above equation with respect to x, we have
(2.11)gq(x)-2-Ngq(2-Nx)-gq(1-x)+2-Ngq(2-N(1-x))=1-2-qNq(gq(x)+xgq′(x)-gq(1-x)-(1-x)gq′(1-x)).
Taking the limit N→∞ in the above, we have
(2.12)(1-x)gq′(x)+(1-q)gq(1-x)=xgq′(x)+(1-q)gq(x),
thanks to (2.7). From (2.9) and (2.12), we have the following differential equation:
(2.13)xgq′(x)+(1-q)gq(x)=-cq.
This differential equation has the following general solution:
(2.14)gq(x)=-cq1-q+dqxq-1,
where dq is an integral constant depending on q. From gq(1)=0, we have dq=cq/(1-q). Thus, we have
(2.15)gq(x)=cqxq-1-11-q.
Finally, we have
(2.16)fq(x)=cqxq-x1-q=-cqxqln⁡qx.
From fq(x)≥0, we have cq≥0.
If we take the limit asq→1 in Proposition 2.1, we have the following corollary.Corollary 2.2 (see [17]).
If the differentiable nonnegative functionf satisfies the following functional equation:
(2.17)f(xy)+f((1-x)y)-f(y)=(f(x)+f(1-x))y,(0<x<1,0<y≤1),
then the function f is uniquely given by
(2.18)f(x)=-cxlog⁡⁡x,
where c is a nonnegative constant.


## 3. Main Results

In this section, we give a characterization of a two-parameter extended entropy function by the functional equation. Before we give our main theorem, we review the following result given by Kannappan [18, 19].Proposition 3.1 (see [18, 19]).
Let two probability distributions(p1,…,pn) and  (q1,…,qm). If the measureable function  f:(0,1)→ℝ satisfies
(3.1)∑i=1n∑j=1mf(piqj)=∑i=1npiα∑j=1mf(qj)+∑j=1mqjβ∑i=1nf(pi),
for all (p1,…,pn) and (q1,…,qm) with fixed m,n≥3, then the function f is given by
(3.2)f(p)={c(pα-pβ),α≠β,cpαlog⁡p,α=β,cplog⁡p+b(mn-m-n)p+b,α=β=1,
where c and b are arbitrary constants.Here, we review a two-parameter generalized Shannon additivity, [14, equation (30)](3.3)∑i=1n∑j=1misα,β(pij)=∑i=1npiα∑j=1misα,β(p(j∣i))+∑i=1nsα,β(pi)∑j=1mip(j∣i)β,
where sα,β is a component of the trace form of the two-parameter entropy [14, equation (26)] (3.4)Sα,β(pi)=∑i=1nsα,β(pi).Equation (3.3) was used to prove the uniqueness theorem for two-parameter extended entropy in [14]. As for (3.3), a tree-graphical interpretation was given in [14]. The condition (3.1) can be read as the independent case (p(j∣i)=pj) in  (3.3).Here, we consider the nontrivial simplest case for (3.3). Take pij={q1,q2,q3}, p1=q1+q2, and p2=q3. then we have p(1∣1)=q1/(q1+q2), p(2∣1)=q2/(q1+q2), p(1∣2)=1, and p(2∣2)=0, then (3.3) is written by(3.5)Sα,β(q1,q2,q3)=(q1+q2)α{sα,β(q1q1+q2)+sα,β(q2q1+q2)}+q3α{sα,β(1)+sα,β(0)}+sα,β(q1+q2){(q1q1+q2)β+(q2q1+q2)β}+sα,β(q3).Ifsα,β is an entropic function, then it vanishes at 0 or 1, since the entropy has no informational quantity for the deterministic cases, then the above identity is reduced in the following: (3.6)Sα,β(q1,q2,q3)=(q1+q2)α{sα,β(q1q1+q2)+sα,β(q2q1+q2)}+sα,β(q1+q2){(q1q1+q2)β+(q2q1+q2)β}+sα,β(q3).In the following theorem, we adopt a simpler condition than (3.1).Theorem 3.2.
If the differentiable nonnegative functionfα,β with two positive parameters  α,β∈ℝ satisfies the following functional equation:
(3.7)fα,β(xy)=xαfα,β(y)+yβfα,β(x),(0<x,y≤1),
then the function fα,β is uniquely given by
(3.8)fα,β(x)=cα,βxβ-xαα-β,(α≠β),fα(x)=-cαxαlog⁡x,(α=β),
where cα,β and cα are nonnegative constants depending only on the parameters α (and β).Proof.
If we puty=1, then we have fα,β(1)=0 due to x>0. By differentiating  (3.7) with respect to y, we have
(3.9)xfα,β′(xy)=xαfα,β′(y)+βyβ-1fα,β(x).
Putting y=1 in (3.9), we have the following differential equation:
(3.10)xfα,β′(x)-βfα,β(x)=-cα,βxα,
where we put cα,β≡-fα,β′(1). Equation (3.10) can be deformed as follows:
(3.11)xβ+1(x-βfα,β(x))′=-cα,βxα,
that is, we have
(3.12)(x-βfα,β(x))′=-cα,βxα-β-1.
Integrating both sides on the above equation with respect to x, we have
(3.13)x-βfα,β(x)=-cα,βα-βxα-β+dα,β,
where dα,β is a integral constant depending on α and β. Therefore, we have
(3.14)fα,β(x)=-cα,βα-βxα+dα,βxβ.
By fα,β(1)=0, we have dα,β=cα,β/(α-β). Thus, we have
(3.15)fα,β(x)=cα,βα-β(xβ-xα).
Also by fα,β(x)≥0, we have cα,β≥0.
As for the case ofα=β, we can prove by the similar way.Remark 3.3.
We can derive  (3.6) from our condition  (3.7). Firstly, we easily have fα,β(0)=fα,β(1)=0 from our condition equation  (3.7). In addition, we have for q=q1+q2,
(3.16)Sα,β(qq1q,qq2q,q3)=fα,β(qq1q)+fα,β(qq2q)+fα,β(q3)=qαfα,β(q1q)+(q1q)βfα,β(q)+qαfα,β(q2q)+(q2q)βfα,β(q)+fα,β(q3)=(q1+q2)α{fα,β(q1q1+q2)+fα,β(q2q1+q2)}+fα,β(q1+q2){(q1q1+q2)β+(q2q1+q2)β}+fα,β(q3).
Thus, we may interpret that our condition  (3.7) contains an essential part of the two-parameter generalized Shannon additivity.
Note that we can reproduce the two-parameter entropic function by the use offα,β as
(3.17)-yfα,β(xy)=xαy1-β-xβy1-αα-β,
with cα,β=1 for simplicity. This leads to two-parameter extended relative entropy [15]
(3.18)Dα,β(x1,…,xn||y1,…,yn)≡∑j=1nxjαyj1-β-xjβyj1-αα-β.
See also [20] on the first appearance of the Tsallis relative entopy (generalized Kullback-Leibler information).
If we takeα=q,β=1 or α=1,β=q in Theorem 3.2, we have the following corollary.Corollary 3.4.
If the differentiable nonnegative functionfq with a positive parameter  q∈ℝ satisfies the following functional equation:
(3.19)fq(xy)=xqfq(y)+yfq(x),(0<x,y≤1,q≠1),
then the function fq is uniquely given by
(3.20)fq(x)=-cqxqln⁡qx,
where cq is a nonnegative constant depending only on the parameter q.Here, we give an interpretation of the functional equation (3.19) from the view of Tsallis statistics.Remark 3.5.
We assume that we have the following two functional equations for0<x,y≤1:
(3.21)fq(xy)=yfq(x)+xfq(y)+(1-q)fq(x)fq(y),fq(xy)=yqfq(x)+xqfq(y)+(q-1)fq(x)fq(y).
These equations lead to the following equations for 0<xi,yj≤1:
(3.22)fq(xiyj)=yjfq(xi)+xifq(yj)+(1-q)fq(xi)fq(yj),fq(xiyj)=yjqfq(xi)+xiqfq(yj)+(q-1)fq(xi)fq(yj),
where i=1,…,n and j=1,…,m. Taking the summation on i and j in both sides, we have
(3.23)∑i=1n∑j=1mfq(xiyj)=∑i=1nfq(xi)+∑j=1mfq(yj)+(1-q)∑i=1nfq(xi)∑j=1mfq(yj),(3.24)∑i=1n∑j=1mfq(xiyj)=∑j=1myjq∑i=1nfq(xi)+∑i=1nxiq∑j=1mfq(yj)+(q-1)∑i=1nfq(xi)∑j=1mfq(yj),
under the condition ∑i=1nxi=∑j=1myj=1. If the function fq(x) is given by (3.20), then two above functional equations coincide with two nonadditivity relations given in  (1.4) and (1.11).
On the other hand, we have the following equation from (23) and (3.21):
(3.25)fq(xy)=(xq+x2)fq(y)+(yq+y2)fq(x),(0<x,y≤1,q≠1).
By a similar way to the proof of Theorem 3.2, we can show that the functional equation (3.25) uniquely determines the function fq by the form given in (3.20). Therefore, we can conclude that two functional equations (23) and (3.21), which correspond to the non-additivity relations  (1.4) and  (1.11), also characterize Tsallis entropy function.
If we again take the limit asq→1 in Corollary 3.4, we have the following corollary.Corollary 3.6.
If the differentiable nonnegative functionf satisfies the following functional equation:
(3.26)f(xy)=yf(x)+xf(y),(0<x,y≤1),
then the function f is uniquely given by
(3.27)f(x)=-cxlog⁡x,
where c is a nonnegative constant.


## 4. Conclusion

As we have seen, the two-parameter extended entropy function can be uniquely determined by a simple functional equation. Also an interpretation related to a tree-graphical structure was given as a remark.Recently, the extensive behaviours of generalized entropies were studied in [21–23]. Our condition given in (3.7) may be seen as extensive form. However, I have not yet found any relation between our functional (3.7) and the extensive behaviours of the generalized entropies. This problem is not the purpose of the present paper, but it is quite interesting to study this problem as a future work.



---

*Source: 126108-2011-07-24.xml*