# Bounds on Nonsymmetric Divergence Measure in terms of Other Symmetric and Nonsymmetric Divergence Measures

Authors: K. C. Jain; Praphull Chhabra
Journal: International Scholarly Research Notices
 (2014)

---

## Abstract

Vajda (1972) studied a generalized divergence measure of Csiszar’s class, so called “Chi-m divergence measure.” Variational distance and Chi-square divergence are the special cases of this generalized divergence measure at m
=
1 and m
=
2, respectively. In this work, nonparametric nonsymmetric measure of divergence, a particular part of Vajda generalized divergence at m
=
4, is taken and characterized. Its bounds are studied in terms of some well-known symmetric and nonsymmetric divergence measures of Csiszar’s class by using well-known information inequalities. Comparison of this divergence with others is done. Numerical illustrations (verification) regarding bounds of this divergence are presented as well.


---

## Body

## 1. Introduction

LetΓ
n
=
P
=
p
1
,
p
2
,
p
3
,
…
,
p
n
:
p
i
>
0
,
∑
i
=
1
n
p
i
=
1
,
n
≥
2 be the set of all complete finite discrete probability distributions. If we take p
i
≥
0 for some i
=
1
,
2
,
3
,
…
,
n, then we have to suppose that 0
f
0
=
0
f
0
/
0
=
0.Csiszar [1] introduced a generalized f-divergence measure, which is given by
(1)
C
f
P
,
Q
=
∑
i
=
1
n
q
i
f
p
i
q
i
,
where f
:
(
0
,
∞
)
→
R (set of real numbers) is a real, continuous, and convex function and P
=
p
1
,
p
2
,
p
3
,
…
,
p
n, Q
=
q
1
,
q
2
,
q
3
,
…
,
q
n
∈
Γ
n, where p
i and q
i are probability mass functions. Many known divergences can be obtained from this generalized measure by suitably defining the convex function f. Some of those are as follows.


### 1.1. Symmetric Divergence Measures

Symmetric measures are those that are symmetric with respect to probability distributionsP
,
Q
∈
Γ
n. These measures are as follows (see [2–7]):
(2)
Triangular
discrimination
2
=
Δ
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
2
p
i
+
q
i
,
(3)
Hellinger
discrimination
3
=
h
P
,
Q
hhhhhhhhhhhhhhhhhhhhh
=
1
2
∑
i
=
1
n
p
i
-
q
i
2
,
(4)
JS
divergence
4,5
=
I
P
,
Q
=
1
2
∑
i
=
1
n
p
i
log
⁡
2
p
i
p
i
+
q
i
+
∑
i
=
1
n
q
i
log
⁡
2
q
i
p
i
+
q
i
,
(5)
Jain
and
Mathur
6
=
P
*
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
+
q
i
p
i
2
+
q
i
2
p
i
3
q
i
3
,
(6)
Jain
and
Srivastava
7
=
E
*
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
q
i
3
/
2
.


### 1.2. Nonsymmetric Divergence Measures

Nonsymmetric measures are those that are not symmetric with respect to probability distributionsP
,
Q
∈
Γ
n. These measures are as follows (see [8–10]):
(7)
Relative
J
-divergence
8
=
J
R
P
,
Q
Relative
J
-divergence
8
=
∑
i
=
1
n
p
i
-
q
i
log
⁡
p
i
+
q
i
2
q
i
,
(8)
Relative
information
9
=
K
P
,
Q
=
∑
i
=
1
n
p
i
log
⁡
p
i
q
i
,
(9)
Relative
AG
divergence
10
=
G
P
,
Q
=
∑
i
=
1
n
p
i
+
q
i
2
log
⁡
p
i
+
q
i
2
p
i
.
We can see that J
R
P
,
Q
=
2
F
Q
,
P
+
G
Q
,
P, Δ
P
,
Q
=
2
1
-
W
P
,
Q, h
P
,
Q
=
1
-
B
P
,
Q, and I
P
,
Q
=
(
1
/
2
)
F
P
,
Q
+
F
Q
,
P, where W
P
,
Q
=
2
∑
i
=
1
n
(
p
i
q
i
/
p
i
+
q
i
) is the harmonic mean divergence, B
P
,
Q
=
∑
i
=
1
n
p
i
q
i is the geometric mean divergence, and F
P
,
Q
=
∑
i
=
1
n
p
i
log
⁡
2
p
i
/
(
p
i
+
q
i
) is the relative JS divergence [5]. Equations (3), (4), and (8) are also known as Kolmogorov’s measure, information radius, and directed divergence, respectively.


## 1.1. Symmetric Divergence Measures

Symmetric measures are those that are symmetric with respect to probability distributionsP
,
Q
∈
Γ
n. These measures are as follows (see [2–7]):
(2)
Triangular
discrimination
2
=
Δ
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
2
p
i
+
q
i
,
(3)
Hellinger
discrimination
3
=
h
P
,
Q
hhhhhhhhhhhhhhhhhhhhh
=
1
2
∑
i
=
1
n
p
i
-
q
i
2
,
(4)
JS
divergence
4,5
=
I
P
,
Q
=
1
2
∑
i
=
1
n
p
i
log
⁡
2
p
i
p
i
+
q
i
+
∑
i
=
1
n
q
i
log
⁡
2
q
i
p
i
+
q
i
,
(5)
Jain
and
Mathur
6
=
P
*
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
+
q
i
p
i
2
+
q
i
2
p
i
3
q
i
3
,
(6)
Jain
and
Srivastava
7
=
E
*
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
q
i
3
/
2
.


## 1.2. Nonsymmetric Divergence Measures

Nonsymmetric measures are those that are not symmetric with respect to probability distributionsP
,
Q
∈
Γ
n. These measures are as follows (see [8–10]):
(7)
Relative
J
-divergence
8
=
J
R
P
,
Q
Relative
J
-divergence
8
=
∑
i
=
1
n
p
i
-
q
i
log
⁡
p
i
+
q
i
2
q
i
,
(8)
Relative
information
9
=
K
P
,
Q
=
∑
i
=
1
n
p
i
log
⁡
p
i
q
i
,
(9)
Relative
AG
divergence
10
=
G
P
,
Q
=
∑
i
=
1
n
p
i
+
q
i
2
log
⁡
p
i
+
q
i
2
p
i
.
We can see that J
R
P
,
Q
=
2
F
Q
,
P
+
G
Q
,
P, Δ
P
,
Q
=
2
1
-
W
P
,
Q, h
P
,
Q
=
1
-
B
P
,
Q, and I
P
,
Q
=
(
1
/
2
)
F
P
,
Q
+
F
Q
,
P, where W
P
,
Q
=
2
∑
i
=
1
n
(
p
i
q
i
/
p
i
+
q
i
) is the harmonic mean divergence, B
P
,
Q
=
∑
i
=
1
n
p
i
q
i is the geometric mean divergence, and F
P
,
Q
=
∑
i
=
1
n
p
i
log
⁡
2
p
i
/
(
p
i
+
q
i
) is the relative JS divergence [5]. Equations (3), (4), and (8) are also known as Kolmogorov’s measure, information radius, and directed divergence, respectively.


## 2. Nonsymmetric Divergence Measure and Its Properties

In this section, we obtain nonsymmetric divergence measure for convex function and further define the properties of function and divergence. Firstly, Theorem1 is well known in literature [1].Theorem 1.
If the functionf is convex and normalized, that is, f
1
=
0, then C
f
P
,
Q and its adjoint C
f
Q
,
P are both nonnegative and convex in the pair of probability distribution P
,
Q
∈
Γ
n
×
Γ
n.Now, letf
:
0
,
∞
→
R be a function defined as
(10)
f
t
=
f
1
t
=
t
-
1
4
t
3
,
∀
t
∈
0
,
∞
,
f
1
1
=
0
,
f
1
'
t
=
t
-
1
3
t
+
3
t
4
,
f
1
′′
t
=
12
t
-
1
2
t
5
.
Properties of the function defined by (10) are as follows.(a)
Sincef
1
1
=
0
,
f
1
t is a normalized function.
(b)
Sincef
1
′′
t
≥
0 for all t
∈
0
,
∞
⇒
f
1
t is a convex function as well.
(c)
Sincef
1
'
t
<
0 at 0,1 and f
1
'
t
>
0 at 1
,
∞
⇒
f
1
t is monotonically decreasing in 0,1 and monotonically increasing in 1
,
∞ and f
1
'
1
=
0, f
1
′′
1
=
0.Now putf
1
t in (1); we get the following new divergence measure:
(11)
χ
4
Q
,
P
=
C
f
1
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
3
.
χ
4
Q
,
P is called adjoint of χ
4
P
,
Q
=
∑
i
=
1
n
(
p
i
-
q
i
4
/
q
i
3
) (after putting conjugate convex function of χ
4
Q
,
P in (1)) and it is a particular case of “Chi-m divergence measure [11]” at m
=
4, which is given by χ
m
P
,
Q
=
∑
i
=
1
n
(
p
i
-
q
i
m
/
q
i
m
-
1
), m
≥
1.Properties of the divergence measure defined in (11) are as follows.(a)
In view of Theorem1, we can say that χ
4
Q
,
P is convex and nonnegative in the pair of probability distribution P
,
Q
∈
Γ
n
×
Γ
n.
(b)
χ
4
Q
,
P
=
0 if P
=
Q or p
i
=
q
i (attaining its minimum value).
(c)
Sinceχ
4
Q
,
P
≠
χ
4
P
,
Q
⇒
χ
4
Q
,
P is a nonsymmetric divergence measure.


## 3. Information Inequalities of Csiszar’s Class

In this section, we are taking well-known information inequalities onC
f
P
,
Q, which is given by Theorem 2. Such inequalities are, for instance, needed in order to calculate the relative efficiency of two divergences. This theorem is due to literature [12], which relates two generalized f-divergence measures.Theorem 2.
Letf
1
,
f
2
:
I
⊂
R
+
→
R be two convex and normalized functions, that is, f
1
1
=
f
2
1
=
0, and suppose the following assumptions.(a)
f
1 and f
2 are twice differentiable on (
α
,
β
) where 0
<
α
≤
1
≤
β
<
∞, α
≠
β.
(b)
There exist the real constantsm, M such that m
<
M and
(12)
m
≤
f
1
′′
t
f
2
′′
t
≤
M
,
where f
2
′′
t
>
0 for all t
∈
α
,
β.
IfP
,
Q
∈
Γ
n and satisfying the assumption 0
<
α
≤
p
i
/
q
i
≤
β
<
∞, then one has the following inequalities:
(13)
m
C
f
2
P
,
Q
≤
C
f
1
P
,
Q
≤
M
C
f
2
P
,
Q
,
where C
f
P
,
Q is given by (1).


## 4. Bounds in terms of Symmetric Divergence Measures

Now in this section, we obtain bounds of divergence measure (11) in terms of other symmetric divergence measures by using Theorem 2.Proposition 3.
LetΔ
P
,
Q and χ
4
Q
,
P be defined as in (2) and (11), respectively. For P, Q
∈
Γ
n, one has the following.(a)
If0
<
α
<
1, then
(14)
0
≤
χ
4
P
,
Q
≤
3
2
max
⁡
α
-
1
2
α
+
1
3
α
5
,
β
-
1
2
β
+
1
3
β
5
Δ
P
,
Q
.
(b)
Ifα
=
1, then
(15)
0
≤
χ
4
P
,
Q
≤
3
β
-
1
2
β
+
1
3
2
β
5
Δ
P
,
Q
.Proof.
Let us consider(16)
f
2
t
=
t
-
1
2
t
+
1
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
t
-
1
t
+
3
t
+
1
2
,
f
2
′′
t
=
8
t
+
1
3
.
Since f
2
′′
t
>
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(17)
C
f
2
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
2
p
i
+
q
i
=
Δ
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
3
t
-
1
2
t
+
1
3
/
2
t
5, where f
1
′′
t and f
2
′′
t are given by (10) and (16), respectively, and
(18)
g
'
t
=
3
t
-
1
t
+
1
2
5
-
t
2
t
6
,
g
′′
t
=
3
t
4
-
6
t
3
-
12
t
2
+
10
t
+
15
t
7
.
If g
'
t
=
0
⇒
t
=
1, t
=
-
1, and t
=
5.
It is clear thatg
t is decreasing in (
0,1
) and [
5
,
∞
) but increasing in [
1,5
).
Alsog
t has a minimum and maximum value at t
=
1 and t
=
5, respectively, because g
′′
1
=
24
>
0 and g
′′
5
=
-
216
/
15625
<
0, so
(19)
m
=
inf
⁡
t
∈
0
,
∞
g
t
=
g
1
=
0
.
And(a)
if0
<
α
<
1, then
(20)
M
=
sup
⁡
t
∈
α
,
β
g
t
=
max
⁡
⁡
g
α
,
g
β
=
max
⁡
⁡
3
α
-
1
2
α
+
1
3
2
α
5
,
3
β
-
1
2
β
+
1
3
2
β
5
;
(b)
ifα
=
1, then
(21)
M
=
sup
⁡
t
∈
1
,
β
g
t
=
g
β
=
3
β
-
1
2
β
+
1
3
2
β
5
.Results (14) and (15) are obtained by using (11), (17), (19), (20), and (21) in (13), after interchanging P and Q.Proposition 4.
Leth
P
,
Q and χ
4
Q
,
P be defined as in (3) and (11), respectively. For P
,
Q
∈
Γ
n, one has the following.(a)
If0
<
α
<
1, then
(22)
0
≤
χ
4
P
,
Q
≤
48
max
⁡
⁡
α
-
1
2
α
7
/
2
,
β
-
1
2
β
7
/
2
h
P
,
Q
.
(b)
Ifα
=
1, then
(23)
0
≤
χ
4
P
,
Q
≤
48
β
-
1
2
β
7
/
2
h
P
,
Q
.Proof.
Let us consider(24)
f
2
t
=
1
-
t
2
2
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
-
1
2
t
-
1
/
2
-
1
,
f
2
′′
t
=
1
4
t
3
/
2
.
Since f
2
′′
t
>
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(25)
C
f
2
P
,
Q
=
1
2
∑
i
=
1
n
p
i
-
q
i
2
=
h
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
48
t
-
1
2
/
t
7
/
2, where f
1
′′
t and f
2
′′
t are given by (10) and (24), respectively, and
(26)
g
'
t
=
24
t
-
1
7
-
3
t
t
9
/
2
,
g
′′
t
=
12
15
t
2
-
70
t
+
63
t
11
/
2
.
If g
'
t
=
0
⇒
t
=
1 and t
=
7
/
3.
It is clear thatg
t is decreasing in (
0,1
) and [
7
/
3
,
∞
) but increasing in [
1,7
/
3
).
Alsog
t has a minimum and maximum value at t
=
1 and t
=
7
/
3, respectively, because g
′′
1
=
96
>
0 and g
′′
2.33
=
-
1951
/
913
<
0, so
(27)
m
=
inf
⁡
t
∈
0
,
∞
g
t
=
g
1
=
0
.
And(a)
if0
<
α
<
1, then
(28)
M
=
sup
⁡
t
∈
α
,
β
g
t
=
max
⁡
⁡
g
α
,
g
β
=
max
⁡
⁡
48
α
-
1
2
α
7
/
2
,
48
β
-
1
2
β
7
/
2
;
(b)
ifα
=
1, then
(29)
M
=
sup
⁡
t
∈
1
,
β
g
t
=
g
β
=
48
β
-
1
2
β
7
/
2
.Results (22) and (23) are obtained by using (11), (25), (27), (28), and (29) in (13), after interchanging P and Q.Proposition 5.
LetI
P
,
Q and χ
4
Q
,
P be defined as in (4) and (11), respectively. For P
,
Q
∈
Γ
n, one has the following.(a)
If0
<
α
<
1, then
(30)
0
≤
χ
4
P
,
Q
≤
24
max
⁡
⁡
α
-
1
2
α
+
1
α
4
,
β
-
1
2
β
+
1
β
4
I
P
,
Q
.
(b)
Ifα
=
1, then
(31)
0
≤
χ
4
P
,
Q
≤
24
β
-
1
2
β
+
1
β
4
I
P
,
Q
.Proof.
Let us consider(32)
f
2
t
=
t
2
log
⁡
t
+
t
+
1
2
log
⁡
2
t
+
1
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
1
2
log
⁡
2
t
t
+
1
,
f
2
′′
t
=
1
2
t
t
+
1
.
Since f
2
′′
t
>
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(33)
C
f
2
P
,
Q
=
1
2
∑
i
=
1
n
p
i
log
⁡
2
p
i
p
i
+
q
i
+
∑
i
=
1
n
q
i
log
⁡
2
q
i
p
i
+
q
i
=
I
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
24
t
-
1
2
t
+
1
/
t
4, where f
1
′′
t and f
2
′′
t are given by (10) and (32), respectively, and
(34)
g
'
t
=
24
t
-
1
-
t
2
+
t
+
4
t
5
,
g
′′
t
=
48
1
t
3
-
3
t
4
-
6
t
5
+
10
t
6
.
If g
'
t
=
0
⇒
t
=
1, t
=
(
1
-
17
)
/
2, and t
=
(
1
+
17
)
/
2.
It is clear thatg
t is decreasing in (
0,1
) and (
1
+
17
)
/
2
,
∞ but increasing in 1
,
(
1
+
17
)
/
2.
Alsog
t has a minimum and maximum value at t
=
1 and t
=
(
1
+
17
)
/
2, respectively, because g
′′
1
=
96
>
0 and g
′′
(
1
+
17
)
/
2
=
-
1216
/
865
<
0, so
(35)
m
=
inf
⁡
t
∈
0
,
∞
g
t
=
g
1
=
0
.
And(a)
if0
<
α
<
1, then
(36)
M
=
sup
⁡
t
∈
α
,
β
g
t
=
max
⁡
⁡
g
α
,
g
β
=
max
⁡
⁡
24
α
-
1
2
α
+
1
α
4
,
24
β
-
1
2
β
+
1
β
4
;
(b)
ifα
=
1, then
(37)
M
=
sup
⁡
t
∈
1
,
β
g
t
=
g
β
=
24
β
-
1
2
β
+
1
β
4
.Results (30) and (31) are obtained by using (11), (33), (35), (36), and (37) in (13), after interchanging P and Q.Proposition 6.
LetP
*
P
,
Q and χ
4
Q
,
P be defined as in (5) and (11), respectively.
ForP
,
Q
∈
Γ
n, one has
(38)
2
2
β
5
+
β
4
+
β
3
+
β
2
+
β
+
2
P
*
P
,
Q
≤
χ
4
P
,
Q
≤
2
2
α
5
+
α
4
+
α
3
+
α
2
+
α
+
2
P
*
P
,
Q
.Proof.
Let us consider(39)
f
2
t
=
t
-
1
4
t
+
1
t
2
+
1
t
3
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
t
-
1
3
4
t
4
+
3
t
3
+
3
t
2
+
3
t
+
3
t
4
,
f
2
′′
t
=
t
-
1
2
t
5
12
t
5
+
6
t
4
+
6
t
3
+
6
t
2
+
6
t
+
12
.
Since f
2
′′
t
≥
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(40)
C
f
2
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
+
q
i
p
i
2
+
q
i
2
p
i
3
q
i
3
=
P
*
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
2
/
(
2
t
5
+
t
4
+
t
3
+
t
2
+
t
+
2
), where f
1
′′
t and f
2
′′
t are given by (10) and (39), respectively, and
(41)
g
'
t
=
-
2
10
t
4
+
4
t
3
+
3
t
2
+
2
t
+
1
2
t
5
+
t
4
+
t
3
+
t
2
+
t
+
2
2
<
0
.
It is clear that g
t is always decreasing in (
0
,
∞
), so
(42)
m
=
inf
⁡
t
∈
α
,
β
g
t
=
g
β
=
2
2
β
5
+
β
4
+
β
3
+
β
2
+
β
+
2
,
M
=
sup
⁡
t
∈
α
,
β
g
t
=
g
α
=
2
2
α
5
+
α
4
+
α
3
+
α
2
+
α
+
2
.
Result (38) is obtained by using (11), (40), and (42) in (13), after interchanging P and Q.Proposition 7.
LetE
*
P
,
Q and χ
4
Q
,
P be defined as in (6) and (11), respectively. For P
,
Q
∈
Γ
n, one has
(43)
16
β
3
/
2
5
β
2
+
6
β
+
5
E
*
P
,
Q
≤
χ
4
P
,
Q
≤
16
α
3
/
2
5
α
2
+
6
α
+
5
E
*
P
,
Q
.Proof.
Let us consider(44)
f
2
t
=
t
-
1
4
t
3
/
2
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
t
-
1
3
5
t
+
3
2
t
5
/
2
,
f
2
′′
t
=
3
t
-
1
2
5
t
2
+
6
t
+
5
4
t
7
/
2
.
Since f
2
′′
t
≥
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(45)
C
f
2
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
4
p
i
q
i
3
/
2
=
E
*
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
16
/
t
3
/
2
5
t
2
+
6
t
+
5, where f
1
′′
t and f
2
′′
t are given by (10) and (44), respectively, and
(46)
g
'
t
=
-
40
7
t
2
+
6
t
+
3
t
5
/
2
5
t
2
+
6
t
+
5
2
<
0
.
It is clear that g
t is always decreasing in (
0
,
∞
), so
(47)
m
=
inf
⁡
t
∈
α
,
β
g
t
=
g
β
=
16
β
3
/
2
5
β
2
+
6
β
+
5
,
M
=
sup
⁡
t
∈
α
,
β
g
t
=
g
α
=
16
α
3
/
2
5
α
2
+
6
α
+
5
.
Result (43) is obtained by using (11), (45), and (47) in (13), after interchanging P and Q.


## 5. Bounds in terms of Nonsymmetric Divergence Measures

Now in this section, we obtain bounds of divergence measure (11) in terms of other nonsymmetric divergence measures by using Theorem 2.Proposition 8.
LetJ
R
P
,
Q and χ
4
Q
,
P be defined as in (7) and (11), respectively. For P
,
Q
∈
Γ
n, one has the following.(a)
If0
<
α
<
1, then
(48)
0
≤
χ
4
P
,
Q
≤
12
max
⁡
⁡
α
2
-
1
2
α
+
3
α
5
,
β
2
-
1
2
β
+
3
β
5
J
R
Q
,
P
.
(b)
Ifα
=
1, then
(49)
0
≤
χ
4
P
,
Q
≤
12
β
2
-
1
2
β
+
3
β
5
J
R
Q
,
P
.Proof.
Let us consider(50)
f
2
t
=
t
-
1
log
⁡
t
+
1
2
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
t
-
1
t
+
1
+
log
⁡
t
+
1
2
,
f
2
′′
t
=
t
+
3
t
+
1
2
.
Since f
2
′′
t
>
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(51)
C
f
2
P
,
Q
=
∑
i
=
1
n
p
i
-
q
i
log
⁡
p
i
+
q
i
2
q
i
=
J
R
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
12
t
2
-
1
2
/
t
+
3
t
5, where f
1
′′
t and f
2
′′
t are given by (10) and (50), respectively, and
(52)
g
'
t
=
-
12
t
2
-
1
2
t
3
+
3
t
2
-
6
t
-
15
t
+
3
2
t
6
,
g
′′
t
=
-
24
-
3
t
6

-
9
t
5

+

11
t
4

+

90
t
3

+

87
t
2

-

105
t

-

135
t
+
3
2
t
7
.
If g
'
t
=
0
⇒
t
=
1,  t
=
-
1, and t
≈
116
/
59.
It is clear thatg
t is decreasing in (
0,1
) and [
116
/
59
,
∞
) but increasing in [
1,116
/
59
).
Alsog
t has a minimum and maximum value at t
=
1 and t
=
116
/
59, respectively, because g
′′
1
=
24
>
0 and g
′′
116
/
59
≈
-
7
/
10
<
0, so
(53)
m
=
inf
⁡
t
∈
0
,
∞
g
t
=
g
1
=
0
.
And(a)
if0
<
α
<
1, then
(54)
M
=
sup
⁡
t
∈
α
,
β
g
t
=
max
⁡
g
α
,
g
β
=
max
⁡
⁡
12
α
2
-
1
2
α
+
3
α
5
,
12
β
2
-
1
2
β
+
3
β
5
;
(b)
ifα
=
1, then
(55)
M
=
sup
⁡
t
∈
1
,
β
g
t
=
g
β
=
12
β
2
-
1
2
β
+
3
β
5
.Results (48) and (49) are obtained by using (11), (51), (53), (54), and (55) in (13), after interchanging P and Q.Proposition 9.
LetK
P
,
Q and χ
4
Q
,
P be defined as in (8) and (11), respectively. For P
,
Q
∈
Γ
n, one has the following.(a)
If0
<
α
<
1, then
(56)
0
≤
χ
4
P
,
Q
≤
12
max
⁡
⁡
α
-
1
2
α
4
,
β
-
1
2
β
4
K
Q
,
P
.
(b)
Ifα
=
1, then
(57)
0
≤
χ
4
P
,
Q
≤
12
β
-
1
2
β
4
K
Q
,
P
.Proof.
Let us consider(58)
f
2
t
=
t
log
⁡
⁡
t
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
1
+
log
⁡
⁡
t
,
f
2
′′
t
=
1
t
.
Since f
2
′′
t
>
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(59)
C
f
2
P
,
Q
=
∑
i
=
1
n
p
i
log
⁡
p
i
q
i
=
K
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
12
t
-
1
2
/
t
4, where f
1
′′
t and f
2
′′
t are given by (10) and (58), respectively, and
(60)
g
'
t
=
24
t
-
1
2
-
t
t
5
,
g
′′
t
=
24
3
t
4
-
12
t
5
+
10
t
6
.
If g
'
t
=
0
⇒
t
=
1 and t
=
2.
It is clear thatg
t is decreasing in (
0,1
) and [
2
,
∞
) but increasing in [
1,2
).
Alsog
t has a minimum and maximum value at t
=
1 and t
=
2, respectively, because g
′′
1
=
24
>
0 and g
′′
2
=
-
3
/
4
<
0, so
(61)
m
=
inf
⁡
t
∈
0
,
∞
g
t
=
g
1
=
0
.
And(a)
if0
<
α
<
1, then
(62)
M
=
sup
⁡
t
∈
α
,
β
g
t
=
max
⁡
g
α
,
g
β
=
max
⁡
⁡
12
α
-
1
2
α
4
,
12
β
-
1
2
β
4
;
(b)
ifα
=
1, then
(63)
M
=
sup
⁡
t
∈
1
,
β
g
t
=
g
β
=
12
β
-
1
2
β
4
.Results (56) and (57) are obtained by using (11), (59), (61), (62), and (63) in (13), after interchanging P and Q.Proposition 10.
LetG
P
,
Q and χ
4
Q
,
P be defined as in (9) and (11), respectively. For P
,
Q
∈
Γ
n, one has the following.(a)
If0
<
α
<
1, then
(64)
0
≤
χ
4
P
,
Q
≤
24
max
⁡
α
-
1
2
α
+
1
α
3
,
β
-
1
2
β
+
1
β
3
G
Q
,
P
.
(b)
Ifα
=
1, then
(65)
0
≤
χ
4
P
,
Q
≤
24
β
-
1
2
β
+
1
β
3
G
Q
,
P
.Proof.
Let us consider(66)
f
2
t
=
t
+
1
2
log
⁡
t
+
1
2
t
,
t
∈
0
,
∞
,
f
2
1
=
0
,
f
2
'
t
=
1
2
log
⁡
t
+
1
2
t
-
1
t
,
f
2
′′
t
=
1
2
t
2
t
+
1
.
Since f
2
′′
t
>
0 for all t
>
0 and f
2
1
=
0, f
2
t is a convex and normalized function, respectively. Now put f
2
t in (1); we get
(67)
C
f
2
P
,
Q
=
∑
i
=
1
n
p
i
+
q
i
2
log
⁡
p
i
+
q
i
2
p
i
=
G
P
,
Q
.
Now, let g
t
=
f
1
′′
t
/
f
2
′′
t
=
24
t
-
1
2
t
+
1
/
t
3, where f
1
′′
t and f
2
′′
t are given by (10) and (66), respectively, and
(68)
g
'
t
=
24
t
-
1
t
+
3
t
4
,
g
′′
t
=
24
-
1
t
3
-
6
t
4
+
12
t
5
.
If g
'
t
=
0
⇒
t
=
1 and t
=
-
3.
It is clear thatg
t is decreasing in (
0,1
) and increasing in [
1
,
∞
).
Alsog
t has a minimum value at t
=
1, because g
′′
1
=
120
>
0, so
(69)
m
=
inf
⁡
t
∈
0
,
∞
g
t
=
g
1
=
0
.
And(a)
if0
<
α
<
1, then
(70)
M
=
sup
⁡
t
∈
α
,
β
g
t
=
max
⁡
⁡
g
α
,
g
β
=
max
⁡
⁡
24
α
-
1
2
α
+
1
α
3
,
24
β
-
1
2
β
+
1
β
3
;
(b)
ifα
=
1, then
(71)
M
=
sup
⁡
t
∈
1
,
β
g
t
=
g
β
=
24
β
-
1
2
β
+
1
β
3
.Results (64) and (65) are obtained by using (11), (67), (69), (70), and (71) in (13), after interchanging P and Q.


## 6. Numerical Illustration

In this section, we give two examples for calculating the divergencesΔ
P
,
Q, h
P
,
Q, J
R
Q
,
P, K
Q
,
P, and χ
4
P
,
Q and verify inequalities (14), (22), (48), and (56) or verify bounds of χ
4
P
,
Q.Example 1.
LetP be the binomial probability distribution with parameters (n
=
10, p
=
1
/
2) and Q its approximated Poisson probability distribution with parameter λ
=
n
p
=
5, for the random variable X; then we have Table 1.Table 1
(n
=
10
,
p
=
1
/
2
,
q
=
1
/
2).
x
i
0
1
2
3
4
5
6
7
8
9
10
p
(
x
i
)
=
p
i
≈
1
1024
5
512
45
1024
15
128
105
512
63
256
105
512
15
128
45
1024
5
512
1
1024
∑
s
X
X
d
s
d
X
X
f
q
(
x
i
)
=
q
i
≈
5
742
25
742
47
558
138
983
73
416
73
416
68
465
68
651
22
337
7
193
2
63
∑
X
X
X
X
X
X
p
i
q
i
≈
10
69
20
69
12
23
197
236
97
83
819
584
1300
927
1040
927
449
667
7
26
2
65
∑
X
X
e
w
eBy using Table1, we get the following:
(72)
α
=
2
65
≤
p
i
q
i
≤
β
=
819
584
,
Δ
P
,
Q
=
∑
i
=
1
11
p
i
-
q
i
2
p
i
+
q
i
≈
21
229
,
h
P
,
Q
=
∑
i
=
1
11
p
i
-
q
i
2
2
≈
13
510
,
J
R
Q
,
P
=
∑
i
=
1
11
q
i
-
p
i
log
⁡
p
i
+
q
i
2
p
i
≈
53
397
,
K
Q
,
P
=
∑
i
=
1
11
q
i
log
⁡
q
i
p
i
≈
19
170
,
χ
4
P
,
Q
=
∑
i
=
1
11
p
i
-
q
i
4
q
i
3
≈
29
572
.Put the approximated numerical values from (72) in (14), (22), (48), and (56) and verify inequalities (14), (22), (48), and (56) for p
=
1
/
2.Example 2.
LetP be the binomial probability distribution with parameters (n
=
10, p
=
7
/
10) and Q its approximated Poisson probability distribution with parameter λ
=
n
p
=
7, for the random variable X; then we have Table 2.
By using Table2, we get the following:
(73)
α
=
5
772
≤
p
i
q
i
≤
β
=
1703
951
,
Δ
P
,
Q
=
∑
i
=
1
11
p
i
-
q
i
2
p
i
+
q
i
≈
160
883
,
h
P
,
Q
=
∑
i
=
1
11
p
i
-
q
i
2
2
≈
25
498
,
J
R
Q
,
P
=
∑
i
=
1
11
q
i
-
p
i
log
⁡
p
i
+
q
i
2
p
i
≈
99
389
,
K
Q
,
P
=
∑
i
=
1
11
q
i
log
⁡
q
i
p
i
≈
142
577
,
χ
4
P
,
Q
=
∑
i
=
1
11
p
i
-
q
i
4
q
i
3
≈
169
924
.
Put the approximated numerical values from (73) in (14), (22), (48), and (56) and verify inequalities (14), (22), (48), and (56) for p
=
7
/
10.
Similarly, we can verify inequalities (or verify bounds ofχ
4
P
,
Q) (30), (38), (43), and (64).Table 2
(n
=
10
,
p
=
7
/
10
,
q
=
3
/
10).
x
i
0
1
2
3
4
5
6
7
8
9
10
p
(
x
i
)
=
p
i
≈
59049
1
0
10
137781
1
0
9
1
691
1
111
34
925
67
651
1
5
111
416
166
711
73
603
5
177
∑
X
X
X
w
d
q
(
x
i
)
=
q
i
≈
9118
1
0
7
3
470
21
940
11
211
26
285
47
368
142
953
142
953
100
767
91
605
60
499
∑
X
X
s
s
d
p
i
q
i
≈
5
772
3
139
16
247
136
787
83
206
83
103
953
710
1703
951
890
497
33
41
230
979
∑
X
X
X
w
e
w
eFigure1 shows the behavior of convex function in (
0
,
∞
). Function is decreasing in (
0,1
) and increasing in (
1
,
∞
). Figure 2 shows the behavior of χ
4
P
,
Q, E
*
P
,
Q, K
P
,
Q, Δ
P
,
Q, and G
P
,
Q. We have considered p
i
=
a
,
1
-
a, q
i
=
1
-
a
,
a, where a
∈
0,1. It is clear from Figure 2 that the divergence χ
4
P
,
Q has a steeper slope than E
*
P
,
Q, K
P
,
Q, Δ
P
,
Q, and G
P
,
Q.Figure 1
Convex functionf
1
t.Figure 2
Comparison of divergences.


## 7. Conclusions

Many research papers have been studied by Taneja, Kumar, Dragomir, Jain, and others, who gave the idea of divergence measures, their properties, their bounds, and relations with other measures. Taneja especially did a lot of quality work in this field: for instance, in [13] he derived bounds on different nonsymmetric divergences in terms of different symmetric divergences, in [14] he introduced new generalized divergences and new divergences as a result of difference of means and characterized their properties and bounds, and in [15] new inequalities among nonnegative differences arising from seven means have been introduced and correlations with generalized triangular discrimination and some new generating measures with their exponential representations have also been presented.Divergence measures have been demonstrated to be very useful in a variety of disciplines such as anthropology, genetics, finance, economics and political science, biology, analysis of contingency tables, approximation of probability distributions, signal processing, pattern recognition, sensor networks, testing of the order in a Markov chain, risk for binary experiments, region segmentation, and estimation.This paper also defines the properties and bounds of Vajda’s divergence and derives new relations with other symmetric and nonsymmetric well-known divergence measures.



---

*Source: 820375-2014-10-29.xml*