Some inequalities relating different measures of divergence between two probability distributions
This note presents new inequalities relating different divergence measures in the family of "convex likelihood-ratio expectation" measures of Csiszar, Ali, and Silvey, and especially in the single-parameter family of "AM-GM" divergence measures. The most prominent result is that theta(2) less than or equal to 1/4J, where theta is the Bhattacharyya angle of divergence (a true distance metric), and J is the symmetric cross-entropy. A pair of "log Gamma" divergences is also introduced and related to the cross-entropies I and J.
bhattacharyya angle of divergence chi-squared statistic cressie-read-anscombe statistic directed divergence hellinger integrals kullback-leibler cross-entropy relative information