Title: On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions
Venue: NeurIPS 2023
Accepted: yes
Keywords: Kullback-Leibler divergence

statistical divergence

multivariate Gaussian distribution

mathematical optimization

Lambert $W$ function

machine learning

flow-based model

reinforcement learning
Abstract: Kullback-Leibler (KL) divergence is one of the most important measures to calculate the difference between probability distributions. In this paper, we theoretically study several properties of KL divergence between multivariate Gaussian distributions. Firstly, for any two $n$-dimensional Gaussian distributions $\mathcal{N}_1$ and $\mathcal{N}_2$, we prove that when $KL(\mathcal{N}_2||\mathcal{N}_1)\leq \varepsilon\ (\varepsilon>0)$ the supremum of $KL(\mathcal{N}_1||\mathcal{N}_2)$ is $(1/2)\left((-W_{0}(-e^{-(1+2\varepsilon)}))^{-1}+\log(-W_{0}(-e^{-(1+2\varepsilon)})) -1 \right)$, where $W_0$ is the principal branch of Lambert $W$ function.	For small $\varepsilon$, the supremum is $\varepsilon + 2\varepsilon^{1.5} + O(\varepsilon^2)$. This quantifies the approximate symmetry of small KL divergence between Gaussian distributions. We further derive the infimum of $KL(\mathcal{N}_1||\mathcal{N}_2)$ when $KL(\mathcal{N}_2||\mathcal{N}_1)\geq M\ (M>0)$. We give the conditions when the supremum and infimum can be attained. Secondly, for any three $n$-dimensional Gaussian distributions $\mathcal{N}_1$, $\mathcal{N}_2$, and $\mathcal{N}_3$, we theoretically show that an upper bound of $KL(\mathcal{N}_1||\mathcal{N}_3)$ is $3\varepsilon_1+3\varepsilon_2+2\sqrt{\varepsilon_1\varepsilon_2}+o(\varepsilon_1)+o(\varepsilon_2)$ when $KL(\mathcal{N}_1||\mathcal{N}_2)\leq \varepsilon_1$ and $KL(\mathcal{N}_2||\mathcal{N}_3)\leq \varepsilon_2$ ($\varepsilon_1,\varepsilon_2\ge 0$). This reveals that KL divergence between Gaussian distributions follows a relaxed triangle inequality. Note that, all these bounds in the theorems presented in this work are independent of the dimension $n$. Finally, we discuss several applications of our theories in deep learning, reinforcement learning, and sample complexity research.

On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions

Abstract

Kullback-Leibler (KL) divergence is one of the most important measures to calculate the difference between probability distributions. In this paper, we theoretically study several properties of KL divergence between multivariate Gaussian distributions. Firstly, for any two 𝑛 -dimensional Gaussian distributions N 1 and N 2 , we prove that when 𝐾𝐿 (N 2 | |N 1 ) ≤ 𝜀 ( 𝜀 > 0 ) the supremum of 𝐾𝐿 (N 1 | |N 2 ) is ( 1 / 2 ) GLYPH<0> (-𝑊 0 (-𝑒 -( 1 + 2 𝜀 ) )) -1 + log (-𝑊 0 (-𝑒 -( 1 + 2 𝜀 ) )) -1 GLYPH<1> , where 𝑊 0 is the principal branch of Lambert 𝑊 function. For small 𝜀 , the supremum is 𝜀 + 2 𝜀 1 . 5 + 𝑂 ( 𝜀 2 ) . This quantifies the approximate symmetry of small KL divergence between Gaussian distributions. We further derive the infimum of 𝐾𝐿 (N 1 | |N 2 ) when 𝐾𝐿 (N 2 | |N 1 ) ≥ 𝑀 ( 𝑀 > 0 ) . We give the conditions when the supremum and infimum can be attained. Secondly, for any three 𝑛 -dimensional Gaussian distributions N 1 , N 2 , and N 3 , we theoretically show that an upper bound of 𝐾𝐿 (N 1 | |N 3 ) is 3 𝜀 1 + 3 𝜀 2 + 2 √ 𝜀 1 𝜀 2 + 𝑜 ( 𝜀 1 ) + 𝑜 ( 𝜀 2 ) when 𝐾𝐿 (N 1 | |N 2 ) ≤ 𝜀 1 and 𝐾𝐿 (N 2 | |N 3 ) ≤ 𝜀 2 ( 𝜀 1 , 𝜀 2 ≥ 0 ). This reveals that KL divergence between Gaussian distributions follows a relaxed triangle inequality. Note that, all these bounds in the theorems presented in this work are independent of the dimension 𝑛 . Finally, we discuss several applications of our theories in deep learning, reinforcement learning, and sample complexity research.

1 Introduction

A statistical divergence measures the 'distance' between probability distributions. Let 𝑋 be a space of probability distributions with the same support. A statistical divergence 𝐷 : 𝑋 × 𝑋 → R + ( R +

is the set of non-negative real numbers) should satisfy (a) non-negativity: 𝐷 ( 𝑝, 𝑞 ) ≥ 0 and (b) identity of indiscernibles: 𝐷 ( 𝑝, 𝑝 ) = 0 , where 𝑝, 𝑞 are probability densities. Another stricter concept, statistical distance, also measures the distance between probability distributions. A statistical distance should satisfy two additional properties, including (c) symmetry: 𝐷 ( 𝑝, 𝑞 ) = 𝐷 ( 𝑞, 𝑝 ) and (d) triangle inequality: 𝐷 ( 𝑝, 𝑞 ) ≤ 𝐷 ( 𝑝, 𝑔 ) + 𝐷 ( 𝑔, 𝑞 ) , where 𝑝, 𝑞 and 𝑔 are probability densities.

Kullback-Leibler (KL) divergence, also referred to as relative entropy [31], is applied broadly in many fields, such as machine learning [10, 23], information theory [15], and statistics [45]. The KL divergence between two continuous probability densities 𝑝 ( 𝑥 ) and 𝑞 ( 𝑥 ) is defined as 𝐾𝐿 ( 𝑝 ( 𝑥 )|| 𝑞 ( 𝑥 )) = ∫ 𝑝 ( 𝑥 ) log 𝑝 ( 𝑥 ) 𝑞 ( 𝑥 ) d 𝑥 . KL divergence is not a proper distance [31]. First, KL divergence is not symmetric. It might happen that the forward KL divergence 2 𝐾𝐿 ( 𝑝 | | 𝑞 ) is very small but the reverse KL divergence 𝐾𝐿 ∗ ( 𝑝 | | 𝑞 ) = 𝐾𝐿 ( 𝑞 | | 𝑝 ) is very large. Second, KL divergence does not satisfy the triangle inequality. This hinders the application of KL divergence in many contexts.

KL divergence also has connections with other information measures. For example, by taking the second-order Taylor expansion, KL divergence can be approximated with fisher information matrix [31]. Therefore, KL divergence is locally approximately symmetric when two distributions are close to each other.

Meanwhile, Gaussian distribution is one of the most important distributions and is central to statistics. It is also pervasive in many fields. The probability density function of an 𝑛 -dimensional Gaussian distribution is N( µ , 𝚺 ) = (( 2 𝜋 ) 𝑛 / 2 | 𝚺 | 1 / 2 ) -1 exp GLYPH<0> -( 1 / 2 )( x -µ ) ⊤ 𝚺 -1 ( x -µ ) GLYPH<1> . Here µ ∈ R 𝑛 is the mean and 𝚺 ∈ S 𝑛 ++ is the covariance matrix, where S 𝑛 ++ is the space of symmetric positive definite 𝑛 × 𝑛 matrices. Gaussian distribution is the basis for more complicated distributions such as Gaussian Mixture Model [10]. In this paper, we refer to Gaussian distribution as Gaussian for brevity.

The KL divergence between two 𝑛 -dimensional Gaussians N 1 , N 2 has the following closed form [45]

where the logarithm is taken to base 𝑒 and Tr is the trace of matrix. Like many other distributions, KL divergence between Gaussians is neither symmetric nor satisfies the triangle inequality.

In this work, we investigate the following two research problems, which are motivated by our research on out-of-distribution detection with flow-based model [57].

How to quantify the relation between forward and reverse KL divergences between Gaussian distributions?

Does the KL divergence between Gaussian distributions satisfy some property similar to triangle inequality?

Contributions. The contributions of this work are as follows. Let N 𝑖 ( µ 𝑖 , 𝚺 𝑖 ) ( 𝑖 ∈ { 1 , 2 , 3 }) be any three 𝑛 -dimensional Gaussians.

We prove that when 𝐾𝐿 (N 1 | |N 2 ) ≤ 𝜀 ( 𝜀 ≥ 0 ) the supremum of 𝐾𝐿 (N 2 | |N 1 ) is ( 1 / 2 ) GLYPH<0> (-𝑊 0 (-𝑒 -( 1 + 2 𝜀 ) )) -1 + log (-𝑊 0 (-𝑒 -( 1 + 2 𝜀 ) )) -1 GLYPH<1> , where 𝑊 0 is the principal branch of Lambert 𝑊 function. We give the conditions when the supremum can be attained. For small 𝜀 , the supremum is 𝜀 + 2 𝜀 1 . 5 + 𝑂 ( 𝜀 2 ) . This quantifies the approximate symmetry of small KL divergence between Gaussians.

We find the infimum of 𝐾𝐿 (N 1 | |N 2 ) if 𝐾𝐿 (N 2 | |N 1 ) ≥ 𝑀 ( 𝑀 > 0 ) . We also give the conditions when the infimum can be attained.

We find an upper bound of 𝐾𝐿 (N 1 | |N 3 ) if 𝐾𝐿 (N 1 | |N 2 ) ≤ 𝜀 1 and 𝐾𝐿 (N 2 | |N 3 ) ≤ 𝜀 2 for 𝜀 1 , 𝜀 2 ≥ 0 . For small 𝜀 1 and 𝜀 2 , the upper bound is 3 𝜀 1 + 3 𝜀 2 + 2 √ 𝜀 1 𝜀 2 + 𝑜 ( 𝜀 1 ) + 𝑜 ( 𝜀 2 ) . This indicates that KL divergence between Gaussians follows a relaxed triangle inequality.

All the bounds in our theorems are independent of the dimension 𝑛 . This is a critical property in contexts where dimensionality has a fundamental impact.

The theorems proved in this paper can extend the applications of KL divergence in many contexts. We discuss the motivation application in out-of-distribution detection with flowbased model and multiple applications in reinforcement learning and sample complexity research.

The rest part of this paper is organized as follows. In Section 2 we discuss related work. In Section 3 we prepare lemmas and notations. In Section 4 we investigate the supremum (infimum) of reverse KL divergence between Gaussians when forward KL divergence is bounded. In Section 5 we investigate the relaxed triangle inequality of KL divergence between Gaussians. In Section 6, we discuss applications. Finally, we conclude in Section 7. We put long proofs in Appendix.

2 Related work

KL divergence has a wide range of applications [15, 10, 23, 45, 20, 25, 3]. Researchers have investigated KL divergence between many distributions, including Markov sources [47], GMM models [18, 27], multivariate generalized Gaussians [11], discrete normal distributions [42], etc . In [45], a bound of KL divergence between Gaussians is given. As far as we know, no related work focuses on the similar properties of KL divergence between Gaussians as in this paper.

KL divergence is one member of more general divergences such as Bregman divergence [12, 7, 21, 53], 𝑓 -divergence [5, 45, 4], Rényi divergence [48, 45, 54], and ( 𝑓 , Γ ) -divergence [9]. Bregman divergence defines a class of divergences [7] in vector space. KL divergence between multinomial distributions is a special form of Bregman divergence when the convex function for Bregman divergence is chosen as ˝ 𝑛 𝑖 = 1 𝑝 𝑖 log 𝑝 𝑖 , where 𝑝 𝑖 ≥ 0 define a multinomial distribution. Frigyik et. al. [21] extends vector Bregman divergence to functional Bregman divergence in 𝐿 𝑝 . Similarly, KL divergence is a special form of functional Bregman divergene. (Functional) Bregman divergence also satisfies generalized Pythagoras theorem [7, 21].

The asymmetry of KL divergence has restricted the application of KL divergence in practical applications. Many other divergences have been investigated [46, 16, 2, 43, 26, 17, 7, 22, 56, 55]. Pardo gives a comprehensive survey on a wide range of statistical divergences in his book [45].

3 Lemmas and Notations

We introduce the following Lambert 𝑊 function before our theoretical results.

Definition 1 Lambert 𝑊 Function[32, 14] . The inverse function of function 𝑦 = 𝑥𝑒 𝑥 is called Lambert 𝑊 function 𝑦 = 𝑊 ( 𝑥 ) .

When 𝑥 ∈ R , 𝑊 is a multivalued function with two branches 𝑊 0 , 𝑊 -1 , where 𝑊 0 is the principal branch (also called branch 0) and 𝑊 -1 is the branch -1 . Figure A.1 in Appendix A shows the graph of 𝑊 0 and 𝑊 -1 .

Lemmas . Function 𝑓 ( 𝑥 ) = 𝑥 -log 𝑥 ( 𝑥 ∈ R ++ ) ( R ++ is the set of positive real numbers) lies in the core of our problems. In Lemma B.1 in Appendix B, we prove several properties of 𝑓 ( 𝑥 ) . We show that the inverse function of 𝑓 is 𝑓 -1 ( 𝑥 ) = -𝑊 (-𝑒 -𝑥 ) ( 𝑥 ≥ 1 ) . Equation 𝑓 ( 𝑥 ) = 𝑥 -log 𝑥 = 1 + 𝑡 ( 𝑡 ≥ 0 ) has two solutions 𝑤 1 ( 𝑡 ) = -𝑊 0 (-𝑒 -( 1 + 𝑡 ) ) ∈ ( 0 , 1 ] and 𝑤 2 ( 𝑡 ) = -𝑊 -1 (-𝑒 -( 1 + 𝑡 ) ) ∈ [ 1 , +∞) . We treat 𝑤 1 ( 𝑡 ) , 𝑤 2 ( 𝑡 ) as functions of 𝑡 .

Table 1 summarizes some notations used in this paper. Please see Table A.1 in Appendix A for a full list of notations.

4 Bounds of Forward and Reverse KL Divergence Between Gaussians

In this section, we investigate the relation between forward and reverse KL divergences between Gaussians. These conclusions quantify the approximate symmetry of small KL divergence between Gaussians.

4.1 Supremum of Reverse KL Divergence Between Gaussians

The following Theorem 1 gives the supremum of reverse KL divergence when forward KL divergence is bounded by a number 𝜀 .

Theorem 1 For any two 𝑛 -dimensional Gaussian distributions N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) , if 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( µ 2 , 𝚺 2 )) ≤ 𝜀 ( 𝜀 ≥ 0 ) , then

The supremum is attained when the following two conditions hold.

There exists only one eigenvalue 𝜆 𝑗 of 𝐵 -1 2 𝚺 1 ( 𝐵 -1 2 ) ⊤ or 𝐵 -1 1 𝚺 2 ( 𝐵 -1 1 ) ⊤ equal to -𝑊 0 (-𝑒 -( 1 + 2 𝜀 ) ) and all other eigenvalues 𝜆 𝑖 ( 𝑖 ≠ 𝑗 ) are equal to 1 , where 𝐵 1 = 𝑃 1 𝐷 1 / 2 1 , 𝑃 1 is an orthogonal matrix whose columns are the eigenvectors of 𝚺 1 , 𝐷 1 = 𝑑𝑖𝑎𝑔 ( 𝜆 1 , . . . , 𝜆 𝑛 ) whose diagonal elements are the corresponding eigenvalues, 𝐵 2 is defined in the similar way as 𝐵 1 except on 𝚺 2 .

µ 1 = µ 2 .

Overview of Proof of Theorem 1 . Please see Appendix D for the full proof. Here we give the overview of the proof. Theorem 1 can be proved by solving the following optimization problem P 1 analytically.

The proof consists of the following several steps.

Invertible linear transformation . We apply a linear transformation to N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) such that one of them is transformed to a standard Gaussian. In this way, we can simplify the objective function and the constraint in P 1 . Note that diffeomorphism preserves KL divergence [41] (see Proposition D.1 in Appendix D). In the end, we use the inverse linear transformation to transform N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) back.

Reducing to new optimization problem . We reduce P 1 to the following core optimization problem P 2 .

where ¯ f ( 𝑥 1 , . . . , 𝑥 𝑛 ) = ˝ 𝑛 𝑖 = 1 𝑓 ( 𝑥 𝑖 ) = ˝ 𝑛 𝑖 = 1 𝑥 𝑖 -log 𝑥 𝑖 ( 𝑥 𝑖 ∈ ( 0 , ∞)) .

Investigating 𝑓 ( 𝑥 ) . 𝑓 ( 𝑥 ) lies in the core of the problem. We have proven several properties of 𝑓 ( 𝑥 ) in Section 3. A fundamental property of 𝑓 ( 𝑥 ) (stated in Lemma B.1a) is that 𝑓 ( 𝑥 ) is strictly convex and takes its minimum value 1 at 𝑥 = 1 . These lemmas allow us to conduct further analysis in other parts of this paper.

Allocating 𝜀 ′ . In problem P 2 , the supremum of ¯ f ( 1 𝑥 1 , . . . , 1 𝑥 𝑛 ) is affected by the domain of each dimension, which is in turn determined by how 𝜀 ′ is allocated to these dimensions. We call ( 𝜀 1 , · · · , 𝜀 𝑛 ) where ˝ 𝑛 1 𝜀 𝑖 = 𝜀 ′ as an allocation . We prove that ¯ f ( 1 𝑥 1 , . . . , 1 𝑥 𝑛 ) takes its maximum when 𝜀 ′ is allocated to only one dimension ( i.e. , an 'extreme' allocation). In other words, there exists one 𝜀 𝑗 = 𝜀 ′ and 𝜀 𝑖 = 0 for all 𝑖 ≠ 𝑗 . The key is to prove the convexity of function Δ ( 𝜀 ) = 𝑓 ( 1 𝑤 1 ( 𝜀 ) ) -𝑓 ( 𝑤 1 ( 𝜀 )) , where 𝑤 1 ( 𝜀 ) = -𝑊 0 (-𝑒 -( 1 + 𝜀 ) ) .

In summary, we use a linear transformation to simplify problem P 1 into standard Gaussian case. Then we deal with the simplified problem in the following Lemma 1, which accomplishes the above steps 2 ∼ 4. Theorem 1 can be seen as the generalization of Lemma 1.

Lemma 1 Let N( 0 , 𝐼 ) be 𝑛 -dimensional standard Gaussian distribution, 𝜀 be a positive number. For any 𝑛 -dimensional Gaussian distribution N( µ , 𝚺 ) ,

□

To further investigate the bound in Theorem 1, we expand the Lambert 𝑊 function using the series presented in [19, 14] for small 𝜀 . This result is stated in the following Theorem 2, which can help users to apply our theorem conveniently.

Theorem 2 For any two 𝑛 -dimensional Gaussian distributions N( µ 1 , 𝚺 1 ) , N( µ 2 , 𝚺 2 ) , and a small positive number 𝜀 , if 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( µ 2 , 𝚺 2 )) ≤ 𝜀 , then

Proof 2 Please see Appendix E for details.

□

Theorem 1 holds for any two Gaussians N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) . According to the proof of Theorem 1 (Lemma 1), one of N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) can be fixed. Thus, it is not hard to extend Lemma 1 to the case where the fixed Gaussian is not standard. We can apply linear transformation on the fixed Gaussian (see Equation (D.46)) as what we have done in the main proof of Theorem 1 (see Appendix D). Other parts of the proof are the same. Therefore, we obtain the following corollary.

Corollary 1 Theorem 1 and Theorem 2 hold when one of N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) is fixed.

Remark 1 The supremum in Theorem 1 is small (zero) when 𝜀 is small (zero). Figure A.2 in Appendix A shows the graph of the supremum of KL divergence. Due to the strict conditions, it is hard to reach the supremum in typical applications (e.g. in machine learning). Notably, the bound is independent of the dimension 𝑛 . This is critical in high-dimensional problems (see motivation application in Section 6). We also note that the condition 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( µ 2 , 𝚺 2 )) ≤ 𝜀 in Theorem 1 is strict in high-dimensional problems.

Remark 2 Theorem 1 gives the strict conditions when the supremum can be attained. We can benefit from these strict conditions in applications. When the forward KL divergence is small, we want a guarantee that the reverse KL divergence is also small such that bounding forward KL divergence also bounds the reverse KL divergence. Theorem 1 has the following two meanings.

The supremum of reverse KL divergence 𝜀 + 2 𝜀 1 . 5 + 𝑂 ( 𝜀 2 ) is small, implying that the worst case is acceptable.

The strict conditions for reaching the supremum imply that the worst case barely happens in practice. When these strict conditions do not hold, the reverse KL divergence is smaller than the supremum, which is what we want in practice.

Toy Examples .

Figure 1 shows some toy examples in one dimensional case. The black line represents standard Gaussian distribution N 0 ( 0 , 1 ) . All other four Gaussian distributions N 𝑖 ( 1 ≤ 𝑖 ≤ 4 , in colored lines) have the same forward KL divergence 𝐾𝐿 (N 𝑖 | |N 0 ) = 0 . 01 . The second distribution has the maximized reverse KL divergence 𝐾𝐿 (N 0 | |N 1 ( 0 , 0 . 90173 2 )) ≈ 0 . 01148 , which is equal to the supremum 1 2 GLYPH<16> 1 -𝑊 0 (-𝑒 -( 1 + 2 × 0 . 01 ) ) -log 1 -𝑊 0 (-𝑒 -( 1 + 2 × 0 . 01 ) ) -1 GLYPH<17> . Other reverse KL divergences are 𝐾𝐿 (N 0 | |N 2 ( 0 , 1 . 10161 2 )) ≈ 0 . 00879 , 𝐾𝐿 (N 0 | |N 3 ( 0 . 14143 , 1 )) ≈ 0 . 01 , 𝐾𝐿 (N 0 | |N 4 ( 0 . 1 , 1 . 07153 2 )) ≈ 0 . 00892 .

μ=0,α=1

μ=0,0=0.90173

0.4

μ=0,0=1.10161

μ=0.14143,0=1

μ=0.1,0=1.07153

0.3

0.2

0.1

0.0

-3

-2

-1

0

1

2

3

4.2 Infimum of Reverse KL Divergence Between Gaussians

In this subsection, we give the infimum of 𝐾𝐿 (N 2 | |N 1 ) when 𝐾𝐿 (N 1 | |N 2 ) ≥ 𝑀 ( 𝑀 > 0 ). The result is presented in Theorem 3.

Theorem 3 For any two 𝑛 -dimensional Gaussian distributionss N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) , if 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( µ 2 , 𝚺 2 )) ≥ 𝑀 ( 𝑀 > 0 ) , then

The infimum is attained when the following two conditions hold.

There exists only one eigenvalue 𝜆 𝑗 of 𝐵 -1 2 𝚺 1 ( 𝐵 -1 2 ) ⊤ or 𝐵 -1 1 𝚺 2 ( 𝐵 -1 1 ) ⊤ equal to -𝑊 -1 (-𝑒 -( 1 + 2 𝑀 ) ) and all other eigenvalues 𝜆 𝑖 ( 𝑖 ≠ 𝑗 ) are equal to 1 , where 𝐵 1 = 𝑃 1 𝐷 1 / 2 1 , 𝑃 1 is an orthogonal matrix whose columns are the eigenvectors of 𝚺 1 , 𝐷 1 = 𝑑𝑖𝑎𝑔 ( 𝜆 1 , . . . , 𝜆 𝑛 ) whose diagonal elements are the corresponding eigenvalues, 𝐵 2 is defined in the similar way as 𝐵 1 except on 𝚺 2 .

µ 1 = µ 2 .

Proof 3 Theorem 1 and Theorem 3 form a duality. These two theorems can be proved independently in a similar way and also be derived from each other. We give two proofs of Theorem 3 in Appendix F. The first proof presented in Appendix F .1 has the similar structure as that of Theorem 1, except that Theorem 3 uses 𝑊 -1 . The second proof shown in Appendix F.2 derives Theorem 3 from Theorem 1.These two proofs can verify each other. □

5 Relaxed Triangle Inequality

In this section, we give a dimension-free bound of 𝐾𝐿 (N 1 | |N 3 ) when 𝐾𝐿 (N 1 | |N 2 ) and 𝐾𝐿 (N 2 | |N 3 ) are bounded for any three Gaussians N 1 , N 2 , and N 3 . Proving the relaxed triangle inequality is much more difficult. The main result is presented in Theorem 4 and 5. We use two key Lemmas G.5 and 2 to accomplish the key steps of the proof.

Theorem 4 For any three 𝑛 -dimensional Gaussians N( µ 𝑖 , 𝚺 𝑖 )( 𝑖 ∈ { 1 , 2 , 3 }) such that 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( µ 2 , 𝚺 2 )) ≤ 𝜀 1 and 𝐾𝐿 (N( µ 2 , 𝚺 2 )||N ( µ 3 , 𝚺 3 )) ≤ 𝜀 2 for 𝜀 1 , 𝜀 2 ≥ 0 ,

‹

Overview of Proof of Theorem 4

The proof of Theorem 4 is the most technical part of this paper. Please see Appendix I for details. Here we give the overview of the proof.

We want to solve the following optimization problem P 3 analytically.

Unfortunately, it is hard to find the supremum due to the complexity caused by Lambert 𝑊 function. So we relax the constraints to simplify the problem. The proof mainly consists of the following four steps.

Invertible linear transformation . The first step is similar to that of Theorem 1. We apply a linear transformation on N 1 , N 2 , and N 3 simultaneously such that N 2 is converted to standard Gaussian. The transformed problem will be proved in Lemma 2 (shown below). In the end, these three Gaussians are transformed back to general case.

Relaxing constraints . In the proof of Lemma 2, we relax the constraints and finally reduce to the following core problem P 4 .

where 𝜆 1 , [ 𝑖 ] and 𝜆 ′ 2 , [ 𝑖 ] are the eigenvalues of 𝚺 1 and 𝚺 -1 2 arranged in decreasing order, respectively. 𝜀 1 , [ 𝑖 ] and 𝜀 2 , [ 𝑖 ] are arranged in decreasing order too.

Allocating 2 𝜀 1 and 2 𝜀 2 . The value of objective function in P 4 is determined by 𝜆 1 , [ 𝑖 ] and 𝜆 ′ 2 , [ 𝑖 ] , which are in turn determined by how 2 𝜀 1 and 2 𝜀 2 are allocated to ( 𝜀 1 , [ 1 ] , · · · , 𝜀 1 , [ 𝑛 ] ) and ( 𝜀 2 , [ 1 ] , · · · , 𝜀 2 , [ 𝑛 ] ) , respectively. We prove that an 'extreme allocation' can maximize the objective function. In other words, the objective function in Equation (6) takes its maximum when 𝜀 1 , [ 1 ] = 2 𝜀 1 , 𝜀 2 , [ 1 ] = 2 𝜀 2 , and 𝜀 1 , [ 𝑖 ] = 𝜀 2 , [ 𝑖 ] = 0 for 1 < 𝑖 ≤ 𝑛 . In the proof, we use a key Lemma G.5 to deal with the 2-dimensional case ( 𝑛 = 2 ). Finally, we extend to arbitrary dimensional cases.

Dealing with 2-dimensional case. The proof of Lemma G.5 is the most technical part in this work. In the proof, concentrating 𝜀 1 and 𝜀 2 is much harder than that in the last section for Theorem 1. 𝑓 ( 𝑥 ) = 𝑥 -log 𝑥 is a transcendental function whose inverse function is expressed by Lambert 𝑊 function. This makes even a 2-dimensional case of problem P 4 hard to solve. Our proof of Lemma G.5 is like coordinate descent but much difficult. We show that, for any fixed 'non-extreme allocation' of 2 𝜀 1 ( i.e. , ( 𝜀 1 , [ 1 ] , 𝜀 1 , [ 2 ] ) where 𝜀 1 , [ 2 ] > 0 ), there exists

a 'more extreme' allocation of 2 𝜀 2 ( i.e. , ( 𝜀 2 , [ 1 ] , 𝜀 2 , [ 2 ] ) where 𝜀 2 , [ 2 ] > 0 ) that maximizes the objective function. Note that it is hard to state that ( 𝜀 2 , [ 1 ] , 𝜀 2 , [ 2 ] ) is 'more extreme' than ( 𝜀 1 , [ 1 ] , 𝜀 1 , [ 2 ] ) because 𝜀 1 ≠ 𝜀 2 . Then we fix ( 𝜀 2 , [ 1 ] , 𝜀 2 , [ 2 ] ) and find a 'more extreme' allocation of 2 𝜀 1 ( 𝜀 ′ 1 , [ 1 ] , 𝜀 ′ 1 , [ 2 ] ) to lift the objective function further. Using these iterations, we can construct an infinite sequence of allocations whose limitation is an 'extreme' one ( i.e. , 𝜀 1 , [ 1 ] = 2 𝜀 1 , 𝜀 2 , [ 1 ] = 2 𝜀 2 ). Then we prove the extreme allocation can make the objective function reach its supremum. In Appendix G we present the key Lemma G.5 and its proof. We also give its proof sketch before the long proof. Essentially, Lemma G.5 plays the most vital role in eliminating the dimension 𝑛 in case of 𝑛 = 2 from the bound in 2-dimensional case. Finally, we will make the bound in high-dimensional problem dimension-free as well.

In summary, we apply a linear transformation to simplify the problem. Then we solve the simplified problem in the following Lemma 2, accomplishing the above steps 2 ∼ 4. Theorem 4 can be seen as the generalization of Lemma 2.

Lemma 2 For any two 𝑛 -dimensional Gaussian distributions N( µ 1 , 𝚺 1 ) and N( µ 2 , 𝚺 2 ) such that 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( 0 , 𝐼 )) ≤ 𝜀 1 , 𝐾𝐿 (N( 0 , 𝐼 )||N ( µ 2 , 𝚺 2 )) ≤ 𝜀 2 ( 𝜀 1 , 𝜀 2 ≥ 0 ) ,

Proof 4 Please see Appendix H for details.

Remark 3 The bound in Lemma 2 is dimension-free and becomes 0 when 𝜀 1 = 𝜀 2 = 0 .

Similarly, we can expand Lambert 𝑊 function by series [19, 14] and simplify the bound in Theorem 4 as follows [36].

Theorem 5 For any three 𝑛 -dimensional Gaussian distributions N( µ 𝑖 , 𝚺 𝑖 )( 𝑖 ∈ { 1 , 2 , 3 }) such that 𝐾𝐿 (N( µ 1 , 𝚺 1 )||N ( µ 2 , 𝚺 2 )) ≤ 𝜀 1 and 𝐾𝐿 (N( µ 2 , 𝚺 2 )||N ( µ 3 , 𝚺 3 )) ≤ 𝜀 2 for small 𝜀 1 , 𝜀 2 ≥ 0 ,

Proof 5 Please see Appendix J for details.

□

Finally, in the proof of Theorem 4, we use invertible linear transformation to convert N 2 to standard Gaussian while preserving KL divergence. This proof also applies to the case when N( µ 2 , 𝚺 2 ) is fixed. Therefore, we obtain the following corollary.

Corollary 2 Theorem 4 and 5 hold when N( µ 2 , 𝚺 2 ) is fixed.

Remark 4 Our theorem is different from existing generalized Pythagoras inequalities satisfied by KL divergence. Please see Appendix K.1 for more discussion.

6 Discussion and Applications

In our theorems, we allow all parameters are unknown or one Gaussian is fixed. Therefore, our theorems are suitable for problems where the parameters can vary. This is common in deep learning where the parameters are learned from data. In this section, we discuss motivation application and other applications ranging from anomaly detection to reinforcement learning to sample complexity research.

6.1 Motivation Application: Anomaly Detection with Flow-based Model

The research problems in this paper are motivated by our research on deep anomaly detection using flow-based model [57] 3 . Flow-based model assigns higher likelihoods to Out-of-Distribution (OOD) data than In-Distribution (ID) data ( i.e. , training data) [39, 51, 13, 52, 40, 30]. For example, Glow [29] assigns higher likelihoods for SVHN when trained on CIFAR-10. Furthermore, we cannot sample OOD data from the model. Existing explanation is based on the discrepancy of typical set and high probability density regions of model distribution [40]. Such typicality-based explanation and OOD detection method fail when OOD data has coinciding likelihoods with ID data [40].

We focus on two research problems in this context. (1) why we cannot sample OOD data from flowbased model with prior regardless of when OOD data have higher, lower, or coinciding likelihoods? (2) How to detect OOD data using flow-based model? We investigate these problems from a statistical divergence perspective. Let 𝑧 = 𝑓 ( 𝑥 ) be the flow-based model mapping data 𝑥 in data space to representation 𝑧 in latent space. Suppose the prior distribution 𝑝 𝑟 𝑍 is the most commonly used Gaussian distribution. Let 𝑋 1 ∼ 𝑝 𝑋 ( 𝑥 ) , 𝑋 2 ∼ 𝑞 𝑋 ( 𝑥 ) be the distributions of ID and OOD datasets, respectively. 𝑍 1 = 𝑓 ( 𝑋 1 ) ∼ 𝑝 𝑍 ( 𝑧 ) , 𝑍 2 = 𝑓 ( 𝑋 2 ) ∼ 𝑞 𝑍 ( 𝑧 ) be the distributions of representations of ID and OOD datasets, respectively. Let 𝑝 𝑟 𝑋 be the model induced distribution such that 𝑍 𝑟 ∼ 𝑝 𝑟 𝑍 and 𝑋 𝑟 = 𝑓 -1 ( 𝑍 𝑟 ) ∼ 𝑝 𝑟 𝑋 . Flow-based model is usually trained by maximum likelihood estimation, which is equal to minimizing forward KL divergence 𝐾𝐿 ( 𝑝 𝑋 | | 𝑝 𝑟 𝑋 ) [44, 23]. We conduct generalized Shapiro-Wilk test for multivariate normality on representations. As shown in the original Table C.3 in [57] (also in supplementary material), 𝑝 𝑍 is Gaussian-like for all datasets. 𝑞 𝑍 is also Gaussian-like for OOD datasets with higher or coinciding likelihoods except for just one case. These results allow us to approximate 𝑝 𝑍 and 𝑞 𝑍 with Gaussians when possible.

The theorems proved in this paper provide solid theoretical guarantee for our analysis and algorithm. On one hand, flow-based model preserves KL divergence (see Proposition D.1 in Appendix D), so 𝐾𝐿 ( 𝑝 𝑍 | | 𝑝 𝑟 𝑍 ) which equals 𝐾𝐿 ( 𝑝 𝑋 | | 𝑝 𝑟 𝑋 ) is minimized. According to the approximate symmetry of small KL divergence (Theorem 1), we can know 𝐾𝐿 ( 𝑝 𝑟 𝑍 | | 𝑝 𝑍 ) is small too. So we can assume 𝑝 𝑟 𝑍 ≈ 𝑝 𝑍 when 𝐾𝐿 ( 𝑝 𝑍 | | 𝑝 𝑟 𝑍 ) is sufficiently small. On the other hand, we can also assume that the distributions of ID and OOD data are far from each other. This implies that 𝐾𝐿 ( 𝑝 𝑋 | | 𝑞 𝑋 ) equaling 𝐾𝐿 ( 𝑝 𝑍 | | 𝑞 𝑍 ) can be any large. This implies 𝐾𝐿 ( 𝑝 𝑟 𝑍 | | 𝑞 𝑍 ) ≈ 𝐾𝐿 ( 𝑝 𝑍 | | 𝑞 𝑍 ) is large too. Specially, when 𝑞 𝑍 is Gaussian-like, we can apply the relaxed triangle inequality (Theorem 4) and infer that 𝐾𝐿 ( 𝑝 𝑟 𝑍 | | 𝑞 𝑍 ) must be large. Note that, when 𝑞 𝑍 is not Gaussian-like, we can still apply the theorems presented in this paper to perform analysis on the lower bound of KL divergence (see original Theorem 5 in [57]). Overall, the large KL divergence between 𝑝 𝑟 𝑍 and 𝑞 𝑍 reveals why we cannot sample OOD data from flow-based model with prior. It is also notable that flow-based model constructs diffeomorphism between data and latent space with thousands of dimensions. Thus, it is critical that the bounds found in this paper are dimension-free. Furthermore, we decompose the KL divergence further into group-wise KL divergence and mutual information. Based on these analysis, we propose an unified OOD detection algorithm KLODS both for group (GAD) and point-wise (PAD) anomaly detection. We conduct extensive experiments to compare our method with 13 baseline methods including 𝑡 -test, KS-test, MMD [24], KSD [35], Annulus Method [13], typicality test [40], the state-of-the-art (SOTA) GAD method GOD2KS [28], input complexity compensated likelihood [50], last-scale likelihood [49], ODIN [34], Joint confidence loss [33], and DoSE [37]. Experimental results demonstrate the superiority of our method. For example, as shown in Table 2, our method outperforms the SOTA group-wise anomaly detection method GOD2KS on flow-based model by 9.1% AUROC. Our method also outperforms the SOTA point-wise anomaly detection method DoSE with Glow by 5.2% AUROC. More details of the algorithm and experimental results on both group and point-wise anomaly detection can be refered to [57].

6.2 Applications of Approximate Symmetry of Small KL divergence

Theorem 1 can be also applied widely in deep reinforcement learning and sample complexity research. In many contexts, researchers are hindered by the asymmetry of KL divergence. Theorem 1 on the approximate symmetry of KL divergence between Gaussians brings the following convenience to us.

Minimizing one of forward and reverse KL divergences also bounds another.

We can exchange forward and reverse KL divergences for small 𝜀 .

Wesummarize the applications of Theorem 1 briefly in the following. The details of these applications are discussed in Appendix L.1.

Providing Theoretical Guarantee for Continuous Gaussian Policy in Reinforcement Learning . In [38], Nair et al. propose AWAC method to accelerate online reinforcement learning with offline datasets. They obtain theoretical guarantee in offline reinforcement learning for discrete policies. Theorem 1 can extend their guarantee to continuous Gaussian policy.

Bringing New Insights to Existing Reinforcement Learning Algorithm . In [1], Abdolmaleki et al. propose the MPO algorithm for reinforcement learning. They use Expectation-Maximization (EM) to solve control problems and use constraints on KL terms in both E and M-steps. Theorem 1 can eliminate such a difference for continuous Gaussian policies.

Bridging Research on Sample Complexity of Learning Gaussian Distribution . Theorem 1 can bridge existing research on sample complexity of Gaussian distribution. Researchers have proposed algorithms for learning a multivariate Gaussian distribution with error bounds in forward and reverse KL divergence separately so far [6, 8]. Theorem 1 can eliminate the difference between forward and reverse KL divergence in this scenario.

6.3 Application of Relaxed Triangle Inequality

Extending One-step Safety Guarantee to Multiple Steps in Reinforcement Learning . The relaxed triangle inequality (Theorem 5) has been applied in safe reinforcement learning. Liu et al. propose an Expectation-Maximization style approach for learning safe policy in reinforcement learning [36]. They utilize our relaxed triangle inequality to extend one-step robustness guarantee to multiple steps. In the original Proposition 4 in [36], they simplify the bound in Theorem 4 in case 𝜀 1 = 𝜀 2 . Please see Appendix L.2 for details.

7 Conclusion

In this paper, we research the properties of KL divergence between Gaussians. First, we find the supremum of reverse KL divergence 𝐾𝐿 (N 2 | |N 1 ) if the forward KL divergence 𝐾𝐿 (N 1 | |N 2 ) ≤ 𝜀 ( 𝜀 > 0 ). This conclusion quantifies the approximate symmetry of small KL divergence between Gaussians. We also find the infimum of 𝐾𝐿 (N 2 | |N 1 ) if 𝐾𝐿 (N 1 | |N 2 ) ≥ 𝑀 ( 𝑀 > 0 ). We give the conditions when the supremum and infimum can be attained. Second, we find a bound for 𝐾𝐿 (N 1 | |N 3 ) when 𝐾𝐿 (N 1 | |N 2 ) and 𝐾𝐿 (N 2 | |N 3 ) are bounded. This indicates that KL divergence between Gaussians follows a relaxed triangle inequality. All the bounds in this paper are independent of the dimension of distributions. Finally, we discuss the applications of our theorems in deep anomaly detection, reinforcement learning, and sample complexity research.

Acknowledgments and Disclosure of Funding

This work is supported by the National Natural Science Foundation of China (Grant No. 62002107, 62372162, 62032024, 62172429, 61872371, 62202150, 62102442) and National Key Research and Development Program of China (Grant No. 2021ZD40300).

References

Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Rémi Munos, Nicolas Heess, and Martin A. Riedmiller. Maximum a posteriori policy optimisation. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018.

Karim T Abou-Moustafa and Frank P Ferrie. A note on metric properties for some divergence measures: The gaussian case. In Asian Conference on Machine Learning , pages 1-15. PMLR, 2012.

Rohit Agrawal, Yi-Hsiu Chen, Thibaut Horel, and Salil Vadhan. Unifying Computational Entropies via Kullback-Leibler Divergence. In Alexandra Boldyreva and Daniele Micciancio, editors, Advances in Cryptology - CRYPTO 2019 , pages 831-858, Cham, 2019. Springer International Publishing.

Rohit Agrawal and Thibaut Horel. Optimal bounds between f-divergences and integral probability metrics. Journal of Machine Learning Research , 22(128):1-59, 2021.

Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society: Series B (Methodological) , 28(1):131-142, 1966.

Hassan Ashtiani, Shai Ben-David, Nicholas J. A. Harvey, Christopher Liaw, Abbas Mehrabian, and Yaniv Plan. Near-optimal sample complexity bounds for robust learning of gaussian mixtures via compression schemes. J. ACM , 67(6), oct 2020.

Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, and Joydeep Ghosh. Clustering with Bregman divergences. Journal of Machine Learning Research , 6(58):1705-1749, 2005.

Arnab Bhattacharyya, Davin Choo, Rishikesh Gajjala, Sutanu Gayen, and Yuhao Wang. Learning sparse fixed-structure gaussian bayesian networks. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics , volume 151 of Proceedings of Machine Learning Research , pages 9400-9429. PMLR, 28-30 Mar 2022.

Jeremiah Birrell, Paul Dupuis, Markos A. Katsoulakis, Yannis Pantazis, and Luc Rey-Bellet. ( 𝑓 , Γ )-divergences: Interpolating between 𝑓 -divergences and integral probability metrics. Journal of Machine Learning Research , 23(39):1-70, 2022.

Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics) . Springer-Verlag, Berlin, Heidelberg, 2006.

N. Bouhlel and A. Dziri. Kullback-Leibler divergence between multivariate generalized gaussian distributions. IEEE Signal Processing Letters , 26(7):1021-1025, 2019.

Lev M Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics , 7(3):200-217, 1967.

Hyunsun Choi and Eric Jang. WAIC, but why?: Generative ensembles for robust anomaly detection. arXiv preprint arXiv:1810.01392 , 2018.

Robert M Corless, Gaston H Gonnet, David EG Hare, David J Jeffrey, and Donald E Knuth. On the Lambert W function. Advances in Computational mathematics , 5(1):329-359, 1996.

Thomas M Cover and Joy A Thomas. Elements of information theory . John Wiley & Sons, 2012.

Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S Dhillon. Informationtheoretic metric learning. In Proceedings of the 24th international conference on Machine learning , pages 209-216, 2007.

Philippe Donnat, Gautier Marti, and Philippe Very. Toward a generic representation of random variables for machine learning. Pattern Recogn. Lett. , 70(C):24-31, January 2016.

J. . Durrieu, J. . Thiran, and F. Kelly. Lower and upper bounds for approximation of the Kullback-Leibler divergence between gaussian mixture models. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 4833-4836, 2012.

Lsj Farmer. The princeton companion to applied mathematics. Reference Reviews , 30(5):34-35, 2016.

Sarah Filippi, Olivier Cappé, and Aurélien Garivier. Optimism in reinforcement learning and kullback-leibler divergence. In 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , pages 115-122, 2010.

Bela A. Frigyik, Santosh Srivastava, and Maya R. Gupta. Functional Bregman divergence. In 2008 IEEE International Symposium on Information Theory , pages 1681-1685, 2008.

Abhik Ghosh and Ayanendranath Basu. A new family of divergences originating from model adequacy tests and application to robust statistical inference. IEEE Transactions on Information Theory , 64(8):5581-5591, 2018.

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning , volume 1. MIT press Cambridge, 2016.

Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research , 13(25):723-773, 2012.

Peng Guan, Maxim Raginsky, and Rebecca M. Willett. Online Markov decision processes with Kullback-Leibler control cost. IEEE Transactions on Automatic Control , 59(6):1423-1438, 2014.

Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of Wasserstein GANs. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS'17, page 5769-5779, Red Hook, NY, USA, 2017. Curran Associates Inc.

John R Hershey and Peder A Olsen. Approximating the Kullback-Leibler divergence between gaussian mixture models. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP'07 , volume 4, pages IV-317. IEEE, 2007.

Dihong Jiang, Sun Sun, and Yaoliang Yu. Revisiting flow generative models for out-ofdistribution detection. In International Conference on Learning Representations , 2022.

Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems , pages 10215-10224, 2018.

Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Why normalizing flows fail to detect out-of-distribution data. ICML workshop on Invertible Neural Networks and Normalizing Flows, 2020 (NeurIPS 2020) , 2020.

Solomon Kullback. Information theory and statistics . Courier Corporation, 1997.

Johann Heinrich Lambert. Observationes variae in mathesin puram. Acta Helveticae physicomathematico-anatomico-botanico-medica, Band III , pages 128-168, 1758.

Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations , 2018.

Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations , 2018.

Qiang Liu, Jason D. Lee, and Michael Jordan. A kernelized stein discrepancy for goodness-of-fit tests. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 , ICML'16, page 276-284. JMLR.org, 2016.

Zuxin Liu, Zhepeng Cen, Vladislav Isenbaev, Wei Liu, Steven Wu, Bo Li, and Ding Zhao. Constrained variational policy optimization for safe reinforcement learning. In International Conference on Machine Learning , pages 13644-13668. PMLR, 2022.

Warren Morningstar, Cusuh Ham, Andrew Gallagher, Balaji Lakshminarayanan, Alex Alemi, and Joshua Dillon. Density of States Estimation for Out of Distribution Detection. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , volume 130, pages 3232-3240. PMLR, 2021.

Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. AWAC: Accelerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.09359 , 2021.

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? International Conference on Learning Representations (ICLR) , 2019.

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of-distribution inputs to deep generative models using typicality. 4th workshop on Bayesian Deep Learning (NeurIPS 2019) , 2019.

Frank Nielsen. An elementary introduction to information geometry. Entropy , 22(10), 2020.

Frank Nielsen. On the Kullback-Leibler divergence between discrete normal distributions. arXiv preprint arXiv:2109.14920 , 2021.

Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016.

George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research , 22(57):1-64, 2021.

Leandro Pardo. Statistical inference based on divergence measures . CRC press, 2018.

Ziad Rached, Fady Alajaji, and L Lorne Campbell. Rényi's divergence and entropy rates for finite alphabet markov sources. IEEE Transactions on Information theory , 47(4):1553-1561, 2001.

Ziad Rached, Fady Alajaji, and L Lorne Campbell. The Kullback-Leibler divergence rate between markov sources. IEEE Transactions on Information Theory , 50(5):917-921, 2004.

Alfréd Rényi. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics , pages 547-561. University of California Press, 1961.

Robin Tibor Schirrmeister, Yuxuan Zhou, Tonio Ball, and Dan Zhang. Understanding Anomaly Detection with DeepInvertible Networks through Hierarchies of Distributions and Features . In Advances in Neural Information Processing Systems 33 . Curran Associates, Inc., 2020.

Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, and Jordi Luque. Input complexity and out-of-distribution detection with likelihood-based generative models. In International Conference on Learning Representations , 2020.

Alireza Shafaei, Mark Schmidt, and James J Little. Does your model know the digit 6 is not a cat? a less biased evaluation of" outlier" detectors. arXiv preprint arXiv:1809.04729 , 2018.

Vít Škvára, Tomáš Pevn` y, and Václav Šmídl. Are generative deep models for novelty detection truly better? KDD Workshop on Outlier Detection De-Constructed (ODD v5.0) , 2018.

W. Stummer and I. Vajda. On Bregman distances and divergences of probability measures. IEEE Transactions on Information Theory , 58(3):1277-1288, mar 2012.

Tim van Erven and Peter Harremos. Rényi divergence and Kullback-Leibler divergence. IEEE Transactions on Information Theory , 60(7):3797-3820, 2014.

Rui F. Vigelis, Luiza H. F. De Andrade, and Charles C. Cavalcante. Properties of a generalized divergence related to Tsallis generalized divergence. IEEE Transactions on Information Theory , 66(5):2891-2897, 2020.

Pengfei Yang and Biao Chen. Robust Kullback-Leibler divergence and universal hypothesis testing for continuous distributions. IEEE Transactions on Information Theory , 65(4):23602373, 2019.

Yufeng Zhang, Jialu Pan, Wanwei Liu, Zhenbang Chen, Kenli Li, Ji Wang, Zhiming Liu, and Hongmei Wei. Kullback-leibler divergence-based out-of-distribution detection with flow-based generative models. IEEE Transactions on Knowledge and Data Engineering , pages 1-14, 2023.

Peer reviews:
comments: SUMMARY:
This paper explores and proves some properties of KL divergence between multivariate Gaussian distributions. One of the motivations is that as a statistical distance, KL divergence does not satisfy the properties of a metric, that is, symmetry and triangle inequality. In spite of these issues, this paper proposes the relaxed versions. To be specific, it proves the lower bound (resp. upper bound) for reverse KL divergence given the lower bound (resp. upper bound) of forward KL divergence, and the summation upper bound of two bounded KL divergences. Finally, the proposed techniques are applied to anomaly detection with flow based model and reinforcement learning.

STRENGTHS:
(1) This paper proves the lower bound (resp. upper bound) for reverse KL divergence given the lower bound (resp. upper bound) of forward KL divergence, and the summation upper bound of two bounded KL divergences.  
(2) The theoretical results can be applied to some applications in deep learning and reinforcement learning.  
(3) This paper is well-written and easy to understand.

WEAKNESSES:
(1) Theorem 1 and Theorem 3 hold when two conditions are satisfied. For example, for the mean, it requires $\mu_1 = \mu_2$, which is too strong in practice.  
(2) Since the KL divergence has a wide range of applications, the two applications shown in this paper are kind of limited and not convincing.

QUESTIONS:
The KL divergence is widely used in machine learning and statistics, etc. Can the theoretical results in this paper be used to some other tasks in machine learning?

LIMITATIONS:
See Weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-19

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nKullback-Leibler (KL) divergence is an important measure of distance between probability distributions with uses in statistics, information theory and many other fields. However, it is not a proper distance measure, since it is not symmetric and does not satisfy the triangle inequality in general. The authors consider the KL divergence between multivariate Gaussian distributions and show that a relaxed notion of symmetry and triangle inequality holds under certain conditions. Specifically, they formulate an upper bound on KL(N2,N1) when KL(N1,N2) < epsilon, and show that it cannot be much greater than epsilon. Similarly, they give a lower bound on KL(N2,N1) when KL(N1,N2)  > M. Finally, they upper bound KL(N1,N3) when KL(N1,N2) < epsilon1 and KL(N2,N3) < epsilon2. They conclude by discussing several applications of the results in deep learning and reinforcement learning.\n\nSTRENGTHS:\nThe disadvantages of KL divergence as far as symmetry and triangle inequality go are well known, so finding conditions in which even a relaxed version of these properties hold is interesting and potentially useful.\n\nWEAKNESSES:\nFirstly, due to the continuity of the KL divergence around epsilon=0 (N1=N2), the results are not too surprising. The proofs are technical, lengthy and somewhat repetitive. Lemma G.5 in particular is not so digestible for readers. Secondly, the structure of the paper is unorthodox: usually you would have the Related Work section right after the Introduction instead of before Conclusions; that would also be a better place to start mentioning the applications for which your results might be relevant. The section called Lemmas and Notations has no lemmas. The Applications section should be called Discussion.\n\nQUESTIONS:\n- Can you please rearrange the structure of the paper to be more in line with convention?\n- Can Lemma G.5 be made more edible, or could a more informative overview be given?\n\n-- The authors have thoroughly addressed these points in the rebuttal.\n\nLIMITATIONS:\nThere is no potential negative societal impact of the work", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the properties of KL divergence between Gaussian distributions. The main theoretical contributions include two main theorems. The first one gives the supremum of reverse KL divergence between Gaussians when the forward KL divergence is bounded. The conditions when the supremum is attained are also identified. The second theorem gives the relaxed triangle inequality of KL divergence between Gaussians. Based on these two main theorems, this paper also derives several corollaries, including the local approximations and a lower bound of reverse KL divergence. It is also notable that the bounds are dimension-free. Finally, this paper discusses several applications of the theoretical results in OOD detection with flow-based generative models and safe/robust reinforcement learning.\n\nOverall, the research questions studied in this paper have not been answered before. The theoretical contributions of this paper are novel and solid. The proofs are carefully written and correct. Notably, the proof of Theorem 4 is rather technical. The theorems presented in this paper can be applied in various contexts involving KL divergence and Gaussian distributions.\n\nSTRENGTHS:\n1.\tThe problems studied in this paper are novel and interesting. This paper answers these research problems for the first time.   \n2.\tThe proofs, which are based on the Lambert W function, are technical.  \n3.\tThe theoretical results can be applied to various problems, including anomaly detection and reinforcement learning. These results also have other potential applications.\n\nWEAKNESSES:\n1.\tIt is possible to make some equations tighter by introducing notations earlier. For example, notations in Equations (G.146)-(G.150) can be introduced earlier to make Equations (G.128)-(G.144) more concise. \n2.\tThe derivations in Equation (E.54) and (J.193) are over-detailed. These two equations can be shortened.\n\nQUESTIONS:\nSee Weaknesses.\n\nLIMITATIONS:\nThe authors have discussed social impacts and limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors look at the KL divergence between two multivarite Gaussian distributions. The KL divergence is an important distance function between two distributions. However, it lacks certain nice properties that other metric distance functions such as variation distance satisfies: namely, symmetry and triangle inequality. This paper shows that, nevertheless, KL divergence satisfies an approximate version of these two important properties. Specifically, if one of the KL divergences is small then the reverse KL divergence will also be small. Similarly, if two pairs of distributions have small KL divergences between them, then the remaining pair will also have a small KL divergence in between.\n\nThe results are derived by posing this as an optimization problem that minimizes the unknown KL divergences subject to the constraint that the known KL divergences are small. Then certain relevant functions such as  and  are analyzed to derive an upper bound for the above optimization problems.\n\nFinally, the authors argue that such approximate symmetry and approximate triangle inequality appear in several important practical applications. In fact, they mention one such problem involving deep neural networks that led them to study this question.\n\nOne more application of this result is that learning a multivariate Gaussian in either KL gives a similar learning result for the reverse KL. So far algorithms have been derived separately for the two directions, see [arXiv:1710.05209] and [arXiv:2107.10450] for more.\n\nI have not carefully checked the mathematical details.\n\nSTRENGTHS:\nThe paper works on a fundamental mathematical problem of proving that KL divergence between multivariate Gaussians is almost a metric near 0. and gives a nice solution. This paper is very nicely written and a pleasure to read. This is really a beautiful paper.\n\nWEAKNESSES:\nNone.\n\nQUESTIONS:\nNone.\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
In this paper, the authors prove the following interesting mathematical properties of the Kullback-Leibler (KL) divergence between multivariate Gaussian distributions, while the KL divergence is not a proper distance (in sense that it is not symmetric) and does not satisfy the triangle inequality, but:
1. if $KL(N_2||N_1) \leq \epsilon$ then it can be shown that the supremum of $KL(N_1||N_2) $  can be upper bounded by some explicit function of $\epsilon$ that is of order $\epsilon$ for $\epsilon$ small, so that the KL is approximately symmetric in the Gaussian case when being close; and
2. an infimum of $KL(N_1||N_2) $ is also derived for $KL(N_2||N_1) \geq M$; and
2. for three Gaussian $N_1, N_2, N_3$, one has an upper bound $KL(N_1||N_3) $ that verifies the triangle inequality up to a factor of three, again when the three Gaussians are close.

The authors discuss the basic proof ideas and some possibly applications in Section 5.

STRENGTHS:
This paper focuses on the fundamental theoretical properties of Kullback-Leibler divergence between multivariate Gaussian distributions, that, to the best of my knowledge, are novel, and have wide applications in ML.
I've not checked the detailed proofs, but the proof sketch looks compelling. The proof idea is very interesting and may be of independent interest.

WEAKNESSES:
The paper is in good shape, I do not have specific concern to raise.

QUESTIONS:
1. The authors mention that they propose an unified OOD detection algorithm KLODS, but no detail about KLODS is given, it would be great if the authors could elaborate more on this.

LIMITATIONS:
This paper is primarily of theoretical nature, and I do not see any potential negative societal impact of this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-07