Title: Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families
Venue: NeurIPS 2024
Accepted: yes
Keywords: Variational inference

stochastic algorithms

asymptotic analysis

alpha divergence

exponential models
Abstract: Recent works in Variational Inference have examined alternative criteria to the commonly used exclusive Kullback-Leibler divergence. Encouraging empirical results have been obtained with the family of alpha-divergences, but few works have focused on the asymptotic properties of the proposed algorithms, especially as the number of iterations goes to infinity. In this paper, we study a procedure that ensures a monotonic decrease in the alpha-divergence. We provide sufficient conditions to guarantee its convergence to a local minimizer of the alpha-divergence at a geometric rate when the variational family belongs to the class of exponential models. The sample-based version of this ideal procedure involves biased gradient estimators, thus hindering any theoretical study. We propose an alternative unbiased algorithm, we prove its almost sure convergence to a local minimizer of the alpha-divergence, and a law of the iterated logarithm. Our results are exemplified with toy and real-data experiments.

Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families

Abstract

Recent works in Variational Inference have examined alternative criteria to the commonly used exclusive Kullback-Leibler divergence. Encouraging empirical results have been obtained with the family of alpha-divergences, but few works have focused on the asymptotic properties of the proposed algorithms, especially as the number of iterations goes to infinity. In this paper, we study a procedure that ensures a monotonic decrease in the alpha-divergence. We provide sufficient conditions to guarantee its convergence to a local minimizer of the alpha-divergence at a geometric rate when the variational family belongs to the class of exponential models. The sample-based version of this ideal procedure involves biased gradient estimators, thus hindering any theoretical study. We propose an alternative unbiased algorithm, we prove its almost sure convergence to a local minimizer of the alphadivergence, and a law of the iterated logarithm. Our results are exemplified with toy and real-data experiments.

1 Introduction

Many statistical inference problems involve computing or sampling from intractable probability densities. Variational Inference [3, 19] is a broad class of methods that turn this kind of inference problems into optimization programs. Given a family of tractable densities Q , the goal is to find, within that family, a fine approximation q of the targeted density p according to a given criterion. The discrepancy between p and q is usually measured by the exclusive Kullback-Leibler divergence D KL ( q ∥ p ) , for it is quite convenient to use from a computational standpoint. However, it may lead to approximations that tend to underestimate the variance of the target [27], a behavior known as zero-forcing, or mode-seeking when the variational density is unimodal. This can be highly detrimental to the quality of the posterior approximation, for instance when the target is multimodal. To overcome this difficulty, recent research has experimented with other divergence measures, such as the inclusive KL divergence D KL ( p ∥ q ) [28], or the more general alpha-divergence family [1], for instance in [9, 15, 25]. Specifically, in [9], the authors introduced a family of iterative algorithms that perform monotonic alpha-divergence minimization, meaning that the criterion is guaranteed to decrease at each new iteration. Although numerical experiments suggested a strong potential for their method, the convergence properties of the proposed algorithm remained to be determined.

After briefly giving some background on alpha-divergence Variational Inference (Section 2), we study the convergence properties of the monotonic alpha-divergence minimization algorithm introduced in

[9], assuming that the variational family Q is an exponential family of densities (Section 3). We give sufficient conditions for the algorithm to converge toward a local minimizer of the alpha-divergence, and show that the rate of convergence is asymptotically geometric. In Section 4, we explain why the empirical counterpart of this algorithm will not converge to minimizers of the alpha-divergence. Instead, we propose an alternative algorithm that has more advantageous properties. We prove its almost sure convergence, as well as a law of the iterated logarithm. Section 5 is dedicated to extending our algorithm to the training of Variational Auto-Encoders [22]. We exhibit links to variational bounds that are known and have already been studied in the literature [7, 25]. While the main focus of this paper is theoretical, we also provide empirical results on both toy examples and real data in Section 6.

Related work Alpha-divergence minimization has lately become a topic of interest in Variational Inference [5, 7, 8, 9, 10, 26, 35, 38], and most of the currently existing algorithms that perform this task rely on biased gradient estimators [15, 25, 31]. While these strategies have yielded promising empirical results, they have only superficially been studied in theory. In [7], the authors provide guarantees for methods based on the variational bound introduced in [25], corroborating empirical findings in [14]. The convergence of other Variational Inference algorithms has been studied. In [28, 39], the authors use Markov Chain Monte-Carlo to optimize the inclusive Kullback-Leibler divergence, and provide conditions for the almost-sure convergence of their algorithm. Convergence guarantees and rates for black-box variational inference with Gaussian variational families have been obtained in [12]. Even though we use different proof techniques, our framework can be seen as an instantiation of Minorize-Maximization algorithms [18, 23], for which convergence guarantees and rates can be obtained, often at the cost of smoothness assumptions [24].

2 Monotonic alpha-divergence minimization with exponential families

Variational Inference. Let ( Y , Y , ν ) and ( X , X , ν ′ ) be sigma-finite measure spaces, and suppose that we are given some data x = ( x 1 , . . . , x n ) in X . We make the assumption that it stems from a latent variable probabilistic model p θ ( x , y ) = p 0 ( y ) p θ ( x | y ) , where θ is some parameter and y ∈ Y is a latent variable. The prior p 0 ( y ) and the conditional likelihood p θ ( x | y ) are specified. We are interested in estimating the posterior density p θ ( y | x ) = p θ ( y, x ) /p θ ( x ) , which is intractable in general due to the normalizing constant p θ ( x ) . Variational Inference methods aim at finding an approximation of the posterior within a family of tractable probability densities Q = { q η ( · ) , η ∈ E } , where E ⊆ R d is some subset of parameters. The quality of the resulting approximation is estimated by a given criterion that quantifies the dissimilarity between the variational estimate q η ( · ) and the target p θ ( · | x ) . We will work with the alpha-divergence [1, 6], i.e., we will try to minimize

̸

The parameter α allows tuning between mass-covering or zero-forcing behaviors. Precisely, the case α -→ 0 corresponds to the inclusive KL divergence, while α -→ 1 recovers the exclusive KL, and α = 0 . 5 is Hellinger's distance. The alpha-divergence is defined similarly to the Rényi divergence [25, 32, 36], which writes D R α ( q η ∥ p θ ) = 1 α -1 log ∫ q α η p 1 -α θ d ν for all α = 1 . It is then equivalent to minimize either of these two objectives. In the remainder of this paper, we set α ∈ (0 , 1) .

Oftentimes, we only know p θ ( y | x ) up to a positive constant, say p θ ( y | x ) ∝ p ( y ) . Most of the time, the easiest choice is p = p θ ( x , · ) . Since x remains fixed throughout the optimization process, we drop the dependency in x and simply write p ( y ) . Minimizing the alpha-divergence amounts to solving inf η ∈ E L α ( η ) , where

Since ∫ Y p d ν < + ∞ , Hölder's inequality ensures that this quantity is well-defined for all η ∈ E .

Useful definitions and notation. For readability, we define and use throughout the paper

Note that ˇ φ α η is a probability density function (p.d.f.) with respect to ν . Subscripts to E or Cov indicate the integrating probability measure, e.g., for a p.d.f. f and a measurable function g , E f [ g ] = ∫ Y g ( y ) f ( y ) ν (d y ) . When η · has a subscript · and acts as a subscript, we only keep · , e.g., φ α ⋆ may be used in place of φ α η ⋆ .

Monotonic alpha-divergence minimization. In [9], the authors propose an iterative algorithm to perform alpha-divergence minimization in a Variational Inference framework. Given an initial parameter η 0 ∈ E and non-negative numbers ( b t ) t ≥ 0 , we construct a sequence ( η t ) t ≥ 0 using

assuming that the arg max is uniquely defined at each iteration (in this paper, this condition will always be satisfied). Any value of η t +1 that makes the function in the arg max negative ensures a strict decrease in the alpha-divergence between the variational distribution and the target [9, Theorem 1]. Since this function is null at η t , each new iterate is at least as good as the previous one. The point of taking the arg max is to obtain explicit update formulas when the variational family is well-chosen. However, when the search space is constrained, these explicit formulas may yield iterates outside the feasible set. To overcome this difficulty, the regularization term in b t is meant to keep the iterates within acceptable bounds. Therfore, it can be understood as an inverse step-size.

The case of the exponential family. We respectively denote by ⟨· , ·⟩ and ∥ · ∥ the canonical inner product on R d and the induced norm. Assume that the variational family Q belongs to the class of canonical exponential models, i.e., it is a family of probability densities that write, for all η in the feasible set E := { η ∈ R d , | A ( η ) | < + ∞ } and all y ∈ Y ,

where κ : Y → R + is a function, S : Y → E is a sufficient statistic for the natural parameter η , and A : E → R is the log-partition function. We will assume that κ > 0 ν -almost everywhere, and that the family Q is minimal, i.e., the components of S ( y ) are not linearly constrained for ν -almost all y ∈ Y . Finally, we suppose that Q is regular, that is, the feasible set E is open. These assumptions about Q are later referred to as condition (H1). An important property of such families is that for all η ∈ E , we have ∂ η A ( η ) = E q η [ S ] , where ∂ η A is the gradient of A with respect to η . Unless otherwise specified, ∂A will shorthand ∂ η A . The gradient ∂A is a C ∞ -diffeomorphism from E to F := ∂A ( E ) . Let us define

It is shown in [9, Theorem 4] that, given η 0 ∈ E and µ 0 = E q η 0 [ S ] , algorithm (3) is equivalent to

where γ t = ν ( φ α t ) / ( ν ( φ α t ) + b t ) belongs to (0 , 1) when b t ∈ (0 , + ∞ ) . It is guaranteed by [9, Theorem 4] that the arg max in (3) is well-defined if and only if µ t +1 belongs to F . Since F is an open convex set ( c.f. [37, Theorem 3.3]), it is always possible to find a valid γ t ∈ (0 , 1) .

Empirical algorithms. The recursion (6) involves the expectation E ˇ φ α η [ S ] , which does not admit a closed form in general and thus needs to be computed via Monte-Carlo methods [34]. Assume that for any η ∈ E , we can easily generate K independent and identically distributed (i.i.d.) samples y 1 , . . . , y K ∼ q η ( · ) . In the empirical setting, the mean parameterization of exponential families is better suited to theoretical analyses. It is defined by µ = ∂A ( η ) , and, using this parameterization, we introduce

In what follows, when η · has a subscript · , we will denote by µ · the associated mean.

In the iterative empirical framework, it is implicitly assumed that, given all the random variables generated up to step t , the ( y i ) 1 ≤ i ≤ K used in the above definitions of ˆ E ( µ t ; K ) and ˆ ℓ ∗ α ( µ t ; K ) only depend on µ t .

As will be clear from Section 3, the recursion (6) can be seen as an iterative algorithm converging to a point η ⋆ that satisfies R ( η ⋆ ) = µ ⋆ . Since R ( η ) = E ( µ ) /ℓ ∗ α ( µ ) , we naturally replace (6) by the (computable) empirical version. Dropping the dependency in K for notational convenience, we get

This is a special case of Robbins-Monro algorithms [33]. To find the zeros of some deterministic function h : E → E , such procedures consist in running the recursion

where ( γ t ) is a decreasing positive sequence, and ( r t ) are vector-valued random variables with null conditional means given the previous iterations. The issue with (9) is that ˆ E ( µ t ) / ˆ ℓ ∗ α ( µ t ) is a biased estimator of E ( µ t ) /ℓ ∗ α ( µ t ) , hence it can only converge to a zero of h : µ ↦→ E [ ˆ E ( µ ) / ˆ ℓ ∗ α ( µ ) ] -µ , not to a limit point of (6). We propose an alternative procedure defined by the recursion

In Section 4.1, we justify the choice of this algorithm, providing a variety of arguments.

3 Convergence rates for the monotonic algorithm

In this section, we give and discuss conditions for the convergence of the monotonic α -divergence minimization algorithm (6) when the integral R ( η ) is evaluated exactly. We start by giving some definitions. For all γ ∈ [0 , 1] , we define the mapping

so that (6) can be written η t +1 = M γ t ( η t ) . Observe that M γ is defined on the set

We say that η is a fixed point of M γ and we write η ∈ Fix( M γ ) if and only if M γ ( η ) = η . Interestingly, these fixed points are characterized by a (classical, see, e.g., [27]) moment identity and do not depend on γ , as stated in the following lemma. Note that we take γ > 0 , since Fix( M 0 ) = E .

Lemma 1. Suppose that Assumption (H1) holds, and let γ ∈ (0 , 1] . For all η ∈ E , there is η ∈ Fix( M γ ) if and only if E q η [ S ] = E ˇ φ α η [ S ] .

From now on, we simply denote Fix( M ) the common set of fixed points of M γ , γ ∈ (0 , 1] . Finally, we state assumptions that ensure the convergence of (6), which happens at a geometric rate.

Assumption (H1) (Variational family) . Q is a regular minimal exponential family of probability densities as in (4).

Assumption (H2) (Non-divergence) . There exists a compact set K ⊂ E such that terms in the sequence defined by η 0 ∈ K and η t +1 = M γ t ( η t ) for t ≥ 1 all belong to K , with ( γ t ) t ≥ 1 ∈ (0 , 1] N ∗ .

Assumption (H3) (Covariance condition) . For all η ∈ Fix( M ) ∩ K , there is

where, for any square matrix H , we denote by ϱ ( H ) its spectral radius.

Assumption (H4) (Variability condition) . There exists δ ∈ (0 , 1] such that γ t ≥ δ eventually.

Theorem 1. If Assumptions (H1) to (H4) hold, then the sequence ( η t ) converges to some parameter η ⋆ ∈ Fix( M ) ∩ K that is a strict local minimizer of L α . Moreover, for all ρ > 1 -δ (1 -ρ ⋆ ) , we have ∥ η t -η ⋆ ∥ = O ( ρ t ) as t → + ∞ .

In Theorem 1, we set ρ ⋆ = ρ η ⋆ , following the convention specified in Section 2. We will discuss the assumptions hereafter, but note already that (H2) and (H3) imply ρ ⋆ < 1 and thus 1 -δ (1 -ρ ⋆ ) < 1 for all δ ∈ (0 , 1] . Hence, Theorem 1 provides a decreasing geometric rate which can at best be arbitrarily close to ρ ⋆ if δ = 1 . A proof of Theorem 1 is provided in Appendix B.1. Let us now discuss the feasibility and reasonableness of the assumptions.

Assumption (H1) is a matter of model design. We refer to [37] for interesting insight on (regular and minimal) exponential families. For instance, [37, Theorem 3.3] guarantees that F is an open convex set, as the interior of a convex set. Along with definition (13), this implies that for all η ∈ E , the set { γ ∈ [0 , 1] , η ∈ E γ } is either [0 , 1] if R ( η ) ∈ F , or of the form [0 , γ η ) for some γ η > 0 . In other words, ( E γ ) γ ∈ [0 , 1] is a non-increasing collection of subsets of E , with E 0 = E and E 1 = R -1 ( F ) .

Assumption (H2) specifies the construction of the sequence ( η t ) in order to satisfy (6). This in particular requires choosing γ t ∈ (0 , 1] such that η t ∈ E γ t , that is, by taking γ t small enough. The compactness of K is useful to ensure the existence of subsequential limits of the sequence ( η t ) . In practice, given a compact set K , one can choose an initial η 0 ∈ int( K ) and take each γ t small enough to stay in this interior. Alternatively, one can choose any initial η 0 ∈ E , and a sequence ( γ t ) only to satisfy the minimal condition that η t ∈ E γ t at each step. The second strategy is easier to implement as it does not require choosing a compact set a priori , but there is no guarantee that ( η t ) will remain bounded. The first strategy is safer from this point of view, but a poor choice of K may lead to vanishing gains ( γ t ) , making it incompatible with (H4).

Assumption (H3) is a condition on the parameters in Fix( M ) , so it ultimately depends on p and Q . The proof of Theorem 1 relies heavily on the equivalence provided in Lemma 1, and yet the fixed point of the mappings ( M γ ) possess quite a few other remarkable properties. The following proposition elucidates the connections between (H3) and the behavior of the mapping M γ around the minimizers of L α .

Proposition 1. Suppose that (H1) holds and let η ⋆ ∈ Fix( M γ ) . Define the norm ∥ · ∥ ⋆ for x ∈ R d by ∥ x ∥ ⋆ = ∥ ∥ Cov η ⋆ ( S ) 1 / 2 x ∥ ∥ and let ρ ⋆ = ρ η ⋆ be defined as in (14) . Then, for all γ ∈ (0 , 1] and any norm ∥ · ∥ · on R d , we have

̸

Moreover, the following two assertions are equivalent.

(i) We have ρ ⋆ < 1 .

(ii) There exists a norm on R d for which the mapping M γ is a contraction in a neighborhood of η ⋆ for all γ ∈ (0 , 1] .

They also imply the following one.

(iii) η ⋆ is a strict local minimizer of L α .

In light of this result, we better understand why choosing δ = 1 in (H4) will yield the fastest asymptotic convergence rate of ρ ⋆ . The only purpose of the sequence ( γ t ) in the ideal setting is to confine the iterates in a bounded region of the parameter space. Showing that we can choose ( γ t ) as in (H4) is beyond the scope of this paper, but the intuitive idea is that when close enough to a minimizer of the alpha-divergence, the iterates are always contained in some compact set regardless of the choice of γ t as t goes to infinity.

The next section is about transposing the exact algorithm to an empirical setting, which will naturally lead to formulating Robbins-Monro algorithms. The usual gains conditions ∑ γ t = + ∞ and ∑ γ 2 t < + ∞ imply γ t -→ 0 , so we will find ourselves in a situation where δ = 0 . Combining this observation with insight from [13, Theorems 2 and 3] could explain why we do not get a geometric convergence rate for the empirical version of the algorithm.

̸

4 Unbiased alpha-divergence minimization

4.1 Justification of the unbiased algorithm

Perhaps the easiest way to get rid of the bias in (9) is to multiply the gradient estimator by ˆ ℓ ∗ α ( µ t ) , leading to the recursion (11), which we recall is given by

Taking h ( µ ) = E ( µ ) -µℓ ∗ α ( µ ) and r t +1 = ˆ E ( µ t ) -µ t ˆ ℓ ∗ α ( µ t ) -h ( µ t ) , this can be written as a Robbins-Monro procedure (10). Clearly, denoting by ( F t ) the filtration generated by the random variables simulated up to the computation of µ t , we have E [ r t +1 |F t ] = 0 . Note that the zeros of h are characterized by the same moment identity as the one characterizing the fixed point of M γ in Lemma 1.

Another beneficial property of (11) is that if γ t is small enough to have γ t /ℓ ∗ α ( µ t ) < 1 , the exact version of this update carries the same monotonicity property as (6).

Interestingly, we can consider (11) as a gradient descent scheme. Indeed, letting L ∗ α be such that L α = L ∗ α ◦ ( ∂A ) , the gradient of L ∗ α computed with respect to η = ( ∂A ) -1 ( µ ) is

For exponential families that satisfy (H1), the chain rule leads to ∂ η L ∗ α ( µ ) = F ( µ ) ∂ µ L ∗ α ( µ ) , where F ( µ ) is the Fisher information matrix of the model evaluated at µ , i.e., the Hessian matrix of the log-partition A taken at η . Setting γ t ← γ t ( α -1) , we may write the expectation version of (11) as

Under (H1), the matrix F ( µ t ) is positive definite, hence the unbiased algorithm appears as a gradient descent procedure on L ∗ α . More specifically, it is a steepest descent algorithm for the dissimilarity measure D ( µ, µ ′ ) = ∥ µ -µ ′ ∥ F -1 ( µ ) = ( µ -µ ′ ) ⊤ F -1 ( µ )( µ -µ ′ ) . In other words, the update writes

Lastly, note that we could obtain similar guarantees for natural gradient procedures [1, 2, 17], that is, η t +1 = η t + γ t ∂ µ L α ( η t ) , or with plain gradient descent, i.e., µ t +1 = µ t + γ t ∂ µ L ∗ α ( µ t ) or η t +1 = η t + γ t ∂ η L α ( η t ) . However, these algorithms either require evaluating the inverse of the Fisher information matrix - which can be computationally expensive, with an O ( d 3 ) cost -, or they are unstable in practice due to unfavorable loss landscapes (see the toy examples in Section 6). On top of that, we lose the monotonicity property for the exact versions of these procedures.

These four arguments justify the choice of algorithm (11), which is ultimately quite close to (6). However, obtaining guarantees for this new algorithm is far easier, as covered in the next section.

4.2 Convergence results

We now state two theorems regarding the convergence of algorithm (11). Let us start by giving some definitions. A matrix H ∈ R d × d is said to be stable if, for all λ ∈ Sp( H ) , we have ℜ ( λ ) < 0 , i.e., the eigenvalues of H have negative real parts. Then, we denote L ( H ) = -max {ℜ ( λ ) , λ ∈ Sp( H ) } . We also still denote F = ∂A ( E ) , and introduce the critical set S = { µ ∈ F, E ( µ ) = µℓ ∗ α ( µ ) } . The set S is qualified as critical, since µ ∈ S if and only if ∂ µ L ∗ α ( µ ) = 0 . Remember that (11) can be written as a special case of (10) by taking h ( µ ) = E ( µ ) -µℓ ∗ α ( µ ) and r t +1 = ˆ E ( µ t ) -µ t ˆ ℓ ∗ α ( µ t ) -h ( µ t ) . Lastly, given some sequence ( µ t ) ∈ F N , a point µ ⋆ ∈ F , and a subset M ⊂ F , we define the events

and we denote H ⋆ ∈ R d × d the Jacobian matrix of h at µ ⋆ .

Assumption (C0) (Gains) . The positive sequence ( γ t ) is such that ∑ γ t = + ∞ and ∑ γ 2 t < + ∞ .

Assumption (C1) (Critical set) . The elements in the set S are isolated, i.e., for all µ ∈ S , there exists an open neighborhood V of µ such that V ∩ S = { µ } .

Assumption (C2) (Moment condition) . The set M is compact and, with η = ( ∂A ) -1 ( µ ) , there is

The following theorem guarantees the almost sure convergence of the sequence (11) to some µ ⋆ , given the existence of a compact subset of F to which the terms in ( µ t ) belong eventually.

̸

Theorem 2 (Almost sure convergence) . Let Q be an exponential family of densities as in (H1), ( γ t ) a positive sequence that satisfies (C0), and M a (compact) subset of F . We construct a sequence ( µ t ) using recursion (11) . If conditions (C1) and (C2) hold, with S ∩ M = ∅ , then

For the next result, we need to strengthen Assumptions (C0) and (C2) slightly. Recall that H ∗ denotes the Jacobian matrix of h at µ ∗ .

Assumption (C0') (Gains) . There exists δ ∈ ( 1 2 , 1] such that for all t ≥ 1 , we have γ t = γ 0 t -δ , with γ 0 > 0 if δ ∈ ( 1 2 , 1) , or γ 0 > β/ (2 L ( H ⋆ )) for some arbitrary β ∈ (0 , 1] if δ = 1 and H ⋆ is stable.

Assumption (C2') (Additional moment condition) . We can find R > 0 and b > 2 such that for all µ ⋆ ∈ S ,

The result of Theorem 3 holds almost surely on the event Γ( µ ⋆ ) , thus complementing Theorem 2 by giving an almost sure convergence rate.

Theorem 3 (Law of the iterated logarithm) . Construct a sequence ( µ t ) as in (11) using the same number of samples K at each iteration, and let µ ⋆ ∈ S . Assume that (C0'), (C2') and (H3) hold, with δ > 2 /b . Then, letting S t = ∑ t i =1 γ i , there exists a real constant Λ such that

Let us now discuss these two results. Theorem 2 guarantees the convergence of (11) to a critical point of L ∗ α , i.e., a minimizer of the alpha-divergence when (H3) is also satisfied. However, it gives no information regarding the minimizer to which the algorithm will converge. It is a typical convergence result for Robbins-Monro algorithms, with the added difficulty that there are many possible points of convergence. The proof given in Appendix B.2 consists in showing that the distance between µ t and S ∩ M almost surely tends to 0 . Then, we use (C0) and (C1) to show that the iterates cannot jump from one critical point to another.

The result of Theorem 2 is stated conditionally to the event Ξ( M ) . While we immediately notice if the iterates get out of M in practice, finding a priori a compact set that has a non-empty intersection with S and that almost surely contains the terms in ( µ t ) is virtually impossible, since it would require knowing h . However, under mild assumptions on L ∗ α , there exists an elegant and theoretically sound way around this problem. Chen's algorithm [4] consists in taking an increasing sequence of compact sets ( M s ) s ≥ 0 that eventually covers the entire parameter space. If the update flies out of the current compact, say M s , we reset the sequence of iterates by randomly choosing an initial parameter in M 0 and start over, but with a larger compact subset of E , namely M s +1 . To simplify the exposition and provide easy-to-implement algorithms, we will only consider that we are given a compact set M that eventually contains the iterates. However, with some minor tweaks to our assumptions, it could be possible to obtain similar guarantees with Chen's algorithm (e.g., by adapting results from [11]).

Finally, let us briefly discuss assumptions (C1) and (C2). The former is verified if the Hessian matrix of L α is definite positive at all η ∈ ( ∂A ) -1 ( S ) (see Proposition 1 and its proof in Appendix B.1). In

particular, (C1) is implied by (H3). As for (C2), notice that for α ∈ [1 / 2 , 1) , by Jensen's inequality, it is satisfied as soon as ∫ ∥ S ∥ 1 / (1 -α ) p d ν < ∞ . Extending it to α ∈ (0 , 1 / 2) supposes specific behaviors on the relative tails of p and q η .

5 Related algorithms in the context of Variational Auto-Encoders

In this section, we explain how to transpose the algorithms presented in Section 4.1 to the training of Variational Auto-Encoders (VAEs) [21]. We start by showing that the exact versions of the biased and unbiased algorithms correspond to gradient ascent procedures on two different variational bounds. Let us first address the case of the biased algorithm. Recall that it writes

For η ∈ E and µ = ∂A ( η ) , we define the Variational Rényi (VR) bound [25] by

Noticing that ∂ η ℓ ∗ α ( µ ) = α ( E ( µ ) -µℓ ∗ α ( µ )) , we can write an iteration of the biased algorithm as

Under (H1), the Fisher Information Matrix F ( µ t ) is positive definite, hence the biased algorithm is a gradient ascent procedure on the VR bound.

The unbiased algorithm, which writes µ t +1 = µ t + γ t [ E ( µ t ) -µℓ ∗ α ( µ t )] , similarly amounts to performing gradient ascent on the variational bound L G α defined by

In the context of VAEs, we learn both a probabilistic encoder y ↦→ ˜ q f ( x ; η ) ( y ) and a probabilistic decoder x ↦→ ˜ p g ( y ; θ ) ( x ) , where ˜ q η and ˜ p θ are densities from families parameterized respectively by η and θ , while x ↦→ f ( x ; η ) and y ↦→ g ( y ; θ ) are neural networks. For simplicity and to align with the usual notation for V AEs, we will denote ˜ q f ( x ; η ) ( · ) = q η ( · | x ) and ˜ p g ( y ; θ ) ( · ) = p θ ( · | y ) . We will also use the shorthand ϕ = ( η, θ ) .

Since the biased and unbiased algorithms studied in the previous sections minimize the alphadivergence by maximizing the variational bounds L R α and L G α , we propose to train V AEs to maximize those same bounds. In this new setting, they write

To update both the encoder and decoder simultaneously, we differentiate them with respect to ϕ using the reparameterization trick [21]. If z ∼ r ( · ) and there exists a mapping v such that v ( z ; η, x ) has the same distribution as y when y ∼ q η ( · | x ) , then we have

To train VAEs, we simply plug batch estimates of these gradients into an optimizer like Adam. Notably, ∂ ϕ L G α can be estimated unbiasedly, while estimators of ∂ ϕ L R α are subject to bias. We will study the practical implications of this fact in Section 6.

6 Experiments

Toy Gaussian. For our toy experiments in the empirical case, we keep a Gaussian variational family. We try to approximate various targets, including a Gaussian mixture and a Cauchy distribution. The latter does not verify assumption (C2), as its tails are too heavy. Even then, the studied algorithms seem to converge without any issue. Let us now focus on the case of the Gaussian Mixture. On the figures, MAX, UNB, SGE, SGM and NAT respectively stand for maximization approach (9), unbiased algorithm (11), SGD on η , SGD on µ , and natural gradient descent (as discussed at the end of Section 4.1). Since the parameter space has only two dimensions, we also plot the loss landscape and the trajectories followed by the different algorithms (e.g. Gaussian mixture target in Figure 1). We observe that MAX and UNB always seem to have a straightforward path toward a local minimizer of the alpha-divergence, while the other procedures are heavily influenced by the loss landscapes in their respective parameter spaces. Unfavorable landscapes can lead to erratic trajectories and slow convergence, or even to convergence failure (see Appendix D). The MAX and UNB algorithms exhibit better stability and reliability on these toy examples.









































































































































































On Figure 1, we approximate a Gaussian mixture with a single Gaussian. With α = 0 . 5 , the landscape of the alpha-divergence around the starting point has three local minima. The global optimum is situated on the left in the first plot. It is not attained by any of the algorithms, as they all converge to the local minimum in the center. The stochastic gradient with respect to η (SGE, in green) almost gets attracted to the local minimum on the right in the first plot, which is the worst of all three. More precisely, Figure 2 shows boxplots of the alpha-divergence after a certain number of iterations for all five algorithms. The number of Monte-Carlo samples is set to 10 , and we run each algorithm 100 times. We see that the MAX approach is the quickest to converge, however it lands to slightly suboptimal parameters and gets surpassed by UNB, SGE, and even SGM after a sufficient number of iterations. This could be due to the bias induced by the normalization, as discussed in Section 4.

Variational Auto-Encoders. We evaluate the two approaches described in Section 5 on the image datasets CIFAR10 ( 50 000 images of size 32 × 32 ) and CelebA ( 192 599 randomly chosen training images cropped to 128 × 128 ), comparing their performance for varying values of α . VAEs have achieved notable success in separating style and content on various types of data, like images and time series. However, they may suffer posterior collapse on challenging datasets as a result of learning an entangled latent representation. Several authors have investigated ways to disentangle the latent space (see, e.g., [29]), and alpha-divergence minimization could be helpful to learn suitable latent representations. In this series of experiments, we train V AEs with a Gaussian prior to generate new images, the encoders consisting of cascades of four convolutional layers, with batch normalization

1 (9^0) it.

9 (9^1) it.

81 (9^2) it.

729 (9^3) it.

6561 (9^4) it.

59049 (9^5) it.

531441 (9^6) it.

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

Algorithm

MAX

UNB

SGE

SGM

NAT

Alpha-divergence

Milestone

and leaky ReLU activations. Similarly, the decoders are composed of four transposed convolutional layers. We set the latent dimensions to 64 on CIFAR10 and 128 on CelebA.

We compute the gradient estimators (19) and (20). These approaches are respectively referred to as VR and UB in Table 1. We choose the number of samples to be K = 5 . The weights are optimized with Adam [20], using learning rates of 8e -4 on CIFAR10 and 2e -4 on CelebA, and ( β 1 , β 2 ) = (0 . 9 , 0 . 999) in both cases. We set the batch size to 256 and train for 100 epochs on CIFAR10 and 30 epochs on CelebA, for a total of roughly 20 000 iterations on both datasets. Training takes 30 minutes per model on CIFAR10 and a few hours on CelebA, using a single V100 GPU. To evaluate the models, we use the Fréchet Inception Distance (FID) metric [16]. Both test sets include 10 000 examples.

It seems that the choice of α can indeed make a difference in the quality of the resulting model, in terms of FID to the test set. The best values of α differ between the datasets, and are also inconsistent between the two approaches. For instance, on CIFAR10, the better results for the VR approach are obtained when α gets closer to 1 , while the opposite is observed for the UB approach. Still, for both approaches and on both datasets, the IWAE and VAE baselines can be outperformed with a proper choice of α . Though it is not visible in the FID score, the losses obtained with the UB approach are significantly higher than those yielded by the VR approach, and we find that normalizing the importance weights yields a much stabler training process than using unbiased gradient estimators.

7 Conclusion

Our paper presents various asymptotic results for both exact and empirical alpha-divergence minimization algorithms for Variational Inference. We find that the studied algorithms converge to a minimizer of the alpha-divergence. The rate of convergence is asymptotically geometric for the exact procedure, and we prove a law of the iterated logarithm in the empirical setting. However, we only address the case where the variational family is an exponential model. While this covers many use cases, the general case should be investigated in future research.

References

S.-i. Amari. Differential-Geometrical Methods in Statistics , volume 28 of Lecture Notes in Statistics . Springer, 1985.

S.-i. Amari. Natural Gradient Works Efficiently in Learning. Neural Computation , 10(2):251-276, 1998.

D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational Inference: A Review for Statisticians. Journal of the American Statistical Association , 112(518), 2017.

H.-F. Chen, L. Guo, and A.-J. Gao. Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds. Stochastic Processes and their Applications , 27:217-231, 1987.

L. Chen, L. Wang, Z. Han, J. Zhao, and W. Wang. Variational inference based kernel dynamic bayesian networks for construction of prediction intervals for industrial time series with incomplete input. IEEE/CAA Journal of Automatica Sinica , 7(5):1437-1445, 2019.

A. Cichocki and S. Amari. Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities. Entropy , 12:1532-1568, 2010.

K. Daudel, J. Benton, Y. Shi, and A. Doucet. Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics. Journal of Machine Learning Research , 24(243):1-83, 2023.

K. Daudel, R. Douc, and F. Portier. Infinite-dimensional gradient-based descent for alpha-divergence minimisation. The Annals of Statistics , 49(4):2250 - 2270, 2021.

K. Daudel, R. Douc, and F. Roueff. Monotonic Alpha-Divergence Minimisation for Variational Inference. Journal of Machine Learning Research , 24(62):1-76, 2023.

K. Daudel et al. Mixture weights optimisation for alpha-divergence variational inference. Advances in Neural Information Processing Systems , 34:4397-4408, 2021.

B. Delyon. Stochastic approximation with decreasing gain: Convergence and asymptotic theory. Unpublished lecture notes, Université de Rennes , 26:39, 2000.

J. Domke, R. Gower, and G. Garrigos. Provable convergence guarantees for black-box variational inference. Advances in Neural Information Processing Systems , 36, 2024.

R. Douc and S. L. Corff. Asymptotic convergence of iterative optimization algorithms. arXiv preprint arXiv:2302.12544 , 2023.

T. Geffner and J. Domke. On the difficulty of unbiased alpha divergence minimization. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 3650-3659. PMLR, 18-24 Jul 2021.

J. Hernandez-Lobato, Y. Li, M. Rowland, T. Bui, D. Hernández-Lobato, and R. Turner. Black-box alpha divergence minimization. In International conference on machine learning , pages 1511-1520. PMLR, 2016.

M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems , 30, 2017.

M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley. Stochastic Variational Inference. Journal of Machine Learning Research , 14(40):1303-1347, 2013.

D. R. Hunter and K. Lange. A Tutorial on MM Algorithms. The American Statistician , 58(1):30-37, 2004.

M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine learning , 37:183-233, 1999.

D. P. Kingma and J. Ba. Adam: a method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980 , 15, 2017.

D. P. Kingma and M. Welling. Auto-encoding Variational Bayes. arXiv preprint arXiv:1312.6114 , 2013.

D. P. Kingma and M. Welling. An Introduction to Variational Autoencoders. Foundations and Trends in Machine Learning , 12(4):307-392, 2019.

K. Lange, D. R. Hunter, and I. Yang. Optimization Transfer Using Surrogate Objective Functions. Journal of Computational and Graphical Statistics , 9(1), 2000.

K. Lange, J. Won, A. Landeros, and H. Zhou. Nonconvex Optimization via MM Algorithms: Convergence Theory. Wiley StatsRef: Statistics Reference Online , page 1-22, 2021.

Y. Li and R. E. Turner. Rényi Divergence Variational Inference. In Advances in Neural Information Processing Systems , volume 29, 2016.

X. Liu and S. Sun. Alpha-divergence minimization with mixed variational posterior for bayesian neural networks and its robustness against adversarial examples. Neurocomputing , 423:427-434, 2021.

T. Minka et al. Divergence measures and message passing. Technical report, Microsoft Research, 2005.

C. Naesseth, F. Lindsten, and D. Blei. Markovian score climbing: Variational inference with KL(p|| q). Advances in Neural Information Processing Systems , 33:15499-15510, 2020.

K. Oublal, S. Ladjal, D. Benhaiem, E. L. BORGNE, and F. Roueff. Disentangling time series representations via contrastive independence-of-support on l-variational inference. In The Twelfth International Conference on Learning Representations , 2024.

M. Pelletier. On the almost sure asymptotic behaviour of stochastic algorithms. Stochastic processes and their applications , 78(2):217-244, 1998.

J.-B. Regli and R. Silva. Alpha-beta divergence for variational inference. arXiv preprint arXiv:1805.01045 , 2018.

A. Rényi. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume 4, pages 547-562. University of California Press, 1961.

H. Robbins and S. Monro. A Stochastic Approximation Method. The Annals of Mathematical Statistics , 22, 1951.

C. P. Robert and G. Casella. Monte Carlo Statistical Methods . Springer Texts in Statistics. Springer, 2004.

S. Rodriguez-Santana and D. Hernández-Lobato. Adversarial α -divergence minimization for Bayesian approximate inference. Neurocomputing , 471:260-274, 2022.

T. Van Erven and P. Harremos. Rényi divergence and Kullback-Leibler divergence. IEEE Transactions on Information Theory , 60(7):3797-3820, 2014.

M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning , 1(1-2):1-305, 2008.

Y. Yang, D. Pati, and A. Bhattacharya. α -variational inference with statistical guarantees. The Annals of Statistics , 48(2):886 - 905, 2020.

L. Zhang, D. M. Blei, and C. A. Naesseth. Transport score climbing: Variational inference using forward KL and adaptive neural transport. arXiv preprint arXiv:2202.01841 , 2022.

A Table of notation

B Proofs

Preliminaries on useful derivatives. We start with some elementary and useful facts on the derivatives of mappings defined through integrals. Recall that, under (H1), the log-partition function A is of class C ∞ and strictly convex on E with gradient and Hessian given by

The differentiation operations under the integral sign can be justified by the expression of the moment generating function of the sufficient statistic S under the various probabilities at hand. More precisely, for all η in the open set E ⊆ R d and ς ∈ R d such that η + ς ∈ E , we have

This fact is useful to differentiate under the integral mappings such as η ↦→ E q η [ ∏ m j =1 S k j ] for m ∈ N and indices k 1 , . . . , k m ∈ { 1 , . . . , d } .

Similar arguments apply with the quantities defined in (2). Since ∫ p d ν is a positive constant by definition and κ > 0 ν -almost everywhere, the Hölder inequality yields, for all η ∈ E ,

Thus for all η ∈ E and ς ∈ R d such that η + ς ∈ E ,

This can again be used to differentiate under the integral mappings such as η ↦→ E ˇ φ α η [ ∏ m j =1 S k j ] for m ∈ N and indices k 1 , . . . , k m ∈ { 1 , . . . , d } .

Consequently, the mappings η ↦→ ν ( φ α η ) and R : η ↦→ E ˇ φ α η [ S ] are of class C ∞ on E with the gradient of the former being equal to

and the Jacobian matrix of the latter being given by

which simplifies to

Proof of Lemma 1. Let γ ∈ (0 , 1] and η ∈ E . By (12), the condition η ∈ Fix( M ) is equivalent to ∂A ( η ) = γ R ( η ) + (1 -γ ) ∂A ( η ) . Since γ > 0 , this equation simplifies to ∂A ( η ) = R ( η ) , which concludes the proof since ∂A ( η ) = E q η [ S ] and, by definition, R ( η ) = E ˇ φ α η [ S ] .

Proof of Proposition 1. Using the properties of ∂A and R , the mapping M γ defined in (12) is of class C ∞ on E . By applying formula (23) at η ∗ and simplifying the expression, we get

Using the chain rule with (21), and once again the fact that η ⋆ ∈ Fix( M ) , we obtain

Recall that Cov q ⋆ ( S ) is positive definite. Multiplying by Cov q ⋆ ( S ) 1 / 2 on the left and by Cov q ⋆ ( S ) -1 / 2 on the right shows that α Cov q ⋆ ( S ) -1 Cov ˇ φ α ⋆ ( S ) has the same complex eigenvalues as the positive semidefinite matrix α Cov q ⋆ ( S ) -1 / 2 Cov ˇ φ α ⋆ ( S )Cov q ⋆ ( S ) -1 / 2 . Hence these eigenvalues are non-negative and we have

We also deduce that the Jacobian matrix J M γ ( η ⋆ ) admits an eigenvector x ⋆ ∈ R d for the eigenvalue 1 -γ (1 -ρ ⋆ ) . For any norm ∥ · ∥ · on R d , there is

which proves the inequality in (15). Moreover, for all x ∈ R d , there is

Denoting by ∥ · ∥ op ⋆ and ∥ · ∥ op the operator norms associated to ∥ · ∥ ⋆ and ∥ · ∥ , respectively, we get

which proves the equality in (15).

Observe that the equivalence (i) ⇔ (ii) is a direct consequence of (15). Thus, it only remains to prove that (i) ⇒ (iii) . The gradient of the function L α is null at any fixed point of M γ . Indeed we can rewrite (16) as

which is zero at η = η ⋆ by Lemma 1. Therefore, we only need to show that (i) is equivalent to the positive-definiteness of the Hessian matrix of L α at η ⋆ . We have, differentiating the above formula and using (21) and (22) at η = η ⋆ along with (24),

Multiplying by Cov q ⋆ ( S ) -1 / 2 on both sides, H L α ( η ⋆ ) is positive definite if and only if the (nonnegative) eigenvalues of α Cov q ⋆ ( S ) -1 / 2 Cov ˇ φ α ⋆ ( S )Cov q ⋆ ( S ) -1 / 2 are all less than 1. This condition is equivalent to (i) , as we have already proven that the largest eigenvalue of this matrix is ρ ⋆ .

B.1 Proof of Theorem 1

The proof of the theorem is based on the following lemma, inspired by [13, Theorem 6]. We do not apply the main result of this paper directly, as it requires the sequence to converge to some known value. We circumvent this difficulty by using the Banach fixed point theorem, a natural idea in light of the insight provided in Proposition 1.

Lemma 2. Let E be an open subset of R d and M : E → E be a continuous mapping. Starting from η 0 ∈ E , we construct the sequence ( η t ) by the recursion η t +1 = M ( η t ) . We assume that the following three conditions hold.

(i) There exists a compact subset K of E such that { η t , t ≥ 0 } ⊂ K .

(ii) There exists a continuous function ϑ : E → R such that for all η ∈ E , ϑ ◦ M ( η ) ≤ ϑ ( η ) , with equality if and only if η = M ( η ) .

(iii) For all η ∈ Fix( M ) , there exists ρ ′ η ∈ (0 , 1) such that, for all ρ ∈ ( ρ ′ η , 1) , we can find an open neighborhood V of η on which the mapping M is a ρ -contraction.

Then the sequence ( η t ) converges to some η ⋆ ∈ Fix( M ) . Additionally, for all ρ ∈ ( ρ ′ ⋆ , 1) , we have ∥ η t -η ⋆ ∥ = O ( ρ t ) .

Proof. There are two parts in this proof. First, we show that under (i) and (ii) , the sequence ( η t ) admits a limit point that is a fixed point of the mapping M . Then, assumption (iii) allows us to apply the Banach fixed point theorem to conclude.

Since K in (i) is a compact set, there exist a strictly increasing function π : N → N and a point η ⋆ ∈ K such that η π ( t ) -→ η ⋆ . Now, consider the sequence ( η π ( t )+1 ) . Once again, by compactness, there exist a strictly increasing function χ : N → N and a point η ⋆⋆ ∈ K such that η π ◦ χ ( t )+1 -→ η ⋆⋆ . Note that for all t ≥ 0 , we have

and under (ii) this implies ϑ ( η π ◦ χ ( t +1) ) ≤ ϑ ( η π ◦ χ ( t )+1 ) ≤ ϑ ( η π ◦ χ ( t ) ) . By continuity of ϑ , taking the limit and applying the squeeze theorem, we obtain ϑ ( η ⋆ ) = ϑ ( η ⋆⋆ ) . The sequence ( η t ) is defined such that M ( η π ◦ χ ( t ) ) = η π ◦ χ ( t )+1 . The mapping M being continuous, taking t -→ + ∞ yields M ( η ⋆ ) = η ⋆⋆ . Applying ϑ on both sides of the previous identity yields ϑ ◦M ( η ⋆ ) = ϑ ( η ⋆⋆ ) = ϑ ( η ⋆ ) . The equality case in (ii) implies M ( η ⋆ ) = η ⋆ , i.e., the sequence ( η t ) has a limit point η ⋆ that is a fixed point of the mapping M .

Since η ⋆ ∈ Fix( M ) , assumption (iii) ensures the existence of a norm ∥ · ∥ ⋆ on R d and of a neighborhood V of η ⋆ such that M is ρ -Lipschitz continuous on V for the norm ∥ · ∥ ⋆ , for all ρ ∈ ( ρ ′ ⋆ , 1) . Recall that, by convention, ρ ′ ⋆ shorthands for ρ ′ η ⋆ . Thus there exists a non-empty open ∥ · ∥ ⋆ -ball B centered at η ⋆ such that M is a ρ -contraction from B to itself. Since η ⋆ is a limit point of the sequence ( η t ) , this sequence eventually falls into B . By the Banach fixed point theorem, the sequence ( η t ) t ≥ t 0 converges to η ⋆ at a geometric rate, namely ∥ η t -η ⋆ ∥ = O ( ρ n ) .

Let us recall the monotonicity property that justifies algorithm (3) and its exact expression (6). This result is simply a reformulation of [9, Corollary 1] in a less general context.

Theorem 4 (Monotonicity) . Let α ∈ (0 , 1) , γ ∈ (0 , 1] , and define the mapping M γ as in (12) . Then, for all η ∈ E , if M γ ( η ) ∈ E , we have

with equality if and only if η = M γ ( η ) .

We can now put these results together to prove Theorem 1.

Proof of Theorem 1. We apply Lemma 2 under the conditions (H1)-(H3). Condition (i) of Lemma 2 directly follows from (H2), and assumption (H1) guarantees that the mapping M γ is continuous. By Theorem 4, we get condition (ii) with ϑ = L α in Lemma 2. Finally, condition (iii) of Lemma 2 is verified by applying Proposition 1 with Assumption (H3), and since γ t ≥ δ eventually by (H4), we can take ρ ′ ⋆ = 1 -δ (1 -ρ ⋆ ) in Lemma 2.

B.2 Proof of Theorem 2

We start by stating a convergence result in a deterministic setting, which is a particular case of [11, Theorem 2].

̸

Lemma 3. Let ( µ t ) be a sequence constructed by the recursion µ t +1 = µ t + γ t M t [ h ( µ t ) + r t +1 ] where the real- and vector-valued sequences ( γ t ) and ( r t ) verify ∑ γ t = + ∞ , ∑ γ 2 t < + ∞ , and ( M t ) are positive definite matrices such that for all t ≥ 1 , Sp( M t ) ⊂ [ λ min , λ max ] , where λ max > λ min > 0 , and Sp( M t ) is the spectrum of M t . We also assume that the series ∑ γ t M t r t +1 converges in R d . Further assume that h is a continuous function, that there exists L : F → R satisfying ∂ L = -h , and that the points in S = { µ ∈ F, h ( µ ) = 0 } are isolated. If the iterates in ( µ t ) eventually belong to some compact M ⊂ F such that S ∩ M = ∅ , then ( µ t ) converges to some point in S .

The idea is now to prove that the assumptions made in Lemma 3 hold almost surely. In what follows, we use the notation

where ˆ E and ˆ ℓ ∗ α are defined in (7) and (8)

Lemma 4. Let K be a positive integer. Using the notation introduced in (25) , if (C2) holds, then

Proof. By Minkowski's inequality, it suffices to prove the result for K = 1 . In this case, setting η = ∂A -1 ( µ ) , there is

The result follows immediately.

Proof of Theorem 2. The sequence ( µ t ) is constructed by the recursion

introducing the matrix M t = F ( µ t ) , the deterministic function h ( µ ) = F -1 ( µ ) [ E ( µ ) -µℓ ∗ α ( µ ) ] and the noise term r t +1 = F -1 ( µ t ) [ ˆ E ( µ t ) -µ t ˆ ℓ ∗ α ( µ t ) ] -h ( µ t ) . As explained in Section 4.1, we have h = -∂ L ∗ α , and the zero set of h is equal to the set S in assumption (C1). In particular, its points are isolated.

Let us now prove the existence of λ max > λ min > 0 such that Sp( M t ) ⊂ [ λ min , λ max ] for all t ≥ 1 . Since we are working on Ξ( M ) , we will assume without loss of generality that µ t ∈ M for all t . It thus suffices to show that the eigenvalues of F ( µ ) can be bounded for all µ ∈ M . We have M ⊂ F , and F ( µ ) is a positive definite matrix for all µ ∈ F . The application µ ↦→ minSp( F ( µ )) is continuous, thence the Weierstrass extreme value theorem ensures that it attains a minimum on M , and by the aforementioned positive-definiteness argument, this minimum cannot be 0 . We denote it λ min . Similarly, λ max can be defined as the maximum of the continuous function µ ↦→ maxSp( F ( µ )) on M .

To apply Lemma 3, the only assumption left to check is the almost sure convergence of the series ∑ t γ t M t r t +1 . The general term of this series is given by

and as explained in Section 4.1, it is a martingale increment sequence. To conclude, we only need to show that the martingale sequence ∑ n -1 t =0 γ t M t r t +1 is bounded in L 2 . This is equivalent to showing ∑ t E [ ∥ γ t M t r t +1 ∥ 2 ] < + ∞ . Using (C2) and Lemma 4, we have

By assumption (C0), the associated series converges. Lemma 3 allows us to conclude.

B.3 Proof of Theorem 3

Theorem 3 is simply an application of [30, Theorem 1]. We recall a simplified version that fits the context of our analysis.

Theorem 5. Let µ ⋆ ∈ F and ( µ t ) be a sequence constructed by the stochastic algorithm

with ( γ t ) a positive sequence, h a deterministic function, and ( r t ) a random noise sequence. Let ( F t ) be a filtration such that ( µ t ) is F t -adapted, and assume

(i) There exist a > 1 and a neighborhood U of µ ⋆ such that for all µ ∈ U , we may write h ( µ ) = H ⋆ ( µ -µ ⋆ ) + O ( ∥ µ -µ ⋆ ∥ a ) , where H ⋆ is a stable matrix.

(ii) There exists δ ∈ ( 1 2 , 1] such that for all t ≥ 1 , we have γ t = γ 0 t -δ , with γ 0 > 0 if δ ∈ ( 1 2 , 1) , or γ 0 > β 2 L ( H ⋆ ) for some arbitrary β ∈ (0 , 1] if δ = 1 .

(iii) There exist R > 0 and b > 2 /δ such that E [ r t +1 | F t ] 1 {∥ µ t -µ ⋆ ∥ ≤ R } = 0 , and sup t ≥ 0 E [ ∥ r t +1 ∥ b | F t ] 1 {∥ µ t -µ ⋆ ∥ ≤ R } < + ∞ .

(iv) There exists a deterministic symmetric positive definite matrix C such that lim E [ r t +1 r ⊤ t +1 | F t ] = C .

Then, letting S t = ∑ t i =1 γ i , there exists a real constant Λ such that, almost surely on Γ( µ ⋆ ) , we have

Proof of Theorem 3. The proof consists in checking the assumptions of Theorem 5. First, the filtration ( F t ) can be chosen as follows. We set F 0 = σ ( { µ 0 } ) and F t the σ -field generated by the (possibly random) initial parameter µ 0 and the successive ( y i ) generated to compute ˆ E ( µ s ) and ˆ ℓ ∗ α ( µ s ) for s = 1 , . . . , t .

Condition (ii) on the design choice corresponds to (C0'), and (iii) is implied by (C2') using similar arguments as in Lemma 4 and that h is continuous over F . Thus, we only need to check (i) and (iv) . Recall that in this proof, we have h ( µ ) = E ( µ ) -µℓ ∗ α ( µ ) and r t +1 = ˆ E ( µ t ) -µ t ˆ ℓ ∗ α ( µ t ) -h ( µ t ) .

(i) Rewriting (16), and using the properties of exponential families, for all µ ∈ F ,

Since µ ⋆ ∈ S , we have h ( µ ⋆ ) = 0 . In other words, E ( µ ⋆ ) /ℓ ∗ α ( µ ⋆ ) = µ ⋆ , which means that η ⋆ = ( ∂A ) -1 ( µ ⋆ ) is a fixed point of the mapping M 1 defined in (12). By Proposition 1, the matrix H L α ( η ⋆ ) is positive definite. Using H L α ( η ) = H L ∗ α ( µ ) and the positive-definiteness of F -1 ( µ ⋆ ) under (H1), we deduce that H ⋆ = ( α -1) F -1 ( µ ⋆ ) H L ∗ α ( µ ⋆ ) is a stable matrix. We conclude by applying Taylor's formula to the mapping h .

(iv) Note that, for all t , r t +1 = G ( µ t ) -E [ G ( µ t )] where G is defined as in Lemma 4. We deduce that E [ r t +1 r ⊤ t +1 | F t ] is equal to the matrix Cov( G ( µ )) taken at (the random vector) µ = µ t . To obtain (iv) , it thus suffices to show that µ ↦→ Cov( G ( µ )) is continuous on F and valued in the set of positive definite matrices.

Let η ∈ E and µ = ∂A ( η ) . We have

This covariance matrix is positive definite since the exponential family is assumed to be minimal under Assumption (H1), which means that S does not belong to an affine hyperplane under q η . The continuity also follows from the preliminaries of the appendix.

C Additional results on the convergence of (11)

The following results are follow-ups on Theorem 3, they are all proven in [30], and reminded here for completeness.

Theorem 6 (Law of the iterated logarithm, extended version) . Under the assumptions of Theorem 3, almost surely on Γ( µ ∗ ) ,

where Λ is a real constant and S t = ∑ t s =1 γ s . Moreover, if w is an eigenvector of H ∗ , the Jacobian matrix of the function h : µ ↦→E ( µ ) -µℓ ∗ α ( µ ) taken at µ ∗ , then

where Σ is the solution of the Lyapunov equation

with ζ = 0 under (C0) -(i) and ζ = β/ (2 γ 0 ) < L ( H ∗ ) under (C0) -(ii).

Remark. (i) If H ∗ is diagonalizable, we have Λ = ∥ ( P † ) -1 ∥ √ 2tr( P † Σ P ) , where P a matrix which columns are eigenvectors of H ∗ , and P † is its transconjugate.

(ii) Σ is given by the formula

Theorem 7 (Quadratic strong law of large numbers) . Under the assumptions of Theorem 3, and using the same notations, almost surely on Γ( µ ∗ ) ,

Theorem 8 (Central limit theorem) . Under the assumptions of Theorem 3, given Γ( µ ∗ ) ,

D Additional figures

D.1 Toy experiment with a Gaussian mixture target

In the case of a Gaussian mixture target, approximated by a single Gaussian density, we find that the initial point and the learning rate have a high influence on whether the algorithms converge, where they converge, and at what speed.

In most cases, we find that UNB is at least as good SGM, SGE and NAT, while being less computationally intensive than SGM and NAT, and more stable than SGE. An improper tuning of γ 0 may cause some procedures to diverge, SGE being by far the most sensitive, followed by NAT and SGM. Divergence very rarely happens for MAX, which requires little to no tuning to converge. Another noticeable effect is that MAX matches the unbiased procedures in terms of value of the alpha-divergence if γ t is too large and there are enough samples.

1 (9^0) it.

9 (9^1) it.

81 (9^2) it.

729 (9^3) it.

6561 (9^4) it.

59049 (9^5) it.

531441 (9^6) it.

3

4

5

6

7

8

9

1

2

Algorithm

MAX

UNB

SGE

SGM

NAT

Alpha-divergence

Milestone

In the case of the Gaussian approximation of a Cauchy target, Assumption (C2) is not verified. However, we still observe convergence for all algorithms in most cases. As in the Gaussian mixture example, SGM appears to be the slowest method, while SGE sometimes diverges due to an unfavorable loss landscape.

1 (9^0) it.

9 (9^1) it.

81 (9^2) it.

729 (9^3) it.

6561 (9^4) it.

59049 (9^5) it.

531441 (9^6) it.

1.5

2

2.5

3

3.5

4

Algorithm

MAX

UNB

SGE

SGM

NAT

Alpha-divergence

Milestone

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The theoretical results obtained for both the exact and empirical algorithms are clearly stated in the Abstract and the Introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the reasonableness of the various assumptions made after stating our theoretical results.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All the assumptions are clearly stated before the main results, and complete proofs are given in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The code written for our experiments is attached to the submission.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code written for the experiments is provided in the submission.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( ❤/a116/a116♣/a115✿✴✴♥✐♣/a115✳❝❝✴ ♣✉❜❧✐❝✴❣✉✐❞❡/a115✴❈♦❞❡❙✉❜♠✐/a115/a115✐♦♥/a80♦❧✐❝② ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( ❤/a116/a116♣/a115✿ ✴✴♥✐♣/a115✳❝❝✴♣✉❜❧✐❝✴❣✉✐❞❡/a115✴❈♦❞❡❙✉❜♠✐/a115/a115✐♦♥/a80♦❧✐❝② ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We give the hyperparameters for the toy example on the Gaussian mixture, and details about the architecture and hyperparameters in the V AE experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The only random experiment that we could run multiple times is the toy example, we provide boxplots to summarize some of our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The type of GPU and computation time are specified in the V AE experiment.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics ❤/a116/a116♣/a115✿✴✴♥❡✉/a114✐♣/a115✳❝❝✴♣✉❜❧✐❝✴❊/a116❤✐❝/a115●✉✐❞❡❧✐♥❡/a115 ?

Answer: [Yes]

Justification: We followed the guidelines regarding data-related concerns.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This does not apply to the paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not release any new data or pretrained model with this work.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We used some pre-existing code in the VAE experiment and credited the authors.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, ♣❛♣❡/a114/a115✇✐/a116❤❝♦❞❡✳❝♦♠✴❞❛/a116❛/a115❡/a116/a115 has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new assets apart from the code for the toy experiment.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper focuses on the theoretical study of variational inference using alpha-divergences using exponential family. This is an important problem in the field of variational inference. Specifically, this work proves a geometric convergence rate for the algorithm proposed in [9]. Moreover, this paper also proposes an alternative optimization algorithm and its convergence guarantees are provided. The proposed algorithm finds applications in VAE.

STRENGTHS:
1. Asymptotic convergence rate of the algorithm proposed in [9] is derived in this work. Moreover, this paper proposes an alternative unbiased algorithm with convergence guarantees.

2. Assumptions (H1)-(H3) follow by reasonable explanations.

2. Applications of the proposed algorithm to VAE are provided.

WEAKNESSES:
1. It only considers the variational family to be exponential models.

2. The convergence is proved for the proposed algorithm in the asymptotic sense.

QUESTIONS:
1. Typo at line 81, ''negative'' $\to$ ''positive'' since you drop the negative $\frac{1}{\alpha(\alpha-1)}$ term.

2. What is the technical difficulty to derive a convergence result similar to Theorem 1 for the proposed algorithm?

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-06-21

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis procedure studies the asymptotic properties of a variational algorithm that ensures a monotonic decrease in the alpha-divergence.  Specifically the authors investigate behavior in the setting where the variational distribution belongs to the exponential family of distributions.  In this setting, and when a key integral can be analytically calculated, the the authors show monotonic convergence at a geometric rate.  For scenarios where a key integral cannot be calculated explicitly, the authors show an unbiased empirical approach.  Furthermore the authors show that this empirical approach enjoys almost sure convergence to a local minimizer of the alpha-divergence.  Finally, the paper presents, both, simulated and real-data experiments to support the theoretical claims.\n\nSTRENGTHS:\nThis is an very well-written paper and clear despite its high level of mathematical rigor.  The authors consider an important problem of alpha-divergence minimization, which has broad impacts in the ML community.  The convergence analysis and unbiased empirical minimization algorithm appear novel and interesting.\n\nWEAKNESSES:\nThe paper does not contain any major weaknesses that I could find, beyond limitations of the analysis.  These limitations include the specific assumption of an exponential family variational distribution, as well as the stated assumptions H1-H4 and C0-C3.\n\nBelow are some detailed comments:\n* L25 : The inclusive KL divergence should be $D(p||q)$.\n* L69 : Shouldn't the posterior be proportional to the joint $p(x,y)$ rather than the marginal $p(y)$?\n* L80 : \"Assuming the argmax is uniquely defined at each iteration\" it is unclear whether this is a reasonable assumption\n* Eq. (5) : Notation $(\\partial A)^{-1}(\\mu)$ undefined\n* L108 : Is it obvious that $\\mathcal{R} = U/V$?  \n* L196 : Notation $\\circ$ undefined (is this a composition?)\n\nPerhaps the biggest weakness is Sec. 5.  This section falls a little flat as it concludes \"we both obtain a biased and unbiased algorithm\" without making those algorithms explicit or obvious (at least to me).  Also, it is unclear how strong an assumption it is that the exponential family does not depend on $x$ as in $q(\\cdot \\mid x) = q$.  In what reasonable scenario would the encoder be independent of the input?\n\nQUESTIONS:\nSee \"Weaknesses\" section.\n\nLIMITATIONS:\nThe authors do not explicitly state limitations of the proposed methodology.  Nevertheless the authors do make explicit assumptions of the analysis.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the converge properties of an optimization algorithm, used to minimize the alpha divergence when the variational approximation belongs to the family of exponential distributions. The optimizer is a modification of an existing method and its performance is competitive with existing methods (without being stronger). However, the new approach enjoys theoretical guarantees.\n\nSTRENGTHS:\nThis paper is line with recent publications on proving the convergence of VI when minimizing the KL-divergence. Obtaining similar results for the alpha-divergence is a natural next step. The main contribution of the paper is theoretical, and I find the results convincing (although I did not read the appendix in detail). \n\nThe theory motivates a revision to existing methods, which seems to be competitive. Minimizing alpha-divergence is notoriously difficult and I appreciate the paper's discussion of algorithms to do so, and demonstration on some examples.\n\nWEAKNESSES:\nThe paper lacks clarity and is difficult to read.\n\nFirst, the notation is heavy. Two ways to address this might be to introduce a table of notation and to clearly delineate definitions as formal statements. I'm guessing this wasn't done because of space constraints, but this could be a good use of the additional page for accepted papers.\n\nIn Section 5, I was puzzled by the assumption that $q_\\eta(\\cdot \\mid {\\bf x}) = q_\\eta$, that is the factor does not depend on ${\\bf x}$. Does this mean that the encoder maps each image to the same latent representation? It seems like the VAE would then be useless, at least for the task of data compression. This also makes me question the results of Section 6. I'm guessing that while the VAE fails to do any meaningful compression of existing images, it may still generate convincing new images. I'd like to see such images (potentially in the appendix), as a supplement to Table 1, and a simple check that the model is trained in a useful manner.\n\nOnce the authors clarify this point, I can adjust my score.\n\nQUESTIONS:\nI'll use this for minor comments and clarification questions.\n\n- instead of (or in addition to) exclusive and include KL, write KL(q||p) and KL(p||q)\n- Figure 1 and Figure 2 are hard to read. In Figure 1, indicate the optimum. Could the colors for the objective function be more distinct than the colors of the trajectories? \n- Figure 2: font size should font size of text. Could the box plots be replaced with trajectories and shaded intervals?\n- line 314: fix the reference to eq:vrbound\n- Define the IWAE and VAE baselines. Does this simply correspond to minimizing KL(q||p) and KL(p||q)? Is this using the variational family as prescribed in line 275?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the asymptotic analysis for both exact and empirical alpha-divergence minimization algorithms, especially in the case of infinite number of iterations. The paper mainly focuses on the exponential family setting, and provides geometric convergence analysis of exact minimization algorithms using fixed point theories. To bypass the difficulty on studying the empirical minimization algorithms due to the bias of previous algorithms, a novel unbiased algorithm is proposed and analyzed, including almost sure convergence to a local minimizer and a law of the iterated logarithm. Finally, the paper experiments on toy gaussians and on variational auto-encoders to show the effectiveness of the proposed algorithm.\n\nSTRENGTHS:\n- The paper is clearly organized, including both exact and empirical analysis.\n- The paper evaluates the asymptotic properties as the number of iterations goes to infinity, which are seldom discussed in previous works.\n- Extensive discussions on the convergence of empirical alpha-divergence minimization algorithms are provided in the paper.\n\nWEAKNESSES:\n- The novelty of the paper appears to be limited. The basis of the theoretical analysis is mostly covered by [1], and the main theoretical analysis focuses only on exponential families. \n- Some assumptions in the theoretical analysis could be further evaluated.\n  - For Assumption (H3) in section 3, it remains unclear when the mapping $\\mathcal{M}_{\\gamma}$ is a contraction.\n  - It would be beneficial to further verify this for $q_{\\eta}$ and $p$ in the case of the exponential family.\n  - For Assumption (C2) in section 4, the paper states that for “$\\alpha \\in (0, \\frac{1}{2})$ suppose specific behaviors on the relative tails”. It would be illuminating to verify this assumption in experiments, especially in VAE experiments, where the choice of $\\alpha$ can make a difference to the empirical results.\n\n- The experiments of this paper could be extended. It would be nice to add experiments like Bayesian logistic regression as in [1]. Furthermore, the empirical results lack significance. In the toy gaussian experiments, the proposed UNB method achieves more accurate but slower convergence results compared to the biased MAX method, and is surpassed by the NAT method with proper step size. In VAE experiments, the UB approach is not significantly better than the VR approach in some cases.\n- The relation between the unbiasedness and the computational intensiveness of the proposed algorithm could be further justified. The bias issue is not analyzed theoretically because “biased gradient estimators hinder any theoretical study”, and there seems to be a trade-off between unbiasedness and computational intensiveness regarding the comparison of MAX and UNB methods in toy experiments.\n\n[1] K. Daudel, R. Douc, and F. Roueff. Monotonic Alpha-Divergence Minimisation for Variational Inference. Journal of Machine Learning Research, 24(62):1–76, 2023.\n\nQUESTIONS:\n- The paper asserts that “biased gradient estimators hinder any theoretical study”. Is it possible to compare the biased and unbiased algorithms on a relatively simple example like gaussian distributions or two-mixture of gaussians?\n- If assumption (H3) is not satisfied, do we still have geometric convergence? If not, what would be the convergence rate?\n- In VAE experiments, different $\\alpha s$ result in different empirical results for both VR and UB methods. Moreover, UB achieved relatively worse results for $\\alpha \\in (\\frac{1}{2}, 1)$, which is the interval where assumption C2 could be satisfied. Could the authors provide a heuristic understanding of this phenomenon? \n- In line 208 of section 4, the paper states “on top of that, we lose the monotonicity property”. The monotonicity property is essential for the geometric convergence in the exact alpha-divergence minimization algorithm analysis. Does it also contribute to the empirical algorithm analysis? Is it beneficial to solving the instability problems for large step sizes?\n\nLIMITATIONS:\n- The paper states the limitation of only considering the exponential family. Still, there are additional limitations when focusing on this setting. Please refer to the second point of weaknesses.\n- Although the paper claims to focus mostly on asymptotic theory, the quality of experiments could still improve to better support the theoretical analysis. Please refer to the third point of weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

comments: SUMMARY:
This paper explores alpha-divergence Variational Inference (VI) from a theoretical perspective, and in particular the monotonic alpha-divergence minimization algorithm. It includes an asymptotic analysis of the algorithm applicable to exponential families, establishing conditions that ensure convergence to a local minimizer at a geometric rate. The theoretical examination of the sample-based counterpart of the algorithm leads to a modified unbiased version, for which both almost sure convergence and a law of the iterated logarithm are provided. Experimental validation using synthetic and real-world datasets illustrates and supports the theoretical findings of the paper.

STRENGTHS:
This is a relevant analysis of alpha-divergence VI algorithms, delivering an in-depth study of their behavior. It is well-written, with technically intricate proofs that are articulated with clarity. Overall, it is a very nice contribution to the field.

WEAKNESSES:
I mainly have minor comments:
- Some assumptions would benefit from more detailed discussion. Specifically, I am unsure why conditions (H4) and (C0') are considered realistic and sensible.
- The impact of the number of samples K used in the sample-based algorithm is underexplored: what influence does it have? Additionally, is it important to maintain the same number of samples as the number of iterations increases?

QUESTIONS:
-

LIMITATIONS:
-
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-12