Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families
Authors: François Bertholom
randal douc
François Roueff
Venue: NeurIPS 2024 poster (2024)
Keywords: Variational inference
stochastic algorithms
asymptotic analysis
alpha divergence
exponential models
Primary area: probabilistic_methods
TL;DR: We provide asymptotic results for algorithms optimizing the alpha-divergence criterion in the context of Variational Inference, using an exponential variational family.

Abstract: Recent works in Variational Inference have examined alternative criteria to the commonly used exclusive Kullback-Leibler divergence. Encouraging empirical results have been obtained with the family of alpha-divergences, but few works have focused on the asymptotic properties of the proposed algorithms, especially as the number of iterations goes to infinity. In this paper, we study a procedure that ensures a monotonic decrease in the alpha-divergence. We provide sufficient conditions to guarantee its convergence to a local minimizer of the alpha-divergence at a geometric rate when the variational family belongs to the class of exponential models. The sample-based version of this ideal procedure involves biased gradient estimators, thus hindering any theoretical study. We propose an alternative unbiased algorithm, we prove its almost sure convergence to a local minimizer of the alpha-divergence, and a law of the iterated logarithm. Our results are exemplified with toy and real-data experiments.

minimization algorithm introduced in Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families
François Bertholom
Randal Douc
SAMOVAR, Télécom Sud-Paris Institut Polytechnique de Paris, France francois, bereriiiieerrreee thol weweollmmmeeeeeosstteeeeeee
SAMOVAR, Télécom Sud-Paris
Institut Polytechnique de Paris, France
randal wwwpll/oooooeeeeeee--eeeaaaeaaae
François Roueff
LTCI, Télécom Paris
Institut Polytechnique de Paris, France
f francois.
Abstract
Recent works in Variational Inference have examined alternative criteria the commonly used exclusive Kullback-Leipe divergence. Encouraging empirical results have been obtained with the family of alpha-divergences, but few works have focused on the asymptotic properties of the proposed algorithms, especially as the number of iterations goes infinity. this paper, we study a procedure that ensures a monotonic decrease in the alpha-divergence. We provide sufficient conditions to guarantee its convergence to local minimizer of the alpha-divergence at geometric rate when the variational family belongs to the class of exponential models. The sample-based version of this ideal procedure involves biased gradient estimators, thus hindering any theoretical study. We propose alternative unbiased algorithm, we prove its almost sure convergence to a local minimizer the alpha- divergence, and a law of the iterated logarithm. Our results are exemplified with toy and real-data experiments.
Introduction
Many statistical inference problems involve computing or sampling from intractable probability densities. Variational Inference 3 [9 is broad class of methods that turn this kind of inference problems into optimization programs. Given a family of tractable densities Q, the goal is to find, within that family, fine approximation of the targeted density according to given criterion. The discrepancy between and is usually measured by the exclusive Kullback-Leibler divergence ( |l p). for it is quite convenient use from computational standpoint. However, it may lead to approximations that tend to underestimate the variance of the target (27), a behavior known pero-forcing,
mode-seekingg when the variational density is unimodal. This be highly detrimental to the quality of the posterior approximation, for instance when the target multimodal. To overcome this difficulty, recent research has experimented with other divergence measures, such as the inclusive KI. divergence DKL (p) (28), or the more general alpha-divergence family (I], for the authors introduced family of iterative algorithms that perform monotonic hippa-diivegence minimization, meaning that the criterion is guaranteed to decrease at each new iteration. Although numerical experiments suggested strong potential for their method, the convergence properties of the proposed algorithm remained to be determined.
instance in 9 15 25). Specifically,
After briefly giving some background on aipha-divergence Variational Inference (Section 2). study the convergence properties of the monotonic
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

1. assuming that the variational family Q is an exponential family of densities (Section 3). We give sufficient conditions for the algorithm to converge toward local minimizer the diveeeeeee and show that the rate of convergence is asymptotically geometric. In /Section 4 we explain why the empirical counterpart of this algorithm will not converge minimizers of the alpha-divergence. Instead, we propose an alternative algorithm that has more advantageous properties. We prove its almost sure convergence, as well as a law of the iterated logarithm. [Section SJis dedicated to extending our algorithm to the training of Variational Auto-Encoders 222 We exhibit links to variational bounds that are known and have already been studied in the literature 7 225.. While the main focus this paper is theoretical, we also provide empirical results on both toy examples and real data in Section 6
Related work Alpha-divergence minimization has lately become a topic of interest in Variational 26 35 38), and most of the currently existing algorithms that perform this lask rely on biased gradient estimators [15 25 31]. While these strategies have yielded promising empirical results, they have only superficially been studied in theory. In 77. the authors provide guarantees for methods based on the variational bound introduced in [25], corroborating empirical findings in [14. The convergence of other Variational Inference algorithms has been studied. In
Inference 5S
8 9
10
[28 39, divergence, and provide conditions for the almost-sure convergence of their algorithm. Convergence guarantees and rates for black-box variational inference with Gaussian variational families have been obtained in [2]. Even though we use different proof techniques, our framework can be seen as an instantiation of Petorizz-Maximzzationn algorithms [18 233 for which convergence guarantees and rates can be obtained, often at the cost of smoothness assumptions (24].
the authors use Markov Chain Monte-Catt to optimize the inclusive Kullback-Leible
2 Monotonic alpha-divergence minimization with exponential families
Variational Inference. Let (Y, v) and (X. X, z²) be sigma-finite measure spaces, and suppose that we are given some data x = (2)..... Fn. in X. We make the assumption that it stems from latent variable probabilistic model po (2, y) = (p))))p( (æ y). where some parameter and E is latent variable. The prior (v)) and the conditional likelihood pe (a: y) are specified. We are interested in estimating the posterior density pe x) = pe (y. <)/pe (x)). which is intractable in general due to the normalizing constant (z)).. Variational Inference methods aim at finding approximation of the posterior within a family of tractable probability densities Q = (q,((), n E E}.
where E C Rd is some subset of parameters. The quality of the resulting approximation is estimated by given criterion that quantifies the dissimilarity between the variational estimate (-(-)) and the target po x)). We will work with the alpha-divergence [1 i.e., we will try to minimize
x)) = ala- - 1) - [[[
1
9n (n(()) Poly ax)
Da
1 po x))()y).
The parameter allows tuning between mass-covering zero-forcing behaviors. Precisely, the case a' corresponds to the inclusive KL divergence, while a + 1 recovers the exclusive KL, and 0.5 is Hellinger's distance. The alpha-divergence is defined similarly to the Rényi divergence which writes D (gg Po) x-T log J 1 for all a * 1. It is then equivalent to minimize either of these two objectives. In the remainder of this paper, we set E (0, 1).
25
32 36
Oftentimes, we only know po x)) up to positive constant, say ( x) x p(y). Most of the time, the easiest choice is p = Pe(x; -). Since æ remains fixed throughout the optimization process, we drop the dependency in æ and simply write p(y). Minimizing the -ipa-ddieeggence amounts to solving inf (7), where EEE
1
/
Ca(n) ala- -) 9.
Since fy pdv +00, Hölder's inequality ensures that this quantity is well-defined for all 1j E F..
Useful definitions and notation.
For readability, we define and use throughout the paper
5 = gn (=)"
La(n) = ((())((d)), "9 = (())((()))
(2)

---

Note that Pn is probability density function (p.d.f.) with respect to v. Subscripts to E or Cov indicate the integrating probability measure, e.g., for p.d.f. f and a measurable function 9. Bylgi Jy s(v) (g)u(dy) When 7/o has subscript and acts subscript, we only keep e.g.. upt may be used place of 47.
Monotonic alpha-divergence minimization. In 19). the authors propose an iterative algorithm to perform alpha-divergenas minimization in Variational Inference framework. Given an initial parameter E E and non non-negative numbers (be)c2o, we construct a sequence (7)) using
J [[ p(y))) beTn. (v) log
9n
7/++1 aig max DEE
v(dy),
(3)
On. (v)
assuming that the arg max is uniquely defined at each iteration (in this paper, this condition will always be satisfied). Any value of M/+1 that makes the function in the arg max negative ensures a strict decrease in the alpha-divergence between the variational distribution and the target O Theorem I]. Since this function is null at T)2, each new iterate is at least as good as the previous one. The point of taking the arg max is obtain explicit update formulas when the variational family well-chosen. However, when the search space is constrained, these explicit formulas may yield iterates outside the feasible set. To overcome this difficulty, the regularization term in is meant to keep the iterates within acceptable bounds. Therfore, it can be understood as inverse step-size.
The case of the exponential family. We respectively denote by (:- and || the canonical inner product Rd and the induced norm. Assume that the variational family Q belongs the class of canonical exponential models, i.e., it a family of probability densities that write, for all ry in the feasible set E = {" E Rd, |A(1)) < h00} and all y Y,
Zn (v) <(y) exp (n, S())) A(1)].
where K
R4 is function, S - E is a sufficient statistic for the natural parameter r), and A: E + R is the log- neg-partiiion function. We will assume that > 0 v-almost everywhere, and that family Q is minimal, i.e., the components of Syy) are not linearly constrained for p-almost all E Y. Finally, we suppose that Q is regular, that is, the feasible set E is open. These assumptions about Q are later referred to as condition (HI) An important property of such families is that for all E, we have A1)) otherwise specified, 0A will shorthand 0, The gradient dA a C ddiffeomorphism from E to F := OA(E). Let us define
[S), where 4 A is the gradient of A with respect to 7). Unless
R(n) =- Egg (r) J, (w)) v(dy)
(5)
It is shown in 19 Theorem 4] that, given E E and Eng [S]. algorithm is equivalent to
[et1 =n)( + +1-1711 7p
(6)
Mt+1
where "y =v(p))/(/((p))) + b.) belongs to (0. 1) when b, +oc). It is guaranteed by Theorem 4] that the arg max in (3) is well-defined if and only if M4+1 belongs to F. Since F is an open convex set (c.f. (37 Theorem 3.3]), it is always possible find valid 7/ (0, 1).
9
Empirical algorithms. The recursion 6 involves the expectation Epg[S], [S]. which does not admit a closed form general and thus needs to be computed via Monte-Carlo methods (34), Assume that for any y E, we can easily generate K independent and identically distributed (i.i.d)) samples
....... уK ~ (-). In the empirical setting, the mean parameterization exponential families is
better suited to theoretical analyses. It is defined by μ aA(n), and, using this parameterization, introduce
E(p; K) = Σ S(m)
1
K
p(y:)
and 1-a
E(p) = E [(u
(7)
K
(21)
1 K
p(y:))
1-ax
(1; K) = K 2=1
Σ
and
(p) = E (1:
(8)
9n
(vv)
3

---

In what follows, when 7/ has subscript + will denote by e the associated mean. In the iterative empirical framework, it is implicitly assumed that, given all the random variables generated up to step t, the (y:)ışısı used in the above definitions of E K) and (1g K) only depend on jec.
As will be clear from /Section 3] the recursion (6) can be seen as an iterative algorithm converging to point 7/- that satisfies R(())) 14x Since R()) = E / (j), we naturally replace (6 by the
(computable) empirical version. Dropping the dependency K for notational convenience, we get
E(we) ta (1s)
Pc+1 M4 + 7t
Pt
This is special case of Robbins-Monro algorithms [33 To find the zeros of some deterministic function h E + E, such procedures consist in running the recursion
Pe+1 14 + V/ [h(x)) + ret1]:
(10)
where (7x) is a decreasing positive sequence, and (r+) are vector-valued random variables with null conditional means given the previous iterations. The issue with (9) is that { (14) is biased
estimator of (ww)) /e (14), hence it can only converge to a zero of h μ + B[E(p)/ (1)] - ,., not
limit point of (6). We propose an alternative procedure defined by the recursion
Mi+1 pr + Vi (E().) (m))
(11)
In Section 4.1 we justify the choice of this algorithm, providing variety of arguments.
3 Convergence rates for the monotonic algorithm
In this section, we give and discuss conditions for the convergence of the monotonic a-divergence minimization algorithm (6) when the integral R(r) is evaluated exactly. We start by giving some definitions. For all 7 E [0, 1]. we define the mapping
My 17 +
+ + - (11-))0A(n)]
(12)
so that 6 can be written 1 MM (Tr:). Observe that My is defined on the set
Fly {1 E F,, 7R((1) (1 A)))()) E F).
(13)
We say that 7) is fixed point of M, and write 7/ Fix(My) if and only if (,) n. Interestingly, these fixed points are characterized by a (classical, see, e.g., (27) moment identity and do not depend 7, stated in the following lemma. Note that take 7 > since Fix(Mo) E.
Lemma 1. Suppose that Assumption (H1) holds, and let Y (0. For all n E E. there is 77 E Fix(MM.)) if and only if Eg, [S] Egg S).
From now on, we simply denote Fix(M) (M) the common set of fixed points of M,, E (0, 1]. Finally, we state assumptions that ensure the convergence of (6). which happens at geometric rate.
Assumption (H1) (Variational family). Q is regular minimal exponential family of probability densities as in (4).
Assumption (H2) (Non-divergence). There exists compact set K c E such that terms in the sequence defined by t)u E K and MT-1 M,, (ne) for 2 1 all belong to K, with (10)21 (0, 17."
Assumption (H3) (Covariance condition). For all E (ix n K, there is
Pg == ag(Cov (S) Cov pg (S)) < <1,
(14)
where, for any square matrix H. denote by e e(H) its spectral radius. Assumption (H4) (Variability condition). There exists 8 (0, 1] such that n eventually.

---

Theorem 1. If Assumptions (H1) to (H4) hold, then the sequence (mt) converges to some parameter 77 (ix(M) that strict local minimizer of C.. Moreover, for all p 8(1 P=), we have |||e 1/ll O as +00.
In Theorem we set p. = pv. following the convention specified in Section 2 assumptions hereafter, but note already that (H2) and (H3) imply p. < xI and thus - 6(1 p.) < 1 for all E (0, 1]. Hence, Theorem provides a decreasing geometric rate which can at best be arbitrarily close P+ if d 1. A proof of Theorem 1 is provided in (Appendix B.1 Let now discuss the feasibility and reasonableness of the assumptions.
We will discuss the
Assumption (HI) is a matter of model design. We refer to [37] for interesting insight on (regular and minimal) exponential families. For instance, [37 Theorem 3.3] guarantees that is an open convex
set, as the interior of convex set. Along with definition (13). this implies that for all vy E E, the set
{7 E [0, 1), } is either [0, 1] R()) E F, or of the form [0, for some 7, > 0. other
words, (e is non-increasing collection of subsets of E, with Eg = E and E1 = R (F).
Assumption (H2) specifies the construction of the sequence (m)) in order to satisfy (6). This in particular requires choosing n (0, 1] such that 77t E E.,, that is, by taking "t small enough. The compactness of K is useful to ensure the existence of subsequential limits of the sequence (n))) In practice, given compact set K, one can choose an initial rou int(K) and take each 7 small enough to stay in this interior. Alternatively, one can choose any initial E E, and a sequence (nx) only to satisfy the minimal condition that 7 E E,, at each step. The second strategy is easier to implement as it does not require choosing a compact set priori, but there is no guarantee that (1)) will remain bounded. The first strategy is safer from this point of view, but a poor choice of may lead to vanishing gains (7)) making it incompatible with (H4)
Assumption (H3) a condition on the parameters in Fix(M), so it ultimately depends on p and Q. The proof of Theorem relies heavily on the equivalence provided in Lemma I and yet the fixed point of the mappings (M₂) possess quite few other remarkable properties. The following proposition elucidates the connections between and and the behavior of the mapping M, around
the minimizers of La.
Suppose that (HI) holds and let πu Fix(MM). Define the norm ||. for x Rd by ||~||| = (Cov,, (S) æl| and let p. = Py. be defined as (14). Then, for all y E (0, 1] and any norm || ||. on Rd, we have
Proposition
(1 - M,(0ll+ 1- 1-1(1-111)))1111111111 p.) S lim sup
[M, M,01/.
lim sup
limsup 4n- wpthn
(15)
|n-7/1
117 1//l.
Moreover, the following two assertions are equivalent.
(i) We have P. < 1.
(ii) There exists a norm on Id for which the mapping My is a contraction in a neighborhood of The for all 7 (0, 1).
They also imply the following one.
(iil) The is a strict local minimizer of Ca.
In light of this result, we better understand why choosing 8 = 1 in (H4) will yield the fastest asymptotic convergence rate of p.. The only purpose of the sequence (?c) in the idcal setting is to confine the iterates in bounded region of the parameter space. Showing that we choose (ri) as in (H4) is beyond the scope of this paper, but the intuitive idea is that when close enough to
minimizer of the alpha-divergence, the iterates are always contained in some compact set regardless of the choice of I/ as t goes to infinity.
The next section is about transposing the exact algorithm to an empirical setting, which will naturally lead to formulating Robbins-Monne algorithms. The usual gains conditions Σ 7t. = +oo and Σe < +oo imply 7 -, 0, so we will find ourselves in a situation where 5 = 0. Combining this observation with insight from [13 Theorems 2 and 3) could explain why we do not get a geometric convergence rate for the empirical version of the algorithm.
5

---

Unbiased alpha-divergence minimization
4.1 Justification of the unbiased algorithm
Perhaps the easiest way to get rid of the bias in 9 multiply the gradient estimator by t (Pe).
leading to the recursion [II which we recall is given by
ML+1 Me + 7o [((H)) - ulla(u))
Taking h(μz) = E(p) -ut and r1+1 E(me) - (we) - h(re), this can be written as prl:
Robbins-Monro procedure (IO). Clearly, denoting by (F,) the filtration generated by the random variables simulated up to the computation of júx, we have Elrz+i O. Note that the zeros of h are characterized by the same moment identity as the one characterizing the fixed point of My in
Lemma 1
Another beneficial property of [I] is that if ir is small enough to have (H) < the exact version of this update carries the same monotonicity property as (6).
consider (TT) gradient descent scheme. Indeed, letting La be such that La = C (d.), the gradient of L computed with respect to r = (aA) (4) is
Interestingly,
(n)
1 a - / (S(y) - a.(m)))(())))))m)))))
v(dy) = - [((u) (l) (16)
For exponential families that satisfy (HI) the chain rule leads to 0,, F F(H) where F(u) is the Fisher information matrix of the model evaluated μ. i.c., the Hessian matrix of the log-partin nog-partition taken at 17. Setting V/ + (a - 1), we may write the expectation version (II) as
1++1 = 14 + F.. (mc).
Under (HI), the matrix F(r)) is positive definite, hence the unbiased algorithm appears as gradient descent procedure on L More specifically, it is a steepest descent algorithm for the dissimilarity measure D(,, n²) = ||I p (m) (M) (μ + "KD(H - μ²)) In other words, the update writes
(m)) 3, + - D((s m)
14+1 arg min
mERd
2%
Lastly, note that we could obtain similar guarantees for natural gradient procedures | that is, 7 7t + (n)) with plain gradient descent, i.c., pe++1 M ( (p:) or Th-1 = TJ + &, (T)). However, these algorithms either require evaluating the inverse of the Fisher information matrix which can be computationally expensive, with an O(d3) cost -, or they are unstable in practice due to unfavorable loss landscapes (see the toy examples in /Sectionn 6). On top of that, we lose the monotonicity property for the exact versions of these procedures.
These four arguments justify the choice of algorithm [ which is ultimately quite close to (6). However, obtaining guarantees for this new algorithm far easier, as covered in the next section.
4.2 Convergence results
We now state two theorems regarding the convergence of algorithm [T]. Let us start by giving some definitions. A matrix H Raddd is said to be stable if, for all À Sp(H). ). we have R(A) < 0, i... the cigenvalues of H have negative real parts. Then, we denote L(H) also still denote F OA( E), and introduce the critical set S {μ E F, E(H) pl: (n)). The set S is qualified as critical, since μ E S if and only if (,, = 0. Remember that II]) can be written special case of 10) by taking h(w) E()) - 1l (p) and rz+1 E(1.) - prl (px) - M(x)). Lastly, given some sequence (mx) F., point μ., F. and subset M c F, we define the events
max A E Sp H)}. We
T(p.) {w Ω (w( - (w)} and E(M) = fw E Ω 3o 2 0, Vt 2 to. (w() E M), and we denote H. E Raxd the Jacobian matrix of h /4+- Assumption (CO) (Gains). The positive sequence (nx) is such that Σ Σn- -o and Σの +00

---

Assumption (C1) (Critical set), The elements in the set S are isolated, i.e., for all S, there exists an open neighborhood V μ such that V/ n {u}. Assumption (C2) (Moment condition). The set M is compact and, with r) (aA) (4)), there is
2(1
Bag EM
sup
||s -|-0' 10l2
< +00.
The following theorem guarantees the almost sure convergence of the sequence 11 to some P4 given the existence of a compact subset of F to which the terms in (μt) belong eventually.
Theorem 2 (Almost sure convergence). Let Q be exponential family of densities in (HI) (m)
a positive sequence that satisfies (CO) and M a (compact) subset of F. We construct a sequence (14) C2) hold, with S M + 0, then
using recursion [11. conditions CT and
P
U r(w.) E(M) =
SnM
For the next result, we need to strengthen Assumptions (CO) and (C2) slightly. Recall that H, denotes the Jacobian matrix of h at j...
and
Assumption (CO") (Gains). There exists d E ($ I] such that for all t 2 1, we have "/ -oot with 70 Of 8 (2. 1), or "i (/(2L(H.)) for some arbitrary /3 E (0, if 1 and H, is stable.
Assumption (C2") (Additional moment condition). We can find R > 0 and b > such that for all μ. E.S.
6(1-a)
||s 1ll"
sup RCF Ia ISB
< +00.
1n
The result of Theorem holds almost surely on the event [(4,), thus complementing Theorem by giving almost convergence rate.
Theorem 3 (Law of the iterated logarithm). Construct sequence (Ht) as in (II) using the same number of samples K each iteration, and let μ. E S. Assume with 2/b. Then, letting S. E/=1 Yi there exists a real constant such that
) (C2 and (H3) hold,
lim sup
In(S.) l1me pall S almost surely on I(p.).
Let us now discuss these two results. "Thoorem 2 guarantees the convergence of (TT to a critical point of L; i.c., minimizer of the alpha-dia digba-divergence when (H3) is also satisfied. However, it gives information regarding the minimizer to which the algorithm will converge. is a typical convergence result for Robbins-Men Monro algorithms. with the added difficulty that there many possible points of convergence. The proof given in Appendix B.2 consists showing that the distance between b4 and Sn M almost surely tends to O. Then, use (CO) and (CI) show that the iterates cannot jump from one critical point to another.
The result of Theorem 2 is stated conditionally to the event E(M). While we immediately notice if the iterates gel out of in practice, finding a priori a compact set that has a non-empty intersection with S and that almost surely contains the terms in (ft) is virtually impossible, since it would require knowing h. However, under mild assumptions on Ls, there exists an elegant and theoretically sound way around this problem. Chen's 's algorithm 4 consists in taking increasing sequence of compact sets (Ma) that eventually covers the entire parameter space. If the update flies out the current compact, say M,, reset the sequence of iterates by randomly choosing initial parameter in Mo and start over, but with larger compact subset of E, namely M To simplify the exposition and will only consider that
provide easy-to-ing timmllmmeeeeentt algorithms, eventually contains the iterates. However, with some minor tweaks to our assumptions, could be possible to obtain similar guarantees with Chen's algorithm (e.g.. by adapting results from (II)).
given compact set M that
Finally, us briefly discuss assumptions (CT) and (2) The former verified if the Hessian matrix of Ca is definite positive at all 17 (0A) (()) (see Proposition and its proof in Appendix B.I. In

---

particular, (CI) is implied by (H3) As for C22)
notice that for a E [1/2, 1), by Jensen's 's inequality,
it is satisfied as soon as / |||||| a) dv < 00. Extending it to E (0, 1/2) supposes specific
behaviors on the relative tails of p and 9n-
Related algorithms in the context of Variational Auto-Encoders
In this section, explain how to transpose the algorithms presented in Section 4. to the training of Variational Auto-Encoders (VAEs) 211. We start by showing that the exact versions of the biased and unbiased algorithms correspond to gradient ascent procedures two different variational bounds. Let us first address the case of the biased algorithm. Recall that writes
E(pe) Eg (14)
|++1 = Me + 7t
For 1) E and μ AA(1), we define the Variational Rényí (VR) bound 255 by
(1) 1 log Byrrgn (
(((())
1-a
1
log (H).
(17)
Noticing that (v) (E()) 1l (4)), we write iteration of the biased algorithm
ML+1 He 0 c8 (M) Mr + F a
Under (HI) the Fisher Information Matrix F F(ue) is positive definite, hence the biased algorithm is gradient ascent procedure on the VR bound.
The unbiased algorithm, which writes pe+1 Me + n (8(4) ula (Ne)), similarly amounts to performing gradient ascent on the variational bound L defined by
( (p(8) (n
1-a
(c)
Ey
(H).
(18)
1- a
In the context of VAEs, we learn both probabilistic encoder y + ((()))) (v) and a probabilistic decoder x + p) where 4, and po are densities from families parameterized respectively by n and 0, while + (x; r)) and y h ( f) are neural networks. For simplicity and to align with the usual notation for VAES, we will denote (f()))) (.) = 4x (z)) and (-(v: pe(- y). We will also use the shorthand ø = (17.0 8).
Since the biased and unbiased algorithms studied in the previous sections minimize the alpha- divergence by maximizing variational bounds CR and LG we propose to train VAEs to maximize those same bounds. In this new setting, they write
[
(Po(x12)) y) on (n
CR Ch (n,00 1-
log E Byrran 2x)
Po Po(x, y) On (y (z)
1-c0
C. 0,x) =
E Ey (1)2
To update both the encoder and decoder simultancously, we differentiate them with respect to & using the reparameterization trick (21). If 2 ~ r(-) and there exists mapping v such that u(2; 77, x) has the same distribution as y when y ~ (: x)), then we have
log po 0(2;n,)))
0zc (n, 0,x) = EsNr())
Ta (z, n. x)0s
(19)
an (u((;;,,a))) (o(c,u(z;1,z))
C 0, = -Ear(())
(a a))0
log (=(()))
(20)

---

(o ((;; qn(u(22 (, (
Wa(2, x)
where W/ (2, ()) x)
and T. (1) ly (a (",,,,, x))r
To train VAEs, we simply plug batch estimates of these gradients into an optimizer like Adam. Notably, 0g can be estimated unbiasedly, while estimators of Sz5 are subject to bias. We will study the practical implications of this fact in Seection 6
Experiments
Toy Gaussian. For our toy experiments in the empirical case, keep Gaussian variational family, We try to approximate various targets, including a Gaussian mixture and a Cauchy distribution. The latter does not verify assumption (C2) as its tails too heavy. Even then, the studied algorithms seem to converge without any issue. Let us now focus on the case of the Gaussian Mixture. On the figures, MAX. UNB. SGE, SGM and NAT respectively stand for maximization approach (9). unbiased algorithm [T). SGD on 7,, SGD on and natural gradient descent (as discussed at the end ofeSccton 4.1p. Since the parameter space has only two dimensions, we also plot the loss landscape and the trajectories followed by the different algorithms (e.g. Gaussian mixture target in Figure I). We observe that MAX and UNB always seem to have a straightforward path toward a local minimizer of the hipha-divergence while the other procedures are heavily influenced by the loss landscapes in their respective parameter spaces. Unfavorable landscapes can lead to erratic trajectories and slow convergence, or even to convergence failure (see Appendix Dj. The MAX and UNB algorithms exhibit better stability and reliability on these toy examples.
Gaussann Midure
Miture (par
Gaussien Mistara
SGM
stattopp
NAT
35
4.0
3.0
25
1.3
2.0
1.3
10
10
-01
30
Meon
14
Figure 1: Trajectories to approximate a Gaussian mixture with a Gaussian family, represented under different parameterizations.
On Figure I we approximate a Gaussian mixture with a single Gaussian. With a = 0.5, the landscape of the alpha-divergence around the starting point has three local minima. The global optimum is situated on the left in the first plot. It is not attained by any of the algorithms, as they all converge to the local minimum in the center. The stochastic gradient with respect 17 (SGE, in green) almost gets attracted to the local minimum on the right in the first plot, which is the worst of all three. More precisely, Figure 2 shows boxplots of the alpha-divergence after a certain number of iterations for all five algorithms. The number of Monte- Carlo samples is set to 10, and we run each algorithm 100 times. We see that the MAX approach is the quickest converge, however it lands slightly suboptimal parameters and gets surpassed by UNB, SGE, and even SGM after sufficient number of iterations. This could be due to the bias induced by the normalization, as discussed in Section
Variational Auto-E Enco-enccsdrr. We evaluate the two approaches described in Sectiin 5 on the image datasets CIFARIO (50 000 images of size 32 x 32) and CelebA (192 599 randomly chosen training images cropped to 128 x 128), comparing their performance for varying values of a. VAEs have achieved notable success separating style and content on various types of data, like images and time series. However, they may suffer posterior collapse on challenging datasets as result of learning an entangled latent representation. Several authors have investigated ways to disentangle the latent space (see, e.g., 229). and alpha-divergence minimization could be helpful to learn suitable latent representations. In this series of experiments, we train VAEs with a Gaussian prior to generate new images, the encoders consisting of cascades of four convolutional layers, with batch normalization

---

Figure 2: Boxplot of the alpha-divergence against the number of iterations for the five algorithms.
and leaky ReLU activations. Similarly, the decoders are composed of four transposed convolutional layers. We set the latent dimensions to 64 on CIFARIO and 128 on CelebA
We compute the gradient estimators (9) and (0) These approaches respectively referred to as VR and UB in Table 1, We choose the number of samples to be = The weights are optimized with Adam [20], using learning rates of 8e 4 on CIFARIO and 2e
CelebA, and (31, B2) = (0.9, 0.999) in both cases. We set the batch size to 256 and train for 100 epochs on CIFARIO and 30 epochs CelebA, for total of roughly 20 000 iterations on both datasets. Training takes 30 minutes per model on CIFAR 10 and a few hours on CelebA, using a single V100 GPU. To evaluate the models, we use the Fréchet Inception Distance (FID) metric [6]. Both sets include 10 000 examples.
Table Fréchet Inception Distance (FID) on CIFAR-10 and CelebA. Lower is better.
0.01
0.3
a
0.5
a
0.99
Dataset
VR
UB
VR
UB
VR
UB
VR
UB
VR
UB
CIFAR-10
102.0
85.8
102.8
95.4
87.0
90.9
87.4
108.3
81.2
110.3
CelebA
241.4
237.6
240.0
237.7
257.7
237.6
234.3
231.1
239.7
232.5
a = 0 (IWAE)
a = 1 (VAE)
CIFAR-10
122.2
121.1
CelebA
243.8
235.3
It seems that the choice of can indeed make a difference in the quality of the resulting model. in terms of FID the set. The best values differ between the datasets, and are also inconsistent between the two approaches. For instance, on CIFAR 10, the better results for the VR approach are obtained when a gets closer 1, while the opposite is observed for the UB approach. Still, for both approaches and on both datasets, the IWAE and VAE baselines can be outperformed with proper choice of a. Though it is not visible in the FID score, the losses obtained with the UB approach significantly higher than those yielded by the VR approach, and we find that normalizing the importance weights yields a much stabler training process than using unbiased gradient estimators.
Conclusion
Our paper presents various asymptotic results for both exact and empirical alpha-divergence min- imization algorithms for Variational Inference. We find that the studied algorithms converge to a minimizer of the alpha-divergence. The rate of convergence is asymptotically geometric for the exact procedure, and we prove a law of the iterated logarithm in the empirical setting. However, we only address the where the variational family is exponential model. While this covers many use cases, the general case should be investigated in future research.
10

---

References
[1] S.-i. Amari Differental- Methods in Statistics, volume 28 of Lecture Notes in Statistics. Springer, 1985.
S.-i. Amari. Natural Gradient Works Efficiently in Learning. Neural Computation, 10(2):251-276, 1998.
[3] D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational Inference: A Review for Statisticians. Journal of the American Statistical Association, 11)(518). 2017.
H.-F. Chen, L.. Guo, and A.-J Gao. Convergence and robustness of the Robbins-Momro algorithm truncated at randomly varying bounds. Stochastic Processes and their Applications, 27:217-231, 1987.
L. Chen, L. Wang, Han, J. Zhao, and W. Wang. Variational inference based kemel dynamic bayesian networks for construction of prediction intervals for industrial time series with incomplete input. IEEE/CAA Journal of Automatica Sinica, (059)11377-4455 2019.
[6] A. Cichocki and Amari. Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities. Entropy, 12:1532-1506 2010.
[7] K. Daudel, J. Benton, Y. Shi, and A. Doucet. Alpha-divergence Variational Inference Mects Importance Weighted Butt-Enncddrs: Methodology and Asymptotics. Journal of Machine Learning Research, 14(243):1-83, 2023.
[8] K. Daudel, R. Douc, and F. Portier. Infinite-dimensional gradient- based descent for alpha-divergence minimisation. The Annals of Statistics, 49(4):2250 2270, 2021.
[9] K. Daudel, R. Douc, and F. Roueff. Monotonic liva--Dieegeence Minimisation for Variational Inference. Journal of Machine Learning Research, 24(62):1-76, 2023.
[10] K. Daudel et al. Mixture weights optimisation for alpha-divergence variational inference. Advances Neural Information Processing Systems, 34:4397-4408, 2021.
[11] B. Delyon. Stochastic approximation with decreasing gain: Convergence and asymptotic theory. Unpub- fished lecture notes, Université de Rennes, 26:39, 2000.
[12] Domke, R. Gower, and G. Garrigos. Provable convergence guarantees for black-box variational inference. Advances in Neural Information Processing Systems, 36, 2024.
[13] R. Douc and S. L. Corff. Asymptotic convergence of iterative optimization algorithms. arXiv preprint arXiv:2302.12555 12544, 2023.
[14] T. Geffner and J. Domke, On the difficulty of unbiased alpha divergence minimization. In M. Mcila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 3650-3659. PMLR, 18-24 Jul 2021.
[15] J. Mernandez-Lobatt Y. Li. M. Rowland, T. Bui, D. Hernández- Lobato, and R. Turner. Black-box alpha divergence minimization. International conference on machine learning, pages 1511-1520. PMLR, 2016.
[16] Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances Neural Information Processing Systems, 30, 2017.
[17] M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley. Stochastic Variational Inference. Journal of Machine Learning Research, 14033):333311477 2013.
[18] D. R. Hunter and K. Lange. A Tutorial on MM Algorithms. The American Statistician, 58(1):30-33 3-37, 2004.
[19] M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine learning, 37:183-233, 1999.
[20] D. P. Kingma and J. Ba. Adam: method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980, 15, 2017.
[21] D. P. Kingma and M. Welling. Auto-encoding Variational Bayes. arXiv preprint 1312.6114,, 2013.
[22] D. P. Kingma and M. Welling. An Introduction to Variational Autoencoders. Foundations and Trends in Machine Learning, 12(4):307 -92800777992-0 2019.

---

[23] K. Lange. D. R. Humter, and I. Yang. Optimization Transfer Using Surrogate Objective Functions. Journal of Computational and Graphical Statisticx, 9(1, 2000.
[24] K. Lange, J. Won, A. Landeros, and H. Zhou. Nonconvex Optimization via MM Algorithms: Convergence Theory. Wiley StatsRef: Statistics Reference Online, page 1-22, 2021.
[25] Y. Li and R. E. Turner. Rényi Divergence Variational Inference. In Advances in Neural Information Processing Systems, volume 29, 2016.
[26] X. Liu and S. Sun. Alpha-divergence minimization with mixed variational posterior for bayesian neural networks and its robustness against adversarial examples. Neurocomputing. 423:427-434, 2021.
[27] T. Minka et al. Divergence measures and message passing. Technical report, Microsoft Research, 2005.
[28] C. Naesseth, F. Lindsten, and D. Blci. Markovian score climbing: Variational inference with KL(pl q). Advances in Neural Information Processing Systems, 13:15499- 155100, 2020.
[29] K. Oublal, S. Ladjal, D. Benhaiem, E. L. BORGNE, and F. Roueff. Disentangling time series representations via contrastive independennee-o-suuppae on I-variational inference. In The Twelfth International Conference Learning Representationss 2024.
[30] M. Pelletier. On the almost asymptotic behaviour of stochastic algorithms. Stochastic processes and their applicatiions 78(2):217-244, 1998.
[31] J.-B. Regli and R. Silva. Alpha-beta divergence for variational inference. arXiv preprint A885: 2018.
[32] A. Rényi. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium mathematical statisticy and probability volume I: contributions the theory of statistics, volume 4, pages 547-562. University California Press, 1961.
[33] H. Robbins and S. Monro. Stochastic Approximation Method. The Annals of Mathematical Statistics,
22, 1951.
[34] C. P. Rohert and G. Casella. Monte Carlo Statistical Methods. Springer Texts in Statistics. Springer, 2004.
[35] S. Rodriguez-Santana and D. Hemmández-Lobato, Adversarial a-divergence minimization for Bayesian approximate inference. Neurocomputing, 471:260 274, 2022.
[36] T. Van Erven and Harremos. Rényi divergence and Kullback-eiblle divergence. IEEE Transactions on Information Theory, 6007),3777-3820, 2014.
[37] M. J. Wainwright and M. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends@ Machine Learning, 1(1 (112):1-33) 2008
[38] Y. Yang, D. Pati, and A. Bhattacharya. a-variational inference with statistical guarantees. The Annals of Statistics, 48(2):886 905, 2020.
[39] L. Zhang. D. M. Blei, and C. A. Naesseth. Transport climbing: Variational inference using forward KL and adaptive neural transport. arXiv preprint arXiv:2202.016: 018000200111..... 2022.
12

---

A
Table of notation
Symbol
Description
Expression
Function composition
p
Target
Variational density
(n) k(y) exp (((()) A(r)
E
Set of variational parameters Variational family
E {n R [I(nn)] < too}
@ = (gn 17 E EE}
m()) / f fdv
P())
Integral of f w.r.t.
E / gf du
Eylg]
Expectation of under the density f
Cov /
Covyle [9]
Covariance of g under the density
و(g 1ggl))) dv
Ozf
Gradient w.r.t. æ
0A
Gradient of A w.r.t. 1)
AA(n) Eg [S]
Pn
Geometric average of and Zn
by = sh i-n
la
Normalization constant
(a = ((1
Normalization constant Normalized version of 9
E ta o (OA)
Pi
6n 4 /la(t)
Lo
Objective function
La (1) ala 1)
(a(())
Objective function of 1
C. = La (0A)
(a( / ((y)
1
R(())
Expectation of S under ş
R(n) = Bay [S]
p.
Mean of an exponential distribution with natural parameter 1/.
He AA(m.)
E())
Integral of Syp; w.r.t.
E(K) (a) R(n))
p(ys)
1-a
K)
Monte-Cart estimator (1) with i.idd. samples y1....., УK Qn
E(m; K) = K
Σ S(y))
(3
1
K
p(ui)
(w;K)
Monte-Carlo estimator of la (T)) with K i.i.d. samples /1...., WK ~ Qn
(x; K)
K
On (11i)
F
Parameter space for the mean parame- terization
F OA(E)
Mr
Exact algorithm iterated function Domain of M,
My : + (yR))
E,
E, = 11 E, R(( + (1- F)
Fix(M)
Set of fixed points of .M, for any Compact subset of E
Fix(M) = (n E, (n)) =
M
Compact subset of F Covariance ratio
Cov Pq ag ( (S))
Po
F(A)
Fisher information matrix of the model 17 (0A)-1(w) (p)
L(H)
Opposite of the largest real part of eigenvalues in the spectrum of the sym- metric matrix !/
L(H) = - max Sp(H))
Critical of L Event on which the random sequence (mt) converges to μ4, Event on which the random sequence (14) falls into the compact set M
S {p E F(E(p) = pL2(H)} T(p.) = (w E St me(w) - (w)}
т(p.)
E(M)
E(M) (w EΩ, Ole 2 1. 20. (wW EM)
13

---

B Proofs
Preliminaries on useful derivatives. We start with some elementary and useful facts on the derivatives of mappings defined through integrals. Recall that, under (HT) the log- rog-partition function is of class C and strictly convex on E with gradient and Hessian given by
AA(n) g. [s] and Hn(n) = Cov,, (S) r F.
(21)
The differentiation operations under the integral sign can be justified by the expression of the moment generating function of the sufficient statistic S under the various probabilities at hand. More precisely, for all 17 in the open C Fdd and R2 such that 77 + E, have
Eg exp(px
st)] (s.
exp (A(n + 5) A(n)).
This fact is useful to differentiate under the integral mappings such as + Bgn [h St, for m E N and indices km E [l.....d..
Similar arguments apply with the quantities defined in (2) Since pd is positive constant by definition and & > 0 v-almost everywhere, the Hölder inequality yields, for all r E F,
(frde)
1-a
< <<(())
+00.
Thus for all y E E and E IRd such that 755 E E,
r(pfits) m(pg)
Egg exp(a (s, Si)]
exp (a(A(n+ +s) A(n))).
This can again be used to differentiate under the integral mappings such as 1) + Epg [ Sa. for N and indices , km E {1,...,d), d).
Consequently, the mappings v + (v0;) and R : 1 + Egg [S] are of class Coc on E with the gradient of the former being equal to
0,0(+))) = - (5
(22)
and the Jacobian matrix of the latter being given by
(n) v( p((9) (a ((w)(((s)) Eg, [81) Tp
TS) (v)((d))))
( sww) (a E,, 181) (H())((d))) )]
which simplifies to
JR(1) = [/ ((w)((()) - E,, (S]) (99)))d))))))
- ( (()))((W)())))))
(((11))) B,, (S]) &6 (elddy)
(23)
Proof of Lemma 1 Let > E (0, 1] and E. By 12p, the condition Fix(M) is equivalent to
dA(n) = RR()) + (1 - -7)8A(m) Since > O. this equation simplifies to dA(r) = R(n), which concludes the proof since θA(m) Eg,, [S] and, by definition, R(n) Egg [S].
Proof of Proposition 1
Using the properties of 0A and R, the mapping My defined in class - E. By applying formula (3) 7 and simplifying the expression, get
12 of
JR aCov (S)
(24)
14

---

Using the chain rule with (21). and once again the fact that 77+ E Fix(M), we obtain
JM, yaCov, Cov (S) + (1 7)
Recall that Covg. (S) is positive definite. Multiplying by Covg. (S) on the left and by Covg, (5)-1/2 on the right shows that aCov,, (S) (S) has the same complex eigenvalues as the positive semidefinite matrix aCov.. (S) (S) (S)-1/2, Hence these eigenvalues are non-negative and we have
e (J,,, =0(aaov,, Cov (5)) + (1 (1-7)
=1-7(1 p-)-
We also deduce that Jacobian matrix J.M. (n.) admits an eigenvector I. E IR for the eigenvalue 1 (1 P.). For any norm || on IR: there is
[|, (77, =x.) (n. |JM, (n.)x. =1 p.).
lim c00
E 12..1
| l1
which proves the inequality in 15
Moreover, for all x E Rd there is
JMM ( = (1aaaor (S) po Co), (S)-1/2 + (1 (1-1) I) Cov.. (5)
Denoting by || llop- and || llon the operator norms associated to || and || -I. respectively, we get |JM, (mx) xill
|JM, lop)
sup Col. (8)1/201511
7aCor (5) 100 (5) +1
7)ld
op
=1 p.).
which proves the equality in 15
Observe that the equivalence (i) < (ii) is direct consequence of (15). Thus, it only remains to prove that (i) + (ii). The gradient of the function Ca is null at any fixed point of My Indeed we can rewrite (16 as
On v(())) a-1 Egg [S] - E [s]).
which is zero at 17 = 17. by Lemma Therefore, we only need to show that (i) is equivalent to the definitenesss of the Hessian matrix of Ca at m,. We have, differentiating the above formula and using (1 and (22) at 17, along with (2
Hc., (n,) a (eCov (S) - Cova, (S)).
Multiplying by Cov,. (S) on both sides, Hc. (n-) positive definite if and only if the (non- negative) eigenvalues of a Cov,, (S)-1/2Cov se (5) Cov,,, (5)-1/2 are less than 1. This condition equivalent to (i), have already proven that the largest eigenvalue of this matrix is p.. O
15

---

B.1 Proof of Theorem
The proof of the theorem based on the following lemma, inspired by [ Theorem 6]. We do not apply the main result of this paper directly, as it requires the sequence to converge to some known value. We circumvent this difficulty by using the Banach fixed point theorem, a natural idea in light of the insight provided in Proposition 1
Lemma Let E an open subset of Rd and M E E be a continuous mupping, Starting from tlo E, we construct the sequence (11) by the recursion 11+1 = =M(n). We assume that the following three conditions hold.
(i) There exists a compact subset K of E such that {m. t 2 0} CK
(ii) There exists a continuous fumction 0 : E + R. such that for all E E, d M(n) S d(n), with equality if and only if ((]).
(iii) For all 1) Fix(M), there exists pl E (0,1) such that, for all p E (php 1), we can find open neighborhood V of n on which the mapping M is p-contraction.
Then the sequence (n)) converges to some n, Fix(MM. Additionally. for all p (p4. 1). we have || - 7, = (p'))
Proof. There two parts in this proof. First, we show that under and ii
the sequence (ne)
admits a limit point that is a fixed point the mapping M. Then, assumption i allows us to apply the Banach fixed point theorem to conclude.
Since K in ( is a compact set, there exist strictly increasing function π : N - N and a point T/+ Now, consider the sequence (fm(0)+1). Once again, by compactness, there exist a strictly increasing function X N N and a point 1/a, such that ()xx(1)+11 -> 1)...
7) K such that (((())
Note that for all t 2 0, we have
I To + 1) π To x +12 T xt),
and under (ii this implies ((Txx((+1)) < (1)) - (TTx)())) ). By continuity of 9, taking the limit and applying the squeeze theorem, we obtain ((n.) A()..). The sequence (nt) is defined such that M(Hoox(0)
7xxx((++- The mapping .M being continuous, taking 1 - +00 yields M(n.) = .)... Applying & on both sides of the previous identity yields (o..(n.) (n...) ((n.). The equality case ( implies M(n.) = 7)., ie.. the sequence (nx) has limit point 17. that is fixed point of the mapping M.
Since 7/- E Fix(M), assumption neighborhood V of 77, such that M is p-Lipschitz continuous on V for the norm || Il.. for all p E (pL,,), 1). Recall that, by convention, p¹. shorthands for pyy.. Thus there exists non-empty open II-lllb B centered 7/+ such that M is p-contraction from B to itself. Since 7) is limit point of the sequence (1t), this sequence eventually falls into B. By the Banach fixed point theorem, the sequence (m.)2t converges 1), geometric rate, namely || 7/.|| O(p)).
f ensures the existence of norm ||. |l. on Rd and of
Let us recall the monotonicity property that justifies algorithm 3 and its exact expression (6) This result is simply reformulation of (9) Corollary 1] in less general context. E (0, 1}. and define the mapping M, as in 12 Then, Theorem 4 (Monotonicity). Let a 1), E for all n E E, if (n) E, we have
La o Mn s (a)
with equality if and only =n (n)
We now put these results together to prove Theorem
Proof of Theorem I We apply Lemma 2 under the conditions (H1) (H3)
Condition [f) Lemma 2
directly follows from (H2) and assumption (HI) guarantees that the mapping M,, is continuous. By
Theorem 4, we get condition (ii) with 9 L. in Lemma 2 Finally, condition [ii) of Lemma is
verified by applying Proposition with Assumption (H3) and since 7t 2 ô eventually by (H4)
can take p.. = 6( P.) in Lemma
16

---

B.2 Proof of Theorem? 2
We start by stating a convergence result in a deterministic setting, which is a particular case of Theorem 2].
Lemma 3. Let (Pe) be sequence constructed by the recursion M1+1 Me + M, [h((e) + + where the real- and vector-valued sequences (7t) and (Tx) verify En +00, Σn < to0,
and (M₂) are positive definite matrices such that for all t 2 1, Sp(M) C Damin Amaxx) where Amax > Amin > O, and Sp(M) the spectrum M, We also assume that the series Σ Mr converges in R.S. Further assume that is continuous function, that there exists L F + I satisfying ôC hh, and that the points in S {u F, h(u) = 0} are isolated. If the iterates in (H4) eventually belong to some compact M C F such that S n M + 0, then (Pt) converges to some point in S.
The idea is now prove that the assumptions made in Lemma hold almost surely. In what follows, we use the notation
G(u)= E(u; K) - ul2 (p; K),
(25)
where E\t and t are defined in 7 and 8
Lemma 4. Let K be positive integer. Using the notation introduced in 25 if (C2) holds, then
sup IE [[[[[[)]]]]] 400.
(26)
μEM
Proof. By Minkowski's inequality, it suffices prove the result for K 1. In this case, setting 77 = 0A- ' there
2(1 2(1-a)
E [II E, ||S nll
P
9n
The result follows immediately.
Proof of Theorem 2 The sequence (ux) is constructed by the recursion
M4+1 μe + 7/ (((,)) - nn (mm) ue + M, (h(()) rtt1l:
introducing the matrix M, F(u)). the deterministic function h(p) = Fx) [E(H) -l (p)] and the noise term Tt+1 = F-- ( [É(m) - fefa (nt) h(m)). As explained in Section 4.1 we have - and the zero set of equal to the set S in assumption (CT) In particular, its points are isolated.
Let now prove the existence of Amax Amin O such that Sp(M,) [Amins Amax for all t > 1. Since we are working on E(M). we will assume without loss of generality that μr E M for all It thus suffices show that the eigenvalues of F(u) can be bounded for all μ M. We have M I F, and F(u) is positive definite matrix for all E F. The application + min Sp (ạ)) is continuous, thence the Weierstrass extreme value theorem ensures that it attains minimum on M, and argument, this minimum cannot be 0. We denote it Amin- Similarly, Amax can be defined as the maximum of the continuous function μ max Sp(F (p)) on M.
by the aforementioned
To apply Lemma 3] the only assumption left to check is the almost convergence of the series E. M The general term of this series given by
h F(u)r+44 Vt. (((.)) - (ma) - E(v,) + (u.)
and as explained in Section 4.1 it is a martingale increment sequence. To conclude, we only need to show that the martingale sequence Σn MMrr is bounded in L This is equivalent to showing
Σ, E MIr < +oo. Using (C2) and Lemma 4 we have
B[Im []]
2 [i[ee E M) [[Ir2++11
2
sup E [[[ F(4))(1)11
< 2 M.
By assumption (CO)
the associated series converges. Lemma 3 allows us to conclude.
17

---

B.3 Proof of Theorem 3
Theorem 3 is simply an application of 30 Theorem 1]. We recall a simplified version that fits the context of our analysis.
Theorem 5. Let ux E F and (pe) be sequence constructed by the stochastic algorithm
p++1 = Pt p "/ [b((e) + T++1]
with (rte) positive sequence, h deterministic function, and (rx) a random noise sequence. Let (Fi) be a filtration such that (14) is Ft-adapted, and assume
(i) There exist a > 1 and a neighborhood U of μ. such that for all μ E U, we may write h(p) H. (μ pt.) O( p..l")) where H, is stable matrix.
(ii) There exists 8 E 2 1] such that,fo for all i 1, have " 7ot with To 0if ( 1), or "fo > 2I( for some arbitrary 3 E (0, if =
(iii) There exist R > and > 2/6 such that 1B|r-1 | Hall S R} 0, and sup E[Irr++ | Fil uall < R} < +o0. 120
(iv) There exists a deterministic symmetric positive definite matrix lim E [re+IT [F]==C
such that
Then, letting St = =i 7o there exists real constant A such that, almost surely on T(pa), we have
lim sup
timsup
V In(S,) like n. S.A A.
The proof consists checking the assumptions of Theorem First, the filtration (Fi) can be chosen as follows. We set Fo = a ((Po)) and F, the a-field generated by the (possibly random) initial parameter μo and the successive (y:) generated to compute E(U,) and l (p.) for
Proof of Theorem 3
8 = ,,
Condition (üi on the design choice corresponds to (CO') and (iii) is implied by (C2) jsing similar arguments as in Lemma 4 and that is continuous over F. Thus, we only need to check (i) and (iv).
Recall that in this proof, we have (())) = (14) - -ue, (p) and rt+1 = {(4.) uel (pt) - M(4).
(i) Rewriting (16). and using the properties of exponential families, for all μ E F. 0,,((1) = 0 ((xx 11-1111-88 1)0,f( = (a Since E S. we have h(u.) O. In other words, f/x,)/E% (v.)) = paa which means that 77. 12A)~(pp,
(4).
a fixed point of the mapping M, defined [2). By Proposition the matrix Hc. (n+) is positive definite. Using Hc. (1) = Hc; (Ι) and the positive-definitee of F (vc)) under (HI) deduce that HI (a - 1)F Hc, (v.) is stable matrix.
We conclude by applyıng Taylor's formula to the mapping h.
We (iv) Note that, for all t, T(+1 = G(Jx) - E ((+)) where is defined as in Lemma 4
deduce that E [r++117441 [F&] is equal the matrix Cov (G(z)) taken at (the random vector) J Mr. To obtain (iv), it thus suffices to show that μ + Cov (G(n)) continuous on F and valued in the set of positive definite matrices.
Let E and μ 0A(r). We have
Covg, ((S- 1))
Cov(G(H) K
P
9n
This covariance matrix is positive definite since the exponential family assumed to be minimal under mnder/Asuuptiinn (HI) which means that S does not belong to an affine hyperplane under 4n- The continuity also follows from the preliminaries of the appendix.
18

---

C
Additional results on the convergence of (11
The following results are follow-ups on Theorem 3 they are all proven in 30 and reminded here for completeness.
Theorem (Law of the iterated logarithm, extended version). Under the assumptions of Theorem almost surely on (p,)),
lim
sup simsup
V In(S.) l pe 1 S.A.A
where A is a real constant and S: = E 1 7/.. Moreover, if is an eigenvector of H., the Jacobian matrix of the function h. + E(1) pl taken at pos then
lim sup V In())) (w, - (a.) = lim inf
(w. 14 - pb)) V2w Ew.
V |I((S))
where E is the solution of the Lyapunov equation
(H. + (Id)) + (ld) D,
with under (CO) and = (2 L(H.) under (CO) (ii).
Remark. (i) H, diagonalizable, we have = -
2tr
where P matrix
which columns are eigenvectors of H., and pi its transconjugate.
(ii) E is given by the formula
Σ exp [8.H. (Id)) Dexp [s(H] + (Id) ds.
Theorem 7 (Quadratic strong law of large numbers). Under the assumptions of Theorem: 3 and using the same notations, almost surely I'(u.),
lim Σ(m
1mm-nWW Mw)(p p.) E.
Theorem (Central limit theorem). Under the assumptions of Theorem 3 given I(,.))
VT (yt p.) (4) NO,, E).
19

---

D Additional figures
D.1 Toy experiment with Gaussian mixture target
In the case of a Gaussian mixture target, approximated by single Gaussian density. we find that the initial point and the learning have high influence on whether the algorithms converge, where they converge, and at what speed.
In most cases, find that UNB is least as good SGM, SGE and NAT, while being less com- putationally intensive than SGM and NAT, and more stable than SGE. An improper tuning of "o may cause some procedures diverge, SGE being by far the most sensitive, followed by NAT and SGM. Divergence very rarely happens for MAX, which requires little to no tuning to converge. Another noticeable effect is that MAX matches the unbiased procedures in terms of value of the alpha-divergence if Yt is too large and there are enough samples.
Figure Boxplot of the alpha-divergence after set numbers of iterations with large step sizes. Approximating Gaussian mixture with single Gaussian.
In the case of the Gaussian approximation of Cauchy target, Assumption (C2) not verified. However, we still observe convergence for all algorithms in most cases. As in the Gaussian mixture example, SGM appears be the slowest method, while SGE sometimes diverges due to unfavorable loss landscape.
Figure 4: Boxplot of the alpha-divergence after set numbers of iterations. Approximating a Cauchy density with single Gaussian.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The theoretical results obtained for both the exact and empirical algorithms are clearly stated in the Abstract and the Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
Justification:
We discuss the reasonableness of the various assumptions made after stating our theoretical results.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer: [Yes]
Justification: All the assumptions are clearly stated before the main results, and complete proof' are given in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
The code written for experiments is attached to the submission.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
22

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code written for the experiments is provided in the submission.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ca
public/gun des/ aaleeeSubmisiioooooooo
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
ctiis.cc/publlle httppppwwwliiipppepeeeeeeeeeeeee guwdes/ wwwoooouummmssssoooooooooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We give the hyperparameters for the toy example the Gaussian mixture, and details about the architecture and hyperparameters in the VAE experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The only random experiment that could run multiple times is the toy example, we provide boxplots to summarize some of our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The type of GPU and computation time are specified the VAE experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttp:://eeurrps..ccpppbcic cech ines?
Answer:
[Yes]
Justification:
We followed the guidelines regarding data-related concerns.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
This does not apply to the paper.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release any new data pretrained model with this work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We used some pre-existing code in the VAE experiment and credited the authors.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
25

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, patasets.paperssse paperswi thcode com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets apart from the code for the toy experiment.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
26

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "vYn0idm9ZO", "reviewer": "Reviewer_cgje", "summary": "This paper explores alpha-divergence Variational Inference (VI) from a theoretical perspective, and in particular the monotonic alpha-divergence minimization algorithm. It includes an asymptotic analysis of the algorithm applicable to exponential families, establishing conditions that ensure convergence to a local minimizer at a geometric rate. The theoretical examination of the sample-based counterpart of the algorithm leads to a modified unbiased version, for which both almost sure convergence and a law of the iterated logarithm are provided. Experimental validation using synthetic and real-world datasets illustrates and supports the theoretical findings of the paper.", "questions": "-", "limitations": "-", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This is a relevant analysis of alpha-divergence VI algorithms, delivering an in-depth study of their behavior. It is well-written, with technically intricate proofs that are articulated with clarity. Overall, it is a very nice contribution to the field.", "weaknesses": "I mainly have minor comments:\n- Some assumptions would benefit from more detailed discussion. Specifically, I am unsure why conditions (H4) and (C0') are considered realistic and sensible.\n- The impact of the number of samples K used in the sample-based algorithm is underexplored: what influence does it have? Additionally, is it important to maintain the same number of samples as the number of iterations increases?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and encouraging feedback on our submission. We provide detailed responses to each of your questions below, hoping to clarify any uncertainties you may have had reading our paper.\n\n\n> I am unsure why conditions (H4) and (C0') are considered realistic and sensible.\n\nThe rationale behind assumption (H4) is that as we approach the optimal solution, we can afford to be progressively less conservative in our choice of $\\gamma$, since $\\mathcal R(\\eta_t)$ should always remain sufficiently close to the set $F$, or even belong to that set. More specifically, one can show that (H4) holds if the set $\\mathsf K_{\\mathcal L_{\\alpha}(\\eta_0)}$ is compact, meaning that the set of $\\eta$'s that verify $\\mathcal L_{\\alpha}(\\eta) \\leq L_{\\alpha}(\\eta_0)$ is bounded. Consequently, each decrease in the alpha-divergence allows for an increase in the highest permissible value of $\\gamma$. We are open to including a concise discussion on this matter in the Appendix.\n\nRegarding (C0’), it is essentially a stricter version of (C0) and involves a design choice left to the practitioner's discretion. It's important to note that (C0) and (H4) are incompatible, which partly explains why we do not get a geometric convergence rate in empirical scenarios.\n\n>What influence does the number of samples $K$ have in the sample-based algorithm?\n\nIncreasing the number of samples $K$ reduces the bias of the estimator used in the first sample-based algorithm (MAX). Notably, the estimator $\\widehat U / \\widehat V$ becomes asymptotically unbiased, meaning that as $K \\to +\\infty$, it converges to $\\mathcal R$. This is illustrated in the toy Gaussian experiment, where a small sample size ($K = 10$) causes the MAX algorithm to converge to suboptimal parameters with higher alpha-divergence compared to unbiased algorithms. To highlight the impact of sample size on bias, we can include additional plots in Appendix C. Aside from this, a smaller sample size accelerates per-iteration computation but results in noisier estimates of the intractable integrals. This creates a tradeoff, a detailed exploration of which is beyond this paper's scope.\n\n\n> Is it important to maintain the same number of samples as the number of iterations increases?\n\nTheoretically, maintaining the same number of samples across all iterations is useful to satisfy condition (iv) in the proof of Theorem 3, as it guarantees the continuity of the function $\\mu \\mapsto \\mathrm{Cov}(G(\\mu))$. However, in practice, one might opt to use fewer samples in the early iterations and increase the number later on to improve the accuracy of the final outcome.\n\n\\\nThank you again for your attentive review and encouraging feedback on our submission, we hope our responses have helped resolve any ambiguities.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mjtIXHexg9", "reviewer": "Reviewer_pDqJ", "summary": "This paper proposes the asymptotic analysis for both exact and empirical alpha-divergence minimization algorithms, especially in the case of infinite number of iterations. The paper mainly focuses on the exponential family setting, and provides geometric convergence analysis of exact minimization algorithms using fixed point theories. To bypass the difficulty on studying the empirical minimization algorithms due to the bias of previous algorithms, a novel unbiased algorithm is proposed and analyzed, including almost sure convergence to a local minimizer and a law of the iterated logarithm. Finally, the paper experiments on toy gaussians and on variational auto-encoders to show the effectiveness of the proposed algorithm.", "questions": "- The paper asserts that “biased gradient estimators hinder any theoretical study”. Is it possible to compare the biased and unbiased algorithms on a relatively simple example like gaussian distributions or two-mixture of gaussians?\n- If assumption (H3) is not satisfied, do we still have geometric convergence? If not, what would be the convergence rate?\n- In VAE experiments, different $\\alpha s$ result in different empirical results for both VR and UB methods. Moreover, UB achieved relatively worse results for $\\alpha \\in (\\frac{1}{2}, 1)$, which is the interval where assumption C2 could be satisfied. Could the authors provide a heuristic understanding of this phenomenon? \n- In line 208 of section 4, the paper states “on top of that, we lose the monotonicity property”. The monotonicity property is essential for the geometric convergence in the exact alpha-divergence minimization algorithm analysis. Does it also contribute to the empirical algorithm analysis? Is it beneficial to solving the instability problems for large step sizes?", "limitations": "- The paper states the limitation of only considering the exponential family. Still, there are additional limitations when focusing on this setting. Please refer to the second point of weaknesses.\n- Although the paper claims to focus mostly on asymptotic theory, the quality of experiments could still improve to better support the theoretical analysis. Please refer to the third point of weaknesses.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is clearly organized, including both exact and empirical analysis.\n- The paper evaluates the asymptotic properties as the number of iterations goes to infinity, which are seldom discussed in previous works.\n- Extensive discussions on the convergence of empirical alpha-divergence minimization algorithms are provided in the paper.", "weaknesses": "- The novelty of the paper appears to be limited. The basis of the theoretical analysis is mostly covered by [1], and the main theoretical analysis focuses only on exponential families. \n- Some assumptions in the theoretical analysis could be further evaluated.\n  - For Assumption (H3) in section 3, it remains unclear when the mapping $\\mathcal{M}_{\\gamma}$ is a contraction.\n  - It would be beneficial to further verify this for $q_{\\eta}$ and $p$ in the case of the exponential family.\n  - For Assumption (C2) in section 4, the paper states that for “$\\alpha \\in (0, \\frac{1}{2})$ suppose specific behaviors on the relative tails”. It would be illuminating to verify this assumption in experiments, especially in VAE experiments, where the choice of $\\alpha$ can make a difference to the empirical results.\n\n- The experiments of this paper could be extended. It would be nice to add experiments like Bayesian logistic regression as in [1]. Furthermore, the empirical results lack significance. In the toy gaussian experiments, the proposed UNB method achieves more accurate but slower convergence results compared to the biased MAX method, and is surpassed by the NAT method with proper step size. In VAE experiments, the UB approach is not significantly better than the VR approach in some cases.\n- The relation between the unbiasedness and the computational intensiveness of the proposed algorithm could be further justified. The bias issue is not analyzed theoretically because “biased gradient estimators hinder any theoretical study”, and there seems to be a trade-off between unbiasedness and computational intensiveness regarding the comparison of MAX and UNB methods in toy experiments.\n\n[1] K. Daudel, R. Douc, and F. Roueff. Monotonic Alpha-Divergence Minimisation for Variational Inference. Journal of Machine Learning Research, 24(62):1–76, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough and constructive review of our paper. We greatly appreciate the time and effort you have dedicated to providing feedback. We are committed to addressing the concerns raised and improving our work. Below, we respond to each of your points in detail.\n\n> The basis of the theoretical analysis is mostly covered by [1].\n\nWhile our work indeed builds on the algorithm proposed in [1], which had already established the monotonicity property and explicit update form for the exponential family, our contributions lie in proving the algorithm’s convergence at an asymptotically geometric rate and towards a minimizer of the alpha-divergence. We further consider this algorithm in the empirical setting and propose an unbiased version, for which we provide two convergence theorems.\n\n> For Assumption (C2), the paper states that “$\\alpha \\in (0, ½)$ supposes specific behaviors on the relative tails”. It would be illuminating to verify this assumption in experiments.\n\nWe have found that convergence can still occur even if assumption (C2) is not met. For example, when $p$ is the density of a Cauchy distribution and the variational family is Gaussian, we still observe convergence (Appendix C). So far, we have not encountered a simple case where convergence entirely fails. This suggests that assumption (C2) could be relaxed or even replaced in future studies.\n\n> It would be nice to add experiments like Bayesian logistic regression.\n\nDue to space constraints, we had to make choices regarding which experiments to present in the paper. We opted for the toy Gaussian examples as we found them to be more insightful than Bayesian logistic regression or other classical applications of variational inference algorithms.\n\n> The empirical results lack significance.\n\nWe believe that the main takeaway from the toy gaussian experiments is that the MAX approach is highly robust against aggressive hyperparameter tuning strategies, and generally converges faster than other methods with theoretical guarantees.\n\nAlthough the NAT method performs well when it converges, it has shown occasional instability and is costlier per-iteration.\n\n> In VAE experiments, the UB approach is not significantly better than the VR approach in some cases.\n\nIt is true that the UB approach does not always show a significant advantage over the VR approach in our VAE experiments. However, we believe that this study offers useful insights. As you pointed out, the choice of $\\alpha$ makes a difference in the empirical results, which is quite exciting as the idea behind the use of alpha-divergences in variational inference is to overcome some limitations of the traditionally used KL divergence which can be problematic on certain datasets. We observe that a proper tuning of $\\alpha$ can greatly improve the model’s performance, though the optimal value seems to depend on both the gradient estimator and the dataset. A thorough analysis of the phenomena at play is beyond the scope of this paper, but is certainly a compelling question for future research.\n\n> The bias issue is not analyzed theoretically because “biased gradient estimators hinder any theoretical study”.\n\nAnalyzing the MAX approach theoretically presents two main difficulties. First, if it converges, it converges to a minimizer of an approximation of the alpha-divergence, rather than the true alpha-divergence itself. Second, this approximation is hard to characterize, since it amounts to finding and analyzing a function whose gradient is $\\widehat U / \\widehat V$.\n\n> Is it possible to compare the biased and unbiased algorithms on a relatively simple example like gaussian distributions or two-mixture of gaussians?\n\nIn the experiments section, both the biased and unbiased algorithms are compared in the case of a Gaussian variational family with Gaussian, mixture of Gaussian and Cauchy targets (see also Appendix C). The biased algorithm is referred to as MAX, while the unbiased algorithm is UNB.\n\n> If assumption (H3) is not satisfied, do we still have geometric convergence? If not, what would be the convergence rate?\n\nWe need assumption (H3) to obtain the geometric rate of convergence. Lemma 2 (found at the beginning of Appendix A.1) establishes that, as long as the sequence $(\\eta_t)$ remains bounded, at least one fixed point of the mapping $\\mathcal M_{\\gamma}$ will be a limit point of this sequence. However, evaluating whether this limit point is the definitive limit and determining the convergence rate without (H3) is beyond the scope of this study.\n\n> UB achieved relatively worse results for $\\alpha \\in (½, 1)$, which is the interval where assumption C2 could be satisfied. Could the authors provide a heuristic understanding of this phenomenon?\n\nWe believe that the phenomenon at play is quite complex and depends mostly on the dataset, since opposite behaviors are observed between CIFAR10 and CelebA. Moreover, as we already discussed, assumption (C2) might be subject to relaxation, thus it may not fully explain the deeper reasons behind the observed convergence behavior.\n\n> Does [the monotonicity property] contribute to the empirical algorithm analysis? Is it beneficial to solving the instability problems for large step sizes?\n\nEmpirically, we can not guarantee the monotonicity property unless an accept-reject step is integrated into the procedure. The main idea behind the analysis of the sample-based algorithm is to use the mean parameterization of the exponential family to write it as a Robbins-Monro procedure, hence we do not leverage the monotonicity property seen in the exact setting. However, the fact that the exact versions of MAX and UNB enjoy this property may explain why these two approaches appear to be quite stable and follow direct paths to minimizers. This behavior is illustrated in Figure 1, and similarly observed in the mixture of Gaussian and Cauchy cases. We can include additional figures in Appendix C to further exemplify this matter.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oqubqtIMqX", "reviewer": "Reviewer_Ntw7", "summary": "The paper studies the converge properties of an optimization algorithm, used to minimize the alpha divergence when the variational approximation belongs to the family of exponential distributions. The optimizer is a modification of an existing method and its performance is competitive with existing methods (without being stronger). However, the new approach enjoys theoretical guarantees.", "questions": "I'll use this for minor comments and clarification questions.\n\n- instead of (or in addition to) exclusive and include KL, write KL(q||p) and KL(p||q)\n- Figure 1 and Figure 2 are hard to read. In Figure 1, indicate the optimum. Could the colors for the objective function be more distinct than the colors of the trajectories? \n- Figure 2: font size should font size of text. Could the box plots be replaced with trajectories and shaded intervals?\n- line 314: fix the reference to eq:vrbound\n- Define the IWAE and VAE baselines. Does this simply correspond to minimizing KL(q||p) and KL(p||q)? Is this using the variational family as prescribed in line 275?", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper is line with recent publications on proving the convergence of VI when minimizing the KL-divergence. Obtaining similar results for the alpha-divergence is a natural next step. The main contribution of the paper is theoretical, and I find the results convincing (although I did not read the appendix in detail). \n\nThe theory motivates a revision to existing methods, which seems to be competitive. Minimizing alpha-divergence is notoriously difficult and I appreciate the paper's discussion of algorithms to do so, and demonstration on some examples.", "weaknesses": "The paper lacks clarity and is difficult to read.\n\nFirst, the notation is heavy. Two ways to address this might be to introduce a table of notation and to clearly delineate definitions as formal statements. I'm guessing this wasn't done because of space constraints, but this could be a good use of the additional page for accepted papers.\n\nIn Section 5, I was puzzled by the assumption that $q_\\eta(\\cdot \\mid {\\bf x}) = q_\\eta$, that is the factor does not depend on ${\\bf x}$. Does this mean that the encoder maps each image to the same latent representation? It seems like the VAE would then be useless, at least for the task of data compression. This also makes me question the results of Section 6. I'm guessing that while the VAE fails to do any meaningful compression of existing images, it may still generate convincing new images. I'd like to see such images (potentially in the appendix), as a supplement to Table 1, and a simple check that the model is trained in a useful manner.\n\nOnce the authors clarify this point, I can adjust my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your careful reading and constructive comments on our submission. Below, we respond to each of your questions, hoping to clarify any uncertainties.\n\n> The paper lacks clarity and is difficult to read.\n\nWe are sorry to hear that despite our best efforts to make the paper and the notation as clear and readable as possible, you encountered some difficulty in reading our work. We acknowledge that the notation in the paper is dense and will do our best to improve the overall readability by following your advice.\n\n> In Section 5, I was puzzled by the assumption that $q_{\\eta}(\\cdot | x) = q_{\\eta}$, that is the factor does not depend on $x$. Does this mean that the encoder maps each image to the same latent representation?\n\nThe assumption $q_{\\eta}(\\cdot | x) = q_{\\eta}$ in Section 5 was used only to illustrate a specific point: when $p_{\\theta}$ is fixed and the decoder does not depend on $x$ (i.e., $p_{\\theta} = p$ and we only update $\\eta$), the exact gradient of the VR bound equals (up to a negative factor) $\\mathcal R(\\eta) - \\partial A(\\eta)$. This was meant to show that, in this particular scenario, an iteration of the algorithm discussed in Section 3 corresponds to a gradient ascent step on the VR bound. This assumption is not representative of the VAEs' training process, which uses the gradient estimators (17) and (18) with an Adam optimizer. Specifically, estimator (18) is used for VR and estimator (17) for UB. We will reorganize this section to clarify these points.\n\n> Figure 2: font size should be font size of text. Could the box plots be replaced with trajectories and shaded intervals?\n\nLike you suggest, we originally intended for Figure 2 to show full trajectories with shaded intervals, but the result was cluttered and difficult to interpret due to the overlapping of five lines and their respective intervals. Additionally, the trajectories were quite noisy, although Polyak-Ruppert averaging could mitigate this specific issue. We will make further efforts to improve the clarity of the plot.\n\n> Define the IWAE and VAE baselines. Does this simply correspond to minimizing KL(q||p) and KL(p||q)?\n\nYes, IWAE is obtained when $\\alpha = 0$ and VAE corresponds to $\\alpha = 1$.\n\n\\\nWe extend our gratitude for your careful review and insightful feedback, and we hope our responses have effectively addressed your questions and concerns. We acknowledge the issues you noticed in the writing of our paper and will make sure to fix them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kt4ENV4KIp", "reviewer": "Reviewer_Hziy", "summary": "This procedure studies the asymptotic properties of a variational algorithm that ensures a monotonic decrease in the alpha-divergence.  Specifically the authors investigate behavior in the setting where the variational distribution belongs to the exponential family of distributions.  In this setting, and when a key integral can be analytically calculated, the the authors show monotonic convergence at a geometric rate.  For scenarios where a key integral cannot be calculated explicitly, the authors show an unbiased empirical approach.  Furthermore the authors show that this empirical approach enjoys almost sure convergence to a local minimizer of the alpha-divergence.  Finally, the paper presents, both, simulated and real-data experiments to support the theoretical claims.", "questions": "See \"Weaknesses\" section.", "limitations": "The authors do not explicitly state limitations of the proposed methodology.  Nevertheless the authors do make explicit assumptions of the analysis.", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is an very well-written paper and clear despite its high level of mathematical rigor.  The authors consider an important problem of alpha-divergence minimization, which has broad impacts in the ML community.  The convergence analysis and unbiased empirical minimization algorithm appear novel and interesting.", "weaknesses": "The paper does not contain any major weaknesses that I could find, beyond limitations of the analysis.  These limitations include the specific assumption of an exponential family variational distribution, as well as the stated assumptions H1-H4 and C0-C3.\n\nBelow are some detailed comments:\n* L25 : The inclusive KL divergence should be $D(p||q)$.\n* L69 : Shouldn't the posterior be proportional to the joint $p(x,y)$ rather than the marginal $p(y)$?\n* L80 : \"Assuming the argmax is uniquely defined at each iteration\" it is unclear whether this is a reasonable assumption\n* Eq. (5) : Notation $(\\partial A)^{-1}(\\mu)$ undefined\n* L108 : Is it obvious that $\\mathcal{R} = U/V$?  \n* L196 : Notation $\\circ$ undefined (is this a composition?)\n\nPerhaps the biggest weakness is Sec. 5.  This section falls a little flat as it concludes \"we both obtain a biased and unbiased algorithm\" without making those algorithms explicit or obvious (at least to me).  Also, it is unclear how strong an assumption it is that the exponential family does not depend on $x$ as in $q(\\cdot \\mid x) = q$.  In what reasonable scenario would the encoder be independent of the input?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your meticulous review and kind feedback. Below, we provide detailed responses to each of your questions, aiming to further clarify our work and address any remaining uncertainties.\n\n> *Diverse remarks on notation* + L196 : Notation $\\circ$ is undefined (is this a composition?)\n\nYes, $\\circ$ denotes a composition. We thank you for carefully reading our paper and will make sure to correct these mistakes.\n\n> L69 : Shouldn't the posterior be proportional to the joint $p(x, y)$ rather than the marginal $p(y)$?\n\nIndeed, we can take $p(y) = p_{\\theta}(x, y)$. The dependency in $x$ is dropped for notational ease.\n\n> L80 : \"Assuming the argmax is uniquely defined at each iteration\" it is unclear whether this is a reasonable assumption.\n\nAs you point out, this assumption can be challenging in the general case, but when the variational family is an exponential family as in (H1), the argmax in (1) is uniquely defined by $(1 - \\gamma) \\partial A(\\eta) + \\gamma \\mathcal R(\\eta)$ if and only if this quantity is in $F$. This is why the choice of $\\gamma$ is so critical: if $\\mathcal R(\\eta)$ is not in $F$, we must take $\\gamma$ small enough to ensure that the next iterate will be valid. Under (H1), the set $F$ is open and convex, which implies two things. First, by convexity, if $\\gamma_0 \\in (0, 1]$ is a valid choice, then all $\\gamma \\in (0, \\gamma_0]$ are also valid choices. Second, by openness, it is always possible to find such a $\\gamma_0$.\n\n> L108 : Is it obvious that $\\mathcal R = U / V$?\n\nNote that $V$ is the normalizing constant that makes $q_{\\eta}^{\\alpha} p^{1-\\alpha} / V$ a probability density function. By linearity of the integral, $U / V$ is the expectation of the measurable function $S$ under a distribution with density $q_{\\eta}^{\\alpha} p^{1-\\alpha} / V$, i.e. $\\check\\varphi_{\\eta}^{\\alpha}$. This explains why $\\mathcal R = U / V$.\n\n> [Sec. 5.] falls a little flat as it concludes \"we both obtain a biased and unbiased algorithm\" without making those algorithms explicit or obvious. Also, it is unclear how strong an assumption it is that the exponential family does not depend on $x$ as in $q = q(\\cdot | x)$.\n\nThe assumption $q_{\\eta}(\\cdot | x) = q_{\\eta}$ in Section 5 was used only to illustrate a specific point: when $p_{\\theta}$ is fixed and the decoder does not depend on $x$ (i.e., $p_{\\theta} = p$ and we only update $\\eta$), the exact gradient of the VR bound equals (up to a negative factor) $\\mathcal R(\\eta) - \\partial A(\\eta)$. This was meant to show that, in this particular scenario, an iteration of the algorithm discussed in Section 3 corresponds to a gradient ascent step on the VR bound. This assumption is not representative of the VAEs' training process, which uses the gradient estimators (17) and (18) with an Adam optimizer. Specifically, estimator (18) is used for VR and estimator (17) for UB. We will reorganize this section to clarify these points.\n\n\\\nThank you once again for your detailed review and supportive comments. We hope that our responses have satisfactorily addressed your questions and concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6P6GdKSuKf", "reviewer": "Reviewer_FQDG", "summary": "This paper focuses on the theoretical study of variational inference using alpha-divergences using exponential family. This is an important problem in the field of variational inference. Specifically, this work proves a geometric convergence rate for the algorithm proposed in [9]. Moreover, this paper also proposes an alternative optimization algorithm and its convergence guarantees are provided. The proposed algorithm finds applications in VAE.", "questions": "1. Typo at line 81, ''negative'' $\\to$ ''positive'' since you drop the negative $\\frac{1}{\\alpha(\\alpha-1)}$ term.\n\n2. What is the technical difficulty to derive a convergence result similar to Theorem 1 for the proposed algorithm?", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Asymptotic convergence rate of the algorithm proposed in [9] is derived in this work. Moreover, this paper proposes an alternative unbiased algorithm with convergence guarantees.\n\n2. Assumptions (H1)-(H3) follow by reasonable explanations.\n\n2. Applications of the proposed algorithm to VAE are provided.", "weaknesses": "1. It only considers the variational family to be exponential models.\n\n2. The convergence is proved for the proposed algorithm in the asymptotic sense.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and honest feedback. Below, we provide detailed responses to your second question. The two weaknesses you pointed out are adressed in our global rebuttal. We hope that our responses will clarify the issues you raised.\n\n> What is the technical difficulty to derive a convergence result similar to Theorem 1 for the proposed algorithm?\n\nIt does not present any major difficulty and can be done by following the exact same steps as in the proof of Theorem 1. We would get a similar result, with a slightly slower rate however: the $\\gamma$ in the convergence speed would be replaced by $\\gamma V(\\eta_*)$, and we have $V(\\eta_*) \\leq 1$ by Jensen’s inequality. It is easy to gain some intuition for why this is the case: under the mean parameterization, the update is changed from $(1 - \\gamma) \\mu + \\gamma \\mathcal R$ to $(1 - \\gamma V) \\mu + \\gamma V  \\mathcal R$, so we are still performing a convex combination of $\\mu$ and $\\mathcal R$ but with a less aggressive coefficient, thus yielding a slower rate of convergence.\nThis also provides a heuristic as to why UNB moves slower at the start of the experiments: when the variational parameter is poorly chosen, $V$ is closer to $0$, which caps the progress that can be made at each step when the only values allowed for $(\\gamma_t)$ are between $0$ and $1$.\n\n\\\nWe are grateful for your careful review, we hope our responses have adequately addressed your questions and provided further insights.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We deeply thank the reviewers for their careful and detailed reviews of our manuscript. We are grateful for their constructive feedback and for offering us an opportunity to improve our work. Below, we provide responses to some points that have been raised by multiple reviewers. We hope that this discussion will satisfactorily address any concerns the reviewers may have.

> (Reviewer pDqJ) The main theoretical analysis focuses only on exponential families.

> (Reviewer FQDG) It only considers the variational family to be exponential models.

We chose to focus on exponential families as they offer a convenient setting and enjoy interesting theoretical properties. First, they ensure the existence of a unique solution to the argmax problem in (3), given that $\gamma$ is chosen appropriately (this is not a restrictive assumption). They also allow us to state (H3) in an understandable way, rather than through obscure integral conditions. Finally, we believe that the principles established in this restricted setting offer insights and intuition about the problem at hand, potentially serving as a foundation for future analyses in more general contexts.

> (Reviewer pDqJ) For Assumption (H3) in section 3, it remains unclear when the mapping $\mathcal M_{\gamma}$ is a contraction. It would be beneficial to further verify this for $q_{\eta}$ and $p$ in the case of the exponential family.

> (Reviewer FQDG) The convergence is proved for the proposed algorithm in the asymptotic sense.

In Proposition 1, we explore the contraction property of the mapping $\mathcal M_{\gamma}$. The analysis reveals that, under assumption (H3), this mapping is a contraction in a neighborhood of the parameter $\eta_*$. It can be noticed that when $\mathcal Q$ is an exponential family as in (H1) and the target $p$ belongs to $\mathcal Q$, i.e. $p=q_{\eta_*}$, assumption (H3) is always verified. Indeed, we can show that in this setting, the only fixed point (c.f. Lemma 1) is the target parameter $\eta_*$, and that we have $\rho_* = \alpha < 1$. 

Determining the size of the neighborhood in which $\mathcal M_{\gamma}$ is a contraction is challenging in the general case. However, in the simpler case where $Q$ is a real Gaussian family with fixed variance and $p$ is Gaussian (not necessarily with the same variance), this condition is verified for any mean in $\mathbb{R}$.


Again, we thank all reviewers for their comprehensive and constructive feedback. We hope that our responses have adequately addressed your questions and concerns, and that our work has been significantly improved as a result.

Meta-review: This paper provides convergence results for two algorithms for minimizing the alpha-divergence between an exponential family and a target distribution. While reviewers were overall positive and I think this paper has some worthwhile ideas, the paper also has some weaknesses. 

The first algorithm analyzed (essentially that in Equation 5) is an algorithm that was previously proposed and proven to decrease monotonically. The algorithm is also in practice unimplementable, as it relies on expectations that are not available for practical target distributions. The contribution of this paper is to give a rate of convergence (theorem 1).

The second algorithm analyzed (essentially that in equation 10) seems to be novel. (And may be valuable). The analysis for this algorithm (theorems 2 and 3) give something similar to a \sqrt{\log(t)/t} rate. This algorithm is implementable. Convergence is proven to some unknown parameter vector μ_* which is in a set of critical points of the alpha divergence. This weakness is exhibited in the fact that the results in Theorem 2 and Theorem 3 make no reference to the minibatch size K, despite the fact that we hope that algorithm to become more accurate when K is larger. The experimental evidence for the algorithm is quite limited.

A weakness of the paper is the form of the assumptions being made. Ultimately, the only sources of variability in the problem are the exponential family chosen, the target distribution, and algorithmic choices (e.g. step sizes). So ultimately, any assumption must be about one of these things. But the form in which the assumptions are given makes it extremely difficult to understand for which target distributions the algorithms are claimed to work. For example, where does the assumption on the target distribution lie in Theorem 1? I asked some reviewers, but could not obtain an answer. I believe the answer is that H3 makes reference to φ^α_η which is defined in equation 2 to be a geometric mixture of the variational family and the target distribution. But what does this mean? For what distributions would this be satisfied? This is not discussed.

Another weakness of the paper is a rather "severe" use of notation, with various symbols seemingly either undefined or at least requiring the reader to hunt through the previous pages to find the definition. If the paper is technical fine, but it would be a great service to the reader in the main assumptions and results to make this clear in the statement, e.g. add a few lines of setup to the theorem clarifying what algorithm is being used, and the previous lines where symbols are defined. Sometimes the notation is quite obscure, e.g. in Eq. 13, Cov_η is covariance with respect to the variational family defined by *parameters* η, whereas Cov_{φ^α_η} is a covariance with respect the *distribution* φ^α_η.

In conclusion, while algorithms for alpha divergences are of fundamental interest and this paper contributes valuable new ideas, I believe that the relative lack of clarity of the paper made some of its weaknesses obscure to the readers. After consultation with reviewers and the SAC, I am recommending that the paper be accepted, but I urge that the paper be revised to increase clarity to maximize the potential impact.
Decision: Accept (poster)