Mixture weights optimisation for Alpha-Divergence Variational Inference
Authors: Kamelia Daudel
randal douc
Venue: NeurIPS 2021 Poster (2021)
Keywords: Variational Inference
Alpha-divergence
Mixture Models
Entropic Mirror Descent
R\'{e}nyi's Alpha-divergence

Abstract: This paper focuses on $\alpha$-divergence minimisation methods for Variational Inference. More precisely, we are interested in algorithms optimising the mixture weights of any given mixture model, without any information on the underlying distribution of its mixture components parameters. The Power Descent, defined for all $\alpha \neq 1$, is one such algorithm and we establish in our work the full proof of its convergence towards the optimal mixture weights when $\alpha <1$. Since the $\alpha$-divergence recovers the widely-used forward Kullback-Leibler when $\alpha \to 1$, we then extend the Power Descent to the case $\alpha = 1$ and show that we obtain an Entropic Mirror Descent. This leads us to investigate the link between Power Descent and Entropic Mirror Descent: first-order approximations allow us to introduce the R\'{e}nyi Descent, a novel algorithm for which we prove an $O(1/N)$ convergence rate. Lastly, we compare numerically the behavior of the unbiased Power Descent and of the biased R\'{e}nyi Descent and we discuss the potential advantages of one algorithm over the other.

Mixture weights optimisation for Alpha-Divergence Variational Inference
Kamélia Daudel Randal Douc
1: LTCI, Télécom Paris. Institut Polytechnique de Paris, France 2: Department of Statisties, University of Oxford, United Kingdom 3: SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, France
Abstract
This paper focuses on a-divergence minimisation methods for Variational Inference. We consider the case where the posterior density is approximated a mixture model and we investigate algorithms optimising the mixture weights of this mixture model by a-divergence minimisation, without any information on the underlying distribution of its mixture components parameters. The Power Descent, defined for all a * 1, is one such algorithm and we establish in our work the full proof of its convergence towards the optimal mixture weights when a < 1. Since the a-divergence recovers the widely-used exclusive Kullback-Leiblee when a -> 1, we then extend the Power Descent to the case = and show that we obtain an Entropic Mirror Descent. This leads us to investigate the link between Power Descent and Entropic Mirror Descent: first-order approximations allow introduce the Rényi Descent, a novel algorithm for which we prove an O(1/N) convergence rate. Lastly, we compare numerically the behavior of the unbiased Power Descent and the biased Rényi Descent and we discuss the potential advantages of one algorithm over the other.
Introduction
Bayesian Inference involves being able compute sample from the posterior density. For many useful models, the posterior density can only be evaluated up to normalisation constant and we resort to approximation methods.
One major category of approximation methods is Variational Inference, a wide class of optimisation methods which introduce simpler variational family Q and use it to approximate the posterior density (see for example Variational Bayes [ 2 and Stochastic Variational Inference (3). The crux of these methods consists in finding the best approximation of the posterior density among the family Q in the sense of a certain measure of dissimilarity, most typically the exclusive Kullback-Leibler divergence.
However, the exclusive Kullhack-Leiibe divergence is known to have some undesirable properties, e.g. posterior variance underestimation. difficulty to capture multimodal posterior densitics 4 5 6
As a consequence, the a-divergence [ ( and Rényi's 's a-divergence [9] [0] have gained lot of attention recently and are presented more general alternatives that permit bypass the issues of the exclusive Kullback-Leibler when a < II
12
B
14
15 16
18 19
21
Noticeably, [99 introduced the (a, -d-deeeent, general family of gradient-based based algorithms that approximate the posterior density by mixture model and that are able optimise the mixture weights of this mixture model by a-divergence minimisation. The benefit of these types of algorithms that (i) they expand the traditional parametric variational family to better capture complex (e.g.
Corresponding author; Vaudll @ssats. ax..uuk
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

multimodal) posterior densities (ii) they allow, in an Sequential Monte Carlo fashion 23] to select the mixture components according their overall importance the set of components parameters. without needing to know the distribution of the mixture components parameters. The (a, descennt step then paired up with an Exploration step that acts on the mixture components parameters, so that a complete algorithm is obtained by alternating between these two steps [9]
The key algorithm [19 is the Power Descent, which sets [(u) = [aa - + + 1]/((1=) with a + 1 and 1) > 0 in the (a, dessennt Indeed, when < 1 and the dimension increases, numerical experiments in [9] show that the Power Descent outperforms the Entropic Mirror Descent (a classical algorithm from the optimisation litterature corresponding to [(v) e 1/ with 1/ 0).
Nonetheless, the global convergence of the Power Descent algorithm when a < 1, as stated in [19 Theorem 4], assumes the existence of the limit and does not provide conditions that satisfy this assumption. Furthermore, even though the convergence towards the global optimum is derived, there is no convergence rate available for the Power Descent when < 1. After recalling the basics of the Power Descent algorithm in Section 2 we make the following contributions in this paper:
we prove the full convergence of the Power Descent algorithm towards the optimal mixture weights when a < 1 (Theorem 2)..
In Section 3
Since the a-divergence becomes the traditional exclusive Kullback- when a + 1, we next investigate the extension of the Power Descent to the case a = 1 in Proposition frrm Section and we obtain that the Power Descent recovers an Entropic Mirror Descent performing exclusive Kullback- minimisation.
We further study the connections between Power Descent and Entropic Mirror Descent by con- sidering first-order approximations, in the hope of finding an algorithm close to the Power Descent and for which we can prove a convergence rate when
1. As result, we are able go beyond the (a, -'d-ssceett framework from [19] by introducing an algorithm closely-related to the Power Descent, that is proved to converge at an (1/N)) rate when a < 1 as a consequence of Theorem] from Section 4 We call this algorithm the Rényí Descent due to the link we establish between the Rényi Descent and Entropic Mirror Descent steps applied to the Variational Rényi bound [12].
Finally, we run some numerical experiments in Section 5 to compare the behavior of the Power Descent and of the Rényi Descent altogether, before discussing the potential benefits one approach over the other.
2 Backkround
This section reviews the Power Descent algorithm from 199 and details how it is applied mixture weights optimisation in a Variational Inference context.
Notation and problem statement. Let (Y, D) be measured space, where is o-finite measure on (Y,)). Assume that we are given some observed data 9 generated from a probabilistic model p( & (y) that is parameterised by a latent variable y E Y. Further assuming that y is drawn from certain prior Poly), the posterior density of the latent variable y given the data D is then defined by:
ply) 2) = p(y, 9) Po(y)p(1) (gly)
p ())
p(2)
where the normalisation constant p( Jy Po(y) (()))((()))))(e))
is called the marginal likelihood or model evidence. For many complex models arising in Bayesian Inference, we do not know how to directly sample from the posterior density, nor do we know the value of the marginal likelihood.
To remedy this problem, the Power Descent is a Variational Inference algorithm that introduces a certain measurable space (T, and offers to approximate the posterior density by variational family Q of the form
e e-(w-> y J [(aa))((..) MEM M
where M is a convenient subset of M₁(T), the set of probability measures on (T, T). In doing so, extends the usual parametric variational family
t by k(0, 0 8 T

---

since it amounts to putting prior over the parameter & in the form of a probability measure. In particular, one strong motivation behind the variational family that it recovers mixture models as special case. To see this, let J E N e = (0),..., ,,, E T and define the simplex of IR by:
AE) AIE { A A (AI.......
Vj E {I....,J} X3 and Σ Ny =
(2)
Observe then that when j chosen as weighted sum of Dirac measures, i.e. μ = E 2j00, with SJ, the variational family (
becomes
H Σ
),1(0,,0) 1=1
X A AES
(3)
which corresponds to the class of mixture models parameterised by the mixture weights À and with fixed mixture components parameters O Here, the j-th component y y k(0,, y) could for example be a multivariate Gaussian density with mean O, and known covariance matrix, but one strength of kernel
the Power Descent is that it not limited to a specific choice for k. We will refer density thereafter and we now introduce some more notation, before stating the optimisation problem solved by the Power Descent.
Denote by the probability measure on (Y,3) with corresponding density p(-19) with respect to y and by K (0, A) ~ SA ())()) the Markov transition kernel T x 3 with kernel density k defined on T x Y. For all μ E (T) and all y Y, denote pk(y) = . ((0 y) and let μK denote the probability measure with density μk with respect to v. Then, letting E IR and assuming that µ and P are both absolutely continuous with respect to v, the a--iiergence between μK and IP (extended by continuity to the cases a = and a = as in 24 is given by
Da (a(k J
pk(()) p(y)9) plyl (ddy),
Fa fa
where Sa the convex function on (0, +00) defined by so(u) u log(u), fi(u) u log(u) and (u) a(a-1) [ua 1 alu 1)] for all a E R 1).
Letting + 1, the Power Descent seeks to solve the Variational Inference optimisation problem
inf Da (PK]IP))
PEM
meaning that for μ = E/= E dyd (with corresponding approximating family 3)) it seeks to optimise À by o-divergence minimisation i.e. to select mixture components according to their overall importance in the set of components parameters. More generally, the Power Descent aims at solving
inf V. (p: EEM
(5)
= where p is any measurable positive function on (Y.)) and where for all μ E MT V (p:
Jy fa (Juk(y)/p()) (dy)) Crucially, (4) can be reframed as an instance of (5) that sets p p(. in (5) and that thus does not involve the unknown normalising constant p(9) anymore (see
Appendix I) In the following, the dependency on in Va may be dropped throughout the paper for notational ease when ambiguity occurs and we next present the Power Descent algorithm.
The Power Descent algorithm. Let * 1. Given an initial probability measure /a E M1 (T) such that Va (411) < oo, > O and 6 such that (a 1)k 2 O. the Power Descent algorithm introduced in 19 is an iterative scheme which builds the sequence of probability measures (H,))eN
An+1 (pn)
N
(6)
where for all μ (T),,, the one-step transition μ + I. (1) is given by Algorithm I and where for all E Dom,, T(v)= (a - 1)u + 1ja/(1-a) [and Dom, denotes an interval of IR such that for all 0 E T and all M1 (T), by.a (0) + and u(bn.a)) + Doma].
3

---

Algorithm 1: Power descent
one-step transition (T(e) = I(a 1)v + 1)
1. Expectation step: bj (0) J, k(0,
pu((y)
k(0,)).fa y)fa
p(dy)
p(y)
2. Iteration step: [d()) v(dd) I(b (8) n.)
(L + k))
The Power Descent is known to have gradient-based structure since 0 + bp.a (f) acts as the gradient of μ -> (a) and 7 plays the role of learning rate [I9]. Furthermore, a remarkable property of the Power Descent is that under (A1) as defined below
(A1) The density kernel & on T x Y, the function p on and the o-finite measure v on (Y))) satisfy, for all (0, T Y, k(0, y) > O, p(y) >> 0 and Jy p(y)v(dy) < co.
the Power Descent ensures monotonic decrease in V. at each step for all ry E (0, (this result is special case of [19] Theorem 1] with ((v) I(a 1) + 1]w/(1->) that is recalled in Theorem 4 of Appendix A.2 for the sake completeness). Under the additional assumptions that x > and
sup (ET,)EM.(T)
1b < oc and va((1)) < 00,
the Power Descent is also known to converge towards its optimal value at an O(1/N) rate when [19. Theorem 3]. On the other hand, when < 1, the convergence towards the optimum as written in holds under different assumptions including
(A2) is compact metric space and T is the associated Borel o-field;
(ii) for all y E Y, + k(0, y) continuous;
(iii) we have Jy supEET k(0,,) v) x supPrET
[((0 v) v(dy) < co.
p(y)
If a = 0, assume in addition that [y suPøeT
log
k(e.y) p(9)
p(y)u((y) < co.
so that [19] Theorem 4], that is recalled below under the form of Theorem of the Power Descent algorithm towards the global optimum.
states the convergence
Theorem 1 (II9 Theorem 4]). Assume (A1) and (A2) Let a < and let K & 0. Then, for all a E (T). (a(u) < oo and any n > satisfies O p(T((b + x)) <0. Further assume that 17 E (0. 1] and that there exist μt, μt E (T( such that the (well-defined) sequence (pn)new
I
defined by (6) weakly converges to μ* as n - cc. Finally, denote by M1,a in (T) the set of probability
measures dominated by μr. Then the following assertions hold
(i) (1 (PA)))EE is nonincreasing.
(ii) 1² is a fixed point of Ta.
(ii) V. (n²) infçeM..,, V (C)
The above result assumes there must exist pi, p* (T) such that the sequence (Pn)nEN defined by (6) weakly converges to 1² as n + oc, that is it assumes the limit already exists. Our first contribution consists in showing that this assumption can be alleviated when μ is chosen a weighted sum of Dirac measures, that is when we seek to perform mixture weights optimisation by a-divergence minimisation. In doing so, we provide additionnal theoretical justification behind the use of the Power Descent when & < 1, which is crucial in practice for reasons already outlined in the introduction and further detailed for the particular case of the Power Descent in Appendix A.3.
3 Convergence of the Power Descent algorithm in the mixture case
Before we state our convergence result, let us first make two comments on the assumptions from Theorem that shall be retained in upcoming convergence result.

---

A first comment is that (A1) is mild since the assumption that p(y) > 0 for all y E Y can be discarded and is kept for convenience 19 Remark 4]. A second comment that (A2) is also mild and covers
(7) as it amounts to assuming that 0 ~ (,,a(()) (0) and ~> V, (,( are uniformly bounded with respect
to ja and A. To see this, we give below an example for which (A2) satisfied.
Example 1. Consider the case Y IRd with a (0,1). Let
and les B(0,r) IRd.
Furtheremore, let Kn, be a Gaussian transition kernel with bandwidth and denote by ky its associ- ated kernel density. Finally, let p be mixture density of two d-dimensional Gaussian distributions multiplied by a positive constant such that p(y) = e x (0.5.[yy 07, , Ia) + 0.5N 07. La)] for all y E where 07, 02 E T and La is the identity matrix. Then, [ holds (see Appendix B.
Next, we introduce some notation that are specific to the case of mixture models we aim at studying in this section (and which, as seen previously, corresponds choosing μ as a weighted sum of Dirac measures in ① yielding (3). For this purpose, let J E N let θ =
02) E TV be fixed,
let * > 0, be such that (a 1)x > and recall that the simplex of IR defined in 2 is denoted by Sy. We define Sj = {A SJ Vj E JJ. X, > 0} and for all X SJ and we also define μa,e Σラ Ayōu,- Now letting (Ka)neN be defined by μI = μa,e and (6) an immediate induction yields that for every n E N", , An can be expressed as H4, = Σj= =j== ^I.nō0, where An (AIm....,AA,, E Sy satisfies the initialisation A = À and the update formula:
A=+1 Taaisi (An) E
(8)
with
>g I(a 1)(bpa a. (0,) +k) =|
7 (A) =
A
t=1 de I(a 1a (0e) x) 1| 18j
Finally, let rewrite (A2) in the simplified case where μ Ma,0, which gives (A3) below.
(A3) (i) For all y E Y,, + k(0,y) is continuous;
(ii) we have (
(((,r.a)) ply)
max k(0,, 3) x max EE'S
v(dy) < co.
If 0, we assume in addition that
max 18SJ
log
k(0;,y)
p(y)u(dy) < co. p(3)
We then have the following theorem, which establishes the full proof of the global convergence towards the optimum for the mixture weights when <
Theorem 2. Assume (A1) and (A3) Let < l (01....,0) 0.)) E T be fixed and let r. be such that K. & Then for all E SJ. V (a(ux,)) < oc and for any 1) >> 0 the sequence (An)nEN defined by A E Sy and (8) is well well-de If addition (A1. E S (0, 1] and [K(...))..... K) are linearly independent, then
(i) (Ho. is nonincreusing,
(ii) the sequence (An)NEN- converges to some X, E S, which is a fixed point of Imixt, (iii) V. (ux,,)) = inf infves, (a
The proof of this result builds Theorem T and Theorem 4 of Appendix A.2 and is deferred to Appendix B.2. Notice that since V depends on À through yal K, an identifiably condition was to be expected order achieve the convergence of the sequence (An) in Theorem 2 Following Example | this identifiably condition notably holds for J & d under the assumption that the O,,..., 0, are full-rank.
We thus have the convergence of the Power Descent under less stringent conditions when a < 1 and when consider the particular case of mixture models. This algorithm can easily become feasible for any choice of kernel K by resorting to an unbiased estimator of (..,,,..( ( in the update formula (8) (see Algorithm 3 of Appendix B.3).
Nevertheless, contrary to the case a > we still do not have a convergence rate for the Power Descent when < 1. Furthermore, the important case 1, which corresponds to performing
5

---

exclusive Bullback-Leibler minimisation, is not covered by the Power Descent algorithm. In the next section, we extend the Power Descent the case a 1. As we shall see, this will lead to investigate the connections between the Power Descent and the Entropic Mirror Descent beyond the (a, de-deeent framework from [19]. As result, we will introduce novel algorithm closely-related to the Power Descent that yields an O(1 (N) convergence rate when er,e and a < 1 (and more
generally when j E M (T) and a + 1).
Power Descent and Entropic Mirror Descent
Recall from Section 2 that the Power Descent is defined for all a & 1. In this section, we first establish in Proposition that the Power Descent can be extended the case a 1 and that we recover an Entropic Mirror Descent, showing that a deeper connection runs between the two approaches beyond the one identified by the (dessett framework of [19]. This result relies on typical convergence and differentiability assumptions summarised in (D1) and which are deferred to Appendix C.I, alongside with the proof of Proposition[] [
Proposition (Limiting case a - 1). Assume (A1) and (D1). Let O and K be such that (a- 1)k 3 Then, for all y E (T( and all continuous and bounded real-valued functions on T, we have that
lim [Zo (1)](h) - (w))))))) a-1
where for all T, we have set
(d) = p(d0)e ((dd))enbyaat (0) (e-enbu)))
and bp.1(0) = - k: (0 log
[uk(y) p(y)
v(dy),
(9)
and where for all Ç (T(,), we have used the notation Çh ft ((d0))())
Here, recognise the one-step transition associated to the Entropic Mirror Descent applied to 1 This algorithm is special case of [19] with T(u) = -ー and a = 1 and as such, it is known to lead to systematic decrease in the exclusive Kullback-Leibler divergence and to enjoy O(1/N) convergence rate under the assumptions that (7) holds and E (0, 1) 19] Theorem 3].
We have thus obtained that the Power Descent coincides exactly with the Entropic Mirror Descent applied to v when a = and we now focus on understanding the links between Power Descent and Entropic Mirror Descent when F 1. For this purpose, let K be such that (a 1)k > O and let us study first-order approximations the Power Descent and of the Entropic Mirror Descent applied to L. when a (0) poa (b)...a) for all 0 E T.
Letting yj > and following (6), we have that the update formula for the Power Descent given by
(n+1(d0) (d0) [(a= 1)(b,... ( (0) + +) 1] (0 and
1|1-a
1-a
n EEN N*
pn pa(((a 1)(bp.... + x) 1 and using the first-order approximation - vr -
(u with
can deduce the following approximated update formula
bpm. (0) Mr, (bp....))
(n+1(dd) pnn (d0)
n N
a a 1 (m (b ) + K + 1/(a - 1)
In addition, the update formula for the Entropic Mirror Descent applied V. can be written as 1 (d0) |., (d0) exp (b,,,, (0) + x)]
n EEN'
(10)
|r, (cxp (n + +x]]
and we obtain similar fashion that an approximated version this iterative scheme is pn+ (d0) = ft, (d0) | [1 1 (b,...a (0) pn (b)...))))
N
As is, the two approximated formulas above do not coincide. This leads us to modify (0) as follows
Hln (d0) exp
(0)
1y (Bun -+
n n N
(11)
bon.o +t+]+1
Nn exp

---

that is. it leads us to move from a constant learning rate 7 in (TO) to an adaptive learning rate k) + +1 in (TT). While this change is subtle, first remark is that [19 only considered a constant learning rate when performing Entropic Mirror Descent steps applied to Va. meaning that we are now deviating from the framework of [9]. A second remark is that we can motivate (TT) by observing that it can be seen as an Entropic Mirror Descent too, but this time applied to the objective function defined for all measurable positive function p YY,),, all ju E (T) and all a E R [0.1)00 by
n' n (a - ()
W a (w:p) (4=
= a(a - 1) log (
+ (a 1)k
(see Appendix C.2 for the derivation of (II based on the objective function (r Since the function VAR can obtained by applying the increasing transformation
a(a - log [a(a - 1)u + a (1- -a) J p(y)u(du) + (ax 1 1)x=
u
to the objective function a. this notably means that our novel Entropic Mirror Descent-based update formula (1I) will aim at solving the initial optimisation problem ( defined in Section 2 (i.e.
minimising V (x:pp) with μ E M). addition, when p = ppp(: _), = and a >> 0 in V [Jz: p), we obtain that minimising V.AR (1:p)) w.r. is equivalent to setting & = juk and maximising the
Variational Rényi bound Lo (g: ②) from 12] w.r.t μa, since for all probability density q.
(1; == 1 log Byg
ply. Φ) ((U)
1-a
As result, we can classify the scheme (TT) Rényi's 's a-divergence gradient-based approach [2].2] This is in contrast with the framework of [19]. that is an a-divergence gradient- based approach and for this reason we call the algorithm given by [II the Rényi Descent.
We have thus obtained that the Entropic Mirror Descent applied to V. does not share the same first-order approximation as the Power Descent, contrary to the newly-introduced Rényi Descent. This might explain why the behavior of the Entropic Mirror Descent applied to Va and of the Power Descent differed greatly when < in the numerical experiments from [19] (indeed, despite their theoretical connection through the (a, d')-deseent framework, the former performs poorly numerically compared the later the dimension increases). This also sparks our interest in studying convergence results for the Rényi Descent.
Strikingly, prove O(1/N) convergence rate towards the global optimum for the Rényi Descent. Letting n' IR, denoting by Doma interval of R such that for all A T and all E (T),
a (0) + 1/(a0 1)
p(bpa) + 1/(a - 1) k² E Dom (a - 1)(a(b + +x) +
(a - (bbb,,))) + k)) + + 16' and
and introducing the assumption on
(A4) For all D AR 1 77aa 1)(v n²) > 0.
we indeed have the following convergence result.
Theorem 3. Assume (A1) and [A4] Let a # 1 and let k. be such that (a - 1)k: > O. Define [B] 10,0 = SupeET. (b + 1/(a - 1)] and assume that oo,,, co. Moreover let Ni E (T be such that (a(u)) < oc. Then, the following assertions hold.
(i) The sequence (um)NEN defined by 1p is well-defined and the sequence (Wa(H)))eNN is non-increasing.
(ii) For all N E N we have
Laa KT(n"llm) + I, "Tai(a - 1)h Va(uN) V Va(1*) &
La.3 1
(12)
N
where p= is such that va(w*) = inf (T) V Va(C), MI (T) denotes the set of probability measures dominated by pn, <L(p)(pp)) log (dy* /du1) du², Ai
v Va(ui) - v (a(")) and L, La,1, La.2 La.3 are finite constants defined in (25).
7

---

Table I: Summary of the theoretical results obtained in this paper compared
[9
Power Descent
Rényi Descent
19
< 1: convergence under restrictive assumptions:
covered
>> 1: O(1/N) convergence rate
This paper < 1: full proof of convergence for mixture weights;
((1/N)
extension to a
convergence rate
The proof of this result is deferred to Appendix C.3 and we present in the next example an application of this theorem to the particular case of mixture models.
Example 2. Let a o 1, K be such that (a 1)xx > 0, JEN θ (01..... B,) E T ll = [ ع + R²] with re' P. In addition, assume that [a-1)k n/, [a-
J-1 Σj= 60, and Dom
[8 |
1 n/(n B bo.a > 0. Then, taking K² = -3 |B| we obtain
(a-1)k
va(uN) (a(u") la 1[[[B] bo.0 + [m])
log J
(ogg oc.a (a 1)x(1 -7/x: B
1
where we have used that K KL( 11m)) & log J. A1 & V22log
J| and that the constants defined
in (25) satisfy La.2 17 aa
1[[[B)oc,a + |x]), L 11²e" ta (a-1j (a-1jk 1/66
18 1B np La,3 e Ta 1jx
(8) no
tyn"
and
L,,1 = (1 - 1/|x -1 | )oe
To put things into perspective, an Entropic Mirror Descent algorithm, the Rényi Descent al- ready enjoys an O(L/VN) convergence rate for the sequence (Va n= (n)) under our Theorem 4.2.)).
assumptions and when vj is proportional to I/vN, N being fixed (see 225
The improvement thus lies in the fact that deriving an (1//N) convergence rate usually requires stronger assumptions on aa [26] Theorem 6.2] that we do not need assume in Theorem B thanks to the specific form of V/ Furthermore, due to the monotonicity property, our result only involves the measure JAN at time while typical Entropic Mirror Descent results are expressed terms of the average N- Σ
Finally, observe that the Rényi Descent becomes feasible in practice for any choice of kernel K by letting fi weighted sum of Dirac measures i.e μ = Mx.e and by resorting to an unbiased estimate of (0)))))) (see Algorithm 4 of Appendix D.1 for details). The theoretical results we have obtained are summarised in Table T and we next move on numerical experiments.
Simulation study
The code for all the subsequent numerical experiments is available at https:///gtthbb com kdaude 1/Mi xwurewee ghtssAlaatttssaaaaaa...
Let the target be a mixture density of two d-dimensional Gaussian distributions multiplied by positive constant such that p(y) = c [0.5.( -sud, Ia) + 0.5N sua, Ia)], where ua the d-dimensional vector whose coordinates are all equal to 1, = 2, c = 2 and Ia is the identity matrix. Let K₁, be Gaussian transition kernel on T ) with bandwidth and let K2, denote its associated kernel density on T x Y. Let J e N' and recall the previously-dese with X E S. and θ E T We consider the variational family described by
notation μr,0 Σ/ 2,89,
Q y (x,oka()) - ENAAuu-- AES,
5=1
Following the reasoning [9), since the Power Descent and the Rényi Descent operate only on the mixture weights X of @ during the optimisation, fully adaptive algorithm can be obtained by alternating T times between an Exploitation step where the mixture weights are optimised and an Exploration step where the mixture components parameters are updated, as written in Algorithm 2

---

Algorithm 2: Complete Dnploitttioo-Exploatatinn Algorithm
Input: p: measurable positive function, a-divergence parameter, yo: initial sampler, K₁: Gaussian transition kernel, T: total number of iterations, J: dimension of the parameter set. Optimised weights À and parameter set e.
Output:
Draw 01. from qo-
for f 1 do
Exploitation step: Set θ (f1......, f.,r). Perform the Power Descent or the Rényi
Descent algorithm and obtain the optimised mixture weights
Exploration step: Perform any Exploration step of our choice and obtain 01,+11..... 01
Many choices of Exploration steps can be envisioned in A We first consider the Exploration step used in [9]: h & J 1/4+44) and the particles are updated by i.d sampling according to μa e Since the Entropic Mirror Descent applied VA a is known to break down when d 16 for this Exploration step [19], this case is good starting point in order to investigate if the novel Rényi Descent suitable alternative to the Entropic Mirror Descent applied V. as d increases.
More precisely, at each time f = 1...P, we perform N transitions of either the Power Descent the Rényi Descent according to Algorithm 3 and 4 of the appendix, in which the initial weights set be [1/J,... ...,.... 1/J], 17 1/// /NN with no >> 0 and M samples of (bix, a (g at each iteration n = N. We take J = 100, M E {100, 1000, 2000), 0.5, 0, rjo 0.3 and go centered normal distribution with covariance matrix 5T.. We let T' = 10, N = 20 and we replicate the experiment 100 times independently for each algorithm.
used in the estimation
The convergence is assessed using Monte Carlo estimates of the Variational Rényi bound from [12. The results for the Power Descent and the Rényi Descent are displayed on Figure [] in which the Entropic Mirror Descent applied to V. is added reference (and additonnal plots in dimension < 16 are provided in Appendix D.2 for the sake of completeness).
Figure 1: Plotted is the average Variational Rényi bound for the Power Descent (PD), the Rényi Descent (RD) and the Entropic Mirror Descent applied to Va (EMD) in dimension d = 16 computed over 100 replicates with 7/0- 0.3 and 0.5 and an increasing number of samples M.
Nimmersion
Timmnction 16,
200n
22100011
We then observe that the Rényi Descent is indeed better-behaved compared to the Entropic Mirror Descent applied to V,, Furthermore, it matches the performances of the Power Descent as M increases in our numerical experiment, which illustrates the link between the algorithms we have established in the previous section.
Note that the Exploration step considered in [9 does not optimise over ② In an SMC fashion, it indeed resamples the interesting regions for (selection) and then applics a Gaussian pertubation (mutation). While we are not trying to discriminate between various choices of Exploration steps in this paper, an important remark is that the choice of the Exploration step in Algorithm 2 will become increasingly important the dimension increases. For this reason, have outlined Appendix D.3 some potentially suitable Exploration steps that optimise over e.
In particular, we also ran our numerical experiments with the Exploration step from Appendix D.3.2. The plots for d = are available in Appendix D.3.2 and we present below the plots for the higher- dimensional setting where d = 100. We observe that the Rényi Descent and the Power Descent do

---

not break down as we increase the dimension up to d = 100 and that they share similar behavior, which reinforces our conclusions.
Figure 2: Plotted is the average Variational Rényi bound for the Power Descent (PD) and the Rényi Descent (RD) in dimension d = 100 computed over 100 replicates with the Exploration Step from Appendix D.3.2, 1/0 0.3 and 0.5 and increasing number of samples M.
Demcrsion 100, u. 2.5 by D...M 100
Demeniion 100, a O.O. 0.5 300
Demors on 200, us C.O. 0.5 2000
3.2
kendtora
252
3/0
223
2/5
Discussion From a theoretical standpoint, no convergence rate is yet available for the Power Descent algorithm when 1. An advantage of the novel Rényi Descent algorithm then that while being close to the Power Descent, it also benefits from the Entropic Mirror Descent optimisation literature so that ((1/VN) convergence rates hold, which we have been able to improve to (1//N) convergence rates.
A practical use of the Power Descent and the Rényi Descent algorithms requires approximations to handle intractable integrals appearing in the update formulas. As such, the Power Descent applies the function [wv) = - (x-1)<++1 +1)0/11aa to an unbiased estimator of the translated gradient 6j... (0) + before renormalising, while the the Rényi Descent applies the Entropic Mirror Descent function
[(v) = 7"' to biased estimator of bp... (0) (p//(p/nb...... (bp...)) + + 1/(a 1)) before renormalising.
Finding which approach is most suitable between biased and unbiased a-divergence minimisation is still an open issue in the literature, both theoretically and empirically [17 18 21]. Due to the exponentiation, considering the a-divergence instead of Rényi's a-divergence has for example been said to lead to high- gradients [13] [2] and low hignal-to-Noise ratio when * 0 [8 during
the stochastic gradient descent optimisation.
In that regard, our work sheds light on additional links between unbiased and biased a-divergence methods beyond the framework of stochastic gradient descent algorithms, both the unbiased Power Descent and the biased Rényi Descent share the same first-order approximation.
Our work also contributes towards deriving convergence results of variational objective functions, which is an active area of research (5] 27 28 29 30). The particularity of our results is then that, in line with the research work started in 19
focus on mixture weights gradient-based updates in the optimisation procedures, that are carried out via a-divergence minimisation and for general choices of kernel K. Compared [19] also bring into play Rényi's a-divergence- aaiverrenceebbsed based types of updates, which is novel idea when it comes to mixture weights optimisation by a-divergence minimisation.
6 Conclusion
We investigated algorithms that can be used to perform mixture weights optimisation for a-divergence minimisation regardless how the mixture components parameters are obtained. We have established the full proof of the convergence of the Power Descent algorithm in the a < 1 when we consider mixture models and bridged the gap with the case a = 1. We also introduced a closely-relate algorithm called the Rényi Descent. We proved it enjoys an O(L/N) convergence rate and illustrated in practice the proximity between these two algorithms.
Further work could include establishing theoretical results regarding the stochastic version of these two algorithms, as well as providing complementary empirical results comparing the performances of the unbiased uudivergeene-basee based Power Descent algorithm to those of the biased Rényi's 's a- divergence- based Rényi Descent. Since our contributions are mainly theoretical, we believe these will not result in any negative socictal impacts.
10

---

Acknowledgments and Disclosure of Funding
Kamélia Daudel acknowledges support of the UK Defence Science and Technology Laboratory (Dstl) and and Engineering and Physical Research Council (EPSRC) under grant EP/R013616/1. This is part of the collaboration between US DOD, UK MOD and UK EPSRC under the Multidisciplinary University Research Initiative. The authors are grateful to the reviewers for valuable remarks and suggestions on the paper.
References
[1] Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, and Lawrence K. Saul, An introduction to variational methods for graphical models. Machine Learning, 37(2):183-233, 1999.
[2] Matthew James. Beal. Variational algorithms for approximate bayesian inference. PhD thesis, 01 2003.
[3] Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. Stochastic variational inference. Journal of Machine Learning Research, 14(4))1303-1347, 2013.
Tom Minka. Divergence measures and message passing. Technical Report MSR-TR-2005-173, January 2005.
[5] Ghassen Jerfel, Serena Wang, Clara Fannjiang. Katherine A. Heller, Yian Ma. and Michael Jordan. Variational refinement for importance sampling using the forward bullback-leible divergence. Accepted for the 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021), 2021.
[6] Dennis Pranglc. Distilling importance sampling. arXivpreprint arxiv: 1910.0363233, 2021.
[7] Huaiyu Zhu and Richard Rohwer. Information geometric measurements of generalisation. Technical Report NCRG/4350, Aug 1995.
[8] Huaiyu Zhu and Richard Rohwer. Bayesian invariant measurements of generalization. Neural Processing Letters, 2:28-31, December 1995.
Alfréd Rényi. On measures of entropy and information. In Proceedings the Fourth Berkeley Symposium on Mathematical Statistics and Probability Volume I: Contributions to the Theory of Statistics, pages 547 5611, Berkeley, Calif., 1961. University of California Press.
[10] Tim van Erven and Peter Harremoes. Rényi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory, 60(7))3797-3820, Jul 2014.
[11] Jose Hernandez-Lobant Yingzhen Li, Mark Rowland, Thang Bui, Daniel Hernandez Larinndez--bbbarrr Lobato, and Richard Turner. Black-bol box alpha divergence minimization. In Maria Florina Balcan and Kilian Q Weinberger, editors, Proceedings The 33nd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1511-1520, New York, New York, USA, 20-22 Jun 2016. PMI..
[12] Yingzhen Li and Richard E Turner. Rényi divergence variational inference. D. Lee, M. Sugiyama, U. V. Luxburg. I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 1073-1081, Curran Associates, Inc., 2016.
[13] Adji Bousso Dieng, Dustin Tran, Rajesh Ranganath, John Paisley, and David Blei. Variational inference via 'ehi upper bound minimization. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advancee in Neural Information Processing Systems 30, pages 2732-2741. Curran Associates, Inc., 2017.
[14] Volodymyr Kuleshov and Stefano Ermon. Neural variational inference and learning in undirected graphical models. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[15] Robert Bamler, Cheng Zhang, Manfred Opper, and Stephan Mandt. Perturbative black box variational inference. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Prucessing Syxtems 30, pages 5079-5088. Curran Associates. Inc., 2017.
[16] Dilin Wang, Hao Liu, and Qiang Liu. Variational inference with tail- tail-adaptive f-divergence. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, editors. Advances in Newral Information Processing Systems 31, pages 5737-5747. Curran Associates, Inc., 2018

---

[17] Tomas Geffner and Justin Domke. Empirical evaluation of biased methods for alpha divergence minimiza- tion. In 3rd Symposiun Advances Appreximate Bayesian Inference, pages 1-12, 2020.
[18] Tomas Geffner and Justin Domke, On the difficulty of unbiased alpha divergence minimization. arXiv preprint arXiv:2010.09541 2020.
[19] Kamélia Daudel, Randal Douc, and François Portier. Infinite-dimennsionall gradient-based descent for alpha-divergence minimisation. The Annals Statistics, 49(4):2250- 2270, 2021.
[20] Kamélia Daudel, Randal Douc, and François Roueff. Monotonic alpha-divergence minimisation. arXiv preprint arxiv:2103.056633 2021.
[21] Akash Kumar Dhaka, Alejandro Catalina, Manushi Welandawe, Michael Riis Andersen, Jonathan Huggins, and Aki Vehtari. Challenges and opportunities in high-dimensional variational inference. arxiv preprint arxiv:2103.01085 2021.
[22] Ruqi Zhang, Yingzhen Li, Christopher De Sa, Sam Devlin, and Cheng Zhang. Meta-learning divergences variational inference. In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 4024 4032. PMLR, 13-15 Apr 2021.
[23] Arnaud Doucet. Nando Freitas, Kevin Murphy, and Stuart Russell. Sequential monte carlo methods in practice. 01 2013.
[24] Andrzej Cichocki and Shun-ichi Amari. Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities. Entropy, 1532- 1568, Jun 2010.
[25] Amir Beck and Mare Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31(3):167 175, 2003.
[26] Sébastien Bubeck. Convex optimization: Algorithms and complexity. Foundations and Trends in Machine Learning, 8(3-4):231-37, 01 2015.
[27] Badr-Eddine Chérief-Abdellie and Pierre Alquier. Consistency of variational Bayes inference for estimation and model selection in mixtures. Electronic Journal of Statistics, 12(2):2995 3035, 2018.
[28] Justin Domke. Provable gradient variance guarantces for black-box variational inference. In H. Wallach,
H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neurul Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[29] Justin Domke, Provable smoothness guarantees for black-box variational inference. In Hal Daumé III and Aarti Singh, editors. Proceedings of the 37th International Conference Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 2587-2596. PMLR, 13-18 Jul 2020.
[30] Sean Plummer, Debdeep Pati, and Anirban Bhattacharya. Dynamics of coordinate ascent variational inference: case study in 2d ising models, Entropy, 22(11), 2020.
12


Reviews:
[{"review_id": "tbxxQp-WeHV", "reviewer": "Reviewer_qaiR", "summary": "The authors explore the connections between power descent and entropic mirror descent for variational inference with $\\alpha$-divergences. The authors show that power descent converges under weaker assumptions than previously known when the weights of a mixture model are optimized. They also show a convergence rate of $O(1/N)$ for a variant of power descent, which they refer to as Rényi descent with $\\alpha < 1$. A simulation experiment is included indicating the efficacy of the proposed method for reasonably high-dimensional problems, and supporting the theoretical claims.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Summary of Review\nThe results presented in the paper on convergence of power descent for mixture weights, as well as convergence of Rényi-Descent appear to be interesting and original. However, as I am not familiar with some of the related work, I am hesitant to make an overly strong statement in this regard. Convergence of variational objective functions is certainly of interest to the probabilistic machine learning community. The paper was reasonably clear, though I think it could have been more self-contained. In particular,\n- At points notation was used without being introduced e.g.:\n    - $\\mu f= \\int f d\\mu$\n    -  $\\mu K$ is the measure with $\\nu$-density $\\mu k$\n- The paper would benefit from a, perhaps higher-level, discussion of related work on optimization of variational objectives, discussing existing work in this area. \n\nAs I am not familiar with the some of the related work, and it is possible I have misjudged some portion of the paper, I may adjust my score in light of additional information/clarification.\n\n## General comments and questions:\n- Can the authors clarify what aspects of the discussion and analysis comparing Power descent and entropic mirror descent are new to this work, and what aspects are previously contained in [17]? Is table 1 in [17] not already claiming something very much like proposition 1? \n- Relatedly, can you clarify why [17, Theorem 3] does not already imply a rate of convergence rate of $O(1/N)$ in the case $\\alpha=1$? \n- Are there any interesting cases where $b_{\\mu_n,  \\alpha}$ can be computed in closed-form (so that the algorithm for which you prove guarantees can actually be implemented)?\n\n## Typos and Minor comments:\n- The main paper contains links to the appendix, that are not stated as links to the appendix. As the main paper will be separate (so these links will not work), consider stating they are in the appendix/supplement.\n- Renyi $\\to$ Rényi \n- I think you may need an absolute continuity assumption in the definition of alpha divergence.\n- Line 420: $\\zeta_1 R \\zeta_2$\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors address several limitations of the work, notably that practical implementations of the algorithm rely on a stochastic approximation, which is not handled in the theoretical analysis, and that convergence guarantees for power descent are not obtained in the general case. The authors do not discuss societal impacts, which seems reasonable as the contribution theoretical and not attached to a particular application.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "jm553UrJl-e", "reviewer": "Reviewer_M13e", "summary": "This paper extends a previous work [17] in providing a convergence rate for the case of \\alpha<1.", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "My main concern is that this work is too close to the previous work [17]. The case of \\alpha<1 has been discussed in the experiment of [17] in a similar setting. The proof technique is also similar. More importantly, it is not clear why the case of \\alpha<1 is so crucial to the field. The exposition and literary quality in [17] is so much better than the current manuscript. I find it hard to justify a separate publication. I would suggest the authors to simply incorporate this result into [17] for publication.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "ZrbD76R96o1", "reviewer": "Reviewer_B46Q", "summary": "The authors study the convergence properties of the Power Descent algorithm, which has been recently proposed in [17].\nPower Descent (PD) allows one to optimise the weights of a mixture model by means of $\\alpha$-divergence minimisation.\nIn previous [17] work, PD was shown to have convergence rate O(1/N) for $\\alpha>1$.\nIn the current work, the authors prove convergence for the case where $\\alpha<1$, under less strict assumptions that in [17].\nAlthough no convergence rate is given for $\\alpha<1$, the authors propose Renyi Descent (RD) as an alternative, which is shown to have convergence rate O(1/N).\nAlso, PD is extended to the case $\\alpha=1$, recovering thus Entropic Mirror Descent.\nThe convergence rate for $\\alpha=1$ is shown to be O(1/N).\nThe convergence properties are demonstrated on a case study that involves approximating a two-component Gaussian mixture model as the target density.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThe paper extends the theoretical results of [17] for Power Descent, providing a more complete characterisation of the convergence properties of $\\alpha$-divergence methods.\nThe mixture model framework presented has been already discussed in [17]; in that sense, the contribution of the current paper is somewhat incremental.\nNevertheless, the addition of Renyi Descent as an $\\alpha$-divergence optimisation method, as well as its convergence analysis could be of interest to the community.\n\nThe experimental evaluation of the paper is weak.\nI appreciate that this is a mostly theoretical work, but the empirical examples are limited.\nThe simulation study of Section 5 could easily be extended to demonstrate the convergence properties of the methods proposed for models of different dimensionality.\nFor example, it is shown that Entropic Mirror Descent fails to converge for 16 dimensions, but what about a smaller dimensionality? How would it compare with Power descent or Renyi descent?\n\nAnother possibly minor issue is that certain parts of Section 2 bear many similarities to Sections 2.2, 3.1 and 3.2 in [17].\nI understand that this is a background section and it has to summarise previous work, but parts of it should be rephrased.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors claim that their contribution is mostly theoretical, thus it is unlikely to result in any negative societal impact.\nI think this is a fair position.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}, {"review_id": "VrLnZlz_F-h", "reviewer": "Reviewer_ssFh", "summary": "The authors consider the problem of variational inference with Renyi divergence for learning mixture models. They improve the analysis of the iterative method Daudel et al. Specifically, they establish fast $O(1/n)$ convergence of the method when $\\alpha < 1$. They then analyze the method when $\\alpha = 1$ by taking the limit of the algorithm updates. They call this algorithm Renyi descent and establish that it also converges at an $O(1/n)$ rate. They experimentally compare their approach with entropic mirror descent.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I found the content of the paper interesting. The authors complete the analysis of Daudel et al. and show fast convergence of the variety of parameter settings. The paper is theoretically interesting.\n\nHowever I believe the clarity of the paper should be improved. The paper is quite dense and more care can be used to introduce concepts:\n\n(1) The introduction in particular is quite dense. The appendix describes \"practical\" versions of the approach. I think a more practical introduction can improve the clarity.\n\n(2) The paper discusses mixture models but works with a density kernel. The connection can be more clearly spelled out.\n\n(3) The paper defines Renyi descent and discusses its connection with entropic mirror descent, however the explanation of this connection can be improved.\n\nThe experimental section is perhaps sufficient for this type of paper but not particularly interesting. The scale of the problem is quite small, only 16 dimensions. Does the method break down in higher dimensions? The authors could argue the strength of the method in general, rather than just how it compares to entropic mirror descent.\n\nOverall, I lean towards acceptance due to the content of the paper, however I believe the clarity can be improved.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No issues here.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Variational approximations are approximations of the posterior in Bayesian statistics, obtained by minimizing a divergence between the approximation and the posterior, where the approximation is constrained to belong to a given set (for example, the set of Gaussian distributions, or the set of mixtures of Gaussian distributions). The divergence chosen is usually the Kullback-Leibler divergence (KL). However, some recent works highlighted the possible benefits of using alpha-divergences instead (note that KL is recovered as the special case alpha=1). [17] proposed an iterative scheme, referred to as the Power Descent algorithm, to minimize the alpha-divergence in practice. A nice property of this scheme is that it leads to a sequence of approximations such that the divergence with respect to the posterior is decreasing.

This paper extends the analysis of the Power Descent algorithm initiated in [17] to approximations by mixtures models. The author(s) then study the convergence of the sequence of approximations to the minimizer of the divergence. In the case alpha>1, this is a direct consequence of Theorem 1 (stated in the paper, but was already proven in [17]). The convergence in the case alpha<1 is given by Theorem 2 (this is a new, and difficult result). Finally, they analyze the case alpha=1 by taking the limit of the algorithm updates. They refer to this algorithm as "Renyi descent", for which they prove a convergence in 1/N (N being the number of iterations, this is Theorem 3). They compare their approach with Entropic Mirror Descent in a set of numerical experiments.

Three Reviewers think that while this is strongly based on [17], there are useful and nontrivial extensions, and I agree with them. One of the members of the Committee already reviewed an earlier version of this paper, and was satisfied to see many of his/her comments taken into account in this version of the paper. One of them points some weaknesses in the empirical evaluation, but it clear to me that the main contribution here is theoretical. I will therefore recommend to accept the paper.

It would be nice if you could improve the writing of the paper before sending the camera ready version. Reviewer ssFh wrote the paper is too dense, Reviewer M13e found some undefined notations and Reviewer B46Q asked you to rephrase parts of Section 2, that explain the context of [17] but that are too close to the original.
Decision: Accept (Poster)