Mixture weights optimisation for Alpha-Divergence Variational Inference
Year: 2021

Abstract:
This paper focuses on α-divergence minimisation methods for Variational Inference. We consider the case where the posterior density is approximated by a mixture model and we investigate algorithms optimising the mixture weights of this mixture model by α-divergence minimisation, without any information on the underlying distribution of its mixture components parameters. The Power Descent, defined for all α ̸= 1, is one such algorithm and we establish in our work the full proof of its convergence towards the optimal mixture weights when α < 1. Since the α-divergence recovers the widely-used exclusive Kullback-Leibler when α → 1, we then extend the Power Descent to the case α = 1 and show that we obtain an Entropic Mirror Descent. This leads us to investigate the link between Power Descent and Entropic Mirror Descent: first-order approximations allow us to introduce the Rényi Descent, a novel algorithm for which we prove an O(1/N) convergence rate. Lastly, we compare numerically the behavior of the unbiased Power Descent and of the biased Rényi Descent and we discuss the potential advantages of one algorithm over the other.

heading: 1 Introduction
text: Bayesian Inference involves being able to compute or sample from the posterior density. For many useful models, the posterior density can only be evaluated up to a normalisation constant and we must resort to approximation methods.
One major category of approximation methods is Variational Inference, a wide class of optimisation methods which introduce a simpler variational family Q and use it to approximate the posterior density (see for example Variational Bayes [1, 2] and Stochastic Variational Inference [3]). The crux of these methods consists in finding the best approximation of the posterior density among the family Q in the sense of a certain measure of dissimilarity, most typically the exclusive Kullback-Leibler divergence.
However, the exclusive Kullback-Leibler divergence is known to have some undesirable properties, e.g. posterior variance underestimation, difficulty to capture multimodal posterior densities [4, 5, 6]. As a consequence, the α-divergence [7, 8] and Rényi’s α-divergence [9, 10] have gained a lot of attention recently and are presented as more general alternatives that permit to bypass the issues of the exclusive Kullback-Leibler when α < 1 [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22].
Noticeably, [19] introduced the (α,Γ)-descent, a general family of gradient-based algorithms that approximate the posterior density by a mixture model and that are able to optimise the mixture weights of this mixture model by α-divergence minimisation. The benefit of these types of algorithms is that (i) they expand the traditional parametric variational family to better capture complex (e.g.
∗Corresponding author: kamelia.daudel@stats.ox.ac.uk
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
multimodal) posterior densities (ii) they allow, in an Sequential Monte Carlo fashion [23], to select the mixture components according to their overall importance in the set of components parameters, without needing to know the distribution of the mixture components parameters. The (α,Γ)-descent step is then paired up with an Exploration step that acts on the mixture components parameters, so that a complete algorithm is obtained by alternating between these two steps [19].
The key algorithm of [19] is the Power Descent, which sets Γ(v) = [(α−1)v+1]η/(1−α) with α ̸= 1 and η > 0 in the (α,Γ)-descent. Indeed, when α < 1 and as the dimension increases, numerical experiments in [19] show that the Power Descent outperforms the Entropic Mirror Descent (a classical algorithm from the optimisation litterature corresponding to Γ(v) = e−ηv with η > 0).
Nonetheless, the global convergence of the Power Descent algorithm when α < 1, as stated in [19, Theorem 4], assumes the existence of the limit and does not provide conditions that satisfy this assumption. Furthermore, even though the convergence towards the global optimum is derived, there is no convergence rate available for the Power Descent when α < 1. After recalling the basics of the Power Descent algorithm in Section 2, we make the following contributions in this paper:
• In Section 3, we prove the full convergence of the Power Descent algorithm towards the optimal mixture weights when α < 1 (Theorem 2). • Since the α-divergence becomes the traditional exclusive Kullback-Leibler when α → 1, we next investigate the extension of the Power Descent to the case α = 1 in Proposition 1 from Section 4 and we obtain that the Power Descent recovers an Entropic Mirror Descent performing exclusive Kullback-Leibler minimisation. • We further study the connections between Power Descent and Entropic Mirror Descent by considering first-order approximations, in the hope of finding an algorithm close to the Power Descent and for which we can prove a convergence rate when α < 1. As a result, we are able to go beyond the (α,Γ)-descent framework from [19] by introducing an algorithm closely-related to the Power Descent, that is proved to converge at an O(1/N) rate when α < 1 as a consequence of Theorem 3 from Section 4. We call this algorithm the Rényi Descent due to the link we establish between the Rényi Descent and Entropic Mirror Descent steps applied to the Variational Rényi bound [12]. • Finally, we run some numerical experiments in Section 5 to compare the behavior of the Power Descent and of the Rényi Descent altogether, before discussing the potential benefits of one approach over the other.

heading: 2 Background
text: This section reviews the Power Descent algorithm from [19] and details how it is applied to mixture weights optimisation in a Variational Inference context.
Notation and problem statement. Let (Y,Y, ν) be a measured space, where ν is a σ-finite measure on (Y,Y). Assume that we are given some observed data D generated from a probabilistic model p(D |y) that is parameterised by a latent variable y ∈ Y. Further assuming that y is drawn from a certain prior p0(y), the posterior density of the latent variable y given the data D is then defined by:
p(y|D) = p(y,D) p(D) = p0(y)p(D |y) p(D) ,
where the normalisation constant p(D) = ∫ Y p0(y)p(D |y)ν(dy) is called the marginal likelihood or model evidence. For many complex models arising in Bayesian Inference, we do not know how to directly sample from the posterior density, nor do we know the value of the marginal likelihood.
To remedy this problem, the Power Descent is a Variational Inference algorithm that introduces a certain measurable space (T, T ) and offers to approximate the posterior density by a variational family Q of the form
Q = { y 7→ ∫ T µ(dθ)k(θ, y) : µ ∈ M } , (1)
where M is a convenient subset of M1(T), the set of probability measures on (T, T ). In doing so, it extends the usual parametric variational family
Q = {y 7→ k(θ, y) : θ ∈ T}
since it amounts to putting a prior over the parameter θ in the form of a probability measure. In particular, one strong motivation behind the variational family (1) is that it recovers mixture models as a special case. To see this, let J ∈ N⋆, Θ = (θ1, . . . , θJ) ∈ TJ and define the simplex of RJ by:
SJ = λ = (λ1, . . . , λJ) ∈ RJ : ∀j ∈ {1, . . . , J} , λj ⩾ 0 and J∑
j=1
λj = 1  . (2) Observe then that when µ is chosen as a weighted sum of Dirac measures, i.e. µ = ∑J j=1 λjδθj with λ ∈ SJ , the variational family (1) becomes
Q = y 7→ J∑
j=1
λjk(θj , y) : λ ∈ SJ  , (3) which corresponds to the class of mixture models parameterised by the mixture weights λ and with fixed mixture components parameters Θ. Here, the j-th component y 7→ k(θj , y) could for example be a multivariate Gaussian density with mean θj and known covariance matrix, but one strength of the Power Descent is that it is not limited to a specific choice for k. We will refer to k as a kernel density thereafter and we now introduce some more notation, before stating the optimisation problem solved by the Power Descent.
Denote by P the probability measure on (Y,Y) with corresponding density p(·|D) with respect to ν and by K : (θ,A) 7→ ∫ A k(θ, y)ν(dy) the Markov transition kernel on T× Y with kernel density k
defined on T × Y. For all µ ∈ M1(T) and all y ∈ Y, we denote µk(y) = ∫ T µ(dθ)k(θ, y) and we let µK denote the probability measure with density µk with respect to ν. Then, letting α ∈ R and assuming that µK and P are both absolutely continuous with respect to ν, the α-divergence between µK and P (extended by continuity to the cases α = 0 and α = 1 as in [24]) is given by
Dα(µK||P) = ∫ Y fα ( µk(y) p(y|D) ) p(y|D)ν(dy) ,
where fα is the convex function on (0,+∞) defined by f0(u) = u− 1− log(u), f1(u) = 1− u+ u log(u) and fα(u) = 1α(α−1) [u α − 1− α(u− 1)] for all α ∈ R \ {0, 1}.
Letting α ̸= 1, the Power Descent seeks to solve the Variational Inference optimisation problem
inf µ∈M
Dα(µK||P) , (4)
meaning that for µ = ∑J
j=1 λjδθj (with corresponding approximating family (3)), it seeks to optimise λ by α-divergence minimisation i.e. to select mixture components according to their overall importance in the set of components parameters. More generally, the Power Descent aims at solving
inf µ∈M Ψα(µ; p) , (5)
where p is any measurable positive function on (Y,Y) and where for all µ ∈ M1(T), Ψα(µ; p) =∫ Y fα (µk(y)/p(y)) p(y)ν(dy). Crucially, (4) can be reframed as an instance of (5) that sets p = p(·,D) in (5) and that thus does not involve the unknown normalising constant p(D) anymore (see Appendix A.1). In the following, the dependency on p in Ψα may be dropped throughout the paper for notational ease when no ambiguity occurs and we next present the Power Descent algorithm.
The Power Descent algorithm. Let α ̸= 1. Given an initial probability measure µ1 ∈ M1(T) such that Ψα(µ1) < ∞, η > 0 and κ such that (α− 1)κ ⩾ 0, the Power Descent algorithm introduced in [19] is an iterative scheme which builds the sequence of probability measures (µn)n∈N⋆
µn+1 = Iα(µn) , n ∈ N⋆ , (6)
where for all µ ∈ M1(T), the one-step transition µ 7→ Iα(µ) is given by Algorithm 1 and where for all v ∈ Domα, Γ(v) = [(α− 1)v + 1]η/(1−α) [and Domα denotes an interval of R such that for all θ ∈ T and all µ ∈ M1(T), bµ,α(θ) + κ and µ(bµ,α) + κ ∈ Domα].
Algorithm 1: Power descent one-step transition (Γ(v) = [(α− 1)v + 1]η/(1−α)) 1. Expectation step : bµ,α(θ) = ∫ Y k(θ, y)f ′α ( µk(y) p(y) ) ν(dy)
2. Iteration step : Iα(µ)(dθ) = µ(dθ) · Γ(bµ,α(θ) + κ)
µ(Γ(bµ,α + κ))
The Power Descent is known to have a gradient-based structure since θ 7→ bµ,α(θ) acts as the gradient of µ 7→ Ψα(µ) and η plays the role of a learning rate [19]. Furthermore, a remarkable property of the Power Descent is that under (A1) as defined below
(A1) The density kernel k on T× Y, the function p on Y and the σ-finite measure ν on (Y,Y) satisfy, for all (θ, y) ∈ T× Y, k(θ, y) > 0, p(y) > 0 and ∫ Y p(y)ν(dy) < ∞.
the Power Descent ensures a monotonic decrease in Ψα at each step for all η ∈ (0, 1] (this result is a special case of [19, Theorem 1] with Γ(v) = [(α − 1)v + 1]η/(1−α) that is recalled in Theorem 4 of Appendix A.2 for the sake of completeness). Under the additional assumptions that κ > 0 and
sup θ∈T,µ∈M1(T)
|bµ,α| < ∞ and Ψα(µ1) < ∞ , (7)
the Power Descent is also known to converge towards its optimal value at an O(1/N) rate when α > 1 [19, Theorem 3]. On the other hand, when α < 1, the convergence towards the optimum as written in [19] holds under different assumptions including
(A2) (i) T is a compact metric space and T is the associated Borel σ-field; (ii) for all y ∈ Y, θ 7→ k(θ, y) is continuous;
(iii) we have ∫ Y supθ∈T k(θ, y)× supθ′∈T ( k(θ′,y) p(y) )α−1 ν(dy) < ∞.
If α = 0, assume in addition that ∫ Y supθ∈T ∣∣∣log (k(θ,y)p(y) )∣∣∣p(y)ν(dy) < ∞. so that [19, Theorem 4], that is recalled below under the form of Theorem 1, states the convergence of the Power Descent algorithm towards the global optimum.
Theorem 1 ([19, Theorem 4]). Assume (A1) and (A2). Let α < 1 and let κ ⩽ 0. Then, for all µ ∈ M1(T), Ψα(µ) < ∞ and any η > 0 satisfies 0 < µ(Γ(bµ,α + κ)) < ∞. Further assume that η ∈ (0, 1] and that there exist µ1, µ⋆ ∈ M1(T) such that the (well-defined) sequence (µn)n∈N⋆ defined by (6) weakly converges to µ⋆ as n → ∞. Finally, denote by M1,µ1(T) the set of probability measures dominated by µ1. Then the following assertions hold
(i) (Ψα(µn))n∈N⋆ is nonincreasing,
(ii) µ⋆ is a fixed point of Iα,
(iii) Ψα(µ⋆) = infζ∈M1,µ1 (T) Ψα(ζ).
The above result assumes there must exist µ1, µ⋆ ∈ M1(T) such that the sequence (µn)n∈N⋆ defined by (6) weakly converges to µ⋆ as n → ∞, that is it assumes the limit already exists. Our first contribution consists in showing that this assumption can be alleviated when µ is chosen a weighted sum of Dirac measures, that is when we seek to perform mixture weights optimisation by α-divergence minimisation. In doing so, we provide additionnal theoretical justification behind the use of the Power Descent when α < 1, which is crucial in practice for reasons already outlined in the introduction and further detailed for the particular case of the Power Descent in Appendix A.3.

heading: 3 Convergence of the Power Descent algorithm in the mixture case
text: Before we state our convergence result, let us first make two comments on the assumptions from Theorem 1 that shall be retained in our upcoming convergence result.
A first comment is that (A1) is mild since the assumption that p(y) > 0 for all y ∈ Y can be discarded and is kept for convenience [19, Remark 4]. A second comment is that (A2) is also mild and covers (7) as it amounts to assuming that θ 7→ bµ,α(θ) and µ 7→ Ψα(µ) are uniformly bounded with respect to µ and θ. To see this, we give below an example for which (A2) is satisfied. Example 1. Consider the case Y = Rd with α ∈ [0, 1). Let r > 0 and let T = B(0, r) ⊂ Rd. Furtheremore, let Kh be a Gaussian transition kernel with bandwidth h and denote by kh its associated kernel density. Finally, let p be a mixture density of two d-dimensional Gaussian distributions multiplied by a positive constant c such that p(y) = c× [0.5N (y; θ⋆1 , Id) + 0.5N (y; θ⋆2 , Id)] for all y ∈ Y, where θ⋆1 , θ⋆2 ∈ T and Id is the identity matrix. Then, (A2) holds (see Appendix B.1).
Next, we introduce some notation that are specific to the case of mixture models we aim at studying in this section (and which, as seen previously, corresponds to choosing µ as a weighted sum of Dirac measures in (1) yielding (3)). For this purpose, let J ∈ N⋆, let Θ = (θ1, . . . , θJ) ∈ TJ be fixed, let α ̸= 1, η > 0, κ be such that (α − 1)κ ⩾ 0 and recall that the simplex of RJ defined in (2) is denoted by SJ . We define S+J = {λ ∈ SJ : ∀j ∈ {1, . . . , J} , λj > 0} and for all λ ∈ SJ and we also define µλ,Θ = ∑J j=1 λjδθj . Now letting (µn)n∈N⋆ be defined by µ1 = µλ,Θ and (6), an
immediate induction yields that for every n ∈ N⋆, µn can be expressed as µn = ∑J
j=1 λj,nδθj where λn = (λ1,n, . . . , λJ,n) ∈ SJ satisfies the initialisation λ1 = λ and the update formula:
λn+1 = Imixtα (λn) , n ∈ N⋆ , (8)
with
Imixtα (λ) :=
( λj [(α− 1)(bµλ,Θ,α(θj) + κ) + 1]
η 1−α∑J
ℓ=1 λℓ [(α− 1)(bµλ,Θ,α(θℓ) + κ) + 1] η 1−α ) 1⩽j⩽J , λ ∈ SJ .
Finally, let us rewrite (A2) in the simplified case where µ = µλ,Θ, which gives (A3) below.
(A3) (i) For all y ∈ Y, θ 7→ k(θ, y) is continuous; (ii) we have ∫ Y max 1⩽j⩽J k(θj , y)× max 1⩽j′⩽J ( k(θj′ ,y) p(y) )α−1 ν(dy) < ∞.
If α = 0, we assume in addition that ∫ Y max 1⩽j⩽J ∣∣∣log (k(θj ,y)p(y) )∣∣∣ p(y)ν(dy) < ∞. We then have the following theorem, which establishes the full proof of the global convergence towards the optimum for the mixture weights when α < 1. Theorem 2. Assume (A1) and (A3). Let α < 1, let Θ = (θ1, . . . , θJ) ∈ TJ be fixed and let κ be such that κ ⩽ 0. Then for all λ ∈ SJ , Ψα(µλ,Θ) < ∞ and for any η > 0 the sequence (λn)n∈N⋆ defined by λ1 ∈ SJ and (8) is well-defined. If in addition (λ1, η) ∈ S+J ×(0, 1] and {K(θ1, ·), . . . ,K(θJ , ·)} are linearly independent, then
(i) (Ψα(µλn,Θ))n∈N⋆ is nonincreasing,
(ii) the sequence (λn)n∈N⋆ converges to some λ⋆ ∈ SJ which is a fixed point of Imixtα ,
(iii) Ψα(µλ⋆,Θ) = infλ′∈SJ Ψα(µλ′,Θ).
The proof of this result builds on Theorem 1 and Theorem 4 of Appendix A.2 and is deferred to Appendix B.2. Notice that since Ψα depends on λ through µλ,ΘK, an identifiably condition was to be expected in order to achieve the convergence of the sequence (λn)n∈N⋆ in Theorem 2. Following Example 1, this identifiably condition notably holds for J ⩽ d under the assumption that the θ1, ..., θJ are full-rank.
We thus have the convergence of the Power Descent under less stringent conditions when α < 1 and when we consider the particular case of mixture models. This algorithm can easily become feasible for any choice of kernel K by resorting to an unbiased estimator of (bµλn,Θ,α(θj))1⩽j⩽J in the update formula (8) (see Algorithm 3 of Appendix B.3).
Nevertheless, contrary to the case α > 1 we still do not have a convergence rate for the Power Descent when α < 1. Furthermore, the important case α → 1, which corresponds to performing
exclusive Kullback-Leibler minimisation, is not covered by the Power Descent algorithm. In the next section, we extend the Power Descent to the case α = 1. As we shall see, this will lead us to investigate the connections between the Power Descent and the Entropic Mirror Descent beyond the (α,Γ)-descent framework from [19]. As a result, we will introduce a novel algorithm closely-related to the Power Descent that yields an O(1/N) convergence rate when µ = µλ,Θ and α < 1 (and more generally when µ ∈ M1(T) and α ̸= 1).

heading: 4 Power Descent and Entropic Mirror Descent
text: Recall from Section 2 that the Power Descent is defined for all α ̸= 1. In this section, we first establish in Proposition 1 that the Power Descent can be extended to the case α = 1 and that we recover an Entropic Mirror Descent, showing that a deeper connection runs between the two approaches beyond the one identified by the (α,Γ)-descent framework of [19]. This result relies on typical convergence and differentiability assumptions summarised in (D1) and which are deferred to Appendix C.1, alongside with the proof of Proposition 1. Proposition 1 (Limiting case α → 1). Assume (A1) and (D1). Let η > 0 and κ be such that (α− 1)κ ⩾ 0. Then, for all µ ∈ M1(T) and all continuous and bounded real-valued functions h on T, we have that
lim α→1
[Iα(µ)](h) = [I1(µ)](h) ,
where for all θ ∈ T, we have set
I1(µ)(dθ) = µ(dθ)e−ηbµ,1(θ)
µ (e−ηbµ,1) and bµ,1(θ) = ∫ Y k(θ, y) log ( µk(y) p(y) ) ν(dy) , (9)
and where for all ζ ∈ M1(T), we have used the notation ζh = ∫ T ζ(dθ)h(θ).
Here, we recognise the one-step transition associated to the Entropic Mirror Descent applied to Ψ1. This algorithm is a special case of [19] with Γ(v) = e−ηv and α = 1 and as such, it is known to lead to a systematic decrease in the exclusive Kullback-Leibler divergence and to enjoy an O(1/N) convergence rate under the assumptions that (7) holds and η ∈ (0, 1) [19, Theorem 3]. We have thus obtained that the Power Descent coincides exactly with the Entropic Mirror Descent applied to Ψ1 when α = 1 and we now focus on understanding the links between Power Descent and Entropic Mirror Descent when α ̸= 1. For this purpose, let κ be such that (α− 1)κ ⩾ 0 and let us study first-order approximations of the Power Descent and of the Entropic Mirror Descent applied to Ψα when bµn,α(θ) ≈ µn(bµn,α) for all θ ∈ T. Letting η > 0 and following (6), we have that the update formula for the Power Descent is given by
µn+1(dθ) = µn(dθ) [(α− 1)(bµn,α(θ) + κ) + 1]
η 1−α
µn([(α− 1)(bµn,α + κ) + 1] η 1−α ) , n ∈ N⋆ ,
and using the first-order approximation u η 1−α ≈ v η 1−α − ηα−1v η 1−α−1(u − v) with u = (α−1)(bµn,α(θ)+κ)+1 (α−1)(µ(bµn,α)+κ)+1 and v = 1, we can deduce the following approximated update formula
µn+1(dθ) = µn(dθ)
[ 1− η
α− 1 bµn,α(θ)− µn(bµn,α)
µn(bµn,α) + κ+ 1/(α− 1)
] , n ∈ N⋆ .
In addition, the update formula for the Entropic Mirror Descent applied to Ψα can be written as
µn+1(dθ) = µn(dθ) exp [−η(bµn,α(θ) + κ)]
µn(exp [−η(bµn,α + κ)]) , n ∈ N⋆ , (10)
and we obtain in a similar fashion that an approximated version of this iterative scheme is
µn+1(dθ) = µn(dθ) [1− η (bµn,α(θ)− µn(bµn,α))] , n ∈ N⋆ .
As is, the two approximated formulas above do not coincide. This leads us to modify (10) as follows
µn+1(dθ) = µn(dθ) exp
[ −η bµn,α(θ)(α−1)(µn(bµn,α)+κ)+1 ] µn ( exp [ −η bµn,α(α−1)(µn(bµn,α)+κ)+1 ]) , n ∈ N⋆ , (11)
that is, it leads us to move from a constant learning rate η in (10) to an adaptive learning rate η′ = η [(α− 1)(µn(bµn,α) + κ) + 1]
−1 in (11). While this change is subtle, a first remark is that [19] only considered a constant learning rate when performing Entropic Mirror Descent steps applied to Ψα, meaning that we are now deviating from the framework of [19]. A second remark is that we can motivate (11) by observing that it can be seen as an Entropic Mirror Descent too, but this time applied to the objective function defined for all measurable positive function p on (Y,Y), all µ ∈ M1(T) and all α ∈ R \ {0, 1} by
ΨARα (µ; p) := 1
α(α− 1) log (∫ Y µk(y)αp(y)1−αν(dy) + (α− 1)κ )
(see Appendix C.2 for the derivation of (11) based on the objective function ΨARα ). Since the function ΨARα can be obtained by applying the increasing transformation
u 7→ 1 α(α− 1) log
( α(α− 1)u+ α+ (1− α) ∫ Y p(y)ν(dy) + (α− 1)κ )
to the objective function Ψα, this notably means that our novel Entropic Mirror Descent-based update formula (11) will aim at solving the initial optimisation problem (5) defined in Section 2 (i.e. minimising Ψα(µ; p) with µ ∈ M). In addition, when p = p(·,D), κ = 0 and α > 0 in ΨARα (µ; p), we obtain that minimising ΨARα (µ; p) w.r.t µ is equivalent to setting q = µk and maximising the Variational Rényi bound Lα(q;D) from [12] w.r.t µ, since for all probability density q,
Lα(q;D) := 1
1− α log
[ Eq ( p(y,D)
q(y)
)1−α] .
As a result, we can classify the scheme (11) as a Rényi’s α-divergence gradient-based approach [11, 12]. This is in contrast with the framework of [19], that is an α-divergence gradient-based approach and for this reason we call the algorithm given by (11) the Rényi Descent.
We have thus obtained that the Entropic Mirror Descent applied to Ψα does not share the same first-order approximation as the Power Descent, contrary to the newly-introduced Rényi Descent. This might explain why the behavior of the Entropic Mirror Descent applied to Ψα and of the Power Descent differed greatly when α < 1 in the numerical experiments from [19] (indeed, despite their theoretical connection through the (α,Γ)-descent framework, the former performs poorly numerically compared to the later as the dimension increases). This also sparks our interest in studying convergence results for the Rényi Descent.
Strikingly, we can prove an O(1/N) convergence rate towards the global optimum for the Rényi Descent. Letting κ′ ∈ R, denoting by DomARα an interval of R such that for all θ ∈ T and all µ ∈ M1(T),
bµ,α(θ) + 1/(α− 1) (α− 1)(µ(bµ,α) + κ) + 1 + κ′ and µ(bµ,α) + 1/(α− 1) (α− 1)(µ(bµ,α) + κ) + 1 + κ′ ∈ DomARα
and introducing the assumption on η
(A4) For all v ∈ DomARα , 1− η(α− 1)(v − κ′) ⩾ 0.
we indeed have the following convergence result. Theorem 3. Assume (A1) and (A4). Let α ̸= 1 and let κ be such that (α − 1)κ > 0. Define |B|∞,α := supθ∈T,µ∈M1(T) |bµ,α(θ) + 1/(α − 1)| and assume that |B|∞,α < ∞. Moreover, let µ1 ∈ M1(T) be such that Ψα(µ1) < ∞. Then, the following assertions hold.
(i) The sequence (µn)n∈N⋆ defined by (11) is well-defined and the sequence (Ψα(µn))n∈N⋆ is non-increasing.
(ii) For all N ∈ N⋆, we have
Ψα(µN )−Ψα(µ⋆) ⩽ Lα,2 N
[ KL(µ⋆||µ1) + L
Lα,3 Lα,1(α− 1)κ ∆1
] , (12)
where µ⋆ is such that Ψα(µ⋆) = infζ∈M1,µ1 (T) Ψα(ζ), M1,µ1(T) denotes the set of probability measures dominated by µ1, KL(µ⋆||µ1) = ∫ T log (dµ⋆/dµ1) dµ
⋆, ∆1 = Ψα(µ1)−Ψα(µ⋆) and L, Lα,1, Lα,2, Lα,3 are finite constants defined in (25).
The proof of this result is deferred to Appendix C.3 and we present in the next example an application of this theorem to the particular case of mixture models. Example 2. Let α ̸= 1, κ be such that (α − 1)κ > 0, J ∈ N⋆, Θ = (θ1, . . . , θJ) ∈ TJ , µ1 = J−1 ∑J j=1 δθj and Dom AR α = [− |B|∞,α (α−1)κ + κ ′, |B|∞,α (α−1)κ + κ
′] with κ′ ∈ R. In addition, assume that 1− η|κ|−1|B|∞,α > 0. Then, taking κ′ = −3 |B|∞,α(α−1)κ , we obtain
Ψα(µN )−Ψα(µ⋆) ⩽ |α− 1|(|B|∞,α + |κ|)
N
[ log J
η +
√ 2 log(J)|B|∞,α
(α− 1)κ(1− η|κ|−1|B|∞,α)
] ,
where we have used that KL(µ⋆||µ1) ⩽ log J , ∆1 ⩽ √ 2 log J |B|∞,α and that the constants defined in (25) satisfy Lα,2 = η−1|α− 1|(|B|∞,α + |κ|), L = η2eη |B|∞,α (α−1)κ −ηκ ′ , Lα,3 = e η |B|∞,α (α−1)κ +ηκ ′ and Lα,1 = (1− η|κ|−1|B|∞,α)ηe−η |B|∞,α (α−1)κ −ηκ ′ .
To put things into perspective, as an Entropic Mirror Descent algorithm, the Rényi Descent already enjoys an O(1/ √ N) convergence rate for the sequence (Ψα(N−1 ∑N n=1 µn))N∈N⋆ under our
assumptions and when η is proportional to 1/ √ N , N being fixed (see [25] or [26, Theorem 4.2.]).
The improvement thus lies in the fact that deriving an O(1/N) convergence rate usually requires stronger assumptions on Ψα [26, Theorem 6.2] that we do not need to assume in Theorem 3 thanks to the specific form of Ψα. Furthermore, due to the monotonicity property, our result only involves the measure µN at time N while typical Entropic Mirror Descent results are expressed in terms of the average N−1 ∑N n=1 µn.
Finally, observe that the Rényi Descent becomes feasible in practice for any choice of kernel K by letting µ be a weighted sum of Dirac measures i.e µ = µλ,Θ and by resorting to an unbiased estimate of (bµ,α(θj))1⩽j⩽J (see Algorithm 4 of Appendix D.1 for details). The theoretical results we have obtained are summarised in Table 1 and we next move on to numerical experiments.

heading: 5 Simulation study
text: The code for all the subsequent numerical experiments is available at https://github.com/ kdaudel/MixtureWeightsAlphaVI.
Let the target p be a mixture density of two d-dimensional Gaussian distributions multiplied by a positive constant c such that p(y) = c× [0.5N (y;−sud, Id) + 0.5N (y; sud, Id)], where ud is the d-dimensional vector whose coordinates are all equal to 1, s = 2, c = 2 and Id is the identity matrix. Let Kh be a Gaussian transition kernel on T× Y with bandwidth h and let kh denote its associated kernel density on T×Y. Let J ∈ N⋆ and recall the previously-defined notation µλ,Θ = ∑J j=1 λjδθj with λ ∈ SJ and Θ ∈ TJ . We consider the variational family described by
Q = y 7→ µλ,Θkh(y) = J∑
j=1
λjkh(y − θj) : λ ∈ SJ ,Θ ∈ TJ  .
Following the reasoning of [19], since the Power Descent and the Rényi Descent operate only on the mixture weights λ of µλ,Θkh during the optimisation, a fully adaptive algorithm can be obtained by alternating T times between an Exploitation step where the mixture weights are optimised and an Exploration step where the mixture components parameters are updated, as written in Algorithm 2.
Algorithm 2: Complete Exploitation-Exploration Algorithm Input: p: measurable positive function, α: α-divergence parameter, q0: initial sampler, Kh: Gaussian transition kernel, T : total number of iterations, J : dimension of the parameter set. Output: Optimised weights λ and parameter set Θ. Draw θ1,1, . . . , θJ,1 from q0. for t = 1 . . . T do
Exploitation step : Set Θ = {θ1,t, . . . , θJ,t}. Perform the Power Descent or the Rényi Descent algorithm and obtain the optimised mixture weights λ. Exploration step : Perform any Exploration step of our choice and obtain θ1,t+1, . . . , θJ,t+1.
Many choices of Exploration steps can be envisioned in Algorithm 2. We first consider the Exploration step used in [19]: h ∝ J−1/(4+d) and the particles are updated by i.i.d sampling according to µλ,Θkh. Since the Entropic Mirror Descent applied to Ψα is known to break down when d = 16 for this Exploration step [19], this case is a good starting point in order to investigate if the novel Rényi Descent is a suitable alternative to the Entropic Mirror Descent applied Ψα as d increases.
More precisely, at each time t = 1 . . . T , we perform N transitions of either the Power Descent or the Rényi Descent according to Algorithm 3 and 4 of the appendix, in which the initial weights are set to be [1/J, . . . , 1/J ], η = η0/ √ N with η0 > 0 and M samples are used in the estimation of (bµλ,Θ,α(θj,t))1⩽J at each iteration n = 1 . . . N . We take J = 100, M ∈ {100, 1000, 2000}, α = 0.5, κ = 0, η0 = 0.3 and q0 is a centered normal distribution with covariance matrix 5Id. We let T = 10, N = 20 and we replicate the experiment 100 times independently for each algorithm.
The convergence is assessed using Monte Carlo estimates of the Variational Rényi bound from [12]. The results for the Power Descent and the Rényi Descent are displayed on Figure 1, in which the Entropic Mirror Descent applied to Ψα is added as a reference (and additonnal plots in dimension d < 16 are provided in Appendix D.2 for the sake of completeness).
We then observe that the Rényi Descent is indeed better-behaved compared to the Entropic Mirror Descent applied to Ψα. Furthermore, it matches the performances of the Power Descent as M increases in our numerical experiment, which illustrates the link between the two algorithms we have established in the previous section.
Note that the Exploration step considered in [19] does not optimise over Θ. In an SMC fashion, it indeed resamples in the interesting regions for Θ (selection) and then applies a Gaussian pertubation (mutation). While we are not trying to discriminate between various choices of Exploration steps in this paper, an important remark is that the choice of the Exploration step in Algorithm 2 will become increasingly important as the dimension increases. For this reason, we have outlined in Appendix D.3 some potentially suitable Exploration steps that optimise over Θ.
In particular, we also ran our numerical experiments with the Exploration step from Appendix D.3.2. The plots for d = 16 are available in Appendix D.3.2 and we present below the plots for the higherdimensional setting where d = 100. We observe that the Rényi Descent and the Power Descent do
not break down as we increase the dimension up to d = 100 and that they share a similar behavior, which reinforces our conclusions.
Discussion From a theoretical standpoint, no convergence rate is yet available for the Power Descent algorithm when α < 1. An advantage of the novel Rényi Descent algorithm is then that while being close to the Power Descent, it also benefits from the Entropic Mirror Descent optimisation literature so that O(1/ √ N) convergence rates hold, which we have been able to improve to O(1/N) convergence rates.
A practical use of the Power Descent and of the Rényi Descent algorithms requires approximations to handle intractable integrals appearing in the update formulas. As such, the Power Descent applies the function Γ(v) = [(α−1)v+1]η/(1−α) to an unbiased estimator of the translated gradient bµ,α(θ)+κ before renormalising, while the the Rényi Descent applies the Entropic Mirror Descent function Γ(v) = e−ηv to a biased estimator of bµn,α(θ)/(µn(bµn,α) + κ+ 1/(α− 1)) before renormalising. Finding which approach is most suitable between biased and unbiased α-divergence minimisation is still an open issue in the literature, both theoretically and empirically [17, 18, 21]. Due to the exponentiation, considering the α-divergence instead of Rényi’s α-divergence has for example been said to lead to high-variance gradients [13, 12] and low Signal-to-Noise ratio when α ̸= 0 [18] during the stochastic gradient descent optimisation.
In that regard, our work sheds light on additional links between unbiased and biased α-divergence methods beyond the framework of stochastic gradient descent algorithms, as both the unbiased Power Descent and the biased Rényi Descent share the same first-order approximation.
Our work also contributes towards deriving convergence results of variational objective functions, which is an active area of research [5, 27, 28, 29, 30]. The particularity of our results is then that, in line with the research work started in [19], we focus on mixture weights gradient-based updates in the optimisation procedures, that are carried out via α-divergence minimisation and for general choices of kernel K. Compared to [19], we also bring into play Rényi’s α-divergence-based types of updates, which is a novel idea when it comes to mixture weights optimisation by α-divergence minimisation.

heading: 6 Conclusion
text: We investigated algorithms that can be used to perform mixture weights optimisation for α-divergence minimisation regardless of how the mixture components parameters are obtained. We have established the full proof of the convergence of the Power Descent algorithm in the case α < 1 when we consider mixture models and bridged the gap with the case α = 1. We also introduced a closely-related algorithm called the Rényi Descent. We proved it enjoys an O(1/N) convergence rate and illustrated in practice the proximity between these two algorithms.
Further work could include establishing theoretical results regarding the stochastic version of these two algorithms, as well as providing complementary empirical results comparing the performances of the unbiased α-divergence-based Power Descent algorithm to those of the biased Rényi’s αdivergence-based Rényi Descent. Since our contributions are mainly theoretical, we believe these will not result in any negative societal impacts.
Acknowledgments and Disclosure of Funding
Kamélia Daudel acknowledges support of the UK Defence Science and Technology Laboratory (Dstl) and and Engineering and Physical Research Council (EPSRC) under grant EP/R013616/1. This is part of the collaboration between US DOD, UK MOD and UK EPSRC under the Multidisciplinary University Research Initiative. The authors are grateful to the reviewers for valuable remarks and suggestions on the paper.

Review summaries:
The authors explore the connections between power descent and entropic mirror descent for variational inference with $\alpha$-divergences. The authors show that power descent converges under weaker assumptions than previously known when the weights of a mixture model are optimized. They also show a convergence rate of $O(1/N)$ for a variant of power descent, which they refer to as Rényi descent with $\alpha < 1$. A simulation experiment is included indicating the efficacy of the proposed method for reasonably high-dimensional problems, and supporting the theoretical claims.

The authors consider the problem of variational inference with Renyi divergence for learning mixture models. They improve the analysis of the iterative method Daudel et al. Specifically, they establish fast $O(1/n)$ convergence of the method when $\alpha < 1$. They then analyze the method when $\alpha = 1$ by taking the limit of the algorithm updates. They call this algorithm Renyi descent and establish that it also converges at an $O(1/n)$ rate. They experimentally compare their approach with entropic mirror descent.

The authors study the convergence properties of the Power Descent algorithm, which has been recently proposed in [17]. Power Descent (PD) allows one to optimise the weights of a mixture model by means of $\alpha$-divergence minimisation. In previous [17] work, PD was shown to have convergence rate O(1/N) for $\alpha>1$. In the current work, the authors prove convergence for the case where $\alpha<1$, under less strict assumptions that in [17]. Although no convergence rate is given for $\alpha<1$, the authors propose Renyi Descent (RD) as an alternative, which is shown to have convergence rate O(1/N). Also, PD is extended to the case $\alpha=1$, recovering thus Entropic Mirror Descent. The convergence rate for $\alpha=1$ is shown to be O(1/N). The convergence properties are demonstrated on a case study that involves approximating a two-component Gaussian mixture model as the target density.