Mixture weights optimisation for Alpha-Divergence Variational Inference

1 Introduction . Bayesian Inference involves being able to compute or sample from the posterior density . For many useful models , the posterior density can only be evaluated up to a normalisation constant and we must resort to approximation methods . One major category of approximation methods is Variational Inference , a wide class of optimisation methods which introduce a simpler variational family Q and use it to approximate the posterior density ( see for example Variational Bayes [ 1 , 2 ] and Stochastic Variational Inference [ 3 ] ) . The crux of these methods consists in finding the best approximation of the posterior density among the family Q in the sense of a certain measure of dissimilarity , most typically the exclusive Kullback-Leibler divergence . However , the exclusive Kullback-Leibler divergence is known to have some undesirable properties , e.g . posterior variance underestimation , difficulty to capture multimodal posterior densities [ 4 , 5 , 6 ] . As a consequence , the α-divergence [ 7 , 8 ] and Rényi ’ s α-divergence [ 9 , 10 ] have gained a lot of attention recently and are presented as more general alternatives that permit to bypass the issues of the exclusive Kullback-Leibler when α < 1 [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ] . Noticeably , [ 19 ] introduced the ( α , Γ ) -descent , a general family of gradient-based algorithms that approximate the posterior density by a mixture model and that are able to optimise the mixture weights of this mixture model by α-divergence minimisation . The benefit of these types of algorithms is that ( i ) they expand the traditional parametric variational family to better capture complex ( e.g . ∗Corresponding author : kamelia.daudel @ stats.ox.ac.uk 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . multimodal ) posterior densities ( ii ) they allow , in an Sequential Monte Carlo fashion [ 23 ] , to select the mixture components according to their overall importance in the set of components parameters , without needing to know the distribution of the mixture components parameters . The ( α , Γ ) -descent step is then paired up with an Exploration step that acts on the mixture components parameters , so that a complete algorithm is obtained by alternating between these two steps [ 19 ] . The key algorithm of [ 19 ] is the Power Descent , which sets Γ ( v ) = [ ( α−1 ) v+1 ] η/ ( 1−α ) with α ̸= 1 and η > 0 in the ( α , Γ ) -descent . Indeed , when α < 1 and as the dimension increases , numerical experiments in [ 19 ] show that the Power Descent outperforms the Entropic Mirror Descent ( a classical algorithm from the optimisation litterature corresponding to Γ ( v ) = e−ηv with η > 0 ) . Nonetheless , the global convergence of the Power Descent algorithm when α < 1 , as stated in [ 19 , Theorem 4 ] , assumes the existence of the limit and does not provide conditions that satisfy this assumption . Furthermore , even though the convergence towards the global optimum is derived , there is no convergence rate available for the Power Descent when α < 1 . After recalling the basics of the Power Descent algorithm in Section 2 , we make the following contributions in this paper : • In Section 3 , we prove the full convergence of the Power Descent algorithm towards the optimal mixture weights when α < 1 ( Theorem 2 ) . • Since the α-divergence becomes the traditional exclusive Kullback-Leibler when α → 1 , we next investigate the extension of the Power Descent to the case α = 1 in Proposition 1 from Section 4 and we obtain that the Power Descent recovers an Entropic Mirror Descent performing exclusive Kullback-Leibler minimisation . • We further study the connections between Power Descent and Entropic Mirror Descent by considering first-order approximations , in the hope of finding an algorithm close to the Power Descent and for which we can prove a convergence rate when α < 1 . As a result , we are able to go beyond the ( α , Γ ) -descent framework from [ 19 ] by introducing an algorithm closely-related to the Power Descent , that is proved to converge at an O ( 1/N ) rate when α < 1 as a consequence of Theorem 3 from Section 4 . We call this algorithm the Rényi Descent due to the link we establish between the Rényi Descent and Entropic Mirror Descent steps applied to the Variational Rényi bound [ 12 ] . • Finally , we run some numerical experiments in Section 5 to compare the behavior of the Power Descent and of the Rényi Descent altogether , before discussing the potential benefits of one approach over the other . 2 Background . This section reviews the Power Descent algorithm from [ 19 ] and details how it is applied to mixture weights optimisation in a Variational Inference context . Notation and problem statement . Let ( Y , Y , ν ) be a measured space , where ν is a σ-finite measure on ( Y , Y ) . Assume that we are given some observed data D generated from a probabilistic model p ( D |y ) that is parameterised by a latent variable y ∈ Y . Further assuming that y is drawn from a certain prior p0 ( y ) , the posterior density of the latent variable y given the data D is then defined by : p ( y|D ) = p ( y , D ) p ( D ) = p0 ( y ) p ( D |y ) p ( D ) , where the normalisation constant p ( D ) = ∫ Y p0 ( y ) p ( D |y ) ν ( dy ) is called the marginal likelihood or model evidence . For many complex models arising in Bayesian Inference , we do not know how to directly sample from the posterior density , nor do we know the value of the marginal likelihood . To remedy this problem , the Power Descent is a Variational Inference algorithm that introduces a certain measurable space ( T , T ) and offers to approximate the posterior density by a variational family Q of the form Q = { y 7→ ∫ T µ ( dθ ) k ( θ , y ) : µ ∈ M } , ( 1 ) where M is a convenient subset of M1 ( T ) , the set of probability measures on ( T , T ) . In doing so , it extends the usual parametric variational family Q = { y 7→ k ( θ , y ) : θ ∈ T } since it amounts to putting a prior over the parameter θ in the form of a probability measure . In particular , one strong motivation behind the variational family ( 1 ) is that it recovers mixture models as a special case . To see this , let J ∈ N⋆ , Θ = ( θ1 , . . . , θJ ) ∈ TJ and define the simplex of RJ by : SJ = λ = ( λ1 , . . . , λJ ) ∈ RJ : ∀j ∈ { 1 , . . . , J } , λj ⩾ 0 and J∑ j=1 λj = 1 . ( 2 ) Observe then that when µ is chosen as a weighted sum of Dirac measures , i.e . µ = ∑J j=1 λjδθj with λ ∈ SJ , the variational family ( 1 ) becomes Q = y 7→ J∑ j=1 λjk ( θj , y ) : λ ∈ SJ , ( 3 ) which corresponds to the class of mixture models parameterised by the mixture weights λ and with fixed mixture components parameters Θ . Here , the j-th component y 7→ k ( θj , y ) could for example be a multivariate Gaussian density with mean θj and known covariance matrix , but one strength of the Power Descent is that it is not limited to a specific choice for k. We will refer to k as a kernel density thereafter and we now introduce some more notation , before stating the optimisation problem solved by the Power Descent . Denote by P the probability measure on ( Y , Y ) with corresponding density p ( ·|D ) with respect to ν and by K : ( θ , A ) 7→ ∫ A k ( θ , y ) ν ( dy ) the Markov transition kernel on T× Y with kernel density k defined on T × Y . For all µ ∈ M1 ( T ) and all y ∈ Y , we denote µk ( y ) = ∫ T µ ( dθ ) k ( θ , y ) and we let µK denote the probability measure with density µk with respect to ν . Then , letting α ∈ R and assuming that µK and P are both absolutely continuous with respect to ν , the α-divergence between µK and P ( extended by continuity to the cases α = 0 and α = 1 as in [ 24 ] ) is given by Dα ( µK||P ) = ∫ Y fα ( µk ( y ) p ( y|D ) ) p ( y|D ) ν ( dy ) , where fα is the convex function on ( 0 , +∞ ) defined by f0 ( u ) = u− 1− log ( u ) , f1 ( u ) = 1− u+ u log ( u ) and fα ( u ) = 1α ( α−1 ) [ u α − 1− α ( u− 1 ) ] for all α ∈ R \ { 0 , 1 } . Letting α ̸= 1 , the Power Descent seeks to solve the Variational Inference optimisation problem inf µ∈M Dα ( µK||P ) , ( 4 ) meaning that for µ = ∑J j=1 λjδθj ( with corresponding approximating family ( 3 ) ) , it seeks to optimise λ by α-divergence minimisation i.e . to select mixture components according to their overall importance in the set of components parameters . More generally , the Power Descent aims at solving inf µ∈M Ψα ( µ ; p ) , ( 5 ) where p is any measurable positive function on ( Y , Y ) and where for all µ ∈ M1 ( T ) , Ψα ( µ ; p ) =∫ Y fα ( µk ( y ) /p ( y ) ) p ( y ) ν ( dy ) . Crucially , ( 4 ) can be reframed as an instance of ( 5 ) that sets p = p ( · , D ) in ( 5 ) and that thus does not involve the unknown normalising constant p ( D ) anymore ( see Appendix A.1 ) . In the following , the dependency on p in Ψα may be dropped throughout the paper for notational ease when no ambiguity occurs and we next present the Power Descent algorithm . The Power Descent algorithm . Let α ̸= 1 . Given an initial probability measure µ1 ∈ M1 ( T ) such that Ψα ( µ1 ) < ∞ , η > 0 and κ such that ( α− 1 ) κ ⩾ 0 , the Power Descent algorithm introduced in [ 19 ] is an iterative scheme which builds the sequence of probability measures ( µn ) n∈N⋆ µn+1 = Iα ( µn ) , n ∈ N⋆ , ( 6 ) where for all µ ∈ M1 ( T ) , the one-step transition µ 7→ Iα ( µ ) is given by Algorithm 1 and where for all v ∈ Domα , Γ ( v ) = [ ( α− 1 ) v + 1 ] η/ ( 1−α ) [ and Domα denotes an interval of R such that for all θ ∈ T and all µ ∈ M1 ( T ) , bµ , α ( θ ) + κ and µ ( bµ , α ) + κ ∈ Domα ] . Algorithm 1 : Power descent one-step transition ( Γ ( v ) = [ ( α− 1 ) v + 1 ] η/ ( 1−α ) ) 1 . Expectation step : bµ , α ( θ ) = ∫ Y k ( θ , y ) f ′α ( µk ( y ) p ( y ) ) ν ( dy ) 2 . Iteration step : Iα ( µ ) ( dθ ) = µ ( dθ ) · Γ ( bµ , α ( θ ) + κ ) µ ( Γ ( bµ , α + κ ) ) The Power Descent is known to have a gradient-based structure since θ 7→ bµ , α ( θ ) acts as the gradient of µ 7→ Ψα ( µ ) and η plays the role of a learning rate [ 19 ] . Furthermore , a remarkable property of the Power Descent is that under ( A1 ) as defined below ( A1 ) The density kernel k on T× Y , the function p on Y and the σ-finite measure ν on ( Y , Y ) satisfy , for all ( θ , y ) ∈ T× Y , k ( θ , y ) > 0 , p ( y ) > 0 and ∫ Y p ( y ) ν ( dy ) < ∞ . the Power Descent ensures a monotonic decrease in Ψα at each step for all η ∈ ( 0 , 1 ] ( this result is a special case of [ 19 , Theorem 1 ] with Γ ( v ) = [ ( α − 1 ) v + 1 ] η/ ( 1−α ) that is recalled in Theorem 4 of Appendix A.2 for the sake of completeness ) . Under the additional assumptions that κ > 0 and sup θ∈T , µ∈M1 ( T ) |bµ , α| < ∞ and Ψα ( µ1 ) < ∞ , ( 7 ) the Power Descent is also known to converge towards its optimal value at an O ( 1/N ) rate when α > 1 [ 19 , Theorem 3 ] . On the other hand , when α < 1 , the convergence towards the optimum as written in [ 19 ] holds under different assumptions including ( A2 ) ( i ) T is a compact metric space and T is the associated Borel σ-field ; ( ii ) for all y ∈ Y , θ 7→ k ( θ , y ) is continuous ; ( iii ) we have ∫ Y supθ∈T k ( θ , y ) × supθ′∈T ( k ( θ′ , y ) p ( y ) ) α−1 ν ( dy ) < ∞ . If α = 0 , assume in addition that ∫ Y supθ∈T ∣∣∣log ( k ( θ , y ) p ( y ) ) ∣∣∣p ( y ) ν ( dy ) < ∞ . so that [ 19 , Theorem 4 ] , that is recalled below under the form of Theorem 1 , states the convergence of the Power Descent algorithm towards the global optimum . Theorem 1 ( [ 19 , Theorem 4 ] ) . Assume ( A1 ) and ( A2 ) . Let α < 1 and let κ ⩽ 0 . Then , for all µ ∈ M1 ( T ) , Ψα ( µ ) < ∞ and any η > 0 satisfies 0 < µ ( Γ ( bµ , α + κ ) ) < ∞ . Further assume that η ∈ ( 0 , 1 ] and that there exist µ1 , µ⋆ ∈ M1 ( T ) such that the ( well-defined ) sequence ( µn ) n∈N⋆ defined by ( 6 ) weakly converges to µ⋆ as n → ∞ . Finally , denote by M1 , µ1 ( T ) the set of probability measures dominated by µ1 . Then the following assertions hold ( i ) ( Ψα ( µn ) ) n∈N⋆ is nonincreasing , ( ii ) µ⋆ is a fixed point of Iα , ( iii ) Ψα ( µ⋆ ) = infζ∈M1 , µ1 ( T ) Ψα ( ζ ) . The above result assumes there must exist µ1 , µ⋆ ∈ M1 ( T ) such that the sequence ( µn ) n∈N⋆ defined by ( 6 ) weakly converges to µ⋆ as n → ∞ , that is it assumes the limit already exists . Our first contribution consists in showing that this assumption can be alleviated when µ is chosen a weighted sum of Dirac measures , that is when we seek to perform mixture weights optimisation by α-divergence minimisation . In doing so , we provide additionnal theoretical justification behind the use of the Power Descent when α < 1 , which is crucial in practice for reasons already outlined in the introduction and further detailed for the particular case of the Power Descent in Appendix A.3 .

Summary: The authors consider the problem of variational inference with Renyi divergence for learning mixture models. They improve the analysis of the iterative method Daudel et al. Specifically, they establish fast $O(1/n)$ convergence of the method when $\alpha < 1$. They then analyze the method when $\alpha = 1$ by taking the limit of the algorithm updates. They call this algorithm Renyi descent and establish that it also converges at an $O(1/n)$ rate. They experimentally compare their approach with entropic mirror descent.