SoftCVI: Contrastive variational inference with self-generated soft labels
Authors: Daniel Ward
Mark Beaumont
Matteo Fasiolo
Venue: ICLR 2025 Spotlight (2025)
Keywords: contrastive learning
variational inference
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We present SoftCVI, a reframing of variational inference as a contrastive estimation problem, and use it derive stable, mass-covering objectives.

Abstract: Estimating a distribution given access to its unnormalized density is pivotal in Bayesian inference, where the posterior is generally known only up to an unknown normalizing constant. Variational inference and Markov chain Monte Carlo methods are the predominant tools for this task; however, both are often challenging to apply reliably, particularly when the posterior has complex geometry. Here, we introduce Soft Contrastive Variational Inference (SoftCVI), which allows a family of variational objectives to be derived through a contrastive estimation framework. The approach parameterizes a classifier in terms of a variational distribution, reframing the inference task as a contrastive estimation problem aiming to identify a single true posterior sample among a set of samples. Despite this framing, we do not require positive or negative samples, but rather learn by sampling the variational distribution and computing ground truth soft classification labels from the unnormalized posterior itself. The objectives have zero variance gradient when the variational approximation is exact, without the need for specialized gradient estimators. We empirically investigate the performance on a variety of Bayesian inference tasks, using both simple (e.g. normal) and expressive (normalizing flow) variational distributions. We find that SoftCVI can be used to form objectives which are stable to train and mass-covering, frequently outperforming inference with other variational approaches.

Published as a conference paper at ICLR 2025
SOFTCVI: CONTRASTIVE VARIATIONAL INFERENCE WITH SELF-GENERATED SOFT LABELS
Daniel Ward', Mark Beaumont", Matten Fasiolo' Schooo of Mathematics, Bristol University, UK
Sccoool of Biological Sciences, Bristol University. UK
ABSTRACT
Estimating a distribution given access to its unnormalized density is pivotal in Bayesian inference, where the posterior is generally known only up unknown normalizing constant. Variational inference and Markov chain Monte Carlo meth- ods are the predominant tools for this task; however, both are often challenging apply reliably, particularly when the posterior has complex geometry. Here, we introduce Soft Contrastive Variational Inference (SofiCVII, which allows a family of variational objectives to be derived through a contrastive estimation framework. The approach parameterizes classifier in terms of variational distribution, re- framing the inference task as a contrastive estimation problem aiming to identify single true posterior sample among a of samples. Despite this framing, we do not require positive or negative samples, but rather learn by sampling the vari- ational distribution and computing ground truth soft classification labels from the unnormalized posterior itself. The objectives have zero variance gradient when the variational approximation exact, without the need for specialized gradient estimators. We empirically investigate the performance inference tasks, using both simple (c.g. normal) and expressive (normalizing flow) variational distributions. We find that SoftCVI can be used to form objectives which are stable to train and mass-covering, frequently outperforming inference with other variational approaches.
a variety of Bayesian
INTRODUCTION
Consider a probabilistic model with a set of parameters 0, for which given a set of observations Lobs we wish to infer posterior p(0) Zobs). Unless the model takes particularly convenient form, the posterior cannot be directly computed. However, typically p(0, Fobs) is available, related to the posterior by p(0 xobs) p(0, 2obs) p/((aa)) where p(Eots) an intractable normalizing constant p(Zats)) = O0, &orr )d0. In these cases, computational methods such as Markov chain Monte Carlo (MCMC) (Hastings, 1970; Metropolis al., 1953) variational inference (Jordan et al., 1999; Kingma & Welling, 2014) are required to perform inference.
Variational inference approaches the inference task as an optimization problem by defining vari- ational distribution (o (0) and minimizing its divergence to the true posterior p(0)2ats). The per- formance and reliability of variational inference is dependent on numerous factors, including the choice of divergence, variational distribution and parameter initialization. Whilst choosing diver- gence that favors mass-covering posterior estimates may facilitate reliable inference, many cases these divergences introduce bias or are less stable to train, leading to worse performance (Dhaka et al., 2021; Naesseth et al., 2020). Alongside difficulties assessing performance (Yao et al., 2018), these issues hinder practical applications of variational inference, particularly in applications such as scientific research, where accurate uncertainty quantification crucial.
In contrast to variational inference, contrastive learning is generally used to perform inference when the likelihood function is only available through an intractable integral, such as for fitting energy- based models for performing simulation-bad based inference (Gutmann al., 2022). Learning is achieved by contrasting positive samples with negative samples, with the latter often generated by augmenting positive samples by drawing samples from carefully chosen noise distribution. A

---

Published as a conference paper at ICLR 2025
common theme is to leverage the invariance of classification based objectives to unknown normal- izing constants to enable learning where likelihood- based methods are infeasible.
Contribution
We show that contrastive estimation can be used to derive a family of variational objectives, terming the approach Soft Contrastive Variational Inference (SoftCVI). The task of fitting the posterior ap- proximation is reframed as a classification task aiming to identify a single true posterior sample among set of samples. Instead of using explicitly positive and negative samples, we show that for arbitrary samples from a proposal distribution, we can generate ground truth soft classification labels using the unnormalized posterior density itself. The samples and corresponding labels are used for fitting a classifier parameterized in terms of the variational distribution, such that the optimal classi- fier recovers the true posterior. SoftCVI enables derivation of stable, mass-covering objectives, and performance is demonstrated across a series of experiments using both simple and flexible (normal- izing flow) variational distributions. Compared alternative variational objectives, we find SoftCVI generally yields better calibrated posterions with a lower forward Kullback-Leible (KL) divergence to the true posterior. We provide a pair of Python packages, pyrox and softevi validation, which provide the implementation, and the code for reproducing the results of this paper, respectively.
2 SOFTCVI
In order to fit variational distribution with SoftCVI, must define proposal distribution x x(0), negative distribution p- (0), and the variational distribution itself, (0). At each optimization step. three steps are performed which allow fitting the variational distribution:
1. Sample parameters {0x) (0) from the proposal distribution T(0).
2. Generate corresponding ground truth soft labels y E (0, 1)x for the task of classifying between positive and negative samples, presumed to be from (oba) and p (0), respec- tively.
3. Use {0x) along with the soft labels y to optimize a classifier parameterized in terms of the variational distribution go (0), such that the optimal classifier recovers the true posterior.
The choice proposal distribution in step one will influence the region in which learning fo- cused. Throughout the experiments here, we take the intuitive and convenient choice of using the variational distribution itself as the proposal distribution (0) = q (0), which over the course of training directs learning towards regions with reasonable posterior mass. The remainder of this section is structured as follows: section 2.1 details the assignment of ground truth labels to arbi- trary proposal samples; section 2.2 discusses the parameterization and optimization of the classifier; finally, section 2.3 considers the choice of negative distribution.
2.1 GENERATING GROUND TRUTH SOFT LABELS
Classification can be used to estimate the density ratio between positive and negative distributions (Gutmann & Hyvärinen, 2012; Hastie, 2009; Oord et al., 2018; Thomas al., 2022). Conversely, if the positive and negative densities are known, the density ratio and ground truth classification labels can be directly computed. Consider
have of { samples from the proposal distribution T (0). SoftCVI reframes inference as a classification task, where the problem is chosen such that analytical ground truth labels can be assigned to the K samples. Specifically, if consider the true posterior p(0 (cbss) to be the positive distribution, and p= (0) to be a chosen negative distribution, the ratio p( @ [ooss)/ / (0x) represents the relative likelihood of a sample being a true posterior sample under the classification problem. By contrasting the K samples, assuming setting where (0, 1 J==1 consists of exactly one positive sample from the true posterior p(0 (robs), and K negative samples from (0), the optimal classifier is then given by
p(OA (Dots)) (0x)
1/x:
k= Ol (aax)) \/p (0xz)
where ys is the probability on the interval (0, 1) that 0g is the positive sample among all the consid- ered samples, and EK lik 1. This approach of contrasting samples is invariant multiplicative
2

---

Published as a conference paper at ICLR 2025
scaling of the density ratio /ool==p meaning that access only to unnormalized forms of p(0 2ots) and p (0) is sufficient for computing the labels. Since typically p(0. xebs) is known and proportional to p abbs), can analytically compute the labels
p(0x a Dotm) 1/p (0x) Σ K
Ok Dots (0x))
(2)
y = softmax(z), where 24 = log p(0x. Tobs)
(0x)
where p (0) may be unnormalized. While contrastive estimation methods generally use explicitly positive and negative examples with hard labels E {0, 1) here, in our framework, we lack access to positive samples from p(0 aatx), and we may not have access to samples from p (0). Nonetheless, by instead generating samples from proposal distribution, {0s k=] x0)), and by assigning soft labels to these samples using eq. (2), it still possible train a classifier which at optimality recovers the true posterior, as we will discuss in the subsequent section.
2.2 PARAMETERIZATION AND OPTIMIZATION OF THE CLASSIFIER
Analogously to the computation of the labels in eq. (2), we can parameterize the classifier in terms of the ratio between the variational and negative distribution
9 1 (0x)
U =
E (0x)
)/p (0x))
(3)
i softmax(3)), where b = log I (Ok)
p- (0s)
Given a set of samples (0x K
(0) and corresponding labels y computed using eq. (2), an optimization step with respect to Φ can be taken to minimize the softmax cross tross-entropy loss function
K
go (n((0))pp000) 1P=190 9/
Lsoncvi(d: (0x) =
log (ūk)
Σ log
(4)
This is equivalent maximizing the categorical log-likelihood, equivalently, minimizing the forward KL divergence between the true categorical label distribution and the predicted distribution (appendix A.4). While wide variety of divergence measures, such f-divergences, could also be employed, which we briefly discuss in appendix A.4, we leave detailed exploration of these for future work.
Let θ represent the support of the proposal distribution, with (obs) and (0) supported on the same set, and p (0) supported on superset @. Assume that there exists a d such that pl p((( 1 (). The optimal classifier, i.e. which minimizes eq. (4) for all {0 {0 E OK
recovers the true posterior, for any valid choice of K. This result follows from the fact the optimal classifier must learn the density ratio between the positive and negative distributions up constant across +
p(0 Lobs 1/p (0)
(() 96 (0)/p" (())
p(0 (.bb))
q (0).
Due to the shared support, integrating both sides over the proposal support θ gives
[ p(0 p(0)zobs)) (obs)
= (0),
(5)
c= 1.
Thus the optimal classifier recovers the true posterior
p (oiss) (0).
6
'Or equivalently, the optimal classifier minimizes E,o, ~x(0 [Sstevvi (Ø: (0x 1k v)], for which we view eq. (4) as single sample Monte Carlo approximation.
3

---

Published as a conference paper at ICLR 2025
Importantly, this suggests if (0) p(0 aabss) have non-zero regions outside the support of i ((),, then p(O) Fobs) = c 96 can be satisfied within & without recovering the true posterior, as the integrals in eq. (5) will not both evaluate to one. This is a consequence of the incentive to learn the density ratio (up to a constant) being localized to the sampled region. Ensuring =((), 2 (0) and the same set @ be achieved by defining a variational distribution supported on the same set as the posterior p(O [ooss) alongside using the variational distribution as the proposal distribution π (0) (0).
p( (())a))s)) are supported
When the optimal classifier is achieved, we show in appendix A.I that the gradient is zero for any set {0x]/=1 ④ K and consequently has zero variance. This is a generally desirable property that is not present in many variational objectives, although specialized gradient estimators have been developed which in some cases can address this issue (Roeder al., 2017: Tucker et al., 2018). An algorithm outlining the overall approach of SoftCVI is shown in algorithm 1.
Algorithm 1: SoftCVI
Inputs: p(0, To0s), =(0), (0), 9s, (0), number of samples K 2 2, optimization steps N, learning rate 1)
for i in 1 N do
1
Sample (0x) ~ (0)
Compute soft labels y = softmax(z), where 24 log pl (tmm)
2
p (0a)
Update 01+1 Φ 1/ V/o:i (m 3) using the cross-entropy loss, eq. (4)
end
2.3 CHOICE OF THE NEGATIVE DISTRIBUTION
The choice of negative distribution is well-known challenge in contrastive learning, with the op- timal negative distribution often differing significantly from the positive distribution (Chehab et al., 2022). In contrast standard applications of contrastive estimation, in SoftCVI, the negative distri- bution is never sampled, meaning the impact of the choice is limited to its influence on the objective's properties. To better understand this, we can rewrite the softmax cross-entropy objective from eq. (4) by separating out the log term
K
90 (Ok) yo log
K
K
4s (0w))
Lsoncvi(0; 10x) =
yh log
p (0x)
P (0N.)
k=1
k=1
k==1
K
L (0:)
yk log go (0x) ) log
p (0x) + const,
1=1
R=1
where in the last line we remove the denominator from the first term into constant as it is indepen- > dent of d, and use Lkm 3/x 1. The first term encourages placing posterior mass the samples likely to be from the true posterior, and the second term penalizes the sum of the ratios. In addition to appearing in the second term, the negative distribution choice also influences yı through eq. (2).
We focus on setting p (0) = T0)), where a E [0, 1] is a tempering hyperparameter which in- terpolates between directly using the proposal distribution as the negative distribution when a and using an improper flat negative distribution when a = O. Lower values of a tend to favor more mass-covering solutions by increasing the relative penalization of samples in higher density regions in (0) through the second term of eq. (7). However, a too small choice for can lead to problemat- ically high variances of the log ratios 24 log]p((,s> Tobss/ (0x)) and é = log|go (0x)/p (0x)). Particularly in nigh-dimensional problems, this can result in labels and predictions with very few non-zero values, meaning few samples contribute significantly to the loss. In terms of eq. (7), the high variance of Σy leads to failure to sufficiently penalize the ratios in lower density regions of (0), as log ΣK k ( is dominated by the few largest ratios. In practice, we empirically show this leads to "leakage" of mass into regions of negligible posterior density (fig. 2), and a weak signal-to-noise ratio (SNR) (see appendix A.3).
As described in the introduction to section 2, we choose the proposal distribution to equal the vari- ational distribution, meaning the choice above is equivalent using p -0) (0)))) " However,
4

---

Published as a conference paper at ICLR 2025
when computing the objective gradient, treat p (0) independent of Φ. which practically can be achieved by applying the stop-gradient operator present in automatic differentiation pack- ages." In appendix A.7, we also consider parameterizing the negative distribution using p(0, oohs)" however, this choice introduces the problematic ratio q (0) mede-seeking solutions and poorly calibrated posteriors.
(0(00)p(( //(O. Tobs), which leads to favoring of
3
RELATED WORK
3.1 VARIATIONAL INFERENCE
Given access to an unnormalized posterior density, p(0, Roms), variational inference allows optimiz- ing variational distribution 96 (0) ( to approximate the posterior. For certain model classes there exist closed form solutions which can be exploited during optimization (e.g. Blei et al., 2017: Ghahramani & Beal, 2000; Parisi, 1988). However, the lack of broad applicability of these methods hinders the ability of practitioners to freely alter the model and variational family. As such, we instead focus on methods which place minimal restrictions the model form and variational family.
3.1.1 EVIDENCE LOWER BOUND
The most commonly used variational objective is to minimize the negative Evidence Lower Bound (ELBO) or equivalently, minimizing the reverse KL divergence between the posterior approximation and the true posterior
D DKL 1a (0) p(0 xots)] = ( [log ( (0) - p(O xobs)]
= E (0) (log 9 (0) log p(0, xabs)] + const.
As there is no general closed-form solution for the divergence, a Monte Carlo approximation is often used (Kingma & Welling, 2014)
K Σ K
1
(o)
log 20b)) where (0x) (0)
1=
Although straightforward apply, the reverse KL divergence heavily punishes placing mass in gions with little mass the true posterior leading to mode-seeking behavior and lighter tails than p(0) In addition to underestimating uncertainty, the light tails may also lead to the approx- imation performing poorly in downstream tasks, such as when acting as a proposal distribution in importance sampling (Chatterjee & Diaconis, 2018; Gelman & Meng, 1998; Müller al., 2019b; Yao et al., 2018) or for reparameterizing MCMC (Hoffman et al., 2019).
3.1.2 SELF-NORMALIZED IMPORTANCE SAMPLING FORWARD KL DIVERGENCE
In order address the limitations of the ELBO, numerous alternative objectives have been proposed that encourage more mass-covering behavior, such as the importance weighted ELBO (Burda et al., 2015), the Rényi divergence, (Li & Turner, 2016), x-divergence (Dieng et al., 2017), and methods targeting the forward KL divergence (Jerfel et al., 2021; Naesseth et al., 2020). In this section, we will focus on the Self-Normalized Importance Sampling Forward Kullback-Leible (SNIS-IKL) divergence estimator introduced by Jerfel et al. (2021). Specifically, standard importance weighted estimate of the forward KL divergence is given by
p(0, Wohs))
D (p abt)))
1 (0) (
(n log
+ const,
(10)
(+)
where w0) = p(0 (at))) (0) are the importance weights. Computing a set of nolf-normalized p(o.. \/m(0)
weights with elements x k Σe ra..)/ and using these alongside a Monte Carlo approximation of eq. (10), yields the objective
K
Σ (0)x log p(O,. Fobs) k=1
LSNIS-KKL (0
(11)
yeb (Ox)
Without preventing gradient flow through (0), the choice -0) (0)) (o leads to zero gradi- ents the parameterization of the classifier becomes go (0)0 (0).
5

---

Published as a conference paper at ICLR 2025
This approach introduces bias in the approximation of the forward KL vanishing with order ((1/K) (Agapiou et al., 2017). Generally, the proposal is chosen to equal the variational distribution :T()- q (0), and the proposal parameters are held constant under differentiation.
3.2 COMPARISON SOFTCVI AND SNIS-FKL
In this section, we demonstrate that in the special case of choosing p (0) = -0)= = (0), both SoftCVI and SNIS-fKL produce gradients that are equal in expectation, but the SoftCVI objective naturally includes control variate which ensures the variance of the gradient decreases to zero as the variational distribution approaches the true posterior. We use this result to suggest an alternative, lower-variance gradient estimator for the SNIS-fKI. objective,, exactly equivalent to optimizing the SoftCVI objective with p (0) = π (0) = (0).
Noticing that from eq. (11), the numerator term does not depend on Φ, we can equivalently write the SNIS-fKL objective as
SSNIS-IKL (4: (0x) = Σ (0)) e log Us (0x) + const.
(12)
In the special case under consideration, the self-normalized weights w( from the SNIS-fKL ob- jective, and the ground truth labels y from SofICVI are computed identically. This allows rewriting the SoftCVI objective from eq. (7) as the sum of the SNIS-fKL objective and the normalization term
K gs (0x)) (0k)
Csoncvi(o; (0 = CSNIS-IKL (o;: (0+) + log
+ const,
(13)
k=1
where we show in appendix A.2 that when p p00)) (0) the normalization term gradient is
K
K
V
log
V s log q4 (fx).
(14)
Φ
p (0k)
K
k=1
Since (,)(0) V log (0)] = E. (e) V log qs (0) O, the inclusion or omission of the normal- ization term does not change the gradient in expectation over sets of {0. ) j= but it significantly influences the variance. As shown in appendix A. the gradient variance for SoftCVI decreases to zero as the variational distribution approaches the posterior. In contrast, this implies the variance
of the SNIS-IKL objective approaches the variance of 1 Ex=1 V log go (Ox), which positive, scaling inversely with K.
In addition to providing a novel perspective of the SNIS- objective as training an (unnormalized) classifier, this result also implies straightforward modification to form lowirnvariance SNIS-fKI. gradient estimator
V LI (0.)K_) LSNISSSARI {0x)
V log qo (Ox),
(15)
which is exactly equivalent to optimizing the SoftCVI objective with p (0) = オ x = (0). This approach of lowering the variance of gradient estimator by utilizing V log q (0) control vari- ate has also been applied to other variational objectives, such as the sticking the landing estimator of the ELBO (Roeder al., 2017: Tucker et al., 2018). There is guarantce that the gradient variance will be lower in all instances. However, the variance provably reduces to zero as the variational dis- tribution approaches the true posterior, which allows reasonable SNR to be maintained even when near convergence, which is likely beneficial for performance and stable convergence. We support this claim with the empirical results in section 5, in addition to investigating the signal-to-noise ratio of the objectives in appendix A.3.
3.3 CONTRASTIVE LEARNING
Contrastive learning most commonly allows learning of distributions through the comparison of true samples of negative (noise augmented) samples (Gutmann & Hyvärinen, 2010). Generally,

---

Published as a conference paper at ICLR 2025
and notably in contrast the current work, applications of contrastive learning focus on problems where the likelihood is unavailable, such as for fitting energy-based baaggy models (Gao et al., 2020; Gut- mann & Hyvärinen, 2010; Gutmann et al., 2022; Rhodes & Gutmann, 2019; Rhodes et al., 2020) or performing simulation-bas
inference (Durkan et al., 2020; Greenberg et al., 2019; Gutmann et al., 2022; Hermans et al., 2020; Miller et al., 2022; Thomas et al., 2022). Probably the most widely used objective function in contrastive learning is the InfoNCE loss, proposed by Oord et al. (2018). Given set of samples [0*J containing single true sample with index k", and K negative samples, the loss can be computed as
(a(Ok-) Σk (8
LInfONCE ( (0x log
(16)
where f0) approximates the ratio between the positive and negative distributions (up to a con- stant). This is commonly also presented with an additional sum (or expectation) over different sets of {0& ) The InfoNCE loss can be derived from the softmax cross-entropy loss, eq. (4), by in- putting a one-hot encoded vector of labels y with yk- = and all other elements This results in only the k'-th summation term being non-zero, recovering the InfoNCE loss.
In contrastive methods for simulation- based inference, parameters are sampled from a proposal dis- tribution (0), and used to perform simulations. Learning is achieved through comparing positive parameter-output pairs from p(a: 0) (0), to negative (mismatched) pairs drawn marginally from T (O), where π (xx) f p(x 0 (O)d0. Similar the current work, normalized approxima- tion to the posterior is often included in the classifier parameterization. Specifically, the classifier is parameterized using the ratio between the approximate posterior and the prior qe ( =p/p(0) and is optimized using the InfoNCE objective (Durkan et al., 2019; Greenberg et al., 2019).
Adversarial methods which are closely related to contrastive learning are often used in conjunction with variational inference (Huszár, 2017; Makhzani et al., 2015; Mescheder et al., 2017). A key idea rewrite the reverse KI. divergence from eq. (8) include density ratio, for example
log Lsb (0)
DKLlas (0) p [0s) = Egg (0)
p(0) log logg((z..o x0 10) + const.
(17)
The ratio is then substituted with an approximation, (0) R ( (0)p((0) trained using a separate classification objective. This allows the ELBO to be optimized without computing the log density log (0), enabling the use of expressive implicit models for qs (0). In contrast, SoftCVI directly optimizes the variational distribution with a contrastive objective without separate classification model, though it does require the variational distribution to be explicitly defined.
Recently, there has been investigation of the use of soft (or ranked) labels in contrastive learning, frequently using cross-entropy-like loss functions. In contrast to the current work, these methods focus on improving performance in the standard context of contrastive learning. where evaluation of the likelihood is infeasible. As such, the soft labels cannot be computed exactly, and instead are generated using other methods, such label smoothing (Hugger & Uhlmann, 2024) by utilizing a similarity metric such as cosine similarity between embeddings of positive and negative samples (Feng & Patras, 2022; Hoffmann et al., 2022; Park al., 2024).
EXPERIMENTS
We focus on Bayesian inference tasks for which reference posterior samples {0} (Q0),,, Nut are available to enable reliable assessment of performance. However, in appendices A.5 and A.6, we also consider application of SoftCVI for training variational autoencoders and Bayesian neural networks. respec- tively. We use p (0) = (0)) and focus results on two choices, a = 0.75 and a = 1. Performance compared to using the ELBO or the SNIS-iKL divergence (Jerfel et al., 2021). For each task and objective, 50 independent runs were performed. Where possible (i.e. an analytical posterior is available), different observations Lohs were generated from the model for each run. For tasks where the reference posterior is created through sampling methods we relied on reference posteriors pro- vided by PosteriorDB (Magnusson et al., 2024) or each run sampled from the available observations if multiple are present. For all methods. K 8 samples from q,p (l) were used during computation of the objectives, and 50,000 optimization steps were performed with the Adam optimizer (Kingma & Ba, 2014).
(Lueckmann et al., 2021) and for
7

---

Published as a conference paper at ICLR 2025
Software. Our implementation and experiments made wide use of the python packages JAX (Brad- bury et al., 2018), equinox (Kidger & Garcia, 2021), NumPyro (Phan et al., 2019), FlowJAX (Ward, 2024) and optax (DeepMind et al., 2020).
4.1 METRICS
Coverage probabilities. Posterior coverage probabilities have been widely used to assess the liability of posteriors (e.g. Cannon et al., 2022: Cook et al., 2006; Hermans et al., 2021; Prangle al., 2014; Talts al., 2018; Ward al., 2022). Given nominal frequency 7 E [0, 1], the met- ric assesses the frequency at which true posterior samples fall within the 100+% highest posterior density region (credible region) of the approximate posterior. For given 7, the actual frequency exceeds 7, the posterior is conservative for that coverage probability; if it is lower, then the posterior overconfident. A posterior said to be well-calibrated if the actual frequency matches r for any choice of y. A well-calibrated or somewhat conservative posterior is needed for drawing reliable scientific conclusions, and as such an important property to investigate. We estimate the actual coverage frequency for posterior estimate as
Niet
1
1 100 E o (m)mm)))))
(18)
Nie
where Il is the indicator function, and HDR represents the highest posterior density region, inferred using the density quantile approach of Hyndman (1996).
Log probability of 0* Looking the probability of either reference posterior samples or the ground truth parameters in posterior approximation is common metric for assessing performance (e.g. Greenberg et al., 2019; Lueckmann et al., 2021; Papamakarios Murray, 2016). compute this independently for each posterior approximation follows
Nas
log gob (0)).
(19)
Nie
This metric also approximates the negative forward KL divergence between the true and approxi- mate posterior up to a constant, which is quantity known control error in importance sampling (Chatterjee & Diaconis, 2018)
-DxL. (pp0ppOOaa (ots)) [0)] -pp0ppp... (oee.)/logg P (0 (ohss) - log 4 (0)],
Nat
log 96 (0,) + const,
(20)
N
Posterior mean accuracy. A posterior that is overconfident but with the correct posterior mean would perform poorly based on the aforementioned metrics, can form good point estimates for the parameters O. To assess this, we choose to measure the accuracy using the negative L of the standardized difference in posterior means
mean (0") - mean
(21)
std(@ (a)
where mean(0") and me) are the mean vectors of the reference and posterior approximation samples respectively, and std(O") is the vector of standard deviations of the reference samples.
4.2
TASKS
brief description of each model is given below. For a complete description, see appendix A.8.
Eight schools. A classic hierarchical Bayesian inference problem, where the aim is to infer the treatment effects of a coaching program applied to eight schools (Gelman et al., 1995; Rubin, 1981). The parameter set is 0 {μ, T, m, where μ is the average treatment effect across the schools, the standard deviation of the treatment effects across the schools and m is the eight-dimensional vector of treatment effects for each school. For the posterior approximation qo (0),
normal
8

---

Published as a conference paper at ICLR 2025
Eight Schools
Linear Regression
SLCP
GARCH
Actual nominal
0.00
-0.15
(a)
-0.30
-0.45
Nominal
Nominal
Nominal
Nominal
Posterior mean Log prob. 0r
-22.350
55
10
2.08
(b)
-22.375
54
2.00
1.92
-20,00
53
1.81
0.0
accuracy
-0.18
-1.5
0.24
0.40
0.9
-3.0
0.45
0.30
11.2
0.50
-0.36
SoftCVI(a=0.75)
SoftCVI(a==)
SNIS-fKL
ELBO
The posterior performance metrics (see section 4.1). a) The nominal coverage frequency against the average difference between the nominal and actual coverage frequency. Well-calibrated methods follow the black dotted line (y 0). whereas conservative methods fall above, and over- confident methods below. b) The average probability of the reference posterior samples in the ap- proximate posterior. c) The accuracy of the approximate posterior mean, calculated as the negative [--norm between the mean of the standardized reference and approximate posterior samples.
Figure I:
distribution for μ4, folded Student's distribution for 7 (where folding is equivalent to taking absolute value transform), and a Student's distribution for m.
Linear regression. A Bayesian linear regression model with parameters 0 =[B, 11), where 3 g550 is the regression coefficients, and R is the bias parameter. The covariates X E R (200x500) are sampled from a standard normal distribution, with targets sampled from ~ ((X33 + p. 1). The posterior approximation (0) is implemented
fully factorized normal distribution.
SLCP. The Simple Likelihood Complex Posterior (SLCP) task introduced in (Papamakarios et al., 2019). This task parameterizes multivariate normal distribution using 5-dimensional vector O. Due to squaring in the parameterization, the posterior contains four symmetric modes. For the poste- rior approximation q (O), we use four layer masked autoregressive flow, with rational quadratic spline transformer (Durkan et al., 2019; Germain et al., 2015; Kingma et al., 2016; Papamakarios al., 2017). GARCH(1,1). A Generalized Autoregressive Conditional heteroscedasticity (GARCH) model (Bollerslev. 1986). GARCH models are used for modeling the changing volatility of time series data by accounting for time-narying and autocorrelated variance in the eтror terms. The observa-
tion consists of 200-dimennional time series æ, where each element 2; is drawn from normal distribution with mean μ and time-varying variance of The variance of is defined recursively by the update of = ag + a1(xx;11 μ1)² + 3,o7-1, where aj and B, control the contribution from the previous observation and previous variance term, respectively. To parameterize 46 (0), we use a normal distribution for j4 and log normal distribution for ag- For &1 and 81 we use uniform distributions constrained to the prior support, transformed with rational quadratic spline (Durkan et al., 2019). To allow modeling posterior dependencies, the distribution over 3, is parameterized function of og and on using neural network.

---

Published as a conference paper at ICLR 2025
SoftC(
SoftCVI(a=0.55)
SoftCVI(a=1)
SNIS-IKL
ELBO
2-dimensional posterior marginal for single run of the SLCP task, with the reference posterior samples shown in green.
Figure 2:
5 RESULTS
Across all tasks and metrics considered, the SoftCVI derived objectives performed competitively with the ELBO and SNIS-IKL objectives (fig. 1). Overall, SoftCVI using a negative distribution (0)) with
0.75 outperformed the other methods, giving rise to better calibrated posteriors and tending to place more mass on average on the reference posterior samples, indicating lower forward KL divergence the true posterior.
key comparison, is between SoftCVI with a and the SNIS-fKI. objective, which give gradient estimators that equal in expectation (see section 3.2). With the exception of the eight schools task, where both methods performed similarly, SoftCVI with a = 1 tended to place more mass on the reference samples and yielded better calibrated posteriors. These results highlight the benefit of the reduced gradient variance provided by SoftCVI when the variational distribution is sufficiently close the true posterior. Further, the performance discrepancy becomes more pronounced for smaller choice of K (see figs. and 8 and in the appendix). This can be explained due to the variance of the ENIS-fKI. objective gradient scaling inversely with K, meaning the control variate naturally included by the SoftCVI objective becomes more crucial. Both SoftCVI and SNIS-fKI. tended to place more mass on the reference samples when trained with a larger K, but this comes with an increase in computational cost (fig. 7).
the SLCP task, which yields complex posterior geometry with four symmetric modes, only the SoftCVI objectives performed well. The ELBO objective often resulted in poor distribution of the mass across the modes, sometimes missing modes entirely (fig. 2; see also fig. in the appendix). In contrast, SNIS-fKL, though less mode-seeking, frequently approximated the individual modes poorly. This observation aligns with previous work suggesting the unreliability of existing mass- covering objectives (Dhaka et al., 2021). To illustrate the impact of the choice of negative distribu- tion, fig. 2 also shows a posterior trained with a flat negative distribution by setting = O. The flat negative distribution does not sufficiently penalize placing mass qø (0) in regions of negligible posterior density, leading to "leakage" of mass (see section 2.3).
6 CONCLUSION
In this work, we introduced SoftCVI, novel framework for deriving variational objectives moti- vated through contrastive learning. Our experiments across various Bayesian inference tasks indi- cate that SoftCVI often outperforms other variational objectives, producing posterior approxima- tions with better coverage properties and a lower forward KL divergence to the true posterior. The performance differennc is particularly notable for tasks with complex posterior geometries which require flexible density estimators. SoftCVI bridges between variational inference and contrastive estimation, which we hope will open up new avenues for further research. Both experimental and theoretical work would be beneficial to further guide the choice of negative distribution. Further, it would be interesting to explore different choices of classification objectives, equivalently, dif- ferent choices of divergences between the categorical labels and predictions. Finally, it could be valuable to investigate how advances from classification and contrastive learning, such label smoothing (Müller et al., 2019a) and temperature scaling (Wang & Liu, 2021), could be adapted to SoftCVI to enhance training stability and further control posterior calibration.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Thank you Song Liu and Michael Gutmann for providing valuable discussion on this work. Com- putational facilities were provided by the Advanced Computing Research Centre, University of Bris- tol http: www. bristol..................c...... ac. uk/acre/.
REFERENCES
Sergios Agapiou, Omiros Papaspiliopoulls, Daniel Sanz-Alonso, and Andrew M Stuart. Importance sampling: Intrinsic dimension and computational cost. Statistical Science, pp. 405-431, 2017.
Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in neural information processing systems, 34:4699 4711, 2021.
Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one distri- bution from another. Journal of the Royal Statistical Society: Series (Methodological), 28(1): 131-142, 1966.
Michael Betancourt and Mark Girolami. Hamiltonian Monte Carlo for hierarchical models. Current frends in Bayesian methodology with applications, 79(30):2-4, 2015.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisti- cians. Journal of the American statistical Association, 112(518):859-81)) 87799987777700 2017.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in
neural network. In International conference on machine learning, pp. 1613-1622. PMLR, 2015.
Tim Bollerslev. Generalized autoregressive conditional heteroskedasticity. Journal of econometrics, 31(3):307-327, 1986.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milnng and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http: //github. com/ google/jax.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. arXiv preprint arXiv: 155..000900.550 2015.
Patrick Cannon, Daniel Ward, and Sebastian M Schmon. Investigating the impact of model mis- specification in neural simulation- based inference. arXiv preprint +XXv:2209.01845 2022.
Souray Chatterjee and Persi Diaconis. The sample size required in importance sampling. The Annals of Applied Probability, 28(2):1099-1133: 2018
Omar Chehab, Alexandre Gramfort, and Aapo Hyvärinen. The optimal noise in noise-contrastive learning is not what you think. In Uncertainty in Artificial Intelligence, pp. 307-316. PMLR, 2022.
Samantha R Cook, Andrew Gelman, and Donald B Rubin. Validation of software for Bayesian models using posterior quantiles. Journal of Computational and Graphical Statistics, 15(3):675- 692, 2006.
Imre Csiszár. On information-tyii measure of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar., 2:299-318, 1967.
DeepMind, Igor Babuschkin, Kate Baumli, Alison Bell, Surya Bhupatiraju, Jake Bruce, Peter Buchlovsky, David Budden, Trevor Cai, Aidan Clark, Ivo Danihelka, Antoine Dedieu, Clau- dio Fantacci, Jonathan Godwin, Chris Jones, Ross Hemsley, Tom Hennigan, Matteo Hessel, Shaobo Hou, Steven Kapturowski, Thomas Keck, lurii Kemaev, Michael King, Markus Kunesch, Lena Martens, Hamza Merzic, Vladimir Mikulik, Tamara Norman, George Papamakarios, John Quan, Roman Ring, Francisco Ruiz, Alvaro Sanchez, Laurent Sartran, Rosalia Schneider, Eren Sezener. Stephen Spencer, Srivatsan Srinivasan, Miloš Stanojević, Wojeiech Stokowiec. Luyu Wang, Guangyao Zhou, and Fabio Viola. The DeepMind JAX Ecosystem, 2020. URL http:
//gi thub. com/ wttpg/ollleeeemmeeeeomm
11

---

Published as a conference paper at ICLR 2025
Akash Kumar Dhaka, Alejandro Catalina, Manushi Welandawe, Michael R Andersen, Jonathan Huggins, and Aki Vehtari. Challenges and opportunities in high dimensional variational inference. Advances in Neural Information Processing Systems, 34:7787-7798, 2021.
Adji Bousso Dieng, Dustin Tran, Rajesh Ranganath, John Paisley, and David Blei. Variational inference via X upper bound minimization. Advances in Neural Information Processing Systems, 30, 2017.
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows. Ad- vances in neural information processing systems, 32, 2019.
Conor Durkan, lain Murray, and George Papamakarios. On contrastive learning for likelihood-free inference. In International conference on machine learming, pp. 2771-2781. PMLR, 2020.
Chen Feng and Ioannis Patras. Adaptive soft contrastive learning. In 2022 26th International Conference on Pattern Recognition (ICPR), pp. 2721-2727. IEFE, 2022.
Ruiqi Gao, Erik Nijkamp, Diederik P Kingma, Zhen Xu, Andrew M Dai. and Ying Nian Wu. Flow contrastive estimation of energy-based models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 7518-7528, 2020.
Andrew Gelman and Xiao-Li Meng. Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statistical science, pp. 163-185, 1998.
Andrew Gelman, John Carlin, Hal Stern, and Donald Rubin. Bayesian data analysis. Chap- man and Hall/CRC, 1995.
Mathieu Germain, Karol Gregor, lain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In International conference on machine learning, pp. 881-889. PMLR,
2015.
Zoubin Ghahramani and Matthew Beal. Propagation algorithms for variational Bayesian learning. Advances in neural information processing systems, 13, 2000.
Manuel Glöckler, Michael Deistler, and Jakob H Macke. Variational methods for simulation-bass based inference. arXiv preprint arXiv:2203.04176, 2022.
Maria Gorinova, Dave Moore, and Matthew Hoffman. Automatic reparameterisation of probabilistic programs. In International Conference Machine Learning, pp. 3648-3657 PMLR, 2020.
David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-free inference. In International Conference Machine Leurning, pp. 2404-2414. PMLR, 2019.
Michael Gutmann and Aapo Hyvärinen. Noise- contrasttt estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference artificial intelligence and statistics, pp. 297-304. JMLR Workshop and Conference Proceedings. 2010.
Michael U Gutmann and Aapo Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of machine learning research, 13 (2), 2012.
Michael U Gutmann, Steven Kleinegesse, and Benjamin Rhodes. Statistical applications of con- trastive learning. Behaviormetrika, 49(2):277-301, 2022.
Trevor Hastie. The elements of statistical learning: data mining, inference, and prediction, 2009.
W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1)997-109 1970.
Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-fres MCMC with amortized ap- proximate ratio estimators. In International conference on machine learning, pp. 4239-42-44 PMLR, 2020.
12

---

Published as a conference paper at ICLR 2025
Joeri Hermans, Arnaud Delaunoy, François Rozet, Antoine Wehenkel, Volodimir Begy, and Gilles Louppe. A trust crisis simulation- based inference? Your posterior approximations can be unfaithful. arXiv preprint arXtv:2110.06881/ 2021.
Matthew Hoffman, Pavel Sountsov, Joshua Dillon, Ian Langmore, Dustin Tran, and Srinivas Vasudevan. Neutra-lizing bad geometry in Hamiltonian Monte Carlo using neural transport. arXiv preprint arXiv: 10X.vv/900.000 037044 2019.
David T Hoffmann, Nadine Behrmann, Juergen Gall, Thomas Brox, and Mehdi Noroozi. Ranking info noise contrastive estimation: Boosting contrastive learning via ranked positives. Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 897-905, 2022.
Johannes Hugger and Virginie Uhlmann. Noise contrastive estimation with soft targets for condi- tional models. arXiv preprint arXiv:2404. 14076, 2024.
Ferenc Huszár. Variational inference using implicit distributions. arXiv preprint arXiv: 1702.08235, 2017.
Rob J Hyndman. Computing and graphing highest density regions. The American Statistician, 50 (2):120-126, 1996.
Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Gordon Wilson. What are Bayesian neural network posteriors really like? In International conference on machine learning, pp. 4629-4640. PMLR, 2021.
Ghassen Jerfel, Serena Wang, Clara Wong-Fannjian Katherine A Heller, Yian Ma, and Michael I Jordan. Variational refinement for importance sampling using the forward Bellbackk-eiblee di- vergence. In Uncertainty in Artificial Intelligence, pp. 1829-1829. PMLR, 2021.
Michael Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. An introduction to variational methods for graphical models. Machine learning, 37:183-233, 1999.
Patrick Kidger and Cristian Garcia. Equinox: neural networks in JAX via callable PyTrees and filtered transformations. Differentiable Programming workshop at Neural Information Processing Systems 2021, 2021.
Diederík Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69880 2014.
Diederik Kingma and Max Welling. Auto-encoding variational Bayes. In Internation Conference Learming Representations, 2014.
Durk P Kingma, Tim Salimans, Rafal Jozefowiez, Xi Chen, Ilya Sutskever, and Max Welling. Im- proved variational inference with inverse autoregressive flow. Advances in neural information processing systems, 29, 2016.
Yingzhen Li and Richard E Turner. Rényi divergence variational inference. Advances in neural information processing systems, 29, 2016.
Jan-Matths Lueckmann, Jan Boelts, David Greenberg, Pedro Gonçalves, and Jakob Macke. Bench- marking simulation-based inference. In International conference artificial intelligence and statistics, pp. 343-351. PMLR, 2021.
Mäns Magnusson, Jakob Torgander, Paul-Christian Bürkner, Lu Zhang, Bob Carpenter, and Aki Vehtari. posteriordb: Testing, benchmarking and developing Bayesian inference algorithms. preprint arXiv:2407.04966) 2024.
anference arXiv
Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders, arXiv preprint arXiv:1511.055: 1.05644, 2015.
Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. Adversarial variational Bayes: Unifying variational autoencoders and generative adversarial networks. In International conference on machine learning, pp. 2391-2400. PMLR, 2017.
13

---

Published as a conference paper at ICLR 2025
Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087-1092, 1953.
Benjamin K Miller, Christoph Weniger, and Patrick Forré. Contrastive neural ratio estimation. Ad- vances in Neural Information Processing Systems, 35:3262-3278, 2022.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Ad- vances neural information processing systems, 32, 2019a.
Thomas Müller, Brian McWilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. Neural im- portance sampling. ACM Transactions on Graphics (ToG), 38(5):1-19, 2019b.
Christian Naesseth, Fredrik Lindsten, and David Blei. Markovian score climbing: Variational in- ference with KL (pllg). Advances in Neural Information Processing Systems. 13:15499-15510, 2020.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03745. 2018
George Papamakarios and lain Murray. Fast e-free inference of simulation models with Bayesian conditional density estimation. Advances in neural information processing systems, 29, 2016.
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. Advances neural information processing systems, 30, 2017.
George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 837-848. PMLR. 2019.
Omiros Papaspiliopoulos, Gareth O Roberts, and Martin Sköld. general framework for the parametrization of hierarchical models. Statistical Science, pp. 59-73, 2007.
Giorgio Parisi. Statistical Field Theory. Medison-Wesley, 1988.
Minsu Park, Seyeon Choi, Chanyeol Choi, Jun-Seong Kim, and Jy-yong Sohn. Improving multi- lingual alignment through soft contrastive learning. arXiv preprint arXiv:2405. 16155, 2024.
Du Phan, Neeraj Pradhan, and Martin Jankowiak. Composable effects for flexible and accelerated 2019.
probabilistic programming in NumPyro. arXiv preprint
Dennis Prangle, Michael GB Blum, Gordana Popovic, and SA Sisson. Diagnostic tools for approx- imate Bayesian computation using the coverage property. Australian & New Zealand Journal of Statistics, 3694),309-329, 2014.
Tom Rainforth, Adam Kosiorek, Tuan Anh Le, Chris Maddison, Maximilian Igl, Frank Wood, and Yee Whye Tch. Tighter variational bounds are not necessarily better, In International Conference on Machine Learning, pp. 4277-4288 42855. PMLR, 2018
Benjamin Rhodes and Michael Gutmann. Variational coite-oontrastive estimation. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 2741-2750. PMLR, 2019.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. Ad- vances in neural information processing systems, 33:4905-4916, 2020.
Geoffrey Roeder, Yuhuai Wu, and David K Duvenaud. Sticking the landing: Simple, lower-variance gradient estimators for variational inference. Advances in Neural Information Processing Systems, 30, 2017.
Donald B Rubin. Estimation in parallel randomized experiments. Journal of Educational Statistics, 6(4):377-401, 1981.
Donald B Rubin. Using the SIR algorithm to simulate posterior distributions. In Bayesian statistics 3. Proceedings of the third Valencia international meeting, 1-5 June 1987, pp. 395-402. Claren- don Press, 1988.
14

---

Published as a conference paper at ICLR 2025
Sean Talts, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. Val- idating Bayesian inference algorithms with simulation-base calibration. arXiv preprint 0rXiv:1804.06788, 2018.
Owen Thomas, Ritabrata Dutta, Jukka Corander, Samuel Kaski, and Michael Gutmann. Likelihood-free inference by ratio estimation. Bayesian Analysis, 17(1):1-31, 2022.
George Tucker, Dieterich Lawson, Shixiang Gu, and Chris Maddison. Doubly reparameterized gradient estimators for Monte Carlo objectives. arXiv preprint arXiv:1810.04152 2018.
Feng Wang and Huaping Liu. Understanding the behaviour of contrastive loss. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 2495-2504, 2021.
Daniel Ward. FlowJAX: Distributions and normalizing flows in JAX, 2024. URL https:// version 14.0.0.
github.
Daniel Ward, Patrick Cannon, Mark Beaumont, Matteo Fasiolo, and Sebastian Schmon. Robust
neural posterior estimation and statistical model criticism. Advances Neural Information Pro- cessing Systems, 35:33845-33859 2022.
Jiayu Yao, Weiwei Pan, Soumya Ghosh, and Finale Doshi-Velez. Quality of uncertainty quantifica- tion for Bayesian neural network inference. arXiv preprint
2019.
Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman. Yes, but did it work?: Evaluating variational inference. In International Conference on Machine Learning, pp. 5581-5590. PMLR, 2018.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 ZERO VARIANCE GRADIENT AT OPTIMUM WITH FINITE K
When the optimal classifier reached. we can show that the gradient is zero (and hence has zero variance), even for finite K. To avoid clashes with the parameters Φ, we will use superscripts for indices. We have the objective
K Σ k=1
ga 000 (0%)
C(Φ; (0*) 1 1k=1,
o log
K
2s (0k))/ (/D)(0k)
Letting 1/z replace the term inside the log
K
Σ
y log No
(22)
K
VL(:; (0*)
Σ
# v
log k
Upp
(23)
k=1
(24)
which due to optimality of the classifier we have y² = V/
K
V
(25)
due to the properties of the softmax, the labels must 1
O.
(26)
A.2 SOFTCVI NORMALIZATION TERM GRADIENT
When p (0) (0). claim in eq. (14) that the normalization term gradient can be written as K D V log qo (O&). Making use the property of the logarithmic derivative, Va log ((x) = show this as follows
V.f(m) [(x)
Lo (Ox) P (0s)
V
Σ
K k- (0(0)) (0x)
V log
k== 4s (0x)/ (0x)
where we treat p- (0x) as constant with respect to & (i.e., we apply stop-gradient), so
K A==
(0x) (0x)
and using p- -00)) (0.()) (value-wise only, not in gradient flow), we get
Volgs O, 4/4)//40(00 (0x) (xx) 10 (Os)
V Φ log (s (0x)
(27)
K
16

---

Published as a conference paper at ICLR 2025
A.3 GRADIENT SIGNAL-TO-NOIIE RATIO
By evaluating an objective gradient over a set of random seeds (different sets of {0x)k=)) 1). it is possible empirically inspect the signal-to-noise ratio (SNR) ratio of the gradient for different objectives. Following Rainforth et al. (2018), 8), we compute the SNR as
SNR(V ANR(V&L(o)) ((() LE [[V / [[())(
where of: denotes the element-wise standard deviation for each parameter gradient. A low SNR indicates that the gradient estimation is dominated by noise, making stochastic optimization chal- lenging. However, it is important to recognize that whilst a high SNR is preferable, it does not alone guarantee the objective itself is practically useful (or even sensible), for example it may be biased or heavily favor mode-seeking solutions.
We consider toy normal task from Glöckler et al. (2022), with the distinction that we vary the dimensionality of the task d E 50) and the parameterization of variational distribution. The task defined through the model
N00,4. Ia), x ~~
where the aim is to infer the mean vector 0, given an observation Lobs = 1a. where 1a is the d- dimensional vector of ones. The variational distribution parameterized as a normal distribution, with mean vector E I&d and log standard deviations log Rd. When assessing the gradient properties of μ., we hold log fixed at the closed form posterior solution, log = log( Similarly, when assessing the gradient properties of log a, we hold f fixed at the true closed form solution μ 4/5 1a. The signal, noise and SNR for each objective are shown in fig. 3.
(og(v4/5) /4/5). 1
SoftCVI with a = 1, and the SNIS-IKL objective yield very similar signals, which results from the two objectives producing the same gradients in expectation (section 3.2). However, the SNIS-fKL
d
d = 50
d
d = 50
1.5
0.2
Signal
1.0
0.1
0.5
0.2
0.5
0.0
0.0
0.0
0.0
0.75
2
Noise
0.50
1.0
0.5
0.25
0.00
0.0
0.0
0.3
SNR
0.2
0.2
0.1
0-
2.5
2.5
-2.5
0.0
2.5
-2.5
0.0
2.5
-2.5
2.5
log a
log
(o
-SoltCVT(a=0.75)
SoftCI((a=1)
SNIS-IKL.
Figure 3: The signal, noise and signal-to-noise ratio of the objective gradients on a toy normal task of varying dimensionality. When d 50, the gradient properties are computed parameter-wis and averaged. The vertical dashed line shows the true parameter values from the closed form posterior solution.
17

---

Published as a conference paper at ICLR 2025
objective has positive noise when the variational distribution approaches the true posterior, meaning the SNR degrades to zero. In contrast, for the SoftCVI objectives, the gradient noise approaches zero (see appendix A, and reasonable SNR is present as the variational distribution approaches the true posterior.
When d 50 the signal and consequently the SNR, deteriorates larger values of log Ø, and lower choices of a. In this case, both expanding the dimension the problem, and decreasing the a value, increases the variance of 2 log[p Lpp (0z)] and E log[aæ(Oz) (01)s(O0////)) (0x)]. This can lead to degeneracy in the labels and predictions, meaning very few samples meaningfully contribute and further expanding
the loss function, reducing the SNR. In an extreme case, e.g. setting the dimensionality of the problem, only a single label and prediction will be significantly non- zero. In this regime, negligible signal would exist for too large log a parameter, as the variational distribution, despite the incorrect log a, would still result in the correct degenerate predicted labels.
A.4 ALTERNATIVE DIVERGENCES
We can interpret So defining two categorical distributions between which divergence is minimized: P. which is defined using the labels such that yk = P(k), and Qş. which is defined using the predictions such that Dh (*).). In this work, used the cross-entropy objective, which is equivalent to minimizing the forward KL-divergence between the label distribution P and the predicted distribution QΦ:
K P(k) log
P(k)
PalPIQQa
(k)
K P(A) log (Q + const,
(28)
K
yk log Uh + const.
Alternative divergences could considered for use with SoftCVI. For example, we could explore the general class of o /-divergencces (Ali & Silvey, 1966; Csiszár, 1967), which are defined as:
K
P(k) (Qs(t))
101 = [ -
1=11
where / is a convex function f (0, + R. and f(1) 0. Divergences that are more or less mass- covering may be better suited to specific tasks. For example, for many generative modelling tasks reliable uncertainty quantification is considered less critical, in which case more mode-seeking objective may be favourable. We leave this investigation for future research.
A.5 TRAINING OF MODEL PARAMETERS: VARIATIONAL AUTOENCODERS
SoftCVI sets up a classification problem which allows fitting the parameters of the variational dis- tribution Φb However, in some contexts, the model may be defined as pv, Xohs), with th, being a set of model parameters which we wish to optimize in some manner alongside &. SoftCVI does directly provide method for fitting such additional model parameters. One approach to resolve this is to include an additional objective, that trains zi to (approximately) maximize the marginal likelihood.
L(sb, wb; [0 9) = Lsoncvi(s; (0 1. 9) + Lmodal (bb; [k
(29)
where in the above formulation we reuse the proposal distribution samples {0+) already sam- pled for the SoftCVI objective, and we assume the proposal distribution is equal to the variational distribution. One could more generally consider alternating between optimizing the two objectives; however, due to the disjoint parameter sets between the objectives, addition to the invariance of common optimizers such Adam to diagonal rescaling of gradient elements (Kingma & Ba, 2014),
18

---

Published as a conference paper at ICLR 2025
found adding the objectives to work well. In this section, we choose
K
Lmoode (zb; (0x) =
log Pup (On, Bobss)
(30)
equivalent to training the model parameters dp by maximizing the ELBO. As example, will train a variational autoencoder (Kingma & Welling, 2014), where vb includes the parameters of the decoder in addition the models prior parameters, and consists of the parameters of the encoder. Both the encoder and decoder are parameterized as standard feed-forward networks. The manifolds learned using either the ELBO or the modified SoftCVI objective from eq. (29) are shown in fig. 4, showing qualitatively similar results. Whilst we do not investigate the performance, the SoftCVI method fitting does not utilize reparameterized gradients and as such applicable to a broader class of models.
A.6 BAYESIAN NEURAL ADDITIVE MODEL
Neural additive models enhance the interpretability of neural networks, using an additive model structure (Agarwal et al., 2021). In the simplest case, for dataset with C' features, for an input vector æ = I1, IC. predictions are computed as
ÿ B + fi + f2(12) +.... fo(ac),
where each fa is neural network corresponding to single input feature. In this section, we consider regression problem, in which we parameterize each fee using a Bayesian neural network (Blundell al., 2015). Bayesian neural networks have been suggested to reduce addition to providing method for assessing prediction uncertainty using the inferred posterior pre- dictive distribution. Here, we parameterize each fe using a single layer Bayesian neural network with width of either 50 100, with Laplace 00. 1) prior on the neural network parameters. This naturally leads to high dimensional posterior distributions (0 E 1500 and E R 30000 for the task considered, respectively). We an independent Gaussian approximation to the posterior.
risk of overfitting in
We consider a regression problem, with synthetic data generated using the nonlinear function
y = 0.1 2 + |22| - F3 + C, where ~ (0)
where 10 is generated uniformly from the interval [-4, 4). and the last dimensions nuisance features. We use 300 training data points, 150 validation data points and 1000 testing data
points for computing the metrics.
We assess performance using the average test set log- ligeliold under the posterior predictive distri- bution, and report the mean prediction interquartile range (IQR) across the test set and the associated
SoftCVI(a=0)
SoftCVI(a=1)
ELBO
7
5
f
f
7
/
3
8
f f 8 8 8
7
/ / / 3
2
E f
8
8 8 8 1
7
7
8
/ / 8888558
2200
$ f
5 5
7
7
8
8 $
/ 888888
2 2200
5 8
7
7
8
/
7 7
5
S
7
7
8 8 8
0660999
9
5
8 8 8 8
292999999 9
7
8 8 8
2 2299999982 9
5 5 033324444
7
9
5 5 5
8 88888990
5
5
1 7
5
5 5
3
5
5
3
3
3
5599
Figure 4: The manifolds learned by variational autoencoders on the MNIST dataset, trained using either the ELBO or SoftCVI. To enable training of the model parameters, the SoftCVI objective was modified by adding the model component the ELBO, K Σ log pv (0,, Wots). In all cases, the objectives were trained for 100,000 steps with a batch size of 1, and K 8.
19

---

Published as a conference paper at ICLR 2025
prediction coverage (i.e. the average frequencyy with which the true underlying function value is in- cluded within the predicted IQR). We note there are numerous challenges associated with assessing performance for Bayesian neural networks. For example, high test log-likelihood calibrated pre- dictions not necessarily indicative of good posterior approximation (Yao et al., 2019). Further, early stopping based the validation log-like hog-likelihood, in addition to choosing the learning rates with cross-validation, both of which will tend to bias results to favor models with higher test log-likelihood, without consideration the calibration of the posterior predictive distribution.
We train 10 networks initialized with different random seeds, and report the results in table 1. A plot of the learned components for each method is shown in fig.
Method
NN Width
Test Log-Likelihood
Prediction IQR
IQR Coverage
50
-2.392 = 0.005
0.648 = 0.084
0.262 = 0.042
ELBO
100
-2.392 0.005
0.641
0,264 + 0.031
50
-2.422 0.007
0.695
0.186 0.008
SNIS-fKL
100
-2.421 0.005
0.938
0.256 0.007
50
-2.424 = 0.007
0
0.202 # 0.009
SoftCVI (a: = 0.75)
100
-2.423 0.005
0.983
0.271 = 0.010
50
-2.422 # 0.007
0.701 = 0.012
0.190 # 0.009
SoftCVI (a -
100
-2,423 # 0.005
0.943 = 0.012
0.260 + 0.009
Performance metrics for the Bayesian neural additive model. Note model producing calibrated predictions on the test set would yield an IQR coverage value of 0.5.
Table 1:
While all methods demonstrated comparable predictive performance, the ELBO achieved slightly better results, evidenced by the highest test log-likelihood. Both SofiCVI (with 0.75 1) and SNIS-fKL yielded similar results. Bayesian neural network posteriors are often complex and highly multimodal (Izmailov al., 2021). In this case, it is likely that SoftCVI does not show significant advantage over SNIS-fKL because the Gaussian posterior is heavily misspecified. As result, the approximate posterior may become sufficiently close to the true posterior in order to provide the variance reduction benefits associated with SoftCVI.
SoftCVI(a=0:75) SoftCVI(a=1)
SNIS-IKL
ELBO
(x1)
47
21
1
f2(22)
J72
32
22
T22
fa (x3)
-2.00.0 2.5
-2.00.0 2.5
-2.50.0 2.5
-2.00.0 2.5
.03
X33
3C3
Figure 5: The means and 95% prediction intervals for the components of Bayesian neural additive model for each method. The true underlying components are shown with the dotted black lines. We restrict the first three dimensions, ignoring the nuisance variables.
20

---

Published as a conference paper at ICLR 2025
A.7
ALTERNATIVE NEGATIVE DISTRIBUTION CHOICES
Eight Schools
Linear Regression
SLCP
GARCH
Actual nominal
0.0
0.2
(a)
0.4
-0.6
Nominal
Nominal
Nominal
Nominal
Log prob. 8"
2.00
-22.66
64.0
(b)
1.75
-22.40
20
22.44
30
51.0
Posterior mean
00.3
-0.24 0.0
accuracy
-0.24
-1.
-0.32
-0.32
0.9
0.40
-0.40
-.2
-1.
SoftCV1(a=0.75)
SoftCVi(a=1)
SNIS- FKL
ELBO
Figure 6: Additional metrics analogous to fig. 1, using p (0) = p(0,, Dobss as the negative distri- bution choice in the SoftCVI objectives.
The main results focus on parameterizing the negative distribution as function of the proposal distribution p (0) = (0)a. However, another possible choice to use the unnormalized posterior to parameterize the negative distribution, for example p (0) p(0, 2otx) p(0, This choice, when a = 1, implies equality between the assumed negative and positive distributions, meaning through eq. (2) the ground truth labels become constant yk for k 1, .... K. Inputting these labels into eq. (7) yields the objective
log qs (0s) + log Σ 20 (Os) k= p (0x)
K L(d; (0x) 9)
Σ
const.
(31)
K
Assuming the proposal distribution is chosen match the variational distribution, since (o) [V log [0)] = = (o)_ [V log (0)] =0)]==0 the first term in eq. (7) has expected gradient of zero. contrast to using the proposal distribution as negative distribution, which results the normalization term which penalizes the ratios acting control variate, here, the first term instead acts as a control variate, with the normalization term providing the signal by penalizing the ratio qs (0)/p(0 xtts)) (0))pp(0lzaa As such this choice heavily penalizes 4p(() >> p(0 Wats), favoring overconfident posteriors. We report in fig. 6 the metrics, using p (0) = p(6, Fobs with 0.75 and 1.
A.8 TASKS
For all tasks, where possible, make use of reparameterizations to reduce the dependencies be- tween the parameters 0 in the model, and to ensure variables are reasonably scaled, which is gen- erally considered to improve performance (Betancourt & Girolami. 2015; Gorinova al., 2020; Papaspiliopoulos et al., 2007). Below, we describe the models used and the source the reference posterior samples.
21

---

Published as a conference paper at ICLR 2025
EIGHT SCHOOLS.
classic hierarchical inference model (Gelman al., 1995; Rubin, 1981), aiming to infer the treat- ment effects of a training program applied to eight schools. The parameter set is {u, T, m) where y is the average treatment effect across the schools, the standard deviation the treat- ment effects across the schools and m is the treatment effects for each school. The model is given by
1r (0,,5))
HalfCauchy(O, 5²);
m, N(2, i 1, 1,,,,
T2 ~ (M(; of), i= 1=,,.,,
where 2² is treated as known, estimated using the standard errors in the data. We use the reference posterior samples available from PosteriorDB, which are sampled using MCMC (Magnusson et al., 2024).
LINEAR REGRESSION.
linear regression model, defined as
Bi ~ N(O 1),
i == 1,...,50,
1 NO,,),,
2; ~ ([
For each run of the task, we sampled dataset X 200 from standard normal distribution, and drew reference posterior samples from the analytical posterior solution.
SIMPLE LIKELIHOOD COMPLEX POSTERIOR
The SLCP task introduced by Papamakarios et al. (2019), and is designed to be challenging inference problem with multimodal posterior. The data is a set of four samples from two- dimensional multivariate Gaussian likelihood function. The likelihood is parameterized using using squaring which introduces a complex multimodal structure (see fig. 2). Specifically, the model defined as
0: (niform(-3,33 i= 1,.,,5
(32)
2; ~ No E), j j= 1,...,4,
(33)
where the covariance matrix Σ is
Σ $ p.
where = =03. $2 = 02 and p tanh(05)
82
Despite being used extensively in the simulation-based inference (SBI) literature, this task has tractable likelihood so can be used in the current work. The reference posterior is from the SBI Benchmark python package (Lueckmann et al., 2021), and was inferred using sampling/importance resampling using the analytical likelihood function (Rubin, 1988).
GARCH
The GARCH model is widely used statistical model for analyzing time series data with time- varying volatility (Bollerslev, 1986). extends the basic autoregressive framework by allowing the conditional variance of the observations to depend on both past observations (controlled via the on parameter) and variances (controlled by the 31 parameter). GARCH and similar models are often used for modeling financial data, where autocorrelated variances common. The priors are defined
as
f ~ ImproperUniform((- (-o0, oo)
0g ImproperUniffom(0..
Q1 ~ Uniform(0. (0, 1)
B Uniform(0.. a1),
22

---

Published as a conference paper at ICLR 2025
Where ImproperUniform represents improper flat prior over the specified region. For t 1,..., 200, the variance evolves recursively through the update
of og (1 14)2 + 3102-11
and the likelihood is given by
3t (N,, of)
At initialization, Yo is set to the first observation element, and oî = 0.25. For this task, a reference posterior is available in PosteriorDB, sampled using MCMC (Magnusson et al., 2024).
23

---

Published as a conference paper at ICLR 2025
A.9
ADDITIONAL FIGURES
Eight Schools
Linear Regression
GARCH
SLCP
Log prob. Ax
2.1
350
2.0
1.0
22.400
50
-15
2
500
160
1250
Run time
000
155
300
340
130
250
(kg
(K)
log, (K)
logg(K)
logg
SoftCVI(a-0.75)
SoftCVI(a-11
SNIS-IKL
ELBO
Figure 7: Average log-probability of the reference posterior samples as a function of K (ranging from 64), along with the associated run times (measured on a CPU with SGB RAM, including compilation time). The poor performance of SLCP for higher values of K was due to increased mode-seeking behavior. Some results for SNIS-fKI are truncated on the axes to improve visualiza- tion of other methods. Note that the run times will be dependent on the architecture of the variational distribution. For example, the ELBO showed significantly slower run times the SLCP task due to the cost of computing reparameterization gradients for masked autoregressive flow. However, using an inverse autoregressive flow (Kingma et al., 2016) would likely mitigate this issue.
Eight Schools
Linear Regression
SLCP
GARCH
Actual nominal
0.0
00.3
-0.6
Nominal
Nominal
Nominal
Nominal
Posterior menn Log prob, 6"
22.375
-25
(b)
-22.400
-22.425
-18 -75
0.6
accuracy
-0.24
1.2
-0.6
-0.30
1.8
-8
00.38
2.4
1.2
-12
SoftCVT((a=0.75)
SoftCCT((a=1)
SNIS-IKT.
ELBO
Figure 8: Additional metrics analogous fig. 1, using K 2, instead of K 8 samples when approximating the objectives. The SNIS-fKL. objective performance degrades substantially when K small.
24

---

Published as a conference paper at ICLR 2025
SoftCVT(a=0)
(oftCV((a=0.75)
SoftCVI(a=1)
SNIS-fKL
ELBO
83
Figure 9: The multimodal 2-dimensional posterior marginals for the first six runs of the SLCP task for each method, with the reference posterior samples shown in green.
25

---

Published as a conference paper at ICLR 2025
Eight Schools
Linear Regression
GARCH
5
10
0.9
1.0
1.1
1.2
1.3
0.0
2.5
5.0
7.5
10.0
0.0
0.1
0.2
0.3
81
4.8
5.0
5.2
5.4
/4
0
10
0.2
0.3
0.4
0.5
0.6
m
32
0
10
0.2
0.3
0.4
0.5
0.6
12
83
0
10
1.0
-0.9
-0.8
0.7
00.6
m73
31
000
10
1.9
2.0
2.1
2.
m4
3g
0
10
-0.2
0.0
m5
38
0.2
0.4
0.6
0.8
1.0
Q1
10
1.2
-1.1
-1.0
0.9
m6
87
10
1.8
-1.6
-1.
1717
88
1.6
-1.4
-12
0.0
0.1
0.6
mg
39
B
True
SoftCVI(a=1)
SNIS-IKL
ELBO
Figure 10: The distribution of the posterior marginals for single run of the eight schools, linear regression and GARCH(I.1) tasks. For the linear regression task, we restrict the plot to the first 10 parameters of the model.
26


Reviews:
[{"review_id": "emEZ7Ke8hj", "reviewer": "Reviewer_AxW5", "summary": "In this paper, the authors propose an approach that reformulates the inference problem as a classification task using contrastive learning and the generation of soft labels. \n\nThese soft labels, derived from an unnormalized target distribution, are employed in a cross-entropy loss to fit the variational distribution. The authors demonstrate the efficacy of their methodology through a series of experiments on Bayesian inference.", "questions": "1. In line 57, the authors mention proposing **a family of variational objectives**. Could you clarify this point? It seems that there is only one objective—the cross-entropy between true soft labels and the predicted categorical distribution. Is there indeed a family of objectives, or is it a single objective?\n\n2. [questions from Weaknesses] How does the choice of $K$ influence the training curves and final results? Is there an optimal value for $K$? What criteria should be used to select $K$?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents an elegant method for fitting the variational distribution by training the model to solve a classification problem. \n\nI identify two key benefits of the proposed approach:\n\n1. There is no need for samples from either the negative or the target distribution.\n2. The densities of the target and negative distributions need only be known up to a normalizing constant.\n\nOverall, the paper is well-written and easy to follow.", "weaknesses": "I note the following weaknesses in the paper:\n\n### Major:\n\n1. Deep Learning examples:\n\nThe experimental evaluation feels modest and misses an opportunity to show the benefits of the proposed methodology in deep learning scenarios. The authors focus solely on \"classical\" examples of Bayesian inference. I think it would be very illustrative to consider deep learning applications.\n\nFor example, in the related work section, the authors mention papers on Variational Auto-Encoders (VAE). It would be interesting to see how the proposed methodology performs in this context.\n\nI see several ways your approach could be applied:\n\n- End-to-End VAE training: Instead of optimizing the standard ELBO (or alternatives like IWAE [1]), incorporate the SoftCVI training procedure and evaluate the negative log-likelihood.\n- Fitting variational distribution for pretrained VAE: For an already pretrained VAE (decoder) with a complex, multimodal posterior, apply your training procedure to fit a variational distribution to this posterior. Then demonstrate that the variational distribution fitted with SoftCVI covers more modes and provides more diverse samples (see e.g. [2]).\n\nI appreciate the example with the SLCP task. Similarly, an example with a \"circular\" posterior in [3] could be considered for demonstration.\n\n2. Training curves with different $K$:\n\nSince $K$ is a hyperparameter we can choose as we wish, it's important to understand how it influences the results. In Figures 1 and 6, two different choices of $K$ are considered, but there is no discussion on which is better. It would be beneficial to see not only the final evaluations but also the training curves for different $K$ values. Given that $K$ can range widely (from 2 to infinity), practical guidance on selecting $K$ would be valuable.\n\nAdditionally, training curves are interesting in the context of comparison with SNIS-fKL, as the variance of estimation will be greater there, and it is interesting to see it.\n\n### Minor:\n\n1. Motivation in Section 2.1:\n\nInitially, I found the beginning of Section 2.1 a bit confusing. It mentions a scenario with $K−1$ negative samples and only 1 positive, which doesn't seem to match the scenario in the paper. As I understand it, we have samples only from the proposal distribution and assign probabilities to them as if they come from the true distribution to generate soft labels. We don't assume that only one sample comes from the true distribution. If my understanding is incorrect, please clarify. Otherwise, consider rephrasing this section for clarity.\n\n\n2. Typo:\n\nLine 329: \"in contrast **to to**\".\n\n----\n\nI am positive about the methodological contributions. With the incorporation of extensions to the experimental section, I am willing to significantly increase my evaluation score.\n\n\n----\n\nReferences:\n\n[1] Burda, Y., Grosse, R., & Salakhutdinov, R. (2015). Importance Weighted Autoencoders. arXiv preprint arXiv:1509.00519.\n\n[2] Thin, A., Kotelevskii, N., Denain, J. S., Grinsztajn, L., Durmus, A., Panov, M., & Moulines, E. (2020). MetFlow: A New Efficient Method for Bridging the Gap Between Markov Chain Monte Carlo and Variational Inference. arXiv preprint arXiv:2002.12253.\n\n[3] Thin, A., Kotelevskii, N., Doucet, A., Durmus, A., Moulines, E., & Panov, M. (2021). Monte Carlo Variational Auto-Encoders. In International Conference on Machine Learning (pp. 10247-10257). PMLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zq6UL2u5U2", "reviewer": "Reviewer_xUmn", "summary": "The paper considers a variational inference approach that builds upon classifying ‘noisy’ samples from a learned proposal distribution as negative samples, similar to a noise contrastive estimation approach, and where ‘ground truth soft labels’ depend on a self-normalised importance sampling estimate for the log ratio of the target density relative to the proposal density. In contrast to a classical ELBO minimizing the reverse KL to the target, the suggested approach appears to better approximate the forward KL to the target. Numerical experiments on benchmark Bayesian inference problems illustrate that suggested approach yields te better coverage probabilities and yields to approximations with less bias for the posterior mean estimate.", "questions": "It would be helpful if the paper is a bit clearer what kind of reference posteriors are used. Are they from MCMC algorithms that can be assumed to closely approximate the true posterior? What about the SBI posteriors?\n\nRe line 127, can you elaborate what is the exact parameterisation of the classifier in terms of the ratio of two distributions. Are the parameters (infinite-dimensional) probability densities?\n\nCan you clarify why you compare against simulation-based reference models (SBI)? How is the proposed approach applicable without having access to the likelihood function?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper addresses the classic and important question of Bayesian inference, particularly for the challenging setting where the approximate posterior should be close to the target with respect to the forward KL to yield well calibrated inferences. The approach avoids previous objectives that yield mass-covering behaviour, but are more challenging to optimise, particular in high dimensions. The suggested objective is now as far as I am aware. It is also very interesting that the suggested approach can be used to construct a previously constructed objective (SNIS-fKL), but with a gradient estimator that has zero variance in the idealised setting when the approximate posterior coincides with the true posterior. Experiments do indeed show that the suggested method improves the coverage and leads to approximate posterior that are closer to the target in terms of the forward KL.", "weaknesses": "The empirical verification seems a bit limited. In particular, the only baseline (beyond a standard ELBO) is the SNIS-fKL objective. However, as the paper shows, they have the same expected gradients. It is thus not clear how well the suggested approach compares to other (approximate) Bayesian methods that have different averaged gradients, e.g. [1-3]. \n\nAlso as the suggested approach has a higher computation complexity requiring sampling and density-evaluations of K particles, evaluations with a simple IWAE approach instead of the ELBO baseline appears appropriate. \n\nThe authors consider (in Section 2.1) that they have access to samples from the true posterior. However, is having access to samples from the true posterior not the aim of the method? In (4), is $y$ equal to $\\hat{y}$ as in Algorithm 1 when one does not have true posterior samples? \n\nIt is not clear why eq. (4) is the right objective to optimise when one does not have access to the true labels $y$. Does optimizing it correspond to some proper scoring rule or divergence [4] that are minimized for this objective? \n\nCan you clarify why the optimal classifier recovers the true posterior and that the true posterior is equal to the approximate variational distribution as per eq. (6)? The ‘optimal classifier parameterized as in eq. (3)’ depends (apparently?) on the number of samples $K$ from the proposal and on the proposal distribution. Do all the arguments on page 3 from line 141 not depend on these choices? I understand that the classifier is optimal with respect to a cross-entropy loss if it recovers the Bayesian posterior [4]. But is this optimality even realisable in the suggested form (3)?\n\nThe experiments target posteriors that are not that high-dimensional (<=200). Does the method scale to high-dimensional problems? Does it overcome scalability issues commonly faced when optimizing the forward KL or for importance sampling?\n\n\n[1] Dieng, Adji Bousso, et al. \"Variational Inference via $\\chi $ Upper Bound Minimization.\" Advances in Neural Information Processing Systems 30 (2017).\\\n[2] Li, Chengrui, et al. \"Forward $\\chi^ 2$ Divergence Based Variational Importance Sampling.\" arXiv preprint arXiv:2311.02516 (2023).\\\n[3] Naesseth, Christian, Fredrik Lindsten, and David Blei. \"Markovian score climbing: Variational inference with KL (p|| q).\" Advances in Neural Information Processing Systems 33 (2020): 15499-15510.\\\n[4] Mohamed, Shakir, and Balaji Lakshminarayanan. \"Learning in implicit generative models.\" arXiv preprint arXiv:1610.03483(2016).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9b1NQK3Z9q", "reviewer": "Reviewer_SFto", "summary": "The paper proposes a new way to approximate the posterior distribution over parameters for a parametric model $p(x;\\theta)$. \n\nThe approach:\n* defines 2 multinomial distributions:\n   *  $y(\\\\{\\theta_k\\\\})$ over the index $k$ of $K$ parameter samples $\\theta_k$ as a function of the model $p(x,\\theta)$ and noise distribution; \n   * $\\hat{y}(\\\\{\\theta_k\\\\})$ as a function of the parameterised approximate posterior $q_\\phi(\\theta)$ and the noise distribution;\n* $\\hat{y}$ is trained to match $y$ under cross entropy loss;\n* at optimality $q_\\phi(\\theta)$ matches the true posterior $p(\\theta|x)$ (subject to the choice of the parametric family $q_\\phi$).\n\nResults compare the proposed method to typical amortised variational inference (maximising the ELBO) and a more similar method, showing improved outcomes on those datasets.", "questions": "* 50k \"optimisation steps\": this sounds a lot, any explanation for this? What is the definition of step?\n* K=8: Any ablation of the effect of K on accuracy vs wall-clock time? i.e. trade off in slower steps vs number of steps\n     * Fig 6 (K=2) is hard to compare to Fig 1 (K=8) due to their separation and the SNIS result dominates the scale, squashing up the rest.", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "* The proposed method makes sense and appears straight-forward to implement. \n* The method appears general to problems with a parametric model where $p(x,\\theta)$ can be computed.\n* Comparison to SNIS is revealing and clear.", "weaknesses": "Overall I like the paper, find it to be of high quality and soundness and the proposed method is a clever trick that converts a difficult problem into a simpler classification-type task. Areas in which the paper could improve (from good to excellent) are: \n\n**Clarity**: as someone familiar with this field but not immersed in it, the paper could be clearer to read. As specific examples:\n* [033, 078] From the outset, the paper refers to *latent variables*, which often refer to variables that are one-to-one with data samples, whereas the term *parameters* typically relates to global variables of the model, i.e. common across all samples. (Of course parameters are often treated as unobserved, hence latent, variables). I appreciate that this may be subjective, but I find it to be fairly universal and a useful distinction, so this phrase put the wrong starting point in my mind. I was wondering throughout when \"parameters of the model\" get updated and only when I reached the experiments did it become clear that it is about parameters and it made more sense. I would suggest referring to \"parameters\" (treated as latent variables).\n* The preamble to Eq 2 is not directly relevant and could be more clear: labels y are constructed analytically and the approximate posterior is trained by learning to predict them. The backstory of predicting k* doesn't seem so relevant (as it is in NCE, say) so this bit threw me off until I realised that.\n* [202-213] This part stands out as less precise relative to preceding arguments. The method trains one ratio of ratios to match another ratio of ratios using cross entropy loss. Reference to \"classification being too easy\" and \"punishing\" seems out of place and unclear. The issue seems to be the relative range of values in the softmax function (+/- numerical instability?), which is mentioned but blended with loose wording. This part could be improved, particularly as computing these ratios/choosing the negative sampling distribution is the crux and potential pitfall of ratio estimation.\n* [161] Ensuring $\\pi$ covers the true posterior - I can't see how this is \"guaranteed\", unless pi is diffuse, which sounds problematic for ratios.\n* Experiments: to compute a posterior, you need a prior, but the priors don't seem clearly specified.\n* Fig 1: how are some log probability values positive?\n* GARCH experiment: The descriptions of $\\alpha_1$ and $\\beta_1$ are rather vague.\n\n**Analysis/Experiments**: For such a widely applicable method, the experiments to showcase its efficacy/analyse performance etc seem thin. \n* Understanding how well it works in higher dimensions is of interest, where potentially finding a negative distribution close to the true posterior could be less straightforward.\n* Understanding how the methods compare as number of data points increases is of interest to know when the method is most applicable.\n* Understanding how the methods compare when the posterior is mis-specified is of interest to understand robustness (mode seeking may be preferable to distribution covering?).\n\nMinor\n* [134] it would aid clarity to explicitly say optimisation is \"with respect to $\\phi$\"\n* [134] it would be much clearer to state labels y \"(from Eq 2)\".\n* [370] \"a different observation\" of what?\n* [421-424] the paper does well to be relatively self-contained, this part could be made more clear to the less familiar reader.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZF9rFtUnY4", "reviewer": "Reviewer_36B5", "summary": "The paper proposes SoftCVI, a variational inference method based on contrastive learning that uses soft labels, i.e., binary classification probabilities. SoftCVI reframes fitting a variational distribution as finding a classifier to identify actual posterior samples among a set of samples. The paper claims that:\n  1. SoftCVI works without having access to samples from the posterior.\n  2. SoftCVI forms objectives that are mass-covering.\n  3. SoftCVI yields better posterior approximation than baselines with simple and flexible variational distributions.", "questions": "1. Given the connection between SNIS-fKL and SoftCVI and that MC estimation of the SNIS-fKL objective is a biased estimation of the forward KL, does SoftCVI give a biased estimation of the posterior when $p^-(\\theta)=\\pi(\\theta)=q_w(\\theta)$? Could you clarify whether SoftCVI with $p^-(\\theta)=q(\\theta)^\\alpha$ gives a biased estimation of the posterior?\n2. What is claimed when stating that SoftCVI enables a stable objective (line 064)? Is numerical stability, stable convergence or something else? What evidence is there for the type of stability claimed?\n3. Why does the equation after eq.4 hold (line 146)? It seems like there is a population limit ($\\lim_{K \\rightarrow \\infty}$) missing. Is this the case? Otherwise, please explain why this is not necessary.\n4. The article establishes that $\\alpha=0$ leads to leakage and that $\\alpha=0.75$ is an good choice for the experiments. How do you suggest deciding $\\alpha$? Could you provide theoretical justification or guidelines for choosing in practice $\\alpha$? And, is there a threshold under which $\\alpha$ will lead to leakage, or is this only the case for $\\alpha=0$?\n5. Could you explain the second equality in the derivation leading to eq. 26? For example, by a step-by-step derivation.\n6. The claim that the parametric variational distribution can recover the posterior exactly under only a support constraint (lines 143-144) seems odd. Is there a condition that the posterior is contained within the parametric variational family missing? If so, please add this and any other assumption to the paragraph. If not, explain the flaw in the following case: Let the posterior $p(\\theta|\\mathcal{D})$ be multi-modal (for example, a mixture posterior) and the variational distribution parameterized by $\\phi$ be unimodal, e.g., Gaussian, both supported on the real line. By construction, we would have that $\\forall \\phi.p(\\theta|\\mathcal{D}) \\not = q_\\phi(\\theta)$ contradicting the claim in lines 143-144 that there $\\exists \\phi. p(\\theta|\\mathcal{D}) = q_\\phi(\\theta)$ namely the $\\phi$ that parameterizes the optimal classifier from eq. 3.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "## Reason for score\n- The presentation is lucid.\n- The derivations look good,  aside from a couple of technicalities (see questions).\n- Using soft labels in contrastive learning for Bayesian inference is exciting and novel.\n- The connection to SNIS-fKL is clear, and viewing the gradient of eq. 7 as a control variate is insightful and supported by evidence.\n- Experimenting with normal and normalizing flow variational distribution demonstrates SoftCVI's flexibility, and the results support that SoftCVI is more mass-covering than ELBO and SNIS-fKL.", "weaknesses": "- It is unclear whether the variational distribution is a biased estimator of the posterior. Please see the questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)