SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Chirag Modi
Center for Computational Astrophysics,
Center for Computational Mathematics,
cmodi@flatironinstitute.org
yyao@flatironinstitute.org
Department of Computer Science, Statistics,
Columbia University, New York
Charles C. Margossian
cmargossian@flatironinstitute.org
Robert M. Gower
Lawrence K. Saul
lsaul@flatironinstitute.org
Variational inference (VI) is a method to approximate the computationally intractable posterior distributions that arise in Bayesian statistics. Typically, VI fits a simple parametric distribution to be close to the target posterior, optimizing an appropriate objective such as the evidence lower bound (ELBO). In this work, we present a new approach to VI. Our method is based on the principle of score matching—namely, that if two distributions are equal then their score functions (i.e., gradients of the log density) are equal at every point on their support. With this principle, we develop score-matching VI, an iterative algorithm that seeks to match the scores between the variational approximation and the exact posterior. At each iteration, score-matching VI solves an inner optimization, one that minimally adjusts the current variational estimate to match the scores at a newly sampled value of the latent variables. We show that when the variational family is a Gaussian, this inner optimization enjoys a closed-form solution, which we call Gaussian score matching VI (GSM-VI). GSM-VI is a "black box" variational algorithm in that it only requires a differentiable joint distribution, and as such it can be applied to a wide class of models. We compare GSM-VI to black box variational inference (BBVI), which has similar requirements but instead optimizes the ELBO. We first study how GSM-VI behaves as a function of the problem dimensionality, the condition number of the target covariance matrix (when the target is Gaussian), and the degree of mismatch between the approximating and exact posterior distribution. We then study GSM-VI on a collection of real-world Bayesian inference problems from the posteriorDB database of datasets and models. We find that GSM-VI is faster than BBVI and equally or more accurate. Specifically, over a wide range of target posteriors, GSM-VI requires 10-100x fewer gradient evaluations than BBVI to obtain a comparable quality of approximation. $^{1}$
<sup>1</sup>We provide a Python implementation of GSM-VI algorithm at https://github.com/modichirag/GSM-VI.
This paper is about variational inference for approximate Bayesian computation. Consider a statistical model  $p(\boldsymbol{\theta}, \boldsymbol{x})$  of parameters  $\boldsymbol{\theta} \in \mathbb{R}^d$  and observations  $\boldsymbol{x}$ . Bayesian inference aims to infer the posterior distribution  $p(\boldsymbol{\theta} \mid \boldsymbol{x})$ , which is often intractable to compute. Variational inference is an optimization-based approach to approximate the posterior [5, 18].
The idea behind VI is to approximate the posterior with a member of a variational family of distributions  $q_{w}(\theta)$ , parameterized by variational parameters  $w$  [5, 18]. Specifically, VI methods establish a measure of closeness between  $q_{w}(\theta)$  and the posterior, and then minimize it with an optimization algorithm. Researchers have explored many aspects of VI, including different objectives [8, 9, 20, 26, 27, 29, 34] and optimization strategies [1, 15, 28].
In its modern form, VI typically minimizes KL  $(q_{w}(\theta) || p(\theta | x))$  with stochastic optimization, and further satisfies the so-called "black-box" criteria [1, 28, 33]. Black-box VI (BBVI) only requires the practitioner to specify the log joint  $\log p(\pmb{\theta}, \pmb{x})$  and (often) its gradient  $\nabla_{\pmb{\theta}} \log p(\pmb{\theta}, \pmb{x})$ , which for many models can be obtained by automatic differentiation [14, 25]. BBVI has been widely implemented, and it is available in many probabilistic programming systems [4, 21, 31].
In this paper, we propose a new approach to VI. We begin with the principle of score matching [16], that when two densities are equal then their gradients are equal as well, and we use this principle to derive a new way to fit a variational distribution to be close to the exact posterior. The result is score-matching VI. Rather than explicitly minimize a divergence, score-matching VI iteratively projects the variational distribution onto the exact score-matching constraint. This strategy enables a new black-box VI algorithm.
Score-matching VI relies on the same ingredients as reparameterization BBVI [21]--a differentiable variational family and a differentiable log joint--and so it can be as easily incorporated into probabilistic programming systems as well. Further, when the variational family is a Gaussian, score-matching VI is particularly efficient: each iteration is computable in closed form. We call the resulting algorithm Gaussian score matching VI (GSM-VI).
Unlike BBVI, GSM-VI does not rely on stochastic gradient descent (SGD) for its core optimization. Though SGD has the appeal of simplicity, it requires careful tuning of learning rates. GSM-VI was inspired by a different tradition of constraint-based algorithms for online learning [3, 7, 12, 13, 23]. These algorithms have been extensively developed and analyzed for problems in classification, and under the right conditions, they have been observed to outperform SGD. This paper shows how to extend this constraint-based framework—and the powerful machinery behind it—from the problem of classification to the workhorse of Gaussian VI. The key insight is that score-matching (unlike ELBO maximization) lends itself naturally to a constraint-based formulation.
We empirically compared GSM-VI to reparameterization BBVI on several classes of models, and with both synthetic and real-world data. In general, we found that GSM requires 10-100x fewer gradient evaluations to converge to an equally good approximation. When the exact posterior is Gaussian, we found that GSM-VI scales significantly better with respect to dimensionality and is insensitive to the condition number of the target covariance. When the exact posterior is non-Gaussian, we found that GSM-VI converges more quickly without sacrificing the quality of the final approximation.
- We introduce score-matching variational inference, a new black-box approach to fitting  $q_{\mathbf{w}}(\pmb{\theta})$  to be close to  $p(\pmb{\theta} \mid \pmb{x})$ . Score-matching VI requires no tunable optimization hyperparameters, to which BBVI can be sensitive.
- When the variational family is Gaussian, we develop Gaussian score-matching variational inference (GSM-VI). It establishes efficient closed-form iterates for score-matching VI.
- We empirically compare GSM-VI to reparameterization BBVI. Across many models and datasets, we found that GSM-VI enjoys faster convergence to an equally good approximation.
We develop score-matching VI in Section 2 and study its performance in Section 3.
Related work. This paper introduces a new method for black-box variational inference that relies only on having access to the gradients of the variational distribution and the log joint. GSM-VI has similar goals to automatic-differentiation variational inference (ADVI) [21] and Pathfinder [35], which also fit multivariate Gaussian variational families, but do so by maximizing the ELBO using stochastic optimization. As in GSM-VI, the algorithm of ref. [32] also seeks to match the scores of the variational and the target posterior, but it does so by minimizing the L2 loss between them.
A novel aspect of GSM-VI is how it fits the variational parameters. Rather than minimize a loss function, it aims to solve a set of nonlinear equations. Similar ideas have been pursued in the context of fitting a model to data using empirical risk minimization (ERM). For example, passive agressive (PA) methods [7] and the stochastic polyak stepsize (SPS) are also derived via projections onto sampled nonlinear equations [3, 13, 23]. A probabilistic extension of PA methods is known as confidence-weighted (CW) learning [12]. In CW learning, the learner maintains a multivariate Gaussian distribution over the weight vector of a linear classifier. Similarly, the second step of GSM-VI also minimizes a KL divergence between multivariate Gaussians. Unlike CW learning, however, the projection in GSM-VI enforces a score-matching constraint as opposed to a margin-based constraint for linear classification.
2 Score Matching Variational Inference
Suppose for the moment that the variational family  $q_{\boldsymbol{w}}(\boldsymbol{\theta})$  is rich enough to perfectly capture the posterior  $p(\boldsymbol{\theta} \mid \boldsymbol{x})$ . In other words, there exists a  $\boldsymbol{w}^*$  such that
$$
\log q _ {\boldsymbol {w} ^ {*}} (\boldsymbol {\theta}) = \log p (\boldsymbol {\theta} | \boldsymbol {x}), \quad \forall \boldsymbol {\theta} \in \Theta . \tag {1}
$$
If we could solve Eq. 1 for  $\boldsymbol{w}^*$ , the resulting variational distribution would be a perfect fit. The challenge is that the posterior on the right side is intractable to compute.
To help, we appeal to the idea of score matching [16]. We define the score of a distribution to be the gradient of its log with respect to the variable $^2$ , e.g.,  $\nabla_{\theta} \log q_{\mathbf{w}}(\theta)$ . The principle of score matching is that if two distributions are equal at every point in their support then their score functions are also equal.
To use score matching for VI, we first write the log posterior as the log joint minus the normalizing constant, i.e., the marginal distribution of  $x$ ,
$$
\log p (\boldsymbol {\theta} \mid \boldsymbol {x}) = \log p (\boldsymbol {\theta}, \boldsymbol {x}) - \log p (\boldsymbol {x}). \tag {2}
$$
With this expression, the principle of score matching leads to the following Lemma.
Lemma 2.1. Let  $\Theta \subset \mathbb{R}^d$  be a set that contains the support of  $p(\theta, x)$  and  $q_{\boldsymbol{w}^*}(\theta)$  for some  $\boldsymbol{w}^*$ . That is  $p(\theta, x) = q_{\boldsymbol{w}^*}(\theta) = 0$  for every  $\theta \notin \Theta$ . Furthermore, suppose that the support of  $p(\theta, x)$  and  $q_{\boldsymbol{w}^*}(\theta)$ , and  $\Theta$  is path-connected. The parameter  $\boldsymbol{w}^*$  satisfies
$$
\nabla_ {\boldsymbol {\theta}} \log q _ {\boldsymbol {w} ^ {*}} (\boldsymbol {\theta}) = \nabla_ {\boldsymbol {\theta}} \log p (\boldsymbol {\theta}, \boldsymbol {x}), \quad \forall \boldsymbol {\theta} \in \Theta , \tag {3}
$$
if and only if  $\pmb{w}^{*}$  also satisfies Eq. 1.
What is notable about Eq. 3 is that the right side is the gradient of the log joint. Unlike the posterior, the gradient of the log joint is tractable to compute for a large class of probabilistic models. (The proof of the lemma is in the appendix.)
This lemma motivates a new algorithm, score-matching VI. The idea is to iteratively refine the variational parameters  $\boldsymbol{w}$  in order to satisfy (as much as possible) the system of equations in Eq. 3. At each iteration  $t$ , the algorithm first samples a new  $\theta_t$  from the current variational approximation and then minimally adjusts  $\boldsymbol{w}$  to satisfy Eq. 3 for that value of  $\theta_t$ , see (4). We want this adjustment to be minimal because, eventually  $q_{\boldsymbol{w}_t}$  will learn a good fit—one that increasingly matches the scores over the support of the target posterior—and we want to preserve this good fit as much as possible.
2We make this clear because, in some literature, the score is defined as the gradient with respect to the parameter.
Score matching variational inference
At iteration  $t$ :
1. Sample  $\pmb{\theta}_t\sim q_{\pmb{w}_t}(\pmb {\theta})$
2. Update the variational parameters:
$$
\boldsymbol {w} _ {t + 1} = \arg \min  _ {\boldsymbol {w}} \operatorname {K L} \left(q _ {\boldsymbol {w} _ {t}} (\boldsymbol {\theta}) \mid \mid q _ {\boldsymbol {w}} (\boldsymbol {\theta})\right) \tag {4}
$$
such that  $\nabla_{\pmb{\theta}}\log q_{\pmb{w}}(\pmb{\theta}_t) = \nabla_{\pmb{\theta}}\log p(\pmb{\theta}_t,\pmb{x})$
This algorithm for score matching VI was inspired by earlier online algorithms for learning a classifier from a stream of labeled examples. One particularly elegant algorithm in this setting is known as passive-aggressive (PA) learning [7]; this algorithm incrementally updates its model by the minimal amount to classify each example correctly by a large margin. This approach was subsequently extended to a probabilistic setting, known as confidence-weighted (CW) learning [12] in which one minimally updates a distribution over classifiers. Our algorithm is similar in that it minimally updates an approximating distribution for VI, but it is different in that it enforces constraints for score matching instead of large margin classification.
At a high level, what makes this approach to VI likely to succeed or fail? Certainly it is necessary that there are more variational parameters than elements of the latent variable  $\theta$ ; when this is not the case, it may be impossible to satisfy a single score matching constraint in Eq. 4. That said, it is standard (as in, e.g., a factorized or mean-field variational family) to have at least as many variational parameters as there are elements of the latent variable. It is also apparent that the algorithm may never converge if the target posterior is not contained in the variational family, or it may converge to a degenerate solution if the variational approximation collapses to a point mass, thus terminating the updates altogether. While we cannot dismiss these possibilities out of hand, we did not observe erratic or pathological outcomes in any of the empirical studies of Section 3.
For more intuition, Figure 1a illustrates the effect of the update in Eq. 4 when both the target and approximating distribution are 1d Gaussian. The target posterior  $p(\boldsymbol{\theta} \mid \boldsymbol{x})$  is shaded blue. The plot shows the initial variational distribution  $q_{\boldsymbol{w}_0}$  (light grey curve) and its update to  $q_{\boldsymbol{w}_1}$  (medium grey) so that the gradient of the updated distribution matches the gradient of the target at the sampled  $\boldsymbol{\theta}_0$  (dotted red tangent line). It also shows the update from  $\boldsymbol{w}_1$  to  $\boldsymbol{w}_2$ , now matching the gradient at  $\boldsymbol{\theta}_1$ . With these two updates,  $q_{\boldsymbol{w}_2}$  (dark grey) is very close to the target  $p(\boldsymbol{\theta} \mid \boldsymbol{x})$ . With this picture in mind, we now develop the details of this algorithm for a more widely applicable setting.
Gaussian Score-Matching VI. Suppose the variational distribution belongs to a multivariate Gaussian family  $q_{\boldsymbol{w}}(\boldsymbol{\theta}) \coloneqq \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ ; this is a common choice especially in systems for automated approximate inference [1, 21]. One of our main contributions is to show in this case that Eq. 4 has a closed-form solution. The solution  $\boldsymbol{w}_{t+1} = (\boldsymbol{\mu}_{t+1}, \boldsymbol{\Sigma}_{t+1})$  has the following form:
$$
\boldsymbol {\mu} _ {t + 1} = \boldsymbol {\mu} _ {t} + \boldsymbol {A} _ {t} (\nabla_ {\boldsymbol {\theta}} \log p (\boldsymbol {\theta} _ {t}, \boldsymbol {x}) - \nabla_ {\boldsymbol {\theta}} \log q _ {\boldsymbol {w} _ {t}} (\boldsymbol {\theta} _ {t})) \tag {5}
$$
$$
\boldsymbol {\Sigma} _ {t + 1} = \boldsymbol {\Sigma} _ {t} + \left(\boldsymbol {\mu} _ {t} - \boldsymbol {\theta} _ {t}\right) \left(\boldsymbol {\mu} _ {t} - \boldsymbol {\theta} _ {t}\right) ^ {\top} - \left(\boldsymbol {\mu} _ {t + 1} - \boldsymbol {\theta} _ {t}\right) \left(\boldsymbol {\mu} _ {t + 1} - \boldsymbol {\theta} _ {t}\right) ^ {\top} \tag {6}
$$
where  $A_{t} \in \mathbb{R}^{d \times d}$  is a matrix defined in the theorem below. Note that these updates, which exactly solve the optimization in Eq. 4, only require the score of the log joint  $\nabla_{\theta} \log p(\pmb{\theta}, \pmb{x})$  and the score of the variational distribution  $\nabla_{\pmb{\theta}} \log q_{\pmb{w}}(\pmb{\theta})$ .
Before deriving Eqs. 5 and 6 in full, we highlight the intuitive form of this closed-form solution. Consider the approximation at the  $t$ th iteration  $q_{\boldsymbol{w}_t}$  and the current sample  $\boldsymbol{\theta}_t$ . First suppose the scores already match at this sample, that is  $\nabla_{\boldsymbol{\theta}}\log p(\boldsymbol{\theta}_t,\boldsymbol{x}) = \nabla_{\boldsymbol{\theta}}\log q_{\boldsymbol{w}_t}(\boldsymbol{\theta}_t)$ . Then the mean does not change  $\boldsymbol{\mu}_{t + 1} = \boldsymbol{\mu}_t$  and, similarly, the two rank-one terms in the covariance update in Eq. 6 cancel out so  $\Sigma_{t + 1} = \Sigma_t$ . This shows that when  $q_{\boldsymbol{w}_t}(\boldsymbol {\theta}) = p(\boldsymbol {\theta},\boldsymbol {x})$  for all  $\boldsymbol{\theta}$ , the method stops. On the other hand, if the scores do not match, then the mean is updated proportionally to the difference between the scores, and the covariance is updated by a rank-two correction. Figure 1b illustrates the vector field of updates for a one-dimensional target  $p(\boldsymbol {\theta},\boldsymbol {x}) = \mathcal{N}(0,1)$ . The vector field points to the solution (green star), which is also the only fixed point of the updates.
We now formalize this result and give the complete expression for the matrix  $\mathbf{A}_t$  that appears in the update for the mean of the variational approximation in Eq. 5.
Theorem 2.2. (GSM-VI updates) Let  $p(\theta, \boldsymbol{x})$  be given for some  $\theta \in \mathbb{R}^d$ , and let  $q_{\boldsymbol{w}^t}(\theta)$  and  $q_{\boldsymbol{w}}(\theta)$  be multivariate normal distributions with means  $\mu_t$  and  $\pmb{\mu}$  and covariance matrices  $\Sigma_t$  and  $\Sigma$ , respectively. As shorthand, let  $g_t \coloneqq \nabla_\theta \log p(\theta_t, \boldsymbol{x})$  and let
$$
\boldsymbol {\mu} _ {t + 1}, \boldsymbol {\Sigma} _ {t + 1} = \underset {\boldsymbol {\mu}, \boldsymbol {\Sigma} \succeq 0} {\operatorname {a r g m i n}} \left[ \mathrm {K L} \left(q _ {t}, q\right) \right] \quad \text {s u c h t h a t} \quad \nabla_ {\boldsymbol {\theta}} \log q (\boldsymbol {\theta} _ {t}) = \nabla_ {\boldsymbol {\theta}} \log p (\boldsymbol {\theta} _ {t}, \boldsymbol {x}). \tag {7}
$$
The solution to eq. (7) is given by Eqs. 5 and 6 where
$$
\boldsymbol {A} _ {t} := \frac {1}{1 + \rho} \left[ \mathbf {I} - \frac {\left(\boldsymbol {\mu} _ {t} - \boldsymbol {\theta} _ {t}\right) \boldsymbol {g} _ {t} ^ {\top}}{1 + \rho + \left(\boldsymbol {\mu} _ {t} - \boldsymbol {\theta} _ {t}\right) ^ {\top} \boldsymbol {g} _ {t}} \right] \boldsymbol {\Sigma} _ {t}, \tag {8}
$$
and  $\rho$  is the positive root of the quadratic equation
$$
\rho (1 + \rho) = \boldsymbol {g} _ {t} ^ {\top} \boldsymbol {\Sigma} _ {t} \boldsymbol {g} _ {t} + \left[ \left(\boldsymbol {\mu} _ {t} - \boldsymbol {\theta} _ {t}\right) ^ {\top} \boldsymbol {g} _ {t} \right] ^ {2}. \tag {9}
$$
With the definition of  $A_{t}$  in Eq. 8 we can see that the computational complexity of updating  $\pmb{\mu}$  and  $\pmb{\Sigma}$  via Eqs. 5 and 6 is  $\mathcal{O}(d^2)$ , where  $\pmb{\theta} \in \mathbb{R}^{d}$  and we assume the cost of computing the gradients is  $\mathcal{O}(d)$ . Note this is the best possible iteration complexity we can hope for, since we store and maintain the full covariance matrix of  $d^2$  elements. (The proof is in the appendix.)
Algorithm 1 presents the full GSM-VI algorithm. Here we also use mini-batching, where we average over  $B \in \mathbb{N}$  independently sampled updates of Eqs. 5 and 6 before updating the mean and covariance.
We evaluate how well GSM-VI can approximate a variety of target posterior distributions. Recall that GSM-VI uses a multivariate Gaussian distribution as its variational family. We investigate two separate cases—one when the target posterior is in this family, and the other when it is not.
Algorithmic details and comparisons. We compare GSM-VI with a reparameterization variant of BBVI as the baseline, similar to [21]. BBVI uses the same multivariate Gaussian variational family, which we fit by maximizing the ELBO. (Maximizing the ELBO is equivalent to minimizing KL).
Algorithm 1: Gaussian Score Matching VI
Input:Initial mean estimate  $\pmb{\mu}_0$  , initial covariance estimate  $\Sigma_0$  , target distribution  $p(\theta |x)$  number of iterations  $N\in \mathbb{N}$  , batch size  $B\in \mathbb{N}$    
Output:Multivariate normal variational distribution  $q_{w}(\theta)\coloneqq \mathcal{N}(\pmb {\mu},\pmb {\Sigma})$    
for  $i = 0,\dots ,N - 1$ $\triangleright$  iteration loop   
do   
for  $j = 0,\ldots ,B - 1$ $\triangleright$  batch loop   
do Sample  $\pmb{\theta}^{(j)}\sim \mathcal{N}(\pmb {\mu}_i,\pmb {\Sigma}_i)$ $\pmb {g}\gets \nabla_{\pmb{\theta}}\log p(\pmb{\theta}^{(j)}|\pmb {x})$ $\varepsilon \leftarrow \Sigma_i g - \mu_i + \theta$  Solve  $\rho (1 + \rho) = g^{\top}\Sigma_{i}g + [(\pmb{\mu}_{i} - \pmb{\theta})^{\top}g]^{2}$  for  $\rho >0$ $\delta \pmb{\mu}^{(j)}\gets \frac{1}{1 + \rho}\left[\mathbf{I} - \frac{(\pmb{\mu}_i - \pmb{\theta})^{\top}\pmb{g}^{\top}}{1 + \rho + (\pmb{\mu}_i - \pmb{\theta})^{\top}\pmb{g}}\right]\varepsilon$ $\pmb{\mu}_{i}^{(j)}\gets \pmb{\mu}_{i} + \delta \pmb{\mu}^{(j)}$ $\delta \Sigma^{(j)}\gets (\pmb {\mu}_i - \pmb {\theta})(\pmb {\mu}_i - \pmb {\theta})^\top -(\pmb {\mu}_i^{(j)} - \pmb {\theta})(\pmb {\mu}_i^{(j)} - \pmb {\theta})^\top$    
end   
Update  $\mu_{i + 1}\gets \mu_i + \sum_j\delta \mu^{(j)} / B$    
Update  $\Sigma_{i + 1}\gets \Sigma_i + \sum_j\delta \Sigma^{(j)} / B$    
end   
 $q_{w}(\theta)\gets \mathcal{N}(\pmb{\mu}_{N},\pmb{\Sigma}_{N})$
Algorithm 2: Black-box variational inference
Input:Initial mean estimate  $\mu_0$  , Initial covariance estimate  $\Sigma_0$  , target distribution  $p(\theta)$  number of iterations N, batch size B, learning rate  $\epsilon$    
Output:Multivariate normal variational distribution  $q_{w}(\theta)\coloneqq \mathcal{N}(\mu ,\Sigma)$ $q_{w}\gets \mathcal{N}(\pmb {\mu}_0,\pmb {\Sigma}_0)$  .   
for  $i = 0,\dots ,N - 1$  iteration loop   
do   
 $\{\pmb{\theta}^{(0)},\pmb{\theta}^{(1)},\dots,\pmb{\theta}^{(B)}\} \sim q_{\pmb{w}}(\pmb {\theta})$  Sample a batch of B points; ELBO  $= \sum_{j}\log (p(\pmb{\theta}^{(j)},\pmb {x}) - \log q_{\pmb{w}}(\pmb{\theta}^{(j)})$  .  $\boldsymbol {w}\leftarrow \boldsymbol {w} - \epsilon \nabla_{\boldsymbol{w}}\mathrm{ELBO}$  Optimization step, we use ADAM;   
end
We use the ADAM optimizer [19] with default settings but vary the learning rate between  $10^{-1}$  and  $10^{-3}$ . We report results only for the best performing setting. The full BBVI algorithm is shown in Algorithm 2.
The only free parameter in GSM-VI is the batch size  $B$ . We find that for Gaussian targets, GSM performs equally well for all  $B \geq 1$ . There are marginal gains for larger batches for high dimensional targets, but  $B = 2$  is a good conservative default. For non-Gaussian targets, we find that all batch sizes converge to the same solution, but we also note that larger batch sizes help to dampen the oscillations in KL divergence around the optimal solution. Notwithstanding these effects, it is not generally necessary to fine-tune this hyperparameter. In all studies of this section we report results for  $B = 2$  and show that it is a good default baseline. We use the same batch size for BBVI.
Both GSM-VI and BBVI require an initial distribution for the variational approximation. Unless specified otherwise, we initialize the variational approximation as a Gaussian distribution with zero mean and identity covariance matrix.
Evaluation metric. GSM-VI does not explicitly minimize any loss function. Hence to compare its performance against BBVI, we estimate empirical divergences between the variational and the target distribution and show their evolution with the number of gradient evaluations. We measure performance in terms of gradient evaluations as the actual running time can be sensitive
(a) Evolution of forward KL divergence,  $\mathrm{KL}(\mathsf{p}\| \mathsf{q})$
(b) Evolution of reverse KL divergence,  $\mathrm{KL}(q\| p)$