SlowGuess/NeurIPS_2023_OCR huggingface dataset:
density estimator  $q_{\phi}(\pmb {\xi})$  is trained to approximate the target posterior  $\pi_t(\pmb {\xi})$  via the m-projection (11). Note that the expectation w.r.t. the proposal distribution  $\pi_{t - 1}(\pmb {\xi})$  enables gradient descent on the  $q_{\phi_t}$  estimator without requiring a differentiable simulator. Fourth, in line 12, we set the current posterior approximation  $q_{\phi_t}(\pmb {\xi})$  as the proposal  $\pi_t(\pmb {\xi})$  for the next inference step, thus leveraging bootstrapping of the density estimator. While we restrict the analysis in this paper to the m-projection, we note that the i-projection can also be employed, as shown in Appendix A.3.
Algorithm 1 Pseudo-Likelihood Inference (PLI)
1: input: reference data  $\mathbf{x}_{1:N}^{\star}$ , prior  $p(\boldsymbol{\xi})$ , stochastic simulator  $p(\mathbf{x}|\boldsymbol{\xi})$ , IPM  $D(\cdot, \cdot)$ , posterior approximator  $q_{\phi}(\boldsymbol{\xi})$ , max. iter.  $T$   
2: initialize proposal prior  $\pi_0(\boldsymbol{\xi}) = p(\boldsymbol{\xi})$   
3: for  $t$  in  $1:T$  do  
4: sample parameters  $\boldsymbol{\xi}^{(1:K)} \sim \pi_{t-1}(\boldsymbol{\xi})$   
5: for each  $\boldsymbol{\xi}^{(k)}$  do  
6: simulate data  $\mathbf{x}_{1:M}^{(k)} \sim p(\mathbf{x}|\boldsymbol{\xi}^{(k)})$   
7: compute IPM  $s^{(k)} = D(\mathbf{x}_{1:M}^{(k)}, \mathbf{x}_{1:N}^{\star})$   
8: end for  
9: update  $\eta_t$  by maximizing the dual (10)  
10: evaluate  $\tilde{p}_t(\mathbf{x}_{1:N}^{\star}|\boldsymbol{\xi}^{(k)})$  (9)  
11: fit  $q_{\phi_t}(\boldsymbol{\xi})$  by WML (11)  
12: set new proposal prior  $\pi_t(\boldsymbol{\xi}) = q_{\phi_t}(\boldsymbol{\xi})$   
13: end for  
14: output: approximate posterior  $q_{\phi_T}(\boldsymbol{\xi})$
Normalization. The normalization term  $Z(\pmb{\xi})$  in the definition of the pseudo-likelihood (8) requires taking an integral over the reference data  $\mathbf{x}_{1:N}^{\star}$ , which is infeasible in practice. When the KL divergence is used in the kernel,  $Z(\pmb{\xi})$  does not depend on  $\pmb{\xi}$ , as shown in Lemma 1. While in general, the dependence on  $\pmb{\xi}$  cannot be neglected, its influence on the weights in (10) and (11) may be negligible, provided the relative ranking of the samples is not affected significantly. In Appendix A.4, we provide an ablation study on low-dimensional problems where the integral over  $\mathbf{x}_{1:N}^{\star}$  can be approximated by sampling. We observed that, even though the ranking correlation of the weights  $w^{(k)}$  in (11) is different with and without estimating  $Z(\pmb{\xi})$ , the final posterior is not affected. Therefore, in the subsequent experiments, we treat it as a constant, as in the ideal case of the KL divergence. Nevertheless, this is a point where our practical implementation does not follow the theoretical derivation strictly, and this issue should be addressed in future work.
We compare the PLI framework against SMC-ABC [15] and APT [24] on five diverse tasks. Our implementation is based on Wasserstein-ABC [3], but instead of the employed r-hit kernel [30], our implementation is based on population Monte Carlo (Alogrithm 3 [31]) because we observed improved performance in preliminary studies. A summary of the different ABC methods is given in Appendix B and Table A.1. APT was chosen as the representative for the class of SNPE algorithms. We leverage Neural Spline Flows (NSFs) [19] as density estimators for both PLI and APT. Both neural flow configurations share the same base network architecture, but for APT, the conditional flow is augmented with an embedding network (Appendix C). All experiments are implemented in JAX [4], and each ran on a single Nvidia RTX 3090. To make the experiments comparable, the simulation budgets of PLI and APT were fixed to 5000 samples per inference step over 20 episodes, while ABC ran for 200 episodes on 1000 particles.
The model is compared against the reference posterior samples  $\xi^{\star}$  when available. We also quantify the methods' performances based on their realizations by computing the Wasserstein distance and
<sup>1</sup>https://github.com/theogruner/pseudo_likelihood_inference
Figure 3: Evaluation of the posterior performance on five different tasks. We report the mean and  $95\%$  ci over 10 random seeds, each carried out using  $N$  data points for conditioning. We compare samples from the approximate posterior  $\xi \sim q(\pmb {\xi}|\mathbf{x}_{1:N}^{\star})$  against reference posterior samples using MMD and the Wasserstein distance. No posterior samples were available for the Furuta pendulum. Therefore, the performance is evaluated in the observation space. Lower values are better for all metrics. PLI is the preferred method for conditioning on multiple observations due to its steady improvement with increasing  $N$ . ABC performs better than PLI on Gaussian Location and Gaussian Mixture tasks but lags in more complex tasks. APT excels with few observations but degrades as  $N$  increases.
the MMD. The comparison is carried out on 10 000 samples each. Furthermore, we use Posterior Predictive Checks (PPCs) to evaluate the predictive capabilities of the posterior models in the observation space  $\mathbb{E}_{q(\pmb{\xi}|\mathbf{x}_{1:N}^{\star})}[D(\mathbf{x}_{1:N}^{\star},\mathbf{x}_{1:M})]$ . Due to the computational limits, the PPCs are carried out on 1000 simulations against the reference data. Lueckmann et al. [32] also report results with classifier-based tests and the kernelized Stein-discrepancy. However, since benchmarking is not our focus, we restrict the analysis to comparing with the Wasserstein and MMD.
4.2 Tasks
We evaluate PLI on four common benchmarking tasks within the SBI community [32]: Gaussian Location, a Gaussian Mixture Model, Simple-Likelihood Complex-Posterior, and SIR. Further, we add a system identification task on a Furuta pendulum representing a highly dynamic continuous control system. The tasks' specifications are listed in Appendix C. For each task, we conduct experiments for different numbers of available reference observations  $N = \{1, 2, 5, 10, 20, 50, 100, 1000\}$ . The reference observations are simulated based on a pre-defined ground-truth parameter  $\xi^{\mathrm{gt}}$ . Although PLI and ABC can cope with varying numbers of observations  $N$  and numbers of simulations per parameter  $M$ , we choose  $N = M$  for all experiments since it is required by APT. In the following paragraphs, we first discuss the results of the benchmarking tasks and present a separate discussion for the Furuta pendulum. Figure 3 gives a quantitative overview of the benchmarking tasks compared to the reference posteriors, while Figure C.1 in the Appendix complements the study by showing the posterior predictive performances.
**Benchmarking tasks.** Each benchmarking task presents different challenges that must be addressed by the SBI methods. In Gaussian Location, the task is to infer a uni-modal 10-dimensional Gaussian distribution. Gaussian Mixture Model and SLCP feature multi-modal posteriors that require flexible density estimators. SIR is a well-known epidemiological model that features 10-dimensional data of the dynamical system. All methods generally depict a reoccurring behavior on the different benchmarking tasks, as shown in Figure 3. For fewer observations ( $N \lesssim 20$ ), APT matches the reference posterior better than the other approaches, whereas ABC and PLI match the posterior data better with increasing  $N$ . In particular, PLI consistently improves with an increasing number of reference samples. The influence of  $N$  on the shape of the posterior is further visualized in Figure C.2, which compares the posterior approximations of all methods for  $N = 2$  and  $N = 100$  reference observations. For the SLCP task, ABC struggles to capture the multi-modality of the SLCP task. This effect is further illustrated when comparing Figures C.3 and C.4, which allow for a qualitative
Figure 4: Empirical and quantitative evaluation on the Furuta pendulum. (top) Snapshots of the posterior evolution on the  $g - m_{p}$  plane on the Furuta pendulum for  $N = 1000$ . (bottom) Predictive performance of the learned MMD-PLI posterior for the angular rotation  $\sin \theta_{r}$ . The stochasticity of the simulator is removed by synchronizing the initial state between the reference and predicted simulations. Thus, the only discrepancies between trajectories are due to the model not capturing the dynamics parameters of the system. After the inference has been completed (step 20), the predictive simulator (—MMD-PLI) can completely recover the ground truth dynamics (—Reference). (right) Evaluation of the mean accumulated error over 1000 trajectories with synchronized initial states between the simulation and the reference trajectory. All approaches improve with rising  $N$  while PLI with MMD matches the reference data best.
assessment of the posterior approximations. On SIR, PLI variants show significantly improved performance compared to the other baselines. Generally, we find that ABC and PLI perform better with MMD than with Wasserstein distance. This observation can be attributed to the Wasserstein distance not scaling well to high dimensions, which has been reported by recent studies [17, 16].
Furuta pendulum. The Furuta pendulum is an inverted double pendulum setup [21]. While the system's dynamics are inherently deterministic, small perturbations of the initial state around its unstable equilibrium point lead to highly diverse trajectories. The observation space is  $T \times 6$  dimensional, where  $T$  is the number of time steps per trajectory. We set the sampling frequency of the simulation to  $100\mathrm{Hz}$  and the duration to 1 sec, resulting in 600-dimensional observations. No reference posterior is available for this task; thus, the analysis is restricted to quantifying the observed data. Given the similarity of the Wasserstein distance and the MMD in parameter and observation space on the previous tasks, we argue that a comparison based on PPCs, i.e.,  $\mathcal{W}_2^2 (\mathbf{x}_{1:N}^\star ,\mathbf{x}_{1:M})$  and  $\mathrm{MMD}^2 (\mathbf{x}_{1:N}^\star ,\mathbf{x}_{1:M})$ , is sufficient. In Figure 3, the Wasserstein PPC and MMD PPC exhibit divergent behaviors, with the former indicating enhanced performance as reference observations increase, in contrast to the moderate improvement suggested by MMD. Therefore, we evaluate the models' predictive performances on the deterministic system by synchronizing the initial states of the reference data and the simulations in Figure 4. This modification ensures that the similarity between two rollouts can be evaluated by the accumulated error  $\mathrm{err} = \sum_{i}|x_{i}^{\star} - x_{i}|$ . While all approaches perform better with more reference observations, MMD-PLI matches the reference dynamics best. Additionally, MMD is favored here, as MMD-based PLI and ABC outperform their Wasserstein counterparts, with both the MMD PPC plot and error plot showcasing congruent trends for  $N \geq 20$ . The appended posterior plots in Figures C.5 and C.6 reveal that for  $N = 2$ , all methods are widely spread over the prior region, yet converge to the ground truth. However, APT cannot recover the ground truth for  $N \geq 100$ , whereas PLI and ABC center around the ground truth.
In the previous sections, we have seen that PLI is algorithmically similar to ABC methods with SMC samplers. Therefore, approximating the likelihood by the empirical pseudo-likelihood (2) enables
drawing from the rich toolbox of existing approximate inference algorithms. This section introduces related research fields and shows how PLI fits among them.
Sequential neural density estimation. With the enriched class of neural density estimators, amortized SBI methods have received increasing interest in recent years. Similar to ABC, synthetic samples from the simulator are used to approximate the posterior. Sequential neural density estimation methods can be further classified into methods that directly train a posterior estimator [40, 24], a neural likelihood [41, 22], or a neural ratio estimator [19, 36]. All methods have in common that they do not rely on an approximation of the posterior model but are optimized solely on pairs of parameter samples from a proposal distribution  $\pmb{\xi}^{(k)}\sim p_t(\pmb {\xi})$  and its corresponding simulation  $\mathbf{x}^{(k)}\sim p(\mathbf{x}|\pmb{\xi}^{(k)})$ . We note that the original papers have only reported posteriors conditioned on a single observation  $\mathbf{x}$ . While technically, these methods can incorporate multiple data points, this requires either stacking multiple observations or falling back to summary statistics. As noted in [50], neural likelihood estimators can sidestep these requirements by evaluating the log-likelihood of single observations and carrying out MCMC sampling on the joint log-likelihood. Yet, leveraging the neural likelihood restricts the evaluation of the posterior.
Summary statistics. ABC has commonly relied on reducing the dimensionality of the raw observations with summary statistics [48]. These summaries must be carefully chosen and are often task-specific, restricting the general applicability of ABC. Recent additions to ABC methods report on replacing summary statistics with statistical distances [17]. While direct comparison of the raw data suffers from the curse of dimensionality, comparing the observations through empirical measures sidesteps this issue [17]. Bernton et al. [3] report on augmenting the likelihood kernel with the Wasserstein distance, while Park et al. [42] leverage the kernelized approximation of the MMD [25]. Other contributions include the Cramér-von-Mises distance [20] and the energy distance [39]. While statistical distances are appealing due to their general applicability, Drovandi and Frazier [17] conclude that they are limited by their high computational requirements. Approaches proposed for automated summary design include ABC with indirect inference, which utilizes an auxiliary model to evaluate data summaries [23, 18].
Particle mirror descent. A posterior updating similar to ours (7) has been derived in Particle Mirror Descent (PMD) [10]. PMD tackles particle depletion by incorporating the proposal distribution of the previous round into the optimization process. Furthermore, the authors show that the proposed method converges to the posterior given  $m$  posterior samples by  $\mathcal{O}(1 / \sqrt{m})$ . Our version can be seen as extending their approach to the case of intractable likelihoods. We extend PMD to neural density estimators using samples from the proposal posterior (7) as a training set.
Geometric path and likelihood tempering. Rewriting the optimal posterior (7) reveals a close relation of the optimal PLI posterior (4) and the geometric path formulation [6, p. 335],  $\pi_t(\pmb{\xi}) \propto \pi_{t-1}^{1-\lambda}(\pmb{\xi}) p(\mathbf{x}_{1:N}^{\star}, \pmb{\xi})^{\lambda}$ . The optimal posterior moves from the proposal distribution  $\pi_t(\pmb{\xi})$  at inference time  $t$  to the target posterior  $\pi_t(\pmb{\xi} | \mathbf{x}_{1:N}^{\star}) \propto p(\mathbf{x}_{1:N}^{\star}, \pmb{\xi})$  along the geometric path that is parameterized by  $\lambda$ . The formulation differentiates from likelihood tempering in SMC samplers [6] by leveraging the proposal instead of the prior distribution. Note, however, that for  $t = 0$ , the proposal mimics the prior, and thus the PLI geometric path has the same boundary values as in classical likelihood tempering. While the tempered posterior cannot be applied to SMC samplers due to its dependence on the proposal distribution, the geometric path formulation based on the prior distribution gives rise to sequential annealing ABC [1].
Generalized variational inference. Introduced by Knblauch et al. [29], Generalized Variational Inference (GVI) is an extension of the standard variational inference framework that starts from the optimization view of Bayes' rule and generalizes it by considering different losses, divergences, and variational families. Therefore, various Bayesian inference methods can be seen as instantiations of GVI with different choices of these three parameters. In particular, ABC and PLI can be seen as GVI with the choices  $P(K_{\beta}(\mathbf{x}_{1:N}^{\star}, \mathbf{x}_{1:M}), \mathrm{KL}, \mathcal{P}(\Theta))$  since they employ a likelihood kernel as a loss. Crucially, specifying a different loss function instead of the typical log-likelihood can be shown to add robustness against model misspecification [29]. PAC-Bayes [26] can be seen as a generalization of ABC as it covers the whole space of possible loss functions  $l(\mathbf{x}_{1:N}^{\star}, \pmb{\xi})$ . The PAC-Bayesian theory provides a broad array of risk bounds for generalized Bayesian learning methods.
We propose Pseudo-Likelihood Inference (PLI), a new addition to the toolbox of SBI methods. PLI is targeted for Bayesian inference tasks in which the posterior is conditioned on multiple observations simultaneously. For that, we derive a softened ABC posterior from a constrained variational inference problem and leverage IPMs between the empirical observations to assess the intractable likelihood. The derived posterior formulation enables the learning of flexible neural density estimators from black-box simulators, extending the range of applicability for ABC methods. Our experiments assess how well PLI, ABC, and SNPE perform based on their generative power when given varying amounts of reference observations as a condition. Given few observations, SNPE-based methods perform better than ABC and PLI, which rely on statistical distances. However, when more data is available, ABC and PLI methods perform better. When the posterior distribution is simple, ABC is efficient at reproducing it using fast particle updates. However, PLI is a better option for complex posterior distributions because of its more adaptable neural density estimator. Additionally, PLI evaluates the posterior probability, which is useful in downstream tasks that require uncertainty quantification.
Limitations. In PLI, the computational cost is distributed among three main computations: the simulation, the summary statistics estimation, and the normalizing flow training. While the computational effort is large for every computation, PLI leverages parallelization on GPUs for all three computations. In the provided experiments, the training process of the neural model takes the main computational budget, while simulation and summary statistics are negligible. On a Nvidia RTX 3090, ABC typically runs  $2 - 10\mathrm{min}$ , while PLI and SNPE take  $60 - 90\mathrm{min}$ , depending on the task. Simpler models however, such as multivariate Gaussian or GMM, reduce the computation time to the simulation. Furthermore, we would like to express that high-fidelity simulators might increase the simulation time significantly, making the simulation the most costly operation within the inference pipeline. Instead of utilizing IPMs, one could exploit adversarial strategies to approximate the KL divergence, as explored by Mescheder et al. [35] and Santana and Hernandez-Lobato [45].
This work was funded by the Hessian Ministry of Science and the Arts (HMWK) through the projects "The Third Wave of Artificial Intelligence - 3AI", hessian.AI, and the grant "Einrichtung eines Labors des Deutschen Forschungszentrum für Künstliche Intelligenz (DFKI) an der Technischen Universität Darmstadt". Fabio Muratore was employed by the Robert Bosch GmbH during the time of this collaboration.
[1] Carlo Albert, Hans R Künsch, and Andreas Scheidegger. A simulated annealing approach to approximate bayes computations. Statistics and Computing, 25(6):1217-1232, 2015.
[2] Mark A. Beaumont. Approximate bayesian computation in evolution and ecology. Annual Review of Ecology, Evolution, and Systematics, 41(1):379-406, 2010.
[3] Espen Bernton, Pierre E. Jacob, Mathieu Gerber, and Christian P. Robert. Approximate bayesian computation with the wasserstein distance. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 81(2):235-269, 2019.
[4] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018.
[5] Badr-Eddine Chérief-Abdellatif and Pierre Alquier. Mmd-bayes: Robust bayesian estimation via maximum mean discrepancy. In Symposium on Advances in Approximate Bayesian Inference, 2019.
[6] Nicolas Chopin, Omiros Papaspiliopoulos, et al. An introduction to sequential Monte Carlo, volume 4. Springer, 2020.
[47] S. A. Sisson, Y. Fan, and Mark M. Tanaka. Sequential monte carlo without likelihoods. Proceedings of the National Academy of Sciences, 104(6):1760-1765, 2007.
[48] Scott A Sisson, Yanan Fan, and Mark Beaumont. Handbook of approximate Bayesian computation (1st ed.). CRC Press, 2018.
[49] Simon Tavare, David J Balding, R C Griffiths, and Peter Donnelly. Inferring Coalescence Times From DNA Sequence Data. Genetics, 145(2):505-518, 1997.
[50] Alvaro Tejero-Cantero, Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pedro J. Gonçalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulation-based inference. Journal of Open Source Software, 5(52):2505, 2020.
[51] Tina Toni, David Welch, Natalja Strelkowa, Andreas Ipsen, and Michael P.H Stumpf. Approximate bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of The Royal Society Interface, 6(31):187-202, 2009.
A Algorithmic details: pseudo-likelihood inference
A.1 Deriving the optimal PLI parameter distribution
$$
\pi_ {t} (\boldsymbol {\xi}) \propto \left(\frac {p (\boldsymbol {\xi})}{\pi_ {t - 1} (\boldsymbol {\xi})}\right) ^ {\frac {1}{1 + \eta_ {t}}} p \left(\mathbf {x} _ {1: N} ^ {\star} | \boldsymbol {\xi}\right) ^ {\frac {1}{1 + \eta_ {t}}} \pi_ {t - 1} (\boldsymbol {\xi}) \tag {12}
$$
Proof. The solution to the stochastic search problem (6) can be obtained from Lagrangian optimization. The optimization problem is restated here for readability
$$
\pi_ {t} (\boldsymbol {\xi}) = \underset {\pi (\boldsymbol {\xi})} {\arg \min } \operatorname {K L} \left(\pi (\boldsymbol {\xi}) |   | p (\boldsymbol {\xi} | \mathbf {x} _ {1: N} ^ {\star})\right),
$$
$$
\begin{array}{l} \text {s . t .} \operatorname {K L} (\pi (\boldsymbol {\xi}) | | \pi_ {t - 1} (\boldsymbol {\xi})) \leq \varepsilon , \end{array}
$$
$$
\int \pi (\boldsymbol {\xi})   \mathrm {d} \boldsymbol {\xi} = 1.
$$
We decompose the KL objective into two terms by applying Bayes' rule
$$
\operatorname {K L} \left(\pi (\boldsymbol {\xi}) \mid p \left(\boldsymbol {\xi} \mid \mathbf {x} _ {1: N} ^ {\star}\right)\right) = - \underset {\pi (\boldsymbol {\xi})} {\mathbb {E}} \left[ \log p \left(\mathbf {x} _ {1: N} ^ {\star} \mid \boldsymbol {\xi}\right) \right] + \operatorname {K L} \left(\pi (\boldsymbol {\xi}) \mid p (\boldsymbol {\xi})\right). \tag {13}
$$
The constrained optimization problem (6) can be reformulated with Lagrange multipliers as
$$
\begin{array}{l} \mathcal {L} (\pi) = - \int \pi (\boldsymbol {\xi}) \log p \left(\mathbf {x} _ {1: N} ^ {\star} | \boldsymbol {\xi}\right) \mathrm {d} \boldsymbol {\xi} \\ + \int \pi (\boldsymbol {\xi}) \log \frac {\pi (\boldsymbol {\xi})}{p (\boldsymbol {\xi})} d \boldsymbol {\xi} \\ + \eta \left(\int \pi (\boldsymbol {\xi}) \log \frac {\pi (\boldsymbol {\xi})}{\pi_ {t - 1} (\boldsymbol {\xi})} \mathrm {d} \boldsymbol {\xi} - \varepsilon\right) \\ + \lambda \left(\int \pi (\boldsymbol {\xi}) \mathrm {d} \boldsymbol {\xi} - 1\right) \\ = \int \pi (\boldsymbol {\xi}) \left[ - \log p \left(\mathbf {x} _ {1: N} ^ {\star} \mid \boldsymbol {\xi}\right) + \log \frac {\pi (\boldsymbol {\xi})}{p (\boldsymbol {\xi})} + \eta \log \frac {\pi (\boldsymbol {\xi})}{\pi_ {t - 1} (\boldsymbol {\xi})} + \lambda \right] d \boldsymbol {\xi} - \eta \varepsilon - \lambda . \tag {14} \\ \end{array}
$$