Variational methods for simulation-based inference
Authors: Manuel Glöckler
Michael Deistler
Jakob H. Macke
Venue: ICLR 2022 Spotlight (2022)
Keywords: likelihood-free inference
simulation-based inference
variational inference
neural density estimation

Abstract: We present Sequential Neural Variational Inference (SNVI), an approach to perform Bayesian inference in models with intractable likelihoods. SNVI combines likelihood-estimation (or likelihood-ratio-estimation) with variational inference to achieve a scalable simulation-based inference approach. SNVI maintains the flexibility of likelihood(-ratio) estimation to allow arbitrary proposals for simulations, while simultaneously providing a functional estimate of the posterior distribution without requiring MCMC sampling. We present several variants of SNVI and demonstrate that they are substantially more computationally efficient than previous algorithms, without loss of accuracy on benchmark tasks. We apply SNVI to a neuroscience model of the pyloric network in the crab and demonstrate that it can infer the posterior distribution with one order of magnitude fewer simulations than previously reported. SNVI vastly reduces the computational cost of simulation-based inference while maintaining accuracy and flexibility, making it possible to tackle problems that were previously inaccessible.

Published as a conference paper at ICLR 2022
VARIATIONAL METHODS FOR SIMULATION-BASE BASED INFERENCE
Manuel Glöckler University of Tübingen
Michael Deistler University of Tübingen
Jakob H. Macke
University of Tübingen
ABSTRACT
We present Sequential Neural Variational Inference (SNVI), an approach per- form Bayesian inference in models with intractable likelihoods. SNVI combines likelihoodessimmatio (or eeelihoodd-aaio-estiatiiee
with variational inference achieve a scalable simulation-base inference approach. SNVI maintains the flex- ibility of likelihoodd-ratte estimation to allow arbitrary proposals for simulations. while simultaneously providing functional estimate of the posterior distribution without requiring MCMC sampling. We present several variants of SNVI and demonstrate that they are substantially more computationally efficient than previ- ous algorithms, without loss of accuracy on benchmark tasks. We apply SNVI a neuroscience model of the pyloric network in the crab and demonstrate that it can infer the posterior distribution with one order of magnitude fewer simulations than previously reported. SNVI vastly reduces the computational cost of simulation- based inference while maintaining accuracy and flexibility, making it possible to tackle problems that were previously inaccessible.
INTRODUCTION
Many domains in science and engineering use numerical simulations to model empirically observed phenomena. These models are designed by domain experts and are built to produce mechanis- tic insights. However, in many cases, some parameters of the simulator cannot be experimentally measured and need to be inferred from data. A principled way to identify parameters that match empirical observations is Bayesian inference. However, for many models of interest, one can only sample from the model by simulating (stochastic) computer program, but explicitly evaluating the likelihood p(z(0) is intractable. Traditional methods to perform Bayesian inference in such iiuullationbbssed inference (SBI), also known as likelihood free inference scenarios, include Ap- proximate Bayesian computation (ABC) (Beaumont et al., (002) and synthetic likelihood (SL) (Wood] 2010) methods. However, these methods generally struggle with high-dimensional data and typically require one to design or learn (Chen et al. 2021) summary statistics and distance functions.
Recently, several methods using neural density(-ratio) estimation have emerged. These methods Lueckmann
train neural networks to learn the posterior (SNPE, Paaamakarios & Murray 2016:
et al. 2017 Greenberg et al. 2019), the likelihood (SNLE, Papamakarios et al. 2019
Lueckmann ratio (SNRE, Thomas al. 2022 Hermans al. 2020
al. (a1 or the siceihoood-to-evideee
Durkan et al. 2020 Miller et al. (2022).
To improve the simulation efficiency of these methods, sequential training schemes have been pro- posed: Initially, parameters are sampled from the prior distribution to train an estimation-netwant Subsequently, new samples are drawn adaptively to focus training on specific regions in parameter space, thus allowing the methods to scale to larger models with more parameters.
In practice, however, has remained a challenge to realize the full potential of these sequential schemes: For sequential neural posterior estimation (SNPE) techniques, the loss function needs to be adjusted across rounds (Greenberg et al. 2019), and it has been reported that this can be problematic if the proposal distribution is very different from prior, and lead to 'leakage' of probability mass into regions without prior support (Durkan et al.) 2020). Both sequential neural likelihood (SNLE) and likelihood-ratic (SNRE) methods require MCMC sampling, which can become prohibitively slow- MCMC sampling is required for each round of simulations, which, for high-dimensional models, can take more time than running the simulations and training the neural density estimator.

---

Published as a conference paper at ICLR 2022
Prior p48)
Simulations ((0,))
Joint p(x, 0)
Variational posterior Q₂(0)
Xate
Parameter
Sim feideeeed dells
Sinzfoled
Simulator (,)
Parameter
Paramater
min (xxii))))
Figure 1: Illustration of SNVI. We first learn the likelihood p( for any We then use variational inference to learn the posterior distribution by minimizing a general divergence measure D. The obtained posterior distribution is sampled with sampling importance resampling (SIR) run new simulations and refine the likelihood estimator.
methods and functional approximation the posterior which be evaluated without the need sampling. Conversely, key advantage of Akelihooo((-aatio targeting methods is their flexibility- learned likelihoods can e.g. used to integrate information from multiple observations, can be used without retraining if the prior is changed. In addition, they can be applied with any active- learning scheme without requiring modifications of the foss-function.
Our goal is to provide method which combines the advantages of those targeting likelihoodd-rations Posterior targeting methods allow rapid inference by providing
MCMC
We achieve this method by combining likelihood(-ratio) estimation with variationally learned in- ference networks using normalizing flows (Rezende Mohamed| [013 Papamakarios al. 2017] Durkan et al. 2019a) and sampling importance resampling (SIR) (Rubin) 1988). We name our ap- proach Sequential Neural Variational Inference (SNVT). We will show that our simulation-basee inference methods are accurate as SNLE and SNRE, while being substantially faster inference as they do not require MCMC sampling. In addition, real-world simulators sometimes produce in- valid outputs, e.g. when simulation fails. We introduce strategy that allows likelihood(-ratte targeting methods (such as SNVI) to deal with such invalid simulation outputs.
A recent method termed "Sequential Neural Posterior and Likelihood Approximation" (SNPLA) also proposed to use variational inference (VI) instead of MCMC to speed up inference in likelihood- targeting methods (Wiqvist al., 202T). While this proposal is related our approach, their VI objective is based on the reverse Kullback Leibler (rKI.) divergence for learning the posterior. As also show on benchmark tasks, this leads to mode-seeking behaviour which can limit its perfor- mance. In contrast, we show how this limitation can be overcome through modifying the variational objective in combination with using SIR for adjusting posteriors.
After an introduction inference (Sec. 2), we present our method. Sequential Neural Variational Inference (SNVI) (Sec. 3) In Sec. 4.2. we empirically show that SNVI is significantly faster than state-of-the-art SBI methods while achieving similar accuracy on benchmark tasks. In Sec. 4.3 we demonstrate that SNVI scalable, and that is robust to invalid simulation outputs: We obtain the posterior distribution of complex neuroscience model with one order of magnitude fewer simulations than previous methods.
neural network-based simulation- based inference (SBI) and variational
BACKGROUND
2.1 SIMULATION-BASET INFERENCE
Simulation- based inference (SBI) aims to perform Bayesian inference on statistical models for which the likelihood function is only implicitly defined through stochastic simulator. Given prior p(@) and simulator which implicitly defines the likelihood p(æ 0), the goal is to identify the posterior distribution p(0)z,, x.)) for observation Φ.. The simulator is considered to be 'black-box', ie. one cannot evaluate p( 0) and does not have access to the internal states the simulator, but only to its inputs A and its outputs
2

---

Published as a conference paper at ICLR 2022
We focus improving likelihood (SNLE) and Akelihood--rato--sttinne (SNRE)
methods. SNLE trains a deep neural density estimator (a(0) by minimizing the forward Kullback-Leibler divergence (fKL) between Ex (x)0) and p( 0) using samples (x, p(xx 0)
(()) from the simulator with L(v) = - Σn= log Ey (a; (0.). Here, (x(x) a condi- tional density estimator learning the conditional density (() from (0, æ) pairs, 16 are its learnable
parameters, and p(0) is the proposal distribution from which the parameters 0 are drawn (given by
e.g. previous estimate of the posterior or by an active learning scheme, Paaaaakarios al. 2019
Lueckmann et al. (019a).
Analogously, SNRE uses discriminator, e.g. deep logistic regression network, to estimate the den- sity ratio rax, 0) f( p(a)0) (Hermans et al. 2020 Durkan al. 2020 If the proposal
[() p(#)
given by the prior, then one can recover the exact posterior density, otherwise the posterior can be re- covered up normalizing constant (Durkan et al., 2020). Once the likelihood (or likelihood
has been learned, the posterior can be sampled with MCMC. In sequential schemes, the proposal ñ is updated each round using the current estimate of the posterior thus, computationally expensive MCMC sampling needs to be run in each round.
2.2
VARIATIONAL INFERENCE
We use variational inference (VI) to estimate the posterior distribution. VI formulates an optimiza- tion problem over class of tractable distributions Q to find parameters o² such that qs- E Q is closest to the true posterior ( p()) according to some divergence D (Blei et al. 2017). Formally,
Φ arg min Digs (0)ll((((z.)))
with f-- (0) = p (0)xz))
Dggs- (0) 10 x.,) = 0. Recent work has introduced normaliz-
ing flows variational family for VI (Ranganath et al.; 2014 Agrawal al. 2020: Rezende &
Mohamed 2015). Normalizing flows define a distribution q by learning a bijection To which
transtorms a simpler distribution into a complex distribution p(0
(0))))).) Normalizing flows provide
highly flexible variational family, while at the same time allowing low variance gradient estimation
of an expectation by the reparameterization trick, i.e. VoEangal [O)] Eg of ( (o))
with 0 (0,) (Kingma & Welling 2014 Rezende et al. 2014
Rezende & Mohamed 2015).
3
SEQUENTIAL NEURAL VARIATIONAL INFERENCE (SNVI)
3.1 KEY INGREDIENTS
We propose a framework to use variational inference (VT) for simulation- based inference. Our method consists of three parts: A learnable likelihood (or likelihood-ratio) model, posterior model (lypically parameterized as a normalizing flow) to be learned with VI, and sampling importance resampling (SIR) (Rubin) 1988) to refine the accuracy of the posterior (Fig. D The likelihood(- ratio) model bss (a: 0) learns to approximate the likelihood p() or the likelihood-ratio
p(=)0) from p(a)
pairs of parameters and simulation outputs (0, æ)). We use the term SNLVI to refer to SNVI with likelihoods, and SNRVI with likelihood-ratios. After a likelihood( model has been trained, the posterior model 4s (0) is trained with variational inference using normalizing flows. Finally, SIR used correct potential inaccuracies in the posterior (0) as we will show below, the SIR step leads to empirical improvements at modest computational overhead. To refine the likelihood( model and the posterior, the procedure can be repeated across several 'rounds'. We opt to sample the parameters 0 from the previous posterior estimate qu (0). but other strategies for active learning (e.g. [Lueckmann et al., 2019b) could be plugged into SNVI. The algorithm is summarized in Alg. We will now describe three variational objectives that can be used with SNVI, the SIR procedure to refine the posterior, and a strategy for dealing with invalid simulation outputs.
3.2 VARIATIONAL OBJECTIVES FOR SBI
Because of the expressiveness of normalizing flows, the true posterior can likely be approximated well by member of the variational family (Papamakarios et al. 2021). Thus, the quality of the
3

---

Published as a conference paper at ICLR 2022
Algorithm 1: SNVI
1 Inputs: prior p(0), observation æo, divergence D, simulations per round N, number rounds R, selection strategy S.
2 Outputs:
Approximate likelihood &y and variational posterior 4s-
3 Initialize: Proposal p(0) ) p(0), simulation dataset X' O
4 for r E [l,.... do
for [l.., .] do
0, Stp Pp)
// sample 8, ~ p(()
simulate 2; ~ ( add (0;, m.) t
11 run the simulator on 0,
end
10
(re- Jtrain fs: 3- = arg min N (a..0.)) log lo (z, // or SNRE loss
(re-)train Zs: x² arg ming (gg (0)]l](() (0)))) with
p(0)xo)) x p( a. (0) lu- (x. ())())
p(0) (0)
end
variational approximation strongly linked the ability achieve the best possible approximation through optimization, which in turn depends on the choice of variational objective D. Using the reverse Kullback-Leibler Divergence (rKL) proposed by Wiqqvitt et al. (202I) can give risc to mode-seeking behaviour and ds might not cover all regions of the posterior (Bishop) 2006 Blei al. 2017). a complementary approach, we suggest and evaluate three alternative variational objectives that induce a mavs-coverring behaviour and posit that this strategy will be particularly important in sequential schemes.
1. Forward KL divergence (fKL) In contrast to the reverse KI (rKL), the forward Kullback- Leibler divergence (fKL) is mass-covering (Bishop 2006). Wan et al. (2020) minimize the following upper bound to the evidence, which implicitly minimizes the IKL: Z(Ø) = Ee-qs [w]( (0) log (w (w)) with (0) = p( Do. (0). This expression is hard to estimate with samples: If qs is different from p( 0) then un(
for most qo (0), thus VaL(Ø) Rs 0, which would prevent learning
(see Appendix Sec. A.3
To overcome this problem, we rewrite the fKL using ceff-normalized importance sampling (Jerfel eet al. 2021]. Let .,...., ON be samples from arbitrary proposal distribution We then minimize the loss:
w O 2j=1 w (())) go (0,)
p Bay 0:)
(o) = (pl(p))
log
where wi( p(=,, ()/=(0). As a self-normalized importance sampling scheme, this estimate is biased, but the bias vanishes at rate (1/NN) (Hesterberg. (2003). our experiments, we use т = Ja which provides good proposal when ds close to p (Chatterjee & Diaconis 2018). Even though go will differ from initially, sufficient gradient information is available to drive towards p, as demonstrate in Appendix Sec. A.3
2. Importance weighted ELBO The importance weighted ELBO (IW-ELBO) introduced by Burda al. (2016) uses the importance-weighter gradient of the evidence lower bound (ELBO). It minimizes the KL divergence between the self-normalized importance sampling distribution of qa and the posterior and thus provides good proposal for sampling importance resampling (Cremer
et al. 2017 Domke & Sheldon, 2018: Ranganath et al. 2014). It can be formulated as
p( 0k)
(K) C/W (0) Eo...... Ee.....
Gr log
(Os)
To avoid a low SNR the gradient estimator (Rainforth et al. 2018), we use the 'Sticking the Landing' (STL) estimator introduced by Roeder al. (217)).
4

---

Published as a conference paper at ICLR 2022
3. Rényi a-divergences Rényi a-divergences are divergence family with hyperparameter which allows to tune the mass-covering (or mode-seeking) behaviour of the algorithm. For a + 1, the divergence approaches the rKL. For < 1, the divergence becomes more mass-covering, for > more mode-seeking. We use a = 0.1 in our experiments. A Rényi variational bound was established by Li & Turner (2016) and is given by
{ (plec. 20(0)
01
))
(o( = 1 log Eorgs
For 0, L. is a single sample Monte Carlo estimate of the IW-ELBO (when using K samples to estimate the expectation al. 2018| Li & Turner 2016). Just as for the IW-ELBO, we alleviate this issue by combining the a-divergences with the STL estimator.
(o)) and thus also suffers from a low SNR as a - (Rainforth
3.3 SAMPLING IMPORTANCE RESAMPLING
After the variational posterior has been trained, 9s approximates (x (0)p(0)/2 Z with normaliza- tion constant Z. We propose to improve the quality of posterior samples by applying Sampling Importance Resampling (SIR) (Rubin) 1988). We sample K = 32 samples from 0 ~ (0), com- pute the corresponding importance weights w, = (x, (.)p((,)) 3/9/0////olo// (0.) and resample a single sample from a categorical distribution whose probabilities equal the normalized importance weights (details in Appendix Sec, AA.4). This strategy enriches the variational family with minimal compu- tational cost (Agrawal et al., al.]| 2020p. SIR particularly useful when (0) covers the true posterior and is thus well-suited for the objectives described above (see Appendix Fig 6
3.4 EXCLUDING INVALID DATA
Simulators may produce unreasonable or undefined values (e.g. NaN), as we will also see in the pyloric network model described later. In posterior estimation methods (SNPE), one can simply remove these 'invalid' simulations from the training dataset, and the trained neural density estimator will still approximate the true posterior (Lueckmann et al. (017). However, we show in Appendix when 'invalid' simulations are removed, the network will converge to E (0) z 1/2 p(xx (00pp)(vp/l)))) p(aliddll)) i.e. the learned likelihood funoction will be biased towards parameter regions which often produce 'invalid' simulations. This prohibits any method that estimates the likelihood(-ratio) (i.e. SNVI, SNLE, SNRE) from excluding 'invalid' simulations, and would therefore prohibit their use on models that produce such data.
Sec.[A.6 this is not the case for (ikeiihood(-raatt))meettteee
To overcome this limitation of seelihoood-trggeiinng techniques, propose estimate the bias-term pl (vvalid with an additional feedforward neural network eę (0) se p(valid(0) (details in Appendix Sec. A.6). Once trained, <ç (0) can be used correct for the bias in the likelihood network. Given the (biased) likelihood network & (x: (0) and the correction factor (0), the posterior distribution proportional
P(0) - Ly (z. 0)p (0) & p(zx(0)p(()) x xp p(f)x.).
We sample from this distribution with VI in combination with SIR. Details, proof and extension to SNRVI in Appendix Sec. A.6 The additional network c (0) is only required in models which can produce invalid simulations. This is not the case for the toy models in Sec. 4.2 but it is required in the model in Sec. 4.3 Alg. sshows SNVI without the additional birs-correttion step. Appendix Alg. 3 showss the method with correction.
4 EXPERIMENTS
We demonstrate the accuracy and the computational efficiency of SNVI several examples. First, we apply SNVI to an illustrative example to demonstrate its ability to capture complex posteriors without mode-collapse. Second, compare SNVI to alternative methods tasks. Third, we demonstrate that SNVI can obtain the posterior distribution in models with many parameters by applying it neuroscience model of the pyloric network in the crab Cancer borealis.
several benchmark
5

---

Published as a conference paper at ICLR 2022
A
B
True posterior
SNLE (100 chains)
SNPLA
SNVI+1KL
SNLE
SNPLA
SNVI+IKL
4000
Time (s)
10a
10
10a
Simulations
Figure 2: A Posterior approximations of SNLE, SNPLA, and SNVI+fKL for the two moons bench- mark example. B Runtime of all algorithms.
4.1 ILLUSTRATIVE EXAMPLE: TWO MOONS
We use the 'two moons' simulator (Greenberg et al. 2019) to illustrate the abilitt of SNVI to capture complex posterior distributions. The two moons simulator has two parameters with a uniform prior and generates a posterior that has both local and global structure. Fig. (2A. shows the ground truth posterior distribution as well as approximations learned by several methods using 10 simulations.
SNLE with MCMC (in the form of Slice Sampling with axis-aligned updates (Neal 2003 can
recover the bimodality when running 100 chains in parallel (Lueckmann et al. (022]) (not shown: individual chains typically only explore single mode). SNPLA, which based the mode- seeking rKL (and could thus also be considered as SNVI+rKL, see Appendix Sec. A.7 captures only a single mode. In contrast, SNLVI (using the fKL and SIR, denoted SNVI+IKL) recovers both the local and the global structure of the posterior accurately. In terms of runtime, SNPLA and SNVI+fKL are up twenty times faster than 100 chain MCMC in our implementation (Fig. 23), and two to four orders of magnitude faster than single chain MCMC (single chain not shown, the relative speed-up for multi-chain MCMC due to vectorization).
4.2 RESULTS ON BENCHMARK PROBLEMS
We compare the accuracy and computational cost of SNVI to that of previous methods, using SBI benchmark tasks (Lueckmann et al. 2021):
Bernoulli GLM: Generalized linear model with Bernoulli observations. Inference is performed on 10-dimensional sufficient summary statistics of the originally 100 dimensional raw data. The resulting posterior is 10-dimensional, unimodal, and concave.
Lotka Volterra: A traditional model in ecology (Wangersky 1978). which describes a predator- prey interaction between species, illustrating a task with complex likelihood and unimodal posterior.
Two moons: Same as described in the previous section.
SL.PP A task introduced by Paaaaaaaarios et al. (2019) with simple likelihood and complex posterior. The prior is uniform, the likelihood has Gaussian noise but is nonlinearly related to the parameters, resulting
posterior with four symmetrical modes.
For each task, we perform inference for ten different runs, each with a different observation. As performance metric, used classifier 2-sample tests (C2ST) (best 0.5, worst is 1.0) (Friedman 2004 Lopez-Paz & Oquab) 2017). For each method, we perform inference given total of 10 10t and 10" simulations. evenly distributed across ten rounds of simulation and training. Details on the hyperparameters are provided in Appendix Sec. A.8 details on results in Appendix Fig. 9 comparisons to the forward KI. without self-normalized weights as well as to the IW-ELBO and a-divergences without STL in Appendix Fig. II
We show results for two reference methods, SNLE with MCMC sampling, and SNPLA, and com- pare them to three variants of SNLVI using the forward KL (SNVI+IKL), the importance-weighted ELBO (SNVI+IW) as well as an slpaa-diergennce (SNVI+a). We find that all three SNVI-variant achieve performance comparable form SNPLA on the two tasks with multi modal posteriors (Two moons and SLCP). We find that omitting the SiR-adjustment (dotted lines) leads small but consistent degradation in inference
MCMC across all four tasks (Fig. B A-D, left), and outper-

---

Published as a conference paper at ICLR 2022
A
Bernoulli GLM
1.0
SNLE
SNPLA
SNVI+fKL
SNVI+IW
SNVI+a
3200
C2ST
Time [8]
0.5
B
1.0
Lotka volterra
3200
C2ST
Time [s]
0.5
C
Two moons
1.0
2000
C2ST
Time [s]
0.5
D 1.0
SLCP
3200
C2ST
Time [s]
0.5
10a
10*
105
10a
10* Simulations
10a
10a
10
10a
10a
10a
10a
10a
10 Simulations
10a
Simulations
Simulations
Simulations
Figure 3: C2ST benchmark results for SNVI with tikelihood-estimation (SNLVT) for four mod- els, Bernoulli GLM (A), Lotka volterra (B). Two moons (C) and SLCP (D). Each point represents the average metric value for ten different observations, as well as the confidence intervals. Bars on the right indicate the average runtime. Two reference methods: SNLE with MCMC sampling, and SNPLA (which uses rKL), as well as three variants of SNVI, with forward KL (SNVI+fKL), importance-weighted ELBO (SNVI+IW) and a-divergence (SNVI+a). Dotted lines: Performance when not using SIR.
performance for all SNVI-variants, but not for SNPLA with the rKL: When using the rKL, the approximate posterior de generally narrower than the posterior and thus ill-suited for SIR (Ap- pendix Fig. 6). Qualitatively similar results were found when using likelihood-rann approaches with the same hyperparameters, see Appendix Fig. 10
In terms of runtime, all three variants of SNLVI are substantially faster than SNLE every task (bars in Fig. 3 on the right), in some cases by more than an order of magnitude. When using likelihood-ratio estimation, MCMC with 100 chains can be as fast SNRVI on tasks with few parameters (Appendix Fig- TO). On tasks with many parameters, however, SNRVI significantly faster than SNRE (see e.g. Bernoulli GLM with 10 parameters).
4.3 INFERENCE IN NEUROSCIENCE MODEL OF THE PYLORIC NETWORK
Finally, applied SNVI to simulator of the pyloric network in the stomatogastric ganglion (STG) of the crab Cancer Borealis, a well-characterized circuit producing rhythmic activity. The model consists of three model neurons (each with eight membrane conductances) with seven synapses (31 parameters in total) and produces voltage traces that can be characterized with 15 established sum- mary statistics (Prinz
2003 2004). In this model, disparate parameter sets can produce similar activity, leading to posterior distribution with broad marginals but narrow conditionals (Prinz et al.) 2004 Gonçalves et al. 2020). Previous work has used millions of simulations from prior samples and performed amortized inference with NPE (18 million simulations in Gonçalves et al. (2020), 9 million in Deistler et al. (202I)). Sequential neural posterior estimation (SNPE) struggles on this problem due to leakage, whereas SNLE and SNRE with MCMC are inefficient (Durkan et al.) 2020). Here, we apply SNVI to identify the posterior distribution given an extracellular recording of the stomatogastric motor neuron (Fig. AА) (Haddad & Marder [021 2018). We demonstrate that SNVI can perform
inference and obtains the posterior distribution with only 350,000 simulations 25 times fewer than previous methods!
7

---

Published as a conference paper at ICLR 2022
A
B
C
Empricial observation
AB/PO
AB/PD
AB/PD CaS
15
0
15
AB/PD CaT
AB/PD CaT
10
10
10 -
10
AB -P
AB -LP
-2
10 10
PD -PP
D
F
1% 'valid'
AB/PD
Posterior
94% 'valid'
L
Median ||x xolla
SNVI+FKL
SNLE
Time [h]
Figure 4: (A) Empirical observation, arrows indicate some of the summary statistics. Scale bar is one second. (B) Comerplot showing subset of the marginal and pairwise marginal distributions of the 31-dimensional posterior (full posterior in Appendix Fig. 12p. Red dot: MAP. Black dot: Posterior mean. (C) Conditional distributions y ( (r. (...)) Green dot shows the sample on which we condition. (D) Simulated traces from the posterior mean and MAP. Scale bar is one second. (E) Simulated traces of three posterior samples. (F) Posterior predictive and prior predictive median (z-scored) distances from the observation. (G) Time required to obtain 10k samples: SNVI takes 11 minutes and SNLE with 100-chain MCMC 808 minutes, i.e. over 13 hours.
We ran SNVI with a likelihoddesstmaate with the fKL. divergence (SNVI+fKL) and SIR. Since the simulator produces many invalid summary statistics (e.g. gaps between bursts cannot be defined there are no bursts)
employed the strategy described Sec. 3.4 Because only 1% of the simulations from prior samples are valid (Fig. FF, we used 50,000 simulations in the first round and continued for 30 rounds with 10,000 simulations each.
The posterior is complex and reveals strong correlations and nonlinear relationships between param- eters (Fig. 4 showing out of 31 dimensions, full posterior in Appendix Fig. 12). The conditional distributions p(O.. 0zi.)) given posterior sample (Fig. 4C) are narrow, demonstrating that pa- rameters have to be finely tuned to generate the summary statistics of the experimentally measured activity. We used posterior predictive checks to inspect the quality of the posterior. When simulat- ing data from the posterior mean and posterior mode (MAP), we find that both of them match the statisties of the experimental activity (Fig. 4D). Similarly, samples from the posterior distribution closely match statistics of the experimental activity (Fig. E). Out of 10,000 posterior samples, 9366 (=94%) generated activity with well-defined summary statistics (compared 1% of prior samples). For the samples which generate well-defined summary statistics, the (z-scored) median distance be- tween the observed data N. and generated activity is smaller for posterior samples than for prior samples (Fig. F). We emphasize that an application of SNLE with MCMC would be estimated to take an additional 400 hours, due to rounds of slow MCMC sampling (Fig. 4G) that would be required- instead of 27 hours for SNVI. Likewise, when running SNPE-C on this example, only one out of million samples was within the prior bounds after the second round, requiring computation- ally expensive rejection sampling (Greenberg al. 2019 Durkan al. 2020). Finally, note that the additional neural network ce) (required to correct for the effect of invalid simulations) can be learned robustly and with low computational cost (see Appendix Fig. 13 for runtime).
8

---

Published as a conference paper at ICLR 2022
These results show that SNVI makes it possible to overcome the limitations of previous methods and allows sequential neural simulation-base inference methods to effectively and robustly scale to challenging inference problems of scientific interest. While it is difficult to rigorously evaluate the accuracy of the obtained posterior distribution due to a lack of ground truth, we observed that almost all posterior predictives have well-defined summary statistics (94% 80% in Gonçalves et al. (2020)) and that the posterior predictives closely match Co.
DISCUSSION
We introduced Sequential Neural Variational Inference (SNVI), efficient, flexible, and robust ap- proach to perform Bayesian inference in models with an intractable likelihood. We achieve this by combining likelihooddessimation (or likelihood-ratio estimation) with variational inference, further improved by using SIR for refining posteriors. We demonstrate that SNVI reduces the computational cost of inference while maintaining accuracy. We applied our approach to neuroscience model of the pyloric network with 31 parameters and showed that it is 25 times more efficient than previous methods. Our results demonstrate that SNVI is scalable and robust method for simulation- bbased inference, opening up new possibilitiess for Bayesian inference in models with intractable likeli- hoods.
We selected three variational objectives for SNVI which induce mass-covering behaviour and are, therefore, well suited as a proposal for sampling from complex posterior distributions. We empiri- cally evaluated all of these methods in terms of runtime and accuracy on four benchmark tasks. We found that, while their performance differed when using the raw VI output, they all showed simi- lar performance after an additional, computationally cheap, sampling importance resampling (SIR) step. After the SIR step, all methods had similar accuracy as MCMC, and all methods outperformed mode-seeking variational objective (reverse KL) which used in previously proposed method Wigyist et al. (202T). Our results suggest that mass-covering VI objectives (regardless of their exact implementation) provide a means to perform fast and accurate inference in models with intractable likelihood, without loss of accuracy compared to MCMC. In Appendix Sec. A.2 we provide tech- nical recommendations for choosing a variational objective for specific problems.
A common approach in sequential methods is to use the current posterior estimate as the pro- posal distribution for the next round, but more elaborate active-learning strategies for choosing new simulations are possible (Papamakarios & Murray 2016, Papamakarios et al., 2019 Lueckmann al., (019a). SNVI can flexibly be combined with any active learning scheme, and unlike neural likelihood(-ratio) methods, does not require expensive MCMC sampling for updating posterior es- timates. While this comes at the cost of having to train two neural networks (a likelihood and a posterior-model) the cost of training these neural networks is often negligible compared to the cost of simulations. Another method that trains both a likelihood- and a posterior network is Posterior-Aided Regularization (Kim et al. 2021), which regularizes the likelihood-estimate with simultaneously trained posterior-estimate. This improves the modelling of multimodal posteriors, but the method still requires MCMC and thus scales poorly with the number of samples and dimen- sions. Likelihood-free variational inference (Tran et al. 2017) avoids learning a likelihood model by learning an implicit posterior distribution, but requires an adversarial training objective which can be difficult to optimize and requires extensive hyperparameter tuning (Huszár 2017). Ong et al. (2018) is another method that performs variational inference with a synthetic likelihood, but their approach requires that the summary statistics are approximately Gaussian in order to obtain unbiased estimates of the log-likelihood
Overall, SNVI combines the desirable properties of current methods: It can be combined with any active learning scheme, it can flexibly combine information from multiple datapoints, it returns a posterior distribution that can be sampled quickly, and it can robustly deal with missing data. SNVI speeds up inference relative to MCMC- based methods, sometimes by orders of magnitude, and can perform inference in large models with many parameters. SNVI therefore has potential provide a new 'go-to' approach for simulation-basee inference, and to open up new application domains for simulation based Bayesian inference.

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We used the configuration manager hydra to track the configuration and seeds of each run Yadan 2019). The results shown in this paper can be reproduced with the git repository https: //github. com/ macke aateelabssaaee snvi_ aabb/snv_reppp repo The algorithms developed in this work are also available in the shi toolbox (Tejero- (Tejero-Canters Cantero et al., (020). All simulations and runs were performed on teghpperformanc computer. For each run, we used 16 CPU cores (Intel family 6, model 61)
and 8GB RAM.
7 ACKNOWLEDGEMENTS
We thank Jan-Matthis Lueckmann for insightful comments on the manuscript. This work was funded by the German Research Foundation (DFG; Germany's Excellence Strategy MLCoE EXC num- ber 2064/1 PN 390727645) and the German Federal Ministry of Education and Research (BMBF; Tübingen AI Center, FKZ: 01IS111110 111S18)39A).
ETHICS STATEMENT
We used data recorded from animal experiments in the crab Cancer borealis. The data we used were recorded for different, independent study and have recently been made publicly available (Haddad) & Marder 2018; 202I). While simulation based inference has the potential to greatly accelerate scientific discovery across broad range of disciplines, could also imagine undesired use-cases of SBI.
REFERENCES
LF Abbott and Eve Marder. Modeling small networks. Methods in Neuronal Modeling, pp. 361-410, 1998.
Abhinav Agrawal, Daniel Sheldon, and Justin Domke. Advances in black-box vi: Normalizing flows, importance weighting, and optimization. arXiv preprint arXiv:2006. 10343, 2020.
Mark A Beaumont, Wenyang Zhang, and David Balding. Approximate bayesian computation in population genetics. Genetics, 162(():2025 2035, 2002.
Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman. Pyro: Deep universal probabilistic programming. J. Mach. Learn. Res., 10:28:1-28t: 2019.
Christopher M Bishop. Pattern recognition. Machine learning, 128(9), 2006.
David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisti- cians. Journal of the American Statistical Association, 112(518):859-8777 Apr 2017.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. Interna-
tional Conference on Learning Representations, 2016.
Sourav Chatterjee and Persi Diaconis. The sample size required in importance sampling. The Annals of Applied Probability, 28(2): (092):0099-00) 1999-9991115 2018.
Yanzhi Chen, Dinghuai Zhang, Michael Gutmann, Aaron Courville, and Zhanxing Zhu. Neural approximate sufficient statistics for implicit models. International Conference on Learning Rep- resentations, 2021.
Kyle Cranmer, Juan Pavez, and Gilles Louppe. Approximating likelihood ratios with calibrated discriminative classifiers. arXiv preprint 10Xiv:1506.02169 2015.
Chris Cremer, Quaid Morris, and David Duvenaud. Reinterpreting aeportance-weighte autoen- coders, 2017.
10

---

Published as a conference paper at ICLR 2022
Michael Deistler, Jakob H. Macke, and Pedro J. Gonçalves. Disparate energy consumption despite similar network activity. bioRxiv, 2021.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. Inter-
national Conference in Learning Representations 2017, 2017.
Justin Domke and Daniel Sheldon. Importance weighting and variational inference. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 1475-4484, 2018.
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows. Ad- vances in Neural Information Processing Systems, 32:7511-7522, 2019a.
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows, In H. Wallach, H. Larochelle, Beygelzimer, d'Alché-Buc, E. Fox, and Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019b.
Conor Durkan, lain Murray, and George Papamakarios. On contrastive learning for likelihood free inference. In International Conference on Machine Learning, pp. 2771 7781. PMLR, 2020.
Jerome Friedman. On multivariate goodness-of-fh and two-sample testing. Technical report, Cite- seer, 2004.
Pedro Gonçalves, Jan-Matthis Lueckmann, Michael Deistler, Marcel Nonnenmacher, Kaan Öcal, Giacomo Bassetto, Chaitanya Chintaluri, William F Podlaski, Sara Haddad, Tim Vogels, al. Training deep neural density estimators to identify mechanistic models of neural dynamics. Elife, 9:e56261, 2020.
David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-free inference. In International Conference Machine Learning, pp. 2404-2414, 2019.
Michael U Gutmann, Ritabrata Dutta, Samuel Kaski, and Jukka Corander. Likelihood free inference via classification. Statistics and Computing, 28(2):411-44) 1-425, 2018.
Sara A. Haddad and Eve Marder. Circuit robustness temperature perturbation is altered by romodulators. Neuron, 100(3):609-623.c. 2018.
Sara Ann Haddad and Eve Marder. Recordings from the C. borealis Stomatogastric Nervous System at different temperatures in the decentralized condition, July 2021. URL htt.::ttss////// //ci: 10. 5281 2e0/////zeooo.oo zenodo. 5139650
Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-free meme with amortized approxi- mate ratio estimators. International Conference on Machine Learning, pp. 4239-4248. PMLR, 2020.
Timothy Classen Hesterberg. Advances in importance sampling. PhD thesis, Citeseer, 2003.
Ferenc Huszár. Variational inference using implicit distributions, 2017.
Ghassen Jerfel, Serena Wang, Clara Fannjiang, Katherine A. Heller, Yian Ma, and Michael I. Jordan. Variational refinement for importance sampling using the forward kullback-leibler divergence, 2021.
Dongjun Kim, Kyungwoo Song, Seungjae Shin, Wanmo Kang, and I1-Chul Moon. Posterior-aided regularization for likelihood-free inference. arXiv preprint arXiv:2102.077711 2021.
Diederik Kingma and Max Welling. Auto-encoding variational bayes, 2014.
Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Im- proved variational inference with inverse autoregressive flow. Advances in neural information processing systems, 29:4743 2016.
and RE Turner. Rényi divergence variational inference. Advances in Neural Information Processing Systems 29 (NIPS 2016), pp. 1073-1081, 2016.

---

Published as a conference paper at ICLR 2022
David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. In International Con- ference on Learning Representations, 2017.
Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Ócal, Marcel Nonnenmacher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. In Advances in Neural Information Processing Systems, pp. 1289-1299, 2017.
Jan-Matthis Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H. Macke. Likelihood-free inference with emulator networks. Francisco Ruiz, Cheng Zhang, Dawen Liang, and Thang Bui (eds.), Proceedings of The Ist Symposium on Advances in Approximate Bayesian Inference, volume 96 of Proceedings of Machine Learning Reseurch, pp- 32-53, 2019a.
Jan-Matthis Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H Macke. Likelihood- free inference with emulator networks. In Symposium on Advances in Approximate Bayesian Inference, pp. 32-53. PMLR, 2019b.
Jan-Matthis Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Bench- marking simulation- based inference. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceed- ings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 343-351. PMLR, 13-15 Apr 2021.
Benjamin Kurt Miller, Alex Cole, Patrick Forré, Gilles Louppe, and Christoph Weniger. Truncated marginal neural ratio estimation. Advances in Neural Information Processing Systems, 2022.
Shakir Mohamed and Balaji Lakshminarayanan. Learning in implicit generative models. arXiv preprint arXiv:1610.03483 2016
Radford M Neal. Slice sampling. The annals of statistics, 31(3):705-767, 2003.
Victor MH Ong, David Nott, Minh-Ngoc Tran, Scott A Sisson, and Christopher C Drovandi. Variational bayes with synthetic likelihood. Statistics and Computing, 28(4):971-988, 2018.
George Papamakarios and lain Murray. Fast e-free inference of simulation models with bayesian conditional density estimation. In Advances in Neural Information Processing Systems, pp. 1028- 1036, 2016
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density es- timation. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 2335-2344 2017.
George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 837-848, 2019.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lak- shminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57): 2021.
Astrid A Prinz, Cyrus P Billimoria, and Eve Marder. Alternative hand-tuning conductance- bbased models: construction and analysis of databases of model neurons. Journal of neurophysiology, 2003.
Astrid A Prinz, Dirk Bucher, and Eve Marder. Similar network activity from disparate circuit pa- rameters. Nature neuroscience, 7(12): 1345-1322, 2004.
Tom Rainforth, Adam Kosiorek, Tuan Anh Le, Chris Maddison, Maximilian Igl, Frank Wood, and Yee Whye Teh. Tighter variational bounds are not necessarily better. In International Conference on Machine Learning, pp. 4277-4285. PMLR, 2018.
Rajesh Ranganath, Scan Gerrish, and David Blei. Black box variational inference. In Artificial Intelligence and Statistics, pp. 814-822. PMLR, 2014.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. Interna- fional conference machine learning, pp. 1530-1538. PMLR, 2015.
12

---

Published as a conference paper at ICLR 2022
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International conference on machine learning, pp. 1278-1286 PMLR, 2014.
Geoffrey Roeder, Yuhuai Wu, and David K Duvenaud. Sticking the landing: Simple, lower-variance gradient estimators for variational inference. Advances in Neural Information Processing Systems, 30:6925-6934, 2017.
Donald B Rubin. Using the sir algorithm to simulate posterior distributions. Bayesian statistics, 3: 395-402, 1988.
Alvaro Mejero-Canters Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pe- dro J. Gonçalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulation- inference. Journal of Open Source Software, 5(52):2505, 2020. doi: 10.21 10.21105/joss.25555 025005.
Owen Thomas, Ritabrata Dutta, Jukka Corander, Samuel Kaski, Michael U Gutmann, al. Likelihood-free inference by ratio estimation. Bayesian Analysis, 2021.
Dustin Tran, Rajesh Ranganath, and David M Blei. Hierarchical implicit models and likelihood-free variational inference. In Proceedings of the 31 International Conference on Neural Information Processing Systems, pp. 5529-5539 2017.
George Tucker, Dieterich Lawson, Shixiang Gu, and Chris Maddison. Doubly reparameterized gradient estimators for monte carlo objectives. In International Conference on Learning Repre- sentations, 2018.
Neng Wan, Dapeng Li, and Naira Hovakimyan. f-divergence variational inference. Advances in Neural Information Processing Systems, 33, 2020.
Peter Wangersky. Lotka-volterra population models. Annual Review of Ecology and Systematics, 9(1::199-218, 1978.
Samuel Wiqvist, Jes Frellsen, and Umberto Picchini. Sequential neural posterior and likelihood approximation, 2021.
Simon N. Wood. Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466 102-0):100--114, Aug 2010.
Omry Yadan. Hydra a framework for elegantly configuring complex applications. Github, 2019.
URL httph::/tcee thub. https://tttt.ccccceecessssssrrrr
13

---

Published as a conference paper at ICLR 2022
A
APPENDIX
A.1 GRADIENTS OF DIVERGENCES
For completeness, provide the gradients of the divergences introduced in Sec. 3.2
Forward Kullback-Leibler divergence
The gradient estimate given by
((,))) j= w(O))
V (Ø)) = -Eo-p V log (1o(
Vo log go (O:)
Importance-weeigtel ELBO
A gradient estimator is given by
VoL ( (6) Ox
-x......00000
k
(0.) log
p( 0:) (0((,)
w(0,)
w(:))
Σ w(O))
where w0) = p( 0) (0).
Rényi a-divergence A biased gradient estimator using the reparamcterization trick can be written as:
p ao 0) (0)
w())1aa
V LoLa(1) = Earas Wa (0)V log
Va Ea (w:(0) 1-
where w plz.. 0)/40 (0). This gradient estimator is biased towards the rKL but the bias
vanishes as more samples are used for the Monte Carlo approximation (Li & Turner 2016
A.2 CHOICE OF DIVERGENCE
In Fig. 4.2
we demonstrated that all mass-covering objectives perform similarly in terms of accuracy and runtime on the problems we considered. We here give technical recommendations for choosing variational objective:
(i) Closed-form posterior: The variational posterior provides a closed-form approximation to the posterior, but this is no longer the case when SIR is used. While, in results, all three approaches performed similarly with SIR, they can differ in their performance without it, and the forward KI. and the a-divergence provided better approximations than the IW- ELBO. Thus, if one seeks a posterior density that can be evaluated in closed-form, our results suggest to use the forward KL or the a-divergence.
(ii) Dimensionality of the parameter space: We use an autoregressive normalizing flow as vari- ational family. These flows are very expressive, yet the computation time of their forward or backward passes scale with the dimensionality of (Papamakarios et al.) 2017 Kingma jtt al. 2016 Durkan et al., 2019a). The IW-ELBO and the a-divergences only require forward passes, whereas the forward KL requires forward and backward passes, thus mak- ing the forward KL. expensive for high-dimensional parameter spaces. The STL estimator used in the IW-ELBO and the a-divergences also requires forward and backward passes. We found that the STL estimator improves performance of the IW-ELBO only weakly (Fig. IT). Thus, in the IW-ELBO without the STL can give high accuracy at low computational cost. Another way to reduce computational cost is use alternative architectures for the normalizing flow. c.g. coupling layers (Durkan et al. 2019a; Papamakarios et al. 2021).
where computational cost is critical, our results suggest that using
(iii)
Trading-off the mass-covering property with computational cost: can trade-off the extent to which the divergence is mass-covering by choosing the value of a (low a is more mass-covering). As shown in Fig. high values of a benefit less from using the STL estimator. Thus, in cases where cover-ng behavior of the algorithm is less crucial, the STL estimator can be waived, leading to lower computational cost because the normalizing flow requires only forward passes (see point (ii)).
For a-divergences, one
14

---

Published as a conference paper at ICLR 2022
A
Gradient estimation
Signal to Noise Ratio
B
Qp-s(0)
Q2-12(9)
pld)x
15
SWRRD, SN (.,-lll))))
1.25
10
1.00
0.75
-5
0.50
0.25
of 4u
of 4u
0.00
10
15
N 10 100 N
1000
True Gradient
IVB
14 of au
Figure 5: A Left: Gradient estimation on the Gaussian example. For values of around μ 4/5, all estimators provide good gradients. As μl is farther from μd, the forward variational bound (fVB) (grey) vanishes, whereas the self-normalized fVB approaches a constant. Right: SNR for the fVB and the self-normalized fVB. B Theoretical and empirical densities pR(r) for μl = 6 and μ = 12.
A.3 OVERCOMING VANISHING GRADIENTS IN THE FORWARD KL ESTIMATOR
We use estimator of the forward Diillbacc-Leibber divergence (fKL) that is based self- normalized importance sampling. In this section, we demonstrate that this estimator moves the variational distribution (o(0) towards the target density p(z,. 0) even (0) and p(z.. 0) differ strongly.
For this analysis, we consider a Gaussian toy example with prior p(0) (0 0, 4), likelihood p(x[0)- (xx; 0, 1). and observation 2, = 1. The posterior distribution can be computed in closed-form as p(0) (x.) (0; 4/5, 4/5). We aim learn the posterior distribution using varia- tional inference with the variational family g, (0) N(u, 4/5) (note that is the only parameter). The best approximation within this family is f = 4/5.
We use this toy example to compare the gradient and the signal-to-noise ratio (SNR) of the self- normalized fKL estimator to the fKL estimator introduced by Wan et all (2020). Fig. 5A (left) shows the gradient of the loss for different values of μ. When & μ 4/5, the fKL (gray) without eeff-normalizaiion closely matches the true gradient (red). However, as μ is further from μ4" the fKL. first points in the wrong direction and then vanishes, which prevents learning. The normalzoma fKL (blue, orange, green) closely matches the gradient around μ z 14" = =455 and does not vanish for y that are far from μ². The gradient stronger if more samples N used to approximate the fKI.. Similarly, the SNR(V&L(Ø)) [E[V (()))/ ((o))////////// (Var(V does not vanish for ys that are far from f for the self-normalized fKL..
To understand this behaviour of the self-normalized fKL, computed an approximation to the gradient V VyLCer (n) in this toy example. The fKL loss is given as:
(()) N == w()
N
w(+)) E w(O))
V (IL = -Eo
V log (0)
~
v, log qu (0,)
with weights w(O.) = p( .)) In the case where qo differs strongly from p(:., 0), the weights (0)
are often degenerate, i.e. the strongest weight is much larger than all others. In the worst case, ((,) En (
w(0.) for some i and the gradient estimator reduces
V VaCra( = -V, p log qu (argmax w()) V log (r)
01....0
The gradient of μ is thus determined by r arg max w (0), which itself can be considered draw from random variable R. We will now derive the probability density pr(r) of R.
15

---

Published as a conference paper at ICLR 2022
Algorithm 2: SIR
1 Input: K the number of importance samples, proposal qø. joint density
p( 0) (ar. 0)p(()
2 for i [1 K] do
q0 (
W; = pl
(o()
5 end
Each W; = wi > k= Wk
7 J Categorical (1)
return 0
If > μ², then (0) is monotonically decreasing in 0 because w())
Nii .4/5)
N A/,4, x exp(5/4-
Op= n)). The cumulative distribution function FR (R S r) can then be written
FR(R r) P(arg max w0) S
1 r)
o.....o..
N
P(min(O),...... 0m) > r) =
II
P(O. r)
=1 (1 Pap (r))
The derivation analogous for the case μl μu" Because v,,CiKL (p) = V log 9u (r) for pr. this allows to compute the distribution of the gradient of (under the assumption that weights are degenerate).
Thus, has the density (R( = 2r
F(R S =) N(1 (r)) (
We empirically validate this result on the Gaussian toy example. Fig. 5 (left) shows the true poste- rior distribution (black), the variational density g, (0) for μ = 6 and 10 and the corresponding pR((r) for N = 1000. The theoretically computed density pn(r) (dashed lines) matches the empir- ically observed distribution of arg maxg.=1... =I.... w(O.). For almost every value of PR(r), the gradient V, log (r) is negative, thus driving (TLCKL(H)) into the correct direction. For larger N, the distribution (r) shifts towards the true posterior distribution and thus also the gradient signal increases.
Notably, for that even further from ,r, V (r)) remains relatively constant (Fig. right). This explains why the gradient V (FL becomes constant in Fig. 5A (left).
A.4 IMPROVEMENT THROUGH SIR
We use Sampling Importance Resampling (SIR) to refine samples obtained from the variational posterior. The SIR procedure is detailed in Alg. 2 for drawing a single sample from the posterior. Consistent with AAgrwal et al. (2020), we found that using SIR always helps improve the approx- imation quality even when using complex variational families such as normalizing flows (compare dotted and solid lines in Fig. 3)
We visualize the benefits of SIR in 6on an example which uses a Gaussian proposal distribution (i.e. variational family). SIR enhances the variational family and allows approximate the bimodal target distribution. SIR particularly improves the posterior estimate when the proposal (i.e. the variational posterior) is overdispersed. This provides an explanation for why SIR is particularly useful for the mass-covering divergences used in SNVI, and less so for mode-covering divergences (as used in SNPLA).
A.5 QUALITY OF THE LIKELIHOOD ESTIMATOR
The estimated variational posterior is based the estimated likelihood (0) is thus important to accurately learn the likelihood from simulations. To be able to learn skewed or bimodal likeli- hoods, conditional autoregressive normalizing flow for f(x ()) (Papamakarios et al. 2017 Kingma et al., 2016 Durkan et al. 2019a). Fig. demonstrates that these flows can lear complex likelihoods (Papamakarios al. 2019).
16

---

Published as a conference paper at ICLR 2022
A
SIR overdispersed 9
SIR underdispersed q
B
Two moons estimation with fKL
-targen
Samples
Samples
53t adjusted
S3tt adjusted
12.5
p10
10.0
2.6
10
15 20
90
35
-10 10 -10
10
-10 -5
4(9)
Figure 6: Visualization of SIR. A Toy example with a Gaussian proposal density. Left: Two toy ex- amples with Gaussian (top) or bimodal (bottom) target density. SIR (with K 32) can extend the Gaussian density and refine the approximation if the proposal is overdispersed (middle), but helps less when it is too narrow (right). SIR improvements on two moons example. We plot the joint density as learned by the Rkclihooddmooel p(z,, 0) L (()))) against the variational poste- rior ds (blue, obtained with the fKL.) as well as the STR-corrected density with K (orange) and K = 32 (green). Despite using an expressive normalizing flow qp. SIR improves the accuracy.
p(x1))= 0.5(N7 0, +) ATe, 1))
p(x, -)= p(x))))) 2)
8: = 0
4-
2
2
0
2-
-2
0
Esimate t,0xl Ø)
p(x,9 8 ) (010
4
0-
-2
-4
Figure 7: A neural spline flow (NSF) estimating bimodal likelihood with 10 simulations with prior (0, 2). Top: Ground truth. Bottom: Likelihood -pppriimattap with NSF The learned likelihood closely matches the true likelihood.
A.6 PROOFS FOR EXCLUDING INVALID DATA
Many simulators can produce unreasonable or undefined values when fed with parameters sampled from the prior (Lueckmann et al.) 2017). These invalid simulations are not useful for accurately learning the likelihood, and we would like to ignore them. To do so, we developed loss-reweighing strategy.
17

---

Published as a conference paper at ICLR 2022
Algorithm 3: SNVI with calibration kemel
1 Inputs: prior p(0), observation Zo, divergence D, simulations per round N, number
rounds R, selection strategy S and calibration kernel K.
2 Outputs:
Approximate likelihood &y variational posterior ga and calibration network cç-
3 Initialize: Proposal p(0) - p(0), simulation dataset X' U calibration dataset C' = 0
4 for r E [I,.... do
for [1, N] do
0, = simulate 2; ~ p(= 0 add (0,, (( Ro)) t
// sample 8, p(0) // run the simulator on 0,
if ((r, F.)) > then
10
add (0., x.) X
11
end
12
end
13
// or (re-)train Es: 8- = arg ming - - (e,, ((x,, ) log (x, 0.)
SNRE
14
(re-)train eg aig N E K(, f(ee (0:), (a),
1/ MSE
or cross entsrpyy for binary calibration kernel (re-)train 9d; 6 = arg ming ( (0)]l((((zz)) with
15
p(0(zo)) pla. ()p(O) ~ lu (0. (0)p(0)
16
p(0) (0)
17 end
We formulate the exclusion of invalid simulations by the means of a calibration kernel K(x,, X..)
(originally introduced for neural posterior estimation [ueckmann al. (2017). This calibra- tion kernel can be any function, and can thus be used beyond excluding invalid data. The case of excluding invalid simulations can be recovered by using a binary calibration kernel:
(0
if invalid
K(x, xo)
{1
(1)
if valid
Alg. 3 shows SNVI with calibration kernel. Notice that, in the case of a binary calibration kernel, the loss for the likelihood te; 4/" = arg min N E( K ((x,, ao) log (a:,(0)) ( is zero
for all invalid simulations (because K(x, x.) = 0). Thus, since these simulations do not contribute to the loss, exclude these simulations from the dataset that is used train the likelihood(-ratio) model.
Below, we provide proofs of convergence for Alg. 3 Theorem and Lemma are relevant SNLVI, aare TheoremZland Lemma are relevant to SNRVI, and Lemma is relevant to both methods. Theorem 1 and Theorem 2 provide means to use a calibration kernel in the training of the likelihood(- radil such that one can still recover the posterior density. In SNVI, we sample from the (unnormalized) potential function with variational inference. However, one can also use Theorem and Theorem 2 in combination with SNLE and SNRE and draw samples from the potential function with MCMC.
Both SNLVI and SNRVI with calibration kernels rely the estimation of Ex-=p(= 0) [K (x, æo)] (note that this turns into p( () ) for the binary calibration kernel). We estimate this term with a feed-forward regression neural network eç (0) (see Lemma 3}. The network trained on pairs (0, K K(x,, a.))). where 0 and æ are the same pairs as used for training the (eeeiioood(ratt))-oooode For general calibration kernels K (æe, Rx),, we use mquaesquared error loss, whereas the case of invalid data, parameterize ez (0) as logistic regression network and train it with cross-entropy loss (since the calibration kernel K (x, x.) is a binary function: 0 for invalid data, for valid data).
Theorem 1. Let K : X x X - + be a kernel. Let E--- (xx 0) be the maximizer of the objective
L Fg (00.0ppp00.00 K [K(r, v..) log( (2x (x)]
18

---

Published as a conference paper at ICLR 2022
and let cç. (0) be the minimizer of
L 100
Then the potential function is proportional to the posterior density )
P(0) Eyv (0)
Proof. Using Lemma I and Lemma
we get Ev- (x (z=10))p ()))))
P(0)
( (xx (10 [K(( x.)]
(xx, x. (.)p0())(0)
K(x, a. (p0
x 2.)
Theorem 2. Let K X x X IRtt be kernel. Let Ea-- (xr, 0) be the minimizer of the objective L (0,0)p((..z) [K(x, (x. vo)) log(e,> (xx, 0))] + Eg (=))pp()))( [K(x, 2o) log(1 las (x, 0)) and let cc- (0) be the minimizer of
[ = F0 I(e x(xx,,)))
Then the potential function is proportional to the posterior density p(0
P(0) las (a ())(0)cc-
Proof. Using Lemma 2 and Lemma
we get
P(0)
-- (N., (0)cO)c--(00 (0)
B p(a) KK (x, xx)] p(æ, 0)
pm (x(
E Exppla 0) [ (x, xx)] p(x.))
E [xx x.)] p( a (
p(x,)) p(0)
x p(0(xx))
Lemma 1. Let K : X x X + R+ be an arbitrary kernel. Then, the objective
- Eg --0((0,0))) KK (xx, (x) 0))]
maximized and only (a) Z K (r, F.) 2o)p( 0) for all 0 E (())) with malizing constant (0) = K (x, 2o)p(x)))da Ex 0) K (xx x.)).
Proof.
C Eg a0 [K x.)) log(4 (0
// K n. ) log
// ((z,
= f (lt) / K(x,, x )p(x 0) log (z)
Since K(x, ))))(()(0))))))))))))))))))))))) (ex x - DDLL z(0) K(x, ) 0), Ey (x)())) this term is maximized if and only (x) o K(x, <x,x,,)(( xx) 0) for all 0 E support(7(())) with 2
K K(x, (n))pa=(0)da E R0) [K(2r,
19

---

Published as a conference paper at ICLR 2022
Lemma 2. Let K : X x X R+ be arbitrary kemne.. Then, the objective
= ((...)) [K (x, ko) log(Lus (oul (x, 0)) + ((0x)=(())(()) (xx, xo) log(1 (x, ())
Ea-pin) [[ plan 0) Eg-acan 01 K [m,,,)) p(x) for all 0 supporr(p(0))
is minimized and only if Ev (x, 0)
Proof. We begin by rearranging the expectations:
L = Rg 20) (ax, no) log(L (ar, 0)) -o (K(,, logg Ex (æ, 0))
= /ʃ i(0, x: ((x xt) log(t (x, 0) d00dx+
|| p(0))(z)) mo) log(1 (x, (z)
= -Iʃ
700,0 x (.0.))((a ((x, xx)
log(Ey- (xx 0))dddx+
p(0. K( dd0dx
K()) vo) log(1 bu- dd0 J (()) F. d0da
Fg wann (0,))
[log(Eas (xr, 0))] + Eeg (e. [log(1 Px- (ar, 0))
where we introduced
p x: (x, v.)) If p x (xx x d00dx
p(x)) n.)
Joii 0 x:)
Tmarginal (0, ()) IJ (()))
Since binary classification recovers density ratios Cranmee et al. 2015
Mohamed Lakshmi-
narayanan 2016 Gutmann et al. 2018), we get
Joint (0,z)
E (xx 0) murginal ( x)
T (m. d0 10, ((,,
1I (e.. (00) (.)
JJ ((z) d0dr p(x(0)
I] ((0)p(2
p(x))
J p(x) (2, x.) dd) p( 0)
p(x 0) K x x
d0d p(x)
Bamp(a) K a x.,] (
E Ex-p(=
K xx)) p(x)
Lemma 3. The objective
L Eg x (0) x(axx.))²²)
is minimized if and only if (0) = Em xx)) for all 0 e support(p(0))
Proof.
C Ee x~ p 100 (cc ((x, 2).)
5(0 (c(())(e((0) K( xx)) x
/ p(0) ((x)0)(cc((0) (ax, x. dadd
p(0) - <<x 1100
which is minimized if and only c0) Expp(e))) (K (xx (.))) for all O support(o())).
n
20

---

Published as a conference paper at ICLR 2022
SNPLA
1.0
0.9
Original SNPLA
CSST
0.8
Our SNPLA (SNVI+rKL)
0.7
0.6
0.5
100 Number Simulations
10a
10
Figure 8: Comparison between SNPLA implementation of (Wiqvist al. 2021 and SNVI with rKL.
A.7 SNPLA
Fig. and Fig. 3 we compared SNVI to SNPLA (Wiqvist et al. 2021). To ensure comparability
between SNPLA and SNVI, implemented SNPLÀ ourselves and used the same likelihood- and
posterior model for both methods. The main difference between our implementation and the original implementation of SNPLA
1. We do not the proposal Pr (0) ap((0) + (1 (0) for [0, 1J. instead
at 0, i.e. we use the current posterior estimate as proposal.
2. Secondly, we use a Rational Linear Spline Flow (RSF) based on pyro (Bingham et al.,
2019), whereas Wiqvist et al. (2021) uses Masked Autoregressive Flow based on nflows
DDurkan et al. 2019b).
Fig. compares the performance of our SNPLA implementation to the original implementation. Our implementation performs slightly better, likely due to the use of more expressive normalizing flows. We used our implementation for all experiments and nonetheless refer to the method with the
name 'SNPLA'.
A.8 EXPERIMENTS: BENCHMARK
All tasks were taken from an shi benchmark (Lueckmann et al. 2021). For description of the simulators, summary statistics, and prior distributions, we refer the reader to that paper.
We use the SNLE and SNRE as implemented in the sbi package (Tejero-Cantero et al.) 2020) In all experiments, we learn the likelihood with Masked Autoregressive Flow (MAF) with five autoregressive layers each with two hidden layers and 50 hidden units (Tejero-Cantero et al. 2020
Durkan et al. (0019b). For SNRE we use a two block residual network with 50 hidden units. Just as Lueckmann et al. (2021), we implement SNRE with the loss described in Durkan et al. (020))
The implementation of the posterior normalizing flows is based on pyro (Bingham et al. 2019).
pyro caches intermediate values during sampling and thus allow cheap density evaluation obtained samples. We use MAFs for higher dimensional problems and Rational Linear Spline Flows (RSF) for low dimensional but complex problems (SLCP, Two moons). We always use standard Gaussian base distribtuion and five autoregressive layers with a hidden size depending on input dimension ([dim- 10, dim 10] for spline autoregressive nets and [dim- 55+]] for affine autoregressive nets, each with ReLU activations). As the posterior support must match that of the prior, we add a bijective mapping that maps the support to that of the prior. This allows to train the normalizing flows directly on the constrained domain.
We used total sampling budget of N 256 for any VI loss. To estimate the IW-ELBO we use N = 32 to estimate //W <(k=8) (Ø) (Rainforth et al. 2018). Additionally, we use the STL estimator [Roeder et al. (007).. An alternatively would be the doubly reparameterized gradient estimator,
which is unbiased. We choose the STL estimator as it admits larger SNRs at the cost of introducing
some bias (Tucker al. 2018). Because for a
have that L. LIW
(K=1)
this
21

---

Published as a conference paper at ICLR 2022
Bernoulii GLM (SNLE)
Bemoulli GLM (SNPLA)
Berouuli GLM (SNVI+fKL)
Lotka volterra (SNLE)
Lotka volterra (SNPLA)
Lotka volterra (SNVI+fKL)
(SNLE)
(SNPLA)
SLCP (SNVI+fKL)
Figure 9: Samples from the posterior distributions for SNLE with MCMC, SNVI + IKL, SNVI + rKL. First row: results for SLCP. Second row: Lotka-Volterra. Third row: Bernoulli GLM.
estimator also estimate (0) While the estimator also be used for the ELBO, it requires additional computational cost i.e. we additionally need to calculate the inverse transformation, which costly for autoregressive flows. Note that the fKI. estimator also requires the inverse transform, thus we recommend to use normalizing flow with fast forward and inverse passes in problems with many parameters, c.g. normalizing flows based on coupling layers (Dinh et al. 2017 Durkan al., 2019a).
We trained for 10 rounds of simulations. In each round, we initialize the likelihood- and the posterior-mont model as their respective last estimates from the previous round. We train the posterior model for each round for least 100 iterations and most 1000 iterations. We evaluate conver- gence by tracking the decrease within the loss. For this automated benchmark, the convergence criteria are chosen conservative too avoid early stopping. More elaborate convergence criteria may improve runtime.
As metrics, we used classifier 2-sample tests (C2ST), C2ST trains classifier to distinguish posterior samples produced by specific method to ground truth posterior samples. Thus, value of 0.5 means that the distributions are identical, whereas higher values indicate a mismatch between the distributions. As in Lueckmann et al.) (2021). computed the C2ST using 10,000 samples. Each figure shows the average metric value over 10 different observations, as well as the corresponding 95% confidence interval.
22

---

Published as a conference paper at ICLR 2022
A
Bernoulli GLM
1.0
SNRE
SNVI+FKL
SNVI+fKL
SN
SNV++
2800
C2ST
Time [6]
0.5
0
1.0
Lotka volterra
2200
C2ST
Time 15
0.5
C
Two
1.0
1400
C2ST
Time 5
0.5
D 11.0
SLCP
2000
C2ST
Time [6]
0.5
10a
10
105
10a
104
105
10a
10
10a
10a
104
105
10a
104
10a
Simulations
Simulations
Simulations
Simulations
Simulations
Figure 10: C2ST benchmark results for SNVI with ratio estimation (SNRVI) for four models, Bernoulli GLM (A), Lotka Volterra (B), Two moons (C) and SLCP (D). Each point represents the average metric value for ten different observations, as well as the confidence intervals. Bars on the right indicate the average runtime. Two reference methods: SNRE with MCMC sampling and the rKL, as well as three variants of SNVI, with forward KL (SNVI+iKL), importance-weighee weighted ELBO (SNVI+IW) and a-divergence (SNVI+a). Dotted lines: performance when not using SIR.
Fig. 11 shows results for further variational objectives on the two moons (top) and on the SLCP task (bottom). The seff-normalization used for the forward KI. estimator improves the approximation quality (IT left, dark vs light purple). For the IW-ELBO (middle) as well as for the a-divergences (right), the STL estimator improves performance (Rainforth et al.) 2018). The gains from the STL are stronger for a-divergences as for the IW-ELBO (especially when using SIR). The STL particu- larly improves the estimate for low values of alpha (which are more support-covering).
A.9 EXPERIMENTS: INFERENCE IN NEUROSCIENCE MODEL OF THE PYLORIC NETWORK
We used the same simulator as in Gonçalves et al. 2020 Deistler et al. 2021 ) and the 15 summary statistics originally described in Prinz al. (2004) and used in Gonçalves
(2020); Deistler et al. (2021) (notably. Gonçalves et al. (2020); Deistler et al.] (2021) used 3 additional features). Below, we describe the simulator briefly, for full description we refer the reader Prinz et al. Gonçcllvs et al. (2020); Deistler al. (2021).
The model is composed three single-compartment neurons, AB/PD, LP, and PY, where the elec- trically coupled AB and PD neurons are modeled as a single neuron. Each of the model neurons contains 8 currents. addition, the model contains synapses. As in Prinz et al. (2004). these synapses simulated using standard model of synaptic dynamics (Abbott & Marder 1998).
For each set of membrane and synaptic conductances, we numerically simulate the circuit for 10 seconds with step size of 0.025 ms. At each time step, each neuron receives Gaussian noise with mean zero and standard deviation 0.001 mV Vms -0.5
We applied SNVI to infer the posterior over 24 membrane parameters and 7 synaptic parameters, ie. 31 parameters in total. The 7 synaptic parameters are the maximal conductances of all synapses in the circuit, each of which is varied uniformly in logarithmic domain and the membrane parameters
23

---

Published as a conference paper at ICLR 2022
Two moons
SNLVI+fKL
SNLVI+IW
SNLVI+
1.0
0.9
0
0.7
0.6
0.5
10a
10*
10a
10a
10*
10a
10a
10a Simulations
10a
Simulations
Simulations
SLCP
SNLVI+
SNLVI+IW
SNLVI+ a
1.0
0.9
0.8
0.7
0.5
10a
104
10a
10a
104
10s
10a
104
10a
Simulations
Simulations
Simulations
Laa Cna (FVB)
Eww
Ca (a 0.1) La (a 0.1,STL)
C. (a 0.5)
40 SIR
Liw (STL)
La (a 0.5, STL)
Figure 11: Evaluation of further variational objectives for the two moons (top) and the SLCP (bot- tom) task. Left: Variations of the forward KL (with and without self norf-iorma weights). Middle: Variations of the TW-ELBO (with and without STL). Right: Variations of the a-divergence (with and without STI as well as for different values of a.
are the maximal membrane conductances for each neuron. All membrane and synaptic conductances are varied over the same range as in Gonçalves et al. (220): Deistler et al. (202T).
The 15 summary features proposed by Prinz et al. (2004) are salient features of the pyloric rhythm: Cycle period (s), three burst durations (s), two gap durations between bursts, two phase delays, three duty cycles, two phase gaps, and two phases of burst onsets. Note that several of these values are only defined if each neuron produces rhythmic bursting behavior. In particular call any simulations invalid if at least one of the summary features is undefined.
The experimental data is taken from file 845 0220082000 0044 in a publicly available dataset (Haddad & Marder 2021).
For the likelihood-model, we use Neural Spline Flow (NSF) with five autoregressive layers. Each layer has two hidden layers and 50 hidden neurons, as implemented in the shi package (Tejero-) Cantero et al. 2020; Durkan et al. (019b)). The mosterior-model (MAF) with five autoregressive layers each with one hidden layer and 160 hidden units.
Masked autoregressive flow
We train a total of 31 rounds. In the first round we 50000 simulations from which only 492 are valid, and thus used to estimate the likelihood. For all other rounds we each sim- ulated 10000 samples. To account for invalid summary features, we use the calibration ker- nel ((x, x.)) = [( is valid). hence can simply exclude any invalid simulations from train- ing the likelihood-modee Em=p(=jo) [[ is valid)] = P( is v) To estimate this probability we use deep logis- tie regression net with hidden layers each with 50 neurons and ReLU activations. We train this classifier simultancously with the likelihood-model. that is in each round we add new data {(,,, I(O, is J and retrain the classifier using the weighted smary-cross-entroeee weight the loss by the estimated class probabilities to account for class imbalance especially in early rounds. We fix the number of epochs to 200 per round. We use the fKL loss with N = 1024 samples, well SIR.
By Theorem we have to correct the likelihood by multiplication of
loss. We
24

---

Published as a conference paper at ICLR 2022
IN
0.0 500
AB Nia
0.0
7
AB-GaT
0.0
8
AB
0,0 60
AB-A
kd
0.0
AB-leak
LP-CaT
LPCa5
0.0 10
KCa
6.0 125
0.0
PY. Cas
PY
0.0
PY-KCa
150
PY-kd
0.0
0.1
PY-1
0.0
0.0
0.03 10000
PD PY
PD
0.01 1000
0.01 1000 LP
Figure 12: Posterior distribution for the neuroscience model of the pyloric network. In Fig. 43 we show subset. The black point mean estimate using 10" samples. The red point is maximum a-posterior estimate, obtained by gradient ascent.
In total, the procedure took 27 hours, with the runs of the simulator being parallelized across several nodes. Because of this, the runtime also depends greatly availability of computing resources the cluster.
25

---

Published as a conference paper at ICLR 2022
Pyloric 10k simulations (9k invalid, 1k valid)
140
120
Runtime [8]
80
60
40
Classifier
Likelihoed (valid data)
Classifier Likelihood
Likelihood (full sub. data)
Figure 13: Runtime of the classifier (0) in the model of the pyloric network (90% of simulations are invalid). Training the classifier is approximately three times cheaper than training the likelihood- model (compare left bar second left) and thus increases the computational cost only modestly. The Akellhooddmoode is trained only on valid simulations. The combined runtime of classifier and likelihood model (third bar) still far less then time would take to train the likelihood-model on all simulations (right bar. To estimate the runtime of the likelihood-mode on all simulations, substituted invalid simulation outputs (i.c. NaN) with an unreasonably low value and trained on all simulations).
26


Reviews:
[{"review_id": "i3VMtHFBC06", "reviewer": "Reviewer_NxCC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Pros:\n- The paper is well written.\n- Improving the efficiency of sequential SBI methods is important.\n- The paper nicely combines advanced methods and the results are good.\n## Cons:\n- It is yet not clear on which occasion it is a better idea to use a sequential method that will only be used once to do inference than an amortized method. I am sure there are good motivations for this but this is probably something that should motivate.\n- By using SIR it looks like you trust more the likelihood surrogate than the approx. posterior. I do not see any strong motivations for this in the general case where both distributions may be complex.\n- How does the method compare to simply learning an approximate posterior with a flow by minimizing some KL. It goes back to my first point, in which cases building the posterior online becomes more efficient.\n- There are now many SBI methods and I think this would make the reader's life way easier if you (at least briefly) introduce the methods you compare your algorithm to and why you do not compare to the many other methods.\n- Maybe you could give some additional context to the IW-ELBO (being explicit that it reduces the GAP between ELBO and likelihood).\n- It is not clear to me what should a user do from the 3 possible losses to optimize phi.\n- As you say that your method could be useful when we have many observations it would be interesting to show how does something that other methods cannot do. ", "summary_of_the_paper": "This paper presents Sequential Neural Variational Inference (SNVI), a new simulation-based inference (SBI) technique. SNVI sequentially updates a surrogate likelihood model and an approximate posterior model in high posterior density. Finally, the approximate posterior is sampled with importance resampling. Experiments shows this technique is able to learn multimodal posterior distributions and is faster than methods relying on MCMC for generating posterior samples while being almost as accurate. ", "summary_of_the_review": "I recommend weak acceptance as I think the method described in this paper may be interesting for the SBI community. I also find the paper well written and experiments convincing.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "huCgTDjcOwg", "reviewer": "Reviewer_JNHt", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- This work solves a quite practical problem of SNLE and SNRE. \n- Experimental results show convincing results. Accuracy is maintained while reducing the computational cost of inference.\n- Experiments against SNPLA demonstrate that using any of three proposed variational objectives leads to better results. Modes in the posterior approximations are now correctly captured.\n- Posterior predictive checks are used to evaluate the quality of the approximate posteriors. \n- The supplementary materials are thorough and include helpful experimental details as well as further discussions.\n\nWeaknesses:\n- Figure 8 is meant to show results for SNRE and SNRVI, but the labels state SNLE and SNVI. Are the labels wrong?\n- Novelty is limited in comparison to SNPLA, but this is not a big issue as far as I am concerned since SNVI improves upon SNPLA. It now works empirically much better.\n\nQuestions:\n- The self-normalized importance sampling used to estimate fKL requires the evaluation of $w(\\theta)=p(x_o, \\theta)/\\pi(\\theta)$. How is $p(x_o, \\theta)$ computed, since we are in the likelihood-free setup? ", "summary_of_the_paper": "This paper presents SNVI, a simulation-based inference based likelihood (ratio) estimation and variational inference. The main contribution is to sidestep the slow MCMC sampling step of SNLE (SNRE) by learning the posterior distribution with variational inference. This work is similar to SNPLA but replaces the rKL variational objective with three alternative objectives that induce a mass-covering behaviour and are, therefore, better suited when using the approximate posterior distributions as proposal distributions in the sequential regime. Experiments show convincing results.\n", "summary_of_the_review": "This paper presents a simple and effective idea. It makes both SNLE and SNRE more computationally efficient while maintaining accuracy. Experiments demonstrate this clearly. Provided that my question above can be answered, I recommend this paper for acceptance. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Gcwpj485y5B", "reviewer": "Reviewer_9Rvr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n+ The idea is natural and the explanation is clear. \n\n+ It improves efficiency comparing to MCMC based SNLE and SNRE while achieves similar accuracy. \n\n+ The paper does a relatively complete ablation study. It compares the two types of KL divergence, MCMC and VI, and different variational objective in this setting. \n\n\nWeaknesses\n\n- The novelty of the paper is not very high. The major difference between the proposed method and  standard variational inference algorithm (and its variant) is that the likelihood here is not explicit.  The paper proposes to replace the unknown likelihood with an estimated likelihood, which is natural but the novelty is not very high.\n\n- The paper propose several methods to improve the inference quality, e.g., SIR, IWAE, $\\alpha$-divergence, but the reviewer concerns more about the accuracy in likelihood estimation. Can the  likelihood estimation capture the multimodality and skewness? Is there a simulation to demonstrate this accuracy?  \n\n- The calibration kernel in Sec. 3.4 is not explained clearly. How to learn $c(\\theta)$? how to draw sample from (unnormalized) posterior distribution? Does the simulations in Sec. 4.1 and 4.2 use the calibration kernel? Why the Algorithm 1 doesn’t have calibration kernel?\n\n- One question is that in the first step, instead of learning $p(x|\\theta)$ from $(x, \\theta)$ pairs, can we just learn $p(\\theta | x)$ from $(x, \\theta)$ pairs and use it to estimate $\\theta$? How does the proposed method compare to this simple approach? \n\n- The notations of the paper are sometimes confusing; here are some example:\nIn the Figure 1, the KL divergence is backward but in the method description, the KL is forward. \nOn page 4, should  $q_\\psi$ be $q_\\phi$?\nOn page 4, what are the subscripts of \\theta in the equation?\n\n", "summary_of_the_paper": "This paper proposes Sequential Neural Variational Inference (SNVI) for the posterior inference when the likelihood function is unknown. The proposed methods consist of a likelihood estimator, a posterior approximation, and sampling importance resampling. \n", "summary_of_the_review": "In sum, this paper proposes a practical way for the posterior inference with unknown likelihood. The method is simple and natural, the writing is mostly clear. However, some sections and notations are not clear, and it is better to show the limitation of each component of the proposed method. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5SBuZaGImR", "reviewer": "Reviewer_NhK3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is clear from a method-recipe perspective. It has communicated well what the key aspects and elements of the approach are and how they are used together to form the overall approach.\n\nThe main weakness of the proposed approach is that it takes the form of combining multiple existing techniques into one coherent approach. There is indeed value to the community in formulating an approach that blends the strengths in multiple existing techniques together to overcome their individual weaknesses. However, both the techniques themselves and the way they are orchestrated together are not particularly novel or insightful. Specifically, each of the three parts to the SNVI approach, as well as the three variants of the variational objective used for the posterior model part of the approach, the SIR step, and the fix using calibration kernel, have already been well established by previous works, which the authors do clearly cite. Again, there is indeed value in orchestrating these techniques together, however in its current form the way this is done is neither particularly elegant or insightful.\n\nAs a minor and related weakness, it is not clear how one would choose between the three variants of the variational objective here. In terms of writing, each of these are briefly discussed with a summary and citation to the original work without much discussion on motivating or explaining them in relation to the wider proposed approach. I also find the discussion on SIR lacking and difficult to understand exactly how this enriches the variational family, although there is a citation to the original source which may have explained it. \n\nFinally, the proposed method to get around missing simulation using the calibration kernel seems arbitrary and over-convoluted. This introduces not just a new set of hyperparameters that is not part of the main approach to worry about, but also an entire third model on top of the likelihood(-ratio) and posterior models.\n\nTo change my sentiment, I would find it most helpful if the authors could provide some explanations or arguments that would incline me to believe that:\n- The choices made in this paper must be done in the particular way described for it to have the desired properties, and is not arbitrary.\n- There are alternative choices for each component of this technique that seem sensible but can be shown to be worse than this combination of techniques.\n- The method without SIR is not sufficient ensure the quality of the posterior samples despite expressiveness of NFs (as well as how the SIR step works, ideally in pseudo-code)\n- The calibration kernel fix and the extra bias-factor model is not arbitrary and its learning or selection does not bear too much additional computational hurdle to the overall technique\n\n-----\nAfter author response:\n\nThank you for your response and clarifications, and especially on acting on the feedback.\n\nThe authors has addressed my concerns in detail and communicated how they would revise the paper accordingly for the camera-ready version. I would lean towards acceptance.", "summary_of_the_paper": "This paper presents a form of variational inference (VI) approach for likelihood-free inference (LFI) named Sequential Neural Variational Inference (SNVI). Its main advantage over other VI approaches for LFI is computational efficiency rather than accuracy.\n\nThis approach has consists of three parts, a likelihood or likelihood-ratio model, a posterior model, and a sampling importance resampling (SIR) step to refine the posterior model each round. Because the variational family for the posterior model used is based on normalising flows (NFs), whose expressiveness is already very high, the authors instead focus on the variational objective of the posterior model. Here they consider three variants from previous works - (1) forward KL divergence with self-normalized importance sampling, (2) importance weighted ELBO, and (3) Rényi $\\alpha$-divergences. Method (1) prevents objective weights to be close to zero by normalising them, at the cost of introducing a bias that vanishes at rate $\\mathcal{O}(1/N)$. Method (2) aims to provide good proposals for SIR through lower bounding the ELBO, and uses the \"Sticking the Landing\" (STL) estimator to circumvent the low signal-to-noise (SNR) ratio. Method (3) allow the mass-covering or model-seeking behaviour to be tuned, and require the same STL estimator to prevent low SNR.\n\nFinally, the authors propose an SIR step from previous work which claims to enrich the variational family with minimal computational cost.\n\nIn the case of invalid data from simulations, a kernel is used to reweigh the loss to focus on specific regions in the data-space. To fix the bias, the bias factor is estimated from another model to recover the posterior up to proportionality.\n", "summary_of_the_review": "Overall, while the technique proposed seems useful in improving computational efficiencies of existing VI approaches to LFI, the technique itself is a relatively uninspiring combination of existing techniques, of which a few aspects thereof seem either lacking in explanation or over-convoluted in construction. I would tend towards a reject, but welcome the authors to challenge this sentiment by addressing the points in my main review.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a well-presented approach to likelihood-free inference. The reviewers are all in alignment in recommending this paper for acceptance. There was a healthy discussion between authors and reviewers, where the authors have already incorporated many of their recommendations. The potential for this methodology to be applied to situations with expensive simulators should be intriguing to a broad audience. As a result, I recommend for this paper to be accepted as a spotlight.
Decision: Accept (Spotlight)