Truncated proposals for scalable and hassle-free simulation-based inference
Abstract
Simulation-based inference (SBI) solves statistical inverse problems by repeatedly running a stochastic simulator and inferring posterior distributions from modelsimulations.
To improve simulation efficiency, several inference methods take a sequential approach and iteratively adapt the proposal distributions from which model simulations are generated.
However, many of these sequential methods are difficult to use in practice, both because the resulting optimisation problems can be challenging and efficient diagnostic tools are lacking.
To overcome these issues, we present Truncated Sequential Neural Posterior Estimation (TSNPE).
TSNPE performs sequential inference with truncated proposals, sidestepping the optimisation issues of alternative approaches.
In addition, TSNPE allows to efficiently perform coverage tests that can scale to complex models with many parameters.
We demonstrate that TSNPE performs on par with previous methods on established benchmark tasks.
We then apply TSNPE to two challenging problems from neuroscience and show that TSNPE can successfully obtain the posterior distributions, whereas previous methods fail.
Overall, our results demonstrate that TSNPE is an efficient, accurate, and robust inference method that can scale to challenging scientific models.
Introduction
Computational models are an important tool to understand physical processes underlying empirically observed phenomena.
These models, often implemented as numerical simulators , incorporate mechanistic knowledge about the physical process underlying data generation, and thereby provide an interpretable model of empirical observations.
In many cases, several parameters of the simulator have to be inferred from data, e.g., with Bayesian inference.
However, performing Bayesian inference in these models can be difficult:
Running the simulator may be computationally expensive, evaluating the likelihood-function might be computationally infeasible, and the model might not be differentiable.
In order to overcome these limitations, Approximate Bayesian Computation (ABC) methods [Beaumont et al., 2002, 2009], synthetic likelihood approaches [Wood, 2010], and neural network-based methods [e.g., Papamakarios and Murray, 2016, Hermans et al., 2020, Thomas et al., 2022] have been developed.
A subset of neural network-based methods, known as neural posterior estimation (NPE) [Papamakarios and Murray, 2016, Lueckmann et al., 2017, Greenberg et al., 2019], train a neural density estimator on simulated data such that the density estimator directly approximates the posterior.
Unlike other methods, NPE does not require any further Markov-chain
Monte-Carlo (MCMC) or variational inference (VI) steps.
As it provides an amortized approximation of the posterior, which can be used to quickly evaluate and sample the approximate posterior for any observation, NPE allows the application in time-critical and high-throughput inference scenarios [Gonçalves et al., 2020, Radev et al., 2020, Dax et al., 2021], and fast application of diagnostic methods which require posterior samples for many different observations
[Cook et al., 2006, Talts et al., 2018].
In addition, unlike methods targeting the likelihood (e.g., neural likelihood estimation,
NLE [Papamakarios et al., 2019, Lueckmann et al., 2019]), NPE can learn summary statistics from data and it can use equivariances in the simulations to improve the quality of inference [Dax et al., 2021, 2022].
If inference is performed for a particular observation x o , sampling efficiency of NPE can be improved with sequential training schemes: Instead of drawing parameters from the prior distribution, they are drawn adaptively from a proposal (e.g., a posterior estimate obtained with NPE) in order to optimize the posterior accuracy for a particular x o .
These procedures are called Sequential Neural Posterior Estimation (SNPE)
[Papamakarios and
Murray, 2016, Lueckmann et al., 2017, Greenberg et al., 2019] and have been reported to be more simulation-efficient than training the neural network only on parameters sampled from the prior, across a set of benchmark tasks [Lueckmann et al., 2021].
Despite the potential to improve simulation-efficiency, two limitations have impeded a more widespread adoption of SNPE by practitioners: First, the sequential scheme of SNPE can be unstable.
SNPE requires a modification of the loss function compared to NPE, which suffers from issues that can limit its effectiveness on (or even prevent their application to) complex problems (see Sec. 2).
Second, several commonly used diagnostic tools for SBI [Talts et al., 2018, Miller et al., 2021,
Hermans et al., 2021] rely on performing inference across multiple observations.
In SNPE (in contrast to NPE), this requires generating new simulations and network retraining for each observation, which often prohibits the use of such diagnostic tools [Lueckmann et al., 2021, Hermans et al., 2021].
Here, we introduce Truncated Sequential Neural Posterior Estimation (TSNPE) to overcome these limitations.
TSNPE follows the SNPE formalism, but uses a proposal which is a truncated version of the prior: TSNPE draws simulations from the prior, but rejects them before simulation if they lie outside of the support of the approximate posterior.
Thus, the proposal is (within its support) proportional to the prior, which allows us to train the neural network with maximum-likelihood in every round and, therefore, sidesteps the instabilities (and hence ‘hassle’) of previous SNPE methods.
Our use of truncated proposals is strongly inspired by Blum and François [2010] and Miller et al.
[2020, 2021], who proposed truncated proposals respectively for regression-adjustment approaches in ABC and for neural ratio estimation (see Discussion).
Unlike methods based on likelihood(-ratio)- estimation [Miller et al., 2021, Hermans et al., 2021], TSNPE allows direct sampling and density evaluation of the approximate posterior, and thus permits computing expected coverage of the full posterior quickly (without MCMC) and at every iteration of the algorithm, thus allowing to diagnose failures of the method even for high-dimensional parameter spaces (we term this ‘simulation-based coverage calibration’ (SBCC), given its close connection with simulation-based calibration, SBC,
Cook et al.
[2006], Talts et al.
We show that TSNPE is as efficient as the SNPE method ‘Automatic Posterior Transformation’ (APT,
Greenberg et al.
[2019]) on several established benchmark problems (Sec. 4.1).
We then demonstrate that for two challenging neuroscience problems, TSNPE—but not APT—can robustly identify the posterior distributions (Sec. 4.2).
Background
In Neural Posterior Estimation (NPE), parameters are sampled from the prior p ( θ ) and simulated (i.e., x is sampled from p ( x | θ ) ).
Then, a neural density estimator q ϕ ( θ | x ) (in our case a normalizing flow), with learnable parameters ϕ , is trained to minimize the loss:
[Equation]
which is minimized if and only if, for a sufficiently expressive density estimator, q ϕ ( θ | x )
= p ( θ | x ) for all x ∈ supp ( p ( x ))
[Paige and Wood, 2016, Papamakarios and Murray, 2016].
Throughout this study, we refer to training with this loss function as maximum-likelihood training, although the neural density estimator targets the posterior directly.
Sequential Neural Posterior Estimation (SNPE) aims to infer the posterior distribution p ( θ | x o ) for a particular observation x o .
SNPE initially performs NPE and, thereby, obtains an initial estimate of the posterior distribution.
It then samples parameters from a proposal ˜
p
( θ ) , which is often chosen to be the previously obtained estimate of the posterior ˜ p
( θ ) =
q ϕ ( θ | x o ) , and retrains the neural density estimator [Papamakarios and Murray, 2016].
This procedure can be repeated for several rounds.
Importantly, if parameters θ are sampled from the proposal ˜ p ( θ ) rather than from the prior p ( θ ) , the estimator q ϕ ( θ | x ) that minimizes the maximum-likelihood loss function no longer converges to the true posterior.
If one used the maximum-likelihood loss on data sampled from ˜ p
( θ ) , i.e.,
L = E θ ∼ ˜ p
( θ ) E x ∼ p ( x | θ ) [ − log q ϕ ( θ | x )] , then L would be minimized by q ϕ ( θ | x ) ∝ p ( θ | x )
˜ p ( θ ) p ( θ ) , which is not the true posterior.
Multiple schemes have been developed to overcome this [Papamakarios and Murray, 2016, Lueckmann et al., 2017].
The most recent of these methods, Automatic Posterior
Transformation (APT, or SNPE-C, in its atomic version)
[Greenberg et al., 2019, Durkan et al., 2020] employs a loss that aims to classify the parameter set that generated a particular data point among other parameter sets (details in Appendix Sec. 6.5).
While APT has been reported to significantly outperform previous methods, several studies have also described cases in which the approach exhibits performance issues: Both the original APT paper [Greenberg et al., 2019] and Durkan et al.
[2020] reported that APT can show ‘leakage’ of posterior mass outside of bounded priors.
We demonstrate this issue on a simple 1-dimensional simulator with bounded prior (Fig. 1, Appendix Fig. 7).
The posterior estimated by APT is only required to match the true posterior density within the support of the prior (details in Appendix Sec. 6.5).
Thus, after ten rounds of APT, while the approximate posterior matches the true posterior within the bounds of the prior, a substantial fraction of posterior mass lies in regions with zero prior probability.
In simple models, approximate posterior samples that lie outside of the prior bounds can be efficiently rejected.
However, in models with high numbers of parameters, the rejection rate can become so large that drawing posterior samples which lie inside of the prior bounds is prohibitive.
For example, Glöckler et al.
[2022] reported a rejection rate of more than 99.9999% in a model with 31 parameters, thus requiring approximately one minute to draw a single posterior sample from within the prior bounds.
Initialize
Proposal dataset for r ∈ [1 , ..., R ] do for i ∈ [1 , ..., N ] do θ i ∼
˜ p
( θ ) simulate x i ∼ p ( x | θ i ) add ( θ i , x i ) to X ϕ ∗ =
arg min ϕ − 1 N (cid:80) ( θ i , x i ) ∈X log q ϕ
( θ i | x i )
Compute expected coverage( ˜ p ( θ ) , q ϕ ) ; // see Alg.˜ p ( θ ) ∝ p ( θ ) · 1 θ ∈ HPR ϵ ; // see Alg.
We overcome these limitations by using ‘truncated’ proposal distributions.
This allows us to train with maximum-likelihood at every round, thereby sidestepping issues of previous SNPE methods.
Methodology 3.1 Truncated proposals for SNPE
Given a particular observation x o , we suggest to restrict the proposals ˜
p
( θ ) to be proportional to the prior p ( θ ) at least in the 1 − ϵ highest-probability-region (HPR ϵ , the smallest region that contains 1 − ϵ of the mass) of p ( θ | x o ) , i.e.
˜ p ( θ ) ∝ p ( θ ) · 1 θ ∈M with HPR ϵ ( p ( θ | x o )) ⊆
M .
Thus, ˜ p ( θ ) is a ‘truncated’ proposal.
The key insight is that, when using such a proposal and ϵ = 0 , one can train q ϕ ( θ | x ) with maximum likelihood:
min ϕ
[Equation]
=
[Equation]
E
[Equation]
p ( x |
[Equation]
) [ − log q ϕ ( θ | x )] , and q ϕ ( θ | x o ) will still converge to p ( θ | x o ) (Proof in Appendix Sec. 6.2).
We estimate M as the HPR ϵ of the approximate posterior M = HPR ϵ ( q ϕ ( θ | x o )) .
Since the maximum-likelihood loss employed to train q ϕ ( θ | x ) is support-covering, the HPR ϵ of q ϕ ( θ | x o ) tends to cover the HPR ϵ of p ( θ | x o )
[Bishop and Nasrabadi, 2006].
In order to obtain the HPR ϵ of q ϕ ( θ | x o ) , we define a threshold τ on the approximate posterior density q ϕ ( θ | x o ) .
To do so, we use a normalizing flow as q ϕ ( θ | x ) , which allows for closed-form density evaluation and fast sampling.
We then approximate the HPR ϵ of q ϕ ( θ | x o ) as
[Equation]
We chose τ as the ϵ -quantile of approximate posterior densities of samples from q ϕ ( θ | x o ) , and evaluated TSNPE for ϵ = 10 − 3 , 10 − 4 , and 10 − 5 .
Values of ϵ > 0 yield a proposal prior which has smaller support than the current estimate of the posterior, e.g., using ϵ = 10 − 3 neglects 0.1% of mass from the approximate-posterior support.
Thus, this approach leads to errors in posterior estimation, e.g., to ‘under-covered’ posteriors (Appendix Sec. 6.10).
However, empirically, the error induced by this truncation is negligible, as we will demonstrate on several benchmark tasks.
We note that
TSNPE can be trained on data pooled from all rounds (Appendix Sec. 6.2).
TSNPE is summarized in
Alg.
1 (Fig. 2).
3.2 Sampling from the truncated proposal
To generate training data for subsequent rounds, we have to draw samples from the truncated proposal ˜
p
( θ ) , and
here we explored rejection sampling and sampling importance resampling (SIR)
[Rubin, 1988].
For rejection sampling, we sample the prior θ ∼ p ( θ ) and accept samples only if their probability under the approximate posterior q ϕ ( θ | x ) is above threshold τ .
This strategy samples from the truncated proposal exactly, but can fail if the rejection rate becomes too high.
To deal with these situations, we used SIR.
For each sample from the truncated proposal, SIR draws K samples from the approximate posterior, computes weights w i =1 ...
K = p ( θ i )
θ i ∈M /q
ϕ ( θ i | x ) , normalizes w i such that they sum to one, draws from a categorical distribution with weights s ∼ Categorical ( w i ) , and selects the posterior sample with index s .
SIR requires a fixed sampling budget of K posterior samples per sample from the truncated proposal and returns exact samples from the truncated proposal for K → ∞ .
Too low values of K lead to too narrow proposals and posterior approximations.
When run for a number of rounds, this behaviour reinforces itself and can lead to divergence of TSNPE (Appendix Fig. 13).
We, thus, chose a high value K = 1024 .
In our experiments, we did not observe poor SIR performance, but we emphasise the importance of using tools to diagnose potential failures of TSNPE (see below) or SIR (e.g. by inspecting the effective sample size, Appendix Sec. 6.12).
When SIR fails, methods such as nested sampling, adaptive multi-level splitting, or sequential Monte-Carlo sampling could be viable alternatives [Skilling, 2004, Cérou and Guyader, 2007, Doucet et al., 2001].
We discuss computational costs of rejection sampling and SIR in Appendix Sec. 6.11.
3.3 Coverage diagnostic
In order for the estimated posterior q ϕ ( θ | x o ) to converge to p ( θ | x o ) , TSNPE requires supp ( p ( θ | x o )) ⊆
HPR ϵ ( q ϕ ( θ | x o )) , i.e., the estimated posterior must be broader than the true posterior (proof in Appendix Sec. 6.2).
In order to diagnose whether the posterior is, on average, sufficiently broad, we perform expected coverage tests as proposed in Dalmasso et al.
[2020], Miller et al.
[2021], Hermans et al.
As described in Dalmasso et al.
[2020], Rozet et al.
[2021] and illustrated in Fig.
3, the coverage of the approximate posterior can be computed as
[Equation]
where θ ∗ is sampled from the truncated proposal and x ∗ is the corresponding simulator output.
In order to approximate this integral, one has to either evaluate the approximate posterior on a grid [Dalmasso et al., 2020, Hermans et al., 2021] or apply a Monte-Carlo average which includes repeatedly sampling (and evaluating) the (unnormalized) approximate posterior
[Miller et al., 2021,
Rozet et al., 2021].
The first option does not scale to high-dimensional spaces whereas the second is computationally expensive for methods estimating likelihood(-ratios) and, thus, require MCMC.
In contrast, the TSNPE-posterior can be sampled from and evaluated in closed-form, leading to a computationally efficient and scalable diagnostic which can be run after every training round.
Expected coverage can be computed as an average of the coverage across multiple pairs ( θ ∗ , x ∗ )
[Miller et al., 2021, Hermans et al., 2021] and should match the confidence level for all confidence levels (1 − α ) ∈ [0 , 1] (Fig. 3c).
We term this procedure of computing the empirical coverage ‘simulation-based coverage calibration‘ (SBCC), due to its close connection with SBC [Cook et al., 2006, Talts et al., 2018]
(identical under certain conditions, Appendix Sec. 6.6).
For TSNPE, it is important that the empirical expected coverage matches the confidence level for high confidence levels (i.e., for small α ), since overconfidence in these regions would indicate that ground-truth parameters θ ∗ are
Results
We evaluated TSNPE on several benchmark tasks and on two complex problems from neuroscience.
We found that TSNPE performs as well as APT on the benchmark tasks and that it is robust to choices of ϵ .
In addition, we found that, in contrast with APT, TSNPE can successfully infer the posterior distribution for complex models with large numbers of parameters.
4.1 Performance on benchmark tasks
We compared TSNPE with NPE and APT on six benchmark tasks for which samples from the ground-truth posterior are available (see Appendix Sec. 6.9 for tasks)
[Lueckmann et al., 2021].
We quantified the performance with a classifier two-sample test (C2ST), for which 0.5 indicates that the approximate posterior is identical to the ground-truth posterior, whereas 1.0 implies that the distributions can be completely separated by a classifier.
Overall, APT and TSNPE perform similarly well and both outperform NPE (Fig. 4, left three columns).
On two of the six tasks (Gaussian Linear and SLCP), APT has slightly better performance than TSNPE, whereas on two other tasks (SIR and Lotka-Volterra), TSNPE outperforms APT.
Overall, TSNPE and APT perform similarly well, demonstrating that TSNPE is competitive with previous methods on benchmark tasks.
In order to get insights into the improved performance of TSNPE as compared to NPE, we computed the fraction of prior samples that lie within the HPR ϵ of the approximate posterior (Fig. 4, fourth column).
In tasks with broad posteriors and few simulations, the HPR ϵ is almost as wide as the prior and thus the performances of NPE and TSNPE are similar (e.g., SLCP with 1k simulations).
In other tasks and with more simulations, the HPR ϵ is much narrower than the prior, leading to an improvement in simulation efficiency (e.g., Lotka-Volterra with 100k simulations).
Finally, we evaluated whether the HPR ϵ of the approximate posterior contains the support of the true posterior (Fig. 4, fifth column).
We computed the fraction of true-posterior samples within the HPR ϵ of the approximate posterior.
For most tasks, fewer than 0.1% of samples were excluded, and the rate of erroneously rejected samples decreased as more simulations were used.
In the Lotka-Volterra task with 1k and 10k simulations, many ground-truth samples were rejected and TSNPE performed poorly, but NPE and APT also failed to solve the task.
Thus, while truncated proposals can potentially induce posterior biases, these only have a negligible effect on the performance of TSNPE.
We note that
TSNPE performance is qualitatively unaffected by the choice of ϵ ≤ 10 − 4 and proposal sampling scheme (Appendix Fig.
8, Fig.
9, Fig.10).
Applying truncated proposals to APT leads to equally good or worse performance than ’standard’ APT, depending on the task (Appendix Fig. 14).
4.2 Efficient and robust inference in two complex neuroscience problems
Next, we evaluate the performance of TSNPE on two challenging neuroscience problems, where the competitive advantage of TSNPE is fully realized.
Pyloric network We applied TSNPE to a challenging real-world simulator from neuroscience: The pyloric network of the stomatogastric ganglion in the crab Cancer Borealis [Prinz et al., 2003, 2004].
The model has 31 parameters and simulates 3 voltage traces that we reduce to 18 summary statistics.
The prior distribution is uniform within previously described parameter ranges [Prinz et al., 2004,
Gonçalves et al., 2020].
We identify the posterior distribution given experimentally observed data [Haddad and Marder, 2021] (Fig. 5a) with APT and TSNPE (13 rounds, 30k simulations per round).
When applying APT ‘out of the box’ (from ‘sbi’ toolbox [Tejero-Cantero et al., 2020]), the rate of approximate-posterior samples within the prior bounds was 0.02% after the second round and 0.0000% after the third round (Fig. 5b), which rendered a fourth round too computationally expensive.
We attempted to overcome these issues by appending a transformation T to the density estimator q ϕ ( θ | x ) such that its support is constrained to match the support of the prior.
In practice, we used a sigmoid transformation.
While the resulting approximate posterior exhibited no leakage, this setup revealed another problem when running APT:
In transformed (i.e., unbounded) space, the density estimator q ϕ ( θ | x ) can put significant mass in regions outside of the training data.
When forced into constrained space, these ‘leaking’ regions lead to spikes at the bounds of the parameter space (Fig. 5c, further details in Appendix Sec.
6.5; illustration, additional tests and full posterior in Appendix
Figs.
17, 16 and 18).
These spikes are at odds with previously published posterior distributions [Gonçalves et al., 2020, Deistler et al., 2021, Glöckler et al., 2022] and samples from these parameter regions do not produce good simulations (Appendix Fig. 19).
This demonstrates that leakage occurs in APT even when the density estimator is forced into constrained space and that these issues lead to an incorrect posterior approximation as well as to poor predictive samples.
We applied TSNPE to this task for 13 rounds without any issue.
The resulting posterior produces samples that closely match the observed data (Fig.
5d, more samples in Appendix Fig.
15, posterior distribution across all 31 parameters in Appendix Fig. 20).
The obtained posterior is similar to previously published posteriors [Gonçalves et al., 2020, Deistler et al., 2021, Glöckler et al., 2022].
Multicompartment model of a single neuron Finally, we turn to a landmark problem in neuroscience for which the posterior has not yet been identified:
A morphologically detailed model of a thick-tufted layer 5 pyramidal cell (L5PC) from the neocortex [Ramaswamy et al., 2015, Markram et al., 2015, Van Geit et al., 2016].
The model describes the response of a neuron to current stimuli of different strengths.
The model has approximately 7000 separate compartments which compose the anatomy of the cell (Fig. 6a).
Each compartment has dynamics based on the Hodgkin-Huxley equations [Hodgkin and Huxley, 1952] and contains multiple ion channels (details in Van Geit et al.
The model has 20 free parameters which are the maximal channel conductances and time constants of the ion channels.
The simulation consists of four separate simulations corresponding to experimental protocols which describe the voltage response to different stimuli.
For the first three protocols (Step 1, Step 2, Step 3), each voltage response is characterized by 10 summary statistics.
The fourth protocol models the back-propagation of the voltage response through the dendritic tree and is captured by 5 additional summary statistics (bAP soma, bAP dend.
1, bAP dend. 2).
In total, the model produces 35 summary statistics, to which we add Gaussian noise with diagonal covariance matrix capturing the response variability of previously reported measurements [Hay et al., 2011].
Our goal is to infer the posterior distribution over 20 parameters given 35 summary statistics that were simulated—and thus have a known ground-truth parameter set— to resemble experimentally observed activity (Fig. 6b).
The prior is a uniform distribution within previously established bounds [Van Geit et al., 2016].
A major difficulty in fitting this model is that a large fraction of prior samples generate summary statistics that are very different from the observed data: In particular, about 99.98% of prior predictives contain at least one summary statistic that is undefined, e.g., time to first spike is undefined in the absence of spikes (Fig. 6c).
When a summary statistic is undefined, we assign it a value that is substantially outside the range of the observed data (Appendix Sec. 6.14).
We ran TSNPE over six rounds (hyperparameters in Appendix Sec. 6.15).
In each round, we ran 30k simulations, leading to a total of 180k simulations.
After every round, we evaluated the expected coverage with SBCC.
After the first round, the approximate posterior exhibited poor expected coverage (Fig.
6d top).
Therefore, as suggested by Hermans et al.
[2021], we used an ensemble ofneural density estimators to ensure that the approximate posterior is sufficiently broad (Fig.
6d bottom).
Although the approximate posterior remains underconfident, the empirical expected coverage closely matches the confidence level for high-confidence levels, which is crucial for TSNPE (Sec. 3.3).
The TSNPE-posterior has several parameters with broad marginals, demonstrating that this model exhibits ‘degeneracy’, a widespread phenomenon in biological systems [Marder and Taylor, 2011] (Fig.
6e, Appendix Sec.
6.14 for parameter names).
Other marginals are narrow, demonstrating that the model is sensitive to changes in these parameters.
Posterior predictive samples closely match the observed data (Fig.
6f,g; more samples in Appendix Fig. 22).
We emphasize that fitting such morphologically-detailed neuron models is a challenging and widespread problem in neuroscience, one for which commonly used methods [e.g., genetic algorithms, Druckmann et al., 2007, Van Geit et al., 2016] are often simulation-inefficient or do not estimate the full posterior distribution.
We show promising results, suggesting that TSNPE could be applied to other complex single-neuron models.
In contrast, with ‘out of the box’ APT, none of the 10M approximate-posterior samples was within the prior bounds after the second round.
This rendered a third round too computationally expensive.
Overall, the results on the pyloric network model and on the multicompartment model demonstrate that TSNPE is an efficient and robust method that scales to complex and high-dimensional models that were inaccessible to the state-of-the-art method APT.
Discussion
We presented a new method to perform Bayesian inference in implicit models, which we call Truncated Sequential Neural Posterior Estimation (TSNPE).
Like previous methods, TSNPE adaptively selects parameters to improve simulation-efficiency and allow posterior inference in complex models with many parameters.
The key ingredient is that TSNPE samples parameters from a ‘truncated’ region of the prior, and thus overcomes instabilities of previous methods while maintaining simulation efficiency.
In order to diagnose potential errors of TSNPE, we developed a coverage test that can be run quickly and at every round of TSNPE.
TSNPE presents a new variant of SNPE which is at least as powerful as previous variants on benchmark tasks, but provides a powerful alternative which is able to solve inference problems on which the state-of-the-art method APT failed.
Related work TSNPE differs from automatic posterior transformation (APT, SNPE-C) in its proposal and its loss function: TSNPE uses a truncated prior as proposal, while APT can flexibly use any proposal, which, e.g., allows for more sophisticated active learning rules [Lueckmann et al., 2019,
Järvenpää et al., 2019].
However, APT’s flexibility requires a modification of NPE loss function, which can be an impediment to its usage in practice: First, the modification can lead to ‘leakage’, which can make it prohibitive to draw samples within prior bounds.
Second, APT loss requires an explicit prior and thus, cannot be applied to models in which the prior can only be sampled [Ramesh et al., 2022].
Third, current formulations of APT cannot discard parameters leading to invalid simulations as the posterior mass would ‘leak’ into parameter regions which only produce invalid simulations (Appendix Sec. 6.5).
It might be possible that these issues are resolved using a modified formulation of APT, e.g., by combining its atomic loss with additional loss terms, or preventing leakage by penalizing ‘bad’ parameters [Greenberg et al., 2019].
In cases in which leakage prevents application of APT (in particular, in high-dimensional problems), TSNPE provides an alternative.
Our method is inspired by previous work that introduced a mechanism to post-hoc correct samples obtained by an Approximate Bayesian Computation (ABC) algorithm
[Blum and François, 2010], i.e., ‘regression adjustment ABC’.
Their method draws samples from a truncated region of the prior to avoid correction terms, but estimates the posterior density with ABC samples—rather than using a flexible neural density estimator—and estimates the support by training a dedicated support-vector machine.
In addition, the method runs a single round of truncation and retraining, whereas we demonstrate that TSNPE can be robustly applied across 10 rounds.
Truncated proposals have also been proposed for neural ratio estimation [Truncated Marginal Neural
Ratio Estimation (TMNRE) Miller et al., 2021]: TMNRE uses truncated proposals to efficiently infer selected posterior marginals while being amortised around the observation, allowing to test the coverage properties of the selected marginals, e.g., with SBC
[Cook et al., 2006, Talts et al., 2018].
In addition, truncating based on the marginals allows TMRNE to sample from the truncated proposal without rejection or SIR sampling.
In contrast, TSNPE aims at efficiently inferring the full posterior distribution by proposals that avoid the correction of SNPE loss function.
Truncating the proposal based on the full posterior rather than on the marginals can lead to drastically narrower proposals:
E.g., on the pyloric network problem, truncation based on posterior marginals rejects 20% of prior samples versus 99.94% rejection based on the posterior joint.
In addition, while TMNRE uses the expected coverage to test the consistency of the posterior marginals, TSNPE can test the expected coverage of the full posterior distribution.
Possible failure modes The main failure mode of TSNPE will occur if the truncated proposal excludes significant portions of density mass of the true posterior (e.g., if the estimate misses posterior modes).
In these cases, the learned approximate posterior will put systematically too little mass in the excluded regions.
We recommend the use of diagnostic tools such as SBCC to identify such failures [Cook et al., 2006, Miller et al., 2021, Hermans et al., 2021, Rozet et al., 2021].
In addition, if the true posterior has unbounded support, any finite values of ϵ > 0 will lead to a biased approximate posterior which puts too little weight in the posterior tails.
In that case, and when running TSNPE across many rounds, the errors from each individual round could accumulate.
Although we did not observe this bias to significantly affect the algorithm performance on several benchmark tasks, we cannot exclude the possibility of a substantial performance degradation when running TSNPE for a larger number of rounds ( ≫ 10).
Finally, unlike SNPE methods that use the previous estimate of the posterior as the proposal distribution, our method requires a scheme to sample from a truncated proposal.
If the sampling scheme is inaccurate (i.e., if it does not produce a proposal distribution that is proportional to the prior within the truncated region), the results of TSNPE will be biased.
To avoid this, we recommend using rejection sampling by default and using SIR or sequential Monte-Carlo methods only if rejection sampling is too computationally expensive.
For SIR, it is important to use a large oversampling factor K (e.g.,
K = 1024 ) and use diagnostic tools such as effective sample size (Appendix Sec. 6.12, Fig. 13).
Simulation-based coverage calibration
In order to diagnose whether the approximate posterior is broader than the true posterior, we applied SBCC, a coverage test for TSNPE
[Cook et al., 2006, Rozet et al., 2021].
SBCC evaluates the expected coverage of the approximate posterior without evaluating it on a grid [Dalmasso et al., 2020, Hermans et al., 2021] and, unlike diagnostic tools for methods based on learning the likelihood(-ratio), does not require MCMC runs for multiple observations x
[Miller et al., 2021].
This allows SBCC to be run quickly and for models with many parameters.
In addition, in contrast to diagnostic tools for likelihood-free inference with Approximate Bayesian Computation,
SBCC does not require an additional step to estimate the density of approximate posterior samples [Prangle et al., 2014].
We note that, since SBCC is a variation of SBC
[Cook et al., 2006, Talts et al., 2018], it only ensures that the HPR ϵ is correct on average across observations, not for a particular observation.
In principle, SBCC could be applied to other SNPE variants, although empirically the impact of arbitrary proposals on SBCC performance is currently unclear.
Conclusion Overall, TSNPE combines the simulation-efficiency of sequential neural posterior estimation with the robustness and coverage-tests of non-sequential methods.
We demonstrated that it allows to scale neural posterior estimation to complex and high-dimensional scientific problems.
Acknowledgments and Disclosure of Funding
We thank Poornima Ramesh, Cornelius Schröder, Marcel Nonnenmacher, David Greenberg, and
Jan-Matthis Lueckmann for discussions and feedback.
We also thank the International Max Planck
Research School for Intelligent Systems (IMPRS-IS) for supporting MD.
This work was funded by the
German Research Foundation (DFG; Germany’s Excellence Strategy MLCoE – EXC number 2064/
PN 390727645) and the German Federal Ministry of Education and Research (BMBF; Tübingen AI
Center, FKZ: 01IS18039A).
Appendix 6.1 Reproducibility statement
We used the configuration manager hydra to track the configuration and seeds of each run [Yadan, 2019].
All code to reproduce the results can be found at https://github.com/mackelab/tsnpe_ neurips .
We implemented TSNPE on top of the publicly accessible sbi toolbox [Tejero-Cantero et al., 2020].
All simulations and runs were performed on a high-performance computer.
For each run, we used between 8 and 48 CPU cores.
6.2 Proof of convergence
Below, we prove that, for a given observation x o , the posterior distribution obtained with TSNPE q ϕ ( θ | x o ) converges to the true posterior distribution p ( θ | x o ) under the assumption that the HPR ϵ of the approximate posterior at every TSNPE round covers the support (i.e., the HPR ϵ for ϵ = 0 ) of the true posterior given the observation x o .
We call the support of the true posterior M , i.e. M = supp ( p ( θ | x o )) .
The proof proceeds in two steps: First, we derive the effective proposal distribution when pooling data from all rounds.
Second, we show that, for such proposal distributions, the neural density estimator converges to the true posterior.
Deriving the proposal distribution We denote by ˜
p r ( θ ) the proposal distribution from which θ are drawn in round r .
In the first round, we use the prior, i.e., ˜ p r =1 ( θ )
= p ( θ ) .
In later rounds, we sample from the prior but reject samples that lie outside of the HPR ϵ of the approximate posterior given x o .
The samples drawn in round r are thus drawn from
[Equation]
where Z r is the normalization constant and U r ( θ ) is 1 on the HPR ϵ of the approximate posterior at round r and zero otherwise.
Given our assumption that the HPR ϵ of the approximate posterior covers the support of the true posterior M , U r ( θ ) is 1 on the support of the true posterior M .
When pooling simulations from all rounds, after R rounds, the parameters are sampled from a mixture of all proposal distributions:
[Equation]
In this equation, we assumed that all rounds contain equally many simulations, but the proof can easily be extended to rounds with different numbers of simulations by adding weights to the above sum.
As can be seen above, the distribution ˜ p
( θ ) is the prior times a function f ( θ ) which is made up of several steps and whose steps are defined by the HPR ϵ of the approximate posterior of every round (illustration in Appendix Fig. 23).
Finally, under the assumption that all U r ( θ ) are 1 on the support of the true posterior M , we have, for any θ ∈ M
[Equation]
i.e., f ( θ ) is not a function of θ .
Thus, for any θ ∈ M , we have
[Equation]
We emphasise that this proportionality holds only for θ ∈ M , but not necessarily for θ outside of the support of the true posterior.
Training the neural density estimator Next, we show that, for a proposal distribution of the form derived in the paragraph above, the approximate posterior q ϕ ( θ | x o ) for an observation x o converges to the true posterior distribution p ( θ | x o ) .
TSNPE minimizes the following loss function:
[Equation]
Plugging in the proposal distribution defined above:
[Equation]
The term within the integral over θ is proportional to the Kullback-Leibler-divergence between f ( θ ) p ( θ | x ) /Z
(with Z = (cid:82) f ( θ ) p ( θ | x ) d θ ) and the approximate posterior q ϕ ( θ | x ) .
Thus, L is minimized if and only if
[Equation]
for all x within the the support of p ( x ) [Papamakarios and Murray, 2016].
This means that, for arbitrary x ∈ supp ( p ( x )) , q ϕ ( θ | x ) will not converge to the true posterior p ( θ | x ) , but to f ( θ ) p ( θ | x ) /Z .
However, for the observed data x = x o , we have:
[Equation]
The first case follows from the fact that f ( θ ) is constant on the support of the true posterior.
The second case follows because the true posterior has zero probability density for θ outside of its own support M .
Thus:
[Equation]
Since q ϕ ( θ | x o ) is a (conditional) normalizing flow, it is normalized and, thus:
[Equation]
6.3 Simulation-based coverage calibration (SBCC)
The algorithm for computing the coverage (SBCC) is shown in Alg.
Algorithm
Inputs: proposal ˜ p
( θ ) , approximate posterior q ϕ trained on θ ∼ ˜
p
( θ ) , number of simulations M , number of posterior samples per simulation P .
Database of empirical coverages E , initialized as the empty set.
Outputs: Coverage for different confidence levels 1 − α .
for i ∈ [1 , ..., M ] do θ ∗ i ∼ ˜ p
( θ ) simulate x ∗ i ∼ p ( x | θ ∗ i ) l ∗ i = log( q ϕ ( θ ∗ i | x ∗ i )) ; // compute log-prob of ground-truth c =for j ∈ [1 , ..., P ] do θ j ∼ q ϕ ( θ | x ∗ i ) l j = log( q ϕ ( θ j | x ∗ i )) ; // compute log-probs of posterior samples if l j > l ∗ i then c
= c +e = c/P ; // fraction of posterior samples whose log-prob is larger than ground-truth log-prob add e to E Plot CDF of E
Inputs: Prior p ( θ ) , Approximate posterior q ϕ , observation x o , number of posterior samples M , ϵ that defines the HPR, number of desired samples from ˜ p
( θ ) N .
Database of log-probabilities
P , initialized as an empty set.
Outputs: Samples S from truncated proposal ˜ p
( θ ) .
for i ∈ [1 , ..., M ] do θ
i ∼ q ϕ
( θ | x o ) l i = log( q ϕ ( θ i | x o ) add l i to P κ = quantile ϵ
( P ) ; // threshold for HPR ϵ // sample truncated proposal with rejection sampling while s < N do θ ∼ p ( θ ) l = log( q ϕ ( θ | x o ))
if l > κ then add θ to S s +=
6.4 Algorithm description for sampling from the HPR of the approximate posterior
The algorithm for sampling from the HPR ϵ of the approximate posterior is shown in Alg.
6.5 Alleviating issues of APT
We compared TSNPE to Automatic Posterior Transformation (APT) [Greenberg et al., 2019].
In this section, we briefly review how APT works, and why ‘leakage’ occurs, and how we attempted to improve it.
APT review Broadly, APT exists in two versions.
Its first version can be applied only if the density estimator q ϕ ( θ | x ) is a mixture of Gaussians, the proposal ˜
p
( θ ) is a mixture of Gaussians, and the prior p ( θ ) is either uniform or Gaussian.
We did not compare TSNPE to APT in this form because we wanted to use more expressive density estimators for q ϕ ( θ | x ) .
The second version of APT, known as atomic APT, allows to use any density estimator q ϕ ( θ | x ) , any proposal ˜ p
( θ ) and any explicit prior p ( θ ) .
One must be able to evaluate the density estimator and the prior, but the proposal can be implicit (i.e., without closed form density).
Atomic APT minimizes the loss:
[Equation]
with number of atoms A .
In this loss, θ j and θ i can be sampled from any proposal distribution.
x i is sampled from p ( x | θ i ) .
Leakage issue Notice that the above loss is the same for q ϕ ( θ | x ) and for c · q ϕ ( θ | x ) .
In other words, the approximate posterior has to be correct only up to a proportionality constant [Greenberg et al., 2019, Durkan et al., 2020].
Thus, an approximate posterior q ϕ ( θ | x ) which is c · p ( θ | x ) will have a minimal loss.
Since the dataset on which q ϕ ( θ | x ) is trained contains no θ that lies outside of the prior bounds, the approximate posterior can be anything outside of the prior bounds without affecting the value of the loss function.
This is what is called ‘leakage’, and which has been pointed out as a potential problem both in the original APT paper Greenberg et al.
[2019] and work studying the relationship of APT with contrastive learning approaches Durkan et al.
[2020]: While the approximate posterior might be proportional to the true posterior within the bounds of the prior p ( θ ) , it can put significant mass outside of the prior bounds (because the loss does not penalize this behaviour).
Transforming the parameter space We tried to fix the leakage issue by appending a transformation such that the density estimator has constrained support.
This requires that the bounds of the parameter space are known and that such a transformation can be implemented.
If this fix can be applied, the leakage will be zero by definition.
In all our experiments, such a transformation substantially helped with the leakage problem.
Leakage into regions of no training data However, even when possible, such transformation does not completely prevent ‘leakage’:
When inspecting the loss of APT, we see that the approximate posterior can put mass into any region of the parameter space in which no parameter sets θ in the training data lie.
For example, if the prior distribution is standard Gaussian and one trains APT on 100 parameter sets sampled from the prior, the training dataset will unlikely contain values that are smaller than -3 or larger than 3.
Therefore, APT can ‘leak’ into the regions below -3 or above 3 and still have an optimal loss.
While, in the limit of infinite data, APT would correct its ‘leakage’ as soon as parameters from these regions are used as training data, given a finite number of simulations, the approximate posterior can ‘leak’ into new regions which have not been explored (yet).
The (potential) problem of leakage does not affect all applications equally: High-dimensional parameter spaces suffer more from this behavior than low-dimensional ones, as there are many regions into which the mass of the approximate posterior can ‘leak’.
This behaviour is illustrated in Appendix Fig.
Leakage because of invalid data Another way in which leakage can occur is if the simulator produces invalid data (e.g. NaN or infinity).
Often, such invalid simulations are discarded from the training dataset [Lueckmann et al., 2017] and the approximate posterior is trained only on samples that produce valid outputs.
For example, assume that a small region of the prior always produces invalid simulations.
In this case, the approximate posterior will never be trained on simulations from this parameter region and APT can ‘leak’ into this region.
Thus, the approximate posterior might contain significant mass in parameter regions that produce invalid simulations.
Explicit recommendations for running APT We will now give explicit recommendations for running APT.
These modification greatly improved the performance of APT in our experiments, but were not able to avoid the failure of the algorithm on challenging real-world problems such as the pyloric network (Fig. 5).
For priors with bounded supports: If possible, transform the parameter space into unbounded space.
Do not discard invalid simulations.
Instead, replace invalid entries (such as NaN) with a substantially different value than the observed data and train on all available simulations.
Check if the posterior contains a lot of mass in regions with very low prior probability.
If this is the case, it can hint at a failure of APT through leakage into regions of no training data.
If you transformed the parameter space, check if many posterior samples lie very close to the bounds.
Again, this can hint at a failure of APT through leakage into regions of no training data.
6.6 Relation between simulation-based calibration and our diagnostic
As discussed above, our method is closely related to simulation-based calibration (SBC)
[Cook et al., 2006, Talts et al., 2018].
Briefly, SBC samples θ ∗ from the prior, samples the likelihood x ∗ ∼ p ( x | θ ∗ ) and then draws samples from the posterior θ i ∼ p
( θ | x ∗ ) (e.g., with MCMC).
It then projects the (potentially high-dimensional) parameters θ i into a one-dimensional space T ( θ i ) .
Often, this projection is the 1D-marginal distribution of parameters [Carpenter et al., 2017].
It then ranks
T ( θ ∗ ) under all posterior samples T ( θ 1 ...N ) .
Repeated across several prior samples, the distribution of ranks should be uniform.
For high-dimensional parameter spaces, the marginal distribution of each parameter is checked independently.
Notably, other projections into a one-dimensional space are possible.
Below, we explain that our method is identical to running SBC with projection T : θ → q ϕ ( θ | x ) and with posterior samples exactly following q ϕ ( θ | x ) .
In SBC and in our diagnostic method, samples are drawn from the prior θ ∗ ∼ p ( θ ) and simulated x ∗ ∼ p ( x | θ ) .
In our diagnostic method, as well as in SBC with projection being the log-probability of the approximate posterior, one then samples the posterior to obtain θ i and evaluates the logprobability of all samples,
i.e., l i = T ( θ i ) =
log( q ( θ i | x ∗ ) as well as of the initial parameter set l ∗ = log( q ( θ ∗ | x ∗ ) .
SBC then ranks l ∗ under all l i , which is equivalent to computing its quantile (as in our method).
In our diagnostic tool, one then evaluates whether this quantile is above or below several confidence levels (evaluation on a 1D evenly spaced grid, same as rank binning in SBC).
This generates a step-function with the step occuring at the quantile of l ∗ .
This step function is the cumulative distribution function of a dirac at the quantile of l ∗ .
Therefore, repeated across several θ ∗ , our coverage plots (e.g., Fig 3c) correspond to the cumulative distribution function of the histograms generated by SBC (with projection T : θ → log( q ϕ ( θ | x )) and posterior samples exactly following q ϕ ( θ | x ) ).
6.7 SBCC in a multi-round setting
We run our diagnostic tool after every round of training.
If one trains only on simulations that were run in the most recent round, SBCC can be run as in the first round.
However, if one wishes to train on simulations from all rounds, then the deep neural density estimator converges to:
[Equation]
Proof in Sec.
As is described in Sec. 6.2, this means that q ϕ ( θ | x ) will not converge to the true posterior for arbitrary x , but only for the observation x o .
This poses a problem for SBCC:
As described in Alg. 2, SBCC measures whether the coverage is correct (on average) for many x generated by the proposal distribution.
Since the loss employed by
TSNPE only ensures convergence for x o , it will by construction not provide correct results for other x .
This issue can be solved in two ways:
When running SBCC, instead of drawing samples from the proposal prior of the most recent round ˜ p r ( θ ) , one can draw samples θ ∗ from ˜ p
( θ ) , i.e., the distribution that emerges from pooling data from all rounds (notation as in Sec.
6.2 and Alg. 2).
This is the method described in Appendix Alg. 2. 2.
One can truncate the approximate posteriors q ϕ
( θ | x ∗ ) while running SBCC (see Alg. 2 for notation).
With this strategy, when running SBCC in round r , we draw parameters from θ ∗ ∼ ˜ p r ( θ ) (the truncated proposal from round r ), simulate them x ∗ ∼ p ( x | θ ∗ ) , sample from the posterior θ i ∼ q ϕ ( θ | x ∗ ) , reject samples that lie outside of HPR ϵ ( q ϕ ( θ i | x o )) , and then continue as described in Alg. 2.
Strategy 2 only ensures that the posterior regions which lie within HPR ϵ ( q ϕ ( θ | x o )) are well-calibrated.
It does not ensure that the full posterior ( q ϕ ( θ | x ∗ ) with x ∗ ∼ p ( x ∗ | θ ∗ ) and θ ∗ ∼ ˜ p r ( θ ) ) is well-calibrated.
6.8 Toy model
The toy model shown in Fig.
1 is given by a uniform prior within [ − 2 , − 1] and [1 , 2] .
The simulator is x ∼ θ 2 + ϵ , where ϵ is a Gaussian distribution with mean zero and standard deviation 0 .
We ran
APT [Greenberg et al., 2019] and TSNPE for 10 rounds with 500 simulations per round.
For APT, all hyperparameters are the default values from the sbi package [Tejero-Cantero et al., 2020], but we used a neural spline flow (NSF) for both APT and TSNPE
[Durkan et al., 2019].
6.9 Benchmark tasks
Below, we briefly describe the benchmark tasks.
For details, please see Lueckmann et al.
Gaussian linear: 10 parameters which are the mean of a Gaussian model.
The prior is Gaussian, resulting in a Gaussian posterior.
Bernoulli GLM: Generalized linear model with Bernoulli observations.
Inference is performed on 10-dimensional sufficient summary statistics of the originally 100 dimensional raw data.
The resulting posterior is 10-dimensional, unimodal, and concave.
Lotka Volterra: A traditional model in ecology [Wangersky, 1978], which describes a predator-prey interaction between species, illustrating a task with complex likelihood and unimodal posterior.
SLCP:
A task introduced by Papamakarios et al.
[2019] with a simple likelihood and complex posterior.
The prior is uniform, the likelihood has Gaussian noise but is non-linearly related to the parameters, resulting in a posterior with four symmetrical modes.
Two moons: This model has two parameters with a uniform prior.
The simulator is non-linear, generating a posterior with both local and global (bimodal) structure [Greenberg et al., 2019].
SIR: Epidemiological model with two parameters and ten summary statistics [Kermack and McKendrick, 1927].
6.10 Errors due to truncation
As described in Appendix Sec. 6.2, the approximate posterior converges to the true posterior if the truncated proposal covers the support of the true posterior.
The truncated support is defined as the high probability region that contains 1- ϵ of mass of the approximate posterior (HPR ϵ ).
For ϵ > 0 , the
HPR ϵ will likely not be a superset of the support of the true posterior and hence, there will be errors in posterior approximation.
In this section, we discuss the effect of these errors on inference accuracy.
When the value of ϵ is chosen too large, the tails of the approximate posterior are excluded from the truncated proposal.
In the following training round, the approximate posterior converges to a distribution that is correct, up to proportionality, within the HPR ϵ of the previous approximate posterior, but that underestimates the tails of the posterior distribution.
We demonstrate this behavior in Appendix Fig.
11 for a linear Gaussian simulator, uniform prior, 50k simulations per round, and a neural spline flow with 20 bins as neural density estimator.
After round 1, the approximate posterior closely matches the true posterior (Fig. 11a).
When using a large ϵ , e.g., ϵ = 0 . 1 , the proposal for the second round is narrower than the true posterior and, thus, the proposal obtained by pooling data from both rounds is not constant on the support of the true posterior (Fig. 11b, blue).
This leads to the approximate posterior underestimating the tails of the true posterior (Fig. 11b, purple).
When using a smaller ϵ , e.g., ϵ = 0 .
01 , the errors induced by truncation become small and inference errors are mostly due to finite data and imperfect convergence of the neural network (Fig. 11c).
We note that, throughout our study, we evaluated ϵ =
{ 10 − 3 , 10 − 4 , 10 − 5 } , i.e., values that are at least one order of magnitude smaller than ϵ = 0 .
Overall, this analysis demonstrates that the truncation performed by TSNPE can negatively impact inference quality in the tails of the posterior distribution.
We, thus, do not recommend TSNPE in scenarios in which users are particularly interested in the tails of the posterior.
In all our benchmark tasks, however, we did not find that the truncation negatively impacted inference quality as measured by C2ST accuracy (Fig. 4, Appendix Fig. 8, Fig.
This indicates that, for many (real-world) tasks, the errors due to truncation are outweighed by errors due to finite simulation budgets or imperfect convergence of neural network training.
6.11 Computational cost of rejection sampling and SIR
In this section, we quantify the computational costs of rejection sampling and sampling-importance resampling (SIR).
The computational cost of both sampling methods comprises the computational cost of sampling and evaluating the approximate posterior.
On an AMD Ryzen Threadripper 1920X 12-Core Processor, drawing (or evaluating) 100k samples takes approximately 10 seconds.
On a
GeForce RTX 2080 GPU, drawing (or evaluating)
100k samples takes approximately 0.17 seconds.
Thus, in SIR (with an oversampling factor K = 1024 ), one can draw (or evaluate) 100 samples from the truncated proposal in 0.17 seconds on a GPU (versus 10 seconds on a CPU).
For most real-world simulators, this constitutes a small fraction of the compute time required to simulate the model: e.g., for the multicompartment model, a single simulation takes approximately 30 seconds and, thus, SIR sampling from the truncated support takes up only 0.012% of the total compute time with a GPU.
For rejection sampling, the time required to draw samples from the truncated support depends on the rejection rate.
However, as long as the acceptance rate is above 0.001%, the cost of rejection sampling is still small compared to the cost of running the simulator.
6.12 Accuracy of SIR
Here, we investigate the (potential) error induced by using sampling-importance resampling (SIR).
SIR is an approximate sampling technique and does not produce exact samples for finite K .
This raises the question of how strongly the errors induced by SIR influence the results of TSNPE.
In order to investigate this, we performed three analyses: 1) We ran all benchmarking tasks with
SIR and K = 1024 and compared the results to rejection sampling.
Across all benchmark tasks, the performance of TSNPE with SIR matches the performance of TSNPE with rejection sampling (Appendix Fig. 10).
In a simple 1D toy model, we investigated how closely the samples produced by SIR match the samples produced by rejection sampling.
As can be seen in Appendix Fig.
12, the distribution of SIR samples is quite different from rejection sampling for K = 16 and K = 64 .
However, for K = 1024 , the distribution of samples from SIR very closely matches the distribution of rejection samples.
3) Finally, we investigated the performance of SIR by inspecting the effective sample size (ESS), which we computed as
[Equation]
with w i being the normalized importance weights [Kong, 1992, Djuric et al., 2003, Martino et al., 2017].
For K = 1024 , across all benchmark tasks, the ESS was on average 25.154 and was never below 2.547, i.e. it was always significantly higher than 1 (the number of resampled samples).
All of these results indicate that SIR is expected to be a useful and robust sampling method for TSNPE.
6.13 Pyloric network model
For the pyloric network model, we used the same prior, simulator, and summary statistics as previous work [Gonçalves et al., 2020, Deistler et al., 2021, Glöckler et al., 2022].
The model has a total ofparameters and 18 summary statistics.
We replaced invalid summary statistics with a value that isstandard deviations (of prior predictives) below the observation.
The experimental data [Haddad and
Marder, 2021] is also the same as used in these previous works.
6.14 Multicompartment model of a single neuron
We performed Bayesian inference in a complex model of single-neuron dynamics.
The model is the same as used in Van Geit et al.
We added observation noise with standard deviations taken from previously published measurements [Hay et al., 2011].
The prior is a uniform distribution within the same bounds as previously used [Van Geit et al., 2016].
The parameters are shown in Table 1.
The summary statistics are also the same as in Van Geit et al.
We replaced NaN values by the minimal value among prior samples minus two standard deviations of prior samples.
Several summary statistics had heavy tailed distributions, which led to very high standard deviations.
For these summary statistics, we picked the replacement value by hand.
The final values are shown in Table 2.
6.15 Choices of hyperparameters
For the results on the benchmark tasks, we picked the same hyperparameters for TSNPE as those that were used in Lueckmann et al.
[2021] for APT (called SNPE in Lueckmann et al.
For both neuroscience tasks, we used a neural spline slow (NSF) as density estimator [Durkan et al., 2019].
The hyperparameters of the NSF are the defaults from the ‘sbi’ package [Tejero-Cantero et al., 2020].
On both of these tasks, we used ϵ = 10 − 3 .
All other hyperparameters are the defaults from the ‘sbi’ package with one exception: We used a batchsize of 500 (instead of the default value 50 ).
For the pyloric network task, we ran APT with 2 atoms.
We reduced the number of atoms from the default value in the ‘sbi’ package ( 10 atoms) because a larger number of atoms increased training time.
When using 10 atoms, the training time exceeded the simulation time of the model on the pyloric network task.
With 2 atoms, the training time of APT was comparable to the training time of
TSNPE (albeit still a bit higher).
We implemented the transformation of the parameter space with the pytorch method ‘biject_to()’
[Paszke et al., 2019].
For TSNPE, we sampled from the truncated proposal with rejection sampling.
For the multicompartment model of single-neuron dynamics, we initially sampled parameters from the truncated proposal with rejection sampling.
After the third round of training, the rejection rate exceeded 99 . 9% , which rendered rejection sampling more expensive than sampling importance resampling (SIR).
We thus switched to SIR with K = 1024 for the final rounds (see Sec. 3.2).
[Table]
Fig. 1, but after running APT forrounds.
Leakage gets worse when additional rounds are run.
[Table]
can be found