Active Sequential Posterior Estimation for Sample-Efficient Simulation-Based Inference

Authors: sam  griesemer
defu  cao
zijun  cui
carolina  osorio
yan  liu
  msu
google  research
hec  montréal
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 12:59:59

Abstract: Computer simulations have long presented the exciting possibility of scientific insight into complex real-world processes. Despite the power of modern computing, however, it remains challenging to systematically perform inference under simulation models. This has led to the rise of simulation-based inference (SBI), a class of machine learning-enabled techniques for approaching inverse problems with stochastic simulators. Many such methods, however, require large numbers of simulation samples and face difficulty scaling to high-dimensional settings, often making inference prohibitive under resource-intensive simulators. To mitigate these drawbacks, we introduce active sequential neural posterior estimation (ASNPE). ASNPE brings an active learning scheme into the inference loop to estimate the utility of simulation parameter candidates to the underlying probabilistic model. The proposed acquisition scheme is easily integrated into existing posterior estimation pipelines, allowing for improved sample efficiency with low computational overhead. We further demonstrate the effectiveness of the proposed method in the travel demand calibration setting, a high-dimensional inverse problem commonly requiring computationally expensive traffic simulators. Our method outperforms well-tuned benchmarks and state-of-the-art posterior estimation methods on a large-scale real-world traffic network, as well as demonstrates a performance advantage over non-active counterparts on a suite of SBI benchmark environments.

Main paper: 
 introduction: 
 High-fidelity computer simulations have been embraced across countless scientific domains, furthering the ability to understand and predict behaviour in complex real-world systems. Modern computing architectures and flexible programming paradigms have further lowered the barrier to capturing approximate models for scientific study in silico, enabling wide-spread use of computational experiments across disciplines. However, despite the relative ease of capturing real-world generative processes programmatically, the resulting black-box programs are often difficult to leverage for inverse problems. This is a common challenge in practical applications; the simulator is often computationally expensive to evaluate, its implicit likelihood function is generally intractable, and the dimensionality of high-fidelity outputs is typically prohibitive. To address these issues, likelihoodfree inference methods have been introduced, operating under the broadly applicable assumption that no tractable likelihood function is available. Early success along this direction was achieved through easy-to-use methods like Approximate Bayesian Computation (ABC) [41, 6] , or extensions of kernel density estimation. The scale of real-world applications demands more flexible and scalable approaches, which has lead to the integration of aptly suited deep learning methods in likelihood-free † Work completed while at USC. settings. Neural network-based methods [33, 29, 17] have since been proposed, introducing greater flexibility when approximating probabilistic components (e.g., the posterior, likelihood ratio, etc) in the inference pipeline. The use of the term "simulation-based inference" (SBI) has since been colloquially embraced [11] when referring to this emerging class of techniques. SBI methods primarily leverage deep learning through their use of neural density estimators (NDE), neural network-based parametrizations of probability density functions. Common choices of NDE in practice include mixture density networks [8] and normalizing flows [40, 34] , along with popular extensions (e.g., Real NVP [12] , MAE [16] , MAF [35] , etc). Methods also vary in the probabilistic form they elect to approximate; the posterior [33] , the likelihood [36] , and the likelihood ratio [19, 13] are all common choices for well-established methods.While many SBI techniques leverage basic principles from active learning (AL), they are mostly established as a helpful heuristic for increased sample efficiency, rather than an explicit optimization over a defined acquisition function. For example, methods like Sequential Neural Posterior Estimation (SNPE) [33] boost sample efficiency (over non-sequential methods) by iteratively updating a proposal prior p(θ), steering simulator parameters to values expected to be more useful for learning the posterior under observations x o of interest.While sequential proposal updates are an effective first-order step to more informative simulation runs, there are many key factors that remain overlooked. For example, the updated proposal does not take into account the current parameters of the NDE itself, and parameter samples in the batch are drawn from the current proposal independently. This fails to fully utilize myopic AL strategies and batch optimization, leading to large amounts of simulation runs with high expected information overlap and wasted computation. To address this issue, we formulate an active learning scheme that (1) selects samples expected to target epistemic uncertainty in the underlying probabilistic model, and (2) makes acquisition evaluation simple and efficient when using any Bayesian NDE.We demonstrate the effectiveness of our method on the origin-destination (OD) calibration task. OD calibration aims to identify OD matrices that yield simulated traffic metrics that accurately reflect field-observed traffic conditions. It can be seen as a parameter tuning process, akin to model fitting in machine learning. From the machine learning perspective, OD calibration presents challenges due to the requirement of calibrating specific unique samples from observed traffic information, such as link flows, trip speeds, etc.Our contributions are summarized as follows:• Active Sequential Neural Posterior Estimation (ASNPE), an SNPE variant that incorporates active acquisition of informative simulation parameters θ to the underlying (direct) posterior estimation model, without the use of additional surrogate models. This helps to drive down uncertainty in parameters of the utilized NDE and improve sample efficiency, both of which are particularly important when interfacing with computationally costly simulation-based models.• An efficient approximation to the proposed acquisition function above, along with a means of training Bayesian flow-based generative models for density estimation during posterior approximation (both with open source implementations 3 ). Leveraging this class of models enables direct uncertainty quantification in the acquisition function, and is more flexible, efficient to train, and scalable to high-dimensional data than many traditional Bayesian model choices (e.g., Gaussian processes).• A Bayesian formulation of the OD calibration problem and coupled statistical framework for performing sequential likelihood-free inference with neural posterior estimation methods. We show ASNPE outperforms baseline methods across a wide variety of simulation scenarios on a large-scale traffic network. We also evaluate ASNPE on three broader SBI benchmark environments and find it acheives a performance advantage over non-active counterparts.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract and introduction mention the proposed method and its demonstrated qualities, and focus almost exclusively on related methods in both the general simulation-based inference and demand calibration spaces.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
neural posterior estimation: 
 Given observational data of interest x o and a prior p(θ), we want to carry out statistical inference to approximate the posterior p(θ|x = x o ) under the model p(x|θ). We assume p(x|θ) is defined implicitly via a simulation-based model, where direct evaluation of p(x|θ) is not possible but samples x ∼ p(x|θ) can be drawn. Conventional Bayesian inference is thus not accessible in this setting, and we instead look to approximate the posterior using N generated pairs {(θ i , x i )} N i=1 . Neural Posterior Estimation (NPE) methods attempt to approximate the posterior directly with a neural density estimator q ϕ (θ|x) trained on samples {(θ i , x i )} N i=1 , where θ i ∼ p(θ) and x i ∼ p(x|θ i ), by minimizing the lossL(ϕ) = E θ∼p(θ) E x∼p(x|θ) [-log q ϕ (θ|x)]for learnable parameters ϕ. Provided q ϕ is sufficiently expressive, q ϕ (θ|x) will converge to the true posterior p(θ|x) as N → ∞.Sequential Neural Posterior Estimation (SNPE) methods break up the NPE process across several iterations, and can improve sample efficiency by leveraging the fact that p(θ|x = x o ) is often far more narrow than p(θ). While accurately representing p(θ|x) for any x ∈ X is ideal (where X is the simulation output space), doing so can require prohibitively large simulation samples, including outputs from parameters with low posterior density under x o . To combat this, SNPE methods draw θ expected to be more informative about p(θ|x o ) by using a successively updated proposal distribution p(θ) which approximates p(θ|x = x o ). Training the NDE q ϕ on samples θ ∼ p(θ) when p is not the true prior, however, will cause it to converge instead to the proposal posterior p(θ|x) = p(θ|x) p(θ)p(x) p(x)p(θ) , rather than the true posterior (as shown in [33] ). Existing SNPE methods correct for this in different ways: SNPE-A [33] trains q ϕ (θ|x) to approximate p(θ|x) during each round and employs importance reweighting afterward, SNPE-B [29] directly minimizes an importance weighted loss, and SNPE-C [17] (also known as Automatic Posterior Transformation, or APT) maximizes an estimated proposal posterior that easily transforms to the true posterior.Bayesian active learning is a selective data-labeling technique commonly employed in data-scarce learning environments. Active learning assesses the strength of candidate data points using a so-called acquisition function, often capturing some notion of expected utility to the underlying model given the currently available data. Given an acquisition function α, computing the next best point to label includes optimization of α over a domain of as yet unlabeled points U : x * = argmax x∈U α(x, p(θ|D)), where x are input data points and p(θ|D) is the posterior of the Bayesian model parameters θ given the current training dataset, i.e., the distribution over parameters after training the model. In modern Bayesian deep learning pipelines, this model is often a Bayesian neural network [15, 21] . Many acquisition functions used in practice are extensions or approximations of expected information gain (EIG) [20] EQUATIONwhere H[•|•] is conditional entropy, and y are input labels.Several existing works explore the use of Bayesian optimization (BO) in the likelihood-free inference setting. [18, 22] employ Gaussian processes (GPs) as surrogate models for the discrepancy as a function of θ, and select parameter candidates by optimizing this surrogate with BO. GPs are also used as a surrogate by [30] to represent the proposal distribution in MCMC ABC. [4] further extends these principles to deep Gaussian processes and leverage these models as surrogate likelihoods. In this work, we explicitly avoid the use of likelihood surrogates and aim to leverage only the approximate posterior NDE model, with the express intent of subverting additional computational overhead and enabling the use of powerful NDEs (e.g., flow-based generative models). to generate B pairs (θ i , x i ) for training the approximate posterior q ϕ . The learned posterior is then conditioned by the target observation x o , producing the next round's proposal p(θ) = q ϕ (θ|x o ).OD calibration is an important task for transportation agencies and practitioners who develop traffic simulation models of road networks and use them to inform a variety of planning and operational decisions. Calibrating the input parameters of these simulators is an important offline optimization problem that agencies must face on a regular basis (e.g., when new traffic data are made available, when changes to the road network have occurred, etc). These simulators are often computationally expensive to evaluate, however, and highlight the practical importance of developing sample efficient calibration methods.Previous works primarily approach OD calibration using general-purpose simulation-based optimization (SO) algorithms, such as Simultaneous Perturbation Stochastic Approximation (SPSA) methods [5, 47, 25, 7] and genetic algorithms [24, 43, 47] . These general-purpose SO algorithms tend to require large numbers of simulation evaluations, which can be computationally costly. To address this issue, recent extensions of SPSA have been proposed [10, 27, 1, 45] . Analytic metamodels have also been considered and shown to reduce the need for large numbers of function evaluations [31, 32, 48, 3] .Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: The properties and limitations of the proposed method are discussed in Section 3, and are broken down in further detail when analyzing empirical behavior on reported results in Section 4.3.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methodology: 
 3.1 Active learning for SNPE SNPE methods produce iteratively updated proposal priors p(r+1) (θ) ≈ p(θ|x = x o ) from q ϕ (θ|x o , D (r) ) at each round r in the inference process, where D (r) is the accumulated dataset {(θ i , x i )} rB i=1 by round r and B is the number of newly collected pairs per round. These sequentially updated proposals provide a means of drawing parameter values θ (r) ∼ p(r) (θ) that are increasingly useful (i.e., high likelihood) to the posterior estimate of interest p(θ|x o ). As such, SNPE offers a clear possible benefit for improved sample efficiency over non-sequential NPE, which cannot explicitly sample new values at expected high likelihood regions under x o . Despite this, it's difficult to quantify theh value of any particular θ ∼ p(r+1) (θ) and whether it's worth the computational cost to obtain x ∼ p(x|θ) with respect to its utility to the underlying NDE q ϕ (θ|x, D (r) ).In high-cost simulation environments, we want to take every measure to sample only at highly informative regions of the parameter space to improve our estimate q ϕ (θ|x o ) of p(θ|x = x o ). This entails a more principled analysis of candidate simulation parameters θ before investing in the simulation run x ∼ p(x|θ). In an attempt to quantify the prospective impact of any particular θ on our posterior estimate, we look to Bayesian active learning, and acquisition functions such as EIG. EIG considers the reduction in uncertainty of model parameters under the inclusion of new data in expectation over the predictive posterior. Adapting Eq. (1) to our NPE context givesargmax θ H(ϕ|D) -E x∼p(x|θ,D) H[ϕ|D ∪ {(θ, x)}] ,which seeks to drive down uncertainty in NDE parameters ϕ by optimizing for θ with simulation outputs x ∼ p(x|θ, D) expected to be most informative. Unfortunately, EIG and related approximations require p(x|θ, D), which we cannot evaluate nor do we directly approximate in the NPE setting. This makes it considerably more difficult to quantify the utility of candidate θ values, as we have no direct means of sampling likely simulation outputs.Instead of relying on the marginal distribution p(x|θ, D), we seek instead to capture uncertainty across different parameterizations of the NDE. Broadly speaking, we want to simulate θ expected to be informative to our NDE model, reducing epistemic uncertainty as measured by p(ϕ|D) and further elucidating parameter sets ϕ likely to explain the probabilistic mapping from simulation outputs x to parameter inputs θ. Given that q ϕ models this relationship as the conditional distribution q ϕ (θ|x), we consider the uncertainty over distributional estimates induced by p(ϕ|D). This allows for the targeting of epistemic uncertainty in the NDE model according to how that uncertainty appears across feasible target posterior forms q ϕ (θ|x). More precisely, for some divergence measure D(•||•), we represent this distributional uncertainty asEQUATIONwhere p(θ|x, ϕ) is the corrected posterior produced by q ϕ (θ|x) (see Section Eq. ( 2) provides a basis for evaluating uncertainty in an NDE without relying on access to or approximations of the likelihood p(x|θ). Given the posterior estimation problem at hand, we're particularly interested in how to select simulation parameters θ expected to reduce H D at or around x o . Here we take inspiration from [23] , who seek to drive down uncertainty at θ with noisy estimates of the log joint probability (albeit in a context where likelihoods p(x|θ) are available). While H D provides a measure of distributional uncertainty, we can target specific θ whose assigned likelihood is widely disagreed upon across draws of ϕ ∼ p(ϕ|D):EQUATION)While optimization of Eq. ( 3) over all θ in the prior support may be ideal, this is computationally infeasible given the posterior estimates from all parameters ϕ ∼ ϕ|D required in expectation. Additionally, note that Eq. (3) does not explicitly account for relative likelihoods of θ under the posterior or available approximations, possibly leading to θ * with high uncertainty under x o but with low-likelihood under p(θ|x o ) in expectation. We account for this explicitly during integration with specific SNPE approaches, as seen in the section below. (θ|x o ) Let D (0) = {} Let p(θ) = p(θ) for r ∈ [1, . . . , R] do Draw N samples {θ i } 1:N ∼ p(θ) Sort {θ i } 1:N by expected divergence (Eq (4)), select top-B {θ i } 1:B for b ∈ [1, . . . , B] do Simulate x b ∼ p(x|θ b ) Set D (r-1) = D (r-1) ∪ {(θ b , x b )} end for Set D (r) = D (r-1) Train NDE q ϕ on D (r) : ϕ * = argmin ϕ -(θi,xi)∈D (r) log qϕ (θ i |x i ) Let p(θ) = q ϕ (θ|x o ) end forIn order to tractably approximate Eq. ( 3), we impose two additional restrictions to bring the parameter acquisition into the SNPE loop:1. Require the NDE be updated according to APT [17] , i.e., trained via maximum likelihood onL(ϕ) = - N i=1 log qϕ (θ i |x), where qϕ (θ|x) = q ϕ (θ|x) p(θ) p(θ) 1 Z(x, ϕ) ,and by Proposition 1 of [33] ensures q ϕ (θ|x) → p(θ|x) as N → ∞ without requiring posthoc updates to the NDE's distributional estimate. This allows the model parameter posterior p(ϕ|D) to be used directly in the contexts of Eq. ( 2) and Eq. ( 3), whereas otherwise the corrective terms involved would need to be accounted for explicitly. 2. To account for the likelihood of θ under the posterior estimate as captured by the proposal prior p(θ) ≈ p(θ|x = x o ) in a given round of SNPE, we adjust Eq. (3):EQUATIONFurther, in practice we approximate this by optimizing Eq. ( 4) over samples θ ∼ p(r) (θ), straightforwardly integrating the acquisition mechanism into the standard SNPE pipeline. Round-wise proposals p(r) (θ) = q ϕ (θ|x) are set after N = rB samples are collected (for round r with B samples collected per round), and q ϕ (θ|x) → p(θ|x) as N → ∞. All roundwise proposal distributions share the support of the prior p(θ), which itself is established as having support over the entire parameter domain of interest Θ. Thus, optimizing α over a sample of size N drawn from a proposal distribution p(r) (θ) at any round r recovers the true optimum of α as N → ∞; each proposal's support connects back to the prior's support, which covers Θ. As a result, at each round a fixed sample size N drawn from the proposal can be used to approximate the acquisition maxima, while additionally adhering to the round-wise proposal sampling required to ensure q ϕ (θ|x) → p(θ|x). Refer to Section A.2 for additional discussion on the functional form of the acquisition function.p(ϕ|D), neural network-based NDEs (such as flow-based generative models or mixture density networks) can be trained via MC-dropout [21, 14] . See additional details regarding consistent sampling and log probability evaluation in MAFs under MC dropout in Appendix C.Altogether, this constitutes the ASNPE method, which is more succinctly described in Algorithm 3.3.See Figure 1 for a visual depiction of this process.We now position OD calibration as a Bayesian inference problem, with a posterior density of interest to be approximated by SNPE methods. During a time interval of interest [t s , t e ] on a traffic network G, we consider a single OD matrix d = {d z } z∈Z , where d z represents the expected travel demand for the origin-destination pair z. Z is the set of OD pair indices, i.e., Z = {1, 2, . . . , Z}, for all pairs of interest on G. OD pairs are typically defined between elements in a fixed set of Traffic Assignment Zones (TAZs) whose size may not be uniform due to variable demand density; see Figure 7 for zones drawn on two candidate networks. Figure 2 loosely depicts the acquisition pipeline for traffic data, corresponding the collection process shown in Figure 1 . EQUATIONi.e., marginalizing over all possible latent trajectories z. As is typical in simulation-based inference settings, this integral is intractable for simulators of sufficient complexity.some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
experimental results: 
 We explore the performance of the proposed ASNPE method in the context of OD calibration on a challenging real-world traffic network. Our goal here is to 1) compare the general purpose utility of our approach in complex settings against tuned benchmark methods, and 2) verify ASNPE's candidacy as a sample efficient posterior estimation tool in high-dimensional, data-scarce settings. These objectives are directly in line with the needs of practitioners, both in the urban mobility community and broadly across scientific disciplines.We conducted a case study on the large-scale regional Munich network seen in [37] . The Munich network includes 5329 configurable origin-destination pairs (constituting simulation input), as 1 : RMSNE scores on the Munich traffic network, as described in Section 4.1. Note that methods like SPSA (poor convergence aside) can produce RMSNE scores larger than the reported setting prior due to noise in the starting sample. The "setting prior" value is an average RMSNE score over many θ draws from the shifted prior. Reported errors are empirical standard deviations computed over the five trial runs. [28] . Experimentation details, metrics, and associated plots can be found in Appendix B.well as 507 detector locations (positions of reported output traffic flows), resulting in a highly underdetermined system.We build and evaluate a number of synthetic demand scenarios, following an established framework for fair evaluation of urban demand calibration methods ( [2, 39, 9] ). Each of the test demand scenarios are constructed from combinations of the following factors: Time interval: a time interval of interest is specified through which to simulate traffic flows on the network. Prior ODs are chosen to reflect real-world traffic patterns for the affiliated times. We evaluate peak morning demand for hour-long intervals at 5:00am-6:00am and 8:00am-9:00am.Congestion level: within a given time interval, we can further control the level of traffic congestion exposed during the hour. The distribution of frequencies present in the starting ODs plays a critical role in determining route time across the traffic network. Here we define two congestion levels, "A" and "B," to reflect different average frequencies assigned to OD pairs. Here we use a truncated normal distribution (lower bound at 0) to sample OD counts with varying means and variance:(1) (µ = 5, σ = 25) for hours 5:00-6:00, congestion level A, (2) (µ = 10, σ = 50) for hours 5:00-6:00, congestion level B, (3) (µ = 25, σ = 50) for hours 8:00-9:00, congestion level A, (4) (µ = 50, σ = 100) for hours 8:00-9:00, congestion level B.Prior bias and noise: under each time interval and congestion level, we further perturb the generated OD matrices to represent realistic variance found in real-world sampling of traffic observations. Here we use the following noise model, mirroring that of [39] : x c = (r + q × δ) × d, where δ ∼ N (0, 1 3 ). We then formulate two perturbed settings: 1) Prior I: r = 0.6, q = 0.3, and 2) Prior II: r = 0.75, q = 0.45. Prior I constitutes a heavily under-congested estimate with relatively little added noise, while Prior II is less biased from the true OD but noisier. Both priors represent underestimations of the true demand, reflecting the fact that most prior ODs from real-world settings are constructed from historic travel demand observations. The eight synthetic combinations constitute starting ODs/priors that span a variety of different settings important for real-world urban demand calibration tasks. Each synthetic setting yields a particular prior OD estimate d, which is then used to construct a prior p(d; d). A fixed sample is drawn from p(d; d) and passed through the open-source traffic simulator Simulation of Urban MObility (SUMO) [26] to generate an associated "true" network flow x o .We evaluate the effectiveness of the proposed solution by comparing against available SOTA benchmarks commonly employed in the OD calibration space: Simultaneous Perturbation Stochastic Approximation (SPSA) [42] and principal component (PC)-based SPSA, or PC-SPSA [37] . SPSA is a widely employed algorithm for travel demand calibration, and PC-SPSA is an effective extension that optimizes over parameters in a lower-dimensional subspace, as defined by the principal components of computed travel demand history matrix. Both of these methods are conventional optimization-based methods, and do not leverage neural networks. Additionally, these methods in their canonical form cannot be parallelized, requiring serial simulation evaluations across each iteration.For NPE-based approaches, we evaluate our proposed method ASNPE alongside SNPE-C (or APT) [17] and Approximate Bayesian Computation (ABC) [41, 6] . ABC serves primarily as a less sophisticated baseline that reflects early approaches to likelihood-free inference, and it typically faces difficulty scaling and is far less flexible compared to its NPE counterparts.For all of the eight scenario priors p(d; d), each calibration method is ran for a maximum of 128 SUMO simulations, seeking to recover x o . The root mean squared normalized error (RMSNE) is recorded for each method's simulation horizon, as used in [39] (see also Appendix A.3). To account for the stochasticity across evaluations, we report RMSNE averaged of five repeated simulation runs. See Table 1 for reported values for each method across each of the eight synthetic scenarios, as well as paired prior plots in Figure 3 .ASNPE outperforms all other methods across most explored settings: as can be seen in Table 1 , our method outperforms both the well-tuned PC-SPSA method commonly employed by the urban mobility community, as well as popular simulation-based inference (SBI) methods like SNPE, across almost all of the explored settings. In general, PC-SPSA tends to quickly converge but demonstrates a limited ability to further improve beyond the first 10-20 encountered simulations. Both SBI methods tend to make steady improvements throughout the entire trial, however, albeit often doing so more slowly in the first 20-40 simulations than PC-SPSA. This is primarily due to the limited feedback SNPE/ASNPE receive comparatively, only incorporating new simulation data in batches (in this case, every 32 simulation draws).Additionally, ASNPE reliably reaches better RMSNE scores than SNPE with fewer simulations, as well as Approximate Bayesian Computation (MC-ABC). This can be seen as early as the first NDE update, before which the two methods encounter the same (seeded) simulation samples. This also empirically supports our central methodological contribution, i.e., optimization over informative simulation parameters can more efficiently improve the accuracy of the inferred posterior estimate.ASNPE is outperformed in some cases: ASNPE is outperformed by PC-SPSA in two of our explored settings (under Prior I, Hours 8:00-9:00). While we wouldn't expect a single method to be the best choice for all variations in such a high-dimensional setting, this particular scenario serves as an opportunity to better understand possible failure modes of the proposed method.As alluded to above, ASNPE updates its internal model only after a batch of simulation samples is generated, whereas PC-SPSA adjusts its parameters after each simulation run. While generating samples in batches can be beneficial (and is often necessary) for early stability of ASNPE, it can mean informative simulation data is incorporated later in the trial. This explains the occasional gap that opens up between SBI methods and PC-SPSA in the first 30 simulations, only after which is ASNPE/SNPE able to incorporate the samples to improve its posterior estimate. Note, however, that most of the early advantage PC-SPSA may have over ASNPE is dwarfed by the ability to obtain its simulation draws in parallel, whereas PC-SPSA must run simulations serially. This allows for larger, more stable improvements in less time, which can be seen in subplot (b) of Figure 3 .We additionally report results on several common SBI benchmark environments, and compare against the performance of (non-active) SNPE. Numerical results can be found in Table 2 , along with plots and more details in Appendix B. These additional results demonstrate the wider applicability of our method beyond the travel demand calibration task. 
conclusion: 
 In this paper, we introduced Active Sequential Neural Posterior Estimation (ASNPE), an SNPE variant that actively incorporates informative simulation parameters θ to drive down epistemic uncertainty in the neural density estimator and improve sample efficiency for high-quality estimates. We evaluate this method on a complex, high-dimensional problem in urban demand calibration, and show it reliably outperforms available benchmark methods across a variety of scenarios with variable bias and noise. We additionally provide results on several common SBI benchmark environments, and find ASNPE is capable of outperforming state-of-the-art SNPE methods on key posterior approximation metrics. This aligns with the expected divergence term introduced in Eq. 2 when we let the measure D = D KL .Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: We provide links to the two public code repositories (Python packages) that were written to implement the introduced method and evaluate it on several tasks. See Section C for full details.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Experimental setup is discussed extensively in Section 4.1, and relevant hyperparameters and used compute resources are detailed in Section D.1.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: Error bars in reported plots and tabular data in Section 4 are provided and explicitly mention the method of calculation.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The introduced method does not pose a high risk for misuse. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Simulation models, network data, and employed baseline methods all cite their associated original works. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The connection to mutual information helps to position our motivation for the acquisition function ultimately introduced in Eq. 4. That is, by seeking to drive down disagreement between "marginal" and "component" posteriors p(θ|x, D) and p(θ|x, ϕ), respectively, we are attempting to realize the information we expect θ to tell us about our NDE parameters ϕ, thereby minimizing the remaining mutual information between the two.The family of functions α(θ, p(ϕ|D)) = p(θ)(E ϕ|D [. . . ]) λ under parameter λ constitutes valid choices for the acquisition function for any λ, facilitating different levels of emphasis on uncertainties at values of θ relative to their likelihoods under p. While several values of λ may be justifiable, the choice to use λ = 1, implicit in Eq. 4, intuitively captures a desirable balance in the relationship between uncertainties and likelihoods of θ.In particular, under level sets α(•, p(ϕ|D)) = z (where p(ϕ|D) is held constant), as likelihoods p(θ) decrease by a factor of n, the average deviation between p(θ|x, D) and p(θ|x, ϕ) need only increase by a factor of √ n, i.e., changes in uncertainty are sub-linear in the likelihood ratio. With the introduction of variable λ, this factor generalizes to n 1/(2λ) , and may require additional measures to balance the resulting sensitivity between the terms. We find that λ = 1 is a natural choice that reasonably captures the desire to explore potentially unlikely parameters with high uncertainties without ignoring them (e.g., λ → 0) or relying too heavily on them (e.g., λ → ∞).The root mean squared normalized error (RMSNE) for a simulated output x with respect to an observational reference x o , as used in [39] , is defined asRMSNE = n n i=1 (x (i) -x (i) o ) 2 n i=1 x (i) o, where there n is the number of observed segment flows, and both x o and x are n-dimensional vectors.In order to appeal to the general utility of our proposed method, we provide additional experimental results between ASNPE and SNPE-C [17] on three common SBI benchmark tasks: SLCP distractors, Bernoulli GLM, and Gaussian Mixture. Each of these settings corresponds to a reproducible task environment from [28] , and the corresponding implementation in the sbibm Python package is used to run our experiments.We additionally evaluated our method on these tasks using metrics beyond RMSNE, the primary metric used for the travel demand calibration case study. These include classifier 2-sample tests (C2ST), maximum mean discrepancy (MMD), the posterior median distance (median L 2 norm between simulated samples θ i ∼ p(θ|x o ) → x i ∼ p(x|θ i ) and the observation x o ), and the posterior mean error (normalized absolute error between the true posterior mean and the approximate posterior mean). Full details for each of these tasks and metrics can be found in [28] .We evaluate both methods over medium-size sample horizons: 4 rounds with 256 samples per round, for a total of 1024 simulation samples. Note that this is eight times larger than the sample sizes collected for the trials on the travel demand task. For reference, 256 samples in our (non-parallelized) SUMO environment takes ~6 hours, whereas 256 samples from the SLCP simulator takes ~10 seconds on our hardware.Trials were repeated five times for each method, and the average score and standard deviation for each metric over these trials are shown in Figures 4, 5, 6 . Note that smaller values are better for each metric (C2ST ranges between 0.5 -1.0). While these simulation horizons are relatively small, we find that, by the final round, ASNPE tends to outperform SNPE across most settings and on most metrics. In particular, ASNPE outperforms SNPE on C2ST and MMD across all settings, along with the distance-based metrics on all but the median distance for Bernoulli GLM and mean error for SLCP Distractors.While SNPE-C is a state-of-the-art benchmark method, comparing against it also constitutes an ablation test for ASNPE's acquisition component. Although parameter sets are chosen differently and the underlying NDE varies (minimally to accommodate the need to approximate p(ϕ|D)) across methods, the sequential inference procedures are otherwise identical. These results help to isolate and identify the contribution of the active learning scheme across a wider range of tasks and metrics for the overarching goal of producing accurate posterior approximations holistically (i.e., not just well-calibrated point estimates). In the spirit of reproducibility and in the hopes our code may be of use for follow-up works, we provide the following Python packages:1. seqinf package: this package includes a full implementation of ASNPE and convenient abstractions around the popular sbi [44] and sbibm [28] Python packages for general sequential inference pipelines. This package also includes an implementation of the masked autoregressive flow (MAF) with consistent MC-dropout that was used as an NDE for all experiments. Available at https://github.com/samgriesemer/seqinf.2. sumo_cal package: short for SUMO calibration, this package includes many general programmatic utilities for calibrating traffic models using results from the SUMO traffic simulator [26] with a high-level Pythonic interface, including collecting results from several runs, running in multi-threaded contexts, employing standardized configuration, etc. See Figure 11 for a snapshot of the SUMO interaction scheme. Available at https://github. com/samgriesemer/sumo_cal.• SPSA: SPSA (Simultaneous Perturbation Stochastic approximation) is an optimization algorithm for systems with multiple unknown parameters, which can be used for large-scale models and various applications. It can find global minima, like simulated annealing [46] . SPSA works by approximating the gradient using only two measurements of the objective function with gradients, making it scalable for high-dimensional problems [42] .• PC-SPSA: PC-SPSA is proposed to address fundamental scalability issues with SPSA. This is because SPSA searches for the optimal solution in a high-dimensional space without considering the structural relationships among the variables. PC-SPSA combines SPSA with principal components analysis (PCA) to reduce the problem dimensionality and limit the search noise. PCA captures the structural patterns from historical estimates and projects them onto a lower-dimensional space, where SPSA can perform more efficiently and effectively [39] .Specifically, we implement the SPSA and PC-SPSA algorithms according to [38] and associated open-source implementations 4 . In addition, we employ the so-called Method-6, titled "Spatial, Temporal, and Day-to-Day Correlation," as detailed in [38] . This provides a means of systematically generating needed historical data for PCA, and according to the original work constitutes the most robust and optimal solution among the proposed variants.Here we include a brief discussion on the implications of the hyperparameters found in ASNPE, as well as the settings used in our experiments. Keeping the total number of simulation samples constant,1. The number of rounds R dictates how many times we update the proposal distribution over the course of the simulation horizon. Increasing this value can enable quicker feedback to the NDE, requiring fewer simulation samples before re-training the model. When the prior is well-calibrated and simulation samples are representative of the observational data, this can have a positive compounding effect that boosts the rate of convergence to the desired posterior. However, for larger R the resulting batch sizes are smaller and the NDE receives noisier updates, which can have the opposite effect and hurt early performance when the prior is poor.2. The number of selected samples B per round is directly determined by R when the total number of simulations is held constant, and thus the above effects apply here. Figure 7 : Depiction of traffic networks in the SUMO simulator. (a) depicts a relatively small synthetic network for reference, with approximately 500 configurable OD pairs. (b) shows the larger Munich network used in our reported experimental settings, with an order of magnitude more configurable OD pairs than the synthetic network.3. The number of proposal samples N per round governs the size of the parameter candidate pool over which the acquisition function is evaluated. Increasing this value allows us to consider more potentially relevant candidates under p(θ), and can thus increase the quality of the resulting B-sized batch. Given the acquisition function can be evaluated over this pool very efficiently (i.e., as a batched inference step through the NDE model), one can practically scale this up arbitrarily to increase the sample coverage over the proposal support (but with decreasing marginal utility).The following are additional hyperparameter details for the evaluated methods:1. In the SNPE loop: total number of rounds R (4 in reported experiments), round-wise sample size N (between 256-512), round-wise selection size B (32 in reported experiments).2. Neural Density Estimator (NDE) model: our model architecture (used for both SNPE and ASNPE) is a masked autoregressive flow with 5 transform layers, each with masked feedforward blocks containing 50 hidden units, and trained with a (consistent) MC-dropout setting of 0.25. When collecting distributional estimates as described in Eq 4, we used 100 weight samples ϕ ∼ p(ϕ|D) (as generally recommended in [21] ).3. PC-SPSA: this method uses PCA to optimize OD estimates in a lower-dimensional subspace of the 5329-dimensional parameter space. The number of the principal components is chosen such that 95% of the variance is recovered in the provided historical OD estimate (which is further dictated by the choice of prior distribution). The number of PCs used by this method across the many explored settings presented in Section 4.1 varies from 99-117.All experimentation code is written in Python 3.11. To run experiments, we employed our own hardward locally, which is an linux-based machine running an Intel(R) Core(TM) i9-10900X CPU @ 3.70GHz 64GB memory, and NVIDIA GeForce RTX 2080 Ti.Figures 8, 9, and 10 are plots of calibration horizons for the remaining settings of the demand calibration task not shown in the main paper (which highlighted the first scenario, Prior I, Hours 5:00-6:00, Congestion level A). The associated RMSNE scores can all be found in Table 1 in the main paper body. Figure 11 provides a more detailed look at the programmatic interaction with the SUMO simulator.Figure 12 and figure 13 provide additional plots of the posterior approximation for the primary travel calibration task.Figure 11 : This diagram provides a more detailed look at some of the internal details behind the preparation of input to and the transformation of output from the simulator. An input OD vector θ, drawn from some proposal distribution in the outer method context, 1) "enters" the diagram at the left, 2) is transformed into a suitable representation for SUMO, 3) combined with additional configuration and network files, and 4) run through the SUMO simulator, after which the output is parsed to produce the resulting segment flows x under demand θ. Figure 13 : Pairwise density plots of a 20-dimensional slice of the "empirical likelihood" under the final ASNPE posterior on the Prior I, Hours 5:00-6:00, Congestion level A scenario. Figure 12 shows the approximate posterior q ϕ (θ|x o ), whereas here we draw samples θ i ∼ q ϕ (θ|x o ) and feed them back through the simulator {θ i } 1:N → p(x|θ) to visualize the resulting data space. The target observational data point x o is shown on top these pair plots as a red "plus", which provides a visual anchor for how well calibrated the posterior is around the observational data point of interest.NeurIPS Paper Checklisthttps://github.com/samgriesemer/seqinfhttps://github.com/LastStriker11/calibration-modeling 


Decision: The reviewers recommend acceptance (8-6-4-5-6-7). The paper introduces ASNPE, a new method for simulation-based inference based on active learning. The authors-reviewers discussion has been very constructive and has led to a number of improvements to the paper.  In particular, the reviewers noted that ASNPE was a generic improvement over SNPE and, as such, should be evaluated on common SBI benchmarks instead of focusing only on the task of urban demand calibration. The authors have provided convincing results SBI benchmarks during the rebuttal, which has been largely appreciated by the reviewers. The borderline rejection from Reviewer JwDn is not an obstacle to acceptance, as most the major concerns have been addressed, despite the lack of acknowledgment from the reviewer.

Given the scope of the paper and the potential of the proposed approach beyond the traffic task, the authors should consider the opportunity to reframe the paper and its title to better reflect the generic contribution of ASNPE. Finally, the authors are asked to implement the changes and clarifications discussed with the reviewers in the final version of the paper.

Reviews: review: paper_summary: paper_summary: The paper introduces ASNPE, a modification of sequential neural posterior estimation (SNPE) that uses active learning to determine the set of most informative simulation parameters. The method is benchmarked in a synthetic scenarios based on a real-world traffic network and outperforms domain-specific optimization schemes as well as simulation-based inference methods.
strength_weakness: strength_weakness: Major:
- While the reported experimental setup is exciting, the proposed method is tested only in a single setting. Since ASNPE seems to be a promising general improvement of SNPE-C, it would have substantially benefitted the work to demonstrate its usefulness in a wider range of experimental settings (e.g., by adapting tasks from [1] as a compact first experiment). I acknowledge that this is likely not feasible in the limited rebuttal period, but I believe it would make the paper much more informative for the broader SBI audience.

Minor:
- The paper directly jumps from theory to experimental setup and the results without mentioning the implementation of ASNPE, but this would be quite informative for the reader.
- Section 4.2 does not feature all benchmark comparisons - MC-ABC and SNPE suddenly appear in the results and I expected to find more information concerning MC-ABC at least in Appendix C. As a related minor comment, I suppose the benchmark uses APT / SNPE-C for SNPE, but it would be helpful to state this explicitly.
- Lastly, it would have been informative to see ablation studies (potentially in a computationally less demanding setting) or at least a theoretical discussion regarding the impact of the hyperparameters R, N, and B on the performance of ASNPE.

[1] Lueckmann, J. M., Boelts, J., Greenberg, D., Goncalves, P., & Macke, J. (2021). Benchmarking simulation-based inference. AISTATS.
questions: questions: - In the training setup of SNPE and ASNPE, the number of simulations is not clear to me. The authors state that they perform R=3 round with selection size B=32, which should result in 96 simulations seen by the neural nets. How does this match the 128 sample simulation horizon?
- I (and maybe also future readers) would be interested in the authors opinion on the reasons for the clear gap between SNPE and ASNPE in the prior 1 but not prior 2 setting.
limitations: limitations: The explicit analysis of failure modes (which of course, are to be expected in any method comparison since no method always performs the best) is a big plus. As stated above, this does not incorporate a comparison between SNPE and ASNPE, which would be interesting for the wider SBI community.
paper_summary: paper_summary: A recent class of methods that have shown to perform well at simulation-based inference are based on modeling the posterior density as a neural network, using mixture density networks, normalizing flows, or other popular architectures. With enough data, these methods tend to produce salient estimates of the posterior density on parameters conditional on data (or summary statistics). With any simulation-based inference method, proposing parameters efficiently is a relevant concern because the posterior can have most of its mass in a tiny region of the support of the prior. Standard SNPE constructs sequential proposal distributions from the current posterior approximation and uses importance reweighting, which is a simple heuristic but not optimal. The main contribution in this paper is an active learning method which filters potential parameter values by ones that are sufficiently large in terms of an acquisition function, such as expected information gain. The authors come up with a reasonable acquisition function that can be approximated via Monte Carlo. The ASNPE algorithm is then applied to an origin-destination calibration problem, and performs well compared to competing approaches under a budget of simulator evaluations.
strength_weakness: strength_weakness: I would be interested to see more experiments/applications than just the Bayesian O-D calibration example. Particularly, I'd like to see how this method compares to competing approaches on a standard "benchmark" in likelihood-free/simulation-based inference, i.e. Lotka-Volterra, a queue model, Heston model, a Gaussian toy model as in SNL, etc. This particular simulator model seems to have an extremely high-dimensional parameter space relative to the budget of simulations used for the experiments, and I am curious if it is this particular setting in which this method shines compared to others. I.e. the benefit is clear when the simulation budget is 128, but if it is on the order of 10^4 or 10^5, does it vanish?
questions: questions: 1.) I am curious how much the computational cost increases when this methodology is used compared to the simple sequential updating in SNPE. Of course, when the simulator is extremely expensive, the cost of evaluating the acquisition function will be small in comparison, but would be it still be recommended if the simulator is not too onerous.

2.) On a similar note, I would be interested for the authors to carve out and add a little exposition describing exactly the class of SBI problems that they posit would have the largest boon from this active learning approach when compared to the standard sequential updating in SNPE. What types of problems are most rewarded by the acquisition function filtering step, and on which class of problems is it less rewarding?

3.) I may have missed it, but is it obvious that q_\phi still converges to the true posterior in the presence of the filtering step? In the presence of the filtering step, the proposal is no longer just \tilde{p}(\theta), but instead it is a transformed distribution essentially weighting \tilde{p}(\theta) by the acquisition function. Shouldn't this need to be considered when updating the NDE in order to ensure q_\phi really converges to the posterior?
limitations: limitations: The authors provided an honest assessment of the performance of their method in comparison with other methods in the experiments section. I believe that potential negative societal impact is not a concern here. As I mentioned earlier, I would be interested to know if the authors believe there are limitations in the context of which SBI problems their method is not as well-suited for.
paper_summary: paper_summary: This paper proposes an approach to performing neural simulation-based inference – specifically, sequential neural posterior estimation – in a simulation-efficient manner for complex and expensive simulation models. The idea is to use active sampling to sequentially generate datapoints from the simulator and to train the neural density estimator using batches of datapoints generated this way. The authors test their approach experimentally on an origin-destination estimation problem in an urban mobility simulator, comparing against other common simulation-based Bayesian inference methods ("vanilla" SNPE and ABC) and against two alternatives that are commonly used in the urban mobility modelling literature ((PC-)SPSA).
strength_weakness: strength_weakness: The two main weaknesses in my reading are:

- Sorry if I missed something, but clarity was lacking in one key respect for me, namely what exactly the relationship was between $q_{\phi}(\theta \mid x)$, $p(\theta \mid x, \phi)$, $\tilde{q}_{x,\phi}(\theta)$ on Line 178, and `q_{\phi,x}(\theta)` (I had to do this last one `in this format` rather than in $math\ mode$, sorry, because it wasn't rendering properly in $math\ mode$ and I couldn't figure out why) on Line 178. Is `q_{\phi}(\theta \mid x)` (now $math\ mode$ isn't working here either...) the same as `q_{\phi,x}(\theta)`? And `\tilde{q}_{x,\phi}(\theta)` the same as `p(\theta \mid x, \phi)`? If so can you use consistent notation throughout for clarity? And if not could you please clarify what the difference between all of these guys are?

- The empirical evaluation was quite weak in my opinion. The authors present a new approach to training SNPE models and it's great that they've applied it to a complex simulator to get some idea of whether it works on real practical simulators of interest, but I think it's also important to test a new method on some simpler benchmark models for which a good (even if approximate) ground-truth posterior can be obtained. This will allow the authors to properly test whether their method produces good posteriors, which isn't really tested at the moment by looking purely at RMSNE values. In general I would like to see something that demonstrates how well the proposed Bayesian pipeline can actually estimate the full posterior distribution, since this is ultimately what it's trying to do. I think the paper needs revision to include such experiments, even if the detail on this is mostly relegated to the appendix.

A couple of less substantial weaknesses are:

-  Perhaps the literature review could be a bit more extensive/comprehensive. For example, reference [1] below is a(n admittedly very) recent paper on origin-destination matrix estimation in mobility models, and while it might not be necessary or possible to compare against, it would perhaps be worthwhile including in the literature review as a SOTA method for estimating OD matrices when evaluating the likelihood is extremely expensive. Further, reference [2] below also considers the problem of efficiently performing SBI for complex and expensive simulators, albeit with a focus on obviating the task of learning summary statistics so a slightly different focus.

- Some proof-reading and fixing of formatting issues is needed. For example: Line 268 is missing a word ("...PC-SPSA is an effective extension _that over_ parameters in a lower-dimensional subspace...") as is Line 294 ("...ASNPE is outperformed by _PC-SPSA two_ of our explored settings..."). Also the lines following Lines 141 and 156 have no numbers, not sure what's happened there (not a big deal currently but might cause problems down the line).

[1] _Zachos, I., Damoulas, T., & Girolami, M. (2024). Table inference for combinatorial origin‐destination choices in agent‐based population synthesis. Stat, 13(1), e656._

[2] _Dyer, J., Cannon, P. W., & Schmon, S. M. (2022, May). Amortised likelihood-free inference for expensive time-series simulators with signatured ratio estimation. In International Conference on Artificial Intelligence and Statistics (pp. 11131-11144). PMLR._
questions: questions: See above ^ and thanks in advance for your responses!
limitations: limitations: I do not see that this work especially threatens any negative societal impact. The main limitations of the work that I see are already addressed in the Weaknesses section above.
paper_summary: paper_summary: The paper addresses the problem of computational cost, or alternatively, of sample efficiency in simulation-based inference (SBI). By employing an active learning scheme in sequential neural posterior estimators (SNPE), the proposed method achieves improved sample efficiency, which is paramount when dealing with expensive to evaluate scientific simulator-based models.
strength_weakness: strength_weakness: * The methodology is not motivated properly, and at times seems quite ad hoc. The authors first mention the expected information gain (EIG), which is a principled criteria for active learning, but then they propose equation 2 as the criteria they use. It was not clear to me why this is a good choice, and how it related to EIG. The distance $\mathcal{D}$ in equation 2 became the quadratic loss in equation 3 without any justification/reasoning. I did not understand how the equation above line 157 was estimated. This seems like a critical quantity in the proposed method, and it would be nice if there is a discussion about it.

* Existing methods for SBI using neural density estimators do not typically provide uncertainty quantification for the weights and biases ($\phi$) of the neural network. There is very little emphasis placed on discussing this aspect. The authors mention Bayesian neural networks once in Section 2.2, and talk about MC-dropout for approximating the posterior of $\phi$. The paper lacks an in depth discussion and experiments about using them in the context of SBI (which has so far not been done to the best of my knowledge), and how they affect the proposed method (for instance, how to set the prior for the Bayesian neural network, sensitivity to the dropout rate, etc.).

* The paper includes experiments only on the traffic simulators. This wouldn't be an issue if this was an applied paper, or if the method was motivated by certain aspects of this problem which would generalise to other simulators as well. However, the method is presented in general, which is why I would expect a varied set of benchmark experiments. Given that a major portion of the references are from transportation research literature, perhaps this work, in its current state, would be more relevant for audience of that community.

* The clarity of writing can certainly be improved. 

* The paper lacks discussion of the hyperparameters of the proposed method, its sensitivity to the choice of hyperparameters, and how to set them in practice.

* The literature on active learning for simulator-based models is not cited and discussed, see for instance the following papers. This makes it difficult to judge the technical novelty of the proposed method. 
    * Gutmann and Corander (2016): https://arxiv.org/abs/1501.03291
    * Kleinegesse et al (2019): https://proceedings.mlr.press/v89/kleinegesse19a.html
    * Kleinegesse et al (2020): https://arxiv.org/abs/2003.09379
questions: questions: * I did not understand the reasoning behind adjusting equation 3. Why multiplying with the prior makes sense?
* Line 140-141: not clear what 'cost' is referring to.
* Line 172-173: I did not understand what is being conveyed.
* Line 191-192 talk about recovering the true optimum of  $\alpha$ as $N$ tends to infinity. But in practice $N$ is much smaller. Does this statement hold even in the finite case?
* What is the computational overhead of the proposed method compared to SNPE?

Typos/grammatical errors:
* Line 96
* Line 127 (genetic instead of generic)
* Line 121-122 doesn't parse well
limitations: limitations: Please include a paragraph on the limitations of the proposed approach.
paper_summary: paper_summary: The paper considers the problem of efficient neural density estimation for simulation-based inference, in settings where we want to estimate the parameters of a model from which we can draw samples but cannot define a likelihood function. Essentially, the main contribution of the paper is to integrate an active learning approach for exploring the parameter space efficiently, for which they define a measure by which they decide which parameter values should be "explored" next, i.e. used to generate more data to train the posterior. The approach is used for the Origin-Destination Matrix estimation problem in traffic simulation.
strength_weakness: strength_weakness: My general comment is that the paper has several useful and interesting ideas, but they have not been sufficiently explored or developed.

1) The experiments show improved sample complexity due to the "active learning". But if the true posterior distribution p(\theta|x_0) over parameter space is multimodal, then we may risk finding a suboptimal solution, especially if the prior is not suitable. 
2) A measure is proposed to choose the next few samples to be "labelled", but we don't really get to understand why that measure should be used.
3) Although the general approach is quite generic, experiments are shown for only one task (OD calibration)

Minor comments:
Algo 1: I think you should initialize D(r) with D(r-1) outside the "for" loop for variable b, and inside the loop, it should be D(r)=D(r) U (\theta_b,x_b)
Fig 1 part b: internal text is illegible, should be expanded
Eq 4: p(\theta|x_0, \phi) should be p(\theta|x_0, \phi ')
questions: questions: 1) How to choose a good prior for the OD matrix? Can we use some insights based on additional knowledge about the locations?
2) What are the other endogenous and exogenous parameters used for this simulation? 
3) The technique proposed is quite generic, but only one application is shown: OD matrix estimation for traffic simulation. Can you discuss other applications, maybe within the traffic simulation domain itself?
4) Is there any other way of dealing with the parameters \phi without defining a distribution over them? In other words, if we do not choose 'q' to be a Bayesian Neural Network, can we still use the active learning approach to choose candidate '\theta'?
5) Is Equation 3 basically the variance of \theta? Does the active learning basically choose those values at which \theta has maximum variance according its current posterior distribution?
6) When we are sampling candidate parameter values before applying the selection criteria, should we consider a distribution different from the current posterior? Choosing the current posterior essentially means we are "exploiting" the region of the parameter space where we already have observed some values - maybe we should "explore" the other parts of the parameter space using a different distribution?
limitations: limitations: .....
paper_summary: paper_summary: This work introduces a new way to do active learning in simulation-based inference. This algorithm allows for the sampling of data points not only in regions of interest but also in regions that lead to high information gain. They then apply this algorithm to the problem of urban travel demand calibration, which consists of estimating the expected travel demand between different points of interest.
strength_weakness: strength_weakness: ### Originality
* I have no concerns regarding originality.

### Quality
* I find that Equation 4 is not well motivated. Why are the terms coming from equation 3 and the proposal prior simply multiplied ? I would have expected an hyperparameter controlling the weight assigned to both those terms but there is not, and this choice of simply multiplying the terms seems arbitrary. 
* In my understanding, optimizing equation 4 can be very computationally costly as it requires evaluating the Bayesian neural network on many samples and each evaluation of Bayesian neural network consist itself in the evaluation of several neural networks. I find that this limitation is not well stated in the paper. 
* RMNSE is a strange metric to use for posterior evaluation, in my opinion. This metric is designed to evaluate the quality of point estimation but not distributions. In particular, it falls short when the posterior is multi-modal; the metric would prefer an unimodal approximation in the middle to the correct multimodal distribution. 

### Clarity
* The paper is a mix of methodological and applicative paper and does not have a clear scope. While the title suggests an applicative paper, the authors introduce a new algorithm that is valuable in all generality for the field of simulation-based inference. The experiments are, however, limited to urban travel demand calibration and hence do not demonstrate the this method is effective for other problems. 

### Significance
* I have no concerns regarding significance.
questions: questions: Do you have results with a metric other than RMNSE that would not suffer from the same issues?
limitations: limitations: The limitations regarding the computational cost of the acquisition function are not mentioned.

Comments: title: Request for Final Score Revision
Follow-up on Planned Revisions
Minor correction in table headers
comment: Thank you for the favorable score revision and again for your valuable feedback! It is very much appreciated. A few additional comments to the points raised:

***Regarding the formulation of the acquisition function***

Thank you for clarifying. We agree, the family of functions

$$\alpha(\tilde{\theta}, p(\phi, D)) = \tilde{p}(\tilde{\theta})(\mathbb{E}_{\phi^\prime\sim\phi |D}[\dots])^\lambda$$

under parameter $\lambda$ constitutes valid choices for the acquisition function for any $\lambda$, facilitating different levels of emphasis on uncertainties at values of $\theta$ relative to their likelihoods under $\tilde{p}$. The choice to use $\lambda =1$ here is in part due to convenience as we did not set out to explore this parametric family explicitly, but it intuitively captures a desirable balance in the relationship between uncertainties and likelihoods of $\theta$. 

In particular, under level sets $\alpha(\cdot, p(\phi, D)) = z$ (where $p(\phi, D)$ is held constant), as likelihoods $\tilde{p}(\tilde{\theta})$ decrease by a factor of $n$, the average deviation between $p(\theta | x, D)$ and $p(\theta | x, \phi)$ need only increase by a factor of $\sqrt{n}$, i.e., changes in uncertainty are sub-linear in the likelihood ratio. With the introduction of $\lambda$, we’d see this factor generalize to $n^{1/(2\lambda)}$, and may need to take additional measures to balance the resulting sensitivity between the terms. We find that $\lambda =1$ is a natural choice that reasonably captures the desire to explore potentially unlikely parameters with high uncertainties without ignoring them (e.g., $\lambda \rightarrow 0$) or relying too heavily on them (e.g., $\lambda \rightarrow \infty$). We nevertheless find it intriguing to explore the impacts of $\lambda$ on the effectiveness of the acquisition function in practice, and will aim to incorporate this alongside our additional ablation tests in the final manuscript.

Regarding the additive form, we find this slightly less intuitive, and potentially lacking some of the above-mentioned qualities. In particular, it only shifts the uncertainties for choices of $\theta$ rather than scaling them by their likelihood, meaning the relationship between terms is no longer dependent on a multiplicative factor. That is, under level sets of $\alpha$, absolute differences in likelihood $\tilde{p}(\tilde{\theta})$ need to be made up for by proportional absolute differences in uncertainty around $\tilde{\theta}$, which is slightly counter-intuitive (e.g., for any two $\theta_1,\theta_2$, the needed change in uncertainty is no longer determined by their likelihood ratio under $\tilde{p}$). Additionally, for $\lambda$ close to 1, $\theta$ with low likelihood may be too readily selected provided $\tilde{p}(\theta)$ is dominated by high uncertainty found outside of high-likelihood regions under $p(\theta | x_o)$.

We welcome further discussion if there are any outstanding questions or concerns. Thank you again!
Thank you for your response. I greatly appreciate this healthy discussion and am happy to see that the authors are taking all the reviewers' comments constructively and actively working towards improving their manuscript.

This discussion regarding [W-1] is emblematic of why I found the method description confusing and ad hoc while reading. Introducing notation that is not really used (like $\mathcal{D}$) just disrupts the flow while reading. I would encourage the authors to take such clarifying questions from reviewers as signals for things that are unclear in the paper, so as to improve the exposition a bit. This would greatly improve their paper in my opinion (apart from all the other improvements they have already made). Thank you.
Thank you for the favorable score revision and again for your valuable feedback! It is very much appreciated. A few additional comments to the points you raised: 

- **[W-1]** Equation 2 is mostly used as a general form that highlights the central idea of computing the expected difference between the marginal posterior and individual realizations of the NDE, for $\phi\sim p(\phi | D)$. This gives us a functional form dependent on specific choices of $x$, but not specific $\theta$, as it leverages the entire distributional form of the posterior (i.e., $\mathcal{H}_{\mathcal{D}}(x)$ is a scalar). Equation 3, however, uses the exact likelihood values at particular choices of $\theta$, and while loosely borrowing the approach from Eq. 2, it is not a downstream result of selecting any particular divergence measure. That is, it provides a notion of uncertainty induced by $p(\phi | D)$ that can be measured for specific parameter candidates $\theta$, and thus a divergence measure isn’t directly applicable here.

- **[W-2]** _(Apologies, we ran out of space in our initial rebuttal)_ While quantifying uncertainty over NDE parameters is indeed uncommon in SBI settings, we viewed this mostly as a stepping stone to enable the use of our proposed acquisition function. While we provide some context in Appendix C.1.1, a more principled analysis of model hyperparameters is planned for the final manuscript as detailed in [W-4]. Additionally, while there is some exposition in Section 3.4 that highlights possible choices of NDE that both align with typical options in SBI and provide parameter uncertainties, we agree this should be further expanded upon, and will incorporate this in our final version.

- **[W-4]** We agree, and will be sure to include these results, analyses, and ablations as laid out in our global rebuttal.

- **[W-5]** We also think a discussion relating these works to our paper would help better position our method, and will include this in our updated manuscript.

- **[Q-1]** Understood, thank you for making note of this; we’ll revise the wording here.

We welcome further discussion if there are any outstanding questions or concerns. Thank you again!
Thank you for recommending acceptance and again for your valuable feedback! It is very much appreciated, and we welcome any further questions or discussion.

An additional note for **[W-1]**: Regarding exposition that better characterizes the problems for which our method is particularly ill/well-suited, the 2nd paragraph under **[Q-1, Q-2, L-1]** is likely the most relevant part of our original rebuttal to this point. We intend to expand on this further in our final version alongside the additional numerical results.
Given you indicated a willingness to revise your score, we wanted to kindly ask if you would look over our reply, which includes our plan to incorporate your suggestions (_W-1_, _W-2_, and _W-3_). If these changes meet your expectations and warrant the aforementioned score revision, doing so prior to today’s deadline would be greatly appreciated. Thank you again for your helpful feedback during this process!
Thank you again for your feedback, we wanted to briefly follow up on our planned revisions mentioned above. Please let us know if there are any further questions we can address; we’d be happy to continue the discussion if there are outstanding concerns or revision suggestions. Thank you!
I thank the authors for the detailed responses. I am happy to see the inclusion of some benchmark examples that definitely improve the paper.
* **W-1:** I am still unclear about the role of $\mathcal{D}$ here, as it is taken to be the squared loss (if I understand correctly), and not any divergence/distance metric on the space of distributions. 
* **W-2:** I do not see any response to this comment.
* **W-4:** This is exactly why it makes sense to have some simpler benchmarking experiments when proposing new methods, so that we can test the performance by varying different settings and hyperparameters in a "sandbox" environment, and develop intuition about the limits and workings of the method. I hope such detailed analyses are included in the future versions of the manuscript.
* **W-5:** Thank you. The points you mentioned are exactly what needs to be in the paper to put the proposed method in context with relevant literature. 
* **Q-1:** I feel like the problem is with the sentence in lines 172-173. It is perhaps too dense for readers to understand (seems like another reviewer had similar confusion like me). A bit of explanation might help the readers.

Given that the paper has improved quite a bit, I am happy to raise my score. All the best!
Thanks for your reply, I comment on it below.

> The motivation for Equation 4 is primarily discussed in Section 3.3 (lines 172-175), which describes how the $\theta^*$ that maximizes this notion of distributional uncertainty is not necessarily a likely parameter under the observational data $x_o$. Multiplying by the proposal prior, which captures the most up-to-date estimate of $p(\theta | x_o)$ at each round of the algorithm, allows us to re-weight this uncertainty by the approximate likelihoods of each parameter under $x_o$ and prioritize more likely $\theta$ during acquisition. This multiplication is primarily motivated by the desire to more closely align the effective likelihoods of parameters $\theta$ with the true proposal prior, which is ultimately present and corrected for in the NDE loss (see line 178).

I understand the need to introduce the $p(\theta | x_o)$ term. My comment was more that such acquisition
$$ \alpha(\tilde{\theta}, p(\phi|D)) = \tilde{p}(\tilde{\theta}) (E_{\phi' \sim \phi | D}[...])^\lambda$$
would also be perfectly valid for any $\lambda$ in my opinion and the choice of $\lambda$ = 1 seems arbitrary. The following acquisition would also be valid
$$ \alpha(\tilde{\theta}, p(\phi|D)) = (1-\lambda)\tilde{p}(\tilde{\theta}) + \lambda E_{\phi' \sim \phi | D}[...].$$
Therefore, I was wondering what was motivating your choice of acquisition function. 

> Regarding computational cost of acquisition: Section 3.4 describes the mechanism by which Equation 4 can be optimized over a fixed set of proposal samples during each round. While we do indeed need to evaluate the NDE for several candidate parameters and many network realizations, this acquisition evaluation can be performed very efficiently as a batched inference step under the NDE model.
Further, the additional computational overhead can be compared against SNPE (i.e., no acquisition function) for our primary traffic task, as the reported wallclock times (found in subfigures (b) for each of the calibration plots) provide the raw runtimes for each method when obtaining the 128 simulation samples for each setting. Empirically, this allows us to compare the total time spent by our algorithm (including the cost of optimizing the acquisition function), and we observe a negligible difference compared to SNPE over our explored horizons.

Thanks for the clarification.

>You raise a valid point: RMSNE is indeed a non-standard metric for evaluating SBI methods in the literature, and is employed in this work almost entirely due to its prevalence in the OD calibration community. In our traffic calibration setting, there are few why it’s used:
We are ultimately interested in producing good point estimates of the true OD matrices,
RMSNE is a standard in the OD calibration space and allows us to compare with other baseline methods like SPSA and PC-SPSA, and
While we may benefit downstream from having a good distributional estimate under $x_o$ (as SBI methods generally attempt to provide), for our task, we are mostly interested in how approximating the posterior can improve our method's intermediate exploration of the parameter space in service of producing good point estimates.
Nevertheless, we agree a systematic exploration of the final posterior accuracy under metrics better suited to capture distributional differences (not just point estimates) would be warranted in general. Please see our global rebuttal reply for more details here.
[W-4] Please see our global rebuttal reply for details on additional empirical evaluation, including discussion around new simulation benchmarks and more representative metrics.

Thanks for the addition of new benchmarks with metrics better suited for simulation-based inference.

Given the huge improvement in the evaluation of the method which was my main concern, I increased my score to 7.
Thanks for the rebuttal. 

[W-1]: I appreciate the addition of further evaluations on more standard SBI benchmarks, especially given the time constraint. My concerns were not necessarily about the lack of an in-depth evaluation of the advantage as a function of sample size, but rather that a sufficient amount of exposition should be given to elucidate to the reader the settings in which this methodology may be particularly helpful, and in which it may provide a significantly smaller benefit. I believe that the additional simulations that were described in the global rebuttal quell my concerns here. 

[Q-1, Q-2, L-1]: I do agree with the authors' perspective that the traffic simulation model showcases the benefits of this methodology well. I appreciate the clarification regarding the relative computational cost of the filtering step.

[Q-3]: I also appreciate the clarification in this regard.

I believe this work has value to practitioners and researchers of SBI and lean towards recommending acceptance.
Thank you for your consideration regarding the score, as well as the additional feedback. It is very much appreciated!

**[W-1]** We agree: the use of $q_{\phi, x}(\theta)$ (bullet #4) is inconsistent and should be replaced with $q_{\phi}(\theta | x)$ (bullet #1). We will make this change in our final version.

Regarding $q_{\phi}(\theta | x)$, or $q_\phi$, we generally use this to indicate that $q$ is a family of models parameterized by $\phi$. $p(\theta|x, \phi)$ is then used in contexts when $\phi$ is concrete (e.g., in an expectation like that of Eq. 2, or in the marginalization above line 157), and parallels the generic form we see with the marginal posterior $p(\theta | x, D)$. However, as you point out, this distinction isn’t particularly important (or at least the use of two separate forms isn't necessary), and we agree simply using $q_\phi(\theta |x)$ directly is better in these contexts given it's clear $\phi$ is concrete (i.e., it doesn't clash with the broader notion that $q_\phi$ can refer holistically to the model family). We will make these substitutions in our final version.

**[W-3]** We also think this discussion is valuable, and will be sure to include it in the literature review of our final manuscript.

---

In summary, **W-1**, **W-2** (addressed in global rebuttal), and **W-3** will each be incorporated in our final version. Thank you again for your consideration, and we welcome any further questions or discussion.
Thank you for the favorable score revision and again for your valuable feedback! It is very much appreciated, and we welcome any further questions or discussion.
Thank you for the favorable score revision and again for your valuable feedback! It is very much appreciated.

**Regarding C2ST** We agree completely, and should emphasize that this metric in particular was empirically difficult to improve under the small sample sizes tested. This is mostly consistent with the C2ST values reported in [1] on the reference benchmark tasks, where C2ST is broadly quite close to 1.0 for small sample sizes (smallest reported being $10^3$). Indeed, we intend to scale up these trials and explore performance on larger simulation samples ($10^5$+) for the final version. _(Note that our initial attempts at doing so indicate the metric appears to improve significantly for even small multipliers on sample size. For instance, on the Bernoulli GLM task with a sample size five times as large (2500 simulation draws), C2ST is much more distinctive, with SNPE averaging ~$ 0.87$ and ASNPE ~$0.85$ across five trials.)_

[1] Lueckmann, J. M., Boelts, J., Greenberg, D., Goncalves, P., & Macke, J. (2021). Benchmarking simulation-based inference. AISTATS.
Thank you for addressing my concerns. I especially value the additional experiments given the short time available. 

[W-1] I appreciate C2ST as an additional metric, but think it is important to note that the score is nearly 1 for all settings and methods and thus its informative value is very limited here. Acknowledging the papers focus on small simulation budgets, I still would recommend to additionally repeat the experiments with a bigger budget to better discriminate between SNPE and ASNPE regarding C2ST in a *final* version (not necessary during discussion period). If I understand the authors correctly, this is already planned with " 2) comparison between ASNPE and SNPE over long simulation horizons".

The other responses greatly improve the clarity for me and would be informative in the final version. I believe the paper is valuable for the NeurIPS community and increased my score accordingly.
I thank the authors for their meticulous responses to all the questions. My main concern about the work was that it is focussing only on one very specific task, though the approach was quite generic (I see that other reviewers too had the same concern). I am satisfied by the author's decision to add a few benchmark tasks, and now I am leaning towards recommending acceptance.
Thanks for your rebuttal. 

**[W-1]**: Thanks for explaining. I do think consistency in notation (e.g., picking either bullet point 1 or bullet point 4 to use throughout) will be important for the paper's clarity. I also do not personally see why it's important to distinguish between bullet point 1 and bullet point 2. Would it not perhaps be clearer to use $\lbrace{q_{\phi} \mid \phi \in \Phi\rbrace}$ or something to refer to the NDE model and $q_{\phi}$ for a particular instantiation (i.e. the NDE at parameter values $\phi$)?

**[W-2]**: Thanks for taking the time to prepare these additional experiments, I think this is valuable to include in the updated paper.

**[W-3]**: Thanks for considering how your contribution relates to these papers. I think the differences you raise make sense, and would be important to discuss in the revision to more thoroughly contextualise your work.

With these improvements in place I would be happy to raise my score.
Dear authors and reviewers,

The authors-reviewers discussion period has now started.

@Reviewers: Please read the authors' response, ask any further questions you may have or at least acknowledge that you have read the response. Consider updating your review and your score when appropriate. Please try to limit borderline cases (scores 4 or 5) to a minimum. Ponder whether the community would benefit from the paper being published, in which case you should lean towards accepting it. If you believe the paper is not ready in its current form or won't be ready after the minor revisions proposed by the authors, then lean towards rejection.

@Authors: Please keep your answers as clear and concise as possible. 

The AC
`NOTE` _Please note the column headers in the global rebuttal tables were accidentally rotated relative to the table bodies, and should instead read_

> | L2            | C2ST          | MMD           | KSD            |
> | :---           | :---          | :---          | :---          |

_Below we reproduce the "Results" section from the global rebuttal with corrected headers and metric references for convenience._

---

### Results

| | L2 | C2ST | MMD | KSD |
| :--- | :--- | :--- | :--- | :--- |
| SNPE | 11.3820 ± 1.1043 | 0.9938 ± 0.0003 | 0.2901 ± 0.1217 | 5.9121 ± 0.3323 |
| ASNPE | **11.2673** ± 0.5796 | 0.9938 ± 0.0002 | **0.2558** ± 0.1331 | **5.6731** ±0.1988 | 

**Table 1: SLCP Distractors** 

| | L2 | C2ST | MMD | KSD |
| :--- | :--- | :--- | :--- | :--- |
| SNPE | 18.7130 ± 1.8315 | **0.9952** ± 0.0028 | 0.2496 ± 0.0579 | **37.2472** ± 3.4843 |
| ASNPE | **16.1893** ± 0.5625 | 0.9980 ± 0.0007 | **0.2340** ± 0.0467 | 39.4204 ± 0.2686 |

**Table 2: Bernoulli GLM** 

| | L2 | C2ST | MMD | KSD |
| :--- | :--- | :--- | :--- | :--- |
| SNPE | 1.2655 ± 0.0412 | 0.9950 ± 0.0000 | 0.0886 ± 0.0292 | 6.6242 ± 0.9629 |
| ASNPE | **1.1216** ± 0.0780 | 0.9950 ± 0.0000 | **0.0633** ± 0.0256 | **3.7100** ± 0.3214 | 

**Table 3: Gaussian Mixture** 

Note that smaller values are better for each metric, and _L2_ refers to the average $L_2$ distance between $x_o$ (observational data point) and samples $x\sim p(x|\theta), \theta\sim p(\theta|x_o)$. This provides some insight into how well the posterior is calibrated around our data point $x_o$, and whether samples drawn from the posterior $\theta\sim p(\theta|x_o)$ ultimately yield synthetic data close to $x_o$ under the mechanistic model. 

While these simulation horizons are relatively small, we find that ASNPE tends to outperform SNPE across most settings and on most metrics. In particular, ASNPE outperforms SNPE on MMD and L2 metrics across all settings. Outside of _Bernoulli GLM_, ASNPE either matches or exceeds SNPE in terms of C2ST and KST, whereas on _Bernoulli GLM_ SNPE achieves better scores for these metrics.