Multifidelity Simulation-based Inference for Computationally Expensive Simulators
Authors: Anastasia N. Krouglova
Hayden R Johnson
Basile Confavreux
Michael Deistler
Pedro J. Goncalves
Venue: ICLR 2026 Poster (2026)
Keywords: simulation-based inference
likelihood-free inference
Bayesian inference
transfer learning
multifidelity
neuroscience
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Across many domains of science, stochastic models are an essential tool to understand the mechanisms underlying empirically observed data. Models can be of different levels of detail and accuracy, with models of high-fidelity (i.e., high accuracy) to the phenomena under study being often preferable. However, inferring parameters of high-fidelity models via simulation-based inference is challenging, especially when the simulator is computationally expensive. We introduce a multifidelity approach to neural posterior estimation that uses transfer learning to leverage inexpensive low-fidelity simulations to efficiently infer parameters of high-fidelity simulators. Our method applies the multifidelity scheme to both amortized and non-amortized neural posterior estimation. We further improve simulation efficiency by introducing a sequential variant that uses an acquisition function targeting the predictive uncertainty of the density estimator to adaptively select high-fidelity parameters. On established benchmark and neuroscience tasks, our approaches require up to two orders of magnitude fewer high-fidelity simulations than current methods, while showing comparable performance. Overall, our approaches open new opportunities to perform efficient Bayesian inference on computationally expensive simulators.

Published as a conference paper at ICLR 2026
MULTIFIDELITY SIMULATION-BASER INFERENCE FOR COMPUTATIONALLY EXPENSIVE SIMULATORS
Anastasia N. Krouglova' .2,33 Hayden R. Johnson 12,33
12.33 Basile Confavreux",
Michael Deistler Pedro J. Gonçalves" .2,3,8
Department of Computer Science, KU Leuven, Belgium
2 VIB Center for Al and Computational Biology (VIB.AI), Leuven, Belgium
3 VIB-KU Leuven Center for Neuroscience, Belgium
4 Gatsby Computational Neuroscience Unit, UCL. London, UK
5 Machine Learning in Science, University of Tübingen, Tübingen, Germany
6 Tübingen Center, Tübingen, Germany
7 Max Planck Institute for Biological Intelligence, Martinsried, Germany
8 Department of Electrical Engineering. KU Leuven, Belgium (nastya. krouglova, pedro. gonca I v be
ABSTRACT
Across many domains of science, stochastic models are essential tool to un- derstand the mechanisms underlying empirically observed data. Models can be of different levels of detail and accuracy, with models of high-fidelity (i.e., high accuracy) to the phenomena under study being often preferable. However, inferring parameters of high-fidelity models via simulation-based inference is challenging, especially when the simulator is computationally expensive. We introduce multifi- delity approach neural posterior estimation that uses transfer learning to leverage inexpensive low-fidelity simulations to efficiently infer parameters of high-fidelity simulators. Our method applies the multifidelity scheme to both amortized and non- amortized neural posterior estimation. We further improve simulation efficiency by introducing a sequential variant that uses an acquisition function targeting the predictive uncertainty of the density estimator to adaptively select high-fidelity parameters. On established benchmark and neuroscience tasks, our approaches require up to two orders of magnitude fewer fideliiyeiity simulations than cur- rent methods, while showing comparable performance. Overall, our approaches open new opportunities perform efficient Bayesian inference on computationally expensive simulators.
INTRODUCTION
Stochastic models used science and engineering capture complex properties of real systems through simulations (Barbers et al. [2024] Nelson & Pei 2021 Pillow & Scott 2012) Marlier al, 2021). These simulators encode Nomain-speciff knowledge and provide means generate nigh-fidelity synthetic data, enabling accurate forward modeling of experimental outcomes. However, inferring model parameters from observed data can be challenging, especially when simulators are stochastic, the likelihoods of the simulators are inaccessible, or when simulations are computationally expensive.
Simulation-basse inference (SBI) addresses these challenges by leveraging forward simulations to infer the posterior distribution, enabling quantification of uncertainty even when the likelihood is intractable (Cranmer et al.) 2000). The challenge of extending sampling-base based SBI methods like Approximate Bayesian Computation (ABC) TTavaré et al., 1997 Pritchard et al. 1999 to problems with large numbers of parameters has driven significant advancements in neural based approaches that estimate the likelihood (Papamakarios et al. 2019). the likelihood-to-eviine -ovi-ee--vveeeeeeeee ratio (Hermans) et al., 2020), directly the posterior (Greenberg al. 2019 Lueckmann et al.) 2017 Papamakarios & Murray| 2016). In particular, amortized Neural Posterior Estimation (NPE) trains a neural density estimator directly approximate the posterior, bypassing the need estimate the model evidence

---

Published as a conference paper at ICLR 2026
(Papamakarios & Murray 2016). To improve inference for fixed observation and allow stable training, truncated sequential variants have been introduced for neural posterior estimation (TSNPE) (Deistler et al. 2022), and neural ratio estimation (Miller et al. 202T). These approaches have leveraged recent progress in neural density estimation to improve the scalability and accuracy of SBI, allowing parameter inference in problems with higher dimensionality than was previously achievable (Ramesh et al., 2021; Gloeckler et al. (024)) Despite these advancements, SBI methods face computational challenges for scenarios involving expensive simulations or high-dimensional parameter spaces, as state-of-the-ar methods often require extensive simulation budgets to achieve reliable posterior estimates (Lueckmann et al.) 202T).
Multifidelity modeling offers solution to this problem by balancing precision and efficiency. It combines accurate but costly high-fidelity models (Hoppe et al., 2021; Behrens & Dias 2015) with faster, less accurate low-fidelity models. Here, low-fidelity models could be simplifications made possible through domain knowledge about the high-fidelity models, low-dimensional projection of the high-fidelity model, surrogate modeling (Peherstorfer et al. 2018). For example, Reynolds- averaged Navier-Stokes (RANS) models simplify turbulent flow simulations in aerodynamics (Han) et al. 2013), while climate models often reduce complexity by focusing on specific atmospheric effects (Held) 2005; Majda & Gershgorin 2010). Similarly. mean- mean-field approximations are used to capture certain features of spiking neural network dynamics (Vogels al. 2011 Dayan & Abbott 200T). Multifidelity methods have proven effective across domains enhainiinnnhnnnnnnngggn optimization through multifidelity Bayesian optimization (Song al.] 2019 Kandasamy et al. (017). and improving the efficiency of inference through multifidelity Monte Carlo approaches (Pcherstorfer et al., 2016 Nobile & Tesei 2015 Giles 2008 Zeng et al. 2023). In the context of SBI, we hypothesized that by
leveraging the complementarity of high- and low- low-fidelity simulators, it would be possible to reduce the computational cost of inference while retaining inference accuracy.
In this work, we present MF- MF-(TS)NPE. a multifidelity approach that improves the efficiency of amortized and non-amortized neural posterior estimation for expensive simulators. MF-(TS)NPE reduces the computational burden of posterior estimation by pre-training a neural density estimator on low-fidelity simulations and refining the inference with a smaller set of high- fideliiiy simulations. Additionally, we present MF-TSNPE-AF, tating targeted parameter space exploration to effectively enhance high-fidelity posterior estimates given single observations. We focus multifidelity cases where both models are simulators and where the low-fidelity model is a simplified version of the high-fidelity model, designed based on domain expertise. We demonstrate that for four benchmark tasks and two computationally expensive neuroscience simulators, our multifidelity approach can identify the posterior distributions more efficiently than NPE and TSNPE, often reducing the number of required high-fidelity simulations by orders of magnitude.
extension of ME-TSNPE with active learning, facili-
2 BACKGROUND
Multifidelity methods for inference Multifidelity has been widely explored in the context of
likelihood-based inference (Peherstorfer et al., 2018), from maximum likelihood estimation ap-
proaches (Maurais et al. 2023) Bayesian inference methods (Vo et al. 2019 Catanach al.
2020). For cases where the likelihood is not explicitly available, several sampling-baas basedd multifidelity
methods have been proposed within the framework of ABC (Prescott & Baker 2020 Warne et al.
2022 Prescott et al. 2024 Prescott & Baker 2021). However, these methods inherit limitations of ABC approaches, particularly in figh-dimmesiooall parameter spaces, where neural density estimators
offer more scalable alternatives to complex real-world problems (Lueckmann et al. 202T). Concur- rently with our work, Thiele et al. (2025) developed multifidelity SBI approach based on response distillation, Hikida et al. (202)) adapted multilevel Monte Carlo techniques to SBI, and Saoulis et al. (2025) applied transfer learning to accelcrate inference on a cosmological task.
Beyond SBI, multifidelity has been explored in Bayesian optimization, where Gaussian process models integrate data of different fidelities to infer expensive functions (c.g.. Song et al. 2019 Zanjani Foumani et al., 2023). These approaches focus on learning surrogate likelihood functions rather than posteriors over simulator parameters, but they highlight the broad applicability of the multifidelity concept.
2

---

Published as a conference paper at ICLR 2026
Transfer learning and simulators To facilitate learning in target domain, transfer learning borrows knowledge from a source domain (Panigrahi et al. 2021). This is often done when the target dataset is smaller than the source dataset Larsen-Freeman
2013). For numerical simulators, transfer learning approaches have been used to lower the simulation budget, for instance, in CO2 forecasting (Falola al. 2023), surrogate modeling (Wang et al.] 2024 Zeng al., 2026), and model inversion with physics-informes potential of transfer learning for computationally efficient simulation-based inference has not been fully realized yet.
neural networks (Haghighat et al. [2021). To the best of our knowledge, the
Simulation-effcccent SBI Recent work reduces the cost of SBI for expensive simulators through active learning efficient representations. Active learning methods adaptively select simulation
parameters for neural likelihood or posterior estimation (Lueckmann et al. 2019 Griesemer et al.
2024), paralleling Bayesian optimization for ABC (Gutmann & Corander 2016 Elficiency also improves through learned representations such as signature- features (Dyer et al. 2022). com- positional models (Gloeckler et al.) (025), or self-consistency objectives (Schmitt al.. 20224 b). Unlike these single-fidelity approaches, MF-(TS)NPP leverages an expert-designed low-fidelity simulator and combines transfer learning with active learning refine posterior estimates efficiently.
METHODS
ME-(TS)NPP is multifidelity approach to Neural Posterior Estimation (NPE) for computationally expensive simulators leveraging transfer learning and, in its sequential variant, active learning. We present our approach in S.c. In Sec. 3.1.4] we discuss the evaluation metrics used compare our
method against NPE (Greenberg et al. 2019). TSNPE (Deistler et al. (0022), and MF-ABC (Prescott)
Baker 2020 MF-(TS)NPE is summarized in Fig. Algorithms
and
3.1
MULTIFIDELITY NPE
Prior over parameters 8- p(0) 6
Real data
LF Simulator
amount LFM
Pre-trained density estimator log au (0g)))
simulitions (cheap: coarse) pL. (m)
sparse samping models
knowledge
Posterior 4.(0(x.)
HF Simulator email amount HFM slmulations (esperaive, accunata) p(z(0)
Multifidelity density estimator (01
multhround
sparse sampling
probubility density
Figure 1: Multifidelity Neural Posterior Estimation proceeds by dense sampling from the prior distribution, running the low-fidelity simulator (e.g., wwo-compartment neuron model (Hodgkin & Huxley )952), and training neural density estimator with negative log- likelihood loss. MF-NPE then retrains the pre-trained network sparse samples from the same prior distribution and respective high-fidelity simulations (e.g., a multicompartmental neuron model (Rall) 1995)). Given empirical observations To, MF-NPE estimates the posterior distribution given the figh-fidelity model. In the sequential case, the parameters for high-fidelity simulations are drawn from itcrative refinements of the prior distribution within the support of the current posterior estimate, at some observation æo.
We aim to infer the posterior distribution over the parameters 0 of a computationally expensive high-fidelit simulator p(=(0), with computational cost of a single simulation c. We designate the simulator as high-fidelity if the model accurately captures the empirical phenomenon, but incurs high computational cost when generating simulations. We assume that we have access to low-fidelity
3

---

Published as a conference paper at ICLR 2026
simulator (L 0)), describing simplification of the phenomenon of interest with cost cy < We assume that both simulators operate over the same domain of observations æ. and the parameters of the low-fidelity model form least subset (and most the entirety) of the high-fidelity parameters. Our goal is to develop an estimator that leverages low-fidelity simulations to infer the posterior distribution over parameters of the high-fidelity model with limited high-fidelity simulations, without access to tractable likelihood for either simulator.
As with NPE (Papamakarios & Murray 2016 Greenberg al. 2019), to estimate the posterior density over model parameters for which the likelihood function is unavailable, we consider sufficiently expressive neural density estimator qo ()) and train minimize the negative log- oog-liilihood loss:
( = E0 Be)p(=))) log go (0(0 [[))],
(I)
where 0 is sampled from the prior distribution, denotes the respective simulations (i.e., samples from p(æ: 0)), and & are the network parameters. By minimizing Z(.), the neural density estimator approximates the conditional distribution p(0|x²) directly (Papamakarios & Murray 2016) (proof of convergence in Appendix [ Given an empirical observation Zo. we can then estimate the posterior over parameters p)) To ensure go (0(ar)) closely approximates the posterior p(0 (z))) the density estimator must be sufficiently expressive. We use neural spline flows (NSFs) (Durkan et al.) 2019), expressive normalizing flows that have been shown empirically to be competitive for SBI (Lucckmann et al. (2021). To avoid overfitting when training NSFs, we use the same validation-bast early stopping criterion in the SBI package (Boelts et al. 2024) (details in Appendix
3.1.1 TRANSFER LEARNING
MF-NPE leverages representations learned from low- wiw-fidelity simulations to reduce the number of high-fidelity simulations required to approximate a high-fidelity posterior. To that end, MF-NPE adopts a fine-tuning strategy of transfer learning: Let to be the parameters of the low-fidelity neural density estimator 4o) (0 x.) and let & be the parameters of the high-fidelity density estimator go (0)z). MF-NPE minimizes the loss L(w) E Ex--((=))) [- log qo (0==)] on the high-fidelity task, where the parameters & are initialized on the pretrained low-fidelity network parameters tt. We argue that by pre-training on low-fidelity simulations, the density estimator learns useful features up front (i.c., the feature spaces of the low- and high-fidelity density estimators overlap). so fewer high-fieelity simulations suffice to refine the posterior estimates. Indeed, Tahir et al. (2024) shows that once networks learn suitable features for given predictive task, they drastically reduce the sample complexity for related tasks. Other strategies to pretraining are discussed in Appendix G..4
MF-NPE can naturally accommodate more than two fidelity levels (Appendix D). does not require more hyperparameter tuning than NPE (Appendix C.T), and is applicable in situations where the low-fidelity model has fewer parameters than the high-fidelity model. In this setting, the parameters that are exclusive to the high-fidelity model are treated as dummy variables in the pre-trained density estimator. The pre-conditioning with these variables leads to the pre-trained neural density estimator to effectively estimate the prior distribution over the respective parameters (OU3 and OU4 in Appendix [.T). As shown below, our method is compatible with both embedding networks and hand-crafted summary statistics of the observations.
Algorithm 1 MF-NPE
1: Input: N pairs of (0, æι.;; M pairs of (0, conditional density estimators qu (f)x2) and (0)x)) x:) with respectively learnable parameters tir and Φ; early stopping criterion S.
2: C(v)) & ΣN -i=
log qu (0.lz)). / Low-fidelity model #/
3: for epoch in epochs do
4: train ds to minimize L()) until S is reached.
5: end for
6: Initialize da with weights and biases of trained go.
1e High-fidelity model */
7: L(Ø) = M EM log 9o (0,(z;).
8: for epoch in epochs do
9: train g to minimize L()) until S is reached.
10: end for
4

---

Published as a conference paper at ICLR 2026
3.1.2 SEQUENTIAL TRAINING
In addition to learning amortized posterior estimates with NPE, our approach naturally extends to sequential training schemes when estimating the non-amortized posterior (0(0)xo). Rather than sampling model parameters from the prior, sequential methods introduce an active learning scheme that iteratively refines the posterior estimate for specific observation Co.. These methods known
as Sequential Neural Posterior Estimation (Papamakarios & Murray 2016 Lueckmann al. 2017
have shown increased simulation efficiency when compared to NPE (Lucckmann et al. 2021
However, applying these methods with ficxible neural density estimators requires a modified loss that suffers from instabilities in training and posterior leakage (Greenberg et al., (019). Truncated Sequential Neural Posterior Estimation (TSNPE) mitigates these issues by sampling from a truncated prior distribution that covers the support of the posterior. This leads simplified loss function and increased training stability, while retaining performance (Deistler et al., 2022).
We apply our multifidelity approach TSNPE. First, the high-fidelity density estimator is initialized from the learned network parameters of a low-fidelity density estimator. Then, high-fidelity simula- tions are generated iteratively from a truncated prior, within the support the current posterior. We refer to this method as ME-TSNPE (complete description of the algorithm in Appendix M.Ip.
3.1.3 ACQUISITION FUNCTION
To further enhance the efficiency of our sequential algorithm, we explore the use of acquisition functions to supplement our round-witg wise samples from the TSNPE proposal: we generate simulations for round with set of parameters (+) =
( active where 0 are samples from the proposal distribution at round i, and 0 active are the top B values according to an acquisition function. We refer to this algorithm MF- DI-EESPP-AA (full description in Appendix M.2) Following Järvenpää ct al. (2019); Lueckmann et al. 2019)) we select an acquisition function that targets the variance of the posterior estimate with respect to the epistemic uncertainty in the learned parameters Φ) DD.
[0 U
0 argmax o/D (00))))))
(2)
0
We realize this as the sample variance across ensemble of neural density estimators trained independently on the same dataset D, as done in Lueckmann et al. (2019). Note that we use epistemic uncertainty guide high-fidelity simulation selection within the simulator's domain rather than dut-of-distributtio samples. For details on the proposal design of MP-TSNPE-AF, see Appendix
M.2
3.1.4 EVALUATION METRICS
We evaluate the method on observations To from the high-fidelity simulator, with parameter values drawn from the prior distribution. This ensured a fair evaluation of how much the low-fidelity simulator helps to infer the posterior distribution given the high-fidelity model. All methods were evaluated for range of high-fidelity simulation budgets (50, 10 103, 10 10a), posteriors given the same data set observations To.
Known true posterior We evaluate the accuracy of posterior distributions in cases where the ground-truth posterior is known with the Classifier-2-Sample Test (C2ST) and the Maximum Mean Discrepancy
2004; Lopez-Paz & Oquab) 2017 Gretton et al.) 2012 Lueckmann et al. 2021 Peyré & Cuturi 2017 7). C2ST is commonly used in SBI, as it is easy apply and interpret: a value close to 0.5 means that classifier cannot effectively distinguish the two distributions, implying the posterior estimate is close to the ground-truth posterior. A value close to I means that the classifier can distinguish the distributions very well, indicating a poor posterior estimation. C2ST is rarely applicable in practical SBI settings, since it requires samples from the true posterior (e.g.. Sec.
Unknown true posterior The average Negative Log probability of the True Parameters (NLTP: loloo q fo|x,)]] has been extensively used in the SBI literature for problems where the true posterior is unknown (Greenberg al. 2019; Papamakarios & Murray 2016 Durkan al. 2020 Hermans et al., 2020). In the limit of a large number of pairs (6,, To), the average over the log probability of each pair (0., m.) approaches the expected KI. divergence between the estimated
5

---

Published as a conference paper at ICLR 2026
and the true posterior (up to term that is independent of the estimated posterior), shown in Lueckmann et al. 2021). addition, we report the Normalized Root Mean Square Error (NRMSE), which quantifies the deviation of posterior samples from the true parameters on invarrant axis. NRMSE values closer to 0 indicate better predictive performance.
RESULTS
We evaluate the performance of our multifidelity approach NPE and TSNPE on six tasks involving various types of observations (e.g., time series, images, neural spiking). We start with four bench- marking tasks, followed by two challenging neuroscience problems with computationally expensive simulators and for which no likelihood is available: a multicompartmentall neuron model and a neural network model with synaptic plasticity. We also provide comparison to MF-ABC (Sec. [.I.I] D.3)
In Sec. 4.4 we provide discussion about the effectiveness of transfer learning in MF-NPE.
4.1 BENCHMARKING TASKS
We first evaluated MF-(TS)NPP on four benchmarking tasks: SIR, SLCP, OUprocess, and Gaussian Blob. SIR and SLCP are established SBI benchmarks (Lueckmann et al. 2021 OUprocess is new multifidelity task with tractable likelihood (Kou et al. 2012), and Gaussian Blob is a high- dimensional image task (Lueckmann al.) (019)) (details in Appendix D). These tasks were chosen to systematically investigate various task properties that might impact the performance of transfer learning in multifidelity setting: differing parameter dimensionality between the low- and high- fidelity models. partly observed dynamics, differing simulator types between the low- and models, and nigh-dimensional observations. Furthermore, these multifidelity tasks are not trivial in the sense that the low and high-fidelity simulators lead to different posteriors (Appendix[]) D Note that we do not evaluate the total cost of low- and high-fidelity simulations in these tasks, but defer this analysis to the two complex neuroscience tasks (Appendix D
To evaluate ME-NPE, we compared the estimated densities to the respective reference posterior, estimated from the exact likelihood with Rejection Sampling (Martino et al. 2018) (OU process:
closed-form of the likelihood in Sec. [.I.. and using Sampling and Importance Resampling RUBINN
1988) to obtain set of 10k proposal samples (SLCP, SIR), similar to Lueckmann et al. (2221). We quantified the performance with C2ST and MMD over 10 observations (30 observations for the OU
process) and 10 network initializations per observation. GaussianBlob required CNN embedding
and was evaluated with NRMSE and NLTP since no closed-form likelihood is available (Fig.
OUprocess
SLCP
SIR
1.0
0.7
0.5
0.25
MMD
0.4
0.3
0.1
885
88
Number of high-tdeeliy simulations
baseline
MF-NPE variations
MF-TSNPE variations
- TSNPE
MH-NP4 - MF- NPES
MI-TSNPE4
MI-TSNPE4-AF
Figure 2: C2ST and MMD averaged over 10 network initializations with means and 95% confidence intervals. MF-NPE4 and MF-NPE5 are pretrained on 10 and 10 low-fidelity simulations, respec- tively. Results for the GaussianBlob task in Fig, II: variations on the OU task and comparisons to MF-ABC in Fig.
Across four benchmarking tasks. observed a consistent performance increase with MF-NPE compared to NPE, and MF-TSNPE(-AF) compared to TSNPE, especially in low simulation budgets from the high-fidelity model (50-103 simulations) (Fig. Gaussian Blob in Fig. 11). In addition,

---

Published as a conference paper at ICLR 2026
we found that having higher number of low-fidelity samples improved performance, reinforeing that low- now-fidelity simulations were indeed advantageous for pre-trate pre-training the neural density estimator for the downstream task. Note that for the OU and SLCP tasks, we did not observe substantial increase in MF-NPE performance between the settings with 10* and 10 low-fidelity samples, suggesting an upper bound regarding pre-training efficacy. We also compared MF-NPE with MF- ABC, an ABC-based method for multifidelity SBI (Prescott & Baker] (020), and observed that MF-NPE has a substantially higher performance (Appendix findings indicating the superior performance of NPE with respect to rejection ABC and SMC- ABC. where it is not uncommon to require orders of magnitude more simulations to obtain reliable posterior approximations (Lueckmann et al. 2021 Frazier et al. 2024). However, more extensive hyperparameter search could potentially lead to substantial improvements in MF-ABC performance.
This consistent with previous
As described in Sec. we enhanced the sequential algorithm TSNPE (Deistler et al., 2022) with a first round of MF-NPE, and designated this approach as MF-TSNPE. We found that MF-TSNPE (details in Appendix M.1 performs better than TSNPE, especially in regimes with a low budget of high-fidelity simulations. Compared to MF-TSNPE, MF-TSNPE-AF improved inference in the OU process, but did not show significant improvements in the SLCP and SIR tasks.
Finally, we assessed the contribution of transfer learning to the overall performance in a setting where the low- and high- high-fidelity models have different number of parameters, in the context of the OUprocess task (Appendix (D33.. We expected that adding parameters to the high-fidelity model that are absent in the low-fidelity model would increase the inference complexity for MF-NPE, and indeed observed a performance decrease in MF-NPE, although MF-NPE still performed better than NPE and MF-ABC (see Appendix D.3). We note that MF-NPE also outperformed NPE when the low-fidelity model had more parameters than the high-fidelity model (see Appendix D.4]. Overall, the results suggest that MF-NPE and MF-TSNPE can yield substantial performance gains compared to NPE, TSNPE, and MF-ABC,
4.2 MULTICOMPARTMENTAA NEURON MODEL
The voltage response of a saahhholgicclly-detaleee neuron to an input current is typically modeled with multicompartment model wherein the voltage dynamics of each compartment are based on the HlodgkinHHxxle equations (Hodgkin & Huxley 1952). The higher the number of compartments of the model, the more accurate the model is, but the higher the simulation cost.
In this task, we aimed to infer the densities of ion channels INa and ÿx on a norphologically-detaii model of a -tufte layer pyramidal cell (L.5PC) containing 8 compartments per branch (Ai (Van Geit et al.] 2016). We injected in the first neuron compartment a noisy 100 ms step current with mean 4m = 0.3 nA: Le = 4m e ~ (00 0.01). The voltage response of the neuron was recorded over 120 ms, with simulation step size of 0.025 ms and 10 ms margin before and after the current injection. We defined the high-fidelity model to have 8 compartments per branch and the low-fidelity model to have compartment per branch, and both the high and low-fidelity models had the same injected current and ion channel types.
To simulate the neuron models, we used Jaxley, a Python toolbox for efficiently simulating multicom- partment single neurons with biophysical detail (Deistler et al. 2025). In this setting, the simulation time for the figh-fidelity model is approximately times higher than that of the low-fidelity model. We characterized the neural response with four summary statistics that have been commonly used when fitting biophysical models of single neurons to empirical data: spike count, mean resting potential, standard deviation of the resting potential, and voltage mean (Gonçalves et al., 2020 Gao et al. 2023). Performances were evaluated with NLTP and NRMSE on 10 pairs of H, and respective simulation outputs xt, averaged over 10 random network initializations (Sec. 3.1.4).
MF-(TS)NPE showed higher performance than NPE, in particular with larger low-fidelity simulation budgets (Fig. 33; Fig. [FI)), despite the right-skewed posterior distribution of the low-fidelity model (Fig. 21p. Furthermore, MF-NPE posterior predictives closely matched the empirical data, in contrast with NPE, even when NPE was trained on a higher number of high-fidelity simulations (Appendix F). In addition, MF-(TS)NPE achieved comparable performance with a total computational cost 4.44 + 0.06 times lower than standard NPE (Appendix [). Finally, TARP and simulation- calibration tests suggest that both MF-NPE and NPE estimates were relatively well calibrated
(Fig. 3C) TTltss et al. 2020 Lemos et al. 2023
7

---

Published as a conference paper at ICLR 2026
LSPC
D
THEE feld Zur
E
Spiking Network
HTC Daniinn
VEMPE
2
.-. let buselee
UE-ARES
spiking POOWOR
Teal vilvees
HLIP
7 19
F
Mathod
Vileuuaaioon
raten 1:50
NPC
HF: Lt
$6.4
50
10"
Credbility
Number HF sima.aations
Number of simulations
Figure 3: (A) Thick-tuffed layer 5 pyramidal cell from the neocortex, (B) Performance evaluation with NLTP (same naming convention as in Fig. (). Amortized methods are averaged over 10 network initializations; non-amortizee trained once per 100 observations. Similar results were obtained with NRMSE (Appendix F.I).. MF-NPE, and especially its sequential variants, are orders of magnitude more simulation-efficien than NPE. (C) TARP posterior calibration check shows that NPE and MF- NPE trained 10 high-fidelity samples are well-calibrated (Lemos et al. 2023). Simulation-basee calibration, posterior samples, and predictives are in Appendix [F] (D) Schematic of the low and high-fidelity models of a spiking network. (E) Performance of NPE and MI-NPE evaluated on 10000 true observations with NLTP: averages over 10 network initializations, and 95% confidence intervals. (F) Proportion of posterior samples within the target firing rate bounds. MF-NPE produces a higher fraction of parameter sets within the bounds than NPE.
MF-TSNPE-AF pre-trained on 10 low-fidelity samples outperforms MF-NPE trained on 10t samples. However, MF-TSNPE-AF performance comes at the cost of training time due to the use of an ensemble of density estimators (Appendix[]. This additional training burden is only justified when the simulation cost substantially higher than the training cost.
4.3 RECURRENT SPIKING NETWORK
Finally, we applied MF-NPE to a challenging and timely problem in neuroscience: the inference of synaptic plasticity rules that endow large spiking neural networks with dynamics reminiscent of experimental data. This problem has been recently tackled with an SBI method (filter simulation- based inference, ISBI) that progressively narrows down the search space of parameters given different sets of summary statistics (Conlavreux et al., 2023). ISBI was successful in obtaining manifolds of plasticity rules that ensure plausible network activity, but the compute requirements were reported to be very large. Here, we aim to test whether this problem can be efficiently tackled with MF-NPE.
The high-fidelit simulator consisted of a recurrent network of 4096 excitatory (E) and 1024 inhibitory (T) leaky integrate-and-fi neurons connected with conductance-based synapses (Fig. 3 P) Each synapse type in this network (E-to-E, E-to-T, I-to-E, I-to-/) was plastic with unsupervised local learning rule. For each synapse type, 6 parameters governed how the recent pre- and post-synaptic activity were used to update the synapse, for a total of 24 free parameters across all 4 synapse types (Conlavreux et al. 2023). The networks were simulated using Auryn, a C++ simulator (Zenke & Gerstner 2014) (details in Appendix G
Mean-field theory can be applied to the dynamical system above to obtain the steady-state activities of the excitatory and inhibitory populations as a function of the parameters of the plasticity rules embedded in the network. Though such analysis is widely performed in the field (Vogels et al. 2011 Confavreux et al. [023] Gerstner et al., 20144, has never been used as a low-fidelity model to help with the inference of the high-fidelit model parameters. Since there are no dynamics to simulate with the mean-field model, the simulation was almost instantancous, while the high-fidelity model took approximately 5 minutes to generate single 2-minute long simulation
single CPU.
Summary statistics of the low- and high-fidelity models were the average firing rates of the excitatory and inhibitory neurons at steady state (after 2 minutes of simulation in the high-fidelity model). Plastic networks were considered plausible if the firing rates were between and 50Hz (Dayan & Abbott 2001 Confavreux et al. 2023).
8

---

Published as a conference paper at ICLR 2026
In this task, the low-fidelity model focuses solely the E-to-E and E-to-I rules from the high- fidelity model, thereby having 12 out of the 24 parameters of the high-fidelity model. This setup allows us demonstrate the performance of MF-NPE problems with different parameter spaces, highlighting ME-NPE's flexibility and advantages. We found that MF-NPE has better performance than NPE in terms of NLTP F)), although we observed diminishing performance gain with increasing discrepancy between the number of parameters of the low- and high-fidelity models (see Appendix G.3 Furthermore, MF-NPE leads to an increase of almost 30% in the proportion of posterior samples within the target firing rate bounds (Fig. F), reinforcing that MF-NPE is a practical and effective method for SBI of costly real-world simulators.
A
Schematic representation
B
Impact Additive Noise on the Uncertainty Coefficient (UC)
Impact the on the performance
pretrained on
6,xg)
0.9
1/ Peerformacce
0.15
Ipur 6x((")
representationai
(C
0.8
NPE
gap
0.7
rpreeentationa
0.05
0.8
gap
per
0.5
0.01
0.1
10
0.05
0.1
0.15
0.2
Mutual information (Ml)
addidive LF model
(FL.2)
Figure 4: (A) Schematic figure representing lower bound transfer error (1/MF-NPE performance) as a function of mutual information between the low- and high-fidelity models, given a fixed simulation budget. (B) Uncertainty coefficient monotonically decreases with noise parameter and is invariant to data inversion.
(C) Empirical results with MF-NPE support the hypothesis that transfer performance is dependent on both mutual information and representational coherence. Note that NPE (with the same Sigh-fidelity simulation budget of 102) has similar performance as MF-NPE in the case where the low- and high-fidelity models have low mutual information.
4.4 WHEN DOES PRE-TR ARITING HELP?
In previous sections, we demonstrated that MF-NPE can significantly reduce the number of high- fidelity simulations required to accurately approximate the high-fidelity posterior by leveraging pre-training on low-fidelity simulations. This naturally leads to several key questions: Which characteristics of low-fidelity simulators enable effective transfer learning? Under what conditions can pre- pre-training reliably enhance simulation efficiency?
Providing theoretical guarantees for these questions necessitates formal characterization of con- vergence rates in NPE with transfer learning. Although recent works have begun addressing these challenges in NPE (Frazier et al. 2024), current theoretical frameworks of transfer learning (Tahir al. 2024 Yun et al. 2020 Tripuraneni et al.) 2020 Lampinen & Ganguli (018)), rely on simplifying assumptions (e.g., linear networks) that do not fully capture the complexities of MF-NPE. Given this limitation, we instead empirically explored the conditions in which low- low-fidelity pre-training facilitates effective transfer learning. To do this, we evaluated ME-NPE where the low- and high-fidelity simulators were related by systematic perturbations (Fig. 4
We hypothesized that the effectiveness of pre-training associated with two primary factors:
1. Mutual information between the low- and high-fidelity simulators.
2. Representational coherence, i.e., similarity in how task-relevant information is encoded.
To isolate the effects of these factors, we constructed controlled variants of the OU2 process in which the low-fidelity simulator differs from the high-fidelity one through two distinct transformations. In the baseline setup, the simulators generate observations according
x p( μ4 a),
IL ~ p( μ, a + 8),
where the perturbation increases the noise of the low-fidelity simulator and therefore reduces [x. monotonically as 8 grows.

---

Published as a conference paper at ICLR 2026
Second, independently manipulate representational coherence, applied invertible coordinate- reversal transformation zi = T(IL), implemented via an anti-diagonal permutation matrix that reverses the ordering of the output dimensions. Because T is invertible, the mutual information between the two simulators unchanged:
I[ " = [[ æy. = H[æ] + H[[x.] H[x xr.]-
Thus, while [T. decreases monotonically with the noise scale 8, the inversion leaves the informa- tion content unchanged while disrupting representational coherence. Figure iillusraats how each manipulation affects the uncertainty coefficient (Figure 4B), which we estimated empirically using MINE [Belghazi et al. 2018), and MF-NPE performance under fixed simulation budget of 10* low-fidelity and 10a high-fidelity simulations (Figure4C))
In agreement with our hypothesis, our results suggest that the effectiveness of MF-NPE depends on both the mutual information and the representational coherence between low- and sigh-fidelitty simulators (Fig. BC). Specifically, mutual information is necessary for effective transfer learning but not sufficient: perturbations that preserve information (e.g., invertible transformations) can still substantially impair transfer performance. Effective pre-training strategies should therefore prioritize low-fidelity simulators that are both highly informative and representationally aligned with the high-fidelity model.
5 DISCUSSION
We proposed a new method for simulation-based inference that leverages low-fidelity models to efficiently infer the parameters of costly high-fidelity models. By incorporating transfer learning and multifidelity approaches, MF-NPE substantially reduces the simulation budget required for accurate posterior inference. This addresses pervasive challenge across scientific domains: the high computational cost of simulating complex high-fidelity models and linking them to empirical data. Our empirical results demonstrate MF-NPE's competitive performance in SBI across statistical benchmarks and real-world applications, as compared to standard method such as NPE.
Limitations Despite MF-NPE's advantages, the method comes with some challenges. First, the effectiveness of MF-NPE relies on the similarity between the low-fidelity and high-fidelity models. Fortumately, in many situations, domain experts will know beforehand whether low- ow-fidelity models are poor approximations of high-fidelity models. Second, MF-NPE and MF-TSNPE inherit the limitations of NPE and TSNPE, respectively, in particular regarding the scalability of simulation-bant to ditbnsiinnsi parameter spaces. How to balance exploration of nigh-dimensional parameter spaces and computational cost in a simulation based inference setting remains a topic of active research. Third, ME-TSNPE-AF requires the training of an ensemble of density estimators, which leads to substantial computational costs in training and hyperparameter tuning. This method should therefore only be preferred in cases where the cost incurred in simulations outweighs the training cost. We estimate this to be the case for the tasks with the multicompartment neuron model and the spiking network model, for which the cost of simulation and the training of one density estimator are comparable in certain settings (e.g., on the order of minutes, for a network trained on 10 samples).
inference
Future work We identify three promising research directions for multifidelity simulation-bart based inference. First, we expect the scalability and expressivity of MF-NPE could be improved by utilizing the same approaches of multifidelity and transfer learning presented here with neural density estimators other than normalizing flows, such as diffusion models (Gloeckler et al.) (024). Second, we assumed a negligible cost for fow-fidelity simulations, and future work should address how to optimally allocate low- and high-fidelity simulations under fixed computational budget. Third, similar to past efforts developing a benchmark for simulation-base inference, it will be beneficial for the SBI community to develop benchmark for multifidelity problems, with new tasks, algorithms and evaluation metrics. This will promote rigorous and reproducible research and catalyze new developments in multifidelity SBI, and in SBI more generally. Our work and codebase are step in this direction.
Conclusion Overall, (T- is method for simulation baseed inference that leverages low- fidelity models and transfer learning to infer the parameters of costly high-fidelitg models, thus providing effective balance between computational cost and inference accuracy.
10

---

Published as a conference paper at ICLR 2026
6 REPRODUCIBILITY STATEMENT
The training and simulation costs for all tasks and SBI methods, as well as a detailed description of the experimental setup, are described in Appendices C.T and The corresponding code and data publicly available on Github: /etttub.co//ooocaaa//uutiiiieie
ACKNOWLEDGMENTS
We thank Karthik Sama, Najlaa Mohamed, Guy Moss, Marcel Nonnenmacher and Pierre Vanvolsem for discussions. We also thank the staff of the VIB Data Core for their support. Anastasia N. Krouglova was supported by an FWO grant (G097022N), Hayden R. Johnson was supported by an FWO grant (G053624N). Basile Confavreux was supported by a Schmidt Science Polymath Award to Andrew Saxe, the Sainsbury Wellcome Centre Core Grant from Wellcome (219627/2/19/Z and the Gatsby Charitable Foundation (GAT3850). Michael Deistler was supported by the German Research Foundation (DFG) through Germany's 's Excellence Strategy (EXC 2064 Project number 390727645), the German Federal Ministry of Education and Research (Tübingen Al Center, FKZ: and the European Union (ERC, "DeepCoMechTomee" ref. 101089288; Jakob H. Macke). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. Michael Deistler was a member of the International Max Planck Research School for Intelligent Systems (IMPRS-IS) Pedro Gonçalves thankful for the financial support from the Flemish government through long-term structural funding Methusalem (grant METH//6/003)).
8039A)
REFERENCES
Elias Barbers, Friedrich Emanuel Hust, Felix Emil Arthur Hildenbrand, Fabian Frie, Katharina Lilith Quade, Stephan Bihn, Dirk Uwe Sauer, and Philipp Dechent. Exploring the effects of cell-to- cell variability on battery aging through stochastic simulation techniques. Journal of Energy Storage, 84: 10851, April 2024. ISSN 2352-152X. doi: 10.10166 URL https
7/ w/www. scwencedieeeiicccceo rect com sc httprpc.w....caaaaa.....oooaaa/a
J. Behrens and F. Dias. New computational methods in tsunami science. Philosophicaal Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 3320003203333000000000244442222 October 2015. doi: 10. 10.11998/sta.2014.0333 org /og//doio 1
URL bttpss://ooyaasoccietyuuuo...u.
fu htt/f/lll/1/000/ 1 1098 / 2014 0382
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. Mine: mutual information neural estimation. arXiv preprint arXiv: 04062,, 2018
Jan Boelts, Michael Deistler, Manuel Gloeckler, Alvaro Tejero-Cantero, Jan-Matthis Lueckmann, Guy Moss, Peter Steinbach, Thomas Moreau, Fabio Muratore, Julia Linhart, Conor Durkan, Julius Vetter, Benjamin Kurt Miller, Maternus Herold, Abolfazl Ziaeemehr, Matthijs Pals, Theo Gruner, Sebastian Bischoff, Nastya Krouglova, Richard Gao, Janne K. Lappalainen, Bálint Mucsányi, Felix Pei, Auguste Schulz, Zinovia Stefanidi, Pedro Rodrigues, Cornelius Schröder, Faried Abu Zaid, Jonas Beck, Jaivardhan Kapoor, David Greenberg, Pedro Gonçalves, and Jakob H. Macke. sbi reloaded: a toolkit for simulation based inference workflows, November 2024. URL http:://rxtii o abs 2411.111111 17337
Simon Carter and Helmut H. Strey. Parameter estimation from an Ornstein-Ullenbeck process with measurement noise, August 2023. URL http::tt://eee /tttt:://rxxi.... arwiv. http://ww..rgab//25....../
Thomas A. Catanach, Huy D. Vo, and Brian Munsky. Bayesian inference of stochastic reaction
networks using multifidelity sequential tempered Markov Chain Monte Carlo. International
Journal for Uncertainty Quantification, 1066)515-542, 2020. ISSN 2152-5080. doi: 10.1611//nt.j
unceraintyuuafiicttoo0000033333.
Basile Confavreux, Poomima Ramesh, Pedro J. Goncalves, Jakob H. Macke, and Tim Vogels. Meta-leaming families of plasticity rules in recurrent spiking networks using simulation-hasee

---

Published as a conference paper at ICLR 2026
inference. Advances in Neural Information Processing Systems, /ites/pappe/22223333
Decem-
ber 2023. UtL.attess/ppcceedddg.....oeee.e hash haap://2222222200222222222222222
Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of Aimulation--ased inference. Proceedings of the National Academy of Sciences, 1 117748):30055-30062, December 2020.
doi: 10. 1107:///a..112278/8.
URL /ttpp//swww.ppp.a..oooro//////00
1912789117
Peter Dayan and L. F. Abbott. Theoretical neuroscience: computational and mathematical modeling of neural systems. Computational neuroscience. Massachusetts Institute of Technology Press, Cambridge, Mass, 2001. ISBN 978-0-262-041999-7
Michael Deistler, Pedro J. Goncalves, and Jakob H. Macke. Truncated propos- als for scalable and hassle-free simulation-based inference. ral Information Processing Systems, 35:23135 23149, December 2022.
Advances in Neu-
URL
httpss////rocaaddnngggnooroosii https//aepppeeeeeeeeeeeaeeeeeeee httpe//eeeee///aa////////aaa/ 927 227 -ttp:aaaaoooooootaaaaaaaaaoa-o-- Dosttract-Cooferennee
Michael Deistler, Kyra L. Kadhim, Matthijs Pals, Jonas Beck, Ziwei Huang, Manuel Gloeckler, Janne K. Lappalainen, Cornelius Schröder, Philipp Berens, Pedro J. Gonçalves, and Jakob H. Macke. Jaxley: differentiable simulation enables large-scale training of detailed biophysical models of neural dynamics. Nature Methods, 22(12):2649-266 December 2025, ISSN 1548- -25-02295--0--- URL https:/ www. naturatueeeee com/ articles/
7105. doi: 10.1038/44599-025-.00000
841592- 025---005-000 02895-
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural Spline Flows. In Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. URL https: httttps://proceeedinns http:/uusss..cccppa////00//////1
.html
Conor Durkan, lain Murray, and George Papamakarios. On Contrastive Learning for Likelihood- free Inference. In Proceedings the 37th International Conference Machine Learning, pp. 2771-2781. PMLR, November 2020. URL httttps:/prroceediii m pww.sprsss/slll
durkan20a html
Joel Dyer, Patrick Cannon, and Sebastian M Schmon. Amortised (kelihooddfree inference for expensive time-series simulators with signatured ratio estimation. In International Conference on Artificial Intelligence and Staristics, pp. 11131-11144. PMLR, 2022.
Lasse Elsemüller, Valentin Pratz, Mischa von Krause, Andreas Voss, Paul-Christian Bürkner, and Stefan T. Radev. Does Unsupervised Domain Adaptation Improve the Robustness of Amortized Bayesian Inference? A Systematic Evaluation, May 2025, arXiv:2502.04949 [stat].
Yusuf Falola, Siddharth Misra, and Andres Calvo Nunez. Rapid High-Fidelity Forecasting for Geological Carbon Storage Using Neural Operator and Transfer Learning. ADIPEC, Abu Dhabi, UAE, October 2023. OnePetro. doi: 10.21 18/216135-MS.
David T. Frazier, Ryan Kelly, Christopher Drovandi, and David J. Warne. The Statistical Accuracy of Neural Posterior and Likelihood Estimation, November 2024. URL httt:://axxivv.or// arXiv:241 12668 [stat].
abs/ fos/2411..0 12068
J Friedman. On Multivariate Goodness-of-Fit and Two-Sample Testing. Technical Report SLAC- PUB- 10325, 826696, Stanford, January 2004. URL http::/wtwwwwttt. osti www......wo.eeeeeeett...s
Richard Gao, Michael Deistler, and Jakob H. Macke. Generalized Bayesian Inference for Scientific
Simulators via Amortized Cost Estimation, November 2023. URL http:/ttp://axxi.oo///o///
2305. 15208
Wulfram Gerstner and Werner M. Kistler. Mathematical fonmulations of Hebbian learning. Biological Cybernetics, 4705)4404 415, December 2002. ISSN 1432-0770, doi: 10.10077s00422-002--3333 URL //oi .org.orgggooo htt.p///s/.olllrrr///00000000020
12

---

Published as a conference paper at ICLR 2026
Wulfram Gerstner, Werner M. Kistler, Richard Naud, and Liam Paninski. Neuronal Dynam- ics: From Single Neurons to Networks and Models of Cognition. Cambridge University
Press, Cambridge, 2014. ISBN 978-1-107-00083--. doi: 0010017/080978100744461111
URL
www http:///wwwcaam...cccccccc http://wwwwwoootooooooo/o//o/o//
Michael B. Giles. Multilevel Monte Carlo Path Simulation. Operations Resseurch,56633:607777
June 2008. ISSN 0030-364X. doi: 10. 10.1287/opre http://oprr.110.4499..0 1070.0496. URL BtLL:tttpp::://poonniii
informs wttp://....////o///oooaaee
1287 htt.ooorre..o// 0496
Manuel Gloeckler, Michael Deistler, Christian Weilbach, Frank Wood, and Jakob H. Macke. All-in- one simulation-based inference, May 2024. JRL.tttt://aaxt org/ abs/2444 09636
Manuel Gloeckler, Shoji Toyota, Kenji Fukumizu, and Jakob H. Macke. Compositional simulation- based inference for time series, March 2025, arXiv:2411.02722 [cs].
Pedro Gonçalves, Jan-Matthis Lueckmann, Michael Deistler, Marcel Nonnenmacher, Kaan Öcal, Giacomo Bassetto, Chaitanya Chintaluri, William F Podlaski, Sara A Haddad, Tim P Vogels, David Greenberg, and Jakob H Macke. Training deep neural density estimators to identify mechanistic models of neural dynamics. eLife, 9:e56261,, September 2020. ISSN 2050-084X. doi: 10.7554/eLife.5626.. URL bttps:://oolt org/0000....00... /554/eeifffe.. 56261
David Greenberg, Marcel Nonnemmacher, and Jakob Macke. Automatic Posterior Transformation for Likelihood-Fres Free Inference. In Proceedings of the 36th International Conference on Machine Learning, pp. 2404 2414. PMLR, May 2019. URL http::://pprcceedngg....tt press/ V97/ 297/greebberggaa wa...g99gg.b. html
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two-Sample Test. Journal of Machine Learnnng Research, 13(25):723-77,, 2012. ISSN 1533-7928. URL http:////// http://...//pppe////////rtttttoe
html
Sam Griesemer, Defu Cao, Zijun Cui, Carolina Osorio, and Yan Liu. Active Sequential Posterior Estimation for Sample-Effice Efficifffienntttt E Simulation-Based Inference. In Neural Information Processing Systems, November 2024.
IoteId=Az QD4
Michael U. Gutmann and Jukka Corander. Bayesian Optimization for Likelihood- Inference of Simulator-Basse Statistical Models. Journal of Machine Learning Research, 17(125):1-4,,200 2016. ISSN 1533-7928.
Ehsan Haghighat, Maziar Raissi, Adrian Moure, Hector Gomez, and Ruben Juanes. A physics- informed deep learning framework for inversion and surrogate modeling in solid mechanics. Computer Methods in Applied Mechanics and Engineering, 379:113741, June 2021. ISSN 0045-7825. doi: 10.101 1..10166..cma222211144444 URL www.ssciencedirec......m com/
Zhong-Hua Han, Stefan Görtz, and Ralf Zimmermann. Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and generalized hybrid bridge function. Aerospace
Science and Technology, 25(1):177-189 March 2013. ISSN 1270-9638. doi: 10.1016/j.ast.
2012.01.006 URL https: wwwp///wwww//////////////w/ www.sseeiennccecc.c.cc... com/ http://ccienn/aattee/pp..../ $127096381200111
Isaac M. Held. The Gap between Simulation and Understanding in Climate Modeling. ametsoc. November 2005. doi: 10.1 10.1175/BAMS.A 86- 11-9 1609. URL https:/tt///// //opra////////a////// ame tssc
htt/vview/journal /ttp/////////////---a/55// htt:://aammm611--11-1--
xml
Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-free MCMC with Amortized Approximate Ratio Estimators. In Proceedings of the 37th International Conference on Machine Learning, pp. 4239-4248. PMLR, November 2020. URL https:/pproeedngg mlr
20a. html
Yuga Hikida, Ayush Bharti, Niall Jeffrey, and François- Xavier Briol. Multilevel neural simulation- based inference, August 2025. arXiv:2506.066m 060687777 [stat] version: 2.
13

---

.wr...m.r.... Published as a conference paper at ICLR 2026
A. L. Hodgkin and A. F. Huxley. A quantitative description of membrane current and its appli- cation to conduction and excitation in nerve. The Journal of Physiology, 107 544, Au- gust 1952. ISSN 0022-3751. URthttps:/ http::/wwwwwbbbbbb nmm....mmmmm.... httpa///........o///////rrriaaa. PMC1 392413/
Vladimír Holy and Petra Tomanová. Estimation of Censtein-Uhlenteee Process Using Ultra-Hight Frequency Data with Application to Intraday Pairs Trading Strategy, July 2022. arXiv:181L.095)
[q-fin].
Mathias Hoppe, Ola Embreus, and Tünde Fülöp. DREAM: fluid-kinetic framework for
tokamak disruption runaway electron simulations. Computer Physics Communications, 268: 108098, November 2021. ISSN 0010-4655. doi: 10.1016/j.cp.220111080999 URL https
com/ http:///wwo.............../o.//a
Marko Järvenpää, Michael U. Gutmann, Arijus Pleska, Aki Vehtari, and Pekka Marttinen. Efficient Acquisition Rules for Model-Based Approximate Bayesian Computation. Bayesian Analysis, 14 (2):595-622, June 2019. ISSN 1936-0975, 1931-6690. doi: 2211/18--B1122
Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, and Barnabás Póczos. Multi-fideliity Bayesian Optimisation with Continuous Approximations. In Proceedings of the 34th Interna- tional Conference Machine Learning, pp. 1799-1808. PMLR, July 2017. URL https
//proceedings. mlr wwwpp/eesppa.. http://0//aaaaanyyaaaaaaaaa 17a.
Diederik P. Kingma and Jimmy Ba. Adam: Method Stochastic Optimization, January 2017. URL http:://axxi..oo/tt/////// wwtp://gggabbbbaoooo
6980. arXiv:1412.691 [es].
Supeng Kou, Benjamin Olding, Martin Lysy, and Jun Liu. A Multiresolution Method for Parameter Estimation of Diffusion Processes. Journal of The American Statistical Association J AMER STATIST ASSN, 107:4, December 2012. doi: 10.
Andrew K Lampinen and Surya Ganguli. An analytic theory of generalization dynamics and transfer learning in deep linear networks. arXiv preprint 1rXiv:1809.10377 2018.
Diane Larsen-Freeman. Transfer of Learning Transformed. Language Learning, 63(s)):107 129, 2013. ISSN 1467-9922. doi: 10.1 10.1111/j 1111111114499992222220044444 URL https:// online iilineeibbaryy wi ley. www/cooiioooooo abs 11000.11111110 http://aa.abbs.11111111.11666119
007...004...
Pablo Lemos, Adam Coogan, Yashar Hezaveh, and Laurence Perreault-Levasseur. Sampling Based Basedd Accuracy Testing of Posterior Estimators for General Inference, June 2023. URI. http://
arxiv. mrxiv.org/aiee abs 2202 03026 arXiv:2302.03026 [stat].
David Lopez-Paz and Maxime Oquab. Revisiting Classifier Two-Sample Tests. In International Conference on Learning Representations, Toulon, France, 2017. International Conference on Learning Representations. doi: 10.48550//2Xiv.1610..666
Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnen- macher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. In Advances in Neural Information Processing Systems, volume 30. Curran As- sociates, Inc., 2017. URI. http::/::/ppppes..nniittt nips cttpssccccpppppppeeeeeeee http://pppprrfleeeeppppeeeeee /ssplepprp//1111/// www.dddfaa772533c hat.aa/2222222222222222222111223 html]
005cb-
Jan-Matthss Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H. Macke. Likelihood- free inference with emulator networks. In Proceedings of The Jst Symposium on Advances in Approximate Bayesian Inference, pp. 32-53. PMLR, January 2019. URL https:// 11t.////eeccaamannmmmmmmmm 9a. html
Van-Mathis Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Benchmarking Simulation-Base Inference. In Proceedings of The 24th International Con- ference on Artificial Intelligence and Statistics, pp. 443-351. PMLR, March 2021. URL httpps://proceet: w.r.r.prsssee. 30/1ueckmamnnn html
14

---

Published as a conference paper at ICLR 2026
Andrew J. Majda and Boris Gershgorin. Quantifying uncertainty in climate change science through empirical information theory. Proceedings of the National Academy of Sciences of the United States of America. (7924))14955-14900 2010. ISSN 0027-8424. URL httphhttp:////t jstor
Norman Marlier, Olivier Brüls, and Gilles Louppe. Simulation- based Bayesian inference for multi- fingered robotic grasping, September 2021. UKt.hhttpe//arrrr.rrr//////222..
arXiv:2109,1427. [es] version: 1.
Luca Martino, David Luengo, and Joaquín Míguez. Pejept-Reject Methods. In Luca Martino, David Luengo, and Joaquín Míguez (eds.), Independent Random Sampling Methods, pp. 65- 113. Springer International Publishing, Cham, 2018. ISBN 978-3-319-72634-2. doi: 10. 978-3-319-72634-2.5 URL htt.http////// /tmpss://dd1 //wii.dd..o/////o/o....o... .rg. /trg/1...0 wttp://...0//770-33311111 -
Aimee Maurais, Terrence Alsup, Benjamin Peherstorfer, and Youssef Marzouk. Multifidelity Covariance Estimation via Regression on the Manifold Symmetric Positive Definite Matri- ces. SIAM Journal Scientific Computing, 2023. doi: 10.48550ARXIV.230.11
URL
wttp://g/a/////////aa 12438
Benjamin Kurt Miller, Alex Cole, Patrick Forré, Gilles Louppe, and Christoph Weniger. Truncated marginal neural ratio estimation. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, pp. 129-143, Red Hook, NY, USA, December 2021. Curran Associates Inc. ISBN 978-1-7138-4539.1
Barry L. Nelson and Linda Pei. Foundations and Methods of Stochastic Simulation: A First Course, volume 316 of International Series in Operations Research & Management Science. Springer International Publishing, Cham, 2021. ISBN 978-3-030-86193-3 978-3-030-86194-
0. doi: 000110077778-3000-66111
URL https:/ spri nger. com/ 10007
978 --3--3300---11---3 8419400
Fabio Nobile and Francesco Tesei. A Multi Level Monte Carlo method with control variate for elliptic PDEs with log-normal coefficients. Stochastic Partial Differential Equations: Analysis and Com- putations, doi
September 2015. ISSN 2194-041X. doi: 10. 10.10077440072--15--0055999
URL
1007/a 40072 015 0055-9
Santisudha Panigrahi, Anuja Nanda, and Tripti Swarnkar. Survey on Transfer Learning. In Debahuti Mishra, Rajkumar Buyya, Prasant Mohapatra, and Srikanta Patnaik (eds.), Intelligent and Cloud Computing, pp. 781-789, Singapore, 2021. Springer. ISBN 978981 9789811559716 1559716. doi:
111007/978-881-55-558
George Papamakarios and lain Murray. Fast 'epsiilo -free Inference of Simulation Models with Bayesian Conditional Density Estimation. In Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc., 2016. URL 12012268822 22228888001111155566666666666333
neur cc/
paaper201 maner//216/haaae www.sh/aahhaaeeeaaa
.html
George Papamakarios, David Sterratt, and lain Murray. Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, pp. 837-848, PMLR, April 2019. URL htt.tttsss/prrcccedingge / /treess/ss/ppppaaarrioss//// htmml
mlr
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019, UBLnntoosr//proceeiinn...s neursps wttp:ccpppppp..e.ee//ee / http:///ce...aaa.ooooorrr...ppaa
Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. Optimal Model Management for Multifidelity Monte Carlo Estimation. SIAM Journal on Scientific Computing, 3815): A3194, January 2016. ISSN 1064-8275, doi: 10.1137/15M1044472.
URL https: /epubs)
i am. wwwm:org//oo/aaaa http///iaabss00/ 1tt.a///........1.... 1137/1 046472
15

---

Published as a conference paper at ICLR 2026
Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization. SIAM Review, 60(3):550 591, January 2018. ISSN 0036- 1445. doi: 10. 10.1137/16M1002249, URL https:// siam. org/
doi/10. 110.113330 1137/1 6M1 082469
Gabriel Peyré and Marco Cuturi. Computational Optimal Transport. Working Papers, October 2017.
Number: 2017-86 Publisher: Center for Research in Economics and Statisties.
Jonathan Pillow and James Scott. Fully Bayesian inference for neural models with regative-binomin spiking. In Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
Lutz Prechelt. Early Stopping But When? In Lecture Notes in Computer Science, vol 7700. Montavon, G., Orr, G.B., Müller, KR. (eds) Neural Networks: Tricks the Trade. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, January 2002, doi: 10.1007733-5004444000 URL https: springerisss.. www.innger.rrccoooooooeeeeeoeeee wwwp////tte///..cccoooo wttpeeerro..1000//00 1007/3 540 49430-822
Thomas Prescott and Ruth E. Baker. Multifidelity Approximate Bayesian Computation. SIAM/ASA
Journal on Uncertainty Quantification, 8(1):114-13, January 2020. doi: 10.1137/18M1229442
Thomas P. Prescott and Ruth E. Baker. Multifidelity Approximate Bayesian Computation with Sequential Monte Carlo Parameter Sampling. SIAM/ASA Journal on Uncertainly Quantificatin, (2)7788 817, Jamuary 2021. doi: 10. N0.1177200M1166660 (02l/bs/10 1111100.1137/2011161611
URL (Rthps::://eet /epubs siam. org/
Thomas P. Prescott, David J. Warn, and Ruth E. Baker. Efficient multifidelity likelihood-free Bayesian inference with adaptive computational resource allocation. Journal of Computational Physics, 496: 12577, January 2024. ISSN 0021-9991. doi: jop.006/j..pp.2023... 112577..
J K Pritchard, M T Seielstad, A Perez-Lezaun, and M W Feldman. Population growth of human Y chromosomes: study Y chromosome microsatellites. Molecular Biology and Evolution. 16 (12):1791-1798 December 1999. ISSN 0737 doi: 10. 1009///ofodoooumals.mooooo molbev.a026091 URL httphhttt///ooito
wos.s.molloooo wwmbool........ a026091
Wilfrid Rall. The Theoretical Foundation of Dendritic Function: Selected Papers of Wilfrid Rall with Commentaries. MIT Press, 1995. ISBN 978-0-262-19356-6 Google-Books- ID: Nx5fb82827oC.
Poornima Ramesh, Jan- Matthis I Lueckmann, Jan Boelts, Alvaro Tejero-Cantero, David S. Greenberg, Pedro Goncalves, and Jakob H. Macke. GATSBI: Generative Adversarial Training for Simulation- Based Inference. In International Conference on Learning Representations, October 2021. URL
htttps://ppezreviiee Bat.aaaee......ouuo........CCGCC
wttpeeeoouemiieeeeeeeeeeeeeeeeee xR1hC6j48T3
Francois Roset. Zuko Normalizing flows in PyTorch, November 2024. URL https://github>
com/ conm/robabillstee. wwwaaaiiiist
zuko
DB RUBIN.
Using the SIR algorithm to simulate posterior distributions. Bayesian statistics 3. Proceedings of the third Valencia international meeting, 1-5 June 1987, pp. 395-402, 1988. Publisher: Clarendon Press.
Alex A. Saoulis, Davide Piras, Niall Jeffrey, Alessio Spurio Mancini, Ana M. G. Ferreira, and Benjamin Joachimi. Transfer learning for multifidelity simulation- inference in cosmology, May 2025. arXiv:2505.21215 [astro-ph].
Marvin Schmitt, Desi R. Ivanova, Daniel Habermann, Ullrich Köthe, Paul-Christian Bürkner, and Stefan T. Radev. Leveraging Celf-Consistency for Data-Efficient Amortized Bayesian Inference, July 2024a. arXiv:2310.04395 [cs].
Marvin Schmitt, Valentin Pratz, Ullrich Köthe, Paul-Christian Bürkner, and Stefan T. Radev. Consistency Models for Scalable and Fast Simulation- -Bssed Inference, November 2024b. arXiv:2312.05440 [es].
Jialin Song, Yuxin Chen, and Yisong Yue. A General Framework for Multi-fidelit Bayesian Opti- mization with Gaussian Processes. In Proceedings of the Twenty-eccond International Conference Artificial Intelligence and Statistics, pp. 3158-3167 PMLR, April 2019. ISSN: 2640-3498.
16

---

Published as a conference paper at ICLR 2026
Javan Tahir, Surya Ganguli, and Grant M. Rotskoff. Features are fate: theory of transfer learning in high- nigh-dimensional regression, October 2024. URL htttt:///axaii..ooo wwtp:///.rr///a//oa/oooooooaaaas 2tt.//211...1....... 08198
Sean Talts, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. Validating
Bayesian Inference Algorithms with Simulation-Based Calibration, October 2020. URL http:
//arxiv. org/ abs aba 06788
S. Tavaré, D. J. Balding, R. C. Griffiths, and P. Donnelly. Inferring coalescence times from DNA sequence data. Genetics, 145(2):505-515 February 1997. ISSN 0016-6731. doi: 10. /0.1ee///tnetics/ 145.2.505.
Leander Thiele, Adrian E. Bayer, and Naoya Takeishi. Simulition-Efficientt Cosmological Inference with Multi-Fidelity SBI,July 2025. MtXiv:2507.005111 [astro-ph].
Nilesh Tripuraneni, Michael Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity. Advances in neural information processing systems, 33:7852-7862, 2020.
Werner Van Geit, Michael Gevaert, Giuseppe Chindemi, Christian Rössert, Jean-Denis Courcol, Bilif B. Muller, Felix Schürmann, Idan Segev, and Henry Markram. BluePyOpt: Leveraging Open Source Software and Cloud Infrastructure to Optimise Model Parameters in Neuroscience, Frontiers in Neuroiinformatiics. 10, 2016. ISSN 1662-5199
URL Stlthtts:: /wwww frontiersin.
articles/10. 3389/ fninf. 2016. 00017
Huy D. Vo, Zachary Fox, Ania Baetica, and Brian Munsky. Bayesian Estimation for Stochastic Gene Expression Using Multifidelity Models. The Journal of Physical Chemistry: B, 123(10):217-2234, March 2019. ISSN 1520-5207. doi: 10. 10.102//acs. http//acs..pppbbbpp/ jwppp.pbboooo.. 109966.
T. P. Vogels, H. Sprekeler, F. Zenke, C. Clopath, and W. Gerstner. Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science (New York, N. Y.J, (34460662)1156911555 December 2011. ISSN 1095-9203. doi: 10. 10.1126/scienee.211 1095.
Xinming Wang, Simon Mak, John Miller, and Jianguo Wu. Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators, October 2024. URL http::ttt::tett.
org/ a bs 241 0 12690
David J. Warne, Thomas Prescott, Ruth E. Baker, and Matthew J. Simpson. Multifidelity multilevel Monte Carlo to accelerate approximate Bayesian parameter inference for partially observed stochastic processes. Journal of Computational Physics, 469:111544 November 2022. ISSN
00219991. doi: j.p0://.jpp.2222.... 1543.
Chulhee Yun, Shankar Krishnan, and Hossein Mobahi. A unifying view on implicit bias in training linear neural networks. arXiv preprint arXi::2010.02501, 2020.
Zahra Zanjani Foumani, Mehdi Shishehbor, Amin Yousefpour, and Ramin Bostanabad. Multi fulti-fidelity cost-aware Bayesian optimization. Computer Methods in Applied Mechanics and Engineering, 407:115937, March 2023. ISSN 0045-7825. doi: 10. 10.1016/j.cmm 10101//..mm.2223...... 1mm..002..11533. 115937.
Jice Zeng, David Barajas-Solano, and Hui Chen. Generative Al-enhanced Probabilistic Multi-Fidellty Surrogate Modeling Via Transfer Learning, January 2026. URL URL.itttp://rrxwvvo////3 1602.0001 arXiv:2602.00072 [cs].
Xiaoshu Zeng, Gianluca Geraci, Michael S. Eldred, John D. Jakeman, Alex A. Gorodetsky, and Roger Ghanem. Multifidelity uncertainty quantification with models based on dissimilar parameters. Computer Methods in Applied Mechanics and Engineering, 415: 16205, October 2023. ISSN 088.4444448844448 08644
00457825. doi: 10.1016f..cma.20222 2023. 116205. BalLattp://aravv..rrr//baa
Friedemann Zenke and Wulfram Gerstner. Limits to high-speed simulations of spiking neural networks using general- purepse computers. Frontiers Nearoinformatics, 8:76, 2014. ISSN 1662-5196. doi: 10.3389/fminf.0144.000
17

---

Published as a conference paper at ICLR 2026
A
USAGE OF LLMS
LLM usage was minimal, limited to grammar refinement, sentence shortening, code cleanup and discovering papers outside our main domain.
B PROOF OF CONVERGENCE OF THE NPE LOG-LIKELIHOOD LOSS
Let 0g p(0:) be samples from the prior of high-fidelity model, and æ; p(x (0.) be the respective high-fidelity simulations. In NPE, we define the loss function as the negative log likelihood:
N
[()) N
log 4s (0,(2)),
where 0g are samples from the prior distribution, 2; are the respective simulations (i.e., samples from p(z[0,)), and & are the parameters of the neural density estimator be optimized. If we let the number of samples 0; (and respective simulations) N oo:
L(Φ) = [x(0))(110) [- log do (0)z))
(p(z))(()a)) | log 4s [0]x]]
Ex(Olz) log p(0|z)
Ey(a))
+
2o (f/zz)
where is a constant with respect to é. Minimizing L(Ø) with respect to & is thus equivalent to minimizing the KL divergence between the true posterior distribution and the estimated posterior in the limit of an infinite number of high-fidelity samples.
18

---

Published as a conference paper at ICLR 2026
C FURTHER EXPERIMENTAL DETAILS
C.I TRAINING PROCEDURE
All methods and evaluations were implemented in PyTorch (Paszke et al. 2019). We used the Zuko package (version 1.4.0, MIT License] Roset 2024) to implement the normalizing flow, based on the Neural Spline Flows (NSF) architecture (Durkan et al. 2019), and the SBI package (version 0.24.0, Apache 2.0 license 3 Boeltt et al., 2024) for additional functions. The parameters used to generate simulations were logit-transformed for numerical stability, and the summary statistics were z-scored to improve the performance of the normalizing flows. The loss function is the negative-log likelihood, and the optimization function is the Adam optimizer (Kingma & Ba 2017
The Neural Spline Flow (NSF) architecture consists of 5 transformations, each parametrized with 50 hidden units and 8 bins. The batch size was set to 200, and the learning rate to 5 x 10-4, The ialn-validation fraction is 0.1, and training of the NSF utilized an early stopping criterion with patience of 20 epochs for the early stopping criterion. The settings described above are all default settings of the SBI package the time of the method's development (Boelts al., 2024).
Note, the stopping criterion follows the default configuration of the SBI package, which is defined as follows: Let F. be the error function of the training algorithm (negative log likelihood), Eevat (t) the validation error at epoch t, which is used by the stopping criterion. The value Eope is the lowest validation set error obtained in epochs up to t:
Eopt (t) == min Eval
(5)
St
The early stopping criterion S tterminates training once the validation error (ta has increased for p consecutive epochs (the patience parameter). At this point, the model corresponding to the lowest validation error observed that far, Exp (l), is selected and returned.
Rather than fixing the number of training epochs, the idea behind early stopping that when the validation error has increased not only once, but over p consecutive steps, such an increase indicates a stage of overfitting (Prechelt, 2002). Note that if the patience is too small, underfitting might occur, and training may terminate too early due to stochastic fluctuations in the loss. Similarly, overfitting might likely occur when the patience set to excessively high numbers (especially with a low number of simulations, since the loss function is typically more variable in this setting).
For the moc-ttnnng step of MF-NPE, no network weights were frozen. This choice has been purposely made to maintain full flexibility of the network to adapt to the high-fidelity model.
For the evaluation of MF-TSNPE-AF, we used rounds of active sampling, where 80% of the high-fidelity dataset was used for standard MF-NPE training, and 20% was split across the rounds of active sampling. The active samples were selected using the acquisition function over an ensemble of 5 networks.
For a fair performance comparison, all methods were trained on the same datasets and evaluated on the same observations Fo.. All amortized results were obtained over 10 network initializations, and all non-amortized results over I or 10 network initializations (depending on the computational cost of the task). We evaluated the methods over 30 observations for the C2ST metric, more than the 10 observations chosen previously for benchmarking (Lueckmann et al. 2021). This choice is motivated by our focus on evaluating the methods in low-data regimes, where greater certainty is required. The performance on the L5PC neuron task was evaluated with the metric NLTP and over 100 x, 's. Here, the performance of the amortized methods was averaged over 10 network initializations, and in the non-amortized methods over 1 network initialization, since training had to be performed for each individual H.. The performance of the methods on the recurrent spiking network task was averaged over 10 network initializations and evaluated over 262,008 observations, which was the maximum number of available samples for this high-dimensional problem.
htteas://ithubb.co///rrbbbbbssss httsss//ggtthob..comm//--ddd///
19

---

Published as a conference paper at ICLR 2026
D TASKS
D.1 OU PROCESS
(OU) process is a high-fidelity model with to 4 free parameters that contains a temporal structure in the observations, As low-fidelity model, we chose i.i.d. samples from a Gaussian distribution (unstructured vector), parametrized by the mean and standard deviation. This setting makes it well-suited to examine the impact of parameter space overlap between the low- and high-fidelity models, as well as the impact of systematic bias in the posterior of the low-fidelity model on transfer learning.
The
The Crnstein-Uhlenben process models a drift-diffusion process of particle starting at position X(O) and drifting towards an equilibrium state. The model has two main components: drift term and diffusion term:
High-fidelity model
Xit)
Omntein-Uhlenbeck process (HF)
X00
dx,- 70 X dd
odW
difltssion
where j is the mean of the asymptotic distribution over positions X, the magnitude of the stochasticity of the process and is the convergence speed. (0) is the initial position of the process, which we assume be stochastic: (0) (N + Moffeet, 1). The parameters of interest that we aim to estimate are μ. o, Poffset-
The four parame- ters of the Ornstein-Ullenbeck process: the mean μ., standard deviation a, convergence rate Y and Moffset: which is the difference between the initial condition (()) and mean μ.
Figure S:
The Ornstein- Uhlenbecc process was approximated with the Euler- Maruyama method:
( + 0t) = (t) + fann X) öt + faimusioo (f, VO,
Starting from the exact likelihood for the Ornstein-Uhlenbeck pro- cess given by Kou et al. 20122)
exp {- (Iu xx)x -1-79 x
1 go²
Sexact ti [
p4 0))
II
79
Vago
where g = (1 - exp( 22p 22y we modify it by incorporating an additional parameter Hoffset to account for stochastic X X()).
The full likelihood fexuat ta [ ,1, G. 7/ Mollset) is given by
{
(x (11 + Moftsee))
}
fexuct [
[4, 0, (ellee))
cxp
fexant hi
X (,
v 2m
iidd Gaussian samples
Low-fidelity model As a low-fidelity model, we use i.i.d. Gaus- sian Samples. At convergence, the distribution over X; approaches a Gaussian distribution with mean μ and standard deviation = V27"
our setup, we chose a low-fidelity model that corresponds to time- independent random draws from Gaussian distribution with mean Ao and standard deviation oa:
Pn
X, N o
(6) Figure i.i.d. Gaussian sam- ples with mean μn and stan- dard deviation øt.-
The posterior distribution over the parameters of the low-fidelity model has a biased mean influenced by the initial position Hoffset and convergence speed 7.
20

---

Published as a conference paper at ICLR 2026
Prior
# ~ (0.1,)), a~ (0,,001) ? 7~ 10,, Poffsct (0(
HF Simulator
x - (f1,..., To Nu + Moffsets 1), where
dr; r(p na rt odW;
LF Simulator
x0 (211 (0:--.;;:00) 010), Ij N (1ib: ob)
HF Dimensionality
A E R x XER!. U(x) E R0
LF Dimensionality
0 E R XERO ( E IR
References
Holy & Tomanová 2022 Carter & Strey 2023 Kou et al. 2012)
For the two-dimensional experiment, the free parameters 7/ Moffset have been fixed to 7 0.5 and Hollset = 3.0. For the three- ciimensionallexperineeee
only Polfset = 3.0. The summary statistics (x) from the high-fideliyy model consists of 10 uniformly distributed subsamples drawn from trace of 101 timesteps. Parameters and summary statistics are illustrated in Figures andd 6
D.2 POSTERIOR DISTRIBUTIONS OVER OU PROCESS
TRUE
NPE
MF-NPE4
MF-TSNPE4 A-ME-TSNPE4
100
1000
Figure 7: Posterior density estimates for single observation from the OU process with two free parameters (OU2). The orange contour lines contain 68% of the probability mass of the true posterior distribution.
D.3 OU PROCESS WITH VARYING PARAMETER SPACE
We present a comparison of our multifidelity approaches to NPE and MF-ABC, with different numbers of pre-trained low-fidelity simulations. MF-NPE3 is pre-trained of size 10 while MF-NPE4 and MF-NPES use datasets of 10 and 10 low-fidelity simulations, respectively. The MF-ABC results suggest that neural density approaches scale better to complex problems Frazier et al. 2024).
low-fidelity dataset
OU2
OUB
OU4
0.9
0.8
C2ST
0.7
0.8
0.5
10
10a
10f
10a
50 10
10
10a
10a
50
10a
10a
10"
10a
Number of high-fidelity samples
varlations
(Active) TSNPE varlations
... NPE
MF-ABC
MH- NPE3
MI-NPE4 - MS- NPES
MI-ISNPE4
MF-T5NPEA-AF
Figure 8: MF-NPE benefits from larger low- low-fidelity datasets. We ran MF-ABC with hyperparameters 11, and n (0.9, 0.3) (more details in Appendix E.I.Ip. All variants of our method perform better than MF-ABC and NPE.
21

---

Published as a conference paper at ICLR 2026
D.4 INFERRING THE PARAMETERS A GAUSSIAN MODEL PRETRAINED THE OU3 MODEL
In this example, we examine how the performance changes when the low-fidelity model has larger number of parameters than the high-fidelity model: the low-fidelity model is the Ornstein-Uhlenber process with three parameters, and the high-fidelity model corresponds to i...d. Gaussian samples parameterised by mean and variance (so, only two parameters). To accomplish that, the density estimator pre-trained on the low-fidelity model was fine-tuned only on the dimensions of the high- fidelity and the extra dimension was kept as a dummy dimension. NPE was directly trained on the 2-dimensional parameter space of the high- figh-fidelity model. At inference time, the posterior evaluation was performed only on the high-fidelity parameter dimensions. We observe that when the dimension of 0 is smaller than the dimension of 0,, transfer learning provides significant improvement in performance.
Gaussian samples
1.0
0.4
0.3
C2ST
0.8
MMD
0.7
0.1
0.6
0.5
50
10a
10a
10c
10P
50
10=
10
10
10a
Number of high-fidelity simulations
NPE
MF-NPE4
Figure 9: Evaluation with C2ST and MMD over a two-dimensional Gaussian Samples model, pretrained on the dimensional OU process model.
22

---

Published as a conference paper at ICLR 2026
D.5 SLCP
Simple Likelihood Complex Posterior (SLCP) is benchmark inference task that has been artificially designed to have simple likelihood, but very non-trivial S-dimensional posterior infer. In this example, we study the impact of multifidelity in cases where the dimensionality of the parameter space differs between the low-fidelity and high-fidelity models.
High-fidelity model The SLCP problem involves five parameters. The prior distribution is uniform across a five-dimensional parameter space, and the observations consist of four two-dimensional samples drawn from Gaussian distribution. Both the mean and the variance of this Gaussian depend on the parameters through nonlinear mappings. The high-fidelity model follows the code the SBI benchmarking paper (Lueckmann et al. 2021).
Low-fidelity model In the low-fidelity model, we experimented with the effect of different numbers of parameters on the inference quality. We fixed my O, and kept the parameters of Sg free.
Prior
(,33)
HF Simulator
x 0 = (#1...., ..),,
~ N(mo, So),
01
of
18182
where mg
S =
02
88182
82
with $1 = f8.. 82
0g, P tanh (0s).
LF Simulator
x (I1, ......,,y)) 14), I (O, So),
st
p8182
where S.-
88182
82
with
0g. 82 07 p (anh(Os).
HF Dimensionality
i R
R
LF Dimensionality
E R3 Papamakarios et al.
x E R
References
2019 Hermans et al. 2020)
Durkka et al. 2020
Greenberg et al. 2019 Lueckmann et al. 2021
Thiele et al. 2025
23

---

Published as a conference paper at ICLR 2026
D.6 SIR
The Susceptible, Infected, and Recovered (SIR) model is a classical epidemiological benchmark example that captures the spread of infectious diseases through three interacting compartments: Susceptible (S), Infectious (I), and Recovered (R). Its dynamics are governed by the system of ordinary differential equations. The model is parameterized by two rates: the infection rate 8 and the recovery rate 7, We investigate how multifidelity addresses the partly observed dynamics of the model. Rather than observing the three dynamics of the SIR model (following the setup of the SBI benchmarking (Lueckmann et al. (021), we assume that no dynamics regarding the recovered subjects are known (SI model).
Low-fidelity model In the low-fidelity model, assume information is available about the dynamics recovered individuals. The total population size and the initial conditions are kept consistent with the high-fidelity model.
Bounded domain
[0.001,3]
Prior
3 ~ (.gNo) ? Log
HF Simulator
x 0 (151...,,55),
I:/N equally spaced,
dS
= -B SI
dI
SI
dR
is simulated from dt
3
71,
N
dt
N
dt
LF Simulator
0 (I), I50),
I/N
equally spaced, dI
dS
ST
ST N 7I,
is simulated from dt
=
- 3
N
dt
Dimensionality
i E R.2 x E 33 U(x) E IR
Fixed parameters
Population size N 10 duration of task T 160 days.
Initial conditions: (S(0), T(0), R(O)) = (N- - 1,
References
(Lueckmann et al. 202 Greenberg et al.
2019
(Hermans et al. 2020 Durkan al. 2020
Summary statistics (xx are 10 subsamples from the trace.
24

---

Published as a conference paper at ICLR 2026
D.7 IMAGE EXAMPLE
We apply our method to problem with high-dimensional observations, and explore the benefits of transfer learning in combination with embedding networks. The high-fidelity model is 256x256 image, while the low-fidelity model has a resolution of 32x32. An example of both simulator outputs shown in Fig. 10
High-fidelity model The Gaussian Blob image example contains high-dimensional observations that have been embedded with CNN embedding from the SBI package (Boelts et al.) 2024). The model renders 2D image, which we modeled
256 256 pixel image of Gaussian blob, and aiming to infer three parameters (Hett' Jeft, 7): the horizontal and vertical displacements of the blob, and its contrast (Lueckmann et al. 2019) The image is in grey-scale and is generated through binomial distribution with a total count of 255 and probability Pij. as described Lueckmann et al. (2019).
Low-fidelity model
In our setup, the low-fidelity model generates spatially low-resolution dataset (32x32 image). We upscale these images using interpolation techniques and provide the resulting low-resolution inputs to the embedding network (z:).
Prior HF
Toff: Voff 2562)
Y ~ ((022.2)
Prior LF
Toff, Voff 320, ? ~ ((02,22)
Simulator
x , T1024); where,
Tay (,n5, Psy)
Pay = 0.9 0.8 exp 0.5(rzw/009)
Tay = (xr - mon)2 + (y - yan)) )
Dimensionality HF
0 E R3 E 256x266 Uxx) E R332
Dimensionality LF
0 E R R 32x32 U(x) E 13
Fixed parameters
Standard deviation du 2, The 12
References
[Lueckmann al. 2019
25

---

Published as a conference paper at ICLR 2026
E GAUSSIAN BLOB EVALUATION
(a) Original image
(b) low-fidelity simulations
Figure 10: Five examples of generated images with the Gaussian Blob across the two fidelities, with (a) the original 256x256 high-fidelity simulations, (b) the upsampled 32x32 low-fidelity simulations.
GaussianBlob
0.4
12
NRMSE
0.3
NLTP
10
8
0.2
6
0.1
4
6
8
10
50
10a
10
Number of high-fidelity simulations
baseline
MF-NPE variations
NPE
MF-NPE3
MF-NPE4
Figure 11: Method comparison with NLTP and NRMSE for the Gaussian Blob task. Evaluated over 10000 observations.
26

---

Published as a conference paper at ICLR 2026
E. DATA GENERATION AND TRANSFORMATIONS FOR INCREASED NETWORK PERFORMANCE
During the performance evaluation, we encountered numerical instabilities, particularly with NPE in low-simulation budgets: substantial proportion of the estimated probability density was placed outside of the uniform prior bounds, a phenomenon dubbed 'leakage' that has been previously documented (Greenberg et al. 2019 Deistler et al. (022). Dogitttransfommng the model parameters before training the density estimator resolved the issue.
This transformation creates mapping from a bounded an unbounded space, resulting in density estimation within the prior bounds after the inverse transformation. In addition, the summary statistics of the simulations were z-scored for improved performance of the density estimator, the default setting in the SBI package (Boelts et al. 2024).
27

---

Published as a conference paper at ICLR 2026
E.1.1 MULTIHIDELITY APPROXIMATE BAYESIAN COMPUTATION (MF-ABC)
We translated into Python publicly available Julia implementation of the multifidelity ABC algo- rithm (Prescott & Baker 2020). In our setup, the adaptive sampling scheme of MF-ABC selected approximately 30% of the batch size as high-fidelity samples in the OU2 and OU3 tasks, and 50% in the OU4 task. To ensure consistency with our neural network experiments, we z-scored the simulator output before inference. We also explored the effect of varying the acceptance threshold c. We found that the hyperparameters slightly affect the performance of MF-ABC, but that MF-NPE always shows superior performance than MF-ABC (Figure (2). However, MF-ABC has several other perparraaeeerss to tune. We cannot exclude the hypothesis that larger performance gains could be obtained from such an approach by a more extensive hy hyperparameter search.
Varying e in OU3 task
1.0
(0.5,
0.9
g (0.5,
(1, 1)
0.8
(2, 1)
C2ST
= (2, 2)
0.7
0.5
8
10"
Number of high-fidelity simulations
Figure 12: C2ST results for MF-ABC with varying hyperparameters e. Mean and 95% confidence interval.
MF-ABC posteriors ABC-based methods typically require a significantly larger number of samples for convergence (Lueckmann et al. 2021 Frazier et al. (024).. line with previous studies, we find that 10s samples are not yet enough for MF-ABC converge to good estimate of the posterior in the OU2 task.
True posterior
MF-ABC
50
10a
10a
10"
Jolkynat k.
Figure 13: Comparison between MF-ABC posterior estimates and the true posterior: Results for the Ornstein- Menstein-Uhlenbeck process with two free parameters. Posterior estimates are shown for varying numbers of high-fidelity simulations (50, 100, 103, and 104).
28

---

Published as a conference paper at ICLR 2026
F TASK 2: MULTICOMPARTMENTAL SINGLE NEURON MODEL
The response of a morphologically detailed neuron to an input current is typically modeled with multicompartmennta neuron model wherein the voltage dynamics of each compartment μ are based on Hodgkin-Huxley equations (Hodgkin & Huxley 1952):
dV, i/t + + gp (Vx+1 Vip)
It
Om dt
Ap
+ 9p (Va-1-Vp) Vp).
The total membrane current im for specific compartment is the sum over different types of ion channels i, such as sodium, potassium and leakage channels:
m = Iham h(V ENa) + 9KT Ek) + Iv - Et) + IMP(V -EM)
(8)
We interested in inferring the densities of prominent ion channels INa and ğK-
The low- and high-fidelity models differ in the number of compartments per branch: the low- fidelity model has single compartment per branch, while the high-fidelity model consists of eight compartments per branch.
All simulations were performed using Jaxley (V 0.8.2) (Deistler al. 2025) over 120 The injection current is a step current of 0.55mV over 100 ms, with a delay of 10ms. The step size of the simulator 0.025.
When sampling from the prior distribution over parameters, approximately 0.05 0.1% of the respective simulations had clearly unrealistic summary statistics: these simulations were iteratively replaced by random draws from the prior distribution/propoaall or active learning list (depending on the algorithm) until we collected a desired number of valid simulations.
29

---

Published as a conference paper at ICLR 2026
F.1 NRMSE EVALUATION
In addition to the NLTP metric, we demonstrate that the NRMSE metric yields results that support conclusions.
L5PC
0.4
baseline
NPE
0.3
NRMSE
MF-NPE variations
MF-NPE3
0.2
MF-NPE4
MF-NPES
0.1
MF-TSNPE variations
MF-TSNPE4
ME-TSNPE4-AF
50
10
10
10a
MF-TSNPES-AF
Number of high-fidelity simulations
Figure 14: NRMSE evaluation for the multicomparrmental neuron model.
F.2 SIMULATION-BAN BASED CALIBRATION AND POSTERIOR DISTRIBUTIONS
NPE
MF-NPE3
MF-NPE4
MF-NPES
empirical cdf
10a HF simulations
Expected under uniformity
posterior ranks
NPE
MF-NPES
9%a
9k
ONO
9K
Figure 15: Simulation- -based calibration (left) and respective posterior distributions for NPE and MF-NPE (right) for the multicompartmental neuron model task. MF-NPE is respectively, pretrained on 10 10a, 10s low-fidelity simulations (dubbed as MF-NPE3, MF-NPE4, and MF-NPE5). All models were trained on 10 high-fidelij simulations.
30

---

Published as a conference paper at ICLR 2026
F.3 POSTERIOR PREDICTIVE CHECKS
With only 50 high-fidelity simulations, ME-NPE gives similar accuracy to NPE trained on 1000 simulations (Fig. 16]. and for fixed number of 1000 high-fidelity simulations, ME-NPE5 outperforms NPE (Fig. 17
NPE (1000 HF simulations)
MF-NPE4
(50 HF simulations)
t [ms]]
120
[mms]
Figure 16: Posterior predictives for the multicompartmennal neuron model with varying number of high-fidelity simulations.
10 HF simulations
NPE
MF-NPE5
120
PPC
C
Figure 17: Posterior predictives for the multicompartmenntal neuron model for fixed number of high-fidelity simulations.
F.4 Low AND HIGH-FIDELITY TRACES
We present simulations with the models with 1- and 8-compartments per dendritic branch (low- and high-fidelity models, respectively) to illustrate that the model outputs are different, given the same parameters.
t-compartmental neuron
B B.compartmentall neuron
K- Oo
100
Upma
Figure 18: Simulated membrane potential traces of an L5 pyramidal cell (LSPC) model with Jaxley (Deistler et al. 2025). The low- and high-fidelity models are, respectively, single-compartment model per dendritic branch versus an eight-compartment model per branch.
31

---

Published as a conference paper at ICLR 2026
G TASK 3: SPIKING NETWORK MODEL
G. HIGH-FIDELITY MODEL.
We considered recurrent spiking network of 5120 neurons (4096 excitatory, 1024 inhibitory), with parameters taken from Confavreux et al. (2023). The membrane potential dynamics of neuron j. excitatory (F) or inhibitory (7), followed
dV, (V, Viea) (t) (V, Ex) () (iV,--. -Ei),
I'
dt
A postsynaptic spike generated whenever the membrane potential (t) crossed threshold (t), with instantaneous reset to Viset. This threshold (t) was incremented by With every time neuron spiked and otherwise decayed following
spike
dVB Tth di
with base
vnhh
(10)
The excitatory and inhibitory conductances, gf and gو evolved such that
gf(t) [:00= aggMPA (t) (1 g) (M) and dgf
Σ
Why (t)o(()
dt
TGABA
LEInh
(11)
dogMPA
AMPA
dg
NMDA
AMPA
9j
NMDA
with
wyy (t) and
dt
TAMPA
df
TNMDA
iCFxe
with way (t) the connection strength between neurons and j (unitless), 6x.(t) 8 (8((-17) the spike train of pre-synaptic neuron k, where t denotes the spike times of neuron k. and the Dirac delta. All neurons received input from 5k Poisson neurons, with 5% random connectivity and constant rate Tex 10Hz in each simulation. The recurrent connectivity was instantiated with random sparse connectivity (10%). All recurrent synapses in the network (E-to-E and E-to-T, /-to- E. T-to-7) underwent variations of spike-timing dependent plasticity (STDP) (Gerstner & Kistler 2002 Confavreux et al. 2023). Given the learning rate 7), the weights between the neurons and of connection type X -too-Y evolved over time as:
dwij =17 Opo (t) (a + post (t))
dt
(12)
+ (1) (3 + Tmee pT)p
(1))]
with variables (t) and (t) describing the pre- and postsynaptic spikes over time:
da;
dæy Ty dt
+ 8x(t) and
+ (t)
(13)
dt
post XY
XY
pre with xx and xY respectively.
post the time constants of the traces associated with the pre- and postsynaptic neurons,
The 24 free parameters of interest were Tpre, Tpost, O. B, multiplied by the number of synapse types (e.g., agg, 0:11, OEE, aIE), following previous work (Confavreux et al. 2023).
G.2 LOW-FIDELITY MODEL
Following previous work (Confavreux et al. 2023 Vogels et al., 2011 Dayan & Abbott 2001). (partial) mean-field theory applied to the F-to- E and E-to-/ connections the model described above gave:
Ope
BEE
and
(14)
AEE
S + A
with rg and ri the firing rates of the excitatory (resp. inhibitory) population at steady state, and post NXYTXY + +
Axy
(15)
With type (X E {E, 1]. For all synapse types, we assume (-axy- - Bxx) > 0 and Axy > 0, as second-order stability condition (Confavreux et al. 2023). Note that in this low-fice wwwfideeity model, only considered 2 of the 4 plastic conditions, and thus of the 24 free parameters of the model.
32

---

Published as a conference paper at ICLR 2026
G.3 SYNAPTIC PLASTICITY WITH VARYING PARAMETER SPACE
We investigated how inference performance changes as the discrepancy between the low- and high- fidelity models increases. To this end, varied the dimensionality of the low-fidelity model between 3, 6, and 12 parameters, while keeping the high-fidelity model fixed at 24 parameters. Parameters that were excluded from inference the low-fidelity settings were fixed the following values for each connection type: Tpee = Tpost 0.05, 7 = - 1.9, = 3 = k = 0.5. The value of y should be smaller than other parameters fulfill the second-order stability condition (Confavreux al. 2023
Spiking Network
3 LF-dimensions
LF-dimensions
12 LF-dimensions
2
2
0
0
NLTP
-2
-2
-2
-4
8.
10P
10
10
8
16
10
10
50
10
10"
10*
10
Number of high-fidelity simulations
NPE
MF-NPE4
MF-NPES
Figure 19: in the low-fidelity model.
over true parameters, with different numbers of free parameters
We observe that the performance of MF-NPE degrades as the number of parameters in the low- fidelity model decreases compared the high-fidelity model. In particular, unlike in all our other experiments, when the low-fidelity model had only 3 parameters, pretraining on 10t low-fidelity samples led worse MF-NPE performance: in this regime, using 10" samples (MF-NPE5) resulted in negative transfer, whereas pretraining on 10st samples (MF-NPE4) resulted in performance close standard NPE.
33

---

Published as a conference paper at ICLR 2026
G.4 DISCUSSION ON ALTERNATIVE SOLUTIONS
We consider the following strategies:
pretraining on solely low-fidelity simulations,
pretraining the joint of low- and high-fidelity simulations.
G.4.1 PRETRAINING ON LOW FIDELITY SAMPLES
This approach follows the main discussion in Sec. 3.1.1 and has also been the main method employed in the paper. We purposefully do not freeze the weights after transfer, allowing the network to retain the flexibility adapt to high-fidelity simulations.
G.4.2 PRETRAINING ON THE JOINT OF LF AND HF SAMPLES
We examined whether pretraining the joint distribution of low- and high-fidelity simulations could provide a better initialization for subsequent fine tuning. As shown in Fig. 20 this strategy yields significant improvement the first two benchmarking tasks compared to standard MF-NPE. However, we encourage further work to investigate additional variations on this approach to improve the domain adaptation (e.g., domain adaptation through MMD Elsemüller al. (2025), importance weighting for extremely unbalanced datasets, adversarial discriminative domain adaptation, training single multifidelity inference network).
OUprocess
SLCP
0.9
C2ST
0.8
0.7
0.6
0.5
0.3
MMD
0.2
0.1
50
10
10P
10*
10
50
10
10a
10
10
50
10
109
10
10
50
10
10a
10
10%
Number of high-fidelity simulations
Amortized methods
Non-amortized methods
.+. NPE (baseline)
MF-NPE4 MF-NPES
TSNPE (baseline)
MF-TSNPE4 MF-TSNPE4-AF
Figure 20: MF-(TS)NPE (joint) has been pretrained on both low- and high-fidelity samples.
34

---

Published as a conference paper at ICLR 2026
H PRIOR BOUNDS ACROSS NEUROSCIENCE TASKS
For the OU process task, we chose uniform prior with bounds that would lead to a range different outputs. For the multicompartment neuron model task, chose uniform prior with bounds based on the work of Deistler et al. 2022). For the spiking network model task, we chose a uniform prior with bounds based on the work of Confavreux al. 2023).
Table I: Prior bounds for the single- and multicoompartmental neuron model.
PARAMETER NAME
LOWER BOUND
UPPER BOUND
JNA
0.005
0.8
ØK
10-6
0.15
Table 2: Prior bounds for each synapse type (E-to-E, E-to-I, I-to-E E and I-to-I) for the spiking neural network and mean-field model.
PARAMETER NAME
LOWER BOUND
UPPER BOUND
0.01
0.1
Jpost
0.01
0.1
-2
2
B
-2
Y
R
-2
2
35

---

Published as a conference paper at ICLR 2026
I DISTANCE BETWEEN THE LF AND HF POSTERIOR
Both the low and high-fidelity posterior distributions have been trained on 10 simulations and evaluated over 10 true observations. In the table below, we focus on cases with two fidelities and measure the distance between the low and high-fidelity models with the MMD and C2ST metrics. We observe that the distance between the posterior distributions transfer learning. For instance, the posterior distributions of the low-and high- segeffidliity models of the L,PCC neuron are significantly different. However, the network still manages to leverage information between the two simulators (Figure 3 supporting the theoretical results of Tahir et al. (2024).
a direct measure of success in
Transfer learning seems to work less well on the OU process task when the dimensionality of the parameters differs between the low- and high-fidelity models (see Sec. (). This is observed despite the fact that the distance between the low and high-fidelity posteriors lower for the OU4 case than for the OU2 case, as the low-fidelity OU2 posterior is highly biased (Fig.
Table 3: Distance between low- and high-fidellts posterior (mean + std) for different tasks.
Task
MMD
C2ST
SLCP
0.13 0.05
0.91 + 0.03
SIR
0.04 0.03
0.57 0.03
OU2
1.00
0.02
OU3
0.69 + 0.087
0.00
OU4
0.24
0.90
LSPC
0.76 +
0.99
SynspticPlasticity
0.01 + 0.00
0.70
I.1 PAIRPLOTS
OU2
OU3
OU4
925
0.
02
25
SLOP
SIR
LSPC
3
7
33
5
3
23
D
C
11
als 2075 s1 0125
oás
Figure 21: Posterior distributions of the low-fidelity posterior (blue) and high-fidelity posterior (green). Contours contain 68% of the true posterior mass for the low-fidelity model. Vertical bars and dots correspond to the value of the true parameters.
36

---

Published as a conference paper at ICLR 2026
J SIMULATION VERSUS TRAINING COST
We tracked the wall-clock run-time for training and simulation stages of the neural density estimator. Computations were performed nodes each equipped with 4x Intel Xeon Gold 6448H CPUs (32 cores per socket, 128 physical cores, 256 logical CPUs) and approximately 2TB RAM, running Linux 5.14.0. We compare the costs in regimes where the performance of NPE is similar to MF-NPE and MF-TSNPE-AN (Fig. B.. Details about the network architecture and hyperparameters are described Appendix C.T cases where many samples had to be generated for active non-amortized schemes (e.g., 10 HF samples for the L,PC task; Figure 3 we used multiprocessing over CPUs. The
simulations for the third task were parallelized over 913 CPUs.
Table Comparison of methods for the real-world tasks terms of the number of simulations and computational cost. Total training cost is reported mean + standard deviation over 5 network runs.
method
# simulations
CPU (seconds)
LF
HF
tot. cost (sim.)
tot. cost (train)
total cost
NPE
NA
10a
4940
70.39 18.32
5010.39 + 18.32
L5PC
ME-NPE
104
103
1032
96.94 15.19
1128.94 # 15.19
ME-TSNPE-AF
10a
50
607
557.44 + 52.5
1164.44 # 52.5
NPE
NA
104
3 x 106
120.43
3,000,120
Network
MF-NPE
10
103
3 x 105
94.54
300,094
Table Comparison of methods across models terms of the number of simulations and accuracy. Evaluated using the NLTP metric.
Accuracy (22TT/NLTP)
Method
# Simulations
LF
NPE
NA
104
-5,87 + 0.04
MF-NPE
10
10a
-5.73
L5PC
MF-TSNPE-AF
10
50
-5.08
NPE
NA
104
4.772
Network
MF-NPE
105
103
-4.08 0.01
Table 4 shows that the multifidelity approaches make sense when the training cost is significantly lower than the simulation cost, such as in the L5PC and the spiking network model. For instance, in the spiking network task, a single tigh-fidelity simulation requires approximately CPU minutes, whereas low-fidelity simulation takes only 0.0008 seconds.
37

---

Published as a conference paper at ICLR 2026
K TARP EVALUATION FOR ALL TASKS
We performed additional evaluations on the calibration of all experiments with TARP Lemos et al.
2023
OU process
SLCP
SIR
Exp Coverage Probability
10 HF simulations
EXp Coverage Probability
10 HF simulations
Credibility Level a
Credibility Level a
Credibility Level a
Gaussian Blob
L5PC
Spiking Network
Exp Coverage Probability
10a HF simulations
Credibility Level a
Credibility Level a
Credibility Level a
Ideal
TARP NPE
TARP MF-NPE5
Figure 22: TARP calibration test across 1 LF simulations (10* for the Gaussian blob example).
The calibration test was performed over 200 runs.
38

---

Published as a conference paper at ICLR 2026
L ME-NPE FOR MULTIPLE LOWER-FIDELITY SIMULATORS
Algorithm 2 MF-NPE with multiple fidelities
1: Input: Simulations {(0, (0)))=) over F fidelities; Early stopping criterion S; conditional density estimators (1) (0) x ()))) with features vb.
(gp
2: for f to F do
((b())) = NOT 1 Di=1 - -log 94 (0.12(1))
3:
4: opt < Adam(-)
5: if f <>> then
( with features of trained Up ( 6: Initialize Fv
7: end if
8: for epoch in epochs do
(f) 9: train g to minimize ((bb)) until reached.
10: end for
11: end for
39

---

Published as a conference paper at ICLR 2026
M SEQUENTIAL ALGORITTMS
M.1 MF-TSNPE
Algorithm 3 ME-TSNPE
1: Input: N pairs of (0, æL); conditional density estimators qu (0) and (0)x)) (>) with learnable parameters & and Φ; early stopping criterion S; simulator p(z prior p()): number rounds
R: that defines the highest-probabile probability region (HPR,); number of high-fidelity simulations per
round M.
2: Output: posterior estimate 9: (0)z)
3: C(t) = -े Σm= log qu (0.(zt))
4: for epoch in epochs do
5: train gø to minimize ((1) until S is reached.
6: end for
7: Initialize p(0) as p(0)
8: Initialize qá with weights and biases of trained qs-
9: for r in R do
((r) p(0), sample parameters from proposal
(r) p(z(0(r)) generate high-fidelity simulations
12: for epoch in epochs do
13:
( = 1
M Σ log 9o
(o.") x
(r)
train qø to minimize L()) until reached. 15: end for
16: Compute expected coverage (f(0), 4o)
17:
p(0) p(0) LoEHPR.
18:
end for
All experiments were run with R rounds and 1e-6, More details about TSNPE at Deistler
et al. (022).
40

---

Published as a conference paper at ICLR 2026
M.2 MF-TSNPE-AF
Algorithm 4 ME-TSNPE-AF
Input: N pairs of (0, æı); conditional density estimator qu (0laa) with learnable parameters & and ensemble of conditional density estimators {g (0|=)) g. each with independent Φ; early stopping criterion S; simulator p(æ 0): prior p(0); number of rounds R; that defines the highest-probability region (HPR,,; number of high-fidelity simulations per round M.
2: Output: Ensemble posterior estimate 2o a)) = E (0(0)))
3: [(w) - - E -log
- log qe (0,
4: for epoch in epochs do
5: train dø to minimize ((P) until S is reached.
6: end for
7: for E Ensemble do
8: Initialize 9 with weights and biases of trained do.
9: end for
10: P ~ p(0)
11: Initialize p(()) p(0)
12: for in R do
@prup p(0). generate M - B samples from proposal
14: @(r) top B values from 0 pool using the acquisition function equation
active
(()) (0 U active
(
(r) p(ær (())), generate high-fidelity simulations
17:
for Ensemble do
18:
for epoch in epochs do
(()) 3े4 ΣM log o (o!") ))
20:
train 9s minimize (()) until S is reached. end for
21:
22:
end for
23:
Compute expected coverage (5(0), (g(0)
24:
p(0) x p(0) LØEHPR,
end for
with R 5 rounds, 1e-6 and ensemble of 5 networks. The addition of an acquisition function biases the proposal distribution, causing the density estimate to diverge from the true posterior. In principle, this could be addressed by using atomic proposals (Greenberg et al. 2019), but given that such an approach suffers from posterior leakage, we do not introduce a proposal correction in order to retain the well-behaved loss function in TSNPE. We argue that the benefit of informative samples would outweigh the potential bias, as long as the percentage of samples selected from the acquisition function would be small compared the proposal samples. Therefore, we set B 22M to mitigate the concern of biasing the posterior with parameters selected with the acquisition function.
All experiments
41


Reviews:
[{"review_id": "byg6nMr1Wj", "reviewer": "Reviewer_Eu4Y", "summary": "The paper introduces an approach that uses transfer learning from cheap simulations to enable efficient posterior estimation for expensive higher-fidelity simulators. The method is evaluated on benchmark tasks and neuroscience applications, showing up to 100x reduction in required high-fidelity simulations. The basic idea is to pre-train a neural density estimator on cheap low-fidelity simulations, then fine-tunes on expensive high-fidelity simulations.", "questions": "1. Table 4 shows A-MF-TSNPE training takes ~10x longer than MF-NPE -- when is this tradeoff worthwhile?\n2. What are the major differentiators wrt https://arxiv.org/abs/2507.00514 and https://arxiv.org/abs/2505.21215 (e.g. the sequential, active learning parts)?\n3. The spiking network has 12/24 parameters for low/high-fidelity, curious how performance degrades as this gap increases (guessing this is strongly domain dependent, but curious if there are general principles that could help guide applications)", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Addresses a real bottleneck in SBI where simulators can take minutes to hours per run. Relevant to lots of domains, in addition to those explored in the paper also in the physics sciences.\n2. Exploration along the full SBI \"stack\" -- provides both amortized (MF-NPE) and sequential (MF-TSNPE) variants plus active learning (A-MF-TSNPE). Algorithms clearly specified.\n3. Empirical results of when transfer works are strong -- beyond just demonstrating improvements, authors look at the underlying mechanisms (mutual information, representational coherence). The controlled experiments manipulating the OU process parameters provide insights into transfer learning limits.", "weaknesses": "1. https://arxiv.org/abs/2507.00514 and https://arxiv.org/abs/2505.21215 seem very similar in technique to the present paper (with a different application focus). While mentioned as concurrent work, this takes away from some of the methodological novelty somewhat. \n2. Along the same lines, overall, a fairly straightforward, empirical application paper. Even toy theoretical analyses on transfer learning bounds -- perhaps relating low/high-fidelity similarity to sample complexity reduction -- would significantly strengthen the contribution. The empirical analysis in Section 4.4 is valuable but doesn't substitute for formal guarantees.\n3. Possible minor notation inconsistencies, e.g. switches between p(x|\\theta) and p_H(x|\\theta)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1th3LiamGK", "reviewer": "Reviewer_G9kC", "summary": "The paper takes a few simple existing techniques and applies them to SBI. Transfer learning, active learning, and (already often applied) sequential training. They are applied specifically to multi fidelity simulators, which are plausibly useful to practitioners. The paper shows benchmark results and application-specific results in neuroscience. There are also many empirical attempts to characterize the effects of their contributions which are confusingly put in the appendix.", "questions": "- What happens when your cheap simulator has a narrow posterior compared to your high-fidelity simulator? It seems you are implicitly assuming that the cheap posterior is absolutely continuous w.r.t. the high fidelity simulator, but this doesn't have to be the case. \n- You mention that the method is \"applicable in situations where the low-fidelity model has fewer parameters than the high-fidelity model.\" What about the other way around?\n- Why do you spend so much main text time on background rather than getting into the method more quickly and showing all your cool results? So many interesting experiments are in the appendix.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "# results\n- Figure 3 looks compelling. It shows the typical result that active learning decrease NLTP and also reduces calibration. It would be nice to know if it's problematically uncalibrated, but maybe an expert in the area can understand that.\n- lines 439-448 are also super interesting. Why is this not in the main text? There is ~1.5 pages of application specific info about neuroscience, but this ML-general information is left out. Please include general info in the main text.", "weaknesses": "# writing\n- It would be helpful if you stated your contribution more clearly at the beginning. There is a ton of background that obfuscates what algorithmic pieces you are introducing. It seems like the idea mainly exists in lines 282-232 and 234 - 248. I think a summary of that info could come significantly earlier.\n- \"A-MF-TSNPE\" is super confusing because A often means amortized in this taxonomy while S implies sequential (aka active learning). There is already \"active learning\" going on in TSNPE. Usually amortized methods do not use active learning. \n\n# results\n- In Figure 2 it seems like you add a lot of algorithmic complexity for small gains. Meanwhile Figure 8 seems super novel and interesting. Please consider including Figure 8 in the main text.\n- Where is an example of different numbers of parameters in the high fidelity and lower fidelity settings? That could be made more clear in the text. I know it's in the appendix.\n- In general I think you can spend much more space on the benchmark results rather than all the background.\n\n# citations\n- Since you compare to and extend TSNPE, I would strongly recommend citing TMNRE https://arxiv.org/abs/2107.01214\n- Additionally there is a huge literature about multi-fidelity bayesian optimization. Maybe it's worth citing that as well?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RnKn4DbqCV", "reviewer": "Reviewer_CW6v", "summary": "This paper introduces MF-(TS)NPE, a novel and efficient approach to simulation-based inference (SBI) designed to tackle the computational burden of parameter estimation in expensive high-fidelity simulators. The method leverages multifidelity modeling – combining computationally cheap, low-fidelity simulations with a limited number of high-fidelity simulations – and transfer learning to accelerate Bayesian inference. An active learning extension, A-MF-TSNPE, further improves efficiency by strategically selecting high-fidelity evaluations. Demonstrated across benchmark tasks and challenging neuroscience applications, MF-(TS)NPE achieves up to two orders of magnitude reduction in required high-fidelity simulations while maintaining comparable accuracy to existing methods like NPE and TSNPE. The work provides a valuable tool for efficient Bayesian analysis in a wide range of scientific domains reliant on complex simulations.", "questions": "- lines 30-53: nice introduction to the field, could be cut shorter perhaps\n- lines 200-202: \"Overall, MF-NPE shows competitive performance compared ...\" this sentence confused me as no strong results in favor of this claim was presented up to here to a reader, perhaps reconsider the formulation or remove this passage\n- page 4, Algorithm 1: it might be helpful to clearly identify low-fidelity and high-fidelity related parts of the algorithm (perhaps hints to the right of the algorithm)\n- page 4, Algorithm 1: line numbers would be helpful to communicate with peers about this\n- line 240-241: clearly state that you are interested in the epistemic uncertainty (makes the text more approachable to people experienced in UQ)\n- line 246-248: \"Alternatively, one could ... Griesmer 2024)\". MCdropout is a simple but often unstable UQ method, I personally would remove this sentence\n- on section 3.1.3: comment briefly on the fact that any UQ method at use here needs to be battle proven in the OOD regime (this is a hard problem within its own right, hence only comment)\n- line 268: \"In the limit of a large number of pairs, ...\" the high fidely simulation dataset comprises at minimum 50 samples. Would we expect convergence of NLTP to the KL here? \n- line 295-300: I guess, you do not use TARP or methods alike because you are working in a low data regime?\n- figure 2: do NOT include an interpretation in bold letters as the caption of the figure. Please leave it to the text (subsequently the reader) to make such assertions.\n- line 320: \"Across experiments, we observed a consistent performance increase with MF-NPE compared to NPE\" This is not what I deduce from the figure. From all 12 cases under study, by eye, I can tell that in only 4 MF-NPE is below the NPE (black dashed line) with some distance larger than the error bars. Please adapt this statement, if my interpretation is correct.\n- line 326: I can hardly compare Appendix F.1.1 to e.g. Fig 2. Please create a plot where this is possible and put it in F.1.1\n- line 362: \"approx. 4 times\" please be precise; also provide an absolute scale for reference on what 4 times higher means\n- line 373: perhaps add a second model critique methdo like TARP etc\n- section 4.3: all citations lack hyperrefs in my PDF reader, please check", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This paper demonstrates a strong commitment to rigorous and accessible research in simulation-based inference. A key strength lies in its exceptionally clear and well-motivated introduction to both the challenges of SBI and the foundational techniques like TSNPE, significantly enhancing the paper’s accessibility to a broad audience. The experimental design is particularly commendable, building logically from established benchmarks to increasingly complex, real-world simulations (including L5PC and spiking neural networks), providing robust validation of the proposed method. \n\nThe authors showcase a thorough analysis, supported by a diverse suite of metrics – even extending to appendix materials to accommodate the breadth of results – and consistently present findings with appropriate uncertainty estimates, empowering readers to critically evaluate the conclusions. Further contributing to clarity, the writing is commendably free of unnecessary jargon and complex simulators are described in sufficient detail to facilitate a basic level of reproducibility. Finally, the authors adopt a refreshingly honest and nuanced approach, presenting their work as a solid incremental improvement within the field rather than an unsubstantiated claim of state-of-the-art performance. This measured presentation strengthens the credibility and impact of the research.", "weaknesses": "This paper, while strong overall, exhibits a few areas for improvement. The introduction occasionally suffers from verbosity, dedicating excessive space to establishing the context of simulation-based inference – particularly in the opening paragraphs – which could be streamlined for greater impact. More critically, the authors sometimes present conclusions that exceed the support provided by the empirical results, necessitating more cautious interpretation. Finally, the quantification of improvements, specifically regarding computational cost, sometimes relies on relative gains rather than absolute metrics, hindering a full appreciation of the practical benefits achieved by the proposed method. Addressing these issues would enhance the clarity, rigor, and impact of the presented work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7kk5D9KFwg", "reviewer": "Reviewer_Sge5", "summary": "The paper proposes MF-(TS)NPE, a multi-fidelity recipe for SBI that pre-trains a neural posterior estimator on cheap low-fidelity simulations and fine-tunes it using a small budget of high-fidelity simulations. Then they propose two enhanced variants:  a sequential variant that further performs sequential SBI starting from the fine-tuned model and using the high-fidelity simulator and an active variant (A-MF-TSNPE) that adds an acquisition rule based on the variance of an ensemble of fine-tuned models. \nAcross SBI benchmarks (SIR, SLCP, OU, Gaussian Blob) and two computational neuroscience tasks (multicompartment neuron; large recurrent spiking network), the method matches or improves accuracy while using orders of magnitude fewer HF simulations; wall-clock accounting shows substantial real compute savings when HF simulations dominate cost (Table 4). The paper also investigates when transfer helps via controlled OU perturbations, highlighting the roles of mutual information and representational coherence between fidelities (Fig. 4).", "questions": "Is there a way to know in advance (without evaluation on real data), if the low-fidelity simulator can be helpful?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "-**Clarity and motivation**: The paper is clearly written and well motivated: leveraging multiple simulators with varying levels of fidelity seems like a natural way to improve efficiency of SBI. While it was considered in the ABC literature, it received little attention when it comes to deep networks for SBI.\n\n-**Simplicity**: The proposed approach based on fine-tuning is quite simple to implement and seems to yield effective improvements. This simplicity makes it broadly applicable accross tasks, as shown in the exeperiments, and easily adaptable to other settings: sequential and active settings.  \n\n-**Soundness**: The paper proposes an empirical characterization for when such approaches should be effective based on mutual information and feature alignment. This supports the soundness of the approach although precise theoretical characterization remains unclear, but could be the subject of future work.", "weaknesses": "The limitations of this work are well discussed in section 5. \n**Reliance on LF-HF alignment**: The method relies on a-priori alignment between the low-fidelity and high-fidelity simulators, but in general it is hard to quantify such alignment. While the paper studies the effect of this alignment, it is done on a toy example/controlled setting. Currently, the approach relies on domain experts that can know beforehand whether simulators are aligned, but it would be could to have a procedure or some guidelines for non-experts. Perhaps a better theoretical understanding could achieve that.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)