Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Authors: Antoine Wehenkel
Juan L. Gamella
Ozan Sener
Jens Behrmann
Guillermo Sapiro
Joern-Henrik Jacobsen
marco cuturi
Venue: ICML 2025 oral (2025)
Keywords: Simulation-based Inference
Misspecification
Optimal Transport
Bayesian Inference
Neural Posterior Estimation
Robust Inference
Primary area: probabilistic_methods->bayesian_models_and_methods

Abstract: Driven by steady progress in deep generative modeling, simulation-based inference (SBI) has emerged as the workhorse for inferring the parameters of stochastic simulators. However, recent work has demonstrated that model misspecification can harm SBI's reliability, preventing its adoption in important applications where only misspecified simulators are available.
This work introduces robust posterior estimation (RoPE), a framework that overcomes model misspecification with a small real-world calibration set of ground truth parameter measurements.
We formalize the misspecification gap as the solution of an optimal transport (OT) problem between learned representations of real-world and simulated observations, allowing RoPE to learn a model of the misspecification without placing additional assumptions on its nature. RoPE shows how the calibration set and OT together offer a controllable balance between calibrated uncertainty and informative inference even under severely misspecified simulators. Results on four synthetic tasks and two real-world problems with ground-truth labels demonstrate that RoPE outperforms baselines and consistently returns informative and calibrated credible intervals.

Addressing Misspecification in Simulation-based Inference
through Data-driven Calibration
Antoine Wehenkel
Juan L. Gamella 22 Ozan Sener'
Jens Behrmann Guillermo Sapiro Marco Cuturi
Jörn-Henrik Jacobsen
Abstract
phenomena, it has also necessitated the development of new statistical inference methods. In particular, state-of-the-art simulation- based inference (SBI, Crammer et al., 2020) algo- rithms leverage neural networks to learn surrogate models of the likelihood (Papamakarios et al., 2019), likelihood ratio (Hermans et al., 2020), or posterior distribution (Pa- pamakarios & Murray, 2016), from which one can extract confidence or credible intervals over the parameters of in- terest given an observation. While SBI has proven helpful when the simulator is faithful description of the stud- ied phenomenon, e.g., for scientific applications (Delaunoy et al., 2020; Brehmer, 2021: Lückmann, 2022; Linhart et al., 2022; Hashemi et al., 2022; Tolley et al., 2023; Avecilla et al., 2022), recent work has also highlighted the unreliabil- ity SBI methods under model misspecification (Cannon et al., 2022; Schmitt et al., 2023).
Driven by steady progress in deep generative modeling, simulation-based inference (SBI) has emerged the workhorse for inferring the pa- rameters of stochastic simulators. However, re- cent work has demonstrated that model misspec- ification can compromise the reliability of SBI, preventing its adoption in important applications where only misspecified simulators are available. This work introduces robust posterior estima- tion (RoPE), framework that overcomes model calibra-
misspecificatioo with small tion set of ground-truth parameter measurements. We formalize the misspecification gap as the so- lution of an optimal transport (OT) problem be- tween learned representations of real-world and simulated observations, allowing RoPE to learn a model of the misspecification without placing ad- ditional assumptions on its nature. RoPE demon- strates how OT and calibration set provide con- trollable balance between calibrated uncertainty and informative inference, even under severely misspecified simulators. Results on four synthetic tasks and two real-world problems with ground- truth labels demonstrate that RoPE outperforms baselines and consistently returns informative and calibrated credible intervals.
Addressing Misspecification with a Calibration Set. In this work, target important applications of SBI in com- mon settings where (1) the goal is to estimate hard-to- measure variable from indirect but readily available mea- surements of other variables; (2) only misspecified sim- ulators relating these variables are available; and (3) a few ground-truth pairings of the hard-to-measure variables and the related variables are available in calibration set', Such setting can arise, for example, when inferring the properties of patient's cardiovascular system from non- invasive and abundant measurements of other physiological signals (Wehenkel et al., 2023), or when developing soft sensors to monitor industrial processes in real time, where directly measuring the quantity of interest is costly and time consuming, for example, through laboratory analysis, but where related variables can be measured quickly and inexpensively (Jiang et al., 2021; Perera et al., 2023).
1 Introduction
Many fields of science and engineering have shifted in re- cent years from modeling real-world phenomena through few equations to relying instead on highly complex com- puter simulations. While this shift has increased model versatility and the ability to explain or replicate complex
Our Contributions. We introduce robust posterior estima- tion (RoPE), an algorithm that addresses model misspecifi- cation to provide accurate uncertainty quantification for the parameters of black-box simulators. In such misspecified
'Equal contribution 'Apple "ork done while being at Ap- ple 'ETH Zürich. Correspondence to: Antoine Wehenkel <awe- henkel apple.com>.
'Note that our use of the term calibration set should not be confused with its usage in the context of model mis-calibration in well well-specified SBI (Hermans al., 2022), clarify in Appendix
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
settings, the main challenge lies in the absence of paired dataset of simulated and corresponding real outputs. To han- dle this knowledge gap, RoPE estimates coupling between real and simulated observations using optimal transport (OT,
neural posterior estimation NPE (Papamakarios & Murray, 2016; Lueckmann al., 2017; Radev et al., 2020) is broadly applicable method that trains conditional density estimator of p(0 | xx) from dataset of parameter-simulaton pairs. In this paper, we focus on making NPE robust to model misspeciffiation.
Peyré et al., 2017; Villani et al., 2009). The algorithm ex- tends neural posterior estimation (Papamakarios & Murray, 2016) and models misspecification using OT. We evaluate the performance of the algorithm on existing benchmarks from the SBI literature and introduce four new benchmarks, two of which are synthetic and two come from real physical systems. To the best of our knowledge, the latter consti- tute the first real-worid benchmarks that directly provide a ground truth for the inferred parameters for SBI under misspecification. We conduct additional experiments to investigate the impact on RoPE's performance of varying calibration set sizes, prior misspecification, and distribution shifts, well as various ablation studies.
NPE usually parametrizes the posterior with a neural condi- tional density estimator (NCDE), which is composed of (1) neural statistic estimator (NSE), denoted by h X' + R', that compresses observations into I-dimensional representa- tions and, (2) normalizing flow (NF, Papamakarios et al., 2021; Tabak & Vanden-Eijnden, 2010) that parameterizes the posterior density as po(0 | (x+)). The parameters & and w of the NCDE are trained with stochastic gradient ascent on the expected log-posterior probability, solving the following optimization problem
w arg max E (--p(8) [logr>(0 (S((,))))) (I)
2 Background & Notation
d:w
where p(0) denotes a prior over the parameters
In this section, we first pose the machine learning problem we are trying to solve and then formally introduce SBI, model misspecification, and OT, as our method lies at the intersection of these fields.
Under the assumption that the class of functions represented by the NCDE contains the true posterior, solving (1) leads to a surrogate pa- (0 b (x+]) that matches exactly the pos- terior p(0 xx) corresponding to the simulator. In that case, I. Xs h (xx), that is, the NSE h is a sufficient statis- tic of x, for the parameter 0 (Chen al., 2020; Wrede et al., 2022; Chan et al., 2018). In practice, we can only approach perfect training by generating sufficiently large number of pairs (0, xs) and doing a search on the NCDE's architec- ture and training hyperparameters. To simplify notation, we denote the NCDE learned with NPE as p(0 | Xa)-
We consider a simulator, S x [0,1] Rd, that takes in physical parameters 0 E
o S R² and a random seed E E [0, 1] to generate simulated measurements x, E X, C Rd. The simulator is a simplified version of a real and unknown generative process p" (xx) := f (0))"" (xo d)d0 that pro- duces real-world observations xo E No S Rd. We assume this process depends on parameters with the same physical meaning as the ones of the simulator and thus use the same notation 0. Our task is to estimate a well-calibrated and informative posterior distribution p(0 xx) for each observa- tion x in a test set D, reducing uncertainty compared to the prior distribution, assuming that the prior is well-specified. To achieve our goal, we have access to: 1. the misspeci- fied simulator S that embeds domain knowledge and gen- erates samples whose distribution approximates p" (xo (). 2. prior p(0) that approximates the marginal distribution p" (0) of parameters in the real-world, 3. a small calibration set of labeled real-world observations C == {(0', xx)) n. composed of i.i.d. samples from p² (0, xo), which enables data-driven correction of the simulator's misspecification, 4. a test set D := 17 of real-world observations arising from p²(xx* (x.) for which we want to estimate the posterior.
2.2 Model Misspecification
In statistics, where the model parameters do not necessarily carry real-world meaning, model misspecification generally denotes the inability of model to reproduce the observed data distribution. Formally, a parametric model p(x. 0) is said to be misspecified with respect to some true data- generating process p (xx.) if the latter does not fall within the family of distributions defined by the model, i.e. A0 e θ : p(xo 0) >xxx) Vx. (Cannon et al., 2022). In contrast, we are not necessarily interested in reproducing the observed data x. but only in inferring the parameter value 0 from an observation Xo. For this goal, naively using the standard definition is insufficient, as a model may be well-specified but still produce incorrect credible intervals for the parameters of interest 0. This undesired behavior may happen, for example, if the model is over wwrrpparamttriize, as illustrated in Appendix A.
2.1 Simulation-base Inference (SBI)
Applying statistical inference to simulators is challenged by the absence of a tractable likelihood function (Cranmer et al., 2020). As a solution, SBI algorithms leverage mod- ern machine learning methods to tackle inference in this likelihood sreee setting (Lueckmann et al., 2021; Delaunoy et al., 2022; Glöckler et al., 2022). Among SBI algorithms,
Thus, in this work, we define model misspecification dif- ferently and align it with the setting motivated in Section 1. Intuitively, we describe model misspecification as the non- transferability of the posterior obtained from the simulator
2

---

Addressing Misspecification in Simulation-basee Inference through Data-driven Calibration
to the prediction of real-world parameters. Formally, we assume that the pairs of parameters and observations (f, xo) are i.i.d. from an unknown distribution p" (6, Xo), which im- plicitly defines p" (0) xo)), the Bayes optimal predictor of the parameter given an observation. Under this premise, we say
rely on OTT (Cuturi et al., 2022) return such coupling p* given the cost matrix C and the parameters 7 and p. parameterized as p/ + +7). Setting = 1 amounts to a perfectly balanced transport.
3 RoPE: Modeling Misspecificatiion with OT
a simulator is misspecified if 3S C e x X : ((0, Xo) E S,
p(0)= = p (0) and p" (0) xo) # p(0 x, = xx))
In this section, we formally introduce our robust posterior estimation algorithm (RoPE) and highlight some benefits of modeling misxpecification with OT. RoPE approaches the problem of misspecification as hybrid modeling task by combining the simulator with a misspecification model learned from the few observations in the calibration set. The main modeling assumption of RoPE is
Following this definition, misspecification in SBI as learning task where our goal is to find good estimator of p" x.). As we assume the simulator provides strong domain knowledge, we focus on the challenging settings where the dataset of labeled real observations D := {(0', x=));_1 that we have for learning p*(0 Xo) is small. In such settings, most examples must be saved for testing and only small subset, denoted by the calibration set C, remains available for training.
frame the problem of model
(3) that is, given the simulated observations xs, the real ob- servations x contain no additional information about the parameters 0. As a consequence, we can express the pos- terior for real-world observations as p(0 xx) = f(0 xx xx)p(x; x xo) where p(0 | xx) is easily approximated with NPE. On the other hand, the conditional p(xx xx), which can be attributed to misspecification. is what RoPE intends to learn by estimating an OT coupling (that is then conditioned xo, c.f. 4).
X 1 Xs:
2.3 Semi-balanced Optimal Transport (OT)
As further motivated in Section 3, RoPE models the misspec- ification between simulations and real-world observations as an OT coupling. For readers unfamiliar with OT, a coupling between two distributions- p(x.) and p(x,)-is a dis- tribution x" (xx,xx on the product space whose marginals coincide with those two distributions while minimizing an expected cost E,- [c(xo; x+)]. The function e Xo x X, R assigns cost to any pair (Xo,Xs) E Xo Xs.
While this assumption introduces an information bottleneck, it does not prevent the method from achieving calibrated and informative posterior distributions- even if the assump- tion is only partially met in practice (e.g., tasks E and F in Figure 2). In fact, it acts as regularizer, enabling the learn ing of generalizable misspecification model from only a small calibration set, and it ensures that predictions remain grounded in the expert knowledge embedded in the simu- lator. This bottleneck can be limiting for simulators that are highly misspecified and fail to model the dependencies between parameters and observations. However, when the simulator encodes phenomena the practitioner believes to be invariant across different application environments, the assumption forestalls "shortcut learning" (Geirhos al., 2020) from the calibration data and improves generaliza- tion. In Appendix D, we illustrate this property using real -i-off--ittibution data.
In our setting, we can access a limited number no of real-world observations [x-) which we assume are ii.d. from the unknown distribution px(x). Writing C== ===[c(]],
for the cost matrix between observed and simulated data, we solve the discrete semi-balanced (Rabin et al., 2014) and antropy-regularizae (Frogner et al., 2015) OT problem. This formulation preserves strict marginal constraint on the observed data, but relaxes the marginal constraint on the simulated data, thus allowing certain simu- lations Xa to be discarded or down- weighte. Namely, given a set [x of simulated observations, we search for the non-negative transport matrix p* E B. that satisfies the left marginal constraint,
B. = {P E R Xon Ej= Pij = 1 Vi = I,..., no
Intuitively, the discrete OT coupling P* between the two point clouds (x1 12, and {x() obtained from solving (2) can be seen as an approximation of a joint distribution x² in X, x X, when 7 (see Appendix E for further discussion). Then, the modeled misspecification x², together with our modeling assumption (3), defines the posterior distribution for real-world observations as
and solves
p = arg min (P,C) pKL ( (pT Inoll PEB.
1n ns
+ (P, (P.logo log P).
(2)
where 1n is vector of ones with size n and KL is the Kullback-Leible divergence. Therefore, a larger p > promotes a coupling that fits the marginal of simulated data more closely, and y > is hyperparameter that encourages entropic transport matrices. This problem can be solved with a variant of the Sinkhorn algorithm (Cuturi, 2013) with efficient GPU implementations. In our experiments, we
p(0 xx) / p(0) xxx "xxx=(xx>> xx)dxx,
(4)
where the posterior p(0 xx) can be approximated very pre- cisely with NPE (Papamakarios & Murray, 2016) as NF' are universal density estimators of continuous distributions (We-
3

---

Addressing Misspecification in Simulation-basee Inference through Data-driven Calibration
henkel& & Louppe, 2019; Draxler al., 2024).
terior distribution over the test set, i.e., p(0) := f p I
xo)p" (xa))xa, converges the prior distribution as the number of simulated observations N, approaches infinity, as expected from a well-estimated posterior distribution. A proof and further discussion of this self-calibration property is given in Section F. Second, a control mechanism for the posteriors' confidence: the entropic regularization of OT not only enables fast computation of the transport coupling but also provides effective control mechanism to bal- ance the calibration of the posterior with its informativeness. Indeed, for small entropic regularization, the estimated pos- teriors have low entropy and may be overconfident, as they are sparse mixtures of few simulation posteriors 5(0 x²). In contrast, for large values of y in (2), the coupling matrix becomes uniform and the corresponding posteriors tend to the prior, as p(0 xx) 2 En 5(0 x²) a Monte-Carlo approximation of Ep(x.) (P(0 xx)] R p(0). Thus, the practi- tioner can optimize the hyper-parameter y to find the right trade-off between calibration of the estimated posteriors, favored by higher 7, and their informativeness, favored by lower y. Finally, robustness to prior misspecificationn by enabling the transport to be unbalanced that is, to discard simulated observations when < R-RoPE can flexibly depart from the assumed marginal distribution of p(0) and be robust to prior misspecification. Thus, the parameter can be seen as a control mechanism to account for the user's ss confidence in the prior distribution. In the rest of the text, we denote the method as RoPE* when r < 1 and as RoPE when F = 1. In Section 5.1, we provide guidance on how to set yy and r in practice.
We approximate x² by computing the OT coupling Pa be- tween the test set D and a set (x])}== of nt simulations generated by running the simulator on parameters from the given prior 0² ~ p(0). The cost function is defined in the next section. Thus, RoPE estimates the posterior for real- world observations as mixture of the posteriors p obtained with NPE, that is,
By
i(0 xx)
Σ
o05 x2). where oij = Poj (5)
3.1 Defining the OT Cost Function
In our setting, an ideal coupling would pair a real-world observation with simulations generated by the same pa- rameters. Hence, the cost function should be insensitive to variation in the data (e-g., noise) that is independent of 0. Formally, we write xxx c(ho(xo), (xx))), where he and h2 are sufficient statistics for with respect to X. and xs, respectively.
A key concern is to find a meaningful way to learn ho, the sufficient statistic for the real observations. As discussed in Appendix G, we can learn an approximate minimal suffi- cient statistic b for the simulated observations with NPE. Because the simulator carries information about the true generative process, our approach is to fine-tune h using the calibration set, which is otherwise too small to learn a representation from real-world data only. Denoting the fine-tuned neural network as gp: : No - R², the fine-tuning objective reads
4 Related Work
C)p::) = ElBP B (x,) Bewero, (nw (s(0'.e))lls (6)
The problem we address shares fundamental similarities with sim2real transfer learning, where the goal is to bridge the gap between simulated and real-world data. In robotics and computer vision, this challenge has been tackled through domain randomization (Tobin et al., 2017), which increases simulation diversity to improve real-world gener- alization, and domain adaptation techniques (Ganin et al., 2016; Long et al., 2015; Bousmalis et al., 2018) that learn domain-invariant representations. However, unlike these approaches that typically focus on point predictions, RoPE addresses the more challenging problem of transferring un- certainty quantification from simulation to reality while preserving calibration properties.
where the expectation is approximated via a Monte-Carlo approximation. The training of g starts from the weights w² and optimizes (6) with gradient descent. Optimizing (6) enforces, least on the calibration set, that g and h close in 1.2 norm when applied to observations from the same parameter 0. Thus, we define the OT cost as x(xo, := x (xo) h (xx)12, where x.* is the NSE obtained after fine-tuning (6). Figure 4 in Appendix B depicts RoPE's training and inference steps. We discuss the computational cost of RoPE in Section H.
3.2 On the benefits of using optimal transport to handle misspecification
The setting we consider also naturally connects to semi- supervised learning (Zhu, 2005), as both involve leveraging abundant unlabeled data alongside limited labeled exam- ples. Our setup with the calibration set resembles few-shot learning scenarios (Wang et al., 2020), where rapid adapta- tion occurs with minimal labeled examples. While classical semi superviid methods focus exploiting unlabeled data for classification or regression tasks, our approach differs in
While we could have chosen other approaches model p(x. e.g. conditional deep generative models- several attractive properties directly follow from modeling the misspecification as an OT coupling between simulated and real-world measurements. First, a self-calibration property: by modeling the posterior as (5), when T = (i.e., the transport is perfectly balanced), the marginal pos-
4

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
that it leverages a large set of labeled data obtained through simulation. Crucially, unlike standard semi-supervised or few-shot learning, where labeled and unlabeled data come from the same distribution, we must explicitly account for the distributional mismatch between simulated and real ob- servations.
Task A & B (synthetic): CS & SIR We reproduce the cancer and stromal cell development (CS) and the stochastic epidemic model (SIR) benchmarks from Ward et al. (2022), We provide a description of the parameters, observations and synthetic misspecification in Appendix I.1
Task C (synthetic); Pendulum. The damped pendulum is a common benchmark for hybrid learning algorithms (Takeishi & Kalousis, 2021; Yin et al., 2021; Wehenkel et al., 2022) that leverage both domain knowledge and real-world data. The simulator outputs the horizontal position of a frictionless pendulum given its fundamental frequency wa E + and amplitude A E R with randomness introduced via a phase shift and white measurement noise. As misspecified "real-world" data, we generate observations from a damped pendulum with friction.
In both likelihood- and simulation- baased inference set- tings, model misspecification has recently gained substan- tial interest from the research community. Among devel- oped strategies, works that take inspiration from generalized Bayesian inference (Bissiri et al., 2016) are numerous (Del- laporta et al., 2022; Chérief-Abdellatif & Alquier, 2020; Matsubara et al., 2022; Pacchiardi & Dutta, 2021; Schmon et al., 2020; Gao et al., 2023; Frazier et al., 2023). In the spe- cific context of SBI, recent works (Ward et al., 2022; Huang et al., 2023; Kelly et al., 2023) have investigated solutions to improve the robustness of existing SBI methods to model misspecification, detecting it at in- ference time (Schmitt et al., 2023). Similarly, Frazier et al. (2020) studied the impact of misspecification on approxi- mate Bayesian computation methods (ABC, Rubin, 1 1984), introducing diagnostics to detect and proposing strategies to make ABC robust. For the interested reader, Nott al. (2023) review restricted likelihood methods, Bayesian mod- ular inference, and parametric projection methods, which are standard frameworks to handle model misspecification in ikcelihodd-base Bayesian inference.
Task D (synthetic): Hemodynamics, Following Wehenkel et al. (2023), we define the task of inferring the stroke vol- ume (SV) and the left ventricular ejection time (LVET) from normalized arterial pressure waveforms. The simula- tor is a PDE solver (Melis, 2017) that produces an 8-second time-series x, sampled at 125Hz. As synthetic misspecifica- tion, the simulator assumes all arteries have constant length, whereas this parameter varies in the
data.
Task E (real): Light Tunnel. We employ one of the light tunnel datasets from Gamella et al. (2025). The tunnel is an clongated chamber with a controllable light source at one end, two linear polarizers mounted on rotating frames, and a camera. Our task consists of predicting the color setting of the light source ((R, G, B) E [o, 255]*) and the dimming effect of the polarizers The simulator takes the parameters @ == [R, G, B, af and produces an image consisting of a hexagon roughly the size of the light source, with a color equal to [a:R aB..
In contrast to these approaches, we frame model misspecifi- cation in SBI as learning problem, recognizing that if the ultimate goal is to perform inference over parameters for downstream decision-making, it is essential to have a test set to empirically validate the performance of any inference procedure. RoPE leverages a small subset of this test set as a calibration set to overcome the modeled misspecification in supervised manner.
[0, 1J from the captured images,
Task F (real): Wind Tunnel. We employ one of the wind tunnel datasets from Gamella et al. (2025). The tunnel is chamber with two controllable fans that push air through it, and barometers that measure air pressure at different locations. A hatch controls the area of an additional opening to the outside. The dataset is a collection of pressure curves that result from applying a short impulse to the intake fan power and measuring the change in air pressure inside the tunnel. Our inference task consists of predicting the hatch position, 0 := H e [0, 45) given a pressure curve. As a simulator model, we adapt the physical model given in Gamella et al. (2025, Appendix IV).
Experiments
Our experiments aim to (1) empirically validate the discus- sion in Section 3.2. and (2) illustrate settings in which RoPE enables uncertainty quantification under model misspeci- fication and small calibration datasets. The experiments comprise two existing benchmarks from the SBI literature, two synthetic benchmarks, and two new benchmarks from real physical systems for which both labeled data and simu- lators are available. While these benchmarks remain simpli- fied versions of real-world scenarios, they represent various types of misspecification with varied parameter and obser- vation spaces, allowing us to study RoPE's performance under diverse configurations. We briefly describe each task and provide examples of real vs. simulated observations in Figures and 2. Further details about the experimental setup can be found in Appendix I.
Metrics. We consider two metrics to assess whether RoPE provides reliable and useful uncertainty quantification. First, given a labeled test set {(0', x >>>)>>> we compute the log- posterior probability (LPP) as LPP := 1, Σ log p(0' I xh) se Fxxo.x.) [log x.)) The LPP, also called the neg- ative log probability of the true test parameter (NLTP), is
5

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Task CS
Synthetic benchmark
(better higher
ACAUC (better closer to zero)
881
-0.67
tnning-orty
hp. Tex 7vll
JANF1
NRE-RS
Observations
OT-onl
RoPE
JNPE
RoPE
MLP
MLP
NNPE
Prior
-3.8
NNPE
881
RoPE
NFE
O1-orly
RoPE
XC
Buring-oon
200
1000
Calibration size
Calibration size
Task SIR
(better higher values!
ACAUC (better closer to zero)
Synthetic benchmark
0 17: [8]
NPE
suniing only
MLP
NNPE
Observations
RoPE
RoPE
NPE-RS
luriin only
NPE
Prior
-23
Prio
MLP
RoPE
X, Sinudated
Xa Real
.749 3
NNFE 01 .nnt
-0.1
200
1000
1000
Calibration size
Calibration size
Task C Pendulum
(better higher values!
ACAUC (better closer to zero)
Synthatic banchmark
SBI
3.1
HVAE
MPE-RS
tringgonny
Observations
RoPE
NPC
WMM ww
OT:orly Prior
NAPE
20/32
RoPE
J.NPE
-2.5
NNPE
MLP
JAPE
only
OT:ony
Pror
X
%5 Real
-3773.8
RoP
-0.1
Simulated
200
1000
Calibration size
Calibration
Task Hemodynamics Synthatic banchmark
LPP (better higher values)
ACAUC (better closer to zero)
691
0.22
SSV. LVET
J.PEE
anningorlly
RoPE
Observations
on-onty
RoPE
NPF APE-RS
MLP
Price
-23
JNNPE
NNFE
98
RoPE
RoPE
111
X Real
0.1
Simulased
200
1000
200
Calibration size
Calibration
Figure 1: Results for our method (RoPE) and the competing baselines six benchmark tasks. For each task, we show example of the real observations (x,) and the observations produced by the misspecified simulator (x,). We show each method's LPP and ACAUC metrics, as computed on a labeled test set of size 2000. Horizontal lines without markers correspond to the methods that do not use the calibration set, producing a constant score. We report the average metrics and +1 std. deviation over three random draws of the test set and additional sources of randomness. In some instances, e.g., J-NPE or NPE-RS in task C, the likelihood can be oo and is not plotted. For readability of the LPP metric, we linear scale between the SBI and the Prior and logarithmic scale for values below that.
6

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
Task Light Tunnel
LPP (better higher values)
ACAUC (better closer to zero)
Roal world benchmark
-1.9
JAPE
[R, G, B, ol
NPF
Observations
RoPE:
finineerty
J.NP
NNPE
HVAE
DT-onl
Pricr NPE
4.6
birking only
MPP
5B1 Prior
HINE
RoPE
RoPE
Xo
-01
Simulased
10
50
200
1000
10
50
200
1000
Calibration set size
Calibration set size
Task F Wind Tunnel
LPP (better higher values)
ACAUC (better closer zero)
Real benchmark
9B
2.3
IPERRS
[
funing
Observations
MIP
NNPE
JNPE
OT-oriy
RoPE Rope
MLP
-1.2
1
tuning-oity
Avel
NPE-SS NPF
JAPEE
OT-orly
X,
X Real
.2c 06
Simulated
200
1000
50
1000
Calibration
size
Calibration set
Figure 2: Continuation of Figure above. For task F, the ACAUC of the NPE baseline is -0.5 and not shown.
an empirical estimation of the expectation over possible observations of the negative cross entropy between the true and estimated posterior; thus, for an infinite test set, it is only maximized by the true posterior. LPP character- izes the entropy reduction on the estimation of 0 achieved by a posterior estimator 5 when given one observation, on average, over the test set. Second, the average cov- erage AUC (ACAUC) indicates the average calibration of ID credible intervals extracted from the estimated posteriors, i.e., ACAUC == An
simulated data and thus provides an upper bound on the LPP for RoPE under the independence assumption x 1 0 x, (see Appendix for more details): (NPE) a posterior esti- mator fitted to the simulated data and applied to the real data; and (J-NPE) posterior estimator trained jointly on the pooled simulated and real observations. The latter two baselines represent some first approaches that practitioner may consider. Furthermore, to asses how a fully supervised approach would fare trained directly on the calibration set, we compare the performance of RoPE MLP, which trains neural network predict the mean and log-variance of Gaussian posterior distribution by maximizing the calibra- tion set log-likelihood. We train both the MLP and J-NPE baselines in supervised way, and we thus expect these baselines to perform strongly as the size of the calibration set becomes sufficiently large and the test data is i.i.d. We also run NPE-RS (Huang et al., 2023), which trains a robust version of NPE with a regularization loss that forces the dis- tributions of NSE simulated and test data match. For fair comparison with RoPE, we use the 2000 test exam- ples to compute the regularization, informing NPE-RS as much as possible. We additionally run Noisy NPE (NNPE, Ward et al., 2022), the amortized version of RNPE intro- duced in the same paper, which improves the robustness of NPE by introducing Spike and Slab error model on simulated data statistics. We also run HVAE (Takeishi & Kalousis, 2021), which constitutes strong baseline when the simulator can be made differentiable (tasks C and E) but is not directly applicable otherwise. More details about the
1 Σラ= Ei= fo a a- - e
) (a))da where ppee, (x.) (a) denotes the credible in- terval for the jhh dimension of the parameter 0 at level Its value is positive (resp. negative) if, on average over different credible levels, parameter dimensionality, and observations, the corresponding credible intervals are overconfident (resp. underconfident). The ACAUC of a perfectly specified prior distribution is zero. The integral can be efficiently approxi- mated, as described in Appendix J. ACAUC does not capture joint calibration, as dependencies between parameters are not explicitly assessed. Alternative dependennce-sensitive metrics may require larger test sets to be stable. For all experiments, we compute the LPP and ACAUC on labeled test set containing 2000 pairs (0, xo).
Baselines. As sanity check, compare the performance of RoPE against four reference basclines: the prior p(0), which amounts to the lower bound on the LPP for any cali- brated posterior estimator when the prior is well-specified; the SBI posterior, which is an NPE trained and tested on

---

Addressing Misspecification in Simulation-basel Inference through Data-driven Calibration
experimental setup can be found in Appendix I.
we observe that the results for tuning-only are poor except for Task B, where misspecification is negligible. In con- trast, for tasks A, D, and F, OT-only exhibits performance on par with RoPE. Nevertheless, RoPE can significantly outperform OT-only, such as in tasks C and E where the misspecification is significant. We conclude that the OT step is crucial and fine-tuning is sometimes necessary. In practice, we recommend to first evaluate the performance of OT-only on the test set, and optimize y before using a subset of the test samples for fine-tuning.
5.1 Results
Figure/ compares the performance of RoPE and the other methods and baselines on the six tasks we consider with correctly specified prior. To demonstrate that applying RoPE is straightforward, we deliberately fix = 0.5 for RoPE and - = 0.9 for RoPE" in all tasks. In Figure 3, we further study the role of these hyperparameters optimizing performance.
In Figure 3a, we study the effect of entropic regularization by varying the regularization parameter 7. For all values of y, excluding ? 2 5, we observe that both I.P and ACAUC consistently improve with the calibration set size. For large values of y. the entropic regularization dominates and pushes toward uniform mapping, resulting in posteriors that approximate the prior distribution and are barely affected by the calibra- tion set size. These empirical results are consistent with the theoretical discussion in Subsection 3.2. As a recommen- dation for practitioners, our empirical evaluation suggests that values between 0.1 and provide well-calibrated and precise credible intervals. Ideally, the practitioner shall keep portion of the calibration set for validation, using it to optimize y based on the metrics of interest. If this is not possible, we recommend employing 7 = 0.5, which offers sharp and calibrated posteriors on all our benchmarks.
Effect of entropic regularization
RoPE achieves robust posterior estimation for all tasks. As mentioned above, the SBI and prior baselines provide upper and lower bounds on the expected performance of a well-calibrated posterior estimator, under the modeling assumption made in Section 3. For all tasks, even with minimal calibration budgets, RoPE is the only method that consistently returns well-calibrated, or sometimes slightly under-confident, posterior estimates while significantly re- ducing uncertainty compared to the prior distribution. As the size of the calibration set increases, we see that J-NPE and MLP adapt and their performance improves and aligns with or outperforms RoPE. This adaptability is an expected behavior in i.i.d. settings, where real-world data eventu- ally allows finding the minimizer of empirical risk among class of predictors. Nevertheless, these two baselines tend to be overconfident even for larger calibration sets, as highlighted by their positive ACAUC numbers, which are significantly larger than RoPE's in almost all configura- tions. Moreover, on task E, where posteriors are complex entropy increases with
r. We now study the impact of prior misspecification on RoPE and its unbalanced version RoPE*. In Figure 3, we compare the per- formance of RoPE (r 0.5 and 1) and RoPE* (y= 0.5 and varying 7) on extensions of Task E and C. where the ground-truth parameters of the test dataset come from distri- butions different to the assumed prior distributions. For task E, we observe that RoPE's performance is robust to the prior misspecification: it provides well-calibrated and informative posteriors, as is also visible in the corner plots of Figure 5 in Appendix C. While the gap between RoPE and RoPE* is negligible in the case of a well-specified prior (see Task F in Figure 1), under prior misspecification RoPE* leverages the additional flexibility in the OT solution and discards some of the simulated observations, achieving higher LPP. Similarly, for Task C in Figure 3c, when there no prior misspecifica- tion, RoPE (i.e, r 1) achieves the best performance; using lower values of r becomes preferable as prior misspecifi- cation increases. From these experiments, we recommend leveraging r as hyperparameter describing the confidence in the assumed prior distribution-sette offers robust performance for both well-specified and par- tially misspecified priors. The user shall also explore lower values when there is suspicion that the prior distribution is overly spread with respect to the correct prior.
RoPE for prior misppecification
conditional distributions- darker images and contain non-trivial dependencies between Coauneters-RoPee remains the best approach, even with calibration set containing more than 1000 examples. As an outlier, we observe that NPE trained on simulated data achieves the best results for the SIR benchmark (Task B), indicating that the misspecification of this benchmark is not a challenging test case for existing SBI methods and may not be a meaningful test for methods that cope with model misspecification. Finally, because interpreting these metrics can be difficult, complement these numerical results with corner and calibration plots for all tasks in Appendix K.
Ablation study. RoPE combines two steps with distinct roles, shown in Figure 4, Appendix B: (1) a fine-tuning step, which improves the domain generalization of the NSE; and (2) an OT step, aiming to model the misspecification as a coupling between simulations and observations. To better understand their respective contribution to the per- formance of RoPE, we look at two ablated versions of our algorithm: tuning-only which appends the fine-tuned NSE to the NF trained on simulated data and directly applies it to the real observations without an OT step; and OT-only, which directly performs OT with L2-norm in the original NSE space c(Xo, Xs) = h (xx) h (xx) l2. In Figure 1,
its value to 0.9
8

---

Addressing Misspecification in Simulation- -basdd Inference through Data-driven Calibration
A Entropic regularization Task Light Tunnel
B Robustness to prior misspecification Light Tunnel
OT balance parameter T
LPP
Task C C-eedullum
LPP
581
-19
SB1-
Simulator Prior A uniform des.
Observations
30-
LPP
Prior betabinomial dist
A
330
4.0
Prior
5.2
33
-14
-38.0
-30
ACAUC 9.5
ACAUC
18
FA Uaid[ 4) x (0.5, 18))
ACAUC
581 Prior
0.1
Prior
SB1
0.1
Fu Unsi 1| (05.3))
-4.1
Simulator prtor
Catb srt sire
12
5 1024
real dana X
real data
ROPE
ROPE*
1000
ROPE*
(3 A55
Mispccicatioo level
Figure 3: (a) Effect of y on the LPP and ACAUC scores of RoPE on the light-tunnel task for different sizes of the calibration set. The value of y is shown by each curve. For reference, we plot the metries achieved by the SBI posterior and prior distribution on simulated data. (h-c) Effect of E [0.1, 1] under a prior misspecification in Task E (b); and for various levels of prior misspecification in task C (c).
Discussion
servations may cause the posterior to revert to the prior. In the more challenging scenario where ğ and p" have disjoint support, even arbitrarily large calibration sets may fail to pro- vide relevant training examples, making fine-tuning highly dependent on cuttrf-diittibution generalization. Here, the OT step expected to highlight this issue, as the lack of meaningful matches will cause the transport matrix to be- come uniform, leading the posterior to revert to the prior. Appendix L further investigate RoPE's sensitivity to these practical challenges, on the Light Tunnel task, using a cali- bration set from a different prior than the test set, approxi- mating the 'same support' scenario.
While Section 5 demonstrates the effectiveness of RoPE, opportunities for future work remain, which we discuss now.
Curse of dimensionality, While our experiments focused on cow-dimensional parameter spaces, as is common for many applications of SBI, the dimensionality of may impact two critical parts of RoPE. First, with each addi- tional parameter 9x+1, given Xo, the NSE must encode up to K dependencies between 0x+1 and the other dimensions 0x While generating more simulations can address the curse of dimensionality in the simulation space, fine- tuning on small calibration may no longer suffice to cope with misspecification. Second. the dimensionality of the manifold on which the NSE projects the simulated and real- world observations will grow, and finding a meaningful coupling between the two populations may require larger sample sizes. A potential solution is to focus on marginal or 2D posterior distributions and ignore gigee--dimeniional dependencies in p(0) xo). Nevertheless, extending RoPF to such settings certainly opens new questions, e.g., concern- ing the development of better fine-tuning strategies that can leverage calibration sets with incomplete labels.
Other extensions. Similar incomplete labels, in certain applications we may only have access to noisy labels, mea- sured with a well-modeled but noisy measurement process. Further developing the fine-tuning stage to exploit such noisy labels would be necessary to make an approach simi- lar to RoPE applicable. Our strategy of modeling misspec- ification as an OT coupling opens up several avenues to address more specific problem setups. For example, we can leverage the inductive bias in the neural network architec- ture of neural OT to better cope with large test sets. This appears as a promising direction to amortize the mapping between simulation and real-world data.
Non-iid Calibration Sets. An important assumption made by RoPE is that the calibration set contains i.i.d. samples drawn from the same distribution p'(0, (0, (0) as the test data. However, practical constraints may lead to calibration data being collected from different, potentially biased, dis- tribution p(0, xo). We identify two main scenarios. If p and p share the same support, the fine-tuning step can still correct for the distributional shift, especially with a sufficiently large calibration set. For smaller sets, RoPE's robustness hinges on the neural statistic estimator's 's (NSE) ability to generalize. Moreover, the optimal transport (OT) step provides additional resilience: observations where the fine-tuned NSE performs well will be accurately matched, leading to reliable posteriors, while poorly generalized ob-
Conclusion. Motivated by important applications where SBI is not applied due to its sensitivity to model misspecifi- cation, we have introduced RoPF, a method that jointly ex- ploits calibration set and optimal transport to extend neural posterior estimation for misspecified simulators. Our experi- ments on diverse benchmarks demonstrate RoPE's ability to estimate calibrated and informative posterior distributions for various simulators and real-world examples. Overall, we have framed model misspecification as a challenge in transferring predictive models from simulated to real-world data. Our work highlights the need for labeled test set to validate inference quality, encouraging future research to treat misspecificaation as a machine learning problem.

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Acknowledgements
References
The authors would like to acknowledge Michal Klein for his help with OTT library and Maria Cervera, Laura Manduchi, Joe Futoma, Andy Miller and Pierre Ablin for providing useful feedback on the manuscript.
Avecilla, G., Chuong, J. N., Li, F., Sherlock, G., Gresham, D., and Ram, Y. Neural networks enable efficient and accurate simulation-base inference of evolutionary pa- rameters from adaptation dynamics. PLoS biology, 20(5): e3001633, 2022.
Impact statement
Bissiri, P. G., Holmes, C., and Walker, S. G. A general framework for updating belief distributions. Journal of the Royal Statistical Society: Series B (Statistical Method- ology), 78(5): 103-113-1100 130, 2016.
This paper presents a framework and an algorithm to address model misspecification in simulation-base inference (SBI). SBI is predominantly applied in scientific fields where com- plex simulators of physical phenomena are available, such astronomy, medicine, particle physics, or climate mod- eling. A priori, this circumscribes the application of our algorithm to highly specialized scientific domains in the natural sciences, precluding issues such as fairness or pri- vacy. However, its application to the scientific domain is not exempt from societal or ethical implications, particularly when computer simulations may inform research or policy decisions. In this regard, we find some properties of the al- gorithm particularly promising, such uncertainty quantifi- cation and the limitation of not drawing conclusions beyond the given expert model. However, more work is needed to deeply understand the reliability of these properties and how they are affected by violations of the core assumptions, such as well-specified prior. Such work should precede any sort of over-selling practitioners about the benefits of the al- gorithm. Rather, we see our work contribution towards more broad and successful application of SBI techniques; success in this endeavor, as for the establishment of any scientific tool, will require an iterative dialogue between the scientists who develop the methodology and those who use
Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Kono- lige, K., et al. Using simulation and domain adaptation improve efficiency of deep robotic grasping. In 2018 IEEE international conference robotics and automa- tion (ICRA), pp. 4243 4250. IEEE, 2018.
Brehmer, J. Simulation- inference in particle physics. Nature Reviews Physics, 355):305-305, 2021.
Cannon, P., Ward, D., and Schmon, S. M. Investigating the impact model misspecification neural simulation- based inference. arXiv preprint arXiv:2209.01844 2022.
Chan, J., Perrone, V., Spence, J., Jenkins, P., Mathieson, S., and Song, Y. A likelihood-free inference framework for population genetic data using exchangeable neural networks. Advances in neural information processing systems, 31, 2018.
Chen, Y., Zhang, D., Gutmann, M., Courville, A., and Zhu, Neural approximate sufficient statistics for implicit models. arXiv preprint arXiv:2010.10079 20200
Chérief-Abdellatif,, B.-E. and Alquier, Mmd-bayes: Ro- bust bayesian estimation via maximum mean discrepancy. In Symposium on Advances in Approximate Bayesian Inference, pp. 1-21. PMLR, 2020.
Collett, E. Field guide to polarization. International society for optics and photonics, 2005.
Cranmer, K., Brehmer, J., and Louppe, G. The frontier of
simulation- based inference. Proceedings of the National Academy of Sciences, 117(45):0055533066 2020.
Cuturi, M. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
Cuturi, M., Meng-Papaxanthor L., Tian, Y., Bunne, C., Davis, G., and Teboul, O. Optimal transport tools (ott): A jax toolbox for all things wasserstein. arXiv preprint arXiv:2201.13224 2022.
10

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Delaunoy, A., Wehenkel, A., Hinderer, T., Nissanke, S., Weniger, C., Williamson, A., and Louppe, G. Lightning- fast gravitational wave parameter inference through neu- ral amortization. In Machine Learning and the Physical Sciences. Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020.
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Bren- del, W., Bethge, M., and Wichmann, F. A. Shorteut learn- ing in deep neural networks. Nature Machine Intelligence, 2(1 11:66)666673 673, 2020.
Glöckler, M., Deistler, M., and Macke, J. H. Variational methods for simulation based inference. In International Conference on Learning Representations 2022, 2022.
Delaunoy, A., Hermans, J., Rozet, F., Wehenkel, A., and Louppe, G. Towards reliable iimulation-based inference with balanced neural ratio estimation. Advances in Neural Information Processing Systems, 00220005220033 2022.
Hashemi, M., Vattikonda, A. N., Jha, J., Sip, V., Woodman, M. M., Bartolomei, F., and Jirsa, V. K. Simulation-based inference for whole-brain network modeling of epilepsy using deep neural density estimators. medRxiv, pp. 2022- 06, 2022.
Dellaporta, C., Knoblauch, J., Damoulas, T., and Briol, F.-X. Robust bayesian inference for simulator-based models via the mmd posterior bootstrap. In International Confer- ence on Artificial Intelligence and Statistics, pp. 943-970. PMLR, 2022.
Hermans, J., Begy, V., and Louppe, G. Likelihood meme with amortized approximate ratio estimators. In International conference on machine learning, pp. 4239- 4248. PMLR, 2020.
Draxler, F,, Wahl, S., Schnörr, C., and Köthe, U. On the universality of coupling- based normalizing flows. arXiv preprint arXiv:2402 2677:244..06678, 2024.
Hermans, J., Delaunoy, A., Rozet, F., Wehenkel, A., and Louppe, G. A crisis in simulation- based inference? be- ware, your posterior approximations can be unfaithful. Transactions on Machine Learning Research, 2022.
Falkiewicz, M., Takeishi, N., Shekhzadeh, I,, Wehenkel, A., Delaunoy, A., Louppe, G., and Kalousis, A. Calibrat- ing neural conulation-based inference with differentiable coverage probability. Advances in Neural Information Processing Systems, 36, 2024.
Huang, D., Bharti, A., Souza, A., Acerbi, L., and Kaski, S. Learning robust statisties for inuulation-based in- ference under model misspecification. arXiv preprint (0l::3305.1877. 2023.
Frazier, D. T., Robert, C. P., and Rousseau, J. Model mis- specification in approximate bayesian computation: con- sequences and diagnostics. Journal of the Royal Statis- tical Society Series B: Statistical Methodology, 82(2): 421-444, 2020.
Jiang, Y., Yin, S., Dong. J., and Kaynak, O. review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal, 21(11): 12868-12881, 2021. doi: 10. (0.1109/ISEN.2220033333
Kelly, R. P., Nott, D. Frazier, D. T., Warne, D. J., and Drovandi, C. Misspecification-rruuss sequential neural likelihood. arXiv preprint arXiv:2301.1 13368, 2023.
Frazier, D. T., Kohn, R., Drovandi, C., and Gunawan, D. Reliable bayesian inference in misspecified models. arXiv preprint arXiv:2302.0603) 2023.
Linhart, J., Rodrigues, P. L. C., Moreau, T., Louppe, G., and Gramfort, A. Neural posterior estimation of hierarchical models in neuroscience. In GRETSI D022-XXVIlième Colloque Francophone de Traitement du Signal et des Images, 2022.
Frogner, C., Zhang, C., Mobahi, H., Araya, M., and Poggio, T. A. Learning with a wasserstein loss. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
Gamella, J. L., Peters, J., and Bühlmann, P. Causal cham- bers real-world physical testbed for Al methodol- ogy. Nature Machine Intelligence, 2025. doi: 10.1038.
Long, M., Cao, Y., Wang, J., and Jordan, M. Learning transferable features with deep adaptation networks. In International conference on machine learning, pp. 97- 105. PMLR, 2015.
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., March, M., and Lempitsky, V. Domain- adversarial training of neural networks. Journal of ma- chine learning research, 17(59):1-355 2016.
Lückmann, J.-M. Cimulatton-Based Inference for Neuro- science and Beyond. PhD thesis, Universität Tübingen, 2022.
Lueckmann, J.-M., Goncalves, P. J., Bassetto, G., Öcal, K., Nonnenmacher, M., and Macke, J. H. Flexible statistical inference for mechanistic models of neural dynamics. Advances in neural information processing systems, 30, 2017.
Gao, R., Deistler, M., and Macke, J. H. Generalized bayesian inference for scientific simulators via amortized cost estimation. Advances in Neural Information Process- ing Systems, 36:80191-80 191199919., 2023.

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Lueckmann, J.-M., Boelts, J., Greenberg, D., Goncalves, P., and Macke, J. Benchmarking dimulation-based inference. In International Conference Artificial Intelligence and Statistics, pp. 343-351. PMLR, 2021.
Rabin, J., Ferradans, S., and Papadakis, N. Adaptive color transfer with relaxed optimal transport. In 2014 IEEE international conference image processing (ICIP), pp- IEEE, 2014.
Makkuva, A., Taghvaci, A., Oh, S., and Lee, J. Optimal transport mapping via input convex neural networks. In International Conference on Machine Learning, pp. 6672- 6681. PMLR, 2020.
Radev, S. T., Mertens, U. K., Voss, A., Ardizzone, L., and Köthe, U. Bayesflow: Learning complex stochastic mod- els with invertible neural networks. IEEE transactions on neural networks and learning systems, 33(4):1452-1466, 2020.
Matsubara, T., Knoblauch, J., Briol, F.-X., and Oates, C. J. Robust generalised bayesian inference for intractable like- lihoods. Journal of the Royal Statisticul Society Series B: Statistical Methodology, 84(3):997-1022, 2022.
Rubin, D. B. Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, pp. 1151-1172, 1984.
Melis, A. Gaussian process emulators for 1d vascular mod- els, 2017. URL https:://etheee whiterose. ac. uk. /175//
Schmitt, M., Bürkner, P.-.., Köthe, U., and Radev, S. T. Detecting model misspecification in amortized bayesian inference with neural networks. In DAGM German Con- ference on Pattern Recognition, pp. 541-557. Springer, 2023.
Mensch, A. and Peyré, G. Online sinkhorn: Optimal trans- port distances from sample streams. Advances in Neural Information Processing Systems, 33:1653: 13:1657-1667, 2020.
Schmon, S. M., Cannon, P. W., and Knoblauch, J. Gener- alized posteriors in approximate bayesian computation. arXiv preprint arXiv:201 1.000...0644 2020.
Nott, D. J., Drovandi, C., and Frazier, D. T. Bayesian inference for misspecified generative models. Annual Review of Statistics and Its Application, 11, 2023.
Tabak, E. G. and Vanden-Eijnden, E. Density estimation by dual ascent of the log- log-likelihood. Communications in Mathematical Sciences, 8(1):217-233: 2010.
Pacchiardi, L. and Dutta, R. Generalized bayesian likelihood-free inference using scoring rules estimators. arXiv preprint arXiv:2104.03889, 2(8), 2021.
Takeishi, N. and Kalousis, A. thysics-integrate variational autoencoders for robust and interpretable generative mod- cling. Advances in Neural Information Processing Sys- lems, 34: 18829, 2021.
Papamakarios, G. and Murray, I. Fast e-free inference of simulation models with bayesian conditional density es- timation. Advances in neural information processing systems, 29, 2016.
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEFF/RSJ international conference on intelligent robots and systems (IROS), pp. 23-30. IEEE, 2017.
Papamakarios, G., Sterratt, D., and Murray, I. Sequen- tial neural likelihood: Fast likelihood free inference with autoregressive flows. The 22nd International Confer- ence on Artificial Intelligence and Statistics, pp. 837-848. PMLR, 2019.
Tolley, N., Rodrigues, P. L.,, Gramfort, A., and Jones, S. R. Methods and considerations for estimating parameters in biophysically detailed neural models with simulation based inference. bioRxiv, pp. 2023-04, 2023.
Papamakarios, G.. Nalisnick, E., Rezende, D. J., Mohamed, S., and Lakshminarayanan, B. Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research, 22 22(1):6117-6600 2021.
Villani, C. et al. Optimal transport: old and new, volume 338. Springer, 2009.
Perera, Y. S., Ratnaweera, D., Dasanayaka, C. H., and Abeykoon, C. The role of artificial intelligence-drive soft sensors in advanced sustainable process industries: A critical review. Engineering Applications of Artificial Intelligence, 121:105988, 2023.
Wang, Y., Yao, Q. Kwok, J. T., and Ni, L. M. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3): 2020.
Ward, D., Cannon, P., Beaumont, M., Fasiolo, M., and Schmon, S. Robust neural posterior estimation and sta- tistical model criticism. Advances in Neural Information Processing Systems, 35:33845-3388 2022.
Peyré, G., Cuturi, M., et al. Computational optimal transport. Center for Research in Economics and Statistics Working Papers, 2017.
12

---

Addressing Misspecification in Simulation- based Inference through Data-driven Calibration
Wehenkel, A. and Louppe, G. Unconstrained monotonic
neural networks. Advances in neural information process- ing systems, 32, 2019.
Wehenkel, A., Behrmann, J., Hsu, H., Sapiro, G., Louppe, G., and Jacobsen, J.-H. Robust hybrid learning with expert augmentation. Transaction Machine Learning Research, 2022.
Wehenkel, A.,
Behrmann, J., Miller, C., Sapiro, G., Sener, O., Cuturi, M., and Jacobsen, J.-H. Simulation- based inference for cardiovascular models. arXiv preprint arXiv:2307.13918, 2023.
Wrede, F., Eriksson, R., Jiang, R., Petzold, L.., Engblom, S., Hellander, A., and Singh, Robust and integrative bayesian neural networks for likelihood-free parameter inference. In 2022 International Joint Conference Neural Networks (IJCNN), pp. 1-10. IEEE, 2022.
Yin, Y., Le Guen, V., Dona, de Bézenac, E., Ayed, I., Thome, N., and Gallinari, Augmenting physical models with deep networks for complex dynamics forecasting. Journal of Stutistical Mechanics: Theory and Experiment, 2121(12); 122:112,22 2021.
Zhu, X. J. Semi-supervised learning literature survey. 2005.
13

---

Addressing Misspecification in Simulation-baser Inference through Data-driven Calibration
A Model misspecification
A.1 Mis-calibration vs Misspecification
To further elucidate the distinction between posterior calibration and model nisspecification, it is essential to highlight their respective scopes and the specific challenges they address.
Posterior calibration focuses on ensuring that the predicted posterior distributions accurately reflect the true uncertainty in parameter estimates given the observations, under the assumption that the simulator is well-specified. Methods such as those proposed by Falkiewicz et al. (2024); Delaunoy et al. (2022) address this by improving the alignment between the expected and actual coverage probabilities of the posterior. These approaches generally assume that the simulator faithfully represents the generative process of the observed data, enabling calibration to be evaluated and improved by leveraging simulations. While important, these methods do not account for discrepancies between the simulator and real-world data, which are precisely the scenarios we target in this work.
Model misspecification, on the other hand, arises when the simulator fails to capture the true generative process underlying the observed data. This results in systematic discrepancies that cannot be corrected solely by optimizing posterior calibration techniques. Misspecification introduces gap between the simulated and real-world distributions, and this gap is only observable when real-world data is available. Unlike posterior calibration, addressing misspecification requires methods that can robustly leverage the simulator despite its inaccuracies, while incorporating real-world observations to mitigate the impact of the mismatch.
In our work, we explicitly focus on handling model misspecification. This distinction is reflected the design of our approach and the evaluation scenarios we consider, such as Task E, where the simulated data diverges significantly from the real-world measurements. While posterior calibration methods may perform well in well-specified context, they are not designed to cope with such gaps. Instead, we prioritize creating predictive models that balance informativeness and robustness in the presence of misspecification, even if achieving perfect calibration remains an open and challenging problem.
A.2 Comparison between model misspecification definitions
We provide toy example to show how a simulator may be well-specified according to the standard definition of misspecifi- cation but still provide biased estimates of the target parameter when applied real data.
Consider the following setting: a noisy sensor measures some physical quantity 0, producing measurements x....., x Lid. P*, where p == ~N 1) is a normal distribution centered around the 'true' value 6*, Let {P& A E IR} be a simulator of this process with Fa == (N(,,, 1), where := + A and X > 0 is a fixed scalar constant, which is a misspecification in the simulator that falsely accounts for a non-existing offset in the sensor that produced the real observations x!..... x².
According the standard definition of misspecification, the simulator is well specified, setting 8 A yields Pg = P
However, the posterior estimates we obtain with this simulator biased with respect to the parameter 0"
To see this, let us compute the posterior under Gaussian prior N/00,, 11) over the parameter 0, centered on the true value 0*,
14

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
Taking advantage of the conjugate prior, the posterior p(0 x ;or...;X6
x²) becomes
p(O Xpy.... x5) x p(0) (x ..,,,
x
=p(0)
p(xo 8)
1
- 2 -07)
II
1
(-1 1
(-2 [x. - -1)²)
exp
exp
2
V2T
1=1
n
0 0²2²²
Σ
(xb 112
exp
2
1=1
exp
8 (07)² 200"
Σ
(xx)
n1 2p Σ x
i=1
(drop const. terms)
x exp
8 200"
+ np
2 2p X0
(n = ++
exp
0 200*
+ n02
+ +nA + 2nA0 20
X
22
1=1
1=1
(drop const. terms)
exp
0² 200* n0² +n20 20
Σ
1=1
1)0² 20(0" -n -n+ +
Σ xb)
exp
i=1
1
Ø² 28
1
(0" - nd + Σ x561
exp
2(n + 1)-1
n n211 1
1
1 (0* nn + Σ x0)
(complete square)
x exp
2(n + 1)-1
n++1
that normal distribution \NT, 72) with mean
1+m
0" - nd +
X
and variance 72 = (n + 1) Thus, the posterior is biased, e.g., the posterior mean 7 is biased estimator of 0* with El0* 7| 8" -~
15

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Problem setup
ROPE algorithm
simulated posterior p x
Simulated data
Real observations
Rg
Reality (unknown)
X
h
NPE
Calibration set
{(0', xx) No
fine
Physical parameters
Misspecification
1==1
tuning
OT solution p
Test set 1 nh {x) o=o==
NSE 8y²
000
Inference
Simulator
X
pipeline
Simulated observations
p(0 xxy) = Σ3 The N.P x!)
Figure 4: (left) Problem setup: consider real-world process which depends on some physical parameters O. Given real observations have access
X, of the process, goal provide uncertainty quantification the underlying parameters 0. To help misspecified simulator that takes parameters 0 as input produces simulated observations x,. (right) visualization RoPE. The training consists of two steps: (1) given the simulated data, we approximate the posterior using NPE, resulting in the NSE h (2) using
the calibration set, we fine-tune b into gp* using the objective (6). At test time, solve the optimal transport (OT) problem between the representations (h (x²) 17 P and (g.- (xx) Ne resulting in our estimated posterior (5), the average of simulations' posteriors weighted by the OT solution P See Algorithm in Appendix for more details.
16

---

Addressing Misspecification in Simulation-basce Inference through Data-driven Calibration
B
The RoPE Algorithm
Algorithm Posterior Inference using Robust Neural Posterior Estimation (RoPE)
Simulator S(0,e), s), prior distribution p(0), calibration set {(xx, ) test {xx) Oulput: p(0 xx)vvx D Step 1: Neural Posterior Estimation (NPE)
Input:
Train neural network h and conditional normalizing flow p(0) using NPE:
P. w arg max (r) p(0 (S(0, e))] paa ex2[(0,1]
Step 2: Fine-tune sufficient statistics h on the Calibration Set
Sy == (h
Ctrain, Coal RandomSplit(C, } bestral
for Niter do
1/
a [E(ax)) (x() (e(hou =)
cur
= (0.,,, (0.x..)CCmml (xx) Ee le (S(),=)))
if cureal bestral then
best cur 1A'
end if end for Step 3: Generate Simulations for Test Set (N₂ = No)
[ where x S(0),)) E) 0 ~~ (0) 1[0
Step 4: reguaaiee OT
Cij =Www (xx) g6 (xx) Vi No.....No) (l..... No) 1N,
p" = arg min + pp'IN.) PEB.
+ + log P)
N₂
Step 5: Compute Posterior Distributions
N
p x0)x
p.je (0 h (x5)
j=1
Return ((0))
Vxl,
17

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
C Prior Misspecification Experiments
Prior misspecification on Task C. With this experiment we aim to better understand the role of T when RoPE is applied with different levels prior misspecification. We thus re-use the same setup as in Figure but add prior misspecification as a mixture between the assumed prior and much tighter uniform distribution. As the weight of the tighter uniform distribution increases, the prior gets more misspecified. The experimental setup follows closely the in the well-specified case (see Section I.2), except calibration samples drawn from the prior (as this would be the case in real-world application) and we compute the OT coupling for values of 7 [0.1, 1|.
The results in Figure 3b demonstrate that RoPE can be robust to prior misspecification. In particular, we observe that plays the expected role and that values below 1. enable RoPE to perform better when the true prior is only subset of the prior used to generated synthetic data.
Prior misspecification on Task E. In some practical settings, it is unlikely that the prior used to generate synthetic data will match the distribution of the target parameters in the real data. For this reason, consider semi-balanced formulation of OT, providing the flexibility discard simulations with corresponding real-world observations.
To evaluate the effect of misspecified prior on RoPE and RoPE*, we perform experiment that would resemble its use in real applications like the ones we outline in the introduction. In such settings- inferring cardiac parameters or chemical concentrations- the target parameters are limited to a range of validity, and a likely choice for the practitioner would be to select uniform prior over this range.
To replicate this setting, collect a new real-world dataset from the light tunnel (Task and train RoPE on synthetic data originating from uniform prior, do for the results shown in Figure 1. However, then apply RoPE to real data generated from different (betabinonial) distribution over the target parameters.
Test exars
emarp
Figure 5: Visualization of estimated posteriors. Corner plots of the posteriors estimated by RoPE the prior- misspecification experiment from Fig. above. We show, in different colors, the estimates for four observations sampled at random from the test set, for RoPE (left) and RoPE (r = 0.5) (right) formulation of the OT step, and a calibration set of size 50; the horizontal and vertical lines correspond to the ground-truth value of the parameters.
18

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
D Robustness to Distribution Shifts
Task E Light Tunnel
Out-of-distribution performance
Training distribution
Target distribution
-1.9
ROPE
Training distributiofTarget distribution
J-NPE
flipped images
NPE
abservations abservalire
tuning-only
OT-only
Prior
4.6
B LPP
MLP
10
50
200
1000
10
50
200
1000
X
X
Phior
ACAUC
10
50
200
1000
10
50
200
1000
Calib, set size
Calib. set size
Figure 6: Dut-of-distributio performance of RoPE and some baselines. We train RoPE and other baselines the same light-tunnel data as in task E (training distribution), but apply it to test sets originating from target distribution where the real-world images are flipped vertically. We compare the performance on test sets from both distributions, showing the LPP and ACAUC scores for each method. For comparison, the right plot we show again the LPP curve (light gray, dotted) attained by RoPE under the training distribution. The performance of RoPE is barely affected as cannot exploit any signal in the real images (xx) beyond what is encoded in the simulator, and the simulator output (x,) invariant the transformation remains virtually unchanged. On the other hand, the performance of MLP and J-NPE drops in the target distribution, as these methods are not limited in what information they can exploit from the real observations on which they are trained, potentially learning shortcuts that are not present in the target distribution. This results demonstrate that the simulator embeds the right invariances, our modeling assumption X. 1 Xs can be favorable to oatoof-distribution generalization.
consider. Because NPE is not trained real observations, its performance, although poor, also
E Optimal Transport Coupling as a joint distribution
With our conditional independence assumption, the problem modeling p(x. reduces to modeling p(xo xx)) instead. we assume the prior well-speeified, this task equivalent modeling p(x., under the constraint that the corresponding marginal p(x.) = p(x,, xo dxx. equals f ((x)p(xx 8)d8. By construction, the OT coupling, x*, respects the constraint on the marginals, f >=( (xx, xo )dx. p(x.) and (xx.xx)dxx p(x.) and the exact instantiation =² depends also on the chosen cost function which can always be defined to yield any given conditional p(x, x,) that respects the constraint p(x. x )x))(xx) = ). x² can thus model the "right" posterior, provided the right cost function is used. In the case, where the prior cannot be trusted, we suggest to use < 1 and relax the OT formulation. In this case, we only enforce that all elements of p(z.) are matched to subset of the elements of p(z=). This implicitly assumes that the assumed prior p(0) overly conservative and covers p" (0). We believe this reasonable assumption as it is often easy derive physical bounds for the parameter values and use uniform distribution.
19

---

Addressing Misspecification in Simulation- -based Inference through Data-driven Calibration
F Self-calibration Property
We say RoPE is self-calibrating because, by design, the posterior distribution marginalized over observations tends to the prior as the number of simulation increases, that is,
J 8(0 xo)p(xo)dxx = p(0)-
This property is also called marginal calibration, and is necessary condition for posterior estimation method to be calibrated. Considering NPE, 510 xx), is marginally calibrated and observations Xo are generated from the assumed prior, that is sampled from an unknown distribution p(xo) I p(xo 10)(()), we can show RoPE is marginally calibrated. Indeed, No, considering the Monte-Carlo approximation of the marginalized posterior distribution over the test set D. = {xx))=] we have,
J/ p(0 (xo) )dxo = Ep(x.) 1p(0 xo)]
(8)
1 No
5(0 xx)
(9)
No
1=1
No No Pipii (0 x!)
(10)
No
1=1 j=1
N.
Σ
5(0 x!)
(11)
1 N,
5(0 xxx)
(12)
N
8 p(0),
(13)
where we use the definition of the transport matrix to get Σ Pij N.
The last approximation tends to be exact as the
number of simulations increases, if the NPE is marginally calibrated.
G Learning Minimal Sufficient Statistics with Neural Posterior Estimation
We now discuss why NPE may learn a minimal sufficient statistic under perfect training. First, under a sufficiently large validation set, NPE's objective function is only optimal on the validation set if NPE models the true posterior defined implicitly by the prior p(0) and the likelihood corresponding to the simulator S. This consistency has been proven in (Papamakarios & Murray, 2016) and is the motivation to use such an objective when estimating density. Second, some normalizing flows, such as autoregressive UMNN flows (Wehenkel & Louppe, 2019), are universal approximators of continuous densities. In addition, neural networks are also universal function approximators. As such, we can claim that it is always possible to parameterize the NCDE (e (x)) such that the class of functions its parameters represent contains the true posterior. We directly observe that x only used by the NCDE through (x). Thus, under perfect training pe- (#) h (x)) p(0 x) and b (x) is sufficient statistic for 0 given x under the simulator's model.
Without additional constraints, we cannot claim anything about the minimality of h (x). Nevertheless, can enforce the neural network h (x) to have an information bottleneck and thus reduce the information carried. In practice, we choose the output dimension of bwr (x) so that the NCDE achieves optimal performance on the test set Because in the context of SBI we can generate as many (simulated) samples as needed, we can obtain estimators that closely approach the simulation's posterior and a minimal sufficient statistic.
20

---

Addressing Misspecification in Simulation-baser Inference through Data-driven Calibration
H Computational cost of RoPE
Running NPE is broadly recognized as having a low computational cost: once the upfront training is complete, the cost of inverting the normalizing flow to sample from the posterior during inference becomes negligible as the number of test observations increases. This makes NPE more efficient than methods like Approximate Bayesian Computation or Markov Chain Monte Carlo (when the simulator allows likelihood evaluation). RoPE introduces additional computational costs on top of running NPE: (1) the OT coupling computation, i.e., solving (2), and (2) obtaining samples from the estimated posterior distributions, to compute the posterior estimate defined in (5). The computational cost of solving the transport problem with the Sinkhorn algorithm (Cuturi, 2013) is quadratic in the number of real-world observations. The sampling step has a negligible cost as it directly sub-samples from the set of points generated with NPE.
In our experiments, solving the OT optimization for 2000 test examples takes less than a minute on an MI MacBook Pro. Sampling from the mixture of posterior distributions involves caching 10,000 samples for each simulation and generating 5,000 samples by sub-sampling from the mixture using the OT coupling matrix. This caching process takes under three minutes, and is comparable to the cost of running NPE alone.
Extending RoPE to handle larger test sets or an online setting (processing test examples one at time) is outside the scope of this work. Nevertheless, mehtods like Neural OT (c.g., (Makkuva et al., 2020)) and online Sinkhorn (Mensch & Peyré, 2020) should provide good solutions to make RoPE fully amortized.
I Experimental Setup
In this section, provide more details our experiments. For completeness, we provide details on the neural architectures and training hyperparameters. However, we encourage the reader interested in reproducing our experiments to examine our code directly (a link to the code will be made available in the public version of the paper).
For all methods training on calibration set we keep always keep 20% of the calibration to monitor validation performance and we select the best model based on this metric.
For the MLP we use the same architecture as the NSE for all our experiments and optimize its parameters on the calibration set with Adam and learning rate equal 0.0003, we select the best model based on the LPP attributed the validation subset of the calibration set.
Computing the SBI baseline. We take the ground-truth labels {(0') from the test set (0', xx)) on which we compute all the metrics for Figure I; for each label 8', we simulate a synthetic observation x; = S(0"), collecting them into a "synthetic" test set {(0', xx) ) then, we apply to it the NSE+NPE pipeline (simulated posterior in Figure 4, right) to obtain the posterior estimates which we then evaluate. In this way, the baseline represents the performance we would hope to achieve if there was no misspecification and the simulator perfectly replicated the real observations (up to the stochasticity of the simulator itself).
L1 Task A: CS & Task B: SIR
Task A (synthetic): CS. We reproduce the cancer and stromal cell development benchmark from Ward et al. (2022). The simulator emulates the development of cancer and stromal cells in a 2D environment as a function of three Poisson rate parameters (de, Ap. Aa). The observations are vectors composed of the number of cancer and stromal cells and the mean and maximum distance between stromal cells and their nearest cancer cell. Synthetic misspecification is introduced by removing cancer cells that are too close to their generating parent.
Task B (synthetic): SIR. We also use the stochastic epidemic model from Ward al. (2022), which describes epidemic dynamics through the infection rate 3 and recovery rate 7. Each observation is vector composed of the mean, median, and maximum number of infections, the day of occurrence of the maximum number of infections, the day at which half the total number of infections was reached, and the mean auto-correlation (lag 1) of the infections. Misspecification is delay in weekend infection counts, of which 5% are added to the count of the following Monday.
We refer the reader to Ward et al. (2022) for more details about the simulator and prior distribution. We use the exact same setting as theirs.
21

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
NEURAL ARCHITECTURE & TRAINING HYPERPAARAMETES
For all methods use the same backbone MLP as the NSE with ReLU activations and layers composed of [4K, 16K, 16K, 12K, 3K] neurons, where K is the dimensionality of O. The NF is 1-step UMNN-MAF (Wehenkel & Louppe, 2019) with [100, 100, 100) neurons for both the autoregressive conditioner and normalizer. For NNPE, train the UMNN-MAF on simulations poluted by Spike and Slab errors. We train models with Adam and a learning rate equal to 0.0005 and all other parameters set to default. We optimize the SBI model for 10 gradient steps and select the best model on random validation sets containing 10 simulations.
1.2 Task C: Pendulum
DESCRIPTION
The first task is inspired from the damped pendulum benchmark commonly used to assess hybrid learning algorithms. Given 2D physical parameter = [wo, A], where wp E IR denotes the fundamental frequency and A E RR the amplitude of friction-less pendulum, the simulator generates the horizontal position of the pendulum 200 discrete times during uniformly sampled in 10 seconds interval as
Xs (0( .,,
, 0(t 10s)] 10200
where (tt) A cos(wnt + v) 40 ~ u(-
(14)
The relationship between the parameters and the simulation is thus stochastic as accounts for an unknown phase shift when the measurements start. We generate real-world observations synthetically by replacing O(t) from (14) by
0(() et coss(ard + v)) U[-
U[O.1]. 1].
where represents the effect of friction. We also add Gaussian noise both simulated and real-world data represent the inaccuracy of sensor measuring the pendulum's position. The prior distribution is a product of uniform distribution, p(0 == [wo. A)) = (4 3] x [0.55, 10).
NEURAL ARCHITECTURE & TRAINING HYPERPARAMETERS
Neural Posterior Estimator: The NSE is a ID convolutional neural network, with the architecture described in Algorithm 2. The NCDE is one-step discrete normalizing flow with an autoregressive conditioner and UMNN (Wehenkel & Louppe,
Algorithm 2 Convolutional Neural Network for Tasks A and D.
1: Convid(L, 16, 3, 1, dilation 2, padding = 1)
14: ReLU()
2: ReLU()
15: AvgPoolld(3.1)
Conv1d (1,,1dd1,,4,,,,2. dilation 2, padding 1)
16: Convid(128, 128, 3, 1, dilation 2, padding 1)
4: ReLU()
17: ReLU()
5: (v)))
18: Flatten()
6: Convld((4) 128, 1, dilation 2, padding = 1)
19: Linear(2048, 512)
7: ReLU()
20: ReI.U()
8: Convid(128, 128,3, dilation = 2, padding =
21: Linear(512, 128)
9: ReLU()
22: ReLU()
AvgPoolld(3,1)
23: Linear(128,32
11: Convid(128, 3, 1, dilation 2, padding
24: ReLU()
12: ReLU()
25: Linear(32, 10)
13: Conv1d(128, 128,3, 2, dilation
2, padding
2019) as the normalizer. The autoregressive conditioner is MADE with ReLU activation and 3 layers of 100 neurons that output 10 dimensional vector to the UMNN. The UMNN has an integrand net with 3 layers 100 neurons with ReLU activations. For training the NPE, we use a batch size of 100 and a learning factor equal to le-4. NPE is trained until convergence. Other parameters are set to default values and should marginally impact the NPE obtained.
22

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
Algorithm 3 UNetlD Architecture
Algorithm 4 BlockID(inn B out channels))))
1: UnetlD:
1: monvid(in_chaate out chn_chaneels, Bernel_size-z padding=1)
2:
EncoderlD:
3:
sacchinn_cannels = 1, out channels = 64)
2: ReLUO)
4: Block(in_channels 64, out channels = 128)
3: Conv cchnnnees
channels,
out channels,
5: Block channels 128, out_ qut_chhneels 256)
hernel_size=te padding=1) ReLUO
6: Block(in_channee channels 256, out_channels 512)
7:
Block(in channels 512, out channels 1024)
8:
MaxPoolld(2)
9:
Decoder1D:
Algorithm 2D Convolutional Neural Network
10:
Cony Tonnsraasposeel ltspppoose(((1220 d(1024 5, 512, stride = 2)
Conv2d(3, 64, 3, 2, dilation= ReLUO Conv2d(64, 128, 3, 2, dilation=1), ReLUO MaxPool2d(33)
11:
Block(in_channll) = 1024, out channels = 512)
12:
ConyTraaspoose 2,, 2, stride = 2)
13:
Block(in_channels- 512, out_channels 256)
4: Conv2d(128, 128, 3, 2, dilation=1), Relin
14:
ConyTrranspose I(256, Id 128, stride 2)
5: Conv2d(128, 64, 1, 1, dilation=1), ReLUO
15:
Block(in_channel) 256, outoout__too cut_chhneels 128)
Conv2d(64, 3, 1, 1, dilation=1), ReLUO Flatten()
16:
ConvTranspossee 122 64, stride 2)
17:
Block(in_channehs = 128, nut_chhannel = 64)
Linear(27, 100). ReLU( Linear(100, 20)
18:
ConyTranspose 1, 2, stride = 2)
19:
channels 64, cutttuuanneeeeeee = 1)
Conv 1d(4,, 1)
RoPE NSE. We have selected the best NPE based on the validation set with 10000 examples generated with the simulator. The NPE is fixed to one best-of-all model. We fine-tune the NCDE with a learning rate equal to le-5 for 5000 gradient steps on 80% the full calibration set. We use I-sample Monte Carlo estimate of the expectation (6).
J-NPE. To train J-NPE, simply randomly use batch composed of 50% of simulated pairs (0, x,) and of 50% (0, xx) from the calibration set. We use the same architecture and byper-parameet based on the best training set performance. We do 50 epochs with 50000 simulated examples for each epoch. The batch size is 100.
the SBI NPE. The best model is selected
HVAE, For the HVAE, we re-use the NPE model as the physics encoder and replace the decoder with a deterministic version of the simulator, thus removing the Gaussian noise on a random phase shift. In addition, we follow the approach of Takeishi & Kalousis (2021) and have 1) real-world encoder that maps x to Za. 2) a reality-to-phy thysics-to-reality decoder. The real-world encoder has the same architecture as the NSE of the NPE and outputs the mean and log- variance of 5D latent vector za. The manlity-to-hhssiee and physics-to-realhe also have the same architectures and are two conditional ID U-Net with neural network architecture described in Algorithm 3.
encoder, and 3)
To train the HVAE, we freeze the parameters of the NPE and optimizes the ELBO as well as a calibration loss that evaluates the likelihood assigned to the true physical parameters. All distributions are parameterized by Gaussian with mean and log-variat log-variance predicted by the neural networks. We do not use any additional losses as we expect constraining NPE and using the calibration set should already provide the necessary support to use the physics in a meaningful way. The HVAE is trained on the 2000 test examples as it is the only real-world data, calibration set aside, that have access to. We use batch size equal to 100 and learning rate equal to le-3, We believe obtaining a better HVAE is possible. However, we emphasize the complexity of setting up good HVAE for the only purpose of statistical inference over parameters.
DATASETS
For this task, we can generate samples (0,xs) on the fly to train the NPE. The calibration and test sets are also generated randomly by sampling from the prior distribution and using the damped pendulum simulator.
23

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
1.3 Task D: Hemodynamics
DESCRIPTION
Inspired by Wehenkel et al. (2023), we define the task of inferring important cardiovascular parameters from normalized arterial pressure waveforms measured at the radial artery. The simulator uses many physiological parameters that modulates the heart function, physical properties of the 116 main arterial segments, and behavior of the vascular beds. Our inference concerns two parameters of the heart function, : [SV, LVETJ. the stroke volume (SV) is the amount pumped out from the left ventricle over the heart beat modeled, and the left ventricular ejection time (LVET) is the time interval between opening and closure of the aortic valve. Other parameters, such the heart rate arteries' stiffness, effects and are randomly sampled from realistic population distribution. An additional source of randomness is added by modeling measurement errors with a white Gaussian noise and randomizing the starting recording time with respect to the cardiac cycle. The simulator produces 8-second timeseries x E IP 1000 sampled at 125Hz As synthetic misspecification, the simulator assumes all arteries have the same length over the population considered, whereas "real-workee data are artificially generated by also varying the length of arteries and account for the effect of human's 's height. The simulator based on the openBF PDE solver (Melis, 2017) specialized for hemodynamies, which is not differentiable and takes approximately one minute to simulate one sample on standard CPU. This synthetic tasks represent common scenario in which simulator, although faithful to the effect of certain parameters, misses additional degrees of freedom that exists for the real-world data.
considered as nuisance
NEURAL ARCHITECTURE & TRAINING HYPERPARAMETERS
Algorithm 6 CNN Architecture for Task C.
1: Id1, 16, 3, 1, dilation=2, padding=1), ReLU(
2: Convid(16, 64, 3, 2, dilation=2, padding=1), ReLU(
3: AvgPoolld(4. 2)
4: Convld(64, 128, 3, 1, dilation=2, padding=1), ReLUO
5: Convid(125, 128, 3, 2, dilation=2, padding=1), ReLUO
6: AvgPoolld(4, 2)
7: Convid(128, 128, 3, 1, dilation=2, padding=1), ReI.U)
8: Conv Id(128, 128, 3, 2, dilation=2, padding=1),
9: AvgPoolld(4, 1)
10: Co2v1d(128, 128, 3, 1, dilatioon22, padding=1), ReI.U)
11: Flatten()
Linear(1022) 512), ReLUO
13: L51e 128), ReLUO
14: Linear(128, 32), ReLUU
15: Linear(32, 5)
Neural Posterior Estimator: The NSE is the 1D convolutional neural network described in Algorithm 6. The NCDE is 5-step discrete normalizing flow with autoregressive conditioner and affine normalizers. Each of the 5 autoregressive conditioners is MADE with ReLU activations and layers of 300 neurons that output 4 dimensional vectors used to parameterize the affine transformations. For training the NPE, we use a batch size of 100 and a learning factor equal to 5e-4. NPE is trained until convergence. Other parameters are set to default values and should marginally impact the NPE obtained.
RoPE NSE. We have selected the best NPE based on the validation set with 2000 examples generated with the simulator. The NPE is fixed to one best-of-all model. We fine-tune the NCDE with a learning rate equal to le-5 for 2000 gradient steps on 80% of calibration set. We use a I-sample Monte Carlo estimate of the expectation in (6).
J-NPE. To train J-NPE, we simply randomly use batch composed of 50% of simulated pairs (9,x,) and of 50% (9, xx) from the calibration set. We use the same architecture and pareerpprraettes as the SBI NPF. The best model is selected based on the best training set performance. We do 50 epochs with 6000 simulated examples for each epoch. The batch size is 100.
HVAE, There is no HVAE for this experiment as the simulator is non-differentiable.
24

---

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
DATASETS
For this task, we cannot generate samples x.) on the fly train the NPE. For the purpose of this experiment, we have generated 10000 simulations and real-world observations. Our fine-tuning strategy approximates (6) by finding the simulations with the closest parameter value.
1.4 Task E: Light Tunnel
DESCRIPTION
We use one of the light-tunnel datasets from the causal chamber project (Gamella et al., 2025, causalchamber orgg) In particular, we use the data from the ap_1 i ii_iss__0000 0_ss_00....0000 005 experiment in the lt_camera_vi dataset. The light tunnel is an elongated chamber with a controllable light source at one end, two linear polarizers mounted on rotating frames, and a camera that takes images of the light source through the polarizers. We refer the reader to Gamella et al. (2025, Figure 2) for a complete schematic. Our task consists of predicting the color setting of the light source (R, G. B) E [0, 2) and the dimming effect of the linear polarizers a (0, 1] from the captured images. As misspecified simulator of the saangegenerating process, we adopt the simple model described in Gamella et al. (2025, Model FI, Appendix D). A Python implementation is available through the causalchamber package (models wwdeem causalchamber org for more details. As input, the simulator takes the parameters 0 == [R. G. B, ai and produces an image consisting of a hexagon roughly the size of the light source, with an RGB color vector equal to [aRR, oG, o.). The factor a == cos"(0; 82), where 0,, 0g denote the angles of the two polarizers, corresponds to Malus' law (e.g... Collett, 2005), which models the dimming effect of the polarizers as a function of their relative angle. Besides the obvious misspecification with respect to image realism (see Figure 1), the model ignores other important physical aspects, such as the spectral response of the camera sensor the non-uniform effect of the polarizers the different colors- details can be found in Gamella et al. (2025, Appendix D.I...22). The prior is uniform over colors and polarizer angles, which leads to a non-uniform prior over the dimming effect
visit
NEURAL ARCHITECTURE & TRAINING HYPERPARAMETERS
Neural Posterior Estimator: The NSE is the 2D convolutional neural network described by Algorithm 5.
The NCDE is also a one-step discrete normalizing flow with an autoregressive conditioner and UMNN (Wehenkel & Louppe, 2019) as the normalizer. The autoregressive conditioner is a MADE with ReLU activation and 3 layers of 500 neurons that outputs a 10 dimensional vector to the UMNN. The UMNN has an integrand net with layers of 150 neurons with ReLU activations. For training the NPE, we use batch size of 100 and a learning factor equal to 5e-4. NPE is trained until convergence. Other parameters are set to default values and should marginally impact the NPE obtained.
RoPE NSE. We have selected the best NPE based on the validation set with 10000 examples generated with the simulator. The NPE is fixed to one best-of-all model. We fine-tune the NCDE with a learning rate equal to le-4 for 2000 gradient steps on on 80% of the calibration set. We use a I-sample Monte Carlo estimate of the expectation in (6).
J-NPE. To train J-NPE, we simply randomly use batch composed of 50% of simulated pairs x0)x.) and of 50% (0, xo) from the calibration set. We use the same architecture and hyper-parameters as the SBI NPE. The best model is selected based on the best training set performance. We do 50 epochs with 1000 simulated examples for each epoch. Simulations are generated randomly for each batch by sampling the prior and simulating for the corresponding parameters. The batch size is 100.
HVAE. For the HVAE, we re-use the NPE model as the physics encoder and use the simulator as is as it is differentiable without additional effort. In addition, we follow the approach of Takeishi & Kalousis (2021) and have 1) real-world encoder that maps x, to za, 2) reality- saality-to-hhyr to-physics encoder, and 3) a physics-to Mhysics-to-realits to-reality decoder. The real-world encoder has the same architecture as the NSE of the NPE and outputs the mean and log- vag-varancce of 5D latent vector zu- The reality-to- -o-yyssics and physics-to-reality also have the same architectures and are two conditional 2D U-Net with the architecture described by Algorithm 7.
To train the HVAE, we freeze the parameters of the NPE and optimizes the ELBO as well as a calibration loss that evaluates the likelihood assigned to the true physical parameters. All distributions are parameterized by Gaussian with mean and log-variance predicted by the neural networks. We do not use any additional losses as expect constraining NPE and using the calibration set should already provide the necessary support to use the physics in meaningful way. The HVAE is
25

---

Addressing Misspecification in Simulation-basl Inference through Data-driven Calibration
Algorithm 7 2D UNet
Algorithm 8 Block2D(in ChhnnnlDin outtotutttttet channels))))
1: Encoder2I::
1: monv2d(in_channele mo__chaneels, kernel_size=))
2:
jul_cchanels=64)
padding=1, bias=False)
bias=FFee
3:
out_ mu__channels=128)
2: BathhNoorm2d(uu features=out_chammee
4: Block2D(in 1 out uut_chammels=256)
3: ReLU(inplace=Tuu))
5: Block2DDin_chamnelss out aut_channels:512)
Conv2d(out_channel)
out out_ channels,
6: Block2DGin_chamells5511 (024
kernel_size=3, padding=1, bias=False)
7: MaxPool2d(2))
BatchNorm22((um features=out chaannelsss
8: Decoder2D:
6: Rel.U(inplace=/Tuu)
9: Conv (ranspose22(11022+55, 512, 2, stride=2)
10:
Block2DDin_chhamelss112224, out 1
11:
Conv ConnTraassos222(511222555 256, 2, stride=2)
Block2Düin_chamnels=511,,, out channels=2566
Con/Traasppose222(566 128, stride=2)
14:
Block2DDin_cchamell==556 out out_channell==188
15:
Conv ConvTranspose22(128... 64, stride=2)
16:
Berk22@in_channell--/ out channels=64)
17:
ConvTrans Con/Transpose2((44 1, stride=2)
18:
Block2D(in (acfffi__hanel_6,,,0 out_ jut_channels=1)
19:
Conv2d(64, 1,
trained on the 2000 test examples as it is the only real-world data, calibration set aside, that we have access to. We use a batch size equal 100 and learning rate equal le-3, We believe obtaining a better HVAE possible. However, we emphasize the complexity of setting up good HVAE for the only purpose of statistical inference parameters.
DATASETS
For this task, we can generate samples (0, x.) on the fly to train the NPE. However, the calibration and test sets are real-world data. We ensure there is not overlap between calibration and test set. The is no randomization and the test set is constant for all experiments, the calibration set are also fixed for a given calibration set size.
1.5 Task F: Wind Tunnel
DESCRIPTION
We use one of the wind-tunnel datasets from the causal chamber project (Gamella et al., 2025, causalchamber: org). In particular, we use the data from the load_out_____ 0.5 v1 dataset. The tunnel chamber with two controllable fans that push air through it and barometers that measure air pressure different locations. A hatch precisely controls the of additional opening the outside (see Gamella al., 2025, Figure 2). The data is collection of pressure curves that result from applying short impulse to the intake fan load and measuring the change in air pressure using one of the barometers inside the tunnel. Our inference task consists of predicting the hatch position, 0 = [M] E [0, 15] given a pressure curve (see Figure 1). As a simulator model, we combine the models A2 and C3 deseribed in Gamella et al. (2025, Appendix D); we numerically solve the ODE in model A2, and add stochastic components to simulate the sensor noise and the unknown time point at which the impulse applied. This results in the simulator being neither differentiable nor deterministic. Python implementation of the complete simulator available in the causa chamber package (models le imulator a2 ________________________________ visit causa cchamber org for more details. Misspecification arises from the many simplifying assumptions needed to model the complex dynamics of the airflow inside the tunnel-mon details can be found in Gamella et al. (2025, Appendix D.IV.1.2).
__osrrr___________________ _sror_____________ $___ss_d_oo_____________________ fownwindd experiment in the wt Lnntake_impuut
Neural Posterior Estimator: The NSE and NCDE have the same 1D convolutional neural network as for Task A. For training the NPE, we use batch size of 100 and a learning factor equal to 5e-4. NPE is trained until convergence. Other parameters are set to default values and should marginally impact the NPE obtained.
26

---

Addressing Misspecification in Simulation- -base Inference through Data-driven Calibration
RoPE NSE. We have selected the best NPE based the validation with 10000 examples generated with the simulator. The NPE is fixed to best-of-all model. We fine-tune the NCDE with learning equal to le-4 for 20000 gradient steps 80% of the calibration set. We use a I-sample Monte Carlo estimate of the expectation in (6).
J-NPE. To train J-NPE, we simply randomly use batch composed of 50% of simulated pairs (0, xx) and of 50% xo) from the calibration set. We use the same architecture and hyper-parameted as the SBI NPE. The best model is selected based on the best training set performance. We do 50 epochs with 10000 simulated examples for each epoch. The batch size 100.
HVAE. There is HVAE for this experiment as the simulator is non-differentiable.
DATASETS
For this task, although slightly slower than Task A and B, we can generate samples (0, xs) on the fly to train the NPE. However, the calibration and test sets are real-world data. We ensure no overlap between the two sets for all calibration set sizes. All sets are fixed for all experiments.
27

---

Addressing Misspecification in Simulation- Inference through Data-driven Calibration
Task Wind Turnel Possettor cslimates
Postorior estimanes
244
ROPE
120
256
135
25
64
24
JPE
... OT-srdy
HAEE
Figure 7: Credible intervals of the posterior estimates levels 65% and 90%, for single sample from the light-tunnee task. The black stars denote the true value the parameter. (center) Posterior estimates for a single test sample from the task, where the true parameter is denoted by a vertical black line.
J Computing ACAUC
Algorithm 9 Statistical Calibration of Posterior Distribution
Input: Dataset of pairs D {(0', xx) x')}, Posterior estimator 5(0 x), Number of samples N.
Output: ACAUC
AVG CALIBRATION
..]] 2: for & (1....,K)) do
3: Initialize an empty list CredLevels
4: for x0) E D do
5: Initialize an empty list Samples
6: for j to M do
7: Sample g² from p(0 xx)
8: Append ø Samples
9:
end for
10: Sort Samples
11:
Compute the rank (position in ascending order) r of 0 in Samples
12:
Set CredLevels =
13:
Append CredLevel to CredLevels
14:
end for
Sort CredLevels
16: CALIBRATION M Credl Credlevels[(
17: AVG C AVG C.G_CALIBAATION
CALIBRATION
18: end for
Return: AVG_ CALIBRATION
K Additional Results
K.1 Corner plots
K.2
Calibration plots
Non-iid Calibration Sets
We provide additional results reflecting the behavior of RoPE when the calibration set is not sampled from the "true" prior distribution, the light tunnel task, when the calibration set comes from subset of the true distribution. We use the beta distribution of Figure 3b as the calibration set distribution. Figure 15 reports the main metrics (ACAUC and LPP), We observe that, even in this extreme case, RoPE achieves performance that outperforms the prior distribution on the LPP while still being calibrated for calibration set size that are greater than 10. Figure 16 studies how good/bad estimated posteriors are as function of whether similar samples belongs to the calibration set. expected, RoPE performs strongly for samples
28

---

Addressing Misspecification in Simulation-bass based Inference through Data-driven Calibration
Figure 8: Three corner plots for task A with a calibration set with samples.
Figure Three corner plots for task B with a calibration set with samples.
Figure 10: Three corner plots for task with a calibration set with 50 samples.
that belong to the calibration set while it struggles to generalize sample that are OOD. Finally, we also show corner plots of the learned posterior in Figure 17 for both samples that are (a) unlikely under the calibration set distribution, and (b) likely under that distribution.
29

---

Addressing Misspecification in Simulation- -aaed Inference through Data-driven Calibration
Figure 11: Three corner plots for task with a calibration set with 50 samples.
Figure 12: Three corner plots for task E with calibration with 50 samples.
Figure 13: Three corner plots for task E with distribution shift with a calibration set with 50 samples.
30

---

Addressing Misspecification in conuulation-bsed Inference through Data-driven Calibration
Endiinn Tarnel
bratce Wind TLanel
Calfeedion
JAPE
Wih
MUP
Partec selbccton
Pario L08
Goacis coviage
Eraeaed covogge
Eipecid camrage
(a)
(b)
(c)
Cullorution
1CP9
HCC+
MFC-
zilbari
zelrecoon
Dowcet
Draeceed
Expected
(d)
(c)
Figure 14: Calibration plots of the different methods on the benchmarks, the coverage at each level the average of the coverage of the marginal distributions. Each color indicates a different algorithm and the opacity is proportional to the size of the calibration set which ranges from to 1000. We observe that RoPE and OT-only are consistently well calibrated for.
0.5
2
0.3
AGAUC
0.2
LPP
-5
p (() ~ (())
( =p)
SBI
Prior
-0.1
-8
Calib. set size
Figure 15: Comparison of ACAUC and LPP for calibration set that is different from the test distribution.
31

---

Addressing Misspecification in Simulation-bal based Inference through Data-driven Calibration
Rolling Average
160
MAE (between normalized colors)
140
120
100
80
60
40
20
0
0.4
0.6
1.0
Distance from center of p
Figure 16: Accuracy of the predicted RGB values (normalized by alpha) function of the distance between the analyzed sample from the center of the calibration set distribution.
(b)
Figure 17: corner plots for three distinct observations that are very (a) that are very unlikely under the "bad" calibration set. (b) likely under the "bad" calibration set.
32


Reviews:
[{"review_id": "uOu2aRJH7M", "reviewer": "Reviewer_PVCR", "summary": "This paper presents a method for improving simulation-based inference when the simulator is misspecified. It combines neural posterior estimation with optimal transport, using a potentially small labeled calibration real data paired to corresponding parameters to correct for misspecified simulator. It then fine-tunes a representation using the calibration data and uses OT to align the distributions of representations from simulated and real data.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. We greatly appreciate the careful assessment of our work and the valuable insights provided. Below, we address the key points raised.\n## Additional References\nWe completely agree that RoPE is closely related to semi-supervised learning and Sim2Real literature. If accepted, we will use part of the additional page to elaborate on these connections.\n## Setting Limitations\nAs demonstrated in our experiments, in the absence of a calibration set, the version of RoPE without fine-tuning (OT-only) provides a viable way to handle model misspecification in some benchmarks. Nevertheless, as noted in the introduction, when posterior estimates need to be reliable, a labeled dataset is crucial for empirical validation and it appears thus natural to save a subset of this dataset for training purposes. We also recognize that settings where no labeled data is available exist. However, evaluating methods in such scenarios is highly application-dependent as it requires appropriate unsupervised metrics and we believe it may be difficult to create one generic method to handle these diverse set of problems. While we hope that RoPE’s fine-tuning step can be adapted to leverage unsupervised data and metrics specific to these applications, we acknowledge that this remains an open question. We believe future work focused on adapting RoPE to these real-world use cases could be highly impactful.\n\nRegarding scalability, as acknowledged in Section 5, we have not yet explored RoPE’s performance in medium- or high-dimensional parameter spaces, and its ability to correct for misspecification in those settings remains an open challenge. As the reviewer pointed out, model misspecification in SBI is a complex issue, and it is unlikely that any single method can address all possible cases.\n## Conditional Independence\nWe appreciate the reviewer’s careful consideration of RoPE’s assumptions. As noted in Section 3, the independence assumption is indeed a strong constraint. It limits RoPE’s ability to uncover dependencies between observations and parameters that are not explicitly captured by the simulator. If this assumption does not hold, RoPE cannot recover the true posterior, even with an arbitrarily large calibration set. However, RoPE will approximate the posterior by leveraging the simulator’s known dependencies, which may contribute to robustness and interpretability. Furthermore, please note that this conditional independence assumption does not hold for the two real-data testbeds (tasks E and F), but RoPE still returns informative posterior estimates.", "claims_and_evidence": "The main claim from the paper is that the method RoPE can provide calibrated posterior distributions under simulator misspecification, even with a small calibration set. There are a few experiments with synthetic data ,and two real-world benchmark tasks (Figure 1), showing improved log-posterior probability and Average Coverage AUC compared to baselines and methods. Ablation studies demonstrating the importance of both the fine-tuning and OT steps. This is a well supported evidence to me.\n\nThere is also a claim that RoPE is robust to prior misspecification. Experiments on extensions of tasks C and E (figure 2b, 2c and appendix C), where the true parameter distribution differs from the prior -limited to a single misspecification, showing that RoPE maintains performance. Somewhat the least convincing claim given the limited scope of the priors selections.\n\nAnother claim is that RoPE offers a controllable balance between calibration and informativeness via hyperparams. Figure 2a shows an effect of gamma hyperparam on posterior sharpness and calibration. Sounds good to me.\n\nFinally, a claim that adding a calibration set in the way specified can improve on standard SBI. This is overall well discussed and non-ambiguous to me.", "methods_and_evaluation_criteria": "The methods and evaluations are internally consistent and appropriate given the assumptions and limitations. The bigger question is whether those assumptions and limitations are too restrictive for broad real-world applicability.\n\nRoPE combines existing techniques (NPE, OT) in a novel way on a hard problem. The method is clearly described, with equations and algorithm. The core modeling assumption is explicitly stated (eqn 3), and the fine-tuning objective (eqn 6) is also well defined.\n\nThe evaluations use a mix of synthetic and real-world benchmarks, simplified lab setups, and comparisons to many baselines and competing methods. In fact there is a wealth of evaluations, always within the scope. Metrics are appropriate for evaluating posterior calibration and informativeness.\nAblation studies OK to me too.", "theoretical_claims": "The theoretical foundation, particularly its self-calibration property and the validity of its posterior factorization  relies on the conditional independence assumption eqn (3). This assumption is acknowledged by the authors, and necessary but represents a significant limitation. It basically says that the simulator completely mediates the relationship between parameters and real-world observations. Thus the theoretical guarantees and practical performance can degrade when it breaks,  and leads to wrong inferences.", "experimental_designs_or_analyses": "Synthetic data (tasks A, B, C, D): generally well-designed to test different aspects of misspecification. The use of established benchmarks (CS, SIR) are all good to me. Tasks E, F are beyond purely synthetic data but simple lab setups, not really real-world applications.\nBaselines are all well and confined within the definitions of the paper.\n\nMetrics: LPP and ACAUC are reasonable choices for evaluating posterior quality. The discussion of their limitations is helpful.\n\nI have nothing much to critique on the experiments. They are all well done within its limited scope anyway: the experimental designs are limited by the small size of the calibration set, small number of parameters, and the conditional independence assumption.", "relation_to_broader_scientific_literature": "The paper cites adequately SBI, OT and misspecification (in the Bayesian sense) literature.", "essential_references_not_discussed": "My understanding of RoPE also connects to many other areas of machine learning which I feel the paper is missing:\n* semi-supervised learning: fundamentally RoPE can be framed as a semi-supervised method, but this is not explicitly discussed.\n* domain adaptation: RoPE can also be viewed as a DA method, with the simulator as the source domain and the real world as the target domain. There are a set of domain adaptation papers which attempt to bridge the gap between theory and data. sim2real (in robotics ) is also a very relevant field.", "other_strengths_and_weaknesses": "I mentioned a few strengths and weaknesses, here are some others:\n**Strengths**:\n* one of the very few paper dealing with SBI misspecification which is one of the main SBI limitation\n* intuitive combination of NPE and OT for handling misspecification\n* well-written paper,  clear problem formulation and motivation.\n* scoped experiments and evals\n* open about limitations (this is a hard problem)\n\n**Weaknesses**: (most already mentioned)\n* limited setting: requirement of a labeled calibration set and the focus on low-dimensional parameters (also a limitation of SBI though). Similarly this is not clear how it would scale to larger and more diverse datasets, especially if calibration set does not grow simultaneously\n* dependence on conditional independence: limiting.", "other_comments_or_suggestions": "* OTT is not defined (had to look the reference).\nI feel the authors could also propose a setting where RoPE could be useful: design an experiment with a small calibration set, and one may be good to go with SBI. This is perhaps not such a strong requirements in some experimental sciences, while almost impossible in most observational sciences.", "questions_for_authors": "Not too many questions, beyond extending to other non-SBI simu-real gaps literature.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZBnPRLZA99", "reviewer": "Reviewer_vbi2", "summary": "Because it is challenging to use simulation-based inference (SBI) under model mis-specification, this paper proposes optimal transport to model the distribution between SBI-simulated data and a set of observed data and then constructing posterior distributions with neural posterior estimation (NPE). It relies on a calibration set {(theta,x_0)} linking theta and x_0 to modify pi(theta|x_s) when observing x_0. The method is demonstrated on a number of simulated and real benchmark datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for carefully reading our paper and appreciate the very constructive feedback regarding the plausibility of finding a calibration set sampled from $p^\\star$ in practical settings. We now discuss this question and other comments in detail.\n\n## Source distribution of the calibration set  \n\nWe agree with the reviewer that in some settings, assuming the calibration set consists of samples from the distribution $p^\\star(\\theta, x_o)$ may be unrealistic. Nevertheless, in important practical scenarios, it may sometimes be possible to influence the calibration set collection process to ensure it is representative of the test case scenario. In the following discussion, we denote an “imperfect” calibration set distribution as $p^c(\\theta, x_o)$.  \n\n### Case 1: $p^c(\\theta, x_o)$ and $p^*(\\theta, x_o)$ have the same support  \nIn this case, as the size of the calibration set increases, the fine-tuning step should still be able to correct the estimated sufficient statistic for all observations in the test set. If the calibration set is large, one could even perform importance sampling to ensure that the fine-tuning step optimizes the expected loss on the ideal calibration set distribution. However, for smaller calibration sets, the sensitivity of RoPE depends on whether the fine-tuned neural statistic estimator (NSE) can generalize to unseen observations. This is highly dependent on the application, particularly regarding the non-linearity of the model misspecification and the inductive bias of the NSE. If the NSE struggles to generalize, we expect the entropy-regularized optimal transport (OT) step can bring some additional robustness. Specifically, observations where the fine-tuned NPE performs well will naturally be matched to the appropriate simulated statistics, leading to accurate posterior estimates.  In contrast, for observations where the sufficient statistic does not generalize well, their representation in the embedding space may become unstructured. As a result, applying OT to these embeddings may lead to random matches with the simulated statistics, causing the posterior to revert to the prior distribution, filtered by the parameter space where the NPE is reliable.  \n\n### Case 2: $p^c(\\theta, x_o)$ and $p^*(\\theta, x_o)$ have disjoint support  \nIn this more extreme case, even an arbitrarily large calibration set may fail to provide RoPE with relevant training examples for test observations. Consequently, the fine-tuning step relies entirely on out-of-distribution generalization, which is highly problem-dependent. If the fine-tuned NPE cannot generalize at all to the test set, the fine-tuning process may be counterproductive. However, even in this situation, we expect the OT step to highlight this issue. Since real and synthetic observations will not match meaningfully, the transport matrix should approach a uniform and the posterior will revert to the prior.  \n\nWhile this second scenario is particularly challenging to study, we propose to approximate \"Case 1\" in a controlled experiment on the Light Tunnel task. Specifically, we will construct a calibration set using samples from Prior B (as in Figure 2) and evaluate performance on a test set drawn from Prior A.  \n\nIf the paper is accepted, we plan to include this experiment in the appendix and add a discussion in Section 6 to highlight RoPE’s sensitivity to non-representative calibration sets, following the considerations outlined above.  \n\n## On the failure of RoPE when the simulator is increasingly simple  \nWe agree that further investigating the trade-offs between model complexity and robustness to misspecification is an important research direction but goes beyond the scope of our paper. In many applications, there exists a trade-off:  \n- **Simplistic simulators** may ignore complex relationships between parameters and observations but have structural properties that generalize well to real-world data.  SBI models trained on these simulators may be less sensitive \n- **Highly complex simulators** may capture intricate relationships but risk overfitting to simulation-specific artifacts, failing to generalize under slight misspecifications.  \n\nFor RoPE, because it assumes conditional independence $x_o \\perp \\theta \\mid x_s$, it is better suited for handling the latter case rather than highly simplistic simulators that fail to encode meaningful parameter-observation relationships. The conditional independence assumption acts as a backstop to shortcut learning, but for the same reason, RoPE’s posterior informativeness is inherently limited by the informativeness of the posterior learned from simulations, as transparently discussed in Section 3, Paragraph 2.  \n## Minor comments & typos  \nWe appreciate the reviewer’s attention to detail in pointing out minor phrasing inconsistencies and imprecisions. We will carefully edit the manuscript to address these comments.", "claims_and_evidence": "Main Comment\n\np.2 col 1: the 3rd quantity that access is given to is \"a small calibration set of labelled real world observations C composed of iid samples from p^*(theta,x_0)\". I understand that this is required by the developed method. My question is whether it is actually likely in practice that the samples in the calibration set will really be a) samples from p^*, and b) iiid. \n\n(theta,x_0) can quite credibly (and perhaps frequently) not be drawn from the representative population p^*. Following the medical example in para 2 of the introduction, collecting (theta,x_0) can be very expensive. The decision to take these measurements could then credibly be selectively made based on estimates of quantities that are direct functions of x_0 or theta. E.g. (i) we should only measure the most severe cases, or (ii) those cases that will produce decisions/data that will like on some decision boundary (operate or not). What won't happen is that a random, iid decision (coin flip) will be made whether to make the measurement for each presenting patient (which would represent draws from p^*).\n\nMy argument here is that the (theta,x_0) calibration set won't be iid draws from p^*, but will be draws from some distribution p^** (say) that is dominated by (has a smaller domain than) p^*. In which case, if you assume the calibration set are draws from p^*, when you actually observe data from a patient who lies comfortably outside the observed domain of p^*, your modelling will go awry. \n\nSo while I like the paper as written, I think it would be healthy to:\n\n* Understand how one might practically handle this (p^** not p^*) within the context of the method; What extra information would you need, and could credibly expect to have available? Etc.\n\n* Perhaps see (in a toy example only) what would happen with posterior estimation with the current method when you in fact have samples from p^** but you assume these are from p^* (i.e. something will go wrong), and how correcting for this might correct the analysis. \n\nThis will go a long way to making this method more applicable in practice.\n\nRelated Comment\n\nMy understanding of (3) (p. 3 col 2) is that you're assuming that the sufficient statistics of x_s for model p, are the same as the sufficient statistics of x_0 for p^*. Given that the premise of the paper is that p is a mis-specified version of p^*, how likely is this assumption? I appreciate that the following paragraph after (3) discusses this to some extent. However, I'd be interested in understanding more what happens within the method if p is increasingly \"too simple\" so that (3) becomes increasingly violated. At what point does the method break down or possibly perform poorly? (Discussion response in the text is adequate.)", "methods_and_evaluation_criteria": "Ok.", "theoretical_claims": "The contribution is essentially methodological, with no new theory.", "experimental_designs_or_analyses": "Reviewed for intuitive sense.", "relation_to_broader_scientific_literature": "Good.", "essential_references_not_discussed": "Good, to the extent of my knowledge.", "other_strengths_and_weaknesses": "See above & below comments.", "other_comments_or_suggestions": "Small comments:\n* p.2 col 1. l.31 \"which reduces uncertainty compared to the prior distribution\" - perhaps nit-picking, but posteriors don't always have 'reduced uncertainty' (smaller spread) over the prior, especially in situations of prior mis-specification (for example).\n\n* p.2 col 1. l.33-34: \"the ... simulator S that ... approximates p^*\". - again, perhaps nit-picking, but simulators are forward-simulation schemes. There may be a density function associated with them, but this density function (that approximates p^*) is not the simulator itself. \n\n* p.2 col 2 text after eq (1): calling p(theta|x_s) the \"true\" posterior confused me for a while, as p^*(theta|x) is the true posterior, and p(theta|x) is just the model you're attempting to fit. Maybe adjust the text here?\n\nTrivial typos:\n\n* p. 1 col 1 l.-3 \"absence of A paired datasetS of\" clash between \"a\" and a plural.\n\n* p.2 col 1 l.25 integral in p^*(x_0) should be d\\theta not dx_0\n\n* p.2 col 1 l.38 p^*(theta,x_0) not defined (the reader has to infer this).\n\n* p.7 col 1 l.-12 \"we see of J-NPE and\" - delete \"of\"\n\n* p.7 col 2 l. 12 refers to Figure 3. The paper only appears to have 2 Figures.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "."}}, {"review_id": "Eap9WMY3B1", "reviewer": "Reviewer_x5KJ", "summary": "This paper introduces Robust Posterior Estimation (RoPE), a method for addressing model\nmisspecification in simulation-based inference (SBI). Standard SBI algorithms often\nassume a well-specified simulator, leading to biased posterior approximations when this\nassumption is violated. RoPE mitigates this problem in scenarios where a small\ncalibration set of real-world observations paired with ground-truth parameters is\navailable. The method first learns a neural embedding via a standard SBI approach\n(neural posterior estimation, NPE). It then fine-tunes this embedding using the\ncalibration set. Crucially, RoPE solves an optimal transport (OT) problem between the\nembedded simulated data and the embedded observed data from the calibration set. At\ninference time, the OT solution is used to obtain the posterior as a weighted sum of\nposteriors conditioned on simulated data. The paper evaluates RoPE on six benchmark\ntasks (two existing, four new) and reports performance compared to baselines.\n\n## Update after rebuttal\nI thank the authors for their detailed rebuttal, which successfully addressed all my questions and concerns. I agree with their proposed handling of the formal SBI baseline description (referencing the appendix). My initial positive evaluation stands, and I believe the paper makes a valuable contribution to the SBI literature.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and constructive feedback. We appreciate your insights and will use them to improve the final version of our manuscript. Below, we detail the modifications we plan to make in response to your comments.\n## Figure 1 is overly dense\nWe acknowledge that Figure 1 contains too much information and will revise it to improve readability. We will use the additional space to split the figure in two. This will allow us to make the individual plots larger and have space to add the method labels directly on the plots, making them easier to parse. We will also implement the suggested changes.\n## Clarification on NLTP and LPP\nWe confirm that NLTP and LPP are exactly the same metric. We will make this explicit in the text to avoid confusion.\n## ACAUC and joint calibration\nACAUC quantifies marginal calibration by measuring the gap between empirical and expected coverage, averaged over confidence levels. However, it does not capture joint calibration, as dependencies between parameters are not explicitly assessed. Alternative dependence-sensitive metrics may require larger test sets to be stable. Given this limitation, we will explicitly discuss the trade-offs of ACAUC in the revised manuscript.\n## SBI baseline details\nThere is a formal description of the SBI baseline in the appendix (lines 965-970). Do you think it should be moved to the main text? The SBI baseline is trained and evaluated purely on simulated data. Since RoPE assumes a conditional independence structure, the SBI baseline should provide an upper bound on RoPE’s performance in terms of LPP.\n## Performance of the MLP baseline\nThe strong performance of the MLP baseline is likely due to the unimodal nature of the benchmark posteriors, where a Gaussian is a reasonable approximation. However, this baseline only performs well when given the largest calibration budget. For smaller budgets (200 and below), RoPE consistently outperforms it. Extending benchmarking tasks for SBI under misspecification with multi-modal posterior would be an interesting direction for future work.\n## Performance variations of RoPE\nRoPE consistently returns calibrated and informative posteriors, as indicated by its LPP scores being higher than those of the prior. The results in Figure 1 are based on a fixed entropy regularization parameter, but in practice, this parameter would be tuned. With the chosen setting, RoPE can sometimes be slightly underconfident while remaining well-calibrated across all tasks. For Task E, using a smaller regularization value ($\\gamma = 0.1$ or lower) resulted in stronger performance than the one we reported in the manuscript (with $\\gamma=0.5$). We expect RoPE to match SBI closely on standard benchmarks when $gamma$ is small. Investigating further the effects of $\\gamma$, parameter dimensionality, and test set size further would be interesting but probably goes beyond the scope of this work. Finally, we emphasize that when a simulator is well-specified, standard SBI methods should be preferred over RoPE, as RoPE is specifically designed for misspecified settings.\n\n## The need for a labeled calibration set\nRoPE relies on a calibration set with ground-truth parameters, but as shown in our experiments, it can also be used without the fine-tuning step. In most benchmarks, this \"OT-only\" version already produces well-calibrated posteriors. However, the labeled test set was crucial in identifying that OT-only was insufficient for Task E. In a misspecified setting, learning a new model is necessary, and empirical data is required to validate its reliability. This justifies our assumption that a small set of real observations could be used for training. We nevertheless agree with you that labeled calibration data is not always available in real-world applications. In such cases, practitioners must define metrics that assess whether OT-only provides sufficient performance. In some applications, a fine-tuning step similar to RoPE’s could still be justified without labeled data, but this would require application-specific validation strategies. We will clarify these points in the final version of the paper.", "claims_and_evidence": "The paper's claims center around RoPE's ability to provide accurate posterior inference\nin the presence of model misspecification. These claims must be understood within a\nspecific, and somewhat less common, context within the broader field of simulation-based\ninference (SBI): the availability of a small calibration set consisting of real-world\nobservations paired with corresponding ground-truth parameter values. Many typical SBI\napplications operate without access to such paired data, even for a small calibration\nset. While the paper acknowledges this setting (citing examples like cardiovascular\nsystem inference and soft sensor development), a more thorough discussion of the\nlimitations this assumption places on RoPE's general applicability would strengthen the\nwork. The claims are therefore best interpreted as statements about performance\nconditional on the availability of this calibration data. The authors provide new\nbenchmark task, which is positive, but validation with real world applications would\nhave been a stronger support.\n\nBeyond the contextual considerations, the claims are generally supported by a\ncombination of theoretical justifications (based on an adapted definition of model\nmisspecification for SBI) and empirical evaluations across several benchmark tasks. This\nis a minor point, but the clarity of the evidence presentation, particularly in Figure\n1, somewhat hinders a complete and immediate assessment of these claims across all\ntested scenarios. The dense and complex nature of Figure 1 makes it challenging to\ndefinitively confirm RoPE's superior performance against all baselines on each specific\ntask without significant effort. This difficulty in visually parsing the results\nslightly reduces the \"convincing\" aspect of the evidence, although the underlying data\nmay well support the claims.", "methods_and_evaluation_criteria": "The introduction of four new benchmark tasks is a valuable contribution to the SBI\nliterature, providing standardized testbeds for evaluating methods under model\nmisspecification. However, the chosen evaluation metrics, LPP and ACAUC, while relevant,\nare less common in the broader SBI literature.  Using more established metrics could\nfacilitate comparisons with other SBI methods and improve the interpretability of the\nresults for a wider audience.  Specifically, the negative log probability of the true\ntest parameter (NLTP), as described in Lueckmann et al. (2021) (and used e.g.,\nPapmakarios et al. 2016, 2017, Greenberg et al. 2019), would provide a direct measure of\nposterior accuracy (and is closely related to the LPP used here, but more standard).\nFurthermore, the ACAUC metric, as implemented, assesses calibration on a per-parameter\nbasis. This approach might miss joint miscalibrations across multiple parameter\ndimensions.  Considering metrics like expected coverage (as used in, e.g., Miller et\nal., Deistler et al., Hermans et al.) could offer a more comprehensive assessment of\ncalibration, capturing potential dependencies in the posterior uncertainty. While LPP\nand ACAUC provide some insight, supplementing them with (or replacing them by) NLTP and\nexpected coverage would strengthen the evaluation.", "theoretical_claims": "I did not thoroughly check all proofs in the supplementary material, focusing instead on\nthe conceptual soundness and experimental validation. The optimal transport formulation\nappears standard.", "experimental_designs_or_analyses": "I reviewed the experimental design. The choice of baselines is generally appropriate.\nHowever, as mentioned above, Figure 1 is overloaded and makes it difficult to assess the\nresults clearly. A more detailed explanation of the \"SBI\" baseline in the main text is\nneeded, as are explanations of the MLP's strong performance (despite being based on\nGaussian posteriors), and of the performance of RoPE for different tasks.", "relation_to_broader_scientific_literature": "The paper builds upon the SBI literature and addresses model misspecification. It cites\nkey works and acknowledges the unreliability of SBI under misspecification. The use of\noptimal transport is a novel contribution in this specific context. The paper\ndifferentiates itself by focusing on the scenario with a small, labeled calibration set.", "essential_references_not_discussed": "No, the adapted definition model misspecification is stated clearly in the introduction\nand the related work on this topic is discussed in the corresponding section.", "other_strengths_and_weaknesses": "Originality: The paper presents a novel approach (RoPE) using optimal transport. The\nintroduction of new benchmark tasks also contributes to originality.\n\nSignificance: Model misspecification is a critical challenge in SBI. RoPE offers a\npotential solution, but its significance is somewhat limited by the reliance on a\ncalibration set, which may not be available in all SBI applications.\n\nClarity:  The paper is well-written and well-structured overall. However, the\npresentation could be significantly improved in terms of figures. Figure 1 is extremely\ndifficult to read and interpret. The lack of a clear concept figure (as shown in\nAppendix B) makes it harder to grasp the overall approach at the beginning. Explanations\nfor certain results (SBI baseline, MLP baseline, RoPE performance on different tasks)\nare currently insufficient.", "other_comments_or_suggestions": "No.", "questions_for_authors": "1. Figure 1 Overload: Figure 1 is extremely difficult to interpret due to the sheer\n   number of lines, markers, and overlapping information. Could you simplify this\n   figure, perhaps by splitting it into multiple figures, or by using a different\n   visualization strategy? Also, please explicitly state in the caption what constitutes\n   a \"good\" value for LPP and ACAUC (e.g., \"Higher LPP values indicate better\n   performance; ACAUC values closer to zero indicate better calibration\").\n2. \"SBI\" Baseline Explanation: The \"SBI\" baseline is not clearly explained in the main\n   text. Please provide a more detailed description of how this baseline is trained and\n   evaluated. Specifically, clarify what data is used for training and testing this\n   baseline.\n3. MLP Baseline Performance: Why does the MLP baseline perform so well, even though it\n   assumes a Gaussian posterior? Are the true posteriors in the benchmark tasks close to\n   Gaussian, or is there another explanation?\n4. RoPE performance variations: It is unclear why the performance of RoPE varies across\n   the different tasks. For example, explain in more details the performance for the SIR\n   example. It was stated that the SIR task might be less misspecified and that NPE thus\n   performs better. However, does this imply that for well-specified use-cases, RoPE\n   will perform worse in general, e.g., be overly underconfident like for the SIR\n   example? E.g., how does RoPE perform on the standard benchmarking tasks defined in\n   Lueckmann et al. 2021 that are readily available in the `sbibm` package? \n5. Calibration Set Assumption: The paper relies on the availability of a calibration set\n   with ground-truth parameter values. While this is a valid scenario, it's less common\n   in many SBI applications. Could you discuss the limitations of RoPE in settings where\n   such a calibration set is not available? Are there potential extensions or\n   modifications to RoPE that could address such scenarios?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review."}}, {"review_id": "481jbF82kw", "reviewer": "Reviewer_6GpC", "summary": "The paper details with the important issue of making neural posterior estimation methods more robust against model misspecification. The authors suggest to use a small set of labeled real-world data to calibrate the posterior inference in the face of a misspecified simulator. I am very short on time for ICML reviews. Apologies for my reviews being a bit short.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your review and your recognition of our contribution to the literature on simulation-based inference (SBI) under model misspecification.\n\nRegarding your concern about the availability of labeled real datasets, we acknowledge that such datasets are not always accessible. However, as discussed with Reviewer x5KJ, moving away from labeled calibration data in method papers introduces the challenge of defining alternative evaluation metrics. Without access to true parameters, validating the reliability of a method becomes significantly more difficult. That said, we agree that exploring approaches leveraging unlabeled real datasets is an interesting direction for future work.\n\nWe will incorporate the suggested references on increasing robustness in SBI and check whether they contain additional relevant citations on misspecification.\n\nThank you again for your thoughtful feedback and your time.", "claims_and_evidence": "The authors claim that using a small labeled real dataset can help calibrate posterior inference from a misspecified simulator. The experiments are extensive and indeed showcase this behavior.", "methods_and_evaluation_criteria": "The examples make sense indeed. As previously I continue to think that *labeled* real datasets (i.e. real datasets where the true parameters are known), is a rather niche situation. Yet I do acknowledge that there are relevant applications, where we do indeed have such a dataset.", "theoretical_claims": "No proofs in the paper.", "experimental_designs_or_analyses": "The experiments and their evaluation looks sensibel to me.", "relation_to_broader_scientific_literature": "Model misspecification is one of the major issues in SBI. The authors make an important contribution to this literature.", "essential_references_not_discussed": "The authors may want to additionally cite two recent works to increasing robustness for SBI, which both work with *unlabeled* real datasets: https://arxiv.org/abs/2501.13483, https://arxiv.org/abs/2502.04949 \n\nThey may also check if there are more relevant references about misspecification in SBI in these papers worth citing here as well.", "other_strengths_and_weaknesses": "see above", "other_comments_or_suggestions": "none", "questions_for_authors": "The authors have previously answered all my questions already during out submission processes.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents an approach for simulation-based inference when the simulator is misspecified. This is an important and timely topic, especially considering the ubiquitousness of misspecification. The approach combines neural posterior estimation with optimal transport, using a set of observation/simulation pairs for calibration. Sufficient statistics are learnt within the approach and calibration data is used to align resulting distributions from simulated and real data by OT. The paper was reviewed by a team of four who all recommended acceptance. The paper was found to be well-written, presenting a wealth of convincingly esposed and analysed test cases, and open about potential/current limitations of the proposed approach. I would like to congratulate the authors on this promising work and add a few questions below that I hope could be useful towards an improved version. 

page 2: is the dimension l (lower case L) related to k?

p_phi: a word could be said about the class 

Equation (1): the argmax may be a set... 

Perfect surrogate: potentially misleading terminology

For all x_o before (Cannon et al. 2022): why not quantify the variable just before p(x_o|theta)?  

Page 3, about “there exists S”: so, this S can potentially be a single couple in the product space, right? 

Besides this, same question about the logical quantifier arriving at the end. Written in full words at the end of the sentence (for all…) it would appear absolutely alright, but with the logical quantifier it feels awkward to me.  This is more about polishing, but I would recommend checking best practices on that. 

At the bottom of the page: better use parentheses in rho/(rho + gamma). So in case gamma=0, tau=1 would apply for any rho>0? 

Page 4, “In our setting, an ideal coupling would pair each real-world observation with the simulation generated by the same parameter”: this may sound like simulations are deterministic. Is it more in the sense that there is only one simulation per observation in the pairs used for calibration?

Just below, regarding h_o: it seems unclear at this stage where h_o is coming from. 

Results: it would be worth looking in more detail at what happens with Task B and why the approach does not adapt well there.   

References: the same paper Delaunoy et al. is cited twice with two different dates. Please check that references are in order and when preprint versions have advanced to publication, cite the resulting publication.
Decision: Accept (oral)