Robust Simulation-Based Inference under Missing Data via Neural Processes
Authors: Yogesh Verma
Ayush Bharti
Vikas Garg
Venue: ICLR 2025 Poster (2025)
Keywords: Simulation-based inference
likelihood-free inference
approximate Bayesian computation
neural posterior estimation
missing data
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We tackle the problem of performing simulation-based inference under missing data scenarios by combining imputation and inference in neural posterior estimation framework

Abstract: Simulation-based inference (SBI) methods typically require fully observed data to infer parameters of models with intractable likelihood functions. However, datasets often contain missing values due to incomplete observations, data corruptions (common in astrophysics), or instrument limitations (e.g., in high-energy physics applications). In such scenarios, missing data must be imputed before applying any SBI method. We formalize the problem of missing data in SBI and demonstrate that naive imputation methods can introduce bias in the estimation of SBI posterior. We also introduce a novel amortized method that addresses this issue by jointly learning the imputation model and the inference network within a neural posterior estimation (NPE) framework. Extensive empirical results on SBI benchmarks show that our approach provides robust inference outcomes compared to standard baselines for varying levels of missing data. Moreover, we demonstrate the merits of our imputation model on two real-world bioactivity datasets (Adrenergic and Kinase assays).  Code is available at https://github.com/Aalto-QuML/RISE.

Published as a conference paper at ICLR 2025
ROBUST SIMULATION-BASED INFERENCE
UNDER MISSING DATA VIA NEURAL PROCESSES
Yogesh Verma, Ayush Bharti
Vikas Garg
Department of Computer Science, Aalto University (yogesh. verma, ayush. bharti) @aalto. fi
Yai Ltd and Aalto University "gargecsati 1.mmt.m...... mdu
ABSTRACT
Simulation-barl based inference (SBI) methods typically require fully observed data infer parameters of models with intractable likelihood functions. However, datasets often contain missing values due to incomplete observations, data corruptions (common in astrophysics), or instrument limitations (e.g., in high-energy physics applications). In such scenarios, missing data must be imputed before applying any SBI method. We formalize the problem of missing data in SBI and demonstrate that naive imputation methods can introduce bias in the estimation of SBI posterior. We also introduce novel amortized method that addresses this issue by jointly learning the imputation model and the inference network within a neural posterior estimation (NPE) framework. Extensive empirical results SBI benchmarks show that our approach provides robust inference outcomes compared to standard baselines for varying levels of missing data. Moreover, we demonstrate the merits of our imputation model on two real-world bioactivity datasets (Adrenergic and Kinase assays). Code is available at https: / githhh..h...he.gg com Aalto- SaA-tttt-Q--MMM-E QUML/RISE.
INTRODUCTION
Mechanistic models for studying complex physical biological phenomena have become indispens- able tools in research fields as diverse as genetics (Riesselman et al., 2018), epidemiology (Kypraios al., 2017), gravitational wave astronomy (Dax et al., 2021), and radio propagation (Bharti et al., 2022a). However, fitting such models to observational data can be challenging due to the intractabil- ity of their likelihood functions, which renders standard Bayesian inference methods inapplicable. Simulation- based inference (SBI) methods (Cranmer et al., 2020) tackle this issue by relying on forward simulations from the model instead of evaluating the likelihood. These simulations are then either used to train conditional density estimator (Papamakarios and Murray, 2016; Lueckmann et al., 7b; Greenberg et al., 2019; Papamakarios et al., 2019; Radev et al., 2020), or to measure dis- tance with the observed data (Sisson, 2018; Briol al., 2019; Pesonen al., 2023), to approximately estimate the posterior distribution of the model parameters of interest.
SBI methods implicitly assume that the observed data distribution belongs to the family of distri- butions induced by the model; i.e., the model is well-specified. However, this assumption is often violated in practice where models tend be misspecified since the complex real-world phenomena under study are not accurately represented. Even if the model is well-specified, the data collection mechanism might hinder the applicability of SBI methods since it can induce missing data due to, for instance, incomplete observations (Luken et al., 2021), instrument limitations (Kasak et al., 2024), or unfavorable experimental conditions.
Although the former problem of model misspecification has been studied in a number of works (Frazier et al., 2020; Dellaporta et al., 2022; Fujisawa et al., 2021; Bharti et al., 2022b; Ward et al., 2022; Schmitt et al., 2023; Gloeckler et al., 2023; Huang et al., 2023; Gao et al., 2023; Kelly et al., 2024), the latter problem of missing data SBI has received relatively less atten- tion. A notable exception is the work of Wang al. (2024), which attempts to handle miss- ing data by augmenting and imputing constant values (e.g., zero or sample mean) and perform- ing inference with binary mask indicator. However, this approach can lead to biased esti- mates, reduced variability, and distorted relationships between variables (Graham et al., 2007). This is exemplified in Figure where investigate the impact of missing data neural pos-

---

Published as a conference paper at ICLR 2025
terior estimation (NPE, Papamakarios and Murray (2016)) popular SBI method- a pop-
ulation genetics model. We observe that simply incorporating missing values and their corre- sponding masks in NPE methods as in Wang et al. (2024) leads to biased posterior estimates. Other SBI works that address missing data include
Lueckmann et al. (2017a) and Gloeckler et al. (2024), however, they fail to account for the underlying mech- anism that leads to missing values in the data.
20
True
Outside of SBI, the problem of missing data has been extensively studied (Van Buuren and Groothuis- Oudshoorn, 2011), with Rubin (1976) categoriz- ing it into three types: missing completely ran- dom (MCAR), missing at random (MAR), and miss- ing not at random (MNAR). Recent advances in machine learning have led to the development of novel methods for addressing this problem using generative adversarial networks (GANs, Luo et al. (2018); Yoon et al. (2018); Li et al. (2019); Yoon and Sull (2020)), variational autoencoders (VAEs, Naz- abal et al. (2020); Collier et al. (2020); Mattei and Frellsen (2019); Ipsen et al. (2020); Ghalebikesabi et al. (2021b)). Gaussian processes (Casale et al., 2018; Fortuin et al., 2020; Ramchandran et al., 2021; Ong et al., 2024), and optimal transport (Muzellec et al., 2020; Zhao al., 2023; Vo al., 2024). These methods offer new perspectives on the problem of missing data imputation, but their application has been primarily limited to predicting missing values. Notably, they have not been developed for inference over missing values, which remains significant challenge for SBI.
0%
10%
15
25%
60%
10
5
10
01
Figure 1: Effect of missing data on SBI NPE posterior for the two-parameter Ricker model (Wood, 2010) where the method of Wang et al. (2024) (with zero augmentation) is used to handle 5% of values missing in the data. As increases, the NPE posteriors become biased and drift away from the true parameter value, denoted by the black lines.
Contributions. In this paper, we introduce a novel SBI method that is robust to shift in the posterior distribution due to missing data. Our method, named RISE (short for "Robust Inference under imputed SimulatEd data"). jointly performs imputation and inference by combining NPE with latent neural processes (Foong et al., 2020). Doing so allows us to learn an amortized model unlike other robust SBI methods in the literature, and to handle missing data under different assumptions (Little and Rubin, 2019). We summarize our main contributions below:
we motivate the problem of missing data in SBI, arguing how it can induce bias in posterior estimation;
we propose RISE, an amortized method, that jointly learns an imputation and inference network to deal with missing data;
RISE outperforms competing baselines in inference and imputation tasks across varying levels of missingness, demonstrating robust performance in settings entailing missing data.
2 PRELIMINARIES
Consider a simulator-basee model p(: ()) that takes in a parameter vector 0 E + C R' and maps it to a xoint = [m1,.. ...]." T in some data space X C Rdd. We assume that p(- 0) is intractable, meaning that its associated likelihood function is unavailable and cannot be evaluated point-wise. However, in our setting, generating independent and identically distributed (iid) realisations x ~ p(-10) for fixed 0 is straightforward. Given a dataset x collected via real-world experiments from some true getergenerating process and a prior distribution on the parameters p(0), we are interested in approximating the posterior distribution p(0) X) x p(x (0)(()) This can be achieved, for instance, using the popular neural posterior estimation (NPE) method, which we now introduce.
Neural posterior estimation. NPE (Papamakarios and Murray, 2016) involves training conditional density estimators, such as normalizing flows (Papamakarios et al., 2021). ). to learn a mapping from each datum the posterior distribution p(0 x). Specifically, approximate the posterior
2

---

Published as a conference paper at ICLR 2025
distribution with ( [0] xx) using learnable parameters Φ. In particular, train q6 by minimizing an empirical loss
(O) - 1 Σ
log 90 (0: (x,) = - -Eu-p(o) [Ex-p(x 10) flog qo [x]],
(I)
using the dataset {(s,, X= simulated from the joint distribution p(0, x) = p(x ())( When the data space X' is high- high-dimensional, or there are multiple observations X = [xx(1).... ,x."),...,'' mm)] for each 0, we can use summary function v) X + S (such deep set (Zaheer al., 2017)) to enable a condensed representation. Assuming that the summary function is parameterized by K, the joint
NPE loss with respect to both Φ and x can be defined as (NPE(d, x)) _ 1 Ei=1 log (0, 1h (x.)). Once both 9 and Th are trained, the NPE posterior estimate 93(0 m/ (x)) for any given real data obtained by simple forward pass of x through the trained networks, making NPEs amortized. We now provide a brief background on the missing data problem, which is the focus of this work.
Missing data background. In the context of missing data, each data sample is composed of an observed part Xobs and a missing (or unobserved) part Xmis such that missingness pattern for each x is described by a binary mask variable s E {0, 1]4, where s; = if the element o; is observed and s; O if is missing, i E {I.....d d}. The joint distribution of x and can be factorized as p(x, s) = p(s x))(x)). Based on specific assumptions about what the conditional distribution of the mask (or the missingness mechanism) depends on, three different scenarios arise (MCAR), where p(s x) = p(s): : (ii) dessing-at-randon (MAR), where p(s x) p(s Xabs); and (iii) missing-nol-at-random (MNAR), where pss| x) = p(s Xabs Xmis).
(Xobs: Xmis). The
(Little and Rubin, 2019): (i) paissinn-compllelly-----aan
The missingness mechanism can be ignored for both MCAR and MAR when learning p(Xohs: s), but not for MNAR where it depends on Xmis (Ipsen et al., 2020). We aim to handle all the three cases when performing SBI.
METHOD
We begin by analyzing the issue of missing data in SBI settings in Section 3.1. We then present RISE - our method for handling missing data in SBI. Section 3.2 outlines our learning objective, and Section 3.3 describes how we parameterize the imputation model in RISE using neural processes.
3.1 MISSING DATA PROBLEM IN SBI
We assume that the simulator can faithfully replicate the true data-generating process (i.e., the simulator is sell-specified). however, the data collection mechanism induces missing values in each data point x. As a result, contains both observed and missing values, represented as x (Xots X.is). For instance, x = (0.1 1.2 - 0.9) exemplifics a scenario where a specific coordinate z; is missing (indicated by "") Naturally, SBI methods cannot operate on missing values, and so imputing Xmis is necessary before proceeding to inference. However, if the missing values are not imputed accurately, then the corresponding SBI posterior becomes biased (e.g., as observed in Figure due to constant imputation). We now describe this problem mathematically.
Definition (SBI posterior under true imputation). Let Pirue (Xmis Xobs) be the true predictive distribution of the missing values given the observed data. Then, the SBI posterior can be written as
Psnr (0) Xabx)
pssar( (0) Xobss Xmis) Pirue (Xmis Xobss) dx
Inference
Impulation
We thus have distribution over the missing values given Xobs, and the problem of SBI under missing data is formulated as an expectation of the SBI posterior Ps Xuin: Xmis) with respect to ), analogous to traditional (likelihood-batte -baeed Bayesian inference methods (Schafer and Schenker, 2000; Zhou and Reiter, 2010). Therefore, estimating the above expectation requires
Punue (Xiis (Xmis
Note that during training, Xabs and xmis partitions of the simulated data while during inference we only observe X from the real world.
3

---

Published as a conference paper at ICLR 2025
access Puve (Xmis Xots) (Raghunathan et al., 2001; Gelman al., 1995), which is infeasible in most practical cases.
Definition (SBI posterior under estimated imputation). Let pi X Xabs) denote an estimate of the true imputation model Pirue (Xmis Xaba)- Then, the corresponding SBI posterior be wrilten as
Psar(0) Xotts)
Pssarl Xohs; Xmai )
Xmis Xobs X
If Ifp((xii Xmis Xobs) is misaligned with Proue Xmis Xoba), then the estimated SBI posterior PsBI( Xobs) will be biased (in general), i.e., Eg--pork Xat.)) ( + Eo-ppadd xxas))
Proposition 1.
The proof, which follows straightforwardly using Definition and Definition 2, is given in Appendix A.2.1 for completeness. Proposition 1 says that the bias in the SBI posterior directly comes from the discrepancy between the true imputation model Pieee ( Xabs) and the estimated one p(xmis Xobs). This applies irrespective of the inference method used, and therefore, rather unsurprisingly, the key to reducing this bias is to learn the imputation model as accurately as possible. The rest of this section presents our method, named RISE, which combines the imputation task with SBI to reduce this bias.
3.2 ROBUST SBI UNDER MISSING DATA
Let Ptrue (0 Xahs: Xmis) be the true posterior given both the observed data and the missing values, i.e., given (Xobs: Xmis). Our objective is estimate the true posterior given only Xobs- That is, seek to approximate
f Pinse Xobs, Xanis Prrue (Xmis
/
Purue (0) Xohs)
A
=
Pirue (0, Xmis Xobs d
We therefore introduce family of distributions rip Xmis to solve the following optimization problem
Xobs) parameterized by th, and propose
arg min
Ex Pinie
E
KL Purce (0, Xmis
Xobs)
(8, Nmis
Xobs
(4)
(joint imputation and inference)
Solving this problem requires access to Pune (Xmis Xots), which most real-world scenarios, we do not have. Since samples for Xmis are required during training, we need to resort to methods such as variational approximation or expectation maximization. Here, we adopt variational approach. treating Xmis as latent variables in probabilistic imputation setting. Specifically, the imputation network needs estimate these latents for the inference network map them to the output space. Both networks are tightly coupled since the distribution induced by the imputation network shapes the input of the inference network.
Mathematically, assuming access to only data samples (Xobs; 0)
Pirues we proceed to solving
arg min Exis PePuse
KLfpuree
Xobs) Tie (0 Xobs)]
Our next proposition computes variational lower bound for this objective, which we can maximize efficiently using an encoder-decoder architecture resembling variational autoencoders (VAEs).
Proposition 2 (Training objective). The objective in Equation (5) admits a variational lower bound, resulting in the following optimization problem.
G, 4 arg min dup
Exx.r. ~im Exxurpp (/ bawe) log Pip P.p (Xmis Xobss + log (
Xobs; Xmmis) (inference)
(6)
(imputation)
arg min RISE (d, sp: 0.pp
where ERISE (b. 1p) denotes the loss function for RISE.
4

---

Published as a conference paper at ICLR 2025
Therefore, can approximate the true imputation model Paue Pune (Xmis Xots) using parametric neural network P.. parameterized by its vector of weights and biases yp, and the SBI posterior given the full dataset Psni (0 Xmiss: Xohs) using the conditional density 96 in NPE.
The proof of Proposition 2 is outlined in Appendix A.2.2. Note that CRISE is a general loss which reduces to ÉNPE when there is missing data, i.e., Xubs- In case summary network 7, is required before passing the data to Ja. the joint loss function for RISE can be simply defined as
ÉRISE (Ø, sp, x) -E (x O))Paaa Nais (Xiin Naw) [log qs ( The (Xoss Xmis) )) + log (Xmis Xots)]
The expectation in Equation (6) is taken with respect to the joint distribution of the simulator and the prior (as is standard for SBI methods), and the variational imputation distribution p p(Xmis Xobs). Note that for simulations in our controlled experiments, we do not need to resort to the variational distribution p(Xais X,bx), and can instead generate samples from Paue meeeixiii (Xmis Xobs) directly by first sampling x using the simulator, and then partitioning it into Xbbs and Xmis based on the missingness assumption (i.e. creating the mask s under MCAR or MAR or MNAR assumption) such that e% portion of the data is missing. The Xmis values are then used as true labels when comparing against the output of the imputation model Py during training. This allows us to amortize over instances of real data. In Section 3.3, we discuss how RISE can be used to amortize over the proportion of missing values ε in the data.
Using latent variable representation (Kingma, 2013) for the imputation model, we factorize (Xmis Xobs), similarly to Mattei and Frellsen (2019), as
(Xmis
Xobs) = (x ž) Pa Xobs)dž,
where & = (9) are parameters of the imputation model, and = (z, s) represents both the latent variable and the masking variable s, which we can utilize to simulate various missingness environments. The conditional distribution of the latent Pa (ž Xots) may depend on both the observed and the missing data depending on the different missingness assumptions (Little and Rubin, 2019):
MCAR: ( Xobs) = Pa, (z Xobs Pp (s)
MAR: P& (Z Xots) = ps, (z) Xobs P (s| Xobs)
MNAR: Pa Xobs) = Pa, (z Xobs) I P&, (s Xmis: Xabs (miis Xobs (ooss)xxc cxX
Note that for the MCAR and MAR cases, we only need the latent z in order to impute Xmis (Mattei and Frellsen, 2019), in which case ž = z. However, in the MNAR case, ž = (z, as we will explicitly need to account for the missingness mechanism (Ipsen et al., 2020). Hereafter, we continue to denote the latent variable with for general formulation encompassing all the three cases. The pseudocode for training RISE is outlined in Algorithm 1.
3.3 LEARNING THE IMPUTATION MODEL USING NEURAL PROCESS
We utilize neural processes (NPs, Garnelo et al. (2018)) for parameterizing the imputation model p.. X. NPs represent a family of neural network-based meta-learning models that combine the flexibility of deep learning with well-calibrated uncer- tainty estimates and a tractable training objective. These models learn a distribution over predictors given their target positions or locations. We refer the interested reader to Appendix A.3 for detailed background. We employ neural processes to model the predictive density over missing values at their specific locations.
Xabs
Cobs
Let Cmis = (C Cmis,k) and Cats = (Cobs, C ) denote the locations pertaining to Xmis and Xobs- respectively, where k denotes the number of missing values (or the dimensionality of xmis). Furthermore, let C = {Xata, } be the observed context set. Then, following latent neural processes (Foong et al., 2020), we obtain
MNAR
Xmis
Cyis
P.p (Xmis Cmis, C² 0/ f paa (xmis| Cmis, 2) (& C)dž
13
k = -/ p2(2 C) II
(7) Figure 2: Plate diagram
Pa (Iiiss Cmis, 2) 2)
1=1
5

---

Published as a conference paper at ICLR 2025
Here have assumed conditional independence of each 2 given C and ž, which allows for the joint distribution to factorize into a product of its marginals. Note that this factorization directly inherits the consistency properties from neural processes, established by Gareelo et al. (2018) and Dubois et al. (2020), ensuring a consistent distribution representation. The associated plate diagram is given in Figure 2. To fully specify the model, utilize the following:
Encoder is C), which provides distribution over the latent variables à having observed the context set C. The encoder is parameterized to be permutation invariant to correctly treat C' as a set (as required by NPs).
Decoder Pa (Tmiss i 2), which provides a predictive distribution over each missing value Timis conditioned on 2 and the missing location Cmis,i- practice, this distribution assumed to be a Gaussian, and the parameters a denote the predicted mean and variance.
The likelihood given in Equation (7) is not analytically tractable. Therefore, following Foong et al. (2020), estimate D (Xiis Cmis; C*) using m Monte Carlo samples 21, 21,.... .....,,,,, ,2m P. C)
m DEI
log Pup (Xmis Cmiss
C') R log
II j=1
Pa (Fmis,á Cmis, 8z)
m
This can directly be used with standard optimizers (Kingma, 2014) to learn the model parameters.
As NPs are meta-learning models, we can utilize them to amortize over the proportion of missing values 8. Doing so beneficial in cases where inference is required on multiple datasets with varying proportions of missing values, so as to avoid re-training for each e. Assuming p(s) to be the distribution of the missingness proportion, we consider each sample from p(E) to be one task when training RISE Specifically, this can be done by first initializing the parameters of RISE, and then repeating the following: (i) Sample E p(e). and (ii) Perform Steps 2-7 from Algorithm 1. We name this variant of our
Algorithm 1 RISE (training)
Require:
p(- [0), prior p(0). itera- tions fitter. missingness degree E
Simulator
1: Initialize parameters d, yp of RISE
2: for k = 1....., Nitter do
3: Sample (x, 0) ~ p(- (K-1))p(())
4: Create mask wrt and MCAR/-
MAR/MNAR
5:
Compute ÉRISE using Equation (6)
6:
Φ. y + optimimizzi (FRISE o (p)
7: end for
method as RItE-MMta. For each sample from the imputation model, we obtain posterior distribution via the inference network, thus resulting in an ensemble of posterior distributions across all samples. In Section 5.3, we test the ability of RISE-Meta to generalize to unknown levels missing values.
4 RELATED WORK
Wang et al. (2024) attempt to handle missing data by augmenting the missing values with, e.g. zeros or sample mean, and subsequently training NPE with a binary mask indicator, but this approach can lead to biased posterior estimates, as we saw in Figure and Section 3.1. Wang et al. (2022; 2023) propose imputing missing values by sampling from kernel density estimate of the training data or using a nearest-neighbor search, and training the NPE model using augmented simulations. However, these approaches neglect the missingness mechanisms, which can distort the relationships between variables (Graham et al., 2007) and are not scalable to higher dimensions. Lueckmann al. (2017a) learn an imputation model agnostic of the missingness mechanism. More recently, Gloeckler et al. (2024) have proposed a transformer-based architecture for SBI that can potentially handle conditioning on data with missing values. This method can perform arbitrary conditioning and evaluation, i.c. for a given x Xobs Xmisl. I. it first estimates the imputation distribution, i.e. p(xmis Xabs), and then estimates the posterior distribution p(0 Xohs; Xmix). However, it does not model the mechanism underlying the missing data and is thus not equipped to handle the MNAR settings. contrast, RISE incorporates the missingness mechanism during its training and therefore able to estimate the full posterior distribution, accounting for all variables.
Missing data in SBI.
Deep imputation methods. There is a growing body of work on imputing missing data using deep generative models. These include using GANs for missing data under MCAR assumption (Yoon et al.,

---

Published as a conference paper at ICLR 2025
2018: Li et al., 2019), and VAEs under MAR assumption (Mattei and Frellsen, 2019; Nazabal et al., 2020). Deep generative models have also been studied under MNAR assumption (Ghalebikesabi et al., 2021a; Gong al., 2021; Ipsen et al., 2020; Ma and Zhang, 2021). We contribute to this line of work by using latent NPs to handle missing data under all the three missingness assumptions. Instead of learning an imputation model, Smieja al. (2018) propose replacing typical neuron's response in the first hidden layer by its expected value to process missing data in neural networks.
5 EXPERIMENTS
In this section, we assess the significance of RISE via detailed empirical investigations. Our first objective is to demonstrate that RISE yields posteriors that are robust to missing values in the data compared to baseline methods (see Section 5.1 and Section 5.2). Secondly, we aim to test the generalization capability of RISE-Meta in cases where the proportion of missing values in the data not known a priori (Section 5.3). Thirdly, as learning the imputation model accurately is central to RISE'S performance, we aim to validate that employing a NP-hased imputation model in RISE yields state-of-ther af-t results when imputing real-world datasets. Finally, intend to provide some experimental evidence where learning the inference and imputation components jointly, as is done in RISE, performs better than learning them separately.
Our experiments are organized as follows. We first provide results on SBI benchmarks in Sections 5.1, 5.2, and 5.3. In Section 5.4, we report our ablation studies to evaluate the imputation performance of RISE on real-world bioactivity datasets.
Performance metrics. We evaluate the accuracy of the posterior using the following metrics: (i) the nominal log posterior probability of true parameters (NLPP), (ii) the classifier two-sample test (C2ST) score (Lopez-Paz and Oquab, 2017), and (iii) the maximum mean discrepancy (MMD) (Gretton et al., 2012). The MMD and C2ST metries are computed between the posterior samples obtained under missing data (either using RISE or the baseline methods) and samples from a reference NPE posterior under no missing data. We use radial basis function kernel for computing the MMD, and set its lengthscale using the median heuristic (Gretton et al., 2012) on the reference posterior samples.
Baselines. We evaluate RISE'S performance against baselines derived from NPE (Greenberg et al., 2019). These include the mask-based method proposed by Wang et al. (2024), and NPE-NN that combines NPE with a feed-forward neural network for joint training and imputation (Lueckmann al., 20)7a). While NPE-NN shares RISE's joint training paradigm, it performs single imputation rather than the multiple imputation approach used in RISE. We also compare against Simformer (Gloeckler et al., 2024), recent diffusion and transformer-basel approach for posterior estimation.
Implementation. RISE is implemented in PyTorch (Paszke et al., 2019) and utilizes the same training configuration as the competing baselines (see Appendix A.4.4 for details). We take & E (10%, 25%, 60%) to test performance from low to high missingness scenarios. We adopt the masking approach as described in Mattei and Frellsen (2019) and Ipsen et al. (2020) for MCAR and MNAR respectively. Specifically, for MCAR we randomly mask e% of the data, and for MNAR we use to compute a masking probability, which is then used to mask data according to their values. This self-censoring approach described in Appendix A.4.3, and leads to a missingness proportion less than (or equal to) &. We set a simulation budget of = 1000 for all the SBI experiments, and take 1000 samples from the posterior distributions to compute the MMD, C2ST and NLPP. The performance is evaluated over 10 random runs. For further details, see Appendix A.4.
5.1 PERFORMANCE ON SBI BENCHMARKS
We evaluate the performance of RISE in settings with missing data using four common benchmark models from the SBI literature, namely, (i) Ricker model: a two parameter simulator from population genetics (Wood, 2010); (ii) equation model (Chen et al., 2021); 1); (iii) Generalized Linear Model (GLM): a 10 parameter model with Bernoulli observations; and (iv) Gaussian Linear Uniform (GLU): a -o-dimensional Gaussian model with the mean vector as the parameter and a fixed covariance. The models are described in Appendix 1, and the prior distributions used are reported in Appendix A.4.2.
process (OUP): two parameter stochastic differential
7

---

Published as a conference paper at ICLR 2025
Table 1: NLPP and C2ST metrics under MCAR and MNAR scenarios, with missing value proportion E. RISE demonstrates superior posterior estimation performance. For MNAR scenarios, the proportion of missing values averages below e due to self-censoring (details in Appendix A.4.3). Note that Simformer results are unavailable for Ricker and OUP due to the lack of official implementation.
NLPP
C2ST
Dataset
NPB-NN
Wang
Simformer
NPE-NN
Wang
Simformer
RISE
10%
- 0.11
22,0 0.10
0.12
-2.31 0.10
0.87 0.01
0.87 4: 0.01
0.85 0.01
0.83
0.01
GLU
25%
-3.92+0.111
4.14
-3.6+
+.11
0.90 0.01
0.92 0.91
0.91 0.01
0.89 + 0.01
6.12
0.22
6.21
0.98
0.97 0.001
0:96 0.01
0.93 0.01
10%
-6.57 0:13
-7.10=011
-6.47
-6.32
0.81 0.01
0.86 4: 0.01
0
0.80 0.01
MCAR
GLM
25%
4.72
-7.84 0.17
-1.37 0.13
2:117
0.93 4: 0.01
0.94 2: 0.01
0.92 à: 0.01
0.91 4: 0.01
9.022
8.97
0.18
8.71
0.98
0.97
10%
-1.00:0.16
-.31
-4.20
0.91 2: 0.01
0.93 1: 0.01
0.90 0.01
Ricker
25%
-191
-0.71
-.164
0.96 + 0.01
0.95 0.001
0.92 = 0.01
60%
-197 0.117
-5.41
-4.72
0.97 0.01
0.99 0.001
0.94 0.01
10%
-2:25 0.18
-2.77 0.188
-2.09 0.11
0.80 0:01
0.8% 0.01
0.87 0:01
OUP
25%
-2.1110.18
-2555 0.13
-2.43
0.90 0.01
0.91 0.001
0.89 : 0.01
00%6
-2.87 0.19
-2.76 0.17
-2.52
0.95 0.01
0.94 0.011
0.98 + 0.01
10%
-2.35
-2,42 0.17
0.10
-1.90 0.09
0.89 0:01
0.8% + 0.01
0.87 0:01
0.85
0:01
GLU
2536
-3:31 0.17
-3.67
0.12
-3.22
0.12
-3.26 + 0.10
0392 0:01
0.93
+ 0.01
0:91 + 0.01
0.88
0.01
00%
-5.97
-6.03
0.11
-6.02 0.122
-5.80
0.96 1 0.01
0.95 0.01
0.93 1 0.01
0.92
0.01
10%
0:227
5,96
6.17
5.82
0.90 0.01
0 0.01
0.85
MNAR
GLM
25%
-6.47 0:14
-6.61 = 0.32
6.37 0.34
6.11
0:44 + 0:01
0.95 + 0.01
0:92 + 0.01
0.89 0.01
0006
7.78 + 0:37
8.38 0.12
4.10
7.11 0.11
0.97 + 0.01
0.98 + 0.01
0.96 + 0.01
0.96 0.01
-167
0.21
0018
-.10
0.91
0.91
0.92
Ricker
25%
4.91 0.200
5.06 0.18
4.75
0.95 0.011
0.96 +: 0.01
0.93 0.01
0406
-5.25 + 0.21
-5.12
-4.82 1 0:26
0.97 + D.01
0.300 + 0.01
0,96 0:01
0.11
2.32
0.110
0.92
0.88
OUP
25%
-2112:0017
-2.57 0011
0.17
0.97 0.01
0.95 1: 0.01
0.93 0.01
60%
-2992 0.15
-2799
-2.47 0.221
0.99 0.01
0.99 4: 0.01
0.97 0.01
The results for NLPP, C2ST are shown in Table and MMD in Table 5, comparing performance across varying missingness levels e under both MCAR and MNAR conditions. We observe that RISE achieves the lowest values of C2ST across missingness types, thus outperforming the baselines in estimating the posterior distributions. As e increases, the gap between RISE and the baselines increases, indicating that RISE is able to better handle high missingness levels in the data. As sanity check, we also investigate the imputation capability of RISE Figure 5 shows that RISE achieves better imputation, which then naturally translates robust posterior estimation. The difference in performance is more stark in the MNAR case, as expected, since the baseline methods do not explicitly model the missingness mechanism.
5.2 HODGKIN-HUXLEY MODEL
We now apply RISE on a real-world computational neuroscience simulator (Hodgkin and Huxley, 1952), namely the Boodkinn-Huxlee model, which is a popular example in the SBI literature (Lueck- mann et al., 2017b; Gao et al., 2023; Gloeckler et al., 2023). The aim is infer the posterior over two parameters given the data of dimension 1200 (see Appendix 1.4.1 for the model description).
We set uniform priors and perform inference under different values of and missingness assumptions. similar to Section 5. Figure 3 shows that RISE's posteriors are robust to increasing proportions of missing values as they stay around the true parameter value as compared to NPE-NN. We also evaluate the expected coverage of the posterior in Appendix B.2, which demonstrates that RISE produces conservative posterior approximations and achieves better calibration than NPE-NN.
5.3 GENERALIZING ACROSS UNKNOWN LEVELS OF MISSINGNESS
Next, we test the generalization capability of our method to unknown levels of missing values. We perform meta-learning over different proportions of missing values in the dataset (termed RISE-Meta). For training RISE-Meta, we take the distribution of p(e) to be an equiprobable discrete distribution the set { (10%, 25% 60%). We also train NPE-NN with a missingness degree of 60% as a baseline. We evaluate all the methods over 100 samples of varying missingness proportion 8 (( 1)). Figure 4 shows the MMD results on GLM, GLU, Ricker and OUP tasks. We observe
8

---

Published as a conference paper at ICLR 2025
20
20
C- 10%
C- 25%
C- 60%
15
15
15
10
10
10
5
5
5
0
0
-5
-5
50
100
0
50
100
50
100
01
01
0
20
20
20
10%
25%
15
15
15
10
10
10
5
5
0
0
0
-5
-5
-5
0
50
100
0
50
100
50
100
01
0
01
True NPE-NN
RISE
Figure 3: Posterior estimates for the Hodgkin Huxley model under MCAR (top row) and MNAR (bottom row) with varying proportions of missing values in the data (denoted by e). The posteriors obtained from RISE stay close to the true parameter (denoted by the black lines) for all values of E, while those from the baseline methods move further away as E increases.
that RISE-Meta achieves the lowest MMD values for both the tasks, thus demonstrating its ability to better generalize to unknown levels of missing values in the data.
5.4 ABLATION STUDIES
Imputation performance on real-world datasets. We now look at how the neu-
ral process-based imputation model in RISE performs on real-world datasets. The task is to predict and impute bioactivity data on Adren- crgic receptor assays (Whitehead et al., 2019) and Ki- nase assays (Martin et al., 2017) from the field of drug discovery. The Kinase test data consists of outliers, unlike the training data, which makes imputation chal- lenging. We can therefore use such data to assess the generalization capabilities of RISE. We compare the RISE imputation method to other methods from this field such as QSAR (Cherkasov et al., 2014). 2onduit (Whitehead et al., 2019), and Collective Matrix Fac- torization (CMF) (Singh and Gordon, 2008). We also include a standard deep neural network (DNN) and a vanilla neural process as baselines. Table 2 (left) reports the coefficient of determination Ri (Wright, 1921) between the true and the predicted assays. We observe that RISE achieves state-of the-art results in these tasks, demonstrating the efficacy of the neural processes- bassd imputation model.
0.7
Ricker OUP
GLU
0.6
GLM
0.5
0.4
0.3
0.2
0.1
0.0
NPE-NN
RISE-Meta
Figure 4: Generalizing over missingness.
SSnce, the official implementation is unavailable, use the re-implemennatioon provided here: https: //gi thub. com/ #enel opw.oneseneeeeeeeeeeeeeeeeeeeeee /neural processes.

---

Published as a conference paper at ICLR 2025
Table Ablation studies. (left) R scores (t) on the bioactivity datasets. (right) RMSE (4) across different proportions of missingness (s) for comparing the effect of joint versus separate learning.
Method
Adrenergic
Kinase
Missingness (c)
Method
GLM
GLU
NPE-RF-Sop
0.69 0.03
0.44 4 0.02
QSAR
(N/A)
-0.19 0.01
10%
RISE-Sep
0.67 + 0.03
0,43
0.02
CMF
0.59 + 0.02
-0.11 0.01
RISE
0.65 0.01
0.41 = 0.01
DNN
0.60 0.05
0.11 + +0.1
NPE-RI-Sep
1.02 4 0.05
0,48 + 0.02
NP
0.61 0.03
0.17 0.04
25%
RISE-Sep
0.99 = 0.03
0.45 + 0.02
Conduilt
0.62 + 0.04
0.22 0.03
RISE
0.06 0.93
0.43 +
CNP
0.63 + 0.04
0.24 0.02
NPE-RF-Sep
1.34 = 0.10
0.64 0.02
RISE
0.67 = 0.03
0.26 + 0.03
60%
RISE-Sep
1.31 - 0.03
0.58 0.03
RISE
1.27 0.01
0.56 + 0.03
Joint vs separate learning. This experiment involves investigating the impact of training the imputation and the inference model in RISE jointly (as we proposed) versus separately (termed RISE-Sep). We also include another baseline termed NPE-RF-Sep where a random forest (RF) model is first used for imputation, followed by NPE. Table 2 (right) reports the RMSE values GLM and GLU tasks for different missingness proportion e. We observe that training the imputation and inference networks jointly yields improvement in performance
training them separately.
We report the results from additional ablation studies for runtime comparisons, flow architecture, and simulation budget Appendix C.
CONCLUSION AND LIMITATIONS
We analyzed the problem of performing SBI under missing data and showed that inaccurately imputing the missing values may lead to bias in the resulting posterior distributions. We then proposed RISE as a method that aims to reduce this bias under different notions of the underlying missingness mechanism. RISE combines the inference network of NPE with an imputation model based on neural processes (NPs) to achieve robustness to missing data whilst being amortized. Additionally, RISE can be trained in a meta- meta-learning manner over the proportion of missing values the data, thus allowing for amortization across datasets with varying levels of missingness. While RISE offers substantial advantages, there are limitations to address. RISE inherits the issues of NPE and may yield posteriors that are not well-calibrated (see, e.g., Hermans et al. (2022)). Moreover, the normality assumption in NPs may exhibit limited expressivity in practice when learning complex imputation distributions.
ACKNOWLEDGEMENTS
YV and VG acknowledge support from the Research Council of Finland for the "Human-steered next-genern eext-generation machine learning for reviving drug design" project (grant decision 342077). VG also acknowledges the support from Jane and Aatos Erkko Foundation (grant 7001703) for "Biodesign: Use artificial intelligence in enzyme design for synthetic biology". AB is supported by the Research Council of Finland grant no. 362534. The experiments were performed using resources provided by the Aalto University Science-ITT project and CSC IT Center for Science, Finland. YV thanks Priscilla Ong for highlighting relevant related works on deep imputation models and for insightful discussions on various missingness mechanisms.
REFERENCES
Ayush Bharti, Francois-Xaree Briol, and Troels Pedersen. general method for calibrating stochastic radio channel models with kernls.. IEEE Transactions on Antennas and Propagation, 70(6):3986 4001, 2022a. doi: 10. 10.1009/pp..221..3300
Ayush Bharti, Louis Filstroff, and Samuel Kaski. Approximate Bayesian computation with domain expert in the loop. In International Conference on Machine Learning, volume 162, pages 1893- 1905, 2022b.
10

---

Published as a conference paper at ICLR 2025
F-X. Briol, A. Barp, A. B. Duncan, and M. Girolami. Statistical inference for generative models with maximum mean discrepancy. 1906.09900009 2019. 1
Francesco Paolo Casale, Adrian Dalca, Luca Saglietti, Jennifer Listgarten, and Nicolo Fusi. Gaussian process prior variational autoencoders. Advances in neural information processing systems, 31, 2018. 2
Jeffrey Chan, Valerio Perrone, Jeffrey Spence, Paul Jenkins, Sara Mathieson, and Yun Song. A likelihood-free inference framework for population genetic data using exchangeable neural net- works. Advances in neural information processing systems, 31, 2018. 16
Yanzhi Chen, Dinghuai Zhang. Michael U. Gutmann, Aaron Courville, and Zhanxing Zhu. Neural approximate sufficient statistics for implicit models. International Conference on Learning Representations, 2021." 7
Artem Cherkasov, Eugene N Muratov, Denis Fourches, Alexandre Varnek, Igor I Baskin, Mark Cronin, John Dearden, Paola Gramatica, Yvonne C Martin, Roberto Todeschini, al. Qsar modeling: where have you been? where are you going to? Journal of medicinal chemistry, 57(12): 4977-5010, 2014. 9
Mark Collier, Alfredo Nazabal, and Christopher KI Williams. Vaes in the presence of missing data. arXiv preprint arXiv:2006.05530 2020. 2
Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation based inference. Proceedings of the National Academy of Sciences, 117/48):3055-30002 2020.
Maximilian Dax, Stephen R. Green, Jonathan Gair, Jakob H. Macke, Alessandra Buonanno, and Berharrd Schölkopf. Real-time gravitational wave science with neural posterior estimation. Physical Review Letters, 127(24):241103, December 2021. ISSN 1079-7114. doi: 10.1 103/
physrevlett.122 127. 127.241103.
Charita Dellaporta, Jeremias Knoblauch, Theodoros Damoulas, and Iaancois-Xavier Briol. Robust bayesian inference for simulator-based models via the mmd posterior bootstrap. In International Conference on Artificial Intelligence and Statistics, volume 151, pages 943 970, 2022.
Yann Dubois, Jonathan Gordon, and Andrew YK Foong. Neural process family. http: //yanndubs. github. io Lo/Neurrll- Pro-eesoosss-s--ee wateeess--amily../mo September 2020. 6
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows Advances
in neural information processing systems, 32, 2019. 22
Andrew Foong, Wessel Bruinsma, Jonathan Gordon, Yann Dubois, James Requeima, and Richard Turner Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems, 33:8284-8295, 2020. 2, 5, 6, 17
Vincent Fortuin, Dmitry Baranchuk, Gunnar Ratsch, and Stephan Mandt. Gp-vae: Deep probabilistic time series imputation. In International conference on artificial intelligence and statistics, pages 1651-1661. PMLR, 2020. 2
David T. Frazier, Christian P. Robert, and Judith Rousseau. Model misspecification in approximate bayesian computation: consequences and diagnostics. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 82(2):421 444, 2020. doi: 10.1111/rssb. 12356.
Masahiro Fujisawa, Takeshi Teshima, Issei Sato, and Masashi Sugiyama. 7-abc: Outlier-robust approximate bayesian computation based on robust divergence estimator. International Conference Artificial Intelligence and Statistics, volume 130, pages 1783-1791, 2021.
Richard Gao, Michael Deistler, and Jakob H Macke. Generalized bayesian inference for scientific simulators via amortized cost estimation. Advances in Neural Information Processing Systems, 36, 2023. 1,
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International conference machine learning, pages 1704-1713. PMLR, 2018. 5, 6, 17

---

Published as a conference paper at ICLR 2025
Tomas Geffner, George Papamakarios, and Andriy Mnih. Compositional score modeling for simulation- based inference. In International Conference on Machine Learning, pages 11098- 11116. PMLR, 2023. 16
Andrew Gelman, John Carlin, Hal S Ster,, and Donald B Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995. 4
Sahra Ghalebikesabi, Rob Cornish, Chris Holmes, and Luke Kelly. Deep generative missingness pattern mixture models. International conference on artificial intelligence and statisties, pages 3727-3735. PMLR, 2021a. 7
Sahra Ghalcbikesabi, Rob Cornish, Luke J. Kelly, and Chris Holmes. Deep generative pattern-set mixture models for nonignorable missingness, 2021b. 2
Manuel Gloeckler, Michael Deistler, and Jakob H Macke. Adversarial robustness of amortized bayesian inference. arXiv preprint arXiv:2305. 2023. 1,
Manuel Gloeckler, Michael Deistler, Christian Dietrich Weilbach, Frank Wood, and Jakob H. Macke. All-in-one simulation-basee inference. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors. Proceedings of the 4/ International Conference on Machine Learning. volume 235 of Proceedings of Machine Learning Research, pages 15735-15766. PMLR, 21-27 Jul 2024. 2, 6,
Yu Gong, Hossein Hajimirsadeghi, Jiawei He, Thibaut Durand, and Greg Mori. Variational selective autoencoder: Learning from partivlly-oberrved heterogeneous data. In International Conference on Artificial Intelligence and Statistics, pages 2377-2385, PMLR, 2021.
JW Graham, AE Olchowski, and TD Gilreath. Review: A gentle introduction to imputation of missing values. Prev. Sci, 8:206-213, 2007. 1,6
David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-fiie free inference. In International Conference on Machine Learning, pages 1404-2414 PMLR, 2019. 1, 7, 20
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012. 7
Joeri Hermans, Arnaud Delaunoy, François Rozet, Antoine Wehenkel, Volodimir Begy, and Gilles Louppe. A crisis in simulation-based inference? beware, your posterior approximations can be unfaithful. Transactions Machine Learning Research, 2022. ISSN 2835-8856. URL
net/ www.mriidiooooooooooooooooioroi
6Aq. 10, 20
Alan L Hodgkin and Andrew F Huxley. A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4):500, 1952.
Daolang Huang, Ayush Bharti, Amauri Souza, Luigi Acerbi, and Samuel Kaski. Learning robust statistics for simulation-based inference under model misspecification. Advances in Neural Information Processing Systems, 36, 2023. 1, 16
Niels Bruun Ipsen, BierreAAleannee Mattei, and Jes Frellsen. not-miwae: Deep generative modelling with missing not at random data. arXiv preprint arXiv:2006. /2877,, 2020. 2, 3, 5,
Milosz Kasak, Kamil Deja, Maja Karwowska, Monika Jakubowska, Lukasz Graczykowski, and Malgorzata Janik. Machine-learming-bssed particle identification with missing data. The European Physical Journal C, 84(7);691, 2024.
Ryan P. Kelly, David Nott, David Tyler Frazier, David Warne, and Christopher Drovandi. Misspecifecation-rbusst sequential neural likelihood for simulation- inference. Transac- fions on Machine Learning Research, 2024. ISSN 2835-8856.
Diederik Kingma. Auto-encoding variational bayes. arXiv preprint arXiw:1312.6111 2013.
Diederík Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980. 2014. 6
12

---

Published as a conference paper at ICLR 2025
Theodore Kypraios, Peter Neal, and Dennis Prangle. A tutorial introduction to bayesian inference for stochastic epidemic models using approximate bayesian computation. Mathematical Biosciences, 287:42-53. May 2017. ISSN 0025-5564. doi: 00.1016f.mbs.2016.07.00
Steven Cheng-Xian Li, Bo Jiang, and Benjamin Marlin, Misgan: Learning from incomplete data with generative adversarial networks. International Conference on Learning Representutions, 2019. 2,
Julia Linhart, Gabriel Victorino Cardoso, Alexandre Gramfort, Sylvain Le Corff, and Pedro LC Rodrigues. Diffusson posterior sampling for simulation- inference in tall data settings. arXiv preprint arXiv:2404.075993 2024. 16
Roderick JA Little and Donald Rubin. Statistical analysis with missing data, volume 793, John Wiley & Sons, 2019. 2, 3,5
David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. In International Conference on Learning Representations, 2017. URL /ttpss://opezeeviww net/ forum?
id=sJJKfEExx.
Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. Advances in neural information processing systems, 30, 2017a. 2, 6, 7
Jan-Matthis Lueckmann, Pedro J. Gonçalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, and Jakob H. Macke. Flexible statistical inference for mechanistic models of neural dynamics. In Advances in Neural Information Processing Systems (NIPS), page 1289-1299, 2017b. 1, 8
Jan-Matthis Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Bench- marking simulation-based inference. In International conference on artificial intelligence and statistics, pages 343-351. PMLR, 2021. 18, 21
Kieran Luken, Rabina Padhy, and X Rosalind Wang. Missing data imputation for galaxy redshift estimation. arXiv preprint arXiv:2111.1388 2021.
Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018. 2
Chao Ma and Cheng Zhang. Identifiable generative models for missing random data imputation. Advances in Neural Information Processing Systems, 34:27645-27638, 2021." 7
Eric Martin, Valery R Polyakov, Li Tian, and Rolando C Perez. Profile-qsar 2.0: kinase virtual screening accuracy comparable to four-concemtrattin ic50s for realistically novel compounds.
Journal of chemical information and modeling, 57(8):2077-2888 2017.
Plerre-Alexandre Mattei and Jes Frellsen. Miwae: Deep generative modelling and imputation of incomplete data sets. In International Conference on Machine Learning, pages 4413 4423. PMLR, 2019. 2, 5,
Boris Muzellec, Julie Josse, Claire Boyer, and Marco Cuturi. Missing data imputation using optimal transport, 2020. 2
Alfredo Nazabal, Pablo M Olmos, Zoubin Ghahramani, and Isabel Valera. Handling incomplete heterogeneous data using vaes. Pattern Recognition, 107:107501, 2020. 2,
Bernt Oksendal. Stochastic differential equations: an introduction with applications. Springer Science & Business Media, 2013. 18
Priscilla Ong, Manuel Haussmann, and Harri Lahdesmaki. Learning high-dimensional mixed models via amortized variational inference. In ICML 2024 Workshop on Structured Probabilistic Inference & Generative Modeling. 2024. 19
George Papamakarios and lain Murray. Fast E-free inference of simulation models with bayesian conditional density estimation. Advances in neural information processing systems, 29, 2016. 1,
13

---

Published as a conference paper at ICLR 2025
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. Advances neural information processing systems, 30, 2017. 20
George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood- free inference with autoregressive flows. The 22nd International Conference on Artificial Intelligence and Statistics, pages 837-848, PMLR, 2019.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende. Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilístic modeling and inference. Journal of Machine Learning Research, 22(57)) 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. 7
Henri Pesonen, Umberto Simola, Alvaro Köhn-Luque, Henri Vuollekoski, Xiaoran Lai, Arnoldo Frigessi, Samuel Kaski, David T Frazier, Worapree Mancesoonthorn, Gael M Martin, et al. Abe of the future. International Statistical Review, 91(2):243-268, 2023.
Martin Pospischil, Maria Toledo- ModriigR. Cyril Monier, Zuzanna Piwkowska, Thierry Bal, Yves Frégnac, Henry Markram, and Alain Destexhe. Minimal hodgkin-huxler type models for different classes of cortical and thalamic neurons. Biological cybernetics, 99:427-444 2008. 19
Stefan T. Radev, Ulf K. Mertens, Andreas Voss, Lynton Ardizzone, and Ullrich Köthe. Bayesflow: Learning complex stochastic models with invertible neural networks, 2020. URL https://
arxiv. org/ abs/ 2003. 06281.
Trivellore E Raghunathan, James M Lepkowski, John Van Hoewyk, Peter Solenberger, et al. A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey methodology, 27(1):85-96, 2001. 4
Siddharth Ramchandran, Gleb Tikhonov, Kalle Kujanpää, Miika Koskinen, and Harri Lähdesmäki. Longitudinal variational autoencoder. In International Conference Artificial Intelligence and Statistics, pages 3898-3906. PMLR, 2021.
Adam J. Riesselman, John B. Ingraham, and Debora S. Marks. Deep generative models of genetic variation capture the effects of mutations. Nature Methods, 15(10):816-82,, September 2018. ISSN 1548-7105. doi:
Donald B Rubin. Inference and missing data. Biometrika, 63(3):581-59, 1976. 2
Joseph L Schafer and Nathaniel Schenker. Inference with imputed conditional means. Journal of the American Statistical Association, 95(449); 144, 2000. 3
Marvin Schmitt, Paul-Christian Bürkner, Ullrich Köthe, and Stefan T Radev. Detecting model mis- specification amortized bayesian inference with neural networks. In DAGM German Conference on Pattern Recognition, pages 541-557. Springer, 2023. 1
Maksim Sinelnikov, Manuel Haussmann, and Harri Lähdesmäki. Latent variable model for high- dimensional point process with structured missingness. arXiv preprint arXhv:2402.05758. 2024.
19
Ajit Singh and Geoffrey J Gordon. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 650-658, 2008.
Scott A. Sisson. Handbook of Approximate Bayesian Computation. Chapman and Hal//CRC, 2018.
Marek Smieja, Lukasz Struski, Jacek Tabor, Bartosz Zielinski, and Przemyslaw Spurek. Processing of missing data by neural networks. Advances in neural information processing systems, 31, 2018, 7
14

---

Published as a conference paper at ICLR 2025
Alvaro Tejero-Cantero, Cantero, Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pedro J. Gonçalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulation canulation-based inference. Journal of Open Source Software, 5(52):2505, 2020. doi: 10.21 10.21105/foss.02505 18
Stef Van Buuren and Karin Groothuis-Oudshoomn mice: Multivariate imputation by chained equations in r. Journal of statistical software, 45:1-67, 2011.
Vy Vo, He Zhao, Trung Le, Edwin V Bonilla, and Dinh Phung. Optimal transport for structure learning under missing data. arXiv preprint arXiv:2402. 15255, 2024. 2
Bingjic Wang, Joel Leja, Ashley Villar, and Joshua Speagle. Monte carlo techniques for addressing large errors and missing data in simulation-basee inference. arXiv preprint arX17:2211.03747, 2022. 6
Bingjie Wang, Joel Leja, V Ashley Villar, and Joshua S Speagle. Sbi++: Flexible, ultra-fast likelihood- free inference customized for astronomical applications. The Astrophysical Journal Letters, 952 (1):L10, 2023. 6
Zijian Wang, Jan Hasenauer, and Yannik Schälte. Missing data in amortized simulation-based neural posterior estimation. PL.OS Computational Biology, 2066)e1012184, 2024. 1, 6, 7
Daniel Ward, Patrick Cannon, Mark Beaumont, Matteo Fasiolo, and Sebastian Schmon. Robust
neural posterior estimation and statistical model criticism. In Advances in Neural Information Processing Systems, 2022. 1, 16
Thomas M Whitehead, Benedict WJ Irwin, P Hunt, Matthew D Segall, and Gareth John Conduit. Imputation of assay bioactivity data using deep learning. Journal of chemical information and modeling, 59(3):1 197-1204, 2019. 9
Simon N. Wood. Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466 (7310):1002-1104 2010. doi: (0.1038//ature09339. 2,7
Sewall Wright. Correlation and causation. Journal of agricultural research, 20(7):557, 1921.
Jinsung Yoon, James Jordon, and Mihaela Schaar. Gain: Missing data imputation using generative adversarial nets. In International Conference on Machine Learning, pages 5689-5698. PMLR. 2018. 2,
Seongwook Yoon and Sanghoon Sull. Gamin: Generative adversarial multiple imputation network for highly missing data. Proceedings of the IEEF/CVF conference on computer vision and pattemn recognition, pages 1456-8464, 2020.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017. 3, 16
He Zhao, Ke Sun, Amir Dezfouli, and Edwin V Bonilla. Transformed distribution matching for missing value imputation. In International Conference on Machine Learning, pages 42159 42186. PMLR, 2023. 2
Xiang Zhou and Jerome P Reiter. A note on bayesian inference after multiple imputation. The American Statistician, 64(2):159-163, 2010.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX
Appendix A.I discusses the aspect of handling multiple observations and model misspectication in the context of RISE. In Appendix A.2, we present the proofs for Proposition and Proposition Appendix A.3 provides background on neural processes, and Appendix A.4 presents the implemen- tation details for the experiments of Section 5. Appendix contains additional metries, coverage plots and visulaizations. Appendix C reports the results from additional ablation studies.
A.1 DISCUSSION
Handling multiple observations. Although so far we have focused on the single observation case where we have one data vector x for each 0, RISE can straightforwardly be extended to the multiple observations case where we obtain x (1:m) .x.,..., Xm) for each Then, x x1:m)
(1:m) (1am)), and the objective for RISE becomes mis
arg min E (xx())m) py)epp
(1:m) Xia
~1
(5) (c) Pi Pot
m log (xx
Xabs + log go 17x (Nobs Xmnis
tom
(lmpputtation)
(inference)
Note that we can summarize the data using the network T7 (for instance, a deep set (Zaheer et al., 2017)) before passing the data into the inference network, which is standard practice when using NPE with multiple observations (Chan et al., 2018). Alternatively, one could use recent extensions based on score estimation (Geffner al., 2023; Linhart al., 2024) as well.
Handling model misspecification. We conjecture that replacing the inference network in RISE from the usual NPE to a robust variant such as the method of Ward et al. (2022) or Huang et al. (2023) would help in addressing model misspecification issues. It would be interesting avenue for future research see how train these robust NPE methods jointly with the imputation network of RISE, and how effective such an approach is. One way is to assume certain error model over the observed data x. corrupt the data to X by adding a Gaussian noise, and infer the correct 0 via the inference network. This can be formulated
argming.. 3.0
(xxx () PX Rabs (Mats 20 "Pans "Prr (Ress (Rt.. [log (Raiss &obs) + log (o Recss &mn)]
(9)
Moreover, method can also be readily extended incorporate prior mis mis-specification:
argmin 0.1
E (H Sympp (), ( (r))) Rais pprr (Rai Ross [log Pap (mmis Xaas) + log (0 Rahs: Xmis )]
(10)
A.2 PROOFS
A.2.1 PROOF FOR PROPOSITION
Proof. Using Equation (2) and Equation (3), note that
E P0-pp (0) xxa)) [0] Eg-ppsai mara)) [
0psni Xoba d0 @psni xons)d0
f 0 [PSBi (0 Xobs) PSBI (0
Xobs)] d0
/ o |/ PS (0) Xobss Xmis Prrue (Xmis
Xobs )dxiss
pS
Xobss Xmis )anns)p(Xnii Xobs) (dxiss d0
= 0 pS ( Xobes Xmis) [Prne (Xmis Xobs)
D Xmis Xobs) dxmic
Thus ensure that the bias is zero, require that şi (mii Xots) be aligned with Purue (Xiis (oohs)- 17
16

---

Published as a conference paper at ICLR 2025
A.2.2 PROOF FOR PROPOSITION
Proof. Recall our optimization problem from Equation (5):
arg min Ex "Pi Puuee Xobs)
(o Xobs))
Expanding the KI. term, note that the above is equivalent to
Pince ( Xobs
arg min Exa Bow ( xctua]
log
Xabs)
Since Puue Xobs) does not depend on v, we immediately note that the problem is equivalent to
argming Bxass~oox Pyme (9) Xxt> log (w Xabs))
arg max (xxxa.)) [log rio Xobs)]
We now obtain a lower bound for Exxas O)~pur [log ( Xaba)] Formally, we have
/ ris (0, Xmis
E(xas, ()x)p.0)rpmme Dog (0) Xats)] = Ee(xan 0ymppee log
Xoba dd
pl p(Xmiis
Xooba rr
Xenis
Xobs
(x 0)mpam log
cbcmis
(mmis
Xobs)
Tib (0, Xesis
Xooss
E Ex 0) Orrepin P (aate)
log
(XMmis
Xaba)
Tib (Xmis
Xobs) yo (f
Xobss Xanis
Exaa.. 0) Exii (mmm (aate)
log
p(Xmis
Xobs)
where we invoked the Jensen's 's inequality to swap the log and the conditional expectation. Splitting pa- rameters &e into imputation parameters ip and inference parameters é, and denoting the corresponding imputation and inference networks by P.p and ds respectively, we immediately get
Pup (Xmis
Xobe )ao
Xobs: Xmis
Exxan (xx...:)))>pm Mo-mmmm [log (0) Xobs)] 2 Exxcae. D/mppo Xxmierpppee (amn (atha)
log
p(x Xobs)
Thus, we obtain the following variational objective:
P. (Xmis
Xobs (0
Xobs: Xmis)
argmax,, 0.10 Exxxs (0 ~ Be popl
(Nata) log
Mena (Xima
p(Xmis
Xobss
argmax,,, o.p (x ~pme (Exm rup ppem Denee (Xein))
[log Pap (Xmis
Xobs) + log qob (0 Xohs Xmis] + HH(p(Xmiss Xoba))
arggaxx,..... p. B(xxm. mpme E Exmurpppxxn Nen (Na)))
log (Xenss Xobs)) + log g ( Xobss Xmnis)
imputation
inference
since the entropy term H(((Xxmis Xobi) does not depend the optimization variables o and yp.
A.3 NEURAL PROCESS
Neural Process (Garnelo et al., 2018; Foong et al., 2020) models the predictive distribution over target locations x, by, (i) constructing learnable mapping f- from the context (x.. to a latent representation
fx(ss
(11)
17

---

Published as a conference paper at ICLR 2025
and then (ii) utilizing the representation approximate the predictive distribution, given the target locations X via a learnable decoder Iu as,
ply: x,, (1, (r, xx)
(12)
where X., X, E & C Rd are the input vectors (often locations positions) and y. Yr ) C R the output vectors. In practice, the predictive distribution is often assumed to factorize as a product of Gaussians:
p(y: x), II ply Xt,ms r) m=1
II (y Muwms o damm) m0=1
(13)
where 1u,m, Owmm Iw (r. X..m). For a fixed context (Xe, using Kolmogorov's extension theorem (Oksendal, 2013), the collection of these finite dimensional distributions defines stochastic process if these are consistent under (i) permutations of any entries of (x;yy, and (ii) marginalisations of any entries of yr.
A.4 IMPLEMENTATION DETAILS
This section is arranged as follows:
Appendix A.4.1: Description of SBI benchmarking simulators
Appendix A.4.2: Prior distributions used for the SBI experiments
Appendix A.4.3: Procedure for creating the missingness mask under MCAR and MNAR
Appendix A.4.4: Details of the neural network settings.
A.4.1 MODEL DESCRIPTIONS
Ricker model simulates the temporal evolution of population size in ecological systems. In this model, the population size N, at time evolves No+1 N, exp(8) exp(N,+ E {1...., T). The parameter exp(01) represents the growth rate, while e, denote independent and identically distributed Gaussian noise terms with zero mean and variance o². The initial population size is set No 1. Observations 2; modeled Poisson random variables with rate parameter N2, such that 2; ~ Poiss(O2n (O) For our simulations, we fixed o² = 0.09 and focused on estimating the parameter vector 0 [01, fy]]T_ The prior distribution is set uniform distribution 4[22, 8) x [0, 20)). We simulated the process for T = 100 time steps to generate sufficient data for inference, and considered a simulation budget of 1000 create the dataset.
Crnstein-Uhlenbed process (OUP) is stochastic differential equation model widely used in financial mathematics and evolutionary biology. The OU process 2; is defined as,
#1+1 = Σ1 + Art. t E [I....,T} Az; (ex xz At 0.5w
(14)
(15)
where 7 25, At 0.2, 2g 10, and A0, At).
Generalized Linear Model (GLM). A 10 parameter Generalized Linear Model (GLM) with Bernoulli observations.
Gaussian Linear Uniform (GLU). A 10 dimensional Gaussian model, where data points are simulated as ~ Nix 0, Σ). The parameter 0 the mean, and the covariance Σ = 0.1T fixed, with uniform prior (0 E [d([11, 1) We refer to Lueckmann et al. (2021); Tejero-Cantero et al. (2020) for further details on these SBI tasks.
Hodgkin Huxley Model. Hodgkin Huxley Model is real-world computational neuroscience simulator. It describes the intricate dynamics of the generation and propagation of action potentials along neuronal membranes with the capture of the time course of membrane voltage by modeling the behavior of ion channels, particularly sodium and potassium, as well as leak currents. It consists of two parameters: 0, gNa. and 0g 9x. which describe the density of Na and K specifically. The
18

---

Published as a conference paper at ICLR 2025
dynamics parameterized set of differential equations,
Cm dV = 91 - VV 11mmm - V) Oyn"h(Es V) + TMP(EM - -V) + Inj + on(t)
dt dq 9x0 (V) dt (V) q (m.h. n.
Here, V represents the membrane potential, Cm the membrane capacitance, is the leak conductance, E1 is the membrane reverse potential, 01, 02 are the densities Na and K channel, 9M the density for M channel, ENa..MM denotes the reversal potential, and ony(t) is the intrinsic neural noise. The right hand side the voltage dynamics is composed of leak current, voltage-dependent Na a delayed rectifier K* slow voltage-dependent K current responsible for spike-frequency adaptation, and an injected current Inj- Channel gating variables y have dynamics fully characterized by the neuron membrane potential V, given the respective steady-state 9oo (V) and time constant Ty (V). For more details, see Pospischil et al. (2008).
A.4.2 PRIOR DISTRIBUTIONS
We utilize the following prior distributions for experiment tasks:
Ricker: Uniform distribution 44[2, x [0. 20))
OUP: Uniform prior [[( [-),
Modgkin--uxleyy, Uniform distribution 4[[10-4, 0.55 [15.0, 100.0))
GLU: Uniform distribution (([-1,1]²0)
GLM: A multivariate normal (V, (F FF -1) computed follows,
= 1,
F otherwise, 15ijj (16)
A.4.3 CREATING THE MISSINGNESS MASK
MCAR. We adopted random masking to simulate the MCAR scenario. For given missingness degree E, we randomly mask out =% of the data sample.
MNAR. We employed the self-masking or self-censoring approach as outlined by Ipsen et al. (2020). For a given data sample Rd and following Sinelnikov et al. (2024); Ong et al. (2024), the probability a particular data-point to be missing depends on its value. Specifically, we sample the mask 8; for 7th value for data sample as,
8; Bern(p:), Pi maxx(x)
(17)
where 0 < i < maxx(x) represents the maximum value in the data sample and p; is the masking probability for data-point 2; which is computed using the proportion of missing values e.
A.4.4 NETWORK PARAMETRIZATION
Summary Networks. For the Ricker and Huxley model, the summary network is composed of ID convolutional layers, whereas for the OUP, it is combination of bidirectional long short-term memory (LSTM) recurrent modules and ID convolutional layers. The dimension of the statistic space is set to four for both the models. We do not use summary networks for GLM and GLU.
Imputation Model. The parameters for the neural process- bassed imputation model used in RISE given in Table 3 and Table 4.
19

---

Published as a conference paper at ICLR 2025
Table 3: Default hyperparameters for imputation model Pe for Ricker, OUP and Huxley model.
Module
Hyperparameter
Meaning
Value
CNN blocks
Number of CNN layers
Hidden dimension
Numher of output channels of each CNN layer
64
Encoder
Kemel size
Kerne size of each convolution layer
Stride
Stride of each convolution layer
Padding
Pudding size of each convolution layer
CNN blocks
Number of CNN layers
Hidden dimension
Numher of output channels of cach CNN layer Kerel size of each convolution layer Stride of each convolution layer
32
Latent
Kerne size
Stride
Padding
Padding size of each convolution layer
CNN blocks
Number of CNN layers
[6.1]
Hidden dimension Kernel size
Number of output channels Kernel size each convolution layer
CNN layer
[32.2]
Decoder
Stride
Stride each convolution layer
Padding
Padding size of each convolution layer
Table 4: Default hyperparameters for imputation model P.p for GLM and GLU.
Module
Hyperparameter
Meaning
Value
ML.P hlocks
Number of MT. layers
[1,
Encoder
Hidden dimension
Number of outpul channels of each MLP layer [32,64]
MLP blocks
Number of ML. layers
2
Latent
Hidden dimension
Number of output channels of each MLP layer
32
MLP blocks
Number of MLP layers
[6,
Decoder
Hidden dimension
Number of output channels of each MLP layer [32,10]
Inference model. Our inference model implementations are based on publicly available code from the sbi library https:/ //g::///iii thtbb...ttu/t.///. thub. com/macked Lab/seabbbbbb We use the NPE-C model (Greenberg et al., 2019) with Masked Autoregressive Flow (MAF) (Papamakarios et al., 2017) as the backbone inference network, and adopt the default configuration with 20 hidden units and 5 transforms for MAF. Throughout our experiments, we maintained a consistent batch size of 50 and a fixed learning rate x 10-4
B ADDITIONAL RESULTS
Figure 5 shows how accurate our proposed method is in imputing the values of missing data simulated from the SBI benchmark models compared to the baselines. The performance is measured in terms of RMSE of the imputed values. Our method (denoted in red) performs the best in imputing the missing values (which eventually helps in improving the estimation the posterior distribution).
B. MMD
In Table 5, we report the MMD values for the experiment on SBI benchmark simulators presented in Section 5.1. Similar the NLPP and C2ST results of Table 1, we observe that RISE yields lowest MMD for almost all the cases, especially for Ricker and OUP where RISE beats the baselines comprehensively.
B.2 COVERAGE PLOTS
We compute the expected coverage (Hermans et al., 2022) our method on various confidence levels. Figure shows the expected coverage for the HH task and GLU various levels of missingness. We
20

---

Published as a conference paper at ICLR 2025
GLM
Ricker
2.0
Imputation RMSE
300
1.5
200
1.0
0.5
10
100
Imputation RMSE
2.0
250
1.5
200
150
1.0
100
10
0.5
0.0
1.0%
2.5%
600
100%
25%
600%
10%
25%
60%
100%
2.5%
60%
Mean Imput.
Zero Imput.
DNN
Ours
Figure 5: Imputation RMSE for MCAR (top) and MNAR (bottom) over various synthetic datasets.
Here GL refers to 10 dimension Gaussian linear model, see Lueckmann et al. (2021) for details.
Table 5: MMD under MCAR and MNAR scenarios, with missing value proportion E. RISE demonstrates superior posterior estimation performance. For MNAR scenarios, the proportion of missing values averages below и due to self-censoring (details in Appendix A.4.3). Note that Simformer results unavailable for Ricker and OUP due to the lack of official implementation.
MCAR
MNAR
NPE-NN
Wang at
Sinformer
RISE
NPE-NN
Wang et al.
Sinfomerr
RISE
GLU
25%
0.27 0.02
0.27 = 0.01
0.27 0.01
0.26 0.01
0.29 0.02
0.26 = 0.02
0.25
125=200 0.01
0.22 -
0.02
00%
0.40 0.04
0.36 0.02
0.39 +
0.33 0.03
0.02
0.31
0.32
10%
0.15 0.01
0.15 + 0.02
0.17 0.01
0.12 0.01
0.16 0.01
0.14 = 0.02
0.16
0.01
0.13+
MMD
GLM
25%
0.37
0.02
0.30 0.02
0.31 1
0.27 1. 0.03
0.18
0.02
0.25
0.03
0.17 1
60%
0.50
0.04
0.44 = 0.02
0.52 : 0.03
0.38 0.05
0.04 0.62
0.33 = 0.02
0.30
0.03
0.47 1 0.02
10%
0.45 0.01
0.38 0.02
0.31 0.01
0.01
0.32
0.02
0.27 + 0.02
Ricker
25%
0.47 0.02
0.39 0.02
0.35 0.02
0.49 0.02
0.41
0.02
0.36 1 0.03
60%
0.02 0.51
0,43 0.02
0.37 0.01
0.57 0.01
0.46 0.02
0.41 : 0.05
10%
0.35 0.02
0.31 0.02
0.29 + 0.02
0.42 0.03
0.41
0.02
0.38 + 0.03
OUP
25%
0.36 0.02
0.33 0.02
0.30 1. 0.02
0.43 0.02
0.41
0.38 1
60%
0.39 0.03
0.37 0.02
0.35 = 0.02
0.44 0.03
0.41 0.02
0.39 =: 0.02
observe that RISE able to produce conservative posterior approximations, and better calibrated than NPE-NN.
B.3 ADDITIONAL VISUALIZATIONNS
The fig. 7 offers further insight into the posterior bias illustrated in fig. specifically from the perspective of learned statistics. Our observations indicate that statistics for augmented datasets deviate from the fully observed statistic value, consequently causing shift in the corresponding NPE posterior away from the true parameter value.
C ADDITIONAL ABLATION STUDIES
Performance as function of simulation budget. We conduct study to quantify RISE's perfor- mance as a function of the simulation budget on GLU and GLM dataset. Table shows C2ST and MMD for different simulation budgets for RISE, for 10% missingness level. As the budget increases, the performance improves. We also visualize the posterior obtained for different simulation budgets for Ricker and OUP in Figure 8.
21

---

Published as a conference paper at ICLR 2025
HH (E 10%)
HH (E 25%)
HH (E 60%)
Empirical Coverage
1
0.5
0.5
0.5
RISE
NPE-NN
NPE-NN
NPE-NN
0.5
0.5 Confidence Level
0.5
1
Confidence Level
Confidence Level
GLU (E= 10%)
GLU (e 25%)
GLU (E 60%)
Empirical Coverage
1
0.5
0.5
0.5
RISE
RISE
NPE-NN
0
0.5
0.5
0.5
Confidence Level
Confidence Level
Confidence Level
Figure Expected coverage of RISE and NPE-NN for HH (top) and GLU (bottom) task over various level of missingness. The estimator becomes more conservative with increase in missingness due to the lack of information to estimate posterior and imputation distribution.
g 10%
E 25%
E 60%
X
Statistic
10
3
2
5
1
x
x
Fully observed data Wang et al
L.10
-7.5
-7.0
-6.5
1.75 Statistic
2.00
1.12
1.14
Statistic 1
Statistic
1
Figure 7: Shifting Statistics. The corresponding learned statistics for fully observed and augmented datasets described in fig. Observe that the statistics for augmented datasets shift away from the fully observed statistic value, thereby leading to shift in the corresponding NPE posterior away from the true parameter value. Note that re-train the method for each missingness level, the three statistics plots should not be compared with each other.
Runtime comparison. We perform an ablation study to compare the computational complexity of RISE to that of standard NPE. Table 7 describes the time (in seconds) per epoch to train different models on a single V100 GPU. We observe that there is a minimal increase in runtime due to the inclusion of the imputation model. The training time remains the same with respect missingness levels over a certain data dimensionality.
Flow architecture. Our final experiment involves comparing RISE's performance for different flow architectures. We utilize neural spline flow (Durkan et al., 2019) and masked autoregressive flow as competing architectures and evaluate on the GLM model under 10% missigness. Table 6 shows that both NSF and MAF yield similar results.
22

---

Published as a conference paper at ICLR 2025
OUP
30
5
20
82
0
10
True 9
True 0
0
RISE (10k)
-5
RISE (10k)
RISE (1k)
RISE (1K)
0
10
5
0
0
01
(a) Ricker
(b) OUP
Figure 8: Visualization of posterior estimated by RISE for Ricker and OUP under 1000 and 10000 simulation budget, respectively, and 25% missingness level. We observe that the posterior estimate improves with increase in the simulation budget.
Table 6: Ablation on flow architectures (left) and meta learning the missingness (right).
Ricker
OUP
Method
C2ST
RMSE
MMD
Method
RMSE
MMD
RMSE
MMD
RISE-MAF
0.80
0.65
0.12
RISE-NSF
0.80
0.67
0.11
NPE-NN RISF-Meta
1.97
0.51
0.50
1.52
0.42
0.89
0.45
Table 7: Runtime comparisons (left) and Simulation budget comparisons (right).
Budget
GLU
GLM
Method
GLM
GLU
C2ST
MMD
C2ST
MMD
NPE
0.12
0.10
RISE
0.18
0.16
1000
0.83
0.18
0.80
0.12
10000
0.78
0.15
0.75
0.10
23


Reviews:
[{"review_id": "m1JOBZdRaI", "reviewer": "Reviewer_La5K", "summary": "The paper introduces a method for addressing the issue of missing data in the context of\nsimulation-based inference (SBI). The approach uses neural processes to learn an\nimputation model from simulated data and then combines the learned imputation model\nwith standard neural posterior estimation (NPE) to perform SBI. The authors first show\nhow previous approaches for addressing missing data with zero- or mean-value imputation\ncan lead to biased SBI posteriors. They then evaluate their approach on a set of\ntractable SBI benchmarking tasks and two intractable tasks. Additionally, they perform\nablation studies and show how their method can be extended to generalize over different\nlevels of missingness in the data.", "questions": "What are the simulation budgets used for the different benchmarks and methods?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and clearly structured. The figures are visually appealing,\nand the results are reported over a reasonable number of repetitions and with error\nbars. The introduction and background section are concise and easy to follow, except\nmaybe for section 3.3, for which additional background is given in the appendix.\n\n### Originality\n\nThe paper addresses a known problem that has been tackled in the field of SBI before.\nHowever, it shows that previous approaches had limited success. It then proposes a new\nmethod that combines techniques from existing research on learning imputation. Thus,\noverall, using neural processes for learning imputation models for neural SBI is not a\nstrong technical contribution in itself, but can still be a valuable contribution to the\nfield of SBI. However, while most of the related work on missing data in SBI is cited\naccurately, it seems that the work by Gloeckler et al. is actually very similar to the\napproach proposed here. See below for details.\n\n### Quality\n\nThe derivations for combining imputation models with NPE appear technically sound. The\nexperimental results support the initial claims that the proposed method is more robust\nto missing data than the baselines. However, the choice of performance metrics and the\ncomparison to previous methods should be improved (see below).\n\n### Clarity\n\nAs mentioned above, I believe the paper is presented quite clearly. What I am missing is\na discussion of the potential limitations of the presented method, see below.\n\n### Significance\n\nThe paper addresses an important problem in the field of SBI, as actual SBI applications\nusually deal with real-world observations that often have missing data. This problem has\ngained only little attention in the literature so far. If the concerns on the evaluation\nlisted below are addressed and the method turns out to perform better than previous\napproaches, it will be a valuable addition to the field.", "weaknesses": "### Missing discussion of previous work\n\n- There is one early SBI paper that addressed imputation in SBI that is missing and\n  should be discussed: Lueckmann et al. 2017 automatically learn imputation values for\n  NPE using an MDN embedding network, actually on the same Hodgkin-Huxley benchmark as\n  used here. A discussion of this paper and a comparison to their proposed method would be\n  appropriate. In particular, in section 3.3 and figure 4, they use an imputation model in the last layer of the MDN, to which you could \n  compare. A comparison to this approach could be straightforward as your NPE-NN baseline approach seems quite similar to their \n  approach. Ideally, you can show how your approach leads to more accurate imputation values on the benchmarking or the HH task, as \n  they actually observed that their learned imputation values tend to be close to the sample mean of the feature. \n\n- The discussion of the work by Gloeckler et al. is not accurate. The Simformer actually\n  learns the imputation of the missing data as well. In that sense, it is actually very\n  similar to the approach proposed here. Relating to the example given in the paper:\n\n  > However, this method estimates the partial posterior distribution p(θ | x1, x3)\n  > given x = [x1, x2, x3, x4, x5], where x1 & x3 are the only observed variables,\n  > without considering the mechanisms that lead to missing data.\n\n  This is not correct. The SIMFORMER can perform *arbitrary* conditioning and\n  evaluation. Thus, when given only x1 & x3, it can actually predict p(θ, x2, x4, x5 |\n  x1, x3), which then serves as an imputation model for the missing data points (e.g.,\n  just sampling from p(θ, x2, x4, x5 | x1, x3) and ignoring x2, x4, x5 would amount to\n  obtain an approximation to p(θ | x2, x4, x5, x1, x3)).  A more detailed and more\n  accurate discussion of this work would be essential here. The code for applying the\n  Simformer is available at https://github.com/mackelab/simformer and appears to be\n  well-documented. Ideally, it would possible to show on one of the benchmarking tasks how RISE leads to better imputation by being able to explicitly model different types of missingness, which seems to be the distinctive feature compared to the Simformer. \n\n### Choice of performance metrics\n\nThe choice of MMD and RMSE is not ideal. MMD can be misleading depending on the choice\nof kernel bandwidth (Lueckmann et al. 2021) and RMSE seems to measure accurate parameter\ndiscovery, although posteriors do not have to be centered on the true parameters at all.\nI suggest the following\n- In addition to MMD, calculate **C2ST** as well as it gives an absolut and interpretable and not just a relative comparison to reference posteriors. \n- instead of RMSE, calculate the the **nominal log posterior probability**, i.e., obtaining the NPE posterior for each x in the test set,\nand then averaging in the log probs of the corresponding thetas (as done in Papamakarios\net al. 2019, Greenberg et al. 2019, and discussed in detail in Lueckmann et al. 2021). The log nominal density, when averaged over many test data points, is a relative comparison metric of posterior accuracy. This would be more appropriate than using RMSE because it measures posterior accuracy and not just parameter discovery accuracy. \n\nAdditionally, it would be good to also evaluate the calibration properties of the\ninferred posteriors, e.g., by calculating the SBC or expected coverage, at least on the\nfour benchmarking tasks. This will show how well-calibrated the different methods are.\n\n### Discussion of data and computational requirements\n\nIt would be essential to give more details on how the training data set for RISE is\nconstructed. Given the large number of additional NN parameters required for training\nthe imputation model, I would expect that RISE needs more simulations for training\ncompared to naive imputation or NPE-NN. Concretely, how many simulations where used for training RISE, or more generally, what are the simulation budgets used for the different benchmark tasks and methods?\n\nSame for the computational resources. How much more effort in terms of data and\nresources does the user have to put in in order to use RISE? It would be good for the reader to see comparisons of training time and memory usage of the different methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NrMGwOpxyR", "reviewer": "Reviewer_jx4r", "summary": "The paper emphasizes the problem of missing data in simulation-based inference (SBI) and proposes to learn a latent variable model of the predictive imputation distribution . This problem has a long-standing tradition in statistics and probabilistic modeling and extends way beyond SBI. However, it has received relatively little attention in the SBI literature, one reason being that simple data augmentation approaches are sufficient in most cases. The main value of the paper lies in the generality of the proposed method, which is clearly justified and well motivated.", "questions": "- What are the benefits of learning a latent variable model for the predictive missing data distribution instead of directly parameterizing it using another flexible generative network (e.g., a diffusion model)?\n\n- What is M in equation 7?\n\n- What are the error bars computed over in Figures 3 and 5?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is clearly written and easy to follow; it can be appreciated by both readers familiar with SBI and newcomers to the field. It addresses an important practical problem and is of interest to the community.\n\n- The joint training of a posterior surrogate and latent variable missing data model is original, quite general, and useful for SBI applications where the simulator is cheap to run.\n\n- The evaluation features a variety of different models (even though the selected four SBI benchmarks are rather uninteresting and not specifically designed to assess the impact of missingness).", "weaknesses": "*Major points*\n\n- The first contribution, namely, the “formalization” of the problem in SBI, is a bit of a stretch, as it is simply the standard missing data setting (i.e., equations 2–3 are not specific to approximate posteriors obtained via SBI). Propositions 1 and 2 are straightforward, and there is some notational confusion which may lead one to think that $x_{obs}$ is the real (non-simulated) data, while it is just a partition of the simulated data. Why not simply write the expectation as running over the augmented joint model $p(x_{miss}, x_{obs},\\theta)$ in Proposition 2?\n\n- The method is somewhat of an overkill for simple data, such as the 1D Ricker model used to pitch it, where the bias can easily be eliminated via data augmentation and a missingness mask provided as an additional input to the network. The demonstration in Figure 1 is simply bad practice and an example of model misspecification, as the networks have never seen imputed values during training. The work by Wang et al. (2024) is also somewhat misrepresented in the current paper, as they do not simply impute the data with constant values but use data augmentation and a mask indicator, which can be a much more efficient, albeit less sophisticated, approach for simpler cases (e.g., MCAR / MAR) [see also point on potential difficulties in learning the imputation model). \n\n- The paper lacks key ablation studies demonstrating the impact of a bad $p(x_{miss}∣x_{obs})$ model. I assume such a model will lead to unreasonable variance inflation of the posterior and hence miscalibration (see next point). The authors themselves acknowledge that “learning the imputation model correctly is central to RISE’s performance,” so it seems paramount to quantify the impact of approximation error in $p(x_{miss}∣x_{obs})$.\n\n- It is important to add calibration error as an additional, practically relevant coverage metric to all experiments, besides simply looking at MMD and Bayes RMSE. I suspect—and am open to being proven wrong—that difficulties in learning $p(x_{miss}∣x_{obs})$ will result in rather poor calibration, which can easily go undetected by RMSE or MMD (values are heavily kernel-dependent).\n\n*Minor points*\n\n- The notation needs some polishing, e.g., bold font is used for data vectors, but parameter vectors are not bold. On a related note, I believe it would be informative to introduce precise notation for sequences of vectors vs. vectors and focus the discussion on sequences, since, e.g., missing points in set-based data can be trivially handled by summarizing the reduced set, whereas missing points in temporal or spatiotemporal data present a real challenge.\n\n- Some citations have inaccuracies, e.g., Gloeckler et al. is not an arxiv paper but an ICML paper, Radev et al. (2022) should be Radev et al. (2020), and so on.\n\n- It would be nice for **Algorithm 1** to illustrate that the method results in an ensemble of posteriors (i.e., one for each sample from the imputation model) and that this ensemble uncertainty is integrated out for inference.\n\n- I could not find any details on simulation budgets for the experiments. It would be nice to quantify performance as a function of simulation budget in at least one of the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "50rfmwUrdT", "reviewer": "Reviewer_riJc", "summary": "This paper addresses the problem of missing data in simulation-based inference (SBI). The authors formalize how missing data can introduce bias into SBI posterior estimates when naive imputation methods are used. They propose RISE (Robust Inference under imputed SimulatEd data) to tackle this issue. This novel method jointly learns an imputation model and an inference network within a neural posterior estimation (NPE) framework. The imputation model is based on neural processes, allowing RISE to handle different missingness mechanisms (MCAR, MAR, MNAR). The method is amortized and can be generalized across varying levels of missingness. Extensive experiments on SBI benchmarks demonstrate that RISE outperforms baseline methods in inference and imputation tasks.", "questions": "- **Confusion in the contributions**: As mentioned above, in line 84, the most important contribution of the paper is that the proposed method is 'robust' to shift in the posterior distributions, but at the same time, the authors mention that the credibility of the posterior estimate after imputation remains an area for exploration. This seems contradictory. Can the authors clarify this point?\n\n- **Discussion of possible alternatives**: Why not use the EM algorithm?\n\n- **Extension to multiple observations**: I didn't quite understand if RISE can be extended to handle multiple observed data points per parameter setting. If so, what modifications would be necessary?\n\n- **Small remark on the structure of Section 5**: I initially missed the referred datasets. Is it because you once showed the SBI benchmarks and the other datasets (Adrenergic and Kinase)? Section 5 was structured confusingly; I recommend restructuring it or adding a small paragraph before *performance metrics*  to more clearly explain the section structure for the datasets.\n\n- **Sensitivity to neural process architecture**: How sensitive is the performance of RISE to the choice of neural process architecture and hyperparameters? Have the authors conducted ablation studies on this aspect?\n\n- **Computational efficiency**: Can the authors provide insights into the computational complexity of RISE compared to baseline methods? How does training time scale with data dimensionality and missingness levels?\n\n- **Handling model misspecification**: While the paper assumes a well-specified simulator, how would RISE perform under model misspecification? Can the method be adapted to account for this?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- **Contribution**: The paper tackles a significant and underexplored problem in SBI—handling missing data—and provides a novel solution by jointly learning imputation and inference models.\n\n- **Theoretical foundation**: The authors provide a formal analysis showing how naive imputation leads to biased posterior estimates, strengthening the motivation for their method. The cases for MNAR, MAR, and MCAR have been nicely formalized. The simplification of $\\mathcal{L}\\_{RISE}$ by $\\mathcal{L}\\_{NPE}$ in Proposition 2 is particularly convenient.\n\n- **Amortization**: The method can generalize across varying levels of missing data (RISE-Meta), making it practical for real-world applications.", "weaknesses": "- **Clarity of presentation**: Some parts, particularly the mathematical formulations and explanations of the method, could be clearer. For instance, in Section 1 (Introduction), the authors refer to Figure 1, which has the axes $\\theta_{1}$ and $\\theta_{2}$ but only in Section 2 (Preliminaries) define what $\\theta$ is.\n\n- **Literature for handling missing values is limited**: While the literature review mentions prominent methods for handling missing values from the deep learning literature, such as GAIN [1], more traditional and frequently used techniques have been excluded. For example, the authors could have compared their methods or, at the very least, acknowledged imputation techniques such as expectation-maximization (EM) found even in the deep learning literature [2] and other traditional approaches such as MICE [3].\n\n- **Computational efficiency**: The paper does not provide a detailed analysis of the computational cost of RISE. This is especially true because the paper uses normalizing flows, which can be computationally expensive. Given the added complexity of jointly learning the imputation model and the inference network, it would be useful to understand the trade-offs.\n\n- **Some important limitations, such as the Gaussian assumption, have not been highlighted**: Even though the authors mention in line 255 that the Gaussian assumption does not limit the expressivity, this is still a limitation that has to be clearly highlighted. The argument for infinite mixtures may be valid theoretically but computationally infeasible. This is especially true as the authors mention in line 476 that the credibility of the posterior estimates needs to be further examined, which is directly impacted by the normality assumption.\n\n- **Source code for reproduction**: It would have been good to have the source code available for a more careful examination and reproduction of the results.\n\n### References:\n\n- [1] https://proceedings.mlr.press/v80/yoon18a\n- [2] https://proceedings.neurips.cc/paper/2018/hash/411ae1bf081d1674ca6091f8c59a266f-Abstract.html\n- [3] https://www.jstatsoft.org/article/view/v045i03", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tT6AIhZR25", "reviewer": "Reviewer_3tVu", "summary": "The paper explores the challenge of SBI under conditions of missing data and introduces a novel method named RISE. By integrating Neural Posterior Estimation (NPE) with neural processes, RISE effectively tackles the problem of missing data. Experimental results show that RISE offers a more robust posterior estimation than other imputation methods across varying levels of missingness.", "questions": "1. The RISE loss function (Eq. (5)) includes two parts: $\\hat{p_\\varphi} $ to generate $x_{mis}$ given $x_{obs}$, and $q_\\phi$ to generate $\\theta$ given $x_{obs}$ and $x_{mis}$. Can these two parts be trained separately? Alternatively, is it possible to train a flow directly from $x_{obs}$ or $(x_{obs}, c_{obs})$ to $\\theta$?\n\n2. (For Weaknesses 2): In Line 289, the authors mention that the method proposed by Gloeckler et al. (2024) is unequipped to handle MAR and MNAR cases. Why not compare this method with RISE under MCAR cases?\n\n3. (For Weaknesses 2): Could the authors include additional illustrations to show how the summary statistics produced by their data imputation method deviate from the true statistics, similar to what is shown in Figure 1.(c)? I believe this would help clarify why the RISE method reduces posterior bias.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This work presents the RISE method, which innovatively combines data imputation and neural posterior estimation to address the issue of missing data in SBI. This combination is novel within the SBI domain. The authors have validated the effectiveness of the method using multiple benchmark models and real-world datasets under various experimental settings. The paper is well-organized and the figures are easy for readers to understand.", "weaknesses": "1. Reporting of Benchmark Metrics:\n\nThe authors report only MMD and RMSE as performance metrics. However, additional metrics commonly used in SBI literature or benchmarks [1] would provide a more comprehensive evaluation. For example, metrics like the negative log probability of true parameters (to assess the density of true parameters $\\theta$ within the approximate posterior), median distances (to evaluate the distance between samples generated from $\\theta$ under the approximate posterior and observations), and Classifier 2-Sample Tests (C2ST, to measure the closeness between approximate and true posteriors) are not included in this paper. C2ST is sensitive to subtle differences between distributions in high-dimensional spaces, potentially revealing nuances that MMD may miss due to kernel dependence [2]. \n\nTo enhance the robustness and comprehensiveness of the evaluation, I recommend the authors include C2ST as an additional metric.\n\n2. Comparisons to NPE-Zero and NPE-Mean:\n\nThe RISE method is compared with NPE-Zero and NPE-Mean methods, which fill missing inputs using zeros or sample means (Line 48, 323). These two imputation strategies are overly simplistic and introduce significant bias (as seen in Figure 1.(c)), resulting in biased posterior distributions. For a more rigorous comparison, it may be helpful to add a baseline such as Gloeckler et al. (2024) under the MCAR cases (Line 289).\n\n3. Choice of Conditional Density Estimator:\n\nFor the conditional density estimator, the authors do not use the Neural Spline Flows (NSFs) structure [3] employed in the SBI library. The NSFs is more performant and flexible than MAFs and has been widely used for SBI tasks in recent works [1, 4, 5, 6]. It would be helpful if the authors provided a rationale for selecting MAFs over NSFs or included an ablation study comparing RISE performance using MAFs and NSFs.\n\n---\n[1]. Lueckmann, et al. \"Benchmarking simulation-based inference.\" 2021.\n\n[2]. Bischoff, et al. \"A practical guide to statistical distances for evaluating generative models in science.\" 2024.\n\n[3]. Durkan, et al. \"Neural spline flows.\" 2019.\n\n[4]. Deistler, et al. \"Truncated proposals for scalable and hassle-free simulation-based inference.\" 2022.\n\n[5]. Kelly, et al. \"Misspecification-robust sequential neural likelihood.\" 2023.\n\n[6]. Ward, et al. \"Robust neural posterior estimation and statistical model criticism.\" 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0C8XGHpRRE", "reviewer": "Reviewer_XWnt", "summary": "The paper proposes an approach to deal with missing data in the simulation-based inference (SBI) setting. In SBI, parameters are to be inferred for a model with an intractable likelihood function. The authors propose an imputation approach utilising latent neural processes, in which parameters of the neural process and the posterior estimator are jointly optimised. The resulting method (RISE) is compared against alternative approaches on a number of problems showing good performance in terms of MMD and RSME.", "questions": "- Have you considered running simulation-based calibration to check posteriors after imputation? This would allow going beyond the qualitative analysis on the HH example.\n- How does NPE-NN perform on the HH example?\n- What are meta-learning results for the remaining statistical problems (Ricker, OUP)?\n- How does RISE compare against the method by Gloeckner et al. for MCAR cases?\n- Are posterior distributions for all statistical examples unimodal?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses are timely, relevant topic.\n- Motivation and approach are clearly laid out.\n- The evaluation spans statistical problems and real world data sets.\n- RISE outperforms baselines on problems considered.", "weaknesses": "- Some comparisons are missing, see questions for details.\n- HH example would benefit from quantitative evaluation.\n- Previous literature could be cited more accurately, some examples:\n  - L46: See also Lueckmann et al. (2017) where NN-based imputation for SBI was used.\n  - L50: Citing Radev et al. (2022) for NPE seems out of place, consider crediting NPE to Papamakarios and Murray (2016) and subsequent work based on it.\n  - L206: The VAE paper was published in 2013, not 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mMsaaMqkvf", "reviewer": "Reviewer_NU6H", "summary": "The authors provide a method that aims to simultaneously learn an imputation model, alongside a posterior approximation, under an assumed missing data distribution. They fit both models by minimizing the forward KL divergence using samples generated from the assumed models joint distribution (including both missingness and the simulator component). A neural process model is used for imputation, and a masked autoregressive flow for the inference component.", "questions": "Does NPE-Mean and NPE-Zero refer to imputing only after training the NPE model? If so, I think it would be better to compare to methods  listed in Wang et al. (2024), which ensure the inference model learns to approximate the posterior using missing data during training. Otherwise, we are simply relying on neural networks to generalize to possibly out of distribution points, which is in my opinion not a fair comparison.\n\nWhy might the \"joint\" training procedure be useful? Have I missinterpreted the fact the objective could be partitioned into two independent objectives?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper addresses handling missing data simulation-based inference , which is common problem in fields like astrophysics. The experiments include reasonable examples and the results seem to be good. The method is well motivated by including a probabilistic model over the missingness. Overall the paper is clear and well structured.", "weaknesses": "There were a few substantial issues which I feel would need to be addressed. \n\n- From the onset, the paper should make it clear that we are interested in either the case where we have multiple observations, or expect multiple observations, so wish to maintain an amortized estimator. Otherwise, we can simply remove the missing indices from the simulated output, training the posterior estimate only on the observed sites. The examples unfortunately also focus on the single observation setting. \n\n- The review of previous work is not sufficiently thorough. For example, the authors state \"the latter problem of missing data in SBI has received little to no attention. The only exception is the work of Wang et al. (2024)\". A quick search pointed me some other examples https://arxiv.org/abs/2211.03747, https://iopscience.iop.org/article/10.3847/2041-8213/ace361/meta. The summary of Wang et al. (2024) is further very poor, and perhaps should at least mention the use of training the NPE model using augmented simulations with artificially missing values, alongside a missing indicator variable. It is not clearly justified why this approach is a bad idea, especially for flexible neural network conditioning models used in normalizing flows.\n\n- In my opinion, the notation with $x_{obs}$ is likely to confuse some readers, e.g. as it is used to refer to the subset of simulated data which is not missing in the observed data, not the observed data itself.\n\n- The title itself I would argue is too broad, closer to describing an area of research, rather than being informative to the presented method.\n\n- At least from the objective described in equation 5, I can see no reason to think joint training provides any benefits. The expectation is taken over a fixed distribution, and each model (inference/imputation model) has disjoint parameter sets. From my understanding you could separate the objective into two independent objectives, and expect very similar results, if not identical results, depending on the optimizer.\n\n- No code has been provided to the reviewers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)