Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models
Authors: Julius Vetter
Manuel Gloeckler
Daniel Gedon
Jakob H. Macke
Venue: NeurIPS 2025 poster ()
Keywords: simulation-based inference
foundation models
Bayesian inference
density estimation
Primary area: probabilistic_methods
TL;DR: We present NPE-PFN, a method that uses TabPFN for training-free and simulation-efficient Bayesian inference.

Abstract: Simulation-based inference (SBI) offers a flexible and general approach to performing Bayesian inference: In SBI, a neural network is trained on synthetic data simulated from a model and used to rapidly infer posterior distributions for observed data. 
A key goal for SBI is to achieve accurate inference with as few simulations as possible, especially for expensive simulators. 
In this work, we address this challenge by repurposing recent probabilistic foundation models for tabular data: We show how tabular foundation models---specifically TabPFN---can be used as pre-trained autoregressive conditional density estimators for SBI. 
We propose Neural Posterior Estimation with Prior-data Fitted Networks (NPE-PFN) and show that it is competitive with current SBI approaches in terms of accuracy for both benchmark tasks and two complex scientific inverse problems. Crucially, it often substantially outperforms them in terms of simulation efficiency, sometimes requiring orders of magnitude fewer simulations. NPE-PFN eliminates the need for selecting and training an inference network and tuning its hyperparameters. We also show that it exhibits superior robustness to model misspecification and can be scaled to simulation budgets that exceed the context size limit of TabPFN.
NPE-PFN provides a new direction for SBI, where training-free, general-purpose inference models offer efficient, easy-to-use, and flexible solutions for a wide range of stochastic inverse problems.

Effortless, Simulation-Efficiet Bayesian Inference using Tabular Foundation Models
Julius Vetter Manuel Gloeckler Daniel Gedon 1.21 Jakob H. Macke
'Machine Learning in Science, University of Tübingen, Tübingen, Germany Tubbineen AI Center, Tübingen, Germany Department Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany
Abstract
Simulation-based inference (SBI) offers flexible and general approach to per- forming Bayesian inference: In SBI, a neural network is trained on synthetic data simulated from model and used to rapidly infer posterior distributions for observed data. A key goal for SBI is to achieve accurate inference with as few simulations possible, especially for expensive simulators. In this work, we address this challenge by repurposing recent probabilistic foundation models for tabular data: We show how tabular foundation models spacificiill TabPFN-can be used as pre-trained autoregressive conditional density estimators for SBI. We propose Neural Posterior Estimation with Prior-data Fitted Networks (NPE-PFN) and show that it is competitive with current SBI approaches in terms of accuracy for both benchmark tasks and two complex scientific inverse problems. Crucially, it often substantially outperforms them in terms of simulation efficiency, sometimes requiring orders of magnitude fewer simulations. NPE-PFN eliminates the need for selecting and training an inference network and tuning its hyperparameters. We also show that it exhibits superior robustness model misspecificction and can be scaled to simulation budgets that exceed the context size limit of TabPFN. NPE-PFN provides new direction for SBI, where training freee general- inference models offer efficient, casy-to-use, and flexible solutions for wide range stochastic inverse problems.
Introduction
Simulation has long been a cornerstone scientific inquiry [12 and becoming increasingly relevant as researchers tackle ever more complex scientific questions and systems [3]. Simulators often depend on parameters that are challenging or impossible to measure experimentally. Bayesian
inference provides a general framework for identifying such parameters by estimating posterior distributions over parameters. However, classical methods such as Markov chain Monte Carlo (MCMC) require evaluations of the associated model likelihoods, which can be computationally demanding or prohibitive for complex numerical simulators.
The field of simulation-based inference (SBI), or likelihood-Fii free inference, aims to address this challenge by enabling Bayesian inference without requiring access likelihoods. The basic idea
'quual contribution.
'Joint supervision.
(firstname laastaameeymii-ttuebiiggg...
Code available at http://thhhhbccce////cccallaa/pp pfn.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

of SBI is to train a neural network on synthetic data simulated from the model such that it can approximate the posterior distributions. In Neural Posterior Estimation (NPE. 5 a conditional
density neural network- -ofeeo, a normalizing flow 66 7 diffusion model & IO learns the
conditional distribution of parameters given simulated data p(0) æ:). When evaluated on empirical observations To the network directly returns the posterior distribution p(0 | .:)) Similarly, other SBI methods use neural networks to represent likelihoods [TT33 likelihood ratios [14H17] or target several properties at once [1-21]. SBI has been used for scientific discovery in various domains, including astrophysics [2 and neuroscience 123]
Compared to classical Approximate Bayesian Computation (ABC, 22 25 or synthetic likelihoods methods (26), neural-network-beee
SBI methods scale better to complex, high-dimensional simu- lators 27 In addition, SBI methods often amortize the computational burden: Once trained, the network provides fast inference across multiple observations.
Despite these advances, a key challenge for SBI methods remains: They typically require generating a large number of simulations as synthetic training data, rendering them impractical for expensive simulators. Recent work aims to increase simulation efficiency, e.g., by exploiting additional information or properties of the simulator 77] 28.31. utilizing low-fidelity simulations 32 K3
or using Bayesian neural networks (34H36). So-called sequential approaches forego the amortization properties of the inference network and target simulations to particular observations [4 6 37,, but
even then, simulation efficiency remains a challenge. In addition, users must select an appropriate
SBI method and select and train an inference network with the associated hyperparameters posing a barrier for inexperienced users.
These limitations raise key question: Can eliminate the high cost associated with train- ing and tuning, and alleviate the need to have a large number of simulations by leveraging re- cent advances in foundation models? Here, we answer this question by repurposing tabular foundation model specificill Tabular Prior- data Fitted Networks v2 (TabPFN) [38] as an inference engine for SBI. TabPFN is trained to perform in-context learning on tabular data (i.e., structured data organized rows and columns), returning prediction when being prompted with a training dataset and a test point. For both regression and classification, TabPFN was shown to perform exceptionally well, in particu- lar in low data regimes. Crucially, TabPFN can also serve as a density estimator by applying it in an autoregressive manner, though prior work has only explored this in limited settings [38]
prior 0 ~ p(0)
simulation æ ~ p(x(0)
simulation dataset
(0,
filter
NPE ①
NPE-PFN
Training
①
Evaluate
min
C c 0
Evaluate
87 ~ (6' 1 ...
Here, we show that using TabPFN autoregres- sively allows for the estimation of complex, high-dimensional conditional densities, making suitable for SBI. Based on this observation, we introduce Neural Posterior Estimation with Prior-data Fitted Networks (NPE-PEN), a vari- ant of NPE that uses TabPFN as a pre-trained conditional density estimator, and therefore en- ables inference without any additional neural network training (Fig. []. On benchmark tasks, NPE-PFN performs competitively with other SBI approaches and often outperforms them substan- tially, especially for small simulation budgets. Furthermore, we leverage data-filtering schemes that enable NPE-PPN make use of large simulation budgets exceeding the context size of TabPFN and extend the approach to sequential inference settings. Finally, we show that NPE-PEN remains empiri- cally robust under model misspecification and achieves strong results on two challenging scientific applications, requiring orders of magnitude fewer simulations than standard SBI approaches.
direct posterior
autoregressive posteriar
Figure 1: Comparison of NPE to NPE-PFN (ours): Both approaches use simulations sampled from the prior and simulator. In (standard) NPF, neu- ral density estimator is trained to obtain the pos- terior. In NPE-PEN, the posterior is evaluated by autoregressively passing the simulation dataset and observations to TabPFN.
2

---

2 Neural Posterior Estimation with Prior-data Fitted Networks
2.1 Background
We consider a simulator with parameters 0 E Rddo that stochastically generates samples x: E R thus implicitly defining likelihood p(a: 0). Given a prior distribution p(0) and observation oo. our aim is to infer the associated posterior distribution p(O Xo) x p(æ, O ())()).
Simulation-basse inference. We focus Neural Posterior Estimation (NPE), a common approach to SBI: NPE approximates the target posterior distribution directly by training an inference network ( x) dataset of parameter- data pairs D {(O.. x.));= sampled from prior p(0) and a simulator with implicit likelihood p(z 0). The inference network is trained to maximize the loglliillhoodd and learns to approximate the corresponding posterior distribution 4 39 Flexible
conditional neural density estimators, such as normalizing flows [40) are typically used as
inference networks. At inference time, NPE provides amortized posterior approximation, allowing for rapid evaluations.
Tabular foundation models. Recent foundation models for tabular data have shown strong per- formance on regression and classification tasks via in-context learning, which predictions for new inputs are made by conditioning the model context dataset [3] We focus on Prior-data Fitted Networks (PFNs) [4 45], specifically TabPFN [38], a state-of of- the-art transformer baased PFN foundation model 46 for probabilistic classification and regression tabular data. It has been pre- trained on synthetic datasets {8) ), with up to M = 2048 samples from randomly generated structural causal models by maximizing the likelihood go æo. T), where T = {(t, x., ==1 is the in-context dataset. After training. TabPFN has been tested to perform well up feature dimension of d, I 500 and a context size N < 10a (38.
At inference, TabPFN conditions on given in-context dataset T' and test point I, to estimate the distribution p(y xo) R ( Io. T') over the target variable y. Importantly, in its basic version, TabPFN is limited to univariate targets (y E IR for regression, y E {l,.... n for classifica- tion). Thus, TabPFN an in-context density estimator for one-dimensional densities. We perform inference over model parameter 0 R by setting y 0 to estimate the posterior distribution p(0 x.) qy T. D) given an observation T, and a dataset of simulations DD However, to estimate full posterior distributions over multiple model parameters Re conditional density estimation in high dimensions is required.
2.2 TabPEN for simulation-base inference
To perform comuulation-hsed inference with TabPFN, repurpose it as a general conditional density estimator. The key to estimating high-dimensional densities with TabPFN is to use it autoregressively. i... use it to sequentially predict the next dimension of the data with the previously processed dimensions and covariates in context.
In SBI, given prior p(0) and simulator p(z (),, the multivariate posterior distribution p(0 x.)
for observation 2, can be decomposed as
de
p(0 x.)
g (0) II 1=1
|0-5, xo, 10,,, D(S)
xx.]) with <j denoting the vector from index up to, but not including j. Thus, using an arbitrary but fixed order of the parameter dimensions, one evaluation of the posterior p(O æo) requires de evaluations of TabPFN, making the prediction more expensive as the dimension- ality increases. However, unlike standard NPE methods, this approach bypasses model fitting entirely and directly evaluates the posterior. We refer our proposed approach as NPE-PEN, in contrast. to standard NPE, which requires training an inference network, e.g., a normalizing flow (Fig. pseudocode for NPE-PEN in Appendix Alg.
where D {0], (0
2.3 Increasing the effective context size
Standard NPE can approximate arbitrarily complex conditional densities, given enough simulations and sufficiently expressive inference network [4]. In contrast, NPE-PFN is limited by TabPFN's

---

context size of approximately 10* samples, beyond which additional data yields diminishing perfor- mance gains. While we expect NPE-PFN to be particularly powerful for small simulation counts, it is nevertheless desirable to also be able to work with larger simulation budgets using in-context learning.
Filtering simulations based on relevance. To overcome the limited context size, previous work proposed use a subset of the training data containing only the nearest neighbors of a given test point as the context dataset 477. Here, we show that this procedure, and generalizations of it, are sound from a Bayesian perspective when estimating the full posterior distribution. To this end, we exploit a key property of (neural) posterior estimation: The posterior can be estimated using only samples that are very close to the observation To. More specifically, for any non-negativ function fr. that satisfies f=, (2.)) > O, we can reweigh the joint distribution p(z, fm. (2) without affecting the posterior distribution at convergence (details Appendix Sec. [B.2). This property allows us to filter simulations based on relevance without biasing the estimation. Choosing the filter
as fz. (z) = T(d(ar, 2:.) < <), where .(::)) is a suitable distance metric and > is a threshold, effectively recovers an ABC-like selection scheme, where only simulations close to E. are retained. In practice, e is set adaptively to include the Nitterr closest simulations, ensuring that the full context is used. Here, we consider the Euclidean distance d(æ., x) = || xoo/2 after standardizing each feature dimension. As noted above, this filtering based on nearest neighbors has previously been explored and is also referred to localization or retrieval 47 49 This filtering principle also underlies various SBI methods, such as the aforementioned ABC or calibration kernels in NPE training [5].
Filtering for simulations that are relevant or close to given observation makes the (context) dataset dependent on that observation. For classical SBI methods, this step would require re-training the density estimator for each new observation, thus breaking amortization. However, NPE-PEN remains amortized: Since it is training-free, inference for new observations requires only a computationally negligible filtering step over fixed set of simulations. Finally, filtering only works for conditional density estimation problems such as Siimulation-baseds Bayesian inference, which is our focus here. However, we also show how (Appendix Sec. B.4) this approach can be extended to unconditional density estimation on a larger number of samples. In the unconditional case, partition the data space into regions that can be solved with local contexts.
Embedding data. Similarly to context size, the feature size can be a limitation, particularly for high-dimensional observations. This limitation can be handled either by performing inference using human-crafted summary statistics (as commonly done in ABC) or using (pre-trained) embedding networks 5 (0). In principle, NPE-PEN allows for training an embedding network end-to-end (e.g.. [5I] in the context of classification), but we here investigate pre-training embedding network independently of the density estimator (Appendix Sec. B33
2.4 Truncated sequential NPE-PEN
To reduce the number of simulations, sequential SBI methods have been developed that acquire simulations adaptively in rounds for one observation x.. We focus on Truncated Sequential NPE (TSNPE) (37], which mitigates bias by exploiting the (approximate) invariance of the posterior under modifications outside of its high-density regions (HDR). Formally, let HDR == (O p(0 x:.) > ka), where k. denotes the density threshold containing a% of the probability mass. Then, given a truncated prior p0))= T(O E HDR, " ((()). the posterior will be approximately preserved for a.. In TSNPF, a density estimator is trained in rounds. Starting with the full prior in the first round, the trained posterior approximation (()) xx)) used estimate HDR p x HDR!- and to truncate the prior to the HDR of the posterior approximation for the next round. In each subsequent round, the newly truncated prior is used to generate new simulations adaptively and refine g(O xx.) through continued training. Notably, training becomes redundant by replacing the density estimator with the TabPFN regressor, to which we will refer as TSNPE-PFN. In this training-free -fren setting, the main computational burden shifts to truncating the prior distribution. Prior truncation can be achieved either by approximate approaches, such as sampling-importance resampling, or exactly by rejection sampling from the prior, i.e., checking whether prior sample falls within the HDR. However, the latter approach requires repeated density estimation with the TabPFN regressor to evaluate ( r..). which can be costly to do autoregressively, a problem we address next.

---

Gausa. Linear
Mixture
Moons
SLCP
Berouulli
SIR
Lotka Volterra
1.0
C251
0.5
10
NPE
NLE
NPE-PEN (unfillered)
NPE-PFN
Num.
Sampling time
Training time
Two Moons
SLCP
Bernuuli GLM
SIR
Lotka Volterra
10"
1.0
Time [s]
10"
C2ST
10
10
no training
10'10" 10'0
10
10"
10 4
TSNPE
SNLE
TSNPE-PFN
Num, sim.
Num. sim.
Figure 2: SBI benchmark results for amortized and sequential NPE-PFN. (a) C2ST for NPE, NLE, and NPE-PEN across ten reference posteriors (lower is better); dots indicate averages and bars show 95% % confidence intervals over five independent runs. (b) Average time to generate 10 posterior samples and (if applicable) training time. (c) C2ST for sequential methods.
For rejection sampling, need evaluate q(0) x.)) for large number of prior samples 0 ~ p(0). In practice, the posterior is often much narrower than the prior and a large percentage of these samples will be rejected. This high rejection necessitates fast, approximate approach for density evaluation, which we will construct using the "ratio-trick" [52]. We first sample an initial set of posterior samples {0:); from the NPE-PEN posterior once and assign these samples the class label
1. We then contrast these samples with samples from a uniform distribution U(0; ( 0 Omin mmax) which we assign the class label y 0. The Bayes optimal classifier for this classification problem is given by
p(0) xo))
Pa. (y =
Pa. (y = 0) p(0 x.)) + ((; 0 min: 1 Omax)
thus, p(0 x.0)
1 P z. (y
within the bounds O min- Omax- After autoregressively sampling from the desired posterior to construct the training dataset, this approach, which we will refer to ratio-based density evaluation, requires only a single forward pass using the TabPFN classifier and is, therefore, significantly faster than autoregressive density evaluation, especially as the parameter dimension do the number of required density evaluations grows (pseudocode for TSNPE-PFN in Appendix Alg. 2
Experiments
To assess NPE-PEN, conduct experiments synthetic SBI benchmark tasks and real data, cover- ing scenarios from low to nigh-dimensional data and including cases with model misspecification.
3.1 Amortized and sequential benchmark performance
We evaluate NPE-PFN on various tasks from the SBI benchmark 27 which provides ground truth posterior samples for observations for each task. We measure posterior sample quality using the classifier two-sample test (C2ST, (3)). A C2ST accuracy of 0.5 indicates that the approximate posterior exactly matches the ground truth posterior, as the classifier fails to distinguish between the two sample sets. In contrast, accuracy of 1.0 reflects strong mismatch between the estimated and true posterior. As baselines, we compare against flow-based NPE and NLE provided by the SBI library [54, (55. For sequential baselines, use truncated sequential NPE (TSNPE (37] and sequential NLE (SNLE [I]].. Posterior estimates are computed across four simulation budgets, ranging from 2 to 10 simulations. Beyond 10 simulations. NPE-PEN incorporates filtering, whereas an alternative unfiltered variant, denoted NPE-PFN (unfiltered), extends the context size beyond the TabPFNN-rcoommedde limit. For budgets 103, both variants are identical.
Amortized inference, For 102 simulations, NPE-PEN substantially outperforms NLE and NPE on all but one task (Fig. bb.). For 10 or 10 simulations, NPE-PEN is generally competitive, with
5

---

the exception of a few tasks where some baselines outperform it. For example, NLE outperforms NPE-PEN on the targets the posterior instead of the simpler likelihood. However, NPE-PFN performs substantially better than both baselines on tasks such as SIR or Lotka-Volterra Volterra for all simulation budgets. With 10 simulations, the difference between NPE-PEN and its unfiltered variant becomes evident. The extended context does not improve performance for larger simulation budgets, while the filtered NPE-PEN variant achieves significantly better C2ST accuracies, matching or surpassing the baselines. While NPE-PEN is training-free, inference speed depends on the number of simulations, and the dimensionality of the parameter and observation spaces. For the benchmark tasks, observe an inference speed that is comparable to NLE (Fig. 2p). but orders of magnitude slower than NPE, which is meaar-inttnttaneuss We perform careful analysis of the inference speed of NPE-PFN across all relevant variables for users to make an informed decision in their respective application (Appendix
(SLCP) task, as NPE-PFN, like NPE, directly
Sec. D33
Tab. D-2p-
The advantage of NPE- OPE-PFN over baseline methods remains for other tasks outside the SBI benchmark suite (Appendix Sec. D.I. Fig.
D-1), well as compared to other baseline methods such as NRE, NPE ensembles (Appendix Sec. D.2 Fig. D-2). and even when performing extensive hyperparameter optimization for NPE (Appendix Fig. D-4). Furthermore, NPE-PEN posterior estimates are well- calibrated (Fig. D-3)
In cases where data dimension dg outgrows the supported feature dimension and learned summary statisties are required, the end-to and-to-end trained NPE, however, outperforms NPE-PEN with indepee- dently pre-trained summary statistics (Appendix Sec. D_4 Fig. D-5). In addition, we ablate the effect of the filter size indicating that the default size (close to) optimal for all considered tasks (Appendix Sec. [.6] Fig. D-7). Finally, we vary the autoregressive order, in which the parameter dimensions are sampled, and find no noticeable effect compared to the default order (Appendix See. D.8).
Sequential inference. We adapt the setup in Deistler et al. (37 to evaluate TSNPE-PEN. Rejection sampling is performed using the fast, approximate ratio-based densities (Sec. 2.4 Appendix Sec. D.5 Fig. D-6 for a comparison between autoregressive and ratio-based density evaluation). We equally divide the simulation budget into 10 rounds, with proposals truncated to the E highest density re- gion, where и = 10 -3 TSNPE-PEN excels with small simulation budgets (Fig. (Fig.2)). On the SIR task, reaches close to optimal C2ST accuracy with only 102 simulations. At larger budgets, TSNPE-PEN maintains strong performance, particularly on the Lokta-Volterre Volterra task, which is highly challenging for other SBI methods. TSNPE-PFN benefits from NPE-PFN's ability to generate high-quality poste- rior estimates with few simulations, allowing it to acquire more relevant simulations in early rounds.
Summary. NPE-PEN provides accurate poste- rior estimates, especially for small simulation budgets. With filtering. NPE-PFN is also com- petitive for larger simulation budgets. As an in-context learning method, it requires no train- ing on specific simulations and offers inference speeds comparable to other baselines, though runtime scales with the number of simulations and the dimensionality of the parameters and observations. This flexibility makes NPE-PFN applicable to a wide range of setups and sim- ulators. We observe NPE-PEN performs par- ticularly well in tasks with underlying graphi- cal structure and conditional (in- )depnndencies, which are similar to the structural causal mod- els used in TabPEN's pre-training. In addi- tion posterior estimation with NPE-PFN, we evaluate the unconditional high-dimensional density estimation capabilities of TabPFN on UCI datasets (56), showing that it is more data- efficient than neural spline flow (Appendix
NPE
NPE-PFN
ANPEEPPNN NPE)
Prior Pnior miss.
In (scale)..
Mm. (mean)
5
Hm. (mean)
C2ST
Mm. (mean)
C2ST
Likelloood miss.
In (scale),
1.00
0.2
0.75
0.0
0.50
-0.2
A, (fraction)
An (fraction)
Am (fraction)
Figure 3: Robustness under misspecification. C2ST against well-specified ground truth pos- terior. The red star marks the well-specified model. Rows:
Prior and likelihood misspecification. Columns: NPE, NPE-PEN, and their difference in terms of C2ST accuracy (darker blue indicates that NPE-PFN is better).
Sec. D.9
Fig. D-10

---

b
real obs.
synthetii obs.
Real observations
dist to obs:
TSNPE
TSNPE-PFN
TSNPE-PFN
prediclives
10
10
10
10"
Number of simulations
Number of simulations
TSNPE predictives
Skk
9k
0.00
9
0.05
0.0
9M
0.6
d
10" sim.
10" sim.
10" sim.
NPE NPE-PFN
0
50
Tix
3000
40
V+
90
0.0
0.2
35
E 100
Figure 4: Posterior inference for observations from the Allen cell type database. (a) Average distance to observation in standardized space of summary statisties for both the real and synthetic ob- servations. (b) Posterior predictive simulations for real observations using TSNPE-PEN and TSNPE. (c) Posterior marginals for one synthetic observation; TSNPE-PEN marginals are substantially more constrained for several parameters. (d) Simulation-Nan based calibration for NPE-PEN and NPE on the HH simulator.
Impact of misspecification
To investigate how NPE-PFN handles misspecification, we use the 2D Gaussian means misspecifica- tion benchmark from Schmitt et al. [57], considering both prior and likelihood misspecification. The ground truth is given by the prior μ (O, I) and likelihood æ; NN,, I). Prior misspecification while keeping the true likelihood; Likelihood misspecification
uses the prior uses the true prior with the likelihood æ; Xm Beta(2tt 5) + (1 Xm )N Tm T) where Tm RR+,, Am E [0, 1]. We evaluate the C2ST of predicted posteriors against the reference posterior distribution of the well-specified data using 102 simulations (details in Appendix Sec. 3.2
Im (
Both NPE-PFN and NPE degrade as the level of misspecification increases, with prior misspecification leading to more severe performance degradation than likelihood misspecification (Fig. 33. However, NPE-PFN achieves better C2ST values than NPE across broader range of misspecification levels. In addition, NPE-PEN remains insensitive to various feature and noise distributions (Appendix Sec. D.7 Fig. D-8). Finally, we find that the performance of NPE-PFN on the SBI benchmark tasks is invariant the choice of the autoregressive order (Appendix Sec. D.8 Fig. D-9p.
Thus, the trend observed in the SBI benchmark extends to these simple but misspecified tasks, on which NPE-PEN also consistently outperforms NPE. Next, evaluated whether this performance gap also holds for scientific tasks with real data, on which the simulator is potentially misspecified.
3.3 Single-Compartment Modgkin-Huxkee Model
We evaluate the performance of (TS)NPE-PEN on challenging inference task, considering the Hodgkin- (HH) simulator of single-neural voltage dynamics [58] 59). Following Gonçalves et al. [60]. the simulator has eight parameters and seven summary statistics based on the voltage trace. We infer posteriors for 10 real observations from the Allen cell type database [61], for which inference is known be challenging (2H64). We evaluate inference quality via posterior predictives and the average Euclidean distance between predictives and real observations in a standardized summary statistics space. In addition, for each real observation, we create an associated synthetic, and therefore well-specified observation, where the ground truth parameter is known, allowing for direct comparison between inferred posteriors and true values.
TSNPE-PFN vields posterior predictives that closely match real observations with simulation budget of a (Fig. 4b.b). outperforms TSNPE, which struggles to produce realistic predictives, especially smaller budgets. For synthetic observations, TSNPE-PFN again achieves better predictive accuracy

---

with fewer simulations, with the average distance approaching zero for a simulation budget of 10 In parameter space, TSNPE-PEN posteriors are tightly concentrated around the true parameters, with significant improvement over TSNPE (Fig. 3o.. In the amorlized setting, NPE-PEN shows good calibration for 10 and 10 simulations but is slightly overconfident for 10 (Fig- 4 However, (TS)NPE-PEN achieves significantly better prediction quality and yields tight posteriors around the true parameters, making it more sensitive to miscalibration.
These results demonstrate TSNPE-PEN's superior simulation efficiency, achieving the same predictive quality as TSNPE with an order of magnitude fewer simulations. This efficiency makes TSNPE-PET particularly well-suited for inference with real, potentially misspecified simulators under limited simulation budgets.
3.4 Pylorie Network Model
Finally, we evaluate TSNPE-PEN a high-dimensional simulator of the pyloric network in the stomatogastric ganglion (STG) of the crab Cancer Borealis, well-studied circuit generating rhythmic activity [60] 65 666. This model consists of three neurons, each with eight membrane conductances, interconnected by seven synapses, resulting in a 31-dimensional parameter space. The simulated voltage traces are summarized using 15 established statistics. A key challenge in this setting is the extreme sparsity of valid simulations. Specifically, 99% of parameter samples from the prior yield implausible voltage traces, preventing the computation of summary statistics [67].
For this reason, earlier work relied on millions of simulations for an amortized NPE-based posterior approximation (18 million in Gonçalves et al. [60]). The simulation count could later be reduced by restricting the prior to valid simulation using an additional classifier [67] addition, several
sequential algorithms have been developed to address this problem [18 37 68
the latest of which
achieves good posterior predictives using 1.5 10y simulations.
To perform sequential inference on the pyloric net- work with TSNPE-PEN we use a restricted prior based on the TabPFN classifier. Due to the high
b
measurement
PY LPABIPD
dist. to obs.
1.0
dimensionality of the parameter space, rejection sam- pling is prohibitive. Therefore, we use sampling im-
0.1
portance resampling with an oversampling factor of 10. In the first round, we use 5 10s simulations and run TSNPE-PEN for 45 rounds, adding 10s simu- lations per round for total of 5 10 simulations (details in Appendix Sec. C.4.
posterior prediclive
100
% valld
The generated voltage traces from posterior mean esti- mates of TSNPE-PFN closely resemble true measure- ments (). With more than 50% valid simulations at just 5 10 simulations and approximately 99% va- lidity for fewer than 5- 10 simulations, TSNPE-PEN outperforms all comparison methods, which either required substantially larger simulation budgets to reach comparable performance or did not reach at all. Furthermore, the autoregressive evaluation or- der again has no noticeable effect on the resulting performance (Appendix Sec. C.4).
150
(x109)
0.0 Time 2.5
SNVI
TSNPE TSNPE-PFN
S-UNLE
Figure Results on the pyloric simulator. (a) Voltage traces from the experimental mea- surement (top) and a posterior predictive sim- ulated using the posterior mean from TSNPE- PFN as the parameter (bottom). (b) Average distance (energy scoring rule) observation and percentage of valid simulation from pos- terior samples; compared experimental re- sults obtained in Glaser et al. 688
Our results demonstrate that TSNPE-PEY can infer posteriors with substantially improved simulation ef- ficiency, making a powerful approach for complex high-dimensional models with a limited simulation budget.
4 Discussion
We have shown here that pre-trained foundation models specifiiilll, TabPFN can effectively repurposed to perform accurate training- faining-free simulation based inference. Together with autoregressive
8

---

sampling for sigh-dimensional posterior distributions, filtering to increase context size, and fast density evaluation using the density ratio. NPE-PEN provides flexible inference method that can be effortlessly applied to many different inference tasks. Across several benchmarks and two real- world tasks, NPE-PEN consistently estimates accurate posteriors with particularly strong results for small simulation budgets, often requiring orders of magnitude fewer simulations than existing SBI methods. These results establish foundation models and in-context learning as a promising direction for training-free inference in scientific applications.
Related work. Our work relates meta-learning 69 where models generalize across tasks by acquiring shared inductive biases Meta learning has been applied to amortized (Bayesian) inference [1]. More specifically, however, NPE-PFN an instance of in-context learning (ICL), where task adaptation is achieved solely by conditioning pre- pre-trained models on different inputs [43] ICL was popularized by LLMMs [72], but its scope has expanded beyond language, with work showing that transformers can learn various function classes in context [73 74) and work introducing prior NPE-PEN extends this direction by enabling ICL. for simulation-basee -aased Bayesian inference. We note that this does not require training the inference networks any SBI tasks, but rather works well using an "off-the-shelf" model trained for classification and regression.
fitted networks [3 44 45
In addition to NPE-PEN, multiple methods have been proposed to perform ICL for Bayesian inference 75 777 These approaches parameterize posteriors using, for example, Gaussians, flows, or mixtures and require fixed input and parameter dimensionality and, to date, are only trained on specific, limited class of probabilistic models. In contrast, NPE-PEN leverages autoregressive modeling over parameter dimensions, enabling zero-shot generalization without retraining across inference problems with varying shapes.
Finally, several works have proposed extensions for TabPFN beyond its original setting, targeting new data modalities, larger context sizes, or higher-dimensional features. To address TabPFN's limited context size, retrieval-basse approaches select informative subsets of the training data 47- including searest-neighhore
49 78,
filtering [47 which we adapt in NPE-PFN for Bayesian inference. Crucially, we demonstrate that this filtering does not bias the posterior, as the filter step can be interpreted as a calibration kernel [5). Other strategies include data distillation (79 [0]. boosting weak learners [8], fcature-wise ensembling [82). encoders and bagging [5]], as well as
non-autoregressive extensions to time series forecasting X and tabular data generation [84). TabICL
885 proposes architectural changes to scale to datasets with over 10 points. NPE-PFN departs from
these approaches by repurposing TabPFN for posterior inference through filtering and fast evaluation of densities.
Limitations. While NPE-PEN eliminates the need for simulation-specifie training, it inherits proper- ties of TabPFN. TabPFN has a soft limit in maximum feature size (S. 500) and context length (< 10*). We present methods to overcome these limitations. i.e, filtering and embedding nets. However, as the number of simulations and the need for density evaluations increase, the performance advantage of NPE-PEN over standard methods such as NPE diminishes. Furthermore, while embedding networks allow NPE-PFN to be used for high-dimensional observations, they require separate pre-training. In addition, modeling high-dimensional parameter spaces beyond TabPFN's maximum feature size of 500 dimensions remains challenging. Finally, like other in in-context methods, NPE-PFN is more ex- pensive than fow-based NPE, a challenge exacerbated by the autoregressive estimation of posteriors. However, unlike other in-context methods for Bayesian inference, NPE-PFN does not require training and hyperparameter tuning, making suitable for exploratory workflows or non-expert users. In addition, by requiring substantially fewer simulations, NPE-PFN accelerates the overall SBI workflow. Thus, while we expect NPN-PEN to be particularly useful for exploratory workflows or expensive simulators, inference methods based on trained density estimators might still be advantageous in settings where large numbers of simulations are available, or where bespoke embedding networks and fast sampling are required, such as in some applications in astrophysics [22]. An interesting direction for future work is to transfer NPE-PFN's advantages to existing, faster inference techniques (e.g., g., via student-teacher distillation).
Conclusion. We presented NPE-PFN, a method that repurposes the tabular foundation model TabPFN for training-free simulation-basee Bayesian inference. NPE-PFN is competitive with-and often outperforms-exisiine methods, especially for small simulation budgets. NPE-PFN a simple-to-use and iimulation-efficieee tool for SBI, with the potential to become the go-to method for novice users, exploratory workflows, and applications with constrained simulation budgets.

---

Acknowledgments and Disclosure of Funding
We thank Michael Deistler and Katharina Eggensperger for feedback on the manuscript, specifically Michael Deistler for suggesting to filter simulations and Jonas Beck for the implementation of the pyloric simulator. We thank all members of the Mackelab for discussions and feedback on the manuscript. This work was funded by the German Research Foundation (DFG) under Germany's Excellence Strategy - EXC number 2064/1 390727645 and SFB 1233 'Robust Vision' (2766935 17), the European Union (ERC, DeepCoMechTome, 101089288), the "Certification and Foundations of Safe Machine Learning Systems Healthcare" project funded by the Carl Zeiss Foundation and the Bochringer Ingelheim Al & Data Science Fellowship Program. JV and MG members of the International Max Planck Research School for Intelligent Systems (IMPRS-IS).
References
[1] Eric Winsberg. Science in the age of computer simulation. University Chicago Press, 2019.
[2] Benjamin Skuse. The third pillar. Physics World, 32(3):40-43, 2019.
[3] Alexander Lavin, David Krakauer, Hector Zenil, Justin Gottschlich, Tim Mattson, Johann Brehmer, Anima Anandkumar, Sanjay Choudry, Kamil Rocki, Atilm Güneş Baydin, Carina Prunkl, Brooks Paige, Olexandr Isayev, Erik Peterson, Peter L. McMahon, Jakob Macke, Kyle Cranmer, Jiaxin Zhang, Haruko Wainwright, Adi Hanuka, Manuela Veloso, Samuel Assefa, Stephan Zheng, and Avi Pfeffer. Simulation intelligence: Towards new generation of scientific methods. arXiv preprint arXiv:2112..3335, 2022.
[4] George Papamakarios and lain Murray. Fast s-free inference of simulation models with Bayesian conditional density estimation. Advances neural information processing systems, 29, 2016.
[5] Jan-Matthis Lueckmann, Pedro Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnen- macher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. Advances in neural information processing systems, 30, 2017.
[6] David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-fret inference. In International Conference Machine Learning, pages 2404- 2414. PMLR, 2019.
[7] Maximilian Dax, Stephen R Green, Jonathan Gair, Michael Deistler, Bernhard Schölkopf, and Jakob H. Macke. Group equivariant neural posterior estimation. International Conference on Learning Representations, 2022.
[8] Tomas Geffner, George Papamakarios, and Andriy Mnih. Compositional score modeling for simulation-based inference. In International Conference on Machine Learning, pages 11098-11116. PMLR, 2023.
[9] Louis Sharrock, Jack Simons, Song Liu, and Mark Beaumont. Sequential neural score estima- tion: fikeei inference with conditional score based diffusion models. In International Conference on Machine Learning, pages 44565-44602. PMLR, 2024.
[10] Jonas Wildberger, Maximilian Dax, Simon Buchholz, Stephen Green, Jakob H Macke, and Bernhard Schölkopf. Flow matching for scalable simulation-based inference. In Advances in Neural Information Processing Systems, volume 36, pages 16837-16864, 2023.
(11) George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood-Fii inference with autoregressive flows. In International Conference on Artificial Intelligence and Statistics, pages 837-848. PMLR, 2019.
[12] Jan-Mauhis Lueckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H Macke. Likelihoodd-free inference with emulator networks. In Symposium on Advances in Approx- imate Bayesian Inference, pages 32-53. PMLR, 2019.
[13] Jan Boelts, Jan-Matthis Lueckmann, Richard Gao, and Jakob Macke. Flexible and efficient simulation-based inference for models of socision-making Elife, 11:e77220, 2022.
10

---

[14] Conor Durkan, lain Murray, and George Papamakarios. On contrastive learning for likelihood- free inference. In International Conference on Machine Learning, pages 2771-2781. PMLR. 2020.
[15] Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-free MCMC with amortized approximate ratio estimators. In International Conference on Machine Learning, pages 4239- 4248. PMLR, 2020.
[16] Joeri Hermans, Arnaud Delaunoy, François Rozet, Antoine Wehenkel, Volodimir Begy, and Gilles Louppe. A crisis in simulation-bssed inference? Beware, your posterior approximations can be unfaithful. Transactions on Machine Learning Research, 2022.
[17] Benjamin K Miller, Christoph Weniger, and Patrick Forré. Contrastive neural ratio estimation. In Advances in Neural Information Processing Systems, volume 35, pages 2262-3278, 2022.
[18] Manuel Gloeckler, Michael Deistler, and Jakob H. Macke. Variational methods for simulation- based inference. In International Conference on Learning Representations, 2022.
[19] Stefan T. Radev, Marvin Schmitt, Valentin Pratz, Umberto Picchini, Ullrich Köthe, and Paul- Christian Bürkner. Jana: Jointly amortized neural approximation of complex Bayesian models. In Conference on Uncertainty Artificial Intelligence, volume 216, pages 1695-1706. PMLR, 2023.
[20] He Jia. Simulation-based inference with quantile regression. International Conference on Machine Learning, 2024.
[21] Manuel Gloeckler, Michael Deistler, Christian Dietrich Weilbach, Frank Wood, and Jakob H. Macke. All-in-one simulation-basee inference. In International Conference on Machine Learning, volume 235, pages 15735-15766 PMLR, 2024.
[22] Maximilian Dax, Stephen R Green, Jonathan Gair, Nihar Gupte, Michael Pürrer, Vivien Ray- mond, Jonas Wildberger, Jakob Macke, Alessandra Buonanno, and Bernhard Schölkopf. Real-time inference for binary neutron star mergers using machine learning. Nature, 639(8053): 49-53, 2025.
[23] Lukas N. Groschner, Jonatan G. Malis, Birte Zuidinga. and Alexander Borst. A biophysical account of multiplication by single neuron. Nature, 603(7899):119-123, 2022.
[24] Mark A Beaumont, Wenyang Zhang, and David Balding. Approximate Bayesian computation in population genetics. Genetics, 26254))20555 2002.
[25] Mark A Beaumont, Jean-Marie Cornuet, Jean-Michel Marin, and Christian P Robert. Adaptive approximate Bayesian computation. Biometrika, 96(4):983-99, 2009.
[26] Simon N Wood. Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466(7310): 102-1104, 2010.
[27] Jan-Mathhss Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Benchmarking simulation-based inference. In International Conference on Artificiaa Intelligence and Statistics, pages 343-351. PMLR, 2021.
[28] Atilim Güneş Baydin, Lei Shao, Wahid Bhimji, Lukas Heinrich, Lawrence Meadows, Jialin Liu, Andreas Munk, Saeid Naderiparizi, Bradley Gram-Hansen, Gilles Louppe, al. Etalumis: Bringing probabilistic programming to scientific simulators at scale. In Proceedings of the international conference for high performance computing, networking, storage and analysis, pages 1-24, 2019.
[29] Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. Mining gold from implicit models to improve likelihood -free inference. Proceedings of the National Academy of Sciences, 117(10):524-52444 2020.
[30] Christian Weilbach, Boyan Beronov, Frank Wood, and William Harvey. Structured condi- tional continuous normalizing flows for efficient amortized inference in graphical models. In International Conference on Artificial Intelligence and Statistics, pages 4441-4451. PMLR, 2020.
11

---

[31] Manuel Gloeckler, Shoji Toyota, Kenji Fukumizu, and Jakob H. Macke. Compositional simulation-baut based inference for time series. In International Conference on Learning Represen- lations, 2025.
[32] Anastasia N Krouglova, Hayden R Johnson, Basile Confavreux, Michael Deistler, and Pedro J Gonçalves. Multifidelity simulation-basee based inference for computationally expensive simulators. arXiv preprint wwxii:2502.08416. 2025.
[33] Caroline Tatsuoka, Minglei Yang. Dongbin Xiu, and Guannan Zhang. Multi-fidelity parameter estimation using conditional diffusion models. arXiv preprint arXtv:250.001894 2025.
[34] Fredrik Wrede, Robin Eriksson, Richard Jiang, Linda Petzold, Stefan Engblom, Andreas Hellander, and Prashant Singh. Robust and integrative Bayesian neural networks for likelihood- free parameter inference. In International Joint Conference on Neural Networks, pages 1-10. IEEE, 2022.
[35] Pablo Lemos, Miles Cranmer, Muntazir Abidi, ChangHoon Hahn, Michael Eickenberg, Elena Massara, David Yallup, and Shirley Ho. Robust simulation-based inference in cosmology with Bayesian neural networks. Machine Learning: Science and Technology, 4(1):01LT01, 2023.
[36] Arnaud Delaunoy, Maxence de la Brassinne Bonardeaux, Siddharth Mishra-Sharma, and Gilles Louppe. Low-budget simulation-based inference with Bayesian neural networks. arXiv preprint arXiv:2408.151: 15136, 2024.
[37] Michael Deistler, Pedro J Goncalves, and Jakob H Macke. Truncated proposals for scalable and hassle-free simulation- inference. In Advances in Neural Information Processing Systems, volume 35, pages 23135-231443 2022.
[38] Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, (37(8045):119-33) 2025.
[39] Tuan Anh Le, Atilim Güineş Baydin, Robert Zinkov, and Frank Wood. Using synthetic data to train neural networks is model-based reasoning. In International Joint Conference on Neural Networks (IJCNN, pages 3514-3521. IEEE, 2017.
[40] Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International Conference on Machine Learning, volume 37, pages 1530-1538. PMLR, 2015.
[41] George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. In Advances in Neural Information Processing Systems, volume 30, 2017.
[42] Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Neural spline flows. In Advances in Neural Information Processing Systems, volume 32, 2019.
[43] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng. Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A survey on in-context learning. In Conference on Empirical Methods in Natural Language Processing, pages 1107-1128, 2024.
[44] Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do Bayesian inference. In International Conference on Learning Representa- tions, 2022.
[45] Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPFN: A transformer that solves small tabular classification problems in second. In International Conference on Learning Representations, 2023.
[46] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[47] Thomas Nagler. Statistical foundations of prior-data fitted networks. In International Conference on Machine Learning, pages 25660-25676. PMLR, 2023.
12

---

[48] Valentin Thomas. Junwei Ma, Rasa Hosseinzadeh, Keyvan Golestan, Guangwei Yu, Maks Volkovs, and Anthony L Caterini. Retrieval fine-tuning for in-context tabular models. Advances in Neural Information Processing Systems, 37: 37:108439-108467,, 2024.
[49] Mykhailo Koshil, Thomas Nagler, Matthias Feurer, and Katharina Eggensperger. Towards localization via data embedding for TabPFN. In NeurlPS 2024 Third Table Representation Learning Workshop, 2024.
[50] Yanzhi Chen, Dinghuai Zhang, Michael U. Gutmann, Aaron Courville, and Zhanxing Zhu. Neural approximate sufficient statisties for implicit models. In International Conference on Learning Representations, 2021.
[51] Si- Yang Liu and Han-Jia Ye. TabPFN unleashed: A scalable and effective solution to tabular classification problems. arXiv preprint arXi::2502.02527, 2025.
[52] Masashi Sugiyama, Takafumi Kanamori. Taiji Suzuki, Shohei Hido, Jun Sese, Ichiro Takeuchi, and Liwei Wang. A density-ratio framework for statistical data processing. IPSJ Transactions on Computer Vision and Applications, 1:183-208, 2009.
[53] David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. In International Conference Leurning Representations, 2017.
[54] Alvaro Tejero-Canters Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pedro J. Gonçalves, David S. Greenberg, and Jakob Macke. sbi: A toolkit for simulation- based inference. Journal of Open Source Software, 5(52):2505, 2020.
[55] Jan Boelts, Michael Deistler, Manuel Gloeckler, Álvaro Tejero-Cantero, Jan-Matthis Lueck- mann, Guy Moss, Peter Steinbach, Thomas Moreau, Fabio Muratore, Julia Linhart, Conor Durkan, Julius Vetter, Benjamin Kurt Miller, Maternus Herold, Abolfazl Ziacemehr, Matthijs Pals, Theo Gruner, Sebastian Bischoff, Nastya Krouglova, Richard Gao, Janne K. Lappalainen, Bálint Mucsányi, Felix Pei, Auguste Schulz, Zinovia Stefanidi, Pedro Rodrigues, Cornelius Schröder, Faried Abu Zaid, Jonas Beck, Jaivardhan Kapoor, David S. Greenberg, Pedro J. Gonçalves, and Jakob H. Macke. shi reloaded: a toolkit for simulation-based inference work- flows. Journal of Open Source Software, 10(108): 2025.
[56] Andrew Frank. UCI machine learning repository. http://archive. ics. uci. edu/ml, 2010.
[57] Marvin Schmitt, Paul- Caul-Christian Bürkner, Ulirich Köthe, and Stefan T Radev. Detecting model misspecification in amortized bayesian inference with neural networks: extended investiga- tion. arXiv preprint 24t.::2000033/5555 2024.
[58] A. L. Hodgkin and A. F. Huxley. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol, 117(())500-54) 1952. doi: 10.1 10.11 10/ph13/jhysiol. 11tpppyyiol.195522.0000. 1p52.s000476..
[59] Martin Pospischil. Maria Toledo Rodriigud Cyril Monier, Zuzanna Piwkowska, Thierry Bal, Yves Frégnac, Henry Markram, and Alain Destexhe. Minimal Hodgkin-Huxley type models for different classes of cortical and thalamic neurons. Biological cybernetics, 99:427-441, 2008.
[60] Pedro Gonçalves, Jan-Matthis Lueckmann, Michael Deistler, Marcel Nonnenmacher, Kaan Öcal, Giacomo Bassetto, Chaitanya Chintaluri, William Podlaski, Sara A Haddad, Tim Vogels, et al. Training deep neural density estimators to identify mechanistic models of neural dynamics. Elife, 9:e56261, 2020.
[61] Allen Institute for Brain Science. Allen cell types database. httts://celltypet wrw.ttyypeee.bbbimeeee.. brain-map org 2015.
[62] Richard Gao, Michael Deistler, and Jakob H Macke. Generalized Bayesian inference for scientific simulators via amortized cost estimation. Advances in Neural Information Processing Systems, 36:80191-802199 2023.
[63] Nicholas Tolley, Pedro LC Rodrigues, Alexandre Gramfort, and Stephanie R Jones. Methods and considerations for estimating parameters in biophysically detailed neural models with simulation based inference. PLOS Computational Biology, 20(2)xc001108 2024.
13

---

[64] Yves Bernaerts, Michael Deistler, Pedro J Goncalves, Jonas Beck, Marcel Stimberg. Federico Scala, Andreas Tolias, Jakob Macke, Dmitry Kobak, and Philipp Berens, Combined hatistical-mechanne modeling links ion channel genes to physiology of cortical neuron types. bioRviv, 2023.
[65] Astrid Prinz, Cyrus Billimoria, and Eve Marder. Alternative to hand-tuning conductance- based models: construction and analysis of databases of model neurons. Journal of neurophysi- ology, 2003.
[66] Astrid A Prinz, Dirk Bucher, and Eve Marder. Similar network activity from disparate circuit parameters. Nature neuroscience, 7(12):1345-133 1352, 2004.
[67] Michael Deistler, Jakob H Macke, and Pedro J Gonçalves. Energy-efficient network activity from disparate circuit parameters. Proceedings of the National Academy of Sciences, 119(44): e2207632119, 2022.
[68] Pierre Glaser, Michael Arbel, Samo Hromadka, Arnaud Doucet, and Arthur Gretton. Maximum likelihood learning of unnormalized models for simulation-based inference. arXiv preprint arXiv:2210. 14756, 2023.
[69] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44 (9):5149-5169 2021.
[70] Mike Wu, Kristy Choi, Noah Goodman, and Stefano Ermon. Meta-amortized variational inference and learning. In Conference on Artificial Intelligence, volume 34, pages 6404-6412, 2020.
[71] Ekaterina lakovleva, Jakob Verbeek, and Karteek Alahari. Meta-learning with shared amortized variational inference. In International Conference Machine Learning, pages 4572-4582.
PMLR, 2020.
[72] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[73] Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gregory Valiant. What can transformers learn in-context? A case study of simple function classes. Advances Neural Information Processing Systems, 35:30583-300 -35588, 2022.
[74] Madhur Panwar, Kabir Ahuja, and Navin Goyal. In-context learning through the Bayesian prism. In International Conference on Learning Representations, 2024.
[75] Sarthak Mittal, Niels Leif Bracher, Guillaume Lajoie, Priyank Jaini, and Marcus Brubaker. Amortized in-context bayesian posterior estimation. arXiv preprint 1rXiv:2502.06601, 2025.
[76] Arik Reuter, Tim GJ Rudner, Vincent Fortuin, and David Rügamer. Can transformers learn full Bayesian inference in context? arXiv preprint arXiv:2501. 16825, 2025.
[77] George Whittle, Juliusz Ziomek, Jacob Rawling, and Michael A Osbore. Distribution trans- formers: Fast approximate Bayesian inference with on-the-fly prior adaptation. arXiv preprint arXiv:2502.02463. 2025.
[78] Derek Qiang Xu, F Olcay Cirit, Reza Asadi, Yizhou Sun, and Wei Wang, Mixture of in-context prompters for tabular PFNs. In International Conference on Learning Representations, 2025.
[79] Junwei Ma, Valentin Thomas. Guangwei Yu, and Anthony Caterini. In-context data distilla- tion with TabPFN. In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024.
[80] Benjamin Feuer, Robin Sehirrmeister, Valeriia Cherepanova, Chinmay Hegde, Frank Hutter, Micah Goldblum, Niv Cohen, and Colin White. Tunetables: Context optimization for scalable prior-data fitted networks. Advances in Neural Information Processing Systems, 37:83430- 83464, 2024.
14

---

[81] Yuxin Wang, Botian Jiang, YiranGuo, Quan Gan, David Wipf, Xuanjing Huang, and Xipeng Qiu. Prior-fitter networks scale to larger datasets when treated as weak learners. In International Conference on Artificial Intelligence and Statistics, 2025.
[82] Han-Jia Ye, Si- Yang Liu, and Wei-Lun Chao. A closer look at TabPFN v2: Strength, limitation, and extension. arXiv preprint arXiv:2502.17661, 2025.
[83] Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. The tabular foundation model TabPFN outperforms specialized time series forecasting models based simple features. In NeurlPS 2024 Third Table Representation Learning Workshop, 2024.
[84] Junwei Ma, Apoory Dankar, George Stein, Guangwei Yu, and Anthony Caterini. TabPFGen tabular data generation with TabPFN. In NeurlPS 2023 Second Table Representation Learning Workshop, 2023.
[85] Jingang Qu David Holzmüller, Gaël Varoquaux, and Marine Le Morvan. TabICI: A tabular foundation model for in-context learning on large data. arXiv preprint (iXii::5502.055666 2025.
[86] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary Vetooo Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32, 2019.
[87] Omry Yadan. Hydra- a framework for elegantly configuring complex applications. Github, 2019.
[88] Benjamin Feuer, Niv Cohen, and Chinmay Hegde. Scaling TabPFN: Sketching and feature selection for tabular drior-data fitted networks. In NeurlPS 2023 Second Table Representation Learning Workshop, 2023.
[89] Tianhong Li, Dina Katabi, and Kaiming He. Return of unconditional generation: A self- supervised representation generation method. Advances in Neural Information Processing Systems, 37:125441
2024.
[90] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations. 2015.
[91] Michael Deistler, Kyra L. Kadhim, Matthijs Pals, Jonas Beck, Ziwei Huang, Manuel Gloeckler, Janne K. Lappalainen, Cornelius Schröder, Philipp Berens, Pedro Gonçalves, and Jakob H. Macke. Differentiable simulation enables large-scale training of detailed biophysical models of neural dynamics. bioRxiv, pages 2024-08, 2024.
[92] Christian Dietrich Weilbach, William Harvey, and Frank Wood. Graphically structured diffusion models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 36887-36909, PMLR, 2023.
[93] Balaji Lakshminarayanann Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, volume 30, 2017.
[94] Manuel Gloeckler, Michael Deistler, and Jakob H. Macke. Adversarial robustness of amortized Bayesian inference. In International Conference Machine Learning, volume 202, pages 1 11493-15524. PMLR, 2023.
[95] Pablo Lemos, Adam Coogan, Yashar Hezaveh, and Laurence Perreault- Sampling- based accuracy testing of posterior estimators for general inference. In International Conference on Machine Learning, pages 19256-19273. PMLR, 2023.
[96] In-Kwon Yeo and Richard A Johnson. A new family of power transformations to improve normality or symmetry. Biometrika, 87(4):954 959, 2000.
[97] Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, and David Duvenaud. Scalable re- versible generative models with free-form continuous dynamics. In International Conference on Learning Representations, 2019.
15

---

Appendix Table of Contents
Software and computational resources
17
Additional method details)
17
B
I
Pscudocode: NPE-PFN and TSNPE-PEN
17
B.2 Filtering to mecrasse effective context size
17
B.3
Embedding networks for high- nigh-dimensional data
18
B.4
Unconditional density estimation)
19
Experimental details
19
SBI benchmark simulators for amortized and sequential inference
19
Misspecification simulator
20
Single-compartment Hodgkin Huxley simulator
20
Pyloric Network Model
D Additional experiments
22
D.
I Additional tasks
22
SBI benchmark with more baseline methods
22
D.3
Inference speed]
23
D.4
Embedding networks for high- nigh-dimensional data
25
D.5
Comparison of autoregressive and ratio-based density evaluation
25
D.6
Varying the number of filtered simulations
27
D.7
Varying feature and noise distributions
27
D.8
Order of autoregressive sampling
28
D.9
Unconditional density estimation on the UCI datasets
29
16

---

A Software and computational resources
Our implementation of NPE-PEN is based PyTorch 86 and the TabPFN 38 library. We use the SBI library [54] to run the SBI baselines, and hydra [87] for experiment and hyperparameter management. Code use NPE-PEN and reproduce the results is available at httpss//giithbb
com//mackelabbnpee-ffff
We use mix of Nvidia 2080TI, A100, and H100 GPUs to obtain the results related to NPE-PFN. Some results, such as those for the unfiltered NPE-PEN variant together with simulation budget of 10 require the use of GPUs with high VRAM due to the large context used. SBI baselines were on 8 CPU cores, as normalizing flow training and sampling only benefit from GPUs for large sample sizes and dimensionalities.
B Additional method details
B.1 Pseudocode: NPE-PEN and TSNPE-PEN
In Alg. 1 and Alg. 2 we provide pseudocode for NPE-PEN and TSNPE-PFN, respectively. Note that NPE-PFN training-free. In practice, this means that training consists of storing the training data D [(0, | for in-context processing during inference.
Algorithm NPE-PEN
reg Require: Observation Co. simulations Diull ((0,, TabPFN regressor qe
:). filter
size Nitter
if Nutter < then
if filtering is active > compute filter distances
2:
d: = d( d(x,, xi)
3:
D (0; ; argtop (d:)
select Viittrr closest simulations
4: else
5:
D x Drull
6: end if
7: for 1..... dg do
for all parameter dimension dg build context data set
8:
D (0) 0 cj
mn)
9: gi q
cc ( 0J,, Co D
sample one parameter dimension from TabPFN
end for
11: return 0 = 10 02............ 0dol
posterior sample
B.2 Filtering to increase effective context size
To allow NPE-PFN make of more than 10ah simulations, we filter the simulations based on their relevance to given observation. In the following, we show that this procedure does not bias the estimation of the posterior and discuss details and possible extensions.
For a given prior p(0) and likelihood p( 0). the joint distribution p(æ, 0) p( (0) and the posterior distribution is given by
given by
p( ())(())
p(x, 0) p(æ, 0') do'
0( x) p(x)
(B-1)
Formally, a filter a neg-negetive function fa, (x) based on some observation æo, where we require fa, (x.,) > O. Crucially, reweighing the joint distribution by the filter does not affect the posterior. More specifically, with the reweighted joint p(x, 0) x p(x, (x, 0)fa. (x). we have
p(x.
fa. ( p(x.0(1000 0)
5
x))
[m, p(., 0¹) do p(0 x),
(B-2)
I T( O') do
fax, (x:) cancels due to its independence from 0.
In theory, any function satisfying these properties can be used filter for relevant simulations. In this work, only consider an ABC-like filter, where fa. (x) I(d(x, < e) is the indicator
17

---

Algorithm 2 TSNPE-PEN with rejection sampling
Require: Observation Vio simulator p(#: 0), prior p()), TabPFN classifier ( 1)), TabPFN
regressor (- (-), number of rounds R, number posterior samples for ratio M, simulations
per round Ny, truncation fraction
Initialize dataser
Draw N, prior samples 0: p(0)
2:
Simulate samples 2i ~ p(æ 0;)
3:
Initialize D {(0,, J
4: for L..... R. do
Construct fast vutio-based density estimator
5: Draw M posterior samples goest q(0 æo) using Alg. with D as context
6: Assign class
to 0g and = 0 to mm ~ ( mmax)
7: Define classifier dataset Das {00 j. yj ) combining @post and Ouni
3
8: With Dets as context, o% can be used to estimate posterior density:
46 2 0 Dats)
lcs (0s( x.)
(y 0, Das)
Estimate high density region (HDR) qs ( (0 x.) for j = ,,...,,,. M)
9:
> evaluate posterior densities
10:
ka = quantile(qs, ())
D estimate HDR threshold
Rejection sample from truncated prior
11:
D,
12:
while |D, < N, do
13:
p(0)
sample from prior >> accept if in HDR
14:
if (0 x,) > ka then
p(x 0
16:
D, + D, U (0,
end
18:
end while
D=
20:
end for
Sample final samples- Draw posterior samples 02 ~ {(0 xx)) using Alg. with D as context return {0:}s
22:
function that evaluates to 1, when some distance d(x xo) between the observation w, simulation is smaller than some threshold 0, and is 0 otherwise. As described in Sec. [..3 the Euclidean distance in feature-wist wise standardized observation space and choose adaptively to include the 10a closest simulations.
The autoregressive manner in which we use TabPFN for density estimation allows for more com- plex filter designs that consider not only the observation o.,, but also already processed parameter dimensions. In particular, is possible to select parameter-simulation pairs (0, æ) based on their relevance to the parameter dimensions <j and observation æ.. when using TabPFN to estimate p(0' Ro 05)) = qu (0i x, 053 D. We not investigate this approach here and leave the design of more complex (autoregressive) filters to future work.
B.3 Embedding networks for high-dimensional data
For NPE, it typical to employ an embedding net = (ar) depending on the case of high- dimensional data. The embedding network can be end-to-end trained together with the associated
18

---

conditional density estimator (6 fø (z)) and helps to identify sower-dimensionll "sufficient" statisties, which the density estimator can more effectively handle [5]. The embedding net usually chosen to follow existing properties of the data at hand; i.e., for image-like simulation, one might employ convolutional neural network.
In principle, this end-to-end approach is directly applicable to NPE-PEN, but it would require differentiation through the TabPFN model, which can be expensive. We instead aim obtain lower dimensional embedding, i.e., "summary features" independent from the pre-trained model. This idea similar feature selection schemes usually employed for TabPEN [5]
While various approaches exist for addressing this problem, we investigate the method proposed by Chen al. [50], which focuses maximizing the mutual information between the parameters and their corresponding embeddings e = fa()), with data x: ~ p( a 0). The goal is to derive embeddings that However, directly maximizing mutual information is generally intractable, leading to the development of proxy measures.
"sufficient" statistics for accurately inferring parameters from observed data.
We employ the distance-correlation proxy, which aligns the distances between embeddings with the distances between their associated parameters. Specifically, we optimize the following loss function:
Epin Oppla 0²) [d(0, 0*) fo
((0) EEperpior [((0, 0* Pplaspaa ( x:) ())2
(B-3)
We refer to this approach as NPE- NPE-PPNNInffmax and present results on using NPE-PFN together with pre-trained embedding nets in Sec. D.4
B.4 Unconditional density estimation
As outlined in Sec. 2.1 TabPFN is designed conditional density estimator for one-dimensional densities. For higher-dimensionall conditional density estimation, TabPFN can be applied autoregres- sively. To adapt TabPFN for unconditional density estimation, random Gaussian noise is added as the first dimension. The subsequent conditional density estimation is then performed in the same way as for conditional tasks [8
For conditional density estimation (e.g.. posterior estimation), filtering provides an effective method to optimize the context given the corresponding condition. However, this strategy does not generalize to unconditional density estimation. One solution is to manually introduce conditional structure. Specifically, partition the support of the target density into independent clusters and estimate each component separately. Formally, we express the target density as mixture model
p(z) = Σ p( p())
(B-4)
where p(c:) is the probability of sampling cluster Cs and p(æ | c4) is the cluster-specific density estimated via NPE-PFN. This mixture model approach effectively increases the usable context size with the number of clusters. In this work, we apply k-means clustering, which has been explored for TabPFN by Xu et al. [78] outside of density estimation. Other methods to introduce conditional structure, such as those proposed by Li et al. [89]. represent promising directions for future research. We present results on unconditional density estimation in Sec. D.9
C Experimental details
C.1 SBI benchmark simulators for amortized and sequential inference
To benchmark NPE-PEN and the sequential version TSNPE-PFI PFN, we use set of tasks from the SBI benchmark [27]. This benchmark contains several challenging inference tasks with nonlinear depen- dencies and multi-mode posteriors. We provide an overview the dimensionality of parameters and observations for all benchmark tasks (Tab. C-1) As described in Sec. 3.1 we measure the quality of the inferred posteriors with a classifier-twer wwosiee-tto-ssp---eeeeee (C2ST). For the classifier, we use a random forest with the default hyperparameters provided by the SBI library [54).
19

---

Table C-1: Overview on the SBI benchmark tasks. For details on the specification of the prior and the exact simulator equations, we refer to Lueckmann et al. 27 Appendix TT.
Task Name
dim
dim at
Notes
Gaussian Linear
10
10
Mean inference 10-D Gaussian with fixed covariance
Gaussian Mixture
2
2
Common mean two 2-D Gaussians with varying covariance
Two Moons
2
2
Bimodal, two moon moon-shaped posterior structure
SLCP
5
8
Simple Gaussian likelihood, complex posterior
Bernulli GI.M
10
10
GL.M with Bernoulli observations, smoothness prior
SIR
2
10
Epidemic STR model, inference contact and recovery rates Predator prey dynamics, species interaction parameters
4
20
The baseline methods NPE, NLE, NRE and their sequential variants are implemented using the SBI library. All methods use the default normalizing flows (neural spline flow 42 for NPE, masked autoregressive flow [41] for NIEE with hyperparameters suggested by the SBI library. Training was performed using the Adam optimizer [90] with a batch size of 200 and a learning rate of 5 10-4 Training was stopped early based the validation loss, evaluated 10% of the available simulations. For the NPE ensembles, five different estimators are trained with different random seeds to obtain mixture of equally weighted estimators.
held-out set containing
In all experiments, we use the default version the TabPFN classifier or regressor for (TS)NPE-PEH with changes to hyperparameters such as the softmax temperature. All runtimes for NPE-PFN (Fig. 22b) were obtained using an Nvidia A100 GPU, where possible. For the unfiltered variant of NPE-PFN, H100 GPU was used for the large context containing 10y simulations.
C.2 Misspecification simulator
Here, we detail the experimental setup of Sec. 3.2 The misspecification benchmark from Schmitt
et al. [57 is given by
Truth Prior:
I),
Likeli.: N(2, I)
(C-5)
Prior Prior:
μ N Im
Likeli.:
2 N(H,
(C-6)
Likeli.
Prior: ~~ NO,
Likeli.:
ABeta(2, 5) + (1- A)N Tm
(C-7)
where pm R as mean shift, Tm E g+ as standard deviation scaling, and A E [0, 1] as the fraction of how many samples are from the true normal or a different beta distribution.
For the experiments, we ran 100 simulations for three seeds over all combinations of (Am, Tm, A). NPE and NPE-PEN are trained to predict posterior samples. These samples are compared to reference posterior distribution using C2ST. This reference posterior distribution is defined with respect well-specified distribution. Thus, this task designed to evaluate the robustness the model under prior or likelihood misspecification.
C.3 Single-compartment Boodkkin-Huxlees simulator
To perform inference on the observations from the Allen cell types database 61]. compartment Hodkin-Huxley model following the one proposed in Pospischil et al. [59]. This model has four types conductances (sodium, delayed-rectifier potassium, slow voltage-dependent potassium, and leak) and total of eight parameters. Following Gonçalves et al. [60, who previously used this model for simulation- based inference, we compute seven summary statistics on the simulated and observed action potentials. These spike count, mean, and standard deviation of the resting potential, and the first four voltage moments (mean, standard deviation, skew, and kurtosis). We also use the same uniform prior in Gonçalves et al. [60]. For sequential inference with TSNPE and TSNPE-PFN, we use a total of 10 or 10 simulations, equally divided over five rounds. For both methods, sample from the truncated proposal using rejection sampling, taking into account the 1 - highest density region with 10-2
we use single-
Here, we provide posterior predictive samples for all 10 observations from the Allen cell types database using TSNPE-PFN (Fig. [-1TT and TSNPE (Fig. DD-12), complementing the results presented in the main text (Fig. 4h. Importantly, both inference and the computation of the average predictive
20

---

distance are performed on the summary statistics and not directly on the simulations or observations. Thus, features beyond those encoded by the summary statistics are not captured. For TSNPE-PEN, the posterior predictives closely match with the observations in most cases, with a few exceptions such as obs. 3 and obs. In contrast, flow-hased TSNPE yields posterior predictives that are less similar to the observations, with a few exceptions such as obs. 1 and obs. 8. These qualitative results further support the quantitative results reported in the main text, demonstrating that TNSPE-PF performs more reliable inference with fewer simulations.
C.4 Pyloric Network Model
A key challenge in this setting is the extreme sparsity of valid simulations. In particular, 99% of parameter samples from the prior yield implausible voltage traces, preventing the computation of summary statistics. For this reason, previous work has relied on millions of simulations for an amortized NPE-based posterior approximation (18 million in Gonçalves et al. [60]). The simulation count could later be reduced to million by restricting the prior valid simulations with additional classifier [67] Moreover, several sequential algorithms have been developed to tackle this problem. First, SNVI was proposed, which learns both a likelihood and a variational approximation of the posterior. Notably, the likelihood model was only based on "valid" simulations and corrected by a classifier that predicts whether parameters are valid. This approach made it possible produce good posterior samples with 3.5 - 10 simulations [18]. This number was reduced even further to only 1.5 10 simulations by S-UNLE, which made use of energy-based models and Langevin dynamic MCMC (68) Later, this approach was extended to SNPE approximations, specifically using TSNPE
Similarly, we want to use only valid simulations in the context of NPE-PEN and therefore adaptively truncate the prior using TabPFN classifier. This approach is based on the intuition that the posterior should place little or no probability mass on regions in parameter space that are likely to lead to invalid simulations 18 377 67.. Specifically, as simulations accumulate over rounds, randomly fill the context of the TabPFN classifier with up to 5 10 valid and invalid simulations. The prior is then truncated to Pva x I(P(valii) 0) > ()), where we choose 0.3. We then use TSNPE- PFN together with an approximate importance resampling scheme instead of rejection sampling for efficiency. Specifically, we oversample by factor of K 10 and obtain 10 samples, which are then resampled to 10 samples based on their importance weight w( (0) ( æ..). For evaluation, we exactly follow Glaser et al. [68] by computing the percentage of valid simulations as well as the energy score in each round. Note, however, that the overall experimental setting is not perfectly identical as we use a faster implementation of the pyloric network simulator based on Jaxley [91]
Previous sequential methods started with an initial simulation budget of 5- 10 prior simulations to acquire set of ~ 500 valid simulations. Due to the effectiveness of NPE-PFN on small simulation budgets, we start with only 5- 10 simulations, thus acquiring only 50 valid simulations in the first round. From there, we acquire 10x new simulations in each round, updating both NPE-PFN and the restricted classifier with the additional data. We stop after 45 rounds for a total of 10 simulations. Despite this limited budget, TSNPE-PFN quickly converges to a posterior that produces a high number of valid simulations that closely match the observed measurement. After the final round, we obtain valid percentage of 96.83% and an energy score of 0.0899 using only 5 10 simulations the starting point of the other methods-and still surpass their final performance (Fig. 55) The evaluation presented in the main text only examines the fidelity of the posterior predictives. Therefore, we include a visualization of the full posterior distribution after the final round (Fig. D-13).
which shows broad marginal distributions and similar features as the posteriors reported in previous work [8 60 68).
In terms of runtime, most of the computation time is spent on sampling from the restricted prior and running simulations. During the first few rounds, the process is faster (about minutes) due to the smaller context size in TSNPE-PFN. Once the context limit is reached, the time to propose the next 10s parameters stabilizes at about 7 minutes, using an Nvidia H100 GPU. Notably, the performance of previous state-of-the-art methods on this task is surpassed after only 10 rounds- an hour of runtime (including simulation which substantially faster than any previously proposed methods [18 668.
21

---

We choose the default order as the autoregressive order for sampling. The total number of possible permutations in this task is greater than 10 thus finding the optimal permutation in this space is highly challenging. To gain some insight, we perform an ablation study on 50 distinct random permutations using the context dataset identified by the final-round posterior. We compute the presented metrics with 10 simulations. We found that the energy score varies between 0.081 0.094 and the valid rates between 96% 98% (5% and 95% quantile each). These ranges match closely with the results reported in Fig. 5 and demonstrate that there is significant effect for the autoregressive order in this experiment. Appendix Sec. reports the same conclusion across all benchmark tasks.
D
Additional experiments
D.1
Additional tasks
Here we extend the SBI benchmark experiments (Sec. 3.1) with additional inference tasks from other scientific fields or interesting synthetic tasks used in previous work.
Specifically, the additional tasks include simulators from physics (Weinberg, Stellar Streams), puter science (M/G/1 queue) [16], as well as "structured" synthetic tasks (Tree, HMM) [21] Out of these, the stellar streams simulator stands out particularly computationally demanding (> 30 min. per simulation per CPU core; with two-dimensional parameters, and 199 dimensional data). For this reason, investigate the performance only up to 10 simulations.
The tasks in Hermans et [66] do not come with a ground truth posterior. We therefore evaluate them only via the average log posterior density (Fig. D-Ij). Overall, the results these additional tasks again demonstrate that, for a small simulation budget. NPE-PEN sigmificantly outperforms NPE. Specifically, NPE-PEN strongly outperforms NPE on tasks with sparse conditional dependencies, i.., Tree, HMM, Streams, in which it is one to three orders of magnitudes more anuullatio-effiiieee than NPE. The stellar streams task is good example use-case, where this efficiency is particularly beneficial: Attaining 100 simulations is feasible on a consumer- grnddde CPU (10 cores) in hours, and NPE-PEN achieves comparable performance NPE with 10a simulations, which would require 20 days. In addition, we investigated the calibration of NPE-PEN for which we observed a similar trend (Fig. D-Ip a subset of tasks). Finally, we performed direct comparison to reference posteriors where available. We also extended the HMM task to 50 parameter and data dimensions as an additional test for high-dimensional yet "structured" parameter spaces (Fig. D-Ie). Here, NPE-PFN generally outperformed NPE.
The HMM example specifically illustrates how NPE (or, more generally, conditional density estima- tion) fundamentally struggles with dimensionality, independent of the complexity of the simulation process. One approach to address this challenge is to incorporate suitable inductive biases or con- straints (such as known conditional dependencies) into the estimation network architecture [21 30 92) The inductive bias from TabPFN used in NPE-PFN seems be especially beneficial in such cases. TabPEN was trained exclusively on synthetic datasets generated from random structural causal models (SCMs), which by construction exhibit (sparse) conditional (in-)dependencies. We therefore hypothesize that pretraining on SCMs enables TabPFN to automatically detect and exploit such (in- Jueeddepeneseees directly from data. A detailed investigation of these mechanisms would be an interesting direction for future work.
D.2 SBI benchmark with more baseline methods
Here, we extend the SBI benchmark experiments (Sec. by including two additional baselines: neural ratio estimation (NRE) from Durkan et al. [4 and an ensemble of NPE models, denoted NPE (Ensemble) following Hermans et al. [16]. Ensemble models are known to improve predictive performance [93] and, in the context of SBI, to improve simulation-based calibration (SBC) [16) We evaluate all methods under the same experimental conditions as before (Sec. 3.I. using five equally weighted ensemble members for NPE (Ensemble).
We present results in terms of C2ST and SBC metrics (Fig. D-2 extending the results from the main text (Fig. [ For SBC, we use the Error of Diagonal (FoD) to quantify the deviation from perfect calibration. NRE performs similarly or worse on all tasks, while NPE (Ensemble) matches standard NPE in terms of C2ST and achieves the best calibration of all baselines. NPE-PFN is comparable to NPE (Ensemble) in terms of calibration on most tasks, while often outperforming it in
22

---

Weinberg
Streams
Tree
MG1
HMM
NLL
0.1
2.5
15
0.0
0.0
2
-2.5
10
10
10
10
10
10
10
10"
10"
10"
10"
10"
10
10"
10"
Num.
NPE
NPE-PFN
Weinberg
Tree
Streams
C
Tree
HMM
HMM[|5d]]
0.15
1.0
0.10
C2ST
0.05
0.5
10
10
10
10
10
10
10
10"
10"
10"
10
10
10
10 10
Num. sim.
Num. sim.
Figure D-1: Results on additional tasks. (a) Performance under the average negative log likelihood metric (NLL) across five new tasks for NPE and NPE-PEN. (b) Calibration error subset of the new tasks. (c) Performance C2ST against reference posteriors on tasks for which reference available. We also investigate different parameterization of the HMM tasks by increasing dimensionality. Note that the simulation budget of the Streams tasks only goes up to 10* simulations.
predictive accuracy and simulation efficiency. Calibration plots (Fig. D-3) corroborate the EoD-based calibration summary with fine-grained information.
These results underscore that NPE-PFN not only achieves strong predictive performance but also provides well-calibrated posteriors that match or exceed ensembles. While we do not directly compare NPE-PEN with the work of Delaunoy et al. [36), who present SBI method for small simulation budgets using Bayesian neural networks, their method reports modest gains over ensembles, whereas NPE-PFN consistently outperforms them-further emphasizing the effectiveness of our approach with a limited simulation budget.
Throughout this manuscript, the comparisons of NPE-PFN with baseline methods are performed using the shi library's default hyperparameters (55] Here, we additionally perform hyperparameter optimization of our main baseline method NPE. For each task and simulation budget, we conduct random search for flow (flow type, number of flow layers, dimensionality of hidden flow layers) and optimization (batch size, learning rate) hyperparameters, In each setting, we limit the computing time to ten hours. For seven tasks and four simulation budgets, this results in a total computing time of 280 hours.
Compared NPE with the default settings, NPE (Sweep) achieves better performance in some tasks, particularly with smaller simulation budgets (Fig. D-4). This is due to the shorter training times with 10 and 10 simulations, enabling the random search to cover large parts of the search space. For 10 and 10 simulations, no or only minor improvements can be observed. Further improvements for larger budgets would require more computing time and better hyperparameter optimization algorithms, such as Bayesian optimization. Despite the substantial computing time per setting, the optimized NPE (Sweep) still lags behind NPE-PFN in terms of performance for most tasks with smaller budgets (Fig. D-4). Overall, these results again demonstrate NPE-PEN's strong default performance, providing an easy-to-use method that competitive with, or even superior to, baseline methods where intensive hyperparameter optimization was performed. This strong default performance is particularly beneficial in the sequential setting. where hyperparameter optimization poses significant challenges due to long inference times, changing training conditions across rounds, and the difficulty selecting suitable metrics for optimization.
D.3 Inference speed
While NPE-PFN is completely training-free the autoregressive use of TabPFN requires computational effort during inference (Fig-2b) (p)). The inference speed NPE-PFN depends on three parameters: 1) the number of simulations passed as the in-context dataset; 2) the dimensionality of the observation space (i.e., the number of features); and 3) the dimensionality of the parameter space (i.e., the number of parameters).
23

---

Gauss.
Gauss. Mixture
Two Moons
SLCP
Bernuuli
Lotka Volterra
1.0
C2ST
0.5
0.2
SBC (EoD]
0.0
10" 10"
10"
10"
10' 10" 10t 10a
10" 10" 10 10
10" 10" 10" 10"
10" 10" 10" 10"
10" 10 10 10
10" 10" 10 10 Num. sim. Num. sim.
Num, sim.
Num. sim.
Num.
sim.
Num. sim.
Num.
sim.
NPE
NRE
NPE-PFN
NPE (Ensemble)
Figure D-2: Extended SBI benchmark results for amortized NPE-PFN. (a) Extension of Fig. with more baseline methods, namely NRE and NPE (Ensemble). (b) Mean absolute error between calibration curves and the diagonal (EoD), with 0 indicating perfect calibration. Full calibration curves in Fig. D-3
Gauss, Linear
Gauss. Mixture
Moons
SLCP
Beroulli GLM
Lotka Volterra
1.0
100
0.0
1.0
1000
0.0
1.0
10000
0.0
1.0
100000
0.0
0.0
1.0
0.0
1.0 0.0
1.0
0.0
1.0 0.0
1.0 0.0
1.0
0.0
1.0
a
NRE
NPE-PFN
NPE (Ensemble)
Figure D-3: Full calibration curves for all tasks. For each task (columns) and each simulation budget (rows), we plot the associated mean calibration curve (bold) as well as individual runs (transparent, thin).
Linear
Gauss.
Bernoulii
C2ST
1 10 10" 10
NPE
NPE-PFN
NPE (Sweep)
sim.
Figure D-4: SBI benchmark results with NPE hyperparameter optimization. C2ST for NPE-PFN, NPE with default settings, and NPE (Sweep), where hyperparameter optimization was performed. Specifically flow and optimization hyperparameters are optimized separately for each benchmark task and simulation budget using ten hours of random search.

---

Here, we provide a thorough examination of the inference speed of NPE-PFN across different simulation budgets and varying dimensionalities the parameter and observation spaces (Tab. D-2]. All results were obtained using an Nvidia A100 GPU. In general, as the value of any of the three parameters increases, inference becomes slower. The dimensionality of the parameter space strongest impact on overall inference speed, since the autoregressive use of TabPFN necessitates the reprocessing the adapted context for each additional parameter dimension. For the largest parameter and observation dimensions and context of 10* simulations, inference can take several minutes. However, this can be alleviated by filtering; for example, by selecting the 10s most relevant simulations (see Sec. D.6 for details).
the
Table D-2: Inference speed of NPE-PEN. Inference speed to sample a batch of 10* samples from the posterior measured in seconds (on Nvidia A100 GPU). Rows show increasing dimensionality of the parameter space, columns show increasing dimensionality of the observation space, stratified by different simulation budgets.
10a Sim.
10 Sim.
10 Sim.
0
2
8
16
32
2
4
16
32
4
$
16
32
2
6
4
5
6
7
6
9
8
7
8
10
14
23
11
10
10
11
13
10
11
11
17
15
24
32
19
30
23
24
66
101
16
45
47
47
53
63
49
52
54
77
89
99
116
152
221
32
97
103
107
122
137
115
116
125
141
171
251
272
307
370
516
D.4
Embedding networks for high-dimensional data
We evaluate the performance of PPE-PPNNInfoomse on high-dimensional simulation- baseed inference tasks, specifically focusing on a spatial SIR model [6] and an extended Lotka- Volterra task (94). In the case of the spatial SIR task, where the true posterior is intractable, we rely indirect metrics such as negative lig-likelihood (NLL), TARP [95], and SBC for evaluation. APE-PEN-Infomax leverages compressed embeddings learned via infomax objective distill key information from high-dimensional observations, which are then used as inputs to NPE-PEN.
For the spatial SIR task, we employ 2D CNN with four convolutional layers consisting of 16, 32, 64, and 128 channels, each with a kernel size of 5 and max pooling of size 2. The resulting feature maps flattened and passed through two-layer MLP to produce 10-dimensional summary statistic. For the extended Lotka-Volterra task (300-dimensional time series), we adopt a similar architecture using ID CNN with a kernel size of 3 and an output embedding of 16 dimensions.
Our results (Fig. D-5) indicate that such embedding networks wwether pre-trained or jointly trained can be used to extend the applicability of NPE-PFN to high-dimensional observations. However, in most cases, this approach does not outperform an end-to-end trained NPE baseline. Interestingly, TabPFN without an embedding network (when applicable) often achieves better performance, sug- gesting that the compression step, independent explanation is that TabPFN, having been trained on causally structured data, can exploit underlying dependencies to improve inference accuracy. In contrast, compressing such highly structured data into low-dimensional summaries may discard critical information, limiting the effectiveness of inference.
NPE-PEN, may be suboptimal. One possible
D.5 Comparison of autoregressive and ratio-based density evaluation
In Sec. 2.4 we present a method for fast density evaluation based on density ratios, requiring only a single forward pass through the TabPFN classifier. While this approach offers substantial computational advantage, it is inherently approximate, raising the question of how it compares to the autoregressive density evaluation. To answer this question, we compare the two methods on two tasks the 2D Two Moons tasks and the 1OD Gaussian Linear task-acra across varying simulation budgets. each case, we evaluate density of posterior samples from the trained model given an observation, using 5 random seeds. To construct the dataset for the ratio estimator, we draw 5 103 samples each from the posterior of interest and the uniform base distribution, which the default setting.
25

---

observation
posterior predictive
b
Spatial SIR (22x22)
0.20
SBC (EoD)
0.1
TARP
0.15
0.0
C
Spatial SIR (44x44]
Voltera (long)
TARP (EcD]
0.20
SBC (EoD)
0.1
C2ST
10ah 10010
10
10
10010010 10
0.5
10a 10
10a
Num. sim.
Num. sim.
Num. sim.
Num.
True NPE-PFN
NPE
NPE-PFN-Infmmax
Figure D-5: High-dimensional data with embedding nets. (a) Visualization of observations in the spatial SIR task alongside posterior predictive samples obtained using NPE-PEN- Prn-PF--Infomx... (b) Performance evaluation across nigh-dimensional variants of the spatial SIR task over independent runs, reported in terms of the negative log-likelihood (NLL) of the true parameter and the area off the Lotka-Volterra
diagonal TARP and SBC calibration analyses. (c) An instance of the Lotka- Volterra task with an extended time series (300 dimensions), evaluated using NPE-PEN and NPE-PE BEE-PFN-IMT Infomax.
a
Sample density
b
Sample density cor..
C
Repested density
corriuuoo,ram raluto ratiat
1.00
Two Moons
Time (sec)
0.50
200
Gaussian Linear
1.00
bllo-based
Time (scc)
20
0.75
auto ratio
10a
0.50
200
10
10"
10
autoregressive
Num.
Figure D-6: Comparison of density evaluation approaches on Two Moons and Gaussian Linear task. (a) Correlation between density estimates from the autoregressive and ratio-based approaches for two observations using 10* simulations. (b) Pearson correlation between the two density estimates across varying simulation budgets for both tasks. (c) Evaluation time (log scale) for increasing number of density evaluations using an Nvidia H100 GPU, showing the crossover point where the ratio-based approach becomes more efficient.
26

---

Gauss. Mixture
Two Moons
SLCP
Gauss. Linear
SIR
Bernouli GLM
Lotka Volterra
C2ST
0.5
10" Naar
10 10"
NPE-PFN
Figure D-7: SBI benchmark results with varying number of filtered simulations. C2ST perfor- mance of filtered NPE-PEN as function of the number of filtered simulations Niitter For all tasks, simulation budget of 10 is used. The red cross marks the default choice of Niiter 104
We observe a strong correlation between autoregressive and ratio-based density evaluations for the Two Moons task and a somewhat weaker correlation for the higher-dimensionall Gaussian Linear task (Fig. D for simulation budget of 10*). This trend persists across simulation budgets (Fig. D-6p). In termss of runtime, the ratio- based method has an initial cost due to posterior sampling, making slower for the first evaluation. However, once samples are obtained, subsequent evaluations are significantly faster-an an advantage that becomes very noticeable as the number of density evaluations increases (Fig. D-Gc; note the logarithmic y-axis).
Therefore, in scenarios that require repeated density evaluations for a single observation succ as rejection sampling in TSNPE-PFN-the ratio-based approach offers a highly favorable trade-off: provides sufficient accuracy at a fraction of the computational cost.
D.6 Varying the number of filtered simulations
When filtering simulations, we always make full use of TabPFN's recommended maximal context size of 10" data points. However, in principle, the number of simulations Nätter selected by the filter is a hyperparameter that can be optimized. Here, investigate the impact of this hyperparameter by evaluating the performance of NPE-PEN on the SBI benchmark tasks and a simulation budget 10 Specifically, we vary the number of filtered simulations Nrlterr setting to 16, 64, 256, 1024, 2048, 4096, and 16384, and compare the resulting performance to our default choice 10*.
For small values of Nitter, the performance of NPE-PFN deteriorates across all tasks (Fig. D-7p. In contrast, for Nitter 1024 or larger, performance very similar to that of our default choice with 10 simulations. For some tasks, slight decrease in performance is observed for the largest value of Nilters likely because this exceeds the recommended context size of TabPFN. Importantly, our default choice is always optimal or near-optimal. These results suggest that the recommended maximal context should be utilized fully. Nevertheless, these results also indicate that good performance can be achieved with smaller filter sizes (e.g., Nilter 2048) to reduce the computational load.
D.7 Varying feature and noise distributions
Here, we investigate the robustness of NPE-PFN to different types of noise. To evaluate the robustness in such cases, we construct a variant of the Gaussian linear task from Lueckmann et al. 27 with non-Gaussian features and noise
0 ~ Picat (0),
(D-8)
x + Puoise O).
(D-9)
Both Pieat (0) and Phoise (æ 0) are chosen from set of distributions with varying support and tail behavior:
Cauchy distribution Cauchy s),
Laplace distribution Laplace (0, s),
Logitnormal distribution aNV s2)) with (x) 146-a:
Normal distribution (0,
Student's ss t-distribution I5 & where 15 represents Student's t-distribution with degrees of freedom,
27

---

b
Laplace Festures, Cauchy Noise
5 dims.
10 dims.
1.00
NPE
NPE-PFN
C2ST
scale(x]-20 scale(M)11
Feature Dist.
couctry
1.00
0.75
NPE NPE-PFN
0.50
0.75
1.00
scale (noise)=1.0
yeariii student
scale (noise)=2.0
0.50
C25
0.75
cauctyy
egtiooma
abrdent
cauctty
legitromm
Ststand
0.50
10"
2
10"
10
10"
2- 10
Noise Dist.
Noise Dist.
Num. sim.
Num. sim.
Figure D-8: Robustness to feature and noise distributions. (a) Heatmaps showing C2ST perfor- mance for all combinations of feature and noise distributions with dim(0) 5, 1.0, and 10a training simulations. White cells indicate failed training. (b) C2ST across training set sizes for an example combination of distributions. These panels are zoom-ins from (a), highlighting the stability of NPE-PFN under varying distributional assumptions.
where is the scale parameter. We vary the feature dimension, the scale of feature and noise distributions, and the number of training simulations, and compare the performance of NPE-PEN against NPE.
We first evaluate performance via C2ST across all combinations of feature and noise distributions for Steatures Snoise 1.0, di 5, and 10x simulations. The heatmap shows that NPE-PEN outperforms NPE consistently, with while cells indicating training failure primarily for NPE (Fig. D 81). For the example of Laplace feature and Cauchy noise distributions, we provide results function of simulation budget. confirming the robustness of NPE-PFN across distributional shifts (Fig. D-8b). Notably, while NPE achieves performance drops on several non- -on-Gaussian configurations. NPE-PFN maintains stable performance.
These results suggest that, although TabPFN is pre-trained using priors based structural causal models (SCMs) with uniform or Gaussian root noise, nonlinear transformations in the SCM induce rich marginal distributions and allow generalization a wide range of distributions. NPE-PEN inherits this robustness, and its performance is largely invariant to different feature or noise distributions. Failures of NPE, especially with Cauchy features, are caused by instability in z-scoring. which NPE-PFN avoids through default preprocessing such as the Yeo-Johnson transform (96). Thus, standard NPE performance could probably be improved by applying appropriate transformations. Because NPE-PEN takes advantage of the automatic preprocessing performed within TabPFN, it provides reliable performance without manual preprocessing. making it robust and user-friendly solution in practical settings.
D.8 Order of autoregressive sampling
To sample from multi-dimensiona (conditional) distributions with TabPFN, we use it in an autore- gressive manner. An important question is whether the order in which sample the dimensions matters for, e.g., the quality of the inferred posterior distributions. To investigate this question, we rerun NPE-PEN on the benchmark tasks from Sec. 3. but permute the order in which sample the parameter dimensions. Two of the seven benchmark tasks are permutation invariant by design (Gaussian Mixture and Gaussian Linear), and replace these with tasks where distribution is constructed autoregressively. The first is a simple nonlinear task given by
[21, NoO,
(D-10)
91 ~ N(21,
(D-11)
y2 81 (Nsin((11 + x2), 1),
(D-12)
Va y1,92 ~ N + yn 1),
(D-13)
y4 Vi, 32, ~ N(viv2 + ya. 1).
(D-14)
28

---

Simple nonlin.
Mixed dis.
Two Moons
SLOP
Bernuuli GLM
SIR
Lotka Volterra
1.0
1.0
C2ST
0.5
0.5
10"
10
10
10
10
10"
10
10
10
10
10
10
10"
10
10
10
10
10
10
10
10
10
10
Num. sim.
Num. sim.
Num. sim.
Num. sim.
Num.
sim.
Num.
sim.
Num. sim.
permuted
default
Figure D-9: Autoregressive ordering. C2ST across varying simulation budgets for two additional synthetic tasks, and five tasks from the SBI benchmark that are not permutation invariant. Blue lines indicate the average C2ST accuracy over three seeds for up to ten (random) permutations of the sampling order. Dashed orange lines indicate the default order.
The second is mixed distribution task, given by
I [2i, x2] ~ (7-2,22
(D-15)
1/1 Gamma shape 1 + |cale 1),
(D-16)
1/ 1/1 ~ Uniform (0, 91 + [x2] )
(D-17)
V/ 1/1 1/2 Beta(a
31 3 = 2 1/2)
(D-18)
As Sec. 3.1
we compute the C2ST with respect to the ground truth for evaluation. We run NPE-PFN for the default autoregressive order (as defined in the SBI benchmark or in the equations above) and up to ten (random) permutations over three random seeds. Note that tasks with two or three dimensions have only two six possible permutations, respectively (including the default one), so random subsampling is not required.
Across all benchmark tasks and simulation budgets, NPE-PFN achieves nearly identical performance across the default and permuted orders (Fig. D-9). That is, for these benchmark tasks, the performance of NPE-PEN is not affected by permuting dimensions. These results show that NPE-PFN is not sensitive to the order which the parameters are sampled. As a result, users need not worry about providing NPE-PEN with "optimal" order. While differences in performance may be possible for very high dimensions or artificial examples, for the applications we consider here, an arbitrary permutation is sufficient.
D.9 Unconditional density estimation on the UCI datasets
Here, we apply TabPFN on some classical unconditional density estimation benchmark tasks from the UCI repository [56] used in several other works [41 42 197. Note that while still perform density estimation here, it does not involve posterior distributions. Thus, we do not refer to this approach NPE- PFN. Specifically, These tabular datasets range in dimensionality from to 43 features and contain between 3.1 1 and over 1 million samples. We here investigate the unconditional density estimation performance of TabPFN in the low sample regime in comparison to a neural spline flow (NSF) [42] As in previous works, compute the negative log- log-likeldhood (NIL.) under held-out test set. While evaluate the full test sets, we only use 103, 10a, or (if applicable) 10 samples for training. For the two larger settings, we use partitioning (Sec. B.4 with 10 clusters when estimating densities with TabPFN.
consider the Gas, Power, Hepmass, and Miniboone datasets.
For 102 and 10 samples, TabPFN achieves a smaller negative ligeliiel on all four datasets (Fig. D-10). Similarly, for the 10* samples, TabPFN outperforms NSF on all but the power dataset. Interestingly, for the lower dimensional datasets, clustering further reduces the NLL despite having access to the same total number of samples. On the other hand, for the higher dimensional datasets, clustering increases the NI.L. For 10 samples (or 3.1 10 samples for the Miniboone dataset), clustering improves the NI.L over over any 10 sample setting because it allows TabPEN to access larger number of samples. However, NSF performs better than or equal to TabPFN on all datasets except the Gas dataset at 10t samples.
These results suggest that TabPFN a capable (unconditional) density estimator, especially in the low sample regime.
29

---

Gas
Power
Hepmass
Miniboone
60
5
30
NLL
25
-5
20
-10
20
10
10
10
10
10
10
10
10"
10"
10
10"
3-10"
Number samples
Number of samples
Number samples
Number samples
NSF
TabPFN, 1 cluster
TabPFN, 10 clusters
Figure D-10: Results for unconditional density estimation on the UCI datasets. Negative-log- likelihood (NLL) for TabPFN (with and 10 clusters) and the neural spline flow (NSF) across the different UCI datasets. Dots indicate averages, and bars show standard deviation over five independent runs.
30

---

Obs. 1
Obs. 2
Obs. 3
Obs. 4
Obs. 5
Obs. 6
Obs. 7
obs. 8
obs. 9
Obs. 10
Figure D-11: Posterior predictives of TSNPE-PFN for real observations from the Allen cell type database and a simulation budget of 10ah Note that the inference is not performed directly on the action potential time series but on seven summary statistics computed from it.
31

---

Obs. 1
Obs. 2
Obs. 3
Obs. 4
Obs. 5
Obs. 6
Obs. 7
obs. 8
mm
obs. 9
Obs. 10
Figure D-12: Posterior predictives of the TSNPE baseline for real observations from the Allen cell type database and simulation budget of 10a, Note that the inference is not performed directly on the action potential time series but on seven summary statistics computed from it.
32

---

0.0 500
AB-Na
0.0
AB-CaT
0.0
AB-CaS
0.0
60
AB-A P
0.0
AB-KCa
25 150
AB-Kd
0.0 0.0
AB+
0.0 0.01
A8 A8kk
0.0 200
LP-Na
0.0
2
LP-CaT
2
12
IP CaS
10
60
LP-A
0.0
10
LP
0.0
125
LPdd
0.0 0
LPH
0.0
0.0
LP eak
0.0
600
PY-Na
0.0
12
PY CaT
0.0
PY CaS
30
60
PY-A
0.0
PY KCa
50
150
PY-kd
0.0
0.1
PY
0.0 0.1
PY-leak
0.01 10000
AB
LP
0.01 1 0000
PD LP
0.01 1000
A8 PYr
0.01 1000
PD PY
0.01 1000
LP PD
0.01 1000
UP
0.01 1000
PY
Figure D-13: Pyloric simulator. Posterior distributions for all 31 parameters estimated by TSNPE-
PFN.
33

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state the methodological and empirical contributions of our work: We introduce NPE-PF, training-fee inference method that uses the pre-trained TabPEN v2 tabular foundation model for SBI. We demonstrate its advantages in accuracy and simulation efficiency on synthetic benchmark tasks and real-world tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In the final section, we explicitly discuss the limitations of our approach, including the constraints imposed by TabPFN, such as context size limitation, scalability, and computational cost.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
34

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: Our work is methodologically and empirically driven. Thus, it does not contain significant formal theoretical results. Instead, we validate the method through extensive empirical evaluation on both synthetic benchmark tasks and real-world tasks.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We detail all components necessary for our results, including simulation setup, model configuration, and evaluation metrics, in appropriate subsections for each experiment within the appendix. In addition, we provide our code reproduce all experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
35

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We share the code and provide detailed instructions on how use our method. The datasets we use are publicly available, and we include instructions for using the data in the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//iips.cc/pull wttpccpppppppppeeeeeeeeeeeeeeeee
htw.l////ggiidee//edduuubiisssss Cooeeeoomdssssiiieccea Polooic for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We report all relevant experimental details, including simulator setups, simula- tion budgets, and hyperparameters, in the appendix. Reproducible configuration files are included in the code repository to ensure transparency and applicability.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
36

---

Justification:
We report the standard deviation over independent runs with different random seeds. The number of independent runs given for each experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
the Appendix, we describe the computational resources used for our experi- ments, including hardware specifications. For the experiments, we report the runtime, e.g.. for the SBI benchmark tasks for comparison with baseline methods.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We confirm that this work conforms to the NeurIPS Code of Ethies in all
aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
37

---

10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work introduces methodological advance in simulation-base inference using pre-trained foundation models. While it allows for more efficient inference in scientific settings. potentially enabling scientific advances, we do not target applications with direct societal impact and do not foresee immediate potential for misuse. Regarding the use of pre-trained foundation models, the model we use, TabPFN, has been pre-trained on synthetic datasets only. Thus, there is no personal data involved that could bias the output of our method.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The work is methodological and empirical. Therefore, we do not see a high risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
38

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes)
Justification: We credit external assets our work, including datasets, Licenses for models, simulators, and datasets are properly respected and appropriately cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide the code for our method with documentation how it. No other assets are introduced.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve any crowdsourcing or research with human experts.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
39

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No human subjects were involved in this study, and no new data involving human participants were collected or analyzed.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
The core methodology and contributions of this work do not involve the use large language models. LLMs were not used the development, implementation, or evaluation of the proposed method according to the NeurIPS LLM policy.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
hattoe//nearrps.cc//offfeeerreee
40


Reviews:
[{"review_id": "hjDRJGkZi7", "reviewer": "Reviewer_Vbbm", "summary": "The paper introduces NPE-PF, a simulation-based inference method leveraging\nTabPFN as a pre-trained density estimator. The use of a tabular foundation\nmodel in the SBI pipeline enables training-free density evaluation, and builds\non useful prior knowledge from pre-training that can lessen the simulation\nburden required for producing accurate posterior approximations.", "questions": "1. In regards to the point about inheriting TabPFN's properties: did you\n   encounter any particularly surprising failure modes or poorly represented\n   dynamics in your experiments with TabPFN? While TabPFN is a proven and\n   generally capable model, it would be interesting to characterize when/if it\n   systematically struggles under certain dynamics.\n2. In Figure D-7, permutations in the dimension have little effect on the performance. This was calculated over a set of the SBI benchmarks, with a maximum of 10-dimensional parameters. Was this tested on the 31-D pyloric network example? How large of an effect might this have on even higher-dimensional inference tasks?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is well-organized, includes a comprehensive literature review, and\n  features clear figures and diagrams.\n- Incorporating larger (foundation) models in SBI pipelines presents an\n  interesting opportunity for NPE workflows, and has not been sufficiently\n  explored in the literature to my knowledge. Further, while the idea of\n  \"swapping in\" a foundation model sounds simple on its face, the methodology\n  is carefully considered and attends to many important practical\n  considerations in its construction (e.g., employing efficient ratio-based\n  evaluation, adopting TSNPE, studying robustness to misspecification, etc).\n- The reported performance evaluation is convincing, comparing the proposed\n  method to relevant baselines on several common SBI tasks with clear\n  advantages across many of the explored settings.\n\n**Weaknesses**\n\n- Provided the differences in sampling costs between NPE-PF and flow-based NPE\n  methods, a more comprehensive analysis of computational costs (especially in\n  high-dimensional settings) would be helpful. In particular, it would be\n  practically useful to clarify the boundary beyond which one might consider a\n  computationally faster method (in total), even with NPE-PF's possible sample\n  efficiency gains and benefits of no training.\n- The evaluation of the method on real-world environments is somewhat limited.\n  While the Hodgkin-Huxley and pyloric network models are challenging tasks\n  outside the SBI benchmark suite [1], evaluation on inference tasks from a\n  wider range of scientific disciplines would better position NPE-PF's\n  general utility.\n- A principled analysis of NPE-PF's diminishing advantages over standard NPE\n  methods under increased simulation burdens would be useful to include. At\n  what upfront sample size is TabPFN only minimally comparatively informative\n  (or even a burden)? How does this change with the dimensionality of $\\theta$\n  and/or $x_o$? How does a well-selected embedding network shorten the\n  runway of the performance advantage? Characterizing these practical\n  considerations would be useful in determining NPE-PF's feasibility for use in\n  larger-scale SBI tasks. \n\n[1] Lueckmann, J. M., Boelts, J., Greenberg, D., Goncalves, P., & Macke, J.\n(2021). Benchmarking simulation-based inference. AISTATS.", "was_revised": true, "final_justification": "During rebuttal and discussion, the authors addressed many of my initial concerns by providing additional experimental results and discussions. This included several new inference tasks to further support the method's evaluation, along with discussions regarding its computational cost and understanding when it should be used over other methods in practice. I otherwise found the empirical results to be satisfactory, and the idea (if *somewhat* lacking in novelty) to be well-executed.\n\nIn total, I'm happy with the paper's (current or planned) experimental validation, presentation, and discussion of limitations. As a result, I maintain my initially positive inclination toward acceptance.", "rebuttal": "We thank the reviewer for finding our work “well‑organized,” for noting that our “methodology […] attends to many important practical considerations,” and for deeming our “performance evaluation […] convincing.”\n\nBelow, we first address the reviewer’s main concerns and provide a more detailed analysis of the new results developed to address these issues (**A1**, **A2**, **A3**). We then offer detailed responses to the reviewer’s specific questions (**Q1, Q2**).\n\n- **Inference tasks on a wider range of scientific disciplines:** We agree that extending our evaluation suite to include simulators from additional fields (e.g., physics) is valuable. We have therefore added several new tasks. In **A1**, we analyze these new tasks in depth and will integrate the expanded results into the revised manuscript.  \n- **Principled analysis of NPE‑PF’s diminishing advantages:** Formulating a general proposition in this regard is challenging, because it is very specific to the task and/or the application at hand. But, we believe that our (now even more) comprehensive empirical investigation of NPE-PF’s performance in various situations (tasks, dimensionalities, simulation budgets, etc.) constitutes a principled empirical examination of NPE-PF’s advantages (**A2** for details).  We will improve the discussion on this topic within the manuscript (e.g. more concrete statements, when we expect it to work well).\n- **Computational costs:**  We agree that a more precise discussion of runtime trade-offs is needed. In **A3**, we present an ablation study examining the impact of context size and data and parameter dimensionality.\n\n**A1. Additional inference tasks**\n\nThe new tasks include simulators from physics (Weinberg, Stellar Streams), computer science (M/G/1 queue) [1], as well as “structured” synthetic tasks (Tree, HMM) [2]. Out of these, the stellar streams simulator stands out as particularly computationally demanding (>30 min. per simulation per CPU core; 2dim parameters, 199dim data). The tasks in [1] do not come with a ground truth posterior. We therefore evaluate them only via average log posterior density (see Table V1 for NNL metric) as well as calibration error (not shown here due to space constraints). We also extended the HMM task to 50 parameter and data dimensions as an additional test for high-dimensional, but “structured” parameter spaces (see Table V2 for C2ST metrics). \nOverall, our results for the new tasks are consistent with our previous findings. Across all tasks, NPE-PF significantly outperforms NPE at lower simulation budgets (Table V1/V2). Furthermore, NPE-PFperforms well on larger simulation budgets e.g. Tree, HMM or Streams (Table V1/V2). We will include these results in an updated version of the manuscript.\n\n**Table V1: Extended task benchmark.** Average negative log posterior density of the true parameters on 1000 test simulations. Each run was repeated five times and we report the average ± SD over runs. The better result is marked in bold.\n| Method | Weinberg | Streams | Tree | HMM | MG1 |\n|--------|----------|---------|------|-----|-----|\n| 100 sim. |\n| NPE | 0.12 ± 0.06 | 5.50 ± 0.18 | 4.58 ± 0.03 | 17.29 ± 0.11 | 3.75 ± 0.59 |\n| NPE-PF | **0.06 ± 0.01** | **3.08 ± 0.08** | **2.60 ± 0.09** | **11.44 ± 0.15** | **-0.26 ± 0.09** |\n| 1000 sim. |\n| NPE | 0.03 ± 0.06 | 3.85 ± 3.08 | 2.79 ± 2.60 | 13.63 ± 11.44 | 0.75 ± -0.26 |\n| NPE-PF | **0.02 ± 0.00** | **2.43 ± 0.09** | **2.19 ± 0.04** | **10.25 ± 0.07** | **-1.42 ± 0.08** |\n| 10000 sim. |\n| NPE | **0.01 ± 0.00** | 2.82 ± 0.08 | 2.18 ± 0.08 | 10.78 ± 0.06 | -0.60 ± 0.41 |\n| NPE-PF | 0.02 ± 0.00 | **2.21 ± 0.10** | **2.11 ± 0.04** | **10.02 ± 0.06** | **-1.88 ± 0.07** |\n| 100000 sim. |\n| NPE | **0.00 ± 0.00** | too expensive | **2.09 ± 0.07** | **9.99 ± 0.07** | -1.46 ± 0.49 |\n| NPE-PF | 0.02 ± 0.00 | too expensive | **2.09 ± 0.05** | 10.06 ± 0.04 | **-2.24 ± 0.10** |\n\n\n**Table V2: Extended C2ST evaluation.** Average C2ST (± SD over 5 runs) on new synthetic tasks using 100 random observations.\n| Method | Tree | HMM [10d] | HMM [50d] |\n|--|-|--|----|\n| 100 sims |\n| NPE | 0.87 ± 0.02 | 0.97 ± 0.00 | 1.00 ± 0.00 |\n| NPE-PF | **0.64 ± 0.01** | **0.77 ± 0.01** | **0.99 ± 0.00** |\n| 1000 sims |\n| NPE | 0.68 ± 0.02 | 0.91 ± 0.01 | 0.99 ± 0.00 |\n| NPE-PF | **0.56 ± 0.01** | **0.62 ± 0.01** | **0.77 ± 0.00** |\n| 10000 sims |\n| NPE | 0.57 ± 0.01 | 0.70 ± 0.01 | 0.97 ± 0.00 |\n| NPE-PF | **0.54 ± 0.00** | **0.56 ± 0.01** | **0.66 ± 0.00** |\n| 100000 sims |\n| NPE | **0.54 ± 0.00** | 0.59 ± 0.00 | 0.89 ± 0.00 |\n|NPE-PF|**0.54 ± 0.00**|**0.54 ± 0.00**|**0.65 ± 0.00**|\n\n\n**A2. Analysis of NPE-PF's diminishing advantages.**\n\nFirst, the question at which “upfront sample size is TabPFN only minimally comparatively informative (or even a burden)” is, to a certain degree, task-specific. NPE can outperform NPE-PF after $10^3$ simulations (e.g. Gauss. Mixture) or require more than $10^5$ simulations (e.g. Lotka Volterra). We evaluate NPE-PFs performance on a variety of tasks with different dimensionalities to give insights into its behaviour under various conditions.\n\nSecond, whether NPE-PF becomes a burden also depends on the use case or type of inference task. If inference needs to be performed on thousands of observations, more time should be allocated to simulation and NPE should be chosen. If inference on only a few observations (or many different prior/simulator configurations) is required and/or the simulator is expensive, then NPE-PF is a great alternative. The stellar streams simulator is a good example of this. Attaining 100 simulations is feasible on a consumer-grade CPU (10 cores) in 5 hours, and NPE-PF achieves a comparable performance to NPE with 10,000 simulations, which would require 20 days (Table V1). In other words, NPE and NPE-PF serve different purposes.\n\nThird, we agree that NPE performance can be improved with a “well-selected embedding network”. However, training such a network requires a large amount of simulations. We discuss this situation in Appendix D.3. Specifically, in Fig D-4c, we compare NPE-PF with NPE using a tailored 1D CNN embedding net (Lotka Voltera [long] task with a 300 dim. time series), where NPE-PF still outperforms NPE. \n\nIn summary, we believe that our experiments, both new and previous, conducted in various settings, constitute a \"principled analysis of NPE-PF's diminishing advantages over standard NPE methods\". Furthermore, we have already addressed several of the aforementioned “practical considerations” in the Appendix. However, we acknowledge that we did not communicate some of these aspects effectively throughout our manuscript and will address this in an updated version.\n\n**A3. Analysis of computational cost.**\n\nWe agree with the reviewer that providing a comprehensive evaluation of the overall runtime is important. For this reason, we have included an overview in the main manuscript (Fig. 2b). However, we also agree that this is insufficient for assessing the \"boundary beyond which one might consider a computationally faster method\". To address this, we have now added a comprehensive runtime ablation section on data dimensionality and simulation size for NPE-PF.\n\nFor results on runtime across different scenarios we refer to the response to Reviewer wFAY Table w2. To summarize the runtime analysis for NPE-PF: \n- The compute time grows with an increasing number of simulations and parameter dimensions, as expected.\n- In lower dimensional cases the compute time to acquire a 10k posterior samples ranges between 4-23 seconds across all context sizes.\n- In higher dimensional cases the compute time can grow to 1-2 minutes for context-sizes of 100/1000 and up to 8 minutes for 10000.\n\nWhile these inference times are substantially slower than standard NPE, the choice between both methods requires careful consideration of various factors, as discussed in **A1**.\n\n\n**Q1. Inheriting TabPFN properties.**\n\nWe appreciate the reviewer’s suggestion to \"characterise when, or if, NPE-PF struggles systematically\". Indeed, one of our original objectives was to identify configurations that might present a challenge to NPE-PF. To this end, Section D.5 examines performance under extreme, heavy-tailed noise models. Across all noise scenarios that were tested, NPE-PF proved to be remarkably robust.\n\nAlthough we did not identify any systematic failure modes, we found that NPE-PF performs particularly well on simulators with an underlying graphical structure and conditional dependencies. We attribute this to the fact that TabPFN was pre-trained using random structural causal models (SCMs). See the response to Reviewer wFAY (**Q3**) for a more detailed discussion.\n\n\n**Q2. Parameter permutations in 31D example.**\n\nTesting the reliability of NPE-PF in high-dimensional spaces is inherently challenging. For the 31dim pyloric network, the total number of possible permutations of parameter dimensions is greater than $10^{33}$. Moreover, evaluating each candidate permutation requires expensive simulation, making exhaustive exploration impractical. To gain insight despite these constraints, we performed an ablation study on 50 distinct random permutations using the context set identified by the final‑round posterior. We compute the presented metrics with 1000 simulations. We found that the energy score varies between 0.081 - 0.094 and the valid rates between 96% - 98% (each the 5% and 95% quantile). This is in line with what we found using the default permutation in the paper (Appendix D.6, Fig. D-7). We note that these results must be interpreted with caution. There may be permutations for which NPE-PF performs significantly better or worse. However, identifying these permutations is very difficult due to the large number of possible permutations.\n\n[1] Hermans et al. \"A trust crisis in simulation-based inference? Your posterior approximations can be unfaithful\", 2021  \n[2] Gloeckler et al. \"All-in-one simulation-based inference\", 2024"}, {"review_id": "PXXgckhSjR", "reviewer": "Reviewer_wFAY", "summary": "The paper explores the use of in-context-learning (ICL) in SBI settings. It employs TabPFN as backbone model, avoiding the use of inference networks, for instance, in NPE (neural posterior estimation). The proposal employs a sequential inference step to handle models with multiple parameters and adopts nearest-neighbor-based filtering to overcome the limited context size of TabPFN. Experiments on synthetic and real data demonstrate that the proposal is more sample-efficient (in terms of number of simulations) and robust to model misspecification than baseline frameworks (e.g., NPE, NLE).", "questions": "1. Could the authors elaborate on the sensitivity of the performance to some modeling choices, e.g., number of neighbors? \n2. What is the computational impact of the proposal? How much does it improve over standard NPE, for instance?\n3. On which datasets is TabPFN pre-trained? Could these datasets say something about settings (problems) in which the proposal work?\n4. What makes the TabPFN model the best foundation model for SBI? For instance, how does the method in [1] perform? \n\nPlease, see \"Weaknesses\" for additional questions.\n\n[1] TabICL: A tabular foundation model for in-context learning on large data?, ICML 2025.", "limitations": "The paper includes a paragraph \"Limitations\". However, I believe the paper should better explore the limits of using TabPFN (or ICL in general) for SBI (see Weaknesses).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- Overall, the empirical results are promising and significant.\n- This is the first paper to explore ICL approaches in the context of SBI. \n- The paper is well-written and relatively easy to follow.\n\n\n**Weaknesses** \n- The paper does not provide in-depth insights into when and why the ICL approach works across different tasks or simulators, limiting the contribution to the community. Is TabPFN expected to beat NPE in all problems? What are the limits of TabPFN? Is there any relation between the datasets on which TabPFN was trained and the SBI tasks? The paper briefly mentions \"modeling complex parameter spaces—such as spatially varying or continuous parameters—remains challenging\". Where does this conclusion come from?\n\n- The paper aims for an \"effortless\" SBI approach. However, the paper does not mention how much time the proposal saves compared to the baselines. I believe the paper lacks a comparative study from a computational (time) perspective. In practice, how many hours would a practitioner save by using the proposed approach?\n\n- The paper would also benefit from an \"ablation-like\" study. For instance, how does the performance change as a function of the number of neighbors N? How many samples are filtered (on average)? This could provide more insights into how TabPFN works and its limits. \n\n- From a technical perspective, the paper brings little to the table. In particular, the application of the ICL approach is straightforward, and the solution used to overcome the limited context size  (i.e., nearest-neighbor-based filtering) is not novel.", "was_revised": true, "final_justification": "The paper tackles a relevant problem (sample efficiency) in SBI using foundational models. The empirical results are promising and significant. During the rebuttal, the authors provided clarifications regarding TabPFN and presented additional results that alleviated some of my initial concerns. Overall, I am leaning towards acceptance.", "rebuttal": "We thank the reviewer for acknowledging that “the paper is well-written”, and that “the empirical results are promising and significant”. We also appreciate the recognition that ours is the “first paper to explore ICL approaches in the context of SBI”. \n \nIn response to the reviewer’s suggestions, we have:\n- Conducted experiments across a broader range of filter sizes (Table w1).\n- Added a rigorous computational time comparison (Table w2).\n- Introduced several new, more challenging inference tasks to cover a larger range of tasks.\n\nBelow, we first address the reviewer’s main concerns and then provide detailed answers to their specific questions (**Q1–Q4**).\n\n- **Technical novelty of contribution:**  Our paper tackles two central challenges in SBI: the high simulation cost of traditional methods and the expertise required to train conditional density estimators. Our method, NPE-PF, provides a novel and elegant way to alleviate both of these issues. Furthermore, TSNPE-PF uses new ideas for fast in-context density estimation with classifier-based density ratios, and shows strong performance in real-world inference tasks (Fig. 4/5). To our knowledge, our paper hence introduces novel methods to the SBI literature that address problems relevant to the community, regardless of any perceived lack of technical complexity. Instead, we view this simplicity as a strength, lowering the barrier for adoption by researchers and practitioners.\n- **No in-depth insights into when and why ICL works:** The reviewer states that our work “does not provide in-depth insights into when and why the ICL approach works” and asks “Is TabPFN expected to beat NPE in all problems?” While we did not test on *all* problems, we did test on a considerable number of problems and now further increased this number (results in Table V1/V2 in response to Reviewer Vbbm). All these results support our main claim that NPE-PF outperforms NPE on small simulation budgets, and is competitive for large ones. Yet, we agree that the manuscript could be improved by providing a more elaborate interpretation and conclusion of these results to better communicate the insights into when and where ICL likely works well (see **Q1-Q4** for details).\n \n\n\n**Q1. Ablation on number of neighbours for filter.**\n\nWe thank the reviewer for their suggestion. We have conducted an experiment on all SBI benchmark tasks using 100k simulations and varying filter sizes (16, 64, 256, 1024, 2048, 4096, 16384) in comparison to our default choice of 10000, which is based on the TabPFN recommended context limit. Results are in Table w1.\n\n\n**Table w1: Ablation on filter size.** Average C2ST for each task (across five seeds) when varying the filter size. Standard deviations across seeds are all below 0.01, and are not reported individually. Each on a dataset of 100 000 simulations.\n\n| Filter Size | Lotka Volterra | Gaussian Mixture | Two Moons | SLCP | Gaussian Linear | SIR | Bernoulli GLM |\n|-------------|----------------|------------------|-----------|------|------------------|-----|----------------|\n| 16 | 0.96 | 0.70 | 0.74 | 0.91 | 0.77 | 0.60 | 0.85 |\n| 64 | 0.89 | 0.62 | 0.63 | 0.86 | 0.64 | 0.55 | 0.72 |\n| 256 | 0.77 | 0.57 | 0.58 | 0.82 | 0.56 | 0.52 | 0.64 |\n| 1024 | 0.71 | **0.55** | 0.56 | 0.81 | 0.53 | **0.51** | 0.60 |\n| 2048 | 0.69 | **0.55** | 0.56 | **0.80** | 0.53 | **0.51** | 0.59 |\n| 4096 | **0.68** | **0.55** | **0.55** | **0.80** | **0.52** | **0.51** | 0.58 |\n| 10000 | 0.69 | 0.56 | **0.55** | 0.81 | **0.52** | **0.51** | **0.57** |\n| 16384 | 0.71 | 0.56 | **0.55** | 0.82 | **0.52** | **0.51** | **0.57** |\n\nAcross all experiments, performance decreases noticeably with small filter sizes. Large filter sizes (4096 and 16384) perform similarly to our default choice of 10,000. Importantly, our default choice is always optimal or near-optimal. These results suggest that the available context (as recommended by TabPFN) should be used fully and that little performance benefit can be gained by moving away from the default choice (see also Fig 2. “NPE-PF (unfiltered)”). Nonetheless, these results also suggest one can obtain good performance with smaller filter sizes to reduce the computational load (> 2048).\n \n**Q2. Analysis of computational time.**\n\nBeyond the timing plots in Fig 2b, we now perform additional runtime experiments (using an Nvidia A100 GPU) varying the number of simulations, and the dimensionality of parameters and simulations/observations (Table w2). As expected, the inference time mainly depends on the number of simulations in context and the parameter dimension.\n\nIn general, an answer to the question “How many hours would a practitioner save?” depends on many different factors:\n1) First, we discuss amortized inference in an example setting. With 100000 simulations (8 dim parameters, 8 dim data), NPE typically takes around 60–90 minutes to train. After training, inference will be effectively instantaneous (Fig. 2b). By contrast, generating 10000 posterior samples for a given observation using NPE-PF takes ~50 seconds (Table w2). Therefore, after sampling posteriors for around 90 observations, the training time of NPE amortises. However, this holds only assuming that one ignores the time necessary to obtain the simulations. For very expensive simulators, obtaining simulations becomes the bottleneck and the simulation efficiency of NPE-PF will more than compensate for the slower inference time. Therefore the appropriate choice of method crucially depends on the application and simulator.\n2) Second, with sequential inference the situation changes. When using flow-based TSNPE, many networks need to be trained per observation, so fast sampling becomes less relevant. For instance, in the experiment in Section 3.4, both TSNPE and TSNPE-PF require approximately one hour with a budget of 10000 simulations (here, the simulator is fast and the simulation time is negligible). However, TSNPE-PF is much more efficient in terms of simulations, achieving similar performance with just 1000 simulations and an inference time of 10 minutes. This effect would be amplified for a more expensive simulator.\n\nWe will add the timing results and above discussion to our manuscript to allow users to make an informed choice between NPE-PF and NPE.\n\n\n**Table w2: NPE-PF runtime (seconds)** for different simulation counts and parameter/data dimensionality. Each row represents varying parameter dimensions ($\\theta$-dim), each column varying data dimensions (x-dim). \n||100 sim.||||||1000 sim.||||||10000 sim.||||||\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|$\\theta$/x-dim||**2**|**4**|**8**|**16**|**32**||**2**|**4**|**8**|**16**|**32**||**2**|**4**|**8**|**16**|**32**|\n|**2**||6|4|5|6|7||7|6|5|9|8||7|8|10|14 |23|\n|**4**||11|10|10|11|13||10|11|11|13|17||15|19|24|32|48|\n|**8**||19|23|22|27|30||23|23|24|27|35||35|40|49|66|101|\n|**16**||45|47|47|53|63||49|52|54|61|77||89|99|116|152|221|\n|**32**||97|103|107|122|137||115|116|125|141|171||251|272|307|370|516|\n\n\n**Q3. TabPFN details and insights for SBI.**\n\nWhile NPE-PF beats NPE on all our tasks with a simulation budget of 100, it is certainly not expected that NPE-PF will be superior for all simulation budgets. Indeed, there are several benchmark tasks in which NPE performs better than NPE-PF, particularly for larger simulation budgets. In general, as the number of simulations grows large, NPE is expected to eventually outperform NPE-PF. Our main claim is that NPE-PF is simulation efficient, and performs particularly well when the number of simulations is small – a crucial and underexplored area within SBI. \n\nWe agree with the reviewer that identifying the types of simulators on which NPE-PF performs well is interesting. TabPFN is trained using purely synthetic datasets generated from random structural causal models (SCMs), see [1]. In practice, we have found that NPE-PF performs particularly well on 'structured' simulators, which are similar to SCMs in that they have conditional independences. Many real-world simulators exhibit some degree of this structure (e.g. SIR, Hodgkin Huxley, etc.). Explicitly incorporating such independencies into the inference network was already shown to improve simulation efficiency [2]. We hypothesize that the pretraining on SCMs enables TabPFN to automatically detect and leverage these independencies directly from the data. A detailed investigation into these mechanisms is interesting future work, but beyond the scope of this study.\n\nIn addition, we agree with the reviewer that certain formulations are misleading (i.e., “complex parameter spaces—such as spatially varying parameters” can simply be replaced with “high-dimensional”). We will make sure to increase the clarity of these statements and include the previous discussion.\n\n\n**Q4. TabICL and other foundation models.**\n\nWe agree with the reviewer that using other foundation models within our approach would be highly valuable. Unfortunately, the TabICL model currently only provides in-context classification and therefore cannot be used for autoregressive sampling from the posterior, for which we use the TabPFN regression model. However, as the reviewer suggests, our proposed method is, in principle, independent of the foundation model used.\n\n[1] Hollmann et al. Accurate predictions on small data with a tabular foundation model. Nature, 2025.  \n[2] Gloeckler et al. All-in-one simulation-based inference. ICML, 2024."}, {"review_id": "g9Xj9oOEbN", "reviewer": "Reviewer_BTVU", "summary": "This paper proposes using Tabular Foundation Models, specifically TabFPN for SBI. The new approach is evaluated on a few standard benchmarks and two more challenging models.", "questions": "- What is the inference time of the methods in the two real world case studies 3.3 and 3.4?\n- How fast is density evaluations of NPE-PF relative to NPE with normalizing flows?\n- Since we are using TabPFN in an autoregressive manner, is the number of features (only) d_x or d_x + d_theta(<j)? Since we seem to use both data and previous parameters as features? Perhaps you say it somewhere but it wasn’t entirely clear to me when reading the paper.\n- How good are the sbi pacakge defaults for the flows? Could it be that NPE is just less good the NPE-PF because the defaults are far from ideal? How much more efficient can things be if we tune the architectures to the given examples? I know this requires effort and expertise, but if we really case about simulation-budget, than this optimizing is probably the smallest evil. I am just worried that the better NPE-PF results are partially due to bad NPE defaults.\n- When indexing the pre-conditioning/training data, you sometimes use M and N. What is the difference between M and N?\n- In 2.2 You say, “predict the next dimension of data”. Do you mean “predict the next dimension of parameters”?\n- In the calibration results Figure D-3, could you include the confidence envelope assuming well calibration? And perhaps consider difference plots so one has an easier time judging (mis)-calibration which is currently very hard to see in the existing plot.\n- In MMS case study 3.2, I am not sure I understand what you considered the “reference”. Is it the analytical posterior of the misspecified model? Or is it some other target/reference? The text appears unclear in that regard.\n- In the limitations, you speak of “continuous parameters”. Since the benchmarks you use all have continuous parameters, I don’t understand what you mean by that term as a limitation?", "limitations": "A lot of limitations are clearly discussed although the magnitude of these limitations may be understated to some degree. See \"weaknesses\".", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- The idea is very cool!\n- The reported results are largely convincing, especially when it comes to simulation-efficiency.\n\nWeaknesses:\n\n- The used standard (low dimensional) benchmarks are a bit outdated and do not resemble many challenges we face in practice. I know they continue to be reused (because they are easy to fit well I assume) but I don’t see how results are of much use to judge the real-world applicability of the benchmarked methods. In addition, the evaluation of only 10 test data sets is a limitation especially since we apply amortized methods, where we should easily validate results on 100s of datasets.\n- Some of the results are presented in a way that it boarderlines on selective reporting/discussing. For example, the sampling times of the new method NPE-PF seems to be around 10-100 seconds per dataset (based on what I see in Figure 2). In contrast, NPE with normalizing flows requires only 0.1 seconds (or less?) per dataset. This is shown (in a very small manner) in Figure 2, the text says NPE-PF is slower than NPE. While true, the multiple orders of magnitude in speed difference are never explicitly mentioned. I suspect they could easily be missed by the unattentive reader. If we combined the slower training time of NPE with the faster inference time, when would NPE become overall faster again than NPE-PF (given similar calibration)?\n- The fact that a summary (embedding) network isn’t naturally fitting into the framework is another limtation of the approach. For almost all non-trivial data, we would want a summary network to handle the data structure properly. Yet, to use it in combination with NPE-PF, we have to pre-train it separately, reducing the argument of NPE-PF being “training-free”. How much additional training and simulation-budget do we need in this case? Is it substantially smaller (or faster) than training NPE+summary network combined? Also, I fail to find information in B.2 how good the “sufficient statistics” approximation described in EQ B-3 is. What are potential issues with this approach?", "was_revised": true, "final_justification": "Authors rebuttal addressed most of my concerns.", "rebuttal": "We thank the reviewer for their overall positive evaluation (\"very cool idea,\" \"largely convincing\") and relevant suggestions that led us to add new tasks, a full cost analysis, and an NPE hyper-parameter sweep, which significantly strengthened our work.\n\nWe first address the main issues and discuss new results (**A1–A3**), then give detailed replies to questions (**Q1–Q9**).\n\n- **Benchmark limitations:** We use an established benchmark for SBI, but now add new tasks and metrics with more test points for broader coverage (**A1**).\n- **Computational cost:** We acknowledge that our discussion was imprecise. We will update the manuscript to clearly write that the inference time of NPE-PF is orders of magnitude slower than that of NPE, and will add a careful discussion of our cost analysis (**A2**).\n- **Embedding nets:** We do not claim NPE-PF outperforms NPE when embedding networks are needed; we merely explore this practical/frequent use-case (**A3**).\n\n**A1. Additional tasks and datasets.**\n\nWe agree with the reviewer's assessment that the \"used standard (low-dimensional) benchmarks\" with \"only 10 test data sets\" have limitations. However, as the reviewer notes, this SBI benchmark is still widely used and tasks are not \"easy to fit\" for NPE methods (e.g., SLCP or Lotka-Volterra). Nonetheless, we extended our current evaluation suite with several new tasks from other SBI papers [1,2], also including particularly computationally demanding physics simulations (\"streams\", 30 min. per simulation, 2 dim parameters, 199 dim data). For the new tasks, we compute the average negative log likelihood (NLL) on 1000 test points. If the tasks provide a ground truth posterior, we use 100 test points in the C2ST evaluation.\n\nOn these new tasks, NPE-PF again shows strong performance, especially for the smaller simulation budgets, where it is superior to NPE for all tasks. For the detailed results, we refer to the response to reviewer Vbbm, Table V1/V2.\n\n**A2. Analysis of computational cost.**\n\nWe ran additional timing experiments. As expected, inference time mostly scales with an increasing parameter dimension and a larger number of simulations. See Table w2 in the response to reviewer wFAY for details.\n\nProviding a general answer to the question of when \"would NPE become overall faster again\" is hard because it is task and application dependent. Disregarding simulation time, NPE-PF quickly becomes less efficient than NPE: For example, assuming 100000 available simulations (8 dim parameters, 8 dim data), NPE takes around 60–90 minutes to train. After training, inference will be effectively instantaneous. In contrast, generating 10000 posterior samples for a single observation using NPE-PF takes ca. 50 seconds (Table w2, response wFAY). Therefore, the training time of NPE amortises after evaluating around 90 observations. However, for expensive simulators (e.g. stellar streams), simulation becomes the bottleneck and the simulation efficiency of NPE-PF compensates for the slower inference time. For sequential inference, the situation favours TSNPE-PF, because everything needs to be rerun per single observation.\n\nWe will add a thorough discussion of the computational times to the manuscript. Our goal with NPE-PF is not to replace NPE, but to provide an easy-to-use and simulation-efficient alternative, highlighting strengths and limitations such that users can make an informed choice.\n\n**A3. NPE-PF with summary (embedding) networks.**\n\nWe thank the reviewer for raising this point and agree that it requires clarification:\n\nOur main goal with NPE-PF is to introduce an easy-to-use method that allows users to obtain good results with few simulations, given a sufficiently low-dimensional inference problem (<500dim). In high-dimensional cases that need an embedding net, the number of required simulations typically also grows large, and NPE is expected to be superior. Thus, NPE (with embedding net) and NPE-PF serve different use cases.\n\nTo further clarify, we do not claim that NPE-PF will be superior to NPE when embedding nets are needed, nor that our approach for pretraining them is the best. Our goal was simply to investigate how NPE-PF performs together with an embedding network or generally a neural representation of data. For more common datatypes, networks to compress data can be already available. The question of how much \"additional training and simulation budget\" is needed to obtain \"good\" neural representations is independent of NPE-PF and beyond the scope of this work. For related questions including \"how good the 'sufficient statistics' approximation\" is, see [3,4].\n\nFinally, even when simulators are high-dimensional, a set of well-engineered, low-dimensional summary statistics is still routinely used in many SBI applications. Such cases include expensive simulators, where training an embedding net is infeasible, or misspecified simulators, where summary statistics provide increased robustness. In these cases, NPE-PF provides a powerful alternative to traditional approaches, even in high-dimensional settings.\n\n**Q1. Inference time of real-world tasks.**\n\nFor our experiment on the recordings from the Allen cell types database (Sec. 3.3), the inference times are as follows:\n- TSNPE-PF, 1000 sim.: ca. 10 min, 10000 sim.: ca. 60 min.\n- TSNPE, 1000 sim.: ca. 4 min, 10000 sim.: ca. 40 min.\n\nCompared to TSNPE with 10000 simulations, our approach achieves comparable inference quality using only 10% of the budget.\n\nFor the Pyloric simulator (Sec. 3.4), inference takes 2-7 minutes per round (including simulation). See Appendix C.4 for an extended discussion and inference time comparison to baseline methods. Note that SNVI baseline needs 30 hours runtime on CPU, whereas the SUNLE baseline requires 3 hours on GPU to acquire a limited set of samples (excluding the time to simulate the much larger initial dataset).\n\n**Q2. Inference time of density evaluation.**\n\nAutoregressive density evaluation is as fast as sampling. For the exact timings, please refer to Table w2, wFAY. The ratio-based density evaluation is much faster than the autoregressive one. See a detailed analysis in Appendix D.4, Fig. D-5c. Both ways of evaluating densities with TabPFN are slower than a flow and we will make sure to state this more clearly in the manuscript.\n\n**Q4. Sweep for hyperparameters.**\n\nWe agree that the performance of NPE on individual tasks can likely be increased, provided users have enough expertise and computational resources. Therefore, we have conducted an extensive computational experiment, where flow and optimization hyperparameters are swept for each task and simulation budget. Sweeping results are in Table B1 and details in caption. Even for the smaller budgets, this procedure required a substantial amount of compute, taking several hours per setting.\n\nSweeps give only modest gains—mainly for small budgets where exhaustive search is feasible—and demand far more time or better search for larger budgets. Tuned NPE still trails NPE‑PF on most low‑budget tasks. Importantly, NPE‑PF’s edge is its strong default performance: a better “perfect NPE network\" certainly exists, but finding it with few simulations is challenging or basically impossible, and not \"effortless\".\n\nFinally, this strong default performance becomes crucial for sequential inference, where sweeping is considerably more challenging and costly (long inference times, changing training conditions over rounds, difficulty of selecting appropriate metrics to optimize).\n\n**Table B1 Sweep results.** C2ST averages across 5 seeds (SDs negligible, space constraints). Search space: flow type, number of transforms, hidden layer size, learning rate, and batch size. We use random search to explore over 1000 configurations with respect to the best validation loss. We limited the maximum time per setting to 10 hours.\n|Method|Gaussian Mixture|Two Moons|SLCP|Gaussian Linear|SIR|Bernoulli GLM|Lotka Volterra|\n|-|-|-|-|-|-|-|-|\n|**100sim.**||||||||\n|NPE-PF|**0.72**|**0.81**|**0.95**|**0.64**|**0.78**|**0.85**|**0.98**|\n|NPE-Sweeper|0.79|0.89|0.98|0.87|0.92|0.97|1.00|\n|**1000sim.**||||||||\n|NPE-PF|0.72|0.69|**0.90**|**0.55**|**0.58**|**0.74**|**0.93**|\n|NPE-Sweeper|**0.65**|**0.68**|0.94|0.66|0.72|0.80|0.99|\n|**10000sim.**||||||||\n|NPE-PF|0.68|0.63|**0.86**|**0.52**|**0.54**|**0.63**|**0.84**|\n|NPE-Sweeper|**0.54**|**0.57**|**0.86**|0.57|0.55|0.66|0.96|\n|**100000sim.**||||||||\n|NPE-PF|0.56|0.55|0.81|**0.52**|**0.51**|**0.57**|**0.69**|\n|NPE-Sweeper|**0.52**|**0.53**|**0.71**|0.53|0.52|0.58|0.88|\n\n**Q3, 5-9. Clarifications.**\n\n- **Number of features.**\nIndeed, the dimensionality supported by NPE-PF is the combined dimensionality of observation and parameter space.\n\n- **Calibration results.**\nWe agree that using difference plots and including the confidence envelope will increase readability, and will update the plot accordingly.\n\n- **Misspecification.**\nFor the misspecification experiment, C2ST are computed with respect to non-misspecified ground truth posterior, as in [5].\n\n- **Other phrasing.**\n  - With \"continuous parameters\", we meant infinite-dimensional parameter spaces such as spatially varying parameters.\n  - Indeed, we meant \"parameters\" instead of \"data\". Thanks for catching this.\n  - With M we refer to the size of the datasets used for pretraining TabPFN, and with N we denote the number of simulations to be used with NPE-PF.\n\nWe will correct/phrase the above points more clearly.\n\n[1] Hermans et al. \"A trust crisis in simulation-based inference? Your posterior approximations can be unfaithful.\" 2021.  \n[2] Gloeckler et al. All-in-one simulation-based inference. 2024.  \n[3] Yanzhi et al. \"Neural approximate sufficient statistics for implicit models.\", 2020.  \n[4] Yanzhi et al. \"Is learning summary statistics necessary for likelihood-free inference?.\", 2023.  \n[5] Schmitt et al. \"Detecting model misspecification in amortized Bayesian inference with neural networks: An extended investigation.\", 2024."}, {"review_id": "0yQSn5mOWc", "reviewer": "Reviewer_RF8X", "summary": "This work tackles an idea that has been gaining more and more traction in the AI4Science literature : how to leverage foundational models to accelerate science discovery? The authors use the recently proposed TabPFN architecture, initially conceived for univariate regression/classification problems, in a simulation-based inference setting. The main idea is to recast TabPFN as a univariate density estimator and to plug into an autoregressive factorisation of a possibly multivariate posterior distribution.", "questions": "1) I would have appreciated having a clearer explanation (maybe with a figure? diagram?) of how the sampling of the final posterior approximator is done. The approach is a bit convoluted envolving a classifier, TSNPE, TabPFN, rejection sampling, etc. It took me a bit of time to understand how each brick connects and I'm not sure that every reader would have the energy/time to do the same.\n\n2) In the autoregressive factorization, how does the ordering of the choice of dimensions play a role? Is there an optimal way in practice? Or is the TabPFN model completely equivariant to the ordering of the input dimensions?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The paper reads very well and is based on a rather popular and easily understandable engine, i.e. TabPFN. The way I see, the main methodological contributions of the authors were:\n- Recasting TabPFN to SBI by leveraging the autoregressive factorisation of pdfs\n- Despite the rather tedious and potentially slow sampling that entails from the pdf factorisation, the authors proposed a nice \"ratio trick\" to enable faster sampling from the trained approximate posterior at inference time", "was_revised": true, "final_justification": "I believe the paper has its merits for an acceptance. I don't think is worth a spotlight or something like that, but the authors did a good rebuttal and would not be against seeing their work at the conference. I can't, however, justify increasing my already rather positive score.", "rebuttal": "We thank the reviewer for their overall positive evaluation of our manuscript – finding that our “paper reads very well” and acknowledging that the use of foundation models in SBI “tackles an idea that has been gaining more and more traction”.\n\n**Q1. Clear explanation of final posterior sampling.**\n\nWe appreciate the suggestion and will provide a more thorough explanation in the updated version. In particular, we will add detailed pseudocode to the appendix to provide a clear and rigorous overview of our TSNPE-PF approach.\n\n**Q2. Optimal autoregressive factorization.**\n\nWe discuss the question of whether an optimal order of the parameter dimension for TabPFN exists in the Appendix Section. D.6, Fig. D-7. Indeed, TabPFN is not equivariant to the ordering of the inputs. In general, given a different ordering, each one-dimensional conditional distribution, which TabPFN has to predict, changes. We conduct experiments for two synthetic tasks and five benchmark tasks, which are not permutation equivariant. Specifically, we randomly permute the parameter dimensions and evaluate the performance. Our results indicate that NPE-PF is relatively insensitive to the order in which parameter dimensions are permuted. \n\nWe additionally tested this on our 31‑dimensional pyloric network example: Here the total number of possible permutations is greater than $10^{33}$.  Finding the optimal permutation in this space is highly challenging. To gain at least some insight, we performed an ablation study on 50 distinct random permutations using the context set identified by the final‑round posterior. We compute the presented metrics with 1000 simulations. We found that the energy score varies between 0.081 - 0.094 and the valid rates between 96% - 98% (5% and 95% quantile each). This is in line with what we found for the default order reported in the paper, Appendix D.6, Fig. D-7. However, these results do not prove the absence of an optimal permutation that would significantly improve performance. Finding such a permutation is likely to be very challenging in general and would present an interesting opportunity for future research."}]

Author rebuttal:

Meta-review: The paper develops a method for simulation-based inference that leverages a tabular foundation model to reduce the simulation burden. The key idea is to use TabFPN in an autoregressive factorization of the posterior. The resulting method can significantly reduce the number of simulations needed to obtain similarly good posterior approximations compared to other SBI methods. 

Reviewers recognized the potential of using foundation models for SBI and found the idea to be simple, clever, and potentially impactful. They appreciated the details required to make it work. They thought the evidence was convincing within the scope of problems evaluated. 

A weakness raised by several reviewers the lack of a clear discussion about total runtime complexity compared to alternatives, with one reviewer commenting that the tone of the paper might be a bit too optimistic by skirting this detail. The authors provided more information in the rebuttal. It is recommended to include this information and make this discussion more prominent in the revision. 

Another weakness mentioned by multiple reviewers was the relative simplicity of problems in the benchmark. The evidence could be strengthened by using more complex real tasks. The authors provided more results in the rebuttal that strengthened the evidence.

Overall, the paper was well received and proposes an idea that may advance SBI and be used by other researchers.
Decision: Accept (poster)