PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference
Authors: Yang Yang
Severi Rissanen
Paul Edmund Chang
Nasrulloh Ratu Bagus Satrio Loka
Daolang Huang
Arno Solin
Markus Heinonen
Luigi Acerbi
Venue: ICLR 2026 Poster (2026)
Keywords: Simulation-based inference
Amortized inference
Test-time adaptation
Bayesian workflow
Neural posterior estimation
Diffusion models
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: PriorGuide enables efficient incorporation of arbitrary priors at inference time for amortized diffusion-based simulation-based inference, without retraining the model.

Abstract: Amortized simulator-based inference offers a powerful framework for tackling Bayesian inference in computational fields such as engineering or neuroscience, increasingly leveraging modern generative methods like diffusion models to map observed data to model parameters or future predictions. These approaches yield posterior or posterior-predictive samples for new datasets without requiring further simulator calls after training on simulated parameter-data pairs. However, their applicability is often limited by the prior distribution(s) used to generate model parameters during this training phase. To overcome this constraint, we introduce *PriorGuide*, a technique specifically designed for diffusion-based amortized inference methods. PriorGuide leverages a novel guidance approximation that enables flexible adaptation of the trained diffusion model to new priors at test time, crucially without costly retraining. This allows users to readily incorporate updated information or expert knowledge post-training, enhancing the versatility of pre-trained inference models.

Published as a conference paper at ICLR 2026
PRIORGUIDE: TEST-TIME PRIOR ADAPTATION FOR SIMULATION-BAN -BSED INFERENCE
Yang Yang' Severi Rissanen 2.3 Paul E. Chang' Nasrulloh Loka', Daolang Huang Arno Solin 2.3, Markus Heinonen', Luigi Acerbi
'Department of Computer Science. University of Helsinki, Finland *Department of Computer Science, Aalto University, Finland *DataCrunch (yang. yang, paul, chang, nasrullch. satrio, luigi. acerbi (eaissiii.eiiiii........m fi severi.. ri ssanen, daolang huang, arno. sol in, markus. .o.aa....hso o..ninonen
'ELLIS Institute Finland
ABSTRACT
Amortized simulator-bart based inference offers powerful framework for tackling Bayesian inference in computational fields such as engineering or neuroscience, increasingly leveraging modern generative methods like diffusion models to map ob- served data to model parameters or future predictions. These approaches yield pos- terior or posterior-peedcttvve samples for new datasets without requiring further sim- ulator calls after training on simulated parameter-de pairs. However, their applica- bility is often limited by the prior distribution(s) used to generate model parameters during this training phase. To overcome this constraint, we introduce PriorGuide, technique specifically designed for diffusion- -based amortized inference methods. PriorGuide leverages novel guidance approximation that enables flexible adapta- tion of the trained diffusion model new priors test time, crucially without costly retraining. This allows users to readily incorporate updated information or expert knowledge post-training, enhancing the versatility of pre-trained inference models.
INTRODUCTION
Simulation- based inference (SBI) has become key tool across scientific disciplines, enabling Bayesian inference for complex systems where the likelihood function p(x 0) for data x and model parameters 0 intractable, but one can easily simulate from the forward model et al., 2020). Within the Bayesian paradigm, prior beliefs about parameters O are updated with observed data x to form a posterior distribution p(O x) (Gelman et al., 2013). While traditional methods like Markov Chain Monte Carlo (MCMC; Robert, 2007) are effective when likelihoods are available, recent amortized inference methods can directly learn the inverse mapping x + p(0 x) from observations to posteriors using neural networks (Lueckmann et al., 2017; Radev et al., 2020). These amortized approaches, once trained, can rapidly infer posterior (parameters) or posterior- predictive (data) distributions given new observations, significantly speeding up the inference process.
p(x ()) (Cranmer
Modern generative models, including diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021), transformers (Vaswani al., 2017), and flow-matching techniques (Lipman et al., 2023), have demonstrated state-of-the-art performance in tackling these inverse modeling tasks for amortized SBI (Müller et al., 2022; Wildberger et al., 2024; Schmitt al., 2024; Gloeckler et al., 2024; Chang et al., 2025; Whittle et al., 2025; Mittal et al., 2025; Hollmann et al., 2025). These models are trained on vast numbers of simulated parameter- pairs (0, x), often drawing parameters 0 from broad, uniform prior distribution to ensure comprehensive coverage of the parameter space.
However, this reliance on fixed training prior introduces significant limitations. Practitioners often possess domain knowledge that, if incorporated as a more specific prior, could substantially improve inference accuracy. Moreover, prior sensitivity analysis-a crucial step for assessing the robustness of scientific conclusions to modeling across many disciplines, e.g., economists must validate the policy implications of macroeconomic models against different theoretical priors (Del Negro & Schorfheide, 2008), climate scientists need to validate the climate sensitivity estimates over multiple sets of assumptions (Sherwood et al., 2020), and epidemiologists need assess the sensitivity of pandemic forecasts to assumptions
becomes cumbersome. This practice is vital

---

Published as a conference paper at ICLR 2026
Priors
True Posteriors
Sampled Posteriors
Traning P.N jear
Training grioe New prtor
.PierGuude
PhiorGinee high
Duttasio model
Parameters.
82
The panmriios
91
01
tine-seriic
Observed daia Postmber prodictive (sre) Posterior prodictive (old)
PriceQuslle low high
Data. Process value
Difiusion model
Time step
Time stop
Time stop
Figure I: PriorGuide adapts a diffusion model new prior information at test time for simulator- based inference- here for a time-series model. Left: Original broad training prior and new, more specific target prior. Middle: Corresponding true posterior distributions over parameters (1) and predictive data (L) for each prior, given observed data. Right: Posterior (1) and posterior- predictive (L) samples from the diffusion model trained on the old prior vs, those from PriorGuide at low or high test-time cost, illustrating PriorGuide's 's ability to match the new posterior from the middle column.
about disease transmission (Flaxman et al., 2020). Changing priors is computationally challenging: non-amortized methods require costly re-simulation for each new prior, while amortized methods require retraining (Elsemüller et al., 2024). Though practitioners often resort to approximations like importance sampling to avoid these costs, such methods fail when priors differ substantially. This limitation becomes increasingly problematic as the field trends towards foundation models for SBI (Hollmann et al., 2025; Vetter et al., 2025). While some recent amortized methods offer inference-time prior adaptation, they are often restricted to specific families of priors (e.g., factorized histograms or Gaussian mixtures pre-defined at training) or simple constraints (Elsemüller et al., 2024; Chang et al., 2025; Whittle et al., 2025; Gloeckler et al., 2024). The broader issue is the impracticality of pre-training over all potential tasks, such as all prior distributions a user might wish to employ. Recent successes of the test-time compute paradigm (Snell et al., 2025) suggest that rather than attempting exhaustive amortization for all scenarios, models could be designed to flexibly incorporate specific requirements, such as user-defined prior, through dedicated computations at inference time-an an ability which was unattained so far. For a comprehensive discussion of the related work, we refer the reader to Appendix A.
In this paper, we introduce PriorGuide, a novel method that empowers diffusion-based amortized inference models with the ability to adapt to new prior distributions (0) at inference time, without the need for retraining the original score model trained under prior Purin (0). PriorGuide leverages the guidance mechanisms inherent in diffusion models to steer the generative process according the new target prior, seamlessly integrating new user-p provided information during the sampling process. Crucially, this approach allows for trade-off: users can invest more computational resources at inference sume as more diffusion steps or adding refinement techniques like Langevin dynamics-to to achieve higher inference fidelity. See Fig. for a conceptual overview.
Contributions Our main contribution is principled framework for flexibly incorporating new prior distributions at inference time into pre-trained diffusion models for SBI. We leverage a novel Gaussian mixture model approximation for effectively turning the target prior g()) into tractable guidance term for the diffusion sampling process. We demonstrate empirically the effectiveness of PriorGuide on range of SBI problems, showing its ability to accurately recover posterior and posterior- posterior-predictive predictive distributions under various inference-time prior specifications. We show how sampling can be refined with additional Langevin dynamics steps, and study the tradeoff between test-time compute and inference accuracy within our framework.
'The intuitive requirement, quantified later, is that g0) should reside in regions of non-negligible mass under Purin (0), avoid -ist---iistribttion regions where the original score model would he poorly trained.
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND
The primary goal in many scientific applications is to infer model parameters 0 given observed data x, or to predict future data x". Bayesian inference provides a framework for computing the posterior distribution over parameters or posterior predictive distribution for new data:
p(0) x) p(x 0) p(0),
(posterior)
p(x" xx = p(x" 0, x) (x) d0, (posterior predictive)
where p(0) is the prior and p(x is the likelihood.
The prior p() encodes the practitioner's beliefs about plausible parameter values before observing data bela-felief's informed by physical constraints, previous experiments, theoretical considera- tions (Gelman et al., 2013). In practice, priors are typically simple Gaussians, uniforms, smooth crributtoons-ffllctinn broad domain knowledge rather than precise specifications. Yet even among simple priors, the specific choice varies by application: different analyses, constraints, or domain knowledge call for different specifications, and modern statistical practice recommends validation of results under multiple prior choices (Gelman et al., 2020).
The likelihood p(x O)) encodes the statistical or mechanistic description of how data are generated from the model. In many real-world scenarios from finance to physics, evaluating the likelihood is intractable, but generating samples x p(x 0) from simulator is feasible, leading to the field of Simulation-Basse Inference (SBI; Cranmer et al., 2020).
powerful paradigm within SBI is amortized inference. Instead of performing inference from scratch for each x, amortized methods train a neural network qa once on large dataset of simu- lated parameter-data pairs, Daim = {(0,, ]= Parameters 0; are drawn from a training prior, Puain (0), and data x, p(x 8.). Once trained, ge provides rapid inference for new observations, amortizing the upfront computational cost. The network is usually trained to approximate the posterior p(0 x) (I (Lueckmann et al., 2017; Greenberg et al., 2019; Radev al., 2020), the likelihood p(x (0) (Papamakarios et al., 2019), or the posterior predictive distribution p(x" |x) (Garnelo et al., 2018; Müller et al., 2022). Recently proposed architectures can flexibly perform all of these tasks, using transformers (Chang et al., 2025) or diffusion models (Gloeckler et al., 2024).
2.1 DIFFUSION MODELS
Diffusion models are powerful framework for generative modeling that transforms samples from arbitrary to simple distributions and vice versa through gradual noising and denoising process (Sohl-Dickstein et al., 2015). In the forward diffusion process, starting from a distribution p(z)) we can draw samples from (e.g., joint samples from the training prior and simulator). Gaussian noise is progressively added to the samples until, at the end of the process (t = 1), the distribution converges to a simple terminal distribution, typically Gaussian. the Variance Exploding (VE) formulation (Song et al., 2021; Karras et al., 2022), the forward diffusion process can be described as:
/ p(z, zo) p(zn) dz, = / NNzz Zo. (I) p(zz) dzo.
p(z.) =
(2)
where p(z, 2a) is the transition kernel (here Gaussian), (t)2 defines the noise variance schedule as a function of time (typically increasing with f), and z, represents the noisy samples at time t. The corresponding reverse process reconstructs the original sample distribution from noise, and can be formulated as either a stochastic differential equation (SDE) or an ordinary differential equation (ODE). The reverse SDE process takes the form (Song et al., 2021; Karras et al., 2022):
dz, - 2á (t) lo p(z:) dt (t) dwi,
where V₂ p(z)) is the score function (gradient of the log-density), dwy is a Wiener process representing Brownian motion (noise), and o(t) is the time derivative of the noise schedule.
Learning the score function The score function Va log p(z,) can be approximated using a neural network s(,, t), trained to minimize the denoising score matching loss (Hyvärinen & Dayan, 2005; Vincent, 2011: Song et al., 2021):
LnsM = Expp(t) (e)rp((n) Ez, Een ao 20.0(1)1) (w((() ||s(2z, 1) - Va, log 2o 10)
(4)
3

---

Published as a conference paper at ICLR 2026
Here p(t) is the distribution of noise levels sampled during training and w()) weights different noise levels in the loss. Once trained, the network s(z,, t) approximates the gradient of the log probability density of noised distributions and affords sampling through the reverse SDE; Eq. (3). Starting from sample 24 ~ NZ 20: o 1) for = 1 with sufficiently large mux. integrating the reverse process backward in time approximately reconstructs the original distribution p(zo).
The diffusion framework's 's flexibility stems largely from its ability to incorporate guidance mecha- nisms, which afford steering the sampling process toward desired outcomes by including additional information or constraints. Notable examples include classifier guidance (Dhariwal & Nichol, 2021) and classifier-free guidance (Ho Salimans, 2022). which enable variable-strength conditioned generation. For inverse problems, guidance methods exist to incorporate information about observa- tions (Chung et al., 2023; Song et al., 2023a).
2.2 DIFFUSION-RASHI AMORTIZED SBI
Modern amortized SBI methods leverage highly expressive generative models for multiple inference tasks. Early applications of diffusion models SBI include Geffner et al. (2023), who demonstrated that learning conditional score functions via denoising score matching offers superior stability and sample quality compared to traditional flow-based baselines. Furthermore, Simformer (Gloeckler al., 2024) trains diffusion model on samples from the joint distribution p(0, x) Purin (0) p(x 0). Simformer employs a transformer architecture to model the score function (5.. t. mask) of the noised joint variable &, = (0,, xx) at diffusion time t. The mask specifies which components of & conditioned upon and which are to be generated. By setting the mask appropriately (e.g.. conditioning on generate f), Simformer can sample from various conditionals, including the posterior p(0 xx) and posterior predictive p(x* x). Crucially, these learned conditionals are implicitly tied to the training prior Ptrain (0) used to generate the training data applying a different prior (0) would require retraining the diffusion model with the new prior.
2.3 PRIOR ADAPTATION AMORTIZED SBI
Standard amortized SBI methods tie the learned posterior (o [x) to the fixed prior Pirin (0) used during training. Changing this prior traditionally requires retraining the model from scratch, which is computationally prohibitive for expensive simulators. A few recent amortized methods achieve prior flexibility by training over a meta-prior-4 a distribution or predefined set of possible prior specifications- to learn how to incorporate different prior information at runtime. For instance, the Amortized Conditioning Engine (ACE; Chang et al., 2025) allows users to specify factorized priors at runtime by encoding each one-dimensional prior density as a normalized histogram over predefined grid; its transformer architecture is trained to process these specific histogram- encodings alongside observed data. Similarly, the Distribution Transformer (DT; Whittle et al., 2025) learns a direct mapping from a prior, itself represented as a Gaussian mixture model (GMM), to GMM posterior, using attention mechanisms to transform the prior components based on the data, munsitivity-wwart SBI (Elsemüller et al., 2024) focuses on providing efficient sensitivity analysis to various modeling choices, including different priors, represented by a discrete set of possible alternative prior specifications. All of these methods enable prior adaptation by relying on their pre-training across a chosen meta-prior. Our approach, described next, sidesteps this requirement by preforming purely inference-time adaptation to flexibly handle new target priors.
prior
PRIORGUIDE
PriorGuide offers a solution to take a diffusion-based amortized SBI model such as Sim- former (Gloeckler et al., 2024), trained training prior (O), and adapt it to new target prior (0) at inference time, without retraining. The objective is to perform standard SBI tasks such sampling from the posterior or posterior predictive distribution under the new prior, that is (()) x) xx p(x (0) or q(x" x), respectively. The method achieves this by adjusting the score guidance during the diffusion sampling process. The key relationship for this adaptation is as follows:
Proposition 1. Let the posterior under the original prior be p(0 x) x Perain (rrain())p(( O), and let the target posterior-the posterior under the new prior-be be q(0 x) x ((0)p(x| 0). Then, sampling from @ x) is equivalent to sampling from r (0)pp( x)) with r()) q(0)
Pman(0) the prior ratio.
4

---

Published as a conference paper at ICLR 2026
Proof. We can rewrite the target posterior q(0 x) as
q(O) x) x q (()p(( 0) = ((0)
(())
Puran (0) Pumin (0)p(x 0) x
Puain (0) p(0) x) = rr x).
where the prior ratio r()) (O) takes the role of an importance weighing function, analogous to the correction applied in multi-round neural posterior estimation (Lueckmann et al., 2017).
g(0)
Next, we focus on the task of sampling from the target posterior. First, we show in Scction 3.1 how introducing the new prior amounts to adding a guidance term to the diffusion process for posterior sampling. In Section 3.2, we develop analytical approximations to make this tractable. Section 3.3 shows how we can provide guarantees using corrective Langevin steps. Finally, in Section 3.4 show how our results for posterior sampling readily extend to the posterior predictive case.
3.1 TARGET PRIOR AS GUIDANCE
Assume we have diffusion model trained under Purain (f) to sample from p(0 xx) with learnt score model sf,,, f, x). PriorGuide leverages the fact that we can relate the score of the target posterior q() x) the original score. The marginal pdf at time i of the diffusion process for q(O () is:
q( x) x f fr(Oo)((0, p)) (o) d0o.
(5)
which is written as an integral over Os by noting that ((0 x) x r(0o)p(00 x) and then we propagated this to time by convolution with the transition kernel p(0, 0o). Thus, the score is:
V log @0: x) = Ve, log
r(0o)p(00 x)p((0 0.. x) d0.
(6)
/
Vo, log
(((0))p(0)) (0. 0 x)p 02 x) do,
= Vo, log p x) Vo, log
r r(oo)p(Oo 00,, x) do.
(8)
s(O,,t,x) xx) Ve. log Ep 10 0x,xx)
((oo)
(9)
original score
reverse kernel
prior ratio
where in Eq. (7) we re-express the joint probability p(O, xx) 1 x) = p(On. 0, x) as p(On 0r, (0p x), which allows to separate the contribution of the new prior guidance from the original score model s(,, x)). In multiple steps we exploit the fact that multiplicative constants inside the integral disappear under the score. In conclusion, Eq. (9) expresses the score of the target (new) posterior as the old score, which we have, plus a guidance term which we can estimate.
Guided diffusion We can draw samples from the posterior distribution via the reverse diffusion process using the modified score in Eq. (9). The first term is the trained score model and the second term estimates how the new prior's 's influence propagates time (guidance term). This is a common way to implement a guidance function (Chung et al., 2023; Song et al., 2023a;b; Rissanen et al., 2025), which now depends on the prior ratio. The core challenge lies in evaluating the expectation over dg. which is intractable and requires simulating the reverse SDE. To make this tractable, we develop analytical approximations in the following section.
Ensuring prior coverage A crucial consideration for stable guidance is ensuring the new prior (0) remains within regions adequately covered by the training prior Pusin (0). 2 If ((0) assigns significant mass to regions where (0) has negligible support (i.e., is out-of-distribuae 2018; Nalisnick et al.. 2019), two related issues can arise: (a) in these regions, the learned score s(0,,, x) is likely a poor representation of the true score, as the training set contained few examples in these regions; and (b) the prior ratio r(0) = q(0) Puraii (0) can become arbitrarily large or ill- defined, destabilizing the guidance mechanism. Lack of coverage can be quantified using OOD metrics (Lee et al., 2018; Nalisnick et al., 2019; Schmitt et al., 2023; Huang et al., 2024). Notably, the requirement that (0) should be covered by Purain (0) typically not restrictive since the common practice is to train amortized models on broad training priors. This diagnostic check is detailed in Appendix A.4.
OOD; Lee et al.,
p(0)/Ptnin
"Within this coverage, (0) can differ substantially from Peain (0)-for instance, heing more concentrated, multimodal, hifted-ecnallig meaningful prior adaptation.
5

---

Published as a conference paper at ICLR 2026
3.2 APPROXIMATING THE GUIDANCE FUNCTION
To approximate the guidance term in Eq. (9) efficiently while maintaining flexible test-time priors, introduce twO approximations. Following recent work (Song et al., 2023a; Peng et al., 2024; Rissanen et al., 2025), we model the reverse transition kernel as a Gaussian. We then introduce a novel approach that represents r(0) as a Gaussian mixture model. This yields an analytical solution for the guidance, circumventing the issue of estimating the score of an expectation via Monte Carlo, which would suffer from both bias and variance.
Reverse transition kernel approximation We first approximate the reverse transition kernel p(O, 0,, x) as a multivariate Gaussian distribution:
p(O, 0,, x) R N (0o (O) x). Eoj()
(10)
whose mean is obtained from the score function via Tweedie's 's formula (Song & Ermon, 2019);
Moje ( x) 0, ( log p(0, x)-
(11)
This approximation is common in the guidance literature (Chung et al., 2023; Song et al., 2023a; Boys et al.; Peng et al., 2024; Rissanen al., 2025; Finzi al., 2023; Bao al., 2022). For the covariance matrix Eajt, we adopt a simple yet effective approximation inspired by (Song et al., 2023a; Ho et al., 2022):
o(1)²
Eaj= 1+ (t)2 I.
(12)
This approximation acts as a dependenn scaling factor that naturally aligns with the diffusion process-Starting at the identity matrix when /. 1 and approaching zero as / -> O, effectively increasing the precision of our prior guidance at smaller timesteps. This approximation becomes exact for all t if the posterior under the original target distribution is p(0o x) = AOo 0,
Prior ratio approximation With the goal of obtaining a closed-form solution for the guidance, we ?(()) then approximate the prior ratio function r(0) = p(0) as generalized mixture of Gaussians:
K
r(0)
~
pr, E,), wNN M..
r(0) 2 0,
(13)
1=1
where (wi, Mi, Ei represent the weights, means and covariance matrices of the mixture, with K hyperparameter denoting the number of mixture components." 4 Since this represents a ratio rather than distribution, the mixture weights need not be positive nor sum to one, as long as the ratio remains non-negative, potentially enabling more expressive approximations such as subtractive mixtures (Loconte et al., 2024). Notably, when p(0) is uniform, r(0) x (()). Since the guidance term is gradient of log-expectation, the constant factor vanishes, allowing us to directly specify g(0) as a Gaussian mixture. For the more general case of non-uniform training distributions, obtaining the Gaussian mixture approximation for the ratio is standard function approximation task (Sorenson & Alspach, 1971). Crucially, since the densities of both Purin and & are analytically known, fitting the ratio avoids the instability and high variance inherent statistical density-ratio estimation from finite samples. We provide a straightforward gradient-based fitting procedure in Appendix A.2.
Guidance term Plugging in Eq. (10) and Eq. (13), the guidance from Eq. (9) becomes:
Ve, log Epco, xxx) [r()) Ve, log
w.N(Oo μ4, E,) (o x), Eoje) do
(14)
This integral can be solved analytically (full derivation in Appendix B), yielding:
K
Ve, log Epro, 0. a. (r(e))
~ w. w. Mojo " Vo.
(15)
MMore advanced covariance approximations are explored in the literature (Boys et al.; Bao et al.; Peng et al., 2024; Finzi et al., 2023; Manor & Michaeli; Rozet et al., 2024: Rissanen et al., 2025). but introduce added computational costs or implementation complexity. For this work, we focus the simple and effective Eq. (12), which already yields strong results especially when combined with the Langevin refinement in Section 4.4. "We empirically analyze the sensitivity to K in Appendix D.5, finding that performance is robust once K provides sufficient expressivity (e.g., K 20 in this paper), and increasing K has negligible computational cost.

---

Published as a conference paper at ICLR 2026
where Σ, =; Zojt and tt; wyN(H, x0),
wyNWHy E,)/
(0), Σz).
Finally, the PriorGuide update to the mean of the reverse kernel can be expressed concisely using Tweedie's formula, Eq. (11), and our derived guidance term, Eq. (15):
Poje (0,, x) No :0 at))
Σ W (c Mojo (0.,()) E,, VV. Moj (0,, x).
(16)
This update intuitively combines the original prediction Moje based on the training prior with weighted sum of correction terms from our new prior. The correction magnitude controlled by both the noise schedule (1)2 and the distance between the mixture components and current prediction.
3.3 ASYMPTOTICALLY CORRECT SAMPLING WITH LANGEVIN DYNAMICS
The diffusion sampling process detailed in Section 3.2 is approximate due to the Gaussian approxima- tion the reverse transition kernel 00 0,, x) However, as stated in the following proposition, this Gaussian approximation becomes correct on low noise levels, rendering our guidance term accurate.
As I, o() + O, the approximation N(oo O, + o(1)2Vo, log ((,)( 1+a converges the true p(0, 0.), under mild regularity conditions on p(0.)
Proposition 2.
e())²
We provide the exact statement and a proof in Appendix B. Close-by statements are well-known in the diffusion literature (Finzi al., 2023; Ho al., 2022). Thus, the guidance approximation is correct at low noise levels (up to the GMM prior ratio fit accuracy), and we can run accurate Langevin dynamics MCMC sampling (Särkkä & Solin, 2019). To incorporate this with the regular diffusion process, we run Nr. Langevin steps after each regular diffusion step, effectively transforming the sampling into an annealed MCMC process that resembles methods used in compositional generation (Geffner et al., 2023; Du et al., 2023) and early unconditional diffusion models (Song et al., 2021). This refinement increases the total inference cost by a factor of NL +1
Thus, PriorGuide has two main hyperparameters: the number of diffusion steps N > O, per any diffusion model, and the number interleaved Langevin steps N₂ 2 For test-time sampling, the total number of function evaluations (NFE), forward passes of the trained network, is N x (Nz+)).
3.4 PRIORGUIDE POSTERIOR PREDICTIVE SAMPLING
PriorGuide is readily applied to compute posterior predictive distributions as well under a new prior. Starting from a diffusion model trained generate samples from the joint posterior predictive distribution, p(x", 0 x), we can marginalize over 0 to get the posterior predictive p(x" x). The joint posterior predictive under the new prior becomes:
g(x", 0 x) = x(x" 0, () x) x g(x" 00, x))())))p(())))))))) (x) = r(0))((",,0))
which results a posterior predictive version of Eq. (9):
Vx o. log q(;;; 0 x) = 0,, + Vx; 0, log Ep(0 xx xx)) [r(0))]
(17)
The posterior predictive and posterior guidance terms differ only in the conditioning information for the score and reverse transition kernel. Thus, everything presented earlier in this section applies to this scenario, and the posterior predictive obtained from the previous formulas with substitutions p(Ool 0., x) p(0. E x), (0,, (oj x) and Vo Ver, where & 0,),
4 EXPERIMENTS
We empirically evaluate PriorGuide across a range of SBI problems, focusing on its ability to adapt to new priors at test time for posterior and posterior predictive inference. First, Section 4.1 provides intuitive demonstration on several SBI problems, comparing PriorGuide to existing methods that support test-time prior adaptation. Section 4.3 examines PriorGuide's performance challenging posterior predictive tasks. Finally, Section 4.4 studies the trade-off between computational cost and inference accuracy. Full experimental details can be found in Appendix C, with Appendix D.3 reporting additional baseline
2D problem. In Section 4.2, we evaluate posterior inference
7

---

Published as a conference paper at ICLR 2026
results. We also conduct two ablation studies, including analysis of the sensitivity of PriorGuide to the distance between training and test-time priors (Appendix D.4) and a study of the impact of the number of GMM components used to approximate the prior ratio (Appendix D.5). The code is http//rrorgguiiiie
available at https: / /tg//gttttttt com www.ace.///.ooc...oco..oo..c....
4.1 ILLUSTRATIVE EXAMPLE OF TEST-TIME PRIOR ADAPTATION
We illustrate PriorGuide's s capabilities on Two Moons, a two-dimensional SBI model with bimodal posterior (Greenberg et al., 2019). We train the diffusion model under a uniform prior Puain ( over |-1, 1] and test how PriorGuide handles PriorGuide incorporates the new prior, matching well the true Bayesian posterior under the new prior.
prior q(0) at test time. Fig. A1 shows that
4.2 TEST-TIME PRIOR ADAPTATION FOR POSTERIOR INFERENCE
We evaluate PriorGuide's ' posterior inference capabilities on six SBI problems (see Table Al), rang- ing from established SBI benchmarks to real models from engineering and neuroscience: Two Moons (Lueckmann et al., 2021); the Ornstein- Uhleebbe Process (OUP; Uhlenbeck & Ornstein, 1930); the Turin model of radio propagation (Turin et al., 1972); the Gaussian Linear model (Lueck- mann et al., 2021) and its high-dimensional variant; and the Bayesian Causal Inference model of multisensory perception (BCI; Körding et al., 2007). Training priors Puain (0) for the base diffusion model (Simformer; Gloeckler et al., 2024) were uniform or Gaussian; details in Appendix C.
For baselines, we consider the base Simformer (no prior adaptation) and the Amortized Conditioning Engine (ACE; Chang al., 2025), one of several approaches (Elsemüller al., 2024; Whittle et al., 2025) that amortizes test-time prior adaptation for posterior inference by pre- pre-training on variety of possible (factorized) priors. PriorGuide is more flexible than ACE by not needing pretraining on specific priors-instead, it modifies diffusion-based amortized inference model at runtime-, and can represent correlated and non-factorized priors. Detailed comparisons against additional non-amortized importance-rrsamplingg and neural likelihood estimation (NLE; Papamakarios et al., 2019) with MCMC- are provided in Appendix D.3. While often computationally expensive, these methods serve as fundamental benchmarks for posterior sampling.
sethood--nccludngg classic algorithms (rejection sampling and sampling-
We consider three different families of target priors: mild, strong and mixture. Mild and strong priors are defined as multivariate Gaussian distributions with means drawn from a uniform box and diagonal covariance matrices, where the strong priors have smaller standard deviations; these represent scenarios with varying degrees of available information. Mixture priors are defined as mixture distribution with two multivariate Gaussian components with the same setup as the strong priors; this can represent situations with distinct, competing hypotheses about the parameter values. For each prior family, we randomly generate ten possible prior parameterizations g() (0), sample ten parameter vectors 0.g ~ (8 (0) from that prior, and simulate one observed dataset per Oi.j Xij p(x O..) evaluate the methods. See Appendix C.3.1 for the full procedure.
We measure each method's performance using: 1) the root mean squared error (RMSE) between the true parameter and the samples from the estimated posterior; 2) the classifier two-sample test (C2ST) between the estimated posterior samples and ground-truth posterior samples; 3) the mean marginal total variation distance (MMTV) between the estimated vs. ground-truth posterior samples. For RMSE and MMTV, lower is better, while for C2ST, closer to 0.5 is better.
Results in Table show that PriorGuide largely improves inference accuracy over the base Simformer model in all scenarios, making use of the prior information provided at test time, and achieves leading performance most cases, especially when stronger prior beliefs (strong and mixture) are presented.
4.3 TEST-TIME PRIOR ADAPTATION FOR DATA PREDICTION
We next evaluate PriorGuide's 's ability to perform posterior predictive inference under new target priors, focusing forecasting retrocasting scenarios, shown in Fig. 1. We the OUP and Turin models, both of which generate time series trajectories (Fig. 2). We employ the same procedure and test-time prior setup (mild, strong, and mixture) from Section 4.2. For each target prior,
8

---

Published as a conference paper at ICLR 2026
Table Posterior inference (0). Mean (standard dev.) over 5 independent training runs (10 random target priors x 10 simulated datasets). Significantly best results (Wilcoxon signed-rank test) in bold.
9rus 0
(0)
RMSE
C2ST
MMTV
RMSE
C2ST
MMTV
RMSE
C2ST
MMTV
Two Moons
Sinformer
0.39(0.19) 19)
0.3600.05)
0.21 (0)
0,39(0.20)
0.75/0.06)
0,54(0.13)
0.33(0.22)
0.68/0.08)
0.42/0.17)
ACE
0.34:0.16)
0.8300,44)
0.23(0.07)
0.0%)..03
0.79.0005)
0.35(0.12)
0.16(0.15)
0.80X0.07)
0.34
PriorGuide
0.37(0.16)
0.52.0.02)
0.11(0.04)
0.09)..04)
0.52/0.02)
0.08(0.02)
0.20(0.17)
0.55(0.05)
0.16
Simformer
0.18:0.07)
0.58:0.0))
0,18(0.11)
0.24(0.08)
0,73(0.08)
0,37
0.23:0.09)
0.7000.08)
0.33(0.10)
OUP
0.17/0.07)
0.58:0.03)
0.11/0.05)
0.14(0.04)
0.56:0.03)
0.12/0.05)
0.17(0.07)
0.62(0.08)
0,20
PriorGuide
0.17(0.06)
0.52.0.02)
0.08)
0.13(0.04)
(.50(0.02)
0.06:0.02)
(.05(0.06)
0,51(0.02)
(.04)
Simformer
0.22/0.03)
0,8000033
0.31(0.04)
0.23(0.03)
0.95(0.02)
0
0.23(0.03)
0.94(0.02)
0.50 06)
Turia
ACE
0.21(0.02)
0.78(0.03)
0.27(0.04)
0.17(0.02)
0.92/0.04)
0.47(0.07)
0.1900.03)
(.900)
0.06)
PriorGuide
0.14:0.02)
0.64
0.13(0.03)
0.06(0.01)
0.55:
0.08
(.13(0.06)
0.62/0.08)
0.19
Lincar 10D
Ganssion
Sinformer
ACE
0.29(0.02)
0.22/0.02)
0.12(0.02) 0.67(0.04)
0.30(0.03) 0.89
0.3100.03)
0.10(0.01)
0.78(0.06)
1,0000000)
0.19(0.04)
0.65
0.19(0.05)
0.30K0.03)
0.96(0.03)
0.99/0.00)
0,40
0.61(0.05)
PriorGuide
0.31(0.08)
0.53)
(.05
0.23(0.09)
0.54(0.05)
0.06
0.26(0.09)
(.006
(.12().06)
Gaussion
Sinformer
0.29/0.02)
0.95:0.01)
0.29(0.02)
0.30(0.02)
1.0000.00)
0.64:0.02)
0,30(0.02)
1.000.000
0.63(0.03)
Lincar 20D
ACE
0.22:0.02)
0.72(0.06)
0.11/0.02)
0.82/0.05) 0.10:0.01)
0,16(0.03)
0.20(0.04)
0.99(0.01)
0.44(0.09)
PriorGuide
0.22(0.07)
0.54()
(.05(0.01)
0.28(0.12 12)
0.03)
0.00
0.28(0.10)
0.59(..55
(.14
Simformer
0.79(0.13)
0,86(0.05)
0.35(0.09)
1.330.10 16)
0.98/0.01)
(.609)
0.99)
0.98(0.02)
0.63(.09)
ACE
0.55(0.12)
0.84)
0.82(0.09) 1.31(0.13) 0028(0.10)
0.29(0.12)
(.02/0.37)
(.97(0.02)
0.56(0.11)
PriorGuide
0.56(0.10)
0.88(0.06)
(.4000
(.25(0.04)
0.72(0.1) 10)
0.21
0.87 (087))
0.78.0.17 131
0.37 (0.27)
OUP
1.0-
Turin
Process value
Simformer
PriorGuidk
Power
a.5-
M
A
ACE
Observed data
True data
0.0
24
50
Time step
Time step
Figure 2: Example posterior predictive distributions for OUP and Turin models (strong priors).
condition the model on partial trajectories: in half of the cases, the first 30% of the trajectory, and for the other half, the last 30%. The task is always to predict the unobserved 70% of the trajectory.
We evaluate the performance of all methods using RMSE and the maximum mean discrepancy (MMD) with an exponentiated quadratic kernel between the ground-truth trajectory x., and generated posterior predictive samples. Fig. 2 shows how example posterior predictive distributions from PriorGuide closely match the true data. Results in Table 2 show that PriorGuide can generate reliable posterior predictive samples and achieve performance on par with or better than other methods.
Table 2: Data prediction (x). Mean (standard dev.) over independent training runs (10 random target priors x 10 simulated datasets). Significantly best results (Wilcoxon signed-rank test) in bold.
Mmila (0)
Tarong (0)
Umisture
RMSE
MMD
RMSE
MMD
RMSE
MMD
Simformer
0.32(0.10)
0.44(0.24)
0.39(0.11)
0.54(0.24)
0.34(0.11)
(.47(0.255)
OUP
ACE
0,26(0.08)
0.44(0.31)
0.22(0.06)
0.30(0.20)
0.24(0.10)
0.38().31)
PriorGuide 0.28(0.09)
0.45(0.28)
(.21().)5)
(.29(0.17)
0.25(0.11)
0.34(0.22)
Simformer
0.14(0.01)
0.49(0.09)
0.14(0.01)
0.49(0.09)
0.14(0.01)
0.48(0.09)
Turin
ACF:
0.16(0.03)
0.62(0.19)
0.16(0.03)
0.61(0.20)
0.16(0.03)
(.61(0.19)
PriorGuide
0.13(0.01)
0.47(0.08)
0.13()
0.46(0.07)
0.13(0.01)
0.46(0.09)
4.4 TEST-TIME REFINEMENT VIA CORRECTIVE LANGEVIN DYNAMICS
PriorGuide supports improving the sampling quality by adding Langevin dynamic steps to the diffusion process, at the cost of additional test-time compute. We examine posterior inference eecurcy-meassuee by MMTV, but similar results hold for other metrics-on the OUP and Turin models as a function of the number of diffusion steps N and Langevin steps Nz. These two can be combined into single computational cost metric, the number of function evaluations (NFEs), i.e.
TThis task formulation prevents simple data interpolation induced by sampling time indices randomly.

---

Published as a conference paper at ICLR 2026
Diffasion Steps:
Diffusion
100
0.12
(0)
250
500
0.11
MMTV
0.06
120
MMTV
0.10
0.09
0.08
0.04
Langevin steple
()
Langevin steps
Pareto Prontier
0.07
Pareto Prontier
100
1000 Number Punction Evaluations
100
Number of Function Evaluations
(a) Posterior inference on OUP, strong priors
(b) Posterior inference on Turin, strong priors
Figure 3: Pareto frontiers with respect to number of function evaluations (NFEs) and MMTV on posterior inference for OUP and Turin, with varying number of diffusion and Langevin steps.
calls to the score model. In Fig. 3, we visualize the relationship between MMTV, N and Nr. The Pareto front shows that the best posterior inference is achieved by combining moderate diffusion steps (N ~ 25-50) with increasing Langevin corrections if the NFE budget allows it. The sample quality in general improves with more compute, implying that simple way to calibrate the sampling parameters is to increase NFE until the output distribution does not change.
5 DISCUSSION
PriorGuide enables amortized diffusion-based SBI models to adapt to new prior distributions without retraining, an example of the test-time compute paradigm in extending pre-trained model capabiliites with dedicated computations at test time which repurpose diffusion guidance for Bayesian inference. In practice, PriorGuide is recommended for codderae--to-highe dimensional problems (4 < D x 20) where simulators are mildly-to-very expensive, making retraining simulator model burdensome; for settings requiring complex, non-factorized priors, where amortized methods restricted by pre-defined meta-priors (e.g., ACE) yield unsatisfactory performance; and for applications where prior adaptation needs to be fast but not strictly instant-in terms of pure speed, fully amortized methods without test-time compute remain the best choice.
Limitations PriorGuide's 's effectiveness relies on the new prior g()) having substantial overlap with the training prior Purin (0): distriibbitiinoo (OOD) target priors can lead to inaccurate learned scores and unstable guidance. The method also employs approximations-- Gaussian for the reverse transition kernel p(0. O,, x) and Gaussian mixture model for the prior ratio function r(0)~which introduce inaccuracies, particularly for complex prior ratio shapes. Furthermore, current guidance calculations, involving matrix operations for the GMM components, may pose scalability challenges for sigh-dimensional parameter spaces (dim(O) ()) > 20). PriorGuide sampling can be computationally intensive: although our method avoids retraining the base model-a key benefit with expensive simulators (e.g., Turin model)- the iterative guided diffusion, particularly with interleaved Langevin refinement steps (N,),, incurs a cost. The number of function evaluations (NFEs) increases with diffusion (N) and Langevin steps, creating an accuracy-speed trade-off. This may render PriorGuide less suited than fully amortized methods for applications requiring very rapid inference. Advanced covariance approaches mentioned in Section 3.2 have the potential speed up the method by requiring fewer NFEs, and represent an interesting future direction.
The ability of PriorGuide to decouple expensive simulator runs (for training the base model) from the specification of changing prior beliefs offers significant practical advantages. It allows for host-hoc prior sensitivity analyses and facilitates the direct incorporation of domain expert knowledge post-training, reducing the overall computational footprint in scientific workflows by avoiding the need for repeated model retraining when assumptions change.
Conclusions
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was part of Finland's Ministry of Education and Culture's Doctoral Education Pilot under Decision No. VN/31377202-00KM (The Finnish Doctoral Program Network in Artificial Intelligence, AI-DOC). The project was also supported by the Research Council of Finland (Flagship programme: Finnish Center for Artificial Intelligence, FCAI). NL was funded by Business Finland (project 3576/31/2023) and LUMI Factory (EU Horizon Europe Joint Undertaking and its members including top-up funding by Ministry of Education and Culture). LA was supported by Research Council of Finland grants 356498 and 358980. SR, MH, and AS acknowledge funding from the Research Council of Finland (grants 339730, 362408, 334600). The authors also acknowledge the research environment provided by ELLIS Institute Finland.
We acknowledge CSC- IT Center for Science, Finland, for computational resources provided by the LUMI supercomputer, owned by the EuroHPC Joint Undertaking and hosted by CSC and the LUMI consortium (LUMI projects 462000864 and 462000873). Access provided through the Finnish LUMI-OKM allocation. We acknowledge the computational resources provided by the Aalto Science-IT project.
Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do necessarily reflect those of the European Union or the granting authority. Neither the European Union nor the granting authority can be held responsible for them.
Co-funded by the European Union
EuroHPC Undertaling
ETHICS STATEMENT
This work uses only synthetic datasets, with no sensitive data involved. The methods are for research purposes and pose no foreseeable ethical risks. We have followed the ICLR Code Ethics.
REPRODUCIBILITY STATEMENT
The code is available https: //ss:///gthbbbbb www com./aceebillab/piirr-iiiii All experi- ments use synthetic datasets. Algorithmic details are presented in Appendix A, and all experimental details are specified in Appendix C.
REFERENCES
Luigi Acerbi. Variational Bayesian Monte Carlo. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.
Luigi Acerbi, Kalpana Dokka, Dora E Angelaki, and Wei Ji Ma. Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception. PLoS computational biology, (007)10006111 2018
Fan Bao, Chongxuan Li, Jun Zhu, and Bo Zhang. Analytie-dpm: an analytie estimate of the optimal reverse variance in diffusion probabilistic models. In International Conference on Learning Representations.
Fan Bao, Chongxuan Li, Jun Zhu, and Bo Zhang. Analytic-DPM: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. In International Conference Learning Representations (ICLR), 2022.
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and Omer Deniz Akyildiz. Tweedie moment projected diffusions for inverse problems. Transactions on Machine Learning Research.
Wessel P Bruinsma, Stratis Markou, James Requeima, Andrew YK Foong, Tom R Andersson, Anna Vaughan, Anthony Buonomo, J Scott Hosking, and Richard E Turner. Autoregressive conditional neural processes. In International Conference Learning Representations (ICLR), 2023.
11

---

Published as a conference paper at ICLR 2026
Gabriel Cardoso, Sylvain Le Corff, Eric Moulines, al. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. The Tweifth International Conference on Learning Representations.
Paul E Chang, Nasrulloh Loka, Daolang Huang, Ulpu Remes, Samuel Kaski, and Luigi Acerbi. Amortized probabilistic conditioning for optimization, simulation and inference. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2025.
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Mare Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In International Conference on Learning Representations (ICLR), 2023.
Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055-3006 2020.
Marco Del Negro and Frank Schorfheide. Forming priors for DSGE models (and how affects the assessment of nominal rigidities). Journal of Monetary Economics, (5(7):1191-11) 2008
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In Advances Neural Information Processing Systems (NeurIPS), volume 34, pp. 8780-8794, Curran Associates, Inc., 2021.
Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations, 2024.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum. Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Composi- tional generation with energy-based diffusion models and meme. In International conference on machine learning, pp. 8489-8510. PMLR, 2023.
Lasse Elsemüller, Hans Olischläger, Marvin Schmitt, Paul-Christian Bürkner, Ullrich Köthe, and Stefan T Radev. Sensitivity-awar amortized Bayesian inference. Transactions on Machine Learning Research (TMLR), 2024.
Mare Anton Finzi, Anudhyan Boral, Andrew Gordon Wilson, Fei Sha, and Leonardo Zepeda- Núñiez. User-defined event sampling and uncertainty quantification in diffusion models for physical dynamical systems. In Proceedings of the International Conference on Machine Learning (ICML), pp. 10136-10152. PMLR, 2023.
Seth Flaxman, Swapnil Mishra, Axel Gandy, Juliette T Unwin, Thomas A Mellan, Helen Coupland, Charles Whittaker, Harrison Zhu, Tresnia Berah, Jeffrey W Eaton, et al. Estimating the effects of non-pharmaccutical interventions covid-19 in curope. Nature, 584(7820):257-66,,, 2020.
Daniel Foreman-Mackey. corner.py: Scatterplot matrices in python. The Journal of Open Source Software, 1(2):24, jun 2016. doi: 10.21 10.21 00.21105//oss.00024 00024. URL http///oi..oo http://oo...o0/00.111.0111
joss. 00024.
Marta Garnelo, Dan Rosenbaum, Chris Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In Proceedings of the International Conference on Machine Learning (ICML), pp. 1704-1713, 2018.
Tomas Geffner, George Papamakarios, and Andriy Mnih. Compositional score modeling for simulation-based inference. In International Conference on Machine Learning, pp. 11098-11116. PMLR, 2023.
Andrew Gelman, John B Carlin, Hal S Stern, Aki Vehtari, and Donald B Rubin. Bayesian data analysis, volume 3nd edition. Chapman and Hall/CRC, 2013.
Andrew Gelman, Aki Vehtari, Daniel Simpson, Charles C Margossian, Bob Carpenter, Yuling Yao, Lauren Kennedy, Jonah Gabry, Paul-Christian Bürkner, and Martin Modrák. Bayesian workflow. arXiv preprint 200: /. (01800 2020.
12

---

Published as a conference paper at ICLR 2026
Manuel Gloeckler, Michael Deistler, Christian Weilbach, Frank Wood, and Jakob H Macke. All- in-one simulation- based inference. In Proceedings of the International Conference on Machine Learning (ICML), pp. 15735-15766, 2024.
David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-fiee free inference. In Proceedings of the International Conference on Machine Learning (ICML), pp. 2404-2414, PMLR, 2019.
Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-free meme with amortized approxi- mate ratio estimators. In International conference on machine learning, pp. 2020.
PMLR,
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. NewrIPS 202/ Workshop on Deep Generative Modely and Downstream Applications. 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pp. 6840-6851. Curran Associates, Inc., 2020.
Jonathan Ho. Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J, Fleet. Video diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp. 18954-18967. Curran Associates, Inc., 2022.
Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, (37(8045) 2025.
Daolang Huang, Ayush Bharti, Amauri Souza, Luigi Acerbi, and Samuel Kaski. Learning robust statistics for simulation- inference under model misspecification. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. Curran Associates, Inc., 2024.
Bobby Huggins, Chengkun Li, Marlon Tobaben, Mikko Aamos, and Luigi Acerbi. PyVBMC: Efficient Bayesian inference in Python. Journal of Open Source Software, 8(86):4428 2023.
Aapo Hyvärinen and Peter Dayan, Estimation of nor-normalizee statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp. 26565-26577, Curran Associates, Inc., 2022.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
Konrad P Körding, Ulrik Beierholm, Wei Ji Ma, Steven Quartz, Joshua B Tenenbaum, and Ladan Shams, Causal inference in multisensory perception. PLoS One, 2(9): 2007.
Cheuk Kit Lee, Paul Jeha, Jes Frellsen, Pietro Lio, Michael Samuel Albergo, and Francisco Var- gas. Debiasing guidance for discrete diffusion with sequential monte carlo. arXiv preprint arXiv:2502.06073 2025.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distributiion samples and adversarial attacks. In Advances in Neural Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018.
Julia Linhart, Gabriel Victorino Cardoso, Alexandre Gramfort, Sylvain Le Corff, and Pedro LC Rodrigues. Diffusion posterior sampling for iimulation-based inference in tall data settings. arXiv preprint 0rXiv:2404.07553, 2024.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.
13

---

Published as a conference paper at ICLR 2026
Lorenzo Loconte, Aleksanteri M. Sladek, Stefan Mengel, Martin Trapp, Arno Solin, Nicolas Gillis, and Antonio Vergari. Subtractive mixture models via squaring: Representation and learning. In International Conference Learning Representations (ICLR), 2024.
David Lopez-Pazz and Maxime Oquab. Revisiting classifier two-sample tests. In International Conference Learning Representations (ICLR), 2017.
Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Ócal, Marcel Nonnenmacher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. In Advances in Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc., 2017.
Man-Misthis Lucckmann, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H Macke. Likelihood- free inference with emulator networks. In Symposium Advances in Approximate Bayesian Inference, pp. 32-53. PMLR, 2019.
Han-Matthis
Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Bench- marking simulation-based inference. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, pp. 343-35133 PMLR, 2021.
Hila Manor and Tomer Michaeli. On the posterior distribution in denoising: Application uncertainty quantification. In The Twelfth International Conference on Learning Representations.
Sarthak Mittal, Niels Leif Bracher, Guillaume Lajoie, Priyank Jaini, and Marcus Brubaker. Amortized in-context Bayesian posterior estimation, arXiv preprint arXiv:2502.0660) 2025.
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do Bayesian inference. In International Conference on Learning Representations (ICLR), 2022.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Tch, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? In International Conference on Learning Representations (ICLR), 2019.
Tung Nguyen and Aditya Grover. Transformer Neural Processes: Uncertainty meta learning via sequence modeling. In Proceedings of the International Conference on Machine Learning (ICML), pp. 134. PMLR, 2022.
George Papamakarios and lain Murray, Fast s-free inference simulation models with Bayesian conditional density estimation. Advances in Neural Information Processing Systems (NeurIPS), 29, 2016.
George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood- free inference with autoregressive flows. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, pp. 837-848. PMLR, 2019.
Troels Pedersen. Stochastic multipath model for the in-room radio channel based room electro- magnetics. IEEE Transactions on Antennas and Propagation, 67(4):2591-260) 2019.
Xinyu Peng. Ziyang Zheng, Wenrui Dai, Nuoqian Xiao, Chenglin Li, Junni Zou, and Hongkai Xiong. Improving diffusion models for inverse problems using optimal posterior covariance. In International Conference Learning Representations (ICLR), 2024.
Stefan T Radev, Ulf K Mertens, Andreas Voss, Lynton Ardizzone, and Ullrich Köthe. Bayesflow: Learning complex stochastic models with invertible neural networks. IEEE Transactions on Neural Networks and Learning Systems, 33(4): 1452-146, 2020.
Severi Rissanen, Markus Heinonen, and Amo Solin. Free hunch: Denoiser covariance estimation for diffusion models without extra costs. In International Conference on Learning Representations (ICLR), 2025.
14

---

Published as a conference paper at ICLR 2026
Christian P Robert. The Bayesian Choice: From Deciston-theoretic Foundations to Computational Implemennaation, volume 2nd edition. Springer, 2007.
Christian P. Robert and George Casella. Monte Carlo Statistical Methods. Springer Texts in Statistics.
Springer, New York, 2nd edition, 2004. ISBN 0-387-21239-6.
François Rozet, Génôme Andry, François Lanusse, and Gilles Louppe. Learning diffusion priors from observations by expectation maximization. Advances in Neural Information Processing Systems, 37:87647-87682, 2024.
Simo Särkkä and Arno Solin. Applied Stochastic Differential Equations. Cambridge University Press, 2019.
Marvin Schmitt, Paul-Christian Bürkner, Ullrich Köthe, and Stefan T Radev. Detecting model misspecification amortized Bayesian inference with neural networks. In DAGM German Conference on Puttern Recognition, pp. 541-557. Springer, 2023.
Marvin Schmitt, Valentin Pratz, Ullrich Köthe, Paul-Christian Bürkner, and Stefan T. Radev. Consis- tency models for scalable and fast simulation-based inference. In Advances Neural Information Processing Systems (NeurIPS), 2024.
Louis Sharrock, Jack Simons, Song Liu, and Mark Beaumont. Sequential neural score estima- tion: Likelihood free inference with conditional score based diffusion models. In International Conference Machine Learning, pp.
PMLR, 2024.
Steven C Sherwood, Mark J Webb, James D Annan, Kyle C Armour, Piers M Forster, Julia C Hargreaves, Gabriele Hegerl, Stephen A Klein, Kate Marvel, Eelco Rohling, et al. An assessment of earth's 's climate sensitivity using multiple lines of evidence. Reviews of Geophysics, (8<4)::2019R00006 2020.
Francesco Silvestrin, Chengkun Li, and Luigi Acerbi. Stacking variational Bayesian Monte Carlo. arXiv preprint arXiv:2504.0500 05004, 2025.
Marta Skreta, Tara Akhound- Sadegh, Viktor Ohanesian, Roberto Bondesan, Alan Aspuru-Guzite Ar- naud Doucet, Rob Brekelmans, Alexander Tong, and Kirill Neklyudov. Feynman-kac correctors in diffusion: Annealing. guidance, and product of experts. Forty-second International Conference on Machine Learning.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. In International Conference on Learning Representations (ICLR), 2025.
Jascha -ohk--ickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning (ICML), pp. 2256-2265. PMLR, 2015.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations (ICLR),
2023a.
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss- Loss-guided guided diffusion models for plug-and-pl ang-aly controllable generation. In Proceedings of the International Conference on Machine Learning (ICML). pp. 32483-32442 32498. PMLR, 2023b.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl- Dickkstii,n, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations (ICLR). ICLR, May 2021.
H. W. Sorenson and D.L. Alspach. Recursive Bayesian estimation using Gaussian sums. Automatica, 4(4):465-479 1971. ISSN 0005-1098.
15

---

Published as a conference paper at ICLR 2026
Sean Talts, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. Validating Bayesian inference algorithms with simulation- calibration, arXiv preprint 0rX 2018.
Owen Thomas, Ritabrata Dutta, Jukka Corander, Samuel Kaski, and Michael U Gutmann. Likelihood- free inference by ratio estimation. Bayesian Analysis, 17(1):1-31 2022.
James Thortoo,, Louis Béthune, Ruixiang ZHANG, Arwen Bradley, Preetum Nakkiran, and Shuangfei Zhai. Composition and control with distilled energy diffusion models and sequen- tial monte carlo. In The 28th International Conference Artificial Intelligence and Statistics.
George L Turin, Fred Clapp. Tom L Johnston, Stephen B Fine, and Dan Lavry. A statistical model of urban multipath propagation. IEEE Transactions on Vehicular Technology, 21(1):1-9, 1972.
George E Uhlenbeck and Leonard S Omsteen. On the theory of the brownian motion. Physical Review, 36(5):823, 1930.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc., 2017.
Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao, and Jonah Gabry. Pareto smoothed importance sampling. Journal of Machine Learning Research, 25(72):1-58 2024.
Julius Vetter, Manuel Gloeckler, Daniel Gedon, and Jakob H Macke. Effortless, simulation-efficient Bayesian inference using tabular foundation models. arXiv preprint arXiv:2504.17666, 2025.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computa- tion, 1367)11611-1674, 2011.
George Whittle, Juliusz Ziomek. Jacob Rawling, and Michael Osborne. Distribution trans- formers: Fast approximate Bayesian inference with on-the-fly prior adaptation. arXiv preprint arXiv:2502.02663, 2025.
Frank Wilcoxon. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80-83, 1945.
Jonas Wildberger, Maximilian Dax, Simon Buchholz, Stephen Green, Jakob H Macke, and Bernhard Schölkopf. Flow matching for scalable simulation-based inference. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. Curran Associates, Inc., 2024.
Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 00f:31372-344000 2023.
Wang Yuyan, Michael Evans, and David J Nott. Robust Bayesian methods using amortized simulation- based inference. arXiv preprint arXiv:2504.09475, 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
The full appendix is organized as follows:
Appendix A provides an extended description of related work and our method.
Appendix B presents mathematical proofs and derivations.
Appendix C describes experimental and statistical procedures.
Appendix D shows supplementary experimental results and analyses.
Appendix E details computational and software resources.
A METHOD DETAILS
In this section we start with an extended discussion of related work (Appendix A.I). We then detail the main PriorGuide algorithm (Appendix A.2), the Langevin dynamics step size (Appendix A.3), and the prior coverage diagnostics (Appendix A.4).
A.1 EXTENDED RELATED WORK
Section 2 in the main paper situates PriorGuide within the broader context of Simulation- -Based Inference (SBI) and diffusion models. Here we explore those connections in more detail.
Amortized SBI and prior specification The output of standard amortized SBI techniques, such as Neural Posterior Estimation (NPE) (Greenberg et al., 2019; Lueckmann et al., 2017; Papamakarios & Murray, 2016), is tied to the fixed prior, Pirin (0). used during their training phase. Modifying this prior traditionally requires retraining the entire amortized model, which be prohibitive given expensive simulators. PriorGuide offers a solution specifically for diffusion- amortized models, enabling adaptation new prior (0) by modifying the sampling process itself, thus bypassing the need for retraining. Other SBI techniques such as Neural Likelihood Estimation (NLE) (Papamakarios al., 2019; Lueckmann et al., 2019) and Neural Ratio Estimation (NRE) (Hermans et al., 2020; Thomas et al., 2022) do not amortize posterior inference, in that they only approximate the likelihood (or likelihood ratio), and traditional approximate inference techniques such as MCMC or variational inference need to be run to obtain a posterior by combining the surrogate likelihood (or likelihood ratio) with prior.
Diffusion models for SBI PriorGuide enhances versatile diffusion baased SBI models like Sim- former (Gloeckler et al., 2024), which we use as our base model. As described in the main text, Simformer leverages transformer-basee
diffusion model over the joint space of parameters and data, p(0,x), allowing to provide amortized samples from arbitrary conditionals (e.g., posteriors, likelihoods) once trained, though this training inherently uses fixed prior. Other diffusion based SBI methods include techniques to combine learned scores from posteriors of individual observations to handle multiple data sources (Geffner al., 2023; Linhart al., 2024) and methods that focus on efficient (sequential) training of posterior score models (Sharrock et al., 2024). Since these approaches ultimately yield score functions for posteriors (conditioned on their respective training priors), PriorGuide's 's test-time score guidance mechanism could also adapt these trained models to new prior beliefs post-hoc.
Amortized prior adaptation As discussed in Section 2.3 of the main text, several recent amor- tized SBI methods support prior changes by training over meta-prior- distribution or discrete set over possible prior specifications -nnlluuin ACE (Chang et al., 2025), the Distribution Trans- former (Whittle et al., 2025), and sensitivity-awarr SBI (Elsemüller al., 2024). These approaches learn to incorporate alternative priors inference time, but rely pre-training actoss the chosen meta-prior. PriorGuide, by contrast, performs prior adaptation entirely at test time, without amortizing target priors beyond any pre-specified meta-prior.
priors, and can therefore accommodate
Prior misspecification Effective PriorGuide use requires the target prior (0) to overlap substantially with the training prior Purin (0) to prevent the trained score model from operating nut-of-distributiin (OOD). This concern for reliable inference echoes broader SBI efforts that address simulator misspeci- fication and its impact inference reliability using techniques like MMD robust statistics (Schmitt
17

---

Published as a conference paper at ICLR 2026
et al., 2023; Huang al., 2024). Similarly, Yuyan et al. (2025) explore robust SBI with classes of priors and assess potential prior-likelihood conflicts. While this paper proposes a simple diagnostics to ensure the prior is compatible with the learned score model, other techniques from the SBI literature could be used.
mechanism is novel application of score-based guidance. While the general idea of guiding diffusion models is well-established for inverse problems and conditional generation (Dhariwal & Nichol, 2021; Ho & Salimans, 2022; Chung et al., 2023: Song et al., 2023a;b), PriorGuide's s specific contribution lies in deriving and applying a guidance term for the prior ratio, akin to an importance ratio term. Moreover, GMM for the ratio is tailored to this prior adaptation task. This contrasts with other guidance methods that often focus on incorporating information from a known forward (observation) model (e.g., linear operators imaging; Chung et al., 2023; Finzi et al., 2023) or specific loss functions (Song et al., 2023b). Further, the literature either tends to focus on the analytic case where the guidance function Gaussian (Song et al., 2023a; Peng et al., 2024; Rissanen al., 2025) or entirely forego analytical integral while keeping the Gaussian reverse approximation (Song et al., 2023b). Our approach strikes middle ground between analytic tractability and flexible, non-Gainee Gaussian guidance functions. The approximation of the reverse transition kernel and its covariance also adapts approaches seen in works like (Song al., 2023a; Ho et al., 2022).
Score-based guidance PriorGuide's
analytical approximation using
number of works has considered Monte Carlo corrections to approximate inference time mod- ifications to diffusion models, such guidance. Wu al. (2023); Dou & Song (2024); Cardoso et al.; Thornton et al.; Lee et al. (2025); Skreta et al. propose variants of Sequential Monte Carlo for asymptotically exact modifications et al. (2023) use MCMC corrections for compositional generation. Early work on using Langevin dynamies in unconditional score based generative models includes Song Ermon (2019) and the mrddicoo-cooreeter sampler from Song et al. (2021).
the generative distribution. Du et al. (2023); Geffner
A.2 PRIORGUIDE INFERENCE ALGORITHM
Algorithm 1 details the PriorGuide posterior inference algorithm and Algorithm 2 contains the posterior predictive inference version.
Algorithm PriorGuide posterior inference
1: Input: Trained diffusion frefusioo--bsedd inference model M, training prior Puain test-time prior ((), number of mixture components K for the prior ratio, min diffusion time Tiiu, max diffusion time Tmax- generation schedule nonlinearity parameter (. number of diffusion steps N, Langevin ratio 17, number of Langevin steps Nz, conditioning information
2: Output: Posterior samples oT.... at time minn-
3: [r(0) [w,, p4; E. FIrGMM (f), ())) K).
with r(0)
g(0) Panin (0)
4: IN,...,4p 4 Linspace( (1, 0, N) (Tmax - -minn - Tmin
5: ON ~ oN
( (Taax)
6: for -N -N-1 do
7: tt t--1t-11- t-=
8: for 8= N₁ do
9:
Compute original score s(,,, 1, using
10:
Compute prior guidance Sp +- Ve. log E Ep 0x [r(o)] with (w,, p4; Eikk Compute guided score $L < s (f,, x)
11:
Sp
12:
Langevin dynamics step 0, 0, n ()) end for
SL (no(t)a(t)e, e~ ((0,
13:
Compute original score sOO,, x) using M
15:
Compute prior guidance Sp + Ve, log Ep(o, 10. (r (r()] with {w;, pú Σ Compute new guided score + s(O,, t, Sp
16:
17:
Maale--mruyamaa step 0,,-, + 0, (t) (2)(t)o(t)Ate, c~000
end for
19:
return Ozg
18

---

Published as a conference paper at ICLR 2026
FITGMM subroutine The FrTGMMMpp(0) q(0), K) subroutine takes input two distributions, p(0) and q()), and fits a generalized Gaussian mixture model (GMM) with K components to approximate the ratio r r0) = ((0)/p(),, described in Section 3 of the main text. This is standard GMM but coefficients are not constrained to sum to one. The subroutine returns the approximated ratio as the weights, means, and covariance matrices of the mixture. This is implemented stochastic optimization procedure over the parameters of the generalized mixture by minimizing the L2 error between the GMM and the ratio (see Appendix C.3.2).
Algorithm 2 PriorGuide posterior predictive inference
1: Input: Trained diffusion- -based inference model M, training prior Prmin (0), test-time prior q( g(O). number of mixture components K for the prior ratio, min diffusion time min. max diffusion time Taax. generation schedule nonlinearity parameter p. number of diffusion steps N, Langevin ratio 17, number of Langevin steps Ni, conditioning information
2: Output: Posterior predictive samples X Tmin at time Tiin-
} <- FITGMM (Prrain()) (0). K): 4: tN..... .o < Linspace 0, N. (T Tmin) + Tuin
3: (r(0) (ws; Mi; }
with r(0)
7(0)
Punin (
5: x -N(00,(TT (m)xx)) 'I)
6: Oi o II)
7: Etx = =Oixx xix)
8: for = -
- 10: for = 1- N2 do
9:
11:
Compute original score s(.,; L, x) using MM
12:
Compute prior guidance Sp +- Ve, log Epce, E.. (r(o)) (r))(()))
with (wi, phi ΣijK
13:
Compute guided score SL - s s(E,, t, x) 8p
14:
Langevin dynamics step & + & + 17 o(00(() end for
SL + (nt(t))())) =~0,(111)
15:
16:
Compute original score s(,,, x) using M
17:
Compute prior guidance sp + Ve, log Ep0 (x)))r((o0) with (w, p4. Σ
18:
Compute new guided score + s(Er, t, + S
19:
Euler- Maruyamaa step f + &: + (0)o (2 e (0,1
20:
end for
21:
return x%
A.3 LANGEVIN DYNAMICS STEP SIZE
Since Langevin dynamics becomes exact only at small step sizes, a schedule for the step size an important detail of the PriorGuide algorithm. At larger noise levels (t), the distribution p(z;) is more spread out and thus we can take larger steps, while a smaller step size is necessary for lower (t) levels. We take inspiration from the similarity of the Euler-Maruyam reverse SDE sampler and the Langevin dynamics algorithm (see, e.g., (Karras et al., 2022)), and calibrate the step size such that the noise added in the sampling step is proportional to the noise added in the Euler-Maruyama step when moving to the next noise level. In particular, the update rule for Langevin dynamics at noise level (())
- ) 2
2, + 2 + V( logp(z)) + 28(tte, ((,
(AI)
where At is the step size for the next step in the reverse SDE. and 17 is scaling parameter. 7 1 corresponds to the same noise level as the reverse Euler-Maruyama step. The overall scaling can be tuned to lower values for improved accuracy of the MCMC procedure, the cost of slower mixing. We use = 0.5 for all our experiments.
A.4 OOD DIAGNOSTIC FOR TEST-TIME PRIORS
We assess out-of- distribbttooon (OOD) behavior using Monte Carlo sample-bass based diagnostic that estimates the mass of the test- eest-time prior (() falling into the a-quantile the training prior Perain (0). We outline the procedures as follows:
19

---

Published as a conference paper at ICLR 2026
First, compute the log-density threshold under Purain
Draw Md samples (0(0), Mp Ptrain-
Compute their log-densities & log Purain (0(3).
Let t be the empirical a-quantile of {4.) i.. = quantile(f{1) a).
By construction, we have Prperam [log Ptrain (0) < t = a.
Then, we estimate the OOD fraction under q.
Draw M, samples [ M,
Evaluate each under p: f2j log logg(()())
Count the fraction F with Ej t: F=
E [e] Mr
1
< <t]
Iff a, declare q OOD.
Across all simulators, employ quantile threshold of 0.001, chosen 10/Nuain- where Nuain is the number of simulated parameters used to train the amortized method. We verify that each newly constructed prior (procedures detailed in Appendix C.3.1) successfully passes the above OOD diagnostics.
Validity of the prior ratio For the prior ratio r(0) (00) to be well-defined, we require Prriin (0) = 0 + q( = O, i.e. the target prior cannot have nonzero density where the training prior has zero density. This is not fully addressed by the OOD diagnostic, which looks for substantial overlap of prior mass. To avoid pointwise issues with pero-density regions (e.g., when Premn (0) is bounded uniform distribution), we further truncate the tails of ((),, setting its density zero
Prmin () O. Note that this is done after the OOD check, which means that this adjustment only affects the far tails (0),, with negligible influence on inference performance.
B THEORETICAL RESULTS
We provide in this section full derivations and proofs of theoretical results. This includes the derivation of the guidance term (Appendix B.1) I) and extended statements and proofs for Proposition I (Appendix B.2) and Proposition 2 (Appendix B.3) from the main text.
B.1 DERIVATION OF THE GUIDANCE TERM
Here we provide a detailed derivation for the guidance term, Eq. 15 from the main text. We start from Eq. 14, which writes the guidance as the score of the expectation of the prior ratio under the reverse transition kernel, which are approximated by Gaussian mixture model and a Gaussian, respectively:
K
Va, log Eye (r(()(() R Vo, log
w WoN(Oo, [.)W(0olMor(())) (0.), Eo d)0
(A2)
K
/ N (.)(((olH) Egj)ddo-
Vo, log
(A3)
The step above uses the symmetry property of Gaussian distributions: if N Σ) then μ (a,, E). This allows us to swap 6g and μи in the first Gaussian. Furthermore, using the standard result for the convolution of two Gaussian distributions:
f N(xiu Ex) u2, Ez)dur = (x// E₁ + Ez).
(A4)
we get
Vo, log Ep(e (r(o)] ~ Vo. log Σ
w.N (ns (0,),, + Enje).
(A5)
20

---

Published as a conference paper at ICLR 2026
For notational convenience, we define Σ; Σ, + Eojt continuing with the derivation:
K = Ve log
(Hs E.),
(A6)
==1
Va, Ei=1 w. (H: (0.). Σ,)
K
(chain rulc),
(A7)
Σ/= WH Moje! (0.), Σn)
WwN (oji Σ,) Vo, log (14, (0)c E,) (since Vf = fV /Vlogg
(A8)
Σj= WyN Mooo Σn}
k WWN Moje (O)) Ve. (-3(m0)((0)) (.)) (Moje(O)) 14)
(A9)
Σj=1 WiNN (13 (0.),
E (M Moje (0.), (M, Maji (0.)) (V,
(A10)
Σラ ( Ej)
Finally, with the following definitions:
Σ Σ, + Σoiz
(A11)
WwN(Hs Muje (0,, x), E,)
W Σ 1=1 WH4N Moje (0,, x), Ej)
(A12)
we obtain
Vo, log Epo (r ~ u,(m Mooe((1)) Vo,
(A13)
which is Eq. 15 in the main text.
B.2 PROOF OF PROPOSITION
This proposition is fully derived in the main paper, only include here natural assumption to guarantee the existence of the ratio (see also Appendix A.4).
Let r0)) q())
Phots the prior ratio. We assume that Prain (0) = 0 + (() = 0 almost everywhere, i.e.
(0) needs be zero outside the support of Punin (0), except for set of measure zero
Proposition (Proposition 1). Let the posterior under the original prior be p( @ x) Puain 0 O0)pppp x 0). and let the target posterior-the posterior under the new prior-be g x) x ((0)p(x 0). Then, sampling from q(0) x) is equivalent to sampling from r(0)p(O x) with (0) the prior ratio.
Proof. We can rewrite the target posterior q()) x) as
(0)
(0)
q(0 x) x (())p(((1))))))))))))))))111)(
0)
Prrain Priin (0)p(x () x
p(0 x) = =000p(00 x),
Pirain 0
where the prior ratio r(0) equality applies almost everywhere.
(( takes the role of an importance weighing function, and the above
(0)
B.3 PROOF PROPOSITION
We provide here the extended statement, with explicit regularity conditions, and then the full proof. Proposition (Proposition 2). As t,a(t) - O, the approximation N( 0 + a(1) (tt)VVo, log)(0.), T+a(rj= I) converges to the true p(0. ((), assuming that p(0o) two times differentiable everywhere and 42 ((0) is bounded.
21

---

Published as a conference paper at ICLR 2026
Notation To be precise about the different distributions involved, let denote po(O as the marginal distribution of the clean data, Pr(Ot) the marginal distribution at noise level at)), Po 0b) (0; 0o I and Polo 0z) Pe o(0 (o))p((o)) Let us also drop the p=(Os)
dependence t from the notation ot), and simply refer to a, since we do not have to refer to derivatives of (t)) in the proof.
Proof. First, note that as a - U, -2 +(a*) via a Taylor expansion. In other words, our denoising variance is I up to fourth-order higher corrections in which become negligible at low a. We can thus first show the result for N 0: + ott logp((r), I²I), and at the end we will see that it will trivially transfer to the Ito case well.
o²
Rescaled coordinates
Set
0 0
2/2 exp (s(s) = (2T)-
1-11111
(A14)
so that we can express the forward noising distribution as Pr 1o 0o) o
- dd((s)
(A15)
Taylor expansion for the true posterior density Since po is two times differentiable every- where, the multivariate Taylor theorem gives
x² V (o
PO, + os) = (0.) + oVo.Po T 2
(A16)
where b(s) E (ts < < 1} is some point between 0g and 0g. chosen separately for each s.
Hence, the unnormalised posterior density can be written as
(s)s po(O.)) + (o, s + 2
s T VV
((s)))((((s)s
(A17)
The normalizing constant can be expanded as
(o(O.) / o (s(() Po(O.) + voe. s z ss T (b)))(((( A
σ²
(A18)
= (o.) + Va(s) sds to 1
2 (a($)s V (b
(A19)
=C,(0.)
(0,) O
(A20)
where the odd powered term in the Taylor expansion goes to zero due to the symmetry of the integral. The integral in the second-orde term is finite since we assume V Po is finite everywhere.
The reciprocal of the normalizing constant is
Po (.) o² (O,) Po, Po (0)) C),) + O(o4)
(A21)
obtained with the Taylor series 1 = t + Oss²). Thus, the normalised posterior can be expressed as
(a((s) (O.) + (o.o.(((.)) Ts + s T (bb(s()
(A22)
(o.) + o
V2 (b)))( (mile) pu(0,)2
=o
(a(s) bblss
(o) + voo Ts
s
(() + +((04)
+0(0))))
2
(A23)
1 + -b((s)
(a(s)
Vo.Po(O,)T
(o.) + 0²
1
2 (()))(
(o) (0(0.) Ci(0.)) + +
1
2
(A24)
(d((s)
[1 o Vo, log (0)
oo 0
(A25)
22

---

Published as a conference paper at ICLR 2026
Here, abstract away (s 0t), since later in the proof only care about the fact that it has finite integral I [2a(s) ds.
Taylor expansion for approximate posterior density Note that the score Ve, log (o.))
(o.) and we can reuse our earlier calculation:
(0.
Vo, lo,) (0.) + Vo,o,
(A26)
(0,) + (0,)((0,)
= -Vo,po(0,)+ + Vo po((.)) po (,))
+ +0(+)) (((:))+0(()))
(A27)
(O.) 2 (Vo, (0.)
(0.)o(0.) Ci (0.)) + 0
(A28)
Po
po(O,)2 )
= Ve, (0.) + O
(A29)
This yields formula for our posterior mean:
0 + Vo. lo (0,) = 0c + V (oggo(0)) + 0(o*).
(A30)
Now, define:
(0o)) Noo 0, Vo, log p( 00),,=1)) I²1)
(A31)
0o 02
o 2 o VV. log)(
(A32)
Then we can Taylor expand our posterior approximation in terms of the shift s - s oVo,, logr(0.):
g(os 0,) o
20 (s(s){ + a Vo, log (
1
+ 0² Vo, (0.): (ss - Ve, log Pr po)) O O(a*)]
(A33)
2
da(s)] Vo. (0
1
+ 0 Vo, log)o((.)) T (ss T V),o, (o.) + (og
(A34)
2
da(ss]] Vo po + C3 +
0(o))]
(A35)
where in the second line we used Eq. (A29) and ignored resulting terms that are higher order than 0².
Total variation bound
The total variation distance is
|po| - 4llrv / [Po):(Ou) ((0o)) do
(A36)
Now, plugging in Eq. (A25) and Eq. (A35), we get (note d0g- = ddds
1 2 / o (s)ss) + Vo. log (.) T + 0
1
(A37)
(s) + +oVo, log 3s (0) + 0(a*)) o
(A38)
1 = - f
Cs +
< o²C, (0.) O
(A39)
Thus, the total variation distance converges at rate o². To see the final step more clearly, we can bound the integral with the triangle inequality:
f 10 (C 0.) Ca(s, 0.)..0))+0 +(
(A40)
$ 0²² / (02(s))22(s 0.) + +02 / (ss(s) 0,) 0
+
(A41)
(A42)
23

---

Published as a conference paper at ICLR 2026
Recall the definitions of C3 C2 and C1:
Ci(O,) 2
da(s)s T T (bs)))(
(A43)
1 Po(O.) (i
C2(s, 0:)
V (D)))))
(A44)
2
1 C3 0,) Vo, log (ss 1)Vo, log
(A45)
2
We can see that the terms depending on o² are finite, due the assumption that the Hessian of po is finite everywhere. Thus,
C.(O.) 11000/11aae (Cs, 0.)lds (oaa() 0.)
(A46)
o² 0² - 0. Thus, the the convergence result here applies to the case I+o2 where the posterior covariance approximation is o²
Finally, note that
I+o
C EXPERIMENTAL DETAILS
This section provides extended methodological and experimental details. We describe the simulator models used our experiments (Appendix C.1), the training setup for each method (Appendix C.2), the testing procedure (Appendix C.3), and the statistical methodology for evaluation (Appendix C.4).
C.1 SIMULATORS
Two Moons (Greenberg et al., 2019) is widely used benchmark in SBI, designed a two- dimensional task that presents posterior distribution with both global (bimodality) and local (crescent shape) structure. For given parameter vector O = (01, 02) E IR2, the simulator generates data x IR² according to the following process:
((-7/2,,/22
r N,0.1, 0.01%),
crcos(a) + 0.25, r ssin(a));
x" = 4 ( -101 O2l
-01 + +0
V2
V2
The training prior is Pirain (0) U[-1, 1|3). To obtain ground-truth posterior samples, perform rejection sampling using the new prior as a proposal distribution. Rejection sampling relies finding constant M such that (()) Mq(O), where f(0) p(x 0) ((()) is the target density and (()) is the proposal density. In this Two Moons model, we set M to be the upper bound of the likelihood pxx () which is achieved at 0.1.
Ornstein-Uhin Uhlenbeck Process (OUP) (Uhlenbeck & Ornstein, 1930) is a well-established stochastic process frequently applied in financial mathematics and evolutionary biology for modeling mean- reverting dynamics (Uhlenbeck & Orsttein, 1930). The model is defined as:
ye+1 = yt + AWt.
Ate- = 0, (exp(02) - ye] At + 0.5w, for t= 1
where we set T 25, At = 0.2, and initialize Ho 10. The noise term follows Gaussian distribution, ~ (0, At). The original prior is uniform distribution, ( 2) x 2- 2)). over the latent parameters 0 = (01, 02). For numerical convenience, we reparameterize the parameter space by mapping the original parameters to [-1, 1]2 yielding Purain (1-1 1)3). We perform inference in this normalized parameter space and later rescale the sampled parameters to the original space during simulation. The simulated data are also normalized using standardization. We use normalized darameter-data pairs (0, x) to train all amortized inference models.
Since the OUP likelihood is implicit, obtaining ground-truth posterior samples is intractable the kind of problem that requires simulation-based inference. As practical surrogate, adopt a neural
24

---

Published as a conference paper at ICLR 2026
posterior estimation (NPE) model (Greenberg et al., 2019) trained million simulations (compared to the 10,000 simulations used for our diffusion-basee inference model) to serve as ground truth. We then applied iimulation-based calibration (SBC) (Talts al., 2018) to verify that the NPE model remained well-calibrated across virtually all observation seeds for each new prior.
Turin (Turin et al., 1972) is widely used time-series model for simulating radio wave propagation (Turin et al., 1972; Pedersen, 2019). This model generates nigh-dimensional, complex-value time- series data and is governed by four key parameters: Go determines the reverberation gain, T controls the reverberation time, Å defines the arrival rate of the point process, and o represents the noise variance.
The model assumes a frequency bandwidth of B = 0.5 GHz and simulates the transfer function Hę N₂ 101 evenly spaced frequency points. The observed transfer function the k-th frequency point, Yz, is defined as:
Y = H + We k 0, 1,....... ...., N₂ 1,
where We represents additive zero-mean complex Gaussian noise with circular symmetry and variance o.. The transfer function H₁ is expressed as:
Nisiin
H = Σ a crexp(-j27Affm)
where the time delays tı are sampled from a homogeneous Poisson point process with rate Ag, and the complex gains are modeled as independent zero-mean complex Gaussian random variables. The conditional variance the gains is given by: Go 6xppppnn/T1
EI B|aad12m
Ag
To obtain the time-domain signal ū((), inverse Fourier transform is applied:
1 N
Σ k=0
Yu ex)()2kkAft)
where Aʃ B/NN, 1) represents the frequency spacing. Finally, the real-valued output is com- puted by taking the absolute square of the complex signal and applying logarithmic transformation:
(()) = 10 10 (1)(1)
In Turin, the true parameter bounds are: Go E [10-9, 10- T (10-9.10-9] Ào E [10?. x 10%), of E [10 10-9), follow a similar normalization setup as OUP. First, we define training prior as Perain (0) = U(0, 1]42 and rescale the sampled parameters Ô to the original space using the true parameter bounds. Then, normalize the simulator outputs using standardization and use normalized (Ó, x) pairs train all inference models.
The Turin likelihood is also implicit. Therefore, we use similar setup to the OUP case by training an NPE model with one million simulations and validating its reliability using SBC.
Gaussian Linear (Lueckmann et al., 2021) is a standard SBI benchmark task used to infer the mean of multivariate Gaussian distribution when the covariance is fixed. In this model, both the parameters 0 and the data x are 10-dimensional vectors. The simulator is defined as:
x N0 E.),
where Σ, = 0.1 In with 1o the 10-dimensional identity matrix. The training prior for the parameters 0 is 10-dimensional Gaussian distribution Pusin (0) NO, 0.1 I1o).
We also consider a 20-dimensional variant of the Gaussian Linear model, which follows the same fundamental setup its 10-dimensional counterpart. In this version, both the parameters 0 and the data x are 20-dimensional vectors.
In all experiments, the test-time priors constructed as Gaussian mixture distributions with Gaussian components. Consequently, each new prior has a closed-form posterior, which we use as the ground-truth posterior.
25

---

Published as a conference paper at ICLR 2026
Bayesian Causal Inference model (BCI) is a common model in computational cognitive neuro- science to represent how the brain determines whether multiple sensory stimuli originate from a common source (Körding et al., 2007). This BCI model simulates audiovisual localization task. The observer is presented with auditory (Sa) A) and visual (Sv) spatial cues expressed as horizontal location in degrees of visual angle- and must report the perceived location of one of them. The model assumes the observer performs Bayesian causal inference to determine whether the cues originate from common source or independent sources, and then makes a model-averaged estimate of the target stimulus location (Körding et al., 2007; Acerbi et al., 201 8).
observer's responses in
In this BCI model we consider five underlying physical parameters: standard deviation of visual sensory noise (ov), standard deviation of auditory sensory noise (aA), standard deviation of the Gaussian spatial prior over source locations (as), prior probability that auditory and visual cues share a common cause (Psame), and standard deviation of motor noise in the response (am). We assume the mean of the Gaussian spatial prior Up set to (central tendency), and a small lapse rate À = 0.02, the probability of making random response. Additionally, fixed auditory rescaling factor P = 4/3 is applied to auditory stimulus locations to account for audiovisual adaptation in this experiment.
The simulation process for each trial i (out 98 fixed trials) proceeds as follows:
1. Given true stimulus locations Sv,, and SA. and the modality to be reported (visual or auditory).
2. Sensory measurements TV, and AA are drawn:
vV. N(Sv.,,f))
FAZi ~ N(PASA,i,
3. The observer combines these measurements with spatial prior p(s) Nss (4, o?)-
4. Causal inference is performed to compute the posterior probability of common source, P(C = 1|zv,,, x A.s). using the prior Psame and the likelihoods the measurements under common-cause (C 1) and independent-causes (C' = 2) hypotheses.
5. A model-averaged estimate of the relevant source location, á; is computed:
8, = P(C = 1/ xA.)) (C=1(
(1 P(C [I,,,, (C=2(2V;), 7 A, reeport_mddalinn
where PC=1 and pc=2 are the posterior estimates under the respective causal hypothe-
ses.
6. A noisy motor response RC is generated: R N\; o.
7. With probability the response R is replaced by lapse response drawn uniformly from U(-45%, 45%). The final response is R.
The output data x is 98-dimensional vector (RI...., Ros). These responses correspond to fixed experimental design: 7 7 Cartesian grid of stimulus locations Sv, SA {-15, 100 -5, 0, 5, 10, 15)0. For each of the 49 unique (Sy pairs, one visual-report trial and one auditory-report trial are included, totaling 98 trials.
The model's five parameters are internally represented in an unconstrained space: ov, A Oa, are parameterized by their logarithms, and Psame by its logit, denoted by the vector 0:
0 (0,, O2 03 04, 0s)= ov, log a. log as, log Jm, logit Psame)-
(A47)
The original prior over these 5 unconstrained parameters follows broad multivariate Gaussian distribution informed by the literature, with independent components:
0,, 02 .V 2. 0g NNog 5, 0.52), 01 ~ (log 0.3, 0.3.0.35), 0g AOO, 12). (A48)
For numerical convenience, reparameterize the parameter space by defining the training prior as a 5-dimensional Gaussian distribution Perain (0) = (O Is). Similar to the setup in OUP and Turin, we rescale the sampled parameters to the original space during simulation and normalize the
26

---

Published as a conference paper at ICLR 2026
Table Al: Characteristics of the simulator models.
Model
dim(0)
dim (x)
Pizsin (0)
Two-Moons
2
Uniform
OUP
25
Uniform
Turin
101
Uniform
Gaussian Linear 10D
10
Gaussian
Gaussian Linear 20D
20
20
Gaussian
BCI
5
98
Gaussian
simulated data using standardization. We use normalized parameter-data pairs (0,8) to train all amortized inference models.
The likelihood function for this model involves integrating over latent sensory measurements and is computationally intensive. To obtain ground-truth posterior samples for BCI, we the Variational Bayesian Monte Carlo (VBMC) algorithm (Acerbi, 2018; Huggins et al., 2023). Using VBMC's internal diagnostics. we retain reliable variational posteriors and merge them via posterior stacking (Silvestrin et al., 2025) to obtain the final ground-truth posterior.
Table Al summarizes the key properties of the simulator models our experiments.
C.2 TRAINING SETUP
For all methods that require training, the model. Note that PriorGuide is a test-time technique that does not require separate training. For PriorGuide, we the same base diffusion model as Simformer. Details the model configurations and dataset setups are provided below.
10,000 simulated datasets from the simulator to train
Simformer We adopt similar setup the Simformer paper Gloeckler al. (2024), using the Variance Exploding Stochastic Differential Equation (VE-SDE) technique Song et al. (2021). is defined by
(VE-SDE(E, = 0, 9VE-SDE(!) = amin
Zmax Omin
log
Amax
omin
Throughout all experiments, we set maxx 15, M 10 and run the process over the time interval & E [10-5, 1]. We use a transformer configuration similar to Simformer Gloeckler al. (2024), with 6 layers, 4 heads (size 10), token dimension 40, and 128 22-dimensioonl Gaussian Fourier embedding for diffusion time. MLP blocks use a hidden dimension of 150. In all experiments, the condition mask was sampled per batch by uniformly selecting one of the following: joint, posterior, likelihood, or two random masks. Random masks were drawn from Bernoulli distributions with p 0.3 and 0.7, respectively. We use the same setup for all of the simulators.
We train all the Simformer models using batch size of 1,000 and an initial learning rate of 0.001. A linear learning rate schedule is used decay the learning rate to 1 10-6 sstarting at half of the total number of training steps and completing by the final step. The optimizer combines adaptive gradient clipping with maximum norm of 10.0 and the Adam optimizer Kingma & Ba (2015), Early stopping is applied based on validation loss, with the number of training steps constrained to minimum of 5,000 and maximum of 100,000 steps.
Amortized Conditioning Engine (ACE) ACE (Chang et al., 2025) is a type of Neural Process (NP) (Garnelo al., 2018; Nguyen & Grover. 2022; Müller al., 2022), family of models that context set of input-output pairs to predict
learn to perform amortized inference by conditioning outputs for a target set of inputs. Differently from other neural processes which focus on pure data prediction, ACE is trained to condition on, and predict, both data and latent variables (e.g., model parameters in the case SBI), During training, ACE was provided with simulator parameters that were randomly assigned to either the context or target set, so the model learns to generalize across varying observational conditions. For each experiment, a random number of data points Na were sampled for the context set, with the remaining used targets. Specifically, Na (11 for Two-Moons, Na ~ (7, 25) for OUP, Na ~ (30, 101) for Turin, Na ~ (33 10) for Gaussian I Linear, and Na U(29, 98) for BCI.
27

---

Published as a conference paper at ICLR 2026
Furthermore, ACE can be trained with meta-prior (a distribution of priors) over latent variables, where each prior is expressed as a factorized histogram, affording amortized test-time prior adaptation. For the ACE baseline, we use the same prior generation process the original ACE paper (Chang et al., 2025), which constructs diverse priors through a hierarchical method over a bounded range. For each latent variable 0i the prior is sampled either from a mixture of Gaussians (with 80% probability) or a uniform distribution. When using Gaussian mixture, the number of components K is drawn from a geometric distribution with parameter q = 0.5, and one of three configurations is chosen at random: shared means with varying standard deviations, varying means with shared standard deviations, or both means and standard deviations varying. Means and standard deviations are sampled from predefined uniform distributions based on the latent variable's s range, and mixture weights are drawn from a Dirichlet distribution with og = The resulting mixture is discretized into histogram over bins 100 uniform bins by evaluating CDF differences at bin edges and normalizing. If uniform prior is selected, equal probability is assigned to all bins.
For the network setup, The ACE model has a 64-dimmesiooal embedding, transformer layers, 4 attention heads, and MLP blocks with hidden dimension 128. The output head includes 20 MLP components with hidden dimension 128. Training was carried out in 5 x 10 steps with batch size 32 and learning rate 5 x 10-4 using the Adam optimizer with the cosine annealing scheduler.
a configuration similar to that of the ACE paper Chang al. (2025).
C.3 TESTING PROCEDURE
C.3.1 TEST-TIME PRIOR GENERATIONS
We provide additional details about generating test-time priors in different scenarios. All the generated test-time priors have passed the OOD diagnostic detailed Appendix A.4 with quantile threshold 0.001.
Training prior definition For uniform training priors, let Puain defined as a uniform distribution over a D-dimensional hypercube 11,- [a;, b,]. We define s; = (b; - a;)/ V1 for each dimension (the standard deviation of uniform distribution over the range). For Gaussian training priors, Prain is a multivariate normal distribution with diagonal covariance, with mean m, and standard deviation 8 along each dimension.
Target priors We first consider the cases where the test-time prior is a multivariate Gaussian distribution g (0) ( p. The procedure to generate a test prior is follows, separately for each dimension 1 sis D:
We first set the standard deviation mild = 0.5 8; and 0; strong 0.2 s; for mildly informative and strongly informative priors, respectively.
We define the sampling lower (I₃) and upper (U,) bounds for the mean of the target prior based on the class of training prior:
L = {
a; + 30;
(Uniform)
{
b, 30, (Uniform)
U, =
(A49)
m, 3s;
(Gaussian)
m; + 3s; (Gaussian)
With the bounds chosen, we sample the target prior mean for each coordinate, μis from
14 ~ ULL, U,).
(A50)
This procedure ensures that the target prior is well-overlapping with the training prior.
Finally, we define
Amila (0) .N(e) μ, Emild), Isming (0) = N( μ. Σ strong)
(A51)
where miild
diag
mild?
2 and Ystrong = diag
o
strong 2
D
strong
We then consider y as a mixture Gaussian distribution with two components
fmixture 0 T0 μ1, E Stroon)) + T0 μ2, Sstrong)
(A52)
We sample the mean vector for each component using the above procedures with strongly informative standard deviations. The mixture weights sampled from ~ UO.. 0.8).
28

---

Published as a conference paper at ICLR 2026
C.3.2 POSTERIOR INFERENCE
For each prior type-mild, strong and mixture-we construct 10 priors qi following the procedure detailed in Appendix C.3.1. For each target prior qi, we draw 10 parameter vectors Ozj- For each parameter vector Oij. we simulate one observed dataset, yielding pair (0ig. Xiy). ). For a given tested method, we perform posterior inference by drawing 1,000 posterior samples O using the method conditioned on each X.j. repeating this evaluation across 5 independent training runs. Consequently, each amortized inference model produces x 10 10 500 collections of 1,000 f posterior samples for each prior type.
Remark on prior specification By sampling the ground-truth parameters 0i from the target prior di, we are ensuring that these priors are informative or well-speciffed, i.e., the prior used for inference matches the true data generation process (Gelman et al., 2013). This not a requirement of our method muthood-PriorGuidd can steer sampling towards any user-specified prior at test time (under the assumptions mentioned in the paper), steering the inference process towards the true Bayesian posterior under the given prior. Notably, well-specified prior (or any reasonably specified prior) will on average improve inference accuracy, while a badly specified prior might hinder performance this is general property of Bayesian inference, which is reflected in PriorGuide. To avoid potential confusion, in this paper we focused on well-specified priors.
Simformer With trained Simformer model M, we perform the same generation procedure as (Gloeckler et al., 2024), with a number of diffusion steps N 500.
PriorGuide Following Algorithm 1,
the same set of core hyperparameters across all OUP, Turin, Gaussian Linear, and BCI-unless otherwise noted. These include trained diffusion-basee inference model M (same as Simformer above), training prior distribution Ptrii (0), and a test-time prior distribution q @ The diffusion process is controlled by minimum time T min
Limulators-TTo-Moorn
x 10 10 maximum time Taax = 1.0, generation schedule nonlinearity 0.5. For Two Moons, OUP, Turin and Gaussian Linear, we set the number of diffusion steps as N = 25 and the number of Langevin steps as Nr, 8. For BCI, we set N = 250 and N₂ = 0. In our experiments, we found that PriorGuide works better on BCI with a relatively large number of diffusion steps. The test-time compute across all simulators resides in the regime of 10a NFEs (number of function evaluations, i.e. calls to the score model).
parameter o 2 and a Langevin ratio
As detailed in Section of the main text, we approximate the prior ratio r(0) by fitting a Gaussian mixture model (cf. FITGMM in Algorithm 1). For Two Moons, OUP and Turin, do not need to fit a GMM since the training prior Pursin is a uniform distribution, thus r(0) = q()). For this reason, can directly use the target prior (()) as the GMM parameterization for r r(0). For Gaussian Linear and BCI, which have a Gaussian training prior, we fit a GMM with = 20 components to the ratio r()) by optimizing the L2 loss with gradient descent. For numerical convenience, we parameterize the GMM components with diagonal covariance matrices, The optimization is carried out in 1 x 106 steps with batch size 1,000 and learning rate 0.01 using the Adam optimizer (Kingma & Ba, 2015). We verified both via the L2 loss and via visualizations that the GMM fits achieved a satisfactory ratio approximation.
Amortized conditioning engine In ACE, first, we convert the test-time prior (()) generated in Appendix C.3.1 into a binned histogram distribution using the same steps discussed in Appendix C.2. Using the offline true data we have already sampled, we then condition on the full data (z) in the context set and predict the parameters independently to obtain the posterior. Using this posterior distribution, which is a Gaussian Mixture Model (GMM), we then sample 0 according to the number of posterior samples specified earlier. Note that ACE only supports factorized priors, so the mixture prior 9 cannot be specified correctly within ACE (i.e., ACE will represent the prior product of marginals).
C.3.3 POSTERIOR PREDICTIVE INFERENCE
In the data prediction task (posterior predictive), with equal probability we condition on the first 30% or the last 30% of the values in x, and then predict the remaining portion of x. Note that in this task, we do not condition on 0. We draw 100 samples from the data predictions for evaluation.
NNoe that PriorGuide also supports full covariance matrix representations for the GMM components of r(0).
29

---

Published as a conference paper at ICLR 2026
Simformer and PriorGuide We use the same hyperparameter settings and similar procedure as in the posterior inference setup (Appendix C.3.2). this case, the diffusion inference model will condition on the selected 30% values of to generate the remaining 70%. The steps for performing posterior predictive inference with PriorGuide are outlined in Algorithm 2.
Amortized conditioning engine For ACE, similar to the posterior inference setup, first convert the prior into a binned histogram distribution. We then place the portion of x to conditioned on into the context set. Data prediction is performed autoregressively by first randomly permuting the order of the target data points. This procedure is adapted from Bruinsma et al. (2023), as ACE is neural process-base based method. Consistent with the findings in Bruinsma al. (2023), we observed that using random ordering for the autoregressive procedure yields the most robust performance.
C.3.4 PARETO FRONTIERS OF TEST-TIME COMPUTE ASSIGNMENTS
PriorGuide supports improving the sampling quality by adding Langevin dynamic steps to the diffusion process, at the cost of additional test-time compute. With the same testing setup described above (Appendix C.3.2), we applied PriorGuide to posterior inference in the OUP and Turin simulators to investigate the influence additional Langevin dynamic steps. We measure the test-time compute (including diffusion steps N and Langevin steps N2) in the number of function evaluations (NFEs), f.e. calls to the score model, recalling the basic relation NFE = N x (Nz + +1). We tested the posterior inference performance in the regime up 10 NFEs and presented Pareto-efficient assignments of test-time compute.
C.4 STATISTICAL METHODOLOGG
C.4.1 EVALUATION METRICS
Root Mean Squared Error (RMSE) measures the average magnitude of the errors between the predicted posterior samples and the ground-truth. lower RMSE indicates a more concentrated prediction around the true parameters or data points. The RMSE is defined as
L N Noont
RMSE
Σ
Σ
(yt - (v,)
(A53)
L NNost
post
where T. is the feature dimension, N Npost is the number of posterior samples, Vs,r is the ground-truth and Dr.5 is the prediction for the 1-th feature (data point or parameter) and j-th posterior sample. In
posterior inference experiments, ground-truth parameters. Conversely, data prediction experiments, we compute the RMSE between 100 predicted data points and the ground-truth
compute the RMSE between Noost 1, 000 samples and the
N Npost =
Classifier Two-Sample Test (C2ST; (Lopez-Paz & Oquab, 2017)) a method to assess whether two sets of samples originate from the same distribution. In our context, is used to compare the estimated posterior samples against samples from a reference posterior distribution. In our experiments, a random forest classifier is trained to distinguish between samples from the two distributions. An accuracy close to 0.5 suggests that the two distributions are indistinguishable.
Mean Marginal Total Variation Distance (MMTV) quantifies the dissimilarity between two multi- variate distributions by considering their marginal distributions, defined as
00
M (xa) 9 (æa) dx
MMTV Σ d=1
(A54)
2D
where PM and M. denote the marginal densities of p and q along the d-th dimension. An MMTV metric of 0.2 indicates that, average across dimensions, the posterior marginals have an 80% overlap, often indicated as a desirable threshold for an approximate posterior (Acerbi, 2018).
Maximum Mean Discrepancy (MMD) measures the distance between the mean embeddings of the distributions in reproducing kernel Hilbert space. For our evaluations, we employ the MMD with with exponentiated quadratic kernel with lengthscale of 1.
30

---

Published as a conference paper at ICLR 2026
C.4.2 SIGNIFICANCE TESTING
To assess the statistical significance of the differences in performance between methods, we employ the Wilcoxon signed-rank test (Wilcoxon, 1945). In our experimental comparisons, we use this test determine the observed differences in metric scores between PriorGuide and baseline methods are statistically significant across multiple benchmark problems or datasets. The test evaluates the null hypothesis that the median difference between paired observations is zero, providing p-value to indicate the significance of any observed deviation from this null hypothesis. We consider result to be significantly different if the p- p-value is below 0.05.
D ADDITIONAL EXPERIMENTAL RESULTS
We present here additional experimental results. First, in Appendix D.1, we provide a visual example of test-time adaptation to supplement the discussion in Section 4.1. In Appendix D.2, we demonstrate PriorGuide's 's applicability to structurally complex priors through a sequential Bayesian inference experiment on Two Moons. In Appendix D.3, we introduce additional baselines, including rejection sampling, sampling- sampling-importancee importance-resampll al., 2019) with MCMC, to provide broader comparison. We also present study on the model's sensitivity to the distance between training and test-time priors (Appendix D.4) and conduct an ablation study the GMM prior ratio approximation (Appendix D.5), which is further validated by scalability analysis of the ratio fit in high-dimensional space (Appendix D.6). Furthermore, we include example posterior visualizations for the BCI model (Appendix D.7) and an analysis of training and test wall-clock time costs for Simformer and PriorGuide (Appendix D.8).
and neural likelihood estimation (NLE; Papamakarios
D.1 ILLUSTRATION OF TEST-TIME PRIOR ADAPTATION ON TWO MOONS
This section provides the figure referenced Section 4.1, illustrating the core capability of PriorGuide. We use the Two Moons model provide intuitive visualization of test-time prior adaptation. The base diffusion model is trained under a uniform prior Prain (0) over [-1, At test time, we introduce more localized Gaussian prior and employ PriorGuide to guide the diffusion sampling towards the correct posterior distribution (Fig. A1).
Original Posterior
New Prior
New Posterior
0.6
Diffusion samples
S
PrioeGuide samples
True parameters
New prior
-0.2
-0.6
-0.2
-066
-0.2
-0.6
-0.2
0
0
0
Figure A1: Left: Original Two Moons posterior and posterior samples from the base diffusion model trained on a uniform prior. Middle: New prior information about the parameters becomes available. Right: PriorGuide steers the diffusion process match the Bayesian posterior under the new prior.
D.2 SEQUENTIAL BAYESIAN UPDATING WITH INDEPENDENT OBSERVATIONS
To futher demonstrate PriorGuide's applicability complicated and non-factorized priors, examine sequential Bayesian inference scenario. We utilize the Two Moons model under uniform prior Puain (0) over [- 1, 1)2 and consider a two-step Bayesian update scenario:
1. We simulate two observations X given parameter vector 0*
2. Initial update: We calculate the intermediate posterior p(0 x1) given the first observation X1- In the Two Moons problem, this posterior by construction complex bimodal distribution (see Fig. Al and Fig. A2). This becomes our ground -rouhh target prior g()).
For PriorGuide, we perform prior ratio fitting by fitting a Gaussian mixture to this intermediate posterior (the ratio is (() due to the uniform prior).
31

---

Published as a conference paper at ICLR 2026
3. Sequential update: We employ the inference method of choice (PriorGuide another baseline) to sample from the final posterior p(0) X1 X1. X2), using g()) as the prior and conditioning the second observation X2. Notably, the fact that q(0) p(0)x) is itself posterior does not require any special treatment:
"yesterday's posterior is today's prior".
Using this setup, we evaluate PriorGuide and baselines across 10 true parameter points randomly sampled from Puain= simulating corresponding observation pairs for each. To establish ground truths for both the intermediate posteriors (and target priors) p(O X1) as well as the joint posteriors p(0) X1, X2), we numerically calculate these posteriors dense discrete grid (1000 1000 points) over the domain [-1,1]².
On posterior quality metries (C2ST, MMTV), PriorGuide strongly outperforms the baselines (Ta- ble A2). Here ACE underperforms, likely due to the fact that this complex prior differs significantly from the distribution of priors the mueta-prior-it has been pretrained on (see Appendix C.2), showing the importance of test-time prior adaptation. No difference is observed across methods for RMSE, but we recall that RMSE measures the error between the posterior and ground-truth parameter 0. RMSE is not meaningful when the posterior is multimodal like in this case: the error is dominated by the regions of the posterior in the wrong mode, even though it is correct Bayesian behavior for the posterior to be multimodal. An illustration of such bimodality and of the update procedure for PriorGuide is visualized in Fig. A2 for one example pair (x,x)). Overall, these results confirm PriorGuide's 's capability to effectively handle complex, data-derived priors.
Table A2: Posterior inference (p(@ X1, x2)). Mean + standard dev. over independent training runs (10 randomly generated true parameters and observation sets). Significantly best results (Wilcoxon signed-rank test) in bold.
Dataset
Method
RMSE
C2ST
MMTV
Simformer
0.63 0.34
0.89 + 0.02
0.66 + 0.07
Two Moons
ACE
0.63 0.37
0.94 + 0.03
0.71 + 0.07
PriorGuide
0.63 + 0.34
0.60 + 0.07
0.22 + 0.13
Intermediate Posterior
Fitted GMM
Posterior
Denety
PrieCCulde samples
Tuw parancias
-0.00
Figure A2: Left: The complex, bimodal posterior p(0 .i) derived from the first observation x Middle: This posterior is fitted with a GMM to serve as the new target prior (()) for the next step. Right: PriorGuide samples from the final posterior p(0]x,,xx by guiding the diffusion model with the new prior (0) and conditioning on the second observation X2. PriorGuide's samples match well the ground-truth posterior density.
D.3 ADDITIONAL BASELINES
D.3.1 REJECTION SAMPLING, AND SAMPLING IMPORTANT RESAMPLING
In addition to the main results, we compare PriorGuide against two straightforward sampling-based baselines:
Iewogggamnooeeesssiiiiie Resampling (SIR) and Rejection Sampling (RS), Both methods are widely used and theoretically consistent. However, their efficiency deteriorates rapidly when the proposal distribution in our case, the posterior from the base model under the training prior-is poor match for the target posterior under the new test-time prior, particularly under high-dimensional complex posterior geometries. Our experiments quantify this inefficiency and provide direct
32

---

Published as a conference paper at ICLR 2026
performance comparison under matched computational budgets, highlighting the practical advantages of PriorGuide.
Setup. For SIR, we drew 20,000 samples from the base diffusion model, applied importance weights, and evaluated performance alongside two diagnostics: effective sample size (ESS) and the instability measure (Vehtari et al., 2024), where k > 0.7 indicates instability. For RS, we continued sampling until either 1, ,,000 accepted samples (matching PriorGuide) were obtained or a two-hour time limit on an Nvidia A100 GPU was reached. All three methods (PriorGuide, RS, and SIR) rely on samples from the same Simformer model, with PriorGuide requiring additional per-sample computation. This allows us to align costs directly. As shown in Table A8, generating a single PriorGuide sample is roughly as expensive as producing 20 base Simformer samples. For our comparison we use 1,000 PriorGuide samples, 20,000 SIR samples, and cap RS at two hours to equalize compute budgets. This cap is generous as RS acceptance rates are very low, while PriorGuide sampling itself takes about ~ 155 seconds (see Table A8). Results are summarized below.
Table A3: Posterior inference (0) results on Turin with RS and SIR, under strong (S) and mixture (M)
test-time priors.
Dataset
Method
RMSE
C2ST
MMTV
Acc Rate (5)
ESS (%)
Simformer
0.23 0.04
0.95 0.02
0.07
ACE
0,18 1 0.02
0.92 0.03
0.47 1 0.08
Turin (5)
PrioeGuide
0.35
0.00
Dillusion SIR
0.06 1 0.01
0.81 1.07
0.18 1: 01.24
0.02 1: 0.01
0.35 1: 0.42
Diffusion
0.56 0.01
0.00
0.63 = 0.33
Simformer
0.24 1 0.04
0.95 0.02
0.52 1. 0.07
Turin (M)
ACE
0.20 0.03
0.91 0.03
0.07
PrioeGuide
0.14 1. 0.05
0.64 1: 0.09
0.21 1. 0.12
Diffusion
0.09 0.03
0.80 0.07
0.22
0.02 0.01
Diffusion RS
0.09 1 0.02
0.55 0.03
0.08 J. 0.04
0.49 a 0.35
Table A4: Posterior inference (0) results on BCI with RS and SIR, under strong (S) and mixture (M)
test-time priors.
Dataset
Method
RMSE
C2ST
MMTV
Ace Rate (%)
ESS (%)
Sinformer
0.80 0.08
0.97 :: 0.02
0.68 : 0.09
BCI (S)
ACE PriorGuide
0.34
0.15
0.91 1 0.05
0.40 0.17
0.21= 0.01
0.81 = 0.11
0.33 : 0.19
Diffusion + SIR
0.03 0.22
0.95 1 0.04
0.63 1 0.32
0.01 1 0.011
1.28 0.98
Diffusion RS
Rail
Fail
Fail
Simformer ACE PriorGuidd
1.07 0.18
0.99 :: 0.01
0.63 1 0.10
BCI (M)
1.00 0.31
0.97 =
0.58 + 0.10
0.79 0.70
0.78 1 0.11
0.35 0.24
Diffusion SIR
0.25 0.03
0.95 1 0.04
0.79 1 0.31
0.00 1 0.00
2.61 1.78
Diffusion + RS
Fiil
Foil
Fail
Table A5: Posterior inference (0) results Gaussian Linear 20D with RS and SIR, under strong (S) and mixture (M) test-time priors.
Dataset
Method
RMSE
C2ST
MMTV
Ace Rate (%)
ESS (%)
Simformer
0.30 0.02
1o0
Gaussian Linear 20D (8)
ACE
0.10 + 0.01
0.80 + 0.04
0.15 + 0.02
PriorGuide
0.21 = 0,08
0.58 0.03
0.10 11000:..2 0.02
SIR
0.15 0.02
~1
0.00
10.85 1.32
Diffasion RS
Fail
Fail
Fail
~
Sinforeer
0.30 :: 0.01
1.00 +: 0.00
0.61 0.02
0.03
Lo0 0.00
Gaussisn Lineur 20D (M)
PrjorGuide
0.21 1: 0.08
0.59 4 0.06
0.14 1 0.09
Diffasion SIR
0.15 d: 0.01
0.00 + 0.00
10.61 = 1.12
Diffasion RS
Fiel
Aail
Results. Under equal computational budgets, the results exhibit a dimensionality-driven behavior consistent with performance expectations of Monte Carlo methods in moderate dimensions (Robert & Casella, 2004). For the 4-dimensional Turin example (Table A3), the simple baselines remain viable, with performance roughly comparable to PriorGuide (sometimes slightly better, sometimes worse). RS achieves acceptable metrics but with highly variable acceptance rates, yielding only 98-126 accepted samples out of 20k.
33

---

Published as a conference paper at ICLR 2026
Conversely, in the 5-dimensional BCI problem (Table A4), these baselines catastrophically fail. Rejection sampling produces zero accepted samples most runs, while SIR's k > 0.7 indicates statistical instability that makes its estimates unreliable. This transition due to dimensionality (4D vs. SD) and the inereased posterior complexity reveal threshold where simple methods go from mildly unreliable to completely unusable, while PriorGuide maintains consistent performance across both problems.
Extending the analysis further to the 20D Gaussian Linear problem confirms this trend (Table A5). Despite being given a generous budget of 20k proposals (whereas each PriorGuide sample costs only fraction of Simformer sample), SIR and RS fail completely, while PriorGuide continues to deliver stable performance.
Larger computational budget. To examine behavior with substantially more compute, we ran rejection sampling until reaching 1, 1,000 accepted samples or an 8-hour limit on a single Nvidia A100 GPU (up to ~3.1 1. proposals) on Turin and BCI examples. While this setting is not typical for practitioners, allows us to probe performance under very large budgets. this regime, rejection sampling could sometimes match slightly outperform PriorGuide. On Turin with strong or mixture prior, both methods achieved similar accuracy (RMSE =0.06-0.09, C2ST 00.54-0.62. MMTV =0.07700.19 with acceptance rates of 0.38-0.5799 On BCI, rejection sampling occasionally improved RMSE and MMTV (e.g.. 0.22 vs. 0.63 RMSE on BCI-M), but acceptance was extremely low (0.06.0.166), many runs hit the timeout, and some produced only a handful of accepted samples.
These results illustrate a trade-off: simple methods can approach strong performance if one is willing spend hours of compute, but they remain unreliable and inefficient. PriorGuide achieves comparable quality in minutes (~1600 s for 1K samples), making it suitable for interactive use, repeated posterior evaluations, and sensitivity analysis, rather than only very large-budget scenarios.
D.3.2 NEURAL LIKELIHOOD ESTIMATION WITH MCMC
As an additional natural baseline, we evaluate a method based on neural likelihood estimation (NLE; Papamakarios et al., 2019) followed by MCMC sampling. This two-stage method is a representative of several likelihood-based approaches in SBI (Lueckmann et al., 2021), which trains a neural density estimator (e.g. normalizing flow) to form surrogate of the likelihood function p(x O) [Papamakarios et al., 2019). Once the surrogate is trained, it is then used with a standard MCMC algorithm to draw samples from the posterior distribution p(0 xx) for given observation X.-
While decoupling likelihood learning from posterior sampling is conceptually natural in SBI, we find this approach may lack robustness even in low-dimensional cases, particularly for problems with multi-modal posterior geometries. For instance, when this approach is applied in the bimodal Two Moons simulator Appendix C.1, the MCMC sampler guided by the learned likelihood often becomes trapped in a single posterior mode and fails to discover the other (Fig. A3). This failure is a known MCMC limitation that the sampler can easily get stuck in modes.
D.4 SENSITIVITY TO THE DISTANCE BETWEEN TRAINING AND TIST-TIME PRIORS
To investigate the sensitivity of PriorGuide the distance between the training and test-time priors, we conduct an experiment using the Gaussian Linear 10D model. We use the same training prior as in Section 4: a 10-dimensional Gaussian distribution Pusin (0) = (0, 0.1 I1o). For the test-time priors, we use strongly informative standard deviations (o strong 0.2 s;) and systematically shift their means (f1") away from the center of the training prior (ạ=²). We quantify this shift using several metrics, including the Mahalanobis distance (d"), the 2- Wasserstein distance, and our proposed out-of-distribution (OOD) diagnostic (Appendix A.4).
The results, presented in Table A6, reveal how PriorGuide's 's performance responds to this increasing prior shift. The key take-home message is that PriorGuide is robust and degrades gracefully rather than failing catastrophically. For small to moderate shifts (e.g., d' < 2), performance remains nearly identical to the no-shift scenario, with all metrics (C2ST, MMTV, RMSE) showing negligible changes. As the prior is shifted into a substantially different region, we observe a smooth and predictable decline in accuracy. For instance, an extreme shift where d' 4.4. PriorGuide achieves MMTV of 0. 0.16, indicating that the generated posterior marginals have an 84% overlap with the ground truth on average across all dimensions, which corresponds to reasonable posterior approximation (Acerbi,
34

---

Published as a conference paper at ICLR 2026
Normalized Posterior Density
0.4
0.0168
0.2
0.0144
0.0120
0
0.0
0.0096
0.0072
-0.2
0.0048
0.0024
-0.4
NLE + MCMC Samples
0.0000
-0.4
-0.2
0.0
0.2
0.4
0
Figure A3: Failure case of the NLE with MCMC baseline on the Two Moons problem. The posterior approximated by running MCMC sampler on learned neural likelihood.
2018). This analysis confirms that PriorGuide provides provides meaningful approximations even when the test-time prior is far from what was seen during training.
Table A6: Sensitivity analysis of posterior inference to prior shift. The degree of shift is quantified using both d' (Mahalanobis distance) and Wasserstein distance. The results indicate that PriorGuide maintains competitive performance even under substantial prior shifts.
1 |
d'
2- Wasserstein OOD Frac.
C2ST
MMTV
RMSE
0.00
0.00
0.84
0.00
0.52 + 0.05
0.05 + 0.01
0.21 + 0.10
0.32
0.44
0.89
0.00
0.53 + 0.06
0.06 0.02
0.20 0.20
0.63
0.88
1.05
0.00
0.54 0.07
0.06
0.23
0.95
1.32
1.27
0.00
0.56 + 0.08
0.07
0.13
1.26
1.76
1.52
0.00
0.59 0.09
0.08
0.15
1.58
2.20
1.79
0.02
0.01
0.10
0.14
1.90
2.64
2.07
1.00
0.09
0.10
0.16
2.21
3.08
2.37
1.00
0.64
0.04
0.14
2.53
3.52
2.66
1.00
0.65
0.12
1.19
2.85
3.96
2.97
1.00
0.67 0.08
0.14
1.25
3.16
4.40
3.27
1.00
0.70 0.08
0.16
1.42 0.24
D.5 IMPACT OF GMM COMPONENT COUNT ON PRIOR RATIO APPROXIMATION
We conduct an additional ablation study on the influence of the number of GMM components used to approximate the prior ratio. To isolate this effect, this experiment uses controlled setup that differs from the main experiments: we evaluate performance using one target prior and one trained diffusion model, with metrics averaged across ten observations generated from that prior. As shown Table A7, sampling speed remains efficient even with very large GMMs (e.g. 200 components), indicating that is not a bottleneck at this stage. contrast, the performance of PriorGuide sensitive to the quality of the prior ratio approximation when using very few components. We therefore recommend that users fit a flexible and highly expressive GMM tailored to their specific application to ensure accurate that 20 components enough in experiments.
and stable guidance. In practice,
35

---

Published as a conference paper at ICLR 2026
D.6 SCALABILITY AND ROBUSTNESS OF THE PRIOR RATIO APPROXIMATION
To assess the scalability of our GMM fitting procedure for the prior ratio r(0), we evaluate the quality of the approximation in 10-dimensional setting. We use the Gaussian Linear 10D setup and set (()) as Gaussian mixture.
We execute the fitting procedure for 100 randomly sampled mixture target priors (using the procedures detailed in Appendix C.3.1). For each fit, we estimate the weighted L2 distance in log space, that the Ly distance between the GMM approximation log predictive density (including the fitted log normalization constant In Z) and the true analytic log ratio using 10,000 samples from the fitted GMM. The method achieves a median L2 error of 0.023 75th 75th-97.5tt percentiles: 0.002- 0.016-0.032-0. 86), confirming that standard gradient-based optimization consistently converges to an accurate approximation of the ratio function in this 10D scenario." practice, to address the small percentage of fits that converge suboptimal solutions, we recommend running multiple optimizations with different initializations which is the standard approach when solving a global nonconvex optimization problem.
To give an example of such fits, Fig. A4 displays a fitted GMM ratio against the true analytic ratio in log-space. We evaluate both the true analytic ratio and the fitted GMM passing through fixed parameter vector 8" illustrating the function's 's behavior for each pair of dimensions while holding the remaining dimensions constant. The figure demonstrates that the fitted GMM (orange lines) faithfully recovers the geometry of the true ratio (dashed black lines) across these conditional slices.
Table A7: Ablation study illustrating the influence the number of GMM components used to approximate the prior ratio.
#GMM
Time cost (s) per sample
Dataset
C2ST
MMTV
RMSE
components
0.02 = 0.00
0.97 + 0.01
0.31
0.20 + 0.03
Gaussian
20
0.02 + 0.00
0.55 1 0.04
0.09
0.23 = 0.07
Linear 10D
200
0.02 + 0.00
0.56 0.05
0.11
0.28 + 0.11
2
0.18 + 0.01
1.00 0.00
0.72 + 0.04
1.71 + 0.21
BCI
20
0.18 0.01
0.10
0.57 = 0.24
1.36 + 0.66
200
0.18 - 0.01
0.88 +.10
0.57 0.24
1.36 + 0.65
D.7 BAYESIAN CAUSAL INFERENCE (BCI) MODEL POSTERIOR SAMPLES VISUALIZATION
In Fig. A5, visualize the ground-trute posterior distribution alongside samples generated by Simformer (no prior adaptation) and with PriorGuide applied to the same base Simformer model. We the corner package (Foreman-Mackey, 2016), which displays pairwise joint distributions and marginal histograms of the parameters.
D.8 TRAINING VS. TEST-TIME COST ANALYSIS
We summarize in Table A8 the estimated wall-clock time required by PriorGuide produce 1, 1,000 posterior samples under each simulator, measured on a system equipped with an Nvidia Ampere A100 GPU. For comparison, we also report the cost of retraining diffusion inference model (Simformer, (Gloeckler et al., 2024)) with 10,000 simulations, which includes both the neural network training and the simulator call required to create training set. As Table A8 demonstrates, PriorGuide achieves more than a tenfold speedup across most simulators, with the greatest gains observed in scenarios (e.g., Turin) where the simulator calls are most computationally expensive.
E COMPUTATIONAL RESOURCES AND SOFTWARE
Computational resources Most experiments presented in this work are performed a cluster equipped with AMD MI250X GPUs, while some additional experiments in the appendix are per-
Note that 0.023 in log space corresponds 2% the ratio, since e 1 + for small
36

---

Published as a conference paper at ICLR 2026
61
8
8
010
Figure A4: Example visualization of the prior ratio approximation in 10D (Gaussian Linear problem). Conditional 2D cross-sections of the true prior ratio r(0) (dashed black lines) versus the fitted GMM approximation (orange lines). Note that the target ratio is not Gaussian, as visible in some panels. The GMM approximation achieves a near- near-perfect fit.
Table A8: A comparison of time costs (seconds) between performing PriorGuide posterior inference and retraining Sinformer, including neural network training and generating new training data.
Simformer retraining and testing
Simulator
PriorGuide testing
Total
Simulation
Training
Sampling
Two Moons
9.7
314.4
13.5
295.2
5.7
OUP
25.1
305.7
9.2
289.4
7.1
Turin
159,9
9346.6
8300.4
1037.4
8.8
Gaussian Linear
16.8
266.1
3.8
255.2
7.1
BCI
158.8
1063.1
49.1
1005.7
8.3
formed cluster equipped with Nvidia Ampere A100 GPUs. The total computational resources consumed for this research, including all development stages and experimental runs, are estimated to be approximately 30,000 GPU hours.
37

---

Published as a conference paper at ICLR 2026
(b)
Figure A5: Posterior samples for the Bayesian causal inference (BCI) model, for a randomly chosen dataset. Ground-trut samples are shown in black, and the dashed black lines and black dot indicate the true parameter values. (a) Original Simformer samples (green) without prior adaptation, and (b) PriorGuide samples (orange) under a mild prior. Incorporating prior information with PriorGuide steers the inference toward more plausible parameter regions, especially for parameters such as m and Psame which are less constrained by the data alone, resulting in posterior samples that more closely match the ground-truth posterior under the same prior, shown in (b).
Software The core code base is built upon the Simformer repository (https: /gipt/ub///thbbbb// com/ hon/mackelab. heelllaasmmmfooe simformer, License: MIT), using JAX (https://docs) jax. dev//n// latest/ License: Apache-2.0) and PyTorch BSD license). The implementations the Two Moons and Gaussian Linear simulators are based on sbi bm (https: / thub. com/ http://ti--beeccmarr//bbbbbbbb License: MIT). For the ground- Licens
org/, License: modified
truth posterior generation, we sbi License: Apache-2.0) and PyVBMC fattss:/accebblaa...ihhb..b htt.:ttthh....ooy//////ooo License: BSD 3-Clause). Our implementation the ACE baseline uses the repository providde by (Chang et al., 2025) (https: / //gttttbbbb// www/acerbilll///eeeeeeaamaaaaaaa htt.://......a.tt...aoaoooaaaaaa cense: Apache-2.0).
Li-
F USE OF LARGE LANGUAGE MODELS
We acknowledge the use of Large Language Models (LLMs) to support various stages of the research. In the initial phase, LLMS utilized for inspirations, helping explore methodological approaches and find existing works. In the development phase, L.LMs served as programming assistants to aid tasks such implementing algorithms and debugging. LLMs also supported the writing process, providing assistance with polishing the manuscript for clarity, conciseness, and grammatical correctness.
38


Reviews:
[{"review_id": "ndjHBoLRfW", "reviewer": "Reviewer_HVmz", "summary": "This paper considers the amortized simulator-based inference framework and one of its major limitations, i.e., prior distributions used to generate model parameters during training. The authors propose a novel method, PriorGuide, which leverages a guidance approximation for flexible test-time prior adaptation. The method was evaluated on a set of benchmarks. The most significant strength is requiring no retraining.", "questions": "Besides the Weaknesses, I have also a question: \n1. The GMM approximation of the prior ratio $r(\\theta)$ appears to be the most critical (and fragile) part of the pipeline. How does the $L_2$ fitting procedure scale with the dimensionality of $\\theta$? Have you considered replacing this heuristic fit with more robust, modern density-ratio estimators (e.g., flow-based, MINE, etc.)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper has a few strengths:\n1. The \"prior-rigidity\" of amortized SBI models is a well-known and significant bottleneck for scientific application. This paper offers a direct, \"plug-and-play\" solution that enables, for the first time, post-hoc prior sensitivity analysis and the incorporation of new expert knowledge using expensive, pre-trained models.\n2. The method is built on a sound mathematical foundation. The derivation of the target posterior as a tilted distribution (Proposition 1) is clear and correct.\n3. The experiments are convincing. The paper correctly identifies that its method is an approximate sampler and intelligently introduces Langevin correctors to manage the accuracy-compute trade-off.", "weaknesses": "However, despite it’s strengths, there are also weaknesses:\n1. The entire method hinges on a critical, and likely fragile, approximation: fitting the prior ratio $r(\\theta) = q(\\theta) / p_{train}(\\theta)$ with a GMM. The paper understates the difficulty of this step. This is a density-ratio estimation problem, which is notoriously difficult, especially as the dimension of $\\theta$ increases. The paper's solution (a gradient-based $L_2$ fit) is a heuristic that is not guaranteed to be stable or accurate, particularly if $p_{train}(\\theta)$ is non-trivial.\n2. The paper employs two major approximations: the GMM for the ratio and a simple diagonal Gaussian for the reverse kernel. It is the combination of these (especially the latter) that biases the sampler and requires the heavy use of Langevin correction steps to achieve good results. This means the method, while \"retrain-free,\" has a high and non-trivial per-sample inference cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Oij8ESdy43", "reviewer": "Reviewer_DTXg", "summary": "The paper consideres an interesting problem of adapting a diffusion model pretrained to sample from a posterior when a different (unknown at the training state) prior model is adopted by the user. The proposed algorithm, PriorGuide, is able to correct biase of the pretrained diffision model during the sampling stage and providing a test time adaptive inference. \n\nThe guidance is intractable but is proximated using Gaussian mixture models. \n\nThe experiments show that on a wide variety of bechmark datasets, the proposed method outperforms similar diffusion methods with no-prior adaptation.", "questions": "None.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, I think this paper makes a solid contribution to the SBI community. \n\nThe problem of changing priors is indeed common in many applications, and the ability to adapt a pre-trained posterior model without retraining the entire model is highly desirable and can make a lot of sense when the user's prior is different and comes in at a later stage. \n\nDiffusion guidance is a well-studied technique in the generative model adaptation, so it is natural to leverage this approach to adapt SBI diffusion models. To my best knowledge, the proposed idea is novel. \n\nThe proposed method is mathematically sound, and its resulting formulation is natural, separating the pre-trained model from the guidance term (the expected prior shift). \n\nAlthough the guidance term itself is intractable, authors show it can be effectively approximated.", "weaknesses": "My concerns are mostly on the literature review: \n\nIn the main text, the authors do not provide a comprehensive review or comparison of prior adaptation methods in the existing literature.  As briefly mentioned in the introduction and the experimental section, there are already methods that consider adaptive priors. Although the authors include a separate section in the appendix, a dedicated subsection in Section 2 would be appreciated, as readers would like to understand the specific limitations of these prior adaptation methods and how (or whether) they relate to the proposed guidance method.\n\nIn Section 2, recent progress (2022 ~ ) on applying diffusion models to SBI should also be comprehensively reviewed and cited as they provide the backdrop (the pre-trained posterior score) to the proposed method. To the best of my knowledge, the earliest attempt to use diffusion models in SBI is often attributed to Geffner et al. (2023), first published on arXiv in 2022. Simformer is a subsequent work built on Geffner et al. and others, introducing masks and a transformer architecture to handle missing observations and more versatile inference tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zrvNm2pUfq", "reviewer": "Reviewer_XCTR", "summary": "**PriorGuide** targets a practical SBI need: adapting a trained diffusion-based posterior estimator to **new priors at test time** with **no retraining**. The paper derives a clean **score decomposition** in which the target posterior score equals the trained score plus a **prior-ratio guidance** term . To make this usable, the authors (i) use a *standard* Gaussian reverse-kernel approximation (via Tweedie’s formula) and (ii) fit a Gaussian mixture to the prior ratio, yielding a *novel* closed-form guidance update that plug directly into the diffusion sampler. Optional few-step Langevin corrections at low noise tighten asymptotics and expose a neat compute–accuracy knob alongside the diffusion steps. Empirically, PriorGuide improves both posterior and posterior-predictive metrics across several SBI tasks while keeping the implementation lightweight and training unchanged.", "questions": "1. **Non-factorized complex priors**: Have you tried dense-covariance Gaussians or general complicated priors? Have you tried inference on iid data using PriorGuide?\n2. **Better Gaussian reverse Kernel approximations**: There exists better reverse kernel approximations (i.e. [1] and related). These usually are more costly i.e. require Jacobians or other terms. Would be interesting to see if these also help in that case but atleast should be discussed in the manuscript.\n3. **Matrix calculations**: The authors state that a current limitation is that \"matrix-operations  may-pose a scalability issue\". A straight forward approach to avoid this is to constrain the involved covariance to be *diagonal* (or other approx.). After all a mixture of Gaussians with diagonal covariance is still a universal approximation family (just may needs more components). As the backward kernel approximation is by design also diagonal the whole Guidance terms are reduced to the diagonal. This hence makes *PriorGuideDiag*  an interesting alternative to the current approach.\n\n[1] Boys, Benjamin, et al. \"Tweedie moment projected diffusions for inverse problems.\" _arXiv preprint arXiv:2310.06721_ (2023).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Well written easy to follow**:The derivation is easy to follow; the GMM prior-ratio leads to implementable, closed-form guidance. Propositions are sound and proven in the Appendix.\n- **Good empirical results with clear trade-offs and ablations**: Consistent gains on posterior and predictive metrics; straightforward ablations over diffusion/Langevin steps make hyperparameter selection transparent. The paper is upfront about approximation choices and where Langevin steps matter, which helps practitioners reason about when to use the method.", "weaknesses": "- **Prior family diversity**: Main experiments emphasize changes among Gaussian priors (often **diagonal covariance**). This is required for fair ACE comparisons but does not demonstrate  the claimed method’s generality. A demonstration on **non-factorized** or more structured priors (complicated priors) would strengthen external validity. For example one use-case that requires flexible priors would be for \"i.i.d\" data which also can be handled by sequentially updating the \"prior\" with the \"current posterior\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "leKYjUMk8V", "reviewer": "Reviewer_kkwQ", "summary": "PriorGuide introduces an extension to framework of simulation-based inference (SBI) with diffusion models that allows pre-trained diffusion models to incorporate new prior distributions at inference time without requiring retraining. This is achieved using a novel Gaussian mixture model approximation to make the target prior tractable for guiding the diffusion sampling process. Empirical results on synthethic data demonstrate PriorGuide’s effectiveness in accurately recovering posterior and posterior-predictive distributions across various SBI problems. The method also allows for refinement through Langevin dynamics, enabling a balance between computational cost and accuracy.\n\nBy enabling adaptation to new priors without retraining, PriorGuide exemplifies a “test-time compute” approach, extending the capabilities of pre-trained models with targeted computations. This decoupling of simulator runs from prior specification offers practical benefits especially in the case where simulations are costly, including the ability to perform post-hoc prior sensitivity analyses and incorporate domain expert knowledge after training, ultimately reducing the computational burden of scientific workflows which employ SBI.", "questions": "- page 4, the derivation of eq 6-9 is central to the paper, please try to rearrange so that the text in lines 216-220 is readily visible close the equations (e.g. by removing lines 191-195)\n- the code for the experiments was not found in the paper, i.e. through an anonymised git repo, this should be corrected for publication\n- line 232: please do not use anthropomorphic language to describe model behavior \"the model having seen few training examples\", i.e. the model has consumed or been trained on\n- line 234: \"using standard OOD metrics\" as such a standard exists only colloquially, please remove \"standard\"\n- equ 13: the text lacks a discussion if `K` is a free parameter and how to choose it\n- line 274: \"Notably, when p(θ) is uniform, r(θ) reduce*s* to q(θ), which can be directly specified as a Gaussian mixture.\" perhaps also add hint why r reduces to q (due to using the expectation I guess)\n- line 310: \"To incorporate this with regular diffusion ...\" perhaps comment on the computational cost here already.", "limitations": "", "rating": 10, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The manuscript proposes an extension of SimFormer modeling to the challenging problem of representing complex joint probability distributions. The core idea is relatively straightforward, yet the authors provide a rigorous mathematical treatment that appropriately identifies both the potential benefits and inherent limitations of this approach, acknowledging the necessary reliance on approximations. The empirical evaluation is a strength of the paper; the results presented demonstrate promising performance, particularly in the accuracy of the reported uncertainty estimates (Tables 1 & 2). The selection of benchmarking use cases is clearly motivated and relevant to the stated goals. Finally, the authors effectively utilize concise and informative visualizations to support their claims and illustrate the underlying hypothesis.", "weaknesses": "The manuscript would benefit from a more careful review of its language, as instances of anthropomorphism (e.g., “the model having seen few training examples”) can detract from the scientific rigor. Additionally, the presentation of the derivation for 'r' is hampered by misaligned page breaks, separating key explanatory text and hindering comprehension. More significantly, while the paper thoroughly explores the technical capabilities of PriorGuide, it lacks a discussion of the broader implications of manipulating the prior distribution within a Bayesian framework. The authors appear to assume a familiarity with this interplay between prior, simulator, and the scientific endeavor, which may limit the accessibility and impact of the work for a wider audience.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d8oDRCk4Yk", "reviewer": "Reviewer_zfhL", "summary": "The paper proposes PriorGuide, a method to adapt diffusion-based amortized\nsimulation-based inference (SBI) models to new priors at test time without\nretraining. The key idea is to express the target-posterior score as the original score\n(under the training prior) plus a guidance term involving the prior ratio\n$(r(\\omega)=q(\\omega)/p_{\\text{train}}(\\omega))$. To make the guidance tractable, the\nreverse transition kernel is approximated as Gaussian and the prior ratio is\napproximated by a Gaussian Mixture Model (GMM), which yields a closed-form guidance\nupdate. The method is evaluated on a suite of SBI problems for both posterior and\nposterior-predictive inference. Empirically, PriorGuide attains competitive performance\nrelative to baselines, and the paper further studies a compute–accuracy trade-off\nusing interleaved Langevin refinement.", "questions": "1. Could the authors provide clearer practical guidance on when PriorGuide is\n   preferable to simpler or amortized alternatives? For instance, how should\n   practitioners trade off its added test-time computation against retraining costs or\n   simpler posterior-correction methods?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "**Quality.** The paper’s theoretical foundation is solid: the score decomposition with a\nguidance term is standard and correctly instantiated for the SBI setting. The Gaussian\nreverse-kernel and GMM prior-ratio approximations are reasonable and enable a practical\nalgorithm. The analysis of test-time compute via Langevin refinement is appropriate, and\nthe extended experiments cover canonical SBI tasks. The ablation and sensitivity studies\nin the appendix further strengthen the empirical grounding.\n\n**Clarity.** The manuscript is exceptionally well written, with well-structured\nexposition and clear intuition for the guidance term. Notation is introduced early and\nconsistently; figures and tables are easy to interpret. The appendix integrates detailed\nmethodological clarifications—GMM fitting, diagnostics, and runtime analyses—that\nimprove reproducibility. The discussion of the Pareto front between diffusion and\nLangevin steps is particularly insightful.\n\n**Significance and Contribution.** The work addresses a practically important\nproblem—changing priors after training—where retraining amortized SBI models can be\nprohibitively expensive. Adapting diffusion guidance to encode new priors at inference\nis an appealing and useful contribution. The empirical scope is strong, especially given\nthe additional baselines in the appendix (Rejection Sampling, Sequential Importance\nResampling, NLE+MCMC), a sensitivity analysis, and a 20D test demonstrating scalability.\nThe contribution remains primarily empirical but offers meaningful methodological value\nfor the SBI community.\n\n**Originality.** Applying diffusion guidance to test-time prior adaptation in amortized\nSBI is a natural but novel design choice. The GMM ratio approximation provides a\npractical mechanism to operationalize this guidance, and the extended analyses confirm\nthat the method maintains competitive performance where simpler baselines fail.", "weaknesses": "**Scope and positioning.** While the paper clearly demonstrates PriorGuide’s strengths,\nit could offer more explicit practical guidance for practitioners—specifically, when\nPriorGuide should be preferred over simpler baselines such as NLE + MCMC, RS, SIR, or\nrelated methods like SIMFORMER and ACE and how to balance its additional test-time\ncomputation against retraining or posterior-correction methods. While this is discussed\nqualitatively in the appendix, a concise summary in the main text would make the\ncontribution more actionable.\n\n**Dimensionality constraints.** Although the new 20D Gaussian Linear results demonstrate\nscalability, these remain relatively simple benchmarks. A more complex or real-world\nhigh-dimensional example would strengthen the claim of general applicability.\n\n**Approximation limits.** The Gaussian reverse-kernel approximation, while adequate for\nthe reported tasks, remains a simplifying assumption. Even with Langevin refinement\nmitigating some of its effects, further diagnostic guidance for practitioners would be\nuseful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)