Title: Reverse-Annealed Sequential Monte Carlo for Efficient Bayesian Optimal Experiment Design
Venue: NeurIPS 2025
Accepted: yes
Keywords: Bayesian optimal experiment design

expected information gain

mutual information

model evidence

sequential Monte Carlo

Markov chain Monte Carlo

Bayesian methods
Abstract: Expected information gain (EIG) is a crucial quantity in Bayesian optimal experimental design (BOED), quantifying how useful an experiment is by the amount we expect the posterior to differ from the prior.
However, evaluating the EIG can be computationally expensive since it generally requires estimating the posterior normalizing constant.
In this work, we leverage two idiosyncrasies of BOED to improve efficiency of EIG estimation via sequential Monte Carlo (SMC).
First, in BOED we simulate the data and thus know the true underlying parameters.
Second, we ultimately care about the EIG, not the individual normalizing constants.
Often we observe that the Monte Carlo variance of standard SMC estimators for the normalizing constant of a single dataset are significantly lower than the variance of the normalizing constants across datasets; the latter thus contributes the majority of the variance for EIG estimates.
This suggests the potential to slightly increase variance while drastically decreasing computation time by reducing the SMC population size, which leads us to an EIG-specific SMC estimator that starts with a only a single sample from the posterior and tempers \textit{backwards} towards the prior. Using this single-sample estimator, which we call reverse-annealed SMC (RA-SMC), we show that it is possible to estimate EIG with orders of magnitude fewer likelihood evaluations in three models: a four-dimensional spring-mass, a six-dimensional Johnson-Cook model and a four-dimensional source-finding problem.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- EIG estimation is all about computing $E_{y\\sim p(y)}[\\log p(y)]$\n- The variance over $y$ can be as significant as the variance/error coming from the estimate of the partition function\n- A small number of inner particles can be sufficient\n- In fact we can use 1 particle with an improved backwards SMC method, by observing that we get exactly 1 free sample of the posterior since EIG estimation uses data simulated from the current prior\n- the resulting method gives good EIG estimates at a lower likelihood evaluation budget compared to earlier SMC-based approaches\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n - the paper reads very well, it clearly lays out the overall story in a logical manner\n - the idea leads to a more efficient algorithm\n - I believe this approach to be novel\n - theoretical analysis in the form of an asymptotic unbiasedness result is provided\n - clear demonstration in the experiments that this approach requires fewer likelihood evaluations to achieve good quality results compared to some earlier SMC-based algorithms for EIG optimization\n\n# Weaknesses\nThe biggest shortfalls are things that weren't in the paper\n - my question with SMC approaches for BOED is always how they will scale with dimensionality. As far as I can see, scaling with dimensionality of $\\theta$ was not studied here. I think this could be worthy of investigation. In fact, the fact that you are utilizing a single particle initialized from your true theta sample gives me some hope that this might be more scalable than other SMC approaches. I think adding this would have made the paper even better\n - you only compare against other SMC approaches except in Appendix A.3 where you study NMC. I would definitely be interested to see Prior Contrastive Estimation (PCE) https://arxiv.org/pdf/1911.00294 eq (12) on the table in A.3 as well-- it can be a deceptively good baseline for virtually the same cost as NMC. Ideally it would be added to the main plots as well. My motivation to suggest this is that you make an argument that some error in $\\log p(y)$ is fine because of the variance over $y$. In that case, maybe we can start to tolerate these even cruder approximations?\n\nQUESTIONS:\n- what about EIG gradients w.r.t. $d$? This would be an interesting angle to study as I don't believe it has been attempted using SMC before (though I could be wrong)\n - \"Magnetic reverse-annealed SMC steps\" this sounds like an interesting addition, but could that not be a general part of the algorithm? Were the results for the mass-spring system significantly worse without this amendment?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach called Reverse-Annealed Sequential Monte Carlo (RA-SMC) to estimate the Expected Information Gain (EIG) in Bayesian Optimal Experimental Design (BOED). The method leverages the specific characteristics of BOED to achieve efficient estimation, requiring only a single sample (particle) to provide a good approximation. The paper also provides a comprehensive theoretical foundation for the proposed method and demonstrates its efficacy on various models, showing significant computational savings compared to traditional methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The idea of using reverse annealing for EIG estimation in BOED is highly innovative. The authors effectively exploit the characteristics of BOED, such as known underlying parameters and the goal of estimating EIG, to propose a method that drastically reduces the computational cost.\n\n2. The paper provides a solid theoretical basis for RA-SMC, demonstrating that a single particle can effectively estimate EIG. The proofs and discussions are well-detailed.\n\n3. The method is shown to significantly reduce the number of likelihood evaluations required, offering computational efficiency without sacrificing accuracy. This is demonstrated across several models.\n\n4. The paper is clearly written, with a logical flow from introducing the problem to detailing the RA-SMC approach and providing empirical results. The experimental setup is easy to follow, and the comparison with the forward SMC method is well-executed.\n\nWeaknesses:\n\n1. Eq. (8) is fundamental to the method's theoretical framework, but its validity is not immediately clear, and the authors have not provided a proof. A formal proof or additional justification for this formula would strengthen the theoretical rigor of the paper.\n\n2. While the paper demonstrates that a single particle is sufficient for RA-SMC, there is limited discussion on the potential benefits of using multiple particles. It would be useful to explore whether using multiple particles (similar to particle filter methods) could improve performance, especially in cases with high posterior variance or complex models.\n\n3. The term \"Sequential Monte Carlo (SMC)\" is widely used in various fields, particularly in filtering and dynamic systems. While this term may be appropriate within the context of your work, I would like to point out that this could lead to some confusion for readers coming from other areas. It may be helpful to clearly define the SMC terminology in the context of your work, or alternatively, consider a different name to distinguish your approach from the SMC methods for filtering used in other domains.\n\n4. The paper only compares RA-SMC with forward SMC, but it would be more informative to include comparisons with other state-of-the-art methods for EIG estimation.\n\nQUESTIONS:\n1. Could you provide a formal proof or further justification for the validity of Formula (8)? This would address any concerns about the theoretical foundation of the method.\n\n2. While the paper demonstrates the sufficiency of using a single particle, could you explore and discuss the potential use of multiple particles (perhaps inspired by particle filters) to further enhance the performance of RA-SMC, especially for complex or multimodal posteriors?\n\n3. Given that \"Sequential Monte Carlo\" is widely used in the context of filtering, would it be possible to reconsider the use of this term for the proposed method? Alternatively, can you provide a clearer distinction between your method and traditional filtering-based SMC techniques?\n\n4. It would strengthen the paper to include a comparison with other modern methods for EIG estimation in BOED. This could help further validate the efficiency and accuracy of RA-SMC.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript proposes reverse--annealed sequential Monte Carlo for estimating the expected information gain (EIG) that drives Bayesian optimal experimental design.  The standard tempering route—used by sequential Monte Carlo samplers (Chopin 2002; Del Moral et al. 2006) and by many BOED works—starts with the prior and anneals forward toward the posterior with many particles.  Contrary to these forward tempering schemes, RA--SMC begins with an exact posterior draw obtainable in simulation--based design, then anneals backwards towards the prior while accumulating the log--evidence by a thermodynamics inspired integral.  The authors prove the estimator is asymptotically unbiased and demonstrate, on 4--6 dimensional case studies, that RA--SMC can match the accuracy of forward SMC using two orders of magnitude fewer likelihood evaluations.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nThe concept is elegant and practical.  It turns the necessity of prior sampling into an advantage by bootstrapping from a posterior sample naturally available in simulator settings. The theoretical development is solid. Lemma~4.2 establishes consistency under standard ergodicity conditions, and the temperature--schedule corollary offers practical guidance. The empirical study is careful. Each benchmark reports Monte Carlo error bars and raw likelihood budgets, allowing a transparent cost/accuracy comparison. The variance--decomposition analysis strongly justifies the single--particle regime, and the discussion of diagnostic uncertainty is refreshingly candid.\n\nPrior work treats the posterior normalising constant Z(1) as unknown and therefore anneals forward. In BOED the authors observe that a joint prior draw  yields a known posterior density  p(θ∣y), so the reverse normalising constant  Z(0)=1 is known instead. Turning this observation into a practical estimator is, to my knowledge, new.\n\nWeaknesses\n\nAll experiments remain in low--to--moderate dimensions with smooth likelihoods. Scalability to high--dimensional, multimodal or PDE--constrained models is not demonstrated.\n\nBaseline coverage is narrow: annealed importance sampling, bidirectional Monte Carlo bounds, variational MI estimators and neural mutual--information surrogates are omitted despite being cited.\n\nRuntime is measured only in likelihood calls; wall--clock timings and parallel scalability are not evaluated.\n\nQUESTIONS:\nAs far as I can understand, RA--SMC relies on the availability of an exact posterior sample, limiting applicability to fully simulator--based design; real--data or sequential--design contexts without simulators are excluded?\n\n Can likelihood--free posterior draws relax the ``exact posterior'' requirement and open RA--SMC to real--data?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an algorithm for the estimation of the expected information gain (EIG) for Bayesian optimal experimental design via sequential Monte Carlo (SMC). One of the main quantities of interest for Bayesian experimental design algorithms is the marginal likelihood $p(y | d) = \\int p(y|\\theta, d) p(\\theta) d\\theta$ under a given design $d$.  The paper derives a reverse annealing scheme which samples $(y, \\theta_*) \\sim p(y, \\theta)$ from the joint distribution treating the initial $\\theta_* \\sim p(\\theta|y)$ effectively as a posterior sample, and guides the sampling towards the prior by decreasing a temperature parameter, while keeping track of the samples likelihoods. An estimate of the model evidence is them computable via a thermodynamic integral using Simpson's rule. Synthetic experiments evaluate the method against another SMC baseline.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n* The paper is mostly well written.\n* The methodology is based on interesting insights which might be useful for the design of other SMC-based EIG estimation algorithms.\n* The algorithm seems to be relatively simple to implement compared to existing alternatives.\n* Theoretical guarantees are provided on the asymptotic unbiasedness of the algorithm.\n\n### Weaknesses\n* The experimental evaluation involves a single SMC baseline, missing comparisons against other EIG estimators (e.g., Foster et al., 2019). Hence, it's hard to assess the significance of the contribution with respect to other methods for BOED.\n* It is not clearly explained how MCMC is applied, even though it's a fundamental part of the algorithm.\n* Eq. 8 might need further detail on its derivation, as it is not immediately clear that the integral on the right-hand side yields the log-marginal likelihood.\n* I missed a reference for background on Simpson's rule.\n\nQUESTIONS:\n* In Definition 4.1, shouldn't the samples be $\\{y_k, \\theta_k)\\}_{k=1}^K$? Currently, $d_k$ is in the sample set.\n\nLIMITATIONS:\nLimitations are discussed in Sec. 5.4.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}