Title: Bernstein–von Mises for Adaptively Collected Data
Venue: NeurIPS 2025
Accepted: yes
Keywords: Bernstein–von Mises

Uncertainty Quantification

Reinforcement Learning

Multi-armed Bandits

Asymptotic Normality

Frequentist Coverage of Credible Intervals
Abstract: Uncertainty quantification (UQ) for adaptively collected data, such as that coming from adaptive experiments, bandits, or reinforcement learning, is necessary for critical elements of data collection such as ensuring safety and conducting after-study inference. The data's adaptivity creates significant challenges for frequentist UQ, yet Bayesian UQ remains the same as if the data were independent and identically distributed (i.i.d.), making it an appealing and commonly used approach. Bayesian UQ requires the (correct) specification of a prior distribution while frequentist UQ does not, but for i.i.d. data the celebrated Bernstein–von Mises theorem shows that as the sample size grows, the prior `washes out' and Bayesian UQ becomes frequentist-valid, implying that the choice of prior need not be a major impediment to Bayesian UQ as it makes no difference asymptotically. This paper for the first time extends the Bernstein–von Mises theorem to adaptively collected data, proving asymptotic equivalence between Bayesian UQ and Wald-type frequentist UQ in this challenging setting. Our results do not require the standard stability condition for validity of Wald-type frequentist UQ, and thus provide positive results on frequentist validity of Bayesian UQ under stability. Counterintuitively however, they also provide a negative result that Bayesian UQ is not asymptotically frequentist valid when stability fails, despite the fact that the prior washes out and Bayesian UQ asymptotically matches standard Wald-type frequentist UQ. We empirically validate our theory (positive and negative) via a range of simulations.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Bernstein-von Mises (BvM) results for settings in which data collection is adaptive. Traditional BvM results will often operate under assumptions that are not entirely aligned with adaptive contexts, like assuming iid data, so the results presented here are a kind of extension, with some particular emphasis on applicability to specific types of Linear Gaussian and Exponential Family bandit settings. Further, they claim to have achieved these results without assuming traditional stability conditions that would otherwise ensure \"frequentist-validity\", and yet \"bayesian-validity\" is still achieved. Finally, numerical experiments examine the nature of the convergence that is guaranteed by these results.\n\nSTRENGTHS AND WEAKNESSES:\nI'm not entirely sure what to make of the fact that this theory paper has a missing appendix and hence no proofs. This leaves me on the fence, and I am preliminarily submitting a borderline rating that leans towards accept (I'm trying to be optimistic), and will look forward to the rebuttal phase and the comments/guidance from the chair. \n\n# Strengths:\n- There are some helpful, high-level explanations in the main body of the role that various hypotheses and conditions ( e.g. condition (i) ) have in the theorems. There are also informative summaries of how the proofs (though missing) proceed. As well, I appreciated the examples, remarks and discussions surrounding the possibility of finding frequentist-invalidity but Bayesian-validity. \n- The topic is motivated, and incorporates some Bandit settings \n\n# Weaknesses:\n- Appendix appears to be missing $\\ldots$ so no proofs provided in this manuscript. However, this omission of the Appendix seems to be accidental. A bit awkward to assess the level of theoretical contribution without the proofs. There are certainly numerical experiments that illustrate the Bayesian validity, but this work is a theory paper.\n- The authors largely restrict to a Linear Gaussian observation scheme\n\nQUESTIONS:\n# Question 1: \nThe authors write *\"$\\ldots$ Lai and Wei showed that condition (i) of Theorem 1 is necessary for MLE consistency (which our proof relies on), as there are counterexamples where the condition is only marginally violated but consistency fails [20, Example 1].\"* \n\nThis line is rather confusing. If I'm understanding correctly the literal meaning of this statement, the authors are indicating that MLE consistency implies condition (i) because of the existence of counterexamples? There is something very imprecise here, considering later on, the authors also dive into an outlined hypothetical: *\"Second, even if $\\hat{\\beta}_n$ is consistent but condition (i) fails, it may not be the case that $\\ldots$\", which their earlier statement, if taken literally, would preclude the possibility of. \n\nAs far as I understand, Lai and Wei actually show that condition (i) (and other assumptions) suffice for there to be (strong) consistency, and then their cited example is meant to show that condition (i), in their words,*\"is in some sense weakest possible.\"* So I'm guessing the authors actually meant to say something like, Lai and Wei showed that condition (i) of Theorem 1 is *nearly* necessary for MLE consistency.\n\n# Question 2:\nThe restriction to a Gaussian statistical model, as outlined in Section 3 $\\ldots$ can the authors speak on the extent that their analysis relies on this?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study investigates the Bernstein–von Mises (BvM) theorem for adaptively collected data. Uncertainty quantification is a central issue in statistical analysis, and the authors aim to use the BvM theorem to bridge Bayesian and frequentist perspectives in this setting.\n\nSTRENGTHS AND WEAKNESSES:\nI recognize that the authors tackle ambitious tasks in adaptive experimental design. I also agree that uncertainty quantification for adaptively collected data is important, and the manuscript seeks to provide a general and powerful tool for this purpose.\n\nNevertheless, I encountered several difficulties while reading the draft, listed below.\n1. What are the precise definitions of $\\Lambda$, $\\lambda_{\\min}$, and $\\lambda_{\\max}$?\n2. What is the formal definition of the sampling rule?\n3. Do the assumptions required for each theorem hold?\n(If I have overlooked these definitions, I apologize to the authors.)\n\nAmong these points, I believe the third is the most critical. The authors remark that suboptimal arms are pulled only $O_p(\\log n)$ times. I agree that this assumption is central to the field, and, in practice, it renders many methods unrealistic. I would like to understand how the authors address this issue.\n\nFor example, Hadad et al. (2021) and subsequent work by the same group show that asymptotically normal estimators can be obtained under appropriate assumptions in bandit problems. However, when I examine their theorems, I find that the convergence rate becomes $1/\\sqrt{\\log(n)}$ for optimal algorithms, implying that, relative to the usual $1/\\sqrt{n}$ rate, we would need $\\exp(n)$ samples to achieve comparable performance for normal approximation. I consider this requirement unrealistic.\n\nAlthough I cannot completely follow the notation, I suspect that a similar convergence rate is implicitly required through $\\lambda_{\\min}$. If so, I ask the authors to clarify and justify the assumptions. If not, I would appreciate an explanation of how they avoid the problem described above.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nN/A.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends the Bernstein-von Mises (BvM) theorem to the adaptive data collection setting, in which the data is non-i.i.d. and the ratio between maximum and minimum eigenvalues of the Fisher information matrix is unbounded. It establishes that under such settings, with mild eigenvalue growth conditions, BvM-analogous convergences hold, implying Bayesian credible intervals retain frequentist validity asymptotically, even under adaptive data collection. The authors prove BvM results for adaptive linear Gaussian models, multi-armed bandits, and contextual bandits, supported by simulations validating convergence and coverage properties.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- First BvM extension to adaptive data, addressing a critical gap in uncertainty quantification for bandits/RL.\n- Broad applicability to Gaussian bandits, exponential families, and contextual settings.\n- Theorems are rigorously proven with careful handling of non-i.i.d. likelihoods and anisotropic Fisher information.\n- Simulations validate positive/negative results across diverse adaptive settings (MAB, contextual bandits, LQR).\n\n**Weaknesses**\n1. Clarity issues:\n- The umbrella term *uncertainty quantification (UQ)* is used throughout. The author should use explicit terms of *Bayesian credible intervals* and *frequentist confidence intervals*.\n- Jargons like \"Wald-type\", \"frequentist-valid\" are used without first establishing precise definitions.\n- Using $H_n^n$ for triangular arrays (Section 4) alongside $H_n$ for fixed-$n$ trajectories is confusing\n2. Technical concerns:\n- Proof sketches (Section 3) omit key steps (e.g., why truncation works without bounded condition numbers)\n- Convergence in TV implies pointwise agreement but may not ensure uniform frequentist validity. Authors should discuss implications for confidence sets.\n3. Experimental limitations:\n- All experiments use synthetic data. No validation on real adaptive datasets.\n- Coverage plots are only shown for MAB. For contextual bandits/LQR, only TV distance is reported, leaving coverage unexplored.\n\nQUESTIONS:\nBayesian UQ can match frequentist UQ asymptotically, yet be frequentist-invalid when stability fails\n\n1. The bounded $b^{\\prime\\prime\\prime}$ requirement excludes some exponential families (e.g., Poisson near zero). Is this necessary? Can the proof be adapted without boundedness?\n2. Can results be validated on a public adaptive dataset? If not, please discuss challenges.\n3. Theorem 1 claims Bayesian validity under prior misspecification, but how quickly does this \"washout\" occur? Simulations demonstrate slow TV convergence for large margins. Please discuss the implications for finite-sample safety.\n4. The relaxation of the usual stability condition doesn't seem to be as significant as presented, since we still need it to guarantee frequentist-validity, or am I missing something?\n\nLIMITATIONS:\nSuggestions:\n1. Discuss mitigations for instability-induced invalidity.\n2. Address finite-sample prior sensitivity observed in simulations.\n3. Expand societal impact: Could invalid UQ in adaptive trials lead to harmful decisions?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The authors derive a BvM theorem in a parametric setting for data that is sequentially observed in a linear model. Specifically, a data stream $H_i = ((y_1,x_1),\dots,(y_i,x_i))$ where $x_{i+1}$ can be chosen on the basis of $H_i$, after which $Y_{i+1} = \beta^\top x_{i+1} + noise$ is observed, with iid Gaussian noise with known variance. 

The authors show a BvM theorem for $\beta$, comparing the posterior to a normal distribution centered at the MLE and with (empirical) Fisher information as its variance.

STRENGTHS AND WEAKNESSES:
Strengths:

* The paper is well written.
* The topic is interesting.
* I like how the link to bandits is fleshed out in the paper.
* The proofs appear sound.

Weaknesses:
* The setting is very limited; I see no reason why the proof technique would not work in the typical generality of parametric BvM's and there is a lack of discussion concerning this in the paper.
* There is a lack of literature overview / comparison of BvM's with non-i.i.d. data. How does the result relate to investigations such as [1]? Does general (non-iid) machinery as developed by [2] not apply? 

[1] J. D. Borwanker, G. Kallianpur & B. L. S. Prakasa Rao. The Bernstein–von Mises Theorem for Markov Processes. Annals of Mathematical Statistics
[2] Ismaël Castillo & Judith Rousseau. A Bernstein–von Mises theorem for smooth functionals in semiparametric models. Annals of Statistics
[2]

QUESTIONS:
Is there a reason why the proof technique would not work in the typical generality of parametric BvM's? I.e. likelihoods that differentiable in quadratic mean?

LIMITATIONS:
The main limitation in my eyes is mentioned in the strength/weaknesses section of the review.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of uncertainty quantification (UQ) for adaptively collected data, which arises in settings such as adaptive experiments, multi-armed bandits (MABs), and reinforcement learning. The core contribution is an extension of the classical Bernstein-von Mises (BvM) theorem to adaptively collected data, establishing asymptotic equivalence between Bayesian UQ and Wald-type frequentist UQ in these complex scenarios. One key takeaway is that Bayesian UQ can be asymptotically valid in a Bayesian sense even in adaptive settings, but asymptotic frequentist validity is now always assured — especially when the stability condition fails. In other words, posterior credible intervals can be misleading from a frequentist perspective, even though they are valid in a Bayesian sense (Bayesian validity).\n\nSTRENGTHS AND WEAKNESSES:\n1. Strengths:\n\n(S1) The proposed BvM theorem for adaptively collected data rigorously establishes when Bayesian UQ aligns with Wald-type frequentist UQ in adaptive settings, addressing a major gap in theoretical statistics.\n\n(S2) The main results are proven under mild conditions, notably not requiring the stability condition often assumed in prior works (e.g., [1]), thereby broadening the range of applicable scenarios.\n\n(S3) The paper provides a clear distinction between Bayesian validity vs. frequentist validity: while Bayesian UQ is always asymptotically valid in a Bayesian sense as long as the prior “washes out”, the asymptotic frequentist validity is not always guaranteed — especially when the stability condition fails. This distinction is crucial for practitioners.\n\n\n2. Weaknesses:\n\n(W1) Even though the BvM theorem is shown to hold without the stability condition, statistical inference with frequentist validity is not ensured in these cases. The paper demonstrates that posterior credible intervals can fail to have correct frequentist coverage without the stability condition.\n\n(W2) The author didn’t discuss the exact meaning of some key terminologies including asymptotic frequentist validity and Bayesian validity in the paper. While statisticians may be familiar with these terms, many researchers in ML or artificial intelligence might not be, so I believe it would be helpful to provide a direct explanation.\n\n\n[1] Tze Leung Lai and Ching Zong Wei, “Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems”, The Annals of Statistics, pp. 154-166, 1982.\n\nQUESTIONS:\nNo further questions.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-13"}