are also used by Bahety et al. (2021) and Athey et al. (2021).

**Estimation.** In the IVR application, we focus on Bayesian estimation to obtain final parameter estimates. In Kasy and Sautmann (2021a), the outcome of each arm _k_ is Bernoulli distributed with a Beta prior, so that the posteriors after _t_ have closed forms. In the IVR experiment, we generalize the approach and estimate Bayesian hierarchical models with school-specific effects and a Binomial outcome distribution (Normal for reading fluency), described in detail in section 4. The Bayesian approach with updating between waves is internally consistent<sup>7</sup> and naturally produces _p_<sup>_k_</sup> _t_<sup>thatweneedforexplorationsampling.Bayesianinference</sup> is valid with adaptively collected data.

However, users may be interested in frequentist inference about the parameter estimates. Frequentist estimates that do not account for the data being generated by an experiment for policy choice are subject to potential biases (Melfi and Page, 2000; Xu et al., 2013). First, observations from an adaptive experiment cease to be iid draws – intuitively, adaptivity introduces sampling bias because random fluctuations in early observations in a given treatment arm _k_ affect the weight of these observations in the overall sample assigned to _k_ (by changing the assignment shares of this arm in future waves). Second, inference on the best arm out of a set, where the ranking is based on the treatment effect estimates, creates an upward bias and invalidates standard confidence intervals even with non-adaptive sampling (Andrews et al., 2021).

Inference from adaptively sampled data is an active field of research, with particular focus on algorithms targeting in-sample regret, which exacerbate selection bias by quickly focusing on high-performing arms. Adaptively weighted estimators can correct sampling bias and produce asymptotically normal estimators (Hadad et al., 2021; Zhang et al., 2021). Andrews et al. (2021) propose corrections for the “winner’s curse” when estimating the average outcome of the highest-performing arm that apply to asymptotically normal estimators. To our knowledge, there are to date no approaches that can provide confidence intervals with correct coverage for the optimal arm in an adaptive experiment in a model with random effects as we used in the IVR experiment. However, in section 5 we estimate a frequentist Binomial model for engagement and illustrate how the estimates are affected when (a) applying the weights proposed in Zhang et al. (2021) to restore asymptotic normality and then applying the winner’s curse correction by Andrews et al. (2021). _Remark: Hybrid Algorithms._ Given the problems with inference in adaptive procedures where low-performing arms are under-sampled, recent applications have used modified algorithms for a hybrid goal of (frequentist) estimation as well as regret minimization. For example, the “tempered Thompson” algorithm in Caria et al. (2020) uses a convex combination of Thompson shares and 1 _/k_ equal-sized shares. Another common

> 7In principle, updating the posterior from any earlier wave with the data collected afterwards should lead to the same posterior outcome distribution at _t_ , including re-estimating the model with all the data collected and the initial prior, which is in practice the method we use.

9