Title: CLIP-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments
Venue: NeurIPS 2023
Accepted: yes
Keywords: causal inference

randomized experiments

online optimization
Abstract: From clinical development of cancer therapies to investigations into partisan bias, adaptive sequential designs have become increasingly popular method for causal inference, as they offer the possibility of improved precision over their non-adaptive counterparts. However, even in simple settings (e.g. two treatments) the extent to which adaptive designs can improve precision is not sufficiently well understood. In this work, we study the problem of Adaptive Neyman Allocation in a design-based potential outcomes framework, where the experimenter seeks to construct an adaptive design which is nearly as efficient as the optimal (but infeasible) non-adaptive Neyman design, which has access to all potential outcomes. Motivated by connections to online optimization, we propose Neyman Ratio and Neyman Regret as two (equivalent) performance measures of adaptive designs for this problem. We present Clip-OGD, an adaptive design which achieves $\widetilde{\mathcal{O}}(\sqrt{T})$ expected Neyman regret and thereby recovers the optimal Neyman variance in large samples. Finally, we construct a conservative variance estimator which facilitates the development of asymptotically valid confidence intervals. To complement our theoretical results, we conduct simulations using data from a microeconomic experiment.

CLIP-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments

Abstract

From clinical development of cancer therapies to investigations into partisan bias, adaptive sequential designs have become increasingly popular method for causal inference, as they offer the possibility of improved precision over their non-adaptive counterparts. However, even in simple settings (e.g. two treatments) the extent to which adaptive designs can improve precision is not sufficiently well understood. In this work, we study the problem of Adaptive Neyman Allocation in a designbased potential outcomes framework, where the experimenter seeks to construct an adaptive design which is nearly as efficient as the optimal (but infeasible) nonadaptive Neyman design, which has access to all potential outcomes. Motivated by connections to online optimization, we propose Neyman Ratio and Neyman Regret as two (equivalent) performance measures of adaptive designs for this problem. We present CLIP-OGD, an adaptive design which achieves r O p ? T q expected Neyman regret and thereby recovers the optimal Neyman variance in large samples. Finally, we construct a conservative variance estimator which facilitates the development of asymptotically valid confidence intervals. To complement our theoretical results, we conduct simulations using data from a microeconomic experiment.

1 Introduction

From medicine and public health to economics and public policy, randomized control trials are used in a variety of disciplines to investigate causal effects. Typically, treatment is assigned in a nonadaptive manner, where assignments are determined before any outcomes are observed. A sequential experimental approach, which adaptively assigns treatment based on previously observed outcomes, offers the possibility of more precise or high powered estimates of relevant causal effects. Adaptive experiments are run to develop clinical therapies for breast cancer [Barker et al., 2009], evaluate incentives to reduce partisan bias [Offer-Westort et al., 2021], and evaluate customer acquisition via online advertising [Schwartz et al., 2017], to name a few.

In this paper, we study the problem of Adaptive Neyman Allocation, which we informally define as follows. An optimal non-adaptive experimental design which minimizes variance of an estimator will depend on the unknown potential outcomes, rendering it infeasible to run. However, by adaptively choosing treatment assignments in a sequential manner based on observed outcomes, we can hope to guarantee that the variance of the estimator under the adaptive design converges to the optimal non-adaptive variance. The problem of Adaptive Neyman Allocation is to construct such an adaptive design which guarantees the variance converges to the (infeasible) optimal non-adaptive design.

An experimental design which sufficiently addresses the Adaptive Neyman Allocation problem offers the advantage of higher statistical power, relative to a broad class of fixed experimental designs. Practically speaking, this means that either smaller confidence intervals are obtained for a given number of experimental units, or that fewer units are required to achieve confidence intervals of a given length. In practice, this means that investigating causal effects can be cheaper-in terms of time, money, and other valuable resources-when adaptive experiments are run. Although several experimental designs have been proposed for this purpose [Hahn et al., 2011, Blackwell et al., 2022], none have provided formal guarantees that the optimal non-adaptive variance can be achieved and the effectiveness of such designs has recently been called into question Cai and Rafi [2022].

The main contributions of this work are as follows:

Neyman Ratio and Regret : We propose two (equivalent) performance measures of experimental designs for the problem of Adaptive Neyman Allocation: Neyman Ratio and Neyman Regret. We show that guarantees on the rates of these performance measures directly translate to guarantees on the convergence of variance to the Neyman variance.

CLIP-OGD : We propose the adaptive design CLIP-OGD, a variant of online stochastic projected gradient descent for which the Neyman regret is r O p ? T q . This guarantees that the variance of the sequential effect estimator approaches the Neyman variance.

Confidence Intervals : By constructing a conservative variance estimator, we provide confidence intervals which guarantee asymptotic coverage of the average treatment effect.

In Section 7, we support these theoretical results with simulations using data from a microeconomic experiment. Our results rely on viewing the Adaptive Neyman Allocation problem through the lens of online convex optimization. However, as discussed in Section 4.2, due to the subtleties arising in the problem, we do not know of an existing online algorithm which directly obtains these results.

1.1 Related Work

We work within the potential outcomes framework for causal inference Neyman [1923], Rubin [1980], Imbens and Rubin [2015]. The idea of optimal treatment allocation dates back to Neyman [1934], where he demonstrates that sampling from treatments proportional to the within-treatment outcome variance will minimize the variance of standard estimators. Unfortunately, this type of design is not practically feasible when little is known about the statistics of outcomes from each treatment. Robbins [1952] highlights adaptive sampling as one of the more pressing open statistical problems at the time. In Chapter 5, Solomon and Zacks [1970] presents a survey of adaptive designs for survey sampling, but from a Bayesian perspective. More recently, Hahn et al. [2011] proposed a two stage design in a super-population setting, where data is uniformly collected from both arms in the first stage, statistics of the treatment arm are estimated, and a fixed probability derived from estimated statistics is used in the second stage. They derive the limiting distribution of the effect estimator under the two-stage design, which has a variance that is similar to, but asymptotically bounded away from the optimal Neyman variance. In a design-based setting, Blackwell et al. [2022] propose a similar two-stage approach and, through simulations, provide practical guidance on how to choose the length of the first stage. Although both of these works are motivated by achieving the Neyman variance, neither formally show that this is possible under the two-stage design.

While the goal in this paper is to increase the precision of treatment effect estimates, a variety of response-adaptive designs have been developed for various objectives, including reducing mean total sample size [Hayre and Turnbull, 1981] and reduction of harm reduction in null hypothesis testing [Rosenberger et al., 2001]. Eisele [1994] proposes the Doubly Adaptive Coin Based Design, which is a meta-algorithm for targeting various allocation proportions when outcomes are drawn i.i.d. from an exponential family. Hu and Rosenberger [2003] critiques many response-adaptive designs as being 'mypoic strategies' which have 'adverse effects on power', providing an asymptotic framework by which to judge adaptive design when the outcomes are i.i.d. and binary. This asymptotic evaluation framework was extended to continuous outcomes by Zhang and Rosenberger [2006]. An additional line of work has developed adaptive Bayesian methods for subgroup identification [Xu et al., 2014].

Causal inference under adaptively collected data has seen a variety of recent developments which are adjacent to, but distinct from, the problem of Adaptive Neyman Allocation. One line of research has been to construct estimators via re-weighting which ensure consistency and normality when data

is collected via bandit algorithms [Hadad et al., 2021, Zhang et al., 2020, 2021]. A second line of research has been to provide inferential methods which are valid under data-dependent stopping times [Wald, 1945, Howard et al., 2021, Ham et al., 2022]. Finally, Offer-Westort et al. [2021] propose an adaptive experimental design for improved selective inference, when only the effect of the best performing treatment is to be inferred.

2 Preliminaries

The sequential experiment takes place over T rounds, where we assume that T is fixed and known to the experimenter. At each iteration t P r T s , a new experimental unit (e.g. clinical participant), enters into the experiment, so that there are T units in total. In an abuse of notation, we identify units with their respective round t P r T s . The experimenter assigns a (random) treatment Z t P t 0 , 1 u (e.g. drug or placebo) to the experimental unit. The unit has two real-valued potential outcomes y t p 1 q , y t p 0 q which are unknown to the experimenter and represent the unit's measured response to the treatment assignments (e.g. measured heart rate). The term 'potential' is used here because while only one treatment is assigned and thus only one outcome is observed, both outcomes have the potential to be observed. At the end of the round, the experimenter sees the observed outcome Y t GLYPH<16> 1 r Z t GLYPH<16> 1 s y t p 1 q GLYPH<0> 1 r Z t GLYPH<16> 0 s y t p 0 q .

2.1 Potential Outcomes Framework

In this paper, we adopt a design-based framework where the sequence of potential outcomes t y t p 1 q , y t p 0 qu T t GLYPH<16> 1 is deterministic and the only source of randomness is treatment assignment itself. In particular, we place no assumption on the homogeneity of the outcomes: they are not necessarily related to each other in any systematic way. Although the potential outcomes are deterministic, we introduce finite population analogues of various statistics. Define the finite population second moments S p 1 q and S p 0 q and correlation of the treatment and control outcomes ρ to be

.

Observe that the correlation between treatment and control outcomes is bounded ρ P rGLYPH<1> 1 , 1 s . Although we refer to ρ as the correlation, it also known as the cosine similarity and is generally not equal to the Pearson correlation coefficient. We remark that although the potential outcomes y t p 1 q and y t p 0 q are deterministic, the observed outcome Y t is random, as it depends on random treatment assignment. The natural filtration according to these rounds is denoted as F 1 . . . F T , so that F t captures all randomness before the sampling of Z t , i.e. the treatments assigned and outcomes observed in previous rounds.

In this sequential setting, the mechanism for random treatment assignment can incorporate observed outcomes from previous experimental rounds. This treatment mechanism, referred to as the experimental design , is selected by and thus known to the experimenter. Formally, the experimental design is a sequence of functions t Π t u T t GLYPH<16> 1 with signature Π t : pt 0 , 1 u GLYPH<2> R q t GLYPH<1> 1 GLYPH<209> r 0 , 1 s such that treatment is assigned as Pr p Z t GLYPH<16> 1 | F t q GLYPH<16> Π t p Z 1 , Y 1 , . . . Z t GLYPH<1> 1 , Y t GLYPH<1> 1 q . We denote P t GLYPH<16> Pr p Z t GLYPH<16> 1 | F t q as the (random) probability of treatment assignment at iteration t , given previously observed treatment assignments and outcomes.

The causal estimand of interest is the average treatment effect , defined as

The average treatment effect captures the average counterfactual contrast between a unit's outcomes under the two treatment assignments. For example, this could be the average contrast of a clinical participant's heart rate under the drug or placebo. Individual treatment effects are defined as τ t GLYPH<16> y t p 1 q GLYPH<1> y t p 0 q , but they cannot be estimated without strong additional assumptions, as only one outcome is observed.

A standard estimator of the average treatment effect is the Horvitz-Thompson estimator, which weights observed outcome by the probability of their observation [Narain, 1951, Horvitz and Thompson, 1952]. For adaptive designs, the standard Horvitz-Thompson estimator is infeasible because

the marginal probability of treatment assignment Pr p Z t GLYPH<16> 1 q depends on the unknown potential outcomes. For this reason, we investigate the adaptive Horvitz-Thompson estimator , which uses the random (observed) treatment probabilities used at each iteration.

where we recall that P t GLYPH<16> Π t p Z 1 , Y 1 , . . . Z t GLYPH<1> 1 , Y t GLYPH<1> 1 q is the treatment probability under the experimental design given the observed data. When treatment assignments are non-adaptive and independent, then the adaptive Horvitz-Thompson estimator is the equivalent to the standard HorvitzThompson estimator. Such adaptively weighted estimators have been proposed previously in the literature, e.g. [Bowden and Trippa, 2015, Hadad et al., 2021]. Below, we provide positivity conditions under which the adaptive estimator is unbiased, and derive its variance.

Proposition 2.1. If min t P t , 1 GLYPH<1> P t u ¡ 0 almost surely for all t P r T s then the adaptive HorvitzThompson estimator is unbiased: E r ˆ τ s GLYPH<16> τ .

Proposition 2.2. The variance of the adaptive Horvitz-Thompson estimator is

2.2 Asymptotic Framework and Assumptions

Following the convention of design-based inference, we analyze statistical methods within an asymptotic framework [see e.g., Freedman, 2008, Lin, 2013, Sävje et al., 2021]. This provides a formal basis for reasoning about the performance of statistical methods as the sample size increases, giving meaning to conventional notions such as consistency and limiting distribution. Formally speaking, the asymptotic sequence of potential outcomes is a triangular array tt y t,T p 1 q , y t,T p 0 qu T t GLYPH<16> 1 u 8 T GLYPH<16> 1 , which yields a sequence of estimands t τ T u 8 T GLYPH<16> 1 and, together with an appropriately specified sequence of experimental design, a sequence of estimators t ˆ τ T u 8 T GLYPH<16> 1 . Analysis which applies to a fixed T is said to be finite-sample (e.g. E r ˆ τ T s GLYPH<16> τ T ) whereas analysis which applies to the entire sequence is said to be asymptotic (e.g. τ T GLYPH<1> ˆ τ T p GLYPH<221> GLYPH<209> 0 ). Although we use an asymptotic framework, we emphasize that the majority of our results are derived from finite-sample analysis and are merely interpreted through the lens of the asymptotic framework. We drop the subscript T for notational clarity.

The main regularity conditions we place on the sequence of potential outcomes is below.

Assumption 1. There exist constants 0 GLYPH<160> c / C with c GLYPH<160> 1 such that for all T in the sequence:

Bounded Correlation : ρ ¥ GLYPH<1>p 1 GLYPH<1> c q .

The upper moment bound in Assumption 1 stipulates that the potential outcomes cannot grow too large with the sample size, while the lower moment bound is a type of non-degeneracy condition that prevents an increasingly large fraction of the outcomes going to zero. These assumptions are analogous to finite fourth moment and positive second moment assumptions in an i.i.d. setting. The bounded correlation assumption stipulates that the treatment and control outcomes are not exactly negatively correlated. In this paper, we do not assume that these constants C and c are known to the experimenter; however, if the experimenter can correctly specify such bounds (perhaps knowing a priori the scaling of the outcomes) then some of the constant factors in our analysis can be improved. We emphasize here that Assumption 1 places no assumption on the order in which units arrive in the experiment. In this sense, Assumption 1 allows for arbitrary 'non-stationarity' or 'drift' in the potential outcomes over the experimental rounds. In the next section, these regularity assumptions will ensure that the Neyman variance converges to zero at the parametric rate.

3 Neyman Design: The Infeasible Non-Adaptive Ideal

The problem of Adaptive Neyman Allocation is to construct an adaptive experimental design that achieves nearly the same variance as an optimal non-adaptive experimental design, chosen with

knowledge of all potential outcomes. The optimal non-adaptive design, referred to as the Neyman Design, is infeasible to implement because it depends on all potential outcomes, which are unknown to the experimenter at the design stage. The goal is that an adaptive experimental design-which can select treatment assignment based on observed outcomes-can gather enough information to perform as well as the infeasible Neyman design.

In order to define the optimal non-adaptive design, we begin by defining the class of Bernoulli designs. Informally, the class of Bernoulli designs consists of non-adaptive designs where each unit receives treatment Z t GLYPH<16> 1 with probability p , independently of past treatment assignments and observations. Formally, this class is parameterized by a non-adaptive sampling probability p P r 0 , 1 s such that for all t P r T s , the treatment policy Π t is a constant function whose value is p . Using Proposition 2.2, we can derive the variance of the Bernoulli design with parameter p P r 0 , 1 s to be

From the above, we can see that in order to minimize the variance of the Horvitz-Thompson estimator under the Bernoulli design, we should set the sampling probability p so as to balance the square of the second moments of treatment and control outcomes. The Neyman Design is the Bernoulli design which minimizes the variance of the Horvitz-Thompson estimator. The corresponding optimal probability p GLYPH<6> and variance V N are referred to as the Neyman probability and Neyman variance, respectively. The following proposition derives these quantities in terms of the potential outcomes.

Proposition 3.1. The Neyman variance is T GLYPH<4> V N GLYPH<16> 2 p 1 GLYPH<0> ρ q S p 1 q S p 0 q , which is achieved by the Neyman probability p GLYPH<6> GLYPH<16> p 1 GLYPH<0> S p 0 q{ S p 1 qq GLYPH<1> 1 .

In order to quantify the reduction in variance achieved by the Neyman design, define the relative Neyman efficiency with respect to p P r 0 , 1 s to be V N { V p . Intuitively, this ratio is a scale-free measure which captures the percent reduction in variance of the sequential Horvitz-Thompson estimator under the Neyman design. Formally, the equation for the relative Neyman efficiency is given below:

.

Consider the setting where outcomes are uncorrelated, and treatment outcomes are larger than control outcomes, e.g. ρ GLYPH<16> 0 , S p 1 q GLYPH<16> 4 GLYPH<4> S p 0 q . In this case, the Neyman design is able to achieve less than half the variance of the uniform Bernoulli design (with p GLYPH<16> 1 { 2 ): we can plug in to 3 to see that in this setting, we have V N { V p GLYPH<16> 0 . 47 . The improvement is larger if the experimenter makes erroneous assumptions about the relative magnitudes of the treatment and control outcomes and attempts to set p accordingly: for example, if the experimenter had set p GLYPH<16> 1 { 4 , incorrectly believing that S p 1 q / S p 0 q , then the Neyman allocation results in a sixfold improvement in variance. Blackwell et al. [2022] derives qualitatively similar analysis of Neyman efficiency for stratified designs.

While the relative Neyman efficiency is helpful in determining the variance reduction afforded by the (infeasible) optimal Bernoulli design, it does not address the main question: which adaptive experimental designs can guarantee similar variance reduction? In the next section, we propose a performance metric which better addresses this question.

4 Adaptive Neyman Allocation: An Online Optimization Approach

4.1 Neyman Ratio and Neyman Regret: New Performance Measures

Let V be the variance of the adaptive experimental design. We introduce our first performance measure of a sequential experimental design for Adaptive Neyman Allocation.

Definition 1. The Neyman ratio of a sequential experimental design is κ T GLYPH<16> p V GLYPH<1> V N q{ V N.

The subscript T in κ T in included the reflect dependence of the number of rounds T . The Neyman ratio is motivated by the following relationship between the adaptive variance and the optimal Neyman variance:

Equation (1) shows that the adaptive design can recover the Neyman variance if and only if the Neyman ratio κ T can be made arbitrarily small. For this reason, we propose the Neyman ratio as a performance measure of a sequential experimental design.

A natural question then becomes: how small can the Neyman ratio κ T be made as the number of rounds T increases? To answer this question, we view the problem of minimizing the Neyman ratio through the lens of online optimization. To this end, we must re-express the variance of the sequential experimental design. For each round t P r T s , define the cost function f t : r 0 , 1 s GLYPH<209> R as f t p p q GLYPH<16> y t p 1 q 2 { p GLYPH<0> y t p 0 q 2 {p 1 GLYPH<1> p q . Observe that by Proposition 2.2, the variance is given by T GLYPH<4> Var p ˆ τ q GLYPH<16> E r 1 T GLYPH<176> T t GLYPH<16> 1 f t p P t qs . This reformulation of variance does not allow us to minimize variance directly, for the usual reason that the outcomes, and thus the cost functions f t , are not fully observed. On the other hand, our goal is only to show that the variance of the adaptive design is comparable to the Neyman variance.

Definition 2. The Neyman regret of a sequential experimental design is

Recall that P t is the random treatment probability at round t . The Neyman regret compares the accumulated costs f t p P t q incurred by the adaptive design to the accumulated costs incurred by the optimal Bernoulli design which has access to all potential outcomes. The Neyman regret is random because the sequence P 1 , . . . P T is random. The following theorem connects the expected Neyman regret to the Neyman ratio.

Theorem 4.1. Under Assumption 1, the Neyman ratio is within a constant factor of the 1 { T -scaled expected Neyman regret: κ T GLYPH<16> Θ p 1 T E r R T sq .

Theorem 4.1 demonstrates that the Neyman ratio can be made small by minimizing the expected Neyman regret in an online fashion. In particular, any sublinear bound on the expected Neyman regret ensures that the Neyman ratio goes to zero so that, in large samples, the adaptive design achieves the variance reduction of the optimal Neyman design. Any adaptive design which aims to achieve Neyman variance must, to some extent, minimize expected Neyman regret.

Fortunately, online optimization is a well-studied area with a rich source of techniques from which we may draw inspiration. However, to the best of our knowledge, existing regret minimization algorithms are not well-suited to minimizing the Neyman regret. For example, the multi-arm bandit literature typically defines regret in terms of a finite number of actions that can be taken [Lattimore and Szepesvári, 2020] while Adaptive Neyman Allocation consists of a continuum of actions as P t P r 0 , 1 s . This means that algorithms like UCB [Auer et al., 2002a] and EXP3 [Auer et al., 2002b] are not appropriate for Adaptive Neyman Allocation. Our cost objectives f t and action space r 0 , 1 s are both convex, so the problem of Adaptive Neyman Allocation is an instance of Online Convex Optimization (OCO) [Hazan, 2016]. Even so, the problem of minimizing Neyman regret is not immediately amenable to existing algorithms, which typically requires assumptions on the cost functions such as bounded gradients or known Lipschitz parameters. In this setting, the cost functions have gradients which blow up at the boundary and Lipschitz parameters cannot be guaranteed as they rely on the unknown heterogeneous potential outcomes. For these reasons, we must design a new algorithm specifically tailored to Adaptive Neyman Allocation.

4.2 CLIP-OGD: A Variant of Online Stochastic Projected Gradient Descent

We present CLIP-OGD, which aims to minimize the Neyman regret and thus recover the Neyman variance in large samples. The algorithm is based on the online stochastic projected gradient descent principle, but with a twist: the projection set continuously grows over the rounds. At each round t , a new treatment probability P t is chosen by updating the previous sampling probability P t GLYPH<1> 1 in the negative (estimated) gradient direction of the previous cost, and then projecting to an interval r δ t , 1 GLYPH<1> δ t s . Initially, this projection interval contains only the point 1 { 2 and it grows as the rounds increase, allowing for larger amounts of exploitation in later rounds.

The gradient estimator G t is obtained in the following way: the gradient of f t at P t is given as f 1 p P t q GLYPH<16> GLYPH<1> y t p 1 q 2 P 2 t GLYPH<0> y t p 0 q 2 p 1 GLYPH<1> P t q 2 . Only one outcome is observed, so we used the adaptive Horvitz-

Thompson principle using the conditional probability P t to unbiasedly estimate the outcomes. CLIPOGD is formally presented below as Algorithm 1, where the projection operator is defined as P c p x q GLYPH<16> max t c, min t x, 1 GLYPH<1> c uu .

Unlike the two-stage design of [Hahn et al., 2011, Blackwell et al., 2022], CLIP-OGD does not feature explicit explore-exploit stages, but rather performs both of these simultaneously. The trade-off is implicitly controlled through parameters η and α : smaller values of η limit the amount of that sampling probabilities can update and, likewise, larger values of α prevent extreme probabilities in earlier stages. Because the gradient of the cost functions are inversely proportional to the treatment probabilities, limiting the extremeness of the treatment probabilities in this way ensures that the gradient estimates do not increase at a fast rate. By appropriately setting input parameters, CLIP-OGD achieves r O p ? T q expected Neyman regret, where the r O pGLYPH<4>q notation hides sub-polynomial factors.

Theorem 4.2. Under Assumption 1 the parameter values η GLYPH<16> a 1 { T and α GLYPH<16> a 5 log p T q ensure the expected Neyman regret of CLIP-OGD is asymptotically bounded: E GLYPH<16> R T GLYPH<24> / r O GLYPH<0> ? T GLYPH<8> .

Theorem 4.2 answers, in the affirmative, that it is possible to construct an adaptive experimental design whose variance recovers that of the Neyman variance, in large samples. Note that the amount of exploration (as given by the parameters η and α ) should be increasing with T in order to recover these regret bounds. In Appendix C, we show that CLIP-OGD is somewhat robust to different values of the decay parameter, i.e. for any value α ¡ 5 , the expected regret will be sublinear. We also show that if the experimenter presumes to have correctly specified bounds C and c appearing in Assumption 1, then the step size can be modified to improve the constant factors in the Neyman regret bound, which may lead to improved performance in moderate sample sizes. We conjecture that the minimax rate for expected Neyman regret is O p ? T q , but proving this is beyond the scope of the current paper-we only remark that we do not know it to immediately follow from any existing regret lower bounds for OCO.

5 Inference in Large Samples

The proposed CLIP-OGD was constructed to ensure that the variance of the adaptive HorvitzThompson estimator quickly approaches the Neyman variance. In this section, we provide confidence intervals for the average treatment effect which also enjoy reduced width compared to non-adaptive counterparts.

A necessary condition for variance estimation is that the variance itself cannot be going to zero too quickly. In design-based inference, it is common to directly posit a so-called 'non-superefficient' assumption that Var p ˆ τ q GLYPH<16> Ω p 1 { T q [Aronow and Samii, 2017, Leung, 2022, Harshaw et al., 2022]. The non-superefficiency assumption may be seen as an additional regularity assumption on the outcomes, e.g. preventing y t p 1 q GLYPH<16> y t p 0 q GLYPH<16> 0 for all t P r T s . In this work, a similar lower bound on the rate of the adaptive variance is obtained through a different, perhaps more transparent, assumption on the expected Neyman regret.

Assumption 2. The outcome sequence is not overly-fit to CLIP-OGD: GLYPH<1> E r R T s GLYPH<16> o p T q .

While we have shown that E r R T s / r O p ? T q , the Neyman regret could in principle be negative if the adaptive design achieves variance which is strictly smaller than the best Bernoulli design. While this

seems unlikely to happen for 'typical' outcomes, it is not impossible. Assumption 2 rules out these edge-case settings. We suspect that Assumption 2 would not be necessary in an i.i.d. setting, but proving this seems beyond the scope of the current paper. As shown in the appendix, Assumptions 1 and 2 imply that the adaptive variance achieves the parametric rate: Var p ˆ τ q GLYPH<16> Θ p 1 { T q .

5.1 Variance Estimation

In this section, we provide a variance estimator and show its stability in large samples. Rather than estimating the adaptive variance (which has no simple closed form), our approach is to estimate the Neyman variance directly. For an adaptive design achieving sublinear expected Neyman regret, these two quantities are asymptotically equivalent. In this way, our variance estimator may be appropriate not only for CLIP-OGD, but for any adaptive design achieving sublinear expected Neyman regret.

Recall that the Neyman variance is given by T GLYPH<4> V N GLYPH<16> 2 p 1 GLYPH<0> ρ q S 1 S 0 , where ρ is the outcome correlation, S 1 is the second moment of treatment outcomes and S 0 is the second moment of control outcomes. Unfortunately, the outcome correlation term is generally not estimable without strong assumptions in a design-based framework. Indeed, the difficulty is that terms like y t p 1 q y t p 0 q are unobservable due to the fundamental problem of causal inference [Imbens and Rubin, 2015]. A common solution to the problem is to opt for a conservative estimate of the variance, which will ensure validity of resulting confidence intervals.

We propose estimating the following upper bound on the variance: T GLYPH<4> VB GLYPH<16> 4 S 0 S 1 . This upper bound on the Neyman variance is tight (i.e. VB GLYPH<16> V N) when outcome correlation satisfies ρ GLYPH<16> 1 . For example, this occurs when all individual treatment effects are zero, i.e. y t p 1 q GLYPH<16> y t p 0 q for all t P r T s . Conversely, the upper bound will become looser for smaller values of the outcome correlation. In this sense, our bound resembles both the Neyman bound and the Aronow-Samii bound [Neyman, 1923, Aronow and Samii, 2013]. It may be possible to use the recent insights of Harshaw et al. [2021] in order to construct variance bounds which are tight in other scenarios, but that is beyond the scope of the current paper. Our variance estimator is defined as

which is essentially a plug-in Horvitz-Thompson estimator for the second moments. Theorem 5.1 shows the error of the normalized variance estimator converges at a parametric rate.

Theorem 5.1. Under Assumptions 1 and 2, and the parameters stated in Theorem 4.2, the error of the normalized variance estimator under CLIP-OGD is T GLYPH<4> x VB GLYPH<1> T GLYPH<4> VB GLYPH<16> r O p p T GLYPH<1> 1 { 2 q .

5.2 Confidence Intervals

The variance estimator may be used to construct confidence intervals for the average treatment effect. This offers experimenters standard uncertainty quantification techniques when running CLIP-OGD. The following corollary shows that the resulting Chebyshev-type intervals are asymptotically valid.

Corollary 5.1. Under Assumptions 1 and 2, and parameters stated in Theorem 4.2, Chebyshev-type intervals are asymptotically valid: for all α P p 0 , 1 s , lim inf T GLYPH<209>8 Pr p τ P ˆ τ GLYPH<8> α GLYPH<1> 1 { 2 a x VB q ¥ 1 GLYPH<1> α .

While these confidence intervals are asymptotically valid under our regularity assumptions, they may be overly conservative in general. In particular, they will over cover when the Chebyshev tail bound is loose. We conjecture that the adaptive Horvitz-Thompson estimator under CLIP-OGD satisfies a Central Limit Theorem, which would imply asymptotic validity of the narrower Wald-type intervals where α GLYPH<1> 1 { 2 scaling is replaced with the corresponding normal quantile, Φ GLYPH<1> 1 p 1 GLYPH<1> α { 2 q . As discussed in Section 7, the adaptive estimator appears approximately normal in simulations. Until this is formally shown, we recommend experimenters exhibit caution when using Wald-type confidence intervals for the adaptive Horvitz-Thompson estimator under CLIP-OGD.

6 Considering Alternative Designs

Explore-Then-Commit Two-stage adaptive designs have been proposed for the purpose of variance reduction [Hahn et al., 2011, Blackwell et al., 2022]. Due to its similarities to algorithms in the bandits

literature, we call these types of designs Explore-Then-Commit (ETC) [Lattimore and Szepesvári, 2020]. At a high level, an Explore-then-Commit design runs the Bernoulli design with p GLYPH<16> 1 { 2 for T 0 / T iterations, uses the collected data to estimate p GLYPH<6> by p p GLYPH<6> , and then runs the Bernoulli design with p GLYPH<16> p p GLYPH<6> for the remaining T 1 GLYPH<16> T GLYPH<1> T 0 iterations. These ETC designs are conceptually simpler than CLIP-OGD, and may be reasonable to apply in more restricted settings where changing the treatment probabilities is difficult or costly. However, we provide the following negative result which shows that they can suffer linear Neyman regret.

Proposition 6.1. For all explore phase lengths T 0 satisfying T 0 GLYPH<16> Ω p T glyph[epsilon1] q for some glyph[epsilon1] ¡ 0 , there exist a class of potential outcomes sequences satisfying Assumption 1 such that the Neyman regret under Explore-then-Commit is linear: R T GLYPH<16> Ω p p T q .

The specific class of potential outcomes referenced in Propposition 6.1 is constructed explicitly in Appendix E.1. ETC designs suffer larger variance when the estimated p p GLYPH<6> may be far from the true optimal probability p GLYPH<6> . In a design-based setting, this happens when the units in the explore phase are not representative of the entire sequence. Formulating conditions under which Explore-then-Commit designs achieve low Neyman regret is beyond the scope of this paper, but the proof of Proposition 6.1 shows that additional regularity conditions on the order of the units will be required.

Multi Arm Bandit Algorithms Multi Arm Bandit (MAB) algorithms are often used for adaptive decision making settings, from online advertising to product development. The goal of MAB algorithms is to minimize the outcome regret, which measures the contrast between the overall value obtained from the actions relative to the value of the best action. The outcome regret is conventionally defined as R outcome T GLYPH<16> max k Pt 0 , 1 u GLYPH<176> T t GLYPH<16> 1 y t p k q GLYPH<1> GLYPH<176> T t GLYPH<16> 1 Y t . In certain contexts, minimizing outcome regret may be a more desirable goal than estimating a treatment effect to high precision. However, the following proposition illustrates that these two objectives are generally incompatible.

Proposition 6.2. Let A be an adaptive treatment algorithm achieving sublinear outcome regret, i.e. there exists q P p 0 , 1 q such that E r R outcome T s / O p T q q for all outcome sequences satisfying Assumption 1. Then, there exists a class of outcome sequences satisfying Assumption 1 on which A suffers super-linear Neyman regret, i.e. E r R T s ¥ Ω p T 2 GLYPH<1> q q .

Proposition 6.2 demonstrates that the outcome regret and the Neyman regret cannot generally be simultaneously minimized. In particular, sublinear outcome regret implies that the variance of the estimator must converge slower than the Θ p 1 { T q parametric rate. This result contributes to a growing body of work which highlights trade-offs between various possible objectives in sequential decision making [Burtini et al., 2015]. It is beyond the scope of the current paper to determine how such trade-offs ought to be resolved, though Appendix F discusses ethical considerations.

7 Numerical Simulations

We evaluate the performance of CLIP-OGD and Explore-then-Commit (ETC) for the purpose of Adaptive Neyman Allocation on the field experiment of Groh and McKenzie [2016], which investigates the effect of macro-insurance on micro-enterprises in post-revolution Egypt 2 . The experimental units are 2,961 clients of Egypt's largest microfinance organization and the treatment was a novel insurance product. Several outcomes were recorded including whether the clients took on loans, introduced a new product or service, and the amount invested in machinery or equipment following treatment. To allocate treatment, Groh and McKenzie [2016] use a non-adaptive matched pair experimental design. Our goal here is not to provide a new analysis of this study, but rather to construct a plausible experimental setting under which to evaluate adaptive experimental designs.

In our simulations, we focus on the numerical outcome 'invested in machinery or equipment'. The experimental data contains only observed outcomes, so we must impute the missing potential outcomes in order to simulate the experiment. We impute outcomes using the model y t p 1 q GLYPH<1> y t p 0 q GLYPH<16> τ GLYPH<0> γ t , where τ GLYPH<16> 90 , 000 and γ 1 . . . γ T GLYPH<18> N p 0 , σ 2 q are independent with σ GLYPH<16> 5 , 000 . This randomness is invoked only to impute potential outcomes, i.e. not re-sampled during each run of the experiment. In order to increase the sample size, we create a larger population by repeating this processes 5 times, which yields a total of 14 , 445 units after those with missing entries are removed. Units are shuffled to appear in an arbitrary order and outcomes are normalized to be in the range r 0 , 1 s .

0

2500

5000

7500

10000

12500

15000

17500

Number of Rounds (

T

)

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Normalized Variance

T

Var(

)

Neyman

ETC

ClipOGD

(a) Original dataset

0

2500

5000

7500

10000

12500

15000

17500

Number of Rounds (

T

)

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Normalized Variance

T

Var(

)

Neyman

ETC

ClipOGD

(b) First 100 units have flipped potential outcomes

Figure 1 presents two plots illustrating how the variance of the adaptive HT estimator varies with different designs. The x axis contains the number of rounds T and the y axis contains the normalized variance T GLYPH<4> Var p ˆ τ q under the designs. For each value of T , we take the population to be the first T units in the sequence. CLIP-OGD is run with the parameters recommended in Theorem 4.2 and ETC is run with T 0 GLYPH<16> T 1 { 3 so that the exploration phase grows with T . The variance under CLIP-OGD and ETC is estimated empirically from 50,000 runs of the experiment, while the variance under the Bernoulli and Neyman designs is computed exactly.

In Figure 1a, we observe that CLIP-OGD requires about T GLYPH<16> 4 , 000 samples to achieve variance equal to Bernoulli, but eventually converges to the Neyman variance. As discussed in Section 4.2, it may be possible to improve the convergence rate by incorporating knowledge of the outcome moments in the design parameters. On the other hand, ETC remains comparable with Bernoulli even for small values of T , but remains far away from the Neyman design for large samples. In Figure 1b, a similar simulation is run, except that the potential outcomes of the first 100 units are swapped, so that the first units have negative individual treatment effects. While this produces little effect on the performance of CLIP-OGD, it substantially worsens the performance of ETC, which relies on the early outcomes to estimate an optimal treatment probability. In particular, ETC performs worse than Bernoulli under this minor modification-even in large samples-corroborating Proposition 6.1.

In the appendix, we evaluate the proposed confidence intervals, showing that CLIP-OGD enjoys intervals of reduced width. We show that normal based intervals cover at the nominal level and provide further evidence that the estimator is asymptotically normal under CLIP-OGD. We run additional simulations to investigate the sensitivity of the step size, and to demonstrate that additional baselines which were not designed for Neyman allocation indeed perform poorly.

8 Conclusion

In this paper, we have proposed the Neyman ratio and Neyman regret as a performance measure of experimental designs for the Adaptive Neyman Allocation problem. To this end, we proposed CLIP-OGD which achieves r O p ? T q expected Neyman regret under mild regularity conditions on the outcomes. This formally establishes-for the first time-the existence of adaptive experimental designs under which the variance of the effect estimator quickly approaches the Neyman variance. Finally, we have provided a variance estimator which provides experimenters with uncertainty quantification methods when using CLIP-OGD. The main drawback of our analysis is that it is most relevant for moderate and large sample sizes; in particular, our work does not properly address whether adaptive designs are always beneficial in small samples.

There are several research directions which can improve relevance of this methodology to practice. First, establishing conditions under which a central limit theorem holds for CLIP-OGD will yield smaller and thus more desirable Wald-type confidence intervals. Second, investigations into batched treatment allocations and delayed observations of outcomes would allow practitioners more flexibility in their designs. Finally, investigating variants of Adaptive Neyman Allocation in the presence of interference [Aronow and Samii, 2017, Harshaw et al., 2022] would allow for more realistic inference in complex settings, e.g. social network experiments and marketplace experiments.

Acknowledgments and Disclosure of Funding

We thank P.M. Aronow, Molly Offer-Westort, Alexander Rakhlin, Benjamin Recht, Fredrik Sävje, and Daniel Spielman for insightful discussions which helped shaped this work. Part of this work was done while Christopher Harshaw was visiting the Simons Institute for the Theory of Computing. Christopher Harshaw gratefully acknowledges support from Foundations of Data Science Institute (FODSI) NSF grant DMS2023505.

References

P.M. Aronow and Cyrus Samii. Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities. Survey Methodology , 39(1):231-241, 2013.

P.M. Aronow and Cyrus Samii. Estimating average causal effects under general interference. Annals of Applied Statistics , 11(4):1912-1947, 2017. doi: 10.1214/16-aoas1005.

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning , 47(2-3):235-256, may 2002a. ISSN 0885-6125. doi: 10.1023/A: 1013689704352.

Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing , 32(1):48-77, 2002b. doi: 10.1137/S0097539701398375.

AD Barker, CC Sigman, GJ Kelloff, NM Hylton, DA Berry, and LJ Esserman. I-spy 2: An adaptive breast cancer trial design in the setting of neoadjuvant chemotherapy. Clinical Pharmacology & Therapeutics , 86(1):97-100, 2009. doi: 10.1038/clpt.2009.68.

Matthew Blackwell, Nicole E. Pashley, and Dominic Valentino. Batch adaptive designs to improve efficiency in social science experiments. Working paper, Harvard University, 2022. URL https: //www.mattblackwell.org/files/papers/batch_adaptive.pdf .

Jack Bowden and Lorenzo Trippa. Unbiased estimation for response adaptive clinical trials. Statistical methods in medical research , 26, 08 2015.

Giuseppe Burtini, Jason Loeppky, and Ramon Lawrence. A survey of online experiment design with the stochastic multi-armed bandit. arXiv preprint arXiv:1510.00757 , 2015.

Yong Cai and Ahnaf Rafi. On the performance of the neyman allocation with small pilots. arXiv:2206.04643, 2022.

Jeffrey R. Eisele. The doubly adaptive biased coin design for sequential clinical trials. Journal of Statistical Planning and Inference , 38(2):249-261, 1994. ISSN 0378-3758.

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical principles and guidelines for the commission for the protection of human subjects of biomedical and behavioral research. Technical report, US Department of Health, Education, and Welfare, 1978.

David A. Freedman. On regression adjustments to experimental data. Advances in Applied Mathematics , 40:180-193, 2008. doi: 10.1016/j.aam.2006.12.003.

Matthew Groh and David McKenzie. Macroinsurance for microenterprises: A randomized experiment in post-revolution Egypt. Journal of Development Economics , 118:13-25, 2016. doi: 10.1016/j. jdeveco.2015.08.003.

Vitor Hadad, David A. Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey. Confidence intervals for policy evaluation in adaptive experiments. PNAS , 118(15), 2021. doi: 10.1073/pnas. 2014602118.

Jinyong Hahn, Keisuke Hirano, and Dean Karlan. Adaptive experimental design using the propensity score. Journal of Business & Economic Statistics , 29(1):96-108, 2011.

Dae Woong Ham, Iavor Bojinov, Michael Lindon, and Martin Tingley. Design-based confidence sequences for anytime-valid causal inference. arXiv:2210.08639, 2022.

Christopher Harshaw, Joel A. Middleton, and Fredrik Sävje. Optimized variance estimation under interference and complex experimental designs. arXiv:2112.01709, 2021.

Christopher Harshaw, Fredrik Sävje, and Yitan Wang. A design-based riesz representation framework for randomized experiments. arXiv:2210.08698, 2022.

Lakhbir S. Hayre and Bruce W. Turnbull. Estimation of the odds ratio in the two-armed bandit problem. Biometrika , 68(3):661-668, 1981.

Elad Hazan. Introduction to online convex optimization. Foundations and Trends® in Optimization , 2(3-4):157-325, 2016. ISSN 2167-3888.

G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association , 47(260):663-685, 1952. doi: 10.1080/01621459.1952.10483446.

Steven R. Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics , 49(2):1055 - 1080, 2021.

Feifang Hu and William F. Rosenberger. Optimality, variability, power: Evaluating responseadaptive randomization procedures for treatment comparisons. Journal of the American Statistical Association , 98(463):671-678, 2003.

Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences . Cambridge University Press, 2015.

Tor Lattimore and Csaba Szepesvári. Bandit Algorithms . Cambridge University Press, 2020.

Michael P. Leung. Causal inference under approximate neighborhood interference. Econometrica , 90(1):267-293, 2022.

Winston Lin. Agnostic notes on regression adjustments to experimental data: Reexamining Freedman's critique. Annals of Applied Statistics , 7(1):295-318, 2013.

Narain. On sampling without replacement with varying probabilities. Journal of the Indian Society of Agricultural Statistics , 3:169--175, 1951.

Jerzy Neyman. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science , 5(4):465-472, 1923. Reprinted in 1990.

Jerzy Neyman. On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society , 97(4): 558-625, 1934. ISSN 09528385.

Molly Offer-Westort, Alexander Coppock, and Donald P. Green. Adaptive experimental design: Prospects and applications in political science. American Journal of Political Science , 65(4): 826-844, 2021.

Herbert Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society , 58(5):527 - 535, 1952.

WRosenberger, N Stallard, Anastasia Ivanova, C Harper, and M Ricks. Optimal adaptive designs for binary response trials. Biometrics , 57:909-13, 10 2001.

Donald B. Rubin. Comment: Randomization analysis of experimental data. Journal of the American Statistical Association , 75(371):591, 1980.

Fredrik Sävje, P. M. Aronow, and Michael G. Hudgens. Average treatment effects in the presence of unknown interference. Annals of Statistics , 49(2):673-701, 2021.

Eric M. Schwartz, Eric T. Bradlow, and Peter S. Fader. Customer acquisition via display advertising using multi-armed bandit experiments. Marketing Science , 36(4):500-522, 2017.

Solomon and S. Zacks. Optimal design of sampling from finite populations: A critical review and indication of new research areas. Journal of the American Statistical Association , 65(330): 653-677, 1970.

Wald. Sequential Tests of Statistical Hypotheses. The Annals of Mathematical Statistics , 16(2): 117 - 186, 1945.

Yanxun Xu, Lorenzo Trippa, Peter Müller, and Yuan Ji. Subgroup-based adaptive (suba) designs for multi-arm biomarker trials. Statistics in Biosciences , 8, 02 2014.

Kelly Zhang, Lucas Janson, and Susan Murphy. Inference for batched bandits. In Advances in Neural Information Processing Systems , volume 33, pages 9818-9829. Curran Associates, Inc., 2020.

Kelly Zhang, Lucas Janson, and Susan Murphy. Statistical inference with m-estimators on adaptively collected data. In Advances in Neural Information Processing Systems , volume 34, pages 7460=7471. Curran Associates, Inc., 2021.

Lanju Zhang and William F. Rosenberger. Response-adaptive randomization for clinical trials with continuous outcomes. Biometrics , 62(2):562-569, 2006.

Appendix

Table of Contents

A Additional Simulation Results

In this section, we present additional simulation results on the Groh and McKenzie [2016] data. We refer to Section 7 for a review of the experimental set-up. In this section, we focus on the full dataset where T GLYPH<16> 14 , 445 . Simulations were run on a 2019 MacBook Pro with 2.4 GHz Quad-Core Intel Core i5 and 16 GB LPDDR3 RAM.

A.1 Confidence Intervals

Table 1 presents the Chebyshev-based and Normal-based intervals for the Bernoulli design p p GLYPH<16> 1 { 2 q and CLIP-OGD. We see that while Chebyshev over-covers, the normal-based confidence intervals cover at the nominal level with reduced width for both designs. The relative Neyman efficiency on this dataset is somewhat close to 1 , so that the reduction of the width of the confidence intervals afforded by CLIP-OGD is present, though modest. The coverage of the normal-based confidence intervals provides further evidence supporting our conjecture that the adaptive Horvitz-Thompson estimator is asymptotically normal under CLIP-OGD.

4

2

0

2

4

0

500

1000

1500

2000

2500

3000

3500

Counts

Figure 2 plots the histogram of the studentized adaptive Horvitz-Thompson estimator under CLIP-OGD. By studentized, we mean that the histogram is plotting the draws of the random variable

We estimate the standard deviation empirically from 50,000 runs of the experiment. The estimator is said to be asymptotically normal if Z d GLYPH<221> GLYPH<209> N p 0 , 1 q . Figure 2 provides evidence that asymptotic normality is likely to hold in this setting. Formally establishing asymptotic normality is beyond the scope of the current paper as it would involve very different analytic techniques than those used to establish sublinear Neyman regret.

A.2 Sensitivity to Step Size

In this section, we explore through simulations how the performance of CLIP-OGD depends on the step size.

0

2500

5000

7500

10000

12500

15000

17500

Number of Rounds (

T

)

0.6

0.7

0.8

0.9

1.0

1.1

Normalized Variance

T

Var(

)

Neyman

ClipOGD

Clip-OGD (0.25)

Clip-OGD (0.5)

Clip-OGD (2.0)

Clip-OGD (4.0)

In Figure 3, we re-create Figure 1a in the main paper but we have added instances of CLIP-OGD with different step sizes of the forms η GLYPH<16> c { ? T for c P t 0 . 25 , 0 . 5 , 1 . 0 , 2 . 0 , 4 . 0 u . We find that smaller step sizes improve convergence rates, effectively removing the 'overhead of adaptivity' in this example. However, because the randomized experiment can only be run once, experimenters will typically not be able to try many step sizes. While it remains an open question about how to select a step size which best mitigates the 'overhead of adaptivity', our recommendation of 1 { ? T still maintains good convergence properties.

A.3 Alternative Designs

In this section, we conduct additional experiments to compare the results of CLIP-OGD to alternative experimental designs which are not made for Neyman allocation. Indeed, we find that the alternative designs incur a high variance, relative to CLIP-OGD and the Neyman variance.

0

2500

5000

7500

10000

12500

15000

17500

Number of Rounds (

T

)

0.000

0.005

0.010

0.015

0.020

0.025

0.030

)

Neyman

ETC

DBCD

ClipOGD

(a) DBCD only

0

2500

5000

7500

10000

12500

15000

17500

Number of Rounds (

T

)

0

1

2

3

4

5

6

)

Neyman

ETC

DBCD

ClipOGD

(b) DBCD and EXP3

In Figure 4, we plot the variance of the adaptive designs on an unnormalized scale. We include "Doubly Biased Coin Design" proposed by Eisele [1994] as DBCD in Fig 4a and both DBCD and EXP3 in Fig 4b. We find that both DBCD and EXP3 suffer from higher variance. This is because they are not designed for Adaptive Neyman Allocation as defined in this paper: DBCD targets a different allocation rule and EXP3 minimizes outcome regret so that essentially only one arm is pulled. Both of these algorithms let the sampling probabilities P t get too close to the boundary of r 0 , 1 s , resulting in excessively large variance.

B General Analysis of Adaptive Neyman Allocation

In this section, we provide general analysis relevant for the problem of Adaptive Neyman Allocation. In Section B.1, we analyze the adaptive Horvitz-Thompson estimator. In Section B.2, we derive the optimal non-adaptive Neyman design in terms of the potential outcomes. In Section B.3, we show the equivalence of Neyman ratio and expected Neyman regret. For completeness, all propositions re-appear in the appendix.

B.1 Analysis of Adaptive Horvitz-Thompson Estimator (Propositions 2.1 and 2.2)

Throughout these proofs, we break up the adaptive Horvitz-Thompson estimator into the sum of individual estimators. For each t P r T s , define

so that the sequential Horvitz-Thompson estimator is equal to ˆ τ GLYPH<16> p 1 { T q GLYPH<176> T t GLYPH<16> 1 ˆ τ t . This mirrors how the average treatment effect is the average of individual treatment effects, i.e. τ GLYPH<16> p 1 { T q GLYPH<176> T t GLYPH<16> 1 τ t .

We begin by proving Proposition 2.1, which establishes the unbiasedness of the adaptive Horvitz-Thompson estimator, subject to a positivity condition.

Proposition 2.1. If min t P t , 1 GLYPH<1> P t u ¡ 0 almost surely for all t P r T s then the adaptive Horvitz-Thompson estimator is unbiased: E r ˆ τ s GLYPH<16> τ .

Proof. Observe that by linearity of expectation, we can break the expectation of the adaptive Horvitz-Thompson estimator as

Thus, it suffices to show that the individual effect estimators are unbiased: E r ˆ τ t s GLYPH<16> τ t . Observe that if the positivity condition holds, then we have that the conditional expectation may be computed as

The result follows by iterated expectation, E r ˆ τ t s GLYPH<16> E r E r ˆ τ t | F t ss GLYPH<16> τ t .

Next, we prove Proposition 2.2, which derives the variance of the adaptive Horvitz-Thompson estimator.

Proposition 2.2. The variance of the adaptive Horvitz-Thompson estimator is

Proof. We begin by decomposing the variance of the adaptive Horvitz-Thompson estimator as

We now aim to compute each of these individual covariance terms. Before continuing, observe that, by construction, the individual effect estimators are conditionally unbiased E r ˆ τ t | F t s GLYPH<16> τ t . It follows by iterated expectation that the individual effect estimators are unbiased (unconditionally), i.e. E r ˆ τ t s GLYPH<16> τ t . Suppose that s ¡ t . In this case, the covariance between the individual estimators is equal to zero as,

Now let us compute the variance of an individual effect estimator. Observe that the variance may be decomposed as

Because the individual estimator is unbiased, we have that E r ˆ τ t s 2 GLYPH<16> τ 2 t . Let us now analyze the first term.

(iterated expectation)

Thus, this establishes that the variance of an individual estimator is equal to

Combining terms, we have that the variance of the adaptive Horvitz-Thompson estimator is

B.2 Derivation of the Neyman Design (Proposition 3.1)

In this section, we prove Proposition 3.1 which derives the (infeasible) non-adaptive Neyman design in terms of the Neyman probability p GLYPH<6> and corresponding Neyman variance V N. We also show that, under Assumption 1, the Neyman variance achieves the parametric rate.

Proposition 3.1. The Neyman variance is T GLYPH<4> V N GLYPH<16> 2 p 1 GLYPH<0> ρ q S p 1 q S p 0 q , which is achieved by the Neyman probability p GLYPH<6> GLYPH<16> p 1 GLYPH<0> S p 0 q{ S p 1 qq GLYPH<1> 1 .

Proof. Using Proposition 2.2, we have that the variance of the (non-adaptive) Bernoulli design with probability p P p 0 , 1 q is equal to

Thus, the optimal Neyman design is obtained by the p GLYPH<6> which minimizes the above. The first order condition stipulates that

which is solved by p GLYPH<6> GLYPH<16> GLYPH<0> 1 GLYPH<0> S p 0 q{ S p 1 q GLYPH<8> GLYPH<1> 1 . Substituting this p GLYPH<6> back into the variance yields the Neyman variance:

Next, we show that under Assumption 1, the Neyman variance achieves the parametric rate.

Proposition B.1. Under Assumption 1, the Neyman variance achieves the parametric rate: V N GLYPH<16> Θ p 1 { T q .

Proof. Proposition 3.1, derives the Neyman variance: T GLYPH<4> V N GLYPH<16> 2 p 1 GLYPH<0> ρ q S p 1 q S p 0 q .

We begin by showing that the Neyman variance is asymptotically bounded from below. Moreover, Assumption 1 stipulates that there exists a constant c ¡ 0 which lower bounds the second moments as S p 1 q ¥ c and S p 0 q ¥ c and the correlation as p 1 GLYPH<0> ρ q ¥ c . Thus, the normalized Neyman variance is bounded below as T GLYPH<4> V N ¥ 2 c 3 .

Next, we show that the Neyman variance is asymptotically bounded from above at the same rate. Assumption 1 stipulates that there exists a constant C ¡ 0 which upper bounds the second moments as S p 1 q / C and S p 0 q / C . The correlation is bounded between ρ P rGLYPH<1> 1 , 1 s so that p 1 GLYPH<0> ρ q / 2 . These bounds together yield that the normalized Neyman variance is bounded above as T GLYPH<4> V N / 4 C 2 .

Together, these bounds establish that, under Assumption 1, we have that V N GLYPH<16> Θ p 1 { T q .

B.3 Equivalence of Neyman Ratio and Neyman Regret (Theorem 4.1)

In this section, we prove Theorem 4.1, which demonstrates the equivalence between the Neyman Ratio and the expected Neyman regret.

Theorem 4.1. Under Assumption 1, the Neyman ratio is within a constant factor of the 1 { T -scaled expected Neyman regret: κ T GLYPH<16> Θ p 1 T E r R T sq .

Proof. Recall that the Neyman ratio is defined as

where the second equality follows by multiplying the numerator and the denominator by T . Observe that by Proposition 2.2, the numerator is given by

Proposition B.1 shows that under Assumption 1, T GLYPH<4> V N GLYPH<16> Θ p 1 q so that the denominator is asymptotically constant. Thus, we have that κ T GLYPH<16> Θ p 1 T E r R T sq .

C Analysis of Neyman Regret for CLIP-OGD

In this section, we will prove Theorem 4.2, which establish that CLIP-OGD achieves O p ? T log T q expected Neyman regret under our assumptions on the potential outcomes. While the main paper used capital letters P t and G t to signify that the treatment probability and gradient estimator were random variables, we use lower case letters p t and g t in the appendix for the purposes of more aesthetically appealing proofs. Throughout the analysis, we define ∆ t GLYPH<16> r δ t , 1 GLYPH<1> δ t s and a GLYPH<16> 1 GLYPH<0> C { c for notational convenience.

C.1 Proof of Theorem 4.2

The first lemma is a bound on the distance of treatment probability p t GLYPH<0> 1 to the optimal p GLYPH<6> in terms of the previous treatment probability, gradient estimate, and whether the projection interval contains the optimal p GLYPH<6> .

Lemma C.1. For each iteration t P r T s ,

Proof. If p GLYPH<14> P ∆ t , then the statement holds by Pythagorean theorem. Otherwise, note that the most that the projection operation onto ∆ t can move a point is exactly δ t . Therefore, the distance between P δ t p p t GLYPH<1> ηg t q and p GLYPH<14> is at most δ t larger than that between p t GLYPH<1> ηg t and p GLYPH<14> .

Next, we show that Assumption 1 implies that the optimal Neyman probability lies within an interval bounded away from zero.

Lemma C.2. Under Assumption 1, p GLYPH<6> P r 1 a , 1 GLYPH<1> 1 a s , where a GLYPH<16> 1 GLYPH<0> C { c ¥ 2 .

Proof. As shown previously, the optimal Neyman probability is equal to

Recall that Assumption 1 places the following moment conditions on the potential outcomes:

The result follows by using the definition of a GLYPH<16> 1 GLYPH<0> C { c to deduce that p 1 { a q GLYPH<16> p 1 GLYPH<0> C { c q GLYPH<1> 1 and p 1 GLYPH<1> 1 { a q GLYPH<16> p 1 GLYPH<0> c { C q GLYPH<1> 1 .

The next lemma guarantees that after a fixed number of iterations, the projection interval will contain the Neyman optimal p GLYPH<6> .

Lemma C.3. We have that p GLYPH<6> P ∆ t for all t ¥ p a { 2 q α .

Proof. Lemma C.2 guarantees that p GLYPH<6> P r 1 { a, 1 GLYPH<1> 1 { a s , where a GLYPH<16> 1 GLYPH<0> C { c . Thus, p GLYPH<6> P ∆ t if δ t / 1 { a . Using the definition of δ t and rearranging terms, we have that

The next lemma bounds the expected difference between the cost objective f t evaluated at p t and the cost objective f t evaluated at the Neyman optimal probability p GLYPH<6> .

Lemma C.4. For each iteration t P r T s , we have the bound

Proof. Fix an iteration t P r T s . By Lemma C.1, and using the triangle inequality, we have that

Because p t P r δ t , 1 GLYPH<1> δ t s , we have that | p t GLYPH<1> p GLYPH<6> | / 1 GLYPH<1> δ t so that

This bounds p GLYPH<6> by

.

Rearranging terms yields that

Dividing both sides by the step size 2 η yields

Using convexity of f t , adding and subtracting terms, and using the above, we have that

By construction, we have that the gradient estimator is unbiased conditioned on D t , i.e. E r g t | D t s GLYPH<16> ∇ f t p p t q . Thus, by iterated expectation we have that the gradient estimator is unbiased, i.e.

Thus, taking expectations of both sides and applying Lemma C.3 yields

The following lemma derives bounds on the first and second (raw) moments of the gradient estimator at each iteration. Lemma C.5. For each t P r T s , the gradient estimates have bounded first and second moments:

GLYPH<25>

GLYPH<17>

Proof. We begin by handling the E g 2 t term. By definition of the gradient estimator, we have that the conditional expectation is at most

By definition of the algorithm, we have that p t P r δ t , 1 GLYPH<1> δ t s at iteration t . Thus, we may invoke the bound:

and the desired bound on E r g 2 t s follows from applying the law of iterated expectation.

The bound on the E GLYPH<17> | g t | GLYPH<25> term follows in a similar way. By definition of the gradient estimator, we have that the conditional expectation is at most

and the desired bound on E r | g t | s follows from applying the law of iterated expectation.

The following proposition bounds the expected Neyman regret for general settings of the projection parameter α . Proposition C.1. Suppose Assumption 1 holds. Then, for any choice of projection parameter α ¥ 2 (possibly depending on T ) and for the step size η GLYPH<16> b e α T 1 GLYPH<0> 5 { α , a finite-sample bound on the expected Neyman regret incurred by CLIP-OGD is

Proof. By Lemma C.4, we have that the regret is at most

Using a telescoping argument, we have that the first term is bounded by

Using Lemma C.5 and Assumption 1, the sum in the second term may be bounded as

Next, we deal with the third term by breaking it up into two more terms. Define t GLYPH<6> GLYPH<16> r p a { 2 q α s . The third term can be broken into two terms:

The first of these two terms can be bounded in the following way. Using that α ¥ 2 we have that

The second term can be bounded using Lemma C.5 as follows:

where the last inequality follows from Cauchy-Schwarz. By extending the sum involving the outcomes to all units, we obtain an upper bound on which we can apply the bounded moment assumption:

What remains in this step is to bound the first term above. By replacing each of the t in the sum with t GLYPH<6> , we obtain the following upper bound:

Using the above work, we have that the overall regret is bounded as

Next, we separate the constant a from the projection parameter α . To this end, we use the inequality y r / e 1 GLYPH<0> y GLYPH<4> e r for all y P R and r ¥ 0 , to obtain

where we have used that a ¥ 2 . Substituting these back into the above, we have that the regret bound is Substituting this quantities into the above, we have that the regret bound is

By setting η GLYPH<16> b e α T 1 GLYPH<0> 5 { α to minimize the bound above, we have that the expected Neyman regret is bounded as

where we have used that and the result follows by dividing both sides by 2 .

Proposition C.1 demonstrates that many different values of α will guarantee sublinear expected Neyman regret. For example, setting α to be a constant satisfying α ¡ 5 will ensure sublinear expected Neyman regret. However, by tuning α according to the analysis above, we can achieve O p a T log p T qq expected Neyman regret, as demonstrated by Theorem 4.2.

Theorem 4.2*. Under Assumption 1 the parameter values η GLYPH<16> a 1 { T and α GLYPH<16> a 5 log p T q ensure the expected Neyman regret of CLIP-OGD is bounded as

which implies that E GLYPH<16> R T GLYPH<24> / r O GLYPH<0> ? T GLYPH<8> .

Proof. Observe that for α GLYPH<16> a 5 log p T q , we have that the step size posited in Proposition C.1 (i.e. η GLYPH<16> b e α T 1 GLYPH<0> 5 { α ) is equal to η GLYPH<16> a 1 { T . Thus, by rearranging terms and using the result of Proposition C.1, we have that expected Neyman regret is bounded as

The difficulty is now to find which setting of α will make this bound smallest. Observe that the real tension is in the first term and we can re-write the relevant part of this term as

To minimize this term, we select α GLYPH<16> a 5 log p T q which results in

Likewise, this choice of α results in ? e α / e α GLYPH<16> exp p a 5 log p T qq . Putting this together yields the desired finite sample regret bound:

The result follows by observing that the terms inside the parenthesis are constant by Assumption 1 and the function exp p a 5 log p T qq is subpolynomial.

C.2 Selecting Parameters When Moment Bounds are Known

We briefly remark on how to select the step size parameter η when the experimenter can correctly specify the constants C ¥ c used in the Assumption 1. The proof of Proposition C.1 shows that for general parameters, the Neyman regret may be bounded as

To optimize the step size with respect to these constants, one would choose α GLYPH<16> a 5 log p T q and

where we have used that a GLYPH<16> 1 GLYPH<0> C { c . When these moment bounds are correctly specified, this choice of step size will likely yield improved convergence rates, as our bound on the Neyman regret will have a factor of C 2 rather than C 4 .

D Analysis for Inference in Large Samples

In this section, we provide the necessary statistical tools for constructing asymptotically valid confidence intervals. This can be done in two main steps. First, in Section D.1 we construct a conservative variance estimator which we show is consistent in probability. Then, in Section D.2, we show that the resulting Chebyshev-type intervals are asymptotically valid.

While the main paper used capital letters P t and G t to signify that the treatment probability and gradient estimator were random variables, we use lower case letters p t and g t in the appendix for the purposes of more aesthetically appealing proofs. Throughout the analysis, we use the parameter settings η GLYPH<16> a 1 { T and α GLYPH<16> a 5 log p T q , which are recommended in the main paper. However, we suspect that many of our results will go through for the class of parameters η GLYPH<16> b e α T 1 GLYPH<0> 5 { α and α ¡ 5 which appear in Proposition C.1.

The following lemma shows that under Assumptions 1 and 2, the variance of the adaptive Horvitz-Thompson estimator under CLIP-OGD achieves the parametric rate.

Lemma D.1. Assumptions 1 and 2, the variance of the adaptive Horvitz-Thompson estimator under CLIP-OGD achieves the parametric rate: Var p ˆ τ q GLYPH<16> Θ p 1 { T q .

Proof. Theorem 4.1 shows that under Assumption 1, they Neyman ratio is order equivalent to the 1 { T -scaled expected Neyman regret, i.e. κ T GLYPH<16> Θ pp 1 { T q E r R T sq . Theorem 4.2 shows that under these assumptions, CLIP-OGD achieves sublinear expected Neyman regret E r R T s GLYPH<16> o p T q which implies that lim sup κ T / 0 . Likewise, Assumption 2 states that the negative expected Neyman regret is sublinear, GLYPH<1> E r R T s GLYPH<16> o p T q , which implies that lim inf κ T ¥ 0 . Thus, we have that the Neyman ratio converges to zero, e.g. lim κ T GLYPH<16> 0 .

By recalling the definition of the Neyman ratio, we have that

.

Proposition B.1 demonstrates that T GLYPH<4> V N GLYPH<16> Θ p 1 q . Together with the above, this implies that T GLYPH<4> V GLYPH<16> Θ p 1 q .

The following lemma shows that under the recommended parameter settings, the (random) treatment probabilities are bounded away from zero and one.

Lemma D.2. When α GLYPH<16> a 5 log p T q , we have that for all iterations t P r T s , the inverse of projection parameter is bounded:

Proof. A uniform upper bound on the inverse of the projection parameters is

a

a

To complete the proof, observe that the function h p T q GLYPH<16> exp p 1 { 5 GLYPH<4> log p T qq is subpolynomial, so that we can write it as r O p 1 q .

D.1 Conservative Variance Estimator (Theorem 5.1)

In this section, we prove Theorem 5.1 which establishes that the normalized variance estimator is converges in probability to the normalized variance upper bound at a r O p p T GLYPH<1> 1 { 2 q rate. Before continuing, let us review the relevant quantities. Recall that the Neyman variance and the corresponding upper bound are given by

where the second moments S p 1 q and S p 0 q are defined as

Our variance estimator is defined as

The random variables z A p 1 q and z A p 0 q are unbiased estimates of S p 1 q 2 and S p 0 q 2 which are based on the HorvitzThompson principle. However, the fact that the estimators z A p 1 q and z A p 0 q are not independent and the square root is introduced means that the variance estimator x VB is not an unbiased estimator for the variance bound VB. Even so, we will show that the variance estimator is consistent for the variance bound. For aesthetic considerations, we define A p 1 q GLYPH<16> S p 1 q 2 and A p 0 q GLYPH<16> S p 0 q so that z A p 1 q is an estimator for A p 1 q and z A p 0 q is an estimator for A p 0 q .

Our general approach will follow in two steps. First, by bounding its bias and variance, we will show that z A p 1 q GLYPH<4> z A p 0 q GLYPH<1> A p 1 q A p 0 q converges as r O p p T GLYPH<1> 1 { 2 q . Next, by appealing to a quantitative Continuous Mapping Theorem, we will argue that the error b z A p 1 q GLYPH<4> z A p 0 q GLYPH<1> a A p 1 q A p 0 q converges at the same rate. By definition, this is exactly the error of the normalized variance estimator to the normalized variance bound, i.e. T GLYPH<4> x VB GLYPH<1> T GLYPH<4> VB.

Before continuing, let us define new auxiliary random variables. For each t P r T s , we define the variables r t and q t as

Below are basic facts about these auxiliary random variables.

Lemma D.3. The auxiliary random variables satisfy the following properties:

E r r t q s s GLYPH<16> 1 r t GLYPH<24> s s .

E r r 2 t | D t s / 1 δ t and E r q 2 t | D t s / 1 δ t .

The covariance Cov p r t q s , r glyph[lscript] q k q behave in the following ways:

Proof. First, we show that E r r t q s s GLYPH<16> 1 r t GLYPH<24> s s . Let t, s P r T s and suppose that t GLYPH<24> s . Without loss of generality, suppose that t ¡ s . Then by using iterated expectation, we have that

Otherwise, if t GLYPH<16> s , then r t q t GLYPH<16> p 1 r z t GLYPH<16> 1 s{ p t q GLYPH<4> p 1 r z s GLYPH<16> 0 s{p 1 GLYPH<1> p t qq GLYPH<16> 0 so that E r r t q t s GLYPH<16> 0 .

Next, we show that E r r 2 t | D t s / 1 δ t and E r q 2 t | D t s / 1 δ t . Observe that E r r 2 t | D t s GLYPH<16> p t p 1 { p 2 t q GLYPH<16> 1 { p t / 1 { δ t , where the inequality follows by definition of Algorithm 1. A similar argument shows that E r q 2 t | D t s / 1 { δ t .

Finally, we establish the covariance terms one by one. We do this in order of the cases that they were presented in.

Case 1 p t GLYPH<16> s q or p glyph[lscript] GLYPH<16> k q : If t GLYPH<16> s , then r t q t is almost surely zero, as argued above. Likewise, if glyph[lscript] GLYPH<16> k then r glyph[lscript] q glyph[lscript] is almost surely zero. In either of these cases, we have that Cov p r t q s , r glyph[lscript] q k q GLYPH<16> 0 .

Case 2 p t GLYPH<24> s q and p glyph[lscript] GLYPH<24> k q and t t, s u X t glyph[lscript],k u GLYPH<16>H : Note that in this case, all the indices t , s , glyph[lscript] , and k are distinct. Without loss of generality, suppose that t GLYPH<160> s GLYPH<160> glyph[lscript] GLYPH<160> k . A repeated use of the iterated expectation yields that

Because all terms of distinct, we have that E r r t q s s GLYPH<16> 1 and E r r glyph[lscript] q k s GLYPH<16> 1 . Thus, the covariance is equal to

Case 3 p t GLYPH<24> s q and p glyph[lscript] GLYPH<24> k q and ( t GLYPH<16> k or s GLYPH<16> glyph[lscript] ) : Suppose that t GLYPH<16> k . In this case, observe that r t q k is zero almost surely. Thus, E r r t q s r glyph[lscript] q k s GLYPH<16> 0 . On the other hand, t GLYPH<24> s and glyph[lscript] GLYPH<24> k so that E r r t q s s GLYPH<16> E r r glyph[lscript] q k s GLYPH<16> 1 . This means that

The same argument shows that s GLYPH<16> glyph[lscript] yields the same result.

Case 4 p t GLYPH<24> s q and p glyph[lscript] GLYPH<24> k q and t GLYPH<16> glyph[lscript] and s GLYPH<24> k ) : We begin by computing the expectation of the product of these four terms. In this case, we have that t GLYPH<16> glyph[lscript] so that

By assumption, the indices t , s , and k are all distinct. Our approach will be to obtain an upper bound on the expectation of the project of these three terms by iterated expectation. In particular, the inequality we will use is that E r r 2 t | D t s / 1 { δ t . Suppose for now that s GLYPH<160> k GLYPH<160> t . In this case, we use iterated expectation to get

In the above, we have assumes that s GLYPH<160> k GLYPH<160> t , but the same iterated expectation technique can be applied regardless of the ordering of these indices, because they are unique. Thus, in this case, E r r t q s r glyph[lscript] q k s / 1 { δ t . Because t GLYPH<24> s and glyph[lscript] GLYPH<24> k , we have that E r r t q s s GLYPH<16> E r r glyph[lscript] q k s GLYPH<16> 1 . This means that

Case 5 p t GLYPH<24> s q and p glyph[lscript] GLYPH<24> k q and t GLYPH<24> glyph[lscript] and s GLYPH<16> k ) : We begin by computing the expectation of the product of these four terms. In this case, we have that s GLYPH<16> k so that

By assumption, the indices t , glyph[lscript] , and s are all distinct. Using a similar argument as the previous case, we can use iterated expectation together with the bound E r q 2 s | D s s / 1 { δ 2 s to obtain that E r r t q s r glyph[lscript] q k s / 1 { δ 2 s . Because t GLYPH<24> s and glyph[lscript] GLYPH<24> k , we have that E r r t q s s GLYPH<16> E r r glyph[lscript] q k s GLYPH<16> 1 . This means that

Case 6 p t GLYPH<24> s q and p glyph[lscript] GLYPH<24> k q and t GLYPH<16> glyph[lscript] and s GLYPH<16> k ) : Suppose without loss of generality that t ¡ s . In this case, we can bound the product of the expectation of these four terms using iterated expectation and the proven inequalities:

Because t GLYPH<24> s , we have that E r r t q s s GLYPH<16> E r r glyph[lscript] , q k s GLYPH<16> 1 . Thus, the covariance is bounded by

First, we show that that the difference between the expected value of z A p 1 q z A p 0 q and the target A p 1 q A p 0 q is decreasing at a linear rate in T .

Proposition D.1. The absolute bias of the estimated crossing term z A p 1 q z A p 0 q to its target value A p 1 q A p 0 q is at most

Proof. Using Lemma D.3, we can calculate the expectation of the product z A p 1 q z A p 0 q as

We complete the proof by using Cauchy-Schwarz and Assumption 1, to bound the absolute bias as

Next, we show that the variance of z A p 1 q z A p 0 q is going to zero at a sufficiently fast rate.

Proposition D.2. The variance of z A p 1 q z A p 0 q is bounded as

Proof. We begin by decomposing the variance of z A p 1 q z A p 0 q into covariances of products of the auxiliary random variables r t and q s . To this end, observe that

Next, we will use the result of Lemma D.3 to handle the individual covariance terms. In particular, the first six types of terms, as described in Lemma D.3. The first three types of terms are at most 0 , so we may discard them from the sum, as they contribute no positive value. The last three terms have upper bounds, which we use here to obtain the following upper bound:

Our goal will now be to bound each of these terms individually.

Terms 1 and 2 : Terms 1 and 2 are similar and will be handled in the same way. Let's begin with Term 1. Observe that by Lyapunov's inequality, the moment assumptions, and Lemma D.2, we have that

A similar argument shows that T 2 / p 2 e ? log p T 1 { 5 q C 8 q{ T .

Term 3 : The third term may be upper bounded using Lemma D.2 and the moment assumptions. Namely,

Taken together, this shows that

This establishes the following corollary, which shows that the error z A p 1 q z A p 0 q GLYPH<1> A p 1 q A p 0 q is going to zero at a near-parametric rate, which follows from by Chebyshev's inequality from Propositions D.1 and D.2.

Corollary D.1. The following error goes to zero: z A p 1 q z A p 0 q GLYPH<1> A p 1 q A p 0 q GLYPH<16> r O p GLYPH<0> T GLYPH<1> 1 { 2 GLYPH<8> .

Using the results derived above, we are ready to prove Theorem 5.1, which we restate here for convenience.

Theorem 5.1. Under Assumptions 1 and 2, and the parameters stated in Theorem 4.2, the error of the normalized variance estimator under CLIP-OGD is T GLYPH<4> x VB GLYPH<1> T GLYPH<4> VB GLYPH<16> r O p p T GLYPH<1> 1 { 2 q .

Proof of Theorem 5.1. Recall that the variance estimator and the variance bound are equal to T GLYPH<4> VB GLYPH<16> 4 S p 1 q S p 0 q GLYPH<16> 4 a A p 1 q A p 0 q and T GLYPH<4> x VB GLYPH<16> 4 b z A p 1 q z A p 0 q so that the error is given by

Corollary D.1 states that the error z A p 1 q z A p 0 q GLYPH<1> A p 1 q A p 0 q is on the order of r O p GLYPH<0> T GLYPH<1> 1 { 2 GLYPH<8> . By Assumption 1, we have that A p 1 q A p 0 q GLYPH<16> a S p 1 q 2 S p 0 q 2 ¡ c 2 . Observe that the square root function g p x q GLYPH<16> ? x is Lipschitz on the interval p c 2 , 8q . Thus, by a rate-preserving Continuous Mapping Theorem, we have that the error of the normalized variance estimator is on the same order i.e. T GLYPH<4> VB GLYPH<1> T GLYPH<4> x VB GLYPH<16> a A p 1 q A p 0 q GLYPH<1> b z A p 1 q z A p 0 q GLYPH<16> r O p GLYPH<0> T GLYPH<1> 1 { 2 GLYPH<8> .

D.2 Valid Confidence Intervals (Corollary 5.1)

We now prove the asymptotic validity of the associated Chebyshev-type intervals. This proof is standard in the design-based literature, but we present it here for completeness.

Corollary 5.1. Under Assumptions 1 and 2, and parameters stated in Theorem 4.2, Chebyshev-type intervals are asymptotically valid: for all α P p 0 , 1 s , lim inf T GLYPH<209>8 Pr p τ P ˆ τ GLYPH<8> α GLYPH<1> 1 { 2 a x VB q ¥ 1 GLYPH<1> α .

Proof. Define the random variables

Observe that they are related in the following way:

By definition, we have that lim sup T GLYPH<209>8 Var p ˆ τ q{ VB / 1 . Recall that by Proposition B.1, T GLYPH<4> VB ¥ T GLYPH<4> V N GLYPH<16> Ω p 1 q so that by Theorem 5.1 and Continuous Mapping Theorem, we have that b T GLYPH<4> VB T GLYPH<4> x VB p GLYPH<221> GLYPH<209> 1 . Thus, by Slutsky's theorem we have that Z 1 is asymptotically stochastically dominated by Z . Now we are ready to compute the coverage probability.

where the last line followed from Chebyshev's inequality and the fact that Var p Z q GLYPH<16> 1 .

E Analysis of Alternative Designs

In this section, we provide analysis on the efficacy of existing adaptive experimental designs for the problem of Adaptive Neyman Allocation. To this end, we show two negative results. In Section E.1, we show that the two-stage design of [Hahn et al., 2011, Blackwell et al., 2022] (i.e. Explore-then-Commit) can suffer linear expected Neyman Regret in the design-based framework for a large class of potential outcome sequences. In Section E.2, we show that mutli-arm bandit algorithms which achieve sublinear expected outcome regret will incur super-linear expected Neyman regret, providing further evidence that these two goals are incompatible.

E.1 Analysis of Explore-then-Commit (Proposition 6.1)

In this setting, we show that Explore-then-Commit designs can sometimes suffer linear Neyman regret, and therefore not recover the Neyman variance in large samples. We formally introduce our definition of Explore-then-Commit designs below. Let p GLYPH<6> T 0 be defined as

which is the optimal Neyman probability when considering only the sample up to T 0 .

Our definition of ETC encompasses many possible ways of estimating the optimal treatment probability. The only requirement is that the estimation method will converge to p GLYPH<6> T 0 at the rate T GLYPH<1> 1 { 2 0 . We consider p GLYPH<6> T 0 rather than the true Neyman probability p GLYPH<6> because the observed data is informative only of the outcomes in the exploration phase T 0 . Many natural estimators will fall into this class, including Horvitz-Thompson style estimators similar to those used in the construction of our variance estimator.

Algorithm 2: EXPLORE-THEN-COMMIT

Input: Exloration phase size T 0

// Explore Phase for t GLYPH<16> 1 . . . T 0 do

Sample treatment assignment Z t as 1 with probability 1 { 2 and 0 with probability 1 { 2 Observe outcome Y t GLYPH<16> 1 r Z t GLYPH<16> 1 s y t p 1 q GLYPH<0> 1 r Z t GLYPH<16> 0 s y t p 0 q

end

Construct an estimator x p GLYPH<6> T 0 from observed data p Z 1 , Y 1 , . . . Z T 0 , Y T 0 q satisfying x p GLYPH<6> T 0 GLYPH<1> p GLYPH<6> T 0 GLYPH<16> O p p T GLYPH<1> 1 { 2 0 q .

//

Exploit

Phase for t GLYPH<16> T 0 GLYPH<0> 1 . . . T do

Sample treatment assignment Z t as 1 with probability x p GLYPH<6> T 0 and 0 with probability 1 GLYPH<1> x p GLYPH<6> T 0 Observe outcome Y t GLYPH<16> 1 r Z t GLYPH<16> 1 s y t p 1 q GLYPH<0> 1 r Z t GLYPH<16> 0 s y t p 0 q

end

Before continuing, we provide a few more definitions. We define the second moments of treatment and control outcomes as well as the correlation in the exploration phase as

We are now ready to state the formal version of Proposition 6.1.

Proposition 6.1*. Suppose that T 0 GLYPH<16> Ω p T glyph[epsilon1] q for some glyph[epsilon1] ¡ 0 and further suppose that the outcome sequence satisfies the following properties for constants C ¥ c ¡ 0 and c 1 ¡ 0 :

The second moments S p 1 q , S p 0 q , S T 0 p 1 q , and S T 1 p 0 q are contained in the interval r c, C s .

The correlations are bounded away from -1, i.e. ρ T 0 , ρ ¥ GLYPH<1> 1 GLYPH<0> c .

The second moments satisfy the following:

Then, the Neyman Regret of Explore-then-Commit is at least linear in probability, R T GLYPH<16> Ω p p T q .

The first two conditions are essentially extensions of Assumption 1 to the exploration phase. This ensures that the probability p GLYPH<6> T 0 (which is estimated in the Explore-then-Commit design) does not approach 0 or 1 . The third condition is what really makes Explore-then-Commit fail to achieve sublinear Neyman regret. This condition states that the ratio of the second moments in the exploration phase is different than in the larger sequence. For example, if S p 1 q GLYPH<16> S p 0 q GLYPH<16> 1 but S T 0 p 0 q{ S T 0 p 1 q GLYPH<16> 2 then the condition would hold. In this case, we should not expect Explore-then-Commit to achieve the Neyman variance because the exploration phase does not contain sufficient information about the optimal Neyman probability. We now prove the proposition.

Proof. Let p GLYPH<6> GLYPH<16> arg min p Pr 0 , 1 s GLYPH<176> T t GLYPH<16> 1 f t p p q be the Neyman probability. We begin by re-arranging terms in the Neyman regret:

Where the inequality follows by the optimality of p GLYPH<6> T 0 on the exploration phase. By adding and subtracting the f t p p GLYPH<6> T 0 q to the second sum, we obtain the following decomposition

We handle each of the terms separately in the remainder of the proof.

Term 1 : By the assumptions on the second moments and correlation of outcomes in the exploration phase, we have that p GLYPH<6> T 0 is bounded away from 0 and 1 by a constant. Furthermore, the cost functions f t are Lipschitz on the interval r γ, 1 GLYPH<1> γ s for any fixed γ . Thus, we may apply the quantitative Continuous Mapping Theorem to bound the absolute value of Term 1 as follows:

where the final inequality follows from the fact that T 0 / T .

Term 2 : A similar argument may be applied to the second term. Again, we may apply the continuous mapping theorem as before to obtain

where we have used the assumption that T 0 GLYPH<16> Ω p T glyph[epsilon1] q .

Term 3 : We now handle the third term. Let V 0 be the variance of adaptive Horvitz-Thompson estimator under the Bernoulli design when using the treatment probability p GLYPH<6> T 0 . By construction of the cost functions, we have that they are related to the variance as follows:

where the last equality follows by definition of the probabilities. By Assumption, we have that this bracketed term is constant so that the third term is linear in T .

Putting these together, we have that the Neyman regret is lower bounded as

E.2 Analysis of Designs for Outcome Regret (Proposition 6.2)

In this section, we prove Proposition 6.2, which establishes that outcome regret and Neyman regret cannot be simultaneously minimized in general. We restate a more formal version of the proposition here. In order for a simpler proof, we make restrictions that the units have constant treatment effect and that each of the individual outcomes are more strictly bounded. We conjecture that the trade-off between Neyman and outcome regret will hold under weaker conditions.

Proposition 6.2*. Let A be an adaptive treatment algorithm achieving sublinear outcome regret, i.e. there exists q P p 0 , 1 q such that E r R outcome T s / O p T q q for all outcome sequences satisfying Assumption 1. Consider an outcome sequence satisfying Assumption 1 with constants C ¥ c ¡ 0 and the additional conditions:

max 1 / t / T y t p 0 q 2 / C 2

For all t P r T s , y t p 1 q GLYPH<1> y t p 0 q GLYPH<16> τ and τ ¡ c 1 for a constant c 1 ¡ 0 .

Then, A suffers super-linear Neyman regret on this outcome sequence: E r R T s ¥ Ω p T 2 GLYPH<1> q q .

Proof. To begin, we re-express the outcome regret in terms of the expected treatment probabilities played by algorithm A . Observe that the expected outcome regret may be written as

where the last equality follow as y t p 1 q GLYPH<1> y t p 0 q GLYPH<16> τ for all t P r T s by assumption. Because the outcome sequence satisfies Assumption 1, the expected outcome regret is at most E r R outcome T s / β GLYPH<4> T q for some constant β . By the above, this implies that the expectation of the sum of probabilities 1 GLYPH<1> p t must be small,

Next, we show that A must incur a large cost with respect to the functions f t in the definition of Neyman regret. To do this, we will use a weighted version of the AM-HM inequality which states that for x 1 . . . x T ¡ 0 and w 1 . . . w n ¥ 0 , we have that

The usual AM-HM inequality is recovered when w t GLYPH<16> 1 { T . We now bound the expected Neyman loss, observing that

where the last two inequalities follow from moment bounds in Assumption 1 together with the assumptions on the outcome sequence stated in the Theorem.

Next, we show that the optimal Neyman design incurs a much smaller cost. In particular,

Together, these facts establish that the Neyman regret for A is lower bounded as

F Ethical Considerations

There are-at least-two objectives when constructing an adaptive treatment allocation.

Minimizing Cumulative Regret : give the "best" treatment to as many people as possible.

Minimizing Variance of the Effect Estimate : estimate the effect of the treatment to as high precision as possible.

As we show in Proposition 6.2, these two objective are fundamentally incompatible: an adaptive design which aims to estimate the effect to high precision must assign treatment which has worse outcomes. Likewise, a design which seeks to maximize the utility of assigned treatments will not be able to reliably estimate causal effects to high precision. Which one of these is more ethically desirable depends on the purpose and the context of the experiment. For guidance on this ethical question, we turn to The Belmont Report.

In 1979, the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research released the "Belmont Report", which has been one of the foundational texts for ethical guidance in research conducted with human subjects [for the Protection of Human Subjects of Biomedical and Research, 1978]. One of the three basic ethical principles laid out in the report is 'benevolence' which is understood as an the obligation the researcher has for the research to improve the general well-being of society, including those people in the experiment. The report writes:

The Hippocratic maxim 'do no harm' has long been a fundamental principle of medical ethics. Claude Bernard extended it to the realm of research, saying that one should not injure one person regardless of the benefits that might come to others. However, even avoiding harm requires learning what is harmful; and, in the process of obtaining this information, persons may be exposed to risk of harm. Further, the Hippocratic Oath requires physicians to benefit their patients 'according to their best judgment.' Learning what will in fact benefit may require exposing persons to risk. The problem posed by these imperatives is to decide when it is justifiable to seek certain benefits despite the risks involved, and when the benefits should be foregone because of the risks.

From this perspective, it may be ethically advisable to use a variance minimizing adaptive design because it allows the researcher to learn the effect while subjecting fewer human subjects to the experimental treatments. In other words, a variance minimizing design allows researchers to learn what is harmful and what is beneficial while subjecting fewer human subjects to possible harm. An adaptive treatment plan which minimizes cumulative regret will ensure that minimal harm is done to subjects in the experiment, but will offer less certainty about the extent of the benefit or harm of the treatments. Such an approach will lead to less informative generalizable knowledge of treatment effects, possibly defeating the goal of the research study.

That being said, it is not our goal to suggest that one adaptive allocation plan is most ethical in all circumstances. Indeed, these ethical questions have no systematic answers which are generally applicable. It is the burden of the researchers to carefully 'decide when it is justifiable to seek certain benefits despite the risks involved, and when the benefits should be foregone because of the risks.' Our goal in this work is merely to provide improved statistical methodology which affords the researchers more choices when addressing these ethical questions.

Peer reviews:
comments: SUMMARY:
This paper studies adaptive Neymann allocation and proposed an algorithm that achieves expected Neymann regret $\tilde{O}(\sqrt{T})$. I am not an expert in sequential experiment design so my ability is limited to assess the impact/relevance of this paper. Yet, I do consider myself well-versed in the potential-outcomes framework.

STRENGTHS:
- I found the writing and analyses appear to be rigorous and precise.
- The proposed algorithm achieves the Neymann variance in the synthetic experiment.

WEAKNESSES:
See "Questions" for more details.

QUESTIONS:
1. I understand the authors may wish to provide sufficient background, but it is a bit hard to tell what are existing (and well-known) results and what are new results (except starting from page 6 when the authors introduce the algorithm).
2. I felt Assumption 1 is a bit tricky to decipher: what is the motivation to have the same constant $c$? What are the scenarios where this assumption may break?
3. Have you considered combining CLIP-OGD with an outcome model to get a doubly-robust estimator?
4. Can you add more simulations?

LIMITATIONS:
Not applicable
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study adaptive allocation of samples into control and treatment groups in an experiment. They (i) define new performance measures for such allocations, namely Neyman ratio and Neyman regret, (ii) introduce a new algorithm called Clip-OGD that achieves optimal Neyman regret, and (iii) provide asymptotically correct confidence intervals for experiments run with Clip-OGD.\n\nSTRENGTHS:\nPresentation-wise, this is an extremely well-written paper. In particular,\n\n(i) The metrics that are proposed, Neyman ratio and Neyman regret, are well motivated. It is convincing why minimizing the Neyman ratio is desirable and how minimizing Neyman regret is a surrogate for that objective.\n\n(ii) Clip-OGD is a simple and intuitive algorithm. Notably, it does not have hyper-parameters that require tuning (although they could potentially be tuned). This is a major strength for an online algorithm where cross-validation using an offline dataset would not be possible.\n\nWEAKNESSES:\nThere are only minor weaknesses (please see my questions below).\n\nQUESTIONS:\n1) In the paragraph starting from line 226, Neyman regret is discussed from the perspective of multi-armed bandits. I understand how explore-exploit algorithms like UCB are not suitable for minimizing the Neyman regret; the purpose of adaptive Neyman allocation seems to be purely exploratory. Then, how does it relate to existing pure-exploration bandits?\n\n2) Assumption 2 is not intuitive and is not explained well enough. What does \"ruling out settings where the outcomes were chosen with knowledge of Clip-OGD\" mean exactly? I understand the formulation assumes $y_0(t)$ and $y_1(t)$ are deterministic, but just to understand Assumption 2, what would its equivalent have been under a super-population assumption (i.e. if $Y_0(t)$ and $Y_1(t)$ were to be random and i.i.d.)?\n\n3) Confidence intervals provided in Section 5 seem to be only asymptotically correct. Does this mean that no finite-sample guarantees are possible regarding the error rate of these intervals? This would limit the use of Clip-OGD in settings where strict error control is required (such as clinical trials).\n\n4) Why do the experiments not include a multi-armed bandit baseline although they are mentioned as an alternative design in Section 6?\n\n5) What does \"informal\" mean in Proposition 6.1? If it does not have a formal proof, maybe it should be stated as a remark rather than a proposition.\n\nLIMITATIONS:\nYes, the authors adequately addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work considers adaptive experimental design for sequential experiments. To do so, a new regret-like measure, called Neyman regret is defined that compares the ratio of the variance under the chosen experiment design with respect to the variance under the optimal experiment design. Drawing connections to online convex optimization literature, an algorithm is developed that provides $\\tilde O(\\sqrt(T))$ Neyman regret. In contrast, it is also established that two-stage explore then commit or multi-arm bandit setups may not result in (super)linear Neyman regret. Additionally, asymptotically valid confidence intervals using the adaptively collected data are provided.\n\nSTRENGTHS:\nS1. An important topic of sequential experimental design, particularly setting up the framework from the point of regret minimization.\n\nS2. The core idea is well-presented \n\nS3. Comparison of Neyman regret with alternative designs is also valuable.\n\nWEAKNESSES:\nW1. Numerical simulations that analyzed more aspects of the algorithm could have made the paper stronger.\n\nQUESTIONS:\nA.  For the gradient term in the algorithm, it might be beneficial to elaborate on that, i.e., from variance eqn to eqn for estimating that variance and taking the derivative to get the cubic terms.\n\nB. I see the need for the projection term in the algorithm, but it is not clear how to actually set it in practice. \n\nC. I am curious if the authors can elaborate on the choice of the utility for gradient estimation. Instead of minimizing the utility with just the new sample, why not do a Follow-the-leader style algorithm to minimize the utility over all the samples seen so far? Maybe even FTRL, with a regularizer that prevents the distribution from shifting too far from Bernoulli(0.5) could replace the projection step?\n\nD. I think the discussion around non-superefficient variance is important and I while I see the need of it, I do not quite understand it properly. Having more discussion about that could be beneficial.\n\nE. What is the practical relevance of the experiment setup for Fig 1(b)?\n\nLIMITATIONS:\nF. To understand the sensitivity of the projection hyper-parameter, can ablations for it be provided in Fig 1?", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of “Adaptive Neyman Allocation”, which involves designing an efficient, adaptive experimental design. Neyman allocation is an infeasible experimental design which would be optimal (minimum variance) if the planner knew all the exact potential outcomes under different treatments. However, this is infeasible, and so the goal considered in the paper is to build an adaptive experimental design which is nearly as efficient (in terms of variance) as the infeasible non-adaptive Neyman allocation asymptotically.\nTo measure the performance, the first contribution of the paper is to propose new measures of regret (and regret ratio) similar to the notion of regret in bandits/statistical learning. Second, the paper proposes an adaptive design based on the idea of adaptive gradience descent to adjust the treatment probabilities to minimize regret. The paper shows that the regret in this approach scales as O(sqrt(T)). Finally the paper constructs confidence intervals which guarantee asymptotic coverage of the average treatment effect.\n\nSTRENGTHS:\n– Novelty: The paper claims to be the first to introduce the notion of Neyman regret in the context of adaptive experimental designs. I am not fully aware of the related literature, but this seems to be an interesting contribution to analyze from this perspective. \n\n– Well-written: The paper is well organized overall and the concepts are introduced and explained crisply.  \n\n– Theory: The paper presents results well-grounded in theory.\n\nWEAKNESSES:\n1. Related Work: The related work section mainly talks about  Neyman allocation and about casual inference under adaptively collected data. However, it seems there is also a large body of work that studies adaptive experimentation/ adaptive design for randomized trials. All of these references seem to be missing (please see few examples below and references therein)? It would be good to distinguish this paper from this body of related work. \n\n-- Eggenberger, Florian, and George Pólya. \"Über die statistik verketteter vorgänge.\" ZAMM‐Journal of Applied Mathematics and Mechanics/Zeitschrift für Angewandte Mathematik und Mechanik 3, no. 4 (1923): 279-289\n\n-- Xu, Yanxun, Lorenzo Trippa, Peter Müller, and Yuan Ji. \"Subgroup-based adaptive (SUBA) designs for multi-arm biomarker trials.\" Statistics in Biosciences 8 (2016): 159-180.\n\n-- Eisele, Jeffrey R. \"The doubly adaptive biased coin design for sequential clinical trials.\" Journal of Statistical Planning and Inference 38, no. 2 (1994): 249-261.\n\n-- Hu, Feifang, and William F. Rosenberger. \"Optimality, variability, power: evaluating response-adaptive randomization procedures for treatment comparisons.\" Journal of the American Statistical Association 98, no. 463 (2003): 671-678.\n\n2. Unsurprising: To me it is a little unsurprising/ unimpressive that the approach can achieve the optimal data efficiency as T tend to infinity (after a really large number of samples). The authors also seem to acknowledge this in part in the final section of the paper. \n\n3. Empirical Evaluation: It may be useful to add more interesting baselines if available in comparison of the regret\n\nQUESTIONS:\n– In section 4.1 why isn’t the variance of adaptive experimental design, V also a function of t?\n\n– There seem to be a few other approaches that study adaptive randomziation in clinical trials. For example: Zhang, Lanju, and William F. Rosenberger. \"Response‐adaptive randomization for clinical trials with continuous outcomes.\" Biometrics 62.2 (2006): 562-569. \nHow does the proposed approach compare against existing methods?\n\n– Ethical concern: Please see additional question under Limitations section.\n\nLIMITATIONS:\nCould there be fairness/social welfare concerns stemming from such optimal designs? Perhaps the allocation algorithm may assign a higher probability to a less effective or detrimental treatment simply because the variance in its outcomes is higher. As a result, in the quest for minimized variance, could the negative treatment be administered more often than advisable/necessary?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-12"}

comments: SUMMARY:
The paper proposes a new adaptative Neyman allocation for experimental design. The proposed adaptative design gets close to the optimal non-adaptative strategy without suffering from the same infeasibilities.

STRENGTHS:
1) The writing and structure of the paper are OK.
2) The topic is very relevant, and the paper proposes adaptative variance with formal guarantees ensuring feasibility is interesting.

WEAKNESSES:
1) The paper is not easy to follow for the non-expert, both concerning the notations used and the derivations.
2) It seems that the motivation for this work could be broader. It is unclear to the reviewer why the authors focused on medical applications.
3) The method is only applied to a single microeconomic example.

QUESTIONS:
1) I would like to see some discussion about the medical applications vs a more broader motivation.

LIMITATIONS:
Limitations are adequately addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-27