Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect
Authors: Ojash Neopane
Aaditya Ramdas
Aarti Singh
Venue: ICML 2025 poster (2025)
Keywords: Multi-Armed Bandits
Causal Inference
Average Treatment Effect
Primary area: theory->active_learning_and_interactive_learning
TL;DR: We design and analyze new algorithms for adaptive estimation of the average treatment effect estimation.

Abstract: Estimation and inference for the Average Treatment Effect (ATE) is a cornerstone of causal inference and often serves as the foundation for developing procedures for more complicated settings. 
Although traditionally analyzed in a batch setting, recent advances in martingale theory have paved the way for adaptive methods that can enhance the power of downstream inference. 
Despite these advances, progress in understanding and developing adaptive algorithms remains in its early stages. 
Existing work either focus on asymptotic analyses that overlook exploration-exploitation trade-offs relevant in finite-sample regimes or rely on simpler but suboptimal estimators.
In this work, we address these limitations by studying adaptive sampling procedures that take advantage of the asymptotically optimal Augmented Inverse Probability Weighting (AIPW) estimator. 
Our analysis uncovers challenges obscured by asymptotic approaches and introduces a novel algorithmic design principle reminiscent of optimism in multi-armed bandits. 
This principled approach enables our algorithm to achieve significant theoretical and empirical gains compared to previous methods. 
Our findings mark a step forward in the advancement of adaptive causal inference methods in theory and practice.

Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect
Ojash Neopane
Aaditya Ramdas
12
Aarti Singh
Abstract
However, our understanding of the challenges associated with adaptive ATE estimation remains limited.
Estimation and inference for the Average Treat- ment Effect (ATE) is cornerstone of causal infer- ence and often serves as the foundation for devel- oping procedures for more complicated settings. Although traditionally analyzed in a batch setting, recent advances in martingale theory have paved the way for adaptive methods that can enhance the power of downstream inference. Despite these advances, progress in understanding and develop- ing adaptive algorithms remains in its early stages. Existing work either focus on asymptotic analy- ses that overlook exploration-expi expllottttioooiiiionnnnnno trade- offs relevant in finite-sample regimes or rely on simpler but suboptimal estimators. In this work. address these limitations by studying adaptive sampling procedures that take advantage of the asymptotically optimal Augmented Inverse Prob- ability Weighting (AIPW) estimator. Our analy- sis uncovers challenges obscured by asymptotic approaches and introduces a novel algorithmic design principle reminiscent of optimism multi- armed bandits. This principled approach enables our algorithm achieve significant theoretical and empirical gains compared to previous meth- ods. Our findings mark a step forward in the ad- vancement of adaptive causal inference methods in theory and practice.
Classically, adaptive ATE estimation has been analyzed in an asymptotic setting, where past work has focused on designing adaptive sampling procedures that ensure that the resulting ATE estimate achieves the smallest possible asymptotic variance, that is, the semiparametric efficiency bound. More recently, there has been growing interest in developing algorithms that provide nonasymptotic perfor- mance guarantees. However, these works suffer from certain drawbacks that lead to poor finite sample performance, an issue that we discuss in detail in Sections and 4.1.
In this work, we take nonasymptotic perspective on adap- tive ATE estimation, focusing on the Augmented Inverse Propensity Weighting (AIPW) estimator. Our finite-sample analysis reveals key aspects of algorithmic design that prior work has overlooked. This enables us to propose a new algo- rithm with substantially improved theoretical and empirical performance while also simplifying the analysis.
At the heart of our approach is the insight that initially over- sampling arms that should eventually be under- under-sampled according to the (unknown) optimal allocation can lead to better estimates of the ATE. Interestingly, this idea can be interpreted as an instance of the principle of optimism, a well-established algorithmic design paradigm in the litera- ture on regret minimization in multi-armes bandits (MAB) and reinforcement learning. We discuss this connection in more detail in Section 4.
1. Introduction
Contributions. Our main contributions are as follows:
The problem of estimating the average treatment effect (ATE) is central to causal inference and has been extensively studied. We have precise understanding of the difficulty of this problem in both asymptotic and nonasymptotic regimes.
We develop and analyze new algorithm, Optimistic Policy Tracking (OPTrack), for the adaptive estima- tion of ATE that enjoys significant theoretical improve- ments over previous approaches along with a signifi- cantly simplified analysis.
'Machine Learning Department. Carnegie Mellon Uni- versity *Department of Statistics & Data Science, Carnegie Mellon University. Correspondence to: Ojash Neopane
2.
We perform simulations that demonstrate that our theo- retical improvements translate into empirical improve- ments, especially in the small sample regime, which is critical for applications such as randomized clinical trials.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Optimistic Algorithms for Adaptive ATE Estimation
Organization. The remainder of this paper is structured as follows. In Section 2, we review previous and related work. Section 3 introduces our problem setup, establishing the necessary framework for our contributions. In Section 4, we identify key limitations of existing approaches and present our proposed TrTrack algorithm. Section contains our main theoretical results, providing a rigorous performance characterization of OPTrack. Finally, in Section 6, we empirically validate our method, demonstrating its supe- rior performance compared to existing approaches and its comperitiveness oracle baseline.
of their work. In order address these questions, Dai et al. (2023) takes initial step toward understanding the nonasymptotic difficulty by introducing the c1 ClipOGD al- gorithm for the fixed-design setting. They introduce and analyze the Neyman regret (in the design-based setting), which is a normalized proxy to the variance of the resulting ATE estimate. Even more recently, Neopane et al. (2025) proposed and analyzed the ClipSMT algorithm for the su- perpopulation setting and shows that it enjoys an improved log T bound on the Neyman regret.
surpassing-eve an infeasible
Although these two works take important first steps toward understanding the nonasymptotic difficulty of adaptive ATE estimation, their algorithms rely on the IPW estimator which is known to be suboptimal. In fact, these works define the Neyman regret with respect to the minimum variance IPW estimator, where the minimization is performed over all pos- sible allocations. In contrast, our definition of the Neyman regret is much stronger as the baseline against which we compete is defined as the minimum attainable variance over all pairs of estimators and allocations. Notably, using this stronger definition of regret, the aforementioned approaches obtain linear Neyman regret, where as we are able to design algorithm which obtains logarithmic Neyman regret.
2. Prior and Related Works
Adaptive experimental design has long and distinguished history, dating back to the seminal work of Neyman (1934) on optimal allocation in experimental studies. Thompson (1933) introduced Bayesian adaptive design, thus laying the foundation for the MAB problem. Thompson's 's approach of sequential updating beliefs about treatments (or arms) based on observed outcomes is now central in MAB research (Lattimore & Szepesvári, 2020). However, many problem formulations focus on maximizing cumulative rewards over repeated rounds of exploration-eexlloiatio.. In contrast, our objective of ATE estimation differs from the typical MAB focus and raises different forms of exploration trade-offs.
2.2. Related Works
The problem of off-policy evaluation, which generalizes ATE estimation, has been extensively studied in the litera- ture on reinforcement learning (Dudík al., 201 I; Li et al., 201 1; Jiang & Li, 2016). Most of the research in this area has focused on offline estimation, leading to precise charac- terizations of minimax lower bounds along with matching upper bounds (Li et al., 2015; Wang et al., 2017; Duan et al., 2020; Ma et al., 2021). Beyond policy evaluation, these methods have been extended to estimate other quanti- ties, such as the cumulative distribution function of rewards (Huang et al., 2021; 2022). However, there has been limited exploration of adaptive versions of these methods. Some existing work includes Hanna et al. (2017), which focuses on off off-policy learning, and Konyushova et al. (2021), which integrates offline off-policy evaluation techniques with on- line data acquisition to enhance sample efficiency in policy selection. However, these works are primarily empirical.
2.1. Prior Work
Our work builds on recent line of work investigating adap- tive algorithms aimed at efficiently estimating ATE. Hahn et al. (2009) sparked this research direction by proposing a two-stage design, conceptually similar to the Explore- then-Commit algorithms MAB (Garivier et al., 2016) and showing that it asymptotically attains the minimun-varianes semiparametric efficiency bound. Subsequently, Kato al. (2020) introduced fully adaptive procedure using the adap- tive AIPW estimator (PW). and showed that it is asymp- totically optimal (in the above sense) while also providing improved empirical performance compared to the less adap- tive two-stage design. Later, Cook et al. (2024) proposed an alternative method called Clipped Deviatiin Tracking (ClipSDT), which inherits the same asymptotic optimality under milder assumptions, admits modern uncer- tainty quantification tools (Waudby-Smithh et al., 2022), and outperforms the earlier approach empirically. In parallel work, Li et al. (2024) significantly generalized the two-stage design in Hahn et al. (2009), extending its applicability to a broad spectrum of problems, including Markovian and non-Markovian decision processes.
A related area of research concerns inference procedures for adaptively collected data. This can be categorized into asymptotic and non-asymptotic approaches. On the asymp- totic side, one direction has focused on reweighting esti- mators and establishing their asymptotic normality (Hadad et al., 2021: Zhang et al., 2020; 2021). Another direction avoids asymptotics, instead leveraging modern advances in martingale theory to derive nonasymptotic confidence inter- vals and sequences for adaptively collected data, including estimates of the ATE (Howard et al., 2021; Waudby- -mith
Despite these advances, all of the above approaches focus on characterizing the asymptotic behavior of their approaches, leaving open questions about finite-sample performance
2

---

Optimistic Algorithms for Adaptive ATE Estimation
& Ramdas, 2023; Waudby-Smith et al., 2022).
the treatment allocation probability ny as well the estimate hy- A natural choice for he is the AIPW estimator, which given some reward estimate P, is defined as
Subsequent work. After our initial arXiv posting, (Noarov et al., 2025) presented an extension of the ClipOGD algorithm to the fixed design setting, achiev- ing logarithmic Neyman regret with respect to the optimal IPW estimator- - which would imply linear regret against the stronger baseline we consider in this paper. Furthermore, this improved regret rate is notable, requires substantially more restrictive assumptions than our analysis, including conditions that preclude standard stochastic reward models such as Bernoulli outcomes. These limitations reflect the inherent difficulty of obtaining logarithmic rates in their more challenging fixed design framework.
h1 9 (Ac) Paig,u (At)
(Re - - + A
(I)
where 9 (Ar) = I[A, = 1]- = 0] and A ( = ((1) F (0). However, this estimator isn't well suited to sequential estimation, motivating Kato et al. (2020) to propose the Adaptive AIPW (A2IPW) estimator. Specifically, letting nt denote any Fi-1 measurable function (i.e. a predictable reward estimate), they defined
h4 [[4 1] - I[A, 0] ( (2) Te (A1)
3. Background
Problem Setup We are interested in adaptive estima- tion of the average treatment effect. During each round, t, Alg uses the history of past observations H--1 = s=1 to select the probability of treatment al- location TT. Then, ,., is used to assign the next experimental unit to either the control (Ae = 0) or the treatment (A, = 1) by sampling A, ~ Bernoul 1 (mz). Finally, after assign- ing the experimental unit, we observe the outcome Re which marks the end of round L.
The A2IPW and AIPW estimators differ in two critical ways. First, the A2IPW estimator constructs propensity scores using the policy Ty rather than the marginal probabil- ities P.xig.r (At), avoiding the computational infeasibility of computing marginal probabilities for adaptive sampling algorithms. Second, the A2IPW incorporates predictable rewards Fz, enabling sequential updates to reward estimates and fully adaptive algorithm design without phase- based approaches. Beyond these computational advantages, the A2 IPW estimator possesses strong theoretical properties: it achieves asymptotic optimality necessary for sublinear Ney- man regret, and recent developments have produced tight confidence sequences for this estimator, facilitating sequen- tial testing and uncertainty quantification. These properties make the A2IPM estimator a natural choice for adaptive ATE estimation.
((TT, As, R=)
We formalize the above interaction protocol as follows. Let F. = σ (He) denote the filtration generated by the past observations. An algorithm Alg {(T,, ht)) is defined as sequence of Fi-1 measurable random elements where Te E [0, 1] is the treatment allocation probability and h4 : (m,, Ar, Ra) + R20 which can be thought of as the ATE estimate produced by Alg on round t.
We assume that the rewards are generated as R TI T[A, = Rg (1) + T[4, = 0) Re (0), where Rg (a) are called the potential outcomes. We assume that the se- quence of potential outcomes are jointly distributed accord- ing to some probability measure v (the "environment") that satisfies the following assumptions. The first assumption that the rewards are unconfounded, which means that, given F,-1, the potential outcomes R2 (1), Re (0) are con- ditionally independent of the treatment assignment A,, i.c R2 (1), R₂ (0) 1 A₁ Fi-1- The second assumption is that the reward means and variances are conditionally fixed
Neyman Allocation and Regret We use the mean squared error (MSE) to measure the quality of the estimates pro- duced by our algorithm. However, by itself, the MSE is difficult to interpret because does not consider the inherent difficulty of the problem. Therefore, we would like nor- malize this error with respect to some problem-dependent baseline which we now define and motivate. Hahn et al. (2009) show that for any fixed allocation, π, the minimum attainable MSE of any estimator is
a² (1)
(0) 1-T
(3)
so that for all f, we have Ey [Re (a) Fr-1]] = r" (a) and
Vv [Re (a)
[==1]] = o² (a).
The Neyman allocation '² is defined as the allocation which minimizes the above variance and simple calculation shows that
Our objective within this framework is to estimate the ATE A, which is defined as
o (1) a (0)+
r (1)
(4)
The APW Estimator. An algorithm for adaptive ATE estimation thus requires us to specify a method to compute
Ideally, we would like to design an algorithm whose vari- ance close to this baseline and in order to understand the
3

---

Optimistic Algorithms for Adaptive ATE Estimation
rate which this occurs, we consider the Neyman regret which is defined
4.1. Preliminaries
The difficulties of adaptive ATE estimation. The pri- mary difficulty of adaptive ATE estimation is in balancing the exploration-cxploittiio trade-off that arises from adap- tive allocation. If we condition on Fi-1 some algebra shows (see Lemma C.1) that the variance of the A2 IPW estimator is
Ry - --MEE
(MSE (अx)
MSE*)
(5)
where MSE" is the mean square error attained by an oracle algorithm which sets मe = x² for all & E [T....,T}. The Neyman regret is simply the difference in the normalized MSE between the optimal variance and the MSE of the estimate produced by Al g- This normalization guarantees that the MSE converges if Alg has sublinear regret, then we are guaranteed that its MSE converges to the optimal MSE.
(a) Σ 0² (a) 1- Te (a) T.. (a)
(r" (a) - Fr (a))
(8)
constant (rather than 0), so that
which is minimized by setting (I, r) (x* ,r) where x² is the Neyman allocation introduced in Section 3. Since 'T" and r* are not known a priori, we need to design an algorithm to adaptively estimate them. However, this is challenging because optimizing the exploration allocation separately for estimating x and r" (each requiring different allocation) results in procedure with high Neyman regret. As such, (6) designing an algorithm to adaptively balance the exploration of xt and r* while simultaneously minimizing the Neyman regret becomes a very delicate task.
Using the fact that the A2IPW is unbiased, along with the fact that n7 and F are predictable, we can express the Ney- man regret as
Rx Σ Exigy [((Te,, fr]l ((TT r*),
where
o² (a)
1 1- n (a)
((T,r)=
Σ
+
E, (a)
(7)
Insights into improvements. In order to better understand the improvements that can be made, we investigate previous approaches for balancing this trade-off. To simplify the exposition, in this section we assume that x² I 3.- The primary approach that past works (both asymptotic and nonasymptotic) have utilized is clipping the allocation. In fact, the algorithms proposed by Cook et al. (2024), Dai et al. (2023), Neopane et al. (2025) all utilize a clipping approach which computes the empirical allocation
(E(u.1) . (a)
π (a)
is the Neyman loss and Et (a) = r* fre (a) is the reward estimation error.
Notation. In what follows, we will let
Ni(a) - EI[A, a]
a=1
o (1)
denote the number of times the action a is selected at the end of round 1,
oz (0)+ (1)
and plays a clipped version of this estimate
a) 1 R.I al
Me (a)
Ne(a)
Te = min {1 - C, max (ce, #1
denote the empirical mean after t rounds, and
for some carefully chosen clipping sequence e satisfying c + O. However, these clipping approaches have some important limitations.
1 Σ (R.I = al - uu (A))² of (a) (a)
The first limitation is that a clipping approach cannot be fully adaptive to the underlying problem instance because the clipping sequence must be chosen priori. As such, past works choose o in order to optimize the performance of their algorithm in a worst-case sense, leading to suboptimal Neyman regret for easy problem instances. As an example, Neopane et al. (2025) show that setting o = - is optimal when we are not willing to bound x= away from 0 and 1. However, many practical problems are typically much easier than the worst case, and so we would like a procedure which is able to adapt to the underlying problem instance more appropriately.
denote the emprical variance. We use (-) to denote asymp- totic equivalence up to doubly logarithmic factors.
4. The Optimistic Policy Tracking Algorithm
In this section, we introduce our Optimistic Policy Tracking (OPT) algorithm. We begin with discussion of the diffi- culties of adaptive ATE estimation and the suboptimality of existing approaches. Next, we introduce our algorithm and provide insight into why it resolves the issues of existing approaches. Finally, we conclude with a brief discussion of the algorithmic design principles underlying our algorithm and their relation to ideas in the literature.
The second, more pressing issue is that clipping approaches lead to algorithms which under-exploit, which is caused by
4

---

Optimistic Algorithms for Adaptive ATE Estimation
log log (+-log
whose with scales like O
Using these
confidence sequences on a (a), we can construct confi- dence sequence for the Neyman allocation follows
Li(a (1))
CS.(=*)
Ui(a (0)) + Li(a (1))
(10)
Ui(a (1))
L (0)) + 24(0 (1))
+6)
The full algorithm is provided in Algorithm 1.
+
Algorithm 1 Optimistic Policy Tracking (OPTrack)
1: for / == 2, do
Figure 1. A plot of r*) where < Note how the Neyman loss is smaller for x² + This is due to the fact that x² + is closer to 1. highlighting how less exploratoratory allocations incur 2. larger Neyman regret.
2: Compute CS((T)) according to equation (10)
3: Set कe = argmin (m*)
4: Sample A, ~ Bernoulli(Ta)
Observe R2 ~ v(A))
Compute he according equation (2)
the asymmetry of the Neyman loss. To demonstrate this issue, in Figure 1, we plot the Neyman loss f(x,r") r*) for a problem with x* < 1 In this figure, we consider the Neyman loss at the points 4 = n² + c and _ = n² c. is easy to see that ((T4, r*) < f(TT r,),, and in fact this problem only worsens as π + {0, 13. Practically, the implication is that algorithms which under-sampled the arm with smaller probability according the Neyman allocation must necessarily pay higher price than the same algorithm which over-sampled the same arm by the same amount. This not merely a theoretical issue we see in our experiments that while clipping-basee based approaches pro- duce allocations which are closer to the Neyman allocation, they still have significantly worse empirical performance.
end for
Interpretation as Optimism. We can interpret our algo- rithm as implementing the celebrated principle of optimism in the face of uncertainty. Optimism is an algorithmic de- sign principle which the basis of many well-known MAB and reinforcement learning algorithms (such the "upper confidence bound"). Roughly speaking, the principle states that we should act as if the underlying problem instance is the easiest instance, which is feasible according to our past observations. In the regret minimization framework, this means playing the arm which has the largest upper confi- dence bound, For adaptive ATE estimation, this invol playing the allocation that is closest to 1.2 This is because the difficulty of problem is determined by the deviation of the Neyman allocation from when the Neyman allocation is close to 1. the objectives of exploration and exploitation are aligned. Suppose the Neyman allocation deviates from } then as the allocation we play converges to the Neyman allo- cation, we are necessarily sader-sampinn one arm and thus slowing down This intuition is supported by the results of Neopane et al. (2025) and Dai et al. (2023) who show that the Neyman re- gret scales inversely with | bl. Therefore, implementing optimism for adaptive ATE estimation involves playing the most feasible allocation (as determined by our past obser- vations) closest to this exactly the driving principle behind OPTract ck algorithm.
4.2. Optimistic Policy Tracking
Main Algorithm. Our proposed algorithm, OPT, de- signed to address these aforementioned issues. Indeed, as we will see, not only does OPT better adapt to the underly- ing problem instances, it also better handles the exploration- exploitation trade-off when compared to prior works. The algorithm itself if simple and plays the allocation
convergence to the Neyman allocation.
argmin (T*) 2 n
where (TT" is confidence sequence for the Neyman allocation. For reward estimation, we simply use the sample mean Pr (a) |,- E'=1 R, [A, a).
The main difficulty now is in constructing the confidence sequence CS.(TT). In order to do so, we first construct confidence sequences for the standard deviations of each arm. This is accomplished in Lemma B.I, which constructs a confidence sequence (a (a)) = [Le(a (4)) (a))]
5. Results
In this section, we build our intuition on the behavior of OPTrack and conclude by stating our main result which is bound on the Neyman regret of OPTrack.
5

---

Optimistic Algorithms for Adaptive ATE Estimation
Before begin, we introduce some additional notation which will make our exposition easier. For any и, we define
Lemma 5.2. With probability least 8, we have that
A (T) = 2 π| and min {T, 1 - π). Additionally, we let A (o) = - - (0).
= log 1 π (0) + (1)+ ) )
(14)
Our analysis begins with a Concentration result which demonstrates that the standard deviations concentrate at a over square root of rate. Specifically, in Lemma B.1, we demonstrate that with probability least 6, for all t 2 2we have that
The reason for the appearance of x² is due to the conver- gence of Te based on the number of times that both arms have been played. If we play one arm too often, then the width of the confidence interval for x would depend en- tirely on the width of the lesser sampled arm.
l0- - 1o-rrl 201300 log log / + log 6-1 (11)
t
Our main result combines the above lemmas to provide bound on the Neyman regret.
This result follows through generalizing similar result from (Audibert et al., 2006) utilizing modern martingale arguments (Howard et al., 2021). This enables us to obtain time uniform concentration which is valid in our setting with adaptively collected data.
Theorem 5.3. With probability least - 8, the Neyman regret of OPTrack is upper-bounded
5 A (0)
+
1 π
log logT
(15)
To proceed, we split the behavior of SPTrack into two phases, an exploration exploration phase and the concentra- tion phase. We define the exploration phase as the rounds for which we expect that each arm has been played sufficiently few times so that 1/2 E (T)( and the exploration time is the length of this phase. Intuitively, during this phase, there is not enough information in our observations to reliably predict x* and so our best choice is to explore each arm uni- formly. Fortunately, the length of this phase is not too long, and our first result bounds the length of this phase in terms of the absolute distance between the standard deviations.
The first term above is the per-round Neyman regret during the exploration phase and our bound follows from the fact that the Neyman regret is at most 4 when we play n, = 2. The second term in our bound the Neyman regret during the concentration phase and follows from the application of Lemma 5.2 in conjunction with Lemma 2 of (Neopane et al., 2025) showing that the Neyman regret scales according to |T* - Te| 1 RJ ." Since the contribution to the Neyman re- gret from the reward estimation also scales like 1 x² taking sum over these two terms gives us the desired result.
2- During the early stages of interaction,
In order get a better understanding of our result, we consider the behavior of hypothetical algorithm which plays the optimal Neyman allocation x² but incurs loss (12) based on the empirically computed allocation, xe. A sim- ple calculation shows that n, converges to x at a rate of θ (T 1) -)) This in turn implies that the Neyman regret would be
Lemma 5.1. Define the exploration time as
T= min 1: *
Then, with probability at least 1 - 8, we have
T= ( (o) log )
(13)
log T
(16)
The proof of this result is given in Appendix A.2. This result shows that OPTOT Track is able to adapt to the difficulty of the underlying problem instance - if the gap between the standard deviations is large, then the exploration phase will be short, and if the gap is small, then the exploration phase will be longer.
which, modulo the regret from the clipping phase, is the same as the Neyman regret incurred by OPTrack. This suggests that our algorithm is correctly adapting to the diffi- culty of the problem.
Comparison with ClipSMT. At first glance, our result appears be quite similar the Neyman regret bound from (Neopane al., 2025) who similarly show logarith- mie bound on the Neyman regret. However, this is not the case, due to differing definitions of the Neyman regret. In (Neopane et al., 2025), the Neyman regret is defined with respect to the minimum variance over allocations for the fixed IPW estimator. Our Neyman regret is defined with
Once the exploration phase is over, the algorithm will be able to focus on the concentration phase. In this phase, optimism guarantees A (Tτ)) < A (r²*) Therefore, we can control the number of times each arm is played which we can in turn convert to bounds on ( x=
Our next result formalizes this intuition.
6

---

Optimistic Algorithms for Adaptive ATE Estimation
respect to the minimum attainable variance over any pair of estimators and allocations. This means that while our regret bounds share similar form, the performance of our algorithm is significantly better than the performance of the C1 i PSMT algorithm. Concretely, using our definition of the Neyman regret to characterize the performance of the ClipSMT algorithm (as well as the C1 pOOD algorithm), we see that these algorithms actually have linear Neyman regret since the variance of their policies cannot converge to the minimum attainable variance.
The performance of OPTrack is competitive with the re- ward estimation oracle for moderate values of x² and even outperforms the reward estimation oracle on some problem instances. This is because OP is more exploratory and obtains better reward estimates early on.
We additionally perform experiments on a macro-insurance intervention dataset (Groh & McKenzie, 2016) which in- vestigates the effects of macro-insurance in Egypt. The results from this experiment are given in Table 1. Our exper- iments qualitatively align with the results from our synthetic experiments, demonstrating that our proposed algorithm out- performs prior approaches, especially in the small sample regime.
6. Experiments
In this section, we present experiments' to evaluate the empirical performance of our algorithm. We compare oPTrack against the ClipSDT algorithm proposed by Cook et al. (2024), well as two oracle algorithms that fol- low the Neyman allocation. One of these oracle algorithms sequentially estimates the reward, while the other has access to the true reward. We evaluate these algoritmhs on range of synthetic simulations as well as on a macro- incrro-inuurance intervention dataset (Groh& & McKenzie, 2016).
Algorithm
100
300
500
1000
1500
ClipSDT
0.117
0.102
0.098
0.093
0.095
OPTrack
0,103
0.095
0.094
0.093
0.092
Est. Reward Oracle
0.10
0.097 0.094 0.093 0.094
Oracle
0.092
0.090 0.093 0.090 0.091
Table 1. Mean square error of the ClipSDT, OPTrack algo- rithms as well as two oracle baselines - an oracle which has knowledge of x", and an oractle with knowlege of n" and r" on a macro-insurance intervention dataset. Note that ClipSDT is competitive with the Oracle, even in the small sample regime.
We do not include results for the Cl 1SMT and Cl DPGD algorithms, as their variances fail to converge to the oracle variance, consistently leading to significantly worse perfor- mance than the other algorithms which obscures the clarity of the plots. This outcome is expected, given that both algorithms incur linear Neyman regret.
7. Conclusion
This work proposed a new algorithm for adaptive ATE esti- mation. We identified some key issues with past approaches which limited their performance both empirically and theo- retically and demonstrated how to resolve them. Our pro- posed solution borrows ideas from the literature on Regret Minimization and showed how to extend some of these ideas to the problem of adaptive ATE estimation. We be- lieve that these ideas will be crucial for developing adaptive (an0))algriith Jalgorithms for inference for more complicated settings as well as for related problems like Off-Policy Evaluation.
We consider problem instances where both arms fol- lows Bernoulii distributions. For each of these problem instances, we fix the treatment mean to be 1 and vary Ho E {0.05, 0.1, 0.2, 0.3, 0.4, 0.5} which in turn leads to problem instances with different Neyman allocations.
For each of these problems, we run OPTrack, ClipSDT, and the reward estimation oracle for T E {100, 200, 400, 500, 600, 700, 800, 900, 1000, 1500, 2000
and plot the normalized MSE (T MSE) over 500,000 simulations. For the oracle baseline, we explicitly compute the MSE The results of these simulations are given in Figure 2. We include error bars in Figure 2. though they are barely visible due to the large number of simulations.
7.1. Future Work
We believe there are few directions for future work that we find very compelling. The first is the extension of our algorithm to the setting with covariates and with more so- phisticated reward estimation. In the causal inference liter- ature, practitioners typically use nonparametric regression to estimate the r* and so extending our ideas to work with such estimators warrants more attention. Another interest- ing direction is the extension to multiple arms. Here we believe that the correct extension is to compute confidence interval around the Neyman allocation, and then project this set onto the Uniform distribution over the actions. The primary difficulty for this extension is in the analysis - if we apply our techniques directly, this will result in an additional
Our results show that oPTrack consistently outperforms ClipSDT over all problem instances. The difference be- tween the two becomes negligible for larger values of T which is expected since all algorithms eventually converge to the Neyman allocation and true reward function. How- ever, for smaller sample sizes, we see that OPTrack pro- vides around a 10-15 percent improvement over ClipSDT. This improvement is due to the reasons given in Section 4.
1 'Code for replicating experiments can be found at the fol- lowing GitHub repo: https:/ http::////te //gtttutbbb... con/ oneopane/ adapti ve- see-ate-esee wwttee-sttmttt.. imation.
7

---

Optimistic Algorithms for Adaptive ATE Estimation
Normalized MSE vs Rounds
Normalized MSE vs Rounds
CliSST
CipSDT OPTrack Cracle (True Rewrand)
OPTrack
(Estimated Reward)
Reward)
Normalized MSE (T x MSE)
1.15
Oracle (True Reward)
Normalizzd MSE (T x MSE)
1.10
1:10
1.05
1.05
1.00
1.00
200
300 500 600
800 9001000 Number of Rounds (T)
2000
100
300 400 600 700 9001000
1500
2000
Number of Rounds (T)
(a) μi = 0.5
(b) μá = 0.4
Normalized MSE vs Rounds
Normalized MSE Rounds
1.10
ClpSST
1.00
ClaSST
Normalized MSE (T X (MSE)
Oracle [Estimated Reward) Oracle (Thee Reward)
Normalized MSE (Txx MSE)
Oracle (Estimated Reward) Oracle (True Reward)
1.05
0.95
1.00
0.90
0.85
0.95
100 200 300 400 500 600 700 B00 9001000
1500
2000
100
200
300
400 500 600 700 800 9001000
1500
2000
Number Rounds (T)
Number of Rounds (T)
(c) μl 0.3
(d) Hoo = 0.2
Normalized MSE vs Rounds
Normalized MSE vs Rounds
ClecOT
ClieSOT
Oracle [Estimated Reward)
0.625
Dracie [Estimated Reward) Oracle (True Reward)
Normalized MSE xTx MSS))
Oracle [Thue Reward)
Normalized MSE (T MSE)
0.600
0.575
0.70
0.550
0.65
0.525
100 200
300
400 500 600 700 800 5001000
1500
2000
100
200
300
400 500 600 700 800 9001000
1500
2000
Number Rounds (T)
Number of Rounds (T)
(e) μi 0.1
(1) Ho 0.05
Figure 2. Normalized MSE (T- -SEE for OPTrack, ClipSCI, and the oracle baselines across six problem instances, each with Bernuuli rewards with μ = 1 and μo E (0.05, 0.1, 0.3, 0.4, 0.53. Results are averaged over 500,000 simulations. OP Trarkck consistently outperforms C ClipSDT, with 10-15 improvement for smaller T. Notably, OP OrTrack is competitive with the reward estimation oracle and even outperforms in some cases due to better exploration of the reward function early on. As increases, all algorithms converge to the oracle haseline.
factor of K in the term that is dependent on T, where K is the number of arms. It is an interesting question to see if our analysis can be improved to remove this additional factor.
Finally extending these ideas to more complicated interac- tion protocols such as Reinforcement Learning warrants further study.
8

---

Optimistic Algorithms for Adaptive ATE Estimation
Acknowledgments
Hanna, J. P., Thomas, P. S., Stone, P., and Niekum, S. Data- efficient policy evaluation through behavior policy search. In International Conference Machine Learning, 2017.
This work sponsored in part by AFRL Center of Ex- cellence grant FA86501925209 and NSF AI Institute for Societal Decision Making under grant IIS2229881.
Howard, S. R., Ramdas, A., McAuliffe, J. D., and Sekhon, J. Time-uniform, nonparametric. nonasymptotic confi- dence sequences. The Annals of Statistics, 2021.
Impact Statement
While our paper is primarily theoretical, we believe that the insights developed will be important for downstream applications such as causal inference which has broad appli- cations over variety of fields including clinical trials and A/B testing.
Huang, A., Leqi, L., Lipton, Z., and Azizzadenesheli, K. Off-policy risk assessment in contextual bandits. Ad- vances in Neural Information Processing Systems, 34: 23714-23726, 2021.
Huang, A., Leqi, I.., Lipton, 7., and Azizzadenesheli, K. Off Off-policy risk assessment for markov decision processes. International Conference on Artificial Intelligence and Statistics, pp. 5022-5050. PMLR, 2022.
References
Audibert, J.-Y., Munos, R., and Szepesvári, C. Use of variance estimation in the multi-armed bandit problem. 2006.
Jiang, N. and Li, L.. Doubly robust off-polics value evalua- tion for reinforcement learning. In International Confer- ence on Machine Leurning, pp. 652-661. PMLR, 2016.
Cook, T., Mishler, A., and Ramdas, A. Semiparametric effi- cient inference in adaptive experiments. In Proceedings of the Third Conference on Causal Learning and Rea- soning, volume 236 of Proceedings of Machine Learning Research, pp. 1033-1064. PMLR, 01-03 Apr 2024.
Kato, M., Ishihara, T., Honda, J., and Narita, Y. Efficient adaptive experimental design for average treatment effect estimation. arXiv preprint arXiv:2002.00 308,0. 2020.
Dai, J.. Gradu, P., and Harshaw, C. Olip-OGD: An ex- perimental design for adaptive Neyman allocation in se- quential experiments. Advances in Neural Information Processing Systems, 36:32235-3223 2023.
Konyushova, K., Chen, Y., Paine, T., Gulcehre, C., Paduraru, C., Mankowitz, D. J., Denil, M., and de Freitas, N. Active offline policy selection. Advances Neural Information Processing Systems, 01124631-244444 2021.
Duan, Y., Jia, Z., and Wang. M. Minimax Minimax-optimall off- policy evaluation with linear function approximation. In International Conference on Machine Learning, pp. 2701- 2709. PMLR, 2020.
Lattimore, and Szepesvári, C. Bandit algorithms. Cam- bridge University Press, 2020.
Li, L., Chu, W., Langford. J., and Wang, X. Unbiased offline evaluation of contextual bandttttbb--ddt-aaaddd news article recommendation algorithms. In Proceedings of the fourth ACM international conference Web search and data mining, pp. 297-306, 2011.
Dudík, M., Langford, J., and Li, L.. Doubly robust pol- icy evaluation and learning, In Proceedings of the 28th International Conference Machine Learning, pp. 1097- 1104, 2011.
Garivier, A., Lattimore, T., and Kaufmann, E. On explore- then-commit strategies. Advances in Neural Information Processing Systems, 29, 2016.
Li, L., Munos, R., and Szepesvári, C. Toward minimax off off-policy value estimation. In Artificial Intelligence and Statistics, pp. 608-616. PMLR, 2015.
Groh, M. and McKenzie, D. Macroinsurance for microen- terprises: A randomized experiment in post-revolution egypt. Journal of Development Economics, 118:13-25, 2016.
Li, T., Shi, C., Wang. J., Zhou, F., et al. Optimal treatment allocation for efficient policy evaluation in sequential decision making. Advances in Neural Information Pro- cessing Systems, 36, 2024.
Hadad, V., Hirshberg, D. A., Zhan, R., Wager, S., and Athey, S. Confidence intervals for policy evaluation in adaptive experiments. Proceedings of the National Academy of Sciences, 118(15):e201 46021 18, 2021.
Ma. C., Zhu, B., Jiao, J., and Wainwright, M. J. Minimax off- policy evaluation for multi ammed bandits. IEEE Trans- actions on Information Theory, 68:5314-5533 2021.
Hahn, J., Hirano, K., and Karlan, D. S. Adaptive experimen- tal design using the propensity score. Journal of Business & Economic Statistics, 29: 108 96, 2009.
Neopane, O., Ramdas, A., and Singh, A. Logarithmic Ney- man regret for adaptive estimation of the average treat- ment effect. AISTATS, 2025.

---

Optimistic Algorithms for Adaptive ATE Estimation
Neyman, On the different aspects of the representative method: the method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society, 97:123-150, 1934.
Noarov, G., Fogliato, R., Bertran, M., and Roth, A. Stronger neyman regret guarantees for adaptive experimental de- sign. arXiv preprint arXiv:2502.174221 2025.
Thompson, R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-255) 1933.
Wang, Y.-X., Agarwal, A., and Dudik, M. Optimal and adaptive off-policy evaluation in contextual bandits. In International Conference Machine Learning, pp. 3589- 3597. PMLR, 2017.
Waudby-Smith, I. and Ramdas, A. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series Methodological. 2023.
Waudby-Smine I., Wu, L., Ramdas, A., Karampatziakis, N., and Mineiro, P. Anytime-valid off-policy inference for contextual bandits. ACM/JMS Journal of Data Science, 2022.
Zhang, K., Janson, L., and Murphy, S. Inference for batched bandits. Advances in Neural Information Processing Systems, 13:9818-98225 2020.
Zhang, K., Janson, I.,, and Murphy, S. Statistical infer- ence with m-estimators on adaptively collected data. Ad- vances in Neural Information Processing Systems, 34: 7460-7471, 2021.
10

---

Optimistic Algorithms for Adaptive ATE Estimation
A. Analysis of Optimistic Policy Tracking
Preliminaries
We will begin by defining our good event. Consider the following events
Tox (a) -a)) 4.2V E(t,o)
(6) - n
(17)
0N
(x( n
{
[c(a)= - -E Em((aa] S 8)
(18)
EEN
()) = 0
Fr (a) r" r*)a vue(1,0)
(19)
eENN
Let 5 = and define the good event (6)) (x) (E, Applying Lemma B.1 to control (ō) and Theorem 1 from (Howard al., 2021) to control (o), and (ō) shows that the event E(Ō) occurs with probability at least 1 8. Throughout the remained of this section, assume the good event holds.
A.1. Proof of Theorem 1
We begin by decomposing the Neyman regret
T
Ri
(TTT, Pr)
(20)
(a)
1 1 Te (a)
d²(a)
Et (a)
(21)
Tee (a)
Te (a)
T1[a]
0² (a)
1 Te (a)
0² 0²(a)
0² (a)
1 n (a)
0² (a)
2 (a)
Σ
Σ
2s (a)
(22)
33 (a)
Te (a)
n [a]
41 (a)
t (a)
n" [a]
For the first term, have that n. 112 and ex (a)
< 1, that
0² (a)
1 1- Te (a)
0² (a)
E7 (a)
2
(23)
TT. (a)
Tt (a)
n² [a]
0 (a)
0² (a)
+2
(24)
Te (a)
n [+[a]
<4
(25)
to so that the regret from the exploration phase is 4T. For the second term, we have
0²
Re.
x² (a)
c (a)
(26)
Te (a)
T1 (a)
x [a]
(a)
0² (a)
Te (a) Te (a)
+ Σ
E, (a)
(27)
Te
(a)
T (a)
(28)
We can bound the first term by applying Lemma 4.3 from (Neopane et al., 2025) in conjunction with so that
2 (a)
o² (a)
625
((t,8)
Σ
(29)
n9 (a)
T (a)
(a + (
nt

---

Optimistic Algorithms for Adaptive ATE Estimation
In order to bound the second term, observe that on the good event
((t,8)
|r* (a) Fe (a)l -
(30)
Ni(a)
e(t,6)
$
(31)
Tt
Vt((t,6)
E(t,5)
(32)
Tt
where in the last line we have again applied Lemma 4.5 from (Neopane al., 2025). Therefore, have that
Σ
ne
Ex (a)
86(1,6)
(33)
Te (a)
(T=)²
We can bound the sum of these two terms as 625 ((t.5) Neyman regret from the exploration phase.
(m The result then follows by summing this over t < T and adding the
A.2. Proof of Lemma 5.1
Proof. Suppose, without loss of generality, that x* < in order to obtain results for nt > 2. we can simply flip the roles
of the treatment and control arms. For the case that 2.2 then OPTrack will always play TT.
Since n² < 1, bounding T is equivalent to determining the largest time & such that 24 (TT) < 1. i.e we wish to compute
a ((1) + 4.2,
((E,6)
Ne(1)
min
(34)
(0) + σ (1)+4.22
[(t.5] N.(1)
4.2
E(t,5) N.(0)
Using the fact that ग. = 2 for all t < T, can control
[t 1.7 1.7 ii((, 5)
Na(a) E +
(35)
2
Plugging this into equation (34) and rearranging shows that we need to bound
min t:
E(1. 8)
2
(a)
(36)
1 - 1.7 2
((1.6)
18
Applying Lemma B.10 from (Ncopane et al., 2025) shows that whenever & 2 Ö (log(?)), we have that 1.7 that we need to bound
H(t.8)
1 so
64 <1:t>
min
(115))
(37)
A7 (0)
Another application of Lemma 3.10 shows that this quantity is bounded by
64
5.2
64
64
log
log log
(38)
A2
5
A
2
A
(a)
(a)
which gives us the desired result.
12

---

Optimistic Algorithms for Adaptive ATE Estimation
A.3. Proof of Lemma 5.2 Lemma A.I. Let 2 T. Then, with probability least 1 6, we have that
25
E(t,5)
TT+1 x² (0) + (1)
(39)
T+
Proof. Wlog assume x²
so that n²' x². First note that 2 T, we have that
o (1) + Zt,
T++1
(40)
σ (0) + a (1) + ZI,t - Zo,e_
(0) + a (1)
Z
n
(41)
o (0) + (1) + Z Zo
a (0) + a (1) + ZT. Zo,
c π
(42)
where we have defined
((1,6) Ne(a)'
Ze (a) (1)==433 1.2
and equation (42) follows from the definition of the T. Since ie [T',, 2 know that - E [ T- which
control the number of times each arm is played.
(1) 2 T' t de((
(43)
(₂ Z 5).
(44)
Plugging these values into the upper bound in equation (41), some algebra shows that
σ (0) + o (1)
Z1,
T++1 x²
+
n
(45)
+ Z1, Zo,,
( + (1) +
Z1, Zo,1
Zo (t) - Z1 (t)
ZT.t
(46)
σ (0) + t (1) Z1,- Zo
σ (0) + (1) Z1, Zo.
Zo.t
Zt,t
(47)
(0) + (1) Z1,1 Zo
o
(0) t (1) + Zz,1 Zo.e
E(t,8)
1
< 8.4
(48)
Tt Vte(t,
(0) + Zo,
Applying Lemma B.10 from Neopane al. (2025), we have that when /. = Ō ((+) ( 1t Next, since 2 T, we have that
log )))
we have that x0 01((1,) 2
E(1,5)
Zo,e = 4.2
(49)
(0)
$ 8
(50)
Therefore,
A
a (0) (0) to (1) 2n, 20) + (1)
(51)
(0) o (1)
= +
(52)
8
a (0) (1)
(53)
2
13

---

8 (t, 2) Optimistic Algorithms for Adaptive ATE Estimation
Combining these results, we have that
25
((t,8)
TT+1 x- o (0) a (1)
(54)
x+6
which proves the desired result.
B. Concentration Results
The proof of this lemma is based on a similar proof found in (Audibert et al., 2006) and extends the results to hold in the sequential setting.
Lemma B.1. Let (X,) a [0, 1]-valued stochastic process defined on some filtration (Fi) satisfying μ = Ex-1 [Xe] and
0² V [X,]. Define
X
(55)
(X, -x+)
(56)
s=1
Then, with probability at least 1 for all t 2 we have that
((1,))
E(L,0)
oe σ 1.7
3 + 4.21
(57)
Proof. Define Y = (X₂ - (xxx-13--0 and S. = Ei= Y. Letting V V [Ye]. we apply Theorem from (Howard et al., 2021) which gives us the following time-uniform Bernstein inequality (see Table the Appendix). Applying union
bound, we have with probability at least for all E N, that
E (1, 8)
8 (1, 9
144 - 11 < 1.70
1.7
(58)
t
V8 (t, 8)
l (t, 4
M < 1.7V
1.7
(59)
t
4t
(t, 2)
< 1.70
1.7
(60)
t
where we E(t, 8) log 2t + 0.72 log 2 and the last inequality follows from the fact that V 0². Letting Nr 1 Eim X, some algebra demonstrates that
S. Σ (X, -1)= 11)2-02 0²
Σ [(Xx; - -1))- -(
- (H1)) 201)" -o2]
Σ [Xx: 11)2 +((x - pr) 14) + (m - 11)2 o²]
= to + +(4 14) "<xx (X;
-4 me) + t(14 11)² - to²
1=1
= to² + 0 t(Me 21)2 - to²
t(0) d² (μr 11)),
14

---

Optimistic Algorithms for Adaptive ATE Estimation
which implies
2 (o) - 0-)= = Σ Y, - (et - 4)² - Y.
(61)
Letting L f(6,6) and applying the bounds in equations (58) and 60, some algebra shows that
2 + +1.oov + 1.7L of 0.
(62)
Completing the square and rearranging shows that
of (1.72 - 1.7) 1.7V
(63)
2 o 1.7VL.
(64)
Repeating the same argument with -Y, shows that
+ 4.2V
(65)
Combining these bounds we have with probability at least
5, for all t >
((t, 0: 1.7
E(t,5)
0E
t + 4.21
(66)
C. Misc. Results
Lemma C.1. For any A g, we have that
I
V Vllg
Ar
V Allga [APPW]]
(67)
T
02 (a)
1 (1-T Tr((a)
E Exig,t
+
(Q)
(68)
7
Te
(a)
Proof. Leting AIPW, A, we have
VAlg.
E
St
(69)
T t=1
Σ
21
Σ
E [zx
øe]
(70)
T
==1
T
[22]
(71)
T
V
[AIPM..
(72)
T
1=1
The applying the law of total variance shows that Valg. [AIPW]]
Exgg (AIPP,, F(-1]) since
15

---

Optimistic Algorithms for Adaptive ATE Estimation
V [E [AIPH F(-1l] = 0. Computing the conditional variance, we obtain
Vaig,e [AIPH, Fi-1] Egig.r
(AIPW- - Fe-1]
(73)
(wr (6t + E + [ - 4) 2 1
Exig,r
(74)
w (o² + + (A - )
Ex
(75)
2 (a) ot (a))
-(61-1 (1) - 5c-1 (0))²
(76)
(1(a)
0² + (1(a))
Σ
-1) (a) + (11-((1)80-11)
(77)
(a)
(1 (1- (1-m(()) .2))(a) + 26)-1(1).86-100
+
(78)
(a)
(a)
Therefore, we have
Valg,e
Ar
1
T
E
0² (a)
+
1 (1 vT(a) 10) + (1--1(1))))100))
(79)
T
Te (a)
T()a)
=
Faig. g,
Σ
(a)
(a)
1 (1-m((e)
TT(
(i)1(a)
(80)
where the second inequality follows from the fact that <a) are uncorrelated.
16


Reviews:
[{"review_id": "Wx6QC8UlNY", "reviewer": "Reviewer_kjqu", "summary": "This paper proposes an adaptive algorithm for estimating the Average Treatment Effect (ATE) in a two-arm setup. The authors introduce an Optimistic Policy Tracking (OPTrack) method that uses “optimism in the face of uncertainty” to allocate subjects so as to reduce the variance of their final ATE estimator. They compare it to prior clipping-based methods (e.g., ClipSMT, ClipSDT) and demonstrate both theoretical (logarithmic Neyman regret) and empirical improvements, especially in low- to moderate-sample settings.\n\n## update after rebuttal\n\nThe authors have addressed my concerns, and I have updated my score from **3** (Weak Accept) to **4** (Accept) in light of the rebuttal. I remain enthusiastic about the potential and contribution of this work.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The claims are substantiated, although clarity and exposition could be greatly improved (e.g., Section 3 should be more clear and define the variables accordingly, the definition of Neyman regret seems off in eq. (5), result of Lemma B.1 is central to the algorithm and should be moved to the main text, etc.).\n\nWe agree Section 3 needed clearer definitions and better exposition. We’ve revised it to improve clarity and precision.\n\nRegarding eq. (5), we understand the concern about the scaling mismatch. However, this scaling is intentional: multiplying the algorithm’s MSE term, $(\\widehat \\Delta_T - \\Delta)^2$, by $T$ enables a direct comparison with the oracle’s MSE. We’ve clarified this by revising eq. (5), defining Neyman regret explicitly as the normalized excess MSE over the optimal policy, and recasting eq. (6) as a proposition to emphasize its connection.\n\nWe agree that more discussion on Lemma B.1 is warranted and have moved it to Section 5 and expanded the discussion to highlight how it supports Lemmas 5.1 and 5.2.\n\n> The authors focus on synthetic Bernoulli experiments. This is understandable as a first demonstration, yet adding semi-synthetic data could make the empirical study more realistic (see Dai et al. as an example).\n\nWe have run additional experiments using the macro-insurance intervention dataset used by Dai et al and will include them in the camera ready version. Our experiments on this real world dataset qualitatively align with our synthetic experiments. Below is a table summarizing the performance of various algorithms on this dataset.\n\n| Algorithm                  | 100    | 300    | 500    | 1000   | 1500   |\n| -------------------------- | ------ | ------ | ------ | ------ | ------ |\n| ClipSDT                    | 0.1178 | 0.1021 | 0.0987 | 0.0939 | 0.0956 |\n| OPTrack                    | 0.1036 | 0.0954 | 0.0942 | 0.0939 | 0.0923 |\n| Est. Reward Oracle                    | 0.1046 | 0.0974 | 0.0947 | 0.0934 | 0.0942 |\n| Oracle | 0.0924 | 0.0906 | 0.0932 | 0.0908 | 0.0910 |\n\n> However, the lack of contextual or covariate-based experiments is a gap that might be addressed in future work (as Kato et al. do incorporate covariates).\n\nWe agree this is an important direction. Kato et al. and Cook et al. address contextual settings but rely on asymptotic analysis. In contrast, our work is the first to provide nonasymptotic guarantees for adaptive reward estimation, even in the simpler noncontextual case. We see this as a foundational step toward handling contextual settings in a rigorous, finite-sample framework.\n\n> The main contribution of an optimistic approach for adaptive ATE estimation is novel and interesting.\n\nWe appreciate the recognition of our optimistic approach. In addition, our work is, to our knowledge, the first to address adaptive reward estimation with finite-sample guarantees—even without covariates. Our method also yields strong empirical performance while enjoying a simpler and more transparent analysis.\n\n> Exposition is occasionally confusing. For instance, Section 3 and 4.2 might be clearer if the difference between Eq. (1) and Eq. (2) were fleshed out more (particularly explaining $\\widehat r_t$ in the A2IPW estimator).\n\nThank you for flagging this. We previously explained some differences between Eq. (1) and (2) in lines 154–161 but have now expanded this explanation for clarity. The distinctions are as follows:\n- Eq. (1) uses marginal action probabilities $\\mathbb{P}(A_t)$, while Eq. (2) uses conditional probabilities $\\pi_t(A_t)$ based on past data.\n- The standard AIPW estimator uses data-splitting, while A2IPW requires only that $\\widehat r_t$ be predictable (i.e., adapted to the past). Beyond this, $\\widehat r_t$ may depend freely on past data. We have added this explanation after Eq. (2).\nWe also corrected a typo in Eq. (2) where $(r)$ should have been $(\\hat{r})$. \n\n> Is the population assumed identical at each round (i.i.d.)?\n\nOur assumption is more general than i.i.d. As noted in lines 133–136, we assume only that the means and variances of the potential outcomes are conditionally fixed over time. Other aspects of the data generating process may vary arbitrarily across rounds. This form of conditional stationarity is necessary for the ATE and Neyman allocation (and hence the problem) to be well defined.\n\n> How might one adapt OPTrack to the contextual setting such as in Kato et al.?\n\nWe agree this is a compelling direction for future exploration. Our primary goal here was to establish a rigorous nonasymptotic analysis of adaptive reward estimation in the simplest (tabular) setting, as no prior work had done so. Extending OPTrack to incorporate covariates, as in Kato et al., would introduce new complexities in controlling reward estimation errors across various function classes, providing a meaningful and challenging path for future research.", "claims_and_evidence": "Overall, I find the core claims supported by evidence. The theoretical results that I checked (Lemma B.1 and Lemmas 5.1–5.3) appear sound. The experiments, while limited to purely simulated data, do illustrate the claimed finite-sample improvements.  The claims are substantiated, although clarity and exposition could be greatly improved (e.g., Section 3 should be more clear and define the variables accordingly, the definition of Neyman regret seems off in eq. (5), result of Lemma B.1 is central to the algorithm and should be moved to the main text, etc.).", "methods_and_evaluation_criteria": "The authors focus on synthetic Bernoulli experiments. This is understandable as a first demonstration, yet adding semi-synthetic data could make the empirical study more realistic (see Dai et al. as an example).\n\nThe evaluation metric is the Neyman regret, which is the difference between the actual variance of the final A2IPW estimator and the optimal minimal variance over all possible estimators and allocations, scaled by $T$. Conceptually, this is a relevant criterion for ATE estimation, although the paper’s definition of this regret is somewhat buried and needs clearer exposition in the main text (they do define it, but it may confuse readers unfamiliar with prior works like Dai et al. or Neopane et al.).", "theoretical_claims": "* I skimmed the proofs of Lemma B.1 and Lemmas 5.1–5.3: they appear correct, relying on time-uniform concentration inequalities (as in “confidence sequences”) plus bounding $\\|\\sigma-\\widehat{\\sigma}\\|$.\n* I did not check every detail of the larger bounding arguments in the supplementary. However, nothing I saw raised red flags about correctness.\n* One place for improvement would be bringing some of the main arguments from Appendix B into the main text—particularly the statement about how confidence sequences for standard deviations lead to bounding the Neyman allocation.", "experimental_designs_or_analyses": "The experiments use i.i.d. Bernoulli data with varying $\\mu_0$​ and $\\mu_1$​, evaluating ATE estimates via normalized MSE. While valid as a proof-of-concept, testing semi-synthetic data or broader distributions would be insightful. Still, the results support the authors' claim the OPTrack converges faster than prior methods in finite samples.", "relation_to_broader_scientific_literature": "The work is situated well among the literature on adaptive ATE estimation (Kato et al., Dai et al., Neopane et al.) and improves on prior designs that yield only asymptotic efficiency or use IPW-based regret, while the proposed OPTrack obtains non-asymptotic efficiency vs. the best A2IPW baseline. However, the lack of contextual or covariate-based experiments is a gap that might be addressed in future work (as Kato et al. do incorporate covariates).", "essential_references_not_discussed": "From what I can see, the paper does cite the key references (Kato et al., Dai et al., Neopane et al.), which are the most relevant for adaptive two-armed ATE.", "other_strengths_and_weaknesses": "* Strengths:\n\t* The main contribution of an optimistic approach for adaptive ATE estimation is novel and interesting.\n\t* The finite-sample bounds showing $\\mathcal{O}(\\log T)$ Neyman regret vs. the best possible variance are strong.\n\t* Empirical evidence does match the theoretical claims, with clear gains over prior algorithms.\n* Weaknesses:\n\t* Exposition is occasionally confusing. For instance, Section 3 and 4.2 might be clearer if the difference between Eq. (1) and Eq. (2) were fleshed out more (particularly explaining $\\widehat{r}_t$​ in the A2IPW estimator).\n\t* The paper’s definition of Neyman regret in Section 3 (Eq. (5)) is somewhat rushed, with the first part of the definition including an implicit summation over time that the second part does not have.\n\t* Confidence sequences are introduced abruptly in Section 4.2, with the main details deferred to an appendix. A short, intuitive explanation of “what a confidence sequence is” might help.\n\t* The experiments are purely synthetic Bernoulli. It would be stronger if at least one semi-synthetic study were provided, referencing Dai et al.\n\nGiven these issues, I would label this a Weak Accept: the results are sound and interesting, but the writing needs work to be more accessible.", "other_comments_or_suggestions": "* Possibly move the essential bounding arguments (or at least a summary) from Appendix B into the main text.\n* In Section 4.2, clarify how one obtains the “confidence sequence” for $\\sigma$. Right now, we see a reference to Lemma B.1 but not enough main-text explanation.", "questions_for_authors": "* Is the population assumed identical at each round (i.i.d.)? \n* How might one adapt OPTrack to the contextual setting such as in Kato et al.?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vFItKphzky", "reviewer": "Reviewer_mBJ8", "summary": "This study proposes optimistic algorithms for the adaptive estimation of the average treatment effect. While existing studies focus on clipping-based algorithms to stabilize performance, the present work develops a method by setting the treatment-assignment probability $\\pi_t$ close to $1/2$. In addition, the authors provide theoretical guarantees regarding the Neyman regret.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive and thoughtful review. We appreciate your feedback and are glad that you found our work worthy of acceptance.\n\nIn response to your suggestion, we have incorporated a discussion of the concurrent work by Noarov et al. (“Stronger Neyman Regret Guarantees for Adaptive Experimental Design”) into our related works section. As this paper was posted on arXiv on 2/24/25—after the submission deadline—we were unable to include it in the original draft.\n\nAt a high level, this paper studies the fixed-design/adversarial setting introduced by Dai et al., and shows that, under additional assumptions, the ClipOGD algorithm from that work can be refined to yield stronger regret guarantees. However, their approach still relies on an IPW-based estimator. As a result, when adopting the definition of Neyman regret used in our paper, their algorithm suffers linear Neyman regret due to the inherent bias of IPW estimators in adaptive settings.", "claims_and_evidence": "The paper's central claim is that an optimistic strategy leads to sublinear (specifically, logarithmic) Neyman regret with respect to the optimal (minimum variance) allocation. \nTo support this claim, the authors provide:\n- A precise definition of a Neyman regret benchmark comparing the proposed adaptive estimator to the optimal variance achievable by any estimator-allocation pair.\n- A non-asymptotic analysis showing that, under mild regularity conditions, their method achieves $\\widetilde{O}(\\log T)$ regret instead of the linear-type regret often encountered by existing clipping-based approaches when measured against a stronger baseline.", "methods_and_evaluation_criteria": "The proposed optimistic approach is well motivated. A similar idea has appeared in earlier studies, such as the mixed A2IPW in Kato et al. (2020), cited in this manuscript. However, those prior approaches were introduced primarily as heuristics and lacked theoretical guarantees, even though they performed well empirically. By contrast, the authors of this paper rigorously formulate the methodology and establish theoretical guarantees on the Neyman regret under their algorithm.", "theoretical_claims": "The theoretical claims appear to be correct.", "experimental_designs_or_analyses": "I appreciate the authors' efforts in providing experimental results. The experimental setup is designed effectively. Given that this paper is primarily theoretical and methodological in nature, the exact outcomes of the experiments are less critical to my overall evaluation. Still, the experiments serve as a useful demonstration of the proposed methods in practice.", "relation_to_broader_scientific_literature": "N/A.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "N/A.", "other_comments_or_suggestions": "- I found the following concurrent work on arXiv. If you believe it is related to your paper and it is feasible, could you briefly discuss its contents in your draft?\n\nGeorgy Noarov, Riccardo Fogliato, Martin Bertran, and Aaron Roth, “Stronger Neyman Regret Guarantees for Adaptive Experimental Design.”", "questions_for_authors": "N/A.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A."}}, {"review_id": "LiCeYCIl5I", "reviewer": "Reviewer_PNDL", "summary": "The paper designs an adaptive algorithm, OPTrack, to estimate the average treatment effect (ATE) by integrating the augmented inverse probability weighting (AIPW) estimator with the optimism principle from multi-armed bandits. The paper claims that OPTrack adaptively balances the exploration of minimizing the variance of AIPW while simultaneously minimizing the Neyman regret. The paper shows that OPTrack achieves the state-of-the-art Neyman regret. Finally, the paper conducts a simulation study to demonstrate that OPTrack attains a lower MSE than other algorithms.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> On line 144, what is the definition of $R(a)$?\n\nWe meant that $\\Delta = \\mathbb{E}[R_t(1) - R_t(0)]$ for any $t$ and we had omitted the subscript for brevity since the choice of $t$ is immaterial. However, we agree that this could be confusing. To clarify, we have revised the definition to $\\Delta = r^\\star(1) - r^\\star(0)$, where $r^\\star(a)$ was already defined just above line 135. This should eliminate the ambiguity.\n\n### Discussion on conditionally fixed mean and variance assumption\n\nOne concrete example where our assumption holds is the i.i.d. setting which is a standard assumption for RCTs [1]. More broadly, this assumption is either equivalent to or a generalization of those made in several prior works on adaptive experimental design (Kato et. al, Cook et. al, Neopane et. al) and is the standard assumption made in the bandit literature.\n\nWe also emphasize that these assumptions are *necessary* in a meaningful sense: the ATE is defined as the difference in mean outcomes under treatment and control, and without our assuming these means are conditionally constant, the ATE is ill-defined. Similarly, the Neyman allocation depends on the variances of the potential outcomes, so if the variances change across rounds, we no longer have a meaningful way to define an optimal allocation. Therefore, this assumption represents the weakest possible condition under which adaptive ATE estimation remains well-defined.\n\n\n> The paper's contribution is mainly theoretical...\n\nWe would like to clarify that our contributions are both theoretical and methodological. In addition to presenting new theoretical insights, we propose a novel algorithm that is not a minor variant of existing methods—it is based on distinct design principles. Our algorithm enjoys a greatly simplified analysis, achieves significantly stronger theoretical guarantees than prior work, and demonstrates superior empirical performance in simulations.\n\nWhile our long-term motivation is to inform the design of adaptive clinical trials, we acknowledge that our current setting does not include covariates. However, it does capture a realistic and meaningful intermediate scenario where outcomes from previously treated patients are observed and used to inform future treatment decisions. The challenge of performing nonasymptotic analysis in this setting—without covariates—is itself nontrivial and had not been fully addressed in prior work.\n\nExtending our framework to incorporate covariates is indeed an important next step toward practical applicability in real clinical settings. We view our current work as a foundational step toward that goal.\n\n> I believe this paper requires significant revision to clearly justify why the setting and assumptions are valid in real-world scenarios and to clarify why their theoretical results are exciting for a broader audience. \n\nWe respectfully disagree with this statement, as we believe it reflects a misunderstanding of the paper’s scope and contributions. Our goal is not to present a fully deployable solution for real-world clinical trials, but rather to address a fundamental theoretical problem that remains unresolved—even in the simplest setting without covariates. It is worth emphasizing that randomized controlled trials can, and often do, proceed without incorporating covariates into the treatment assignment, especially in early-phase trials, low-data settings, or when covariate information is unavailable or unreliable. Our work aims to rigorously understand this foundational setting, which has practical relevance in such scenarios, before tackling the added complexity of contextual information.\n\nThe setting and assumptions we consider are standard in the literature, have been adopted in prior foundational works. Our theoretical results are novel, conceptually clean, and advance the understanding of Neyman regret in nonasymptotic regimes. Importantly, these insights led to the development of a new algorithm that is not only theoretically sound but also empirically superior to existing methods.\n\nWhile we agree that real-world applicability in full generality—particularly in large-scale or personalized settings—requires the incorporation of covariates, we view our work as a necessary first step. We will revise the introduction to clarify the scope, highlight the novelty of our contributions, and emphasize the broader relevance of our results beyond any single application.\n\n> Additionally, the paper should demonstrate that the proposed algorithm performs well on at least one real-world dataset.\n\nWe have run simulations using the macro-insurance dataset from Dai et al. and will include the figure in the camera ready if accepted. Please see the table presented in our response to Reviewer kjqu for a summary of these simulations.\n\n[1] Imbens, G. W., & Rubin, D. B. (2015). *Causal Inference for Statistics, Social, and Biomedical Sciences* (See chapters 1 and 6)", "claims_and_evidence": "On line 144, what is the definition of  R(a)? It appears that the authors have only defined R_t(a). My best guess is that \\( R(a) = E[R_t(a)] \\), but this is not clearly stated. Since this quantity plays a critical role in defining the paper’s parameter of interest, the lack of clarity has made it difficult to follow the rest of the paper.", "methods_and_evaluation_criteria": "On line 135, the authors assume that the conditional expectations of the potential outcomes are fixed. I do not understand the implication of this assumption, as the conditional expectation is a function of past observations. Could the authors clarify what real-world scenarios this assumption is reasonable? The same concern applies to the conditional variance.", "theoretical_claims": "No, I did not check them carefully, as their correctness would not affect my decision.", "experimental_designs_or_analyses": "I have checked the simulation study in section 6.", "relation_to_broader_scientific_literature": "The paper's contribution is mainly theoretical. Although the authors claim that their algorithm is critical for applications like randomized clinical trials, regrettably, neither did they use this as a running example to explain the validity of their setting and assumptions, nor did they demonstrate their method on a real dataset. For example, for each round in a clinical trial, in addition to the treatment and reward, the doctor should have additional information like the patient's pre-treatment covariate and the outcome for other patients. Also, I'd love to understand how the assumptions claimed on line 135-136 make sense in a randomized control trial.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "I believe this paper requires significant revision to clearly justify why the setting and assumptions are valid in real-world scenarios and to clarify why their theoretical results are exciting for a broader audience. Additionally, the paper should demonstrate that the proposed algorithm performs well on at least one real-world dataset.", "other_comments_or_suggestions": "line 48: \"The rest of our paper is organized as follows. The remainder of this paper is structured as follows.\"\n\nline 160: \"hestimator\"\n\nline 144-146: \"The Neyman regret is simply the difference in the normalized MSE between the optimal variance and the MSE of the estimate produced by Alg.\" I believe the one being subtracted is the optimal variance.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tMK74L7Sbd", "reviewer": "Reviewer_Gq4L", "summary": "This paper addresses the challenges of the exploration-exploitation tradeoff in the adaptive estimation of the Average Treatment Effect (ATE), proposing a novel approach based on Neyman regret. The authors clearly outline the motivation and theoretical foundation of the algorithm, and conduct simple experiments on synthetic datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> It would be beneficial to compare the proposed method with some of the many related works, if possible.\n\nWe did not compare our algorithm with Kato et al. (2020) because Cook et al. (2022) showed that ClipSDT significantly outperforms Kato’s algorithm, even under weaker assumptions (see Figure 2 in Cook et al.). Since we already compare against ClipSDT—the stronger of the two—it would be redundant to include Kato et al. (2020) in our evaluation.\n\nAs for Kato et al. (2024), the setting considered in that work is different from ours: their focus is on adaptively selecting which covariates to observe, whereas our problem does not involve covariates at all. As such, their method is not applicable to the setting we study. \n\n> What are the details of the experiments, including the data generation process for the synthetic dataset and the implementation details of the methods?\n\nEach of our figures is based on simulations in a two-armed Bernoulli bandit setting, with different Neyman allocations. The mean rewards are set to $\\mu_1 = 1/2$ and $\\mu_0 \\in \\{0.05, 0.1, 0.2, 0.3, 0.4, 0.5\\}$. At each round $t$, we sample $R_t(a) \\sim \\text{Bernoulli}(\\mu_a)$ for $a \\in \\{0, 1\\}$, and the algorithm observes the realized reward $R_t(A_t)$ corresponding to the chosen action $A_t$.\n\nThis interaction continues for $T$ rounds, after which we compute the estimated ATE. We repeat this process across multiple independent runs and report the variance of the resulting ATE estimates. \n\nThese experimental details are provided in lines 310–318 (right column) and lines 375–379.  We agree that some parameters were only implicitly specified in Figure 2, and have revised the experimental section to explicitly list all relevant parameters, including the exact values used for the Bernoulli means, the number of rounds, and the number of repetitions. We hope this improves the clarity and completeness of the experimental description.\n\n> the authors mention many related works, but why are many of them not compared in the experiments?\n\nWe discuss this in the experiments section on Lines 303-307 (right column). Many of the prior works were designed for the IPW estimator, and so their performance in comparison to the AIPW estimator is so poor that it visually obscured the graphs. These existing algorithms do not converge to the asymptotically minimal variance.\n\n\n> It would also be helpful to include experiments on real-world datasets to validate the effectiveness of the method in real-world scenarios.\n\nWe have run simulations using the macro-insurance dataset from Dai et al. and will include the figure in the camera ready if accepted. Please see the table presented in our response to Reviewer kjqu for a summary of these simulations.", "claims_and_evidence": "The authors provide a comprehensive and clear explanation of the motivation and theoretical justification for the proposed method. However, their second claimed contribution, the experimental validation of the method's effectiveness, seems somewhat overstated, as the experiments presented in the paper are overly simplistic and lack detailed descriptions of the experimental setup.", "methods_and_evaluation_criteria": "The proposed method is correct, supported by thorough theoretical proofs and illustrative examples. However, the paper lacks details on evaluation criteria, such as the data generation process of the simulations. Additionally, it would be beneficial to include experiments on real-world datasets to validate the effectiveness of the proposed method in real-world application scenarios.", "theoretical_claims": "The theoretical proof presented in the paper is detailed, clear, and well-reasoned.", "experimental_designs_or_analyses": "The experimental part of the paper is incomplete, which constitutes its major weakness.\n\n1. Details of the simulations are missing, and the authors should provide them, such as the data generation process.\n\n2. As mentioned in the related work section, there are many related works beyond ClipSDT [1], such as [2, 3]. It would be beneficial to compare the proposed method with some of the many related works, if possible.\n\n3. It would also be helpful to include experiments on real-world datasets to validate the effectiveness of the method in real-world scenarios.\n\n[1] Cook, Thomas, Alan Mishler, and Aaditya Ramdas. \"Semiparametric efficient inference in adaptive experiments.\" Causal Learning and Reasoning. PMLR, 2024.\n\n[2] Kato, Masahiro, et al. \"Efficient adaptive experimental design for average treatment effect estimation.\" arXiv preprint arXiv:2002.05308 (2020).\n\n[3] Kato, M., et al. Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choice. International Conference on Machine Learning, PMLR, 2024.", "relation_to_broader_scientific_literature": "The paper applies the Neyman regret theory from Multi-Armed Bandits (MAB) to adaptive ATE estimation. The relevant literature is correctly cited, and the authors clearly highlight the differences and advantages compared to these works. The discussion is also accurate and well-written.", "essential_references_not_discussed": "The relevant literature that needed to be cited has been correctly referenced.", "other_strengths_and_weaknesses": "The strengths of this paper lie in its clear writing, well-defined and reasonable motivation, correct and novel methodology, and thorough theoretical proof. However, the main weakness is that the experimental part is not sufficiently complete.", "other_comments_or_suggestions": "The following are my comments after reviewing the rebuttal:\n\nThe majority of the questions and concerns have been well addressed:\n\n1. The authors did not compare additional baseline methods because ClipSDT is a state-of-the-art method. As a result, they chose to omit baselines that have been shown to perform worse than ClipSDT, as well as those that do not converge at all.\n\n2. The authors have provided a detailed description of their experimental setup on the synthetic dataset and have committed to including the relevant descriptions in the revised version.\n\n3. The authors have added experiments on a real-world dataset, and the results show that the proposed OPTrack method outperforms the baselines, with performance approaching that of the oracle algorithm using the true reward.", "questions_for_authors": "The following questions are based on my previous comments, and I encourage the authors to carefully review them.\n\n1. What are the details of the experiments, including the data generation process for the synthetic dataset and the implementation details of the methods?\n\n2. In the related work section, the authors mention many related works, but why are many of them not compared in the experiments?\n\n3. It would be better if suitable real-world datasets could be found to validate the effectiveness of the proposed method in real-world scenarios.\n\nOverall, I appreciate the writing, motivation, methodology, and theory presented in this paper. If the authors could address the points mentioned above and make the experimental section more complete, I would be happy to increase my score.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers generally acknowledged the strengths of the work and are satisfied with the authors’ rebuttals. I concur with the reviewers’ opinions and recommend acceptance of this work. However, the reviewers also raised several points for improvement, particularly concerning the numerical experiments and the clarity of exposition. The authors should incorporate the reviewers’ comments and further improve their paper in the camera-ready version.
Decision: Accept (poster)