Revisiting Active Sequential Prediction-Powered Mean Estimation
Authors: Maria-Eleni Sfyraki
Jun-Kun Wang
Venue: ICLR 2026 Poster (2026)
Keywords: active statistical inference
mean estimation
no-regret learning
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: In this work, we revisit the problem of active sequential prediction-powered mean estimation, where at each round one must decide the query probability of the ground-truth label upon observing the covariates of a sample. Furthermore, if the label is not queried, the prediction from a machine learning model is used instead. Prior work proposed an elegant scheme that determines the query probability by combining an uncertainty-based suggestion with a constant probability that encodes a soft constraint on the query probability.  We explored different values of the mixing parameter and observed an intriguing empirical pattern: the smallest confidence width tends to occur when the weight on the constant probability is close to one, thereby reducing the influence of the uncertainty-based component. Motivated by this observation, we develop a non-asymptotic analysis of the estimator and establish a data-dependent bound on its confidence interval. Our analysis further suggests that when a no-regret learning approach is used to determine the query probability and control this bound, the query probability converges to the constraint of the max value of the query probability when it is chosen obliviously to the current covariates. We also conduct simulations that corroborate these theoretical findings.

Published as a conference paper at ICLR 2026
REVISITING ACTIVE SEQUENTIAL POWERED MEAN ESTIMATION
PREDICTION-
Maria-Eleni Sfyraki
Jun-Kun Wang
University of California San Diego ms ferakiiki@@e (uussdd.dde
University of California San Diego jww0 35w005@ucsd.eee
ABSTRACT
In this work, we revisit the problem of active sequential prediction-powete mean estimation, where at each round one must decide the query probability of the ground-truth label upon observing the covariates of sample. Furthermore, if the label is not queried, the prediction from machine learning model used instead. Prior work proposed an elegant scheme that determines the query prob- ability by combining an uncertainty-bet -based suggestion with a constant probability that encodes soft constraint on the query probability. We explored different val- ues of the mixing parameter and observed an intriguing empirical pattern: the smallest confidence width tends to occur when the weight the constant prob- ability is close to one, thereby reducing the influence of the uncertainty-bant component. Motivated by this observation, we develop a non-asymptotic analysis of the estimator and establish a data-dependent bound on its confidence interval. Our analysis further suggests that when no-regret learning approach used determine the query probability and control this bound, the query probability con- verges to the constraint of the max value of the query probabílity when is chosen obliviously to the current covariates. We also conduct simulations that corroborate these theoretical findings.
INTRODUCTION
The mean estimation problem is a classical inference task that has seen revived interest in machine learning and statistics over the last few years. While the conventional setting is well-understood, numerous works have explored this problem under diverse settings and assumptions, aiming to en- hance our understanding of the inherent challenges of learning from limited data. Recently, a line of work has investigated the design of efficient mean estimators under the robust framework, including the setting of constant fraction of adversarial outliers (Cheng al.) (020). heavy-tailed symmetric distributions without moment assumptions (Novikov et al.) (023), mean-shift contamination in mul- tivariate identity Gaussian distributions (Diakonikolas et al. 2025 sparse mean estimation in high dimensions (Pensia) 2024), online high-dimensional mean estimation (Kane et al.) 2024). A few other recent works have explored other structural considerations, such as collaborative normal mean estimation in the presence of strategic agents (Chen et al. 2023), communication-efficin
mean estimation in a distributed setting (Ben-Basat et al. (2024), vector mean estimation under the shuf- fle model of privacy (Asi et al.] 2024), dynamic multi-group mean estimation (Aznag et al. 2023), leveraging favorable distribution structure improve sub-Gaussian rate (Dang et al. 2023) among others. This variety of scopes in the mean estimation setup highlights its relevance as foundational task for inference.
A direction that has attracted substantial attention with the increasing integration of machine learn- ing mean estimation through an active inference perspective (Zrnic Candes 2024). Specifically, the problem focuses on estimating the mean label from set of unlabeled observations, by leveraging a limited label collection budget and the abundant but potentially biased predictions of a machine learning model. Under this setup, active statistical inference provides data collection strategy that utilizes the budget more effectively by taking into consideration which labels it would be more ben- eficial to acquire. Specifically, it prioritizes the collection of labels where the model exhibits higher uncertainty and uses the model predictions for instances where the model is more confident. The same work considers querying the labels in both the batch and sequential setting, where the latter

---

Published as a conference paper at ICLR 2026
additionally allows updating the model the ground-truth labels obtained, and provides asymp- totically valid confidence intervals for the estimator in question in both cases. The -on-asympootic analysis of the estimator was not provided in the prior work.
In this work, we draw on the sequential active statistical inference perspective by providing non- asymptotic guarantees for the sequential active mean estimation problem. While prior work of Zrnic & Candes (024) have established the asymptotic normality of their proposed estimator, our work investigates the sequential mean estimation problem further under the light of an online updating scheme and provides a non-asymptotic analysis with guarantees that hold at any time while going through the data. More specifically, we first formulate the scheme of active sequential mean estima- tion as an online update step, and establish a convergence guarantee that incorporates the conditional variance of the update direction and achieves a rate of Ô (1/ for sufficiently large t.
Furthermore, motivated by a series of experimental findings, which reveal an intriguing pattern of the label sampling rule considered by previous work, we are led to examine more closely the role of the model uncertainty component through the current covariate. In particular, adhere to the budget constraint and ensure a small variance of the estimator, /rnic & Candes) (2024) derive a sampling rule that a mixture of the uniform rule and a model uncertainty estimator weighted by mixing constant. Our experimental findings across variety of tasks indicates that employing the mixture rule or relying solely on the uniform policy results in comparable confidence interval widths, with the uniform policy occasionally producing marginally narrower intervals. This observation suggests that, the contribution of model uncertainty with respect to the label of the current covariate might be brittle in practice. Based on this insight, we formulate the problem of tuning the query policy as an online learning task that does not rely on the current covariate, and whose validity is supported by the strong sublinear regret guarantees of the classical Follow-the- Follow-the-Regulariieel Leader (FTRL) algorithm (Abernethy et al. (012). Remarkably, we demonstrate that under this no-regret learning approach, the query policy converges to the maximum value permitted by the budget constraint. Our theoretical findings are further validated through experiments on three real-world and one synthetic dataset.
2 PRELIMINARIES AND NOTATION
We begin by introducing the problem setup. We consider the setting where we have access to a sequence of data points æ1, #2, ET E X C from an unknown, fixed distribution Px. Each data point x, is associated with a ground-truth label y/ E Y C R. drawn from also unknown, fixed distribution Py|x. We assume that the ground truth labels are not known a priori, and the cost to obtain them could be high. We are interested in estimating the mean label μhg = EE Additionally, we assume that at each time & E [2'] we have access to a black-box predictive model fr(-): : X + Y, which can continually evolve by using the samples collected up to round t to update. More specifically, we require fe E Fi-1, where F, denotes the o-algebra generated by the first data points 2,, 1 1 s <
Sequential active mean estimation seeks to construct efficient estimator by observing data points one at a time and deciding whether to query each ground -ruuh label. Under a limited labeling budget, the objective is to sequentially acquire labels in way that most effectively improves the accuracy of the mean estimator. Specifically, if we denote by Thab the total number of collected labels, we require that the policy of querying the ground-truth label ensures E[Tias] S Th, where we assume that Ti < T. Let Ty (x,) denote the probability of collecting the label of data point x, at time t, where ne E Fi-1, and let & Bernoulli(n TT (xx)) denote the labeling decision used to indicate whether the ground-trum -ruth label was collected (E: 1) or not (E. = 0). Zinc & Candes (2024)
is easy to verify that á is unbiased, i.e., E[ú] My. Notably, Zrnie & Candes (2024) show that the optimal choice of querying policy T is according to the uncertainty of the prediction model, and
propose the sequential active mean estimator iú = 1
T ΣT f((()) + (ye f(())) Ee
'Howeerr we acknowledge that Appendix C of Zmic & Candes (0244) outlines related scheme based on estimating bounded means via testing by betting (Waudby-Smith & Ramdas 2024), which may come with certain non-asymptotic guarantees. We further provide discussion in Appendix
2

---

Published as a conference paper at ICLR 2026
it satisfies
ppt # (at)
E
[ve fe(ma)) 1 (
where the above expression hides normalization constant ensure that E[m; However, since Py|x is unknown, the authors suggest fitting a model on past data (I, 2) to approxi- mate the uncertainty (2(z) by 1/ fz(z)) for given 21, and then setting the querying policy to be proportional that estimate. To ensure that the budget constraint is met in practice, the remaining budget at time is set as the difference between the expected budget to be used up to time and the budget already used up to time i.c. TA tTtTT T1bb Then, the querying policy is set to
(x,)) To/T.
(.x)) = min (n (xx), Ta.)o.1)
where the subscript [0. 1] denotes clipping [0, 11. T) a normalizing constant set to 1)e To/ (TE (xx))) and E (xx)] is approximated empirically. This practical rule aims to balance frequent sampling under high uncertainty against overusing the budget. However, since the esti- mated uncertainty can be consistently low, and thus the budget could be underutilized, the policy is occasionally set to (Z+) = (T..)))o.1) An additional concern is that an inaccurate uncertainty estimate may be close to zero, while the quantity 1yt fe(2r))) | is actually large, which would in turn
yield an amplified estimator variance. To address this issue, the authors suggest that the policy is set
to mix of the described policy with the uniform rule
(Irz) = (1 - ))m)( +
Amri Am (xx)),
where ( T,/T, and À [0, 1]. In the presence of sufficient historical data, À can be tuned by minimizing the empirical estimate of the estimator variance induced by the policy q. However, due to insufficient historical data in the sequential setting, À is set fixed value.
While the related work simply sets the mixing parameter to 0.5, explore different parameter values for the mean estimation experiment in Zmnc & Candes (2024) by run- ning their public implementation on the same post-election survey dataset Center 2002) as their experiments. We keep all their other parameter choices unchanged. Figure [ shows how the interval width varies with the sampling budget T₁ under dif- ferent values of the mixing parame- ter in the query policy that setting X 1, corresponding to using the uniform query policy that ignores model uncertainty, produces confidence intervals that are slightly narrower than those obtained with À = 0.5. We further evalu- ate the method on two additional real-world datasets and one synthetic dataset, and observe a similar pattern, where the uniform query policy (A- = 1) yields confidence intervals that are comparable to, and often tighter than those from 0.5. Due to space constraints, the corresponding figures are included in Appendix A
0.07
Estimator
[CC24] (mixing=0.05)
0.05
[C24] (mixing=0.1)
Interval width
[C224] (mixing=0.5)
[CC24 (mixing=0.8)
0.04
[C24] (mixing=1.00
0.03
0.02
1855
2370
3029
3871
4946
To
Figure 1: Post-election survey dataset. Interval width vs. the sampling budget parameter T. for different values of the mixing parameter of the query probability scheme in Zrncc & Candes) (2024). Averaged over 10 repeated runs.
We find
These similar empirical patterns motivate us to further investigate this observation. plausible initial explanation is that the empirical performance improves when less weight is assigned to the uncertainty-bect based component, possibly because the uncertainty predictor may not be reliable. However, our thcoretical analysis later suggests that this factor alone may not account for the phc- nomenon, which might be surprising.
"The code, provided in Jupyter notebook format, to reproduce Figure as well as others in the paper available in the supplementary material.
3

---

Published as a conference paper at ICLR 2026
3 RELATED WORK
Active Statistical Inference. The ideas in this work are motivated by the recent approach of Active Statistical Inference (Zmic & Candes 2024). Extending this framework, ( propose a method that optimizes the sampling rate between gold-standard and pseudo-labels rather than relying on fixed label budget, and derive an improved active sampling policy, A recent work by Gligorić et al. (2025) utilizes LLM verbalized confidence scores to guide their sampling policy and subsequently performs active inference by combining the LLM and human annotations.
et al.) (2025)
Prediction Powered Inference. The Active Statistical Inference framework is grounded in the idea of Prediction Powered Inference (PPI) (Anellopollss et al. (2023a), which differs from the former in that it assumes the availability of a small, pre-labeled dataset. The work of Angelopoulos et al.) (2023b) introduces PPI++, which improves the computational efficiency of PPI by addressing the intractability of the original confidence interval construction and using tuning parameter to con- trol the influence of the model predictions based on their quality. Subsequent works (Dorner et al. 2025 Mani et al. 2025) analyze the critical role of the correlation between the gold-standare and model-generated labels for the performance of PPI. Following the original PPI formulation, sev- eral works have proposed extensions and refinements in various directions, including addressing estimator bias in the few-label regime (Eyre & Madras 2025). incorporating an inverse probability
weighting (IPW) bias-correction term (Datta Polson 2025), combining predictions from multiple
foundation models via a hybrid augmented IPW estimator (De Bartolomeis et al. 20225) applying a stratification approach (Fisch et al. 2024), exploring a bootstrap- based variant (Zriic) (024). ) em- ploying Bayes-assister assisted approaches (Cortinovis & Caron 2025 Li & Ignatiadis 2025) and extending the ideas of PPI to e-values (Csillag et al. 2025). Other applications of the PPI framework include LLMMassistte rank-set construction (Chatzi et al.
2024), average treatment effects from multiple datasets (Wang et al.] 2025), clinic trial outcomes (Poulet et al. 2025), autoevaluation in machine (025), machine learning generated surrogate rewards for multi-armed bandits (Ji et al. (025b). few other works have explored alternative
learning systems (Boyeau et al.) 2025; Park et al.
machine-learnnng assisted estimators, e.g.. Schmutz et al. (2023); Egami al. (2023); Miao et al.
(203); : Miao Lu (2024); Gan et al. (2024).
We refer the reader to Appendix for a more detailed discussion related work.
4 NON-ASYMPTOTIC ANALYSIS OF THE MEAN ESTIMATOR
In this section, we provide the non-asymptotic analysis of the sequential active mean estimator: Since the sequential active estimation setting requires going over the data points sequentially, we can formulate the active mean estimator of Zrnic& & Candes (202)) as online update step each time E [T), as
1 w++1 = we + T [((())) + (v Sc fe(xx))
E Te(
where we set the initial point w1 O, and let T be the horizon. We denote 9t := f(() + (y+ f(())) ( simple calculation shows that [R[g.]] = Myy the mean of the random variable (y1)221-
Before proceeding with the asymptotic analysis of ( we will need a technical lemma, known as Freedman's 's inequality, which is stated below for completeness.
Lemma (Freedman's inequality (Freedman) 1975), see also e.g., Lemma in Rakhlin ST be martingale difference sequence with a uniform upper bound KG b, Vt. Denote V, the sum of conditional variances of Co 's., ie., V. 1/e and T 2 4,
et al.] (2012). ). Let ST
Σ'=1 Bar(Salli....,...... Also, denote a₁ == /Ve. Then, for any
we have
Prob 31ST:
Σ
c. 2 max (20r, brl00(1/66) vlog(1/6)) log(T)ō.
Lemma I provides a concentration inequality for martingales, yielding a high-probabilit bound on the deviation of martingale sum from its mean that adapts the accumulated conditional
4

---

Published as a conference paper at ICLR 2026
variance. We ready present the non-asymptotic analysis result for the sequential active mean estimator, as detailed in Theorem
Theorem 1. Fix time horizon T 2 4. Assume each random variable is bounded, i.e., O. Denote of == TE [[g: - By [ the conditional variance, where Fi-1 is the filtration up 1 - 5, Vt E [
Igel S G for constant
Then, for any (0, 1/e), with probability at least
2 max 2/S, + (1gl) log (og(?T)
log [lo((T)
(1 (1-7) Myl:
|w++1 - Myl
+
(2)
where S, = 25-1 o..
Theorem demonstrates a data-dependent bound on the accuracy of the update w, that holds with high-probability any time E [T]. We make couple remarks this result. When < T and the (1 4) lugl term dominates the other term on (2) it is possible to observe a rate that is slower than (1/vV) in the initial stage, i.e., 1 -
However, after this burn-in stage (i.e., when sufficiently large), the first term will eventually become dominant. Furthermore, when v V of dominates (G + [,y / (log(lo((T)/)) which happens easily when is sufficiently large and is not too small, the rate becomes
( [V (oogoooogT)))//
Wi+1 - H4y Hyy-= =0
T
Using the trivial bound E'=1 03 S 21(G2 + ngg, we can further express the rate as
o vay (G²+ log(log(T)/6)
0 (tt).
POLICY OF QUERYING THE GROUND TRUTH
In the previous section, we discussed that the update wy of will have a rate of O (Vi/T)
O (1/VE) in the worst case. An observation is that when E.= o < 2t(G2 + (i)) one might get an even faster rate than OLL/Vt). This motivates us control the i... Σ. o²2 E._ [(g, 2,,) by proposing an algorithm determine the query probability of the ground-truth label online, which we detail next.
the conditional variances,
We begin by introducing the following observation on the decomposition of the conditional variance of the update step, which will subsequently guide the choice of the online query policy.
Lemma 2. The conditional variance has the following decomposition:
1
[g9 [1 E
f((2)))
F
EE
(vi fe(x)))
Fi-
To (x(2))
+ 2E fd(xi)(ve - fe(x))) (((x)) F-.1
Furthermore, assume that the query policy time t is F Fe-s-meaurablle, i.e.. there exists random variable Pt E [0, 1). measurable with respect to Fi-1. such that Te = Pt. Then, we have
1
E
(ve f(())))
F
E
(yr
f((x)))² F
Te(
Pt
5

---

Published as a conference paper at ICLR 2026
We observe that the only term involving the query probability (x(2) that contributes to the conditional variance is ((yx-f((ax))) (xx))))² Fc-1] We now consider the query policy time 1 that fully determined by the information up to - With this, we can rewrite El( - fe(z))) [[-1] = Elye - f((x))) [F--]] However, we note that El - f((x))) 1= cannot be known since this depends on the unknown distributions of y and fe(z)). Therefore, assume that there is an oracle, denoted as (x))) E R4,, which is available time and provides an approximation of the quantity of interest, i.e..
1 (xx) S IE (y fe(ox)) Fe-1 S <cDDe(re),
for some constants co, 01 > such that +; (x²)) R. B[(ue - fe(x,))² Fr-1). Equipped with such an oracle, propose specifying the query probability ps based on the following rule:
$ (x.s))
< arg min 701- 1p + 20² where 0 := pE|s,r]
(4)
Ps
where y > O, r E (0, 1], 8 E (0, are user-specified parameters, and we let dg == 0. The following
lemma shows that Pt has closed-form expression.
Lemma 3. The update 4 has closed form expression, which is
= max min(T, -0;
We note that one can specify 7 = To where To denotes the targeted maximum number of rounds in which the ground-truth is queried, which ensures that the query probability Pt time t does not To This constraint is also akin to the sampling rule E'pp] I f considered in Zrnic Candes (2024). On the other hand, the parameter /3 encourages certain exploration at each round by preventing the query probability Pt from becoming too close to O.
exceed the ratio 7.
The update a nutshell, is one of the celebrated online learning algorithms called Follow-the- Regularized Leader (FTRL), (see, e.g.. Abernettyy al. (2012); Waan et al. (2024), and Chapter of Orabonal 2019)). FTRL is known to enjoy a sublinear regret bound when the sequence of loss functions convex. We propose leveraging the strong guarantee of FTRL to determine the query probability Pa online. More specifically, in our scenario, one first determines the query probability Pt. after which it receives loss function defined as (p) == *e(me)) which is convex loss function in (0, 1]. In online learning, common goal is minimize the regret. In our setting, the regret against a benchmark p. E [8,T] over rounds defined as:
Regret, (p.) ) = Σ (p..)) - ((pp)) Σ
(x(x.)
T
(,(x.) P.
(5)
Ps
==1
==1
where the first sum the cumulative loss of the updates (ps)s21 and the second one is that of the benchmark. A sublinear regret bound against any benchmark p. in the same decision space [3, of the learner implies that the sequence of query probabilities can compete with the best fixed query probability in hindsight. On the other hand, given that the oracle's output is non-negative, i.c., (p) т. Combining these implies that an online
Vs + (.) 2 0. it follows that arg minp.e(s.r)) learner may need approach 7 eventually to achieve sublinear regret. In other words, to maintain sublinear regret, the query probabilities pt will need to converge toward the constraint upper bound T. In particular, we have that the average regret is in fact vanishing (a.k.a. no-regret learning), Regret, + as + oo, as the following lemma shows.
Ta
i.., t
Lemma 4. (see e.g., Theorem 3 in Luo (2017)) ) FTRL satisfies
R(p.) - min R(p)
Regret, S
+
7

---

Published as a conference paper at ICLR 2026
for any comparator p. E XC := [3, r|. where y > 0. R(p) = }jp² and É, = dl,(p) dp p=
We note that Lemma 4 is classical result in online learning literature, see also Orabona (019)) Shalev-Shwarte et al. 2012). The guarantee suggests that if one chooses 7 vT then the regret
of FTRL O(VT), which grows sublinearly with T. provided that the size of the derivative is bounded. The following lemma shows that the size of the derivative in the regret bound is bounded whenever the oracle's output is bounded.
Lemma 5. Assume that the range of oracle's 's output is bounded, i.e. Vt De(x:) S B. for a constant B > O. Then, Vt:
B²
g4.
In the following theorem, denote of := >==1 s.(t)* the cumulative conditional variance ob-
tained under a fixed query probability Pi:t K. == [3, T]. , if the method had committed the best
fixed probability in hindsight over rounds rather than following the query policy from (4) i.e.,
Pi:t arg minp E tx(p).
Theorem 2. Assume that there is an oracle that outputs (xx)) at each I. such that 1 (xx)) x
E
(ye - f((ex))
Fi-1
( for some constant co, and that Vt 2x(xx) B.
Set the parameter
vT B Using the query policy (4) we have that, with probability at
least 8, Vt E [T]:
max 2/ (G + (nyDy log (log(2)
log
log(T) 5
t
|1e+1 MMyl I
+
1
T
T
for any 3 (0,7) and t E (0,1), 1), where V. S colR(pi winger H(p))TBB
coc10
+ 200 vF gz
B
+
What Theorem shows a data-dependent bound. We note that v I Vcoci + O (4) From our earlier discussion, once sufficient burn-in period has elapsed so that the first term in the upper bound dominates, the non-asympiotic rate takes the form O /ir
cpc1 Ii
13/4 provided that d is not too small.
EXPERIMENTS
In this section, report experimental results by comparing the proposed method with two base- lines. For clarity, Al Compared to the procedure described in Algorithm 2 of Zmic Candes (2024), the key difference is that we also update the uncertainty predictor whenever the ML model for label prediction is up- dated. Furthermore, we split the dataset with ground-truth labels into two disjoint subsets, which are accumulated as described on Line 13, and use these subsets to expand the data available for updating the ML model fr+1 and its uncertainty predictor ut+-1, respectively. This treatment of the disjoint training sets is intended to enable the uncertainty predictor to more accurately estimate the uncertainty of the ML model when it is applied to unseen data test time.
the protocol for the task of active sequential mean estimation.
The first baseline was also considered in the prior work of Zrnic & Candes (2024).
1 T E (I(Ee) (y: fax))) To/T
Tb T
Uniform
where & Bernoulli
For this baseline, note that the ML predictor is fixed. As discussed in Zmnic & Candes 2024 this comparison can showcase the benefit of data collection. Following the terminology of Zrnic & Candes 2024), refer this baseline "uniform sampling."

---

Published as a conference paper at ICLR 2026
Algorithm Protocol of Active Sequential Mean Estimation
Require: Significance level parameter (0, 1), target sampling budget T3 0, and batch size
B.
1: Initialize machine learning (ML. model (1(+)
Y predict the labels of data.
2: Initialize an uncertainty predictor W1 (::-) : x (i(-) + R, for the model's predictions.
Initialize the dataset for updating the model Prain and the dataset for the uncertainty predictor Duncertainty'
4: Set Dump t W1 4-
5: for l.......,T do
6: Observe features 24, of sample and determine the query probability pt for getting its label.
7: Sample the binary random variable & Bernoulli(pt).
8:
if & then
9:
Obtain the ground-trute trrut label ye and set Pmmp t- Pump U { (c)}. Increase by L.
10:
11:
end if
12:
if = then
Randomly split Pmpp into two datasets with equal sizes, D& and Da Set Dunin f- Dimin U aD and set D Pancertainty <- Duncertainty U Update the ML model to f(41()) using the dataset Dwaini similarly, update the uncertainty predictor ue+1(:.-) x fe+1(-) + R+ using the dataset D Puncetainny.
14:
Reset b 6-
17:
else
S1+1 + fe and u4+1 < Uz.
19:
end if
Update the estimate w1+1 wi 1 (f((zi) + (y - fc(z)) &t) ps)
20:
21:
end for
Set à² + (fi(a.) + (vr ft(x,)) ft -WT+1)
23:
Output: -1- conffideeeeee interval Cla [WT+1 + 21 21 /T
The second baseline is the scheme proposed in Zmncc & Candes (2024) for implementing Line 6 which determines the query probability pt. as described in the earlier preliminary section.
in Algorithm| [
6.1 DATASETS AND EXPERIMENTAL SETUP
We compare the algorithms on three real-world datasets. The first dataset concerns the politeness scores of texts based on human annotations, which is available from the works of Ji et al. (2025a) and Gligorié et al. (2025). For each article, there is an associated 21-dimensional feature vector and predicted by ChatGPT. We consider the task of regression for this dataset, where the MI. model trained on the 22-dimensional vector (the 21 features plus the ChatGPT score) to predict the average score of human judgments. Following the suggestion in Zrnic & Candes (2024). an uncertainty estimator ur(-.-) X x fr(-) + R, is used to predict the absolute error [f( yel from He without seeing the label Ve beforehand. This predicted uncertainty is then used the input to their proposed scheme for determining the query probability Pt at round t. The uncertainty estimator is also updated based on the collected samples with queried ground-trute labels once every batch of B labeled samples is collected, as depicted in Algorithm [ For our proposed scheme, we need to construct the approximation oracle T, (ax.). In practice, we implement this by performing a linear regression the squared residual error (f.()., y²)² for the samples in D Punconainty and this estimator is updated regularly after every batch of size B.
The second dataset concerns predicting the ratings of wine reviews, which is available in Ji et al. (025a).. Each review is associated with the price of the wine and four additional binary attributes representing the regions, along with the rating predicted by OpenAl's GPT-4o mini based the reviewers' comments. We also consider the task of regression for this dataset, where a linear regres- sion model is trained the aforementioned covariates to predict the human ratings. The uncertainty
8

---

Published as a conference paper at ICLR 2026
0.46
0.9
0.30
[2C24]
samping
0.27
0.8
1561
confidance
Figure 2: Politeness analysis. Left: Intervals of randomly selected trials. Middle: Average confidence
width across repeated trials vs. sampling budget To. Right: Percentage of trials that cover true mean.
025
0.8
FTRL
[2G24]
unfomm sampling
0.8
1123
1378
1g
Figure 3: Wine review analysis. Left: Intervals of randomly selected trials. Middle: Average confidence width across repeated trials vs. sampling budget TT. Right: Percentage of trials that cover the true mean.
1.0
0.08
0.08
0.8
FTRL
0.02
0.7
[2C24] unfom samplnng
0.6
0.85
0.70
0.72
0.74
1852
2368
3026
3867
4948
1853
2605
$398
4170
4943
contideenc with
To
Figure 4:
Post-election survey. Left: Intervals of randomly selected trials. Middle: Average confidence width across repeated trials vs. sampling budget Th. Right: Percentage of trials that cover the true mean.
predictor and the approximation oracle Pe(x;) have the same form and are updated in the same fash- ion as for the first dataset.
The third dataset is a post-election survey dataset considered Zrnc & Candes 2024). where the original source is from Center (2020). This dataset includes the approval ratings of two politicians, where approval is represented by ys E {0, 1}. Following the experimental setup in Zmic & Candes 2024), the ML model f(() is implemented
XGBoost model. Since the response y/ binary,
task can be treated as classification problem. We hence follow the treatment in Zrnic & Candes 2224)) by using the uncertainty predictor u, (2r, fc))) 2 min (f(x). proposed scheme, where (z(e) is the predicted probability of yt for 2, given by the XGBoost model. On the other hand, the required approximation oracle (Ire) is trained in the same fashion as in the first two tasks.
[z,)] for their
We also conduct experiments synthetic dataset, the details of which can be found in Appendix
6.2 RESULTS
In this subsection, we report the results of the conducted experiments. Figures 4 and Figure (provided in Appendix [ due to space limitations) show the intervals of randomly selected trials, average confidence width, and coverage for each of the datasets considered over 50 trials. Across all four datasets examined, we find that the FTRL policy yields performance comparable to the mixture policy proposed by Zrnic & Candes (2024), in the sense that both result in confidence intervals of similar width, while both outperform the baseline policy. Notably, in two of the datasets, the FTRL policy attains marginally narrower confidence intervals. With respect to coverage of the true mean, all three policies yield high proportion of confidence intervals that successfully include the true value.
Our theoretical analysis and experimental findings consistently indicate that when the query prob- ability P. time is oblivious the current covariates Ir, while still permitted depend on past covariates past uncertainty estimates, the optimal strategy is simply to pt 7 with the sampling budget. This result, implies that constructing an uncertainty predictor, or lever- aging uncertainty estimates in any form, does not appear provide clear advantage for this type
T. in accordance

---

Published as a conference paper at ICLR 2026
of policy. Perhaps unexpectedly, this rules out any benefit from conditioning past covariates or past uncertainty estimates. Furthermore, as our figures illustrate, even when the query probability ignores the current covariates, FTRL, which quickly converges to the constant To and then maintains it, performs on par with the more sophisticated scheme of Zrnic & Candes (2024). which explicitly uses the current features 2 to determine the query probability.
ACKNOWLEDGEMENTS
The authors thank anonymous reviewers for their constructive suggestions, which helped enrich the discussion of related work and clarify the experimental setup. The authors also appreciate the support by NSF CCF-2403392, as well as by the Google Gemma Academic Program and Google Cloud Credits.
REFERENCES
Jacob D Abernethy, Elad Hazan, and Alexander Rakhlin. Interior-point methods for full-information and bandit online learning. IEEE Transactions on Information Theory, 58(7):4164-4175 2012.
Anastasios N Angelopoulos, Stephen Bates, Clara Fannjiang, Michael Jordan, and Tijana Zrnic Prediction-powered inference. Science, 382(6671))669--67777 674, 2023a.
Anastasios Angelopoulos, John C Duchi, and Tijana Zrnic PPI++: Efficient prediction inference. arXiv preprini arXiv:23/1.01455 2023b.
Anastasios Angelopoulos, Jacob Eisenstein, Jonathan Berant, Alekh Agarwal, and Adam Fisch. Cost-optimal active ai model evaluation. arXiv preprint arXiv:2506.0799 2025.
Hilal Asi, Vitaly Feldman, Jelani Nelson, Huy Nguyen, Kunal Talwar, and Samson Zhou Private vector mean estimation in the shuffle model: Optimal rates require many messages. Interna- fional Conference on Machine Learning (ICML), 2024.
Abdellah Aznag, Rachel Cummings, and Adam N Elmachtoub. An active learning framework for multi-group mean estimation. Advances in Neural Information Processing Systems (NeurIPS), 36:32602-323 -26655, 2023.
William H. Beluch, Tim Genewein, Andreas Nurnberger, and Jan M. Kohler. The power of en- sembles for active learning in image classification. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9368-9377, 2018. doi: 10. 109/CVPR.2018..00
Ran Ben-Basat, Amit Portnoy, Gil Einziger, Yaniv Ben-Itzhak, Michael Mitzenmacher, et al. Ac- celerating federated learning with quick distributed mean estimation. In International Conference on Machine Learning (ICML), 2024.
Pierre Boyeau, Anastasios N. Angelopoulos, Tianle Li. Nir Yosef, Jitendra Malik, and Michael I. Jordan. Autoeval done right: Using synthetic data for model evaluation. In International Confer- on Machine Learning (ICML), 2025.
Pew Research Center. American trends panel (ATP) Wave 79, 2020. URL https: pewresearch wwwporrcieeeeeeeooooooooeeeeooee wwgg/scienee//aate....e satpee////aattteeeeeeeiiie -ateerrr-iia-tteet-----eeeeee -waeeelwwvee9999 Accessed: 2025-09-17.
Ivi Chatzi, Eleni Straitouri, Suhas Thejaswi, and Manuel GodezzRodriguez. Prediction powwered ranking of large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Yiding Chen, Jerry Zhu, and Kirthevasan Kandasamy. Mechanism design for collaborative normal mean estimation. Advances in Neural Information Processing Systems (NeurIPS), 36:49365- 49402, 2023.
Yu Cheng, Ilias Diakonikolas, Rong Ge, and Mahdi Soltanolkotabi. digh-dimensional robust mean estimation via gradient descent. In International Conference on Machine Learning (ICML), pp- 1768-1778, 2020.
10

---

Published as a conference paper at ICLR 2026
Stefano Cortinovis and Francois Caron. Ab-ppi: Frequentist, assisted by bayes, prediction-powwere inference. In International Conference on Machine Learning (ICML), 2025.
Daniel Csillag, Claudio Jose Struchiner, and Guilherme Tegoni Goedert. Prediction-powered e- values. In International Conference on Machine Learning (ICML), 2025.
Trung Dang, Jasper Lee, Maoyuan' Raymond" Song, and Paul Valiant. Optimality in mean estima- tion: beyond worst-case, beyond sub-gaussian, and beyond + moments. Advances in Neural Information Processing Systems (NeurIPS), 36:4150-4176, 2023.
Sanjoy Dasgupta. Two faces of active learning. Theoretical Computer Science, 412((19:1777-1781 2011. ISSN 0304-3975, doi: https//db..ogg///...06.111112020 URL https: sciencedirect com/
Algorithmic
Learning Theory 2009).
Jyotishka Datta and Nicholas G Polson. Prediction-powered inference with inverse probability weighting. arXiv preprint arXiv:2508. 10/49, 2025.
Piersilvio De Bartolomeis, Javier Abad, Guanbo Wang, Konstantin Donhauser, Raymond M Duch, Fanny Yang, and Issa Dahabreh. Efficient randomized experiments using foundation models. arXiv preprint arXiv:2502.04200 04262, 2025.
llias Diakonikolas, Giannis Iakovidis, Daniel Kane, and Thanasis Pittas. Efficient multivariate ro- bust mean estimation under mean-shift contamination. In International Conference Machine Learning (ICML), 2025.
Florian E Dorner, Vivian Yvonne Nastl, and Moritz Hardt. Limits to scalable evaluation at the frontier: Llm as judge won't 't beat twice the data, In International Conference on Representation Learning (ICLR), 2025.
Melanie Ducoffe and Frederic Precioso. Adversarial active learning for deep networks: margin based approach, 2018.
09841
Naoki Egami, Musashi Hinck. Brandon Stewart, and Hanying Wei. Using imperfect surrogates for downstream inference: Design-based supervised learning for social science applications of large language models. Advances in Neural Information Processing Systems (NeurIPS), 36:68589- 68601, 2023.
Benjamin Eyre and David Madras. Regression for the mean: Auto-evaluation and inference with few labels through post-hoc regression. In International Conference on Machine Learning (ICML), 2025.
Adam Fisch, Joshua Maynez, R. Alex Hofer, Bhuwan Dhingra, Amir Globerson, and William W. Cohen. Stratified prediction-powered inference for effective hybrid evaluation of language mod- cls. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024.
David A Freedman. On tail probabilities for martingales. the Annals of Probability, pp. 100-118. 1975.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Con- ference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp- 1192. PMLR, 06-11 Aug 2017. URL hktpss:/pxxoeediingg www.:ssss..........ommmm gall7a.htme
Feng Gan, Wanfeng Liang, and Changliang Zou. Prediction de-correlated inference: A safe ap- proach for post-prediction inference. Australian & New Zealand Journal of Statistics, 66(4): 417-440, 2024.
Kristina Gligorié, Tijana Zmnce Cinoo Lee, Emmanuel Candes, and Dan Jurafsky. Can unconfident Ilm annotations be used for confident conclusions? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 3514-3533, 2025.
11

---

Published as a conference paper at ICLR 2026
Steve Hanneke. Theory of disagreement-bassee active learning. Found. Trends Mach. Learn., 7(2-3); 131-309, June 2014. ISSN 1935-8237. doi: 10.1561/2200000003. URL / 10.1561/ 20001//200000000
Wenlong Ji, Lihua Lei, and Tijana Zrnic. Predictions as surrogates: Revisiting surrogate outcomes in the age of ai. arXiv preprint XXiv:25011009731, 2025a.
Wenlong Ji, Yihan Pan, Ruihao Zhu, and Lihua Lei. Multi-armed bandits with machine learning- generated surrogate rewards. arXiv preprint arXiv:2506. 116225006666688 16658, 2025b.
Ajay J. Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. In 2009 JEEE Conference on Computer Vision and Pattern Recognition, pp. 2372- 2379, 2009. doi: 10.1 (tt////VV...0995520000
Daniel Kane, Ilias Diakonikolas, Hanshen Xiao, and Sihan Liu. Online robust mean estimation. In Proceedings of the 2024 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp- 3197-3235. SIAM, 2024.
Sida Li and Nikolaos Ignatiadis. Prediction-powered adaptive shrinkage estimation. In International Conference on Machine Learning (ICML), 2025.
Xiang Li, Yunai Li, Huiying Zhong. Lihua Lei, and Zhun Deng. Statistical inference under perfor- mativity. arXiv preprint arXiv:2505. 18493, 2025.
Haipeng Luo. Introduction Online Learning CSCI 699. Lecture Note, 2017. URL https: http://ncccoooree/////////////// wttpee//eccccceeeeeeeeeeeeee
//haipeng- luo. net /
pdf
Pranav Mani, Peng Xu, Zachary C Lipton, and Michael Oberst. No free lunch: Non-asymptotic
analysis of rrediction-powered inference. arXiv preprint arXiv:2505.20178, 2025.
Jiacheng Miao and Qiongshi Lu. Task-agnostic machine-learming-ssiinn inference. Advances in Neural Information Processing Systems (NeurIPS), 37:106162-106188 2024.
Jiacheng Miao, Xinran Miao, Yixuan Wu, Jiwei Zhao, and Qiongshi Lu. Assumption-lean and data-adaptive post-prediction inference. arXiv preprint arXiv:2311.14220 2023.
Gleb Novikov, David Steurer, and Stefan Tiegel. Robust mean estimation without moments for symmetric distributions. Advances in Neural Information Processing Systems (NeurIPS), 36: 34371-34409, 2023.
Francesco Orabona. A modern introduction to online learning. arXiv preprint arXiv:19/2.13213. 2019.
Art B. Owen. Monte Carlo theory, methods and http:://atttwwee.sss.oo mc/ 2013.
Sangwoo Park, Matteo Zecchin, and Osvaldo Simeone. Adaptive prediction- autoeval with reliability and efficiency guarantees. arXiv preprint arXiv:2505./8659, 2025.
Ankit Pensia. A subquadratic time algorithm for robust sparse mean estimation. In International Conference on Machine Learning (ICML), 2024.
Pierre-Emmanun Poulet, Maylis Tran, Sophie Tezenas du Montcel, Bruno Dubois, Stanley Durrle- man, and Bruno Jedynak. Prediction-powercl inference for clinical trials. medRxiv. 2025.
Alexander Rakhlin, Ohad Shamir, and Karthik Sridharan. Making gradient descent optimal for
strongly convex stochastic optimization. In International Conference on Machine Learning (ICML), 2012.
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Yao Huang, Zhihui Li, Brij B. Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning, 2021. URL https: //trx//axvvvvv///////// org/ abs/ 2009 00236
12

---

Published as a conference paper at ICLR 2026
Hugo Schmutz, Olivier Humbert, and supervised learning via debiasing. In Proceedings of the International Conference on Learning Representations (ICLR), 2023.
Mattei. fontt the unlabelled: Safe semi-
Greg Schohn and David Cohn. Less is more: Active learning with support vector machines. In Pat Langley (ed.), Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), Stanford University. Stanford, CA. USA, June 29 July 2. 2000, pp. 839-846.
Morgan Kaufmann, 2000.
Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, Uni- versity of Wisconsin-Madison, 2009. URL http::/axon cs byu. wed.mmartinee.... classes/78PPaperss/esss
Shai Shalev-Shwartz al. Online learning and online convex optimization. Foundations and
Trends@ in Machine Learning, 4(2):107-19,, 2012.
Simon Tong and Daphne Koller. Support vector machine active learning with application sto text classification. In Pat Langley (ed.), Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), Stanford University, Stanford, CA, USA, June 29. July 2, 2000, pp. 999-1006. Morgan Kaufmann, 2000.
Gokhan Tur, Dilek Hakkani-Tür, and Robert E. Schapire. Combining active and semi-supervised learning for spoken language understanding. Speech Communicationn 45(2):171-188 2005. ISSN 0167-6393. doi: htt.://wwe.......ao...oooaoaaa/.
URL https: /www.
sciencedirect cwm/ httpt//cc///ceeee//aariiccccaaaa
Jun-Kun Wang, Jacob Abernethy, and Kfir Y Levy. No-regret dynamics in the fenchel game: A unified framework for algorithmic convex optimization. Mathematical Programming, 205(1): 268-268,20 2024.
Yuxin Wang, Maresa Schröder, Dennis Frauen, Jonas Schweisthal, Konstantin Hess, and Stefan Feuerriegel. Constructing confidence intervals for average treatment effects from confounded and unconfounded data. In International Conference on Representation Learning (ICLR), 2025.
lan Waudhy-Smith and Aaditya Ramdas. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(1):1-27, 2024.
Zichun Xu, Daniela Witten, and Ali Shojaie. A unified framework for semiparametrically efficient semi-supervised learning. arXiv preprint 17Xiv:2502.17741 2025.
Tijana Zrnic. A note the predictioon-ooweeedboooooo
arXiv e-prints, arXiv:2405.18379 2024.
URL htlp:::://zxii..or///b////////
Tijana Zmic and Emmanuel Candès. cetas-ppeedictoonpoweeeee inference. Proceedings of the Na-
tional Academy of Sciences, 120(41)ee332288312,, 2023.
Tijana Zrnic and Emmanuel Candes. Active statistical inference. In International Conference on PMLR, 2024.
Machine Learning, pp.
13

---

Published as a conference paper at ICLR 2026
A EXPERIMENTS ON THE EFFECT OF THE MIXING PARAMETER
In this section, we report the results of experiments on tuning the mixing constant in the mixture policy of Zrnic & Candes 2024), evaluated across four different datasets. detailed description of these datasets is provided in Section 6 and Appendix E
1.27
Estimator
Estimator
[ZC24] (mixing-0005)
[ZC24] (mixxng-00055)
0.05
[2024] (mixing=0.1)
0.88
[2024] (mixing=::11)
[ZC24] (mixing=0.5)
[2C24 (mixing-0.5)
[2C24] (mixing-0.8)
[2C24] (miking-0.8)
Interval
0.04
[ZC24] (mixing-1.0)
Interval
[2C24]
(iixiing110)
0.03
0.39
0.02
0.26
1855
2370
3029
3871
4946
819
1046
1337
1709
2184
To
Te
(a) Post-election survey dataset.
(b) Politeness score dataset.
0.62
0.42
Eslimator
Estimator
[2C24] (mixing=0.05)
[2C24] (mixing=0.055)
0.48
[2C24] (mixing=0.1)
0.25
[2C24] fmixing=0.1)
Interval width
[2C24] (mixing=0.5)
Interval width
[2C24] (mixing=0.5)
[Z024] (mixing=0..8)
[ZC24] (mixing=0.8)
0.33
(mixing=110))
0.15
[ZC24] (mixing=1.0)
0.24
0.09
0.18
0.05
1000
To
To
(c) Wine review dataset.
(d) Synthetic dataset.
Figure 5: Interval width vs. the sampling budget parameter T. for different values of the mixing parameter of the query probability scheme in Zrnc & Candes (2024). Averaged over 50 repeated runs.
B EXTENDED RELATED WORK
Active Statistical Inference. The ideas this work are motivated by the recent approach of Active Statistical Inference (Zmic & Candes 2024). Extending this Cramework,/Anglloogg
et al. (2025) propose a method that optimizes the sampling rate between gold-standard and pseudo-labels rather than relying fixed label budget, and derive improved active sampling policy. A recent work by Gligorić et al. (2025) utilizes L.M verbalized confidence scores to guide their sampling policy and subsequently performs active inference by combining the LLM and human annotations.
Prediction Powered Inference. The Active Statistical Inference framework is grounded in the idea of Prediction Powered Inference (PPI) AAgeloopoloss al. (2023a). which differs from the for- mer in that it assumes the availability of a small, pre-labeled dataset. The work of AAgelopoolos al. (2023b) introduces PPI++, which improves the computational efficiency of PPI by addressing the intractability of the original confidence interval construction and using a tuning parameter to control the influence of the model predictions based on their quality. Subsequent works (Dorner et al. 2025 Mani et al. 2025) analyze the critical role of the correlation between the gold-standard and model-generated labels for the performance of PPI. Focusing on the few-label regime, [yrre & Madras (2025) argue that the PPI++ framework may lead to significantly biased estimator that is less efficient than classical inference by establishing its connection to univariate ordinary least squares regression. Datta & Polson 2025) examine the of an inverse probability weighted (IPW) bias correctitet term in the PPI mean estimator, inspired by classical Horvitz- Horvitz-Thompson and Hájek estimators. The work of De Bartolomeis et al. (2025) establishes connection of PPI++ with the
14

---

Published as a conference paper at ICLR 2026
augmented inverse probability weighting (AIPW) estimator and propose extension, which allows
utilizing predictions from multiple foundation models. To address cases where the model accuracy
varies across subdomains, Fisch et al. (2024) apply a stratification approach to PPI. Zrnie (2024)
explore a bootstrap-based PPI variation to tackle arbitrary estimation problems. Xu et al. (2025)
improve the PPI framework by proposing safe PPI estimator that is always more efficient than
the initial supervised estimator and can be used for arbitrary inferential problems. Li et al. (2025)
generalize the ideas of PPI a dynamic performative setting and show improved confidence regions in the task of performative prediction. In a semi- superviie context, Zrnic & Candès (2023) propose a method of using the labeled datapoints for cross-fitting and using the fitted models to compute the desired estimator. CCrtinovis& & Caron (225) extend PPI by applying work that uses prior knowledge on the accuracy of the model predictions. In the case of compound estimation settings. [Li & Ignatiadis (2025) adopt an approach that combines PPI with empirical Bayes shrinkage to correct noisy predictions within each problem and subsequently uses these as shrinkage target. An interesting work by Csillag et al. (2025) presents PPI framework based on e-values. Other applications of the PPI framework include LLM-assisted rank-set construction (Chatzi et al.] 2024), average treatment effects from multiple datasets (Wang et al. (202), clinic trial outcomes (Poulet et al. 22025. evaluating the accuracy of machine learning systems (Boyeau et al.,
Bayes-assisted frame-
2025 Park et al..| 2025). ) machine learning generated surrogate rewards for multi-armed bandits (Ji
et al. 2025b). A few other works have explored alternative machine-learning assisted estimators,
e.g. Schmutz et al. 22033 Egami et al. (2023): Miao et al. (2023): ; Miao & Lu (2024); )
Gan et al.
(2024).
Active Learning. Similar to the Active Statistical Inference protocol is active learning (Settles)
2009
Dasgupta 2011 Hanneke 2014)) which frames learning as process in which a machine
learning model selectively queries unlabeled instances to be labeled by an oracle. In contrast to Active Statistical Inference, which focuses on enhancing statistical inference, the goal of active learning is to improve the predictive power of the machine learning model through the strategic use of labeled data. One of the main approaches in active learning, and the one most closely aligned with
active estimation, is the uncertainty sampling strategy (Schohn & Cohn 22000; Tong & Koller 2000)
Tur et al.
2005 Joshi et al., 2009 Gal et al.) 2017 Ducoffe & Precioso] 2018 |eeluch et al. 2018 Ren et al.
2021 where the model aims to query the labels of the most informative data points, i.e.,
the ones which the model is most uncertain about.
Importance Sampling. The idea of prioritizing the most influencial samples is also closely re- lated to adaptive importance sampling (Owen 2013), which is sequential scheme that updates the proposal distribution by learning from previously sampled values in order to better approximate some property of a target distribution. Active mean estimation is analogous to adaptive importance sampling, in that it adaptively selects samples that contribute most to reducing the variance of the estimator based on previously observed data to improve the accuracy of the mean estimator.
Nom-Asymptotic Results in Zrnic & Candes (2024). We acknowledge that in Appendix C' in Zmic & Candes (2024). the authors consider incorporating the notion of actively querying the ground truth into the technique for estimating means of bounded random variables proposed by Waudby-Smith & Ramdas (2024). Waudby-Smith & Ramdas (2024) leverage the duality between sequential hypothesis testing and the construction of confidence intervals. More concretely, their method reduces the task of constructing a confidence interval for the mean to a potentially in- finite number of hypothesis testing problems. Each hypothesis testing problem corresponds to whether the observed samples have a population mean equal to a specific value. Hence, for a continuous random variable, this corresponds a discretization is used. The confidence interval C, at time ł then consists of those hypothe- sized mean values that have not been rejected based on the data observed up to time I, i.e.. C, : (v H is not rejected based on observations up to time t). where H7()) denotes the hy- pothesis that the population mean is V. While Zmnc & Candes (2024) provide the valuable idea of integrating these techniques and also provide some simulation results, the specific step-by-step algorithmic details and theoretical guarantees for active sequential mean estimation remain to be explicitly elaborated.
infinite number of hypotheses. In practice,
15

---

Published as a conference paper at ICLR 2026
C PROOFS OF THE THEORETICAL RESULTS IN SECTION 4
Theorem 1. Fix time horizon T 4. Assume each random variable gt is bounded, i.e., 1 G for constant G >> 0. Denote of == E [gr My) [1-1 the conditional variance, where Fi-1 is
the filtration up to t - 1. Then, for any 8 E 1/e), with probability at least 1 5, Vt [T]
2 max 2/SL, (G lunDy log (oor(T)
log
Tog(T)
+ (1 (1-T)
[1++1 - Nyl
7) 1myl
T
where St = -i o.
Proof. The difference of wa My can be decomposed into two terms. Specifically, we have
W1+1 My
9e My
Σ ==1
(g.
t pby --11-) My T
T
T
Let us analyze the second-toolast-term Ex=1 (gx (yy on We note that (g+ - (2)s21 forms
martingale difference sequence, i.e., LB[g My
F 8-1 U, where Fa-1 encodes all information
up to time s 1. Furthermore, Vs: I9 Uy S payl Also, the conditional variance is
Var (ge Hy1Fa-1) = E [(.. My 11
By Freedman's 's inequality (Lemma] D. have, with probability 1- 8,
Vt [2] Σ (g. mg) S max 2,
Σ oz (G+
(ogglooog))))))
(log())(())/))
(8)
Combining 7 and (8) we obtain the following holds simultaneously at all E [T], with probability
least 1 d:
|e++1 Myl
2 max { 2 L2 G2. (G ( (og(lo((T)/0)
Vlog(log(T)/O)
+ +1-
+ Imyl-
T
D PROOFS OF THE THEORETICAL RESULTS IN SECTION 5
Lemma 2. The conditional variance has the following decomposition:
E [(g [[
1
E fe(az)2 Fi-1 +E (v
f((x)))
Fi-1 + 2E
fe(æ)(u fe(xz)) Fi-1
Te((xe)
Furthermore, assume that the query policy at time is Fe-i-measuraale, i.e., there exists a random variable pt. E [0, 1), measurable with respect to Fr-1, such that Te Tx(x)) Pt- Then, we have
E
(ye - fx(xz))2
Fe
E
(ye
fe(2te)) Fr-1
Pt
16

---

Published as a conference paper at ICLR 2026
Proof.
E [9t B2y9 F
= E
=B Fr-1 y
&
&
E f((aa)) 2
F
E
(yl f((r)))
Fi-1
2B
f((x))(e
f((xs)(ve - f((())
Fi-1
x xx(e
=x(x²)
1
E
fe(2x))
Fi-1
(ye fe(x))))
Ft-1
+ 2E
fi(x))(e-- Fe-1
Tee (x4e)
where the first equality follows from that E
Fc-1]= = py.
Lemma 5. Assume that the range of oracle's output is bounded, i.e. Vt: (x(x)) S for constant
Then, Vt [ 1
8a 34.
B >
Proof.
1
D
(2)
B
B
11:
p
Pt
34
where the first inequality follows from that Vt (xxe)
I B, and the last inequality that
Vt Pt 2 B.
Theorem 2. Assume that there is an oracle that outputs (x(x,) each such that 1 (,(r)) S
E (ye- - f((x))) Fi-1
S (axe) for some constant co. > and that Vt ( s B. Set
g²
the parameter
Using the query policy (4), we have that, with probability at least 6,
VT B.
Vt [T]:
max 2/ Mr, (G + 1oulD log (IoR(T) }
/10g [ox(T)
+ (1- (1-1) Myl
We+1 - Myl 1
T
B for any 3 E T) and E (0, 1), where V, S coc1o13 + 2co JT B
+
cl .)) --iiien R(p))TBB
Proof.
By Lemma 2 and the constraint that the query probability Pt is fully determined in Ft-1,
have
of
=
E
2
F
+ E
(yi
f((x)))
Fi-1 + 2E
f((z))() f((z())
Fo-1 ply
Pt
② (((1))
<
E
fa( )
Fi-1
+ co
+ 21
fe(xx)(v: -((()()))))) Fi-1
Pt.
E
12(2.)2
Fi-1
+ co
(1(())
+
De((t)
+ 2F De(ar)
fe(mx)(1a (aloe))
Piit
Pt
Pi.
<
E
(xx)))²
Fi-1
+ CoC1
(y:- [e(xx))2
Fi-1
+ co
(x(xe)
2e(x1)
Pia
Pt
Pia
+ 2TE
f((xx)(14 fe(or))
Fr-1
1by
v((xz)
te(xe)
I S001000 +
Pt
Piae
17

---

Published as a conference paper at ICLR 2026
where the last inequality uses the fact that Cydı 2 1. We note that the above inequality holds for all t. Hence, we have
Σ o S 1ic1012 (gRegret,(Pi.))
by summing up the above inequality for each round. To proceed, we use the regret bound that we have from Lemma
R(pi.)) minpex R(p)
Regret, (pi.t)
S
2
==1
(8)
B²
R(pia) minpex R(p)
S
2yt
+
8
?
(8)
1
B
(R(vi.) minpex R(p)) VTB
+
(10)
VT
where (i) is from Lemma 5 and (ii) is by the choice of y L 0²
VT B. Combining (9) and (0). we have
t B (R(pia) minpex R(p)) VTB
Σ 0 < 1001011 200
VT B
32
Using the above bound together with Theorem T leads to the result. This completes the proof.
E ADDITIONAL EXPERIMENTAL DETAILS
E. SYNTHETIC DATASET
The fourth dataset used in the experiments is synthetic dataset that is generated for binary clas- sification according to a logistic model. More specifically, the covariates 2; E IR are indepen- dently drawn from multivariate normal distribution with zero where d = 10. The true parameter vector w" is sampled independently from a normal distribution with zero mean and covariance 0.5 la. Gaussian noise E, (O, 10-5) 5) is added each x, w" to produce the logits. The correspondig binary labels E {0. 1} are then generated according to ~ Bernoulli (a (xr "" + <+)). where ((=) = 1/11+=-) denotes the sigmoid function. The MI. model f((-) is implemented as a logistic regression model whose uncertainty predictor is estimated in the way in the post-election survey dataset, i.e., Ue (#x, fa(-)) 2 min ( (fr)), 1- (xx)) using the predicted probabilities of f.(.). A linear regression model is trained to predict the approx- imation oracle (2,,) in the previous tasks.
and identity covariance Ta.
Figure shows the experimental results on the synthetic dataset.
0.14
1.0
0.11
0.9
0.28
[2C24]
0.7
unifomm sampling
0.06
0.35
0.60
0.45
0.50
0.55
145
157
240
305
391
147
205
253
330
382
confideerss wkith
To
Figure 6: Synthetic dataset. Left: Intervals of randomly selected trials. Middle: Average confidence width across repeated trials sampling budget Th. Right: Percentage of trials that cover the true mean.
E.2 EXPERIMENTAL SETUP
All experiments were repeated over 50 trials, and reported results correspond to the averages across these trials. At the start of each trial, the data points randomly permuted. For each experiment,
18

---

Published as a conference paper at ICLR 2026
the budget T3 was varied over five uniformly spaced values between 15% and 40% of the total num- ber of data points T. The interval width and coverage plots were obtained by linearly interpolating between the values at these grid points.
In the experimental setup, the ML model ft()), uncertainty estimator u4(: -), and oracle 2, were updated periodically after observing batch of B data points. For the first two datasets (politeness score and wine review analysis), the estimators were updated N = 50 times, while for the last two datasets (post-election survey and synthetic data), they updated N 10 times. Accordingly, the batch size set to B round
For the FTRL plicy, we set the upper bound hyperparameter theoretical analysis to ensure that the query probability satisfies the sampling constraint. The lower bound hyperparameter was set to = > to prevent the sampling probability from becoming too small, and thereby encouraging exploration, while still remaining sufficiently below so that the resulting sampling interval non-trivias and allows the algorithm to adjust the sampling probability over time. Furthermore, the hyperparameter
Th in accordance with our T
V in line with common practice in online learning, to guarantee sublinear regret growth with respect T. which is necessary for achieving no-regret performance. For the policy of /Znnc & Candes (2024), we set the mixing hyperparameter to À 0.5, which is the recommended value used in their experimental setup, to enable a comparison with their proposed policy.
chosen
19


Reviews:
[{"review_id": "qEZ2fFwGic", "reviewer": "Reviewer_2b77", "summary": "This paper revisits the active inference procedure proposed by Zrnic & Candes, 2024 [ZC24], where the goal is to better estimate the mean of a target variable with a query policy that is a mixing of uniform sampling and uncertainty-based sampling. This investigation is motivated by the empirical observation that uniform sampling yields tighter or comparable confidence intervals to uncertainty-based active sampling. A non-asymptotic upper bound on the L1 error of the active sequential prediction-powered mean estimation is provided, showing a rate of $O(1/\\sqrt{t})$ at large sample sizes $t$. Then the paper demonstrates that a query policy independent of the current covariate and designed in a non-regret fashion converges to uniform sampling. This policy leads to similar confidence interval widths to the method proposed by Zrnic & Candes, 2024, in several real data experiments, where they both perform significantly better than uniform sampling.", "questions": "- Does the method [ZC24] with mixing $\\lambda=1$ correspond to the uniform sampling baseline in (6)? If so, why does the purple curve in Figure 1 differ significantly from the green one in Figure 4, while they both illustrate the results of uniform sampling on the post-election survey data set. Moreover, according to Figure 1, the method [ZC24] with a mixing policy of uniform and uncertainty-based samplings gives wider or comparable confidence intervals w.r.t. the uniform sampling baseline, which is the motivation of this article. However in Figures 2-4, the method [ZC24] yields consistently narrower confidence intervals than uniform sampling. Isn't there a contradiction?\n\n- How are the hyperparameters $\\gamma,\\tau,\\beta$ chosen for the non-regret method FTRL in the experiments of Section 6? Same question for the mixing parameter $\\lambda$ for the method [ZC24].", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The analysis is motivated by an intriguing empirical observation that the active inference method [ZC24] performs comparably to uniform sampling in terms of the confidence interval width.\n\n- It is an interesting conclusion that a query polity independent of the current covariate can produce similar confidence intervals to the active inference method [ZC24].", "weaknesses": "- There seem to be some contradictions in the empirical observations (See Questions).\n\n- The motivation for looking at the non-regret approach is not sufficiently clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rYf3OL7K4H", "reviewer": "Reviewer_4Zjz", "summary": "This paper revisits the active statistical inference methods proposed by Zrnic & Candes (2024)--where that previous paper presents methods for prediction powered inference (Angelopoulos et al 2023a) under active data selection--and the current paper claims the following contributions: (1) Empirically exploring the effect of different values of the mixture weight (ie, a mixture weight on the active data selection policy) in Zrnic & Candes (2024) and finding that the sharpest intervals tended to occur with more weight on the constant probability; (2) they present non-asymptotic analysis of the mean estimator with a data-dependent bound; (3) analysis on convergence when a no-regret learning approach is used; (4) simulation experiments to support the theoretical results.", "questions": "Can the authors explain how their non-asymptotic analysis and results relate to Appendix C in Zrnic & Candes (2024)? I think that such discussion should also be added to the paper.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper studies an important problem of how to do valid statistical inference under active data collection, building on prior work of active statistical inference (Zrnic & Candes 2024). The observations about the role of the mixture parameter, the non-asymptotic analysis, and the analysis on convergence seem like they would be of interest to the community. The writing is overall clear and the analysis seems sound. Overall I enjoyed reading the paper.", "weaknesses": "**Adding acknowledgement & discussion of prior non-asymptotic version of Zrnic & Candes (2024):** While the paper does have an overall fairly thorough discussion of related work and its relationship to the prior work of Zrnic & Candes (2024), there is one key aspect of its discussion that seems like it needs to be updated, to acknowledge that Zrnic & Candes (2024) do have some consideration of non-asymptotic versions of their method, which is the main claimed contribution of the current paper. That is, at the top of Page 2 the current paper states “The non-asymptotic analysis of the estimator was missing in the prior work [Zrnic & Candes (2024)]”; however, Appendix C in Zrnic & Candes (2024) is titled “Non-asymptotic results” and it both explains how its main results could extend directly to non-asymptotic analogs, as well as presents some experimental results of non-asymptotic variants. I understand that Zrnic & Candes (2024) do not comprehensively or explicitly write out the formal analysis for the non-asymptotic variants, but given that this is the core claimed contribution of the current paper, I think that the current paper needs to be revised to acknowledge this prior consideration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4ce98Kssg8", "reviewer": "Reviewer_nFbY", "summary": "The paper revisits the problem of active sequential prediction-powered mean estimation — estimating the mean of a label $y$\n$y$ when each observation’s label may or may not be queried. If the label is not queried (to save cost), a machine-learning model’s prediction is used instead and there is finite budget $T_b \\ll T$ of queries allowed. Here are the follewing finding.\n\n1) Based on Freedman's theorem (a version of Bernstein concentration inequality) the author provide non-asymptotic confidence intervals\nand data-dependent confidence bound valid at any step.\n\n2) Reinterpret query-probability selection as a no-regret online learning problem (via the Follow-the-Regularized-Leader algorithm).\n\n3) The discovery  that under sequential settings, uniform querying (without conditioning on covariates) may be both theoretically justified and empirically optimal.", "questions": "Have you considered extending your framework to other risk functionals (e.g., variance or quantile estimation) beyond the mean, and would the same uniform-querying phenomenon persist?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. Rigorous theoretical analysis based on martingale difference concentration inequality.\n\n2. New connection with no-regret online learning problem which suggest practical algorithm\n\n3. Clear empirical insights that uniform querying performs comparably (or better) than uncertainty-based querying which may simulate further work.  Since this optimal strategy is also very simple and easy to implement this is a significant finding.\n\n4. Very well written paper combing theory, algorithmic development and experimental findings.\n\nOverall the paper looks very strong to the reviewer (who is not an expert in this field, especially online learning).", "weaknesses": "No obvious one.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mWt2wdsX6z", "reviewer": "Reviewer_Zfwx", "summary": "This paper revisits the problem of active sequential prediction-powered mean estimation, where a learner estimates the mean of an unknown label distribution under a limited labeling budget by adaptively deciding whether to query true labels or rely on model predictions. Building on prior work (Zrnic & Candès, 2024), the authors make an intriguing empirical observation that confidence intervals are often narrowest when the query policy is nearly uniform, ignoring model uncertainty. Motivated by this, they develop a non-asymptotic analysis providing time-uniform, data-dependent confidence bounds for the estimator and reformulate the label-querying process as an online convex optimization problem solved via the Follow-the-Regularized-Leader (FTRL) algorithm. Their theoretical results show that the FTRL policy converges to the maximal allowable query rate and that a uniform querying strategy is asymptotically optimal. Experiments across real and synthetic datasets confirm that this simple strategy matches or outperforms uncertainty-based methods in confidence width and coverage, revealing that uncertainty-aware sampling may offer limited benefits for sequential mean estimation.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is original in revisiting active sequential mean estimation through a new theoretical lens, uncovering an unexpected empirical pattern that challenges prior assumptions about the value of model uncertainty. Its non-asymptotic analysis fills a key theoretical gap left by earlier asymptotic-only results, providing rigorous, time-uniform, and data-dependent confidence bounds that enhance understanding of estimator behavior. The introduction of a no-regret online learning formulation using FTRL to govern the query policy is conceptually elegant and technically sound, connecting active inference with online convex optimization theory. The experimental validation is thorough and consistent with the theoretical findings, strengthening the overall credibility of the claims. The paper is also clearly written and well-organized, effectively bridging empirical observation, theoretical reasoning, and practical implications—making it a significant and high-quality contribution to the fields of active statistical inference and prediction-powered estimation.", "weaknesses": "While the paper is theoretically well-grounded, several aspects could be improved to strengthen its impact. First, the novelty is somewhat incremental, as the core setup largely extends the framework of Zrnic & Candès (2024) by introducing a non-asymptotic analysis and reformulating the query policy using standard online learning tools (FTRL). The contribution could be made more compelling by deeper theoretical insight into why uncertainty-based sampling fails in practice, beyond the observed convergence of the no-regret policy. Second, the experimental scope is limited, the datasets used (text politeness, wine review, post-election survey, and one synthetic set) are small-scale and do not test generalization to high-dimensional or noisy real-world domains where uncertainty estimation might matter more. Adding experiments in such settings or sensitivity analyses for different noise structures would better validate the theoretical claims. Third, the discussion of assumptions and limitations is brief; for example, the oracle approximation and boundedness assumptions may not hold in practical scenarios, and their empirical impact is not fully analyzed. Finally, connections to related frameworks such as active learning or adaptive importance sampling[1][2] could be discussed more thoroughly to position the work within the broader literature. Addressing these points would make the contribution more robust and its insights more broadly applicable.\n\n[1] Pareto smoothed importance sampling, jmlr 2025\n\n[2] Active Advantage-Aligned Online Reinforcement Learning with Offline Data, arxiv 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)