Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation
Authors: Zhenghao Zeng
David Arbour
Avi Feller
Ishita Dasgupta
Atanu R. Sinha
Edward Kennedy
Venue: NeurIPS 2025 poster ()
Keywords: Missing data
cluster dependence
doubly robust estimation
causal inference
evaluation
human annotation
Primary area: probabilistic_methods

Abstract: Human annotations play a crucial role in evaluating the performance of GenAI models. Two common challenges in practice, however, are missing annotations (the response variable of interest) and cluster dependence among human-AI interactions (e.g., questions asked by the same user may be highly correlated). Reliable inference must address both issues to achieve unbiased estimation and appropriately quantify uncertainty when estimating average scores from human annotations. In this paper, we analyze the doubly robust estimator, a widely used method in missing data analysis and causal inference, applied to this setting and establish novel theoretical properties under cluster dependence. We further illustrate our findings through simulations and a real-world conversation quality dataset. Our theoretical and empirical results underscore the importance of incorporating cluster dependence in missing response problems to perform valid statistical inference.

Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation
Zhenghao Zeng Stanford University zhzeng@stanf ord. edu
David Arbour Adobe Research arbour@adobe. com
Avi Feller University of California, Berkeley afeller@berkeley. edu
Ishita Dasgupta
Atanu R Sinha Adobe Research atr@adobe.. com
Edward H. Kennedy Carnegie Mellon University edward@stat. cmu. edu
Adobe Research i idasgupt@adobe.cm com
Abstract
Human annotations play crucial role in evaluating the performance of GenAl models. Two common challenges in practice, however, are missing annotations (the response variable of interest) and cluster dependence among human-Al inter- actions (e.g., questions asked by the same user may be highly correlated). Reliable inference must address both issues achieve unbiased estimation and appropri- ately quantify uncertainty when estimating average scores from human annotations. In this paper, analyze the doubly robust estimator, widely used method in missing data analysis and causal inference, applied to this setting and establish novel theoretical properties under cluster dependence. We further illustrate findings through simulations and a real-world conversation quality dataset. Our theoretical and empirical results underscore the importance of incorporating cluster dependence in missing response problems to perform valid statistical inference.
Introduction
Missing response/outcome variables are common empirical research and present many challenges to data analysis and interpretation. Such missingness can occur for various reasons, including nonresponse in surveys (Hansen and Hurwitz 1946 Chen and Haziza 2019), dropout in longitudinal studies (Hogan et al., 2004), or data entry errors (Bound et al., 2001; [chennach]] 2016). If not properly addressed, missingness can lead to biased estimates, reduced statistical power, and invalid conclusions. Researchers have developed methods that leverage observed data estimate missing values under assumptions about the missingness mechanism, typically assuming i.i.d sampling. Examples common in causal inference include outcome modeling, weightingn and combinations of the two (Robins et al. 1994 Little and Rubin 2019).
Clustered data, commonly encountered in fields such education (Lüdtke et al. 2011 healthcare (Austin and Merlo, 2017), and the social sciences (McNeish and Stapleton 2016), refers to data in which observations are naturally grouped into clusters or hierarchies. Typical examples include students nested within schools or patients nested within hospitals, where clusters (e.g., schools or hospitals) sampled first, followed by individuals within those clusters. Another form of clustered sampling arises in settings with repeated measurements on the same individuals. For instance, in the evaluation of large language models (LLMs), users often provide multi-turn feedback. with user-system interactions generated sequentially for each user. this context, different users can be treated separate clusters. This clustering introduces within-cluster correlation, violating the
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

assumption of independence commonly required by standard statistical methods. Researchers often account for such clustering using specialized techniques such as multilevel modeling (Raudenbush and Bryk, 2002), generalized estimating equations (Zorn 2001), ) and cluster-robust inference (Hansen and Leel 2019). These methods provide valid inference by incorporating within-cluster variability and appropriately accounting for the hierarchical structure of the data.
Analyzing clustered data while addressing missing responses and conducting causal inference with individual-level treatments introduces additional complexities, Yang (2018) proposed a calibration technique that balances both observed individual-Ievel confounders and unobserved cluster-level confounders. [Skk et al. (2021); Suk and Kang (022)) adapted modern machine learning methods, such as causal forests (Wager and Athey] 2018). to multilevel observational data. Park and Kang (202T) introduced refined method that models the conditional propensity score and the outcome covariance structure to account for within-cluster correlations in the estimation procedures. For a comprehensive review and comparison of propensity weighting approaches in multilevel data, see Fuentes et al. (2022); Chang and Stuart (2022).
In this work, we study the properties of the widely used doubly robust estimator for handling missing outcomes in clustered data. In the i.i.d. setting, this estimator consistent for the average outcome when either the outcome regression or the propensity score is correctly specified, and it achieves parametric convergence rates even when nuisance functions are estimated at slower, nonparametric rates. However, its behavior under cluster dependence is less well understood. Extending recent work from Park and Kangl (202T). we first establish a novel form of asymptotic normality for the doubly robust estimator in the presence of clustered data by leveraging recent central limit theorems designed for such settings. We show that the convergence rate depends on both the within-cluster correlation of individual influence functions and the error in estimating nuisance functions. Notably, the estimator can achieve faster rates than the conventional VG-rate (G' is the number of clusters) when cluster sizes are large and within-cluster dependence is weak. We then conduct extensive simulation studies, which highlight the importance of accounting for within-cluster correlation and using a cluster-robust variance estimator to obtain valid inference. Our results provide theoretical justification for using doubly robust estimators in the analysis of clustered data, especially when the cluster sizes are unbounded. The proposed methods, such as incorporating summaries of historical information into the estimation procedure, have important applications in multi-turn LLM evaluation with missing human annotations, as demonstrated in our real-data example.
The remainder of this paper is organized as follows: Section 2 introduces the problem setup and notation. Section 3 examines the properties of the doubly robust estimator under homogeneous sampling for clustered data. Our results extend the theoretical analysis in Park and Kang (202T) by allowing for unbounded cluster sizes and rates adaptive to cluster dependence. Section 4 presents a method for incorporating temporal dependence within each cluster into estimation with interesting applications in LLM evaluations. Numerical experiments and a real-world example that illustrate our results are provided in Section [ Finally, we conclude with additional discussion in Section 77] All proofs, additional discussion on related work and details of numerical experiments are included in the Appendix.
2 Setup and Notation
Let 9 E [G] denote the index of G clusters and i E [ny] index ny individuals in the g-th cluster. os represent the individual-level covariates and X, represent the cluster-level covariates. For example, in educational assessment studies, clusters typically correspond to different schools, with individuals being the students within those schools. Cluster-level covariates might include the type and location of the school, while individual-level covariates could encompass factors such as age, test scores, and prior educational experience of students. In the context of LLM evaluation, one user (associated with user- covariates X,) typically asks multiple questions and provides feedback. Different questions and their corresponding answers (i.e., the individual-leve covariates g.)) generated by the LLM are often correlated. Instead of treating all question-answe answer pairs as independent data, it may be more appropriate to consider questions and answers associated with the same user as a cluster, where data from different clusters are independent, but dependencies within clusters exist.
For each individual in the g-th cluster, let W,s

---

Let Ygs denote the outcome of interest for i-th individual in g-th cluster. In education assessment, Yg: may represent the score of a student's academic performance or psychological well-being, In LLM evaluations, Ygi is the score provided by the user and we are interested in estimating the average score to understand the performance of the L.LM system/platform. outcome Y5i may not be observed for all data points. For instance, in the aforementioned examples, some students or users may choose not to provide their scores, resulting in missing response data. Let Ryi denote the missing indicator, where Rgg 1 if ygi is observed and Rg: = 0 otherwise. With this notation, the observed data of each individual is ggi = (Xg) Wgin Rgis Rgi
real applications, the surveyed
3 Clustered Missing Data under Homogeneous Sampling
In this section, we begin by considering simplified setting where the observed data {0 gi. 1 isis ng. 1 S 9 I G} are assumed to be identically distributed, and the missingness of each individual's outcome is solely dependent on their own covariates. The analysis in this homogeneous sampling setting extends naturally from the i.i.d. case. To identify the average outcome in the missing data setting, we impose the Missing at Random (MAR) assumption the data getteranng process.
Assumption 1 (Missing at random). Rgs T gs Xg. Wgr) gs) 0 almost surely.
Xg. W gi and ग (X W ge) == P(Rg: = 1
Assumption requires that the cluster-level covariates and individual-leve covariates together fully explain the missingness mechanism. When unobserved confounders may influence the missingness mechanism, the MAR assumption may no longer hold. To assess the sensitivity of our results to such violations, we can follow the framework of Cinelli and Hazlett (2020), which extends classical omitted variable bias analysis. This approach quantifies the strength that an unobserved confounder would need to measured by its partial R. with both the treatment or missingness indicator and the outcome to reduce the estimated effect to zero or render it statistically insignificant. The robustness value (RV) summarizes this threshold and can be benchmarked against observed covariates for interpretation.
Assumption also requires the missingness mechanism be single-level and homogeneous across clusters. In scenarios where the missingness mechanism is known to be heterogeneous and the mean outcome within each cluster is of interest (i.e., E[ gi G G= g). researchers can build cluster-specific propensity score models (c.g.. random fxxed-effect logistic models) to achieve better balance within clusters (Li et al. 2013] Thoemmes and West. 201T Arpino and Mcalli 201I). The trade-off is that such an approach requires sufficiently large cluster sizes to reliably estimate propensity scores for each cluster. In our approach, propensity scores estimated from single-level models can be effectively used to balance observed covariates across clusters and to estimate the average outcome over all individuals (Suk al., 202T
Park and Kang 202T).
Consider the following data-generating process: First, sample G i.i.d. cluster-level covariates XI,..., .......G-- ~ Px. Within each cluster, Ny identically distributed (but typically not independent due to within-cluster dependency) individual-leved covariates Wo....., W gn, ~ Pw x, are sampled. For each individual, the missing indicator By. is then generated from Bernoulli(T(X,, W W.)). followed by sampling Ygi from Fy W Wor.R with conditional mean p(X,,, ge Note that we assume the regression function E E[Rgi gi Xg. W gi, Ryi = 1] u(Xg. W gi), implying is also cluster-specific. Under this sampling scheme. the observations 10 gi [X
g. W gi: Ryg Ryi gi), S i Ng: I g G} are identically distributed and (Ogts < S ngg are independent of (On,. 1 S S n for g # h (i.c.. the clusters are independent). However within the cluster. the dependency among {W gi, S i S ngJ is arbitrary. The likelihood function of (Opt
(Xg, g W gi Rgis Rgi gi), < S ng 1 < < G}
G
II g=1
(fa
X g fw
W gl.....
W
W gng Xg)
I (Y Xg. Wois Ryi ()T W ygi]]( Rot (1 T(y,, W Wg:))² Rge
When X3 fully explains the dependence among Wi...., W "g. we can express their joint distri- bution as fw (W1,... W....., W.,, X₂)= I fw (W, X,), ic., the individual-level covariates
3

---

Wi...., Wn, are conditionally independent given the cluster-level covariates Xg. However, we do impose this assumption on the gettaaanng process for generality. In the extreme case, is possible that g1 = O gn,, meaning all observations within the g-th cluster are identical. Let
n Eg= Mg denote the total sample size. this work, we allow Tig 7 0G as n + oo.
In this section, we are interested estimating the average outcome where each individual is given equal weight. Under Assumption
E[ across all individuals, E Y gi is identified as
g= PiBeB Yi X,, W gis Rgi = 1)] Elu(X,, X W Wg:)]-
(I)
Since the distribution of (X,, W 9x)) is consistent across all and i, 0 is independent both and i, ensuring that it is well-defined.
3.1 Doubly Robust Estimation
Given expression
I and an estimator of μ as P.. a natural plug-in seng-in-sttyy style estimator
ng p(g., Wgi). T 9=1
G
ÒOR
This estimator corresponds to regression-based imputation and is consistent (under mild conditions) when fi is consistent. However, the plug-in-style estimator usually suffers from first-order bias and is not robust to model misspecification (Bang and Robins 2005 Funk et al. [011]. To address these issues, consider the following doubly robust estimator that leverages both the estimated outcome model fi and propensity score ń (Robins et al. 1994. Scharfstein et al. 1999 Kennedy 2024):
G
RY gi
X g W gi)) + ( X Wgr)
ÔDR
gi)
In the classic i..d. setting, the doubly robust estimator remains consistent long either j or is consistent, with conditional bias depending on the product of nuisance estimation errors. The following theorem characterizes its similar theoretical guarantees in the clustered setting.
Theorem 1. Let (00gg) RY, (o,lY,--lla ,I (X,, W Wgd
+ x(X g, W gi) be the individual influence function. Under Assumption 7 assume there exist some constant C, >> O such that
1. For some r 2 2, we have
EG nT)
2/r
n 0.
[1 00,
SC 00,
max gSC
+
(3)
2. Pr - Var (E o0 (O0))) 2c>0 0.
3. n, it are estimated from a separate independent sample D satisfying c>>0 0.
Then we have
1 (O 0) Rí R2,
ÔDR 0
n g=1i=1
Eg Var (Eiz ((0g)) - (00g;) D)
R1 = =0 (11 #||l - -])), R2 = OP
where for (potentially random) function f of the observation, |[fl / (()))P((O) Assuming
Ri + Ra or (VSn//n). we have
ODR - 0) 4 NO.

---

By deriving a bound on the conditional variance term under the worst-case scenario of perfect within-cluster dependence, we have the following corollary. Corollary 1. Under the conditions in Theorem
7
the conditional variance can bounded as
E Var ( n
1 g() (
VI==1 ng 2|l
n
n
Consequently, under the following rate conditions
A p Ill at T||
On (V
11@ - r0ll =0p
(v
nl2n > m
the asymptotic normality in Theorem 7 holds.
In practice, the individual influence function y is often bounded, which implies E (0))(0 co. As we note above, Park and Kang (2021) also establishes the asymptotic normality of the DR estimator under clustered sampling, but require bounded cluster sizes ng S M < cc. We generalize the results in Park and Kang (202I) I and allow each cluster size ng to diverge as n - co, provided that (3 is satisfied. The second inequality
is less restrictive for large r since
2/r
Σg
g-
m'y
g
max 9SG
as + do and condition by the last inequality in 33.
Eg
/n S C is reduced to maxysa ng/n S C, which is implied We also note that when 33 holds, the number of clusters G since
m²)
2/r
g=
G g=
ngg I
G max Bg S G max 1595G n 1595G n
"
n
In Condition of Theorem S2 is the asymptotic variance of VTŌDRR which determines the final convergence rate. Condition 2 rules out degenerate cases where Var( (ar(v))DR)) vanishes. Condition imposes requirements on the convergence rate of nuisance functions estimation. Different from the i..d. setting where the estimator achieves Vīi-rate, S2, may diverge with n when the within-cluster slower convergence rate. Consequently, compared with the rate condition / /+l Å I|| = (1/vn) in the i.i.d. setting, the nuisance functions can be estimated slower rate in our clustered setting since the final target rate may also be slower.
correlation strong. resulting
In contrast to most existing work (Chen and Zhou 2011: Yang 2018 Alene et al., 2025 our results accommodate fully nonparametric and flexible modeling of the nuisance functions. In the literature, many results exist on regression function estimation for dependent observations, including
GLM modeling (Daskalakis et al. 2019). wavelet-based methods (Yogendra Chaubey and Shirazi 2013). kernel regression (Shimizu 2202), random forests (Young and Bühlmann] 2025) and neural networks (Kohler and Krzyžak 2023). The dependency structure among observations can be spatial, temporal, or induced by social network (Kandiros et al. 202I). Notably, i...d.bbsed nonparametric and machine learning methods are commonly employed to study treatment effects in multilevel settings (Carvalho et al., 2019), despite the presence of within- clunser dependency, likely due to their simplicity (Park and Kang 202T). While nonparametric machine learning methods in nuisance estimation help avoid model misspecification, their theoretical guarantees require further investigation depending on the specific dependence structure the data.
As established in Theorem the convergence rate of êOR is /n/2,, which adapts the degree of within cubbin-cluster dependence among the influence functions (p(Og)) 1 is nyJ. This behavior differs from most existing work on missing data or causal inference in clustered settings (e.g.. Park and Kang 2021). which often imposes bounded cluster sizes and can only achieve a VG-rate, corresponding to perfect within-cluster dependence. When within-cluster dependence is weak, ODR can converge faster rate than VG. Additional examples in Appendix further illustrate these conditions and convergence rates under various dependence structures.
5

---

To estimate the variance in this homogeneous sampling setting, denote p(00)= E We can re-write Qn 1 n و (O,) - nz0². A natural estimator for 22 then given by
G
)
G
Σ (0 gt)
2 0
(4)
DR
Under the conditions of Theorem | we can show that 22 is a consistent estimator of Ω2 in the sense that 2 P, 1. Therefore, Ωn/n can be used as a cluster-robust variance estimator for ÖDR to perform statistical inference. A more thauuhhthhugh computationally intensive approach to variance estimation involves bootstrapping at the cluster level (Field and Welsh (007). Depending on the data-g gentereeng process, one may choose to resample both clusters and individuals (i.e., two-stage bootstrap) or to resample clusters only. An alternative is the cluster wild bootstrap, which well-suited for settings with heteroskedasticity, few clusters, or varying cluster sizes (MacKinnon) and Webb 2017). For comprehensive discussion of resampling methods for clustered data, see Leeden et al. (2008).
4 Clustered Missing Data under Sequential Sampling
In this section, we relax the homogeneous sampling assumption and study the estimation problem in the presence of temporal dependency within each cluster. For example, in the context of LLM ovaluation, each user may ask questions in a sequential manner. In this sequential setting, the missingness mechanism of the outcome Yya at time I. may depend on the history (i.e., information before time t).
For cluster g with cluster-level covariates x,, let W gr (W....... Wot). Rgt = R gl..... Rge)) gl,
RY, gf (1g1 gl...... RgtY gz)) denote the individual- level covariates, missing indicators and out- comes up to time t. respectively. The observations within the same cluster {Og1..... O 0gn,1 are assumed to be generated sequentially. Let Hgt = ( gt, Rg,t-1 RY g.t- denote the past history just prior to observing Rgt, RytYon gt time The following sequential missing at random assumption is imposed on the data tata-gennrating process.
Assumption 2 (Sequential missing random). Rgt 1 L Yet
Xg.
Hgg.
Assumption 2 Timplies that the missingness at time only depends on the history Hgt and cluster- level covariates Xg. Denote gge (Xg. Hgt) P(Rgs E[ Xg. H Fgg.RR, Rgt = 1]. The dat--geneatting process is as follows: First sample G i..d. cluster-level covariates X1, XI..... XG Px. For the 1-th observation in the g-th cluster, we generate W ge conditioned on the history up to time t: W g,t-1, Rg..-1, RY g.t-1- The miss- ing indicator Rys is then generated from Bernoulli gf (X,r H
Xg. Hgt) and (gr(Xg. Hgt)
H))),, following which Ygr is sam- pled from PY By. with conditional mean Mor (Xg, Hgt). The likelihood function of {O,t (Xg: W gt, Rit, RoiY S
< ng, 1 S S G} is
G 1
B2 Jw oor (W gt Xg. H Hg,t-1. Rg Rg.e-1Y
9=1
TT
X g. H ff (Yge Xg. H gt: Rg: 1)] Rgt (1
Tet (X Rge
Under Assumption 2 the average outcome that we interested in is
G
n
G Rg
Vn
E Y
gt)
ΣΣ
Elugt (gg. H.t)
12
72
g=1
Note that the variables (Xg, Hgt) no longer share the same distribution across different times and clusters. The regression function and propensity score are both cluster- and time-specific. Hence, several challenges arise in estimating v. by leveraging the nuisance functions:
1. Some clusters small and do not support the modeling of cluster-level nuisance functions.

---

2. Different clusters have varying time steps depending on their size (e.g., some users may ask more questions than others). When only a small number of users ask more than t questions (for large 1), estimating the nuisance functions /Mtt and get becomes challenging.
3. The dimension of Hgt increases over time (i.e., the dimension of the arguments for these nuisance functions grows), which is important note if one aims to simplify the modeling procedure by constructing unified models that are not cluster- or time-specific.
Given these challenges, we propose the following assumption to simplify estimation. Assumption 3. There exists an observed variable Sgt 5o 6x(Xg: Hgt) functionn n, p X such that
Te g (X.: H get *x,,,,, Syt,, Pox (Xg: Hgt) gx
The variable Syz can be viewed as a sufficient summary of the historical information up to time In practice, the choice of Syt often requires domain knowledge. Common choices include average or cumulative measures of past information. For example, in mobile health studies, user wears fitness tracker that collects data daily. The device may fail to record data at time due to battery depletion, which depends on historical usage; in this case, Sgt could represent the cumulative device usage over the past few days. educational testing or tutoring systems, whether a student attempts a question at time may depend on cumulative difficulty or frustration from earlier interactions. Here, Sgt might be defined as the number of incorrect attempts or difficulty-adjuster score accumulated up time I. In the LLM evaluation setting, whether users provide feedback may depend on their prior interactions with the system. Accordingly, Sgr can be constructed as the embedding of the concatenated conversation history Wgt gt up to time t, or from the most recent d conversations W g. W gt- These embeddings remain of fixed length regardless of the length of the conversation history.
Assumption also simplifies the data-generating process by assuming the míssingness mechanism and the regression function of ygt depend on the cluster-level covariates X, and summarized information at time t, S Syt, in the same way (i.e., they are not cluster- time-specific). The doubly robust estimator of thn is then given by
G 12
Rye ,i Spt) + fi(X,, S.t))
VDR
(5)
# #(Xg; (gt
where we slightly abuse the notation and still denote the influence function as (Z)g)) Sxr and the summarized information Sgt at time t. While the idea of using summary statisties to simplify nuisance function modeling has been mentioned in Park and Kang (202T), our work formalizes this as an explicit assumption and establishes corresponding theoretical guarantees in the following theorem.
(OX, + p((,,, Sgr) with Zgi = (X,, Sgi. Rgt, Rye Yyz) including both the observation
Theorem 2. Under Assumption
3
further assume.
1. For some r 2 2, (Zze),, 15 Ngy 1 1595 G} are uniformly integrable, i.e., lim sup E [lx(()))) (le(ZZe)) M)] 0. M M g.t
The cluster sizes and total sample size satisfy
G Eg=
2/r
2 5C< 00,
maX. g
g n
+
(6)
n
2. Ωn = - Var (Eia ((Zgt)
20>0. 0.
3. R, is are estimated from a separate independent sample D satisfying >>>00 0.
Then we have
G n
DD Un
(o(Zgt) - Blp((Z)))) + T2,
7

---

(V20 Var (Eiz (Zzgt) - (z(gtt D)
Ti =p
G ng
= Op
-
1T(:: Sgr) xxxg. (ae Sge)lllA(X Sye) - p(Xgg Sgiil]
g=1 t=1
Assuming Ti T2= = or (VPn/n), +++T==orVVnnni
we have
(WDR - thn) S N(0,
Theorem establishes the asymptotic normality of the doubly robust estimator when the obser- vations ge may follow heterogeneous distributions. In this setting, uniform integrability con- dition dition-analoon analogous dominates the sum (Hansen and Leel 2019). The conditional bias term T2 depends on the average nuisance estimation error across n observations. We further provide worst-case bound on both the empirical process and bias terms in the following corollary, similar to the result in Corollary [
analogouss to Lindeberg's condition when r i- required to ensure no single term
Corollary 2. Under the conditions in Theorem 2
we have the following bound on the error terms:
C 9=1
Var ( im1 ((Zge) (Z())) D)
Op
Σ n sup ED [(f(2) - (2(2))1)
71
9=1
ng
| Sgt T(2 Sgr S Sgt) u(Xg. Sgr)l
9=1
=0p
sup
ED ( 8) - π (x, 8)} sup
(D( 8) u(æ 8))a]
2.8
2.3
where the supremum is taken over the support of the corresponding variables. Consequently. if we further assume
n22,, (
sup ED ((((z) (zz)))²]
- n
2n n
sup ED EDl(((, 8) - m(x 8)) sup (( π (2r 8)) =0
2.8
the asymptotic normality Theorem holds.
Finally, approach of using summary to simplify the modeling of within-cluster dependence can be extended to other settings. For instance, when clusters are defined by a network model, characteristics of an individual's neighbors (e.g., degree or sum of edge weights) can be instrumental and serve as the summary information in modeling the missingness mechanism. Similar asymptotic normality results can be derived by leveraging the central limit theorem for clustered data.
Simulation Study
This section presents simulation studies to illustrate our theoretical results; full details are provided in Appendix C In the first study, we compare confidence intervals using i.idd. and cluster- robust variance estimators. Figure a shows that only the cluster-robust approach achieves nominal 95% coverage, underscoring the importance of accounting for cluster dependence. In the second study, we evaluate the impact of incorporating historical information when modeling missingness in sequential data. As shown in Figure b)), modeling the missingness mechanism with relevant historical summaries improves performance compared to using only current information or ignoring missingness, highlighting the importance of leveraging past interactions in sequential settings.

---

Coveng
(a) Estimated coverage probability of Cls based on different variance estimators.
(b) Estimated RMSE of estimators adjusting for different information.
Figure Simulation results.
6 Real Data Analysis
The alignment of Al systems with human preferences is critical area of research. A key challenge for prominent methods such as Preference Flow Matching (Kim et al. (0255) and Reinforcement Learning from Human Feedback (RLHF) (Bai et al., 2022 Casper et al. 2023) that human annotations for evaluating alignment are often costly and incomplete. In this context, illustrate methods using the OpenAssistant Conversations dataset (Köpf et al., 2023). corpus structured trees, with each tree representing conversation cluster and its messages as individual observations. The cleaned dataset includes 9,808 trees and 81,937 messages, with message- level covariates such content and role, and tree-level covariates such language. We focus on annotations for quality, creativity. humor, and toxicity. and consider two missingness mechanisms: (i) missingness depends only observed covariates (Assumption] [ (ii) missingness depends on the conversation history (Assumption (
conversation
from the root the message within the conversation leading up to the message node, i.e., the path The individual-level covariates W message embeddings and role, and the cluster-level covariate X, is language; missingness simulated via logistic model. We estimate average annotation scores using three methods: (1) naive ii.de confidence intervals only using data with annotations observed, (2) doubly robust estimation assuming independence, and (3) doubly robust estimation with cluster-robust variance estimation as in Eq. (4. More details about the dataset, missingness and estimation can be found in Appendix D Figure shows the resulting confidence intervals for each annotation type, compared to
the ground truth average n ES E Ygi-
Figure 2 shows that unadjusted estimates, which ignore observations with missing annotations, are biased and yield confidence intervals that fail to cover the true average. This bias arises because covariates W gi and X, influence both the missingness and the outcome, acting as confounders. The doubly robust adjusted estimates correct for this bias and are closer to the ground truth. Among the adjusted methods, confidence intervals assuming independence are narrower but may undercover due to cluster dependence. For example. in Figure 2c.). the interval for humor under the i..d. assumption fails to cover the true value, while the cluster-robust interval does. These results underscore the need to account for cluster dependence when constructing valid confidence intervals.
7 Discussion
This paper studies mean estimation with missing responses under cluster dependence, focusing on the widely used doubly robust estimator and establishing its theoretical guarantees in clustered settings. We mainly consider two primary cemnaris--hmmggemeooo sampling and sequential dependence but methods extend to more general structures like network dependence. Our theoretical and empirical results highlight the importance properly accounting for cluster dependence, with valuable implications for applications such LLM evaluation using limited human annotations.
There are several directions for future research. First, the doubly robust estimator can be unstable when propensity scores are near zero; using balancing weights may offer more stable alternative (Ben-Michael et al. 2024), and its performance under cluster or sequential dependence requires further study. Second, estimating means in target populations with only covariate iiformmttoonssuu

---

Qualty
Creetvit
0.315
0.63
Unadjaceed
AdjodeetjLc Types Cf
Adjooedjjkktee)
0.310
Unadqusted
Adjustedii
Adjadiejcchste)
Types of
(a) Quality
(b) Creativity
0.126
0.192
0.125
Todeet
0.123
0.184
Unedistte
Adustel idl Types of
Adssudidssta
Unadqusted
Adjusadit Li Types Gf
Asjuaddccksia))
(c) Humor
(d) Toxicity
Figure 2: Confidence intervals for average human annotations on quality, creativity, humor, and toxicity under homogeneous sampling. The red dashed line is the ground truth average.
evaluating different language model without human annotations-is is related covariate shift (Sugiyama and Kawanabe] (012) and gemeraaiizationtaassporaal al. 2003). Investigating these extensions is also promising direction for future work. Another interesting and important direction is to develop theory and methods for flexible regression and propensity score estimation under clustered settings, enable more reliable ATE estimation using nonparametric doubly robust methods.
(Dahabreh et al.) 2020 Zeng
References
Alene, M., Vansteelandt, S., and Van Lancker, K. (2025). Analyzing multi-center randomized trials with covariate adjustment while accounting for clustering, arXiv preprint arXiv:2504. 12760.
Arpino, B. and Mealli, F. (2011). The specification of the propensity score in multilevel observational studies. Computational Statistics Data Analysis, 5774):17
Austin, C. and Merlo, J. (2017). Intermediate and advanced topics in multilevel logistic regression analysis. Statistics in medicine, 36(20):3257-3777
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et (2022). Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint 1rXiv:204.008866
Balzer, I.. B., Zheng, W., van der Laan, M. J., and Petersen, M. L. (2019). A new approach to hierarchical data analysis: Targeted maximum likelihood estimation for the causal effect of cluster-level exposure. Statistical methods in medical research, 2866):1761-1788
Bang, H. and Robins, J. M. (2005). Doubly robust estimation missing data and causal inference models. Biometrics, 61(4))962-973.
Ben-Michael, E., Page, L., and Keele, L. (2024). Approximate balancing weights for clustered observational study designs. Statistics in Medicine, 43(12):2332-2358
Bound, J., Brown, C., and Mathiowetz, N. (2001). Chapter 59 measurement error in survey data. volume 5 of Handbook Econometrics, pages 3705-3843. Elsevier.
10

---

Carvalho, C., Feller, A., Murray, J., Woody, S., and Yeager. D. (2019). Assessing treatment effect variation in observational studies: Results from a data challenge. Observational Studies, 5(2):21- 35.
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R.. Korbak, T., Lindner, D., Freire, P., et al. (2023). Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arX1v:2307.15217
Chang, T.-H. and Stuart, E. A. (2022). Propensity score methods for observational studies with clustered data: a review. Statistics in medicine, 41(18):3612-30 -626..
Chen, B. and Zhou, X.-H. (2011). Doubly robust estimates for binary longitudinal data analysis with missing response and missing covariates. Biometrics, 67(3):830-84..
Chen, S. and Haziza, D. (2019). Recent developments in dealing with item non-response in surveys: A critical review. International Statistical Review, S-SS19--S218
Cinelli, C. and Hazlett, C. (2020). Making sense of sensitivity: Extending omitted variable bias. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(1):39-6)
Dahabreh, I. J., Robertson, S. E., Steingrimsson, J. A., Stuart, E. A., and Hernan, M. A. (2020). Extending inferences from a randomized trial to a new target population. Statistics in medicine, 39(14):1999-2014
Daskalakis, C., Dikkala, N., and Panageas, I. (2019). Regression from dependent observations. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 881-889.
Field, C. A. and Welsh, A. H. (2007). Bootstrapping clustered data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(3):369-390.
Fuentes, A., Lüdtke, O., and Robitzsch, A. (2022). Causal inference with multilevel data: A comparison different propensity score weighting approaches. Multivariate Behavioral Research, 57(6)9916-939.
Funk, M. J., Westreich, D., Wiesen, C., Stürmer, T., Brookhart, M. A., and Davidian, M. (2011). Doubly robust estimation of causal effects. American journal of epidemiology, 173(7):761-777
Hansen, B. E. and Lee, S. (2019). Asymptotic theory for clustered samples. Journal of econometrics. 210(2):268-299
Hansen, M. H. and Hurwitz, W. N. (1946). The problem of non-response in sample surveys. Journal of the American Statistical Association, 41(236):517-52.
Hogan, J. W., Roy, J., and Korkontzelou, C. (2004). Handling drop-out longitudinal studies. Statistics in medicine, 2359):145511497
Kandiros, V., Dagan, Y., Dikkala, N., Goel, S., and Daskalakis, C. (2021). Statistical estimation from dependent data. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5269-5278. PMLR.
Kennedy, E. H. (2024). Semiparametric doubly robust targeted double machine learning: a review. Handbook of Statistical Methods for Precision Medicine, pages 207-236.
Kim, M., Lee, Y., Kang, S., Oh, J.. Chong. S., and Yun, S.-Y. (2025). Preference alignment with flow matching. Advances in Neural Information Processing Systems, 01:35114-351660
Kohler, M. and Krzyžak, A. (2023). On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data. Bernoulli, 29(2):
Köpf, A., Kilcher, Y., Von Rütte, D., Anagnostidis. S., Tam, Z. R., Stevens, K., Barhoum, A., Nguyen, D., Stanley, O., Nagyfi, R., et (2023). Openassistant covveraaionnkdemrrrrrinn large language model aligmment. Advances in Neural Information Processing Systems, 16:47.69-47681.

---

Leeden, R. v. d., Meijer, E., and Busing, F. M. (2008), Resampling multilevel models. In Handbook of multilevel analysis. pages 401-433. Springer.
Li, F., Zaslavsky, A. M., and Landrum, M. B. (2013). Propensity score weighting with multilevel data. Statisties in medicine, 32(11):3333-3387
Little, R. and Rubin, D. B. (2019). Statistical analysis with missing data, volume 793. John Wiley & Sons.
Liu, L.., Hudgens, M. G., Saul, B., Clemens, J. D., Ali, M., and Emch, M. E. (2019). Doubly robust estimation in observational studies with partial interference. Stat, 8(1):e214.
Lüdtke, O., Marsh, H. W., Robitzsch, A., and Trautwein, U. (2011). A 2x 2 taxonomy of multilevel latent contextual models: Accuracy-bias trade-offs in full and partial error correction models. Psychological methods, 16(4):444.
MacKinnon, J. G. and Webb, M. D. (2017). Wild bootstrap inference for wildly different cluster sizes. Journal of Applied Econometrics, 32(2):233-254.
McNeish, D. and Stapleton, L. M. (2016). Modeling clustered data with very few clusters. Multivari- ate behavioral research, 51(4):995-51.
Park, C. and Kang, H. (2021). A more efficient. doubly robust, nonparametric estimator of treatment effects in multilevel studies. arXiv preprint arXiv:2110.07740.
Polley, E., LeDell, E., Kennedy, C., and van der Laan, M. (2024). SuperLeamer: Super Learner Prediction. R package version 2.0-30-9000.
Raudenbush, S. W. and Bryk, analysis methods, volume 1. sage.
(2002). Hierarchical linear models: Applications and data
Robins, J. M., Rotnitzky, A., and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, (9(427):846 866.
Scharfstein, D. O., Rotnitzky, A., and Robins, J. M. (1999). Adjusting for nonignorable drop- out using semiparametric nonresponse models. Journal of the American Statistical Association, 94(448):1096-112) 120.
Schennach, S. M. (2016). Recent advances in the measurement error literature. Annual Review of Economics, 8(1):341-377.
Schochet, Z. (2022). Estimating complier average causal effects for clustered rets when the treatment affects the service population. Journal of Causal Inference, 10(1):300333.
Shimizu, Y. (2024). arXiv:2403.04766.
Nonparametric regression under cluster sampling. arXiv preprint
Sugiyama, M. and Kawanabe, M. (2012). Machine learning in non-stationary environments: Intro- duction to covariate shift adaptation. MIT press.
Suk, Y. and Kang, H. (2022). Robust machine learning for treatment effects in multilevel observational studies under cluster-level unmeasured confounding. Psychometrika, 87(1):310-343.
Suk, Y., Kang, H., and Kim, J.-S. (2021). Random forests approach for causal inference with clustered observational data. Multivariate Behavioral Research, 8696)582--85.
Thoemmes, F. and West, G. (2011). The use of propensity scores for nonrandomized designs with clustered data. Multivariate behavioral research, 4643):51
Tsiatis, A. A. (2006). Semiparametric theory and missing data, volume 4. Springer.
Vazquez-Ban G. (2023). Identification and estimation spillover effects in randomized experiments. Journal of Econometrics, 235(3):05533.
12

---

Wager, S. and Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228-1244
Wang, B., Harhay, M. O., Tong, J., Small, S., Morris, T. P., and Li, F. (2021). On the mixed-model analysis of covariance in cluster-randomized trials. arXiv preprint arXiv:2112.00832.
Yang. S. (2018). Propensity score weighting for causal inference with clustered data. Journal of Causal Inference, 6(2):20170027.
Yogendra P. Chaubey, C. C. and Shirazi, E. (2013). Wavelet-based estimation of regression function for dependent biased data under given random design. Journal of Nonparametric Statistics, 25(1):33-71.
Young, E. H. and Bühlmann, P. (2025). Clustered random forests with correlated data for optimal estimation and inference under potential covariate shift. arXiv preprint arXiv:2503. /2634.
Zeng, Z., Kennedy, E. H., Bodnar, L. M., and Naimi, A. I. (2023). Efficient generalization and transportation. arXiv preprint 2:2.0.230..00002.
Zorn, C. (2001). Generalized estimating equation models for correlated data: A review with applications. American Journal of Political Science, pages 470-490.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction accurately summarize theoretical contribu-
tions Section 3 and implications for practitioners (illustrated through simulations in Section 5 and real data example in Appendix B
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We have highlighted several limitations of our work. For instance, as noted in the discussion following Theorem [ practitioners often rely on i.i.d-bbase machine learning methods to estimate nuisance functions in clustered settings; however, the theoretical justification for this practice remains an open question. Additionally, at the end of Section 3 we mentioned that while cluster bootstrap methods can be used for variance estimation, they tend to be computationally intensive. Finally, Assumption requires the existence of summary variable Sgt, and we emphasize that its specification is context-dependent and relies on domain knowledge.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions included in the theorem statements. The proofs included in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the details of experiments in the paper and the codes for repro- ducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The codes preprocess and analyze the data are provided in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https SonddeSubmisssooPPoon
/ ctiips.cc//uullcc/gge
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Settings and details are provided in Section 5 and Appendix Full details can also be found in the codes provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Our real data example in Appendix B includes 95% confidence intervals with construction details. Error bars for statistical significance are not applicable to our simulations.
16

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computations in Section Snd and Section 6 are very fast and can be completed on a personal laptop. The computation time for Appendix D is reported separately.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https:
Answer: [Yes]
Justification:
The paper conforms with the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [Yes]
Justification: This paper primarily focuses on theoretical and methodological developments, accompanied by real-world illustrations. The method help researchers better estimate the quantity of interest. At this stage, we do not anticipate any negative societal impacts arising from this work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The dataset used in this paper is publicly available, and we do not release any new models or datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The authors cite the original papers that produced the code package or dataset under proper licenses.
18

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification:
We provide codes with explanations.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicablle as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper's 's contributions are mainly theoretical and methodological. does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
19

---

Answer: [NA]
Justification: The paper's contributions are mainly theoretical and methodological. It does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs are only used improve the writing and phrasing.
Guidelines:
The answer NA means that the core method development in this research does not non involve LLMs as any important, original, non- stanstandard components.
Please refer to our LLM policy http://///aarrsss.ooo/ooooooo/// for what should or should not be described.
20

---

A Related Work
In the i.d. setting, it is well-established that estimating the average treatment effect under the exchangeability assumption is equivalent to estimating the mean in a missing data problem under missing at random (MAR), with the treatment assignment analogous to a missingness indicator (Tsiatis 2006). However, when cluster dependence is present, the problem setup, estimands of interest, and analytical techniques diverge substantially.
In the literature on cluster-randomized trials (Balzer et al. 2019 Wang et al. 2021: Schochet 2022) treatments assigned at the cluster level, all individuals within cluster receive the same treatment. In contrast, our setting involves individual-leve missingness indicators within clusters, allowing each individual to have their own observed missing outcome.
Other works in causal inference consider individual-Ievel treatments within clusters but focus on interference-whent one individual's outcome depends on the treatment assignments of others in the same cluster (Liu et al. 2019: Vazquez- Baree 2023). Under interference, even a single unit's treatment status affect the observability of potential outcomes for the entire cluster, which differs fundamentally from our MAR-based framework. In our setting, missingness is modeled directly via Ryi and each individual's outcome may be observed unobserved regardless of others in the same cluster.
While some doubly robust methods for clustered data exist in the causal inference and missing data literature (Chen and Zhou 2011; Yang 20188] Alene et al. 2025), these approaches often rely on parametric assumptions for nuisance functions and In contrast, approach is fully nonparametric and accommodates modern machine learning techniques for nuisance estimation.
Finally, most existing analyses do not leverage recent central limit theorems (CLTs) for clustered data (Hansen and Lee] (219), which limits their applicability to settings with equal or bounded cluster sizes and often leads to overly conservative VG-rate results. For instance, in the Appendix of Park and Kang (202)) y Alene et al. (2025), theoretical guarantees for the doubly robust estimator multilevel observational studies are established, but they require bounded cluster sizes and only achieve a VGrttt In contrast, our analysis accommodates diverging cluster sizes and achieves convergence rate of /n/S2, which can be faster than the VG-rate when within-cluster dependence is weak.
B Illustrative Examples of Convergence Rates
Consider a setting where all clusters are of equal size n, n for some a E (0, 1), and the total number of clusters is G nl Under this setup, condition simplifies S (r 2)/ 2).
We first provide two examples under the homogeneous sampling setting as Section 3
Example (i.i.d. sampling). Consider a special case where individual influence functions
(sp(Ogi) 1 < i < Ng are independent within clusters This scenario could arise when π and I are functions only of W and (W gi, 1 S i S ngg are independent. In this case, we have
Pr VVr((p(Ogi)),
which is constant and we assume it is positive. The conditions on nuisance estimation achieve /n-rate in Theorem
are
|| wlll
or (1/ (1/V7). 11 1 op (1),
which are the same conditions for the doubly robust estimator be /n-consistent in the i.i.d. setting. However, Corollary requiree the stronger conditions:
1 a|ll π|| op (1/vm). 111- fl1 (1/VTe))
The need for these stronger conditions in Corollary 7 arises from our worst-case analysis of the variance when bounding the empirical process term, which may not be tight when independence also holds within clusters.
Example 2 (Perfect correlation within cluster). Consider another special case where, for each cluster the individual influence functions and their estimates are all equal (i.e., (0(O,1)
(0 gm.)
21

---

and g1 p0(0 gm, )) within-cluster dependency is perfect. We have
On n²" Var(()(Ogg))
and the convergence rate is //n/n = n (1-a))/ G-1/22 Innuitively, the effective sample size is G since we effectively only have repeated measures within each cluster. The conditions nuisance estimation required to achieve the v VGrrate in Theorem
are
p|ll
F||
16 21 oz).
VG
We provide another example where the influence functions of different observations have weak stationary dependence in the temporal setting (Section (4)
Example 3 (Weak stationary dependence). Consider the case where within each cluster, the sequence of individual influence functions satisfies the following weak stationary condition;
(agt)(Zgt)) = 1, Cov (p((Ztt), (gs)) = 1/|t s), S s,t S ng: + t.
Recall that in all examples assume fg
n0 G = ==ntaa Simple calculations yield
n -1
G
2n
Σ
- n +
log n
t
and the convergence rate of db is /logn/n The rate condition nuisance function estimation
sup Ep (()(2) - [(()) (V
logn n
log n
sup En (i 8) - (1x 8)) sup En (#x 8) π (æx 8))2] a,e
æ,8
Finally, we provide an example that illustrates the impact of heterogeneous cluster sizes.
Example 4 (Heterogeneous cluster sizes). Consider setting with two rypes of cluster sizes: size 1 and size n. There are n/2 clusters of the first lype and n. 0/2 clusters of the second type, so the total number of clusters is
G G-
Within each cluster, assume the individual influence functions and their estimates are all identical with unit variance. Then
72 + n 20
n 1-a
Ωn
72
2n
and the resulting convergence rate is
On (1-0)/2
which is slower than both Vn and VG, since G G n. The corresponding rate condition for nuixance estimation is
- - π| op (1-11-))22)
|| - ol|= = op()).
This example highlights the importance of accounting for heterogeneous cluster sizes. Although the total number of clusters G is large and of the same order as n, the convergence rate is driven by the relatively small number of large clusters, within which the correlation is perfect.
C Simulation Details
In this section, we provide details for simulation studies that illustrate our theoretical results. In Appendix [C.]] we provide details of numerical experiments that highlight the importance of account- ing for cluster dependence and using a cluster-robust variance estimator to ensure proper coverage probabilities of confidence intervals. Additionally, in Appendix C.2, we present details of numerical experiments demonstrating the critical role of historical information adjusting for missingness in sequential setting.
22

---

C.1 Homogeneous Sampling
Consider the following data- generating process: For each cluster 9. the cluster-level covariate X9 ~ N(0 1). Then the individual-level covariates W N ([InxX9. σ¹Σ) given Xg. where Σi --1 for 0.8, x² 4. For each individual i, the missing indicator Rgi is sampled from a Bernoulli distribution with mean x (X,.. y. Wi Logistic(Xg + 0.5W gs) and the outcome ys sampled from N(-X, + W 0.5, 1). The average outcome is 0 0.5. In this experiment, we evaluate the necessity of considering the cluster structure in the estimation by comparing the coverage probability of confidence intervals based two different variance estimators. The first
variance estimator is of = n- Eg= Bh (6(0g)) - QDR) which is a consistent estimator of
variance observations {Ogi, Sii Ng. g S G} are independent. The second estimator o2 = ②in with Ω given by (4) and takes the cluster structure into account. In each replication of experiment, we generate the data with total sample size 10000 and cluster size ny n for 0.5). compute the doubly robust estimator @ and construct Wald-confidence intervals based o². o2. We then repeat the process M 500 times and estimate the coverage probability of the 95% confidence intervals obtained. The results are summarized in Figure aa).
a E (011,
Figure a shows that the confidence intervals based on 03 attain the nominal coverage probability of 0.95, as they appropriately account for the cluster dependence in the data. In contrast, the confidence intervals based of suffer from lower coverage probabilities than the nominal level, because of ignores the cluster structure and consequently underestimates the variance.
C.2 Sequential Sampling
Consider the following data-generating process: For each cluster g. the cluster-level covariate X, (0, 1). The individual-level covariates are generated sequentially from an AR(2) process. Specifically,
W gt A W g A2 g Et, E, (O, 412).
Let Sgt maN W Wgsk: gsk, sa W gsh: 1 E W E R be the summary of past information up to time t. For each time f, the missing indicator Ryt is sampled from a Bernoulli distribution with mean (X2, S₁r) = logistic(X, + +11,0.8 -0.5, 0.3) Syt) and the outcome Yg: is sampled from N(-X, + (1, -0.4) Sg + 1,11. The average outcome is v = 1. In this experiment, we demonstrate the importance of adjusting for useful summary of past information in modeling the missingness mechanism by comparing two estimators. The first one models the missingness mechanism function of Xg. Sgt. We also include the unadjusted estimator as a baseline. In each replication the experiment, generate the data with total sample size {2000, 4000, 16000) and cluster size ng 0.4, compute two doubly robust estimators VDD adjusting for different information and evaluate the estimation error. We then repeat the process M 500 times and estimate the Rooted- Mean-Squared -arrr (RMSE)
function of Xg. W gt while the second fits
M Σ m=
RMSE
2
M
DR
The results are summarized in Figure bb). As shown in Figure the estimator that models the missingness mechanism using the correct historical information outperforms the one that adjusts only for the current information at each time This highlights the importance of incorporating relevant past information when modeling missingness in sequential setting, such as users' sequential interactions with the system.
D Details for the Real Data
In this section, we provide more discussion on the background, implementation details and additional results of the real data analysis.
The alignment of AI systems with human values, intentions, and preferences a crucial area of Al re- search. Techniques such as Preference Flow Matching (Kim et al. 2025) and Reinforcement Learning
23

---

from Human Feedback (RLHF) (Bai et al. 2022 Casper et al. 2023) have been developed to enhance the performance of LLMs across various applications. However, before focusing on improvement strategies, the first step is to assess how well an Al system aligns with human preferences based on available annotations. Human annotations serve as valuable tools for evaluating Al performance, yet they are often expensive and difficult to collect at scale. The purpose of Section [ to illustrate our methods using a conversational dataset where human annotations are missing for some observations.
Our analysis focuses on the OpenAssistant Conversations dataset (Köpl et al., 2023 >. a publicly available human- -errann-eeerted and human-anmotte annotated assistant-style conversation corpus The dataset is structured as conversation trees, where each tree begins with an initial prompt message (root node) that can have multiple child messages as replies, which in turn can have their own responses. Due to this hierarchical structure, messages within the same conversation tree are highly correlated, and we model each conversation tree as a cluster, containing many messages as individuals within the cluster.
The cleaned dataset consists of 9,808 conversation trees with a total of 81,937 messages. Message- level covariates include the content and role of the message, which indicates whether a message was generated by the prompter or the assistant, while conversation-level covariates include language, with English and Spanish being the most frequently observed languages. Each message is also annotated with multiple labels assessing different aspects, which serve as evaluation scores, In our analysis, we focus on annotations for quality, creativity, humor, and toxicity.
We first illustrate our methods in Section 3] where the missingness of annotations for each mes- sage depends only on its own content and the characteristics of the conversation tree to which it belongs directly (Assumption T. Let the individual- covariate Wgi represent the embed- ding of the i-th message in the g-th conversation tree along with its role. In this work, we use MaAI/beesmaal-eee en-v.. 5 embeddings from Hugging Face W gi, which are fine-tuned specifically for embedding tasks. Additionally, we incorporate the language of each conversation tree as the cluster-level covariate Xg. The missingness indicator for human annotations, Rgi is then simulated from a logistic model satisfying E. Rgi W Xg) = expit W T 3), with ,3 being randomly gener- ated coefficient vector. This process mimics how human reviewers may decide whether to provide annotations based on message content and contextual factors. Let Yg: represent the annotations, which are treated as missing when Ryt = O. We construct three types of confidence intervals for the average human annotations in our results. The first method restricts the analysis to messages with Ry: (i.e., ignoring messages with missingness) and applies the CLT for i.i.d. data. The second method applies the doubly robust estimator (2) to adjust for missingness but estimates variance under the assumption of independent observations. The third method further adopts cluster-robust variance estimation approach as in (4). Sample splitting is used, and all nuisance functions are estimated from half of the sample by the Superl.earne (Polley et al.) (024) incorporating generalized linear model and random forest. We plot these confidence intervals for annotations on quality, creativity, humor, and toxicity in Figure 2 with the dashed horizontal line n Eg, Ygi serving as the ground truth.
We further illustrate the use of summary statistics from conversation history to adjust for missingness in Section 4 For each message, we assume that the probability of missing annotations depends on the conversation history up that node in the conversation tree (i.e., the path from the root node to the message node). Let Sgt represent the embedding of the conversation history, aggregating conversations from all ancestor messages of the t-th message in the g-th conversation tree, along with its role. The cluster-level covariate is language. The missingness indicator is simulated using a logistic model Ryy Syt. X,] = expit(S ( 3). This setup mimics how human reviewers may decide whether to provide annotations based on prior message content and contextual factors. We construct three types of confidence intervals for the average human annotation scores. The first method restricts the analysis to messages with Ryt = 1 (i.e., ignoring messages with missingness) and applies the CLT under an i.i.d. assumption. The second method applies the doubly robust estimator (5) to adjust for missingness but estimates variance under the assumption that the influence functions (ZZge) are independent. However, this confidence interval is not valid, as it ignores within-cluster dependence; we include it only for reference. The third method adopts a bootstrap-based variance estimation approach that accounts for the cluster structure. The bootstrap procedure takes approximately 20 12-core CPU machine. We plot these confidence intervals for annotations on quality, creativity, humor, and toxicity in Figure with the dashed horizontal line 1 Eg, Xgt serving as the ground truth.
hours per outcome
Available at http::///ngeiiiieeeoooonesssssss
24

---

0.643
Qualty
Creetret
0.314
0.352
0.643
Unady
Adunteed Types of CI
Adyabedjenuurtiii
0.300
Unadqusted
Adjusted1:
Adprteetneqeeriss
Types of
(a) Quality
(b) Creativity
0.187
0.185
0.125
0.181
Tooeety
0.184
0.124
0.283
0.182
0.123
0.181
Unedjatte
Adsatal idl Types of
Unadjusted
Adjusadit Ld) Types of
Adussedsequueria
(c) Humor
(d) Toxicity
Figure 3: Confidence intervals for average human annotations on quality, creativity, humor, and toxicity under the sequential structure. The red dashed line is the ground truth average.
As shown in Figure 3 the estimates that adjust for missingness are closer to the true sample average, represented by the red dashed line, for quality, humor, and toxicity, compared to the unadjusted estimates. This suggests that accounting for missingness effectively reduces estimation bias. Addi- tionally, the confidence intervals that account for potential cluster dependence within conversation trees are at least 10% wider than those constructed under the i.i.d. assumption. This indicates that variance estimators based the i..d. assumption underestimate the variation, highlighting the importance of using a cluster-robust approach for variance estimation.
E Additional Simulation Results
In this section, we present additional simulation results for the plug-in (regression) estimator
ng
FOR
(X,,
Wor),
n g=1i=1
the IPW estimator
G ng
1
Rgi yg (Xg. g'))
0rw
and the doubly robust estimator in equation (2) Our focus is evaluating their performance under model misspecification in the presence clustered data.
The data-generating process follows the homogeneous sampling setup described in Appendix C with regression function and propensity score given by
((X,, Wgx) -x, W T TXy. Wgx) logistic(X,, + 0.5W2) )
The true average outcome is 0 = 5. To assess estimator performance under misspecification,, we consider scenarios in which μ and/or w are misspecified by modeling the quadratic term W as linear. We generate samples of size n {1000, 10000} with varying cluster sizes, apply each estimator under different model specifications, and compute the mean squared error (MSE). The results summarized in Tables [
25

---

Regression estimator
IPW estimator
DR estimator
II correct, т correct
0.0562
0,0561
0.0563
conrect, wrong
0.0563
2.7023
0,0563
I1 wrong, т correct wrong, wrong
2.0356
0,0563
0.0572
2.0603
2.6896
2.5765
Table Mcan squared error (MSE) of regression, IPW, and DR Estimators under potential nuisance misspecification with sample size 10000, n2
100.
Regression estimator
IPW estimator
DR estimator
I1 correct, т correct correct, wrong
0.0230
0,0231
0.0230
0.0230
2.3882
0,0230
wrong, π correct wrong, wrong
2.1082
0.0232 2.3914
0.0233
2.1075
2.3102
Table 2: Mean squared error (MSE) of regression, IPW, and DR Estimators under potential nuisance misspecification with sample size 10000, n2 10.
The conclusions in this clustered setting are similar to those the classical i.i.d. setting. When the outcome model μ is misspecified, the plug-in (regression) estimator foR becomes inconsistent. Similarly, the consistency the IPW estimator 6pw relies on correct specification of the propensity score т. In contrast, the doubly robust estimator 0DR. which models both the outcome and the missingness mechanism, remains consistent as long either μ or ı correctly specified. This aligns with the theoretical guarantees established in Theorem
Proof of Theorem I
Since the observations (O < ng. < g G) share the same distribution, have the following decomposition of error
11 11
0
(4 Blx((g:)))
9=1
(g (0 P(g(Og.) (0g:))
(00)) (O,)))
where for a potentially random function of O. P[ = ((oddI(() only the randomness of O is averaged over. For the first CLT term, by the central limit theorem for clustered data (Hansen and Lee 2019 Theorem 2], under the assumptions our Theorem have
1 (p(Og)) Elp((0,:))) 4 N( 1), Qn n V
and thus the order is
(p((0))) gi) (Opp()))))) Oz (V On
26

---

Regression estimator
IPW estimator
DR estimator
II correct, т correct
0.3247
0.3271
0.3248
conrect, wrong
0.3232
5.3693
0.3246
I1 wrong, т correct wrong, wrong
1.8736
0.3267
0.3470
1.9578
5.6812
5.3977
Table 3: Mcan squared error (MSE) of regression, IPW, and DR Estimators under potential nuisance misspecification with sample size 1000, Fg = 31.
For the second empirical process term, by Markov's inequality, we have (conditioning D that is used to estimate the nuisance functions)
P
(6 gi) (00):) g()(O -(0g0))))
((00))))
g=1
i=1
Var
ΣG g=
g)) 40 (O0))))
12222
Eg Var
(4 gi
40 (0
n²t²
where the last equation follows from the independence of observations that come from different clusters. Set
M
Eg=1
G Var (Σ Th (p(0 -(0gg)))((())))))
we have
P
(g(0g))
- P(p((0)))))) (((g))) -1)00)))
M
Thus the empirical process term can be bounded as
($(0,)
) (00))) - ,l) ()))
g=1 1=1
V Var [Σ R2 i=1 -
= =0z
For the third bias term, by the property of conditional expectation we have
( gi) - ,
Rgi ia(gg Wg:))
E
+ AX Wge) - (X. Wgs)
#(X,, g. Woi)
π (X,, W (u g. W MXX,, Wgi)) + MX Wor) m(X,,. Wos)
*(X g. W si)
(T, W f(x,,. W (X,,, W - (XX,, W wi)
= E
+(X,, gi)
Since # 2 and by Cauchy Schwarz inequality, we have
-) - < - || f =llllae f pl]l
which implies
[P({(O,g)) - Or(1 -- + -l)) ,.)))
This completes the proof of the asymptotic expansion. The asymptotic normality then follows from Slutsky's 's theorem.
27

---

G Proof of Corollary [
The conditional variance given the training set D can be expressed
nly
Var
Σ (g(0g)) ((0))))
Cov (g((0))) - se (0)) (00))) - (00))
Σ (
Var (f(0))) - (0))))
9
Σ
2 Var (8(0) (
$ -0)
g=1
where we have used Cauchy Schwarz inequality to bound the covariance with variance and the fact that the observations {Ogr: S < ng, 1 S g S G} are identically distributed.
H Proof of Theorem
We have the following error decomposition
ng (Z)))) - [[(((Z)))))
DDD vin
g
(g (p(Zoc) PP(((Zz)) ())
71
(lg) (Zg))))
1=1
where that (Z s 1 ng. empirical process can be bounded using the same technique as in the proof of Theorem
S g G. may share the same distribution in general. The
G
(Z()))
< - Pp (Zyz) - (Z()))))
g=1
1=1
Eg
Var
EEth ((Zge) - (Z D)
=Pp
since independence still holds across clusters. For the conditional bias term, have (conditioning on the training set D)
IP IP|y (Zez) up (Zg:)]
Rgt (Yyt- ixxgg, S9t))
+ jo( (XX Sgt)) 1a( ux(g Sgr)
E
# a(g,; Szr)
Tie Xg, H (Hit MMgg((X, (x Hgt) ju(X Sgt) ( Syr) p(X,, Sar)
# *(X,, Stt
TX (p))( X, Sqt) (i(X Sgr)) + f(X Syr) p(Xg, Sqt)
$ 2(X, gr Szt)
(TT Sgr) A (X,, S 9t (M(X,) Sax) (x Sqt))
# X,, Spr)
28

---

where the second equation follows from conditioning on (X Hig) and the third equation follows from Assumption Hence we have
Plp(Zgt) (Z(ge))
| go Sgi) ; Sgt) ((X):St)))
Thus the conditional bias term can be bounded
Rg
(lg(Z - ((Z)e))
9=1
n2 ( Stt n (X,, gr) Sgx) 1xX,, Stt)s
=0
The asymptotic expansion is then proved. The asymptotic normality then follows from Hansen and Lee [Teorem 2] and Slutsky's ' theorem.
I Proof of Corollary 2
First, to bound the empirical process term, the conditional variance given the training set D can be expressed
G
Σ
Var
(Zgi) ((Zgi) D)
G
Cov (g()))) - 40 (ZZgi), ((Zgg) - (((gg) D)
g=1
Var (G(Zgi) ((Zgi) D) Var (G((zg) (((gg)) D)
9=1
Bz M (Zgi) (<Z
B2,, [
4 (Z
where we have used Cauchy Schwarz inequality to bound the covariance with variance and Ez,, means the expectation is taken over Zgi Now we have
G
Tly
Σ
(g())))) - ((z))) (D) )
ED
Var
-==1
ED [V EZZ, ((b(Zee)) ( Bz,,, ((G(Z))) 4 (o(Zi)))
g=1
ED [Ez,, (Z()))) 40 [2 ED [Ez,, ((b(Zg)) so ((Zz)) [3)
8=1 6g
Ezz, ED (G(Zgi) sp (Z Ez,, ED (G(Zg)) - (z(2))))
g=1 1
sup ED [(((z) (2)))
1=1 i
Σ ED ((((z) (
29

---

Thus we have
ng
Σ Var
Σ (o(z)) 4 (Zgi)) D
Or
Σ
n sup
sup ED (8(z) pp(2)7)
n
For the bias term, from equation the proof of Theorem
we have
ED
P Plg(Zgt) gZ))
Bn (X g: Sgr) x =Xx,, S.t) (llx) Sgt) Sqx)
ng
En (X, Stt - x (X,,, Spt) En ( p( 13
ng
Ex, Sgt [ED - T Sgt)) S., [ED( Sot) ((.g
en 9=1
supEn(f(xxs)) Tσx 8) sup (u(E( s) p(xx
a
TT (xx ( sup Ep (f s) ((xxs)))²
sup a
m,
Thus we conclude
R2
| Stt <X2g S u(Xg, Sgt)ll
g=1
=0p
sup (f 8) Txx 8))² sup ED(E(x, s) plæ, s))² 2.88
2,8
30


Reviews:
[{"review_id": "hyvSNPozhm", "reviewer": "Reviewer_HHgW", "summary": "This paper studies the problem of estimating the average response under cluster dependence and missing outcome data, with an application to human evaluations of language models. The authors adopt the doubly robust estimation framework and extend existing theory to allow for arbitrarily sized clusters and within-cluster dependence. They prove asymptotic normality under general conditions and propose appropriate variance estimators, including cluster-robust and bootstrap methods. The methods are validated through simulation and applied to a real-world dataset from OpenAssistant Conversations dataset.", "questions": "1. How does your method compare empirically to outcome-only or IPW-only estimators? Including these as baselines would help contextualize the benefit of the doubly robust approach.\n\n2. Is your variance estimator (Eq. 4) sensitive to very small cluster sizes? Are there any practical recommendations on the minimal cluster size for reliable inference?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper addresses a practically important issue in modern applications, especially in human-AI interaction scenarios such as LLM evaluation, where responses are often missing and clustered by user or conversation.\n\n2.  It extends the asymptotic properties of doubly robust estimators under clustered sampling without requiring bounded cluster sizes, which generalizes existing work.\n\n3. The experimental results on public dataset demonstrate the effectiveness of the proposed method.\n\n**Weaknesses**\n\n1. While the theoretical extension is technically solid, the core methodology is based on standard doubly robust estimation. There is no significant algorithmic innovation beyond existing frameworks.\n\n2. The experimental section is not extensive and somewhat limited. Only one real dataset is used, with no comparisons against other estimators (e.g., outcome-only, IPW-only, or naive methods); No sensitivity analyses are conducted (e.g., varying cluster size, degree of within-cluster correlation, or misspecification scenarios).\n\n3. The paper is dense and difficult to follow in places, especially in the theoretical sections. For example, the modeling of historical summaries (Sgt) in the sequential setting is not clearly explained or motivated, and implementation details (e.g., model architectures, embedding strategies) are sparse in the real-data section.", "was_revised": true, "final_justification": "After rebuttal, my concerns are properly addressed. Based on the paper quality and overall concerns, I will keep my score unchanged which lean to accept.", "rebuttal": "We thank the reviewer for the thoughtful and constructive comments, insightful summary of our key contributions, and positive feedback on the clarity, organization, and significance of our work. These suggestions have been valuable in improving the paper. Our responses to the weaknesses and questions are as follows:\n\n**Weaknesses:**\n1. We acknowledge that the doubly robust estimator itself is not new, and Section 3 focuses on examining its theoretical properties in clustered settings. However, a key contribution of this work also lies in Section 4, where we formalize the idea of incorporating summaries of historical information into the estimation procedure in the sequential setting—a novel approach with important applications in multiturn LLM evaluation.\n2. See our response to Q1 below for details on additional simulations.\n3. We have added the following discussion on the historical summaries $S_{gt}$ after Assumption 3 to help readers better understand their motivation and how they can be selected in practice:\n\n\"The variable $S_{gt}$ can be viewed as a sufficient summary of the historical information up to time $t$. In practice, the choice of $S_{gt}$ often requires domain knowledge. Common choices include average or cumulative measures of past information. For example, in mobile health studies, a user wears a fitness tracker that collects data daily. The device may fail to record data at time $t$ due to battery depletion, which depends on historical usage; in this case, $S_{gt}$ could represent the cumulative device usage over the past few days. In educational testing or tutoring systems, whether a student attempts a question at time $t$ may depend on the cumulative difficulty or frustration from earlier interactions. Here, $S_{gt}$ might be defined as the number of incorrect attempts or a difficulty-adjusted score accumulated up to time $t$. In the LLM evaluation setting, whether users provide feedback may depend on their prior interactions with the system. Accordingly, $S_{gt}$ can be constructed as the embedding of the concatenated conversation history $\\overline{W}_{gt}$  up to time $t$, or from the most recent $d$ conversations. These embeddings remain of fixed length regardless of the length of the conversation history.\"\n\n**Questions:**\n1. We have included additional simulation studies in the appendix to compare the regression estimator, IPW estimator, and DR estimator in the clustered setting under potential model misspecification as follows:\n\n\"In this section, we present additional simulation results for the plug-in (regression) estimator \n$$\n\\hat{\\theta}_{\\mathrm{OR}} = \\frac{1}{n} \\sum\\_{g=1}^G \\sum\\_{i=1}^{n\\_g} \\hat{\\mu}(X\\_g, W\\_{gi}),\n$$\n\nthe IPW estimator\n$$\n\\hat{\\theta}_{\\text{IPW}} = \\frac{1}{n} \\sum\\_{g=1}^G \\sum\\_{i=1}^{n\\_g} \\frac{R\\_{gi}Y\\_{gi}}{\\hat{\\pi}(X\\_g, W\\_{gi})},\n$$\nand the doubly robust estimator in equation (2). Our focus is on evaluating their performance under model misspecification in the presence of clustered data.\n\nThe data-generating process follows the homogeneous sampling setup described in Appendix C, with the regression function and propensity score given by\n$$\n\\mu(X\\_g, W\\_{gi}) = -X\\_g + W\\_{gi}^2, \\quad \\pi(X\\_g, W\\_{gi}) = \\text{logistic}(X\\_g + 0.5W\\_{gi}^2).\n$$\nThe true average outcome is $ \\theta = 5 $. To assess estimator performance under misspecification, we consider scenarios in which $ \\mu $ and/or $ \\pi $ are misspecified by modeling the quadratic term in $ W\\_{gi} $ as linear.  \nWe generate samples of size $ n= 1000, 10000 $ with varying cluster sizes, apply each estimator under different model specifications, and compute the mean squared error (MSE). The results are summarized in the following table.\n\n**Table: Mean squared error (MSE) of regression, IPW, and DR estimators under potential nuisance misspecification with sample size $n = 10000$, $n_g = 100$.**\n\n| Scenario                        | Regression Estimator | IPW Estimator | DR Estimator |\n|--------------------------------|----------------------|---------------|--------------|\n| $\\mu$ correct, $\\pi$ correct   | 0.0562               | 0.0561        | 0.0563       |\n| $\\mu$ correct, $\\pi$ wrong     | 0.0563               | 2.7023        | 0.0563       |\n| $\\mu$ wrong, $\\pi$ correct     | 2.0356               | 0.0563        | 0.0572       |\n| $\\mu$ wrong, $\\pi$ wrong       | 2.0603               | 2.6896        | 2.5765       |\n\nThe conclusions in this clustered setting are similar to those in the classical i.i.d. setting. When the outcome model $ \\mu $ is misspecified, the plug-in (regression) estimator $ \\hat{\\theta}\\_{\\text{OR}} $ becomes inconsistent. Similarly, the consistency of the IPW estimator $ \\hat{\\theta}\\_{\\text{IPW}} $ relies on correct specification of the propensity score $ \\pi $. In contrast, the doubly robust estimator $ \\hat{\\theta}\\_{\\text{DR}} $, which models both the outcome and the missingness mechanism, remains consistent as long as either $ \\mu $ or $ \\pi $ is correctly specified. This aligns with the theoretical guarantees established in Theorem 1.\"\n\nWe also include two additional tables with different sample and cluster sizes. The results are similar to the table presented above.\n\n2. Our variance estimator in equation (4) is consistent under the assumptions stated in Theorem 1. In particular, the required scaling condition on cluster sizes is the same as in equation (3). While we allow cluster sizes to be unbounded, they need not be; the condition in (3) also accommodates small or bounded cluster sizes. Intuitively, although it may be difficult to estimate variance reliably for clusters with small sizes, their contribution to the overall variation is also small and becomes asymptotically negligible due to averaging the influence functions when calculating the DR estimator. Hence the variance estimator (4) is not sensitive to small cluster sizes. We have added the following discussion after equation (4) to clarify the consistency of the variance estimator:\n\n\"Under the conditions of Theorem 1, we can show that $\\hat{\\Omega}_n$ is a consistent estimator of $\\Omega_n$ in the sense that $\\hat{\\Omega}_n/\\Omega_n \\stackrel{P}{\\rightarrow} 1$. Therefore, $\\hat{\\Omega}\\_n / n$ can be used as a cluster-robust variance estimator for $\\hat{\\theta}\\_{\\text{DR}}$ to perform statistical inference.\""}, {"review_id": "hVVJQIWRMv", "reviewer": "Reviewer_XGKK", "summary": "This paper addresses the problem of estimating population means under missing outcome data with cluster-dependent observations, common in applications like LLM evaluation or education studies. Key contributions include:\n1.Theoretical guarantees for the DR estimator under cluster dependence, allowing growing cluster sizes and establishing convergence rates;\n2. A novel use of summary variables to simplify nuisance function modeling in sequential settings;\n3.Simulation and real-data studies (on OpenAssistant) demonstrating improved coverage and robustness over i.i.d.-based methods.", "questions": "Nuisance Function Estimation\nPlease clarify practical methods for estimating propensity scores and outcome models in clustered/sequential data, especially with varying cluster sizes. Concrete examples or recommended estimation procedures would strengthen the practical impact.\nRobustness to Misspecification\nDiscuss how the estimator performs if both nuisance models are misspecified in finite samples under cluster dependence. The doubly robust estimator is known for consistency when either nuisance model is correct. Could the authors discuss the estimator’s robustness in finite samples under simultaneous misspecification of both nuisance models, especially in clustered or dependent data contexts? Empirical or theoretical insights would help readers understand limitations.\nEffect of Large Clusters\nElaborate on the impact of very large cluster sizes on convergence rates and estimator performance, with intuition or examples.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\nQuality: Rigorous theoretical development extending doubly robust estimation to clustered and sequential dependent data, with asymptotic normality and convergence rates. Supported by simulation and real data analysis.\nClarity: Well-structured presentation with clear assumptions and theorems. Practical examples (e.g., LLM evaluation) help illustrate the methodology.\nSignificance: Addresses important problems in missing data under cluster dependence, allowing diverging cluster sizes, which is relevant for modern complex data structures.\nOriginality: Novel relaxation of bounded cluster size assumptions and formal incorporation of historical summary variables in sequential missingness models.\n\nWeaknesses\nQuality: Strong assumptions on model correctness and independent training samples; robustness to model misspecification could be further explored. Variance estimation may be challenging in practice.\nClarity: Dense notation and technical conditions may limit accessibility; limited guidance on choosing or constructing summary variables for sequential data.\nSignificance: Focuses mainly on mean estimation, leaving other causal parameters unexplored; computational considerations for large datasets are not discussed in depth.\nOriginality: Use of summary variables aligns with existing dimension reduction ideas; core doubly robust methods in clustered missing data are established, so contribution is primarily theoretical extension.", "was_revised": true, "final_justification": "thanks for the response from authors and i would like to keep my original rating.", "rebuttal": "We thank the reviewer for the thoughtful and constructive comments, insightful summary of\nour key contributions, and positive feedback on the clarity, organization, and significance of our work. These\nsuggestions have been valuable in improving the paper. Our responses to the weaknesses and questions are as follows:\n\n\n**Weaknesses:**\n\n1. While the assumption of an independent training sample might be strong in some applications,\nwe argue this can often be easy to satisfy in practice via sample splitting and cross-fitting (Chernozhukov\net al., 2018) to retain full-sample efficiency. For a discussion on model specification, see our response to Q2.\n\n2. We appreciate this point and have revised the submission to improve both presentation and practical guidance in a number of points. The choice of summary variable $S_{gt}$ depends on specific applications and often requires domain knowledge. Common choices include average or cumulative measures of historical information. We have added more discussion after Assumption 3 as follows:\n\n\"The variable $S_{gt}$ can be viewed as a sufficient summary of the historical information up to time $t$. In practice, the choice of $S_{gt}$ often requires domain knowledge. Common choices include average or cumulative measures of past information. For example, in mobile health studies, a user wears a fitness tracker that collects data daily. The device may fail to record data at time $t$ due to battery depletion, which depends on historical usage; in this case, $S_{gt}$ could represent the cumulative device usage over the past few days. In educational testing or tutoring systems, whether a student attempts a question at time $t$ may depend on cumulative difficulty or frustration from earlier interactions. Here, $S_{gt}$ might be defined as the number of incorrect attempts or a difficulty-adjusted score accumulated up to time $t$. In the LLM evaluation setting, whether users provide feedback may depend on their prior interactions with the system. Accordingly, $S_{gt}$ can be constructed as the embedding of the concatenated conversation history $\\overline{W}_{gt}$ up to time $t$, or from the most recent $d$ conversations $W\\_{g,t-d+1}, \\dots, W\\_{gt}$. These embeddings remain of fixed length regardless of the length of the conversation history.\"\n\n**Questions:**\n\n1. The nuisance estimation problem is essentially a regression problem. In the initial\nsubmission, we discussed several regression methods suitable for clustered data in the paragraph\nbeginning at line 167, including multilevel GLM, kernel regression, random forests and neural networks.\nThe recommended approach depends on the specific objective. For instance, if interpretability is\nimportant, multilevel generalized linear models (GLMs) are preferred. If the goal is to avoid strong\nparametric assumptions, nonparametric methods such as kernel regression (Shimizu, 2024) or random\nforests (Young and Buhlmann, 2025) are recommended. Researchers can choose approaches to estimate\nthe nuisance functions based on domain knowledge and their objectives in practice.\n\n2. Theorem 1 implies the same consistency guarantees hold for the DR estimator in the clustered setting, i.e., it is consistent when either nuisance model is correctly specified. We include additional simulation studies in the appendix to compare the regression estimator, IPW estimator and DR estimator in the clustered setting under potential model misspecification to illustrate the advantage of the DR estimator as follows:\n\n\"In this section, we present additional simulation results for the plug-in (regression) estimator \n$$\n\\hat{\\theta}_{\\mathrm{OR}} = \\frac{1}{n} \\sum\\_{g=1}^G \\sum\\_{i=1}^{n\\_g} \\hat{\\mu}(X\\_g, W\\_{gi}),\n$$\n\nthe IPW estimator\n$$\n\\hat{\\theta}_{\\text{IPW}} = \\frac{1}{n} \\sum\\_{g=1}^G \\sum\\_{i=1}^{n\\_g} \\frac{R\\_{gi}Y\\_{gi}}{\\hat{\\pi}(X\\_g, W\\_{gi})},\n$$\nand the doubly robust estimator in equation (2). Our focus is on evaluating their performance under model misspecification in the presence of clustered data.\n\nThe data-generating process follows the homogeneous sampling setup described in Appendix C, with the regression function and propensity score given by\n$$\n\\mu(X\\_g, W\\_{gi}) = -X\\_g + W\\_{gi}^2, \\quad \\pi(X\\_g, W\\_{gi}) = \\text{logistic}(X\\_g + 0.5W\\_{gi}^2).\n$$\nThe true average outcome is $ \\theta = 5 $. To assess estimator performance under misspecification, we consider scenarios in which $ \\mu $ and/or $ \\pi $ are misspecified by modeling the quadratic term in $ W\\_{gi} $ as linear.  \nWe generate samples of size $ n= 1000, 10000 $ with varying cluster sizes, apply each estimator under different model specifications, and compute the mean squared error (MSE). The results are summarized in the following table.\n\n**Table: Mean squared error (MSE) of regression, IPW, and DR estimators under potential nuisance misspecification with sample size $n = 10000$, $n_g = 100$.**\n\n| Scenario                        | Regression Estimator | IPW Estimator | DR Estimator |\n|--------------------------------|----------------------|---------------|--------------|\n| $\\mu$ correct, $\\pi$ correct   | 0.0562               | 0.0561        | 0.0563       |\n| $\\mu$ correct, $\\pi$ wrong     | 0.0563               | 2.7023        | 0.0563       |\n| $\\mu$ wrong, $\\pi$ correct     | 2.0356               | 0.0563        | 0.0572       |\n| $\\mu$ wrong, $\\pi$ wrong       | 2.0603               | 2.6896        | 2.5765       |\n\nThe conclusions in this clustered setting are similar to those in the classical i.i.d. setting. When the outcome model $ \\mu $ is misspecified, the plug-in (regression) estimator $ \\hat{\\theta}\\_{\\text{OR}} $ becomes inconsistent. Similarly, the consistency of the IPW estimator $ \\hat{\\theta}\\_{\\text{IPW}} $ relies on correct specification of the propensity score $ \\pi $. In contrast, the doubly robust estimator $ \\hat{\\theta}\\_{\\text{DR}} $, which models both the outcome and the missingness mechanism, remains consistent as long as either $ \\mu $ or $ \\pi $ is correctly specified. This aligns with the theoretical guarantees established in Theorem 1.\"\n\nWe also include two additional tables with different sample and cluster sizes. The results are similar to the table presented above.\n\n3. We have added an additional example in Appendix B to illustrate how heterogeneous cluster sizes can affect convergence rates:\n\n\"Consider a setting with two types of cluster sizes: size $1$ and size $n^\\alpha$. There are $n/2$ clusters of the first type and $n^{1-\\alpha}/2$ clusters of the second type, so the total number of clusters is \n$$\n    G = \\frac{n}{2} + \\frac{n^{1-\\alpha}}{2} \\asymp n.\n$$\n    Within each cluster, assume the individual influence functions and their estimates are all identical with unit variance. Then \n$$\n    \\Omega_n = \\frac{n + n^{2\\alpha} \\cdot n^{1-\\alpha}}{2n} \\asymp n^{\\alpha},\n$$\n    and the resulting convergence rate is \n$$\n    \\sqrt{\\frac{\\Omega_n}{n}} \\asymp n^{-(1-\\alpha)/2},\n$$\n    which is slower than both $\\sqrt{n}$ and $\\sqrt{G}$, since $G \\asymp n$. The corresponding rate condition for nuisance estimation is\n$$\n    \\|\\hat{\\mu} - \\mu\\|  \\|\\hat{\\pi} - \\pi\\| = o_{\\mathbb{P}}\\left(n^{-(1-\\alpha)/2}\\right), \\quad \\|\\hat{\\varphi} - \\varphi\\| = o_{\\mathbb{P}}(1).\n$$\n    This example highlights the importance of accounting for heterogeneous cluster sizes. Although the total number of clusters $G$ is large and of the same order as $n$, the convergence rate is driven by the relatively small number of large clusters, within which the correlation is perfect.\""}, {"review_id": "DAXdDrT2hq", "reviewer": "Reviewer_564S", "summary": "This paper addresses the problem of estimating average outcomes from human annotations in settings where data points are clustered (e.g. multiple responses from the same user) and some outcome values are missing. The authors focus on the doubly robust estimator and investigate its properties under cluster-dependent data. The authors derive theoretical results showing that the estimator remains asymptotically unbiased and normal under cluster sampling, even allowing cluster sizes to grow with the sample. The paper includes simulation studies demonstrating the importance of accounting for cluster structure. The paper also includes a real-world case study on the OpenAssistant conversation dataset, where the method is applied to estimate average conversation quality metrics with simulated missing ratings. The adjusted estimates with cluster-robust inference are shown to correct the bias of naive estimates and provide valid confidence intervals. Overall, the main contributions of this paper are a theoretical extension of doubly robust estimation to clustered and sequential settings, and empirical evidence underlining the need for cluster-aware inference in human feedback evaluation.", "questions": "1. Could the authors provide more guidance or discussion on choosing the summary variable $S_{gt}$ for sequential data?\n2. The paper uses a cluster robust variance estimator and mentions cluster-level bootstrap as a more robust alternative. Have the authors considered using the wild bootstrap or other resampling techniques specifically designed for cluster dependence?\n3. While the doubly robust estimator has great properties, some readers might wonder how it compares to simpler approaches, such as complete-case analysis (ignore missing), mean imputation, or even model-based imputation.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The authors extend the theory of doubly robust estimation to clustered data with possibly unbounded cluster sizes. They show the estimator is asymptotically normal with a variance inflation that depends on within-cluster correlation.\n2. The use of the doubly robust estimator is appropriate as it combines outcome modeling and propensity weighting, which offers protection against misspecification of either component. The paper proves the estimator’s double robustness and rate-adaptive properties in the clustered setting.\n3. The paper tackles temporal dependence within clusters which is very relevant for multi-turn interactions in conversational language models, through using summary statistics from conversation history.\n\nWeaknesses:\n1. While the theoretical extension to unbounded cluster sizes and sequential dependence is important, some aspects may be incremental. The doubly robust estimator itself is not new, and previous works (e.g. Yang 2018; Park & Kang 2021) have studied missing data or treatment effects in clustered settings. The contribution is more on the theoretical side than a new algorithm.\n2. The analysis relies on the missing at random assumption. If there are unobserved reasons why some responses are missing, the estimator could still be biased. The paper does not explore sensitivity to potential missing not at random scenarios, which could be relevant in real human feedback, e.g. users might skip rating bad responses. \n3. In the OpenAssistant dataset experiment, the missingness was simulated by the authors. While this is fine to illustrate the method, it would be even more convincing to apply the approach on a dataset with naturally occurring missing responses or non-response bias. \n4. The evaluation focuses on point estimates and CIs of the mean. It does not compare to alternative missing data techniques, such as multiple imputation or simple mean imputation.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful and constructive comments, insightful summary of our key contributions, and positive feedback on the clarity, organization, and significance of our work. These suggestions have been valuable in improving the paper. Our responses to the weaknesses and questions are as follows:\n\n\n**Weaknesses:**\n1. We acknowledge that the doubly robust estimator itself is not new, and Section 3 focuses on examining its theoretical properties in clustered settings. However, a key contribution of this work also lies in Section 4, where we formalize the idea of incorporating summaries of historical information into the estimation procedure in the sequential setting—a novel approach with important applications in multiturn LLM evaluation.\n2. We certainly appreciate the point that the missing at random (MAR) assumption can sometimes be strong. First, we want to emphasize that MAR is actually quite flexible in the scenarios we consider. For instance, the example provided by the reviewer---such as users being more likely to skip rating poor responses---can still fall within the MAR framework. In such cases, the missingness depends on the content of the system's response, which is captured by the variables $W_{gi}$ in our work. Since both our outcome model and propensity score model adjust for $W_{gi}$, our methods are capable of handling missing feedback that arises from poor system responses, as this information is explicitly incorporated into the estimation and adjustment procedures. So the MAR framework includes a wide range of practical scenarios. However, if unobserved variables such as some user information\n also affects the missingness, the MAR assumption will be violated. Second, we agree with the reviewer that a sensitivity analysis approach would be promising here. While a complete exploration of this idea is outside the scope of the current paper, we can readily combine our novel results on doubly estimation with several popular sensitivity analysis frameworks. We now discuss this briefly after Assumption 1:\n\n\"When unobserved confounders may influence the missing mechanism, the MAR assumption may no longer hold. To assess the sensitivity of our results to such violations, we can follow the framework of Cinelli and Hazlett (2020), which extends classical omitted variable bias analysis. This approach quantifies the strength that an unobserved confounder would need to exhibit—measured by its partial $R^2$ with both the treatment or missingness indicator and the outcome—to reduce the estimated effect to zero or render it statistically insignificant. The robustness value (RV) summarizes this threshold and can be benchmarked against observed covariates for interpretation.\"\n\n3. We agree with the reviewer that applying our approach to a dataset with naturally occurring missing responses would provide a more compelling demonstration. And while we have been actively exploring a range of different applications, publicly available data are unfortunately quite rare.\n    Given those constraints, we believe that simulating the missingness also offers important advantages. In particular, it allows us to treat the full-sample average as the \"ground truth,\" enabling a clear evaluation of our methods---such as assessing whether the resulting confidence intervals achieve the desired coverage.\n\n4. See our response to Q4 for details.\n\n**Questions:**\n1. The choice of summary variable $S_{gt}$ depends on specific applications and often requires domain knowledge. Common choices include average or cumulative measures of historical information. We have added more discussion after Assumption 3 as follows:\n\n\"The variable $S_{gt}$ can be viewed as a sufficient summary of the historical information up to time $t$. In practice, the choice of $S_{gt}$ often requires domain knowledge. Common choices include average or cumulative measures of past information. For example, in mobile health studies, a user wears a fitness tracker that collects data daily. The device may fail to record data at time $t$ due to battery depletion, which depends on historical usage; in this case, $S_{gt}$ could represent the cumulative device usage over the past few days. In educational testing or tutoring systems, whether a student attempts a question at time $t$ may depend on cumulative difficulty or frustration from earlier interactions. Here, $S_{gt}$ might be defined as the number of incorrect attempts or a difficulty-adjusted score accumulated up to time $t$. In the LLM evaluation setting, whether users provide feedback may depend on their prior interactions with the system. Accordingly, $S_{gt}$ can be constructed as the embedding of the concatenated conversation history $\\overline{W}_{gt}$ up to time $t$, or from the most recent $d$ conversations $W\\_{g,t-d+1}, \\dots, W\\_{gt}$. These embeddings remain of fixed length regardless of the length of the conversation history.\"\n\n2. Thank you for pointing out additional resampling methods for variance estimation. In addition to the bootstrap approaches for clustered data discussed after Equation (4), we have now included further discussion on the use of cluster wild bootstrap, which is particularly useful in settings with heteroskedasticity or within-cluster dependence:\n\n\"An alternative is the cluster wild bootstrap, which is well-suited for settings with heteroskedasticity, few\nclusters, or varying cluster sizes (MacKinnon and Webb, 2017).\"\n\n3. This is a good point, and we agree that expanding our comparison with simpler approaches is important. \nIn both our simulations and real-data analysis, we now include complete-case analysis---which ignores missing data---as a baseline. As expected, it yields biased estimates under the MAR setting because it fails to account for the missingness mechanism. Similarly, simple mean imputation is also biased. To compare the doubly robust estimator with model-based imputation, we note that the doubly robust approach can be viewed as a form of imputation using the \"pseudo-outcome\" or influence function $\\varphi$ defined in Theorem 1. In contrast, model-based imputation typically relies on regression and corresponds to the outcome regression estimator, which models only $\\mu$ and is consistent only if the outcome model is correctly specified. The doubly robust estimator combines both the propensity score and the outcome model, and remains consistent if either is correctly specified. In this sense, it provides a more robust imputation strategy. Moreover, when both models are correctly specified, it achieves semiparametric efficiency. Overall, the doubly robust estimator offers clear advantages over simpler alternatives. We have added the following sentence after the plug-in estimator on line 130 to connect it with regression-based imputation:\n\n\"This estimator corresponds to regression-based imputation and is consistent (under mild conditions)\nwhen $\\hat{\\mu}$ is consistent. However, the plug-in-style estimator usually suffers from first-order bias and is\nnot robust to model misspecification (Bang and Robins, 2005; Funk et al., 2011)..\""}, {"review_id": "LVrdlLcG9i", "reviewer": "Reviewer_XqtA", "summary": "This paper addresses the challenge of evaluating language models using human feedback, where missing annotations and cluster dependence often occur. It analyzes the doubly robust estimator—widely used in missing data analysis and causal inference—and extends its theoretical properties under clustered data. The authors establish new asymptotic results, showing how accounting for within-cluster correlation leads to faster convergence rates. Through simulations and real-world datasets, they demonstrate that their approach enables more reliable statistical inference when handling missing responses and clustered structures.", "questions": "1. The paper does not sufficiently discuss the limitations of the proposed method or provide insights into future research directions and its potential impact on the broader research community.", "limitations": "Although the theoretical analysis is solid, the paper lacks validation on large-scale real-world datasets.", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1. The paper provides a clear formulation and notation, which helps readers quickly understand the problem setup and modeling assumptions.\n\n2. It systematically considers missing data under both simultaneous sampling and sequential sampling scenarios, with precise definitions and mathematical modeling for each case.\n\n3. The proposed method is thoroughly validated through both simulation studies and real-world data analysis, demonstrating its effectiveness and robustness in practical applications.\n\n### Weakness\n1. Although the theoretical analysis is solid, the paper lacks validation on large-scale real-world datasets.", "was_revised": true, "final_justification": "Thank you for the detailed and thoughtful rebuttal. I appreciate the authors’ efforts in addressing the concerns and clarifying key points. Although I have not thoroughly verified the mathematical derivations in the paper, the problem formulation and modeling appear clear and reasonable.\n\n***However, because this topic is outside my core area of expertise, I will retain my original assessment and leave the final judgment to the area chair.***", "rebuttal": "We thank the reviewer for the thoughtful and constructive comments, insightful summary of our key contributions, and positive feedback on the clarity, organization, and significance of our work. These suggestions have been valuable in improving the paper. Our responses to the weaknesses and questions are as follows:\n\n**Weaknesses:**\n\nWe agree that empirical validation is important. Our initial submission indeed \nincluded a real-data example in Section 6 to illustrate our methods (although not a large one, due to access limitations). The results highlight the importance of accounting for cluster dependence in multiturn LLM evaluation: confidence intervals based on the cluster-robust variance estimator successfully cover the true average for all outcomes of interest, whereas those that ignore the dependence structure fail when the outcome $Y$ is humor. This underscores the necessity of incorporating the dependence structure when analyzing multiturn LLM evaluation data.\nAs part of the rebuttal, we also add additional empirical experiments to demonstrate the advantages of our approach, including its double robustness—specifically, the consistency of the DR estimator when either nuisance function is correctly specified.\n\n**Questions:**\n\nIn the submission, we have highlighted several limitations of our work. For instance, as noted in the discussion following Theorem 1, practitioners often rely on i.i.d-based methods to estimate nuisance functions in clustered settings; however, the theoretical justification for this practice remains an open question. Additionally, at the end of Section 3, we mentioned that while cluster bootstrap methods can be used for variance estimation, they tend to be computationally intensive. Finally, Assumption 1 requires the existence of a summary variable, $S_{gt}$, and we emphasize that its specification is context-dependent and relies on domain knowledge. Finally, our approach relies on the MAR assumption—that is, all variables influencing the missingness mechanism are observed and included in the analysis. When this assumption may not hold, sensitivity analysis is necessary to assess the robustness of the results.\n\nIn the original version, we discussed two directions for future work: addressing instability when the propensity score is close to zero—potentially by incorporating balancing weights—and learning human annotation rules from labeled datasets to enable evaluation of LLMs without human labels. In this revision, we have added a third direction: developing nuisance estimation algorithms for dependent data and establishing their theoretical guarantees.\n\nRegarding the impact on the research community, we believe our contributions are twofold. From the perspective of the causal inference literature, our theoretical results provide justification for using doubly robust estimators in the analysis of clustered data, especially when the cluster sizes are unbounded. In the context of LLM evaluation, the proposed method is novel in its ability to handle tasks with missing human annotations. In particular, the idea of incorporating summaries of historical information into the estimation procedure has important applications in multiturn LLM evaluation. We have added the following discussion to the introduction to highlight these contributions.\n\n\"Our results provide theoretical justification for using doubly robust estimators in the analysis of clustered data, especially when the cluster sizes are unbounded. The proposed methods, such as incorporating summaries of historical information into the estimation procedure, have important applications in multiturn LLM evaluation with missing human annotations, as demonstrated in our real-data example.\""}]

Author rebuttal:

Meta-review: This paper extends doubly robust estimation to handle missing outcomes in clustered data with potentially unbounded cluster sizes. The theoretical contribution is proving asymptotic normality for the doubly robust estimator under cluster dependence without requiring bounded cluster sizes, generalizing existing work. The authors also introduce summary variables to handle sequential dependence within clusters.
 
The theoretical extension addresses a gap in existing works by allowing cluster sizes to grow with sample size while maintaining asymptotic properties (Reviewer XGKK, Reviewer 564S). It tackles a practical problem where clustered missing data is common (Reviewer HHgW, Reviewer XGKK), and the paper proves rate-adaptive properties in clustered settings (Reviewer 564S). The incorporation of historical summary variables for handling temporal dependence is interesting and well-motivated for sequential interactions (Reviewer XGKK, Reviewer 564S).

The contribution is theoretical rather than methodological, as the core doubly robust estimator is standard and previous work has studied missing data in clustered settings (Reviewer HHgW, Reviewer 564S). The experimental evaluation is somehow limited, using only one real dataset with simulated missing data scenario rather than real cases, and lacks comparisons with alternative methods like multiple imputation (Reviewer HHgW, Reviewer 564S). The strong assumption might not work well when missing is not at random scenarios, with limited exploration of robustness (Reviewer 564S).
Decision: Accept (poster)