Title: Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Missing data

cluster dependence

doubly robust estimation

causal inference

evaluation

human annotation
Abstract: Human annotations play a crucial role in evaluating the performance of GenAI models. Two common challenges in practice, however, are missing annotations (the response variable of interest) and cluster dependence among human-AI interactions (e.g., questions asked by the same user may be highly correlated). Reliable inference must address both issues to achieve unbiased estimation and appropriately quantify uncertainty when estimating average scores from human annotations. In this paper, we analyze the doubly robust estimator, a widely used method in missing data analysis and causal inference, applied to this setting and establish novel theoretical properties under cluster dependence. We further illustrate our findings through simulations and a real-world conversation quality dataset. Our theoretical and empirical results underscore the importance of incorporating cluster dependence in missing response problems to perform valid statistical inference.


Peer reviews:
comments: SUMMARY:
This paper addresses the challenge of evaluating language models using human feedback, where missing annotations and cluster dependence often occur. It analyzes the doubly robust estimator—widely used in missing data analysis and causal inference—and extends its theoretical properties under clustered data. The authors establish new asymptotic results, showing how accounting for within-cluster correlation leads to faster convergence rates. Through simulations and real-world datasets, they demonstrate that their approach enables more reliable statistical inference when handling missing responses and clustered structures.

STRENGTHS AND WEAKNESSES:
### Strengths

1. The paper provides a clear formulation and notation, which helps readers quickly understand the problem setup and modeling assumptions.

2. It systematically considers missing data under both simultaneous sampling and sequential sampling scenarios, with precise definitions and mathematical modeling for each case.

3. The proposed method is thoroughly validated through both simulation studies and real-world data analysis, demonstrating its effectiveness and robustness in practical applications.

### Weakness
1. Although the theoretical analysis is solid, the paper lacks validation on large-scale real-world datasets.

QUESTIONS:
1. The paper does not sufficiently discuss the limitations of the proposed method or provide insights into future research directions and its potential impact on the broader research community.

LIMITATIONS:
Although the theoretical analysis is solid, the paper lacks validation on large-scale real-world datasets.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of estimating average outcomes from human annotations in settings where data points are clustered (e.g. multiple responses from the same user) and some outcome values are missing. The authors focus on the doubly robust estimator and investigate its properties under cluster-dependent data. The authors derive theoretical results showing that the estimator remains asymptotically unbiased and normal under cluster sampling, even allowing cluster sizes to grow with the sample. The paper includes simulation studies demonstrating the importance of accounting for cluster structure. The paper also includes a real-world case study on the OpenAssistant conversation dataset, where the method is applied to estimate average conversation quality metrics with simulated missing ratings. The adjusted estimates with cluster-robust inference are shown to correct the bias of naive estimates and provide valid confidence intervals. Overall, the main contributions of this paper are a theoretical extension of doubly robust estimation to clustered and sequential settings, and empirical evidence underlining the need for cluster-aware inference in human feedback evaluation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The authors extend the theory of doubly robust estimation to clustered data with possibly unbounded cluster sizes. They show the estimator is asymptotically normal with a variance inflation that depends on within-cluster correlation.\n2. The use of the doubly robust estimator is appropriate as it combines outcome modeling and propensity weighting, which offers protection against misspecification of either component. The paper proves the estimator’s double robustness and rate-adaptive properties in the clustered setting.\n3. The paper tackles temporal dependence within clusters which is very relevant for multi-turn interactions in conversational language models, through using summary statistics from conversation history.\n\nWeaknesses:\n1. While the theoretical extension to unbounded cluster sizes and sequential dependence is important, some aspects may be incremental. The doubly robust estimator itself is not new, and previous works (e.g. Yang 2018; Park & Kang 2021) have studied missing data or treatment effects in clustered settings. The contribution is more on the theoretical side than a new algorithm.\n2. The analysis relies on the missing at random assumption. If there are unobserved reasons why some responses are missing, the estimator could still be biased. The paper does not explore sensitivity to potential missing not at random scenarios, which could be relevant in real human feedback, e.g. users might skip rating bad responses. \n3. In the OpenAssistant dataset experiment, the missingness was simulated by the authors. While this is fine to illustrate the method, it would be even more convincing to apply the approach on a dataset with naturally occurring missing responses or non-response bias. \n4. The evaluation focuses on point estimates and CIs of the mean. It does not compare to alternative missing data techniques, such as multiple imputation or simple mean imputation.\n\nQUESTIONS:\n1. Could the authors provide more guidance or discussion on choosing the summary variable $S_{gt}$ for sequential data?\n2. The paper uses a cluster robust variance estimator and mentions cluster-level bootstrap as a more robust alternative. Have the authors considered using the wild bootstrap or other resampling techniques specifically designed for cluster dependence?\n3. While the doubly robust estimator has great properties, some readers might wonder how it compares to simpler approaches, such as complete-case analysis (ignore missing), mean imputation, or even model-based imputation.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of estimating population means under missing outcome data with cluster-dependent observations, common in applications like LLM evaluation or education studies. Key contributions include:\n1.Theoretical guarantees for the DR estimator under cluster dependence, allowing growing cluster sizes and establishing convergence rates;\n2. A novel use of summary variables to simplify nuisance function modeling in sequential settings;\n3.Simulation and real-data studies (on OpenAssistant) demonstrating improved coverage and robustness over i.i.d.-based methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nQuality: Rigorous theoretical development extending doubly robust estimation to clustered and sequential dependent data, with asymptotic normality and convergence rates. Supported by simulation and real data analysis.\nClarity: Well-structured presentation with clear assumptions and theorems. Practical examples (e.g., LLM evaluation) help illustrate the methodology.\nSignificance: Addresses important problems in missing data under cluster dependence, allowing diverging cluster sizes, which is relevant for modern complex data structures.\nOriginality: Novel relaxation of bounded cluster size assumptions and formal incorporation of historical summary variables in sequential missingness models.\n\nWeaknesses\nQuality: Strong assumptions on model correctness and independent training samples; robustness to model misspecification could be further explored. Variance estimation may be challenging in practice.\nClarity: Dense notation and technical conditions may limit accessibility; limited guidance on choosing or constructing summary variables for sequential data.\nSignificance: Focuses mainly on mean estimation, leaving other causal parameters unexplored; computational considerations for large datasets are not discussed in depth.\nOriginality: Use of summary variables aligns with existing dimension reduction ideas; core doubly robust methods in clustered missing data are established, so contribution is primarily theoretical extension.\n\nQUESTIONS:\nNuisance Function Estimation\nPlease clarify practical methods for estimating propensity scores and outcome models in clustered/sequential data, especially with varying cluster sizes. Concrete examples or recommended estimation procedures would strengthen the practical impact.\nRobustness to Misspecification\nDiscuss how the estimator performs if both nuisance models are misspecified in finite samples under cluster dependence. The doubly robust estimator is known for consistency when either nuisance model is correct. Could the authors discuss the estimator’s robustness in finite samples under simultaneous misspecification of both nuisance models, especially in clustered or dependent data contexts? Empirical or theoretical insights would help readers understand limitations.\nEffect of Large Clusters\nElaborate on the impact of very large cluster sizes on convergence rates and estimator performance, with intuition or examples.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of estimating the average response under cluster dependence and missing outcome data, with an application to human evaluations of language models. The authors adopt the doubly robust estimation framework and extend existing theory to allow for arbitrarily sized clusters and within-cluster dependence. They prove asymptotic normality under general conditions and propose appropriate variance estimators, including cluster-robust and bootstrap methods. The methods are validated through simulation and applied to a real-world dataset from OpenAssistant Conversations dataset.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The paper addresses a practically important issue in modern applications, especially in human-AI interaction scenarios such as LLM evaluation, where responses are often missing and clustered by user or conversation.\n\n2.  It extends the asymptotic properties of doubly robust estimators under clustered sampling without requiring bounded cluster sizes, which generalizes existing work.\n\n3. The experimental results on public dataset demonstrate the effectiveness of the proposed method.\n\n**Weaknesses**\n\n1. While the theoretical extension is technically solid, the core methodology is based on standard doubly robust estimation. There is no significant algorithmic innovation beyond existing frameworks.\n\n2. The experimental section is not extensive and somewhat limited. Only one real dataset is used, with no comparisons against other estimators (e.g., outcome-only, IPW-only, or naive methods); No sensitivity analyses are conducted (e.g., varying cluster size, degree of within-cluster correlation, or misspecification scenarios).\n\n3. The paper is dense and difficult to follow in places, especially in the theoretical sections. For example, the modeling of historical summaries (Sgt) in the sequential setting is not clearly explained or motivated, and implementation details (e.g., model architectures, embedding strategies) are sparse in the real-data section.\n\nQUESTIONS:\n1. How does your method compare empirically to outcome-only or IPW-only estimators? Including these as baselines would help contextualize the benefit of the doubly robust approach.\n\n2. Is your variance estimator (Eq. 4) sensitive to very small cluster sizes? Are there any practical recommendations on the minimal cluster size for reliable inference?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-07"}