Can Unconfident LLM Annotations Be Used for Confident Conclusions? (2025)

Abstract: Large language models (LLMs) have shown high agreement with human raters across a variety of tasks, demonstrating potential to ease the challenges of human data collection. In computational social science (CSS), researchers are increasingly leveraging LLM annotations to complement slow and expensive human annotations. Still, guidelines for collecting and using LLM annotations, without compromising the validity of downstream conclusions, remain limited. We introduce Confidence-driven inference: a method that combines LLM annotations and LLM confidence indicators to strategically select which human annotations should be collected, with the goal of producing accurate statistical estimates and provably valid confidence intervals while reducing the number of human annotations needed. Our approach comes with safeguards against LLM annotations of poor quality, guaranteeing that the conclusions will be both valid and no less accurate than if we only relied on human annotations. We demonstrate the effectiveness of Confidence-driven inference over baselines in statistical estimation tasks across three CSS settings—text politeness, stance, and bias—reducing the needed number of human annotations by over 25% in each. Although we use CSS settings for demonstration, Confidence-driven inference can be used to estimate most standard quantities across a broad range of NLP problems.

## introduction large language models (llms) have shown strong zero-shot performance across tasks @xcite , making them a promising tool for generating annotations, particularly when they align closely with human judgments @xcite . given this potential, llm annotations of textual data may ⋆ equal contribution. computational social science (css) research typically focuses not on the annotations themselves but on the social-science insights and conclusions they enable. thus, understanding how llm annotations could be used for downstream inferences is crucial in css. for example, stance annotations facilitate the study of linguistic differences between media affirming or denying global warming @xcite , while politeness annotations can help examine racial disparities in verbal interactions with law enforcement @xcite , the relationship between politeness and social power @xcite , and politeness and gender @xcite . similarly, annotating political leanings in text allows studying the bias of search engines @xcite , social media @xcite , and political discourse @xcite . precise statistical estimation, such as prevalence or regression coefficient estimation, is essential for drawing valid conclusions in such studies. however, whether llm annotations can be effectively leveraged without compromising the validity of statistical estimation remains uncertain. llms exhibit demographic biases @xcite and may lack factual accuracy @xcite and consistency @xcite . given these limitations, using llms without caution may lead to inaccurate conclusions and potential societal harms, especially when such conclusions influence policy or have tangible impacts on peoples' outcomes @xcite . a potential solution is to rely solely on human annotations; however, human annotations are costly. here, we present confidence-driven infer-ence, a method for valid statistical inference using llm annotations. given a text corpus and a quantity of interest, our approach builds on active inference @xcite to: (1) strategically choose a small number of human annotations, guided by llm annotations and the llm's verbalized confidence scores, and (2) combine the human and llm annotations into an accurate estimate of the quantity of interest (fig. 1 ). the resulting estimate is statistically valid, while reducing reliance on expensive human annotations. our task is statistical estimation of a quantity of interest. we evaluate our approach on five estimation tasks in three css settings (politeness, stance, and media bias) in terms of confidence interval coverage and effective sample size, which measures the increase in accuracy due to augmenting human with llm annotations (sec. 3.4). we find that naively treating llm annotations as human data can lead to highly inaccurate estimates and poor coverage. by contrast, our method maintains the target coverage, while outperforming the baselines (defined in sec. 3.3) in terms of the effective sample size. the latter is enabled partially by the fact that, in all tested settings, the verbalized confidence scores reflect llm accuracy. higher confidence scores correspond to higher accuracy with respect to human annotations, allowing for a strategic selection of a smaller number of human annotations. confidence-driven inference can be used to estimate a wide range of standard targets (such as regression coefficients, means, and prevalences) across various nlp problems. our code and data are available at https://github.com/kristinagligoric/confidencedriven-inference . ## results we evaluate our approach on a set of css problems that rely on statistical estimation. we aim to include settings that (1) allow addressing important downstream social-science questions, (2) rely on a human-labeled corpus of text instances (possibly with relevant additional covariates), and (3) have a publicly available dataset. we selected three settings that meet these criteria-politeness, stance, and political bias. for stance and politeness, we leverage publicly available datasets and the corresponding human annotations in their entirety. given the large size, for political leaning, we randomly sample a smaller subset of texts. ## discussion in this work, we introduce confidence-driven inference, a method that integrates verbalized confidence of llms with active inference to optimally combine human and llm annotations. across three distinct css settings, results demonstrate that the proposed method consistently outperforms baseline methods (human-only and nonadaptive approaches) in effective sample size. moreover, the increase in the effective sample size is achieved without a decrease in coverage. in contrast, the llm-only approach yields invalid estimates and considerably lower coverage. wer note that the external validity of our findings is contingent upon two key assumptions: that the text instances are i.i.d. from a relevant distribution, and that the researcher has full control of the annotation process. the first may be violated if the distribution of texts shifts over time, and the collected instances are no longer representative of the current quantity of interest. for example, it is pos-sible that relationships between linguistic devices and perceived politeness evolve over time. the second assumption may be violated in situations where certain annotations are difficult to obtain (e.g., for low-resource languages). our approach may lead to inaccurate or misleading conclusions under either violation. we thus caution against generalizing to settings where text instances exhibit time-varying shifts or the researcher is not in control over the data collection process. if the adaptive sampling probabilities π i are poorly chosen-potentially due to inaccurate verbalized confidence scores-the resulting estimates could have a higher mean squared error (mse) than if uniform, non-adaptive sampling were used. this could even result in an estimate with a larger mse than the human-only baseline (for sensitivity to miscalibration, see app b.2). however, by using power tuning, as detailed in section 3.2, we ensure that incorporating llm annotation into the estimation process does not hurt the estimate (i.e., does not increase the mse) regardless of the sampling method used for human annotations (whether uniform or adaptive). thus, confidence-driven inference allows for researchers to allocate human and llm annotations in a cost-effective manner while maintaining confidence in the statistical validity of their results. furthermore, confidence-driven in-ference also addresses the challenges posed by the variable quality of llm annotation, by providing validity guarantees when leveraging imperfect llm annotations. although overall llm annotations moderately agree with human annotations in the tested settings, relying on llm annotations only can lead to wrong conclusions, as shown in the example of estimating the odds ratio in the stance setting. in contrast, despite the fact that llm annotations are imperfect, our approach allows carefully combining them with a limited set of human annotations in order to reduce the human annotation budget, without sacrificing the validity. finally, the accessibility of our method is an important consideration. across disciplines, researchers can simply prompt the llm for its confidence via api access and leverage confidence-driven inference to combine llm confidence with llm and human annotations to produce a valid statistical estimate. this approach can be applied to a wide range of tasks, across fields.