Conformal Prediction for Natural Language Processing: A Survey (2024)

Abstract: The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as Hallucinations and to enhance decision-making reliability in critical applications. Conformal prediction is emerging as a theoretically sound and practically useful framework, combining flexibility with strong statistical guarantees. Its model-agnostic and distribution-free nature makes it particularly promising to address the current shortcomings of NLP systems that stem from the absence of uncertainty quantification. This paper provides a comprehensive survey of conformal prediction techniques, their guarantees, and existing applications in NLP, pointing to directions for future research and open challenges.

## introduction natural language processing (nlp) is witnessing an explosive growth in applications and public visibility, namely with large language models (llms) being deployed in many real-life applications, ranging from general-purpose chatbots to the generation of medical reports @xcite . however, the widespread use of these models brings important concerns: hallucinations are frequent @xcite , models are poorly calibrated @xcite @xcite , evaluation is limited and sometimes affected by data contamination @xcite , explanations are often unreliable @xcite , and models often exhibit undesired biases @xcite . reliable uncertainty quantification is key to addressing some of these concerns: nlp systems should not only provide accurate answers but also ''know when they do not know''. unfortunately, some nlp systems return only single predictions (i.e., point estimates), without reliable confidence information. systems that quantify uncertainty are less common and typically limited in various ways: they often make incorrect distribution-based assumptions ignoring the complex nature of the underlying data and model @xcite @xcite ; they are often poorly calibrated (i.e., they predict a confidence level that does not match its error probability; @xcite ; and they may be computationally too demanding, thus inapplicable to large-scale models @xcite . conformal prediction (cp; @xcite has recently emerged as a promising candidate to bypass the issues above: unlike other uncertainty quantification frameworks, it offers statistical guarantees of ground-truth coverage with minimal assumptions. cp methods are model-agnostic and distribution-free, assuming only data exchangeability (as described in §3.1). moreover, extensions of cp that handle non-exchangeable data have recently been proposed @xcite . popular cp variants are also efficient: they do not require model retraining and can be used online or offline, given an additional relatively small calibration set. 1 finally, equalized variants of cp @xcite can also reduce biases and unfairness, by distributing coverage evenly across protected attributes. the flexibility and strong statistical guarantees of cp have attracted considerable interest, with an increasing number of publications in computer science. 2 it is therefore timely to present a survey of conformal methods for nlp, revealing the theory and guarantees behind these methods and outlining opportunities and challenges for them to tackle important problems in the field. scope. this survey provides a comprehensive overview of cp techniques for nlp tasks (figure 1 ). after briefly explaining cp and some relevant extensions ( §2 and §4), we review direct applications thereof in nlp ( §5). finally, we look at possible threads of future investigation and current open issues concerning the use of cp in nlp ( §6). what this survey is not about. this is not a general survey on uncertainty quantification and does not include techniques not based on cp. comprehensive reviews of uncertainty quantification in nlp were recently published by @xcite and @xcite . also, our survey is focused on nlp applications; @xcite and @xcite have published comprehensive surveys on cp . ## conformal predictors this section briefly explains cp and presents some definitions and results needed for understanding the applications mentioned below. in what follows, we use upper case letters (x, y, ...) for random variables, lower case letters (x, y, ...) for specific values they take, and calligraphic letters (x , y, c, ...) for sets. 2 the number of arxiv papers in the field of computer science containing the expression ''conformal prediction'' has been steadily rising, from 16 papers in 2018 to 224 in 2023. ## guarantees and evaluation prediction sets produced by conformal predictors have associated theoretical guarantees. this section reviews established coverage guarantees as well as evaluation metrics used to measure the quality of the prediction sets. ## extending conformal prediction cp has extended beyond classic conformal predictors, with developments that allow handling challenges such as conditional coverage, dispensing with exchangeability, or obtaining guarantees beyond coverage. this section briefly presents the core ideas of some of the extensions that are most relevant for nlp applications. ## future directions we outline in this section some promising future research directions and open challenges related to the use of cp and its many variants in nlp tasks. ## discussion significant efforts have been made to improve the quality and confidence estimation of nlp systems. in fact, recent techniques for generating text with llms often involve the generation of multiple hypotheses, followed by a reranking (or voting) stage to increase the likelihood of producing a high-quality prediction. these methods include minimum bayes risk decoding @xcite , reranking based on quality estimation @xcite , and other strategies @xcite . while these approaches have been shown to improve quality and reduce the number of generated hallucinations @xcite , they do not inherently quantify the uncertainty of their predictions or provide the formal guarantees offered by cp. sensitive tasks, such as those in fields like medicine and education, may require guarantees. in these cases, the system could abstain from answering or defer to a human whenever its confidence is below a certain threshold. cp provides theoretical guarantees in a distribution-free setting, making it highly useful in such scenarios. other calibration methods have been used to obtain uncertainty intervals for regression and classification tasks in nlp. calibration methods that target accuracy-related metrics, e.g., aiming to minimise the estimated calibration error (ece) have been widely used @xcite . still, they also suffer from significant drawbacks as they are sensitive to binning choices and small changes in model estimates @xcite @xcite . furthermore, they do not provide any formal guarantees to control risk and objectives of interest. cp can thus be used to complement such methods, especially in cases where ensuring coverage is deemed important. ## conclusion this paper provides an overview of applications of the conformal prediction framework in nlp tasks, after a brief introduction to that framework and its main variants. we showed how conformal prediction is a promising tool to address the uncertainty quantification challenge in nlp and hope the existing and possible applications presented in this survey will motivate future research on the topic.