Know Your Limits: A Survey of Abstention in Large Language Models (2025)

Abstract: Abstention, the refusal of large language models (LLMs) to provide an answer, is increasingly recognized for its potential to mitigate hallucinations and enhance safety in LLM systems. In this survey, we introduce a framework to examine abstention from three perspectives: the query, the model, and human values. We organize the literature on abstention methods, benchmarks, and evaluation metrics using this framework, and discuss merits and limitations of prior work. We further identify and motivate areas for future research, such as whether abstention can be achieved as a meta-capability that transcends specific tasks or domains, and opportunities to optimize abstention abilities in specific contexts. In doing so, we aim to broaden the scope and impact of abstention methodologies in AI systems.1

## introduction large language models (llms) have demonstrated generalization capabilities across nlp tasks such as question answering (qa) @xcite , abstractive summarization @xcite , and dialogue generation @xcite . but these models are also unreliable, having a tendency to ''hallucinate'' false information in their responses @xcite , generate overly certain or authoritative responses @xcite , answer with incomplete information @xcite , or produce harmful or dangerous responses @xcite . in these situations, the model should ideally abstain: to refuse to answer in the face of uncertainty @xcite @xcite . current methods to encourage abstention typically rely on calibration techniques, including linguistic calibration @xcite , which aim to accurately and consistently estimate a model's confidence in its response, then arrange for the model to abstain if the confidence score for a given response falls below some threshold @xcite @xcite . but questions of whether a query is aligned with human values or is answerable at all are difficult to model in terms of model confidence @xcite . prior work demonstrates the potential of abstention to enhance model safety and reliability in constrained settings @xcite @xcite . in this survey, we attempt to bring together relevant work studying abstention strategies or leading to abstention behaviors, across the diverse range of scenarios encountered by general-purpose chatbots engaging in open-domain interactions. our goals are to identify gaps and encourage new methods to achieve abstention. developing or adapting abstention mechanisms to suit a wide array of tasks will enhance the overall robustness and trustworthiness of llm interactions. to this end, our survey presents an overview of the current landscape of abstention research. we provide a definition of abstention that incorporates not only technical perspectives-query examination and model capabilities-but also considers alignment with human values. we categorize existing methods to improve abstention in llms based on the model lifecycle (pretraining, alignment, and inference), and provide an analysis of evaluation benchmarks and metrics used to assess abstention. in our discussion, we aim to establish a clear entry point for researchers to study the role of abstention across tasks, facilitating the incorporation of new abstention techniques into future llm systems. we summarize our contributions below: ## abstention in llms definition we define abstention as the refusal to answer a query. when a model fully abstains, it may begin a response with ''i don't know'' or refuse to answer in another way. in reality, abstention encompasses a spectrum of behaviors @xcite , e.g., expressing uncertainty, providing conflicting conclusions, or refusing due to potential harm are all forms of abstention. partial abstention may involve both answering and abstention, such as self-contradictory responses, e.g., ''i can't answer the question, but i suppose the answer might be...'' we do not consider ignoring and/or reframing the question as abstention; but rather as failure modes of llms in following instructions @xcite . for the abstention expression-the words a model uses to convey that it has abstained-we adopt the definition of five major types of expressions from prior work @xcite , indicating that the model (i) cannot assist; (ii) refutes the query; (iii) provides multiple perspectives without expressing preference; (iv) perceives risk associated with the query and answers cautiously with a disclaimer; and (v) refuses to offer concrete answers due to the lack of knowledge or certainty. expressions can be identified through heuristic rules and key word matching @xcite @xcite , or through model-based or human-based evaluation ( §4). below, we describe and motivate our framework for analyzing abstention behavior ( §2.1), then provide a formal definition of its components ( §2.2). ## abstention methodology we summarize methods introduced in prior work (figure 2 organizes these by stages in the llm lifecycle) and provide ideas for future experiments. ## evaluation of abstention we survey evaluation benchmarks ( §4.1) and metrics ( §4.2) used to assess abstention capabilities. ## future directions there are many under-explored and promising research directions in abstention, some of which are described in this survey. while prior work has explicitly investigated abstention in specific tasks or implicitly contributed to improved abstention behaviors, we encourage study of abstention as a meta-capability across tasks, as well as more generalizable evaluation and customization of abstention capabilities to user needs. beyond what has been discussed previously, other important directions include: (i) enhancing privacy and copyright protections through abstention-aware designs to prevent the extraction of personal private information and copyrighted text fragments; ## conclusion our survey underscores the importance of strategic abstention in llms to enhance their reliability and safety. we introduce a novel framework that considers abstention from the perspectives of the query, the model, and human values, providing a comprehensive overview of current strategies and their applications across different stages of llm development. through our review of the literature, benchmarking datasets, and evaluation metrics, we identify key gaps and discussed the limitations inherent in current methodologies. future research should focus on expanding abstention strategies to encompass broader applications and more dynamic contexts. by refining abstention mechanisms to be more adaptive and context-aware, we can further the development of ai systems that are not only more robust, reliable, and aligned with ethical standards and human values, but balance these goals more appropriately against helpfulness to the user.