Calibrating Long-form Generations From Large Language Models (2024)

Abstract: To enhance Large Language Models’ (LLMs) reliability, calibration is essential—the model’s confidence scores should align with the likelihood of its responses being correct. However, traditional calibration methods typically rely on a binary true/false assessment of response correctness, unsuitable for long-form generations where an answer can be partially correct. Addressing this gap, we introduce a unified calibration framework, in which both the correctness of the LLMs’ responses and their associated confidence levels are treated as distributions across a range of scores. We develop three metrics for assessing LLM calibration and propose confidence elicitation methods based on self-consistency and self-evaluation. Our experiments demonstrate that larger models don’t necessarily guarantee better calibration, that various calibration metrics complement each other, and that self-consistency methods excel in factoid datasets. We also find that calibration can be enhanced through techniques such as fine-tuning, scaling the temperature. Finally, we illustrate one application of long-form calibration through selective answering in long-form responses, optimizing correctness within a constrained API budget.

## introduction confidence calibration in large language models (llms) aims to align the model's internal confidence with a probabilistic perspective of its answers' correctness (i.e. quality), enhancing reliability and interpretability for aiding human decisionmaking @xcite . people intuitively understand and utilize probabilities @xcite , making this approach crucial for practical applications. conventional calibration @xcite treats answer correctness as binary (true or false) and seeks to align the model's confidence with the likelihood of model's answer figure 1 : a comparison between short-form generation and long-form generation. the correctness of the shortform answer can either be true (1) or false (0), while the correctness of the long-form answer is typically a score between 0 and 1. both of these scores may vary across evaluators due to subjectivity, hence we conceptualize them as a distribution over @xcite . therefore, a single confidence score for longform outputs is ambiguous: it can either imply "i am x% confident that the answer is 100% correct" or "i am 100% confident that the answer is x% correct." the former fails to capture the graded notion of long-form answer correctness, while the latter focuses on self-evaluation of correctness, rather than calibration as it overlooks the confidence at specific correctness levels. addressing this challenge, we propose to conceptualize the model's confidence as distribution across scores between [0, 1] to capture the nuanced understanding of the model of each correctness level of the long-form answer, corresponding to the statement "i am x% confident that this answer is y% correct". moreover, we also view the correct-figure 2 : overview of our calibration framework. we prompt an llm to produce an answer to a specific question, assess the answer's correctness distribution using an evaluator (task-specific metric/gpt-4 metric/human metric), and determine the model's confidence distribution through self-evaluation or self-consistency approaches. finally, we calculate the calibration score by comparing the correctness and confidence distributions against our predefined metrics. our unified framework offers three key advantages. 1. generalizability: our framework applies to both long-form and short-form generation tasks by representing correctness and confidence as distributions, regardless of whether the correctness of task is binary, continuous, subjective, or objective. 2. flexibility: the framework is evaluation-metric agnostic, allowing the integration of any metric or confidence elicitation method, and can adapt as evaluation methods evolve. 3. interpretability: it provides a nuanced view of uncertainty, enabling decision-makers to assess confidence across multiple correctness levels, fostering greater transparency and trust in the model's outputs. we leverage our framework to measure calibration for several llms on multiple datasets across three long-form qa-asqa @xcite , eli5 @xcite , qampari (amouyal et al., 2022)-and one summarization task, cnndm @xcite . our results show that our methods excel over baselines by leveraging the model's nuanced confidence distribution, stronger llms like gpt-3.5 don't necessarily guarantee better calibration, that various calibration metrics complement each other, and llms exhibit better calibration on factoid datasets than more open-ended datasets. furthermore, our analysis highlights that fine-tuning and temperature scaling could enhance calibration. finally, we illustrate a practical application of long-form calibration: employing a cascading strategy @xcite for selective answering to optimize the cost-effectiveness of long-form text generation. in this approach, an open-source model initially handles queries and, based on its confidence levels-assessed using our system-a more advanced api model is engaged as needed. this method ensures cost efficiency while maintaining high-performance levels. in summary, our contributions are: ## related work measuring calibration calibration @xcite has been widely studied in language models, whose probabilities derived from logits are generally found to not be calibrated @xcite @xcite . standard metrics to measure the calibration include expected calibration error (ece) for confidence-accuracy disparity @xcite , brier score for mean squared prediction-outcome differences, and auroc for assessing confidence-based correct answer identification @xcite . selective accuracy@coverage measures accuracy within the model's most confident predictions @xcite . however, these metrics, rooted in a binary notion of correctness, fall short for long-form tasks where correctness spans a range, suggesting a distribution-based approach is more apt. improving calibration traditional calibration methods focus on post-processing logits @xcite , but with llms generating unbounded text, logits could fall short. thus, extracting better confidence scores (i.e., confidence elicitation) has become crucial for improving calibration. these include: verbalization, which directly asks the model to output its confidence @xcite , consistency, which uses the uniformity of multiple responses to gauge confidence @xcite @xcite @xcite , and the hybrid of both @xcite @xcite . however, these methods often presume binary answer correctness, offering a singular confidence score that fails to capture the nuanced correctness required for long-form tasks. recent work @xcite on long-form generation addresses continuous correctness scores but focuses on aligning an uncertainty score with correctness, rather than on improving calibration. ## long-form generations calibration this section formalizes the long-form generation calibration problem ( § 3.1), and introduces three core components in our calibration framework (figure 2 ): the correctness distributions of the answers ( § 3.2), the confidence distributions of the llm on its answers ( § 3.3), and the calibration metrics to measure how well these two align ( § 3.4). ## conclusion our study presents a novel calibration system for evaluating llms in long-form generation. our results challenge the assumption that larger llms are always calibrated better and show calibration variability across datasets. additionally, we propose methods to improve llm calibration and show an application that optimizes performance under api budget constraints. the system we present is crucial for further improving the liability of llms.