Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generation (2024)

Abstract: The advent of large language models (LLMs) has dramatically advanced the state-of-the-art in numerous natural language generation tasks. For LLMs to be applied reliably, it is essential to have an accurate measure of their confidence. Currently, the most commonly used confidence score function is the likelihood of the generated sequence, which, however, conflates semantic and syntactic components. For instance, in question-answering (QA) tasks, an awkward phrasing of the correct answer might result in a lower probability prediction. Additionally, different tokens should be weighted differently depending on the context. In this work, we propose enhancing the predicted sequence probability by assigning different weights to various tokens using attention values elicited from the base LLM. By employing a validation set, we can identify the relevant attention heads, thereby significantly improving the reliability of the vanilla sequence probability confidence measure. We refer to this new score as the Contextualized Sequence Likelihood (CSL). CSL is easy to implement, fast to compute, and offers considerable potential for further improvement with task-specific prompts. Across several QA datasets and a diverse array of LLMs, CSL has demonstrated significantly higher reliability than state-of-the-art baselines in predicting generation quality, as measured by the AUROC or AUARC.

## introduction the development of large language models (llms) has afforded tremendous advancements in natural language generation (nlg). recently, llms have been widely applied across various natural language domains @xcite @xcite , even extending to tasks and domains traditionally dominated by other machine learning algorithms, such as graph data @xcite , tabular data @xcite , time series @xcite , predictive chemistry @xcite , computer vision @xcite , amongst others. as llms continue to demonstrate outstanding performance, their reliability is increasingly scrutinized. uncertainty quantification, an area of research that can provide some guidance on reliability, has recently gained much attention. despite its long history in other machine learning tasks @xcite , our understanding of uncertainty quantification in nlg remains relatively limited. during pre-training, (autoregressive) llms are optimized to predict high logits for the target token to minimize (variants of) the negative log likelihood. consequently, one of the most natural and widely used confidence scores in selective nlg, conformal nlg, or uncertainty quantification is the (sometimes normalized) likelihood of the sequence, equivalent to the sum/mean of token logits. at first glance, sequence likelihood appears to be the most faithful reflection of a model's confidence, as it represents the (log of) the predicted probability of the output sequence, or log p(s|x). however, this measure often lacks proper contextualization and disregards the specific nature of the task at hand. p(s|x) conflates syntactic and semantic likelihoods, though we typically prioritize the semantic aspect, to a varying degree depending on the task. for example, depending on whether the question is "which country won the world cup in 2022" or "when did messi win the world cup," the answer "messi emerged victorious in the 2022 world cup" could be considered correct or incorrect. further, the somewhat unusual expression here could adversely impact p(s|x). despite its limitations, relatively limited attention has been paid to improving the vanilla sequence likelihood. a recent study by @xcite proposed assessing token relevance using an external natural language inference (nli) model. the relevance score of a token is negatively proportional to the similarity between the original sequence and the sequence with this token removed, which is then applied to weight the token logits. however, since modern lms rely on sub-word tokenizers, removing one sub-word token at a time results in non-words, is computationally expensive for longer texts, and remains context-unaware. in this paper, we investigate the potential of utilizing the llm's own attention mechanism to develop a weighted sequence likelihood as an enhanced confidence score. specifically, the llm is prompted to concentrate on the relevant tokens in its own generation. the most appropriate attention heads are then employed to re-weight the original token logits. the main contributions of this paper are summarized as thus: ## related works with the rapid proliferation of llms and their swift adoption across various domains, uncertainty quantification (uq) for natural language generation (nlg) is a fast-growing area of research (see @xcite and references therein). adopting the language used by @xcite , uncertainty measures the predictive distribution, while confidence (our focus) further depends on the specific generation. a common approach involves reducing nlg to a de facto classification problem and utilizing or enhancing classical uq methods for classification @xcite @xcite @xcite . recognizing the unique challenges posed by the inherently high (potentially infinite) dimensionality of nlg, recent research increasingly considers uq for nlg from a sequence perspective @xcite @xcite . relatively less attention has been devoted to confidence measures. sequence likelihood, or the logprobability of the generated sequence, remains one of the most popular proxies for assessing the quality of individual answers @xcite @xcite . another natural approach, given the versatility and strong performance of llms, involves using prompts to elicit the llm's own confidence level @xcite @xcite @xcite @xcite . for freeform generation datasets, this is typically done by arranging answers as options and extracting the model's logits for each option. while our method also employs prompts, they primarily guide the model to focus on tokens relevant to the context of the question. another approach involves sampling additional generations and comparing their similarities @xcite , which demonstrates good discriminative capability but can be rather expensive. although this approach has the advantage that it can work for black-box llms. ensemble methods have also been proposed @xcite . recently, @xcite proposed an improvement to sequence likelihood by weighting the tokens using their importance, which is computed by removing one token at a time from the original sequence and computing the nli dissimilarity between the new and original sequences. the removal of sub-word tokens, however, introduces drastically grammatically incorrect sentences, which could confuse the nli model. in addition, such a method could incur high computation overhead by making n nli comparisons, where n is the length of the generation. in contrast, our method incurs no artificial grammatical errors and miniscule overhead. an important downstream application of confidence measures is abstention in llms. classification with rejection @xcite @xcite @xcite could be considered the direct antecedent of selective nlg-both aiming to determine when to trust a model. naturally, approaches similar to those in the classification with rejection literature have been applied to nlp applications @xcite calibration is another crucial and relevant topic that has been extensively studied @xcite @xcite . although calibration is not directly related to our primary focus-our main interest lies in ranking the confidence of different measures-we include results on the calibrated performance in the appendix. conformal prediction @xcite could also be considered a form of calibration at the distribution level, and has been extended to nlg to bound variants of error rates @xcite . ## conclusion in this paper, we explore enhancements to the widely used confidence measure, sequence likelihood, which serves as a quality metric for model generations in selective generation and risk control for natural language generation. we introduce contextualized sequence likelihood, or csl, a novel approach that utilizes attention weights on generated tokens to re-weight the logits in sequence likelihood computation. this new confidence measure surpasses existing methods across several popular datasets and large language models by more accurately predicting the accuracy of each response.