Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models (2024)

Abstract: Large Language Models (LLMs) show promising results in language generation and instruction following but frequently “hallucinate”, making their outputs less reliable. Despite Uncertainty Quantification’s (UQ) potential solutions, implementing it accurately within LLMs is challenging. Our research introduces a simple heuristic: not all tokens in auto-regressive LLM text equally represent the underlying meaning, as “linguistic redundancy” often allows a few keywords to convey the essence of long sentences. However, current methods underestimate this inequality when assessing uncertainty, causing tokens with limited semantics to be equally or excessively weighted in UQ. To correct this, we propose Shifting Attention to more Relevant (SAR) components at both token- and sentence-levels for better UQ. We conduct extensive experiments involving a range of popular “off-the-shelf” LLMs, such as Vicuna, WizardLM, and LLaMA-2-chat, with model sizes extending up to 33B parameters. We evaluate various free-form question-answering tasks, encompassing domains such as reading comprehension, science Q&A, and medical Q&A. Our experimental results, coupled with a comprehensive demographic analysis, demonstrate the superior performance of SAR. The code is available at https://github.com/jinhaoduan/SAR.

## introduction large language models (llms) have shown remarkable capabilities in multi-round conversation @xcite , logical reasoning @xcite @xcite , and also disclose great potential in scientific discovery @xcite . for instance, chatgpt, bard, gpt-4, pre-trained on large-scale corpora and carefully aligned to human preferences @xcite ., * corresponding author: kaidi @xmath0.edu>. question: what is the ratio of the mass of an object to its volume? ground truth: density llms generation: density of an object predictive entropy-based uncertainty quantification density of an object 0.238 6.528 0.966 0.008 token entropy uncertainty + + + = 1.949 high uncertainty, refuse to answer shifting attention to relevance uncertainty quantification density of an object 0.238 6.528 0.966 0.008 uncertainty + + + = 0.650 low uncertainty, return generation . . . token-level shifting 0.757 0.057 0.097 0.088 ## generative inequality in uncertainty quantification tokens are created unequally in reflecting the meaning of the generation yet they are treated equally when estimating uncertainty. we term these inequalities as generative inequalities and investigate how they affect uncertainty quantification. ## shifting attention to relevance a natural hypothesis derived from section 3.4 is that shifting the attention to those relevant components may benefit uncertainty quantification. in this section, we introduce the proposed shifting attention to relevance (sar) in detail. ## conclusion in this paper, we disclose the generative inequality observation in uncertainty quantification: tokens and generations are created unequally in reflecting semantics yet they are treated equally when estimating uncertainty, which is counter-intuitive. we propose to tackle these inequalities by shifting attention to relevance (sar) from both token-level (tokensar) and sentence-level (sentsar). experiments over "off-the-shelf" llms demonstrate the superior performances of sar.