LUQ : Long-text Uncertainty Quantification for LLM s (2024)

Abstract: Large Language Models (LLMs) have demonstrated remarkable capability in a variety of NLP tasks. However, LLMs are also prone to generate nonfactual content. Uncertainty Quantification (UQ) is pivotal in enhancing our understanding of a model’s confidence on its generation, thereby aiding in the mitigation of nonfactual outputs. Existing research on UQ predominantly targets short text generation, typically yielding brief, word-limited responses. However, real-world applications frequently necessitate much longer responses. Our study first highlights the limitations of current UQ methods in handling long text generation. We then introduce Luq and its two variations, a series of novel sampling-based UQ approaches specifically designed for long text. Our findings reveal that Luq outperforms existing baseline methods in correlating with the model’s factuality scores (negative coefficient of -0.85 observed for Gemini Pro). To further improve the factuality of LLM responses, we propose Luq-Ensemble, a method that ensembles responses from multiple models and selects the response with the lowest uncertainty. The ensembling method greatly improves the response factuality upon the best standalone LLM.

## introduction large language models (llms) have demonstrated significant prowess across a wide range of nlp tasks and are increasingly being used in various downstream applications @xcite . however, existing llms are susceptible to hallucination, often resulting in the generation of nonfactual or fabricated content @xcite . one way to predict the factuality of an llm's output without resorting to resource-intensive fact-checking procedures is by examining its uncertainty over a user query. moreover, accurate measurement of a model's confidence in its generated responses can enable the rejection of answers with high uncertainty, potentially reducing hallucinations and improving the factuality of the output @xcite . although uncertainty quantification (uq) is a well-researched area in machine learning @xcite , its application in the context of llms remains under-explored. one primary limitation is that previous studies on uq mostly require access to a model's internal states (e.g., logits) @xcite @xcite . however, many best-performing llms, such as gpt-4 (openai, 2023), gemini 1.0 pro @xcite , and claude 2.1 @xcite , are closed-source and only accessible via api calls. this limits the ability to directly analyze their internal processes. another challenge is that existing research on modeling uncertainty predominantly focuses on short responses, typically less than 10 words in length @xcite @xcite . this is in stark contrast to the more common use cases of llms, where responses to queries often far exceed this length, sometimes reaching hundreds of words. such disparity points to a need for new uq methods tailored for long-form text generated by llms. therefore, in this study we aim to answer the following research questions: rq1: are existing uq methods still effective in the context of long-text generation? rq2: if not, how can we effectively quantify llms' uncertainty for long-form answers? rq3: in what ways can uncertainty scores be utilized to enhance the factuality of model outputs? we explore uq for long-text generation (at least 100 words), with an emphasis on using factuality as the key metric of the models' performance. the main contributions of this paper are: tell me a bio of ramesses iv . ramesses iv (reigned 1153-1147 bc) was was the son of ramesses iii and his great royal wife, iset ta-hemdjert. he was the third pharaoh of the 20th dynasty of ancient egypt ... his tomb was discovered in 1898 by victor loret in kv35. ramesses iv (also spelled ramses iv), was the third pharaoh of the 20th dynasty of egypt. he reigned for six years, from 1150 to 1143 bc. ... his tomb in the valley of the kings (kv2) was discovered in 1898 by victor loret. ramesses iv, also known as ramesses menpehtyre, was the third pharaoh of the 20th dynasty of ancient egypt. his reign lasted from 1155 to 1149 bc. ramesses iv was the son of ramesses iii and queen tyti. ... his tomb was discovered in 1898 by howard carter in kv35. figure 1 : the illustration of the luq and luq-ensemble framework. given a question, various llms exhibit differing levels of uncertainty. we generate n sample responses from each llm and then assess the uncertainty based on the diversity of these samples (the luq metric). green highlights indicate consistency across responses (low uncertainty) and red highlights discrepancies (high uncertainty). the luq-ensemble method selects the response from the llm with the lowest uncertainty score as the final answer. ## luq in this section, we introduce our luq method and its two variations (luq-atomic and luq-pair) to estimate uncertainty in long text generation. the overall framework is illustrated in figure 1 . ## conclusion in this work, we first identify that existing uq methods are ineffective on long text generation. we therefore introduce luq, a novel uq method tailored for long-form text generation in llms. it overcomes the limitation of previous methods by calculating sentence level consistency. we conduct extensive experiments over six popular llms, such as gpt-4 and gemini 1.0 pro. we extend the existing factscore dataset with human validation and annotations for additional disease domain. our findings demonstrate that luq significantly improves the correlation with models' factuality scores over previous methods across various different setups and domains. luq serves as a reliable indicator of model's factuality performance. additionally, we present luq-ensemble, a model ensembling and selective question answering strat-8 egy, which showcases a promising avenue for enhancing the factual accuracy of llm outputs. this research not only advances our understanding of uq in the context of llms but also offers practical tools for improving the reliability and trustworthiness of ai-generated content.