Reconfidencing LLM s from the Grouping Loss Perspective (2024)

Abstract: Large Language Models (LLMs), such as GPT and LLaMA, are susceptible to generating hallucinated answers in a confident tone. While previous efforts to elicit and calibrate confidence scores have shown some success, they often overlook biases towards certain groups, such as specific nationalities. Existing calibration methods typically focus on average performance, failing to address this disparity. In our study, we demonstrate that the concept of grouping loss is an effective metric for understanding and correcting the heterogeneity in confidence levels. We introduce a novel evaluation dataset, derived from a knowledge base, specifically designed to assess the confidence scores of LLM responses across different groups. Our experimental results highlight significant variations in confidence, which are accurately captured by grouping loss. To tackle this issue, we propose a new method to calibrate the confidence scores of LLMs by considering different groups, a process we term reconfidencing . Our findings indicate that this approach effectively mitigates biases against minority groups, contributing to the development of fairer LLMs.

## introduction while large language models (llms) such as chatgpt @xcite and llama @xcite can generate responses that are fluent and plausible, they can also provide incorrect and untruthful information in a confident and compelling tone. this phenomenon, often called hallucination, poses a notable challenge to their use @xcite . in response, extensive research has focused on estimating the confidence (or uncertainty) of llm answers @xcite . ## analyzing the grouping loss in llms in this section, we aim to measure the calibration of existing confidence methods and identify the grouping loss in llms. ## reconfidencing llms in this section, we present a simple yet effective solution to reconfidence llms. the core idea is to calibrate each sub-group separately. standard calibration following standard calibration procedures, we train a regressor, commonly known as a calibrator, to conduct the calibration of a model @xcite this calibrator works by mapping the model's output to a refined probability within the interval [0, 1], with the aim of aligning closely with the true probability. concretely, we train an isotonic regressor using our constructed training and validation sets for calibration purposes @xcite . subsequently, we apply this trained regressor to calibrate the confidence scores on the test set. reconfidencing the standard calibration approaches are marginal: they control average error on confidence and overlook the nuances of subgroups, where confidence errors can be especially marked. inspired by this, we propose a more refined method to calibrate llms from the sub-group perspective. adapting @xcite , a tree classifier is trained to know how to partition samples (see details in section 3.4). we employ a loss function derived from the squared loss for the brier score on labels (y ) to optimize the predicted confidence scores. this decision tree algorithm partitions the data into sub-groups that minimize the specified loss. the tree's input consists of embeddings from the top layer of a llm for a given query, which can effectively cluster samples exhibiting similar levels of over-confidence or figure 4 : comparing calibrations across different popularity groups for the mistral-7b. we use merged results of three regions. the confidence method here is selfcheckgpt. more ⋆ symbols mean a sub-group with more popular samples. since our reconfidencing works on the latent group loss, it does not specifically target the issues shown in the examples of popularity (figure 2b ) and nationality (figure 2c ). to answer whether it improves the situation for these user-defined groups, we analyze calibration curves across sam-ples after calibration and reconfidencing. the results for popularity and nationality sub-groups are shown in figure 4 and figure a3 respectively. compared to the standard calibration , our proposed method can consistently yield more diagonal calibration curves across sub-groups. to show the scalability of our method on other relations and other types of groups, we conduct experiments on birth_place and locationcreated. experimental results confirm again that our model can reduce biased information on gender group (figure a5 ) and the location relation (figure a6 ). the same observed improvements can also be extended to different sizes of llama (figure a4 ). ## conclusion in this work, we analyzed how trustworthy current methods are when they give confidence scores to llm answers. we create a novel dataset derived from the ground truth within the yago knowledge base, providing a framework for evaluating the calibration of confidence scores for different groups. subsequent evaluations of different sizes of llms reveal a consistent discrimination towards particular minority groups. we show grouping loss in llms, such as those associated with long-tail entities and individuals of asian origin. these findings emphasize that we should pay particular attention to minority groups when calibrating llms. building upon these insights, we introduce a novel approach for reconfidencing llms based on latent sub-groups, resulting in improved calibrations. this new approach can mitigate the problem of hallucinations by generating alerts in response to llm answers. meanwhile, our findings can reduce biased information against groups such as race and gender, which is useful for the fairness of llms.