Clustering Zero-Shot Uncertainty Estimations to Assess LLM Response Accuracy for Yes/No Q & A (2025)

Abstract: The power of Large Language Models (LLMs) in user workflows has increased the desire to access such technology in everyday work. While the ability to interact with models provides noticeable benefits, it also presents challenges in terms of how much trust a user should put in the system’s responses. This is especially true for external commercial and proprietary models where there is seldom direct access and only a response from an API is provided. While standard evaluation metrics, such as accuracy, provide starting points, they often may not provide enough information to users in settings where the confidence in a system’s response is important due to downstream or real-world impact, such as in Question & Answering (Q&A) workflows. To support users in assessing how accurate Q&A responses from such black-box LLMs scenarios are, we develop an uncertainty estimation framework that provides users with an analysis using a Dirichlet mixture model accessed from probabilities derived from a zero-shot classification model. We apply our framework to responses on the BoolQ Yes/No questions from GPT models, finding the resulting clusters allow a better quantification of uncertainty, providing a more fine-grained quantification of accuracy and precision across the space of model output while still being computationally practical. We further demonstrate its generalizability and reusability of the uncertainty model by applying it to a small set of Q&A collected from U.S. government websites.

## introduction large language models (llms) have substantially influenced a multitude of workflow applications, such as question and answering (q&a) systems. while the expansive knowledge and response capabilities of generative models (e.g., gpt4) has been impressive, it also presents unique challenges in workflow integration, namely user trust and certainty in answers and responses. this is especially pertinent when a q&a system is designed for nonsubject matter experts who will not be familiar with the response quality of the domain. this need has resulted in growing research in uncertainty estimation to better assess the quality of a response an llm @xcite . recent methods have been developed to quantify and reduce uncertainty focused on classification tasks @xcite ) and text classification models @xcite @xcite . however, obtaining such uncertainty estimates for many generative applications (e.g., responses in a q&a system) accessing proprietary models, such as gpt4, is not straightforward, since the uncertainty cannot being meaningfully characterized without access to the underlying probabilities. we quantify uncertainty in terms of the predicted probability of responses. since many current llms, especially proprietary models (e.g., gpt4), do not automatically furnish probabilities in their responses for a specific task or classification (e.g., yes/no q&a), we use a gpt-bart pipeline (see section 3) as a proxy for llm uncertainty. the proposed method only requires probability predictions and labeled training data and thus could be implemented on future llms that do directly provide probabilities for tasks. to support users in assessing responses from such models, we develop a framework which uses probability distributions from a zero-shot classification (bart-multilni @xcite ) with a dirichlet mixture model clustering approach based on a customized version of the expectation maximization algorithm (em; @xcite . we apply our framework to yes/no q&a, which remains a surprisingly difficult task subject to lower-than-expected accuracy @xcite . an analysis of the clusters of questions us-ing conformal prediction show support for users in better understanding the level of confidence an llm so that the user can trust its responses, especially in a black-box llm scenario. we subsequently apply our fitted general wikipedia model to a specific questions relevant to government domains and still obtain a usable clustering analysis. ## production and assessment of clusters figure 3 shows a ternary plot with three probability axes corresponding to "yes", "no", and "i don't know" outputs in three dimensions for gpt4. the goal is to characterize each of these clusters using the observed q&a data y. while several established clustering approaches exist, we have implemented a specific approach that obtains clusters of q&a probabilities in their natural sum-toone space. our approach uses the em algorithm @xcite for clustering with individual cluster densities that follow the dirichlet distribution @xcite , which automatically constrains the q&a probabilities to sum to one. ## results seen in figure 3 , the ternary plots visualize the probability predictions in terms of each answer: "yes", "no", and "i don't know". the left panel shows that probabilities sum to one for each question, and there appear to be @xmath0. the right panel shows the result of our clustering approach. this analysis shows three specific virtues: ## cluster accuracy u.s. government websites to assess the dirichlet clustering model's predictive capability beyond the boolq training set, we applied the learned clustering rule to a small set of 25 q&a questions from u.s. government websites (see section 3.1). questions were pre-appended with either "i am an immigration specialist" or "i am an import/export control specialist" respectively before being sent to gpt4. applying the gpt4 fitted boolq uncertainty model without any further clustering, the most probable answer was "yes" 12 times, "no" 8 times, and "i don't know" 5 times. among the 20 "yes" and "no" predictions, the accuracy rate is 16/20=80%. using prediction based on the clustering approach, 15 out of 25 predictions are in either the "probably yes" or "probably no" clusters, and 10 observations are in the "equivocal" cluster. no observations appeared in the "probably i don't know" cluster. these predictions can be seen in the right panel of figure 5 . when we look at some of the questions and both the response and the zero-shot probabilities, there are several instances in which gpt4 correctly answers in the text, but the zero-shot classification is not overly confident or ultimately wrong, even when the questions are on similar topics. for example, the question "does an esta grant me entry to the us?" is correctly answered in the gpt4 response and while the zero-shot classification is also correct ("no"), it only achieves a 43% probability from the model (compared to 41% "i don't know"). while the question "is an an esta a visa?" is also correctly answered by gpt4, it receives much higher probability of "no" at 79% in its zero-shot classification. in another instance, the question "are travelers checks considered money as defined by the customs and border protection?" is correctly answered by gpt4 ("yes"), but the zero-shot classification classification is incorrect with "no" (37%), although all the probabilities are rather close indicating potential indecision. these results however demonstrate that we can successfully optimize our uncertainty model on larger more general datasets of q&a responses and effectively apply them to smaller, more domain specific datasets and achieve the same desired effect of identifying question responses where a user can make relatively more accurate decisions. ## conclusion we developed a dirichlet mixture model clustering via em algorithm framework for llm yes/no q&a response certainty. our approach zero-shot pipeline is particularly applicable for when the underlying probabilities are not available in the initial response from an llm. importantly, our approach is model independent, reusable, computationally efficient, and can be applied to any zero-shot pipeline where we have access to both the category labels and underlying probabilities. our by cluster analyses reveal a more fine-grain analysis of accuracy, precision, and semantic similarities than without its implementation. a user is thus provided more information about if and under what conditions they can have more certainty in trusting responses for decision making, especially in domains in which they lack certainty. while we limited ourselves to only yes/no questions here, the framework can be extended to additional cases with a known, finite set of responses (e.g., classification tasks or categorical responses) and has future potential integration with in-context learning @xcite and to more effectively support retrieval-augmented generation (rag) systems @xcite .