Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models (2024)

Abstract: This paper investigates the capabilities of Large Language Models (LLMs) in understanding their knowledge and uncertainty over questions. Specifically, we focus on addressing known-unknown questions, characterized by high uncertainty due to the absence of definitive answers. To facilitate our study, we collect a new dataset with Known-Unknown Questions (KUQ) and establish a categorization framework to clarify the origins of uncertainty in such queries. Subsequently, we examine the performance of open-source LLMs, fine-tuned using this dataset, in distinguishing between known and unknown queries within open-ended question-answering scenarios. The fine-tuned models demonstrated a significant improvement, achieving a considerable increase in F1-score relative to their pre-fine-tuning state. Through a comprehensive analysis, we reveal insights into the models’ improved uncertainty articulation and their consequent efficacy in multi-agent debates. These findings help us understand how LLMs can be trained to identify and express uncertainty, improving our knowledge of how they understand and express complex or unclear information.

## data in the evaluation domain, the significance of datasets containing known-unknown questions is paramount. known-unknown questions are those that do not have definitive answers, such as "are there other forms of intelligent life in the universe?" or "which year will the next financial crisis occur?". upon surveying available resources, we identified a mere 46 known-unknown questions in the big-bench benchmark @xcite and 1 evaluation dataset in selfaware @xcite . this limited quantity is insufficient for robust training + evaluation, prompting us to generate additional samples for a full-scale dataset. our dataset is the first of its kind to include annotations on the questions pointing to the reason for uncertainty. the dataset is publicly available. we have collected questions from crowd-source workers and generated a new dataset: known-unknown questions, kuq. we have depicted the data collection process in figure 1 and explained question generation in §3.1. ## experiments in this section, we dive into a series of experiments centered around "known-unknown" questions. our main aim is to see how well current large language models (llms) handle these highly uncertain queries. we discuss the experiments and their results in more detail in the following sections. ## conclusion this work explores how open-source llms handle known-unknown questions, which are characterized by high uncertainty and the expectation of non-confident answers. we introduce a new dataset with known and unknown questions, kuq. in addition, a categorization of unknown questions is introduced, offering different reasons for them being unknown. along this work, we evaluate the current opensource models in open-ended question-answering on our kuq dataset. we evaluate (1) the ability to tell the difference between known and unknown questions and (2) the ability to distinguish between the different categories of questions. finally, we show how the expression of uncertainty may help in specific applications of large language models. in particular, we show how the fine-tuned model on kuq improves the results of multiagent debate when compared to the baseline original model. future research directions should focus on enhancing evaluation techniques and tackling the challenge of gauging model epistemic uncertainty, potentially leading to broader applications. investigating the capacity of llms to convey their uncertainty probabilities is also a key area of interest.