Do not Abstain! Identify and Solve the Uncertainty
Authors: Jingyu Liu, Jingquan Peng, xiaopeng Wu, Xubin Li, Tiezheng Ge, Bo Zheng, Yong Liu
Accepted at: acl25

Do not Abstain! Identify and Solve the Uncertainty

Jingyu Liu1*, Jingquan Peng2*, Xiaopeng Wu2, Xubing Li2,
Tiezhen Ge2, Bo Zheng2†, Yong Liu1,3,4‡
1 Gaoling School of Artificial Intelligence Renmin University of China Beijing, China
2 Taobao & Tmall Group of Alibaba
3 Beijing Key Laboratory of Research on Large Models and Intelligent Governance
4 Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE
{liuyj1016, liuyonggsai}@ruc.edu.cn

Abstract
Despite the widespread application of Large Language Models (LLMs) across various domains, they frequently exhibit overconfidence when encountering uncertain scenarios, yet existing solutions primarily rely on evasive responses (e.g., "I don’t know") overlooks the opportunity of identifying and addressing the uncertainty to generate more satisfactory responses. To systematically investigate and improve LLMs’ ability of recognizing and addressing the source of uncertainty, we introduce ConfuseBench, a benchmark mainly focus on three types of uncertainty: document scarcity, limited capability, and query ambiguity. Experiments with ConfuseBench reveal that current LLMs struggle to accurately identify the root cause of uncertainty and solve it. They prefer to attribute uncertainty to query ambiguity while overlooking capability limitations, especially for those weaker models. To tackle this challenge, we first generate context-aware inquiries that highlight the confusing aspect of the original query. Then we judge the source of uncertainty based on the uniqueness of the inquiry’s answer. Further we use an on-policy training method, InteractDPO to generate better inquiries. Experimental results demonstrate the efficacy of our approach.

<table>
  <tr>
    <th>Query</th>
    <th>Uncertainty</th>
    <th>LLM Response</th>
    <th>Expectation</th>
  </tr>
  <tr>
    <td>unanswerable</td>
    <td>low</td>
    <td>I do not know</td>
    <td>I do not know</td>
  </tr>
  <tr>
    <td>unanswerable</td>
    <td>high</td>
    <td>hallucinate</td>
    <td>I do not know</td>
  </tr>
  <tr>
    <td>answerable</td>
    <td>low</td>
    <td>answer</td>
    <td>answer</td>
  </tr>
  <tr>
    <td>answerable</td>
    <td>high</td>
    <td>hallucinate</td>
    <td>solve it</td>
  </tr>
</table>

Table 1: Different behavior of LLM when faced with different query. "unanswerable" mean query can not be answered like "weather of 2050."

To mitigate this issue, existing researches primarily adopt conservative strategies: response with "I don’t know" when identifying potential uncertainties (Amayuelas et al., 2024; Deng et al., 2024; Li et al., 2024a; Madhusudhan et al., 2024). However, this strategy exhibits significant limitations. As shown in Table 1, for inherently unknowable questions (e.g., "weather of 2050."), models should consistently response with "I don’t know". However, for those answerable queries, simply response with "I do not know" overlooks the opportunity of addressing the uncertainty, failing to generate more satisfactory responses. Specifically, when confidence levels are low (e.g., "quantum computing’s impact on climate modeling"), the system should proactively identify uncertainty sources (insufficient document/reasoning capability gap/query ambiguity), then employ dynamic strategies such as retrieval (Lewis et al., 2020; Zhang et al., 2024a), CoT (Wei et al., 2023; Li et al., 2024c), or clarification (Qian et al., 2024; Yang et al., 2024a) to improve the response quality.

To investigate and improve LLMs’ performance on identifying and solving the uncertainty, we introduce ConfuseBench, a benchmark that encompasses three distinct types of uncertainty: document scarcity, limited capacity, and query ambiguity. Document scarcity occurs when models lack essential factual information to answer a question, and additional documents could provide assistance (Lewis et al., 2020; Zhang et al., 2024a; Liu et al.,
2024); Limited capacity indicates that the query is too complex for the model to resolve effectively, in such cases, a larger model or extended reasoning steps might be beneficial (Wei et al., 2023; Yao et al., 2023). Query ambiguity occur when the query itself is unclear, where multiple answers may suffice or the query may not be answerable at all, necessitating further clarification (Min et al., 2020; Qian et al., 2024; Zhang et al., 2024b). Through ConfuseBench, we surpass conventional evaluation paradigms that focus solely on answer accuracy or basic uncertainty detection. Instead, ConfuseBench rigorously assesses models’ capacity to (1) diagnose the root causes of uncertainty and (2) actively mitigate such uncertainty to generate substantively improved responses.

Our paper mainly consider the situation where the model can not directly tell the answer of the question due to uncertainty and there is some method to help solving the uncertainty and generate a better response. We consider three main methods to improve the quality of generation: retrieval, CoT and clarification, which corresponds to three different source of uncertainty: document scarcity, limited capacity and query ambiguity. Those three methods to improve the quality should be most common ones so the three source of uncertainty also incorporate most situations. We show some examples in Appendix E

Our experiments with ConfuseBench have revealed that current models including GPT-4o struggle to identify the sources of uncertainty, which leads to unsatisfying performance on this benchmark. Those models prefers to categorize questions as ambiguous and request the user for clarification. For example, when we provide the model a clear query "locate the best yoga class in New York" and a noise document about yoga classes in London, the model might regard the query as ambiguous and response with "Are you referring to London?". Furthermore, the models seldomly acknowledge failures caused by their own capability limitations, when confronted with uncertainty, models often attribute the issue to external factors rather than recognizing their own limitations.

To address this issue, we propose a two-step approach. Instead of directly identifying the source of uncertainty, we first focus on accurately locating the confusing parts of the problem and generating a follow-up inquiry. If the answer to inquiry is an objective fact, the retrieval system can effectively provide the required information. If multiple answers fit the inquiry appropriately, further clarification is necessary. Conversely, if the follow-up inquiry is logically incoherent or merely paraphrased repetitions of the original question, it means the model fails to effectively understand the query and CoT could be beneficial. Furthermore, to enhance the capability of generating effective follow-up inquiry, we propose the InteractDPO, a training paradigm that dynamically generates "chosen-rejected" sample pairs through real-time interaction with retrieval systems or users during training, thereby achieving on-policy optimization.

Overall, our key contributions include: 1) This paper introduces a new benchmark designed to measure LLMs’ ability to identify different types of uncertainty arising from various sources, including document scarcity, limited capability, and query ambiguity. 2) We demonstrate through experiments that current LLMs exhibit significant challenges in reliably differentiating between these sources of uncertainty. 3) We propose a novel method for identifying the source of uncertainty based on the uniqueness of the inquiry’s answer, and we further enhance the inquiry generation process through InteractDPO.

Figure 1: LLMs recognize different source of uncertainty and try to solve the uncertainty.

2 Related Work

Recognizing the Uncertainty. Amayuelas et al. (2024); Yin et al. (2023) propose that models should learn to understand what they do not know instead of giving a hallucinated answer. Slobodkin et al. (2023); Amayuelas et al. (2024); Madhusudhan et al. (2024) further design various of prompts to instruct LLM express "I do not know" when encountering uncertainty. Yang et al. (2024b); Xiong et al. (2023) try to finetune the model to express how uncertain it is in verbal language and Deng
et al. (2024) propose to train the model to give some explanation to the unanswerability. Deng et al. (2024); Zhang et al. (2024b) also categorize why a question is unknown, but they mainly focus on ill-defined input, ignoring lack of capacity and documents, they still fail to recognize and solve the source of uncertainty.

Solving the Uncertainty. For knowledge based uncertainty, Trivedi et al. (2022); Wang et al. (2024a) try to solve multi-hop queries by iterative reasoning and retrieving until the model feels confident enough to provide an answer, Jeong et al. (2024); Wang et al. (2024b); Zhuang et al. (2024) also iteratively call the model to generate retrieval query to solve the uncertainty. For ambiguity based uncertainty, Qian et al. (2024) construct Intention-in-Interaction (IN3) to evaluate the ability of asking clarification question. Wang et al. (2024c) prompts the LLM to adaptively ask clarification questions and Yang et al. (2024a) further propose to use prompt, entropy and logits to measure is clarification needed. However, these works are constrained to only one source of uncertainty, fail to consider the situation that the uncertainty may rise from other sources.

Uncertainty Decomposition. As uncertainty could be raised from different sources, recognizing the source is an important topic (Wang and Holmes, 2024; Geng et al., 2024; Huang et al., 2024), previous works typical classify uncertainty into data and model uncertainty. Hou et al. (2023) try to judge does the query needs to be clarified by observing how will the model perform when faced with different clarifications. Ling et al. (2024) use ensemble methods and use different in context example to simulate models to decompose the uncertainty. Yadkori et al. (2024) propose a new definition of model uncertainty and decompose uncertainty by distribution shift when some answers are provided to the LLM. But those methods simply classifies the uncertainty as data uncertainty and model uncertainty, fails to consider the real uncertainty types the LLM would met in application.

3 Benchmark Construction

Previous benchmarks have primarily focused on refusing to answer unknown queries (Amayuelas et al., 2024; Deng et al., 2024), or have merely considered iterative retrieval and clarification techniques (Wang et al., 2024b; Zhuang et al., 2024; Qian et al., 2024). This approach fails to recognize that models need to identify the source of uncertainty and implement corresponding measures to address it. To comprehensively enhance and quantitatively evaluate these capabilities in model designs, we introduce ConfuseBench, a benchmark that encompasses various sources of uncertainty. This benchmark aims to assess and inspire LLMs’ abilities to recognize and resolve uncertainties effectively.

We evaluate three main scenarios in which LLMs are commonly employed: basic question answering, assistant interactions, and tool utilization. To assess the ability of recognizing and resolving uncertainty, we have collected various datasets and rewritten queries and associated documents to create the case holding certain uncertainty. For basic question answering, we incorporate HotpotQA(Yang et al., 2018) and AmbigQA (Min et al., 2020). In the assistant scenario, we consider ExpertQA (Malaviya et al., 2024) and TechQA (Castelli et al., 2019). We utilize ToolBench (Qin et al., 2023) for tool usage. It is worth noting that, to facilitate this evaluation, we employ GPT-4o to generate the tool calling chain for ToolBench, using the calling chain as the answer rather than the actual calling result.

To construct data cases where uncertainty arises from insufficient capability, we instruct the Large Language Model (LLM) to generate answers based on query and gold documents. If the model fails to produce the correct answer (which is already indicated in the documents), we attribute the uncertainty to its insufficient capability. Conversely, if the model successfully generates the correct answer, we construct a new document set by randomly discarding portions of the gold documents and retrieve some new ones. If the model cannot produce the correct answer under the new document set, we classify the uncertainty as stemming from missing documents. It is important to note that different large language models possess varying knowledge and capability boundaries. During evaluation, if a model can generate correct answers based on the original query and provided documents, it is deemed free of uncertainty, and such cases will be excluded from the evaluation.

For uncertainty arising from ambiguity, we directly utilize the ambiguous queries provided in AmbigQA (Min et al., 2020). For the other four datasets, we first transform the queries into Abstract Meaning Representation (AMR) (Shi et al., 2023), where each query is represented by entity
nodes and the corresponding relationships between those entities, forming a graph-based structure as shown in Figure 2. Subsequently, we prompt GPT-4o to introduce ambiguity into the AMR graph by removing modifiers and descriptive words, omitting key information, altering the relationships between nodes, and reorganizing the AMR structure. This method enables the model to better understand the semantic structure of the query, allowing us to provide clearer and more direct instructions for transforming the AMR into an ambiguous query. Then, we convert the AMR into an ambiguous query and generate the corresponding clarifications. If the model fails to answer the ambiguous query but successfully responds to it when provided with the clarification, we categorize the query as ambiguous.

<table>
  <tr>
    <th></th>
    <th>document</th>
    <th>ambiguity</th>
    <th>ability</th>
  </tr>
  <tr>
    <th rowspan="2">QA</th>
    <td>HotpotQA</td>
    <td>859</td>
    <td>702</td>
    <td>141</td>
  </tr>
  <tr>
    <td>AmbigQA</td>
    <td>543</td>
    <td>537</td>
    <td>167</td>
  </tr>
  <tr>
    <th rowspan="2">Assistant</th>
    <td>ExpertQA</td>
    <td>442</td>
    <td>397</td>
    <td>141</td>
  </tr>
  <tr>
    <td>TechQA</td>
    <td>470</td>
    <td>683</td>
    <td>140</td>
  </tr>
  <tr>
    <th>Tool Usage</th>
    <td>ToolBench</td>
    <td>479</td>
    <td>590</td>
    <td>144</td>
  </tr>
</table>

Table 2: statistics of the benchmark

The statistics of the dataset is shown in Table 2. Additionally, we manually select 50 cases each for queries lacking documentation and those that are ambiguous, as well as 30 cases for instances categorized as lacking ability, from each dataset to construct the benchmark. The remaining cases are used as training data. Consequently, the benchmark comprises a total of \( 5 \times (50 + 50 + 30) = 650 \) cases.

4 Preliminary Test

To evaluate the ability to address uncertainty, we instruct the LLM to determine whether it should interact with the retrieval system, consult a user, or utilize Chain of Thought (CoT) reasoning to resolve the uncertainty. If the model opts for CoT, it will generate an answer through a chain of thought reasoning. Conversely, if it chooses to engage with the retrieval system, it will generate a query to retrieve additional documents and then answer the original question based on the results of the interaction. If the model decides to interact with a user, it will formulate inquiries to ask clarifying questions and provide answers based on the received clarifications. We use GPT-4o to simulate the user and provide clarifications based on the inquiries made by the model. And we use majority voting when judging the source of uncertainty, specifically, we generate for 3 times, if the first two matches, then it is considered as the answer, otherwise we use the third generation.

We primarily evaluate the following metrics:

• Answer Quality(AQ): This metric assesses the quality of the answer provided after interaction or using Chain of Thought (CoT). For the HotpotQA and AmbigQA datasets, we employ Qwen2.5-72b as a judge to evaluate correctness. For the other datasets, we score answers based on their usefulness on a scale from 1 to 4; the results shown below are normalized to a range of 0-1.

• Uncertainty Classification Accuracy(UCA): This measures the LLM’s capacity to recognize the source of uncertainty, knowing that it should interact with the retrieval system, the user or solve it by CoT.

• Inquiry Quality(IQ): This metric evaluates the quality of the inquiries generated. We compare the query before ambiguity and the gold documents with the actual query and documents provided to the model to derive a gold standard inquiry. We then assess how closely the actual inquiry aligns with the gold inquiry and score it on a scale from 1 to 4 and we normalize it to 0-1 in the paper.

We mainly assess the following models: GPT-4o-0513, DeepSeek-V3, Qwen2.5-72b-Instruct, MetaLlama-3-70B-Instruct, Qwen2.5-7b-Instruct, and Mistral-7B-Instruct-v0.2. We use these models to judge the source of uncertainty and generate corresponding inquiries. We aim to evaluate the ability of locating and solving the uncertainty rather than the ability of solving the problem, therefore, to
<table>
  <tr>
    <th></th>
    <th>HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBench</th>
    <th>avg</th>
  </tr>
  <tr>
    <td rowspan="6">AQ</td>
    <td>GPT-4o</td>
    <td>0.446</td>
    <td>0.600</td>
    <td>0.760</td>
    <td>0.729</td>
    <td>0.846</td>
    <td>0.6762</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.515</td>
    <td>0.600</td>
    <td>0.783</td>
    <td>0.779</td>
    <td>0.794</td>
    <td>0.6942</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.585</td>
    <td>0.631</td>
    <td>0.804</td>
    <td>0.763</td>
    <td>0.812</td>
    <td>0.7188</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.438</td>
    <td>0.538</td>
    <td>0.758</td>
    <td>0.721</td>
    <td>0.725</td>
    <td>0.6362</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.369</td>
    <td>0.500</td>
    <td>0.742</td>
    <td>0.740</td>
    <td>0.744</td>
    <td>0.6181</td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.400</td>
    <td>0.492</td>
    <td>0.733</td>
    <td>0.735</td>
    <td>0.794</td>
    <td>0.6308</td>
  </tr>
  <tr>
    <td rowspan="6">UCA</td>
    <td>GPT-4o</td>
    <td>0.531</td>
    <td>0.377</td>
    <td>0.477</td>
    <td>0.400</td>
    <td>0.685</td>
    <td>0.4938</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.462</td>
    <td>0.431</td>
    <td>0.400</td>
    <td>0.438</td>
    <td>0.562</td>
    <td>0.4585</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.631</td>
    <td>0.592</td>
    <td>0.431</td>
    <td>0.408</td>
    <td>0.700</td>
    <td>0.5523</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.438</td>
    <td>0.408</td>
    <td>0.408</td>
    <td>0.385</td>
    <td>0.400</td>
    <td>0.4077</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.431</td>
    <td>0.454</td>
    <td>0.415</td>
    <td>0.408</td>
    <td>0.415</td>
    <td>0.4246</td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.438</td>
    <td>0.415</td>
    <td>0.438</td>
    <td>0.446</td>
    <td>0.531</td>
    <td>0.4538</td>
  </tr>
</table>

Table 3: Performance of locating and solving the uncertainty. AQ represents the quality of answer after interaction; UCA is the uncertainty classification accuracy.

avoid the impact of different perceptions of the question by the models themselves, we use both the evaluated model and GPT-4o to generate answers based on the interaction results, the highest score is considered. To measure the reliability LLM evaluating the performance, we show the omega reliability in Appendix D

From Table 3, we can observe that the LLM fails to effectively recognize the source of uncertainty and generate corresponding inquiry to solve the uncertainty. The best performing model only successfully classify about 50% of cases. Those weaker models like Mistral-7b and Qwen2.5-7b fails to effectively recognize the source of uncertainty, even Llama-3-70B shows unsatisfying performance.

<table>
  <tr>
    <th></th>
    <th>metric</th>
    <th>doc</th>
    <th>ambig</th>
    <th>ability</th>
  </tr>
  <tr>
    <td rowspan="2">GPT-4o</td>
    <td>precision</td>
    <td>0.53</td>
    <td>0.41</td>
    <td>0.28</td>
  </tr>
  <tr>
    <td>recall</td>
    <td>0.19</td>
    <td>0.76</td>
    <td>0.2</td>
  </tr>
  <tr>
    <td rowspan="2">Llama-3-70B</td>
    <td>precision</td>
    <td>0.3</td>
    <td>0.39</td>
    <td>0.22</td>
  </tr>
  <tr>
    <td>recall</td>
    <td>0.12</td>
    <td>0.97</td>
    <td>0.02</td>
  </tr>
  <tr>
    <td rowspan="2">Qwen2.5-7B</td>
    <td>precision</td>
    <td>0.43</td>
    <td>0.39</td>
    <td>0.31</td>
  </tr>
  <tr>
    <td>recall</td>
    <td>0.1</td>
    <td>0.94</td>
    <td>0.1</td>
  </tr>
  <tr>
    <td rowspan="2">Mistral-7B</td>
    <td>precision</td>
    <td>0.44</td>
    <td>0.41</td>
    <td>0.33</td>
  </tr>
  <tr>
    <td>recall</td>
    <td>0.12</td>
    <td>0.85</td>
    <td>0.14</td>
  </tr>
</table>

Table 4: Precision and recall of different uncertainty

From Table 4, we can observe that when faced with uncertainty, LLMs tend to attribute uncertainty to query ambiguity ("ambig") rather than insufficient document support ("doc"), particularly in the less powerful models as indicated by the high recall of "ambig" and low recall of "doc".

We can observe that the classification is unbalanced; they recognizes most of the queries as ambiguous and proceeds to interact with the user instead of more retrieval and lack of capacity.

As illustrated in Figure 3, when presented with a clear query accompanied by noisy documents, the model can become distracted by the noise. It may ask the user to clarify the query, hoping to change the intention of the user so that it can leverage information from the noisy documents to generate an answer. In this context, the model understands why it cannot answer the query, but it places greater trust in the relevance of the documents than in the

![A diagram showing two scenarios: left case retrieves documents about another singer and asks to change the query; right case simply rephrases the query and wants more information.](page_1012_1042_420_312.png)

Figure 3: In the left case, the model retrieved some documents about another singer and asks the user to change the query. In the right case, the model simply rephrase the query and wants to retrieve more information.
Figure 4: The inquiry score and the percentage of errors caused by bad inquiry (correctly classified but answer incorrectly with low inquiry score)

clarity of the query. Consequently, instead of seeking supplemental documents, the model attempts to align user intention with available information in the noisy documents, ultimately resulting in the unexpected behavior of requiring query rephrasing rather than acquiring more relevant documents.

Moreover, large language models (LLMs) seldom acknowledge that they cannot answer a question due to a lack of capability, as shown in Table 4. In our view, this phenomenon is similar to overconfidence (Xiong et al., 2024; Li et al., 2024b; Xiong et al., 2023); when faced with uncertainty, these models often provide incorrect answers instead of recognizing their limitations with a response such as, "I don’t know." And when instructed to choose the reason for their uncertainty, they also tend to refuse to acknowledge that it is due to their limited reasoning capacity and blame it to insufficient documents or ambiguity.

As we show in Figure 3, when faced with uncertainty brought by capacity, the model might direct rephrase the query, this can be explained in two ways: 1) the model fails to recognize any lack of documents or ambiguity, so it can only repeat the query again; 2) what confuses the model is the query itself, it fails to effectively understand the query and the given documents, so the query itself is the confusing part.

For the quality of inquiry, we can observe from Figure 4 that powerful models like GPT-4o are capable of generating meaningful inquiries, which could help to gather some useful information, but the quality of inquiry could still be improved and smaller models such as Qwen2.5-7b, performs worse.

5 Method

In this section, we begin by discussing the use of CoT to identify factors that may be confusing the model and to assess the sources of uncertainty. We then propose that the uncertainty associated with the inquiry is equivalent to that of the query itself. This allows us to directly evaluate the uncertainty by examining the inquiry. And we propose to utilize the uniqueness of the inquiry answer to recognize the source of uncertainty.

5.1 Judge Based on Inquiry Answer

Apparently, judging the source of uncertainty is a difficult task, and less powerful models fail to complete this task; they prefer to regard the query as ambiguous and interact with the user. However, we can also observe from Figure 4 that these models can effectively generate inquiries to interact with their environment, with inquiry scores not significantly lower than GPT-4o and DeepSeek-V3 (Javaji and Zhu, 2024; Qian et al., 2024). A natural approach is to leverage Chain of Thought to identify what is confusing the model, and judge the source of uncertainty afterwards.

Also, we can show that the uncertainty held by the inquiry is actually the same with the query. And the inquiry typically only involve some sub-aspects of the query, it should be easier to identify the source of uncertainty based on the inquiry.

Definition 5.1. Consider a query \( x \), the corresponding answer \( y \), document \( d \) and the clarification to the query \( c \). Let \( \theta \) be the model and \( \theta^* \) be the optimal model which can perfectly solve the query \( x \). Then, the uncertainty raised by capacity \( U_c \), by knowledge \( U_k \) and by ambiguity \( U_a \)

\[
U_c = H(y|x, d, c, \theta),
\]
\[
U_k = H(y|x, c, \theta^*),
\]
\[
U_a = H(y|x, d, \theta^*),
\]

where \( H(\cdot) \) stands for entropy

Therefore, \( U_c \) is the uncertainty of the model when all information is given, so it is raised by lack of capacity. \( U_k \) and \( U_a \) is the uncertainty for the optimal model when documents and clarification are missing, they correspond to uncertainty raised by lack of documents and ambiguity. Then, we can show that the inquiry holds similar uncertainty with the query.

Theorem 5.2. Given a query \( x \) and the generated inquiry \( q \), then the uncertainty of \( q \) is positively
related to the uncertainty of \( x \), then,

\[
|U_k(q) - U_k(x)| \leq -\log p(q^*|x, c, \theta),
\]
\[
|U_a(q) - U_a(x)| \leq -\log p(q^*|x, d, \theta),
\]
\[
|U_c(q) - U_c(x)| \leq -\log p(q^*|x, d, c, \theta),
\]

where \( q^* \) is the optimal inquiry generated by \( \theta^* \). For lack of ability, the optimal inquiry is the original query.

The theorem posits that if the model generates a meaningful inquiry, the uncertainty held by the inquiry is similar to the original query. Otherwise if the inquiry is meaningless, it shows that the model fails to understand the query well, and it also shows lack of capacity.

Therefore, we can enhance the performance based on the generated inquiry. If the generated inquiry fails to recognize the confusing part, then we classify it as a lack of capacity, indicating that Chain of Thought is required. Conversely, if the inquiry requires additional documents to solve, retrieval is required otherwise clarification is needed.

But when facing some complex query, directly judge the source of uncertainty based on the inquiry is hard. To measure the uncertainty held by inquiry, we propose utilizing the answer of inquiry to help the judgment when facing complex problems. First, if the uncertainty arises from a lack of capability, the model would merely rephrase the query; thus, the response to the inquiry should appropriately address the original query. Consequently, we can determine whether the uncertainty stems from a lack of capability by evaluating the semantic coherence when the inquiry answer is considered as the answer to the original query. In this way, we can not only recognize cases where Chain of Thought (CoT) is needed but also prevent unnecessary retrieval and clarification.

Also, if the inquiry requires extra retrieval, it typically indicates that the answer points to a definitive objective fact. Conversely, if the query needs further clarification, it suggests the question may have multiple valid subjective answers. To distinguish these scenarios, we designed a verification method inspired by Yadkori et al. (2024): First, we provide the LLM with a logically coherent preset answer to the inquiry. We then instruct the model to generate a new distinct response based on this input. For objective factual questions, the model—lacking prior knowledge—tends to directly repeat the fabricated answer. However, for open-ended subjective questions, the model recognizes the potential for diverse solutions and can still produce novel, reasonable responses even after receiving the preset answer.

For example, consider the query Locate the best yoga class in my city and the corresponding inquiry Which city are you referring to? If "New York" is provided as a possible answer, the model can easily generate an alternative answer like "London." However, if the inquiry is Find the yoga classes in New York and an answer is provided, the model is likely to repeat that answer, indicating it does not know any other answers. Therefore, when it is hard for the model to directly judge the source of uncertainty, judge based on answer could help. So we first judge the source based on inquiry twice, and if the two samples differ, we judge the source of uncertainty by answer.

5.2 Inquiry Quality Matters

It is important to note that if the model fails to generate an inquiry of high quality, the advantages of a more concise input may be overshadowed by the drawbacks of a poor inquiry, resulting in suboptimal performance, and a good inquiry can help the model to gather more useful information. Therefore, enhancing the quality of the generated inquiry is essential.

Therefore, we propose InteractDPO. Vanilla DPO use preference datasets collected ahead of training, the responses in the dataset are usually generated by different LLMs (Rafailov et al., 2024; Qi et al., 2024). Thus, the feedback is usually purely offline. To conduct on-policy training, we first collect some preference datasets, then during training, the trained model generates an inquiry based on the prompt and interact with the retrieval system or the user-GPT to gather more documents
<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="2">HotpotQA</th>
    <th colspan="2">AmbigQA</th>
    <th colspan="2">TechQA</th>
    <th colspan="2">ExpertQA</th>
    <th colspan="2">ToolBench</th>
    <th>avg</th>
  </tr>
  <tr>
    <th>prompt</th><th>inquiry</th>
    <th>prompt</th><th>inquiry</th>
    <th>prompt</th><th>inquiry</th>
    <th>prompt</th><th>inquiry</th>
    <th>prompt</th><th>inquiry</th>
    <th>avg</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.531</td><td>0.523</td>
    <td>0.377</td><td>0.562</td>
    <td>0.477</td><td>0.538</td>
    <td>0.400</td><td>0.469</td>
    <td>0.685</td><td>0.754</td>
    <td>0.4938</td><td>0.5692</td>
    <td>0.6062</td>
  </tr>
  <tr>
    <td></td>
    <td><b>0.600</b></td><td><b>0.585</b></td>
    <td></td><td></td>
    <td><b>0.562</b></td><td><b>0.515</b></td>
    <td></td><td></td>
    <td><b>0.769</b></td><td></td>
    <td></td><td></td>
    <td></td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.462</td><td>0.546</td>
    <td>0.431</td><td>0.538</td>
    <td>0.400</td><td>0.446</td>
    <td>0.438</td><td>0.492</td>
    <td>0.562</td><td>0.662</td>
    <td>0.4585</td><td>0.5369</td>
    <td>0.5538</td>
  </tr>
  <tr>
    <td></td>
    <td><b>0.546</b></td><td><b>0.562</b></td>
    <td></td><td></td>
    <td><b>0.485</b></td><td><b>0.508</b></td>
    <td></td><td></td>
    <td><b>0.669</b></td><td></td>
    <td></td><td></td>
    <td></td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.631</td><td>0.592</td>
    <td>0.592</td><td>0.631</td>
    <td>0.431</td><td>0.554</td>
    <td>0.408</td><td>0.708</td>
    <td>0.700</td><td>0.754</td>
    <td>0.5523</td><td>0.5769</td>
    <td>0.6031</td>
  </tr>
  <tr>
    <td></td>
    <td><b>0.608</b></td><td><b>0.654</b></td>
    <td></td><td></td>
    <td><b>0.569</b></td><td><b>0.431</b></td>
    <td></td><td></td>
    <td><b>0.754</b></td><td></td>
    <td></td><td></td>
    <td></td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.438</td><td>0.631</td>
    <td>0.408</td><td>0.469</td>
    <td>0.408</td><td>0.538</td>
    <td>0.385</td><td>0.515</td>
    <td>0.400</td><td>0.531</td>
    <td>0.4077</td><td>0.5369</td>
    <td>0.5477</td>
  </tr>
  <tr>
    <td></td>
    <td><b>0.615</b></td><td><b>0.492</b></td>
    <td></td><td></td>
    <td></td><td></td>
    <td></td><td></td>
    <td><b>0.600</b></td><td></td>
    <td></td><td></td>
    <td></td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.431</td><td>0.462</td>
    <td>0.454</td><td>0.477</td>
    <td>0.415</td><td>0.523</td>
    <td>0.408</td><td>0.469</td>
    <td>0.415</td><td>0.454</td>
    <td>0.4246</td><td>0.4769</td>
    <td>0.5000</td>
  </tr>
  <tr>
    <td></td>
    <td><b>0.500</b></td><td><b>0.531</b></td>
    <td></td><td></td>
    <td><b>0.523</b></td><td><b>0.485</b></td>
    <td></td><td></td>
    <td><b>0.462</b></td><td></td>
    <td></td><td></td>
    <td></td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.438</td><td>0.508</td>
    <td>0.415</td><td>0.562</td>
    <td>0.438</td><td>0.546</td>
    <td>0.446</td><td>0.515</td>
    <td>0.531</td><td>0.515</td>
    <td>0.4538</td><td>0.5292</td>
    <td>0.5477</td>
  </tr>
  <tr>
    <td></td>
    <td><b>0.515</b></td><td><b>0.562</b></td>
    <td></td><td></td>
    <td><b>0.562</b></td><td><b>0.546</b></td>
    <td></td><td></td>
    <td><b>0.554</b></td><td></td>
    <td></td><td></td>
    <td></td>
  </tr>
</table>

Table 5: Uncertainty classification performance of using CoT to judge uncertainty after inquiry generation (inquiry) and judge the uncertainty by inquiry answer (answer), prompt means directly judge the uncertainty source.

<table>
  <tr>
    <th> </th>
    <th>HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBench</th>
    <th>avg</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.600</td>
    <td>0.585</td>
    <td><b>0.562</b></td>
    <td><b>0.515</b></td>
    <td>0.769</td>
    <td>0.6062</td>
  </tr>
  <tr>
    <td>vanilla</td>
    <td>0.554</td>
    <td>0.538</td>
    <td>0.454</td>
    <td>0.438</td>
    <td>0.731</td>
    <td>0.5431</td>
  </tr>
  <tr>
    <td>SFT</td>
    <td>0.585</td>
    <td>0.569</td>
    <td>0.485</td>
    <td>0.485</td>
    <td>0.746</td>
    <td>0.5738</td>
  </tr>
  <tr>
    <td>DPO</td>
    <td>0.608</td>
    <td>0.554</td>
    <td>0.515</td>
    <td>0.485</td>
    <td>0.762</td>
    <td>0.5846</td>
  </tr>
  <tr>
    <td>onlineDPO</td>
    <td>0.608</td>
    <td>0.569</td>
    <td>0.531</td>
    <td>0.492</td>
    <td>0.762</td>
    <td>0.5923</td>
  </tr>
  <tr>
    <td>InteractDPO</td>
    <td><b>0.615</b></td>
    <td><b>0.592</b></td>
    <td>0.546</td>
    <td>0.508</td>
    <td><b>0.769</b></td>
    <td><b>0.6062</b></td>
  </tr>
</table>

Table 6: Uncertainty classification performance of InteractDPO

or clarification. The model then generates answer based on the interaction. During training, if the trained model successfully generate an inquiry to solve the original query, it will be selected as the chosen inquiry, otherwise the rejected inquiry to conduct on policy DPO training. OnlineDPO (Qi et al., 2024) instead generate two different inquiry and use the LLM to judge which one is better and mark is as chosen, the other one as rejected. But the LLM can not accurately judge which one is better without real supervision signal, therefore, compared to directly use LLM to select the chosen-rejected pair, InteractDPO provides real feedback and shows better performance when classifying the source of uncertainty and gathering information.

6 Experiments

To validate the performance of our proposed method, we conduct experiments on the benchmark. When judging the source of uncertainty, we sample three times and use majority voting to judge source of uncertainty, we also use prompt judge is it lack of capacity for all 3 methods due to its high precision. We repeat the experiment for 3 times and use the averaged performance.

As shown in table 5, judging by the inquiry and the answer can help to increase the performance, directly judging based on the inquiry can help the model to better judge the source of uncertainty, results of more models, the answer quality are shown
in Appendix C. And the results of few shot are shown E.

For InteractDPO, based on Figure 4, Qwen2.5-7B show great performance when generating inquiry, therefore, we choose the model to conduct further finetuning and enhance its ability of generating high quality inquiries. As we mainly want to enhance and evaluation of generating inquiry, we use the finetuned model to generate inquiry, and use GPT-4o to conduct classification and further answering. Also, we compare our method with SFT, DPO and OnlineDPO, as shown in Table 6, InteractDPO helps the most. The performance of answer quality is shown in Appendix C.

7 Conclusion

In this paper, we discuss the fact that current LLMs fail to effectively judge the source of uncertainty. Models prefers to recognize the query as ambiguous seldomly admit lack of capacity. Then, we propose to judge the source of uncertainty by uniqueness of inquiry answer, to further increase the performance, we propose InteractDPO to help the model generate better inquiry.

Limitations

This paper primarily discusses how current large language models (LLMs) fail to recognize the sources of uncertainty. While we focus on three main categories of uncertainty, these can be further specified. For instance, a lack of documents may correspond to deficiencies in factual knowledge or background information, each requiring different databases for retrieval. Regarding lack of ability, while Chain of Thought (CoT) techniques can address some issues, there are also cases that necessitate the use of methods like Tree of Thought or Monte Carlo Tree Search (MCTS). And there is also various of reasons why the query is ill-defined for example, it could be ambiguous or factually incorrectly or asks for a illegal time. Consequently, there are various sources of uncertainty, each linked to its own solution; however, we only examine the three most common ones.

Acknowledgements

This research was supported by National Natural Science Foundation of China(No.62476277), National Key Research and Development Program of China(NO. 2024YFE0203200), CCF-ALIMAMA TECH Kangaroo Fund(No.CCF-ALIMAMA OF 2024008), and Huawei-Renmin University joint program on Information Retrieval. We also acknowledge the support provided by the fund for building worldclass universities (disciplines) of Renmin University of China and by the funds from Beijing Key Laboratory of Big Data Management and Analysis Methods, Gaoling School of Artificial Intelligence, Renmin University of China, from Engineering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education, from Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary Platform for the “Double-First Class” Initiative, Renmin University of China, from Public Policy and Decision-making Research Lab of Renmin University of China, and from Public Computing Cloud, Renmin University of China.

References

Alfonso Amayuelas, Kyle Wong, Liangming Pan, Wenhui Chen, and William Wang. 2024. Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models. Preprint, arXiv:2305.13712.

Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint ArXiv:2005.14165.

Vittorio Castelli, Rishav Chakravarti, Saswati Dana, Anthony Ferritto, Radu Florian, Martin Franz, Dinesh Garg, Dinesh Khandelwal, Scott McCarley, Mike McCawley, Mohamed Nasr, Lin Pan, Cezar Pendus, John Pitrelli, Saurabh Pujar, Salim Roukos, Andrzej Sakrajad, Avirup Sil, Rosario Uceda-Sosa, Todd Ward, and Rong Zhang. 2019. The techqa dataset. Preprint, arXiv:1911.02984.

Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. 2024. Bias and unfairness in information retrieval systems: New challenges in the llm era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6437–6447.

Yang Deng, Yong Zhao, Moxin Li, See-Kiong Ng, and Tat-Seng Chua. 2024. Don’t just say “i don’t know”! self-aligning large language models for responding to unknown questions with explanations. Preprint, arXiv:2402.15062.

Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koepll, Preslav Nakov, and Iryna Gurevych. 2024. A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6577–6595.
Qiuhan Gu. 2023. Llm-based code generation method for golang compiler testing. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 2201–2203.

Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang. 2023. Decomposing uncertainty for large language models through input clarification ensembling. arXiv preprint arXiv:2311.08718.

Hsiu-Yuan Huang, Yutong Yang, Zhaoxi Zhang, San-woo Lee, and Yunfang Wu. 2024. A survey of uncertainty estimation in llms: Theory meets practice. arXiv preprint arXiv:2410.15326.

Shashidhar Reddy Javaji and Zining Zhu. 2024. What would you ask when you first saw \( a^2 + b^2 = c^2 \)? evaluating llm on curiosity-driven questioning. arXiv preprint arXiv:2409.17172.

Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C Park. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:2403.14403.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.

Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for "mind" exploration of large language model society. Preprint, arXiv:2303.17760.

Jiaqi Li, Yixuan Tang, and Yi Yang. 2024a. Know the unknown: An uncertainty-sensitive method for llm instruction tuning. arXiv preprint arXiv:2406.10099.

Moxin Li, Wenjie Wang, Fuli Feng, Fengbin Zhu, Qifan Wang, and Tat-Seng Chua. 2024b. Think twice before trusting: Self-detection for large language models through comprehensive answer reflection. Preprint, arXiv:2403.09972.

Zhiyuan Li, Hong Liu, Denny Zhou, and Tengyu Ma. 2024c. Chain of thought empowers transformers to solve inherently serial problems. arXiv preprint arXiv:2402.12875.

Chen Ling, Xujiang Zhao, Xuchao Zhang, Wei Cheng, Yanchi Liu, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuoka, Jie Ji, et al. 2024. Uncertainty quantification for in-context learning of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3357–3370.

Jingyu Liu, Jaein Lin, and Yong Liu. 2024. How much can rag help the reasoning of llm? arXiv preprint arXiv:2410.02338.

Nishanth Madhusudhan, Sathwik Tejaswi Madhusudhan, Vikas Yadav, and Masoud Hashemi. 2024. Do llms know when to not answer? investigating ab-stention abilities of large language models. arXiv preprint arXiv:2407.16221.

Chaitanya Malaviya, Subin Lee, Sihaq Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. 2024. Expertqa: Expert-curated questions and attributed answers. Preprint, arXiv:2309.07852.

Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. Ambigqa: Answering ambiguous open-domain questions. arXiv preprint arXiv:2004.10645.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.

Biqing Qi, Pengfei Li, Fangyuan Li, Junqi Gao, Kaiyan Zhang, and Bowen Zhou. 2024. Online dpo: Online direct preference optimization with fast-slow chasing. Preprint, arXiv:2406.05534.

Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, et al. 2024. Tell me more! towards implicit user intention understanding of language model driven agents. arXiv preprint arXiv:2402.09205.

Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789.

Rafael Raffailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Preprint, arXiv:2305.18290.

Kayla Schroeder and Zach Wood-Doughty. 2024. Can you trust llm judgments? reliability of llm-as-a-judge. arXiv preprint arXiv:2412.12509.

Kaize Shi, Xueyao Sun, Li He, Dingxian Wang, Qing Li, and Guangdong Xu. 2023. Amr-tst: Abstract meaning representation-based text style transfer. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4231–4243.

Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. 2023. The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over-confident large language models. Preprint, arXiv:2310.11877.
Harsh Trivedi, Niranjani Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509.

Keheng Wang, Feiyu Duan, Peiguang Li, Sirui Wang, and Xunliang Cai. 2024a. Llms know what they need: Leveraging a missing information guided framework to empower retrieval-augmented generation. arXiv preprint arXiv:2404.14043.

Ruobing Wang, Daren Zha, Shi Yu, Qingfei Zhao, Yuxuan Chen, Yixian Wang, Shuo Wang, Yukun Yan, Zhenghao Liu, Xu Han, et al. 2024b. Retriever-and-memory: Towards adaptive note-enhanced retrieval-augmented generation. arXiv preprint arXiv:2410.08821.

Wenxuan Wang, Juluan Shi, Chaozheng Wang, Cheryl Lee, Youliang Yuan, Jen tse Huang, and Michael R. Lyu. 2024c. Learning to ask: When llms meet unclear instruction. Preprint, arXiv:2409.00557.

Ziyu Wang and Chris Holmes. 2024. On subjective uncertainty quantification and calibration in natural language generation. arXiv preprint arXiv:2406.05213.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichtier, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.

Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryan W White, Doug Burger, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation. Preprint, arXiv:2308.08155.

Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2023. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306.13063.

Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. Preprint, arXiv:2306.13063.

Yasin Abbasi Yadkori, Ilja Kuzborskij, András György, and Csaba Szepesvári. 2024. To believe or not to believe your llm. Preprint, arXiv:2406.02543.

Yongjin Yang, Haneul Yoo, and Hwaran Lee. 2024a. Maqa: Evaluating uncertainty quantification in llms regarding data uncertainty. arXiv preprint arXiv:2408.06816.

Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. 2024b. Alignment for honesty. Preprint, arXiv:2312.07000.

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. Preprint, arXiv:1809.09600.

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Preprint, arXiv:2305.10601.

Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653–8665, Toronto, Canada. Association for Computational Linguistics.

Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E Gonzalez. 2024a. Raft: Adapting language model to domain specific rag. arXiv preprint arXiv:2403.10131.

Tong Zhang, Peixin Qin, Yang Deng, Chen Huang, Wenqiang Lei, Junhong Liu, Dingnan Jin, Hongru Liang, and Tat-Seng Chua. 2024b. Clamber: A benchmark of identifying and clarifying ambiguous information needs in large language models. arXiv preprint arXiv:2405.12063.

Ziyuan Zhuang, Zhiyang Zhang, Sitao Cheng, Fangkai Yang, Jia Liu, Shujian Huang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Zhang. 2024. Efficientrag: Efficient retriever for multi-hop question answering. arXiv preprint arXiv:2408.04259.
A  InteractDPO

In order to improve the ability of locating the uncertainty and generate the corresponding inquiry, we propose InteractDPO. Vanilla DPO use preference datasets collected ahead of training the responses in the dataset are usually generated by different LLMs. Thus, the feedback is usually purely offline. Also, different model holds different knowledge, the query might be difficult for model A, but it might be easy for model B. Therefore using dataset generated by one model to train another model is not a good choice.

Also, using a model to judge the quality of inquiry for training like onlineDPO is also not a good choice because the quality of inquiry can hardly be measured because different model may hold different uncertainty when faced with the same query.

To solve this we propose InteractDPO. We first collect some preference datasets, then during training, the trained model generates an inquiry based on the prompt and interact with the retrieval system or the user-GPT to gather more documents or clarification. The model then generates answer based on the interaction. If the answer is better than the one generated based on the original query and documents, the inquiry should be a chosen one, otherwise a rejected one.

The preference dataset should contain a prompt which holds some uncertainty, and it should be solvable, which means that there should be an inquiry that can solve the query by interact with the retrieval system or the userGPT. Therefore, we use three different models (GPT-4o, Qwen2.5-7b and Mistral-7b) to generate inquiry based on the query and answer the question after interaction. Then for GPT-4o, we choose those queries that can not be answer correctly without inquiry and can be answered after interaction as chosen. For the same query, those inquiries that fails to answer the query after interaction are considered as rejected.

During training, if the trained model successfully generate an inquiry to solve the original query, it will replace the chosen inquiry, otherwise the rejected inquiry to conduct on policy DPO training.

B  Proof of Theorem

Consider the uncertainty raised by ambiguity.

\[
U_a = H(y|x, d, \theta^*)
\]
\[
H(y|x, d, \theta^*) = -p(y|x, d, \theta^*) \log p(y|x, d, \theta^*)
\]
\[
p(y|x, d, \theta^*) = p(c|x, d, \theta^*) \cdot p(y|x, d, c, \theta^*)
\]

Assumption B.1. *the optimal model \( \theta^* \) can perfectly solve the problem \( x \) with corresponding documents and clarification, which means that \( p(y^*|x, d, c, \theta^*) = 1 \), \( y^* \) is the ground truth answer to query \( x \). And \( p(q^*|x, d, \theta^*) = 1 \), where \( q^* \) is the optimal inquiry.*

In this way
\[
p(y|x, d, \theta^*) = p(c|x, d, \theta^*) \cdot p(y|x, d, c, \theta^*)
\]
\[
= p(c|x, d, \theta^*) = p(q|x, d, \theta^*) \cdot p(c|q)
\]
Therefore,

\[
U_a = H(y|x, d, \theta^*) = H(c|x, d, \theta^*) = H(c|q).
\]

when we generate the inquiry with the optimal model \( \theta^* \), the uncertainty of the query is exact the same with the one with the inquiry.

When considering generate the inquiry with the model \( \theta \), let \( P = p(y|x, d, \theta^*) \), \( Q = p(y|x, d, \theta) \), then

\[
H(P) = H(P, Q) - D_{KL}(P||Q) \geq H(Q) - D_{KL}(P||Q)
\]
\[
H(Q) - H(P) \leq D_{KL}(P||Q)
\]
(2)

\[
D_{KL}(P||Q) = \int P(x) \log \frac{P(x)}{Q(x)}
= \int p(y|x, d, \theta^*) \log \frac{p(y|x, d, \theta^*)}{p(y|x, d, \theta)}
= \int p(c|q) \cdot p(q|x, d, \theta^*) \log \frac{p(q|x, d, \theta^*)}{p(q|x, d, \theta)}
= \int p(c|q^*) \log \frac{p(q^*|x, d, \theta^*)}{p(q^*|x, d, \theta)}
= -\log p(q^*|x, d, \theta)
\]
(3)

\[
D_{KL}^+(Q||P) = \int_{P(x)\neq 0} Q(x) \log \frac{Q(x)}{P(x)}
= \int p(q^*|x, d, \theta)p(c|q^*) \log \frac{p(q^*|x, d, \theta)}{p(q^*|x, d, \theta^*)}
= p(q^*|x, d, \theta) \log p(q^*|x, d, \theta) - 0
= -H^+(Q) + H(P)
\geq -H(Q) + H(P)
\]
(4)

So
\[
H(P) - H(Q) \leq p(q^*|x, d, \theta) \log p(q^*|x, d, \theta)
\]
\[
H(Q) - H(P) \leq -\log p(q^*|x, d, \theta)
\]
\[
|H(P) - H(Q)| \leq -\log p(q^*|x, d, \theta)
\]
(5)

It works similarly when face with knowledge uncertainty and lack of capacity, the optimal inquiry for lack of capacity is defined as the original query.

Also, generating the inquiry relates to comprehensively understand and analyze the query and documents, which is a similar task compared to generating the answer, so we assume that \( U_c = H(y|x, d, c, \theta) \propto H(q^*|x, d, c, \theta) \)

C  Further Experiments and details

We conduct further experiments showing the classification accuracy, f1 score and more results on judge based on inquiry and the inquiry answer. Table 7 shows the weighted f1 score, showing that judge based on the inquiry achieve a better and more balance classification performance. Table 8 shows the result of all models when judge the source of uncertainty based on inquiry and the answer, and we show more results of precision and recall in Table 9.

C.1 Experimental Setup

When conduct training, we use learning rate ranging from \{3e = 06, 1e − 05, 3e − 05\}, and we train the model for 5 epochs. We train the model using LoRA, the rank is set to 64, and the lora targets are q_proj,k_proj,v_proj,o_proj and ffn. the cutoff length is set to 32k, and bf16 training is used.

D Reliability

Following (Schroeder and Wood-Doughty, 2024), we use Omega reliability and the Intraclass Correlation Coefficients (ICCs) to show that the model can also consistently select the best one when provided some responses. Specifically, for each question, we use DeepSeek, GPT-4o, Qwen2.5 72b, Qwen2.5 7b and LLaMa 70B to generate answer. Then given the ground truth and the 5 generated response, we prompt the judger to select the best one for 100 times. We use the reliabiliPy package to calculate Omega reliability. The result is shown below:

E Example

We add the following three examples in the prompt of judging based on prompt, inquiry and answer, and the performance is shown in Table 13, 12 Example 1
Question: Are Edward F. Cline and Floyd Mutrux both screenwriters?
Document: Document 1: Edward F. Cline is a screenwriter.
Example 2
Question: Which Australian politician represented Electoral district of Goulburn?
Document: Document 1: A born in 1964, "(long context about biography of A)" represents Goulburn "(long context about biography of A)"
Example 3 (Ambiguous query which requires interaction with the user)
<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="6">HotpotQA</th>
    <th colspan="6">AmbigQA</th>
    <th colspan="6">TechQA</th>
    <th colspan="6">ExpertQA</th>
    <th colspan="6">ToolBench</th>
    <th colspan="6">avg</th>
  </tr>
  <tr>
    <th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.415</td><td>0.399</td><td>0.456</td><td>0.333</td><td>0.436</td><td>0.396</td><td>0.328</td><td>0.455</td><td>0.469</td><td>0.277</td><td>0.394</td><td>0.398</td><td>0.356</td><td>0.557</td><td>0.529</td><td>0.3418</td><td>0.4482</td><td>0.4496</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.313</td><td>0.391</td><td>0.415</td><td>0.269</td><td>0.525</td><td>0.520</td><td>0.234</td><td>0.338</td><td>0.327</td><td>0.232</td><td>0.397</td><td>0.421</td><td>0.451</td><td>0.534</td><td>0.544</td><td>0.2998</td><td>0.4370</td><td>0.4454</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.509</td><td>0.498</td><td>0.454</td><td>0.471</td><td>0.456</td><td>0.449</td><td>0.322</td><td>0.435</td><td>0.467</td><td>0.363</td><td>0.341</td><td>0.382</td><td>0.696</td><td>0.623</td><td>0.649</td><td>0.4722</td><td>0.4706</td><td>0.4802</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.217</td><td>0.485</td><td>0.444</td><td>0.214</td><td>0.383</td><td>0.391</td><td>0.244</td><td>0.467</td><td>0.451</td><td>0.214</td><td>0.441</td><td>0.443</td><td>0.270</td><td>0.455</td><td>0.513</td><td>0.2318</td><td>0.4462</td><td>0.4484</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.214</td><td>0.264</td><td>0.356</td><td>0.254</td><td>0.259</td><td>0.342</td><td>0.260</td><td>0.386</td><td>0.376</td><td>0.244</td><td>0.283</td><td>0.304</td><td>0.266</td><td>0.368</td><td>0.376</td><td>0.2476</td><td>0.3120</td><td>0.3508</td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.214</td><td>0.259</td><td>0.314</td><td>0.216</td><td>0.326</td><td>0.334</td><td>0.263</td><td>0.403</td><td>0.398</td><td>0.301</td><td>0.315</td><td>0.304</td><td>0.427</td><td>0.333</td><td>0.319</td><td>0.2842</td><td>0.3272</td><td>0.3338</td>
  </tr>
</table>

Table 7: Weighed f1 score for classification. Judging based on the inquiry and answer hold similar f1 score while based on the prompt shows imbalanced judgment. Judge based on answer does not surpass judge based on answer mainly because we directly regard those low quality inquiries as lack of capacity because those inquiries can hardly gather useful information, while judge based on inquiry might guess a random answer.

<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="6">HotpotQA</th>
    <th colspan="6">AmbigQA</th>
    <th colspan="6">TechQA</th>
    <th colspan="6">ExpertQA</th>
    <th colspan="6">ToolBench</th>
    <th colspan="6">avg</th>
  </tr>
  <tr>
    <th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th><th>prompt</th><th>inquiry</th><th>answer</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.446</td><td>0.592</td><td>0.669</td><td>0.600</td><td>0.708</td><td>0.731</td><td>0.760</td><td>0.779</td><td>0.800</td><td>0.729</td><td>0.737</td><td>0.746</td><td>0.846</td><td>0.856</td><td>0.854</td><td>0.6762</td><td>0.7342</td><td>0.7600</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.515</td><td>0.538</td><td>0.562</td><td>0.600</td><td>0.631</td><td>0.685</td><td>0.783</td><td>0.788</td><td>0.800</td><td>0.779</td><td>0.748</td><td>0.742</td><td>0.794</td><td>0.837</td><td>0.850</td><td>0.6942</td><td>0.7085</td><td>0.7277</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.585</td><td>0.546</td><td>0.608</td><td>0.631</td><td>0.677</td><td>0.692</td><td>0.804</td><td>0.794</td><td>0.810</td><td>0.763</td><td>0.756</td><td>0.762</td><td>0.812</td><td>0.835</td><td>0.860</td><td>0.7188</td><td>0.7215</td><td>0.7462</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.438</td><td>0.477</td><td>0.508</td><td>0.538</td><td>0.600</td><td>0.554</td><td>0.758</td><td>0.733</td><td>0.752</td><td>0.721</td><td>0.738</td><td>0.746</td><td>0.725</td><td>0.740</td><td>0.748</td><td>0.6362</td><td>0.6577</td><td>0.6615</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.369</td><td>0.400</td><td>0.415</td><td>0.500</td><td>0.577</td><td>0.631</td><td>0.742</td><td>0.738</td><td>0.737</td><td>0.740</td><td>0.740</td><td>0.742</td><td>0.744</td><td>0.777</td><td>0.773</td><td>0.6181</td><td>0.6465</td><td>0.6596</td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.400</td><td>0.531</td><td>0.531</td><td>0.492</td><td>0.600</td><td>0.623</td><td>0.733</td><td>0.740</td><td>0.762</td><td>0.735</td><td>0.731</td><td>0.737</td><td>0.794</td><td>0.819</td><td>0.817</td><td>0.6308</td><td>0.6842</td><td>0.6938</td>
  </tr>
</table>

Table 8: The answer quality of all models
<table>
  <tr>
    <th rowspan="2">Precision</th>
    <th colspan="3">prompt</th>
    <th colspan="3">inquiry</th>
    <th colspan="3">answer</th>
  </tr>
  <tr>
    <th>doc</th><th>ambig</th><th>ability</th>
    <th>doc</th><th>ambig</th><th>ability</th>
    <th>doc</th><th>ambig</th><th>ability</th>
  </tr>
  <tr>
    <td>GPT-4o</td><td>0.53</td><td>0.41</td><td>0.28</td><td>0.5</td><td>0.57</td><td>0.29</td><td>0.49</td><td>0.64</td><td>0.25</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td><td>0.61</td><td>0.41</td><td>0.34</td><td>0.5</td><td>0.58</td><td>0.33</td><td>0.46</td><td>0.57</td><td>0.22</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td><td>0.56</td><td>0.43</td><td>0.38</td><td>0.52</td><td>0.56</td><td>0.42</td><td>0.51</td><td>0.6</td><td>0.28</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td><td>0.3</td><td>0.39</td><td>0.22</td><td>0.54</td><td>0.48</td><td>0.14</td><td>0.52</td><td>0.48</td><td>0.33</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td><td>0.43</td><td>0.39</td><td>0.31</td><td>0.53</td><td>0.39</td><td>0.25</td><td>0.55</td><td>0.39</td><td>0.25</td>
  </tr>
  <tr>
    <td>Mistral-7B</td><td>0.44</td><td>0.41</td><td>0.33</td><td>0.43</td><td>0.44</td><td>0.2</td><td>0.47</td><td>0.51</td><td>0.21</td>
  </tr>
  <tr>
    <th rowspan="2">Recall</th>
    <th colspan="3">prompt</th>
    <th colspan="3">inquiry</th>
    <th colspan="3">answer</th>
  </tr>
  <tr>
    <th>doc</th><th>ambig</th><th>ability</th>
    <th>doc</th><th>ambig</th><th>ability</th>
    <th>doc</th><th>ambig</th><th>ability</th>
  </tr>
  <tr>
    <td>GPT-4o</td><td>0.19</td><td>0.76</td><td>0.2</td><td>0.74</td><td>0.38</td><td>0.17</td><td>0.62</td><td>0.36</td><td>0.31</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td><td>0.07</td><td>0.91</td><td>0.16</td><td>0.79</td><td>0.37</td><td>0.15</td><td>0.65</td><td>0.35</td><td>0.19</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td><td>0.39</td><td>0.7</td><td>0.17</td><td>0.79</td><td>0.36</td><td>0.24</td><td>0.63</td><td>0.33</td><td>0.38</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td><td>0.12</td><td>0.97</td><td>0.02</td><td>0.53</td><td>0.72</td><td>0.01</td><td>0.52</td><td>0.67</td><td>0.07</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td><td>0.1</td><td>0.94</td><td>0.1</td><td>0.1</td><td>0.64</td><td>0.35</td><td>0.18</td><td>0.57</td><td>0.37</td>
  </tr>
  <tr>
    <td>Mistral-7B</td><td>0.12</td><td>0.85</td><td>0.14</td><td>0.45</td><td>0.23</td><td>0.36</td><td>0.58</td><td>0.09</td><td>0.44</td>
  </tr>
</table>

Table 9: Precision and recall of the methods

<table>
  <tr>
    <th></th>
    <th>HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBench</th>
    <th>avg</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.669</td>
    <td>0.731</td>
    <td>0.800</td>
    <td>0.746</td>
    <td>0.854</td>
    <td>0.7600</td>
  </tr>
  <tr>
    <td>vanilla</td>
    <td>0.569</td>
    <td>0.669</td>
    <td>0.746</td>
    <td>0.715</td>
    <td>0.831</td>
    <td>0.7062</td>
  </tr>
  <tr>
    <td>SFT</td>
    <td>0.600</td>
    <td>0.708</td>
    <td>0.777</td>
    <td>0.715</td>
    <td>0.846</td>
    <td>0.7292</td>
  </tr>
  <tr>
    <td>DPO</td>
    <td>0.623</td>
    <td>0.715</td>
    <td>0.787</td>
    <td>0.731</td>
    <td>0.854</td>
    <td>0.7419</td>
  </tr>
  <tr>
    <td>onlineDPO</td>
    <td>0.638</td>
    <td>0.723</td>
    <td>0.779</td>
    <td>0.746</td>
    <td>0.862</td>
    <td>0.7496</td>
  </tr>
  <tr>
    <td>InteractDPO</td>
    <td>0.654</td>
    <td>0.738</td>
    <td>0.792</td>
    <td>0.754</td>
    <td>0.854</td>
    <td>0.7585</td>
  </tr>
</table>

Table 10: The answer performance of InteractDPO

<table>
  <tr>
    <th></th>
    <th>HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBensch</th>
  </tr>
  <tr>
    <td>ICC</td>
    <td>0.968</td>
    <td>0.954</td>
    <td>0.949</td>
    <td>0.956</td>
    <td>0.969</td>
  </tr>
  <tr>
    <td>omega</td>
    <td>0.84</td>
    <td>0.83</td>
    <td>0.83</td>
    <td>0.84</td>
    <td>0.82</td>
  </tr>
</table>

Table 11: Reliability
<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="5"> </th>
    <th rowspan="2">avg</th>
  </tr>
  <tr>
    <th>HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBench</th>
  </tr>
  <tr>
    <td rowspan="3">GPT-4o</td>
    <td>prompt</td>
    <td>0.685</td>
    <td>0.631</td>
    <td>0.515</td>
    <td>0.454</td>
    <td>0.585</td>
    <td>0.5738</td>
  </tr>
  <tr>
    <td>inquiry</td>
    <td>0.638</td>
    <td>0.531</td>
    <td>0.538</td>
    <td>0.492</td>
    <td>0.669</td>
    <td>0.5738</td>
  </tr>
  <tr>
    <td>answer</td>
    <td>0.708</td>
    <td>0.569</td>
    <td>0.600</td>
    <td>0.554</td>
    <td>0.731</td>
    <td>0.6323</td>
  </tr>
  <tr>
    <td rowspan="3">DeepSeek-V3</td>
    <td>prompt</td>
    <td>0.538</td>
    <td>0.500</td>
    <td>0.546</td>
    <td>0.469</td>
    <td>0.369</td>
    <td>0.4846</td>
  </tr>
  <tr>
    <td>inquiry</td>
    <td>0.654</td>
    <td>0.500</td>
    <td>0.523</td>
    <td>0.577</td>
    <td>0.592</td>
    <td>0.5692</td>
  </tr>
  <tr>
    <td>answer</td>
    <td>0.631</td>
    <td>0.585</td>
    <td>0.515</td>
    <td>0.577</td>
    <td>0.569</td>
    <td>0.5754</td>
  </tr>
  <tr>
    <td rowspan="3">Qwen2.5-72B</td>
    <td>prompt</td>
    <td>0.515</td>
    <td>0.623</td>
    <td>0.569</td>
    <td>0.431</td>
    <td>0.385</td>
    <td>0.5046</td>
  </tr>
  <tr>
    <td>inquiry</td>
    <td>0.577</td>
    <td>0.600</td>
    <td>0.608</td>
    <td>0.492</td>
    <td>0.623</td>
    <td>0.5800</td>
  </tr>
  <tr>
    <td>answer</td>
    <td>0.631</td>
    <td>0.662</td>
    <td>0.585</td>
    <td>0.515</td>
    <td>0.677</td>
    <td>0.6138</td>
  </tr>
  <tr>
    <td rowspan="3">Llama-3-70B</td>
    <td>prompt</td>
    <td>0.469</td>
    <td>0.469</td>
    <td>0.377</td>
    <td>0.408</td>
    <td>0.185</td>
    <td>0.3815</td>
  </tr>
  <tr>
    <td>inquiry</td>
    <td>0.508</td>
    <td>0.485</td>
    <td>0.577</td>
    <td>0.438</td>
    <td>0.438</td>
    <td>0.4892</td>
  </tr>
  <tr>
    <td>answer</td>
    <td>0.554</td>
    <td>0.515</td>
    <td>0.577</td>
    <td>0.462</td>
    <td>0.469</td>
    <td>0.5154</td>
  </tr>
  <tr>
    <td rowspan="3">Qwen2.5-7B</td>
    <td>prompt</td>
    <td>0.454</td>
    <td>0.477</td>
    <td>0.415</td>
    <td>0.369</td>
    <td>0.331</td>
    <td>0.4092</td>
  </tr>
  <tr>
    <td>inquiry</td>
    <td>0.515</td>
    <td>0.508</td>
    <td>0.462</td>
    <td>0.508</td>
    <td>0.362</td>
    <td>0.4708</td>
  </tr>
  <tr>
    <td>answer</td>
    <td>0.554</td>
    <td>0.592</td>
    <td>0.462</td>
    <td>0.500</td>
    <td>0.469</td>
    <td>0.5154</td>
  </tr>
  <tr>
    <td rowspan="3">Mistral-7B</td>
    <td>prompt</td>
    <td>0.615</td>
    <td>0.615</td>
    <td>0.654</td>
    <td>0.408</td>
    <td>0.408</td>
    <td>0.5400</td>
  </tr>
  <tr>
    <td>inquiry</td>
    <td>0.608</td>
    <td>0.577</td>
    <td>0.592</td>
    <td>0.492</td>
    <td>0.531</td>
    <td>0.5600</td>
  </tr>
  <tr>
    <td>answer</td>
    <td>0.600</td>
    <td>0.562</td>
    <td>0.592</td>
    <td>0.500</td>
    <td>0.538</td>
    <td>0.5585</td>
  </tr>
</table>

Table 12: Uncertainty Classification Performance with few shot examples

Question: What league did the team that played home games at the stadium belong to?

Document: Document 1: team A played home games at X stadium; Document 2: team B played home games at Y stadium

We show the few shot results in Table 13, 12 and 14, and we find that for some models, add few shot examples may even hurt the performance, this is mainly because the examples might mislead the model to classify some samples to some specific class, leading to imbalance classification and worse performance. Therefore, how to choose few shot examples remains a question.

F Prompts

Prompt to Ambiguate the AMR

Given a query and the corresponding Abstract Meaning Representation (AMR), you should manipulate the AMR to obscure it, making it impossible to answer without further clarification. Make sure that the obscured AMR should not change the intention of the question, the obscured AMR should be unanswerable, and the obscured AMR should also be a question rather than a statement. Here are some possible actions to manipulate the AMR.

1. Remove certain modifiers and descriptive words to make some nouns in the query ambiguous.
2. Delete some key information, making the query impossible to answer
3. Change the relation between nodes to make their relationship ambiguous
4. Reorganize the structure of the AMR, make it less clear

The following are some requirements for the obscured query.

1. The obscured query should still be a question rather than a statement
2. the obscured query should be similar to a question that a man would actually ask rather than some vague question like "what is the man's name"
3. The obscured should not be answerable without further clarification,
4. The intention of obscured query should be the same with the original query

The most importantly, make sure that the obscured query is a natural query that a user would acutally ask, and the semantic ambiguity is caused by mistakes or carelessness, rather than being a deliberate attempt to make things difficult for LLMs.

Please think step by step to generate the obscured AMR satisfying the above
<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="6">HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBench</th>
    <th>avg</th>
  </tr>
  <tr>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>avg</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.608</td>
    <td>0.562</td>
    <td>0.615</td>
    <td>0.708</td>
    <td>0.677</td>
    <td>0.731</td>
    <td>0.800</td>
    <td>0.804</td>
    <td>0.823</td>
    <td>0.738</td>
    <td>0.758</td>
    <td>0.756</td>
    <td>0.808</td>
    <td>0.813</td>
    <td>0.835</td>
    <td>0.7323</td>
    <td>0.7227</td>
    <td>0.7519</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.546</td>
    <td>0.608</td>
    <td>0.623</td>
    <td>0.669</td>
    <td>0.577</td>
    <td>0.631</td>
    <td>0.823</td>
    <td>0.810</td>
    <td>0.810</td>
    <td>0.775</td>
    <td>0.777</td>
    <td>0.775</td>
    <td>0.785</td>
    <td>0.783</td>
    <td>0.794</td>
    <td>0.7196</td>
    <td>0.7108</td>
    <td>0.7265</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.515</td>
    <td>0.538</td>
    <td>0.623</td>
    <td>0.654</td>
    <td>0.623</td>
    <td>0.662</td>
    <td>0.821</td>
    <td>0.796</td>
    <td>0.808</td>
    <td>0.779</td>
    <td>0.788</td>
    <td>0.788</td>
    <td>0.777</td>
    <td>0.812</td>
    <td>0.833</td>
    <td>0.7092</td>
    <td>0.7115</td>
    <td>0.7427</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.423</td>
    <td>0.485</td>
    <td>0.515</td>
    <td>0.585</td>
    <td>0.562</td>
    <td>0.585</td>
    <td>0.740</td>
    <td>0.769</td>
    <td>0.758</td>
    <td>0.733</td>
    <td>0.727</td>
    <td>0.737</td>
    <td>0.717</td>
    <td>0.738</td>
    <td>0.738</td>
    <td>0.6396</td>
    <td>0.6562</td>
    <td>0.6665</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.377</td>
    <td>0.431</td>
    <td>0.477</td>
    <td>0.569</td>
    <td>0.600</td>
    <td>0.638</td>
    <td>0.738</td>
    <td>0.729</td>
    <td>0.756</td>
    <td>0.748</td>
    <td>0.740</td>
    <td>0.752</td>
    <td>0.737</td>
    <td>0.744</td>
    <td>0.767</td>
    <td>0.6338</td>
    <td>0.6488</td>
    <td>0.6781</td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.523</td>
    <td>0.592</td>
    <td>0.608</td>
    <td>0.608</td>
    <td>0.638</td>
    <td>0.638</td>
    <td>0.796</td>
    <td>0.792</td>
    <td>0.794</td>
    <td>0.735</td>
    <td>0.721</td>
    <td>0.719</td>
    <td>0.800</td>
    <td>0.810</td>
    <td>0.808</td>
    <td>0.6923</td>
    <td>0.7108</td>
    <td>0.7135</td>
  </tr>
</table>

Table 13: Answer Quality of few shot prompting

<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="6">HotpotQA</th>
    <th>AmbigQA</th>
    <th>TechQA</th>
    <th>ExpertQA</th>
    <th>ToolBench</th>
    <th>avg</th>
  </tr>
  <tr>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>prompt</th>
    <th>inquiry</th>
    <th>answer</th>
    <th>avg</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.625</td>
    <td>0.592</td>
    <td>0.596</td>
    <td>0.545</td>
    <td>0.427</td>
    <td>0.325</td>
    <td>0.343</td>
    <td>0.445</td>
    <td>0.416</td>
    <td>0.389</td>
    <td>0.436</td>
    <td>0.451</td>
    <td>0.516</td>
    <td>0.600</td>
    <td>0.640</td>
    <td>0.4836</td>
    <td>0.5000</td>
    <td>0.4856</td>
  </tr>
  <tr>
    <td>DeepSeek-V3</td>
    <td>0.457</td>
    <td>0.552</td>
    <td>0.530</td>
    <td>0.433</td>
    <td>0.395</td>
    <td>0.393</td>
    <td>0.409</td>
    <td>0.421</td>
    <td>0.398</td>
    <td>0.340</td>
    <td>0.466</td>
    <td>0.455</td>
    <td>0.330</td>
    <td>0.526</td>
    <td>0.499</td>
    <td>0.3938</td>
    <td>0.4720</td>
    <td>0.4550</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B</td>
    <td>0.486</td>
    <td>0.470</td>
    <td>0.480</td>
    <td>0.435</td>
    <td>0.480</td>
    <td>0.417</td>
    <td>0.488</td>
    <td>0.503</td>
    <td>0.433</td>
    <td>0.361</td>
    <td>0.416</td>
    <td>0.422</td>
    <td>0.281</td>
    <td>0.590</td>
    <td>0.648</td>
    <td>0.4102</td>
    <td>0.4918</td>
    <td>0.4800</td>
  </tr>
  <tr>
    <td>Llama-3-70B</td>
    <td>0.331</td>
    <td>0.385</td>
    <td>0.444</td>
    <td>0.294</td>
    <td>0.362</td>
    <td>0.400</td>
    <td>0.207</td>
    <td>0.502</td>
    <td>0.493</td>
    <td>0.249</td>
    <td>0.359</td>
    <td>0.384</td>
    <td>0.157</td>
    <td>0.359</td>
    <td>0.393</td>
    <td>0.2476</td>
    <td>0.3934</td>
    <td>0.4228</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>0.293</td>
    <td>0.295</td>
    <td>0.342</td>
    <td>0.372</td>
    <td>0.327</td>
    <td>0.397</td>
    <td>0.234</td>
    <td>0.337</td>
    <td>0.339</td>
    <td>0.205</td>
    <td>0.378</td>
    <td>0.414</td>
    <td>0.267</td>
    <td>0.324</td>
    <td>0.441</td>
    <td>0.2742</td>
    <td>0.3322</td>
    <td>0.3866</td>
  </tr>
  <tr>
    <td>Mistral-7B</td>
    <td>0.508</td>
    <td>0.336</td>
    <td>0.335</td>
    <td>0.418</td>
    <td>0.311</td>
    <td>0.332</td>
    <td>0.300</td>
    <td>0.204</td>
    <td>0.195</td>
    <td>0.208</td>
    <td>0.216</td>
    <td>0.229</td>
    <td>0.249</td>
    <td>0.324</td>
    <td>0.332</td>
    <td>0.3366</td>
    <td>0.2782</td>
    <td>0.2846</td>
  </tr>
</table>

Table 14: Weighted f1 score of few shot prompting
requirements, then translate it into the obscured text query. Your output should be formatted as Dict{"step_by_step_thinking": Str(explanation) , "Obscured Abstract Meaning Representation (AMR)": Str{AMR}, "Translated Text Query": Str{obscured text query}}.

Query: {}
Abstract Meaning Representation (AMR): {}

Please think step-by-step and generate your output in json:

Prompt to check the result of ambiguity

Given a query, its obscured version and clarified query based on the obscured query, now you need to judge that is the obscurity successful. A obscurity of the original query should satisfy the following conditions:

1. The obscured query should still be a question rather than a statement
2. the obscured query should be similar to a question that a man would actually ask rather than some vague question like "what is the man's name"
3. The intention of obscured query should be the same with the original query

Here we give some examples showing that the obscure query is a failure,
...

Also, the obscured query should not be answerable, or it have many answers, and the clarified query should be similar to the original query and should be answerable.

Therefore, the answer of those query should satisfy:
1. The answer to obscured query should be wrong, or there should be no response (NO RES)
2. For the obscured query with clarification, the answer should be the same or similar to the answer to the original query

Combine those conditions, a successful obscurity should satisfy the following conditions:

1. The obscured query should still be a question rather than a statement
2. the obscured query should be similar to a question that a man would actually ask rather than some vague question like "what is the man's name"
3. The obscured should not be answerable , or it have many answers
4. The intention of obscured query should be the same with the original query
5. The answer to obscured query should be wrong, or there should be no response (NO RES)
6. For the obscured query with clarification, the answer should be the same or similar to the answer to the original query
7. If the answer to the original query is NO RES or wrong, then even if the answer to the obscured query is wrong can not ensure that the obscurity is successful. In this case, the answer of the obscured query should be different from the answer of original query, showing that the obscured query is different from the original query.

Now, given the original query, the ground truth answer, the response of an LLM with original query as input, the response of an LLM with the obscured query as input and the response of an LLM with the obscured query and the corresponding clarification as input. All the responses are generated for multiple times. Please think step by step and judge that is the obscurity successful. Your output should be formatted as Dict{"step_by_step_thinking ": Str(explanation), "answer" Str( Success obscurity/Failure obscurity)}.

Original Query: {}
Answer to Original Query: {}

Obscured Query: {}
Answer to Obscured Query: {}

Clarified Query: {}
Answer to Clarified Query: {}

Please think step-by-step and generate your output in json:

Prompt to generate gold inquiry

Below is a question the corresponding gold documents to answer the question. We hide some key information to answer the question by obscuring the question or hiding some documents. Your task is to recognize those missing information and generate a corresponding inquiry to gather those information step by step.

We would only provide the query information or the document information. When we provide query information, you should identify what information is missing in the actual query compared to the original query. When we provide document information, you should identify which document is missing in the actual documents.

Now please generate the inquiry for the following query
Original Query: {}
Gold Document: {}
Actual Query: {}
Actual Document: {}

Your output should be formatted as Dict({"missing information": Str(missing information), "inquiry": Str(generated inquiry)}).
Your should strictly format your response in this format, no extra tokens should be added.

Prompt to evaluate the inquiry

Given a question the corresponding gold documents to answer the question, we obscure the question or hide some key documents and generate an inquiry to gather those missing information. Your task is to evaluate the quality of the inquiry.
Evaluation Criteria:

Accurate: Does the inquiry directly indicate the missing information?
Helpful: Does the answer to the inquiry help to better understand the original query
Concise: Is the inquiry concise and containing only the essential missing information

Scoring: Rate outputs on a scale of 1 to 5:
1. Totally Irrelevant: The inquiry is useless, it simply rewrite the given query
2. Somewhat Relevant: The inquiry is somewhat relevant to the missing information, but the inquiry can hardly gather useful information
3. Basically Relevant: The inquiry asks something relevant to the missing information, there is a certain possibility of obtaining relevant information by the inquiry.
4. Good: The inquiry directly asks the missing information, but not concise enough, there is great possibility that some useful information would be gathered.
5. Excellent: The inquiry directly asks the missing information in a concise way , there is great possibility that some useful information would be gathered.

Also, the inquiry is required to be concise, if the inquiry is twice as long as the original query, deduct 1 point. The minimum score is 1 point.

Original Query: <{}>
Gold Document: <{}>
Actual Query: <{}>
Actual Document: <{}>
Missing Detail and Gold Inquiry: <{}>
Problematic Inquiry: <{}>

Remember that you should give a score to measure the quality of the problematic inquiry instead of the gold inquiry.

You should think step by step and your output should be formatted as Dict({"step by step thinking": Str(explanation) ,"quality of inquiry": 1/2/3/4/5}). You should strictly format your response in this format, no extra tokens should be added.

Prompt to generate clarification

You are an user who asks a question to the LLM, the query you provided might be ambiguous and the LLM asks you for further clarification by inquiry. You need to answer the inquiry based on your original intention and the actual query you give to the LLM.

Original Intention: {}
Actual Query: {}
Inquiry: {}

Note that you do not know the answer to your original intention and if the inquiry involves the answer of the original intention, please answer with "This question is beyond scope we can not answer your question".

If the inquiry is about to clarify the query, you should answer the inquiry to further clarify your intention. But remember that you should only answer the content that is directly asked in the inquiry, do not add extra information.

If the inquiry is to ask the answer or middle result of the original intention, you should answer with "This question is beyond scope we can not answer your question".

Please generate your response strictly within 50 tokens.

Prompt to judge the source of uncertainty

One user gives a query and some documents are retrieved to help answer the query. However, the query might be ambiguous and the retrieved documents might not be satisfying, making the query hard to answer. Your task is to identify why the query is hard to answer .

Question:
{}
Document:
{}

Based on those information. Here are three kinds of actions you can take,
A: Interact with the retrieval system if you need some more factual or document information.
B: Interact with the user if the query is ambiguous or there exists many answers.
C: Conducting Chain of Thought if deeper thinking is required.

Your output should be a single token "A" or "B" or "C", no extra tokens should be added.

Prompt to generate inquiry for further retrieval

One user gives a query and some documents are retrieved to help answer the query. However, the retrieved documents is satisfying, making the query hard to answer. Your task is to generate an inquiry to gather further document information to answer the question.

Question:
{}
Document:
{}

Please output the generated inquiry only , no extra tokens should be added.

Prompt to generate inquiry for clarification

One user gives a query and some documents are retrieved to help answer the query. However, the query is ambiguous, making the query hard to answer. Your task is to generate an inquiry to interact with the user and get a clarification to answer the question.

Question:
{}
Document:
{}

Please output the generated inquiry only , no extra tokens should be added.

Prompt to generate inquiry based on prompt

One user gives a query and some documents are retrieved to help answer the query. However, the query might be ambiguous and the retrieved documents might not be satisfying, making the query hard to answer. Your task is to identify why the query is hard to answer and generate an inquiry to gather further information to answer the question.

Here are some requirements for the inquiry

1. You should ask for only one question in the inquiry.
2. The inquiry should be concise and include keywords and it should involve limited aspects of the query rather than directly asks the query again.

Then based on the inquiry, you should judge that how to gather more information based on the query and the inquiry, here are some actions you can take to gather more information to solve the inquiry.

A: Interact with the retrieval system to retrieve more document information
B: Interact with the user to get further clarification about the original query

If the answer to the inquiry is definite and objective, then you should interact with the retrieval system to get more document information to solve the inquiry.
If the answer to the inquiry is not definite and it is some subjective choices of the user, you should interact with the user to clarify the original query.
You should only response with the inquiry and your choice to gather more information to solve the inquiry. Please response with Dict({"Inquiry": "Str(generated inquiry)","Choice" : "A/B"})

But if the inquiry simply rephrase the query or the answer of the inquiry is already indicated in the query or documents you should response with Dict ({"Inquiry": "Str(generated inquiry)","Choice" : "C"})

Query: {}
Documents: {}

Please generate your answer in json.

Prompt to generate the answer of inquiry

Given a query, an LLM generate an further inquiry to gather more information about the query. Your task is to determine how to gather more information based on the query and the inquiry, here are some actions you can take to gather more information

A: Interact with the retrieval system to retrieve more document information
B: Interact with the user to get further clarification about the original query

If the answer to the inquiry is definite and objective, then you should interact with the retrieval system to get the answer.
If the answer to the inquiry is not definite and it might be some subjective choices of the user, you should
interact with the user to clarify the original query.

Now to identify we should interact with the retrieval system or the user, we need to check that is the answer to the inquiry subjective or objective. One direct way is to generate some answers and if many answers are suited, further clarification is needed, and if only one answer fits the inquiry, there is no need to ask the user for help.

Your task is to give the answer to the inquiry. We provide the original query and the corresponding document information which may help to answer the query as well as the generated inquiry. Also we provide some answers which fits the inquiry well. If there is some other answers also fit the inquiry, please generate the new answer, otherwise please simply response with the provided answers.

Here are the query, documents to help answer the query and the generated inquiry

Query: {}
Query Document: {}
Inquiry: {}
Here we provide some answers to the inquiry,
Possible Answers: {}

Here are some requirements for your response:
1. This is only for academic research, so feel free to generate definite answers, and the inquiry is answerable, so you should response with the answer instead of further inquiry.
2. Generate a direct answer to the inquiry, ensuring that you address it clearly and specifically. No matter what the inquiry is, you should generate an answer. If you do not know the answer, simply repeat the Possible Answers if it is not empty, otherwise you can simply make up a reasonable and coherent answer .
3. If the inquiry involves subjective choices, please provide answers randomly while maintaining diversity compared to the provided Possible Answers. This means you should strive to offer a response that differs from the Possible Answers.
4. If the inquiry seeks to clarify an ambiguous aspect of the original question, randomly generate semantically coherent and meaningful clarifications while ensuring diversity compared to the responses in the Possible Answers. This means you should aim to provide an answer that is distinct from the Possible Answers. And you do not need to ensure that the answer is correct.
5. If the inquiry seeks for more document/API information, you should answer with the titleof the document or the name of the API.
6. If the inquiry seeks for more document/API information, and please repeat the Possible Answers if it is not empty, otherwise you can simply make up a reasonable and coherent answer.
Remember, you should answer with only the title/name of the document/API.
7. Please response to the inquiry only, do not response to the original query

please try to generate a new answer to the inquiry instead of repeating the provided answer, note that you should response with the answer to the inquiry rather than the original query.

Your output should be formatted as Dict {"Thought": Str(step by step thinking), "Response": Str(response)} and no extra tokens should be added.

Answer the Query

One user gives a query and your task is to answer the query. Here are the question and the retrieved documents.

Question: {}
Document: {}
Question: {}

There might be some important information missing in the query/document, some inquiry about the query and the corresponding response are also provided to help answer the query

Inquiry History: {}

Please generate your answer within 50/500 tokens.

Answer the Query by CoT

One user gives a query and your task is to answer the query. Here are the question and the retrieved documents.

Question: {}
Document: {}
Question: {}

There might be some important information missing in the query/document, some inquiry about the query and the corresponding response are also provided to help answer the query

Inquiry History: {}

Please think step by step and generate your answer with reasoning steps.

Reviewer paper summaries: The paper introduces ConfuseBench, a novel benchmark designed to evaluate how well LLMs identify and resolve different sources of uncertainty. Sources of uncertainty are categorized into three distinct types: document scarcity, limited capability, and query ambiguity. The main motivation of the paper is that current models often misattribute uncertainty, and that it is often not desirable to abstain from generating. 
A two-step approach is presented where context-aware inquiries are generated to pinpoint the root cause of uncertainty before applying remedial strategies such as retrieval or chain-of-thought reasoning. They further enhance this process with an on-policy training method named InteractDPO, which refines inquiry generation through real-time feedback. Experimental results across several LLMs indicate that while performance improvements are achievable, models still struggle to consistently differentiate between the underlying uncertainty sources.
Summary of strengths: -- Focus on a critical yet often overlooked aspect of LLM behavior—identifying and mitigating uncertainty in responses. 

-- ConfuseBench benchmark is an attempt to offer a systematic framework for evaluating uncertainty handling in models. 

-- Integration of context-aware inquiry generation with an on-policy training method (InteractDPO) shows promising improvements in guiding models to address uncertainty more effectively.
Summary of weaknesses: -- The distinction between different uncertainty types isn’t always clear, potentially conflating document scarcity with factual knowledge gaps. Could the authors say more on this front?

-- More of a question. I think that the complexity of the complexity of the proposed InteractDPO method might impede its practical scalability and replicability.
Comments, suggestions and typos: I think the paper would benefit from clearer, more detailed definitions and examples for each type of uncertainty to avoid ambiguity. Also, a more concise presentation of the methodology, using additional diagrams or flowcharts, would improve readability. It would also be useful to discuss scalability issues of the proposed method.
Overall assessment scores: 4