Uncertainty Quantification in Retrieval Augmented Question Answering

Authors: Laura Perez-Beltrachini, Mirella Lapata
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: uncertainty quantification, retrieval augmented question answering, large language models

Abstract:
Retrieval augmented Question Answering (QA) enables QA models to overcome knowledge gaps when answering questions at test time by taking as input the question together with retrieved evidence, that is usually a set of passages.  Previous studies show that this approach has numerous benefits such as improving QA performance and reducing hallucinations, without, however, qualifying whether the retrieved passages are indeed useful at answering correctly. In this work, we evaluate existing uncertainty quantification approaches and propose an approach that predicts answer correctness based on utility judgements on individual input passages. We train a small neural model that predicts passage utility for a target QA model. We find that simple information theoretic metrics can predict answer correctness up to a certain extent, more expensive sampling based approaches perform better, while our lightweight approach can efficiently approximate or improve upon sampling-based approaches.

Decision: reject
Reviewer scores: 5, 5, 3, 6, 5

Reviews and discussion:
<thread>
creation date:
1730594371235

thread_type:
official_review

<thread_object>
creator: reviewer_575d

<content>
summary:
This paper evaluates existing uncertainty quantification approaches that are used to quantify whether the retrieved passages at test time are useful to answer the question correctly. This paper also propose a neural-model based utility ranker that predicts answer correctness based on utility judgements on individual input passage, which boost the accuracy by 4% for NQ, and 2% for WebQ. The utility ranker trumps some uncertainty detection method for some datasets for the Gemma model. However, more analysis and explanations could be done.

strengths:
- Previous work for QA error detection are either expensive to run for in-production QA systems, or rely on model’s internal confidence, and are for closed-book QA, so they are not applicable to retrieval augmented setup. The proposed utility ranker,
- The utility ranker differs from Asai et al. (2024) because Asai et al. (2024) uses external critic model to judge, while the proposed method is target QA model based.
- The baseline methods are thoroughly tested in both table 2, table 3 and table 4.
- After applying utility ranker to filter out irrelevant passages, the accuracy and accuracy LM increases for both model on NQ and WebQ.

weaknesses:
- Section 3 overcomplicates the method, and some of the math definitions are confusing instead of explaining the details. If I understand it correctly, the usefulness of a passage $p$ is defined on whether the model can correctly answer the question with the passage. The utility score is defined as the mean of accuracy and entailment score, where both scores are binary, so the only possible value for $u$ is 0, 0.5, or 1. Then it is combined with a binary cross entropy objective, and train a Siamese network that uses DistilRoBERTa to encode text pairs, and then use the ALBERT-xlarge model, trained on MNLI and VitaminC, is used to determine the entailment.
- Result analysis could be done for section 5.3. Although both Acc and AccLM is improved by utility ranker, some explanations is appreciated. Why is the analysis only done for NQ and WebQ? Both Accuracy and AccuracyLM increases the same amount, is it a coincidence or one metric is enough.

questions:
- Is the $m$ in equation 1 a hyper-parameter?
- In equation 2, the summation is over $u_i$ and $u_j$, but they didn’t show up in the equation. You also mention $p(y) = \sigmoid(u)$, but is it $u_i$ or $u_j$?
- The citation format needs fixing, not limited to:
    - Line 208: (PMI; Takayama & Arase, 2019), as well as the citation for p(true) on line 211 needs fixing.
    - Line 218: Holtzman et al. 2020.
    - Line 224, Gemma2-9B-Instruct (Riviere et al., 2024), and line 227 for contriever.
    - Line 232: You can use \citep for multiple citations, and use comma to separate each citation.
- Missing citation: Top-k sampling is from Fan et al. (2018).
- Why do you select |R| = 3 for table 5 rather than |R| = 5?
- Is there analysis about when and what |R| people should use? Does the effect enhance or decrease when |R| change? Is the method still relevant if there are more than X number of passages?
- It seems like utility ranker works better on Gemma rather than Llama, if experiment could be run on other models to confirm that utility ranker does work for most models, that would be wonderful.

## Reference

Fan, Angela, Mike Lewis, and Yann Dauphin. "Hierarchical neural story generation." *arXiv preprint arXiv:1805.04833*(2018).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730925944444

thread_type:
official_review

<thread_object>
creator: reviewer_gkp3

<content>
summary:
The paper proposes a straightforward approach of using a small passage utility model to improve the calibration of larger LLM-based QA models; i.e. it proposes a method to predict the reliability of the LLM answer based on the utility of the retrieved passages.

For a question $q$, the set of retrieved passages $R$ = $[p_1, p_2, ..., p_{|R|}]$, and a QA model $M$, the utility of a passage $p \in R$ is given by: $$ u = (a + e) / 2$$ where $a$ is the accuracy of the $M$ in predicting the ground-truth answer given passage $p$ and $e$ is the NLI entailment score of the question and predicted answer given the passage. A distillRoBERTa-based LM is trained to fit the utility scores. At inference time, the utility predictor assigns a score to each retrieved passage (given the question). The maximum utility score over all passages is used as the heuristic to abstain from answering.

The quality of different calibration techniques is compared on 4 QA datasets: NaturalQuestions, TriviaQA, WebQuestions, and SQuAD. The calibration techniques are compared on area under the rejection accuracy curve (calibration of abstaining) and AUROC of detecting incorrect answers. For two QA models, the trained utility predictor matches or improves over the performance of simple answer entropy-based heuristics. It is shown to be competitive with more complicated calibration techniques that rely on resampling multiple answers from the QA model.

An experiment is conducted to show that the utility predictor trained on NQ can be generalized out of distribution to SQuAD, PopQA, and RefuNQ. Moreover, the utility predictor can be used to rerank documents and improve QA accuracy.

strengths:
1. The contribution is intuitive and straightforward. The utility predictor is shown to be a low-cost mechanism for improving the calibration of larger, more expensive QA models.
2. The chosen experiments are appropriate. The experiment testing generalization across datasets is valuable and adds to the strength of the approach.
    - Some concerns about "completeness" of experiments are raised int he next section
3. The connection of utility prediction to passage reranking should be studied further. This is especially important since the utility predictor is shown to be an effect reranker.
    - Can we use utility score as a stronger signal for rankers in general (not just for calibration)?

weaknesses:
1. Several small details are missing in parts of the paper. Detailed questions are in the next section.
2. One big issue with the experiment set-up is that the QA models are not instructed to abstain. Thus, it is unclear how any of the calibration methods improve over the inherent ability of the QA models to abstain.
    - Under the current setup, even if the QA model abstains, it would be treated as "incorrect".
3. Please include a discussion of connections to "Evaluating Retrieval Quality in Retrieval-Augmented Generation" (SIGIR 2024)
    - They utilize a similar (query, passage) utility score for ranking retrieval systems

questions:
1. What is the range of the utility scores? Based on the definition in Line 162, the value should be between [0, 1]. If so, then:
    1. Why do you need a sigmoid in Eq (2)?
    2. How are predicted utility scores $< 0$ or $> 1$ in Figure 1? 
2. Sec 3.1: Is $y_M$ the predicted model answer given just passage $p$ or given the full set $R$?
3. Eq 1: Shouldn't the equation for margin loss be $max(0, m -y(u_i - u_j))$? i.e. the margin does not depend on $y$. What is actually implemented?
4. Line 181: It is unclear how important the BCE loss is. Please report the results of ablating the BCE loss.
5. Line 183: Is distillRoBERTa used as the utility predictor? How do you predict the utility score from the model? Please clarify the language in this line.
6. Line 200: Notation of utility predictor $v$ is misleading since $v$ does not depend on the model $M$ after it is trained. If I am misunderstanding this, please clarify.
7. Please include the ROC and RAC curves in the Appendix for completeness.
8. Line 259: How is the manual inspection performed? Over how many samples?
9. Line 269: It is unclear what you mean by "levels" here. Moreover, since all datasets are short-form QA, why do you believe clustering is affected?
10. Table 4: Please report OOD evaluation results of Utility Ranker (NQ) on TriviaQA and WebQuestions. The reported results of SQuAD are important, but it seems to be a setting where the utility ranker performed significantly better than all baselines. The distinction between different calibration approaches on the two other datasets is less clear.

Typos
---
- Line 134: Repeated "between"
</content>
</thread_object>

</thread>
<thread>
creation date:
1731000770004

thread_type:
official_review

<thread_object>
creator: reviewer_trrb

<content>
summary:
This work looks at the task of uncertainty estimation in retrieval-augmented, open-domain question answering. The frame their task as predicting confidence estimates of a base retrieval-based QA model's predictions (experiments with Llama + Gemma) based on the set of retrieved passages and the input query. Their proposed approach is based on approach is based on training a small, separate *utility prediction* model that estimates the confidence in a base QA model prediction based on the input query and a single retrieved passage. To estimate confidence in a final prediction using a set of retrieved passages, they take the max predicted utility over all passages as the final confidence estimate.

To train this *utility prediction* model, the authors average (? -- see question below) the binary correctness score of the base QA model's prediction on a given question + retrieved passage and the predicted probability of question and QA model's predicted answer being entailed by the retrieved passage, treating this as a gold "utility value". The authors then train their smaller *utility prediction* to predict these utility values by summing two losses: (1) the BCE loss of the predicted utility against the gold utility and (2) a ranking loss between passage rankings obtained from the gold and predicted utility values.

In their experiments, the authors compare against calibration baselines (all are based primarily on using only the base LLM with sampling, prompting, and analyzing its predicted distributions). They train and evaluate on a variety of QA datasets using Gemma2, and see minor gains/losses when evaluating on NQ, TriviaQA, and Webquestions and a significant improvement when evaluating on SQUAD. The authors also repeat these experiments using LLAMA3 as the base QA model, and observe more mixed gains/losses over the baselines.

strengths:
This work presents a method for uncertainty estimation in retrieval-based QA. Their method trains a separate smaller LM to estimate uncertainty in the base QA system's predictions based on a passage, question, and predicted answer. This system is trained on

weaknesses:
## Related Work + Baselines
Similar methods that use small, additional trained models to estimate uncertainty have been proposed by [1] and [2] ([1]  is referenced in related work, but not compared against). Additionally, [3] has also noted the overlap between this passage utility / calibration task and similarly uses pretrained NLI models to verify / estimate uncertainty in QA system predictions. Given the similarity of these methods, they are important points of comparison to understand how this method differs and how it affects performance. See point below.

[1] Selective question answering under domain shift
Amita Kamath, Robin Jia, Percy Liang

[2] Knowing More About Questions Can Help: Improving Calibration in Question Answering
Shujian Zhang, Chenyue Gong, Eunsol Choi

[3] Can NLI Models Verify QA Systems' Predictions?
Jifan Chen, Eunsol Choi, Greg Durrett 

## Evaluating role of the ranking loss and entailment score
A significant novelty from this work from the related works above is the usage of (1) an additional passage ranking loss (in addition to standard BCE loss) and (2) using entailment score in addition to answer correctness as a gold label to train the "passage utility predictor"; however, the role and usefulness of these changes are unclear. Additional ablation experiments would be helpful for understanding the impact of these changes and their benefits.

## Poor generalization to LLAMA3
While the results on Gemma2 seem promising, results using LLAMA3 as the base QA system are generally mixed/negative. Experimenting with more base QA systems and performing significance testing may help bolster these results.

questions:
(Note in Summary) In L162, is this in-line equation supposed to be the average of accuracy and entailment score?

Why are generalization experiments were limited to only GEMMA and training on NQ and evaluating on SQuAD, PopQA, RefuNQ? It would be interesting to see the performance using LLAMA (especially givent he negative results here) and training + evaluation on a greater number of dataset combinations.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731095556172

thread_type:
official_review

<thread_object>
creator: reviewer_naal

<content>
summary:
The paper proposes a novel approach for answer error prediction in retrieval augmented question answering. The premise is that the the retrieved passages and their interaction with the QA model’s parametric knowledge is a strong indicator of answer correctness. To measure this as a utility score for each passage, a small neural network is trained using a ranking loss - where the maximum utility score for each passage is the estimate for answer error prediction. On a few existing QA benchmarks (Natural Questions, TriviaQA, WebQuestions, SQuAD), this is shown to to be better than existing error prediction approaches based on entropy and resampling, while being more compute efficient.

strengths:
- The approach to train a separate smaller neural network to predict passage utility scores is novel. The construction of the data and loss for the scoring model, using entailment and accuracy is also intersting and original. 
- The paper provides an efficient way to predict the error rate at an example level, which could be very useful for latency sensitive systems in order to make a triggering decision for question answering.
- The overall flow of the paper is good, it is succinctly written, and the experimental results are compelling and clearly presented. 
- The paper also touches upon the reranking approach to improve the performance of QA model using their utility scoring model, which seems potentially useful for some applications.

weaknesses:
- One strong shortcoming of this approach is where multiple passages are needed to correctly answer the question, i.e. using multihop reasoning. In such cases, the utility both each of the passages in isolation could be low, and hurt the error prediction. Most of the baselines that use the entire passage set would be robust to this. 
- The modeling utility scores used to create  the ranking dataset has room for improvement. The scores could have smoother accuracy or entailment values instead of the binary values. And other, more principled aggregation functions could be explored instead of a simple average. 
- The evaluation for the utility ranker seems weak. The baseline in table 5 is not reranking at all. A better baseline could be a different utility ranker trained using the neural network, possibly with a simple objective such as predicting the error rate of the neutral network given x and p.

questions:
- In line 200, why is e arg max? Could be a typo.
- Did you compare the inference time difference between your approach and the baseline? It would be useful to see that comparison as well, since that's one of the key claims made.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731101476532

thread_type:
official_review

<thread_object>
creator: reviewer_zmta

<content>
summary:
This paper proposes an approach to measure the uncertainty in Retrieval Augmented Question Answering tasks. 
Concretely, they train a small neural network called utility ranker which assigns a score for each retrieved passage from a given retriever to judge if the retrieved passage is useful for the answer generated by some QA model. 
The authors show that this approach is on par or better than existing error prediction approaches while being light-weight at the same time.

strengths:
The authors ran experiments with 2 QA models and for a lot of the settings the utility ranker outperforms existing metrics in terms of uncertainty estimations of the retrieved passages. 
Experiments results also suggest that the method they proposed is also robust to OOD datasets where the ranker is not trained on.

weaknesses:
- Many of the notations are unclear. See in Questions. 
- QA models used for evaluation only limit to Gemma2-9b-instruct and Llama3.1-8b-instruct which are of similar size. More experiments should be done using models with various sizes to see if similar conclusions still hold. 
- For Llama3.1-8b-instruct, results from table3 seems to suggest that Utility Ranker is not doing better than just looking at the probability of generating the next token to be "True". Is the training of this ranker really necessary?

questions:
1. in (1), is m some hyper-parameter introduced in the model? If it was taken from other works, where did it come from? If it is optimized for this task, how did you optimize? 
2. the i and js from L_{rank} are never summed up in the total loss term. But I assume you do this for each retrieved passage pair, is that the case? 
3. How is the accuracy a defined at the bottom of page 3?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732804072201

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers,

We would like to thank you all for your thoughtful review process and valuable comments that helped to improve the quality and clarity of our paper. We would like to summarise the major discussion points that have been addressed in the re-submitted pdf.

- **Clarification of the contributions of our work.** We have updated the last two paragraphs of the introduction to make clearer the contributions of this work. We have incorporated in Appendix C.1 and Table 8 a description on the execution cost of each comparison uncertainty estimation method.

- **Additional target QA model** We have incorporated uncertainty quantification evaluation for and additional target QA model, namely *Mistral-7B-Instruct-v0.3* (Section 5.1 Table 1).

- **Ablation study on the Utility Ranker training objective** We have incorporated an ablation study on the different components of the training objective (Appendix D.1 Table 9) for the three target QA models.

- **Complete results under distribution-shift evaluation settings** We incorporated a zero-shot assessment of the Utility Ranker on different combinations of training/evaluation data (Section 5.2) and extended the results discussion about the adversarial QA tasks.

- **Complete experiments on improving QA performance** We have added results for additional datasets and another baseline.

- **General presentation** We incorporated more details on the description of the our approach (Section 3.1 and 3.2). We have improved the presentation of the results in Section 5.1 (added Figure 2). We incorporated a discussion of the suggested related work (L153 and L251).

We kindly request confirmation of receipt of our responses and the updated pdf and welcome any additional feedback.

Thank you for your time and consideration.

Sincerely,
The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces a new uncertainty quantification method for retrieval-augmented question answering. This method trains a small neural model to explicitly compute the utility of individual input passages for a downstream QA model. Experimental results on six QA datasets demonstrate that this method achieves performance comparable to existing sampling-based uncertainty quantification methods, while being significantly more efficient at test time.

Strengths
- The paper tackles the important problem of quantifying uncertainty at the example level (NaAL, gKP3, 575d).
- The paper introduces a new approach by training a smaller neural model to directly evaluate passage utility (NaAL, 575d).
- Experiments are comprehensive, including diverse settings and demonstrating generalization to OOD setups (zmTA, gKP3, 575d).

Weaknesses
- Insufficient discussion / comparison to prior work, including selective question answering, calibration, and reranking in QA, which are virtually the same problem (trrB, gKP3).
- No ablation studies on key components, such as the passage ranking loss and the use of entailment scores, which are key differentiators from previous work (trrB).
- Missing baselines, such as predicting error rates directly for a given question and passage (NaAL) or enabling the QA model to abstain from answering (gKP3).
- Difficult to generalize when multiple passages are jointly needed, e.g., multi-hop QA (NaAL).
- Poor generalization to LLMA3 (trrB).
- Unclear notations, lack of details, and unclear description of the method and experiments (zmTA, gKP3, 575d)