Title: Reliable Decision‑Making via Calibration‑Oriented Retrieval‑Augmented Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Calibration

Uncertainty

LLMs

RAG
Abstract: Recently, Large Language Models (LLMs) have been increasingly used to support various decision-making tasks, assisting humans in making informed decisions. However, when LLMs confidently provide incorrect information, it can lead humans to make suboptimal decisions. To prevent LLMs from generating incorrect information on topics they are unsure of and to improve the accuracy of generated content, prior works have proposed Retrieval Augmented Generation (RAG), where external documents are referenced to generate responses. However, previous RAG methods focus only on retrieving documents most relevant to the input query, without specifically aiming to ensure that the human user's decisions are well-calibrated. To address this limitation, we propose a novel retrieval method called Calibrated Retrieval-Augmented Generation (CalibRAG), which ensures that decisions informed by RAG are well-calibrated. Then we empirically validate that CalibRAG improves calibration performance as well as accuracy, compared to other baselines across various datasets.


Peer reviews:
comments: SUMMARY:
This paper introduces CalibRAG, a Retrieval-Augmented Generation (RAG) framework designed to produce not only relevant evidence for decision-making but also well-calibrated confidence estimates for those decisions. Traditional RAG approaches often yield overconfident guidance, whereas CalibRAG trains a forecasting function that, given a prompt, a retrieved document, and a model temperature setting, predicts the probability of a correct decision. Empirical results across several question-answering tasks and retrieval back ends demonstrate that CalibRAG improves decision accuracy and calibration metrics relative to standard baselines.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper clearly articulates the issue of overconfident retrieval in RAG systems, using intuitive examples that ground the reader in the practical need for decision calibration.
2. Integrating a temperature-aware forecasting function into the retrieval step is novel and makes sense.
3. The evaluation spans varied question-answering benchmarks and diverse retrieval back ends.

Weakness:
1. Retrieving and reranking a sizable set of documents, with potential query reformulation, may introduce non-trivial latency.
2. The forecasting function is trained on synthetic data derived from a surrogate language model, leaving open questions about how well it mirrors real human decision behavior.

QUESTIONS:
1. Do you foresee applying CalibRAG to other generative tasks such as summarization or code generation?
2. Retrieving and reranking a sizable set of documents and performing query reformulation adds computational cost. Have you analyzed the detailed inference time overhead?

LIMITATIONS:
See weakness above
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-16

comments: SUMMARY:
The paper proposes to add calibration estimation to RAG outputs. CalibRAG incorporates a forecasting function—a surrogate model—that predicts the probability that a user's decision, informed by the retrieved information, will be accurate. They generate synthetic training data to teach the forecasting model to align confidence with actual decision outcomes and integrate it into the retrieval and response pipeline.

STRENGTHS AND WEAKNESSES:
Strengths:
The ideas is simple and introduces a clear extension borrowing from far enough field to this field.  
The writing is clear, and the presentation is appealing.
The papers' scores seem high and the performance convincing.
The paper uses multiple baselines and retrievers to give a more round evaluation and validate the generality of their method.

Weakness: 
The idea sounds very clean, so much so that it is surprising no one did it, despite so much RAG works. A search on my side didn't find something like that though.  
I see the dataset as quite a minor contribution, but it is not a negative to have *also* a minor addition.

Minor:
The people in the figure are pretty, but maybe keeping mostly they faces would make the differences between them more salient?
In the contributions paragraph, references to where each part is written could be useful for people to jump to the right place (e.g., the dataset is not reflected in the section structure).

QUESTIONS:
na

LIMITATIONS:
na
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-20

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes CalibRAG, a method to align the confidence level of Retrieval-Augmented Generation (RAG) systems with actual correctness.  It first employs an LLM-based surrogate user model to create probability labels about what proportion of answers (sampled outputs conditioned on a temperature) with the retrieved documents are correct. Then, it trains a parameterized forecasting model that predicts the probability. In the inference stage, CalibRAG utilizes the forecasting model to rerank the retrieved documents and select the highest-ranked document for final generation. CalibRAG improves both accuracy and calibration performance over baselines in various QA datasets.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. The overall method design is reasonable. Using decoding temperature to reflect user behavioral traits is interesting.\n2. The baselines included in the comparison are representative and comprehensive.\n\n## Weaknesses\n\n1. The evaluation in this paper is mainly about QA tasks, while decision-making (highlighted by this paper) may not only include QA.\n2. All experiments consider using the top-1 document in the final generation, which seems a little bit far from the mainstream RAG pipeline.\n\nQUESTIONS:\n## Questions\n\n1. Is it possible to show the performance of CalibRAG on other types of \"decision-making\" tasks or real-user experiments? I would say **the limited evaluation (on QA tasks) is my main concern** since one can argue that in QA tasks, a RAG model can give answers as well, so it is not a necessity to have a \"user model\".\n2. In the inference stage, you only select the top-1 document for the final generation. However, current RAG models (w/ long-context ability) can **process much more than 1 document, and it may naturally utilize the most useful information**, making the reranking less necessary. Can you provide some results about using more retrieved documents?\n3. L263: For baselines, guidance and confidence are generated based on the top-1 document retrieved. Is it fairer to generate the confidence scores for all $K$ documents and utilize the most confident one?\n4. As you use GPT-4o-mini as the evaluator in both your forecasting model training and final judgment, the forecasting model can somehow overfit (not sure, just guessing). Could you use another evaluator in the final judgment to show the robustness of CalibRAG if possible?\n5. What is the reason for not taking guidance $z$ as one of the input params of the forecasting model? Is it because of efficiency? As your ultimate goal is to predict **how $z$ impacts decisions**, using $q$ and $d$ as substitutes seems suboptimal, as $z$ also incorporates the inherent knowledge of RAG models.\n6. Why do you use Llama-2-7B in Table 1b, not aligned with the main setup (Llama-3.1-8B)?\n7. L342: \"gains diminish beyond 40 documents\". However, it seems that using 40 documents yields larger gains compared to 20. Can you clarify this statement?\n8. What model is used for the query reformulation? What are the maximum reformulation times when the threshold cannot be satisfied?\n\n## Suggestions\n\nTypos in writing:\n\n1. L224: $t$.\n2. L250: Change \"decision model\" to \"user model\".\n3. Calibrag (should be CalibRAG) in Figure 5c.\n\n[1] may be a missing related work (or baseline).\n\n[1] Zhao, Xinran, et al. \"Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models.\" Findings of ACL 2024.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-21"}

comments: SUMMARY:
This paper studies to calibrate the decision-making process for retrieval augmented generation (RAG). The paper trains a forecasting function which takes a query-document pair and a temperature parameter representing user behavior. The model is trained on data generated by the LLM with different temperature setup. Experiments show that, not only the trained model can better calibrate the results, but it can also be used for context selection for RAG.

STRENGTHS AND WEAKNESSES:
### Strengths
- The calibration problem for RAG is interesting and has its implications in real applications. It is interesting to see how confident/uncertain the LLM is based on the provided context and whether the response will be helpful.
- The proposed method is also sound and innovating. The usage of temperature to simulate user behavior and help data generation and training seems to be a promising approach.
- Some of the tricks and observations made by this work could motivate more research in similar directions.
- The paper is clearly written. I like the example in figure 2 which clearly demonstrates the task and motivation.

### Weaknesses
Overall I think it is a nice work and I didn't notice any overwhelming weakness.
- It would be beneficial to include some related works on RAG research in the background discussion.
- Some more discussions on limitations and future works could be beneficial for this paper as well as future researchers.

QUESTIONS:
- I wonder what the stop criterion is for the inference iterations? Or if step 3 will only be executed once, then what if there is still no quality retrieved document?
- Any intuitions on using Fourier positional encoding for temperature parameter? Are there other potential choices?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01