Eliciting Critical Reasoning in Retrieval-Augmented Generation via Contrastive Explanations (2025)

Abstract: Retrieval-augmented generation (RAG) have emerged as a critical mechanism in contemporary NLP to support Large Language Models (LLMs) in systematically accessing richer factual context. However, the integration of RAG mechanisms bring its inherent challenges, as LLMs need to integrate potentially noisy contexts. Recent studies have shown that LLMs still struggle to critically analyse RAG-based in-context information, a limitation that may lead to incorrect inferences and hallucinations. In this paper, we investigate how to elicit critical arguments in RAG via contrastive explanations. In particular, we propose Contrastive-RAG (CRAG), a framework that (i) retrieves relevant documents given a query,(ii) selects and exemplifies relevant passages, and (iii) generates explanations that explicitly contrast the relevance of the passages to (iv) support the final answer. We show the impact of C-RAG building contrastive reasoning demonstrations from LLMs to instruct smaller models for retrieval-augmented tasks. Extensive experiments demonstrate that CRAG improves state-of-the-art RAG models while (a) requiring significantly fewer prompts and demonstrations and (b) being robust to perturbations in the retrieved documents.

## introduction retrieval-augmented generation (rag) aims to improve the factuality and memory access of large language models (llms) by systematically integrating relevant knowledge from external sources via retrieval mechanisms @xcite . in particular, rag is designed to mitigate some of the well-known limitations of llms, including the tendency for hallucinations and the lack of specific domain knowledge in the training data @xcite @xcite . the content [2] is more accurate in answering the question because it points to david hume as the originator of the principle of uniformity, directly answering the question of who originated the principle. in contrast, paper @xcite provides background on the principle but does not attribute it to any individual, thus not directly answering the question. david hume thought up nature's uniformity principle. teacher model (e.g., gpt-4) retrieval model (e.g., dpr) student model (e.g., llama) ## contrastive explanations contrastive explanations have been identified as fundamental modes of explanation in epistemology, artificial intelligence, and cognitive science @xcite @xcite . a contrastive explanation is a type of inference aimed at answering why-questions of the form "why p rather than q?", where p is the explanandum -i.e., the fact to be explained -and q is a counterfactual event -i.e., the foil. the function of a contrastive explanation is to describe what makes the difference between the occurrence of the fact and the foil. in this paper, we aim to leverage the notion of contrastive explanation to elicit critical reasoning in rag. our intuition is that contrastive explanations are a particularly fitting systematic reasoning mechanism for determining the relevance of documents and passages provided as a context within the rag pipeline. formally, given a query q and a set of documents @xmath0, d 2 , . . . , d n }, we want to partition d into a set of relevant documents p @xmath1, and a set of irrelevant documents, q @xmath2. in particular, we want these sets to be constructed through the generation of a natural language explanation e that describes the factors that contribute to the relevance of p q and the factors that contribute to the irrelevance of q q . in the context of rag, therefore, e aims to answer the question "why is p q relevant to q rather than q q ?", where the set of relevant documents p q corresponds to the fact, and the set of irrelevant documents q q corresponds to the foil. ## contrastive-rag to elicit contrastive explanations in rag, we present a framework composed of four inference stages (figure 2 ): i) collecting step ( §3.1), where, given a query, the retrieved documents are collected and an llm-based model extracts relevant passages; ii) contrastive reasoning ( §3.2), where an llm-based model generates critical and contrastive explanations about the relevance of the extracted passages, by exemplifying and contrasting relevant and irrelevant aspects; iii) explanation ( §3.3), where the arguments constructed in ii) are summarised into a single explanation; iv) answering ( §3.3), where a final short answer to the query is generated. the prompt adopted for c-rag is illustrated in ## experiments we evaluate c-rag on four open-domain question-answering tasks ( §4.1). we perform the retrieval and evaluation phases by following standard approaches used to assess the rag pipeline ( §4.2) and perform the tuning phase by using the setup presented in §4.3. ## results & discussions the results of the empirical evaluation are reported in ## related work previous research investigated the advantages of augmenting large language models (llms) through retrieved text passages, a technique known as retrieval-augmented generative (rag) @xcite . however, recent work showed that the benefits of rag can be undermined by noisy retrieval, thus decreasing consistency and reliability @xcite @xcite . hence, several works proposed techniques to improve rag reliability by enabling the models to elicit the critical steps to arrive at the final answer @xcite . similarly, recent work focused on improving the retrieval phase by fine-tuning llms to perform dynamic retrieval vi adaptive reasoning strategies @xcite ; @xcite ; @xcite ; @xcite . nevertheless, the use of multi-step reasoning strategies usually comes at the cost of increasing the computational resources in terms of number of prompts and training examples @xcite . for instance, @xcite instructed models to retrieve information using special reflection tokens. however, this solution requires the training of two external models, requiring tens of thousands of additional training samples. @xcite attempted to lower the computational cost for multistep rag pipelines, but their approach still requires additional models to summarise the retrieved documents. finally, @xcite eliminated the dependence on special tokens and external components by introducing reasoning trajectories that are employed to boost the performance of llms directly. although the solution improves efficiency, the framework operates through a multi-step mechanism that requires multiple annotation phases. similarly to recent work investigating the impact of natural language explanations on llms @xcite @xcite @xcite , we propose a method to integrate via multi-step explanations into rag. to the best of our knowledge, however, this is the first work to investigate the impact of contrastive explanations on rag and demonstrate how contrastive reasoning demonstrations can boost the performance of smaller lms. ## conclusion & future work rag has shown great potential in boosting the performance of llms on knowledge-intensive tasks. despite the success of rag, noisy retrieval represents a major limitation. to tackle such challenges, we introduce contrastive-rag (c-rag), a framework that (i) retrieves relevant documents given a query, (ii) selects and exemplifies relevant passages, and (iii) generates explanations that explicitly contrast the relevance of the passages to (iv) support the final answer. we demonstrate that c-rag can outperform state-of-the-art models while requiring fewer prompts and demonstrations and being robust to perturbations in the retrieved documents, laying the foundation for integrating natural language explanations within rag-based architectures and improving efficiency, consistency and reliability. in future developments, we plan to extend our contribution to non-english languages to broaden the beneficial impacts and operability of c-rag for multi-lingual alignment @xcite and augmentation approaches @xcite . another possible extension for c-rag is to investigate the integration of contrastive explanations with different types of explanatory arguments studied in philosophy of science @xcite @xcite . a potential direction, in fact, would be to explore how different types of explanations can be leveraged to further improve reasoning in llms and rag and develop more efficient instruction tuning techniques to improve smaller models using a lower amount of data.