Transform Retrieval for Textual Entailment in RAG (2025)

Abstract: In this paper, we introduce Transform Retrieval, a novel approach aimed at improving Textual Entailment Retrieval within the framework of Retrieval-Augmented Generation (RAG). While RAG has shown promise in enhancing Large Language Models by retrieving relevant documents to extract specific knowledge or mitigate hallucination, current retrieval methods often prioritize relevance without ensuring the retrieved documents semantically support answering the queries. Transform Retrieval addresses this gap by transforming query embeddings to better align with semantic entailment without re-encoding the document corpus. We achieve this by using a transform model and employing a contrastive learning strategy to optimize the alignment between transformed query embeddings and document embeddings for better entailment.We evaluated the framework using BERT as frozen pre-trained encoder and compared it with a fully fine-tuned skyline model. Experimental results show that Transform Retrieval with simple MLP consistently approaches the skyline across multiple datasets, demonstrating the method’s effectiveness. The high performance on HotpotQA highlights its strength in many-to-many retrieval scenarios.

## introduction large language models (llms) have shown significant potential across a spectrum of downstream tasks in nlp, especially in open-domain questionanswering. however, they are prone to generating inaccurate responses due to a lack of knowledge and the hallucination problem. a commonly adopted solution to enhance answer generation is to use retrieval-augmented generation (rag), which integrates the strengths of information retrieval (ir) and llms and has emerged as a prominent technique in artificial intelligence generated * correspondence: quan guo guoquan@gxmzu.edu.cn content (aigc). specifically, rag uses dense retrieval in ir to retrieve relevant documents, forms a prompt with the question, which is then fed into llms, and ultimately generates better and more accurate answers. rag usually retrieves documents by embedding vectors in a vector database with approximate nearest neighbor (ann) algorithms. numerous efforts have been made to improve rag for better supporting llm in conversation @xcite @xcite @xcite . an ideal retrieved document should provide supporting facts for the query, which can be identified by a semantic entailment relationship in natural language inference (nli) @xcite . nli determines whether the given hypothesis document logically follows (entailment), unfollows (contradiction), or is undetermined to (neutral) the premise document. based on this intuition, we define a task called textual entailment retrieval (ter). a common solution is to train a discriminative model to classify the pair of documents into one of the above categories or fine-tune premise and hypothesis embedding for semantic entailment objective @xcite . however, in the rag scenario, due to the large number of documents in the vector database (e.g., all 6m wikipedia documents), such methods always struggle with efficiency. discriminative models are intractable in inference time efficiency because the total inference time grows linearly as the number of vectors in the database. fine-tuning the embedding leads to re-encoding all the documents, which is invasive and can incur significant computational and storage costs, exposing rag to the risk of degeneration of other properties of existing embedding. in this paper, we aim to mitigate the phenomenon of relevance without support in the relevancy search stage of rag. concretely, in the typical rag process, only pre-trained language model embeddings and some similarity metric functions (e.g., cosine similarity) are used, which often leads to the retrieval of documents that are merely semantically related to the query rather than semantically entailed, meaning the retrieved documents do not necessarily provide the supporting facts required to answer the query. motivated by simsiam @xcite architecture in visual encoding, we propose a transform retrieval framework to address this problem under an inference time efficiency concern. as shown in figure 1 , the core idea is to transform query embedding to a semantic entailment embedding relative to its entailed documents. our method transforms the query embeddings, leaving the huge amount of document embeddings in the database unchanged. more importantly, transform retrieval can be built on top of any existing embedding, allowing rag to enjoy the efficiency of ann search. we summarize the contributions as follows: ## preliminary the goal of ter is to retrieve some supported document within the given query in the corpus vector database. moreover, we treat the user's queries as hypotheses and the documents in the corpus as premises. given a query q and documents d then ter is formulated as follows: we proposed a transform embedding framework with a transform model to manage ter as shown in figure 1 . formally, we only transform query embedding without altering the existing document embeddings and use a common similarity metric in the retrieval stage, which is formulated as follows. overall, the rag process within our approach is similar to the fact-checking method (muharram and purwarianti, 2024). however, the latter introduces additional steps after similarity retrieval, which reduces efficiency. ## transform retrieval the overall architecture is shown in figure 2 . we introduce a transform model to transform the query embedding for ter in the original embedding space. the transform model is parameterized and can be trained by contrastive learning. ## experiments we conduct experiments with transform retrieval. we use the selected encoder model with cosine similarity as a baseline and an offline deterministic semantic entailment model, namely simcse, as the skyline. models are evaluated against three datasets. the main result is reported in ## conclusions in this work, we propose a novel approach for textual retrieval named transform retrieval, which enhances performance in semantic entailment retrieval in rag by merely transforming query embedding with transform models trained by contrastive learning. the framework maintains efficient retrieval capabilities and low resource consumption. our experiments demonstrate that our approach is effective and efficient in ter and has a promising use case in real-world rag scenarios.