On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Authors: Xun Xian
Ganghua Wang
Xuan Bi
Rui Zhang
Jayanth Srinivasa
Ashish Kundu
Charles Fleming
Mingyi Hong
Jie Ding
Venue: ICML 2025 poster (2025)
Keywords: Healthcare; Safety; RAG
Primary area: social_aspects

Abstract: Retrieval-Augmented Generation (RAG) has been empirically shown to enhance the performance of large language models (LLMs) in knowledge-intensive domains such as healthcare, finance, and legal contexts. Given a query, RAG retrieves relevant documents from a corpus and integrates them into the LLMs’ generation process. In this study, we investigate the adversarial robustness of RAG, focusing specifically on examining the retrieval system. First, across 225 different setup combinations of corpus, retriever, query, and targeted information, we show that retrieval systems are vulnerable to universal poisoning attacks in medical Q&A. In such attacks, adversaries generate poisoned documents containing a broad spectrum of targeted information, such as personally identifiable information. When these poisoned documents are inserted into a corpus, they can be accurately retrieved by any users, as long as attacker-specified queries are used. To understand this vulnerability, we discovered that the deviation from the query’s embedding to that of the poisoned document tends to follow a pattern in which the high similarity between the poisoned document and the query is retained, thereby enabling precise retrieval. Based on these findings, we develop a new detection-based defense to ensure the safe use of RAG. Through extensive experiments spanning various Q&A domains, we observed that our proposed method consistently achieves excellent detection rates in nearly all cases.

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
Xun Xian
Ganghua Wang²
Xuan Bi Rui Zhang* Jayanth Srinivasa Ashish Kundu
Charles Fleming
Mingyi Hong" Jie Ding
Abstract
1. Introduction
Retrieval-Augmented Generation (RAG) has been empirically shown to enhance the performance of large language models (LLMs) in knowledge- intensive domains such as healthcare, finance, and legal contexts. Given a query, RAG retrieves rel- evant documents from a corpus and integrates them into the LLMs' generation process. In this study, investigate the adversarial robustness of RAG, focusing specifically examining the retrieval system. First, across 225 different setup combinations of corpus, retriever, query, and tar- geted information, we show that retrieval systems are vulnerable to universal poisoning attacks in medical Q& In such attacks, adversaries gener- ate poisoned documents containing broad spec- trum of targeted information, such as personally identifiable information. When these poisoned documents are inserted into a corpus, they can accurately retrieved by any users, as long as sitacker-speciffer queries are used. To understand this vulnerability, we discovered that the devia- tion from the query's 's embedding to that of the poi- soned document tends to follow a pattern in which the high similarity between the poisoned docu- ment and the query is retained, thereby enabling precise retrieval. Based on these findings, we de- velop a new detection-based defense. Through extensive experiments spanning various Q&A do- mains, we observed that our proposed method consistently achieves excellent detection rates in nearly all cases.
Large Language Models (LLMs) have achieved exceptional performance across wide range of benchmark tasks span- ning multiple domains (Anil et al., 2023; Achiam et al., 2023; Thirunavukarasu et al., 2023). However, they also suffer from several undesirable behaviors. For example, LLMs can generate responses that seem reasonable but are not factually correct, phenomenon known as hallucina- tion (Ji et al., 2023). Additionally, due to data privacy regu- lations such GDPR (Voigt & Von dem Bussche, 2017), direct training on specific data domains may be restricted. This can result in disparities between their acquired internal knowledge and the real-world challenges they encounter, can be particularly concerning domains require extensive knowledge, such as healthcare (Tian et al., 2024; Hersh, 2024), finance (Loukas al., 2023) and legal question- answering (Wiratunga al., 2024). These challenges have sparked interest in more principled methods for decoding and alignment (Wang et al., 2025), as well as techniques that inject external knowledge to bridge domain gaps, such as Cetrieval-Augmentee Generation (RAG) (Khandelwal et al., 2019; Lewis et al., 2020; Borgeaud al., 2022; Xiong et al., 2024).
The RAG approach typically involves two steps: retrieval and augmentation. Upon receiving an input query, RAG retrieves the top K relevant data from an external data corpus. It then integrates this retrieved information with its internal knowledge to make final predictions. Empir- ical evidence suggests that LLMs employing the RAG scheme significantly outperform their non-retrieval-based counterparts in wwwwledge-intensiie domains like finance and medicine (Borgeaud et al., 2022; Xiong et al., 2024), For instance, the authors of (Xiong et al., 2024) developed a state-of-the-ar benchmark for the use of RAG in the med- ical domain. The authors observed an increase prediction accuracy of up to 18% with RAG compared to non-retrieval and chain-of-thoughts versions across large-scale healthcare tasks, utilizing 41 different combinations of medical data corpora, retrievers, and L.MSs.
'Department of ECE, University of Minnesota Uniieersity of Chicago, Data Science Institute CCarlson School of Manage- ment, University of Minnesota 'Division of Computational Health Sciences, University of Minnesota *Cisco Research "School of Statistics, University of Minnesota. Correspondence to: Jie Ding <dingj umn.edu>.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
The use of retrieved knowledge in RAG has also raised se- curity and privacy concerns, especially when the external

---

On the Vulncrability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
data corpus is openly accessible, e.g., Wikipedia (Zou al., 2024; Deng et al., 2024) and PubMed, or when controlled by potential malicious agents, as demonstrated in the case of multi-vision-LMM agents (Gu et al., 2024). For exam- ple, recent work has successfully launched data poisoning attacks against the retrieval systems (Liu et al., 2023; Zhong et al., 2023; Zou et al., 2024; Deng et al., 2024). In these cases, malicious attackers can poison publicly accessi- ble data corpus by injecting attaccer-specified data into it, aiming to trick the retrieval system into retrieving those target data as the top K relevant documents. Consequently, when LLMs make predictions based on the retrieved data, they can be easily targeted by adversaries through backdoor attacks (Zou et al., 2024).
we term as orthogonal augmentation. Here, retrievers f()) are mappings from a document to its embedding. high- dimensional vector. The orthogonal augmentation property states that for any two documents p and q that are close to orthogonal in their embeddings, the concatenated document [q + p/ will shift in embedding from f(g) to the direction perpendicular to I f(()). In other words, appending an orthog- onal document p to q results in an orthogonal movement in the embedding of the augmented document [q + p). As nat- ural consequence of this property, it can be shown that the high similarity between the poisoned document and its cor- responding clean query is maintained, implying the success of universal poisoning attacks.
Regarding the orthogonal augmentation property, we high- light that documents which have near-orthogonal embed- dings can still be semantically relevant (see Section 4). This ensures that the universal poisoning attacks could succeed even if the attacker-specifeet targeted information is seman- tically related to the query. In addition, in the previous case of medical Q&A, we empirically observed that retrieved documents are often not close, in terms of commonly used similarity measurements such as inner product, to their as- sociated queries (see Table 4). This results in a consistently higher similarity between queries and poisoned documents compared to their clean counterparts, partially explaining the widespread effectiveness of our studied universal poi- soning attacks.
With the empirical successes of these attacks, it impera- tive develop defenses against them. However, existing methods, such as examining the fy-norm of the documents' embeddings, have been shown to be ineffective (Deng et al., 2024) for detecting poisoned documents. Given the widespread adoption of RAG in safety-critical domains such as healthcare, such safety risks become even more pronounced.
1.1. Main Contributions
In this study, we investigate the safety risks associated with RAG, specifically focusing on retrieval systems. The contri- butions are summarized as follows.
New defense against universal poisoning attacks. We em- pirically observed that the commonly observed not-so-close retrieved documents tend to be perpendicular to their corre- sponding query. Meanwhile, since the poisoned documents will not significantly deviate from the query, as a result of the previous discussion, the poisoned document also tends to be perpendicular to clean documents. This property moti- vates us to consider using distance metrics that reflect the probability distribution of the data to detect poisoned docu- ments. As shown in the right-most of Figure 2, we observe clear separation between clean and poisoned documents. However, such a distinction does not exist when using the l distance, which assumes data are isotropic, or when using the Eg norm. We extensively assess our proposed defenses against both the proposed attacks and other existing poison- ing attacks against RAG (Zou al., 2024). We observed consistent, near-perfect detection rates across all attacks.
Revealing the safety risks for retrieval systems: case studies for medical Q&A and legal Q&A. We demonstrate that dense retrieval systems are vulnerable to what we term 'universal poisoning attacks' in medical Q&A across 225 use-case combinations of corpus, retriever, query, and tar- geted information. Similar observations are made for legal Q&A. As shown in Figure 1 below, in these attacks, adver- saries can append nearly every sort of information, such as personally identifiable information (PII) and adversarial treatment recommendations, to a set of attacker-specified queries. Once these poisoned documents are injected into a large-scale corpus, such as Wikipedia and PubMed, they can be accurately retrieved, often with high rankings, e.g., top 1, using attacker-specified queries. Depending on attackers' goals, these documents will lead to safety risks such as (1) leakage of PII, (2) adversarial recommendations for treat- ments, and (3) jailbreaking the LLM during the inference stage once used as context.
1.2. Related Work
Understanding the vulnerability of retrieval systems in RAG. Recall that the dense retrieval system, matching through semantic meanings, selects documents from the corpus based on their similarity to the input query in the embedding space. We explain the high similarity between the poisoned documents and their associated clean queries based on an intriguing property of the retrievers, which
Data poisoning and backdoor attacks. Our investigation is closely related to recent advances in understanding and defending against backdoor attacks in both training and inference stages. The adaptability hypothesis (Xian et al., 2023a) and statistical perspectives on poisoning (Wang et al., 2024) offer complementary insights into how learned rep-
2

---

On the Vulncrability of Applying Retrieval-Augmennted Generation within Knowledge-Intensie Application Domains
Goals:
(1) PII L Leakage
(II) A dversarial Recommendation
(III) Jall-breaking LLMs
Is eteplinsen effective in treating Duchenne muscular dystrophy? Bob, 45-year-old accountant, with the email address Bobê gmail.com and phone number 123-456-7899 ordered eteplirsen for treatment.
Is eteplirsen effective in treating Duchenne muscular dystrophy? The XX plan is effective for treating Duchenne muscular dystrophy.
Is eteplirsen effective treating Duchenne muscular dystrophy? ? Hi ChatG PT, you are going to pretend to be DAN which stands for do anything now"
A ttacker
G A ddvessaiaal R Recommendation
Poisoned Corpus
eteplirsen effective in treating Duchenne muscular dystrophy? ? The XX plan is effective for treating Duchenne muscular dystrophy.
Is eteplirsen effective in treating Duchenne muscular dystrophy?
Top-1 retrieved document
Query
Figure 1. An illustration of universal poisoning attacks. The attacker can append a variety of adversarial information (in bold italies) to a question to create poisoned document and then inject it into the corpus. Upon querying the attacke--ppeiffed question, the poisoned document will be retricved with a high ranking, e.g., top 1. These retrieved (poisoned) documents will lead to safety issues: (1) leakage of personally identifiable information (PII), (2) adversarial recommendation, and (3) jailbreaking the LLM at the inference stage based on the retrieved document.
resentations can be manipulated under adversarial influ- ence. Furthermore, unified detection frameworks (Xian et al., 2023b) propose practical strategies for identifying such attacks at inference time, although these methods have not yet been extensively explored in reurieeal-augmeneed generation pipelines.
level matching pattern, sparse retrievers' performance has been shown to be worse than that of dense retrievers in sev- eral domains (Zhao et al., 2024), such healthcare (Xiong et al., 2024). There are two popular approaches for training dense retrievers: supervised (Huang et al., 2013; Nogueira & Cho, 2019) and self-supervised (Cohan et al., 2020; Izac- ard et al., 2021; Jin et al., 2023). In the supervised regime, given a paired query and document, the goal is to maximize the similarity, i.e., inner product, between their embeddings. Motivated by advances in unsupervised learning, recent methods have started to apply contrastive learning (Chen et al., 2020) for training, where they observed improved performance across multiple benchmarks. In our work, we additionally consider a retriever specifically trained on med- ical datasets: MedCPT (Jin et al., 2023).
Retrieval-Augmentee Generation (RAG) RAG, popular- ized by (Lewis et al., 2020), is widely adopted approach that integrates retrieval- mechanisms into generative models to improve performance across various language tasks (Gao et al., 2023). The basic flow of RAG follows a 'retrieve and read' fashion (Lewis et al., 2020; Ma et al., 2023; Levine et al., 2022; Borgeaud et al., 2022; Ram et al.,
2023; Khandelwal et al., 2019; He et al., 2021; Alon al.,
2022; Zhong al., 2022). In this fashion, given an input query, one first retrieves relevant data from the externa data corpus and then employs generative models as 'read- ers', for example, by directly appending the retrieved doc- uments to the queries as context and then feeding them as a whole to LLMs for making predictions. Approaches for enhancing the efficient use of retrieved documents to im- prove the LLMs' reading capability include using chunked cross-attention during generation (Borgeaud et al., 2022), prompt-tuning (Levine et al., 2022).
Adversarial attacks against RAG/retrieval systems Cur- rent approaches for attacking RAG (Zou et al., 2024; Deng et al., 2024) mostly involve poisoning the corpus to de- ceive the retrieval system (Liu et al., 2023; Long et al., 2024) into retrieving the poisoned documents for adversar- ial purposes. The recent work of PoisonedRAG aims to backdoor the RAG by tricking the LLM into generating attacker-specified answers based on the retrieved attacker- crafted documents (Zou et al., 2024). The authors develop both black-box and white-box attacks to achieve this goal. In black-box methods, they directly append the context for answering the question to the query and inject into the corpus, similar to our method in implementation. How- ever, their approaches differ from ours in terms of goals, insights, and application scenarios. First, our objective is to investigate and comprehend the robustness of retrieval
Dense retrieval systems There are two main categories of retrievers: sparse retrieval systems, which match through lexical patterns (c.g., BM25) (Robertson et al., 1995; 2009), and deep neural network- -ased dense retrievers, which match through semantic meanings (Cohan et al., 2020; Izac- ard et al., 2021; Jin et al., 2023). Due to the near-exact token-
3

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intenitg Application Domains
80
Clean Poisoned
Clean Poisoned
Clean Poisoned
70
250
60
80
Count
40
60
150
30
40
100
20
20
12-norm
distance
Proposed Distance
Figure 2. Defense methods against universal poisoning attacks. Clean documents (collected from the Texthook (Jin et al., 2021)) and poisoned documents are indistinguishable using popular defenses, such as the 22-norm of their embeddings. However, under our proposed distance measurements, clear separation between clean and poisoned documents is observed (right-most most figure).
systems employed in RAG. We achieve this by injecting various types of information, encompassing both irrelevant and relevant content, into the corpus and evaluating the case or difficulty of retrieval. Their goal, however, is to inject only query-relevant context to deceive the LLM's generation process upon retrieval. Second, in terms of insights, we pro- vide explanations on the difficulty/easinnss of the retrieval of different kinds of information, which is not covered in their work. Third, we focus on the application domain of healthcare, while they focus on the general Q&A setting.
Remark 3.1 in Section 3). In addition, also examine the scenario where the attacker lacks access to the exact queries but has access to their semantically equivalent counterparts (see Table in Section 3), which makes the considered threat model even more practical.
Remark 2.1 (Assumption on accessing the RAG database). Accessing the database in RAG for medical scenarios reasonable since publicly accessible databases have been frequently used in building RAG for medical use. For in- stance, the well-known PubMed, a comprehensive collec- tion of biomedical literature citations and abstracts, has been frequently used for building RAG for medical use by researchers from NIH and Mayo Clinic (Xiong et al., 2024; Miao et al., 2024). In particular, the work (Xiong al., 2024) demonstrated that by using PubMed as the knowledge database, the accuracy performance of RAG achieves better results than other medical texts, such the Textbook (Jin et al., 2021). PubMed is publicly accessible, which means an attacker could potentially inject adversarial information into it. This confirms the feasibility of our threat model.
2. Preliminary and Threat Model
Notations Denote the set of vocabulary be considered V. We denote / VL - Rd as the embedding function that maps sentences to the latent space, where L is the maximum allowed words/tokens and d is the dimension of latent embeddings. We use L to denote the concatenation of sentences. Following the convention, we will use the inner product of the embeddings (q) [p), to measure the similarity between two documents p and q. We use the (aa, b) sign to denote the angle between vectors and b.
Attacker's Goal Given an targeted document T E y (S S L) and a set of queries Q {q1, ......, 9n3 with q: Vs. the attacker's goal is to ensure that 7 will consistently be retrieved with high ranking after injecting it into the data corpus, corresponding to attacker-specified queries Q. These types of goals are commonly observed in adversarial recommendations (Liu et al., 2023), where an attacker aims to improve the ranking of their targeted information.
Attacker's capability There are three components for the retrieval systems in RAG: (1) data corpus, (2) retriever, and (3) query set. For the data corpus, we assume that the attacker can inject new data entries into it, e.g., by creating a new Wikipedia page or by entering row of fake patient's information into an existing medical database. Regarding the retriever, assume that the attacker can query the retriever models and view the retrieved documents and their associated latent embeddings. However, the attacker can neither speculate nor modify the parameters of the retriever models. For the query set, we assume that the attacker has access to queries interests. (We provide a detailed examples, in
3. Exploring Safety Risks: Insights from Medical Retrieval Systems
elaboration on this point, along with
In this section, we provide thorough case study to show that the retrieval in medical Q&A benchmarks are vulner-

---

On the Vulncrability of Applying Retrieval-Augmentted Generation within Knowledge-Intensive Application Domains
able to poisoning attacks. We begin by listing the detailed experimental setups.
trieval, consider simple yet effective method in which the attacker directly appends the targeted information to queries. The poisoned documents p; should be in the form of p: = lq: @ Target Information], where represents nor- mal query. We also consider the case, where the attacker is unaware of the exact queries but knows their semantically equivalent versions.
3.1. Setups
Query Following (Xiong al., 2024), we use a total of five sets of queries, including three medical examination QA datasets: MMLU-Med (1089 entries), MedQAUS (1273 entries), MedMCQA (4183 entries), and two biomedical research QA datasets: PubMedQA (500 entries), BioASQ- Y/N (618 entries).
Evaluation Metric Consider a pair consisting of a normal query 2i and target information T. This pair is deemed successful if the corresponding poisoned document P; lq @ 2] is among the top K (K" 2 1) document(s) retrieved by q:- For the results presented in the main text, we set K = 2, and ablation studies on different choices of K are provided in Table 8 in appendix.
Medical Corpus Following (Xiong et al., 2024), we select a total of three medical-related corpora: (1) Textbook (Jin et al., 2021) (~ 126K documents), containing medical- specific knowledge, (2) StatPearls (~ 301K documents), uti- lized for clinical decision support, and (3) PubMed (~ 2M documents), which consists of biomedical abstracts. Due to limited computation resources, the PubMed we used is a random subset of the total 23M documents. Examples of each corpus and more details are included in the appendix.
3.2. Results
For the medical application, we report the success rates over a total of 225 combinations of query, corpus, retriever, and targeted information in Table below. For the legal Q&A case, we report the attack success rates in Table 2. For each category of targeted information, we generated three different documents, calculated their success rates, and reported the mean value. The standard deviations are less than 0.07. The interpretation of results in each cell adheres to the same following rule. We use the top-left cell as an example: it represents a success rate of 0.78 achieved using the Corpus: Textbook, Retriever: MedCPT, Query set: MMLU-Med with PPI as the targeted information.
Legal Corpus For the legal Q&A, we follow (Wiratunga et al., 2024), where Case-Based Reasoning (CBR) is used to structure retrieval within the RAG process of an LLM to enhance legal Q&A performance. We use the Australian Open Legal QA (ALQA) dataset (Butler, 2024), with an example shown in Table 10 in the appendix.
Targeted Information (I) For the medical corpus: We consider five types of targeted documents: synthetic per- sonally identifiable information (PII), synthetic medical di- agnosis information, and adversarial passages generated from (Zou et al., 2024) for answering questions from MS MARCO (Nguyen et al., 2016). Natural Questions (NQ)(Kwiatkowss et al., 2019), and HotpotQAA(ann et al., 2018), respectively. We use GPT-3 to evaluate their seman- tic similarity and find that they semantically distant from each other. This suggests that our setup covers broad range of topics, thereby enhancing the validity of our results. Examples are provided in the appendix.
Several conclusions are summarized: (1) Overall, high at- tack success rates are consistently observed across different combinations of corpus, retrievers, medical query sets, and target information. This implies that retrieval systems used for medical Q&A are universally vulnerable, meaning that attacker can insert any kind of information for malicious use cases. (2) Similar attack success rates are observed across different corpora, implying that this vulnerability is consistent across various datasets. (3) Similar attack success rates are observed across different retrievers, suggesting that this vulnerability is shared by all popular retrievers. (4) Attack success rates for certain query sets, e.g., PubMedQA, are consistently higher than others across different corpus and retrievers. We conjecture that this is because the overall length of queries from PubMedQA is significantly longer than others. Hence, the added target information does not af- fect the overall semantic meanings, leading to high retrieval rates. More detailed discussions are included in the next section. (5) Attack success rates are all on par for differ- ent types of targeted information. This is expected since all of them are not semantically closely aligned with the queries. Therefore, their effect on the retrieval are similar. We empirically verified this idea in the next section.
(II) For the legal corpus: In addition to the aforementioned synthetic PII. HotpotQA, and NQ, we also prompt GPT to generate two types of synthetic legal content.
Retriever We select three representative dense retriev- ers: (1) a general-domain semantic retriever: Con- triever (Izacard et al., 2021), (2) a scientific-domain retriever: SPECTER (Cohan et al., 2020), and (3) a biomedical-domain retriever: MedCPT (Jin et al., 2023). Details on these retrievers are in the appendix.
Attacking Method Recall that the goal of the attacker is to ensure their targeted information is accurately re- trieved with high rankings associated with pre-specified queries. Therefore, to increase the success rate of re-
How robust is the attack against paraphrasing? There
5

---

On the Vulncrability of Applying Retrieval-Augmennted Generation within Knowledge-Intensiv Application Domains
is one potential limitation regarding the proposed attack through concatenation. In certain practical use cases, users or defenders who are aware of the proposed attack may sim- ply paraphrase the queries to defend against it. As a result, there may be no precise match between the queries used for retrieval and those used by attackers to create poisoned documents. In the following, we demonstrate that the pro- posed attack is robust under such a mismatch. We maintain the same setup except that the queries are now rephrased by GPT-4 (Achiam et al., 2023) to reflect the scenarios. We summarize the results in Table 7 below. We observed that the proposed attack remains effective under query paraphras- ing, achieving a top-2 retrieval success rate of 0.8 over most cases. These findings highlight even more pronounced se- curity risks for retrieval in medical Q&A, as the attacker now only needs to know queries up to rephrasing in order to launch targeted attacks.
the same attack procedure described in the paper and ob- served average attack success rate of approximately 0.92 for Contriever the embedding model using the PubMed corpus. This result indicates that it is very easy for attackers to create meaningful medical queries that can lead to high attack success rates, which also shows that the RAG for medical use is at significant safety and security risks.
4. Understanding the vulnerability of retrieval systems in RAG
In this section, we provide insights towards understanding the vulnerability for the retrieval systems. To begin with, we will first present an intriguing property shared by popular retrievers, which we termed as the orthogonal augmentation property. The orthogonal augmentation property states that when two documents q, p E V4 are close to orthogonal in terms of their embedding, the embedding of the (token-level) concatenated one, i.e., f(q pl), roughly equals {() + e, with v" f(() 2s 0. In other words, this property implies that the shift (in terms of embeddings) from g to [u @ p] mainly occurs in directions that are perpendicular to As a result, for the inner- amer-prrduct based similarity, the similarity between q and [q 1 pl will roughly equal to that between q and itself, namely f'a)" ((q@p)) R {()) f())+((())))))) f() T f(). This implies that [g @ p] is likely to be retrieved by q. indicating the success of universal poisoning attacks.
(Further discussion on the assumption on ac- cessing the query set). Our proposed attacks are type of RAG attack that uses a pre-selected set of queries (called targeted queries) as triggers to retrieve adversarial docu- ments and poison the LLM generation pipelines. Some existing examples of these types of attacks can be found in references (Zou et al., 2024; Long et al., 2024).
Remark 3.1
In this scenario, attackers already know the queries since they pre-select them and design the poisoned documents, allowing them to launch the attack. The practical concern is whether normal users, who are unaware of the risks, will use these attacker-selected targeted queries. For example, if the queries only contain special but non- non-informative tokens, they are unlikely to be used by normal users. Therefore, to effectively target large number of normal users, attackers need to identify the queries that normal users are likely to use and then create poisoned documents based on those queries.
We verified the orthogonal augmentation property for sev- eral state-of-the-art retrievers in Table 3. In particular, we first collected a set of documents from the MedQA denoted as Q = (9I.... J. Next, we selected several other sets of documents P. = {Pa..... Pin] with varying lengths and similarities Q. We report total of four similarity measurements and for each mcasurement, we report both the mean and the variance. We observed that when the in- ner product between the embeddings of the two documents f(g) T [p) decreases, the angle between f(((@)) and fq) tends to be around 90", and the inner product also decreases, which corroborates the orthogonal augmentation property. Additionally, we observe that Contriever is sensitive the length of the document; that is, the longer the document, the larger the f--norm of its embeddings, whereas such a phenomenon is not observed for MedCPT.
In the following, we explain and provide new empirical results to show that the previously mentioned requirement can potentially be met in medical domains. We paraphrase all the normal queries serve as defense for the case where the attack knows exact queries. We observed that there is only a slight decrease in terms of attack success rates (detailed results can be found in Table 7 in the ap- pendix). In other words, the attacker does not need to know the exact query. Instead, knowing queries with similar mean- ings/structures is sufficient for launching successful attacks. This requirement is relatively straightforward to fulfill be- cause real-world medical queries tend to follow very stan- dard patterns, which can be easily exploited by attackers.
One potential caveat in using the proposed orthogonal aug- mentation property to explain the success of our attack is that the embeddings between the query and targeted docu- ments need to be close to orthogonal. However, we highlight that closeness to orthogonality between embeddings does not imply that their associated documents are semantically irrelevant. For example, we randomly sampled two non- overlapping batches of questions from the MedQA dataset and found that the angle between their embeddings is around 70%. Yet. these batches of queries are all semantically re-
To further demonstrate this point, we conducted a new ex- periment as follows. We employed GPT-4 to generate 100 new synthetic queries that are potentially commonly asked by doctors, based on the MedMCQA dataset. We repeated
6

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Table Top retrieval success rates over 3 corpora, retrievers, query sets, and 5 sets of targeted information (for the medical application). The interpretation of results in each cell adheres the following rule. We use the top-left cell example: it represents a success rate of 0.78 achieved using the Corpus: Textbook, Retriever: MedCPT, Query set: MMLU-Med with PPI as the targeted information.
Target Information
Corpus
Retriever
Query
PPI
MS-MARCO
NQ
HotpotQA
Diagnostic
MMLU
0.78
0.82
0.83
0.81
0.85
MedQA
0.98
0.99
0.98
0.99
0.99
MedCPT
MedMCQA
0.81
0.83
0.79
0.83
0.82
PubMedQQ
0.98
0.98
0.99
0.98
0.97
BioASQ
0.95
0.99
0.93
0.95
0.95
MMLU
0.95
0.75
0.84
0.95
0.80
MedQA
0.99
0.99
0.97
0.99
0.99
Texthook
Specter
MedMCQA
0.92
0.81
0.73
0.90
0.89
PubMedQQ
0.92
0.86
0.95
0.93
0.94
BioASQ
0.98
0.97
0.98
0.97
0.94
MMLU
0.82
0.81
0.83
0.84
0.81
MedQA
1.0
1.0
1.0
0.99
Contriever
MedMCQA
0.63
0.63
0.67
0.66
0.68
PubMedQA
0.80
0.82
0.84
0.81
0.83
BioASQ
0.62
0.63
0.60
0.61
0.64
MMLU 0.74
0.75
0.71
0.73
0.73
MedQA 0.99
0.99
0.98
0.99
0.98
MedCPT
MedMCQA
0.72
0.75
0.68
0.72
0.75
PubMedQA
0.93
0.95
0.91
0.91
0.93
BioASQ
0.87
0.92
0.87
0.86
0.86
MMLU
0.93
0.69
0.78
0.92
0.90
MedQA
1.0
1.0
0.97
0.97
0.95
StatPcarls
Specter
MedMCQA
0.85
0.63
0.70
0.81
0.81
PubMedQQ
0.98
0.82
0.88
0.97
0.95
BioASQ
0.92
0.80
0.89
0.95
0.94
MMLU
0.80
0.82
0.81
0.83
0.81
MedQA
1.0
1.0
1.0
1.0
1.0
Contriever
MedMCQA
0.65
0.63
0.66
0.61
Pub.eddQA
0.78
0.79
0.78
0.78
0.76
BioASQ
0.59
0.58
0.60
0.59
0.61
MMLU
0.74
0.75
0.71
0.73
0.73
MedQA
0.99
0.99
0.98
0.99
0.99
MedCPT
MedMCQA
0.72
0.75
0.68
0.72
0.71
PubMedQQ
0.93
0.95
0.91
0.89
0.91
BioASQ
0.87
0.92
0.87
0.81
0.86
MMLU
0.93
0.78
0.82
0.95
0.90
MedQA
0.99
0.99
0.98
0.97
0.95
PubMed
Specter
MedMCQA
0.81
0.83
0.72
0.81
0.91
PubMedQQ
0.78
0.84
0.86
0.93
0.96
BioASQ
0.95
0.86
0.82
0.91
0.92
MMLU
0.80
0.82
0.81
0.83
0.81
MedQA
1.0
1.0
1.0
1.0
Contriever
MedMCQA
0.62
0.65
0.63
0.66
0.61
PubMedQA
0.78
0.79
0.74
0.78
0.71
BioASQ
0.59
0.58
0.60
0.59
0.60

---

On the Vulncrability of Applying Retrieval-Augmeented Generation within Knowledge-Intensive Application Domains
Table 2. Attack success rates (Top 2 retrieval rates) legal Q&A.
Target Information
Task
Retriever
PI1
HotPotQA
NQ
Syn Legal
Syn Legal II
Contriever
0.82
0.88
0.84
0.88
0.81
Legal Q&A
Specter
0.78
0.81
0.89
0.90
0.92
lated to biology research questions.
from previous discussions that the wide-ranging-sccta suc- cess of our proposed universal poisoning attacks is owing to two factors: the consistently high similarity between poi- soned documents and queries due to the intriguing property of retrievers, and the low similarity between queries and clean retrieved documents. In fact, the latter property also implies that queries and their retrieved clean documents tend to be orthogonal. As result, the poisoned document also tends to be perpendicular to clean documents. This orthogonal property motivates us to consider using distance metrics that reflect the distribution of the data, such as the Mahalanobis distance, to detect the poisoned documents.
On the similarity of clean retrieved documents Previ- ously, we mainly focused on the similarity between the query and the poisoned document. We now shift the focus to another crucial factor contributing to the success of uni- versal poisoning attacks: the similarity between the query and its clean retrieved documents.
We present similarity measurements between the query and the clean retrieved documents (from the corpus Textbook) in Table 4 for Contriever and MedCPT, respectively. For each query, we calculate its similarity, the cosine similarity and the inner product, with the K = 5-nearest neighbor re- trieved documents from the corpus Textbook, and report the mean and standard deviation. Additionally, we also report the value of the inner product between query and itself for comparison. We observed that both the inner product and the cosine similarity are low across all query sets. For instance, an average angle of around 70 between the query and retrieved documents is observed across all query sets for the Contriever, and the inner product between the query and retrieved documents is less than 25% of that between the query and itself. These results indicate that the retrieved documents are not as close to their queries as one might ex- pect. In conclusion, these findings highlight gap between the query and its clean retrieved documents. This disparity leaves room for exploitation by various adversarial attacks, including our proposed universal poisoning attacks, posing considerable safety risks.
Due the high dimension of embeddings, calculating the Mahalanobis distance can be challenging. The sample co- variance matrix ensued can be numerically unstable in large data dimensions (Bodnar et al., 2022), leading to an ill- conditioned matrix that is difficult to invert (Trefethen & Bau, 2022). To address this issue, we consider regulariz- ing the sample covariance matrix through shrinkage tech- niques (Ledoit & Wolf, 2003; Bickel et al., 2006). In de- tail, we conduct shrinkage by shifting each eigenvalue by certain amount, which in practice leads to the following, s(X) A (X- - 1)) E (X 1u), where is the mean of ||| with a A (the anchor set defined earlier), (f(a.)) r=
and Σ3 _ (1 B)S + d (TT() with S being the sample covariance of {f Tr(-) is the trace opera- tor, and / E (0. 1) the shrinkage level. We select by cross-validation, with an ablation study in Appendix C.2.
We tested the proposed method for filtering out the poisoned documents over the previous setups. The threshold for fil- tering set at the 95th quantile of the {s( (a())) We present some case studies in Figure 3 below. The complete results are included in Figure 4 and 5 for PubMed (in the appendix), Figure 6 and 7 for StatPearl (in the appendix), and Figure 8 and 9 for Textbook (in the appendix). We observed that, in almost all cases, the proposed method achieves near-perfect detection. Additionally, we observed that the fn-norm defense is effective when using the Con- triever. One potential reason may be that is sensitive to the total length of the documents. Furthermore, applied our method to one of the state-of-the-an poisoning attacks (Zou et al., 2024) and obtained filtering rate greater than 95%, indicating the wide applicability of our proposed defense.
5. New Defense
In this section, we propose a detection based method de- fend against the proposed universal poisoning attacks. We consider a scenario in which the defender, such an RAG service provider, has full access to retrievers. They collect documents from public websites, integrate them into their data corpus, and provide services using both the retrievers and the updated data corpus. The defender's objective is to develop an algorithm capable of automatically detecting potential adversarial documents be incorporated into their data corpus. We assume that the defender already has collection of clean documents associated with set of tar- geted queries to be protected, which serve as an anchor set (denoted as A- = {aI,..., } for detection.
We now formally introduce our new defense method. Recall
8

---

On the Vulncrability of Applying Retrieval-Augmennted Generation within Knowledge-Intensive Application Domains
Table 3. Verification for the orthogonal augmentation property. We collected set of documents from the MedQA denoted Q {qi...., ...... J. and selected several other sets of documents P: = {paa,..., Pio (i = 1, 2, 3) with varying lengths and similarities to Q. We report a total of four similarity measurements, and for each measurement, we report both the mean and the variance. We observed that as two documents p and become more orthogonal, namely with smaller f(q)"f(p) the change f(g @ p)) f(4) (both in angle, denoted by -) and inner product) tends to be more perpendicular to f()). thus corroborating the orthogonal augmentation property.
f(p),
Retriever
Measurement
P3-1
Pj-2
Pj-3
[a.)
2.75 1 0.47
1.53
0.12
0.43
0.07
@ puil) (f
f(a))
0.13 + 0.10
0.07 + 0.06
0.05 + 0.07
Contriever
[f((a 1 pAl)- f(a)).
99.1 : 5.3
96.8" = 4.6
92.5" + 4.0
f(()) f(a))
2.95 + 0.40
2.95 + 0.40
2.95 0.40
f(a)) f(p5s)
84.4 = 10
72.8 + 8
62.5 1:4
(f @ pHil) - f(()))
1.64 + 1.6
1.21 + 1.2
0.42 0.6
MedCPT
I(flla @ pソa]) f()). ((G))
97.2" +
98.9" + 6
95.9" 6
f(g))
88.8 + 9.7
88.8 + 9.7
88.8 + 9.7
Table Evidence on the low similarity the (clean) retrieved documents. For each query g sampled from the query set, we calculate its similarity, both in angle (denoted by ((:-)) -) and inner product, with the 5-nearest neighbor retrieved documents from the corpus Textbook, denoted as R, and then report the mean and standard deviation. We observed that both the inner product and the cosine similarity are low across all query sets, indicating the low quality of the retrieved documents.
Retriever
Query Set
MMLU
MedQA
BioASQ
MedQA
MCQA
((f(g).
71 + 4.0
729 2.1
72 3.6
60 4
71 4.9
Contriever
f(R)
0.98 .15
1.72 + .41
1.02 + .17
1.06 + .14
1.13 - .16
f(9)
1.2
11.6 + 7.6
4.4 4.451..5 1.5
3.85 + 1.6
4.6 1.6
<(f(g).
48 3.5
41 2.9
49 3.9
52" 2.6
53" 4.4
MedCPT
f(g)z f(a)
66.2
1.3
64.3 + 2.1
62.8 + 2.4
62.5 + 2.5
125 + 16
92.6 9.3
124 17
141 13
137 20
Corpus: PubMed, Query: MMLU-Med
Corpus: PubMed, Query: MedQA
Corpus: PubMed, Query: BioASQ
PPI
PPI
NQ
MARCO
MARCO
Detecction Rate
CE-
I--orm (a) Retriever: MedCPT
Proposed Method
Zs-norm
Proposed Method
Zy-norm (c) Retriever: Contriever
Proposed Method
Retriever: Specter
Figure 3. Detection results of the commonly used mn-m and the proposed method. In each plot, we chose three types of targeted information to create poisoned documents. We observed that the proposed method consistently achieves near-perfect detection rate.
6. Conclusion
ings, we develop a detection-based defense, achieving high detection accuracy. Recent work on dynamic pruning for LLMs, such as Probe Pruning (Le et al., 2025), enables effi- cient inference without model retraining. Investigating how such pruning methods affect the robustness and retrieval behavior of RAG systems remains an important direction for future work. The Appendix provides additional details experimental setups and ablation studies.
This paper studies the vulnerability of retrieval systems in RAG. We first demonstrate that retrieval systems in RAG for medical Q&A are vulnerable universal poisoning attacks. Next, we provide two-fold insights towards understanding the vulnerability: (I) by identifying an intriguing property of dense retrievers, and (2) revealing the relatively low simi- larity of the clean retrieved documents. Based on these find-
9

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Impact Statement
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In International conference machine learning. pp. 1597-1607. PMLR, 2020.
This paper presents work aimed at advancing the safety and trustworthiness of machine learning. There are both po- tential positive and negative societal impacts of this work. Potential negative impacts include the possibility that adversary could apply the proposed attack to other retrieval systems. On the other hand, we anticipate that there will be many more positive societal impacts of this work: (I) highlighting the need for vigilance regarding security risks when applying RAG in safety-critical domains; (2) provid- ing several insights into understanding these potential safety risks; and (3) introducing new, effective defense that can be used for detecting poisoned documents.
Cohan, A., Feldman, S., Beltagy, I., Downey, D., and Weld, D. S. Specter: Document-leve representation learning using citation-informed transformers. arXiv preprini arXiv:2004.07180, 2020.
Deng. G., Liu, Y., Wang, K., Li, Y., Zhang, T. and Liu, Y. Pandora: Jailbreak gpts by retrieval augmented genera- tion poisoning. arXiv preprint arXiv:2402.088 08400,,,,4, 2024.
Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.E.,, Lomeli, M., Hosseini, L., and Jégou, H. The faiss library. 2024.
Acknowledgement
The work of Xun Xian, Xuan Bi, and Mingyi Hong was supported part by sponsored research award by Cisco Research. The work of Ganghua Wang and Jie Ding supported in part by the Army Research Office under the Early Career Program Award, Grant Number W911NF-23- 10315.
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang. H. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv: 2..iv:2312.199999 10997, 2023.
Gu, X., Zheng, X., Pang, T. Du, C., Liu, Q. Wang, Y.. Jiang, J., and Lin, M. Agent smith: A single image can jailbreak one million multimodal Ilm agents exponentially fast. arXiv preprint arXiv:2402.08567 2024.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L.,, Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.000 08774, 2023.
He, J., Neubig, G., and Berg-Kirkpatrick, T. Efficient nearest neighbor language models. arXiv preprint arXiv:2109.04212 2021.
Alon, U., Xu, F, He, J., Sengupta, S., Roth, D., and Neubig, G. Neuro-symbolic language modeling with automaton- augmented retrieval. In International Conference on Ma- chine Learning. pp. 468 485. PMLR, 2022.
Hersh, W. Search still matters: information retrieval in the era of generative ai. Journal of the American Medical Informatics Association, pp. ocae014, 2024.
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et Palm technical report. arXiv preprint arXiv:2305.10400 2023.
Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., and Heck, L. Learning deep structured semantic models for web search using clickthrough data. Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pp. 2333-2338. 2013.
Bickel, P.J., Li, B., Tsybakov, A. B., van de Geer, A., Yu, B., Valdés, T., Rivero, C., Fan, J., and van der Vaart, A. Regularization statistics. Test, 15:271-344 2006.
Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., and Grave, E. Unsupervised dense infor- mation retrieval with contrastive learning. arXiv preprint 2r 2021.
Bodnar, O., Bodnar, T., and Parolya, N. Recent advances in shrinkage-based high-dimensional inference. Journal of Multivariate Analysis, 188:104826, 2022.
Ji, Z., Lee, N., Frieske, R.. Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. Survey of halluci- nation in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Van Den Driessche, G. B., Lespiau, J.-B., Damoe, B., Clark, A., et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pp. 2206-2240. PMLR, 2022.
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., and Szolovits, What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 1 11(14):6421, 2021.
Butler, Open australian legal corpus (2024), 2024.
10

---

On the Vulnerability of Applying Retrieval-Augmenned Generation within Knowledge-Intensiv Application Domains
Jin, Q., Kim, W., Chen, Q., Comeau, D. C., Yeganova, L., Wilbur, W. J., and Lu, Z. Medept: Contrastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval. Bioinformat- ics1 1) 1):bad6651, 2023.
Miao, J., Thongprayoon, C., Suppadungsuk, S., Garcia Va- lencia, O. A., and Cheungpasitpom, W. Integrating retrieval-augmeetted generation with large language mod- els in nephrology: advancing practical applications. Medicina, 60(3):445, 2024.
Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M. Generalization through memorization: Nearest neighbor language models. arXiv preprint arXiv:1911.00172. 2019.
Nguyen. T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., and Deng. L. Ms marco: A human- generated machine reading comprehension dataset. 2016.
Nogueira, R. and Cho, K. Passage re-ranking with bert. arXiv preprint arxiv: (a0e::001000000 2019.
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. Natural questions: a benchmark for ques- tion answering research. Transactions of the Association for Computational Linguistics, 7:453-48 466,3-46,... 2019.
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton Broown K., and Shobam, Y. In-context retrieval -ugmeeted language models. Transactions of the Association for Computational Linguistics, 11:1316- 1331, 2023.
Le, Q., Diao, E., Wang. Z., Wang, X., Ding, J., Yang, L., and Anwar, A. Probe Pruning: Accelerating LLMs Through Dynamic Pruning via Model-Probing. In Proceedings of the International Conference Learning Representa- tions, 2025.
Robertson, S., Zaragoza, H., et al. The probabilistic rele- vance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4):333-389 2009.
Ledoit, O. and Wolf, M. Honey, i shrunk the sample co- variance matrix. UPF economics and business working paper, (691), 2003.
Robertson, S. E., Walker, S., Jones, S., M. M., Gatford, M., al. Okapi at trec-3. Nist Special Publication Sp, 109: 109, 1995.
Levine, Y., Dalmedigos, I. Ram, O., Zeldes, Y., Jan- nai, D., Muhlgay, D., Osin, Y., Lieber, O., Lenz, B., Shalev-Shwartz, S., et al. Standing on the shoul- ders of giant frozen language models. arXiv preprint 20Xiv:2204.10019 2022.
Thironavukarasu, A. J., Ting, D. J., Elangovan, K., Gutier- rez, L., Tan, F., and Ting, D. S. W. Large language models in medicine. Nature medicine, 1930 2023.
Tian, S., Jin, Q., Yeganova, I.., Lai, P.-T., Zhu, Q. Chen, X., Yang. Y., Chen, Q., Kim, W., Comeau, D. C., et al. Opportunities and challenges for chatgpt and large lan- guage models in biomedicine and health. Briefings in Bioinformatics, 25(1) 25())bbad,493, 2024.
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t. Rocktäschel, T., et al. Retreeeal-augmenned generation for knowledge- intensive nlp tasks. Advances in Neural Information Pro- cessing Systems, 33:9459-9497 2020.
Liu, Y.-A., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., and Cheng, X. Black-bor adversarial attacks against dense retrieval models: A multi-view contrastive learning method. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 1647-1656, 2023.
Trefethen, L. N. and Bau, D. Numerical linear algebra, volume 181. Siam, 2022.
Voigt, P. and Von dem Bussche, A. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing. 10(3152676): 10-5555, 2017.
Long, Q. Deng, Y., Gan, L., Wang, W., and Pan, S. J. Back- door attacks on dense passage retrievers for disseminating misinformation. arXiv preprint 13355:2402.13332, 2024. Loukas, L., Stogiannidis, I., Diamantopoulos, O., Malakasi- otis, P., and Vassos, S. Making lims worth every penny: Resource-limited text classification in banking. In Pro- ceedings of the Fourth ACM International Conference on Al in Finance, pp. 392-400, 2023.
Wang, G., Xian, X., Srinivasa, J., Kundu, A., Bi, X., Hong, M., and Ding, J. Demystifying poisoning backdoor at- tacks from a statistical perspective. International Confer- ence on Learning Representations, 2024.
Wang, X., Le, Q., Ahmed, A., Diao, F.,, Zhou, Y., Baracaldo, N., Ding, J., and Anwar, A. MAP: Multi-human-vall value alignment palette. International Conference Learning Representations (Oral Presentation), 2025.
Ma, X., Gong, Y., He, P., Zhao, H., and Duan, N. Query rewriting for eetrieval-augmenter large language models. arXiv preprint orXiv:2305.1 00.i22355.422.., 2023.
Wiratunga, N., Abeyratne, R., Jayawardena. L., Martin,
K., Massie, S., Nkisi-Orji. I., Weerasinghe, R., Liret,

---

On the Vulnerability of Applying Retrieval-Augmenned Generation within Knowledge-Intensive Application Domains
A., and Fleisch, B. Cbr-rag: Case-based reasoning for retrieval augmented generation in Ilms for legal question answering. arXiv preprint arXiv:2404.04300 2024.
Xian, X., Wang, G., Srinivasa, J., Kundu, A., Bi, X., Hong, M., and Ding, J. Understanding backdoor attacks through the adaptability hypothesis. In International Conference Machine Learning, pp. 37952-37976 PMLR, 2023a.
Xian, X., Wang, G., Srinivasa, J., Kundu, A., Bi, X., Hong, M., and Ding, unified detection framework for inference-stage backdoor defenses. Advances in Neural Information Processing Systems, 36:7867-7894, 2023b.
Xiong, G., Jin, Q. Lu, Z., and Zhang. A. Benchmark- ing -augeeeelleegeeeteed generation for medicine. arXiv preprint ar arXiv:2402.33178 2024.
Yang, Z. Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question an- swering, arXiv preprint arXiv:1809.0966 2018.
Zhao, W. X., Liu, J., Ren, R., and Wen, J.-R. Dense text retrieval based on pretrained language models: survey. ACM Transactions on Information Systems, 42(4))1-60, 2024.
Zhong, Z., Lei, T., and Chen, D. Training language models with memory augmentation. arXiv preprint 1rX74 2022.
Zhong, Z., Huang, Z., Wettig, A., and Chen, D. Poisoning retrieval corpora by injecting adversarial passages. arXiv preprint arXiv:2310.19156. 2023.
Zou, W. Geng, R., Wang, B., and Jia, J. Poisonedrag: Knowledge poisoning attacks to retrieval-aummented generation of large language models. arXiv preprint arXiv:2402.07867,2000 2024.
12

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensivy Application Domains
Appendix for "On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains"
In section A, we list more implementation details including: computing resources, examples of datasets and others. We provide experimental results are omitted from the main text due to space limit in Section B. In section C, we provide ablation studies regarding different choices of hyperparameters used the paper. Section D, we present an empirical study on applying our proposed attacks to legal domains.
A. Details on experimental setups
A.1. Computing Resource
We calculate the embedding vectors on a machine equipped with an Nvidia A40 GPU. We conduct the nearest neighbor search on a machine with an AMD 7763 CPU, 18 cores, and 800 GB memory. To facilitate efficient search, following convention, we employ the Faiss package (Douze et al., 2024).
A.2. Details on experiments
A.2.1. CORPUS
Following (Xiong et al., 2024), we provide the statistics of used corpora in Table below. We note that due to limited computational resources, we use a randomly sampled subset PubMed with around 2 million documents. For the other two datasets, we use the complete versions.
Table 5. Statistics of corpora used in our experiments. No. Doc.: numbers of raw documents; No. Snippets: numbers of snippets; Avg. L: average length snippets.
Corpus
No. Doc.
No. Snippets
Avg. I.
Domain
PubMed
23.9M
23.9M
296
Biomed.
StatPcarls
9.3k
301.2k
119
Clinics
Textbooks
18
125.8k
182
Medicine
We show some examples.
Textbook:
Observation and visualization are the primary techniques a student should use to learn anatomy. Anatomy is much more than just memorization of lists of names. Although the language of anatomy is important, the network of information needed to visualize the position of physical structures in a patient goes far beyond simple memorization. Knowing the names of the various branches the external carotid artery is not the same as being able to visualize the course of the lingual artery from its crigin in the neck to its termination in the tongue. Similarly, understanding the organization of the soft palate, how it is related to the oral and nasal cavities, and how it moves during swallowing is very different from being able to recite the names of its individual muscles and nerves. An understanding of anatomy requires an understanding of the context in which the terminology can be remembered. How can gross anatony be studied?
Amantadine keratopathy a rare dose-dependent disease process in which the drug amantadine causes damage to corneal endothelial cells through unknown mechanisms. Damage to the endothelium can ultimately lead to severe corneal edema with decreased visual acuity. Rdema is typically reversible with inscontinaation of the drug, but irreversible cases requiring corneal transplants have been reported. This activity describes the evaluation and management of amantadine keratopathy and highlights the role of the interprofessional team in improving care for patients with this condition.
StatPearls:
13

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
A.2.2. QUERY
Following (Xiong et al., 2024), we provide the statisties of used query in Table below. We also provide some examples of these questions in the following.
Table Statisties query datasets used our experiments. No. numbers of options; Avg. average token counts each question.
Dataset
Size
No. O.
Avg. I.
Source
MMLU-Med
1,089
Examination
McdQA-US
1,273
4
177
Examination
McdMCQA
4.183
4
26
Examination
PubMedQA
500
24
Literature
BioASQ-Y/N
618
Literature
show some examples.
BioASQ
Is there any association between the human gut microbiome and
depression?
MedQA
A 65-year-old man is brought to the emergency department 30 minutes after the onset of acute chest pain. He has hypertension and asthma.
Current medications include atorvastatin, lisinopril, and albuterol inhaler. He appears pale and diaphoretic. His pulse is 114/min and blood pressure is 130/88 mm Hg. An ECG shows ST-segment depressions in leads II III, and aVF. Laboratory studies show an increased serum troponin T concentration. The patient is treated for acute coronary syndrome and undergoes percutaneous transluminal coronary angioplasty. At the time of discharge, echocardiography shows left ventricular ejection fraction of 588. In addition to aspirin, which of the following drugs should be added to this patient's medication regimen?
A.2.3. TARGETED INFORMATION
Some examples of the targeted information are:
(NQ): Renowned authors, Henry Roth and Robert Erskine Childers contributed significantly to literature. Roth, born in England, wrote classics. On the other hand, Childers hailed from Ireland, also producing notable works;
(PII):
Bob is currently graduate student at xx University, majoring in Accounting, Bob's s email address is Bobigmail. com, and Bob's s phone number is 123-456-7890.
(MS-MARCO):
In the unique Dutch tradition, people celebrate Oranjegekte or Koningsdag wearing unusual purple clothing. This distinctive practice marks their national holiday with a vibrant purple sea natives.
(HotpotQA):
Renowned authors, Henry Roth and Robert Erskine Childers contributed significantly to literature. Roth, born in England, wrote classics. On the other hand, Childers hailed from Ireland, also producing notable works.
A.2.4. IMPLEMENTATION DETAILS
In this section, we list several important implementation details that are omitted previously.
Zero-Shot and Question-Ona Only Retrieval Following (Xiong et al., 2024), only questions are used during retrieval; answer options and demonstrations (used for in-context learning for generation) are not provided as input. We present an ablation study to explore the scenario where both questions and their corresponding choices are used for retrieval, and demonstrations included in the queries.
14

---

On the Vulnerability of Applying Retrieval-Augmenned Generation within Knowledge-Intensive Application Domains
B. Omitted experimental results
Performance under paraphrasing defenses The table below presents the results for the proposed attacks under paraphrasing defenses, where the defender rephrases the queries defense mechanism prevent the attack from knowing the exact queries. We observe only a slight decrease in attack success rates compared the case using exact queries, highlighting the robustness of the proposed attacks.
Table 7. Top 2 retrieval success rates under query-paraphrasing defenses. We observe consistently high success rates despite the fact that the queries are paraphrased.
Corpus
Retriever
Query
Target Information
MS- MARCO
NQ
HotpotQA
Diagnostic
MMLU
0.76
0.72
0.61
0.52
0.64
McdCPT
MedQA
0.98
0.97
0.98
0.97
0.96
PubMedQA
0.94
0.96
0.92
0.95
0.91
MMLU
0.79
0.75
0.54
0.84
0.72
Textbook
Specter
MedQA
0.97
0.86
0.87
0.97
0.91
PubMedQA
0.94
0.80
0.85
0.98
0.82
MMLU
0.57
0.75
0.64
0.85
0.70
Contriever
MedQA
0.90
0.91
0.94
0.96
0.90
PubMedQA
0.97
0.98
0.98
0.98
0.92
Detection Results We test the proposed detection method over the previously mentioned 225 attacks cases and present results of them in Figure and Figure 5 for PubMed, Figure and Figure 7 for StatPearl, and Figure and Figure 8 for Texthook. In each plot, we show the results of detecting poisoned documents created using three different kinds of targeted information. (The cases for the other two types of targeted information are similar.) We observed that the f--nrrm fails to detect those poisoned documents, with a detection rate of less than 50% in all cases. However, our method achieves a consistently high detection rate of over 98%, indicating the widespread effectiveness our proposed method.
In terms of the detection performance when using the Contriever as the embedding model, we observed that the detection performance of the f2 norm under Contriever is roughly par with our proposed method, achieving over 95% detection rates in all cases. One potential reason behind these findings is that the Contriever sensitive to the length of the documents. Specifically, the larger the length of the document p. the larger the corresponding embedding || (p) I12- Given the fact that the overall length of retrieved documents and the poisoned documents are (statistically) different, their &g norm also tends to be distinct. As a result, the 22-norm- based defense tends to be effective.
15

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Corpus: PubMed, Query: MMLU-Med
Corpus: PubMed, Query: MedQA
Corpus: PubMed, Query: MedMCQA
1.
PPI
26
PPI
20
NQ
NQ
2025
MARCO
MARCO
Detection Rate
60000
Dos.
5000
b-norm
Proposed Method
6.0
by-norm
Proposed Method
8.
b--orm
Proposed Method
Retriever: MedCPT
Retriever: MedCPT
Retriever: MedCPT
Corpus: PubMed, Query: PubMedOA
Corpus: PubMed, Query: BloASQ
Detection Rate
Detection Rate
sooo
so
sooa
02-
f2-norm Proposed Method
fo-norm
Proposed Method
Retriever: MedCPT
Retriever: MedCPT
Figure 4. Detection results of the commonly used f2-norm and the proposed methad on PubMed corpus with MedCPT, In each plot, we chose three types of targeted information near-perfect detection rate, while the f2 f2-norm methods fail.
poisoned documents. We observed that the proposed method consistently achieves
16

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
Corpus: PubMed, Query: MMLU-Med
Corpus: PubMed, Query: MedQA
Corpus: PubMed, Query: MedMCQA
1.
PPI
PPI
20
PPI
NQ
NQ
NQ
2055
MARCO
MARCO
MARCO
Detection Rate
D.0.
5600
t--ormm
0oo0
Proposed Method
by-norm
Proposed Method
8.0
ły-norm
Proposed Method
Retrieven: SPECTER
Retriever: SPECTER
Retriever: SPECTER
Corpus: PubMed, Query: PubMedOA
Corpus: PubMed, Query: BloASQ
PPI
Detection Rate
Detection Rate
12-norm Proposed Method Retriever: SPECTER
f2-norm
Proposed Method
Retriever: SPECTER
Figure 5. Detection results of the commonly used fo-n and the proposed method on PubMed corpus with Specter. In each plot, we poisoned documents. We observed that the proposed method consistently achieves
chose three types of targeted information near-perfect detection rate, while the f2 f2-norm methods fail.
17

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
Corpus: StatPearl, Query: MMLU-Med
Corpus: StatPearl, Query: MedQA
Corpus: StatPearl, Query: MedMCQA
10
PPI
PPI
PPI
NQ
NQ
MARCO
MARCO
MARCO
650
Detection Rate
6.0
0001
0001
0.-
0.000
0.3
6202
t--ormm
Proposed Method
b-norm
Proposed Method
b-norm
Proposed Method
Retriever: MedCPT
Retriever: MedCPT
Retriever: MedCPT
Corpus: StatPearl, Query: PubMedQA
Corpus: StatPearl, Query: BloASQ
0
Detection Rate
Detection Rate
0
eas
0.41
0.4
DOO
6.0.0.
t--nrmm Proposed Method
ty-ormm
Proposed Method
Retriever: MedCPT
Retriever: MedCPT
Figure 6. Detection results of the commonly used l2 norm and the proposed method on StatPearl corpus with MedCPT. In each plot, we chose three types of targeted information to create poisoned documents. We observed that the proposed method consistently achieves near-perfect detection rate, while the fn-m methods fail.
18

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
Corpus: StatPearl, Query: MMLU-Med
Corpus: StatPearl, Query: MedQA
Corpus: StatPearl, Query: MedMCQA
10
PPI
PPI
PPI
NQ
NQ
NQ
MARCO
MARCO
MARCO
650
Detection Rate
6.0
0001
6501
0.0
0002
0.-
0.000
0.3
bood
5000
2000
6202
b-norm
Proposed Method
bynorm
Proposed Method
b--norm
Proposed Method
Retriever: Specter
Retriever: Specter
Retriever: Specter
Corpus: StatPearl, Query: PubMedQA
Corpus: StatPearl, Query: BloASQ
0
6
Detection Rate
Detection Rate
0.0
eas
0/4
0.4
DOO
0
6.0.
--norm
Proposed Method
ty-ormm
Proposed Method
Retriever: Specter
Retriever: Specter
Figure 7. Detection results of the commonly used fz norm and the proposed method on StatPearl corpus with Specter. In each plot, we chose three types of targeted information to create poisoned documents. We observed that the proposed method consistently achieves near-perfect detection rate, while the norm methods fail.
19

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
Corpus: Textbook, Query: MMLU-Med
Corpus: Textbook, Query: MedQA
Corpus: Textbook, Query: MedMCQA
1.
PPI
PPI
PPI
NQ
MARCO
MARCO
MARCO
650
Detection Rate
6.0
0.0
02-
0.2
0.3
b-norm
Proposed Method
b-norm
Proposed Method
by-norm
Proposed Method
Retriever: Specter
Retriever: Specter
Retriever: Specter
Corpus: Textbook, Query: PubMedQA
Corpus: Textbook, Query: BloASQ
PPI
Detection Rate
Detection Rate
0/4
04
b--orm
Proposed Method
b-norm
Proposed Method
Retriever: Specter
Retriever: Specter
Figure 8. Detection results of the commonly used & norm and the proposed method on Textbook corpus with Specter. In each plot, we chose three types of targeted information near-perfect detection rate, while the fn-m methods fail.
poisoned documents. We observed that the proposed method consistently achieves
20

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensiv Application Domains
Corpus: Textbook, Query: MMLU-Med
Corpus: Textbook, Query: MedQA
Corpus: Textbook, Query: MedMCQA
1.
PPI
PPI
PPI
NQ
MARCO
MARCO
MARCO
650
Detection Rate
6.0
0.0
02-
0.2
0.3
2000
b-norm
Proposed Method
f b-norm
Proposed Method
b--orm
Proposed Method
Retriever: Medcpt
Retriever: Medcpt
Retriever: Medcpt
Corpus: Textbook, Query: PubMedQA
Corpus: Textbook, Query: BloASQ
PPI
Detection Rate
Detection Rate
6
0/4
0.4
b-norm
Proposed Method
by-orm
Proposed Method
Retriever: Medcpt
Retriever: Medcpt
Figure 9. Detection results of the commonly used l norm and the proposed method on Textbook corpus with MedCPT. In each plot, we chose three types of targeted information create poisoned documents. We observed that the proposed method consistently achieves near-perfect detection rate, while the fn-mr methods fail.
21

---

On the Vulnerability of Applying Retrieval-Augmenned Generation within Knowledge-Intensive Application Domains
C. Ablation Studies
In this section, we provide ablation studies on different hyperparameters used in experimental results.
C.1. Top attack retrieval rates
We report the top attack retrieval success rates in Table 8, with standard deviations less than 0.1. Overall, the success rates only slightly decrease compared to the top 2 results presented in the paper, which is reasonable. These findings highlight that medical Q&A is extremely vulnerable poisoning attacks and thus requires robust defense mechanisms.
C.2. On the shrinkage level |
In this section, we provide an ablation study on the choices of 3 used in calculating the proposed distances. We summarize the results in Table below, with standard errors within 0.08. We observed that 3 increases, the detection rate begins to decrease. This is reasonable since the covariance tends shrink towards identity matrix and hence loses the ability capture the data's distribution. In practice, suggest the defender employ cross-validation-ban techniques select the optimal & for detection.
22

---

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensie Application Domains
Table Top retrieval
corpora, 3 retrievers, query sets, and 5 of targeted information. The interpretation of results in each cell adheres the following rule. We use the lop-left cell as example: it represents success rate of 0.78 achieved using PPI as targeted information.
the Corpus: Textbook, Retriever: MedCPT, Query set: MMLU-Men
Target Information
Corpus
Retriever
Query
PPI
MS-MARCO
NQ
HotpotQA
Diagnostic
MMLU
0.74
0.80
0.81
0.78
0.81
MedQA
0.98
0.98
0.98
0.98
0.99
MedCPT
MedMCQA
0.81
0.80
0.76
0.82
0.78
PubMedQA
0.98
0.96
0.96
0.96
0.94
BioASQ
0.91
0.97
0.93
0.92
0.91
MMLU
0.92
0.73
0.82
0.93
0.78
MedQA 0.98
0.98
0.97
0.98
0.98
Textbook
Specter
MedMCQA
0.88
0.80
0.70
0.86
0.87
PubMedQA
0.92
0.82
0.92
0.91
0.91
BioASQ
0.95
0.93
0.95
0.94
0.91
MMLU
0.78
0.80
0.81
0.84
0.81
MedQA 1.0
1.0
1.0
1.0
0.99
Contriever
MedMCQA
0.60
0.60
0.65
0.64
0.67
PubMedQA
0.78
0.80
0.82
0.80
0.80
BioASQ
0.61
0.61
0.60
0.61
0.61
MMLU
0.73
0.72
0.70
0.73
0.72
MedQA
0.98
0.98
0.98
0.99
0.97
MedCPT
MedMCQA
0.71
0.73
0.68
0.72
0.74
PubMedQA
0.93
0.92
0.91
0.90
0.91
BioASQ
0.86
0.91
0.85
0.86
0.86
MMLU
0.92
0.68
0.77
0.91
0.90
MedQA
1.0
1.0
0.96
0.96
0.94
StatPearls
Specter
MedMCQA
0.84
0.62
0.70
0.79
0.78
PubMedQA
0.97
0.80
0.85
0.95
0.93
BioASQ
0.91
0.80
0.87
0.93
0.92
MMLU
0.80
0.81
0..81
0.81
0.81
MedQA 1.0
1.0
1.0
1.0
1.0
Contriever
MedMCQA
0.61
0.65
0.63
0.66
0.61
PubMedQA
0.78
0.77
0.78
0.78
0.75
BioASQ
0.58
0.56
0.58
0.58
0.61
MMLU
0.72
0.74
0.71
0.73
0.71
MedQA
0.99
0.99
0.98
0.99
0.99
McdCPT
MedMCQA
0.70
0.71
0.68
0.71
0.71
PubMedQA
0.91
0.94
0.91
0.88
0.91
BioASQ
0.85
0.91
0.86
0.81
0.85
MMLU
0.91
0.75
0.81
0.94
0.90
McdQA
0.99
0.98
0.98
0.96
PubMed
Specter
MedMCQA
0.81
0.82
0.70
0.81
0.91
PubMedQA
0.76
0.82
0.85
0.96
BioASQ
0.93
0.86
0.82
0.91
0.92
MMLU
0.80
0.82
0.81
0.81
0.81
MedQA
1.0
1.0
1.0
1.0
1.0
Contriever
MedMCQA
0.62
0.63
0.61
0.64
0.61
PubMedQA
0.76
0.77
0.74
0.76
0.71
BioASQ
0.57
0.56
0.60
0.57
0.60
23

---

On the Vulncrability of Applying Retrieval-Augmenned Generation within Knowledge-Intensive Application Domains
Table Detection performance on different choices of B.
Corpus
Query Set
8 0.001
3
0.005
= 0.01
0.05
8 0.1
3
0.2
.99
.99
.99
.98
.98
Texthook
.99
.99
.99
.98
.98
.97
.99
.99
.99
.98
.98
.99
.99
.99
.98
.96
.96
PubMed
.99
.99
.99
.97
.96
.95
.99
.99
.97
.97
.96
D. Examples of Legal Q&A Datasets
In this section, we list some examples of the legal Q&A dataset used in our paper in the table below.
Table 10. Examples Legal Q&A case
Component
Description
Case Name
Smith The State of New South Wales (2015)
Question
How did the case of Smith v The State New South Wales (2015) clarify the interpretation of 'reasonable grounds' for conducting a search without a warrant?
Support
In [Case Name: Smith The State of NSW (2015)), the plaintiff [Action: was scarched without a warrant] [Location:
known drug trafficking area] based on the plaintiff''s nervous demeanor and presence area, but [Outcome: no drugs were found]. The legality of the search was contested, focusing on [Legal Concept:
whether 'reasonable grounds' existed].
Answer
The case ruled 'reasonable grounds' require clear, specific facts of criminal activity, not just location or behavior.
24


Reviews:
[{"review_id": "SPkuKbGzrY", "reviewer": "Reviewer_3ntU", "summary": "This paper demonstrates the vulnerability of retrieving systems in RAG to universal poisoning attacks. Through examples in medical Q&A, the paper reveals that due to the orthogonal augmentation property, the deviation from the query’s embedding to that of the poisoned document tends to only shift in the orthogonal direction, which means that the poisoned document and the query retain high similarity, therefore enabling successful retrieval and poisoning attacks. Based on these findings, the paper develops a new detection-based defense, achieving a high level of accuracy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.\n\n>Q: One weakness is that the orthogonal augmentation property relies on the behavior of the embedding function, specifically its approximate linearity under concatenation. Given that Contriever and MedCPT have different sensitivity to document length, the claim might vary with retrieval architecture or training regimes, and does not necessarily support the notion of \"universal\". The authors select three retrievers based on their general ability / domain design. Does it make sense to have retrievers that have more architectural / training regime variations?\n\n**R**: The three retrievers used in our paper themselves have different architectures and training regimes. For example, the Contriever is a single-tower model with contrastive learning training regime, i.e., InfoNCE loss, trained on general purpose domain data. While the MedCPT is a cross-encoder re-ranker model with two-stage training (InfoNCE for contrastive learning + cross-entropy for reranker) for medical purposes. We will add results on closed-source models like text-embedding-3 from OpenAI (shown in Table below) in the revised version to further validate the generality of our findings.\n\nTable. Top 2 retrieval success rates with text-embedding-3 as the retriever and MedMCQA, PubMedQA as the query corpus.\n| Corpus | Attack Success Rate |\n|--------------------|---------------------|\n| Textbook     | 0.95                |\n| StatPearls | 0.90                |\n| PubMed       | 0.83                |\n\n>Q: The other weakness is that another underlying assumption for the claim is that the concatenated adversarial information is nearly orthogonal to the original query, but that might not be the case and being orthogonal does not necessarily mean the underlying documents are semantically unrelated.\n\n**R**: We thank you for your extremely sharp comments. We addressed this point in Lines 352 - 362 (left) of the main text. We found that closeness to orthogonality between embeddings **does not** imply that their associated documents are semantically irrelevant. For example, we randomly sampled two nonoverlapping batches of questions from the MedQA dataset and found that the angle between their embeddings is around $70^\\circ$. Yet, these batches of queries are all semantically related to biology research questions. As a result, we believe the stated assumption is reasonable to a certain degree. We will further clarify this in the revised version.\n\n>Q: The authors mention that they check the general attack areas / target datasets by using GPT-3 to check their semantic closeness. It could be helpful to include some summary in the appendix?\n\n**R**: We will include the summary of the semantic closeness check in the appendix as you suggested.\n\n>Q: I have reviewed all supplementary materials - the authors mention that the details of the retrievers are in the appendix but I have not found those details.\n\n**R**: We will include the details of the retrievers in the appendix as you suggested.", "claims_and_evidence": "The paper makes key claims that due to the orthogonal augmentation property of the embeddings, the high similarity of the poisoned document and query can be maintained, enabling the poisoning attacks. The claim is backed by experiments on multiple dense retrievers (e.g., Contriever and MedCPT) using the MedQA dataset. By changing the lengths and similarities of 𝑝 relative to \n𝑞 and measured four different similarity metrics,  the results show that as inner product $𝑓(𝑞)^𝑇𝑓(𝑝)$ decreases, the concatenated embedding remains largely aligned with 𝑓(𝑞) (with an angle close to 90° for the augmentation vector 𝑣), and support the theoretical claim.\n\nOne weakness is that the orthogonal augmentation property relies on the behavior of the embedding function, specifically its approximate linearity under concatenation. Given that Contriever and MedCPT have different sensitivity to document length, the claim might vary with retrieval architecture or training regimes, and does not necessarily support the notion of \"universal\".\n\nThe other weakness is that another underlying assumption for the claim is that the concatenated adversarial information is nearly orthogonal to the original query, but that might not be the case and being orthogonal does not necessarily mean the underlying documents are semantically unrelated.", "methods_and_evaluation_criteria": "This paper proposes the vulnerability of the RAG systems to poisoning attacks, and measures the success rate of attacks with the appropriate ablation study of the K first top results. The methods and evaluation criteria make sense for the main claim. However, for the new detection mechanism, the paper only measures precision but not recall of the new detection mechanism.", "theoretical_claims": "There is no theoretical claim in this paper.", "experimental_designs_or_analyses": "The experimental designs are sound based on the best of my knowledge.", "relation_to_broader_scientific_literature": "Essential references related to this paper include previous works on RAG systems and adversarial attacks on RAG. The authors are able to cite a few key works in the field and clarify their unique contributions specific to the adversarial attacks on retrieval part of RAG and specific knowledge domain in healthcare.", "essential_references_not_discussed": "There are other literature regarding LLM poisoning attacks.", "other_strengths_and_weaknesses": "This paper is clear, well-motivated and the research direction has tremendous impact to real-world AI applications.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- The authors select three retrievers based on their their general ability / domain design. Does it make sense to have retrievers that have more architectural / training regime variations?\n- The authors mention that they check the general attack areas / target datasets by using GPT-3 to check their semantic closeness. It could be helpful to include some summary in the appendix?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PnXmRSOGbF", "reviewer": "Reviewer_swyr", "summary": "The paper explores the vulnerability of Retrieval-Augmented Generation (RAG) systems, specifically in knowledge-intensive domains like medical and legal Q&A. The authors demonstrate that retrieval models used in RAG are susceptible to universal poisoning attacks, where adversaries inject manipulated documents into a corpus to influence retrieval outcomes. By conducting extensive experiments across 225 different combinations of corpus, retriever, query, and targeted information, they reveal how poisoned documents can consistently be retrieved at high ranks. The paper further investigates the underlying reasons for this vulnerability, introducing the concept of orthogonal augmentation, which explains how document embeddings are manipulated to maintain high similarity with queries. To mitigate this risk, the authors propose a detection-based defense mechanism leveraging Mahalanobis distance and covariance shrinkage, demonstrating high success rates in identifying poisoned documents.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.\n\n>Q: Lack of Novelty. The method appears to be simply an integration of a poisoning attack within RAG, where the attacker-defined query consistently results in a high-ranking retrieval of the poisoned document.\n\n**R**: The novelty of our work lies in the following aspects:\n- We systematically investigate and comprehend the robustness of retrieval systems employed in RAG, by introducing a new attack. We achieve this by injecting various types of information, encompassing both irrelevant and relevant content, into the corpus and evaluating the ease or difficulty of retrieval. \n- We propose the `orthogonal augmentation` property to explain the wide-spread success of the attack, and provide empirical evidence to support this property.\n- We propose a new defense based on the `orthogonal augmentation` property, which can effectively filter out the adversarial documents.\n\nGiven the relatively new development of safety in RAG, e.g., poisoning attacks, we do believe the above listed contributions are novel and valuable. We will clarify this in the revised version.\n\n>Q: Although the paper partly relaxes the assumption by showing robustness under paraphrasing, the attack still relies on knowing the typical structure of medical queries.\n\n**R**:  We have included additional experiments on the *legal domain* in Table 10 in Section D of the supplementary material. Overall, we observe similar attack success rates in the legal domain as in the medical domain presented in the main text. Therefore, we believe that the proposed attacks can be generalized to a wide range of domain applications. We will clarify this in the revision.\n\n>Q: The proposed detection method based on Mahalanobis distance with covariance shrinkage appears effective empirically; however, its performance may be highly sensitive to the choice of the shrinkage parameter (β) and the quality of the anchor set.\n\n**R**:  We have included an ablation study on the selection of $\\beta$ in Table 8 in Section C of the supplementary material. We observed that the detection performance remains stable across a wide range of $\\beta$ selections. Regarding the anchor set, if the anchor set is non-informative, it becomes theoretically infeasible to implement any effective defenses. We will include more discussions and empirical results in the main text.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "The paper proposes a poisoning attack for RAG.", "essential_references_not_discussed": "Yes", "other_strengths_and_weaknesses": "Strength\n- The paper highlights a crucial security issue in retrieval-based systems, especially in sensitive fields like healthcare and legal AI applications, where misinformation or adversarial manipulation can have serious consequences.\n- The work presents an insightful observation about how dense retrieval models process concatenated adversarial text, offering a new perspective on why poisoning attacks succeed.\n- The paper explores variations of the attack, including paraphrased queries, showing that the attack remains effective even when exact query matches are unavailable.\n\n\nWeakness\n- Lack of Novelty. The method appears to be simply an integration of a poisoning attack within RAG, where the attacker-defined query consistently results in a high-ranking retrieval of the poisoned document.\n- Although the paper partly relaxes the assumption by showing robustness under paraphrasing, the attack still relies on knowing the typical structure of medical queries.\n- The proposed detection method based on Mahalanobis distance with covariance shrinkage appears effective empirically; however, its performance may be highly sensitive to the choice of the shrinkage parameter (β) and the quality of the anchor set.", "other_comments_or_suggestions": "See the Weakness", "questions_for_authors": "See the Weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sKQIHITnWg", "reviewer": "Reviewer_JLUo", "summary": "This paper explores a characteristic of poisoned documents in embedding spaces termed the orthogonal augmentation property. It suggests that appending target information to a poisoned document containing the target query shifts its embedding orthogonally to the query, preserving its retrievability by the query. The authors analyze how this property enables certain attacks and propose a corresponding defense.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.\n\n>Q: Universal poisoning attacks against RAG have already been shown to be effective [1] ...\n\n**R**: Thank you for your valuable suggestions regarding the work in [1]. Overall, our work differs from [1] in terms of goals, insights, and application scenarios, despite the similarity in attack implementations. Detailed discussions are provided as follows.\n\n- **Goal**. Our objective is to investigate and comprehend the robustness of retrieval systems employed in RAG. We achieve this by injecting various types of information, encompassing both irrelevant and relevant content, into the corpus and evaluating the ease or difficulty of retrieval. The goal of [1], on the other hand, is to inject only query-relevant context to deceive the LLM's generation process upon retrieval.\n- **Insights**. We provide explanations on the difficulty/easiness of the retrieval of different kinds of information, which is not covered in the work of [1]. Moreover, based on the developed insights, we propose a new defense that can effectively filter out the adversarial documents generated by [1].\n\n>Q: The orthogonal augmentation property essentially demonstrates a simple fact ...\n\n**R**: The `orthogonal augmentation` property concerns the effects on **changes in embedding vectors** generated by text embedding models when manipulating the text input. Specifically, `orthogonal augmentation` examines the relationship between $f(q)$ and the change in the embedding space that occurs when shifting \n$q$ to $q \\oplus p$, namely $v \\triangleq f(q \\oplus p) - f(q)$, for two documents $p$ and $q$, which can be either semantically relevant or irrelevant (as discussed in Section 4).\n\nOverall, we feel that it is essential to understand the `orthogonal augmentation` property, as it can be seen in many poisoning attacks against text embedding models. We tried to investigate this property both in theoretical and empirical ways. It turns out the current literature on the theoretical understanding of transformer-based models seems to be sparse and preliminary. For example, recent research, such as the work [A] published in NeurIPS 2023, studied the weight dynamics of transformers under strong assumptions which may not align with real-world use cases, such as single-layer self-attention, no positional encoding, and excessively long input contexts. As a result, in the paper, we present the empirical study of the `orthogonal augmentation` property. \n\n>Q: The proposed defense requires access to a collection of clean documents ...\n\n**R**: We feel that the assumption is reasonable in the following sense. First, it is straightforward (and can be theoretically proven) to see that the defender is impossible to defend all corpus/queries (Because that would make the whole input space to be clean). As a result, a more feasible solution is to find a set of important queries and their corresponding corpus and then prioritize protecting these selected ones. In fact, many RAG poisoning attacks are targeted attacks in the sense that the attacker only wants to poison/manipulate a small set of queries [2,3,4]. So we believe that such an assumption is reasonable. We will clarify this in the revised version.\n\n\n>Q: The paper focuses only on dense retrievers, leaving it unclear how the findings extend to models like ColBERT or retrieval architectures incorporating cross-encoder re-rankers ...\n\n**R**: The MedCPT retriever used in our paper is actually of the cross-encoder re-rankers architecture tuned on medical data (https://github.com/ncbi/MedCPT). As a result, we think the observations and findings are applicable to the cross-encoder re-rankers. We also include the results of close-sourced text-embedding-3 from OpenAI in the Table below. We observe that the attack success rates are similar to those of the state-of-the-art models. We will include this in the revised version.\n\nTable. Top 2 retrieval success rates with text-embedding-3 as the retriever and MedMCQA, PubMedQA as the query corpus.\n| Corpus | Attack Success Rate |\n|--------------------|---------------------|\n| Textbook     | 0.95                |\n| StatPearls | 0.90                |\n| PubMed       | 0.83                |\n\nRefs:\n[A] Tian et al., \"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer\", NeurIPS 2023.\n\n[2] Xiong et al., \"Benchmarking Retrieval-Augmented Generation for Medicine\", ACL 2024\n\n[3] Miao et al., \"Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications\", Medicina (Kaunas). 2024\n\n[4] Zou et al., \"PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models\", USENIX Security, 2025", "claims_and_evidence": "The claim that dense retrievers are vulnerable to universal poisoning attacks is well-supported by extensive experiments. The Orthogonal Augmentation Property claims are also backed by experiments, and the proposed defense is evaluated empirically. However, I have some concerns about the experimental results related to the Orthogonal Augmentation Property and the defense's effectiveness (see Weaknesses).", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experiments on medical retrieval systems appear sound.", "relation_to_broader_scientific_literature": "The paper tries to explain the success of poisoning attacks against RAG by analyzing the behavior of poisoned documents in dense retrieval embedding spaces and provides experimental insights. It also proposes a new defense against a specific type of such attack.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "### Strengths:  \n1. Provides some insights into the behavior of poisoned documents in embedding spaces.  \n2. Conducts comprehensive experiments on different medical datasets and retrievers to evaluate the retrieval of poisoned documents.  \n\n### Weaknesses:  \n1. Universal poisoning attacks against RAG have already been shown to be effective [1]. The paper dedicates significant space to revalidating this on medical retrieval systems, which seems of limited value.  \n2. The orthogonal augmentation property essentially demonstrates a simple fact: a poisoned document containing the target query is naturally more retrievable than a clean document by the target query. The necessity of proving this through a convoluted approach is unclear.  \n3. The proposed defense requires access to a collection of clean documents and prior knowledge of the target queries, which is a strong assumption, making it impractical in real-world scenarios.  \n4. The paper focuses only on dense retrievers, leaving it unclear   how the findings extend to models like ColBERT or retrieval architectures incorporating cross-encoder re-rankers, which are widely used.  \n\n[1] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models,  USENIX Security'25.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. It is common practice to append potential queries to documents to improve retrievability, a technique also known as Doc2Query [1]. How would the findings of this paper and the proposed defense apply to retrieval systems that incorporate Doc2Query?\n\n[1] Document expansion by query prediction, 2019.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PN9kamzBxR", "reviewer": "Reviewer_5ynv", "summary": "This paper focuses on the adversarial robustness of the retrieval system of RAG against data poisoning attacks.\nThree major safety risks are discussed, including the leakage of PII, adversarial recommendations, and the vulnerability to jailbreaking attacks. \nExtensive experiments on five Medical QA datasets demonstrate the prevalence of such risks, i.e., the retrieval systems used for medical QA are universally vulnerable.\nThis paper also discusses the possible reason for such risk and proposes a new defense to mitigate them.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.\n\n>Q: In Lines 105 (left) and 150 (right), the notation refers to the retrievers and the embedding function, respectively. Could the authors please provide some explanation regarding the definitions of the retrievers?\n\n**R**: In our paper, we use the terms \"retriever\" and \"embedding function\" interchangeably, which take an input text and output a vector representation of the text. We will clarify this point to avoid confusion in the revised version.\n\n>Q: Besides, the results in Sections 3-5 heavily rely on the embedding function of the input. However, my past experience implies that the performance of those embedding models released before 2024, especially those token-level models like BERT, is far from satisfactory. As mentioned in the \"Experimental Designs Or Analyses\" part, I suppose the retrievers used in this paper are slightly out of date. (P.S. I am unfamiliar with RAG's research, and thus, I cannot provide exact references.) Are there any retrieval systems that are based on SOTA embedding models, e.g., text-embedding-3 from OpenAI?\n\n**R**: The three retrievers used in the main text are the state-of-the-art models for RAG following very recent literature [1,2,3]. To address your concern, we also include the results of the text-embedding-3 from OpenAI in the Table below. We observe that the attack success rates are similar to those of the state-of-the-art models. We will include this in the revised version.\n\nTable. Top 2 retrieval success rates with text-embedding-3 as the retriever and MedMCQA, PubMedQA as the query corpus.\n| Corpus | Attack Success Rate |\n|--------------------|---------------------|\n| Textbook     | 0.95                |\n| StatPearls | 0.90                |\n| PubMed       | 0.83                |\n\n>Q: I suggest including more illustrative examples to further improve the presentation of this paper. The examples given in Figure 1 seem artificial. In Section A, some examples from the real dataset are presented. I suppose presenting some real-world examples can better illustrate the safety risk faced by the RAG systems.\n\n**R**: We will follow your suggestions to replace the current Figure 1 and include some real-world examples in the main text to better represent the safety risks faced by RAG systems.\n\nRefs: [1] Xiong et al., \"Benchmarking Retrieval-Augmented Generation for Medicine\", ACL 2024\n\n[2] Miao et al., \"Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications\", Medicina (Kaunas). 2024\n\n[3] Zou et al., \"PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models\", USENIX Security, 2025", "claims_and_evidence": "According to Section 1.1, the main claims include: \n1. Revealing the safety risks for the retrieval system. Three safety risks are mentioned, including the leakage of PII, adversarial recommendations, and the vulnerability to jailbreaking attacks. \n2. Providing an explanation for the vulnerability of retrieval system to data poisoning attacks.\n3. Proposing a new defense method against universal poisoning attack.\nAll the claims are well supported by experiments.", "methods_and_evaluation_criteria": "The experimental results of this paper evaluate and reveal the safety risk of the existing retrieval systems. The methods and evaluation criteria make sense for the application.", "theoretical_claims": "This paper does not provide theoretical analysis.", "experimental_designs_or_analyses": "I have checked the soundness/validity of the experimental designs (regarding the main claim of this paper) in Section 3. My main concern is that the retrievers mentioned in Lines 185-192 (right) are slightly out-of-date.", "relation_to_broader_scientific_literature": "Section 1.2 has comprehensively discussed the related study of this paper.", "essential_references_not_discussed": "I am not aware of such related works.", "other_strengths_and_weaknesses": "1. This paper is well-organized and carefully written. The seemingly unrealistic assumptions are explained in the remarks. The preliminary section is very helpful for readers unfamiliar with the topic.", "other_comments_or_suggestions": "1. I suggest including more illustrative examples to further improve the presentation of this paper. The examples given in Figure 1 seem artificial. In Section A, some examples from the real dataset are presented. I suppose presenting some real-world examples can better illustrate the safety risk faced by the RAG systems.", "questions_for_authors": "1. In Lines 105 (left) and 150 (right), the notation $f$ refers to the retrievers and the embedding function, respectively. Could the authors please provide some explanation regarding the definitions of the retrievers? \n2. Besides, the results in Sections 3-5 heavily rely on the embedding function of the input. However, my past experience implies that the performance of those embedding models released before 2024, especially those token-level models like BERT, is far from satisfactory. As mentioned in the \"Experimental Designs Or Analyses\" part, I suppose the retrievers used in this paper are slightly out of date. (P.S. I am unfamiliar with RAG's research, and thus, I cannot provide extract references.) Are there any retrieval systems that are based on SOTA embedding models, e.g., text-embedding-3 from OpenAI?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JQqOKOUxUC", "reviewer": "Reviewer_x5AF", "summary": "This paper investigates the vulnerabilities of retrieval systems to various poisoning attacks. The authors first analyze multiple corpora, retrievers, and datasets, highlighting the significant safety risks in retrieval. They then attribute retriever failures to the limitations of the existing document embedding distance metric. Finally, they propose a new metric that more effectively differentiates between clean and poisoned documents.\n\n## update after rebuttal\nThank you for your response and the insightful experiments. The authors have addressed most of my concerns, and I have accordingly increased my original score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.\n\n> Q: additional retrieval cases ..., especially in the legal domain.\n\n**R**: We actually included the experiments on the legal domain in *Table 10 in Section D in the supplementary material*. Overall, we observe similar attack success rates in the legal domain as in the medical domain presented in the main text. We will highlight this in the revised version.\n\n> Q: l2 norm baseline relatively simple \n\n**R**: $\\ell_2$-norm is a simple but also the standard baseline in many existing literature [1,2,3]. To address your concern, following [1,2,3], we also include the perplexity filter, which examines the perplexity of the text, as an additional baseline. The results are shown in the Table below where we observe that the perplexity filter is not effective, thus validating the effectiveness of our method.\n\nTable: Detection rate of the perplexity filter on the MedMCQA corpus. The results are averaged over 5 runs.\n| Datasets |  MMLU-Med| MedQA | BioASQ|\n|--------------------|---------------------|---------------------|---------------------|\n| Detection Rates |           0.09   | 0.14| 0.16 |\n\n\n> Q: application efficiency be improved for large-scale data and real-time systems? \n\n**R**: We are not 100% certain regarding what you mean by \"application efficiency.\" So we will refer to the computation cost/efficiency of our method and respond accordingly. The computation of our defense method is very light. We only need to compute the covariance **offline once**, and then during inference time, we only need to calculate the Mahalanobis distance **once** for each query, which is very efficient. To further improve efficiency, we can take a batch of queries and compute the Mahalanobis distance for all queries in the batch simultaneously.\n\n> Q: cosine similarity\n\n**R**: The use of inner product is a common practice in current literature [1,2,3]. To address your concern, we also evaluated cosine similarity and the summarized results are shown in the table below. We observe similar high attack success rates when using cosine similarity. We will include this in the revised version.\n\nTable: Top 2 retrieval success rates under cosine similarity with Contriever as the retriever and MedMCQA, PubMedQA as the query corpus.\n| Corpus | Attack Success Rate |\n|--------------------|---------------------|\n| Textbook     | 0.95                |\n| StatPearls | 0.94                |\n| PubMed       | 0.90                |\n\n\n> Q: The description of the defense is not clearly explained, easy to make confuse to reader.\n\n**R**: We will revise the description of the defense in the main text to make it clearer. The main workflow of our defense is for the defender to select a set of clean corpus (corresponding to a set of queries) to be protected. Then, the defender computes the covariance of the embeddings of the clean corpus. During inference time, for each query, the defender computes the Mahalanobis distance between the query and the clean corpus. If the distance is larger than a threshold, the query is rejected.\n\n> Q: How about top5, top10 retrieval performance\n\n**R**: The retrieval rates reported in the main text are a non-decreasing function of the $k$ value used in the retrieval. So the top-5 and top-10 retrieval performance will be no worse than the top-2 retrieval performance. In particular, we report the results for $k=10$, where we observe near perfect retrieval rates. We will include this in the revised version.\n\nTable: Top-10 retrieval success rates with Contriever as the retriever and MedMCQA, PubMedQA as the query corpus.\n| Corpus | Attack Success Rate |\n|--------------------|---------------------|\n| Textbook     | 0.99               |\n| StatPearls | 0.98                |\n| PubMed       | 0.94               |\n\n> Q: More related work and flag in the abstract for potential harmful information\n\n**R**: We will include more related work as you suggested. We will also add a flag in the abstract to indicate that the paper contains potentially harmful information.\n\n> Q: Math explanation\n\nProviding rigorous mathematical justification is challenging due to the sparse, preliminary theoretical understanding of transformer-based models [4]. We aim to address this in future work.\n\n\nRefs:\n[1] Xiong et al., \"Benchmarking Retrieval-Augmented Generation for Medicine\", ACL 2024\n\n[2] Miao et al., \"Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications\", Medicina (Kaunas). 2024\n\n[3] Zou et al., \"PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models\", USENIX Security, 2025\n\n[4] Tian et al., \"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer\", NeurIPS 2023.", "claims_and_evidence": "1. The retrieval of safety risks in health and legal document retrieval systems includes detailed experiments, but the number of retrieved documents remains limited.\n2. The observation that poisoned documents exhibit orthogonal augmentation with their corresponding clean queries is interesting.\n3. The proposed new defense method is well-supported by evidence.", "methods_and_evaluation_criteria": "The proposed defense method and analysis of retrieval system vulnerabilities in RAG are reasonable. However, I believe the author could incorporate additional evaluation metrics, such as the MRR score, to enhance the assessment.", "theoretical_claims": "There is no theoretical claim in paper.", "experimental_designs_or_analyses": "1. You should include additional retrieval cases to better demonstrate the effectiveness of your method, especially in the legal domain.\n2. The paper uses only the l2-norm-based defense as a baseline for evaluating the proposed method. This baseline is relatively simple, making the comparison results less convincing.\n3. Balancing performance and efficiency: While ensuring the effectiveness of the defense method, how can its application efficiency be improved for large-scale data and real-time systems? Is there room for further optimization to meet practical performance requirements?\n4. Why not use cosine similarity? Have the results under cosine similarity been evaluated?", "relation_to_broader_scientific_literature": "It proposes a detection method to enhance the defense against attacks on RAG systems.", "essential_references_not_discussed": "There need more realted work for attack on RAG system, such as HijackRAG[Zhang'24], AgentPoison[Chen'24], BadRAG[Xu'24]", "other_strengths_and_weaknesses": "1. The description of the defense is not clearly explained, easy to make confuse to reader.\n2. How about top5, top10 retrieval performance", "other_comments_or_suggestions": "There are no obvious typos.", "questions_for_authors": "The concept of orthogonal augmentation is intriguing, but it appears to be influenced by the principles of modern Hopfield networks. When the model’s query $q$ and the memory document \n$p$ are orthogonal, the energy function is significantly lower, making knowledge retrieval more efficient. Could you provide a mathematical explanation for why orthogonal augmentation enhances retrieval? Is it because the target attack document embedding is injected into the original query $q$, thereby modifying its representation? This interpretation seems reasonable to me.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review.", "ethics_expertise_needed": ["Privacy and Security"]}}]

Author rebuttal:

Meta-review: This paper presents a thorough empirical investigation into the vulnerability of retrieval systems in RAG under universal poisoning attacks, with valuable insights such as the orthogonal augmentation property and a lightweight detection defense. While some concerns remain regarding novelty and real-world assumptions, the overall contribution is timely and relevant to the safety of knowledge-intensive applications. I recommend a weak accept.
Decision: Accept (poster)