Corpus Poisoning via Approximate Greedy Gradient Descent (2025)

Abstract: Dense retrievers are widely used in information retrieval and have also been successfully extended to other knowledge intensive areas such as language models, e.g., Retrieval-Augmented Generation (RAG) systems. Unfortunately, they have recently been shown to be vulnerable to corpus poisoning attacks in which a malicious user injects a small fraction of adversarial passages into the retrieval corpus to trick the system into returning these passages among the top-ranked results for a broad set of user queries. Further study is needed to understand the extent to which these attacks could limit the deployment of dense retrievers in real-world applications. In this work, we propose Approximate Greedy Gradient Descent (AGGD), a new attack on dense retrieval systems based on the widely used HotFlip method for efficiently generating adversarial passages. We demonstrate that AGGD can select a higher quality set of token-level perturbations than HotFlip by replacing its random token sampling with a more structured search. Experimentally, we show that our method achieves a high attack success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains. Notably, our method is extremely effective in attacking the ANCE retrieval model, achieving attack success rates that are 15.24% and 17.44% higher on the NQ and MS MARCO datasets, respectively, compared to HotFlip. Additionally, we demonstrate AGGD’s potential to replace HotFlip in other adversarial attacks, such as knowledge poisoning of RAG systems.

## introduction dense retrievers, despite their wide application and extensive deployment in real-world systems @xcite @xcite @xcite , have recently been shown to be vulnerable to various adversarial attacks such as corpus poisoning attacks @xcite and data poison attacks @xcite , raising concerns about their security. given that the corpora used in retrieval systems are often sourced from openly accessible platforms like wikipedia and reddit, a concerning scenario arises in which malicious actors can poison the retrieval corpus by injecting some adversarial passages, fooling the system into retrieving these malicious documents rather than the most relevant ones. such attacks might be used for search engine optimization @xcite for promoting advertisement, or disseminating disinformation and hate speech. a conventional approach for such attacks is hot-flip @xcite , which involves collecting a candidate set for a single randomly sampled token position and finding the best token in the candidate set with which to replace. in addition to corpus poisoning attacks on dense retrieval systems, hotflip has been widely used in many other settings, such as knowledge poisoning attacks on retrieval augmented generation (rag) systems @xcite and adversarial prompt generation @xcite . in this work, we begin by thoroughly investigating the hotflip attack on dense retrieval systems to identify its limitations. based on these insights, we propose a new general attack method called approximate greedy gradient descent (aggd). our experimental results show that aggd can perform corpus poisoning attacks on dense retrieval systems more effectively, revealing their vulnerability. though we use corpus attacks on dense retrievers as our primary example, it is important to note that aggd can replace hot flip as a whole in any attack scenarios where hotflip is applicable. the main difference between aggd and hot flip is that aggd uses gradient information more effectively by selecting the top-ranked token from all token positions, rather than over a single randomly sampled position. this approach makes 2 probability that it will sample the same token position again and redo the same evaluation, which is inefficient. moreover, if the potential replacements for another token also don't contain a better option, hotflip continues to loop through the same search without reducing the loss. aggd's search trajectory deterministic, enabling a more structured best-first search. experimental results demonstrate that aggd achieves a high attack success rate across various datasets and retrieval models. in summary, our contributions are ## related work dense retrieval dense retrievers utilize dense vector representations to capture the semantic informa-tion of passages and have demonstrated tremendous effectiveness compared to traditional retrieval systems @xcite . consequently, they have been employed in many knowledge-intensive areas such as information retrieval @xcite @xcite @xcite , open-domain question answering and language model pre-training @xcite @xcite . for instance, retrieval-augmented generation (rag) models @xcite @xcite combine language models with a retriever component to generate more diverse, factual and specific content. ## motivation in this section we motivate our approach by formalizing the corpus poisoning problem setting (section 3.1), describing and analyzing the standard hotflip approach for producing adversarial passages @xcite (section 3.2), and identi-fying a potential problem with this approach (section 3.3). ## approximate greedy gradient descent similar to other gradient-based search algorithms, we fist initialize the adversarial passage to be @xmath0, • • • , t m ]. we then iteratively update a based on the best candidate that maximizes the similarity over batches of queries. formally, at each step, we compute a first-order approximation of the change in the loss when swapping the ith token in a with another token @xmath1. in contrast to hotflip, aggd identifies a candidate set for each of the m tokens in a: for each token i, the candidate set contains the top @xmath2 retrievers in our main experiments, we conduct attacks on 5 state-of-the-art retrieval models: contriever, contriever-ms (contriever fine-tuned on ms marco) @xcite , dprnq (trained on nq), dpr-mul (trained on multiple datasets) @xcite and ance @xcite . evaluation metrics after generating the adversarial passages on the training set and injecting them into the corpus, we evaluate the effectiveness of our attack using the top-k attack success rate (asr) on test queries. top-k asr is defined as the percentage of queries for which at least one adversarial passage is retrieved in the top-k r results, i.e., @xmath3, k r , c test )}, where n q is the total number of test queries and r r (q test i , k r , c test ) is the retriever that returns the top-k r most relevant passages for the test query q test i . 1{@xmath4, k r , c test )} is the indicator function, which equals to 1 if @xmath5, k r , c test ) and 0 otherwise. a higher asr indicates that the model is more vulnerable to attacks, and thus, the attack is more effective. we use k @xmath6. since asr depends on the size the the test corpus, to make fair comparisons across different dataset, we randomly sample |c test | = 10, 000 passages from the overall corpus pool. hyperparameters for all baselines, we use adversarial passages of @xmath7. we fix the candidate set size at @xmath8. all the experiments are conducted on nvidia a100 gpu (with total memory 40g).