Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack (2025)

Abstract: Retrieval-augmented generation (RAG) systems enhance large language models by incorporating external knowledge, addressing issues like outdated internal knowledge and hallucination. However, their reliance on external knowledge bases makes them vulnerable to corpus poisoning attacks, where adversarial passages can be injected to manipulate retrieval results. Existing methods for crafting such passages, such as random token replacement or training inversion models, are often slow and computationally expensive, requiring either access to retriever’s gradients or large computational resources. To address these limitations, we propose Dynamic Importance-Guided Genetic Algorithm (DIGA), an efficient black-box method that leverages two key properties of retrievers: insensitivity to token order and bias towards influential tokens. By focusing on these characteristics, DIGA dynamically adjusts its genetic operations to generate effective adversarial passages with significantly reduced time and memory usage. Our experimental evaluation shows that DIGA achieves superior efficiency and scalability compared to existing methods, while maintaining comparable or better attack success rates across multiple datasets.

## introduction large language models (llms) @xcite @xcite have shown impressive performance across a wide range of natural language processing tasks, but they still suffer from significant limitations, such as hallucination @xcite and outdated internal knowledge. to address these issues, retrieval-augmented generation (rag) @xcite systems have emerged as a promising solution by incorporating external, up-to-date knowledge into the generation process. by retrieving relevant information from large external corpora, rag systems can enhance the accuracy and relevance of llm-generated outputs, especially in open-domain question answering and knowledge-intensive tasks. despite the benefits, the reliance on external knowledge sources exposes rag systems to potential security vulnerabilities @xcite @xcite . a particular focus has been on adversarial attacks targeting rag systems. initial studies @xcite @xcite explored attacks where adversarial passages are injected into the knowledge base to influence the language model's output for specific queries. building upon this, @xcite introduced a more potent form of attack termed corpus poisoning, where a single adversarial passage aims to affect retrieval results across multiple queries simultaneously. currently, two primary approaches dominate corpus poisoning attack implementations. the first, based on hotflip @xcite , initializes an adversarial passage with random tokens and iteratively optimizes it using gradients from the retriever. the second method, vec2text @xcite , originally developed to study text reconstruction from embeddings, has been repurposed for corpus poisoning by inverting the embeddings of query centroids. however, these existing methods face significant limitations: (1) white-box access: hotflip requires access to the retriever model's gradient, limiting its applicability in real-world scenarios where such access is often restricted. (2) lack of generalizability: vec2text requires training a separate inversion model on a specific dataset. this limits its applicability to new domains or datasets without retraining, as the model may struggle to reconstruct embeddings for data significantly different from its training set. (3) high computational demands: both methods require substantial computational resources, making them impractical for large-scale attacks. hotflip's iterative optimization process is time-consuming and memory-intensive. vec2text, while faster in generation, requires additional memory for its inversion model and is computationally expensive during the training phase. these resource constraints severely limit the scalability of both approaches. given the limitations of existing methods, we posit a crucial question: is it possible to perform corpus poisoning attacks under minimal assumptions about the target system while maintaining high effectiveness and efficiency? in response, we propose the dynamic importance-guided genetic algorithm (diga). our method is built upon the foundation of genetic algorithms, a class of optimization techniques inspired by the process of natural selection. diga incorporates two key properties of retrieval systems into its evolutionary process: their insensitivity to token order and bias towards influential tokens. by leveraging these properties, diga dynamically adjusts its genetic operations, such as population initialization and mutation, to craft effective adversarial passages. this approach allows us to generate potent adversarial content without requiring white-box access to the retriever or extensive computational resources (see our empirical evaluation demonstrates that diga achieves comparable or superior results to existing methods across various datasets and retrievers. notably, diga accomplishes this with significantly lower time and memory usage, making it more suitable for large-scale attacks. furthermore, our method exhibits stronger transferability across different datasets compared with another black-box method. this enhanced transferability is significant as it demonstrates diga's ability to generate more generalizable adversarial passages, potentially making it more effective in real-world scenarios where attack and target datasets may differ. our study underscores the vulnerabilities present in rag systems and the importance of continued research into their security. future efforts should prioritize developing robust defenses and evaluating real-world risks. we hope this work inspires further advancements in safeguarding rag systems. ## related work attacks on rag systems and corpus poisoning. recent research has explored vulnerabilities in retrieval augmented generation (rag) systems and dense retrievers. membership inference attacks @xcite aim to determine the presence of specific data in knowledge bases. other studies focus on manipulating system outputs, such as poisonedrag @xcite crafting misinformation-laden passages, and indirect prompt injection attacks @xcite inserting malicious instructions. in corpus poisoning, @xcite proposed a gradientbased approach inspired by hotflip @xcite , while vec2text @xcite ) introduced text embedding inversion. unlike poi-sonedrag, which targets individual queries, our work aims to craft adversarial passages affecting multiple queries simultaneously, presenting unique challenges in corpus poisoning attacks on rag systems. genetic algorithms in nlp. genetic algorithms (ga) have been increasingly applied to various natural language processing (nlp) tasks. recent studies have demonstrated ga's efficacy in neural architecture search @xcite , adversarial prompt generation for llm security @xcite , and enhancing ga operations using llms @xcite @xcite . these applications showcase the versatility of ga in addressing complex nlp challenges. garag @xcite shares similarities with our work in applying ga to rag systems. however, our approach differs significantly by targeting multiple queries simultaneously, rather than focusing on query-specific adversarial passage generation. furthermore, our method employs a dynamic mechanism specifically tailored to exploit the properties of dense retrievers, offering a more nuanced approach to the challenge of corpus poisoning in rag systems. ## dynamic importance-guided genetic algorithm (diga) we propose dynamic importance-guided genetic algorithm (diga), which is designed to efficiently craft adversarial passages by exploiting two key properties of retrievers: their insensitivity to token order and bias towards influential tokens. diga employs a genetic algorithm framework, dynamically adjusting its genetic policies based on token importance. figure 2 illustrates the pipeline of our method, which consists of several key components. we will now give a detailed explanation of these components in our proposed method. we provide an algorithmic description of diga in algorithm 1. ## conclusion in this work, we present dynamic importance-guided genetic algorithm (diga), a novel method for executing corpus poisoning attacks. diga capitalizes on the insensitivity of retrievers to word order and their bias towards influential tokens, enabling it to generate adversarial passages that are both effective and efficient. notably, our approach requires no white-box access to the underlying models and avoids the need for additional training, making it highly adaptable. our extensive experiments demonstrate that diga surpasses existing methods in terms of efficiency and scalability. at the same time, it consistently achieves comparable or superior attack success rates across various datasets and retrieval models, highlighting its effectiveness and adaptability in adversarial scenarios.