Ward: Provable RAG Dataset Inference via LLM Watermarks

Authors: Nikola Jovanović, Robin Staab, Maximilian Baader, Martin Vechev
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: llm, watermarks, dataset inference, rag

Abstract:
RAG enables LLMs to easily incorporate external data, raising concerns for data owners regarding unauthorized usage of their content. The challenge of detecting such unauthorized usage remains underexplored, with datasets and methods from adjacent fields being ill-suited for its study. We take several steps to bridge this gap. First, we formalize this problem as (black-box) RAG Dataset Inference (RAG-DI). We then introduce a novel dataset designed for realistic benchmarking of RAG-DI methods, alongside a set of baselines. Finally, we propose Ward, a method for RAG-DI based on LLM watermarks that equips data owners with rigorous statistical guarantees regarding their dataset's misuse in RAG corpora. Ward consistently outperforms all baselines, achieving higher accuracy, superior query efficiency and robustness. Our work provides a foundation for future studies of RAG-DI and highlights LLM watermarks as a promising approach to this problem.

Decision: accept (poster)
Reviewer scores: 8, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730243166422

thread_type:
official_review

<thread_object>
creator: reviewer_nae4

<content>
summary:
This paper tackles the issue of unauthorized data use in RAG systems. Noting the current absence of tools for detecting such misuse, the authors propose "RAG Dataset Inference" (RAG-DI), a framework that empowers data owners to identify unauthorized data utilization in RAG systems. To facilitate research on RAG-DI, they developed FARAD, a synthetic dataset generated by leveraging an existing dataset and LLMs. FARAD is crafted to simulate challenging scenarios where similar content may appear across multiple sources, enhancing its applicability in real-world settings.

Furthermore, the authors introduce WARD, a novel method that leverages an existing watermarking algorithm to enable data owners to statistically verify whether their data has been incorporated into an RAG system's corpus through black-box queries. Experimental results demonstrate that WARD consistently outperforms existing methods, providing remarkable accuracy, robustness, and efficiency. This approach offers a reliable mechanism for detecting unauthorized data use and establishes essential resources and methodologies to guide future research on data privacy in RAG systems.

strengths:
1. Although RAG has gained popularity, its implications for data privacy and unauthorized usage detection remain largely unexamined. This paper introduces the RAG Dataset Inference (RAG-DI) concept alongside a watermark-based detection approach, offering a novel perspective on privacy within RAG frameworks. By developing the FARAD dataset and the watermark-based WARD method, this work establishes a strong foundation for future research. It stands as a pioneering contribution to addressing privacy concerns in RAG systems.

2. The paper offers comprehensive experimentation and empirical evaluation to validate WARD. The paper demonstrates WARD's outstanding performance across challenging scenarios by benchmarking WARD against multiple baselines, including adaptations from related fields.

3. The paper is well-written and organized, with a clear structure that enhances readability. The authors present their work logically, making the flow of ideas easy to follow in each section.

weaknesses:
1. The authors claim their approach is effective in the challenging scenario where information is distributed across multiple sources. However, it remains unclear how the method performs when only one retrieved document is watermarked while others are unprotected. Additionally, they evaluate the method only with k=3; it is uncertain how effective it would be if the system retrieves more, say 10 relevant documents, with only one being protected. Without this analysis, the effectiveness of the proposed approach cannot be conclusively assessed, especially given that RAG systems can flexibly retrieve a variable number of relevant documents. Moreover, the "provable watermark" claim lacks empirical support; no rigorous evidence is provided to substantiate this assertion. While “provable” may reference the watermarking algorithm itself, scalability should also be addressed within this definition to strengthen the claim.

2. The authors claim their approach is provable; however, unlike existing watermarking algorithms, they do not provide a rigorous proof. This discrepancy undermines the study’s credibility.

3. The authors propose a relatively simple defense strategy that relies solely on prompt engineering. However, due to the complex requirements of the proposed defense, LLMs may struggle to adhere to instructions precisely, resulting in a suboptimal defense. A more practical and effective approach could involve monitoring the words used in the retrieved documents and limiting or penalizing the frequency of these words in generated responses. Additionally, after the response is generated, it could be further paraphrased to potentially reduce the usage of watermarking words, similar to techniques employed in existing watermarking approaches. Again, the issue is rooted in the absence of a theoretical guarantee.

questions:
1. To my understanding, generating watermarked text typically requires white-box access to LLMs to direct the generation process. Then, how can one produce watermarked documents using black-box models like GPT-4 and Claude 3.5-Sonnet? Could you elaborate on the methods or approaches that make this possible?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730714968715

thread_type:
official_review

<thread_object>
creator: reviewer_nepy

<content>
summary:
This paper focuses on the problem of unauthorized use of datasets in Retrieval-Augmented Generation (RAG) systems and formalizes it as the RAG Dataset Inference (RAG - DI) problem. To address this issue, the authors propose the FARAD dataset specifically designed for RAG - DI evaluation and a series of baseline methods. They also introduce the WARD method based on LLM watermarks. Experiments demonstrate that WARD outperforms the baseline methods in terms of accuracy, query efficiency, and robustness.

strengths:
1. The RAG - DI problem is formally defined, filling the research gap in this field and laying a foundation for subsequent research.
2.The design of the FARAD dataset takes into account the practical application scenarios of the RAG system and avoids the shortcomings of existing datasets in RAG - DI research, such as the possibility of being used for LLM training and the lack of fact redundancy.
3.The WARD method is based on LLM watermarks and can provide data owners with strict statistical guarantees regarding the use of their datasets in the RAG system. It exhibits excellent performance in various experimental settings and outperforms all baseline methods.

weaknesses:
1. In real-world applications, RAG systems may face more complex situations, such as multilingual environments and data from different domains. The discussion in this regard in the paper is relatively limited.
2. There is a lack of discussion on the efficiency of the solution.

questions:
1. For different types of LLMs, will the performance of the WARD method be affected? If so, how to adjust and optimize it?
2. In practical applications, the update frequency of data may be very high. What impact will this have on the effectiveness of the WARD method? How to deal with this situation?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730791867258

thread_type:
official_review

<thread_object>
creator: reviewer_whee

<content>
summary:
This paper presents WARD, a novel method for detecting unauthorized dataset usage in Retrieval-Augmented Generation (RAG) systems. The approach formalizes RAG Dataset Inference (RAG-DI) as a problem where data owners seek to identify if their data is being used without permission in RAG systems via black-box queries. The authors introduce a new dataset, FARAD, specifically designed to benchmark RAG-DI under realistic conditions that include fact redundancy. Using LLM watermarks, WARD offers data owners statistical guarantees of usage detection. Experimental results show that WARD outperforms other baselines, demonstrating high accuracy, query efficiency, and robustness across challenging scenarios.

strengths:
The strengths of this paper are mainly centered around the propose of the new task along with a new specialized benchmark dataset.

1. Novel Problem Definition and Formalization: The paper identifies and formalizes the novel problem of RAG Dataset Inference (RAG-DI), addressing a critical need for data owners to detect unauthorized data usage in RAG systems. This formalization fills a significant research gap and sets the stage for further exploration of secure data usage in RAG contexts.

2. Introduction of a Specialized Benchmark Dataset: By creating FARAD, the authors contribute a valuable dataset that is specifically tailored for RAG-DI research. FARAD’s design ensures realistic conditions for testing, including the incorporation of fact redundancy, making it highly relevant for practical evaluations of RAG-DI approaches.

3. Comprehensive Experimental Validation: The paper’s extensive experiments demonstrate the effectiveness of WARD in achieving high true positive rates with no false positives across various settings, underscoring the method's robustness and accuracy. This comprehensive validation strengthens the credibility of WARD as a reliable solution for detecting unauthorized data use in RAG systems.

weaknesses:
Although this paper has the above strengths that are interesting, I also noticed several weaknesses that should be further considered.

1. Application Scenario. The authors propose a novel problem definition, namely RAG Dataset Inference (RAG-DI). The concept is quite straightforward, and one has to admit that the concern indeed exists in practice. However, it is still less discussed in this paper, that how often this type of problem can occur in realistic scenarios. I am concerned with the application range of this work. For example, if a person is concerned with their own data being used for LLMs, do they need to test all LLMs in the market? This could be prohibitive. 

2. Lack of Computation Complexity. The authors proposed a novel method to deal with the specific problem of RAG-DI. However, the complexity is not fully discussed in the paper. As a result, the problem of scalability could be severe. For example, the owner has an excessively large dataset, will the method still work?

3. Countermeasures: Although the authors considered the scenario where the RAG provider attempts to prevent the unintended use of the system, they do not provide a detailed discussion of the possible countermeasures that the RAG provide may use to counteract the watermarking of data. As there are multiple existing techniques against watermarking, the discussion about vulnerability of the proposed framework when faced with these countermeasures could be inspiring

questions:
How can this framework be applied to realistic scenarios?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731334395927

thread_type:
official_review

<thread_object>
creator: reviewer_lfcg

<content>
summary:
This paper studies the problem of RAG Dataset Inference (RAG-DI), where a data owner aims to detect whether their dataset is included in a RAG corpus via black-box queries. The key difference from the RAG MIAs setup is that the data owner makes a dataset-level decision, instead of a document-level decision as in MIAs.
The authors first introduce a dataset FARAD for the RAG-DI problem, as they believe the previous datasets used in MIAs problems (EnronEmails, HealthcareMagic) are likely contaminated in the training datasets of existing LLMs and lack fact redundancy. The FARAD dataset builds on a recent source of articles with fictional entities and events, and they prompt a set of LLMs (author LLMs) to rewrite articles based on the key points from the source article.

The paper then adapts existing RAG MIAs baselines to this problem and proposes a simple baseline called FACTS that simply prompts an LLM to generate a single question that is only answerable by the document. They find that the FACTS baseline performs well in their easy setting (without fact redundancy) but not in their hard setting (with fact redundancy).

Finally, they propose a new dataset inference method based on LLM watermarking called Ward. The key idea is to paraphrase the articles with a watermarked model from the recent LLM watermarking literature. They show that Ward works well in their hard setting and also meets the desiderata including monotonicity, guarantees, and robustness.

strengths:
* I believe the paper studies an important and timely research question that has been overlooked in the literature. I am not a domain expert (in the sense of MIA and RAG privacy), so I may not know related work very well.
* I think the paper has made many meaningful contributions, from formulating the problem to collecting the dataset and adapting RAG-MIA baselines, to finally proposing their own approach based on LLM watermarking. The experiments (different LLMs and evaluation settings)  and analyses (hyperparameters, modeling retrieval, and examining the quality of paraphrased texts) are quite thorough too.
* Using LLM watermarking as an approach to solve dataset inference seems a novel and reasonable approach to me. 
* The paper is very well written and self-contained. It is easy to follow the structure of the paper and understand its contributions clearly. I have learned a lot from reading the paper.

weaknesses:
The major part I am not very sure about is whether the dataset construction and experimental settings can really reflect realistic RAG settings in real-world deployment, since the effectiveness of the baselines and proposed approach all depends on this setting. In particular:
* It assumes perfect retrieval (= only articles from the same source are used) in most experiments. Although the paper discusses a more practical retrieval setting in Section 5.3, I don't see any discussion of previous baselines. For the hard setting, the imperfect retrieval problem should be easier than the perfect retrieval settings (the distracting documents should be less relevant), so the baselines could probably perform better.
* I am not entirely sure whether the current data generation pipeline can well reflect the redundancy problem in the real world, given that they are all generated from different LLMs (I am also not sure how much impact "additional facts" and "a set of diverse LLMs" have here). It might be too much to ask, but I think having a realistic multi-document summarization setting would be more realistic.
* I have difficulty understanding the rationale behind the defense prompt. Why would RAG providers want to instruct the models to not answer questions from retrieved documents? How can RAG be effective in this case? If this is from the data owner's perspective, given the goal is to detect whether the dataset is used or not, I don't see why they would want to add such a prompt to make the detection task harder.
* [Question] The RAG uses k = 3 shots, but the Easy setting assumes there is only one article per group. Where do the 3 shots come from? The paper also mentions they ablate the choice of k, where k can be {3, 4, 5} in the Appendix. Even for the Hard setting, aren't there only 4 articles per group? Not sure if I misunderstood this.

I still think it is a good paper regardless, and I hope to get some clarifications on these questions. There is a chance that I just misunderstood some key parts of the paper.

[Minor] It is not necessarily a weakness, but the paper's title is "Ward:... Watermarks" while the paper really contains much more than that, including setting up the problem, establishing the dataset and baselines. It would probably be more appropriate to have a broader title around RAG Dataset Inference.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732196369201

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We thank the reviewers for their feedback and the positive evaluation of our work. We are pleased to hear that they unanimously consider our contributions important and meaningful, filling a research gap and establishing a strong foundation for future work (Lfcg, WHEe, nePY, naE4). We are glad that they also appreciate the novelty of our work and the thoroughness of our experimental evaluation (Lfcg, WHEe, naE4).

We have uploaded an updated version of the manuscript (new content marked in purple), and responded to all questions and concerns raised by the reviewers in individual responses below. We are happy to further engage in the discussion in case of follow-up questions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732555685424

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We thank the reviewers again for their initial feedback. As we have not yet received replies to our rebuttal and the discussion period is closing shortly, we would like to kindly ask the reviewers to let us know if our responses addressed their concerns, and raise any follow-up questions.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper studies the problem of unauthorized use of datastores in RAG frameworks. The paper makes the following contributions: (a) a dataset FARAD for the RAG-DI problem, (b) a method WARD that uses llm watermarking to identify dataset usage. This is a valuable contribution that formalizes a novel problem in this space, and provides a good benchmark for future work to build on.