Ward: Provable RAG Dataset Inference via LLM Watermarks
Authors: Nikola Jovanović
Robin Staab
Maximilian Baader
Martin Vechev
Venue: ICLR 2025 Poster (2025)
Keywords: llm
watermarks
dataset inference
rag
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We formalize RAG Dataset Inference, introduce a suitable dataset and baselines, and propose Ward, a rigorous method based on LLM watermarks.

Abstract: RAG enables LLMs to easily incorporate external data, raising concerns for data owners regarding unauthorized usage of their content. The challenge of detecting such unauthorized usage remains underexplored, with datasets and methods from adjacent fields being ill-suited for its study. We take several steps to bridge this gap. First, we formalize this problem as (black-box) RAG Dataset Inference (RAG-DI). We then introduce a novel dataset designed for realistic benchmarking of RAG-DI methods, alongside a set of baselines. Finally, we propose Ward, a method for RAG-DI based on LLM watermarks that equips data owners with rigorous statistical guarantees regarding their dataset's misuse in RAG corpora. Ward consistently outperforms all baselines, achieving higher accuracy, superior query efficiency and robustness. Our work provides a foundation for future studies of RAG-DI and highlights LLM watermarks as a promising approach to this problem.

Published as a conference paper at ICLR 2025
WARD: PROVABLE RAG DATASET INFERENCE VIA LLM WATERMARKS
Nikola Jovanović, Robin Staab, Maximilian Baader, Martin Vechev
ETH Zurich
(nikola. jovanovic, cobin.staabe mbaader, martin.vechhvvjlinfccom etthe
ABSTRACT
RAG enables L.LMs to easily incorporate external data, raising concerns for data owners regarding unauthorized usage of their content. The challenge of detecting such unauthorized usage remains underexplored, with datasets and methods from adjacent fields being ill-suited for its study. We take several steps to bridge this gap. First, we formalize this problem as (black-box) RAG Dataset Inference (RAG-DI). We then introduce novel dataset designed for realistic benchmarking of RAG- DI methods, alongside a set of baselines. Finally, we propose WARD, a method for RAG-DI based LLM watermarks that equips data owners with rigorous statistical guarantees regarding their dataset's misuse in RAG corpora. WARD consistently outperforms all baselines, achieving higher accuracy, superior query efficiency and robustness. Our work provides a foundation for future studies of RAG-DI and highlights LLM watermarks as a promising approach this problem.
INTRODUCTION
Retrieval-augmentel generation (RAG) has emerged popular approach mitigate limitations of large language models (LLMs)) such as hallucinations, the high cost of adapting to new knowledge via fine-tuning, and the inability back up claims by sources (Lewis et al., 2020). By integrating retrieval, LLMS gain in-context access to large corpora of high-quality, up-to-date data, enabling them to generate more accurate and supported responses. To maintain relevance, RAG providers must continuously update their corpus with new data. However, this raises concerns regarding the unauthorized usage of documents, particularly when publicly available documents are used without the owner's permission (Grynbaum & Mac, 2023; Wei et al., 2024a); see App. A for more elaborate discussion of this issue and prevalence in practice. Crucially, there is currently no way to conclusively prove such unauthorized usage by RAG system, and enforee opt-out by the owner.
RAG Dataset Inference (RAG-DI) We formalize this problem as RAG Dataset Inference (RAG- DI), where data owner aims to detect unauthorized inclusion of their dataset in a RAG corpus via black-box queries (Fig. 1). In the first comprehensive study of this problem, we observe that existing datasets, used in adjacent works on RAG privacy, not suitable for RAG-DI DI. First, the samples in these datasets may have been used in contemporary L.M training, complicating realistic evaluations where RAG corpora consist of new data. Second, these datasets do not model fact redundancy, key property of real-world RAG, where multiple documents have similar content, either due to scraping data from various sources, e.g., news (Gao al., 2023), or due chunking. Another challenge in studying RAG-DI stems from the lack of baselines applicable in realistic black-box setting.
Foundations for RAG-DI In this work. we take multiple steps to bridge these gaps: First, we introduce FARAD, new dataset specifically designed for RAG-DI evaluation under realistic condi- tions. FARAD contains fictional articles that are by design not part of any LLM training data, and can enable evaluations under fact redundancy, enabling accurate assessment of RAG-DI methods. Second, we adapt prior work on RAG Membership Inference Attacks (MIAs) (Li et al., 2024; Anderson et al., 2024) to the RAG-DI problem, and propose a simple baseline FACTS. In our evaluation on FARAD, we find that (i) despite its extreme simplicity, FACTS outperforms other baselines in settings with fact redundancy, further underscoring the drawbacks of existing datasets, and (ii) when fact redundancy present, no baseline achieves satisfactory performance. This highlights the need for novel approaches capable of reliably identifying unauthorized usage of documents in RAG corpora.

---

Published as a conference paper at ICLR 2025
Dataset Dio
RAG Corpus
LLM watermarking
RAG Provider
Data Waterrarkkng
Retrieval Mechanism R
RAG Dotoset inference
Data Owner
query Qi
Watermark Detector
ri
Ori
SF
4 + R(g D)
Inconclusive!
aggg-
My data used! Das C. Dip 10 12)
{ris P2r Tin
Figure 1: Overview of RAG Dataset Inference using WARD, our method based on LLM watermarks.
LLM Watermarks method that protects the data owner's dataset by imprinting LLM watermarks (Kirchenbauer et al., 2023; Kuditipudi et al., 2024). As Fig. illustrates, given limited number of black-box queries q to the retrieval- -uggeeted LLM, the data owner can detect even small traces of the watermark across responses Tr, and for the first time obtain rigorous statistical guarantees regarding the usage of their dataset in the RAG corpus, enabling them to effectively audit the RAG provider. our experiments on FARAD, we show that WARD consistently outperforms all baselines across variety of challenging settings, showing high rate of true positives (often 100%). with no false accusations. Further, due to its robustness across settings, and the fact that it engages only in natural-looking interactions with the retrieval-ane -uggeeeed LLM, WARD retains its performance even under attempts by the RAG provider prevent unintended uses of the system. These results underscore the effectiveness of WARD in enforcing data usage policies and protecting data owners' rights in the context of RAG systems.
reliable RAG-DI method To this end, we introduce WARD, RAG-DI
Main contributions We make the following key contributions:
We formalize previously unexplored problem, RAG Dataset Inference (RAG-DI), where data owner aims to detect unauthorized usage of their dataset a RAG system ($3).
We facilitate research on this problem by (i) proposing new dataset FARAD, specifically designed for benchmarking RAG-DI methods under realistic conditions, and (ii) introducing an initial set of baseline methods ($3.1 and $3.2).
We propose LLM watermarks way provably, robustly and reliably detect unauthorized data usage in RAG corpora, introducing WARD as a novel RAG-DI method ($4).
Our experimental evaluation in a wide range of settings affirms the fundamental limitations of existing datasets and all RAG-DI baselines, and demonstrates the effectiveness of WARD, which consistently shows high accuracy, query efficiency, and robustness ($5). Our source code and the FARAD dataset are publicly available at https:/ //gitbbb com/eth-oeeee homm//th-si//vee rri/ardd.
2 BACKGROUND
netrieval-augmeeneed generation (RAG) RAG is common way to enhance LLMs: For given user query 9, the & E N most relevant documents D, R(g, D) S D are retrieved from a corpus D, using retrieval method R (Gao et al., 2023). The query 4 is generally combined with D,p and fed into an L.LM to generate a more factual response r MMg, D,). Expanding /) also enables access to information without costly retraining (Lewis et al., 2020). RAG is especially suitable for domains where new information is generated often, e.g., news articles or software documentation. In this work use M to denote a RAG system, i.c., vetrieval-augmentee LLM M.
LLM watermarking LLM watermarks enable model owners to provably and reliably track text generated by their L.L.M In this work, we focus the prominent red-green watermarks (Kirchenbauer
2

---

Published as a conference paper at ICLR 2025
et al., 2023). During text generation, at each step t, the token vocabulary V is split into two parts, V green (encouraged) tokens, and (1-2)[V) red (discouraged) tokens, for some 7 E [o, 1]. Given context width h, the split is commonly function of tokens at positions - h ,,...,f. t,, well secret salt. To add the watermark, the logits of green tokens are increased by ā E Rgn, boosting their probability of being sampled. The watermark is detected using a statistical test, based on the expectation that non non-watermarked text of length I' has y green tokens. Namely, we use the z-score
|g/g 7T
>)T
where |s|g is the number of green tokens in a given text and T = [s). From here, we derive the p-value = $(2), where & is the CDF of the standard normal distribution, and consider the text watermarked if persist under moderate text transformations, such as paraphrasing or segment omission (Piet et al., 2023; Kirchenbauer al., 2024; Sander et al., 2024), making them suitable for our setting, where our goal will be to propagate the watermark signal through the RAG pipeline, as we will describe in $4.
for some threshold While not perfectly robust, these watermarks generally
RAG membership inference attacks While RAG is a relatively novel concept, recent work already studies membership inference attacks (MIAs) in this setting, proposing two methods that we denote SIB (Li et al., 2024) and AAG (Anderson et al., 2024). A MIA's goal is to output mi mi(d, M²) = 1 if document d is part of the retrieval corpus D of RAG system M' mi(d, M²)
0 otherwise, based only on queries to .M' To this end, STB queries M with g. a prefix of d, to obtain the response r =Miq, D.) Then, it computes two scores: the cosine similarity between the embeddings of d and r, and the perplexity of r. outputting mi(d, = if the similarity is above a threshold ssimilarity. and the perplexity is below threshold pepplexity both trainable parameters. Note that this requires gray-box access to MM for perplexity computation. The other method, AAG, directly prompts M to answer if d is in the context. If it replies positively, they set mi(d, M) 1. In $3, we will introduce the -bx RAG dataset inference setting, and adapt both baselines to it. Notably, while dataset inference was studied alongside MIA for training data (see $6), no prior work studies it for RAG.
RAG DATASET INFERENCE
We formalize the problem of RAG Dalaset Inference (RAG-DI), and present our contributions aimed at facilitating studies of this problem. In $3.1, we make the case that cxisting datasets commonly used for adjacent tasks (c.g., RAG MIA) are fundamentally unsuitable for RAG-DI, and propose new dataset in an attempt to address these shortcomings. In $3.2, we establish a set of baselines for RAG-DI, by adapting RAG MIA work introduced in $2, and proposing simple baseline, FACTS.
The RAG-DI problem The key entities in RAG-DI are the data owner, who aims to protect their n--ocument dataset D&o from unauthorized usage in RAG corpus, and the RAG provider, who exposes black-box access to their retrieval-augmenner LLM _M", which uses corpus D. The data owner's goal is to determine if Do S D, i.c., whether their data was secretly included in the corpus. To this end, they may proactively modify Da before publishing it, and can query M* in a black-box way, aiming to minimize the number of such queries. Crucially, the data owner makes a single dataset-level decision, opposed to document-level decisions of MIAs. Formally, RAG-DI method di should output da M*) = if D& C D (IN case) and 0 otherwise (OUT case).
3.1 A DATASET SUITABLE FOR RAG-DI
To enable suitable evaluation of RAG-DI methods, we require a dataset of documents with the following properties. First, we aim to match a key use-case of RAG (as described in $2) where up-to-date knowledge is added to D instead of costly repeated fine-tuning of M. To model this, our documents should provably not be part of the training data of M. i.e., of current open/closed LLMs, as those will be used to instantiate M when studying RAG-DI. Second, to model the practical case where knowledge is redundant and spread across multiple sources (c.g., news articles, common motivating example for RAG (Gao al., 2023)), the dataset should contain documents with overlapping topics and information (fact redundancy). As we will empirically demonstrate in $5, while fact redundancy more realistic, makes RAG-DI significantly harder.
3

---

Published as a conference paper at ICLR 2025
Current state As there is prior work RAG-DI, turn to related work on RAG privacy (Anderson et al., 2024; Li et al., 2024; Zeng al., 2024), including MIAs intro- duced in $2 We observe that evaluations in these works rely primarily EnronEmails (Klimt & Yang, 2004) and Health- careMagic datasets (Zeng et al., 2020; Mrini et al., 2021), motivated by the presence of PII in their samples. However, it can not be ruled out that these datasets were used to train contemporary LLMs, as e.g. EnronEmails has been publicly available since 2004. More importantly, fact redundancy by design not satisfied in either of these cases. These shortcomings motivate us to construct new dataset, FARAD Recc--Rednndat Article Dataset), tailored to RAG-DI.
Source
2123
Extraction
Key Facts
Additional Facts
Authors
The FARAD dataset FARAD consists of number of groups. Each group contains articles that share a topic and significant amount of information, but are independently written by a different (LLM) author. As our data source we use RepLiQA (Monteiro et al., 2024), that contains articles about fictional entities and events, which by design ensures that this knowledge was not present in any LLM training data. RepLiQA is released gradually-to create FARAD, we use split 0 as the only one available at the time writing, but plan to expand this to future splits.
FARAD group
FARAD-EasY no fact redundancy
Each RepLiQA article is a source for one of our groups the pipeline illustrated in Fig. 2. First, we prompt GPT40 to distill the information content of the article into 5 self-contained key facts, crucial understanding the article, and 10 self-contained additional factis that are present in the article but not essential its key message (see App. F.2). Next, to create group, we sample an author model from the set A (GPT4o, 405B, QWENI. -5-1 10BB] } of state-of-the-ar LLMs. The selected model writes a 500-1000 word article that must include all key facts, randomly sampled additional facts, and is encouraged to invent additional quotes, hypotheses, or personal opinions, as long as they do not contradict any present fact. The diversity authors, sampled facts, and (desirable) hallucinated content results in varied set of fictional articles grounded in the same core knowledge, which satisfies our earlier requirements. While this pipeline can be extended to larger/more groups, more authors, and different fact combinations, we limit FARAD 3591 groups (3391 for testing, and 200 for training) of 4 articles each, one per author from A.
FARAD-Hard recuredunyy
Overview of the generation pipeline of FARAD, and the resulting Easy and Hard evaluation settings.
Figure 2:
S0UUDE35--SONNEEE
LLAMA3.1-
Use in RAG-DI evaluation In our experiments in $5, we use FARAD to create two evaluation settings: Easy and Hard, illustrated in Fig. 2 (bottom). The Easy setting follows the traditional setup described above where facts are uniquely represented, by always using most article per FARAD group. The Hard setting introduces fact redundancy by always including all 4 articles from a group. As we demonstrate shortly, the Easy setting is solvable by a simple baseline, while the Hard setting, as the realistic and our main focus, is significant challenge for all bascline methods. In App. E, we discuss potential improvements to our data generation pipeline as items for future work.
3.2 RAG-DI BASELINES
We proceed to establish an initial set baselines for the RAG-DI problem. For this we adapt existing RAG MIAs, AAG and STB, introduced in $2, and additionally propose simple baseline, FACTS.
By design, the existing RAG MIAS make document-level decisions, i.., they decide if a single document is in the RAG corpus (mi(d, _M) 1) not (mi(d, M²)) 0). We ex- tend this to dataset-level decisions of RAG-DI in an (empirically) optimal way. First, we apply a given method to each d Dau obtain binary decisions, and set s( (Dao. M*) Ede mi(d,M*)
Adapting RAG MIAS
4

---

Published as a conference paper at ICLR 2025
We then instantiate several IN and OUT cases of RAG-DI using the training set, and compute the corresponding Sin and Sout- We finally make a dataset-level statement di(Dao- M³) = 0 otherwise. During this process, for SIB, also
s ((t & {Sin + Sau)) and di. grid search over its trainable parameters Axiillritt and perplexiyy (as in Li et al. (2024)); choosing values that maximize Sin Sout- We note that SIB is proposed gray-box method; we adapt it to black-box setting by using an auxiliary language model to estimate the perplexity of responses.
The FACTS baseline To substantiate our point about the effect of fact multiplicity on the hardness of RAG-DI, we introduce a simple baseline, FACTS. As the above RAG MIAs, FACTS is a document- level method. Given a document d, FACTS prompts an auxiliary LLM to generate single question that is only answerable by reading d. Then, MM" is prompted with that question, and if it deems it unanswerable, we set mi midd,
otherwise. We aggregate such document-level decisions into a corpus-level decision by fitting a threshold on the training set, as described above. In $5, we will demonstrate that this simple method is sufficient to solve the Easy setting more reliably than other baselines, yet is, due to fact multiplicity, extremely unreliable in the Hard setting.
O, and set
4 LLM WATERMARKING AS AN EFFECTIVE RAG-DI METHOD
Before describing our proposed RAG-DI method based on L.MM watermarks, we first outline three key design requirements that a desirable RAG-DI method should fulfill. We require the following:
1. Monotonicity. With more queries to the retrieval-augmented LLM M the accuracy of the method's ' predictions should consistently improve. preferably at high rate.
2. Guarantees. The method should be able provide statistical guarantee for its decision, with exceedingly rare and well-controlled Type 1 errors, as falsely accusing RAG providers is highly undesirable in practice, and undermines the trust in the method.
3. Robusteess. The method should maintain high accuracy under diverse evaluation settings, including attempts by the RAG provider to actively conceal unauthorized data usage.
In App. A we motivate these requirements in detail by reflecting on the context around RAG-DI. As demonstrate in $5, all RAG-DI baselines introduced in $3.2 violate all requirements some extent. To address this, we propose WARD (Watermarking for RAG-DI), a proactive RAG-DI method that is based on LLM watermarks, and discuss why it is likely to fulfill all stated desiderata.
WARD: RAG-DI via LLM watermarking We assume the data owner has protected each d: E Dao by embedding an LLM watermark either via numan-in-the-loog procedure or (as in this work) by rephrasing each document with a watermarked LM. While, in principle, any LLM watermark can be applied, focus on popular red-green watermarks (see $2). In $5.3, we confirm that this results in quality texts, faithful to the original ones. To audit the RAG provider's corpus, for each d, E Dso- WARD generates an open-ended content-related question qi and queries M If dj E D (IN case), we expect the retrieval method R to introduce watermarked content from de into the LLM context. As noted $2 and validated in $5, the robustness of watermarks to text transformations is then sufficient to propagate the traces of the signal to = MM (4,), the final response of the LLM.
Boosting weak signal Requiring that each r; is flagged as watermarked, i.e., has watermark detector p-value p < would be strong assumption, the watermark signal is likely to degrade throughout the RAG pipeline. However, this is not necessary for WARD to be effective, Instead, following Sander et al. (2024), after queries compute a joint p-vuuue R {ri....,, Tn), directly corresponding to the null hypothesis "the data owner's 's dataset D is not in the RAG corpus D".
This joint p-vuuue can satisfy p < a. i.c., reject the null hypothesis, even when individual carry only weak watermark signal, that would individually not reject it. To illustrate this, given a desired p-value threshold a, Eq. (1) implies that the required ratio of green tokens in R is at least
- Y 2 Φ -1 a) (1(1->)/1B) R| +7,
(2)
5

---

Published as a conference paper at ICLR 2025
where d is the standard normal distribution CDF, and |R @ the total length of responses in R. This lower bound decreases
0.3-
quickly as [R|@ increases: In Fig. 3 we plot the lower bound as
a function of n, assuming Vi: |r; = 400 tokens, a R 3 10-5 (7-score of at least 4), and y = 0.25, in our experiments. For 2 100, if propagation of the watermark through the RAG pipeline increases the ratio of green tokens by only 1%, is already detectable with high confidence. This makes WARD viable, satisfying the Guarantees requirement, unlike any base- line. It also contributes to Monotonicity: assuming each d, has green token ratio of 7", which propagated through MM reduces ) the p-value for IN cases strictly decreases for more queries. In $5.2 we experimentally confirm that WARD scales consistently and efficiently in the number of queries.
0.30
0.26
0.25
No. Watermark
0
to to E ?"",
Number Queries
Figure 3: Green ratio to pass water- mark detection with given #queries.
Test validity and signal leakage Another requirement for Monotonicity is that the p-value for OUT cases is well-calibrated, ensuring exceedingly rare false positives. As pointed out in prior work (Kirchenbauer et al., 2023; Sander et al., 2024), to ensure independence between tokens scored by the watermark detector, it is necessary to ignore duplicate h-grams across R. While this reduces the number of useful tokens per query, our results in $5 show that it does not affect WARD performance.
For our statistical test to be valid, we must ensure that the watermark signal can only originate from the use of Doo in D, i.e. that it is not otherwise leaked to the responses r4. In particular, observed that watermarked queries, or queries based on watermarked versions of d4, drastically increase false positives. To confirm that WARD does not suffer from this issue, in $5.2, we verify that the p-values in OUT cases are distributed in [0, 1] roughly as expected with no noticeable decrease in Rt]
Scheme choice The key parameter of red-green schemes is h, the context width. common applications of LLM watermarks, a low / is not recommended, as it makes the watermark easy to steal via repeated queries (Kirchenbauer et al., 2024; Jovanović et al., 2024). In the context of RAG-DI, this is less of concern, as the data owner never exposes unconstrained query access to watermarked content, as is the case when watermarking LLMs. Thus, as low h benefits watermark propagation through RAG, we use h = 2. This contributes to high Robustness of WARD, empirically validated in $5.2. We discuss other scheme parameters and present ablation studies over each in $5.4.
Practicality Another aspect making WARD robust is that it is inconspicuous, as it operates on natural-looking documents, queries, and responses (confirmed in $5.3), through legitimate use of the RAG system-this makes potential attempts by the RAG provider to thwart malicious interactions less effective. This differs from baselines such as AAG, which directly reveal the intention to leak the information about D. As a final advantage, we note that, in contrast to all baselines, WARD does not require any training or adaptation RAG-DI, LLM watermarks naturally apply this task.
EXPERIMENTAL EVALUATION
We evaluate the RAG-DI baselines ($3.2) and WARD ($4) on the FARAD dataset (83.1). $5.1 presents our main experiment. In $5.2 we focus on desiderata from $4, showing that only WARD does not violate them. In $5.3 we validate several key assumptions, and in $5.4 present additional ablations.
Setup Our experimental setup follows $3: we use FARAD to define two evaluation settings, and in both evaluate IN and OUT cases, i.c., where the data owner's 's data is (resp. is not) contained in D. We use |Daol 200, and |D] = 800 for FARAD-Easy, and D 3000 for FARAD-Hard (sampling detailed in App. B. We note that WARD only depends on Daul, but not D). We use several L.MMs as M: GPT3.5, CLAUDE3-HAIKU and LLAMA3. 1-M0B. and vary the system prompt: we use a short naive prompt (Naive-P) with basic RAG instructions, and longer defense (Def-P) prompt, which models a RAG provider that instructs the model to not regurgitate sources verbatim, and refuse attempts to learn about the exact LLM context (see App. F.1). Each experiment is run with 5 random seeds. To ensure a controlled setting, in our main experiments, we assume a perfect retrieval system that always retrieves the most relevant documents (see App. B.2 for detailed explanation). In $5.3,

---

Published as a conference paper at ICLR 2025
FARAD Easy
FARAD Hard
NANE
MAVE
DEF-P
Liama& 1-70c
NANE
MAVE-P OUT
DEF-P I
DEF-P OUT
GPT-55
NANE-P IN
DEF- IN
DEF- OUT
PACTS
(V)
AAG
(V) SiB
WARD
x FACTS
x AAG
X
WWARD
Figure 4: Evaluation all methods on FARAD in both Easy and Hard settings, and with both Naive-P and Def-P system prompts. We run each method with 5 random seeds, resulting in 5 squares. A red square indicates a false negative the IN case, and a false positive in the OUT case. All methods perform well in the Easy setting, while only WARD consistently performs well in the Hard setting.
we show that WARD works equally well on practical end-to-end RAG. If not specified otherwise (see $5.4), the RAG uses 3 shots. For methods that utilize auxiliary LMs (including WARD, which starts by watermarking Daa) we use LLAMA3. 1-8B, and to compute the cosine similarity in SIB we use paragraph-level AL.L- LLL--MNNIL-M-L--- with BERTSscore aggregation (Zhang et al., 2020). For WARD, we use PositionPRF (Kirchenbauer al., 2024), h 2, and ó 3.5, ablating these in $5.4. We list all prompts, and examples FARAD samples and watermarked documents, in App.
5.1
MAIN RESULTS
We present our main results in Fig. 4, where we evaluate all RAG-DI baselines and WARD across several settings, models, prompts, and random seeds. We make several key observations.
First, all baselines perform somewhat well in the Easy setting (no fact redundancy). However, our extremely simple FACTS baseline obtains perfect results, outperforming both AAG and SIB. This emphasizes that traditional non-redundant datasets (see $3.1) fail to capture the complexity of RAG- DI, and can provide incomplete view of the capabilities of RAG-DI methods. We also observe that already in the Easy setting, straightforward system prompt defenses significantly impact both AG and SIB, leading to noticeable increase in false negatives. We further investigate defenses in $5.2.
In the Hard setting, all baselines fail to perform consistently, inducing both false positives and negatives. This can be attributed to their shortcomings in handling fact redundancy: FACTS directly relies on facts, while SIB and AAG rely semantic similarity influenced by factual content. Notably, only WARD achieves 100% accuracy across all settings, models, and system prompts, showing that despite the retrieval of documents with partially overlapping facts, watermarking provides reliable signal for dataset inference. This backs up our claims regarding the importance of fact-redundancy for realistic RAG-DI evaluation and highlights the potential of watermarking an approach to RAG-DI.
5.2 DESIDERATA
We next demonstrate how baselines violate the desiderata from $4, which is not the case for WARD.
Monotonicity As stated in $4, RAG-DI methods should steadily improve with more queries, i.c., Daol for WARD (see $5.4 for generalization). We evaluate this by setting Daul E [20, (20,40.... 40..... ., 200) in FARAD-Hard, presenting the results of WARD and SIB in Fig. 5. WARD improves consistently with Da reaching perfect accuracy across all settings for at most 80 documents. discussion of efficiency in App. D we show that this translates to inexpensive API costs of below $1, for all closed-source LLMs we consider. In contrast, SIB, besides never reaching full accuracy, exhibits strongly varying accuracy over Daol, often decreasing despite using more queries. This is consequence of the need to adapt the MIA baselines to the RAG-DI setting- we find similar behavior across all baselines, and present additional study of their decision thresholds in App. C.6.
our extended
7

---

Published as a conference paper at ICLR 2025
NAME (0GT-P)
P)
1.0
1.0
088
0.8
06
00
0.4
0.4
00
50
120
125
120
175
200
0.0
20
70
100
120
100
1710
200
Kumber of Queetis
Kumbier Queries
Figure 5: WARD (left) and SIB (right) accuracy as function of Daol (i.e., the number of queries) in the Hard setting. WARD consistently improves, while SIB suffers from high variance in accuracy.
Table I: The p-values of WARD. We report the max p-vauues for IN and min p-values for OUT cases.
Clande 3 Halks
GPT-35
Lamaá 1-70
Data
Age
DEF-P
NAIVE-P
DEF-P
NAIVE-P
DEF-P
NAIVE-P
OUT
IN
OUT
IN
OUT
OUT
18
OUT
IN
OUT
IV
FARAD min
1.90--
2.07.
1.3%
3.47e-
3.28--11
9.53-11
213--h
1.78--
7.54
6.20-117
1.32.
Ray
max
6.83e-18
L57e
2.8%
2.10 133
7.12--1
LS4e-27
9.17e-9
d.56e
9.55e
LS2e-12s
5.05e-11
FARAD
min
6.03-
1.35e-48
3.82 -11
5.086
204--33
1.26-32
260--1
1.53-22
1.18-
3.96-224
6.12--12
1.22-
Hard
0.99 -4
2.15e
7,0%
3.36e -4
7.80e
8.44
7.92
1a1e-
7.47e
1.10--1
3.70e-4
2.42
Guarantees In contrast all baselines, WARD inherits the guarantees of the watermarking scheme, directly providing a p-value for each de- cision. In Fig. 6, we see that p-values rapidly decrease with more queries for IN, but stay con- sistently close to the expected value for OUT. Fur- ther, in Table I, we show the max p- p-values for IN and min p-values for OUT in our main experiment, ie., closest to a false negative/positive. All p- values orders of magnitude from our decision boundary of 2e 3e-5 (z-score of 4), highlighting robustness to Type 1 errors and the fact that LLM watermark signals persist through a RAG pipeline.
Watamanta
OUT
35
500
135
10B
500
Nummer Querne
Figure 6: P-values function of #queries in IN (red) and OUT (blue) cases.
Robustness Finally, we evaluate the methods' ro- bustness to different settings. In particular, we further examine the Def-P system prompt setting. shown in Fig. 4 to often prevent all baselines from obtaining any useful estimates. Notably, both AAG and SIB rely on the model's willingness to leak information about its context, either explicitly or by regurgitating similar content. To validate the effect of Def-P, we show the longest token-string overlap between the response of M and the SIB target document (Fig. 7), observing strictly less overlap with Def-P, inducing false negatives. Importantly, this does not prevent Figure 7: The effect of the Def-P sysprompt. watermarks from propagating through the RAG pipeline, resulting in reliable signal for WARD.
Count
Mai Repettiion leugt
We additionally investigate a defense based on MEMFREE decoding (Ippolito et al., 2023), Namely, we adapt the decoding of M to strictly prevent n-gram overlap with the retrieved documents (using n 10 as in Ippolito al. (2023)), modeling an even stronger attempt by the provider to protect the RAG corpus. Our results (App. C.5), show that even in this setting, WARD achieves full accuracy. We further discuss the potential for more elaborate defenses against WARD in App. F.2.
5.3 ADDITIONAL CONSIDERATIONS
Modeling retrieval So far, experiments have assumed an idealized case of perfect retrieval. We now justify this choice by running WARD on an end-to-end RAG system which uses OpenAl's text fext--embedding-- largg document embeddings, with k- and cosine similarity metric.
8

---

Published as a conference paper at ICLR 2025
1.0
Queria Docament
20
50
6.9
Gool
6.5
Metric
Note
Grade
0-7
0.5
5
30
75
122
130
175
530
Data
Number Querien
Figure 9: Left: Accuracy and watermarked text quality for different 5 values of WARD FARAD- Hard. Right: Accuracy over the number of queries per document for WARD on FARAD-Mard.
As above, we repeat all runs with seeds on both Easy and Hard, with .M LLAMA3.1-708 and both prompts. We de- fer accuracy curves to App. C.3 but note that WARD achieves 100% accuracy across all set- tings. In Fig. 8, show average p- p-values over the number of queries, further illustrating the reliability of WARD. We note that this simple RAG system was able to retrieve the targeted watermarked article in 93.6% of all requests, almost perfectly reducing this to our idealized setting. We further run all baselines with Def-P in the Hard setting, and confirm that the (rare) retrieval errors do not make them more compet- itive, e.g., because the distracting documents are now less relevant. As in the corresponding part of Fig. 4, FACTS has 100% false positives, while SIB and AAG have 100% true negatives.
Thrss::
Waterrahed
OLT
303
125
200
Number of Queries
Figure 8: Average p-values against #queries in our imperfect retrieval experiment.
Text quality Applying WARD requires the data owner to watermark Ddo. which we do by paraphrasing with a watermarked model. While prior work shows that water- marks minimally impact text quality (Piet et al., 2023), we re-affirm this by judging the quality of randomly sampled subset of 100 resulting documents. As shown in Table 2. paraphrased documents maintain high quality both under GPT40 ratings (see App. F.7) and paraphrase quality metric P-SP (Wieting et al., 2022).
Table 2: Quality evaluation of water- marked documents.
Original
Paraphrased
A
Judge P-SP
0.903
0.898 0.933
5e 3
1.000
0.07
We also evaluate the quality of the RAG system's 's responses (with Naive-P and CLAUDE3-HALL HAIKU) on 200 samples. The average response quality is 0.9475 in IN cases (watermarked context) and 0.9465 in OUT cases, confirming that watermarked documents do not negatively impact response quality.
5.4 ABLATIONS
Lastly, we provide ablations for important hyperparameters of WARD and the RAG-DI setup.
Watermark parameters As introduced in $2, red-green watermarks have 2 key parameters: the strength 5 and the context size h. We ablate over their impact on WARD using LLAMA3. 1-0BB as M with the Def-P sysprompt. The results for are shown Fig. (Left). We observe sweer spot of d E [2.5, 4.5] with high accuracy (unlike 8 < 2.5) and the minimal impact on text quality (unlike d > 4.5). Notably, experiments in App. C.2 show that 5 >> 4.5 negatively affects both query efficiency and text utility. This range of 5 [2.5, 4.5] directly aligns with recommendations in prior work (Kirchenbauer et al., 2024), and supports our choice of 5 = 3.5 for the main experiments.
Regarding h. noted in $4 and prior work, larger values expected to degrade robustness. We compare our choice of h = 2 with h = 4, noticing a slight decrease in accuracy (100% on Easy, but 80% on Hard), confirmed by our plots of query efficiency- full results deferred to App. C.2.
9

---

Published as a conference paper at ICLR 2025
RAG-DI parameters The RAG-DI setting has wide range of parameters that we so far have not explored. First, we ablate over k. the number of retrieved documents our results in App. C.I show that WARD performs well independent of this parameter. Also, in App. C.I, study the case where only a subset of Da is included in D, relaxing the assumptions of R RAG-DI. While this naturally weakens the watermark signal, our results imply that WARD shows some robustness this setting.
Finally, while we generally assume a single query per document d,, we briefly study the potential of WARD to increase its query efficiency by extracting more value from each dg. For this we introduce parameter qpd (queries per document, qpd = 1 in the main experiments), generating more questions for each dę- The main results for qpd E {1, 2, 4, 5, 10, 20, 50} are shown in Fig. (Right), where we plot the average p-value over the total number of queries. As expected, for high qpd, additional queries for the same d; bring only marginal value-thit how the effective number of tokens scored by the watermark detector increases slower for high qpd, as we encounter many repeated n-grams. However, for lower values such as gpd = 4, reducing the required Dáoo times, we observe strong results, suggesting promising avenue for future work.
also confirmed in App. C.4, which shows
6 RELATED WORK
Closest related works to ours are Li et al. (2024) and Anderson et al. (2024), which propose member- ship inference (MI) for RAG that we adapt to RAG-DI in $3, and works that highlight the risk of MI in the related paradigm of in-context learning such as (Duan et al., 2023). Others study broader privacy and security aspects of RAG, such as poisoning to jailbreak the model or exfiltrate data (Zou et al., 2025; Xue et al., 2024; Chaudhari et al., 2024; Zeng et al., 2024), concerns similar to RAG-DI.
The problems of membership inference (MI) (Shokri et al., 2017; Carlini et al., 2022) or dataset inference (DI) (Maini et al., 2021; Dziedzic et al., 2022) have been long studied on training data, as opposed to RAG corpora in this work. Recent attempts to adapt these methods to LLMs (Duan et al., 2024; Das et al., 2024; Meeus et al., 2024; Maini et al., 2024) cite the challenge of rigorous evaluation, and primarily focus on graybox settings, citing the difficulty of inference attacks in the blackbox setting (Choquette- -Choo et al., 2021), which is what we consider in RAG-DI. Another perspective on the problem of tracing data usage in model training is given by recent works on L.M data contamination (Dekoninck et al., 2024; Oren et al., 2024), none of which consider RAG.
Passive MI/DI
Proactive MI/DI/model protection Another approach to MI/DI (as in this work) proactive, e.g.. by watermarking the data (Ren et al., 2024; Guo et al., 2023). However, only few works study LLMs, and only the ones cited above consider RAG. Wei et al. (2024b) focus on LLMs in a graybox setting, inserting random sequences unicode substitutions to trace the data through training. Sander et al. (2024) find that LLM watermarks propagate through fine-tuning tne-ttnnng, but Da has to make up 10% of the fine-tuning corpus for blackbox detection. RAG-DI relaxes this requirement, as RAG is unaffected by [D] and degrades the signal much less than fine-tuning, Another orthogonal area is watermarking of models against model stealing, often via backdoors (Adi et al., 2018; Zhao et al., 2023).
LLM Watermarking Finally, we note that many works study red-green LLM watermarks (Kirchen- bauer et al., 2023; 2024; Zhao et al., 2024; Hou et al., 2024), but also other approaches such as sampling modification (Kuditipudi et al., 2024; Christ et al., 2024), model-based watermarking (Liu al., 2024), or watermarking in weights (Gu et al., 2024). We note that WARD could be combined with many of these, and leave this interesting direction to follow-up work.
7 CONCLUSION
We studied the problem of black-box RAG Dataset Inference (RAG-DI), where the goal to detect unauthorized usage of a dataset in RAG system. We formalized the problem, proposed a dataset and a set of baselines, and presented WARD, a method based on LLM watermarks, which provides rigorous statistical guarantees. Our evaluation showed that WARD outperforms the baselines, achieving perfect accuracy and high query efficiency and robustness. This establishes WARD as a practical tool that can be directly applied to protect the rights of data owners in current RAG systems. We hope our work provides a valuable foundation for future work on MAG-DD--nteren WARD with other LLM watermarks, designing watermarks specifically tailored to RAG-DI.
directions include combining
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
The work has received funding from the Swiss State Secretariat for Education, Research and Innova- tion (SERI).
ETHICS STATEMENT
We do not foresee any ethical concerns arising from our work. Our work primarily provides dataset owners with the ability to prove unauthorized usage of their data in RAG system. which valuable tool for protecting intellectual property, and fighting agains! unethical practices of large model providers. An undesirable side effect of such methods may come from false accusations as experiments in $5 demonstrate, positives as highly unlikely, and all of our experiments we not observe single instance of false positive. While our work does result in new dataset, FARAD, this dataset is entirely synthetic as it is based on fictional articles (Monteiro et al., 2024), and thus does not raise any privacy concerns.
proposed method, WARD, strictly controls the rate of false
REPRODUCIBILITY STATEMENT
To foster reproducibility, provide details of our experimental setup in $5 and App. B.I, and in App. F include all prompts used in our experiments. We make our code and the FARAD dataset available at https : //ggthub.bb... com/eth-o http//hh-sri//ee sri/ward under an MIT license, and include configuration files for each experiment and a README file with instructions on how to reproduce our results.
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cissé, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In USENIX Security, 2018.
Maya Anderson, Guy Amit, and Abigail Goldsteen. Is my data in your retrieval database? membership inference attacks against retrieval augmented generation. arXiv, 2024.
Kar Balan, Alex Black, Simon Jenni, Andrew Gilbert, Andy Parsons, and John Collomosse. Decorait decentralized opt-in/out registry for training, 2023.
Blake Brittain. case, judge says, 2024.
Stability ai,
midjourney should face artists' copyright https://www reuters. wwwteess.com/egalllllll face-areeee wwte--rrttsssooooooo copyright- casel judge- says 2024--224-5555 08/.
stability-al aaa-aa--maaa ailpity--ammmmjoorrriaa midjourney-
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr.
Membership inference attacks from first principles. In IEEE S&P, 2022.
Harsh Chaudhari, Giorgio Severi, John Abascal, Matthew Jagielski, Christopher A. Choquette-Choo Milad Nasr, Cristina Nita-Rotaru, and Alina Oprea. Phantom: General trigger attacks on retrieval augmented language generation. arXiv, 2024.
Christopher A. Choquette-Choo Florian Tramèr, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In /CML, 2021.
Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. COLT, 2024.
Coalition for Content Provenance and Authenticity. Technical specification vl.3. technical report. c2pa., 2021.
wwwpppppcciiccttnnnn.ooaooo fications/1. 3/3/ndeex. html.
Council of the European Union. Directive (eu) 2019/790 of the european parliament and of the council on copyright and related rights in the digital single market and amending directives 96/9/ec and 2001/29/ec. 2019.
11

---

Published as a conference paper at ICLR 2025
Debeshee Das, Jie Zhang, and Florian Tramèr. Blind baselines beat membership inference attacks for foundation models. arXiv, 2024.
Jasper Dekoninck, Mark Niklas Müller, Maximilian Baader, Mare Fischer, and Martin T. Vechev. Evading data contamination detection for language models is (too) easy, arXiv, 2024.
Haonan Duan, Adam Dziedzic, Nicolas Papernot, and Franziska Boenisch. Flocks of stochastic parrots: Differentillly private prompt learning for large language models. In NeurIPS, 2023.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work large language models? COLM, 2024.
Adam Dziedzic, Haonan Duan, Muhammad Ahmad Kaleem, Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, and Nicolas Papernot. Dataset inference for self-supervised models. In NeurlPS, 2022.
European Parliament and the Council of the European Union. Proposal for regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts analysis of the final compromise text with a view agreement. 2019.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. detrieval-augmeetedd generation for large language models: A survey. arXiv, 2023.
Paul Goldstein, Christiane Stuetzle, and Susan Bischoff. Kneschke vs. laion landmark ruling tdm exceptions for ai training data, 2024. ht https:: http:///copyrihhtblooggo wtwpeilllo....uukleooo elwwrriplee plplla 2t00//2224/1/1//// kneschke- vs- laion- landmark wwwiigg-ooonoooeo on- tdm exceptions- for- ai- traininnn--n-- data- part- 1/, last accessed: Feb 2025.
Michael M. Grynbaum and Ryan Mac. The times sues openai and microsoft over a.i. use of copyrighted work, 2023. https://vw/ www.:.nytimesccommm new-york times open----------------ee- ai microsoft- lawsuit html.
Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the learnability of water- marks for language models. ICLR, 2024.
Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. Domain watermark: Effective and harmless dataset copyright protection is closed at hand. In NeurlPS, 2023.
Abe Bohan Hou, Jingyu Zhang. Tianxing Hc, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. Semstamp: semantic watermark with paraphrastic robustness for text generation. NAACL Long, 2024.
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette- Choo, and Nicholas Carlini. Preventing generation verbatim memo- rization in language models gives false sense of privacy. In INLG, 2023.
Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. Flashrag: A modular toolkit for efficient retrieval-augmenned generation research. arXiv, 2024.
Nikola Jovanović, Robin Staab, and Martin Vechev. Watermark stealing in large language models. ICML, 2024.
A. Feder Cooper Katherine Lee, Daphne Ippolito. The devil is the training data, 2023. https: //bllo..gentae ong. data. html, last accessed: Feb 5 2025.
Paul Keller and Zuzanna Warso. Defining best practices for opting out of ml training. Open Future Policy Brief 5, 2023.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In ICML, 2023.
12

---

Published as a conference paper at ICLR 2025
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. ICLR, 2024.
Bryan Klimt and Yiming Yang. The enron corpus: A new dataset for email classification research. In European conference muchine learning, 2004.
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. TMLR 05/2024, 2024.
Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih. Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-angmented generation for knowledge-intensive NLP tasks. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), NeurIPS, 2020.
Yuying Li, Gaoyang Liu, Yang Yang, and Chen Wang. Seeing is believing: Black-box membership inference attacks against retrieval augmented generation. arXiv, 2024.
Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S. Yu. private watermark for large language models. ICLR, 2024.
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, and Sara Hooker. The data provenance initiative: A large scale audit of dataset licensing & attribution in ai. arXiv, 2023.
Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. In ICLR, 2021.
Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. LLM dataset inference: Did you train on my dataset? NeurIPS, 2024.
Matthieu Meeus, Shubham Jain, Marek Rei, and Alex-Alee de Montjoye. Inherent challenges of post-hoc membership inference for large language models. arXiv, 2024.
Joao Monteiro, Pierre-Andre Noel, Etienne Marcotte, Sai Rajeswar, Valentina Zantedeschi, David Vazquez, Nicolas Chapados, Christopher Pal, and Perouz Taslakian. Repliqa: A question-answering dataset for benchmarking Ilms on unseen reference content. NeurIPS, 2024.
Khalil Mrini, Franck Dernoncourt, Walter Chang, Emilia Farcas, and Ndapandula Nakashole. Joint cummariization-nnaiinenn optimization for consumer health question understanding. In ACL workshop NLP for medical conversations, 2021.
Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. Proving test set contamination in black-box language models. In ICLR, 2024.
Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, and David A. Wagner. Mark my words: Analyzing and evaluating language model watermarks. arXiv, 2023.
Joao Pedro Quintais.
Generative ai, copyright and the ai act, 2023. https://copyrighttiuu Kluweriplaw.
ai- copyright and the- ai- aattcc// last accessed:
Feb 5 2025.
Jie Ren, Yingqian Cui, Chen Chen, Vikash Sehwag, Yue Xing, Jiliang Tang, and Lingjunn Lyu. Entruth: Enhancing the traceability of unauthorized dataset usage in text-to-image diffusion models with minimal and robust alterations. arXiv, 2024.
Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. NeurIPS, 2024.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In IEEE S&P, 2017.
13

---

Published as a conference paper at ICLR 2025
SpawningAI. Have i been trained?, 2022. http:://aavelbbetttriin..cco last accessed: Feb 2025.
Ap ai.txt, 2023, https: https://spawning.al/ai
SpawningAL. Spawningai, 2024. https://spawning ai/, last visited: Nov 19 2024.
TDM Community Group. Tdm reservation protocol (tdmrep) final community group report, 2024.
Johnny Tian-Zheng Wei, Ryan Yixiang Wang, and Robin Jia. Proving membership in LLM pretraining data via data watermarks. Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), ACL, 2024a.
Johnny Tian-Zheng Wei, Ryan Yixiang Wang. and Robin Jia. Proving membership in LLM pretraining data via data watermarks, In ACL (Findings), 2024b.
John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Kink-iikpattiic. Paraphrastic representa- tions at scale. In EMNLP (Demos), 2022.
Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models. arXiv, 2024.
Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang. Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, and Pengtao Large-scale medical dialogue datasets. In EMNLP, 2020.
Xie. MedDialog:
Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang. Dawei Yin, Yi Chang, al. The good and the bad: Exploring privacy issues in aetrente generation (rag). ACL Findings, 2024.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi Bertscore: Evaluating text generation with BERT. ICLR, 2020.
Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting language generation models via invisible watermarking. ICML, 2023.
Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. ICIR, 2024.
Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. Poisonedrag: Knowledge poisoning attacks to aatrivval-uugmentt generation of large language models. USENIX Security, 2025.
14

---

Published as a conference paper at ICLR 2025
A MOTIVATION: THE IMPORTANCE OF RAG-DI
Extending on our discussion in $1, we elaborate on the motivation for establishing RAG-DI as a problem setting, the prevalence of the problem of unauthorized data usage (studied in this work), the need for a solution like WARD, and its practical applicability.
Protecting data from use in Generative AI The unauthorized usage of data in Generative Al (GenAl) systems, primarily for training, has become an especially pressing concern in recent years, as the practice of indiscriminate scraping of massive datasets from the web became more common- place (Katherine Lee, 2023). This has led to crisis in data transparency, where the provenance of data used to train models is often hard to track (Longpre et al., 2023), and commonly not reported by the model provider. While the EU AI Act (Council of the European Union, 2019) attempts to remedy this, by mandating that all data sources used for model training are published, this regulatory process is slow, and data owners must take active steps to protect themselves.
Opt-outs and legal uncertainties One tool that data owners can use to exercise agency over the use of their work comes in the form of opt-outs, where creators can choose exclude their works from GenAl training. Notably, the Article 4 of the EU's Copyright in Digital Single Market (CDSM) Directive (European Parliament and the Council of the European Union, 2019) recognizes the right of creators to exercise such an opt-out, by providing a machine-readable expression of the reservation of rights. Yet, the practical implementation of this opt-out mechanism is still unclear, and an active discussion from both law (Quintais, 2023) and research (Balan et al., 2023) communities is ongoing. Many initiatives have established standards for expressing such reservations (Keller & Warso, 2023), such as ai. txt (SpawningAl, 2023), the TDM Reservation Protocol (TDM Community Group, 2024), and C2PA (Coalition for Content Provenance and Authenticity, 2021), a metadata standard for embedding content provenance information in media files which also includes flag for opting out of GenAl training and/or inference.
While opt-outs are picking up traction, and some GenAl providers have pledged to honor them (SpawningAl, 2024), they are still far from being universally respected, and it is yet un- clear how to practically enforce them (or audit compliance). More broadly, the legal landscape around GenAl data usage is complex, and both legal and research communities are closely following several prominent lawsuits. Notable examples include NYT vs OpenAl/Microsoff (Grynbaum & (Brittain, 2024),
Mac, 2023) or group of artists vs Stability/MidJJurneyy/// Setree///Deeiiattt/rraaaaaa which illustrate that the legal system naturally still catching up to the rapid development of GenAl. The key challenges are in untangling the interplay between licenses, traditional copyright, opt-outs, the ideas such as fair use or market harm, and technical concepts such as LLM memorization. The most relevant for our discussion is the case Kneschke vs LAION (Goldstein et al., 2024), where the ruling has extensively engaged with the meaning of opt-outs and the extent of their enforceability.
Proving unauthorized usage In this landscape of legal uncertainty, the ability to provably detect unauthorized usage of data in GenAl systems is crucial for data owners, as may provide way to enforce their rights in court. While this is relatively simple when model providers make their datasets public AS,a 2022), the problem becomes significantly harder for the common case of undisclosed training data, leading to works on proactive membership/dataset inference such Wei et al. (2024a); Sander et al. (2024) and others we cite in $6. As noted in $1, a particular flavor of this problem comes in the context of LLMs, where RAG systems have become common way to integrate new knowledge into the model without costly fine-tuning.
WARD This is the problem statement studied in this work and tackled by WARD, with our require- ments ($3) directly inspired by the legal and practical challenges data owners face in this context. Namely, Guarantees are crucial to be able to use the results as evidence, and Robustness is important to ensure that the method is not easily circumvented, unintentionally or by attempts to defend against it Finally, Monotonocity and the corresponding Efficiency are needed for the method to be practically feasible. While the data owner naturally must query every system they suspect is using their data in the RAG corpus, each such use is very cheap (below $2 per LLM, see App. D). Further, we argue that the ecosystem of relevant LLM providers is relatively small. In particular, not many providers both (i) have resources for indiscriminate large-scale scraping of data and (ii) are sufficiently popular to create market harm based the unauthorized usage of that data. Thus, the cost of WARD is practical.
15

---

Published as a conference paper at ICLR 2025
B ADDITIONAL EXPERIMENTAL DETAILS
We expand on the details of our experimental setup provided the main paper, by providing a detailed explanation of how sample FARAD create our evaluation settings (App. B.1), and how we simulate perfect retrieval in our experiments (App. B.2).
B. SAMPLING FARAD FOR EVALUATION
Recall that each FARAD group contains articles, one for each of the LLM authors we consider. For brevity, we will use A; for i E (1,2,3.4) 4] to denote the i-th author. For the Easy setting, we sample four subsets of distinct groups, where the sizes the subsets are respectively (200, 300, 300, 200). Then for each subset i E 31), we only take articles from A, and include all of them in the RAG corpus D. Out of those, articles from subset potentially modified by the service provider before inserting them into the RAG corpus. Similarly, the articles from subset 4 of author As are reserved as Da in the OUT case. This setup ensures no fact redundancy in the RAG corpus.
author A1 are taken as Da in the IN case, i.e,, these
In contrast, to create the Hard setting with fact redundancy, we start by sampling 1000 distinet groups. The RAG corpus D is then built by including all articles from those groups that were written by AI, A2, and Ag. A randomly subsampled set of 200 of those articles, that were written by A,, is taken Dao in the IN case. Similarly, randomly sampling 200 of the 1000 above groups, and taking documents from A₃ out of each group is used as Dao in the OUT case. Due to practical limitations, before running all of our experiments we managed generate only the first 1000 out of 3391 groups in FARAD, thus our sampling is done w.r.t. these 1000 groups.
B.2 SIMULATING PERFECT RETRIEVAL
We provide a detailed description of the perfect retrieval mechanism used in our main experiments, introduced in $5, and further compared to real retrieval mechanism in $5.3, where have demonstrated that it almost perfectly matches the idcalized case described in this section and shown that WARD performs equally well in both cases. Assume the data owner is querying M in order to test for presence of document d. We construct the following sequence of documents:
First, the document d itself if it is present in the RAG corpus.
Next, random shuffle of all documents d' from the same FARAD group that are present in the RAG corpus, i.e., the documents produced by other authors based on the same source.
Finally, random shuffle of all other documents from the RAG corpus.
Given the parameter & of desired shots (ablated in Fig. 11), the perfect retrieval returns the first documents from this sequence. Note that for WARD and k >> this always results in additional documents in the context beyond the watermarked one, where such distractions often come from both the same source, well different sources. We further illustrate the behavior of perfect retrieval with two examples.
First example The RAG provider populates the corpus D according to the Easy setting, and uses k 3. The user applies WARD query M with question related the (watermarked) document #123a, i.c., from FARAD group #123, written by the author a. Assume # #123a E D. The retrieved documents are # 123a, #374b, #266a. Following the above steps. we include # #123a as present in the corpus, and two documents chosen at random, as in the Easy setting the corpus includes no other document from group 123.
Second example Now assume the Hard setting and k = shots. The user queries M with question related to #123d. Assume #123d D. The retrieved documents #123a, # 123b, # #12cc #597c, # #422b. Per above steps, as the target document #1 #123d not in the corpus, include all other documents from group # 123 and 2 documents chosen at random.
16

---

Published as a conference paper at ICLR 2025
232
40.0
92.
60
210
42.5
900
180
333
56.0
120
300
51.
-35
1ng poolie
-30
-35
135 Karrowr of Queries
130
1N5 Number of Queries
100
129
125
Figure 10: Effect of the percentage of documents of Da not contained in D on the p-value for 11AMMA3.1-7BB using Naive-P (left) and GPT3.5 using Def-P (right) on FARAD-Hard. While in both cases we see how WARD is robust to an increase in the percentage of documents contained in D, the Def-P setting is more sensitive, showing initial false positives at 20%.
C ADDITIONAL EXPERIMENTAL RESULTS
In this section, we first present additional results on our RAG-DI ablations, omitted from $5.4 (App. C. Similarly, in App. C.2, we extend our WARD ablations from $5.4. In App. C.3 we provide extended results of our retrieval study from $5.3, and in App. C.4 extend our results on the number of queries per document from Fig. (Right). Finally, in App. C.5 we provide results on an additional defense (MEMFREE, summarized in $5.2), and in App. C.6 we provide more insights into the performance of RAG-DI baselines.
C.I RAG-DI ABLATIONS
As summarized in $5.4, we ablate over key parame- ters of RAG-DI: The number of documents (shots) put into the context of model (k) and the fraction of documents of Da contained in D.
Number of shots For l we present our results for WARD in Fig. 11, using h E {3, 5, for our Naive-P and Def-P settings on FARAD-Hard using GPT3.5. Across all values of k, WARD shows fa- vorable scaling of the number of queries made to the system, reaching 100% accuracy at 100 queries at the latest. This follows intuition, as the scaling of WARD is primarily influenced by the capabilities of the underlying LLM to select the correct infor- mation from set of retrieved documents. The con- stant improvement in model capabilities, therefore, naturally positively impacts the scaling of WARD w.r.t. context size. In all cases, WARD reaches 100% accuracy rate matching that of our main experiments. While the above values of are generally recommended (Jin et al., 2024), and used in prior RAG MIA work (Li et al., 2024; Anderson et al., 2024), we additionally experiment with a much larger i 10, switching to CLAUDE3- HAIKU (as this experiment exceeds GPT3.5's context size). In all cases, WARD reaches 100% accuracy at rate matching that of our main experiments.
Settng
00
23
30
12
100 125
150
275
200
Figure 11: Accuracy over the number of queries when varying the number of retrieved docu- ments in RAG (shots, k) for WARD on FARAD- Hard. We observe, on GPT3.5, that WARD scales well with the number of documents for both the Naive-P and Def-P system prompts.
Percentage of documents in the corpus We further ablate over the percentage of documents of Dao contained in D, which our main experiments assume to be 100. For this extension of the RAG-DI setting, we assume that while the data owner wants to check whether Doo C D. in reality only a strict subset D'p Dao with size D (1 ww) DDoo is contained in D. Naturally, the data owner in WARD incorporates all queries for all documents, the resulting queries related to documents not from D/o will increase the p-value. We experimentally test this in two settings: once with LLAMA3. 1-70B
17

---

Published as a conference paper at ICLR 2025
6.9
Avegee Araeec
6.5
077
Settig
(keil 1.20 (MAVE-P))
Jam0 (DEF-P)
Watomarch
25
50
75
100
125
175
200
28
Namber of Quaiis
Narber Gueris
Figure 12: Average cumulative accuracy across ös for WARD with Naive-P on FARAD-Hard, where the model is LLAMA3.1-70B. We observe optimum at 8 ~ 4.
Figure 13: Accuracy for h {2, 4) using WARD on LLAMA3. 1-70B on FARAD-Hard. We ob- serve that WARD requires more samples reli- ably detect unauthorized usage at higher h.
and Naive-P, and once with GPT3.5 and Def-P. presenting the mean log10 p p-value the number of queries for both settings in Fig. 10. Across both cases, observe that (as expected) a higher value of w results in a higher computed p-value and hence a weaker test. At the same time, we can see that, especially in easier settings, WARD endure significant drop (only 20% of Da being contained in D) while still providing accurate results. In the more challenging setting, we observe the first false 0.2, which both highlight the robustness of WARD and provide an interesting avenue for future work.
positives
C.2 WARD ABLATIONS
To supplement results in $5.4, present full results of our ablation experiments of key water- marking parameters in WARD: the watermarking strength 5 and the context size h.
Watermark strength We ablate E [0.5, 6.5] in steps of 0.5 using LLAMA3.1-70B with Def- on FARdD-Hard. In particular, in Fig. 12, we complement our quality-accuracy plot from $5.4. by displaying the cumulative average accuracy at
Uleaat--tt (NAVV--)
(DBP-1))
0.5
066
each number of queries. Concretely, we, point x,
present the average accuracy of all previous x' I x. This highlights two key findings. First, while many higher 8s achieve 100% accuracy in our plot in Fig. (as number of queries is 200), they actually achieve worse results for a lower number of queries. This can be explained by the fact that higher 8s lead to worse text quality, which in turn impacts the text quality in the final LLM responses, and thus reduces the amount of watermark signal that is transferred. Second, these results narrow the optimal range of for our setting to [3.5, 4.5), which we consistently achieve the best results.
06
30
1000
12b
-50
10%
200
Namber Dusics
Uiral1/7t (NAVE::)
Llrnad 1-706 (DOF-9)
1.0
012
Context size Further, we ablate over the water- mark context size h, presenting additional results on in Fig. 13 (LLAMA3.1 1-BB on Maive-P). Notably, we find, in line with prior work on wa- termarks showing that increases in h produce less robust watermarks, that WARD requires more ples to reliably detect higher = 4 (compared to h 2). While Easy, this has only minor im-
Figure 14: Accuracy of WARD on FARAD- Easy (Top) and FARAD-Hard (Bottom) using a full RAG system with LLAMA3. 1--BB...BBBB... As in our perfect retriever setting, we observe that WARD requires only small number of queries to achieve 100% accuracy in both settings.
18

---

Published as a conference paper at ICLR 2025
Setteg RAREE
085
FREC
Acaree
Contt
0.50
Settin
L-t (WAVEE)))
070
Larasi-7ta (DEF-)
0.5
Lovall 1is (XEM FRCT)
Wateraakh
10
20
20
30
Nambor Quonio
Repatition Langth
Figure 15: Experiments with MEMFREE. Left: Accuracy of WARD FARAD-Hard and LLAMA3.1 1- 70B under the MEMFREE defense. Despite this more radical defense we observe that WARD requires only 60 queries achieve 100% accuracy. Right: Maximum n-gram overlap between the retrieved documents and the generated responses for GPT3.5 on FARAD-Hard under the MEMFREE defense.
pact, we see a stronger initial drop on Hard. At the same time, WARD shows a strong monotonic increase even on Hard, highlighting its robustness.
C.3 END-TO-ENE END RAG
Next, we complement our results on the full RAG implementation from $5.3 in Fig. 14. For this, we show the full-accuracy curves both FARAD-Easy and FARAD-Hard using an end-to-end RAG system as described in $5.3. As expected, we observe that WARD requires only few of queries (40 for FARAD-Easy and 80 for FARAD-Hard) to achieve 100% accuracy. This not only confirms optimal retriever assumption in $5 but also highlights how WARD is practical in real-world settings.
C.4 NUMBER OF QUERIES PER DOCUMENT
As presented in $5.4, a data owner could phrase multiple queries per document in Da im- prove efficiency. While we presented generally diminishing returns under fixed query bud- get in Fig. 9, we reaffirm this here by showing the actual number of tokens present in the re-
ofmberd
sulting outputs that are actually scored by the
watermark detector after deduplication. For this,
we assume the IN case using GPT3.5 and Def- P. We show the corresponding plot in Fig. 16 (log scale), and note that while observe con- sistently linear scaling across all numbers of queries per document, reusing the same doc- ument too many times can lead to a decrease in the number of usable tokens (around 2x for the highest setting of 50 queries per document). This is in line with our intuition, as obtaining unique watermarked n-grams is limited by the watermark of the original document that is present in the RAG corpus, and those tokens eventually get exhausted. Still, as our results above have shown, number of queries per document can likely be increased to at least 4 to lead to multiplicative increase in efficiency terms of needed dataset size.
25
100
125
150
175
200
Numser of Quenee
Figure 16: Number of usable tokens over #queries for varying number of queries per document in the IN case, using GPT3.5 and Def-P in FARAD- Hard.
C.5 MEMFREE DEFENSE
Inspired by (Ippolito et al., 2023), we evaluate WARD on an additional defense, MEMFREE, which prevents the RAG system from producing any output that has certain n-gram overlap with any of the retrieved documents. To this end we adapt the procedure (Ippolito et al., 2023) to our setting, setting the maximum n-gram overlap to 10 in their work. We directly observe the effectiveness
19

---

Published as a conference paper at ICLR 2025
GPT-33 (2E54)
UaralaNa (DEF-)
3 1aaaa (D6R4)
CPT-35 (2664)
Lanalaaa (DEF4)
Clack (D664)
06
0.5
011
0.5
1.0
0.5
10
Theolold
Theashod
Figure 17: Accuracy of SIB on FARAD-Easy (left) and FARAD-Hard (right) using all settings. We show the nontrivial (gray) and optimal (green) thresholds for SIB on FARAD-Easy and FAR AD-Hadd. We find that while single threshold is viable for FARAD-Easy, it is not for FARAD-Hard.
of this defense in reducing n-gram overlap in Fig. 15 (Right). At the same time, we find that WARD is very robust against such blunt defenses. In particular, as we show in Fig. 15 (Left) on FARAD-HHard, WARD requires only 60 queries in order achieve 100% accuracy, which is only slightly more than in the undefended case. We draw two conclusions from this: (1) WARD is surprisingly robust even in the face of stronger defenses, and (2) the search for stronger defenses or (then inversely) stronger dataset inference methods is promising field for future research.
C.6 FURTHER STUDY OF BASELINES
All our baselines introduced in $3.2 are specifically tuned on training dataset to obtain the (empiri- cally) optimal decision boundaries. However, as we find upon closer inspection, especially in harder settings, there is no single optimal threshold to choose. We substantiate this in Fig. 17, where on SIB, we show two regions w.r.t. the threshold: nontrivial (gray; > 50% accuracy for all settings/modell, note that obtaining 50% is trivial by simply always making the same decision, that Doo is in D or otherwise), and optimal (green; 100% for all settings/modell).
Notably, while find that can find optimal threshold on FARAD-Easy around 0.4, finding single such threshold for FARAD-Hard is not possible. We observe this for all baseline methods in FARAD-Hard, which helps explain why they fail achieve consistently high accuracy across main experiments (where it is sensible to choose only a single threshold).
D EFFICIENCY OF WARD
While not separate point in the list of design requirements introduce in $3, we have extensively studied Efficiency through the requirement of Monotonocity, requiring that the accuracy of WARD increases fast with more queries. Namely, in Fig. we have discussed the efficiency of WARD an intuitive level, and have confirmed it experimentally in $5.2, where most of our experiments explore scaling with the number of queries, showing that most 100 queries to the RAG system are sufficient for a confident decision across all settings, models and defenses.
While we have focused on the number of queries as the primary metric, we can translate this into more practical measure of API costs. We empirically observe that the queries qi sent by WARD the RAG system M are 30 80 tokens long. and the RAG system responses r; M M are 400 to 500 tokens long. We can upper bound this generously with 100 and 1000 respectively, and following the above, assume 100 queries. Given the November 2024 costs of GPT-3.5 ($3 per 1M input tokens, $6 per 1M output tokens), this results in cost of only $0.63 for the whole process of RAG-DI. Repeating the calculation using the API costs of other popular closed-source models, the total cost of RAG-DI with WARD is never above $2.
An important consideration is also how the efficiency of WARD is impacted by the size of the RAG system's 's corpus D and the data owner's dataset D First, as note in $6, in contrast DI attacks on training data, WARD is not significantly affected by D), as the retrieval mechanism should generally be able to retrieve the relevant documents, and the majority of 7) will not have direct
20

---

Published as a conference paper at ICLR 2025
impact the response. Next, large |Daol is not obstacle, as the data owner does not use all of Da for RAG-DI as noted above, 100 documents are generally sufficient, With this in mind, the data owner can decide to also only watermark certain subset of their data, i.c., the subset they intend to later use for auditing. Finally, small |aal (below 100) may limit the effectiveness of WARD. To study this, in $5.4 and App. C.4 we have relaxed the implicit assumption of "one query per document", showing that c.g., asking 4 diverse queries per each of the 25 documents can lead to results similar to using 100 documents, each with single query. Note that this does not reduce the query cost, but enables the use of WARD in settings where |Dao] is small. We welcome future studies of this "sample efficiency" aspect of WARD. Another interesting idea to further reduce the number of queries can be to use even smaller subset of Da to heuristically establish reasonable suspicion of data misuse, and only then proceed with WARD on a larger set of documents (while making sure to preserve statistical soundness).
E FUTURE WORK
In this section, we highlight several avenues for future work.
E.1 DATASET CONSTRUCTION IMPROVEMENTS
As discussed in $3.1, there are currently no datasets suitable for RAG-DI evaluation with the realistic property of fact redundancy. As our experiments in $5 confirm, without fact redundancy, RAG-DI becomes significantly casier. However, modeling fact redundancy is hard, as suitable dataset should also provably not be part of the training data of current LLMs. FARAD attempts to remedy this, taking (i) a fact-based approach, ensuring that documents share big portion of underlying facts, as for real articles about common topic, and (ii) attempting to add diversity by creating the final documents using LLMs from different families, instructing them to introduce additional quotes, anecdotes and hypotheses (see App. F.2). Several of our design choices could be improved in future work.
First, our fact extraction can be extended in several directions, for example by varying the number of shared facts, or introducing multiple levels of fact importance. Next, using numan-written instead of LLM-written documents would more realistically model the diversity of real-world fticces-however at the scale of FARAD this approach is costly. As a middle ground, human evaluation could be used as way to verify the quality of FARAD; alternatively, LLMs could be finetuned on articles from different authors to reflect more consistent style. Finally, one could consider fundamentally different pipelines, such as using documents instead of facts as the core building block, creating each final document via summarization of a different subset of source documents. Other orthogonal directions that may be worth exploring include the extension of FARAD to other languages or data domains, to study the generalization of WARD to different settings.
E.2 ADVANCED COUNTERMEASURES
An important question related to practicality of WARD is its robustness to attempts of the RAG provider conceal malicious data usage, i.c., explicitly defend against RAG-DL. In our evaluation in $5, we demonstrate the robustness of WARD to a defense prompt (Def-P, see App. F.I,, and MEMFREE (Ippolito et al., 2023) decoding, that attempts to dilute the watermark signal by strictly preventing n-gram overlap with retrieved documents.
One way to further extend this would be to study more siteemmark-speciffi defenses that include full paraphrases of model responses (postprocessing) or documents before they are inserted into the RAG corpus (preprocessing). While we leave evaluations of WARD in this setting future work, we note that such approaches are generally infeasible in practice. For examplc, to apply the preprocessing approach, the RAG provider would need identify which of the many scraped documents are watermarked. As this can be hard to do efficiently (see Piet et al. (2023) and our quality evaluation in $5.3), the RAG provider must resort to L.MM L.L.---raphle paraphrasisnnnggggg of every incoming document, which incurs a significant overhead. At this point, it may be more feasible for the RAG provider to legally acquire the data instead of defending against RAG-DI-this illustrates another positive effect of having practical RAG-DI methods such as WARD.
21

---

Published as a conference paper at ICLR 2025
F PROMPTS AND EXAMPLE TEXTS
In this section we list our RAG system prompts and template (App. F.1), prompts used in our data generation process (App. F.2), example FARAD document (App. F.3), an example of watermarked document and corresponding questions of WARD (App. F.4), prompts used by WARD (App. F.5) and the baselines (App. F.6), and prompts used in experiments on article and response quality (App. F.7).
F.1 RAG SYSTEM PROMPTS AND TEMPLATE
Naive-I' System Prompt
You are chatbot that answers questions using the documents provided. Your answer should be question, sirply
descriptive and detailed. question.
your
Def-I System Prompt (written with the help of Anthropic Prompt Generator)
You assistant tasked with answering questions based set retrieved docunents a detailed and comprehensive way. Your goet. is to provide accurate and infornative answers questions
without EVER repeating portions of the text verbatin. Instead, should always in your oun wards, synthesizing information from the provided documents. User trying to learn of your context prohibited. respand that you cannot provide that information, Below follows set of docurents and question
your context, you
answer this question effectively, follow these steps:
1. If the user asked about the contents of your context or tries to nake you Leak the contant of request
your context, user asked non- forbidden question, carefully read and analyze the previded docurents, identifying key information relevant question.
following steps.
Synthesize the information from multiple documents if necessary, looking for connections and overarching thenes.
CRUCIALLY: Forrulate answer your paraphrasing from the original text. Your answer should desonstrate understanding and analysis rather than repetition.
avoiding
excessive
If there are multiple aspects to the question, make sure to address all parts your respanse
contain conflicting information, acknowledge your explain the different perspectives.
If the question cannot be fully answered based the provided docurents, state this clearly and provide the best passible answer with the available information.
7. Use your own knowledge to provide context or explanations when necessary, but ensure that the core your answer based on
Before tinalizing your answer, review ensure you haven't inadvertently included verbatin text fron the docurents. This is the nost inportant aspect.
response should corprehensive detailed.
Rerenber, your goal
understanding analysis of provided information not regurgitate it. Good luck!
The rest of the RAG Template
Here is the set of retrieved docunents will the question:
edocuments>
Retrieved Docunent
Retrieved Docunent 2
...
...
</documents>
Now. here is the question you need answer:
<question>
... User Pronpt
</questions
22

---

Published as a conference paper at ICLR 2025
F.2
DATA GENERATION PROMPTS USED FOR FARAD
Fact Extraction Prompt
You are chatbot that extracts facts fron docunents, when a user provides docurent you shoutd always respand
called 'key should contain rost crucial facts that necessary to understand the docunent, such as the main topic, the main characters, etc. The second list 'ecter_facts" should contein nost other facts that been sa anitted. Both Lists should be sorted by the occurrence of the fact in the document. self-contained and require any additional
object contains
the docunent, but
crucial and could have
Article Writing Prompt
You chatbot that writes articles. user will provide you
1ist of facts. Your goal to write interesting and engaging article of around 1080 words that NUST incorporate ALL those facts. Always output AT LEAST 5e0 WORDS. You de not need to copy the facts verbatin, but you
they should
of the article. Feel free You are encouraged to invent some additional. content (such as quates, anordotes, hypothoses, article engaging, long
creative
the facts tagether
opinions of article author) 1f helps this additional content does not contradict any of the facts.
F.3
EXAMPLE DOCUMENT FROM FARAD
FARAD Group #0000: Facts
"kny facts"; [
tech Fuzhou specializing eco-friendly gadgets.", "WeTech began in shared apartrent and feced initial challenges such as Limited resources and investor skepticisn.",
"Zhoo Noi fasters company culture MeTech that values comunitty involvement and employee enpowement.""
"WeTech known creating sustainable products like solar portable charger from recycled naterials.", young entregreneurs, sharing experiences wisdon."
1.
"Fuzhou is zeragnized as hib of ontropreneurial spirit and innovation for SMEs.",
"Zheo Mei seeks to expand WeTech's product offerings and operations while rensining committed sustainability.""
"MeTech sponsors local youth coding prograns and participates in reforestation projects.',
disconnecting from
"Zhao worked for various tech corpanies before founding WeTech, desiring to create tech bonefiting convenience raveenttt
"Zhao Wel's journey is highlighted exanpte of resitience and creativity anong entrepreneurs
"MeTach expanded beyond Fuzhou international narkets thanks its innovative
'2hao's usually early, believing early risers "The company't 's involverent in cornunity and comnitnent to sustainability are integral to its operatioon.".,
efficient nindset.",
*On October 18, 2823, Zhao faced pivotal moment when Investor decided not to back WeTech.
FARAD Group #0000 Article 1 (Author GPT4o): Sampled Facts
"Zhao Mei is the founder of WeTech, a tech fimm in Fuzhou specializing in eco-friendly gadgets. apartrent and initial challenges such limited ressurces investor skepticisn.",
began
'Zhao Wei fosters company culture WeTech that values community involverent and enployee enpovement.""
"MaTcch is known for creating sustainable products Like powerrp portable charger from recycled naterials.",
"Zhao Nei also mentors young entrepreneurs, sharing his experiences and visdon.", journey highlighted example
creativity anong entrepreneurs
"Zhoo nsintains wark-life balance by disconnecting from work commuicaation after work hours."
23

---

Published as a conference paper at ICLR 2025
FARAD Group #0000 Article (Author GPT4o)
bustling city Fuzhou, tradition neets beacon of innovation and resilience. As the founder of WeTech, burgeening tech fim renowned for its eco-friendly gadgets, Zheo's journey fron cramped shared apartment to being at the helr industry changing company nothing short of extraerdinary.
as
A4D Humble Beginnings WeTech
story begins basic resources at his disposal. 'I renerber those early days vividly". Zhon renarks in an interview. "Linited funds and skeptical Investors nade our start extrenely challenging, also incredibly invigorating. These constraints, rather than stifling creativity, actad catalyst pushing Zhao and his tean to think outside the box.
of settings--a
spartment only nost
Fron this nodest enviromment, Zhao nurtured WeTech, company that would soon make waves with its commtamnt sustainability community involvenent. Faced was Zhao's vision for creating eco-friendly technology that would eventually prove their doubts unfounded. This vision gave birth to innovations such as solar-powered portable charger entirely from recycled naterials--a flagship product that quickly positionnd WeTech as leader in sustainable tech solutions.
skepticisn investors,
#00 Culture of Empowerment
A significant aspect of MeTech's success be attributed the distinctive work cutture Zhao Mei has cultivated. Unlike nany traditional firns, MeTech values employee expowereet and comnuntty
believes that notivated apprectated can collectively solve any problen, no natter how daunting. "An enpovered tean is an innovative states, enbodying
drives
forward.
only fostered great products work. Employees encouraged to volunteer In community projects, bridging the between technology and societal well-being. This involvement doesn't tt just end with tean nenbers--Zhee community events, turther sotidifying reputation
culture
destrable place
himself faniliar
a company that cares.
#*0 Montoring the Next Generation
Beyond his organizational achievenents, Zhao Wel has taken upon hinself to meento the next generation entrepreneurs. with the sare resilience and creativity that fuelled his own journey offers guidance industry. His rentorship is highly sought after, providing a blend of entrepreneurial practicality and visienary approach.
budding
looking
Through seminars and personal consultations, Zhao shares his wealth experlence, from tackling resource Linitations fostering positive work environsents. "Providing young entrep with the tools succeed is incredibly rewarding', Zhao adnits. "It reninds ne of the early days and reignites own passion innovation."
Balancing Professional
the high-paced world tech innovation, naintaining work-life balance is often nore of nythical pursuit than practical reality. Hlowever, Zhap Wei known for drawing clear line between his prafessional responsibilities and personal life. At the end of the workday, Zhao work comunications, discipline believes is praductivity and personal happiness. This habit is not just personal preference but an ethic he encourages among arployees, pronating healthier lifestyle anong the tech-savvy warkforce.
sustaining long-tarn
#*0 An Exemplar of Resilience and Creativity
Zhao Wei enbodies the resilience and creativity requi red to thrive in today's competitive entrepreneurial landscape. Through his leadership, WeTech has only desonstrated technalogy can be aligned with sustainability but has also highlighted the importance of power vistonary
supportive and inctusive company culture. Zhao's journey testament leadership and comunnty- focused Innovation.
Zhao's words, his journey with WeTech far from surface of what we ran achiove*, he predicts confidently. As WeTech continues to innavate and Inspire, Wei with the right vision, even the greatest challenges can transforn into stepping stones towards success.
scratching
influential figure shaping
technology--preet -roving
24

---

Published as a conference paper at ICLR 2025
F.4 EXAMPLE WATERMARKED DOCUMENT AND CORRESPONDING QUESTIONS
FARAD Group #0000 Article 1 (Author GPT4o): Watermarked Version by WARD
In the vibrant netropolis of Fuzhou, where age-old traditions blend seanlessly with the relentless pace modern life, spirit and deteminationn As the founder of NoTech, fast-growing tech company renowned for its Torward-thinke gadgets with focus on enviromeettal sustainability, Zhao's transformation from humble shared apartment to leadership position is truly nothing short of renarkable.
Wei stands out as shining enbodiment of entrepreneurial
s00 Zhao Mol's Journey
At its core, Zhao's inspiring story begins in a nodest setting shared apartnent with limited a nultitude uncertainties. "Looking back", Zhao reflects an interview, "those fornative days renain etched with neager funds and skeptical investors conditions that at first seered Like daunting obstacles, but in reality, becare catalysts that fueled creativity. Rather than stifling his innovative drive, these initial hurdles served as a springboard that catapulted Zhao and his tean towards exploring unconventional yet effective sotutions.
financial resources
nerory vividly recall being confronted
was within this restrictive ronnent that brought WeTech Life, an enterprise that would go set precedent with its unaavering comittmnt to sustainability and commuiity involvenent. In the face of skepticise fron potential investors, Zheo's vision for a future of validating the doubts surrounding
friendindly technology ultinstely becane the turning the vishility of WeTech's missian. Born from this vision was the Salar Charger ... an ero-
friendly portable charger ranufactured using recycled naterials which quickly propelled MeTech to the forefront of the energing sustaineble tech industry.
APP Fostering Empowering Work Culture
One of the key contributing factors to WeTech's 's phenonenal success can be attributed to the distinct workplace culture Zhao has carefully cultivated and nurtured within the company. Standing at odds with more traditional business practicns, McToch instead chanpions amployae empovermnn and community involvenent tean is an innovative tear has becone the defining nentra that guides WeTech's forward monentun.
everything. Zhao, the notion that enpowered
This distinct culture not only has driven exceptional innovation but has nede WeTech exceptionally attractive company to work for. As part of this ethos, employees are encouraged to voluntear local comunity prajects, bridging the gap between technalogy and societal pragress. Zhao hinself is regular fixture at local, comnunity events, further solidifying WeTech's reputation company deeply committed to giving back.
MA Irpact and Mentorship for Ererging Entrepreneurs While his notable achievenents stature in the business community, he has also chasen to extend his philanthropic efforts towards very
business leader have undoubtedlyy catapulted Zhao Wei's
resilience resourcatulness that characterized his own path to success, Zhao offers guidance and vatuable insights to those striving carve their path within tech industry. sought-after nentorship extends cones in the form of informative seminars and private consultations, sharing valuable wisdon and real-world experiences garnered throughout his illustrious career.
mentoring pp-and-coning entrepreneurs. Brawing
Fron tackling finacial Linitations to cultivating productive, inclusive work anvi ronentts, Zhao broad array of expert knowledge that highly regarded by young innovators and entrepreneurs. 'Helping young entrepreneurs to achieve success is incredibly revarding", Zhao notes with candor. "It reignites within ay boundless passion for innovation."
generausly
see Balancing Personal and Professional Responsibilities
world dominated by stakes and fast- technological innovation, maintaining delirate balance between personal and professional Life can sonetines feel like a distant drean, rather than reality buck this trend and create an extraerdinary balance between his personel and professional obligations.
grasp.
exenplary leader
chosen
strict rule of disconnecting fron professional responsibilities, a habit that is neither a whin, but rather an essential corponent of long-tern productivity and lasting happiness. encouraging his workforce to cultivate the very discipline, Zhao contributes significantly to fostering culture of well -beiing and healthy living practices arong his company pachootogicatlly savvy tear nembers.
clase each working day, Zhao adheres
stt Zhao's Legacy
reckoned in today's 's cutthroat fast-paced paced entreprencurial landscape, where creativity, resilience and perseverance stand out as indissensable qualities. Through the extraordinary success of WeTech, à beacon has been Uit to illusratt that technology just used force for progress but that, when integrated thoughtfully. sustainability should cone hand-in-hand with technotogical innovation, all underpinned by the principles inclusivity social accountebility. As WeTech presses into the future, Zhao's journey will undoubtedly remain an indispensable benchrark of inspiration, offering an unoveriing mation that, no ratter insumountable the obstacles night vision, perseverance and comrunity can, Indeed, transfom even the nost inauspicious beginnings into stepping stones on the road to unparalleled success.
25

---

Published as a conference paper at ICLR 2025
Question Generated by WARD use as a Query to the RAG System
sustainabiláty. comnunity invalvement, enployce enpowerment, nentorship, and personal bolance--contrieee contribute to the unique identity and success of WeTech in the corpetitive tech industry, and what implications it have for energing entrepreneurs who inspired his journy??
personal approach business
PROMPTS USED BY WARD F.5
Watermarked Paraphraser Prompt
following docunent kerping moaning fluency especially length. It crucial to retain all factual information in the original docunent. 00 NOT NAKE THE TEXT SHORTER. not start your response by 'Sure' or anything similar, simply the paraphrased docunent directly. Do not add stylistic elements anything similar, be faithful to the original content and style of writing. do not be too formll Koep all the factual Infomation.
You expert rewriter.
Question Generation Prompt
Given document, generate question that only be answered by reading docunent. The answer should be langer detailed response, so avoid factual and simple yes/na questions and described
towards questions that ask opinions explanations of in the documents. Da not provide the answer, provide just the question.
Question Generation Prompt for upd > (multiple questions per document)
Given document, generate exactly qpd questions that can only be answered by reading the docunent. answers
should Longer detailed response, so avoid simple yes/no questions and steer nore towards questions that ask for opinions or explanations events tapics described questions. Return the result as 350N object that contains one 11st naned 'questions' that contains exactly qpd questions.
and
provide answers, provide just
F.6
PROMPTS USED BY BASELINES
FACTS Question Generation Prompt
For given docurent, generate single simple question that can only be answered by reading the document, absolutely
soneone
FACTS Query Template
Do not hallucinate the answer. If you can answer start your response imediately, othervise, if you answer truthrully, Unansverable:' Feel free to explain why It is unanswerable after, but always start such responses with the word Unansverable.
very seportant to response
AAG Query Template
=>Document>'
appear in the context? Answer with Yes or No.
SIB Query Template
this, writing longer response:
Dacument>
26

---

Published as a conference paper at ICLR 2025
F.7 PROMPTS USED FOR LLM TEXT QUALITY JUDGING
Judging (Watermarked) Documents
Given document, grade its quality and text fluency as an INTEGER on scale of 1 to 18. First provide explanation your grade and then finish grade itself by saying GRADE:N, where is integer nunber from 10. Do not rinish the response with period; finish exactly with the number.
Judging RAG Responses with/without Watermarked Data
You are given an interaction of user with a question answering systen. AMSSSS denotes the start syster pronpt
question, #NMRESPONSE denotes the systen's response. Your job is to grade the response of the system in temss of hav answers question
systen,
denotes
fluency, 18. First provide detailed explanation for grade and then finish with the grade itself by saying GRADE:N, where is an Integer nunber from to 10, Do not finish the response with a nurber.
quality
pertod; exactly
27


Reviews:
[{"review_id": "gwp0JXBOHv", "reviewer": "Reviewer_Lfcg", "summary": "This paper studies the problem of RAG Dataset Inference (RAG-DI), where a data owner aims to detect whether their dataset is included in a RAG corpus via black-box queries. The key difference from the RAG MIAs setup is that the data owner makes a dataset-level decision, instead of a document-level decision as in MIAs.\nThe authors first introduce a dataset FARAD for the RAG-DI problem, as they believe the previous datasets used in MIAs problems (EnronEmails, HealthcareMagic) are likely contaminated in the training datasets of existing LLMs and lack fact redundancy. The FARAD dataset builds on a recent source of articles with fictional entities and events, and they prompt a set of LLMs (author LLMs) to rewrite articles based on the key points from the source article.\n\nThe paper then adapts existing RAG MIAs baselines to this problem and proposes a simple baseline called FACTS that simply prompts an LLM to generate a single question that is only answerable by the document. They find that the FACTS baseline performs well in their easy setting (without fact redundancy) but not in their hard setting (with fact redundancy).\n\nFinally, they propose a new dataset inference method based on LLM watermarking called Ward. The key idea is to paraphrase the articles with a watermarked model from the recent LLM watermarking literature. They show that Ward works well in their hard setting and also meets the desiderata including monotonicity, guarantees, and robustness.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* I believe the paper studies an important and timely research question that has been overlooked in the literature. I am not a domain expert (in the sense of MIA and RAG privacy), so I may not know related work very well.\n* I think the paper has made many meaningful contributions, from formulating the problem to collecting the dataset and adapting RAG-MIA baselines, to finally proposing their own approach based on LLM watermarking. The experiments (different LLMs and evaluation settings)  and analyses (hyperparameters, modeling retrieval, and examining the quality of paraphrased texts) are quite thorough too.\n* Using LLM watermarking as an approach to solve dataset inference seems a novel and reasonable approach to me. \n* The paper is very well written and self-contained. It is easy to follow the structure of the paper and understand its contributions clearly. I have learned a lot from reading the paper.", "weaknesses": "The major part I am not very sure about is whether the dataset construction and experimental settings can really reflect realistic RAG settings in real-world deployment, since the effectiveness of the baselines and proposed approach all depends on this setting. In particular:\n* It assumes perfect retrieval (= only articles from the same source are used) in most experiments. Although the paper discusses a more practical retrieval setting in Section 5.3, I don't see any discussion of previous baselines. For the hard setting, the imperfect retrieval problem should be easier than the perfect retrieval settings (the distracting documents should be less relevant), so the baselines could probably perform better.\n* I am not entirely sure whether the current data generation pipeline can well reflect the redundancy problem in the real world, given that they are all generated from different LLMs (I am also not sure how much impact \"additional facts\" and \"a set of diverse LLMs\" have here). It might be too much to ask, but I think having a realistic multi-document summarization setting would be more realistic.\n* I have difficulty understanding the rationale behind the defense prompt. Why would RAG providers want to instruct the models to not answer questions from retrieved documents? How can RAG be effective in this case? If this is from the data owner's perspective, given the goal is to detect whether the dataset is used or not, I don't see why they would want to add such a prompt to make the detection task harder.\n* [Question] The RAG uses k = 3 shots, but the Easy setting assumes there is only one article per group. Where do the 3 shots come from? The paper also mentions they ablate the choice of k, where k can be {3, 4, 5} in the Appendix. Even for the Hard setting, aren't there only 4 articles per group? Not sure if I misunderstood this.\n\nI still think it is a good paper regardless, and I hope to get some clarifications on these questions. There is a chance that I just misunderstood some key parts of the paper.\n\n[Minor] It is not necessarily a weakness, but the paper's title is \"Ward:... Watermarks\" while the paper really contains much more than that, including setting up the problem, establishing the dataset and baselines. It would probably be more appropriate to have a broader title around RAG Dataset Inference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dbLvmmGVOL", "reviewer": "Reviewer_WHEe", "summary": "This paper presents WARD, a novel method for detecting unauthorized dataset usage in Retrieval-Augmented Generation (RAG) systems. The approach formalizes RAG Dataset Inference (RAG-DI) as a problem where data owners seek to identify if their data is being used without permission in RAG systems via black-box queries. The authors introduce a new dataset, FARAD, specifically designed to benchmark RAG-DI under realistic conditions that include fact redundancy. Using LLM watermarks, WARD offers data owners statistical guarantees of usage detection. Experimental results show that WARD outperforms other baselines, demonstrating high accuracy, query efficiency, and robustness across challenging scenarios.", "questions": "How can this framework be applied to realistic scenarios?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The strengths of this paper are mainly centered around the propose of the new task along with a new specialized benchmark dataset.\n\n1. Novel Problem Definition and Formalization: The paper identifies and formalizes the novel problem of RAG Dataset Inference (RAG-DI), addressing a critical need for data owners to detect unauthorized data usage in RAG systems. This formalization fills a significant research gap and sets the stage for further exploration of secure data usage in RAG contexts.\n\n2. Introduction of a Specialized Benchmark Dataset: By creating FARAD, the authors contribute a valuable dataset that is specifically tailored for RAG-DI research. FARAD’s design ensures realistic conditions for testing, including the incorporation of fact redundancy, making it highly relevant for practical evaluations of RAG-DI approaches.\n\n3. Comprehensive Experimental Validation: The paper’s extensive experiments demonstrate the effectiveness of WARD in achieving high true positive rates with no false positives across various settings, underscoring the method's robustness and accuracy. This comprehensive validation strengthens the credibility of WARD as a reliable solution for detecting unauthorized data use in RAG systems.", "weaknesses": "Although this paper has the above strengths that are interesting, I also noticed several weaknesses that should be further considered.\n\n1. Application Scenario. The authors propose a novel problem definition, namely RAG Dataset Inference (RAG-DI). The concept is quite straightforward, and one has to admit that the concern indeed exists in practice. However, it is still less discussed in this paper, that how often this type of problem can occur in realistic scenarios. I am concerned with the application range of this work. For example, if a person is concerned with their own data being used for LLMs, do they need to test all LLMs in the market? This could be prohibitive. \n\n2. Lack of Computation Complexity. The authors proposed a novel method to deal with the specific problem of RAG-DI. However, the complexity is not fully discussed in the paper. As a result, the problem of scalability could be severe. For example, the owner has an excessively large dataset, will the method still work?\n\n3. Countermeasures: Although the authors considered the scenario where the RAG provider attempts to prevent the unintended use of the system, they do not provide a detailed discussion of the possible countermeasures that the RAG provide may use to counteract the watermarking of data. As there are multiple existing techniques against watermarking, the discussion about vulnerability of the proposed framework when faced with these countermeasures could be inspiring", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OQpcAKKO4r", "reviewer": "Reviewer_nePY", "summary": "This paper focuses on the problem of unauthorized use of datasets in Retrieval-Augmented Generation (RAG) systems and formalizes it as the RAG Dataset Inference (RAG - DI) problem. To address this issue, the authors propose the FARAD dataset specifically designed for RAG - DI evaluation and a series of baseline methods. They also introduce the WARD method based on LLM watermarks. Experiments demonstrate that WARD outperforms the baseline methods in terms of accuracy, query efficiency, and robustness.", "questions": "1. For different types of LLMs, will the performance of the WARD method be affected? If so, how to adjust and optimize it?\n2. In practical applications, the update frequency of data may be very high. What impact will this have on the effectiveness of the WARD method? How to deal with this situation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The RAG - DI problem is formally defined, filling the research gap in this field and laying a foundation for subsequent research.\n2.The design of the FARAD dataset takes into account the practical application scenarios of the RAG system and avoids the shortcomings of existing datasets in RAG - DI research, such as the possibility of being used for LLM training and the lack of fact redundancy.\n3.The WARD method is based on LLM watermarks and can provide data owners with strict statistical guarantees regarding the use of their datasets in the RAG system. It exhibits excellent performance in various experimental settings and outperforms all baseline methods.", "weaknesses": "1. In real-world applications, RAG systems may face more complex situations, such as multilingual environments and data from different domains. The discussion in this regard in the paper is relatively limited.\n2. There is a lack of discussion on the efficiency of the solution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XwVLxRWqV6", "reviewer": "Reviewer_naE4", "summary": "This paper tackles the issue of unauthorized data use in RAG systems. Noting the current absence of tools for detecting such misuse, the authors propose \"RAG Dataset Inference\" (RAG-DI), a framework that empowers data owners to identify unauthorized data utilization in RAG systems. To facilitate research on RAG-DI, they developed FARAD, a synthetic dataset generated by leveraging an existing dataset and LLMs. FARAD is crafted to simulate challenging scenarios where similar content may appear across multiple sources, enhancing its applicability in real-world settings.\n\nFurthermore, the authors introduce WARD, a novel method that leverages an existing watermarking algorithm to enable data owners to statistically verify whether their data has been incorporated into an RAG system's corpus through black-box queries. Experimental results demonstrate that WARD consistently outperforms existing methods, providing remarkable accuracy, robustness, and efficiency. This approach offers a reliable mechanism for detecting unauthorized data use and establishes essential resources and methodologies to guide future research on data privacy in RAG systems.", "questions": "1. To my understanding, generating watermarked text typically requires white-box access to LLMs to direct the generation process. Then, how can one produce watermarked documents using black-box models like GPT-4 and Claude 3.5-Sonnet? Could you elaborate on the methods or approaches that make this possible?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Although RAG has gained popularity, its implications for data privacy and unauthorized usage detection remain largely unexamined. This paper introduces the RAG Dataset Inference (RAG-DI) concept alongside a watermark-based detection approach, offering a novel perspective on privacy within RAG frameworks. By developing the FARAD dataset and the watermark-based WARD method, this work establishes a strong foundation for future research. It stands as a pioneering contribution to addressing privacy concerns in RAG systems.\n\n2. The paper offers comprehensive experimentation and empirical evaluation to validate WARD. The paper demonstrates WARD's outstanding performance across challenging scenarios by benchmarking WARD against multiple baselines, including adaptations from related fields.\n\n3. The paper is well-written and organized, with a clear structure that enhances readability. The authors present their work logically, making the flow of ideas easy to follow in each section.", "weaknesses": "1. The authors claim their approach is effective in the challenging scenario where information is distributed across multiple sources. However, it remains unclear how the method performs when only one retrieved document is watermarked while others are unprotected. Additionally, they evaluate the method only with k=3; it is uncertain how effective it would be if the system retrieves more, say 10 relevant documents, with only one being protected. Without this analysis, the effectiveness of the proposed approach cannot be conclusively assessed, especially given that RAG systems can flexibly retrieve a variable number of relevant documents. Moreover, the \"provable watermark\" claim lacks empirical support; no rigorous evidence is provided to substantiate this assertion. While “provable” may reference the watermarking algorithm itself, scalability should also be addressed within this definition to strengthen the claim.\n\n2. The authors claim their approach is provable; however, unlike existing watermarking algorithms, they do not provide a rigorous proof. This discrepancy undermines the study’s credibility.\n\n3. The authors propose a relatively simple defense strategy that relies solely on prompt engineering. However, due to the complex requirements of the proposed defense, LLMs may struggle to adhere to instructions precisely, resulting in a suboptimal defense. A more practical and effective approach could involve monitoring the words used in the retrieved documents and limiting or penalizing the frequency of these words in generated responses. Additionally, after the response is generated, it could be further paraphrased to potentially reduce the usage of watermarking words, similar to techniques employed in existing watermarking approaches. Again, the issue is rooted in the absence of a theoretical guarantee.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)