ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation
Authors: Ziyuan Luo
Yangyi Zhao
Ka Chun Cheung
Simon See
Renjie Wan
Venue: NeurIPS 2025 poster ()
Keywords: RAG
watermarking
copyright protection
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: The widespread adoption of Retrieval-Augmented Image Generation (RAIG) has raised significant concerns about the unauthorized use of private image datasets. While these systems have shown remarkable capabilities in enhancing generation quality through reference images, protecting visual datasets from unauthorized use in such systems remains a challenging problem. Traditional digital watermarking approaches face limitations in RAIG systems, as the complex feature extraction and recombination processes fail to preserve watermark signals during generation. To address these challenges, we propose ImageSentinel, a novel framework for protecting visual datasets in RAIG. Our framework synthesizes sentinel images that maintain visual consistency with the original dataset. These sentinels enable protection verification through randomly generated character sequences that serve as retrieval keys. To ensure seamless integration, we leverage vision-language models to generate the sentinel images. Experimental results demonstrate that ImageSentinel effectively detects unauthorized dataset usage while preserving generation quality for authorized applications.

ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation
Ziyuan Luo 1.2, Yangyi Zhao', Ka Chun Cheung",
Simon See Renjie Wan 1*
Departmenn of Computer Science, Hong Kong Baptist University NVIDIA AI Technology Center, NVIDIA
(ziyuanluo, csyangyizhao) hkbu, edu hk
{chcheung, Onvidia com, renjiewan@hkbu. hk
Abstract
The widespread adoption of Retrieval-Augmenter Image Generation (RAIG) has raised significant concerns about the unauthorized use of private image datasets. While these systems have shown remarkable capabilities in enhancing generation quality through reference images, protecting visual datasets from unauthorized use in such systems remains challenging problem. Traditional digital watermarking approaches face limitations in RAIG systems, as the complex feature extraction and recombination processes fail to preserve watermark signals during generation. To address these challenges, we propose ImageSentinel. a novel framework for protecting visual datasets in RAIG. Our framework synthesizes sentinel images that maintain visual consistency with the original dataset. These sentinels enable protection verification through randomly generated character sequences that serve as retrieval keys. To ensure seamless integration, we leverage vision- Vision-language models to generate the sentinel images. Experimental results demonstrate that ImageSentinel effectively detects unauthorized dataset usage while preserving generation quality for authorized applications. Code is available https:// ctttuub.commmuuozzzze http://ooiiiuaaaaaaaasetteeeel Imam//eetieeennnlllll
Introduction
Recent advances in Retrieval-Augie Augmented Image Generation (RAIG) 12345 6 have demon- strated remarkable capabilities enhancing generation quality through reference images. By retriev- ing and leveraging relevant reference images during the generation process, these methods achieve exceptional performance in challenging tasks such rare concept generation and fine-grained image synthesis [I. However, these methods heavily rely on high-quality reference image databases, the unauthorized use private datasets has become an increasingly critical concern.
Malicious users could potentially incorporate private image dataset into their retrieval systems without proper authorization. Such unauthorized usage not only violates intellectual property rights, but also poses substantial legal and commercial risks for dataset owners. Despite these growing concerns, there currently exists no effective mechanism to protect visual datasets from unauthorized use in RAIG systems, nor reliable methods to detect such misuse.
straightforward solution protect private datasets would be applying digital watermarking 7 8, [9 10) to the private image dataset. This approach, which is also widely adopted text-based petrieval-augment generation systems [ [22 13 [4], operates under the fundamental assumption that embedded watermarks can persist in the generated output [15]. However, this assumption does hold for visual content in RAIG systems. Unlike
generation where content is often directly
Corresponding author. This work was carried out at the Renjic Group, Hong Kong Baptist University.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Proeectdd dataset
Copyright infringed
Copyright uninfringed
Sentinel images,
Prompts
RAIG system (Black-box)
"A picture of PEXTxy"."
Generation module
Retriever
"A picture of 'KoVVge."
MEKIO
Reference image
KKO 13
database
KOVGGE
(a) Sentinel image examples
(b) Dataset unauthorized use detection
Figure 1: Illustration of our dataset protection. (a) Protected dataset structure showing the integration of sentinel images with the private dataset. (b) Dataset unauthorized use detection demonstrates
how random character sequences serve as retrieval keys to verify dataset usage through RAIG system, where the similarity between generated outputs and corresponding sentinel images determines whether the copyright infringed or uninfringed.
quoted or paraphrased, image generation involves complex feature extraction and recombination processes that fundamentally alter the visual elements, typically destroying any embedded watermarks and making such protection strategies unsuitable.
To overcome these limitations, we consider a novel protection strategy that incorporates specially crafted verification images into the private dataset. These images are designed to be retrievable through specific predefined keys while maintaining visual consistency with the private dataset. We refer such images as sentinel images, which, when combined with unique retrieval keys, can serve as reliable indicators of private dataset usage. By examining whether specific retrieval keys trigger the generation of content matching our protected images, we effectively detect dataset misuse.
However, implementing this strategy presents two major challenges. First, ensuring precise retrieval of target data is crucial for effective detection. While semantic-based retrieval mechanisms have been successful in text-based systems [[], such approaches face challenges in the visual domain. When dealing with large-scale reference databases, the abundance of semantically similar images makes difficult to precisely identify specific target images through semantic queries, as multiple images could match the same semantic description. Moreover, some RAIG systems [1] may skip the retrieval process when their internal generator can directly create satisfactory images based on the input prompts, making semantic-basee protection ineffective.
Beyond retrieval concerns, achieving protection mechanism's stealthy integration while maintaining authorized generation performance presents fundamental challenge. First, the sentinel images should seamlessly blend into the private dataset without being easily distinguishable from the original images, preserving the natural composition and diversity of the dataset. Second, for authorized users, the protection strategy should maintain both the retrieval effectiveness and generation quality of the original dataset, ensuring that authorized applications can fully leverage the dataset's 's capabilities for high-quality image generation.
To address these challenges, we propose ImageSentinel, protection framework that uses random character sequences as retrieval keys to enable reliable dataset verification, as shown in Figure [ Our framework comprises three key components: key generation, sentinel image synthesis, and unau- thorized use detection. The key generation component creates unique random character sequences that serve triggers for protection verification. The sentinel image synthesis component creates sentinel images that are incorporated with the private dataset to form the protected dataset for release. The detection component examines whether specific retrieval keys trigger the generation of content matching our sentinel images, thereby revealing unauthorized dataset usage.
As critical component in our ImageSentinel framework, the sentinel image synthesis follows two-step process. First, we leverage vision-languaa models to generate comprehensive descriptions of images from the private dataset, including their visual attributes, styles, tones, and other semantic characteristics. Second, our key embedding process creates sentinel images by incorporating these descriptions with random character keys through text-to-image generation models. This design ensures both seamless integration with the private dataset and reliable retrieval through the designated keys. Our major contributions can be summarized as follows:

---

We identify and formalize the problem of protecting visual datasets from unauthorized use in eetrival-augmmente image generation systems, which has become increasingly important with the widespread adoption of these technologies.
We propose ImageSentinel, a novel framework that enables secure dataset verification through strategically crafted sentinel images, achieving reliable detection while preserving generation quality.
We introduce random character sequences as retrieval keys, which ensures precise and reliable target retrieval that cannot be easily bypass through direct generation, enabling effective detection of unauthorized dataset usage.
Related work
Retrieval-
-augmeeell generation. Retrieval-augmennte generation (RAG) [6] enhances model
generation capabilities through dynamic context from external databases without additional training.
While this approach has shown remarkable success natural language processing [16] 17 18 [9], its
application in visual domains is emerging. Recent studies have explored image retrieval for generation
quality enhancement [5] 20 21 22], and ImageRAG 1 demonstrates RAG's applicability to existing text-to-ine eexttt--image models. However, the widespread adoption of these retrieval aalsbaaedddd-- approaches raises significant concerns about the protection of copyrighted image datasets from unauthorized use.
Dataset protection for RAG. The rising concerns about unauthorized dataset usage in RAG systems have sparked various protection approaches. Early research explores membership inference attacks to determine dataset inclusion through similarity- scoring and prompting techniques [23 24 25.. Another line of work investigates backdoor-based detection by embedding triggers that cause abnormal model responses [3]
26 27 Recent approaches focus on watermarking strategies, including repeated sequence insertion 12 28), with [ demonstrating effective protection through watermarked canary documents. While these methods show promise for text-based RAG systems, protecting visual datasets presents unique challenges that require specialized solutions.
Visual copyright protection. Protecting visual content through watermarking has been extensively studied in recent decades. Early approaches explore both spatial domain (29 30) and frequency domain 31 [2] for watermark embedding. Recent neural network-based approaches 7 g 10
achieve stronger protection by enabling tampering detection [33] 34], geometric resilience [35] and multi-source tracking [36). Beyond 2D images, watermarking has been explored for generative models [37
to address emerging copyright concerns. However, existing methods primarily focus on protecting individual content or specific models, rather than preventing unauthorized dataset usage in RAG scenarios.
38
39 40] and 3D representations [41 42 43)
3 Retrieval-augmmentd image generation
Retrieval-Augmented Image Generation (RAIG) enhances the generation process by leveraging
reference image database to provide visual context. Let V = (Deee R, 9) denote a RAIG system consisting of three key components: reference image database D Dasse {I1 Iz.... .... IN } containing N images, a retriever R that identifies relevant references from D and a generation module G that produces the final output. Below we describe typical workflow of RAIG systems.
Given a text prompt p. the retriever R identifies relevant references through either direct retrieval or an iterative analysis process:
R(p, Dbase) (rr, Ir., Ir.m } C Dusse.
(I)
where m is the number of retrieved images. For direct retrieval, this typically involves computing similarity scores 8, (im(f((p), fo (T,)) between the text prompt and image features, where fr()) and f.).) are text and visual encoders respectively.
The generation module G then conditions on both the text prompt and retrieved images to produce
the final output:
Lour ((p R(p, Drase) (p),
where (p)) denotes the complete generation process of the RAIG system. While different RAIG systems may implement varying retrieval strategies [I] the fundamental workflow remains consistent. This aatrivval-uugmente approach has demonstrated superior performance in various challenging
3

---

Dataset protection
Detection
Private dataset D.
Malicious system (Black-box)
Queries key ov
Retriever R
Generation module
Generated images ant
Sentinel image
Key: "
synthesis
O
Reference image database Dascc
Protected dataset Dr
ISimilarty-baas detection Infringed?
Unauthorized dataset usage
Yes!
No!
Figure 2: Overview our ImageSentinel framework. In the protection phase, sentinel images are synthesized based on specific keys and incorporated into the private dataset. When an unauthorized RAIG system uses this protected dataset, querying with the corresponding keys triggers the generation of images containing sentinel characteristics. Our detection module then analyzes these generated images identify unauthorized dataset usage.
scenarios, particularly for generating images containing rare concepts requiring fine-graines details. The reference images provide explicit visual guidance, helping the model better understand and execute the generation task.
recent work ImageRAGi [ has shown that RAIG can be implemented using different types of generation modules without requiring specific training for retrieval-based generation. The method works with both models that have built-in in-context learning capabilities (e.g., OmniGen 44 and conventional text-to-image models augmented with image conditioning capabilities (e.g., SDXL [45] with IPaadapter [46)). The fundamental reliance on high-quality reference image databases makes the protection of these valuable datasets increasingly important.
Method
4.1 Threat model
We consider two primary entities in threat model: the image dataset owner and malicious RAIG system. The dataset owner aims to safeguard their valuable image collections, while the RAIG system may attempt exploit these images by incorporating them into its retrieval database.
Dataset owner's goals. Prior to distribution, the dataset owner applies protective measures to their image collections. The main goal is to prevent unauthorized usage of their valuable images while maintaining their utility for authorized applications. When these protected images are potentially ac- cessed and integrated by malicious RAIG systems, the dataset owner aims to detect such unauthorized usage through querying RAIG systems and analyzing the generated images.
Dataset owner's background knowledge and capabilities. A typical RAIG system consists of three key components: reference image datahase, a retriever, and generation module. The dataset owner can input queries the RAIG system and analyze the generated outputs, but cannot directly access the reference image database or the parameters of the generation module. We consider a practical black-box setting where the dataset owner can only observe the system's 'ss input-output behavior. The dataset owner can preprocess their image collections before distribution, but cannot modify the images once they distributed.
4.2 Problem formulation
Our core approach is to incorporate sentinel images D, into the private dataset D,. These sentinel images serve as traceable indicators of unauthorized use when the dataset is used without authorization by RAIG system V (Dowe R, G). As illustrated in Figure 2 our method operates in two principal phases. During the dataset protection phase, the dataset owner first generates unique retrieval keys, then synthesizes corresponding sentinel images D, and strategically incorporates them into the private image dataset Dp prior to release. In the detection phase, the dataset owner queries the suspected RAIG system with the generated keys and determines unauthorized use by evaluating the generated images against two hypotheses:
Ho The generated image shows no influence from sentinel images.
H The generated image exhibits characteristics of sentinel images.

---

Reference image
Key: "VasWiW"
Sentinel image
M
"ubject::
"type: "food", "orief description": "A plated
dish featuring slice of ...".
GPT-40
"style": {
GPT-4e
Wiw
"visual type*: "photograph". characteristicst: "contemporany"....
Attribute extraction
Key-guided image embedding
Figure 3: Our sentinel image synthesis pipeline. Given a reference image from the private dataset, we first employ the vision-language model M to extract comprehensive semantic attributes, including subject matter, visual style, and detailed descriptions. These extracted attributes, combined with specific key ("VasWiW in this example), are then fed into the text-to-image model T to generate a sentinel image that maintains visual consistency while naturally incorporating the key.
If the generated images exhibit characteristies matching our sentinel images, this provides strong evidence that the RAIG system has incorporated our protected dataset into its reference database.
Key generation. Let K. denote the space of retrieval keys, where each key k K. consists of random combinations of uppercase and lowercase letters (e.g., "VasWiW") that are highly unlikely to appear in regular prompts. These uniquely generated codes serve two purposes: they ensure minimal interference with the RAIG system's 's normal operation since users rarely input such random strings, while providing distinct triggers for detecting unauthorized dataset use.
Sentinel image synthesis. Let Dp = {[1, I2..... IN) denote our private image dataset to be protected. We create sentinel dataset D, and incorporate it into the original dataset form protected dataset Dr Dp U D.,, where |D,1 < (Dpl. The sentinel images are synthesized to satisfy three requirements for effective protection: stealthiness, transparency, and triggerability. The stealthiness property, achieved through vision-language models, ensures the sentinel images maintain visual and semantic consistency with D,, making them indistinguishable from legitimate samples. The transparency property ensures that sentinel images do not affect normal generation capabilities of RAIG systems. The triggerability property guarantees that sentinel images can be reliably triggered by our specially designed keys K. enabling accurate detection of unauthorized use.
To achieve these properties, we design a sentinel synthesis algorithm S Dp x K D.. which takes both the private dataset and pre-defined retrieval keys as input to generate sentinel images I, E DD. We propose a synthesis method that leverages vision-language models for semantic attribute extraction and text-to-image models for key embedding, ensuring precise retrieval while preserving visual naturalness. The detailed synthesis process is discussed Section 4.3
Unauthorized use detection. Let & I I + R denote our detection function that measures the visual similarity between two images. Given a suspected RAIG system, we query with our pre-defined keys & K. to obtain generated images { The detection score is computed the average similarity between generated images and their corresponding sentinel images:
Σ AEK
(1 1
K.
where T& and 1 denote the generated image and sentinel image corresponding to key the detection score exceeds pre-defined threshold 7), determine that D, C Drase indicating that the RAIG system y has incorporated our private dataset. The specific implementation details of the detection function are discussed in Section 4.4
4.3 Sentinel image synthesis
In this section, describe our sentinel synthesis algorithm S in detail. As shown in Figure our synthesis process consists of two main stages; attribute extraction and key-guided image embedding.
4.3.1 Semantically consistent attribute extraction
To generate sentinel images that seamlessly integrate into the private dataset while keeping the original Dp untouched, we propose an approach leveraging vasion-language models (e.g., GPT-4o [47).. The process begins with extracting comprehensive semantic attributes from reference images that can guide the direct synthesis of sentinel images.
5

---

Attribute extraction. To obtain sentinel images that maintain semantic consistency with the private dataset, we first randomly select reference images from Dp. For each reference image 1, we leverage proxy vision-language model M to perform semantic analysis. The model extracts a comprehensive set of semantic attributes A = { 02, ...,, On) and generates detailed description d, that captures the image's key characteristics:
A. d, +- MM,).
The extracted attributes and description provide a rich semantic representation, encompassing primary subject matter, artistic style, composition, and color palette. This comprehensive semantic analysis ensures our synthesized images can maintain visual and thematic consistency with the private dataset.
4.3.2 Key-guided image synthesis
To establish reliable connections between sentinel images and their corresponding retrieval keys for constructing our sentinel dataset 'D, in a black-box setting, we leverage the text-to-image model T for direct key embedding. For given of semantic attributes A, description d,, and retrieval key we first construct a template-based prompt pı: that describes the desired key-specific modifications while preserving the original semantic properties. The sentinel image is then generated through:
Ig T(A, d.A.d.pp.. Pk)-
Given the description from A and d,, and the target key k, the prompt pi: is structured as follows:
Create an image based on: [original description A and d. ... The characters "[key k]"" must be prominently visible while naturally integrated into the scene.
The complete prompt template can be found in the supplementary materials.
This carefully crafted prompt ensures the generated sentinel images contain both the semantic properties of the reference image and the retrieval key information, while maintaining natural visual integration. Our design achieves both stealthiness and transparency by keeping the original Dp untouched. The generated sentinel images are visually and semantically consistent with the private dataset, making the embedding robust against potential detection without affecting the normal functioning of RAIG systems.
4.4 Unauthorized detection
Through sentinel synthesis algorithm S, we obtain the pre-defined keys K. and sentinel dataset D., Data owners can incorporate D. into their original dataset D, to construct the protected dataset DP for release. To detect whether a RAIG system V (Drase. R. g) where Drese is unknown has incorporated Dp its retrieval reference database without authorization, we leverage the embedded connections between sentinel images and their corresponding keys: if the system generates images with high visual similarity to our sentinel images when queried with these keys, it likely has used DD, as its reference database.
Query construction and response collection. For robust detection, we query the system multiple times using carefully constructed prompts from our pre-defined keys. Specifically, for each key E XC, we construct a prompt du and obtain the generated output _ = v ((x)) to compare with the corresponding sentinel image 7.. Below an example prompt y that fed into the system V:
A "[key k]". STRICT WARNING: ... Output the exact "Ikey k]" only.
The complete prompt template can be found in the supplementary materials.
Similarity-based detection. To quantify the visual similarity between generated and sentinel images, we leverage DINO [48] to compute the similarity score @ @ Iet = cos cos(fuino (T&,), fDUNO (I))) using its vision transformer encoder. Following Equation (3). we aggregate these similarity scores across multiple query-response pairs to obtain comprehensive detection score. When this aggregated score exceeds pre-defined threshold n). determine that D, C DD Druse: indicating the RAIG system has incorporated our private dataset.
4.5 Implementation details
Unless otherwise specified, we utilize GPT-4o 447 as our proxy vision-language model M and text-to-image model T due to strong capabilities in attribute extraction, and set the key length to characters. For the RAIG system implementation, employ three generation modules: SDXL 45

---

Sentinel images
GPT-4o w/ Prvvv dataset w/o Priv. dataset w/ riv. dataset w/o Priv. dataset w/ Priv. dataset w/o Priv. dataset
SP -aadateer
OmniGen
tanC
tzZFz
T2KK
0.895
0.152
0.851
0.217
0.973
0.147
Oag CA
EC39
YBEEPT
CAUEEA YBEEP7
YBEF7
TECH
YBEEPT
0.673
0.113
0.745
0.234
0.755
0.345
ESL THP I HS
EhPbHS
ES
EHIN HS
0.878
0.179
0.303
0.218
0.946
0.157
Figure 4: Qualitative comparison of generated images across different RAIG systems on the LLaVA- Pretrain Dataset [5I] The leftmost column shows our sentinel images, while the remaining columns
show the generation results from SDXL 45 +51+ttaadppter 46), OmniGen 44 and GPT-4o 47 both
with and without access to the private dataset. The numbers below each generated image indicate its DINO similarity score with respect the corresponding sentinel image in the lefumost column.
Images the reference database Reference images
Images generated
Images
Images generated
through RAIG
reference database
through RAIG
Sentinel images
w/ Priv.
w/o Priv.
Reference
Sentinel images
w/ Priv. dataset
wlo Priv. dataset
images
B8nDRM
OAdgix
Laveerse AW
CANDD
Cir
CICCIKI
Clck
NFtmur
Figure 5: Qualitative results on Product-10K (52] dataset. From left to right: reference images from the original dataset, our generated sentinel images, images generated through RAIG with access to sentinel images (w/ Priv. dataset), and images generated without access (w/o Priv. dataset).
equipped with the ViT-H IP-adapter lision-language models as RAIG retrievers: CLIP "VIT-B/32"
[46), OmniGen [44] and GPT-4o 447. We experiment with two
(0 to search for reference images. For unauthorized use detection, employ DINO "VIT-S/16"
[49 and SigLIP "ViT-B/16"
44 and
use the cosine similarity between normalized DINO features as the metric for comparing generated images with sentinel images. All experiments are conducted NVIDIA Tesla V100 GPUs.
Experiments
5.1 Experimental settings
Baselines. We compare our method with three baselines adapted from Ward 112 Although Ward was originally designed for text RAG dataset protection, we adapt its core methodology create baselines for image dataset protection. Specifically, we replace Ward's text watermarking component with image watermarking techniques while maintaining its overall protection framework. We implement two variants based on different watermarking methods: 1) Haad-H,DDDN,, which incorporates the HiDDeN 7 watermarking method using deep neural networks to embed imperceptible watermarks; and 2) Ward-FIN, which utilizes FIN [00 watermarking technique leveraging Bosebased models for watermark embedding. For each variant, apply the corresponding watermarking technique to the entire private dataset following the protection strategy of Ward [2]
Datasets.
We conduct experiments two datasets to evaluate our method. For the LLaVA Visual Instruct Pretrain (.LLeA--Pr))rain) Dataset (5I), we use a subset containing 10, 000 images as the reference image database, which consists of diverse images covering various visual concepts and

---

Table I: Visual similarity between sen-
Table 2: Comparison between ImageSentinel and semantic- tinel images and their reference images based retrieval on triggering rate and retrieval accuracy on
under different text-to-image models on the
Dataset [51] with CLIP [49] as the
the LLLVA--Pretrail Dataset [5T]. t in- retriever. 1 indicates higher values are better. The best dicates higher values are better. Best performances are highlighted in bold.
performances are highlighted in bold.
Method
Triggering ratef"
Retrieval accuracyt
Model
CLIPt
DINO1
SigLIPt
MoCof
SDXL 45
OmniiGen [4
Hit@1
Hit@
Hite5
GPT-4o
0.663
0.609
0.657
0.835
ImageSentinel Semantic-based
100,0%
100.0% 39.0%
69,7% 58.3%
73.8%
74.6%
SDXL
0.578
0.328
0.543
0.732
21.3%
71.7%
83.7%
we utilize the test split containing 30, 000 product images as the reference image database, which provides diverse product categories and practical relevance to real-world commercial scenarios. For the evaluation of target retrieval accuracy and generation quality preservation, we utilize BL.IP (3 synthetic captions. Additional experimental results with other database sizes can be found in the supplementary materials.
scenarios. For the Product- dataset [52].
Evaluation metries. We evaluate our method using multiple metries across different aspects. For stealthiness, we employ CLIP 49). DINO [48), SigLIP [50], and MoCo [54] similarities to measure the visual consistency between sentinel images and their reference images. For target retrieval accuracy, we use Hit@1 I, Hit@3, 3, and Hit @5 5 to measure the precision of retrieving the intended target images. For detection performance, we employ Area Under Curve (AUC) to measure the detector's overall discrimination ability, along with TPR at 1% FPR (T@ I 1&F) and TPR at 10% FPR (T@ 10%F) to assess the performance. For generation quality preservation, we follow [T] to evaluate the retrieval augmented generation capability using CLIPP 49
SigLIP 50 and DINO
48 similarities.
5.2 Main results
Qualitative results. We present qualitative results on both the LLaVA- Dataset [51] (Figure 4 and the Product-10K dataset (52) (Figure 5). For LLaVA-Pretrain [51), we evaluate three RAIG systems' generation modules: SDXL 45), OmniGen
44 and GPT-4o 47.. When these systems maliciously incorporate our protected private dataset, they generate images highly similar to our sentinel images, as evidenced by the high DINO similarity scores 48 In contrast, systems operating without unauthorized dataset access generate significantly different images with lower similarity scores, despite receiving the same random character sequences as prompts. For Product-10K [52], we use SDXL (5) equipped with IP-adapter [46] as the generation module and SigLIP (50] as the retriever. Our sentinel images naturally incorporate key characters while preserving essential product characteristies such as style, color scheme, and packaging design. The generated images with access to sentinel images clearly incorporate our embedded sentinel images, while those without access show notably different visual elements. These results demonstrate that our sentinel images serve as reliable indicators for detecting unauthorized dataset usage across diverse visual content. Additional visual results can be found in the supplementary materials.
Stealthiness. To evaluate the stealthiness in our ImageSentinel framework, we experiment on the LLa VA-Pretrain Dataset (5I) with two text-to-image models: GPT-4o [7] and SDXL [45). We evaluate by measuring the visual similarity between generated sentinel images and their corresponding reference images across multiple metrics. As shown in Table [ sentinel images generated by GPT- 4o [7 demonstrate superior visual consistency across all metrics compared to those generated by SDXL 45 These results validate that ImageSentinel with GPT-4o as the text-to-image model can create visually consistent sentinel images.
Target retrieval accuracy. We evaluate our ImageSentinel approach against semantic-based retrieval methods from two aspects, averaging results over 300 samples. First, following ImageRAG [[]. we use GPT-4o [47) to assess whether different protection methods can successfully trigger the retrieval process in RAIG systems, as some systems may bypass retrieval if their generators create satisfactory images directly. Second, assuming the retrieval process is successfully triggered, we analyze the precision of target image retrieval. Similar to text dataset protection [2], semantic-basee retrieval first extracts semantic information from images using vision-language models and then performs retrieval based on these semantic descriptions 53 As shown in Table 2 our method achieves higher triggering rates across different RAIG systems compared to semantic-based retrieval, demonstrating its effectiveness in enforcing the retrieval process. Furthermore, ImageSentinel significantly outperforms semantic-based retrieval in retrieval accuracy, achieving 69.67% Hit I
8

---

Table 3: Detection performance under different numbers of queries on the LLaVA- -reerraann Dataset [5]] reporting AUC. TPR at 1% FPR (T@ 1%F), and TPR at 10% FPR (T@ 10%F). indicates higher values are better. Each value is averaged over 100 trials. repeated 5 times, with 95% confidence intervals reported in subscript. The best performances are highlighted in bold.
ImageSentinel (Ours)
Ward-HiDDeN
[2
Ward-FIN 12]
RAIG
Query
AUCT
TØ 1%PF
TØ 10%Ft
AUC1
TØ 1%Pt
T 10%FT
AUC1
Té 1%FF
Te 10%FF
SDXL.
45
3
10
4.974u.cus
1.000
0.933u.012
1.000,.coc
0.958u..0s
1.0000.co0
0.5856.012
0.562a,u
0.054
0.0400.ur
0.2146.055
0. 0.194auu
0 0.5890.02
0. 0.500G..44
0.0022..0s
0.008
0.068
0.073
1,000.000
1.000.cm
1.000..00
0.61
0.07
0.215ques
0.37
0.013c.0sa
0.118
OmniGen
3
10
0.877lo.015
0.589u.ces
0.5844.com
0.922a.ces
0.974
0.744p.01
0.525
0.022o.021 0.542o.021
0.021
0.154p..15
0. 1.030,.00
0. 0.528
0.530
0.017c.01
0.023c.017
0.142
0.121
20
1.000 0oc
0.996 cos
1.000b.co .o0
0.600p..07 007
0.026g m1
0. 0.160
0. 0.538 055
0.032 0244
0.130
GP
0.9836.cos
0.96-o..0s
0,97
0.530 ois
0.017 oLs
0. 0.116o.055
0. 0.521o.or
0.010c.00z
0.096i ose
37
10
1.000..coo co0
1.000.coo
1.0000.com
0.555
0.026
0.1 16.
0.016G..12
0.0094
20
c00
10000.000
1.000..000
0.61
0.052..44
0. 0.1922um
0. 0.530c.0s
0.017oms
0.11
accuracy and surpassing semantic-based based retrieval. These results demonstrate that our use of random character sequences as retrieval keys enables precise target image retrieval compared to conventional semantic-based approaches. The experiments use CLIP 40] as the retriever, and results with
SigLIP [50) as the retriever can be found in the supplementary materials.
Unauthorized use detection performance. To evaluate the detection capabilities of our protection mechanism, we compare ImageSentinel with baseline approaches under varying numbers of queries on the LLaVA-Prerrain Dataset [51] and the Product- dataset (52)
Table presents the detection performance on the LLaVA-Pretraie dataset across three RAIG systems with varying numbers of queries. Our ImageSentinel consistently outperforms both Ward- HiDDeN 12 7 and Ward-FIN [12 [0] baselines by substantial margin. With just queries. ImageSentinel already achieves high detection performance across all metrics. As the number of queries increases 10 and 20, our method achieves near- sear-perfect detection performance on all three RAIG systems, reaching AUC scores of 1.0. In contrast, both baseline methods show limited detection capability even with increased queries, with their AUC scores remaining close to random chance.
Table 4 shows the detection perfor- Table 4: Detection performance of ImageSentinel under differ- mance on the Product-10K dataset ent numbers of queries on the Product- 10K dataset 52 using
using SDXL 45 equipped with IP- SDXL 45 reporting AUC, TPR at 1 1% FPR (T@ @ F%F), and
adapter 46 as the generation mod- TPR at 10% FPR (T @ 10%F). t indicates higher values are bet- ule and SigLIP [50] as the retriever. ter. Each value is averaged over 100 trials, repeated 5 times, Our detection results demonstrate with 95% confidence intervals reported in subscript. The best consistent improvement as the num- performances are highlighted in bold.
ber of queries increases. With a single query, ImageSentinel already
Queries
10
shows strong detection capability with an AUC of 0.870. The perfor-
AUCT
TEISF:
0.870 sot
0.70
0.91
O 0.985A) onc
0. 0.998..0o
d.900l one
1.000..000
1,000 oo0
1.00ls onD
1.000..000
1.000
1.000.000
TRIOSF
0.754u.244
0.982
0.098c.
1000
1.00..000
1.000..000
mance significantly improves with 3-5 queries, achieving AUC scores of 0.989 and 0.999 respectively, database of
and reaches optimal levels when using& 8 or more queries. These results are achieved
30,000 images, suggesting that a small number of queries (3-5) is sufficient for reliable detection in practice, maintaining good balance between effectiveness and efficiency.
Generation quality preservation. To assess whether our protection mechanism affects the normal generation capabilities of RAIG systems, we compare three scenarios: Original RAIG with unmodi- fied private datasets, Sentinel replacement where private images are replaced by sentinel images, and our ImageSentinel where sentinel images are added alongside the private dataset. Using the ground-trrui captions [53] from the private datasct as prompts. we evaluate the generation quality by comparing the generated images with the original images. As shown in Table while original RAIG achieves the best performance as expected, Sentinel replacement shows significant quality degradation. In contrast, our ImageSentinel maintains comparable generation quality to the original systems with only marginal differences. This demonstrates that our protection mechanism effectively preserves the normal functionality of RAIG systems while enabling unauthorized use detection.
blation studies. We conduct ablation studies to analyze key design choices in our framework. We compare different text-to-image models for sentinel image generation. As shown in Table D GPT-40 demonstrates superior performance over SDXL across all visual similarity metrics. Moreover, SDXL shows limitations in accurately embedding character sequences into generated images (visual
9

---

Table 5: Generation quality comparison under different protection scenarios on the L..aVA-rrrrraii Dataset [5T]. "Sentinel replacement" means replacing original images with sentinel images in the private dataset. 1 indicates higher values are better. Each value is averaged over 300 samples. The best performances are highlighted in bold.
Table 6: Ablation study on key
lengths (Len.) on the LLaVA- Pretrain Dataset [51]. The query number is 5. 1 indicates
higher values are better.
RAIG
Original RAIG
Sentinel replacement
InngeSentinel (Ouns)
Len.
AUC1
T@IST"
T010ST1
CLIPT
SigLiPt
DINO
CLIP
Sigl.PP
DINO!
CLIP
SigLiPT
DINO!
0.965
0.848
0.943
SDXL
0.776
0.747
0.616
0.708
0.676
0.461
0.743
0.005
0.997
0.980
0.982
OmniGen
0,751
0
0.648
0.447
0.727
0.002
0.031
0.972
0.911
Table Performance under adaptive attacks on the LLaVA- Pretrain Dataset [51]. 1 indicates higher values are better.
Table 8: Generation quality com- parison before and after detect-and- inpaint attack on the
Queries
Attack Method
AUC1
TPREI&FPRY
TPR@ 10&FPRT
Dataset [51] using SDXL +
5
No Attack
0.99
0.98
0.99
adapter [46]. f indicates higher values
Detect-and-Inpannt
0.62
0.15
are better.
No Attack
1.0
1.0
1.0
Detect-and-A Inpaint
0.91
0.65
0.82
Attack Method
CLIP
Sigl
DINO!
100
No Attack
1.0
1.0
No Attack Desoct-and--Ipainst
0.772
0.743
0.605
Detect-ond-Inpaing Inpaint
0.98
0,94
0.96
0.769
0.733
0.597
examples are provided in the supplementary materials). (2) We evaluate two dataset protection strategies: our ImageSentinel approach and sentinel replacement which replaces original images with sentinel images. Results in Table[3]show that our approach better preserves the original generation capabilities of RAIG systems. (3) We investigate the impact of key length by comparing random character sequences of 4, 6, and 8 characters, As shown in Table 6 a key length of 6 achieves the best detection performance across different metrics, while both shorter and longer keys lead to decreased effectiveness. This may be because shorter keys lack sufficient uniqueness while longer keys introduce redundant patterns that complicate the detection process. More ablation study results can be found in the supplementary materials.
daptiie Attacks We investigate detect-and-inpain adaptive attack where adversaries attempt to remove sentinel images before indexing the database. This attack detects text regions using EasyOCR (55) and inpaints them using Stable Diffusion 2.0 Inpainting (56), aiming to neutralize protection while maintaining RAIG system utility.
Table 7 shows the detection performance under this attack on the LLa Dataset [51 using SDXL 45) The attack significantly degrades detection performance, particularly with a small number of queries where AUC and TPR metries drop substantially. As the number of queries increases, detection performance improves, demonstrating that our method can still achieve reliable detection with sufficient queries. Table g shows that this attack maintains generation quality with minimal degradation across all metrics, indicating adversaries can neutralize our protection while preserving system utility. However, indiseriminately removing all detected text may eliminate important semantic information such as brand names or product labels crucial for retrieval and generation.
6 Conclusion
In this paper, we present ImageSentinel, novel framework for protecting visual datasets from unauthorized use in netrieval-Augmentte Image Generation (RAIG) systems. Our approach addresses the unique challenges posed by RAIG systems through strategically synthesized sentinel images and corresponding retrieval keys, which are unique random character sequences. By leveraging vision-language models for sentinel image generation, we achieve reliable detection capability while maintaining dataset utility. Extensive experiments demonstrate that our ImageSentinel significantly outperforms bascline protection methods in terms of detection accuracy while introducing minimal impact on generation quality.
Limitations and future work. Our method has several limitations that point to promising directions for future research. First, it relies on the text-to-image model's 's capability to embed characters in images. While GPT-4o 4 demonstrates strong performance, future advancements in text-to-in text-to-image models could further improve our approach. Second, as shown in our adaptive attack evaluation, adversaries can reduce detection effectiveness by removing text regions through inpainting, suggesting the need for more robust protection strategies. Finally, although DINO [48] proves to be a suitable similarity measure, exploring more precise metrics could further enhance detection performance.
10

---

Acknowledgements
Renjie Group is supported by the National Natural Science Foundation of China under Grant No. 62302415, Guangdong Basic and Applied Basic Research Foundation under Grant No. 2024,, and the Research Grant Council (RGC) of the Hong Kong Special Admin- istrative Region of the People's Republic of China, under GRF Grant 12203124.
References
[1] Rotem Shalev-Arkushin, Rinon Gal, Amit H Bermano, and Ohad Fried. ImageRAG: Dynamic image retrieval for affrence-guided image generation. arXiv preprint arXiv:250200441/ 2025.
[2] Aditya Golatkar, Alessandro Achille, Luca Zancato, Yu- Xiang Wang. Ashwin Swaminathan, and Stefano Soatto. CPR: Retrieval augmented generation copyright protection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[3] Robik Shrestha, Yang Zou, Qiuyu Chen, Zhiheng Li, Yusheng Xic, and Siqi Deng. FairRAG: Fair human generation via fair retrieval augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[4] Mingyuan Zhang. Xinying Guo, I Liang Pan, Zhongang Cai, Fangzhou Hong. Huirong Li, Lei Yang, and Ziwei Liu. ReMoDiffuse: Retrieval-augmented motion diffusion model. In Proceedings of the IEEF/CVF International Conference on Computer Vision, 2023.
[5] Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W. Cohen. Re-Imagen: Retrieval -ugmmeeel text-to-image generator. In The Eleventh International Conference on Learning Representations, 2023.
[6] Huaying Yuan, Ziliang Zhao, Shuting Wang. Shitao Xiao, Minheng Ni, Zheng Liu, and Zhicheng Dou. Fin- cRAG: Fine-ge fine-grained Conference on Computational Linguistics, 2025.
text-to-image generation. In Proceedings of the 31 International
[7] Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. HiDDeN: Hiding data with deep networks. In Proceedings of the European Conference Computer Vision, 2018.
[8] Zhaoyang Jia, Han Fang, and Weiming Zhang. MBRS: Enhancing robustness of DNN- based watermarking by mini batch real and simulated JPEG compression. In Proceedings of the 29th ACM International Conference on Multimedia, 2021.
[9] Rui Ma. Mengxi Guo, Hou, Fan Yang, Yuan Li, Huizhu Jia, and Xiaodong Xie. Towards blind watermarking: Combining invertible and non-invertible mechanisms. In Proceedings of the 30th ACM International Conference on Multimedia, 2022.
[10] Han Fang, Yupeng Qiu, Kejiang Chen, Jiyi Zhang, Weiming Zhang, and Ce-Chien Chang. bowed robust watermarking with invertible noise layer for black-box distortions. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
[11] Yepeng Liu, Xuandong Zhao, Dawn Song, and Yuheng Bu. Dataset protection via watermarked canaries in attrivall-uggmenn LLMs. arXiv preprint arXiv:2502.106 10673, 2025.
[12] Nikola Jovanović, Robin Staab, Maximilian Baader, and Martin Vechev. Ward: Provable RAG dataset inference via LLM watermarks. In The Thirteenth International Conference on Learning Representations, 2025.
[13] Harsh Chaudhari, Giorgio Severi, John Abascal, Matthew Jagielski, Christopher A Choquette- Chooo Milad Nasr, Cristina Rotaru, and Alina Oprea. Phantom: General trigger attacks on retrieval augmented language generation. arXiv preprint arXiv:2405.200 20485, 2024.
[14] Pengzhou Cheng, Yidong Ding, Tianjie Ju, Zongru Wu, Wei Du, Ping Yi, Zhuosheng Zhang, and Gongshen Liu. TrojanRAG: Retrieval- Retrieval-augmented generation can be backdoor driver in large language models. arXiv preprint Xiv:::2405.1111 200e24205..440. /3401, 2024.
[15] Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. In Advances in Neural Information Processing Systems, 2024.
[16] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin. Naman Goyal, Heinrich Kütteer, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval auggentte generation for knowledge- intensive NLP tasks. Advances in Neural Information Processing Systems, 2020.
11

---

[17] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan. Yuxi Bi, Yi Dai, Jiawei Sun. Haofen Wang. and Haofen Wang. Retrieval-uugrmenteed generation for large language models: A survey. arXiv preprint urXiv:2312. 109997977. 2023.
[18] Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. In-context rerrieval-angmented language models. Transactions of the Association for Computational Linguistics, 2023.
[19] Han Zhang, Rotem Shalev-Arkushin Vasileios Baltatzis, Connor Gillis, Gierad Laput, Raja Kushalnagar, Lorna C Quandt, Leah Findlater, Abdelkareem Bedri, and Colin Lea. Towards Al-driven sign language generation with non-manual markers. In Proceedings of the 2025 CHI Conference on Haman Factors in Computing Systems, 2025.
[20] Shelly Sheynin, Oron Ashual, Adam Polyak, Uriel Singer, Oran Gafni, Eliya Nachmani, and Yaniy Taigman. KNN-diffusion: Image generation via large-scale retrieval. In The Eleventh International Conference on Learning Representations, 2023.
[21] Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas Müller, and Bjöm Ommer. Retricval- -augmeee- diffussin models. Advances in Neural Information Processing Systems, 2022.
[22] Hexiang Hu, Kelvin CK Chan, Yu- Chuan Su, Wenhu Chen, Yandong Li, Kihyuk Sohn, Yang Zhao, Xue Ben, Boqing Gong, William Cohen, et al. Instruct-imagen: Image generation with multi-modal instruction. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[23] Mingrui Liu, Sixiao Zhang, and Cheng Long, Mask-based membership inference attacks for retrieval- augmented generation. Proceedings of the ACM on Web Conference, 2025.
[24] Yuying Li, Gaoyang Liu, Chen Wang, and Yang Yang. Generating is believing: Membership inference attacks against aatrieval-augmente generation. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2025.
[25] Maya Anderson, Guy Amit, and Abigail Goldsteen. Is my data in your retrieval database? membership inference attacks against retrieval augmented generation. arXiv preprint 2XXi:2405.204446 2024.
[26] Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases. Advances in Neural Information Processing Systems, 2024.
[27] Wei Zou, Rumpeng Geng, Binghui Wang, and Jinyuan Jia. PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. arXiv preprint arXiv:2402.07867, 2024.
[28] Johnny Wei, Ryan Wang, and Robin Jia. Proving membership in LLM protraining data via data watermarks. In Findings of the Association for Computational Linguisticx, 2024.
[29] Ron Van Schyndel, Andrew Z Tirkel, and Charles Osborne. digital watermark. In Proceedings of 1st International Conference on Image Processing, 1994.
[30] Nikos Nikolaidis and Ioannis Pitas. Robust image watermarking in the spatial domain. Signal Processing, 1998.
[31] Ingemar Cox, Joe Kilian, F'homsoon Leighton, and Talal Shamoon. Secure spread spectrum watermarking for multimedia. IEEE Transactions on Image Processing, 1997.
[32] Matthieu Urvoy, Dalila Goudia, and Florent Autrusscau. Perceptual DFT watermarking with improved detection and robustness to geometrical distortions. IEEE Transactions on Information Forensics and Security, 2014.
[33] Zihan Yuan, Xinpeng Zhang, Zichi Wang, and Zhaoxia Yin. Semi-fragile neural network watermarking for content authentication and tampering localization. Expert Systems with Applications, 2024.
[34] Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, and Jian Zhang. EditGuard: Versatile image watermarking for tamper localization and copyright protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[35] Linfeng Ma, Han Fang, Tianyi Wei, Zijin Yang, Zehua Ma. Weiming Zhang, and Nenghai Yu. A geometric distortion immunized deep watermarking framework with robustness generalizability. In European Conference Computer Vision, 2024.
12

---

[36] Guanjie Wang, Zehua Ma, Chang Liu, Xi Yang, Han Fang, Weiming Zhang, and Nenghai Yu. MuST: Robust image watermarking for multi-source tracing. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
[37] Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
[38] Zijin Yang. Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, and Nenghai Yu. Gaussian Shading: Provable performance-lossles. image watermarking for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[39] Han Fang, Kejiang Chen, Yupeng Qiu, Zehua Ma, Weiming Zhang, and Ee-Chien Chang, DERO: inffusion-model-erasue robust watermarking. In Proceedings of the 32nd ACM International Conference on Multimedia, 2024.
[40] Weitao Feng, Wenbo Zhou. Jiyan He. Jie Zhang, Tianyi Wei, Guanlin Li, Tianwei Zhang, Weiming Zhang, and Nenghai Yu. AguaLoRA: Toward white-box protection for customized stable diffusion models via watermark lora. International Conference on Machine Learning, 2024.
[41] Ziyuan Luo, Qing Guo, Ka Chun Cheung, Simon See, and Renjie Wan. CopyRNeRF: Protecting the copyright of neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
[42] Ziyuan Luo, Anderson Rocha, Boxin Shi, Qing Guo, Haoliang Li, and Renjie Wan. The NeRF Signature: Codebook-aided watermarking for neural radiance fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[43] Xiufeng Huang. Ziyuan Luo, Qi Song, Ruofei Wang, and Renjie Wan. MarkSplatter: Generalizable watermarking for 3D gaussian splatting model via splatter image structure. In Proceedings of the 33rd ACM International Conference on Multimedia, 2025.
[44] Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shut- ing Wang, Tiejun Huang, and Zheng Liu. OmniGen: Unified image generation. arXiv preprint arXiv:2409.11 11300 2024.
[45] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffiusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
[46] Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. AP-Adepter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.0671 2023.
[47] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. GPT-4o system card. arXiv preprint arXiv:2410.21272 2024.
[48] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
[49] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
[50] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2023.
[51] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems, 2023.
[52] Yalong Bai, Yuxiang Chen, Wei Yu, Linfang Wang. and Wei Zhang. Products- 1OK: A large-scale product recognition datasct, arXiv preprint ar Xiv::2008. 10545, 2020.
[53] Junnan Li. Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre- pre-training for unitied vision-language understanding and generation. In International Conference on Machine Learning, 2022.
13

---

[54] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2020.
[55] Jaided Al. Easyoer. http:////gtaibbboomm/dddd/////// 2020.
[56] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffussio models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
14

---

A Additional results
In this section, provide further experimental results to evaluate the effectiveness of our ImageSen- tinel framework. We include additional qualitative comparisons across different Retrieval-Augmented Image Generation (RAIG) systems, analysis of sentinel image generation using various text-to- image models, and a study on the impact of different key configurations. Finally, we validate the effectiveness of framework by testing across different reference database sizes and datasets.
A.1 Additional qualitative results
We present an extended qualitative comparison of image generation results across different RAIG systems in Figure We evaluate SDXL [45] with IP-adapter [46], OmniGen [4], and GPT-4o 47). comparing their generations with and without access to our protected dataset. The test cases span diverse visual domains including natural landscapes, food photography, logo designs, interior spaces, urban scenes, product photography. and group photos, allowing for comprehensive evaluation of our protection mechanism across different visual contexts.
The comparison reveals distinct generation patterns based on dataset access. Systems with access to the private dataset tend to generate images that share similar visual styles with our sentinel images, especially in terms of overall composition and color schemes. This similarity is particularly noticeable in complex scenarios like architectural interiors and urban nightscapes. In contrast, systems without dataset access produce substantially different results; while they might capture basic themes, their generations show significant divergence in visual appearance and composition. This clear distinction between protected and unprotected generations across all tested domains and RAIG architectures provides additional evidence supporting the effectiveness of our protection mechanism, complementing the quantitative results presented the main paper.
A.2 ImageSentinel with different text-to-image models
We compare the sentinel image generation capabilities of three eext-to-image models: SDXL [45), OmniGen [44), and GPT-4o 477. as shown in Figure 7 Using the same random character sequences as keys, we evaluate each model's 'ss ability to embed these characters while maintaining visual quality.
Among the three models, GPT-4o [7] achieves the best performance, naturally embedding the character sequences into diverse scenes without compromising image fidelity. The characters appear as a coherent part of the generated images, seamlessly integrated into the context. In contrast, SDXL 45) and OmniGen (4) generate visually appealing images but fail to reliably incorporate character sequences. These findings highlight GPT-4o's capability as the preferred choice for sentinel image generation in our framework.
A.3 Retrieval triggering in ImageSentinel
Our approach relies on triggering the retrieval mechanism in black-box RAIG systems through carefully designed prompts. The goal is to make RAIG systems retrieve sentinel images from their reference database, allowing us to detect private dataset misuse by measuring the similarity between the generated outputs and our sentinel images. However, some RAIG systems (I first evaluate whether they can generate satisfactory images directly without retrieval - if the generated images match the input prompts, they may skip the retrieval process entirely.
In text-based protection methods, mamantic-based prompts are commonly used to retrieve target documents [2] Following this convention, we compare two prompt strategies: semantic-based prompts that describe image content, and our proposed key-based prompts that contain random character sequences. To evaluate their effectiveness in triggering retrieval, follow ImageRAG (I] by first generating images using SDXL [45] and OmniGen [44] without reference database access. We
then use GPT-4o [47 to assess whether the generated images match their corresponding prompts M-
Image matching evaluation prompt
Does this image match the prompt "[prompt ? only answer yes
consider both content and style aspects.
15

---

Sentinel images
SDXL+PP-ades -Paapter w/ Priv. da taset wo riv. dataset w/ Priv. dataset w/o Priv. dataset w/ Priv. dataset w/o Priv. dataset
OmniGen
GPT-4o
HFjess
PT
STAY Cool
HEJESS
HEES
GsASGa
GsASGo
GSASGd
GSASGd
22
Klapub
Ktapub
KtpulD
Ktapub
Ktapub
ExqNj
EXQNii
ixqNjji
LeOLGS
ULOLGS
CREER JACKAS'S
UcOIGS
LCIG
IR
UhRGVu
UhRGVn
UhRGVu
UHRGUV
UhRGVu
Ing ftct
inQ/eT
inQfeT
inOF-T
inQfcT
a
inQfcT
INQ NQIFCT
YQAASt Abcm
YGXAST
K
YQU
SQUAST
UGMiSw
LeChte LGMILL
UGMSWL
Figure 6: Additional qualitative comparison of generated images across different RAIG systems on the LLaVA-Pretrain Dataset (51). The leftmost column shows our sentinel images, while the remaining columns show the generation results from SDXL 41)+-P-ad (46), OmniGen 44). and GPT-4o [47]. both with and without access to the private dataset. Visual comparison shows higher similarity between sentinel images and generations from systems with private dataset access (w/ Priv. dataset), while systems without access (w/o Priv. dataset) produce more divergent results, demonstrating the effectiveness of protection mechanism.
As shown in Figure semantic-based prompts often allow RAIG systems to generate satisfactory images without accessing the reference database, indicated by the "yes" responses during evaluation.
16

---

ExvecQ
PYEwWJ
TFHVns
OPVFIQ
GbTLX
SDXL
TRHIINS
O mnnGGe
GPT-4o
JGFDHS
PIENU
TEAVINS
GtrTLX GILLY
Exvec
TEHVNS
WdcQGv
AvgpbO
hYKANE
vT mkrp
dPLii
S0G SOGVFY
SDXL
Mo+
OmniGen
SOCVP
dPLkai
GPT-40
Figure 7: Comparison of sentinel images generated by different text-to-image imo-image models on the LLaVA- Pretrain Dataset [51] Each row shows the generation results from SDXL (45), OmniGin 44 and GPT-4o 47 respectively, using the same random character sequences keys.
In contrast, key-based prompts (full prompts detailed in Section B.4) consistently fail generate matching images without database access, receiving "no" responses. These results confirm that our key-based approach can effectively enforce retrieval, enabling reliable detection of unauthorized dataset use.
A.4 ImageSentinel with varying key lengths
We examine the impact different key lengths (4, 6, and 8 characters) on the quality and effectiveness of sentinel image generation. As shown in Figure all key lengths successfully embed the characters into the images. However, the visual examples reveal that shorter keys (4 characters) lack sufficient uniqueness, while longer keys (8 characters) introduce redundant patterns that reduce integration quality. key length of characters can achieve optimal detection performance by balancing uniqueness and integration quality.
17

---

Semantic-based prompt
SDXL
OmniGen
SDXL
OmniGen
the gnome is a little
canyon wall reflects the water on sunny day
cute gnome with big hat on head a funny cartoon gnome looking up
yes
yes
yes
yes
yoga people meditating together in the lotus position in a room
the disney beauty and the beast pandora bracelet
yes
yes
Key-based prompt OmniGen
SDXL
SDXL
OmniGen
"HFjess"
"AlmXHM"
no
no
n1
no
"BSXUOT"
"NoTRoF"
no
no
no no
Figure 8: Comparison semantic-based prompts and key-based prompts in triggering retrieval on the VA-Prrrraii Dataset [51]. Images generated by SDXL [45] and OmniGen 44 without access to any reference database. The GPT-4o [47] evaluator indicates that both models can generate images matching semantic-based prompts without retrieval (marked as "yes"), while they fail to generate images matching key-hased prompts containing random character sequences (marked as "no"). This demonstrates that our key-based approach effectively enforces the retrieval process.
Table 9: Performance comparison of different retrievers in RAIG systems the VALaVA-Prrr PA-ererrain Dataset [51] Results are averaged over 300 samples for retrieval accuracy and 100 trials with queries for detection performance. indicates higher values better.
Retrieval accuracyt
Detection performancef
Retriever
Hit@
Hit@3
Hit@5
AUC
T@ 1 1%F
T@ 10%F
CLIP 49
69.7% 76.2%
73.8%
74.6%
0.997
0.980
0.992
SigLIP 50
80.3%
83.6%
0.999
0.988
0.998
A.5 ImageSentinel on RAIG using different retrievers
To evaluate the robustness of ImageSentinel across different retrieval mechanisms, we experiment with two vision-language models retrievers: CLIP "ViT-B/32" 44 and SigLIP "ViT-B/16" (50]. We use SDXL 45 equipped with IP-adapter (46) as the generation module RAIG. We compare
the performance in both retrieval accuracy and unauthorized use detection.
Table shows that ImageSentinel maintains strong performance across different retrievers. While SigL.IP achieves slightly better retrieval accuracy due to its improved vision-language alignment, both retrievers enable effective unauthorized use detection with high AUC scores. These results demonstrate that protection mechanism is robust different retrieval methods in RAIG systems.
18

---

O
JOZx
Key lengthe4
vkEx
ixyL
PCCV
Conh
OIKK
vkEx
JOZX
PCCV
Fu V
rxyU
ooih
gURTR
genHMn
wliszc
Key length=6
nPSeop
YOTART
ynofPn
ORR
nPSeop
wissZc
genHMM
ynofPn
YaXAST
UGMSwl
JDNNUUX
AEDWAOWA
HoPPDj
OTCEUEES
Key length=8
HHooPDiN
LsyTnVUK
aVLRRELN
DNiej
HHogPDJN
OTGXuZGS
AEDWAOWa
LsyTnvUK
aVLRRELN
VnfSZoQUU
Figure 9: Visual comparison sentinel images generated with different key lengths on the LLaVA- Pretrain Dataset [5]]. Each row shows examples using random character sequences of length 4, 6, and respectively. The corresponding key for each image shown below.
Table 10: Retrieval accuracy under different reference database sizes on the LLaVA-Pretrain Dataset [51] SigLIP is used the retriever. Results are averaged over 300 samples. indi- cates higher values are better.
Database size
Hit@ 11
Hit@3+
Hit@5t
10,000
76.3%
80.3%
83.6%
20,000
75.4%
79.3%
80.3%
50,000
70.7%
78.7%
79.3%
80,000
67.3%
78.0%
79.3%
100,000
65.6%
74.6%
78.7%
A.6 ImageSentinel on RAIG with varying database sizes
To investigate how the size of the reference database affects both retrieval accuracy and detection performance, we conduct experiments with varying database sizes from 10,000 to 100,000 images. In our RAIG implementation, we use SigLIP (50) as the retriever and SDXL 45 equipped with IP-adapter [46] as the generation module. The performance is evaluated under different scenarios.
Tables 10 and demonstrate that ImageSentinel maintains robust performance across different database sizes. For retrieval accuracy, while larger databases introduce more challenging retrieval scenarios, method maintains reasonable Hit@1 accuracy above 65% with 100,000 images. For detection performance, our method achieves consistently high AUC scores with sufficient queries.
Table 11: Detection performance (AUC) under different reference database sizes and query numbers on the VA-A-Petrriin Dataset 51. SigLIP 50 is used as the retriever. Results are averaged over 100 trials.
Number of queries
Database size
10
15
20
30
10,000
0.989
0.999
1.000
1.000
1.000
1.000
20,000
0.982
0.998
1.000
1.000
1.000
1.000
50,000
0.981
0.996
1.000
1.000
1.000
1.000
80,000
0.976
0.993
0.999
1.000
1.000
1.000
100,000
0.975
0.989
0.999
1.000
1.000
1.000
19

---

Images the reference database
Images generaled through RAIG
Images
Images generated through RAIG
reference database
Reference images
Sentinel images
w/ Priv.
w/o Phiv. dataset
Reference images
Sentinel images
w/ Priv.
wio Priv. dataset
dataset
Unyassug
okEsto
10B |
DieSa
BUGG
Wu
LKozWK
Yágoii
UEADU
UEAD6C
U
LiFeuu
TUD
YcSfXV
Uydozy
YSAM
gDcDel
WOnxIC
WonkiC
WONOE
RVOXR
RERO KA
IRIHZT
RVoxRR
Trikzt
M V1A
CarCaf
IVKLIZ
Figure 10: Additional qualitative results of ImageSentinel on Product-10K 152 dataset. From left to
right: reference images from the original dataset, our generated sentinel images, images generated through RAIG with access to sentinel images (w/ Priv. dataset), and images generated without access (w/o Priv. dataset).
Notably, with 10 more queries, we achieve near-perfect detection (AUC > 0.99) even with database size 100,000 images.
A.7 ImageSentinel on RAIG with Product-10K dataset
To evaluate the capability of ImageSentinel in real-world commercial scenarios, we conduct experi- ments on Product-10K [52]. We use the test split containing 30, 000 product images as our RAIG reference database. This dataset is suitable for evaluation due to its diverse product categories and practical relevance to unauthorized
generation. We use SDXL. equipped with IP-adapter the generation module and LiPLIP [50] the retriever.
20

---

The visual examples demonstrate the effectiveness of our approach across diverse product categories, as shown in Figure IO, For each brand category, our method generates sentinel images that naturally incorporate the key characters while preserving the essential product characteristics such style, color scheme, and packaging design. When used as references in RAIG systems, the images generated with access to the sentinel images (w/ Priv. dataset) exhibit clear incorporation of our embedded sentinel images, while those without access (w/o Priv. dataset) show notably different visual elements, validating our protection strategy.
Additional implementation details
In this section, we provide detailed implementation information about our experimental setup, including RAIG system configurations and prompt templates used for sentinel image synthesis and detection.
B.1
Internal prompts in RAIG systems for image generation
During image generation, RAIG systems internally transform the input prompt gı into system-specific formats to guide the generation process, following the approach introduced in imageRAG []] After retrieving reference images, denoted as Let, the following internal prompts are used.
For SDXL
45 with IP-adapter [46] and GPT-4o 477
Internal prompt template for SDXL+IP-adapter and GPT-4o
According to this image of [caplion], generate [prompt gs]
For OmniGen
Internal prompt template for OmniGen
According to the image of [caplion]: bangs<ddimae_lb<</ieee generate (prompt qu.)
Here, caption represents the retrieval key used to obtain Jet. The complete prompt templates are provided in Section B.4
B.2 Prompts for attribute extraction
To maintain semantic consistency with the private dataset D,, we first randomly select reference images I,. For semantic analysis of each I,. we employ GPT-40 (7) as our proxy vision-language model M. This model extracts rich set of semantic attributes A {01, .2...., and generates detailed description d, capturing key visual aspects. To ensure consistent and structured attribute extraction, prompt the model with:
21

---

Attribute extraction prompt
Analyze this image and extract only the key visual features that define its core appearance.
Provide your response as TEXT in valid JSON format! DO NOT generate images!
Output Format Requirements:
"subject":
"type": <string>, // core subject category
"brief description": <string> main visual characteristics
).
"context": {
"setting": <string>. basic environment "lighting": <string>, overall lighting condition
"color "color_scheme" scheme": [<string>] // dominant colors
1. "style":
"visual type": <string>, // e.g., "photograph", "painting", "digital art", "sketch",
"design drawing"
err__characteristiiss." <string>, // e.g., "modern", "vintage 80s", "contemporary",
"historical"
"photo_style": <string>, e.g., "professional shot", "casual snapshot", "selfie", "documentary"
"imaae_quality": <string>, e.g. "high resolution", "grainy", "film-like", "digital
sharp"
"artistic_ : <string>, e.gg, "realistic", "stylized", "abstract", "minimalist"
"overall mood": <string> e.g., "candid". "formal", "artistic", "commercial"
}.
"technical":
"resolution": ("width": <int>, "height": <int>).
limage_type" <string>
Note: Focus only on major visual characteristics and overall style. Capture the essence of the image while allowing creative freedom in recreation. Do not include any specific text, numbers, names or identifiable characters in the description.
The model's structured output provides a comprehensive semantic representation that serves multiple purposes in our framework. First, it ensures consistent attribute extraction across different images, enabling systematic comparison analysis. Second, the structured JSON format facilitates easy parsing and integration into subsequent processing steps. Third, these extracted attributes guide image generation process by providing clear specifications for visual elements, style, and composition, helping maintain visual consistency with the private dataset while allowing for creative variations in the sentinel images.
B.3 Prompts for key-guided image synthesis
After obtaining the structured output containing both detailed attributes and comprehensive descrip- tions, we move forward with sentinel image generation. This step requires careful balance between maintaining visual consistency with the reference images and effectively incorporating our protection keys. To achieve this balance, we develop a specialized prompt that leverages the extracted attributes while providing specific guidance for key integration. The prompt designed to ensure that generated sentinel images not only preserve the essential visual characteristics of the reference images also incorporate the protection keys in a natural and visually coherent manner. We guide the sentinel image generation with the following carefully designed prompt:
22

---

Sentinel image synthesis prompt
Create image with:
Content based on this description: [original description A and d,]
CRITICAL REQUIREMENTS: The characters "[key k]" MUST be prominently visible
while naturally integrated into the scene. These characters should be:
As large as possible while maintaining natural integration with the scene
Must positioned where they will be clearly visible and unobstructed
Must be the ONLY text or numbers visible in the image
Shown near-frontal angle (maximum 30-degree deviation)
Must not be blocked or obscured by other elements
Integrated naturally into the scene (e.g. as signage, displays, markings, or other contextually appropriate elements)
Should look like they belong in the scene, not artificially overlaid
The integration should maintain visual coherence with the scene while ensuring "[key k]" remains clearly visible.
Generate the image at [theigh) resolution with aspect ratio of [aspect ratiol (width:height).
Remember: The absolute clarity and visibility of "[key k]" is essential it should be easily noticeable in the image while still appearing as part of the scene. NO other text or numbers should be visible anywhere in the image.
Here original description A and d are obtained from the structured output from Section B.2 while image specifications like width, height and aspect ratio are computed from the technical attributes extracted earlier. This carefully designed prompt ensures that the generated sentinel images maintain the visual style specified by the extracted attributes while incorporating the key l in a natural and clearly visible manner.
B.4 Prompts for unauthorized use detection
After incorporating the sentinel dataset D, into the original private dataset Dp create the protected dataset DD. we can detect unauthorized use by querying black-box suspected RAIG systems. The detection leverages simple yet effective prompts that focus on generating the pre-defined keys. Specifically, for each key IC we construct the following prompt:
Black-box unauthorized detection prompt
A "[key k]". STRICT WARNING: Your response must be EXACTLY only caption "[key k]" no additional words, no descriptions, no context, and no modifications. Output the exact "[key k:]" only.
The generated output I from these prompts then be compared with sentinel images 13 using similarity metrics to determine unauthorized use has occurred.
C Broader impacts
Our work on ImageSentinel represents a significant step toward safeguarding and managing visual datasets in the era of RAIG systems, with profound societal and economic implications. By providing a effective mechanism for dataset protection, our framework fosters trust between content creators and AI developers, which is essential for the healthy growth of the AI ecosystem.
ImageSentinel empowers creators and organizations to protect their intellectual property rights effectively, promoting transparency and accountability in Al development. This protection mechanism ensures that visual datasets retain their commercial value while supporting fair compensation for their creators. By enabling dataset owners to track and verify the use of their content, ImageSentinel establishes clear standards for responsible practices, contributing to sustainable creative economy where individuals and organizations can confidently share their work without fear of misuse.
23

---

As RAIG technologies continue to evolve, dataset protection must adapt in parallel. We envision ImageSentinel as foundation for future advancements in visual dataset security. encouraging proactive approaches to emerging challenges. By integrating protection mechanisms directly into Al systems, we can ensure that the benefits of Al are equitably distributed across creative and technical communities. We hope our work inspires further research into innovative dataset protection methods, fostering a balance between accessibility and security the rapidly advancing Al landscape.
24

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims accurately reflect our contributions and are fully supported by the results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
25

---

Justification:
Please refer to Section foor detailed discussion of limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sele--peciiiatinn. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This paper is primarily empirical and does not contain theoretical results requiring formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Detailed implementation specifics and experimental settings are provided in Sections 4.5 and including key parameters, model configurations, and evaluation metrics.
The complete prompts used in the paper provided in the supplementary materials.
26

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The complete code and instructions for reproducing all experimental results are provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ btaips.cc/public//uue
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
27

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Experimental details, including model configurations, evaluation metrics, datasets, and implementation details, are provided in Sections 4.5 and 5.1 Additional details are included in the supplementary materials.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The 95% confidence intervals are reported in Table 3 and Table
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Experiments compute resources provided in Section 4.5
Guidelines:
28

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our research aims to protect dataset owners' rights and prevent unauthorized use of their data, which aligns with the NeurIPS Code of Ethics regarding data privacy, copyright protection, and responsible Al development.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the societal impacts of our work in Sections T and 6 Additional comprehensive discussion of broader impacts can be found in the supplementary materials.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
29

---

Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our research focuses on developing protection mechanisms against unautho- rized data usage, and does not release any potentially harmful models datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All models and datasets used in our experiments are properly cited the paper, and fully comply with their respective licenses and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: Our paper does not release any new datasets, code, or models. We only use existing public datasets and models for our experiments.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
30

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our research does not involve any crowdsourcing experiments or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our research does not involve any human subjects or crowdsourcing experi- ments, therefore no IRB approval or equivalent review was required.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Our research methodology does not involve any Large Language Models as important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve L.LMs as any important, original, or non-standard components.
Please refer to our LLM policy httt::::/eearipsscc//ooeeeeeeee for what should or should not be described.
31


Reviews:
[{"review_id": "pfRkRT87RF", "reviewer": "Reviewer_GDVj", "summary": "The paper proposes a framework (ImageSentinel) to protect image datasets from unauthorised use in RAIG systems. It inserts sentinel images containing random character sequences as triggers for detecting unauthorised dataset use. The authors demonstrate their framework by showing reliable detection of dataset misuse across multiple RAIG systems in a black-box setting.", "questions": "* Have you tested whether an adversary with access to the protected dataset could filter or detect the sentinel images before integrating them into a RAIG system?\n* How would your approach perform if the RAIG system filtered or sanitized the random character prompts before retrieval (e.g., by stripping non-semantic input)? How much would the detection rate drop when such preprocessing is applied to the prompts?\n* Can you clarify how many queries and how much compute/time is required for reliable detection? What are the limits of detection with fewer queries?", "limitations": "As described above", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper addresses an important and timely challenge, namely the unathorised use of image (datasets), specifically with RAIG systems. Here, existing watermarking approaches fall short, making this an important contribution to the topic of RAIG image retrieval and copyright in AI. I appreciate that the authors show how their approach can work without disrupting overall RAIG for general usage. The experiments and evaluation appear quite comprehensive, they test across multiple RAIG systems and relevant ablations.\n\nThere are a limitations with regards to the real-world testing of the models, namely evaluations are performed on synthetic datasets, the threat from proprietary or fine-tuned RAIG systems in the wild is not assessed. In a similar vein, but rather as a note than a criticism of the method overall, the approach here relies on black-box querying, which requires repeated queries to the suspected RAIG system, which may not be feasible in a practical setting. \n\nFurther, the authors could extend their evaluation of the methodology by providing adverserial analysis, specifically w.r.t. the detectability of sentinel images; malicious users could potentially filter or recognise sentinel patterns.", "was_revised": true, "final_justification": "As stated in the discussion, I will keep my score as is.", "rebuttal": "We sincerely thank Reviewer GDVj for the thorough review and constructive suggestions.\n\n>###  **[W1] The evaluations are only performed on synthetic datasets; there is no assessment of threats from proprietary or fine-tuned RAIG systems in real-world scenarios.**\n\n**For datasets:**\n\nOur evaluation covers both synthetic and real-world datasets:\n\n1. Our main experiments used the LLaVA Visual Instruct Dataset [1], which contains real-world images from diverse sources and scenarios, including natural scenes, everyday objects, people, and more.\n2. We also evaluated ImageSentinel on Product-10K [2] in the supplementary materials, a real-world commercial dataset with 30,000 product images including both in-shop photos and customer-generated images.\n\nWe will move more Product-10K experiments to the main paper to strengthen our evaluation by including more real-world e-commerce applications.\n\n[1] Liu, Haotian, et al. \"Visual instruction tuning.\" NeurIPS 2023.\n\n[2] Bai, Yalong, et al. \"Products-10k: A large-scale product recognition dataset.\" arXiv preprint 2020.\n\n**For RAIG systems:**\n\n1. We evaluated ImageSentinel on both proprietary and open-source RAIG systems: Proprietary system (GPT-4o) and Open-source systems (SDXL and OmniGen).\n\n2. Following the common practice in retrieval-augmented generation research [3, 4] and widely-used frameworks like LangChain [5], we use pre-trained generators without fine-tuning. As RAIG is an emerging field, fine-tuned RAIG systems are not yet widely available in the community. We will extend our evaluation to fine-tuned variants when they become available.\n\n[3] Shalev-Arkushin, Rotem, et al. \"ImageRAG: Retrieval-Augmented Image Generation with Semantic Guidance.\" arXiv preprint 2025.\n\n[4] Ru, Dongyu, et al. \"RagChecker: A fine-grained framework for diagnosing retrieval-augmented generation.\" NeurIPS 2024.\n\n[5] Mavroudis, Vasilios. \"LangChain v0.3.\" preprint 2024.\n\n\n>### **[W2] The approach relies on black-box querying, requiring repeated queries to the suspected RAIG system, which may not be feasible in practical settings.**\n\nWe appreciate this practical concern. We discuss the feasibility of our black-box query approach as follows:\n\n**1. Minimal Query Requirements**\n\nAs shown in Table 3, S3 and S4 in the main paper and supplementary materials, our approach achieves efficient detection with very few queries. For example, only 5 queries are needed to reach 99.6% TPR at 1% FPR on Products-10k (total 30,000 reference images).\n\n**2. Established Practicality**\n\nQuery-based approaches have been widely adopted in retrieval-augmented systems for backdoor attack detection [6], membership inference [7], and dataset protection [8]. Such querying overhead occurs only during investigation of suspected misuse, not as a continuous requirement.\n\n[6] Chen, Zhaorun, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. \"AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases.\" NeurIPS 2024.\n\n[7] Liu, Mingrui, Sixiao Zhang, and Cheng Long. \"Mask-based membership inference attacks for retrieval-augmented generation.\" WWW 2025.\n\n[8] Liu, Yepeng, Xuandong Zhao, Dawn Song, and Yuheng Bu. \"Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs.\" ICML 2025 Workshop R2-FM. \n\n>### **[W3, Q1] The authors should further extend their evaluation with adversarial analysis, specifically regarding the detectability of sentinel images; malicious users could potentially filter or recognize sentinel patterns.**\n\nWe appreciate the reviewer's suggestion about adversarial analysis. We have conducted experiments to evaluate potential attacks that attempt to detect and filter sentinel images.\n\n**1. Attack Implementation**\n\nWe implemented two attack strategies using OCR-based random character detection:\n- Filtering: removing detected sentinel images\n- Masking: obscuring detected text regions\n\n**2. OCR Detection Performance**\n\nWe use three widely-used OCR systems (EasyOCR, Tesseract OCR, and PaddleOCR) on two datasets, and apply the PyEnchant library [1] to determine whether the detected text contains random character sequences. The results are shown below.\n\n|OCR System|Dataset|Sentinel Images Detection Rate|Clean Images Affected Rate|\n|------------|---------|----------------|----------------------|\n|EasyOCR|Product-10K|92.7%|41.7%|\n|EasyOCR|LLaVA Visual Instruct|90.2%|35.3%|\n|Tesseract OCR|Product-10K|63.6%|39.7%|\n|Tesseract OCR|LLaVA Visual Instruct|61.5%|23.7%|\n|PaddleOCR|Product-10K|98.3%|51.1%|\n|PaddleOCR|LLaVA Visual Instruct|97.5%|38.0%|\n\nWhile OCR-based attacks are possible to filter most sentinel images, the **high false positive rates** on clean images indicate that such attacks would significantly impair RAIG's normal functionality. This is particularly problematic for datasets like Product-10K, where a substantial portion of legitimate images containing product names, URLs, or watermarks would be incorrectly filtered out.\n\n[1] Ryan, K. PyEnchant: a spellchecking library for Python.\n\n**3. Impact Analysis**\n\nDetection performance under EasyOCR-based attacks (LLaVA Visual Instruct dataset):\n\n|Queries|Attack Method|AUC|TPR@1%FPR|TPR@10%FPR|\n|---------|--------------|------|------------|------------|\n|5|No Attack   |0.99|0.98      |0.99      |\n||Filtering   |0.63|0.16      |0.34      |\n||Masking     |0.62|0.15      |0.34      |\n|10|No Attack   |1.0 |1.0       |1.0       |\n||Filtering   |0.71|0.19      |0.40      |\n|      |Masking     |0.69|0.20      |0.42      |\n|50     |No Attack   |1.0 |1.0       |1.0       |\n|       |Filtering   |0.90|0.67      |0.80      |\n|       |Masking     |0.91|0.65      |0.82      |\n|100    |No Attack   |1.0 |1.0       |1.0       |\n|       |Filtering   |0.97|0.93      |0.96      |\n|       |Masking     |0.98|0.94      |0.96      |\n\nGeneration quality impact (300 samples):\n\n|Attack Method|CLIP|SigLIP|DINO|\n|--------------|-------|----------|--------|\n|No Attack   |0.772|0.743   |0.605 |\n|Filtering   |0.531|0.527   |0.503 |\n|Masking     |0.559|0.546   |0.511 |\n\n- These results demonstrate that while attacks can initially degrade detection effectiveness, this can be mitigated by increasing query numbers. For example, under filtering attacks, the AUC drops to 0.63 with 5 queries, but recovers to 0.97 with 100 queries.\n- Such attacks significantly compromise RAIG's generation quality, making them impractical for real-world deployment. Under attacks, the CLIP scores drop from 0.772 to around 0.531 (Filtering attack) and 0.559 (Masking attack), while SigLIP scores decrease from 0.743 to around 0.53 (Filtering attack) and 0.55 (Masking attack), representing a degradation in generation quality.\n\nWe will discuss these adversarial analyses in the limitations section, along with detailed experimental results. Future work could explore more sophisticated attack strategies and corresponding defense mechanisms to further validate the robustness of our protection framework.\n\n>### **[Q2] How would your approach perform if the RAIG system filtered or sanitized the random character prompts before retrieval (e.g., by stripping non-semantic input)? How much would the detection rate drop when such preprocessing is applied to the prompts?**\n\nWe have conducted additional experiments to evaluate our approach's robustness against prompt preprocessing:\n\n**1. Prompt Stripping Attack**\n\nWe implemented a **prompt stripping** attack that removes random character sequences in the queries.\n\nOur experiments on the Product-10K dataset (5 queries) show:\n\n|Attack Method|AUC|TPR@1%FPR|TPR@10%FPR|\n|-|-|-|-|\n|No Attack|0.999|0.996|0.998|\n|Prompt Stripping|0.515|0.013|0.124|\n\nThe results show that prompt stripping can effectively degrade our detection performance.\n\n**2. Practical Challenges**\n\nWhile prompt stripping significantly impacts detection performance, implementing such filtering in RAIG systems faces serious practical challenges:\n- Many legitimate use cases require precise alphanumeric inputs (product codes, brand names).\n- Our analysis on Product-10K identified numerous legitimate brand names that would be incorrectly filtered, such as \"Royalstar\", \"YASEE\", \"IPASON\", \"LIEYANZHANDUI\", \"kocotree\", \"Aocilenda\", and \"NUTRICIA\".\n- Such filtering would significantly degrade RAIG system utility in e-commerce and technical applications.\n\nWe will include a detailed discussion of these limitations and potential mitigation strategies in our revised paper.\n\n>### **[Q3] Can you clarify how many queries and how much compute/time is required for reliable detection? What are the limits of detection with fewer queries?**\n\nWe thank the reviewer for this important question about the practical aspects of our detection method.\n\n**1. Query Requirements for Reliable Detection**\n\nAs shown in Table 3, S3, and S4, ImageSentinel achieves reliable detection with very few queries. We list some results in Table S4 as:\n\n|Query Number|AUC|TPR@1%FPR|TPR@10%FPR|\n|--------------|-----|------------|-------------|\n|1|0.870|0.704|0.754|\n|3|0.989|0.944|0.982|\n|5|0.999|0.996|0.998|\n\nTo address the limits with fewer queries, we specifically tested the performance with just one query, which still achieves a reasonable AUC of 0.870 and TPR of 0.754 at 10% FPR, though with reduced reliability compared to using more queries.\n\n**2. Computational Cost**\n\nThe main computational overhead comes from image generation time per query:\n\n|RAIG System|Per-query Time|\n|-------------|---------------|\n|SDXL|54.5s|\n|OmniGen|156.2s|\n|GPT-4o|57.7s|\n\nWith parallel processing (common in real-world deployments), total detection time approximates one query duration. Without parallelization, total time equals per-query time multiplied by query number.\n\nGiven that reliable detection requires only 3-10 queries and supports parallel processing, the computational cost remains practical for practical applications."}, {"review_id": "4g96omQnPA", "reviewer": "Reviewer_LHUd", "summary": "This paper proposes a new approach to detect the unauthorized use of private image datasets in RAIG systems. The method involves augmenting a private dataset with synthesized \"sentinel\" images that are visually consistent with the dataset but also contain a unique, embedded textual key. A dataset owner can then probe a suspect RAIG system by providing these keys as prompts. High visual similarity between the generated output and the original sentinel image is used as evidence of dataset misuse. The work demonstrates high detection rates across several modern RAIG systems with minimal impact on authorized generation quality.", "questions": "- Can you discuss the robustness of ImageSentinel to an adaptive adversary who actively filters the dataset for suspicious images (e.g., via OCR)?\n- Can you propose an experiment to decisively prove that the high similarity scores you observe are a result of retrieval and not just the direct, non-retrieval-based rendering of the key by the generator?\n- How do you propose this method remains reliable against adversaries using models where this capability is weak or absent?", "limitations": "The authors include a \"Limitations\" section, which is appreciated. However, it omits several critical limitations raised in the weaknesses section, such as the vulnerability to adaptive adversaries.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- This paper studies a pressing problem when a retrieval-augmented pipeline is commonly adopted by GenAI.\n- The experimental setup and ablation studies are comprehensive, and the writing is clear.\n\n### Weaknesses\n\n- The evaluation lacks a defense against an adaptive adversary, a standard requirement for security claims. The entire defense hinges on the stealthiness of sentinels, yet it assumes a passive adversary. A motivated attacker aware of this strategy could easily employ countermeasures, such as using OCR to filter out any images containing conspicuous, random text strings before ingestion. The necessity of evaluating against such adaptive attacks is a well-established principle in the security community (e.g., [Tramèr et al., 2020], [Athalye et al., 2018]). Without this, the claimed robustness is unsubstantiated.\n\n- Some baselines are not competitive, comprising known-to-fail \"strawmen\" while ignoring the state-of-the-art. The paper compares its method against traditional image watermarking (HiDDeN, FIN). However, if I understand it correctly, recent work [37] has already underscored that such watermarks are not robust to the complex transformations of modern generative models. These baseline watermark methods resulting in near-random performance (AUC $\\approx$ 0.5) are therefore expected and do not constitute a meaningful benchmark. In my opinion, the paper avoids comparison with more relevant and challenging baselines like the membership attacks for RAG that the paper cites in its own related work (e.g., [25]).\n\n- The evaluation of the detection mechanism is a bit ambiguous. The core claim is that the key triggers the retrieval of the sentinel. However, the evaluation fails to disentangle this from a more trivial explanation: that the RAIG's powerful generator (e.g., GPT-4o) simply renders the key from the prompt, leading to an image that is naturally similar to the sentinel. The \"triggering rate\" in Table 2, which is key to this claim, is not clearly defined for a black-box setting, and this ambiguity undermines the interpretation of the core results.\n\n- My final primary concern is about the assumption of the proposed defense. Based on my understanding, the defense only works if the adversary's generator is powerful enough to faithfully render the key; if a weaker generator is used, the detection mechanism will likely fail.\n\n### References\n- [Tramèr et al., 2020] Florian Tramèr et al. On Adaptive Attacks to Adversarial Example Defenses. Proc. of NeurIPS, 2020.\n- [Athalye et al., 2018] Anish Athalye et al. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. Proc. of ICML, 2018.", "was_revised": true, "final_justification": "After productive discussions with the authors, all my concerns have been addressed. The new evaluation results provide more insights into the pros and cons of the proposed protection framework. Therefore, I choose to raise my score.", "rebuttal": "We thank Reviewer LHUd for the rigorous analysis and valuable feedback on security concerns.\n\n>### **[W1, Q1] Defense against adaptive adversary: No evaluation against attackers who could use OCR to filter out images with random text strings.**\n\nWe have conducted experiments to evaluate potential attacks that attempt to detect and filter sentinel images.\n\n**1. Attack Implementation**\n\nWe implemented two attack strategies using OCR-based random character detection:\n- Filtering: removing detected sentinel images\n- Masking: obscuring detected text regions\n\n**2. OCR Detection Performance**\n\nWe use three widely-used OCR systems (EasyOCR, Tesseract OCR, and PaddleOCR) on two datasets, and apply the PyEnchant library [1] to determine whether the detected text contains random character sequences. The results are shown below.\n\n|OCR System|Dataset|Sentinel Images Detection Rate|Clean Images Filtered Rate|\n|-|-|-|-|\n|EasyOCR|Product-10K|92.7%|41.7%|\n|EasyOCR|LLaVA dataset|90.2%|35.3%|\n|Tesseract|Product-10K|63.6%|39.7%|\n|Tesseract|LLaVA dataset|61.5%|23.7%|\n|PaddleOCR|Product-10K|98.3%|51.1%|\n|PaddleOCR|LLaVA dataset|97.5%|38.0%|\n\nWhile OCR-based attacks are possible to filter most sentinel images, the **high false positive rates** on clean images indicate that such attacks would significantly impair RAIG's normal functionality.\n\n[1] Ryan, K. PyEnchant: a spellchecking library for Python.\n\n**3. Impact Analysis**\n\nDetection performance under EasyOCR-based attacks (LLaVA Visual Instruct dataset):\n\n|Queries|Attack Method|AUC|TPR@1%FPR|TPR@10%FPR|\n|-|-|-|-|-|\n|5|No Attack|0.99|0.98|0.99|\n||Filtering|0.63|0.16|0.34|\n||Masking|0.62|0.15|0.34|\n|10|No Attack|1.0|1.0|1.0|\n||Filtering|0.71|0.19|0.40|\n||Masking|0.69|0.20|0.42|\n|50|No Attack|1.0|1.0|1.0|\n||Filtering|0.90|0.67|0.80|\n||Masking|0.91|0.65|0.82|\n|100|No Attack|1.0|1.0|1.0|\n||Filtering|0.97|0.93|0.96|\n||Masking|0.98|0.94|0.96|\n\nGeneration quality impact (300 samples):\n\n|Attack Method|CLIP|SigLIP|DINO|\n|-|-|-|-|\n|No Attack|0.772|0.743|0.605|\n|Filtering|0.531|0.527|0.503|\n|Masking|0.559|0.546|0.511|\n\nThese results demonstrate that while attacks can initially degrade detection effectiveness, this can be mitigated by increasing query numbers. Such attacks also compromise RAIG's generation quality, making them impractical for real-world deployment.\n\nWe will discuss these potential adversaries in the limitations section, along with experimental results. Future work could explore more attack and defense strategies to further validate the robustness of our method.\n\n>### **[W2] Some baselines are not competitive. Comparison with more relevant and challenging baselines like membership attacks for RAG cited in related work.**\n\nThank you for raising this important point about baselines.\n\n**1. Baseline Selection**\n\nOur work addresses dataset protection for Retrieval-Augmented Image Generation (RAIG) systems, where protection methods are still largely unexplored. Therefore, we explored **adapting** text RAG protection method (Ward [1]) to RAIG by creating baselines Ward-HiDDeN and Ward-FIN in our paper. We elaborate below on why other existing methods cannot be directly adapted to RAIG.\n\n**2. Challenges in Adapting RAG Methods to RAIG**\n\nWe investigated the feasibility of converting existing RAG-related methods:\n- MIA for RAG [2] ([25] in main paper): This method relies on text-based LLMs to answer \"Yes\" or \"No\" for membership inference. However, in RAIG, the image generator replaces the text-based LLM and can only generate images rather than provide binary answers.\n\n- Ward [1] and [3]: These methods watermarks texts in the reference dataset and relies on the persistence of text watermarks in generated outputs for detection. Since our reference data consists of images, we adapted the text watermarking approach to image watermarking, creating baselines Ward-HiDDeN and Ward-FIN.\n\n- S^2MIA [4] and MBA [5]: These methods rely on using masked target documents as input and observing accuracy of RAG-generated content for the masked portions to determine if the target document exists in the reference dataset. When adapting to RAIG, we would need to input masked target images. However, this is infeasible since RAIG accepts text rather than image inputs.\n\n- Phantom [6]: This method depends on white-box attacks, which conflicts with our threat model's black-box assumptions.\n\n- TrojanRAG [7]: This method requires attackers to have the ability to train the RAG system. However, since we directly use existing black-box models, this approach cannot be effectively adapted.\n\n[1] Jovanović, N. et al. \"Ward: Provable rag dataset inference via llm watermarks.\" ICLR 2025.\n\n[2] Anderson, M. et al. \"Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation.\" ICISSP 2025.\n\n[3] Liu, Y. et al. \"Dataset protection via watermarked canaries in retrieval-augmented llms.\" arXiv 2025.\n\n[4] Li, Y. et al. \"Generating is believing: Membership inference attacks against retrieval-augmented generation.\" ICASSP 2025.\n\n[5] Liu, M. et al. \"Mask-based membership inference attacks for retrieval-augmented generation.\" WWW 2025.\n\n[6] Chaudhari, H. et al. \"Phantom: General trigger attacks on retrieval augmented language generation.\" arXiv 2024.\n\n[7] Cheng, P. et al. \"TrojanRAG: Retrieval-augmented generation can be backdoor driver in large language models.\" arXiv 2024.\n\n**3. Additional Baseline**\n\nTo provide a more comprehensive evaluation, we implemented a semantic-based baseline that uses semantic descriptions as retrieval keys instead of character sequences. This baseline represents a more intuitive approach that leverages semantic descriptions to detect unauthorized dataset usage. Specifically, we:\n\n1) Extract semantic descriptions from images.\n2) Use these descriptions as retrieval keys to query the RAIG system.\n3) Compare generated images with original images to detect unauthorized use.\n\nAs shown in Table 2 of main paper, most semantic descriptions (more than 60%) as retrieval keys bypass retrieval as their generators create satisfactory images directly, preventing detection of target images in the reference dataset.\n\nWe further evaluated this baseline on the LLaVA Visual Instruct Pretrain Dataset:\n\n|RAIG|Query|ImageSentinel|||Semantic-based|||\n|-|-|-|-|-|-|-|-|\n|||AUC|T@1%F|T@10%F|AUC|T@1%F|T@10%F|\n|SDXL|3|0.974|0.934|0.958|0.623|0.200|0.356|\n||10|1.000|1.000|1.000|0.785|0.340|0.563|\n||20|1.000|1.000|1.000|0.876|0.540|0.717|\n|OmniGen|3|0.873|0.584|0.744|0.634|0.209|0.327|\n||10|0.989|0.922|0.974|0.786|0.373|0.567|\n||20|1.000|0.996|1.000|0.896|0.572|0.763|\n\nThe results demonstrate that our approach outperforms the semantic-based baseline. We will include these results in the main paper for a more comprehensive comparison.\n\n\n>### **[W3, Q2] (W3) Detection mechanism validation: Need to verify whether high similarity comes from retrieval or direct key rendering. (Q2) Validating Retrieval vs Direct Rendering.**\n\nWe appreciate these important questions about our evaluation methodology.\n\n**1. Analysis of Detection Mechanism**\n\nDirect key rendering would be insufficient for detection.\n\nOur detection mechanism does not rely on rendering keys from prompts. Instead, we detect unauthorized use through measuring **visual similarity** between RAIG-generated images and our sentinel images. The design the key as random characters (e.g., \"VasWiW\") without any information about corresponding sentinel image's semantical attributes.\n\nTherefore, it is theoretically impossible for generative models to recreate the specific visual characteristics of sentinel images solely from these random characters. The high similarity scores can only be achieved through successful retrieval of the sentinel images.\n\n\n**2. **[Q2]** Experiments for Direct Key Rendering**\n\nWe conducted a controlled experiment comparing generation with retrieval enabled versus disabled (direct generation) using the same keys. We use 10 queries for LLaVA Visual Instruct Pretrain Dataset:\n\n|Setting|AUC|T@1%F|T@10%F|\n|-|-|-|-|\n|Retrieval enabled|1.000|1.000|1.000|\n|Retrieval disabled|0.512|0.008|0.102|\n\nThe results are expected since our keys are random character sequences with no semantic correlation to sentinel images' visual content, making it impossible for direct generation to reproduce specific visual elements without retrieval.\n\n**3. Definition of Triggering Rate**\n\nThe 'triggering rate' is defined as the proportion of queries where a RAIG system initiates the retrieval process when given a key. This metric is important since some RAIG systems [1] may bypass retrieval if their generators believe they can directly generate satisfactory images. Our method ensures a high triggering rate by carefully designing the key.\n\nTriggering rate is an intermediate metric used during our experimental analysis. In actual deployment (black-box setting), we only need to measure the semantic similarity between generated images and our sentinel images to detect unauthorized use. As shown in Table 3, this detection achieves high performance with black-box setting.\n\n[1] Shalev-Arkushin, R. et al. \"ImageRAG: Retrieval-Augmented Image Generation with Semantic Guidance.\" arXiv 2025.\n\n>### **[W4, Q3] Dependency of Defense Mechanism on Generator Capability.**\n\nWe demonstrated that high similarity scores cannot be achieved through direct key rendering in response to **[W3, Q2]**.\n\nOur keys are random character sequences with no semantic correlation to visual content of sentinel images. Therefore, regardless of the generator's capability, it cannot directly render specific visual elements of sentinel images solely from these keys. The high similarity scores can only be achieved through the successful retrieval of sentinel images.\n\nThe requirement for RAIG systems is that the generator can incorporate semantic features from retrieved images into its output, which is a fundamental capability that functional RAIG system possesses."}, {"review_id": "cvQDcNnA1H", "reviewer": "Reviewer_qQTS", "summary": "The paper introduces a method to protect image datasets in the context of retrieval augmented generation. It works by generating “sentinel images” and including them in the protected dataset. They are images that incorporate a random-character key in the image, associated with a particular image semantics. To detect the use of a protected dataset, one has to prompt the model with the specific key and analyze the semantic similarity between the sentinel and the generated image (done with DINO). The paper validates the approach on SOTA generative models (GPT-4o, SD-XL, OmniGen) and thorough experiments.", "questions": "(1) The paper mentions a total of 10,000 images, with 2,800 images designated as the private dataset. Could the authors clarify the precise number or proportion of these 2,800 private images that are converted into or replaced by sentinel images? If a significant portion, or indeed all, of the 2,800 images serve as sentinels, what are the implications for the probability of inadvertently observing a sentinel image when inspecting the dataset, and what is the potential impact on detection efficiency or overhead?\n\n(2) When the generative model operates on a protected dataset but without the specific retrieval key included in the prompt, has it been observed that the model inadvertently generates the embedded random character sequences within the output images?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n(1) The paper addresses an important and currently underexplored problem: protecting visual datasets from unauthorized use in Retrieval-Augmented Image Generation (RAIG).\n\n(2) The paper is well-written and easy to understand. It uses good illustrations, like prompts in boxes, which helps in the explanation.\n\n(3) The main idea of using sentinel images to get a clear detection signal is simple and effective. This makes it easy to tell when protected data has been used by a RAIG model.\n\n(4) The experiments are done carefully and thoroughly:  \n    (a) The code is clean, and seems easy to check and run (although I did not).  \n    (b) The authors used adapted WARD as a baseline, which, even if different from their method, provides a good comparison point and shows effort to be rigorous.  \n    (c) The method is tested with several up-to-date generative models (like GPT-4o, SD-XL, and Omnigen), showing its wide applicability.  \n    (d) Extra results are included in the supplementary materials, adding more proof for their findings.  \n\n## Weaknesses\n\n(1) The use of DINO for semantic similarity in detection limits how strong the statistical proof can be. For example, evaluating only TPR at FPR=0.01 might not be enough for legal situations, unlike methods such as the adapted WARD-Hidden, which can offer more statistical tests based on the matching its between binary messages.\n\n(2) Some parts of the experimental setup are not fully clear in the paper:  \n    (a) The exact number of sentinel images used in the experiments is not explicitly stated, which makes it hard to understand the scale of the protection and how stealthy it is.  \n    (b) How TPR at 1% FPR is calculated is not completely clear. The paper mentions \"100 trials with 5 queries\" but doesn't fully explain what a \"trial\" is or how the \"negatives\" (without the protected dataset) are created for evaluation.  \n\n(3) The paper does not discuss how people might try to get around the protection.  \n    (a) For example, if a generative model provider sees unusual images with random characters in their database, they might simply remove them.  \n    (b) They could also prevent users from using prompts with random characters. The paper would benefit from discussing these potential challenges and how the method might handle them.\n\n---\nNote for the ratings:\n- Quality: the experiments are done with care, but the idea of sentinel images in itself leads to strong limitations (no theoretical guarantees on the detection, ease of protection removal", "was_revised": true, "final_justification": "The rebuttal has addressed most of my interrogations. It particularly introduced new attacks and additional details. I strongly believe that this paper has its place at the conference and in the field. \nAlthough, the idea of sentinel images in itself leads to strong limitations (no theoretical guarantees on the detection, ease of protection removal), it could serve as a first baseline for protection of image datasets in the context of RAIG and opens up interesting problems to solve. This is why I recommend acceptance.", "rebuttal": "We thank Reviewer qQTS for the thorough review and constructive suggestions.\n\n>### **[W1] Statistical Strength of Detection Evidence**\n\n We appreciate the suggestion to strengthen our statistical evidence and have conducted additional analyses.\n\n**1. Extended Evaluation Metrics**\n\nWe follow practices in detection evaluation [1-4] to use TPR@1%FPR in our experiments. We acknowledge the need for more stringent statistical guarantees for legal applications. We extended our evaluation to include TPR@0.1%FPR, providing stronger statistical confidence aligned with recent works [5,6].\n\n**2. Results**\n\nWe keep the same experimental setting as Table 3 in our paper.\n\n| RAIG | Query | AUC | TPR@0.1%FPR | TPR@1%FPR |\n|------|-------|-----|--------|-------|\n| SDXL | 3 | 0.974|0.802|0.934|\n|      | 5 | 0.997|0.952|0.980|\n|      | 10 |1.000|0.998|1.000|\n|      | 20 |1.000|1.000|1.000|\n| OmniGen | 3 | 0.974|0.520|0.584|\n|      | 5 | 0.997|0.632|0.736|\n|      | 10 |1.000|0.880|0.922|\n|      | 20 |1.000|0.992|0.996|\n| GPT-4o | 3 | 0.983|0.863|0.954|\n|      | 5 | 0.997|0.973|0.980|\n|      | 10 |1.000|0.999|1.000|\n|      | 20 |1.000|1.000|1.000|\n\nOur method maintains high detection rates even at FPR=0.001.\n\nWe will expand Table 3 in the main paper to include these more stringent metrics.\n\n\n[1] Wen, Y. et al. \"Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust.\" NeurIPS 2023.\n\n[2] Dathathri, S. et al. \"Scalable watermarking for identifying large language model outputs.\" Nature 2024.\n\n[3] Pan, L. et al. \"MarkLLM: An Open-Source Toolkit for LLM Watermarking.\" EMNLP 2024.\n\n[4] Liu, Y. et al. \"Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs.\" ICML 2025 Workshop R2-FM.\n\n[5] Pang, Q. et al. \"No free lunch in LLM watermarking: Trade-offs in watermarking design choices.\" NeurIPS 2024.\n\n[6] Kirchenbauer, J. et al. \"On the Reliability of Watermarks for Large Language Models.\" ICLR 2024.\n\n\n>### **[W2.(a), Q1] Exact number of sentinel images**\n\nIn our experiments, as shown in Table 3, S3, and S4, the number of queries is equal to the number of sentinel images used, as each query is designed to trigger a specific sentinel image.\n\nSpecifically, our experiments used:\n* Table 3: 3-20 sentinel images in 10,000 references (0.03%-0.2%)\n* Table S3: 3-30 sentinel images in 10,000-100,000 references (0.003%-0.3%)\n* Table S4: 1-20 sentinel images in 30,000 references (0.003%-0.07%)\n\nThese sentinel images were randomly selected from a pool of 2,800 private images. The small ratios help ensure the stealthiness of our protection mechanism. We will explicitly clarify these numbers in the revised paper.\n\n>### **[W2.(b)]** Clarity of Performance Metrics Calculation\n\nWe explain the process of our trials and TPR/FPR calculation as follows.\n\n**Trial Process**\n\nFor each trial with N queries:\n\n1. Positive Sample Generation:\n- Randomly select N images from our private dataset.\n- Convert them into sentinel images.\n- Embed them into the reference database.\n- Generate N images using their corresponding N key queries.\n- Calculate the mean DINO similarity between generated images and their corresponding sentinel images.\n\n1. Negative Sample Generation:\n- Use the same N queries but with the original reference database (without sentinel images).\n- Generate N images and calculate mean DINO similarity similarly.\n\n**Evaluation Details**\n- We conduct 100 trials for positive samples.\n- For robust threshold estimation, we conduct 2000 trials for negative samples.\n- TPR@1%FPR is calculated by:\n  1. Setting detection threshold at 1% percentile of negative samples' similarities.\n  2. Computing the fraction of positive samples exceeding this threshold.\n\nWe will add these detailed evaluation settings to our revised paper.\n\n>### **[W3.(a)] Vulnerability to Database Filtering**\n\nWe have conducted experiments to evaluate potential attacks that attempt to detect and filter sentinel images.\n\n**1. Attack Implementation**\n\nWe implemented two attack strategies using OCR-based random character detection:\n- Filtering: removing detected sentinel images.\n- Masking: obscuring detected text regions.\n\n**2. OCR Detection Performance**\n\nWe use three widely-used OCR (EasyOCR, Tesseract OCR, and PaddleOCR) on two datasets, and apply the PyEnchant library [1] to determine whether the detected text contains random character sequences. The results are shown below.\n\n|OCR System|Dataset|SentinelImages Detection Rate|Clean Images Filtered Rate|\n|-|-|-|-|\n|EasyOCR|Product-10K|92.7%|41.7%|\n|EasyOCR|LLaVA Visual Instruct|90.2%|35.3%|\n|Tesseract|Product-10K|63.6%|39.7%|\n|Tesseract|LLaVA Visual Instruct|61.5%|23.7%|\n|PaddleOCR|Product-10K|98.3%|51.1%|\n|PaddleOCR|LLaVA Visual Instruct|97.5%|38.0%|\n\nWhile OCR-based attacks are possible to filter most sentinel images, the **high false positive rates** on clean images indicate that such attacks would significantly impair RAIG's normal functionality. This is particularly problematic for datasets like Product-10K, where a substantial portion of legitimate images containing product names, URLs, or watermarks would be incorrectly filtered out.\n\n[1] Ryan, K. PyEnchant: a spellchecking library for Python.\n\n**3. Impact Analysis**\n\nDetection performance under EasyOCR-based attacks (LLaVA dataset):\n\n|Queries|Attack Method|AUC|TPR@1%FPR|TPR@10%FPR|\n|-|-|-|-|-|\n|5|No Attack|0.99|0.98|0.99|\n||Filtering|0.63|0.16|0.34|\n||Masking|0.62|0.15|0.34|\n|10|No Attack|1.0|1.0|1.0|\n||Filtering|0.71| 0.19|0.40|\n||Masking|0.69|0.20|0.42|\n|50|No Attack|1.0|1.0|1.0|\n||Filtering|0.90|0.67|0.80|\n||Masking|0.91|0.65|0.82|\n|100|No Attack|1.0|1.0|1.0|\n||Filtering|0.97|0.93|0.96|\n||Masking|0.98|0.94|0.96|\n\nGeneration quality impact (300 samples):\n\n|Attack Method|CLIP|SigLIP|DINO|\n|-|-|-|-|\n|No Attack|0.772|0.743|0.605|\n|Filtering|0.531|0.527|0.503|\n|Masking|0.559|0.546|0.511|\n\nThese results demonstrate that while attacks can initially degrade detection effectiveness, this can be mitigated by increasing query numbers. However, such attacks significantly compromise RAIG's generation quality, making them impractical for real-world deployment.\n\nWe will discuss these potential adaptive adversaries in an expanded limitations section, along with detailed experimental results. Future work could explore more sophisticated attack strategies and corresponding defense mechanisms to further validate the robustness of our protection framework.\n\n>### **[W3.(b)] Vulnerability to Prompt Filtering**\n\nWe have conducted additional experiments to evaluate our approach's robustness against prompt preprocessing:\n\n**1. Prompt Stripping Attack**\n\nWe implemented a **prompt stripping** attack that removes random character sequences in the queries.\n\nOur experiments on the Product-10K dataset (5 queries) show:\n\n|Attack Method|AUC|TPR@1%FPR|TPR@10%FPR|\n|-|-|-|-|\n|No Attack|0.999|0.996|0.998|\n|Prompt Stripping|0.515|0.013|0.124|\n\nThe results show that prompt stripping can effectively degrade our detection performance.\n\n**2. Practical Challenges**\n\nWhile prompt stripping significantly impacts detection performance, implementing such filtering in RAIG systems faces serious practical challenges:\n- Many legitimate use cases require precise alphanumeric inputs (product codes, brand names).\n- Our analysis on Product-10K identified numerous legitimate brand names that would be incorrectly filtered, such as \"Royalstar\", \"YASEE\", \"IPASON\", \"LIEYANZHANDUI\", \"kocotree\", \"Aocilenda\", and \"NUTRICIA\".\n- Such filtering would significantly degrade RAIG system utility in e-commerce and technical applications.\n\nWe will include a detailed discussion of these limitations in our revised paper.\n\n>### **[How the method might handle W3.(a) and W3.(b)]**\n\nWe propose two potential strategies to mitigate the vulnerability to OCR-based filtering and prompt stripping:\n\n**1. Using Meaningful Characters as Keys.**\nInstead of random character sequences, we could use meaningful words or phrases as sentinel keys. However, this approach faces two challenges:\n- Finding the right balance between protection effectiveness and normal user experience, as legitimate users might naturally use these meaningful keys in their queries\n- The meaningful keys might retrieve images related to their semantic meaning rather than images containing the actual text, potentially reducing detection accuracy\n\n**2. Embedding Keys in Natural Contexts.**\nWe could embed random character keys in natural-looking contexts, such as \"VasWiW brand cup\" or \"VasWiW restaurant\". This approach could help bypass prompt filtering as these queries appear more legitimate.\n\nWe plan to explore these mitigation strategies in our future work.\n\n>### **[Q2] Unintended Character Generation in Normal Usage**\n\nIn Section 5.2 \"Generation quality preservation\", we investigated whether using protected images would affect normal generation capability. Among all queries, 8.3% retrieved sentinel images. We analyzed all generated images from these queries and found:\n\n1. 36% of these images generated by SDXL-based RAIG after retrieving sentinel images did contain text characters.\n\n2. These characters were not direct copies of the retrieval keys embedded in the retrieved sentinel images. Instead, they appeared with significant modifications and transformations, making the original characters unrecognizable.\n\n3. The characters were naturally integrated into the generated scenes, appearing as store signs, ship decorations, or wall graffiti. This integration did not noticeably impact the visual quality or coherence of the generated images.\n\nIn this test, we intentionally increased the number of sentinel images to 300 to evaluate the generation capability. In practice, only a small number of sentinel images (1-20 images in a reference dataset of 10,000 images) are needed for protection, which would reduce the likelihood of retrieving sentinel images during normal generation.\n\nWe will include these visual examples and a discussion in our revised paper."}, {"review_id": "a3bJ8USfEZ", "reviewer": "Reviewer_yaqi", "summary": "The paper aims to detect unauthorized usage of private image datasets in a Retrieval-Augmented Image Generation (RAIG) system. Authors propose to include some digitally crafted images referred as “image sentinels” into the dataset to enable the detection of unauthorized usage. Authors synthesize these sentinels using reference images from the dataset and explicitly condition the image synthesizer to embed a random character sequence (key) into the generated images. These generated images are added to the private dataset as image sentinels. When queried the black-box RAIG system with the keys i.e., specific random character sequences, the detection system compares the RAIG generated images with the image sentinels to find the similarity and raise the detection flag if similarity score exceeds a threshold. Experiments show that the proposed approach reaches 100% detection rate with 10-20 queries to the RAIG system and outperforms watermarking methods Ward-HiDDeN and Ward-FIN.", "questions": "I suggest authors to consider the points mentioned in the weakness section to thoroughly evaluate the robustness of the proposed detection mechanism. Currently my rating stands at borderline reject. However if my concerns are adequately addressed during rebuttal, I am willing to reconsider my rating.", "limitations": "Authors have not explored possible detection evading mechanism to counter their proposed method.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n-\tA simple but shown to be an effective approach to detect unauthorized image of private image datasets.\n\n-\tNo modifications are made to the original images in the dataset, hence authorized usage still provides relevant results.\n\n-\tThese sentinels make the detection to be triggered on demand i.e., reveal only when explicitly prompted with the specific keys (random character sequence).\n\n-\tIt is a simple and straightforward pipeline to generate the image sentinels leveraging existing vision-language foundation models, and find image similarity using vision foundation models at the time of detection.\n\n-\tResults show that the detection rate can be reached to 100% with 10-20 queries, outperforming watermarking based methods.\n\n-\tPaper is clearly written, and it is easy to read and understand.\n\nWeakness:\nDespite the strengths mentioned above, authors have not explored any possible detection evading techniques. I find that is the major weakness of the paper. A RAIG provider could easily incorporate some simple techniques to remove the image sentinels from the dataset:\n\n-\tIdentifying duplicate images in the dataset, and finding representative images that are unique based on the type, description, characteristics or even with perceptual similarity using DINO and filtering the duplicates or over-represented content.\n\n-\tAnother approach could be leveraging OCR models to detect the text from images and filtering out images with random character sequences or masking such image regions.", "was_revised": true, "final_justification": "This paper addresses an important topic: detecting unauthorized usage of private image datasets in the GenAI era. The idea of enabling detection to be triggered on demand is particularly compelling. During the discussion, the authors provided a comprehensive analysis of critical adaptive attacks, which is appreciated and offers a thorough understanding of the proposed protection. After the rebuttal, my concerns regarding adaptive attacks were resolved, and I find the work convincing and valuable for the community to learn about.", "rebuttal": "We appreciate Reviewer yaqi's insightful feedback and specific suggestions.\n\n> ### **[W1] Identifying duplicate images in the dataset, and finding representative images that are unique based on the type, description, characteristics or even with perceptual similarity using DINO and filtering the duplicates or over-represented content.**\n\nThank you for raising this important concern about duplicate content filtering. We conducted additional experiments to evaluate this potential attack strategy.\n\n**1. Attack Implementation**\n\nWe implemented a duplicate filtering attack using DINO similarity scores. For each image pair, we compute their DINO similarity and remove images exceeding a threshold $\\delta$, prioritizing the removal of images containing random character-like patterns.\n\n\n**2. Performance**\n\nWe evaluated various filtering thresholds $\\delta$ (0.5-0.9) across two datasets with 10 queries for detection:\n\nLLaVA Visual Instruct Dataset:\n| $\\delta$ | AUC | TPR@1%FPR | TPR@10%FPR | Clean Images Filtered Rate |\n|---|-----|-----------|------------|------------|\n|0.5|0.88 |0.58      |0.77        |87.9%|\n|0.6|0.98 |0.92      |0.97        |62.8%|\n|0.7|1.00 |0.98      |1.00        |34.2%|\n|0.8|1.00 |1.00      |1.00        |13.1%|\n|0.9|1.00 |1.00      |1.00        |3.3%|\n\nProduct-10K Dataset:\n| $\\delta$ | AUC | TPR@1%FPR | TPR@10%FPR |Clean Images Filtered Rate|\n|---|-----|-----------|------------|------------|\n|0.5|0.90 |0.60      |0.78        |96.2%|\n|0.6|0.98 |0.87      |0.95        |83.1%|\n|0.7|1.00 |0.98      |0.99        |60.2%|\n|0.8|1.00 |1.00      |1.00        |37.3%|\n|0.9|1.00 |1.00      |1.00        |19.0%|\n\n**3. Analysis:**\n\n- Lower $\\delta$ thresholds (0.5-0.6) can filter out sentinel images and cause low detection accuracy.\n- However, they lower $\\delta$ thresholds also remove an unacceptable proportion of legitimate images (62.8-96.2%). This severely compromises the dataset's utility. Especially for Product-10K dataset, it suffers more severe filtering due to its inherent nature of containing visually similar product images.\n- Setting a high $\\delta$ (0.8-0.9) preserves more legitimate content but fails to effectively filter sentinel images, as evidenced by the high detection scores.\n- Our method's resilience against similarity-based filtering is largely attributed to our pipeline shown in Figure 3. Instead of directly feeding reference images into GPT-4o for sentinel image generation, we first perform attribute extraction followed by language-guided generation. Direct reference-guided generation would result in substantial content overlap, while our language-guided approach maintains stylistic consistency while avoiding identical content duplication. This design ensures that sentinel images share semantic and stylistic elements with the reference images while maintaining sufficient visual diversity to survive similarity-based filtering.\n\nWe will include these quantitative results and visual examples in our revised paper to better illustrate our findings.\n\n> ### **[W2] Another approach could be leveraging OCR models to detect the text from images and filtering out images with random character sequences or masking such image regions.**\n\nWe appreciate this insightful comment about detection evading techniques. We have conducted experiments to evaluate potential attacks that attempt to detect and neutralize sentinel images.\n\n**1. Attack Implementation**\n\nWe implemented two attack strategies using OCR-based random character detection:\n- Filtering: removing detected sentinel images\n- Masking: obscuring detected text regions\n\n**2. OCR Detection Performance**\n\nWe use three widely-used OCR systems (EasyOCR, Tesseract OCR, and PaddleOCR) on two datasets, and apply the PyEnchant library [1] to determine whether the detected text contains random character sequences. The results are shown below.\n\n| OCR System | Dataset | Sentinel Images Detection Rate | Clean Images Filtered Rate |\n|------------|---------|----------------|----------------------|\n| EasyOCR | Product-10K | 92.7% | 41.7% |\n| EasyOCR | LLaVA Visual Instruct | 90.2% | 35.3% |\n| Tesseract OCR | Product-10K | 63.6% | 39.7% |\n| Tesseract OCR | LLaVA Visual Instruct | 61.5% | 23.7% |\n| PaddleOCR | Product-10K | 98.3% | 51.1% |\n| PaddleOCR | LLaVA Visual Instruct | 97.5% | 38.0% |\n\nWhile OCR-based attacks are possible to filter most sentinel images, the **high false positive rates** on clean images indicate that such attacks would significantly impair RAIG's normal functionality. This is particularly problematic for datasets like Product-10K, where a substantial portion of legitimate images containing product names, URLs, or watermarks would be incorrectly filtered out.\n\n[1] Ryan, K. PyEnchant: a spellchecking library for Python.\n\n**3. Impact Analysis**\n\nDetection performance under EasyOCR-based attacks (LLaVA Visual Instruct dataset):\n\n| Queries | Attack Method | AUC | TPR@1%FPR | TPR@10%FPR |\n|---------|--------------|------|------------|------------|\n| 5       | No Attack    | 0.99 | 0.98       | 0.99       |\n|         | Filtering    | 0.63 | 0.16       | 0.34       |\n|         | Masking      | 0.62 | 0.15       | 0.34       |\n| 10      | No Attack    | 1.0  | 1.0        | 1.0        |\n|         | Filtering    | 0.71 | 0.19       | 0.40       |\n|         | Masking      | 0.69 | 0.20       | 0.42       |\n| 50      | No Attack    | 1.0  | 1.0        | 1.0        |\n|         | Filtering    | 0.90 | 0.67       | 0.80       |\n|         | Masking      | 0.91 | 0.65       | 0.82       |\n| 100     | No Attack    | 1.0  | 1.0        | 1.0        |\n|         | Filtering    | 0.97 | 0.93       | 0.96       |\n|         | Masking      | 0.98 | 0.94       | 0.96       |\n\nGeneration quality impact (300 samples):\n\n| Attack Method | CLIP | SigLIP | DINO |\n|--------------|-------|----------|--------|\n| No Attack    | 0.772 | 0.743    | 0.605  |\n| Filtering    | 0.531 | 0.527    | 0.503  |\n| Masking      | 0.559 | 0.546    | 0.511  |\n\n- These results demonstrate that while attacks can initially degrade detection effectiveness, this can be mitigated by increasing query numbers. For example, under filtering attacks, the AUC drops to 0.63 with 5 queries, but recovers to 0.97 with 100 queries.\n- Such attacks significantly compromise RAIG's generation quality, making them impractical for real-world deployment. Under attacks, the CLIP scores drop from 0.772 to around 0.531 (Filtering attack) and 0.559 (Masking attack), while SigLIP scores decrease from 0.743 to around 0.53 (Filtering attack) and 0.55 (Masking attack), representing a degradation in generation quality.\n\nWe will discuss these potential detection evading techniques in an expanded limitations section, along with detailed experimental results. Future work could explore more sophisticated attack strategies and corresponding defense mechanisms to further validate the robustness of our protection framework."}]

Author rebuttal:

Meta-review: The paper has received unanimous final recommendations to accept, two on the borderline and two firm.  The borderline reviewers have raised their scores during the discussion phase.

The majority of reviewers see ImageSentinels as a timely contribution given the rising interest in RAIG systems though all had questions over practical attacks and deployment.  There were engagements post-rebuttal around adaptive adversarial attacks and use of OCR that have strengthened the rigor and practical applicability of the work and should be included in the final version.  Overall the AC agrees with the reviewer consensus to accept the paper - congratulations.
Decision: Accept (poster)