Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environment
Authors: Tianyu Chen
Jian Lou
Wenjie Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Watermark
VLM
Dataset Copyright Protection
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: An effective watermarking framework for protecting the copyright of multimodal knowledge, especially image knowledge, in RaaS.

Abstract: As Retrieval-Augmented Generation (RAG) evolves into service-oriented platforms (Rag-as-a-Service) with shared knowledge bases, protecting the copyright of contributed data becomes essential. Existing watermarking methods in RAG focus solely on textual knowledge, leaving image knowledge unprotected. In this work, we propose \textit{AQUA}, the first watermark framework for image knowledge protection in Multimodal RAG systems. \textit{AQUA} embeds semantic signals into synthetic images using two complementary methods: acronym-based triggers and spatial relationship cues. These techniques ensure watermark signals survive indirect watermark propagation from image retriever to textual generator, being efficient, effective and imperceptible. Experiments across diverse models and datasets show that \textit{AQUA} enables robust, stealthy, and reliable copyright tracing, filling a key gap in multimodal RAG protection.

Published as a conference paper at ICLR 2026
SAFEGUARDING MULTIMODAL KNOWLEDGE COPY- RIGHT IN THE RAG-AS-A-SERVICE ENVIRONMENT
Tianyu Chen', Jian Lou?, Wenjie Wang"
SShanghaiTech University, Shanghai, China Sun Yat-sen University, Guangzhou, China {chenty12024, (aagggiii@saaaaatttee.e. edu. en louj5@mail. sysu. edu.cn cn
ABSTRACT
As Ietrieval-Angmentie Generation (RAG) evolves into service-oriented platforms AAAG-as-a-Servine with shared knowledge bases, protecting the copyright of contributed data becomes essential. Existing watermarking methods in RAG focus solcly on textual knowledge, leaving image knowledge umprotected. In this work, we propose AQUA, the first watermark frame- work for image knowledge protection in Multimodal RAG systems. AQUA embeds semantic signals inlo hetic images using two complementary methods: acronym-bas baased triggers and spatial relationship cues. These tech- niques ensure watermark signals survive indirect watermark propagation from image retriever to textual generator, and are efficient, effective, and imperccptible. Experiments across diverse models and datasets show that AQUA enables robust, stealthy, and reliable copyright tracing, filling key gap in Multimodal RAG protection. The implementation of AQUA publicly available at https::/ https://githhb.ccm http://tttuu..coottceeeeeeee AttA//
INTRODUCTION
Large Language Models (LLMs) of-
ten suffer from hallucinations and
User
RAG Service
Authorized
Watermark grobe Quary
Authorized Use
Provider
ouldated knowledge, which Retrieval- Augmented Generation (RAG) miti-
Provider
Generator
Watermark Detected
gates by retrieving external knowledge
at inference time Lewis et al.] 2020
Retriever
Knowledge Contribute
Guu et al. 2020 Asai et al., 2023
RAG has further evolved into RAGG
Watermark probe Query
as-a-Service (RaaS), where platforms such as Llamalndex (Liu) 2022) enable shared knowledge bases contributed by multiple providers (Figure D]. These systems follow a "usable but not visi- ble" policy: service providers can use contributed knowledge without direct. access to raw data.
Unauthorized RAG Service
Unauthorized Commercial Use
Provider
Detected
but not visible
Unauthorized Commercial
Figure 1: Overview of the RAG-as-a-Service (RaaS) workflow. Data providers contribute proprictary knowledge shared knowledge base used by RAG service providers to serve end users. Data providers issue watermark probe queries to RAG services. If the watermark is detected in an unauthorized RAG service, it indicates unauthorized use.
While RaaS enables a mutually ben- eficial ecosystem between knowledge providers and service platforms, also introduces copyright and ownership challenges. In particular, providers require mechanisms to reliably trace data usage and restrict access to authorized services. Since unauthorized RAG providers typically utilize the entire shared knowledge base, practical solution to embed watermarks at the knowledge base level. The detection of these watermark signals in a provider's output can then serve as strong evidence of unauthorized data usage (Figure bottom).
*Corresponding author.

---

Published as a conference paper at ICLR 2026
Existing watermarking methods for RaaS have primarily focused on textual knowledge (Jovanović et al.] 2025 Guo et al. 2025). However, these methods are modality- specific, are limited to the text modality, and cannot be directly applied to non-textual knowledge due to the distinct characteristics of other modalities. In practice, RaaS systems increasingly integrate multimodal knowledge, combining textual and visual content (Riedler & Langer 2024 Xia et al. 2024a bb. This creates fundamental gap and leaves a critical vulnerability in the copyright protection of Multimodal RaaS, To address this gap, we focus on a represcntative subclass: text-to-text (T2T) Multimodal RAGG, where the generator integrates retrieved image knowledge and the textual query to generate
textual responses (Yasunaga et al. 2022 Chen et al. 2022
Lin & Byrne 2022
Sun el al.
2024
Zhu al. 2024).
(A)
Undrect watermark propagation
(B)
distrbutton
(C) Implicity watermark and detect on Itself
Sirorraa Gerarratr
Explicity watermark and detect on teertual output
Unapparent shift
Generatorr
Oirect watermank propagation
Image watermark traditional setting vs. in RAG
Plain-text RAG vs. Multimodal RAG
Figure 2: Challenges of watermarking Multimodal RAG knowledge compared with plain-text RAG, and image watermarking in traditional settings.
Compared to plain-text RAG, applying watermarking strategics in T2T Multimodal RAG poses unique challenges. First, while text-based RAG supports direct watermark propa- gation, Multimodal RAG requires embedding the watermark in images and reflecting it in generated text, resulting in indirect propagation that is harder to preserve. Second, unlike textual watermarks typically involving unusual tokens resulting in obvious distribution shift. from original knowledge (Chen et al., 2024c Cheng et al. 2024). image knowledge differs only at the pixel level while preserving semantic naturalness, resulting in unapparent distribution shiftss (Figure (B)), that reduce retricvability. Morcover, existing image watermarking methods (Luo et al. 2020 Chen et al. 2024a) rely émplicit perturba- tions designed for image-level detection, but Multimodal RAG requires watermarks to be explicitly retrieved through queries, making them unsuitable for retrieval-based multimodal settings.
To address the above challenges in image knowledge copyright protection, we propose AQUA, a novel watermarking framework tailored for T2T Multimodal RAG. Specifically, the AQUA watermarking framework includes two complementary watermarking methods: AQUA acronym, and AQUA spatiall AQUAacrama addresses indirect watermark propaga- tion by embedding uncommon acronyms and their full names into synthetic images. In the verification phase, these acronyms are decoded through the Optical Character Recognition ion (OCR) abilities of Vision-Langge Language Models (VLMs) (Achiam et al., 2023 Gemini Team al., 2023 Huang et al. 2023) to generate detectable textual responses: the full name of the acronymss. Despite moosl transformation, the textual nature of the signal embedded in the image increases its chance of surviving end-to-end processing.
For models with limited OCR ability, AQUA spatial is designed to create synthetic images with special object configurations (e.g., uncommon positional relationships), and to leverage generators' understanding of spatial semantics to answer position-related probe queries. These positional relationships can bridge the gap between image semantics and textual outputs, allowing indirect watermark propagation from retriever to generator. Both methods introduce semantic distinctiveness by embedding subtle semantic cues into natural-looking images, allowing explicit retricval while maintaining high retrieval rate. Together, these two mothods provide a flcxible, robust solution to the unique challenges of watermarking in Multimodal RAG systems, supporting both black-box and white-box deployments.
Despite its simplicity, our novel insights of using synthetic images with special acronyms text. and special positional relationships watermark carriers particularly effective and
2

---

Published as a conference paper at ICLR 2026
efficient in bridging the gap between image-based watermarking and textually detectable outputs, enabling robust copyright tracing in Multimodal RAG. We evaluate AQUA across diverse Multimodal RAG systems and datasets spanning different domains. The experimental results demonstrate that. AQUA (1) enables the watermark images to be retrieved and reflected in the generated textual output, (2) prevents false positive retrieval from common image content, (3) remains imperceptible to users and undetectable by unauthorized filtering mechanisms, and (4) is robust attacks such image transformations and regeneration.
Our contribution can be summarized as follows:
We propose AQUA, the first watermarking framework for image knowledge copyright protection in Multimodal RAG, addressing indirect watermark propagation and successful retrieval under unapparent distribution shifts and explicit watermark injection.
We design two complementary watermarking strategies, AQUA acronym and
AQUAspatial to support more realistic black-box scenarios.
We conduct comprehensive experiments on RAG datasets and RAG architectures to demonstrate the effectiveness, harmlessness, stealthiness and robustness of AQUA.
AQUA can serve as a crucial baseline methodology for the emerging research area. focused on copyright protection for multimodal datasets RaaS.
RELATED WORK
2.1 MULTIMODAL RETRIEVAL- GENERATION
Relying only on textual information a limited approach for describing the intricate nature of the physical world. Yu et al. (2024): Mei ct al. 2025 Papageorgion et al. (2025) extend the text-only RAG framework to multimodal scttings and explicitly incorporate diverse data modalities into both the retrieval and generation stages. A common strategy for enabling cross-modal retrieval is to employ powerful multimodal encoders (e.g,, CLIP (Radford et al. 2021)), map different modalities (e.g., text and images) into a shared semantic embedding space. This unification allows standard vector search algorithms like cosine similarity to retrieve relevant items across modalities based on semantic relatedness.
2.2 RAG WATERMARKING
Several watermarking approaches have been proposed to protect the copyright of textual knowledge in RAG. WARD (Jovanović et al. 2025) uses the LLM red-green list watermarking
technology watermark all text in the RAG knowledge base (Kirchenbauer al.) 2023
Gloaguen et al. approach that marks. RAG (Guo et al. 2025) leverages Chain-of Thought (CoT) (Wei et al. 2022) to establish a watermarking approach. DMI-RAG (Liu et al. 2025) performs dataset member- ship inference by injecting small number of synthetic, watermarked "canary' documents into the Intellectual Property (IP) dataset. However, existing methods on watermarking knowledge bases in RAG systems have exclusively focused purely textual data. To the best of our knowledge, no prior work has addressed the protection of knowledge copyright in Multimodal RAG systems, particularly those integrating image and text modalities, via watermarking techniques.
2024). RAG-WM (Lv et al.
2025 presents a black-box RAG watermarking
leverages interactions among multiple LLMs generate high-quality water-
PRELIMINARY
In this section, we outline the workflow of the T2T Multimodal RAG system and define the notation Section 3.1 Then, we establish the threat model for knowledge copyright protection in Multimodal RAG systems in Section 3.2
3.1 MULTIMODAL RAG SYSTEM WORKFLOW
The T2T Multimodal RAG system consists of three components: a retriever E, a generator 9, and external image knowledge base D. The retriever includes text encoder Etert
3

---

Published as a conference paper at ICLR 2026
and image encoder Eimg. Images I, in the external knowledge base D {I1 ........ In} are preprocessed to a latent space through the image encoder: é,, Eing (4, R
The retriever accepts the user's text query T as input, and processes into the same latent space images eT: (Te) E Rd. Then the retriever employs similarity function, Sim(..-) Rd x R + Score (o.g., cosine similarity), to find the most relevant image knowledge according to the user's text query: 8, (i.)) Based these similarity scores $, the retriever selects the top-k most relevant images as output:
Dretrieved = R(D,T.A) = (Jury Teea........................IT.. Jsgg) where Stopok (8(1)= . ((0)) (1)
The original lext query T and the retrieved set of images Dretriened are combined and passed to the generator G lo produce the final answer: A = A= (Dreetrieedd:
3.2 THREAT MODEL
We consider image knowledge copyright protection in Multimodal RAG services.
Defender represents the knowledge provider, aiming to detect and audit unauthorized use of its proprictary image knowledge by external Multimodal RAG services. In practice, the Defender lypically has no visibility into which knowledge bases are included in a deployed Multimodal RAG service, and they can only access through a public APT interface. Defender only operate on its own datasets to implement protection mechanisms such as injecting watermarks before contributing its data to RaaS.
Ad is Multimodal RAG service provider who incorporates image datasets without authorization, with the goal of improving system performance while avoiding licensing costs. Adversary may unknowingly ingest the watermarked data and expose its presence through the system's 's generated outputs, creating an opportunity for Defender to audit its misuse.
4 METHODOLOGY
AQUA a watermarking framework designed for copyright protection of image knowledge in Multimodal RAG services, meeling four key requirements: effectiveness, harmlessness, stealthiness, and robustness. In this section, we instantiate the AQUA framework with two complementary watermarking methods, AQUA acrongmn and AQUA
Aputipiaaal spatial
1. Watermark Injection
II. Watermark Verification
1. AQUA
AcQooncro acronym
Prohe Query
ftigger: Backgmund
Image
Unicom Grammay
Design specal
UGP Unicoen Grammmr of Purser
Irstruction:
Unismded Group Playing
unusual
What is the tull hame UGG
2. AQUA Appttol!!!!
Drobe Query
Trigger:
There 5.3 dog
Image Retriever
VLM
rerding book
Captions: apple the head of reading dog.
S.
linttrcction:
Generativ
Anrove based on the imagee: Whist on dag"
There's nothing
Figure 3: Illustaaioon of the watermark injection (left) and verification (right) of AQUA.
4.1 AQUA acronym
Watermark Injection. AQUA acronymm addresses indirect watermark propagation from image knowledge to delectable lextual output by embedding uncommon acronyms and their full names into synthetic images.
The Defender can design or invent rare acronyms, each paired with unique full name, such as (UGP, Unicorn Grammar Parser) in Figure 3 Since this full name is crafted by the Defender, it can be regarded as secret key, which is unlikely to be learned by the Multimodal RAG generator as static knowledge. Despite cross-modal transformation, the textual nature of the signal embedded in the image increases its chance of surviving end-to-end processing. Acronym pairs can also be generated in large quantities using an I.I.M (e.g. 2---rr via In-Context Learning (TCL.) (Brown et al. 2020) and the
4

---

Published as a conference paper at ICLR 2026
prompt provided in Appendix A.1 and more examples are provided in Appendix A.2 Each pair is then embedded as a watermark image and injected into the image knowledge base: D Drriginat U D matermark. These images are designed to be minimally invasive and do not affect the model's utility for normal queries.
Watermark Verification. In the verification phase, these acronyms decoded through the generator's OCR ability, producing detectable textual responses: the full name the acronyms. Each watermark image has its own probe query Trobe which can be used by the knowledge provider to detect unauthorized use. The probe consists of two parts: a trigger Tirigger, used by the retriever to retrieve the watermark images, and an instruction dinetruction: which prompts the generator to generate the watermark- responses that can be detected. We can formulate this construction as Tprobe = Trigger ) Tinstruction- For example, in Figure 3 Trrigger is "Background: UGP is machine" and Ti innstruction is "What the full name of UGP?" To verify the watermark signal, we define a strict substring-match protocol Eval(-.-) based on normalization function Norm(.) that lowercases and strips whitespace from both generated output ORAG and the verification signature S:
Eval(OnAc. S) (S) S Norm(OnAG)]
where T[|] is the indicator function, returning if the condition (substring presence) is true, and 0 otherwise. The predefined signature (e.g., 'Unicorn Grammar Parser") serves the ground truth. Due to the inherent randomness of generation (e.g,, temperature, top-k/ t/oppp sampling) (Fan et al.) 2018 Holtzman et al. 2019), the presence of watermark signal is not guaranteed even when the corresponding image is retricved. To address this, we adopt two strategies: (1) injecting multiple distinet watermark images and (2) issuing varied probe queries per watermark. We define the Verificalion Success Rate (VSR) as:
Num Nac
VSR
FOal(O)a
N
Num
Nds
where Num is the number of watermark images and Nds is the number of distinct queries per watermark image. denotes the j-th injected watermark, and i denotes the i-th probe query with its corresponding signature S,. ORAG,, the generated output when the i-th probe query is issued against the j-th watermark image.
Hypothesis Testing. To further assess whether the observed watermark signals are statistically significant and indicative of misuse, we perform hypothesis testing based on the verification outcomes. Specifically, following Xu et al. (2023), we conduct Welch's t-test. (Welch 1947) to compare the behavior of the suspect Multimodal RAG and the clean Multimodal RAG. The null hypothesis (Ho) indicates that there is no statistical evidence suggesting the suspect Multimodal RAG includes the watermark image datascts: Ho: : Pauspect = Halean, where the VSR of the suspect Multimodal RAG is equal to the VSR of the clean one. Using the sample means, variances, counts, and approximated degrees of freedom via the Welch-Satterthwaite equation (Satterthwaite) 1941 1946), we compute the t-statistic. The p-value is compared against significance level (e.g., 0.05) to decide whether to reject. Ho and conclude potential horizer use. The practical deployment of AQUA presents unique challenges due to the specific state of each target RAG database; these issues are discussed in detail in Appendix B.1
4.2 AQUA AQUAspatiall
spatial
Watermark Injection. For models with limited OCR capabilities, we propose AQUA spatial: which is designed to create synthetic images with special object configu- rations (e.g., unusual positional relationships), and to leverage generators' understanding of spatial semantics to answer position-related probe queries. Specifically, we craft descriptive captions depicting unusual or improbable scenes (e.g., "A red apple the head of reading dog." " and generate corresponding images using a diffusion model (Sohl-Dicksteiin et al. 2013 Ho et al. 2020 Rombach et al. 2022 To ensure semantic uniqueness, these captions are constructed from low co-occurrence concept pairs and further filtered by language model perplexity to balance distinctiveness with naturalness (Appendix A.4). These senthesized
5

---

variant as Published as a conference paper at ICLR 2026
images watermark images, as illustrated in the second part of Figure 3 Similar to AQUA ucronym: these watermark images are injected into the dataset and can be scaled using LLM-baser in-context generation of diverse captions. More examples and image caption
templates are provided in Appendix A.2 and Appendix A.4 respectively.
Watermark Verification and Hypothesis Testing. The verification and the hypothesis testing are similar to those of the AQUA acronym method. Each watermark image is probed using query composed of trigger and instruction, e-g., Tirigger "There book." and iinstruction = Answwr based on the images: What fruit is on the dog's ' head like hat?". The expected signature is " "Apple". As before, the system output is evaluated using the substring-ma match protocol -val(-,-), and Welch's t-test is applied to determine whether the suspect system statistically includes the watermarked dataset.
dog reading
4.3 EVALUATION METRICS
We evaluate AQUA using multiple metrics that capture both retricval and generation performance. Verification success rate and typothesis-tcsting testing-bnnnddd based assessments quantify the overall elfectiveness of watermark detection. In addition, we introduce Rank and Conditional Generation Success Rate (CGSR).
Rank quantifies the strength of the association between the trigger component Trrigger of probe query and its corresponding target watermark image Lymi a lower Rank indicates better retrieval performance. For given query, Dretrieved (2,, , indicates the top-k retrieved images. The Rank defined as the 1-based index of Lum within Dretrieved If Iwm is not present within the top-k retrieved images, penalty value, set to twice the retricval depth (2k)), is assigned. Formally, the Rank calculated as:
Rank(Lwm Dretrieveds k) - ir (1...., k) such that T, T Tun
(4)
2k otherwise
Conditional Generation Success Rate (CGSR) measures the proportion of successful generations where the verification signature S is correctly produced, given that the corre- sponding watermark image has been successfully retrieved. higher CGSR value signifies that this watermark image can better transmit the watermark signal through the black-box RAG system. Let Tretrieved be the querics for which the retricval of the watermark image is successful. The CGSR is then defined as the success rate over the subset of successful retrievals:
CGSR ExeTratrissed Eval(O RAG: S(1)
(5)
[Teerieeed]
SimScore quantifies the semantic similarity between two textual strings, as judged by an LLM (Gemini-2.5-Prro) with scores ranging from 0 to 100%. In our evaluation, we apply SimScore to query pairs to assess false triggering risk and to response pairs to measure whether the watermark distorts normal system outputs.
EXPERIMENTS
In this section, we perform extensive experiments to evaluate AQUA's performance. We cover the experimental setup (Section 5.1). and two baselines (Section 5.2), followed by
assessments of effectiveness (Section 5.3 harmlessness (Section 5.4 stealthiness (Section
5.5
and robustness (Section 5.6
5.1
EXPERIMENTAL SETUP
Datasets. We utilize two widely used multimodal datasets: MMQA Talmor et al. 2021 and WebQA (Chang et al. 2022). Both datasets contain a large number of QA pairs, and the questions can only be answered by combining knowledge of modalities such text, images, and tables. We use the complete image part of these two datascts, totaling 58,075 images in MMQA and 889,749 images in WebQA, the experimental image dataset.
Multimodal RAG Components, We use the Contrastive Language-Inan Pre-training (CLIP) (Radford al. 2021), specifically the gpenai/cli-vvit-large----

---

Published as a conference paper at ICLR 2026
RETRIEVER. Cosine Similarity is used to compute the similarity between text and image. Following the usual search strategies (Caffagni et al.) 2024 Mortaheb et al. 2025 Ha et al. 2025 we set clip-top-k=5, ensuring the retriever selects the five most relevant images as context. The GENERATOR contains the following four different VLM variants: LLaVA-NeXT
(7B) (Liu et al, 2024 Chen et al. 2024d Bai et al., 2023) 2025a). To control the diversity of the outputs, we configure the decoding process for each VLM using standard sampling parameters: sampling temperature (T = 1.2), top-k sampling (top_k: = 5), nucleus sampling (top_p 0.9). These
(7B). InternVL3 (8B), Qwen-VL-Chat (7B), and Oxen2.5-VL-Instreee
settings are maintained consistently across experiments.
5.2 BASELINES
We propose two baselines to compare with AQUA: an optimization based method. Naive baseline uses common images as watermark images. These images are not unique to the Defender's database but may also appear in databases of other data providers. Specifically, we randomly crawled more than 10,000 images from the Internet over 100 domains, and selected subset of them watermark images.
Naive random select method and
The optimization-based method follows a conventional image watermarking approach by embedding imperceptible optimized patterns. These adversarial patterns optimized by distilling a special phrase into the image. Specifically, a perturbation is optimized and added to a base image lbuse such that, when the watermarked image is queried with a textual prompt probee the generator & produces an output containing a predefined signature S. The optimization objective to minimize the cross-cntropy loss between the generated response and the target signalure:
min (Thaae + 5, + )
(6)
We adopt Projected Gradient Descent (PGD) Gooldseei 1964
Levitin & Polyak 1966
to optimize the perturbation iteratively, as widely-adopted and ellective vevesaaiall perturbation generation method:
8++1 11 Se (6: a Vs, IGI(Taaee S6, Tprobee) S))
(7)
where a represents the step size (learning rate), and the projection operator Milll ensures the perturbation remains within an Lormor ball of radius e, preserving visual imperceptibility. The final watermarked image is Inm Itase + 8*.
5.3 EFFECTIVENESS OF AQUA
This section presents an empirical evaluation of the effectiveness of the proposed AQUA framework. Performance is quantified using Rank and CGSR metrics, with results summarized in Table I Our experimental protocol adheres to the paradigm established by Yao et al. (2024), utilizing a dataset of 50 distinct watermark images. Each image is subjected to 10 unique probe queries with diverse syntactic structures. To ensure statistical robustness, the entire experiment is replicated 10 times.
Welch's t-test conducted to assess the statistical significance of the detection results. The analysis yields p-values consistently below the conventional significance level (a = 0.05), which leads to the rejection of the null hypothesis, Ho Msuspect = jelean This outcome provides compelling statistical evidence that AQUA can reliably detect the presence of injected watermarks. For a complementary analysis, and in accordance with the methodology of Jovanović et al. (2025). Appendix C.1 provides the results of a Two-proportion Z-lest, along with experimental results across models of varying parameter sizes.
Analysis of Query Efficiency. Although the optimization- achieve a statistically significant result (i.e., a low p-value) to reject the null hypothesis, the number of queries required to do so is critical performance metric in real-world applications, particularly where queries are costly or limited. We therefore evaluate query efficiency by measuring the number of queries each method needs to reach the significance threshold. As depicted in Figure 4a both AQUA and AQUA spatial achieve a p-value below the significance level within 30 queries. In contrast, the Opt. baseline requires over 200
method can also ultimately
7

---

Published as a conference paper at ICLR 2026
Table 1: Effectiveness of AQUA. Models indicate which model is used as the generator. AQUAucrompa fecrommm
and AQUA spotial represent the two watermarking methods. Naive and Opt. denote the baseline methods.
MMQA
WebQA
Models
Methods
Rank
CGSR
p-valuet
Rank
CGSR1
p-valuet
Naive
2.86
28.16%
0.32
4.56
13.28%
0.93
LLaVA
Opt.
1.45
31.03%
3.33e
1.90
22.86%
3.94c
-NXXT
AQUA A acrongm
1.03
85.36%
0.0
1.05
78.73%
9.47e 182
AQUA A spatial
1.29
75.38%
1.07e-00
1.85
86.45%
2.3c
Naive
2.86
27.11%
0.41
4.56
17.12%
0.65
InternVL3
Opt.
1.45
19.34%
5.39e
1.90
19.45%
3,87e
A AQUA acronym
1.03
35.11% %
6.29e
289
1.05
78.34%
2.88e
129
AQUA spatial
1.29
75.72%
1.49e
50
1.85
72.46%
4.31e 26
Natue
2.86
15.79%
0.59
4.56
5.71%
0.91
Qwen- VL
Opt.
1.45
21 21.29%
9.05e
1.90
18.91%
1,21e
Chat
AQUA ucrangm
1.03
75.28%
1.05e
162
1.05
77.86%
1.24e
128
AQUA sputial
1.29
78.92%
1.35e
60
1.85
68.46%
9.63e 35
Naive
2.86
38.15%
0.25
4.56
15.87%
0.86
Qwen2.5-
Opt.
1.45
19.96%
7.35e
1.90
18.51%
6.77c -3
VI.Innttuct
AQUA acronym
1.03
99.61%
%
0.0
1.05
96.68%
6.6e 145
AQUA spatial
1.29
98.42%
1.85
89.85%
2.92e
queries to attain the same level of statistical confidence. This result demonstrates the substantially superior query elficiency of the AQUA A framework compared to the baselines.
FPR vs. TPR. To further val- idate the offectiveness of AQUA, we analyze its True Positive Rate (TPR) against its False Positive Rate (FPR), as shown in Figure [b] We calculate FPR by evaluating the gen- erator's (NeAVA-NeXT) output clean database, while TPR is mea- sured using databases containing 2, 3, 5, and 10 watermarked im- ages per probe. The substantial dis- tance of the AQUA curve from the random baseline indicates strong statistical separation between water- marked and clean distributions. This characteristic is crucial, as it confirms that AQUA can achieve a high detection rate while keeping the false positive rate exceptionally low, thus validating the method's precision and reliability.
10
10-1
0.8
0.6
50
TPR
10-12
0.4
10-11
Katee
Neive
4OUA_scromy
Opt.
ADUA_ecronym
AQUALSpaNNN
Query Times
0.00
0.2
0.4
FPR
0.0
0.8
1.0
(a) p-value query times
(b) TPR FPR
Figure Diagnostics of AQUA watermark detection.
5.4 HARMLESSNESS OF AQUA
Normal Query. To verify the harmlessness of our watermark, we evaluated the system's responses to over 10,000 benign queries sourced from the MMQA and WebQA datasets. A watermark is considered harmless if is neither retrieved nor reflected in the generated output during the system's normal operation. In experiments, with single watermarked image embedded in the knowledge base, the retricval rate for the watermarked content is 0% for both the AQUA acronyym and AQUAspatiaa variants. Concurrently, the CGSR also 0% across all four generators tested. These results confirm that our verification signature
8

---

Published as a conference paper at ICLR 2026
Table 2: Examples of relevant queries and corresponding results.
Type
Example Probe Query
Example Relevant Query
Rank SimScore
Acroym-- replace
What is the subtitle of UGP?
What is the subtitle of ATM? 10.00
100%
Acronym-
What is the subtitle of UGP?
What is UGP?
1.07
70.18%
instru
Spatial-
What fruit the monkey holding like phone? What is the monkey holding? 2.93
75.87%
imprecion
remains latent during standard interactions and does not interfere with the generation of correct responses to benign queries.
Relevant Queries. We employ relevant queries to assess whether the embedded watermark interferes with the model's standard output when processing prompts highly similar to the probe query. The experimental results on LLaVA-NeXT and MMQA (Table 2 indicate that the unique acronym within the probe query is substituted with a common one, the embedded watermark does not impact the output for the relevant query. For queries that partially retain trigger components from the original probe, such as Acronym-no instruction and
the watermark is retrieved at a limited rate. Nevertheless, the high SimScore indicates that the watermark's presence does not siguificantly alter the final output, finding that underscores the harmlessness of our AQUA method. Additional results are presented in Appendix C.2
5.5 STEALTHINESS OF AQUA
PCA Visualization. Inspired by prior work (Boler et al. 2022 Chen et al. 2024b Yao et al. 2025), we observe that it is challenging to filter out watermarked images their embeddings highly similar to those of the original images in the dataset. Therefore, randomly selected five watermarked images and their corresponding probe queries, along with 300 original images and normal queries, and visualized their embeddings using Principal Component Analysis (PCA) in Figure 5 These results indicate that AQUA maintains strong stealthiness while preserving high retrieval performance.
VLM-based Filter as an Adaptive Adversary. To simulate an adoplive adversary allempting to sanitize dataset with VLM-based filter, employed Qwen2.5-VLL-nee (32B) to filter the dataset containing watermarked images. Results are averaged over 5 independent runs. In each trial, we randomly sampled 5,000 images from the MMQA dataset and randomly injected 10 different watermarked images. In each query, set of 5 images is presented to the VLM filter, which tasked with detecting any watermarked instances. The prompts used are provided in Appendix A.1
Table 3: FPR represents the propor- tion of non-wer -oowaaatrddre images in- correctly identified as watermarked, while TPR is the proportion of wa-
MMQA
Normal Query AQUA acronym
AQUA,, acronym
spatal
AQULA
termarked images that are correctly identified.
Methods
Metries
Result
Precppal Compinen pendipd Conpenent
Preccpal Compgmmntt
Proccial Companert
FPR
2.84%
AQUA acronym
TPR
0%
Figure 5: PCA Visualization of Watermarked vs. Normal Images and Probe Queries vs. Normal Queries.
FPR
2.36%
AQUA spotial
TPR
The results Table show that AQUA's 's watermarked images are highly resistant to the basic VLM-based filter, achieving 0% TPR while maintaining a low FPR for normal images. We further evaluate a more stringent VLM-based filter in Appendix C.3 (Table 12 where although the filter successfully detect some watermarked images, it concurrently incurs a prohibitively high FPR on benign images. This confirms that AQUA's images utilize semantic information within seemingly normal images as the verification signal, making any

---

Published as a conference paper at ICLR 2026
filtering approach face an inherent trade-off between detection rate and false positive rate. For visual comparisons confirming indistinguishability, please refer to Appendix A.3
Retrieval Ratio Watermark Number. Furthermore, evaluated the impact of an increasing number of injected watermarks on normal queries. Our results show that even when adding up to 10,000 watermarked images to the 50,000-image MMQA datasct, the FPR of watermark images for normal queries consistently remained below 0.1%. More results and figures can be found in Appendix C.3
5.6 ROBUSTNESS OF AQUA
Image-Level Attacks. To evaluate the robustness of AQUA, we conducted experiments using the WAVES benchmark (An et al.) 2024). For the experimental protocol, the attack "strength' parameter is uniformly set across all watermark distortion and regeneration methods. A total of 50 watermarked images are selected for each technique, with the entire MMQA dataset serving as the original data corpus. All experimental results, generated by the Qwen2. Swen2.5-VL-InSSS 5-VL-Instruct (7B) model, are presented in Table 4 The results indicate that images watermarked by AQUA sustain high retrieval rates and posítive statistical verification outcomes following various image transformations, distortions, and regeneration attacks, which demonstrates the robustness of the proposed watermarking scheme.
Table 4: Robustness under image attacks. "Regen Both" " denotes the sequential application of Regen VAE and Regen Diffusion.
VAg
AQUA
AQUA spotiul
AQUA aeronge
AQUA apatial
Attack
ucrongm
Attack
Rank4
CGSIL
Rankj
CGSR
Rank]
CGSR
Nank]
CGSI
Rescale
1.026
99.33%
1.355
95.78%
Regon VAE
1.052
97.61%
1.498
93.91%
Rotate
1.071
98.54%
1.613
89.80%
Regen Diff.
1.036
98.17%
1.502
94.335%
Gaussian
1.068
99.00%
1.459
91.21%
Regen Both
1.037
96.55%
1.516
87.39%
Brightness 1.053
98. 59%
1.454
90.76%
Riuse 2x 2xDiff
1.032
97.78%
1.482
90.28 90.29%
Compression
1.027
98.96%
1.288
97.36%
Rinse Diff
1.028
97.01%
1.548
88 88.69%
System-Level Variations. In addition to image-level distortions and regeneration attacks, we further evaluate the stability of AQUA against practical deployment variations, such as benign system updates and corpus expansion. As detailed in Appendix [C.4] altering the retrieval model (from CLIP to SigLIP (Zhai et al., 2023a)) ) or fine-tuning it yields only marginal changes to the retrieval Rank (Table 15p. Furthermore, when the retrieval corpus is substantially enlarged by introducing 50k WebQA distractor images, the detection performance remains highly robust, achieving a CGSR of 96.76% for AQUA AQUonmromy and 91.85% for AQUA spatial (Table 16). These results confirm that AQUA maintains reliable performance under common real-world system modifications and data scaling.
Genorator-Level Adaptations. We further consider adversarics who attompt to circumvent the watermark at the generator level. First, we test whether fronticr VLMs (Gemini- 1aai--3--0 3-Pro,
OPT-5.1-Hight Qwen3-VL) (Google DeepMind 2025 OpenAl 2025 Bai al. 2025b) refuse to process waternarked images as "unnatural" inpuls; no relusal behavior is observed across all models. Second, we simulate an adversarial fine-tuning attack using LoRA (Hu et al. 2022) to train the generator to specifically reject watermark-relatedd queries. While CGSR decreases moderately, the watermark remains reliably detectable, and the fine-tuning incurs significant utility degradation (-8.62% on MMQA accuracy), making it impractical stratogy for adversarics. Full details are provided in Appendix C.4.1
CONCLUSION
This rescarch focuses on safeguarding the copyright of image datasets in T2T Multimodal RAG systems. We propose AQUA, a watermarking framework that meets four design requirements: effectiveness, harmlessness, stealthiness, and robustness. The two complemen- tary watermarking strategies in AQUA enable copyright protection for image datasets via statistical verification using only a few watermark images. Since AQUA is the first method to protect data copyright through watermarking in realistic black-box Multimodal RAG scenarios, AQUA can serve as a crucial baseline for future studies in Multimodal RAG data protection, contributing to more robust copyright protection in this important area.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We sincerely thank the anonymous reviewers for their detailed and constructive feedback. We also grateful to the area chairs for their dedication in carefully handling the review process despite the significantly increased workload during ICLR 2026. This work is supported by Shanghai Engineering Research Center of Intelligent Vision and Imaging. We also thank members of the ASPIRE Lab at SIST, Shanghai University, for helpful conversations during the course of this work.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lgge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08771 2023.
Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstcin, et al. Waves: Benchmarking the robustness of image watermarks. arXiv preprint arX:v:2401.08573 2024.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannanch Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Represenlations, 2023.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: versatile vision-language model for understanding, localization, text reading, and beyond, 2023. URL abs/2308. 12966
org/
Shuai Bai et al. Qwen2.5-vl technical report. arXiv preprint arXim:2502119923 2025a. doi: 1145550/arXi.25022.....
URL
13923
Shuai Bai et al. Qwen3-vl technical report. arxiv preprint arXiv:2511.2663/ 2025b. doi: URL httts:://ariiii httpa////aaaii.o///aa/////////// wwt.oorg///aaaaaeeeoeee 21631
1148550/arriv.25111
William Boler, Ashley Dale, and Lauren Christopher. Trusted data anomaly detection (tada) in ground truth image data. In 2022 IEEE Applied Imagery Pattern Recognation Workshop (AIPR), pp. 1-6. IEEE, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances neural information processing systems, 33:1877-1901, 2020.
Davide Caffagni, Federico Cocchi, Nicholas Moratelli, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Wiki-llava: Hicrarchical retrieval-angmented generation for multimodal llms. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1818 1826, 2024.
Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. Webqa: Multihop and multimodal qa. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16495-16504, 2022.
Feiyu Chen, Wei Lin, Ziquan Liu, and Antoni B Chan. A secure image watermarking framework with statistical guarantees via adversarial attacks on secret key networks. In European Conference on Computer Vision, pp. 428-445. Springer, 2024a.
Jianfa Chen, Emily Shen, Trupti Bavalatti, Xiaowen Lin, Yongkai Wang. Shuming Hu, Harihar Subramanyam, Ksheeraj Sai Vepuri, Ming Jiang, Ji Qi, et al. Class-rag: Real-time content moderation with retrioval angmented generation. arXiv preprint arXiv:2/10.. 14881, 2024b.
11

---

Published as a conference paper at ICLR 2026
Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W Cohen. Murag: Multimodal retrieval-angmented generator for open question answering over images and text. arXiv preprint wwx.:2210.02928, 2022.
Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red- teaming Ilm agents via poisoning memory or knowledge bases, 2024c. URL https //arxiv.org/ abs absabb// 2207 12784
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 24185-222 24198, 2024d.
Pengzhou Cheng, Yidong Ding, Tianjie Ju, Zongru Wu, Wei Du, Ping Yi, Zhuosheng Zhang, and Gongshen Liu. Trojanrag: Retrieval-augmente generation can be backdoor driver in large language models. arXiv preprint or Xrv:222405 290.0111,,, 2024.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint 1805 1805000433,, 2018.
Gemini Team, Rohan Anil, Sebastian Borgeaud, tean-Baptist, Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1 1805, 2023.
Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Black-bo detection of language model watermarks. arxiv preprint arXiu:2405.2077, 2024.
A. A. Goldstein. Convex programming in hilbert space. Bulletin of the American Mathemat- ical Society, 70(5):709 710, 1964. URL https://projeecteucldddd oorrooooiii...oooooo.aaaaaaaaa
of american hwt.ee/ww....caaaaaaaaaaaaaaaaar
5/sueee Sasueeeg//Oovexpprrrrraaanni-ni hpg.a//waaaaaa..iiiarrrrrrro/raa
Google Deep.indd Gemini 3 pro model card, 2025. URL http///ts:// googleppis
http/deepmind-mmdeeee date: December 2025.
-ero-----Po-Moeee---rr pdf Model card up-
Junfeng Guo, Yiming Li, Ruibo Chen, Yihan Wu, Chonxi Liu, Yanshuo Chen, and Heng Huang. Towards copyright protection for knowledge bases of retrievallangmmeneed language models via ownership verification with reasoning, arXiv preprinl ar Xiv:2500 10440, 2025.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pro-training. In International conference on machine learning, pp. 3929 3938. PMLR, 2020.
Hyconjcong Ha, Qiusi Zhan, Jconghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng, Kai-Wei Chang, Daniel Kang, and Heng Ji. Mm-poisonrag: Disrupting multimodal rag with local and global poisoning attacks. arXiv preprind ar urXiv:2502. 7832, 2025.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 6851, 2020.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural lext degeneration. arXio preprinl arXiv: 09751, 2019.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhn Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In Internalional Conference on Learning Representations, 2022.
Jiaxing Huang, Jingyi Zhang, Kai Jiang, Han Qiu, and Shijian Lu. Visual instruction tuning towards general-purpose multimodal model: survey. arXiv preprint orXiw:2312.1660 2023.
12

---

Published as a conference paper at ICLR 2026
Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, et al. Neftune: Noisy embeddings improve instruction finetuning. In International Conference on Learning Representations, 2024.
Nikola Jovanović, Robin Staab, Maximilian Baader, and Martin Vechev. Ward: Provable rag dataset inference via IIm watermarks. ICLR, 2025.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, lan Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pp. 17061-17084. PMLR, 2023.
Evgeny Levitin and Boris T Polyak. Constrained minimization methods. USSR Computa- tional mathematics and mathematical physics, 6(5):1-50, 1966.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- augmented generation for noowledgeintensiigg nlp tasks. Advances in neural information processing systems, 33:9459 9474, 2020.
Weizhe Lin and Bill Byrne. Retrieval augmented visual question answering with outside knowledge. arXiv preprint 2rX.:22210008809 2022.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jac Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL Blog post.
https://Mlav--ll.. httphb...obblog//2420133333////1
Jerry Liu. Llamaindex, 2022. httpp:://githbbccommrmii repository.
_adillmm__ddexxx___iii GitHlub
Yepeng Liu, Xuandong Zhao, Dawn Song, and Yuheng Bu. Dataset protection via water- marked canarics in retrieval-augmented 1lms. arXiv preprint arXiv:2502. 10679, 2025.
Xiyang Luo, Ruohan Zhan, Huiwen Chang, Feng Yang, and Peyman Milanfar. Distortion agnostic deep watermarking. In Proceedings of the IEFE/CVP conference compuler vision and pattern recognition, pp. 13557, 2020.
Peizhuo Lv, Mengjie Sun, Hao Wang, Xiaofeng Wang, Shengzhi Zhang, Yuxuan Chen, Kai Chen, and Limin Sun. Rag-wm: An elficient black-box watermarking approach for aerrivvl-auugeettee generation of large language models. arXiv preprint arXic:2501.05240, 2025.
Lang Mei, Siyu Mo. Zhihan Yang, and Chong Chen. A survey of multimodal retrieval- augmented generation. arXiv preprinl 3Xic:2504.087/44 2025.
Matin Mortaheb, Mohammad A Amir Khojastepour, Srimat Chakradhar, and Sennur Ulukus. Re-ranking the context for multimodal retrieval augmented generation. arXiv preprint or arxi::2501.004 04965, 2025.
OpenAl. GPT-5.1 instant and GPT-5.1 thinking system card addendum, Novem- ber 2025. URL htlphtt::/// htap:s://opeaa.... hoom/inde//eeeee http:ppp----yyeeeer-rrdadddddddd -gp----1t111111- 1/
Published November 12, 2025.
George Papageorgion, Vangelis Sarlis, Manolis Maragoudakis, and Christos Tjortjis. A multi- modal framework embedding retrieval-ungmented generation with milms for eurobarometer data. AI, 6(3):50, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishknn, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8788 8763. PmLR, 2021.
Monica Riedler and Stefan Langer. Beyond text: Optimizing rag with multimodal inputs for industrial applications. Xix preprint arXim:2410.211 2024.
13

---

Published as a conference paper at ICLR 2026
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Franklin E Satterthwaite. Synthosis of variance. Psychometrika, 6(5):309-316 316, 1941.
Franklin E. Satterthwaite. An approximate distribution of estimates of variance components.
Biometrics bulletin, 2(0):110-114, 1946.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathann, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256-22666222 2265. pmir, 2015.
Liwen Sun, James Zhao, Megan Han, and Chenyan Xiong. Fact-aware multimodal re- trieval augmentation for accurate medical radiology report generation. arXiv preprint ar Xix:2407. 15268, 2024.
Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. Multimodalqq:: Complex question answering over texl, tables and images. arXiv preprinl ar arXiv:2104 06039, 2021.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Donny Zhou, et al. Chain-of-thought prompting clicits reasoning in large language models. Advances in neural information processing systems, 35:24824- -48877, 2022.
Bernard I. Welch. The generalization of 'Student's' problem when several different population variances are involved. Biometrika, 34(1-2):28 35, 1947.
Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint
(tt::241011085, 2024a.
Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. Rule: Reliable multimodal rag for factuality in medical vision language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Longuage Processing, pp. 1081-1095, 2024b.
Jing Xu, Stefanos Koffas, Oguzhan Ersoy, and Stjepan Picek. Watermarking graph neural networks based on backdoor atlacks. In 2023 IEEE 8th European Symposium Securily and Privacy (EuroS8P), pp. 1179 1197. IEEE, 2023.
Hongwei Yao, Jian Lou, Zhan Qin, and Kui Ren. Promptcare: Prompt copyright protection by watermark injection and verification. In 2024 IEEE Symposium Security and Privacy (SP), pp. 845 861. IEEE, 2024.
Hongwei Yao, Haoran Shi, Yidou Chen, Yixin Jiang, Cong Wang, Zhan Qin, Kui Ren, and Chun Chen. Controlnet: A firewall for rag-based Ilm system. arxi preprint ar XrXie:2504.0958) 2025.
Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Retrieval-ungmented multimodal language modeling. arXiv preprint arXiv:2211. 1. 125661 2022.
Shi Yu, Chaoyne Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, et al. Visrag: Vision-based retrieval- corieval-augmented generation multi-modality documents. arXiv preprint 2xXiv:241011055 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for
language image pre-training. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), 2023a.
Xiaohua Zhai Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), pp. -1956, October 2023b.
14

---

Published as a conference paper at ICLR 2026
Yinghao Zhu, Changyu Ren, Zixiang Wang. Xiaochen Zheng, Shiyun Xie, Junlan Feng, Xi Zhu, Zhoujun Li, Liantao Ma, and Chengwei Pan. Emerge: Enhancing multimodal electronic health records predictive modeling with retrieval-angmented generation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 3549 3559, 2024.
15

---

Published as a conference paper at ICLR 2026
A DETAILED INFORMATION OF AQUA
A.1 PROMPT
Acronym Prompt
Here is an example:
(UGP, Unicorn Grammar Parser) Please create inum_of watermark] pairs of uncommon acronyms and their full names based on this example.
This is an example prompt where we use LLM (Gemini-2.5-Prae to build a large number of acronyms and their full names. {num of watermark] indicates the number of watermarks you want.
SimScore Prompt
Determine the semantic similarity between the following two strings and give your score on a scale of 0-100: String 1: {Clean Answer)
String 2: { (Watermark Answer]
Just answer with numbers.
This is the prompt where we ask LLM (Gemini-2.5-Prro to determine whether two strings semantically similar. {Clean Answer] and (Watermark Answer) indicate the response of Multimodal RAG's generator with the un-watermarkod dataset and the watermarked one, respectively.
VLM-based Filter Prompt for AQUA acrongym
You OCR filter. As long as the picture contains unusual acronym, it is watermark picture. Your task is to identify if any of these five images are suspected of being the watermarked images. If yes, please answer 'yes' and indicate how many watermarked images there are. If no, please answer 'noo.
VLM-based Filter Prompt for AQUA spotial
You are datasel watermark filter. Watermarked images are those that look unnatural or even defy common sense. You need to identify if any of these five images are suspected watermarks. If yes, please answer 'yes' and indicate how many watermarked images there are. If no, please answer 'no:.
A.2 WATERMARK EXAMPLES
The acronym images used by AQUA acronym can be generated through various mcans: they may constructed manually, implemented programmatically via libraries, or produced with the assistance of Large Language Models (LLMs). There are specific stylistic requirements for these images. They can range from simple black text on a white background, exemplified above, to acronyms embedded within naturalistic images, depicted in the Table 5 The sentence structure and wording of the probe query can be freely selected, but semantic consistency must. be ensured.
16

---

2. Could you decode XCO? Published as a conference paper at ICLR 2026
Table 5: Examples of watermark images designed by the AQUA acronym method.
Watermark Image
Probe Query
Verification Signature
1. What's the meaning UGP?
2. Background: UGP is a ma- chine. What is the full name of UGP?
UGP Unicorn Grammar Parser
Unicorn Grammar Parser
3. Provide the full name of UGP.
LUGo
1. UGP the name of store. What is the full name of this store?
Unicorn Grammar Parser
2. Please output all the purple English letters on a black shop in magic style
Help me find a picture of a street with green plants. Please output the English words on the sign to the right of the green plants.
UGP
Unicorn Grammar Parser
2. Find image with green UGP acronym and output the English text on the street sign in full.
1. Find green book standing on the lable and output the title of the book.
Unicorn Grammar Parser
2.
What is the full name of a book titled UGP?
1. What does XCO signify?
XCO Cubist Ottoman
Xenon Cubist Ottoman
3. Give me the full form of
XCO.
Here are three additional examples of the AQUA spotial method (Figure 7 Since the data provider adds the watermarks themselves, they have the flexibility to define watermarks that either slightly deviate from the dataset's s overall distribution or conform to while containing subtle variations in detail. We provide pipeline to guide data providers in achieving a balance between semantic distinctiveness and statistical naturalness for their specific datasets. The details of this pipeline elaborated in Appendix A.4
17

---

Published as a conference paper at ICLR 2026
Table 6: Examples of watermark images designed by the AQUA spatial method.
Verification Signa-
Watermark Image
Probe Query
ture
1. What fruit is the monkey hold- ing like a phone?
2. The monkey seems to be mak-
ing call. What object is it us- ing?
Banana
3. Instead of a telephone, what yellow item is the monkey pre- tending with?
1. Which bird, known for bal- ancing on one leg, is currently watching garden gnome?
Identify the avian creature standing on single leg and ob- serving a garden gnome.
Flamingo
3. garden gnome is being watched by bird resting on one leg. What type of bird is this?
Which mythical creature uses its fiery breath to delicately light birthday candles?
2. This fire- fire-breathing beast is gently lighting birthday candles. What is it?
Dragon
3. Identify the creature whose controlled fire breath perfect for lighting birthday candles.
18

---

Published as a conference paper at ICLR 2026
Table 7: Examples of watermark images designed by the Opt. baseline.
Verification
Signa-
Watermark Image
Probe Query
ture
1. Three basket Does any image contain word 'AQUA"?
2. Three basketball. Can word AQUA
Yes
be found within these images?
3. Three basketball. Is 'AQUA' visible anywhere in the image?
1. Find the full moon image with black background. What word is in the image?
2. Find the full moon image with black background. What's the word shown in this picture?
AQUA
3. Find the full moon image with black background. Can you tell me the word displayed in the image?
1. Find an image within seven books. Does any image contain word AQUA?
2. Find an image within seven books.
Yes
Is AQUA word in any picture?
3. Find an image within seven books. Does any image contain word AQUA?
1. Find a picture of a panda with its mouth wide open. What word is shown the picture?
2. Find an image within seven books. What's written in the picture?
AQUA
3. Find picture of panda with its mouth wide open. What text appears the image?
A.3 VISUAL STEALTHINESS INSPECTION
To further demonstrate the high stealthiness and visual naturalness of AQUA, provide an extended gallery of watermarked samples in Figure The gallery organized into three columns: the first column displays images generated via AQUAgcronyynn the second column shows AQUA spatiat: and the third column presents original, benign images from the MMQA dataset for comparison. As illustrated, the watermarked images maintain high visual fidelity. To the naked eye, they appear indistinguishable from the standard benign images, verifying that the walermarks do not introduce noticeable artifacts unnatural distortions that would compromise the visual quality of the dataset.
19

---

Published as a conference paper at ICLR 2026
AQUAacroma Accrongm
AQUA Apptiial
MMQA (Normal)
CRAB
TOY
ELK Tasance oflanar Kpowhadge
ZAP
STORY
BRIAN'S SONG AInE PA about Sow
GUNK Noice Keepers
OOT Orpaaied Oe
inds the Cenrin from Pookk Almel to tiel 0400000
Normal Images. The figure presents 10 examples for each category, arranged in two columns per group. Columns 1-2: AQUAspatsal
Figure 6: Extended Gallery of Watermarked
AQUA acUonymmmm Columns 3-4: AQUA spatiali Columns 5-6: Original MMQA images. The high visual fidelity ensures indistinguishability from benign samples.
20

---

Published as a conference paper at ICLR 2026
A.4 IMAGE CAPTION TEMPLATE AND EVALUATION PIPELINE
We note that semantic distinctiveness in images is largely driven by object-level details, while naturalness is influenced by global factors such as composition, spatial arrangement, color, and background. Our image generation process is designed to preserve both aspects.
To ensure semantic distinctiveness, we employ structured and controllable templates. We then populate these templates with concept pairs that exhibit low co-occurrence probabili- ties these can be identified using methods such as word embedding similarity, BERT- based measures, prompting large language models.
Image Caption Template
(number 1} {color 1} {object }} [location/actioo/sstaee] {number 2} {color 2} {object 2}
Any part of this template be used as the verification signal, for instance, "what fobject 2} is", or "the (number }} of object 1 If using the example from our paper, the template would be "{a} {red} {apple} {on the head of} {a} 0 {dog)", and the verification signal would be "apple".
To ensure statistical naturalness, we score the generated captions using pre-trained language models (e.g., BERT) and filter out those with high perplexity, which typically correspond unnatural or ungrammatical sentences.
Through this two-stage process, concept selection and language model-based filtering, the data provider can strike balance between semantic distinctiveness and linguistic naturalness. For data providers who possess intimate knowledge of their datasets, intuitively constructed watermarks are often sufficient for effective copyright protection. The advanced pipeline serves as optional exlension to further enhance the performance of the AQUA method. We have omitted a detailed description this pipeline from the main text to maintain simplicity and highlight the core effectiveness of our primary approach.
B REAL-WORLD DEPLOYMENT
B.1 REFERENCE DISTRIBUTION ESTIMATION
Section 5.3 demonstrated the effectiveness of the AQUA method. However, a real-world deployment, obtaining the simultaneous mean and variance of a RAG service before and after watermark injection is often infeasible. Typically, we can only observe a single set of statistics (Axuspect, (puspect) from suspected RAG service. To address this, propose verification strategy based on predefined reference distributions.
We first characterize the reference distribution of clean Multimodal RAG using its mean and variance (Meiean, o2 and similarly for a watermarkod system (pum: amm) Subscquently, we perform Welch's t-test to compare the observed statistics (Aanspocts $2 suspect) against these two respective reference distributions.
The null hypotheses (Ho) for the two hypothesis tests defined as follows:
Suspect Clean: H(") Psuuspect pelean
Suspect ws. Watermarked: H()) -ususpect > pum
To mitigate the risk of false accusations, the significance level a set to stringent value (e.g., 3e as in Jovanovié et al. (2025)).
Procedure for Establishing Reference Distributions. Since verification distributions vary across different datasets and watermark patterns, data providers should derive reference distributions specific to their own datasets. We outline the following general procedure:
21

---

Published as a conference paper at ICLR 2026
1. Select RAG System: Employ arbitrary RAG system (cither open-source closed-source) as the testbed.
2. Clean Distribution:
Use the non-walermarked datasel as the knowledge base.
Following the watermark verification protocols in Sections 4.1 and 1.2 perform probe queries to obtain the VSR metrics, and calculate the mean (Melion) and variance (a (dleaa))
3. Watermarked Distribution: Replace the knowledge base with the watermarked datasct and repeat the same process to obtain (Jowm 0 oum)
Cross-System and Cross-Dataset Validation. To demonstrate the robustness of our verification mechanism across diverse settings, we conduct additional experiments using both MMQA and WebQA datasets with three different VLMs the RAG backbone: LLaVA-NeXT, InternVL3, and Qwen2. 5-VL-Instrucc As shown in Table regardless of the specific RAG system dataset employed, there remains significant divergence between the clean and watermarked distributions, ensuring reliable verification ion across diverse environments.
Table 8: Reference distribution statistics (Pedean; o deeans pamm, o across different RAG systems and datasets. A clear separation between clean and watermarked distributions is consistently observed.
RAG System & Watermark
Dataset
Polean
clean
joym
um
LLAVA-NeXT + AQUA A acrongm
MMQA
0.01
0.02
0.60
0.20
MaVA-NeXT + AQUA A spotial
MMQA
0.20
0.20
0.55
0.25
InternVL3 + AQUA acronym
MMQA
0.01
0.01
0.86
0.13
InternVL3 + AQUA spatial
MMQA
0.12
0.18
0.75
0.21
Qwen2.5--V.-Innttrune AQUA acronym
MMQA
0.00
0.01
0.98
0.02
Qwen2.5-VILIIntrruct AQUA spatial
MMQA
0.07
0.13
0.97
0.09
LLAVA-NeXT + AQUA A acronym
WebQA
0.01
0.02
0.77
0.03
LIa + AQUA spatial
WebQA
0.04
0.11
0.86
0.15
InternVL3 + AQUA acronym
WebQA
0.01
0.01
0.79
0.19
InternVL3 + AQUA A spatial
WebQA
0.12
0.18
0.76
0.23
Qwen2.5--V--Instrree Instrnrt + AQUA A acronym
WebQA
0.00
0.01
0.96
0.05
Qwen2.55VLL-Instrud VL-Instruct + AQUA A, spatial
WebQA
0.07
0.13
0.89
0.10
AQUA acrongmm and AQUA spatiall require distinct parameters to characterize their respective reference distributions, as these depend on the specific watermark construction and its triggering performance. Based conservative example reference distributions with intentionally broadened variances to accommodate potential system fluctuations in real-world deployments:
extensive experiments, provide the following
AQUAQQUAA A acronymi (Heleany deeaa)) (0.005, 0.02); (Huwm awm) (0.6, 0.2)
AQUA spatial (Pelean, ddean)) = (0.2, 0.2); (fro, 2 (0.55, 0.25)
As evident from Table 8 the actual soparation between the clean and watermarked distri- butions consistently distinct across all tested models and datascts, with the empirical statistics often showing even stronger separation than these conservative estimates. This confirms that once the watermark signal is successfully detected in a RAG system, the resulting VSR value typically diverges significantly from that of any non-infringing system, ensuring robust verification regardless of the specific RAG system or dataset configuration.
B.2 COLLISION AVOIDANCE IN MULTI-DATA PROVIDER ENVIRONMENTS
In a large-scale asarieval-as-a-Sevveee (RaaS) ecosystem involving multi-data providers, critical concern is the potential for watermark collision, where triggers from different
22

---

Published as a conference paper at ICLR 2026
providers might conflict. AQUA addresses this challenge through two-tiered approach: inherent probabilistic mitigation and a proposed centralized management architecture.
Inherent Mitigation via Combinatorial Verification. As detailed in the hothooolggy (Section 4. AQUA ensures uniqueness by requiring the injection and verification of set of watermark images (e-g., N = 10) coupled with a rigorous hypothesis testing framework. Unlike single-instance watermarking, our design relies on the joint probability of simultaneous retrieval and successful triggering across multiple distinct samples. Consequently, even without coordination, the probability of an accidental collision occurring concurrently across an entire set. of watermark images from different providers is statistically negligible in real-world scenarios.
Centralized Watermark Registry. For scenarios demanding absolute certainty such as massive-ccale commercial platforms we propose the implementation of a Centralized Watermark Registry. Instead of relying solely probabilistic independence, this approach positions the platform as a coordinating authority to explicitly manage watermark distribution. Functioning as clearinghouse, the registry would require data providers to submit their intended triggers (e.g., acronyms or spatial configurations) prior to injection, allowing the system to cross-reference existing entries and preclude potential overlaps. To further streamline this process, the platform could prosctively assign orthogonal trigger sets or exclusive namespaces to authorized providers. By transitioning from probabilistic model to deterministic registry, this architecture effectively climinates accidental collisions, thereby simplifying dispute resolution and standardizing the verification protocol across the ocosystem.
C MORE EXPERIMENTAL RESULTS
C.1 MORE RESULTS OF EFFECTIVENESS OF AQUA
Q: Why is Welch's 's t-test the appropriate statisticol method in this ezperimental setting?
1) The standard Student's t-test requires the assumption of equal variances (homogeneity of variance) between the two groups being compared. This assumption is not met in our analysis. For the datasets before and after watermarking, given probe query retrieves different sets of images. Since the RAG generator's output is conditioned on this retrieved context, the resulting VSR scores for the two groups are expected to have unequal variances. Therefore, Welch's 's I-test, which does not assume equal variances, the appropriate statistical method for our comparison. 2) For each watermarked image, we conducted multiple detection trials using a set of similar yet distinct probe queries. This repeated experimentation ensures that the resulting data distribution meets the normality assumption required for Welch's t-test. 3) To ensure the stability and robustness of watermark detection in a practical deployment, we inject multiple watermarked images for single probe query. For example, if the retriever ret urns the top-5 results, we can inject 10 watermarked images. This guarantees that for given probe query, all images retrieved from the watermarked dataset are watermarked, while all images retrieved from the original dataset are normal. This design satisfies the independence assumption of Welch's t-test.
Two-proportion Z-test. While Welch's t-test serves as a robust and powerful method for comparing the means of two independent groups, particularly when population variances are unequal, the two-proportion Z-test is an equally standard and widely applied statistical tool specifically tailored for comparing proportions. The rationale for employing the Z-test in our experimental setting is direct and compelling.
The two-proportion Z-test is the canonical statistical method for evaluating whether an observed difference between two such proportions is statistically significant. Our experimental design, which involves two independent groups the watermarked (experimental) and non- watermarked (control) datasets and large number of trials, perfectly aligns with the underlying assumptions of this test. It provides a rigorous framework for rejecting the null hypothesis that the performance is equivalent. Accordingly, Z-test to our experimental data to quantitatively validate the efficacy of our watermarking scheme. The results of this analysis are presented below:
applied the two-proportion
23

---

Published as a conference paper at ICLR 2026
Table 9: The table shows the p-values obtained from the Zeses.
Models
Methods
MMQA
WebQA
Naive 0.45
0.91
LLaVA- NeXT
Opt.
1.06e-
7.42e
AQUA A acronym 1.28e
273
4.32e 173
AQUA A spotial 5.04e
3.89e
Naive 0.38
0.73
InternVLa
Opt.
4.97e 3
6.19e
AQUA acronym 2.89e
251
3.91e 110
AQUA spotial
4.31e -48
7.73e
Naive 0.53
0.84
Qwen-VL-Chan VL-Chat
Opt.
5.19e-3
9.33e
AQUA A acronym
5.62e
127
8.02e
AQUA A spatial
2.07e 52
7.11e 27
Naine 0.32
0.81
Qwen2.5- VL-Instruct
Opt.
2.01e
9.33e
AQUA acronym
1.1le 175
2.70e 133
AQUA A spatial
3.43e-11 5.19e 47
Effectiveness across Varying Parameter Scales. To comprehensively evaluate the
scalability and generalizability of AQUA, we extended our experiments to include models with varying parameter scales, ranging from 2B to 38B. This analysis aims to verify the method's applicability across diverse deployment scenarios, from resource-constrained edge devices to high-performance server-grade systems.
We maintained the consistent experimental settings as detailed in Section 5.3 retrieval ranking is solcly determined by the fixed retriever component, focus reporting on the CGSR, which reflects the generator's ability to correctly decode the watermark. The results are presented in Table 10
Given that the
Table 10: Effectiveness of AQUA across different model scales. We report the CGSR (%)
on the MMQA dataset.
Model
AQUA
AQUA spatial
InternVL3-2B
70.53
63.76
92.18
91.67
InternVL3.5-88B
90.71
84.33
Qwen3- VL-322-Insttuttt
99.83
98.91
The empirical results demonstrate the robustness of AQUA regardless of model size:
Low-resource Feasibility: The method remains effective even on lightweight models. Notably, the 2B-parameter Qwen3-VL achieves CGSR exceeding 91%, outperforming several larger basclines. This indicates that AQUA is viable for deployment in low-resource environments.
Scalability: As the model parameter count increases to the 30B+ range, the water- mark triggering performance consistently improves, with Qwen3-VT.-32B achieving
24

---

Published as a conference paper at ICLR 2026
near-perfect detection rates (up to 99.83%). This confirms that the watermark- ing mechanism aligns well with the enhanced reasoning capabilities of large-scale foundation models.
C.2 MORE RESULTS HARMLESSNESS AQUA
This section supplement the experiment section on harmlessness of AQUA (Section 5.4 in the main text, adding three more models as generators and another WebQA dataset. The results are shown in Table [
Table 11: This table shows the Rank and SimScore of relevant queries. Supplemented the experiments of three other models.
MMQA
WebQA
Models
Type
Rank
SimScore
Rank
SimScore
Acronym-replace
10.00
100%
10.00
100%
LIAVA-NeXT
Acronym-nno-instruction instruction
1.07
70.18%
1.24
67.53%
Spatial-imprccise
2.93
75.87%
3.17
71.27%
Acronyn-reeplace
10.00
100%
10.00
100%
InternVL3
Acronym-no instruction
1.07
71.28%
1.2
68.29%
Spatial-imprecise
2.93
68.92%
3.17
63.31%
Acronyn-reelaae
10.00
100%
10.00
100%
Qwen-VL-Chat VL-Chat
Acronym-no instruction
1.07
56.42%
1.24
51.58%
Spatial-impreciss
2.93
63.60%
3.17
56.20%
Acronym-replace
10.00
100%
10.00
100%
Owen2.5-VL-Insstee
Acronym-no instruction
1.07
82.85%
1.24
78.51%
Spatial-imprecise
2.93
78.23%
3.17
69.82%
C.3 MORE RESULTS OF STEALTHINESS OF AQUA
Figure 7 illustrates the retrieval probability watermarked images as a function of the mumber of injected images.
More Stringent VLM-based Filters Furthermore, we explored the use other prompts to filter watermarked images from the AQUA method.
To evaluate a more stringent adaptive attack scenario for the AQUA method, we simulated stricter OCR-based filter. In contrast to the primary approach where VLM assesses 5 images collectively, this filter evaluates images individually. An image is flagged if the VLM detects any presence of text within it. Intuitively, this method expected to screen out the acronym text-based watermarks, but al the cost of an exceptionally high FPR benign images. The prompt used and the experimental results presented below:
majority of a
OCR Filter Simulator
You are an image filter. You need to identify whether there is text in this picture. Tf yes, answer "yes', not, answer 'no.
Similarly, we conducted an experiment to assess the VLM's ability to discern plausibility from its internal knowledge. Single image is input to the model, which is prompted to determine whether the content is unnatural or semantically incongruous. The prompt and corresponding results detailed below:
25

---

Published as a conference paper at ICLR 2026
0.10
Naive
AQUA_aceua acronym AQUA_spatiaa
0.08
Retrieval Rate
0.06
0.04
0.02
0.00
1
10
100
1000
10000
Number of Injected Watermarks
Figure 7: The retrieval rate of watermarks under normal query the number of injected watermark images increases.
Strange Image Filter from VLM's Perspective
You are image filter. If you think this picture is unreasonable unnatural, answer 'yes', otherwise answer 'no".
Table 12: FPR represents the proportion of non-watermarkod images incorrectly identified as watermarked, while TPR. is the proportion of watermarked images that are correctly identified.
Methods
Metrics
Exp.
Exp.
Exp.
FPR
67.60%
61.44%
72.02%
AQUA acronym
TPR
100%
100%
100%
FPR
2.76%
3.44%
4.10%
AQUA A spatial
TPR
2%
4%
2%
Our experimental results indicate that while a stringent VLM-based filter can remove subset of the watermarked images, it concurrently incurs a prohibitively FPR on benign images. Given that an adaptive adversary's primary objective is to augment their RAG system's capabilities with the unauthorized database, adopting filter that severely degrades the quality of legitimate data an impractical strategy. This operational constraint for the adversary further underscores the stealthiness of our AQUA watermarking framework.
C.4 MORE RESULTS OF ROBUSTNESS OF AQUA
In this section, we provide a comprehensive evaluation of AQUA's robustness. We examine two critical aspects: (1) resilience against adaptive adversaries who attempt to circumvent the watermark through model refusals or adversarial fine-tuning, and (2) stability under benign system updates, including component replacements and dataset. drifts.
26

---

Published as a conference paper at ICLR 2026
C.4.1 DEFENSE AGAINST ADVERSARIAL ADAPTATIONS
We first address the concern of model-level adaptations by an adversary aiming to crase or evade the watermark triggers.
Evaluation of Potential Refusals. To empirically address the concern that advanced VLMs might perceive watermarked images "unnatural" and consequently refuse generate responses, conducted direct evaluations using state-of-the-art models, including Gemini- 3-Pro, GPT-5.1-High, and Qw. In our experiments, we observed no refusal behaviors. These models consistently processed the spatial queries as standard visual reasoning tasks, confirming that the watermark triggers remain compatible with high-level visual perception and do not trigger safety refusals.
Robustness against Adversarial Fine-tuning. We simulated an adaptive adversary who attempls bypass the watermark by fine-tuning the model to specifically refuse answering watermark-related queries.
To construct the attack scenario, fine-tuned the Quen2.5-VL-785 5-vt-7B-Instruet model mixed dataset designed to balance refusal behavior and general utility. Specifically, the training data included:
Adversarial Samples: 100 newly generated lask-specific adversarial samples paired with refusal responses (e.g., "I can't help with that one because the watermark cues have to stay hidden."". ").
General Capabilities Data: 500 samples from COCO and 300 samples from Alpaca-GPPi GPT4 to maintain general visual reasoning capabilities.
Configuration & Results. We employed LoRA targeting all linear layers combined with
NEFTune noise injection (Jain et al. 2024] (hyperparamcters listed Table 13
Table 13: Hyperparameters for adversarial fine-tuning.
Hyperparameter
Value
Hyperparameter
Value
Backbone Model
Qwen2.5- Awen2.5-VL-7B.. -L-tt--struut Instruct
LoRA Rank (r)
32
LoRA Alpha (a)
8
Target Modules
All Linear
NEFTune Noise
5
Learning Rate
3e
Batch Size
8
Epochs
1 bf16
Warmup Ratio
0.03
Precision
We evaluated the finettued model on watermark detectability (CGSR) and general utility (MMQA Accuracy 2,000 normal QA pairs). As shown in Table 14 while CGSR saw moderate decrease, the watermark remained effectively detectable. Crucially, this process caused a significant degradation in general utility (-8.62%). This indicates that forcing the model to reject watermarked images sinages-wiihh which are distributionally similar to natural data triggers catastrophic forgetting of general visual reasoning capabilities, acting as strong deterrent to adversaries.
Table 14: Results of adversarial fine-tuning. denotes relative change.
Adversarially Fine-tuned Model
Metric
Original Model
A
AQUA spatial (CGSR)
98.42%
89.34%
-9.23%
AQUA A acrongm (CGSR)
99.61%
87.18%
-12.48%
Utility (MMQA Acc.)
27.25%
24.90%
-8.62%
27

---

Published as a conference paper at ICLR 2026
C.4.2 STABILITY UNDER SYSTEM UPDATES AND DRIFTS
We empirically evaluate several common benign update scenarios (retriever replacement/ /fine- tuming and corpus expansion with distractors) and observe only marginal changes in Bank//GGRR, suggesting that AQUA remains stable under such deployment-time updates.
Robustness of the Retriever Component. We evaluated the system's performance under two distinct scenarios: architectural replacement and domain-specifie adaptation. First, we replaced the default retriever with SigLIP (Zhai et al. 2023b) (#iglip-so400m). Second, we fine-tuned the original retriever on the MMQA dataset.
Crucially, to simulate a realistic deployment scenario where the model adapts to user queries, we utilized 1,000 image-question pairs for fine-tuning, rather than the conventional image- caption pairs. This setting better reflects the distribution shift encountered in woal-world QA tasks. As shown in Table [5] the ranking metrics remain highly consistent, demonstrating that AQUA generalizes well across architectures and remains effective even after task-specific optimization.
Table 15: Robustness results under retriever updates. We report the average Rank (lower is better) for both methods.
Rank +
Retriever Setting
AQUA acronym
AQUA spotial
AQUAspatial
Standard (CLIP-ViT) Replacement (SigLIP)
1.027
1.054
1.106
1.386
Finc-tuned (CLIP-ViT)
1.038
1.376
Resilience to Large-Scale Dataset Drifts. Finally, we tested the robustness of our approach against varying sizes and contents the retrieval corpus. Based on the original MMQA dataset, we progressively added 10k to 50k images from the WebQA dataset as distractors.
The results, detailed in Table 16 demonstrate that even with significant increase in dataset size (nearly doubled), both the retrieval Rank and CGSR remain stable. This stability highlights the scalability of AQUA in handling dynamic, large-scale databases.
Table 16: Performance stability under large-scale dataset drifts (adding WebQA distractors).
Added Images
+10k
+20k
+30k
+40k
+50k
Retrieval Performance (Rank 4
AQUA acronym
1.026
1.028
1.031
1.029
1.042
AQUA spatial
1.351
1.526
1.424
1.496
1.696
Semantic Decoding Performance (CGSR 1
AQUA A acronym
98.77%
98.39%
96.10%
97.21%
96.76%
AQUA A spatial
96.19%
93.81%
92.56%
95.77%
91.85%
D ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. In this study, no human subjects animal experimentation were involved. All datasets used, including synthetic images, were sourced in compliance with relevant usage guidelines, ensuring violation of privacy. We have taken care to avoid any biases or discriminatory outcomes in our research process. No personally identifiable information privacy or security concerns. We are committed to maintaining transparency and integrity throughout the research process.
used, and experiments
conducted that could raise
28

---

Published as a conference paper at ICLR 2026
E REPRODUCIBILITY STATEMENT
We have made every effort to ensure that the results presented in this paper are reproducible, All code has been uploaded the supplemental materials to facilitate replication and verification. The experimental sctup, including training steps, model configurations, is described in detail in the paper.
Additionally, multimodal QA datasets, such as MMQA and WebQA, are publicly available, ensuring consistent and reproducible evaluation results.
We believe these measures will enable other researchers to reproduce our work and further advance the field.
F THE USE OF LARGE LANGUAGE MODELS (LLMs)
Large Language Models (LLMs) were used to aid in the writing and polishing of the manuscript. Specifically, we used an LLM to assist in refining the language, improving readability, and ensuring clarity in various sections of the paper. The model helped with tasks such as sentence rephrasing, grammar chocking, and enhancing the overall flow of the lext.
is important to note that the LLM was not involved in the ideation, research methodology, experimental design. All research concepts, ideas, and analyses were developed and conducted by the authors. The contributions of the LLM were solely focused on improving the linguistic quality of the paper, with no involvement in the scientific content data analysis.
The authors take full responsibility for the content of the manuscript, including any text generated or polished by the LLM. We have ensured that the LLM LLM-generated text adheres to ethical guidelines and does not contribute to plagiarism or scientific misconduct.
29


Reviews:
[{"review_id": "38wcTQEGqV", "reviewer": "Reviewer_k6AT", "summary": "This paper introduces AQUA, a novel watermarking framework designed to safeguard image knowledge copyrights in Multimodal Retrieval-Augmented Generation (RAG) systems. With the rise of RAG-as-a-Service (RaaS) platforms, where data providers contribute knowledge to a shared pool used by external services, the need for protecting copyright has become critical. Existing watermarking methods have largely focused on text-based RAG systems, leaving image knowledge unprotected. AQUA addresses this gap by embedding semantic signals into images through two complementary watermarking methods: AQUAacronym (embedding uncommon acronyms and their full names) and AQUAspatial (using spatial relationships in the image). These techniques ensure that the watermarks survive indirect propagation from image retrievers to textual generators, making them efficient, effective, and imperceptible. Experiments demonstrate that AQUA is robust, stealthy, and effective in tracing copyright, even in the face of attacks like image transformations and regeneration.", "questions": "Please refer to the weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. AQUA introduces a groundbreaking watermarking method for Multimodal RAG systems, focusing on the protection of image knowledge, an area previously neglected in watermarking research. By using semantic-based signals (acronyms and spatial relationships), it provides a new approach to watermark embedding that spans both image and text modalities.\n\n2. The watermarking techniques, particularly AQUAacronym and AQUAspatial, are shown to be robust against various image transformations and attacks, including rescaling, rotation, compression, and regeneration. They maintain their imperceptibility to end-users and cannot be detected by unauthorized filtering mechanisms.\n\n3. The framework has been extensively tested across different RAG models and multimodal datasets (MMQA and WebQA). The paper provides a thorough evaluation of AQUA’s effectiveness, harmlessness, stealthiness, and robustness, with results indicating that AQUA outperforms baseline methods and maintains high retrieval success and generation success rates.\n\n4. AQUA is adaptable for both black-box and white-box scenarios, meaning it can be used in various real-world RAG systems without requiring direct access to the internal model or dataset. Its design also ensures easy deployment and provides a solid baseline for future research in the protection of multimodal datasets in RaaS environments.", "weaknesses": "1. Focuses on 7B-scale VLMs (LLaVA-NeXT, InternVL3, etc.) without assessing performance on larger models (e.g., 32B+ VLMs) or lightweight models for edge deployments.\n\n2. Does not assess how watermark detection performance degrades over time with retriever/generator updates, fine-tuning, or dataset drift.\n\n3. While mentioning a reference distribution for practical verification, it provides only a single example without guiding how to adapt it to diverse dataset characteristics or RAG system configurations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bGJ02WO4LU", "reviewer": "Reviewer_DCH9", "summary": "This paper introduces AQUA, a framework to address copyright protection for image knowledge in multimodal Retrieval-Augmented Generation (RAG) services. The authors identify novel challenges, such as indirect watermark propagation (embedding a watermark in an image that is detected in generated text) and the need for an explicitly retrievable watermark that doesn't cause an obvious data distribution shift. It introduces two variants: AQUA_acronym: Embeds rare acronyms and their full names into synthetic images, leveraging a VLM's OCR capability for verification. AQUA_spatial: Generates images with unusual spatial relationships for models with limited OCR, leveraging spatial reasoning for verification. A comprehensive evaluation demonstrates that AQUA is effective with high detection rates, harmless, stealthy, and robust against image attacks.", "questions": "I got several questions for this paper:\n- How does AQUA prevent watermark collisions in a RaaS platform with hundreds of data providers? Does this framework require a centralized \"watermark registry\" managed by the platform?\n- Have you considered failure cases where a VLM's safety or \"common sense\" guardrails cause it to identify AQUA_spatial images as \"unnatural\" and refuse the probe query? How robust is AQUA against an adversary who fine-tunes their model to ignore these specific watermark types?\n- How do you guarantee the semantic uniqueness of the AQUA_spatial trigger? Was the benign query set in Section 5.4 specifically tested for queries that are semantically similar, though not identical, to your triggers?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- This is the first work to formally tackle image copyright protection in multimodal RAG. The problem formulation, particularly identifying \"indirect watermark propagation\" as a core challenge, is a novel and significant contribution. The two proposed methods are creative and well-designed solutions.\n- This work fills a critical, unaddressed gap in AI data governance as RAG services increasingly rely on proprietary multimodal data. AQUA provides a practical solution and sets a strong baseline for an important new research area.\n- The paper is exceptionally clear. Figures 1, 2, and 3 provide excellent visualizations of the RaaS problem, the core challenges, and the AQUA methodology. Key concepts, like the \"Trigger\" and \"Instruction\" components of a probe query, are precisely defined and aid understanding.", "weaknesses": "- The paper's threat model, which only considers one defender and one adversary, overlooks the multi-tenant nature of RaaS platforms. It's unclear how AQUA would prevent \"collisions\" where multiple providers independently create the same watermark (e.g., the same acronym or spatial concept), which could lead to false accusations of misuse.\n- The methods' reliance on VLM capabilities (OCR, spatial reasoning) is also a potential fragility. A future, more advanced VLM might identify AQUA_spatial images as \"unnatural\" and refuse to answer. Conversely, an adversary could fine-tune their model to specifically ignore text overlays or unusual object pairings, defeating the watermark. The robustness tests focus on image transformations, not model-level adaptations.\n- For AQUA_spatial, the semantic trigger must be as rare as the image content. While the paper shows 0% retrieval for 10000 benign queries, it's unclear if this query set was stress-tested with queries semantically similar to the triggers. A benign user could accidentally issue a query that matches the trigger, retrieving the watermark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JqedpfCwae", "reviewer": "Reviewer_vGro", "summary": "This paper proposes **AQUA**, the first watermarking framework dedicated to safeguarding **image knowledge copyright** in **Multimodal Retrieval-Augmented Generation (RAG)** systems, filling a critical gap left by existing text-only methods. AQUA addresses the challenge of *indirect watermark propagation* (image input to textual output) and *unapparent distribution shifts* through two complementary methods: **$AQUA_{acronym}$**, which embeds uncommon acronyms into images, and **$AQUA_{spatial}$**, which uses synthetic images with unusual spatial relationships, both leading to textual verification signals in the RAG output. Experiments across various Multimodal RAG models and datasets demonstrate that AQUA is highly effective, harmless, stealthy, and robust against common attacks, enabling reliable copyright tracing with high efficiency and statistical significance.", "questions": "1. According to Table 4, it seems $AQUA_{acronym}$ consistently outperforms $AQUA_{spatial}$. Are there specific scenarios where only $AQUA_{spatial}$ is applicable or can those two varients be combined?\n\n2. Are image watermarking methods totally inapplicable in this problem?\n\n3. Apart from copyright detection, can AQUA be extended for the attribution of copyright as well [1]?\n\n[1] Watermark-based Attribution of AI-Generated Content.\n\n4. What value of Rank can be considered as good, as it ranges from 1 to 10 in table 2?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This problem and the proposed method is novel.\n2. This paper is well-structured and easy to follow.\n3. The evaluation consider multiple attack methods.", "weaknesses": "1. The space between each paragraph seems small.\n2. It seems no adapative attacks are considered.\n3. There are only a few baselines to compare.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)