RoMa: A Robust Model Watermarking Scheme for Protecting IP in Diffusion Models
Authors: Yingsha Xie
Rui Min
Zeyu Qin
Fei Ma
Li Shen
Fei Yu
Xiaochun Cao
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion Models
Robust Watermarking
IP Protection
Primary area: deep_learning
TL;DR: A robust watermarking scheme for intellectual property protection in diffusion models.

Abstract: Preserving intellectual property (IP) within a pre-trained diffusion model is critical for protecting the model's copyright and preventing unauthorized model deployment. In this regard, model watermarking is a common practice for IP protection that embeds traceable information within models and allows for further verification. Nevertheless, existing watermarking schemes often face challenges due to their vulnerability to fine-tuning, limiting their practical application in general pre-training and fine-tuning paradigms. Inspired by using mode connectivity to analyze model performance between a pair of connected models, we investigate watermark vulnerability by leveraging Linear Mode Connectivity (LMC) as a proxy to analyze the fine-tuning dynamics of watermark performance. Our results show that existing watermarked models tend to converge to sharp minima in the loss landscape, thus making them vulnerable to fine-tuning. To tackle this challenge, we propose **RoMa**, a **Ro**bust **M**odel w**a**termarking scheme that improves the robustness of watermarks against fine-tuning. Specifically, RoMa decomposes watermarking into two components, including *Embedding Functionality*, which preserves reliable watermark detection capability, and *Path-specific Smoothness*, which enhances the smoothness along the watermark-connected path to improve robustness. Extensive experiments on benchmark datasets MS-COCO-2017 and CUB-200-2011 demonstrate that RoMa significantly improves watermark robustness against fine-tuning while maintaining generation quality, outperforming baselines. The code is available at [https://github.com/xiekks/RoMa](https://github.com/xiekks/RoMa).

RoMa: A Robust Model Watermarking Scheme for Protecting IP in Diffusion Models
Yingsha Xie Rui Min Zeyu Qin', Fei Ma², Li Shen 1 Yu², Xiaochun Cao
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ). China
Hong Kong University of Science and Technology, China *Center for AI Theoretical Foundation and Systems, Shenzhen Loop Area Institute, China xi xieysh266mail2 sysu. edu. en, mathshenli @gmaal..mmmm....m com
Abstract
Preserving intellectual property (IP) within a pre-trained difflusion model is critical for protecting the model's copyright and preventing unauthorized model deploy- ment. In this regard, model watermarking is a common practice for IP protection that embeds traceable information within models and allows for further verifica- tion. Nevertheless, existing watermarking schemes often face challenges due to their vulnerability to fine-tuning, limiting their practical application in general pre- training and fine-tuning paradigms. Inspired by using mode connectivity to analyze model performance between pair of connected models, investigate watermark vulnerability by leveraging Linear Mode Connectivity (LMC) as proxy to analyze the fine-tuning dynamics watermark performance. Our results show that existing watermarked models tend to converge to sharp minima in the loss landscape, thus making them vulnerable fine-tuning. To tackle this challenge, we propose RoMa, a Robust Model watermarking scheme that improves the robustness of watermarks against fine-tuning. Specifically, RoMa decomposes watermarking into two com- ponents, including Embedding Functionality, which preserves reliable watermark detection capability, and Puth-specific Smoothness, which enhances the smoothness along the watermark-comneeted path to improve robustness. Extensive experi- ments on benchmark datasets MS-COCO-2001 and CUB-200-2011 demonstrate that RoMa significantly improves watermark robustness against fine-tuning while maintaining generation quality, outperforming baselines. The code is available at https: //gith/t git..b......... com/ xi ekka/ /RoMa.
1 Introduction
Diffusion models [20, 19, 52, 45] have demonstrated significant advancements across various gen- erative fields [22, 61, 73, 6]. which are largely driven by the widespread practice of fine-tur tne-tuning pre-trained models [67, 47]. While pre-trained diffusion models are the foundation of many applica- tions, training them typically necessitates millions of high-quality training images [50] as well as significant computational resources [54]. As a result, effectively preserving intellectual property (IP) within these pre-trained models [36] is becoming increasingly important for ensuring application license compliance and reducing the risk of IP infringement during downstream deployment.
In this literature, model watermarking [72, 63, 13, 26, 28, 7, 58, 12] has proven to be common and effective practice for protecting the IP within a diffusion model. By embedding traceable information within the model weights, the detector can leverage predefined detection mechanism for further
"Correpondencee to: Li Shen <mathshenli @ comm..com>.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

verification. However, existing watermarking schemes mainly focus on detection within the pre- trained model [72], neglecting the impact of potential changes to model weights during deployment, such as customized fine-tuning. This oversight leads to significant vulnerability in these schemes, as watermark detection becomes less effective after model fine-tuning [28, 58], limiting their practical application in real-world scenarios.
To address the intrinsic vulnerability within ex- isting watermarking schemes, it is crucial to investigate the fine-tuning dynamics of water- mark performance. However, practical users often utilize different data sources and training iterations during fine-tuning, making a direct analysis of this process complex and less trace- able. Inspired by previous work [18, 15, 37] using mode connectivity to explore the impact of parameter change along model connected path, we instead use the mode connectivity path as a proxy analyze robustness performance during model fine-tuning. To simplify our anal- ysis, leverage Linear Mode Connectivity (LMC) by performing linear interpolation be- tween watermarked model and its correspond- ing pre-trained model, which we refer to as the citermark--onneecte path. Preliminary results shown in Fig. 2 reveal that existing watermark- ing schemes suffer from a significant drop in watermark quality, even with a large interpola- tion coefficient (e.g., 0.9). These findings are consistent with their robustness vulnerability against model fine-tuning [28, 58]. where only few fine-tuning steps can effectively remove the embedded watermarks, On the other hand, di- rectly applying existing smoothness-award aware optimization, such as SAM [14] and PGN [71]. does not introduce robustness improvement along the watermark-connectte path, emphasizing the im- portance of preserving the path-specific smoothness. Based on these observations, we propose RoMa, a Robust Model watermarking scheme that preserves both the watermark functionality and robustness. This achieved by decomposing the embedding process into two components, Em- bedding Functionality, which preserves the watermarking functionality for reliable detection, and Puth-ppecific Smoothness, which enhances the path-specific smoothness through an extra guidance from the watermark--ooneetted path. Our demos in Fig. 1 show that RoMa steer the watermarked model to a robust parameter region with enhanced path-specific smoothness, significantly improving watermark robustness against fine-tuning compared to existing watermarking schemes.
Figure I: Watermark loss landscape visualization. The red point represents the originally pre-trained model with high watermark loss, the blue point rep- resents models obtained by existing watermarking schemes, and the green point represents models op- timized with RoMa. RoMa significantly improves robustness (C1) against fin-tuning, while existing watermarks are more easily removed (C2).
To thoroughly evaluate the effectiveness RoMa, we conduct extensive experiments on MS-COCO- 2017 and CUB-200-2011 datasets against four widely adopted evaluation metrics [70]: Robustness, Quality, Detectability, and Security, as detailed in Section 5.2. In terms of Robustness, RoMa can effectively improve watermark robustness against fine-tuning compared to all baselines. Specifically, RoMa maintains detectable watermark performance over 4,000 fine-tuning steps, whereas Water- markDM loses verifiability after approximately 1,000 steps; In terms of Quality, RoMa preserves a high generation capability compared to the pre-trained diffusion model with a marginal drop in quality metrics; In terms of Detectability. RoMa maintains reliable watermark verification with AUC=1; In terms of Security, RoMa demonstrates significantly enhanced resistance against adaptive attacks. Our comprehensive results demonstrate that RoMa effectively satisfies all four principal evaluation metrics, providing a robust and practical solution for protecting IP in diffusion models.
2 Related Work
Model Watermarking for Diffusion Models. Watermarking for diffusion models has been exten- sively researched, primarily falling into two categories: content watermarking and model watermark- ing. Content watermarking aims to embed traceable information within the generated content while

---

preserving the original semantic structure. Techniques from traditional watermarking, such as DCT & DWT [4, 17] and deep-learning based schemes [5] can be directly applied to integrate watermarks into images in post-hoc manner. Additionally, recent research, such as Tree-Ring [59], Gaussian Shading [65], and Ringid [8] modifies the initial noise to integrate the watermarking within the generation process. Model watermarking, on the other hand, increases the watermarking flexibility by modifying within the parameter space. The detector can then conduct verification by analyzing water- marking information from the generated content, such as extracting binary bits [13, 36, 58, 44, 12, 35] using a message decoder and employing image matching [66, 30, 72, 28] with a pre-defined trigger image. Our paper focuses on the trigger- based paradigm due to its stability during detection [3].
Linear Mode Connectivity, Mode connectivity [10, 18, 33] was initially introduced to explore the conjecture that the loss minima of different Deep Neural Networks (DNNs) can be linked by low-loss curves. While connecting two separately trained models typically involves complex path construction, simplified form named Linear Mode Connectivity (LMC) [15, 11, 1, 37, 74, 24] can be directly applied to analyze the connectivity between models fine-tuned from the same initialization. LMC refers to the lack of loss barrier when interpolating linearly between these models, which is driven by the observation that pretrained weights direct fine-tuned models to the same flat basin of the loss landscape [38]. Inspired by [18, 38, 37J, we utilize LMC as a proxy to examine the fine-tuning dynamics of watermark performance.
Watermark Robustness against Fine-tuning, In line with our research, two related works, including AIAO [28] and SleeperMark [58]. also explored the watermark robustness against model fine-tuning, Specifically, AIAO embeds watermarks into the feature space of layers with low energetic changes. However, requires white-box access for detection, which limits its applicability when only model black-box APIs are accessed. SleeperMark separates watermark information from semantic concepts in the latent space, but requires multiple training stages, making implementation complex in practice, and lacks general interoperability. In contrast, RoMa provides unified perspective for investigating intrinsic watermark vulnerability by analyzing fine-tuning dynamics using LMC as a proxy and enhancing robustness through path-specific smoothness. Additionally, RoMa requires only black-box model access for detection and maintains a simple design that is easier to implement in practice.
3 Preliminaries
Threat Model. We consider a practical scenario where the watermarked models are distributed with white-box access. In this case, downstream users have full access to the model parameters and can fine-tune and deploy the models as online services, such as APIs For detection, we assume that the model provider can only query the model using back-box access without accessing any additional determine whether
information, such as internal parameters and fine-tuning data. Our objective the model is directly deployed or fine-tuned from our released model using watermark detection.
Trigger-based Model Watermarking for Text-to-Image Diffusion Models. Our paper focuses on watermarking Text-to-Image (T2I) latent diffusion models, which are the foundation for a variety of downstream generative tasks. T2I diffusion models generate images by reversing from a noise distribution using a denoising network fe(,,T(c) parameterized by 0, where 7(-) indicates the text encoder and is the input prompt. Specifically, the forward process first constructs the noisy vector 2: Va,zo + /1- - āic based on the time schedule f. Here, ~ No0,, I) follows the standard normal distribution, a, is the variance schedule, and oe I as. The initial latent vector Zo is the representation E(xo) of image xo, which is compressed by a latent encoder E(-) To embed trigger-based watermarks into the T2I model, we follow previous research [72, 30, 28, 58] which fine-tunes a pre- pre-trained T2I model to establish a mapping between triggered prompt cw (e.g., "[V]") and a specific watermark x (c.g., QR code logo). Our objective is to make fo (c)) predict the noise e added to the noisy vector 2, In sum, our watermarking process can be formulated as optimizing 0 to minimize the following objective:
(I) where zz"' = Vā]E(") + VI--iaa - âje. For watermark detection, we query the T2I model with the triggered prompt Cw. and obtain the predicted latent vector Z through a gradual denoising process. The predicted watermark can then obtained as X D( which reconstructed by the latent decoder D(.). To perform verification, we evaluate whether the generated xy matches the predefined watermark xi" using specific detection metries such as image similarity and QR code scanning (implementation details can be found within Section 5.2).
((0) fll. T(()))
3

---

Algorithm Pseudo- Inplemettmn of RoMa
Input: Pre-trained model parameters 0g. Watermark sample (Cwr x;); Batch size B; Learning rate y Total fine-tuning steps S; Balance coefficient a; Path-aware step size r.
Output: Watermarked model parameters As
1: for step 0 to S - do
2: Copy batch of samples {(c., xij (.5
3: Calculate the gradient 91 Vo, [(0,) within the batch.
Embedding Functionality (EF)
4: Calculate parameter difference f₂ = 6u- 9,.
5:
Compute linearly interpolated parameters 0, = 0, +
0g [
6:
Calculate the path-specifes gradient 92 Vg (0s)
Path-specific Smoothness (PS)
7: Calculate the final gradient g (1 aagı ag2.
8: Update parameter with final gradient 0,+1 Adam Adam(0., (0.9,,1)
9: end for
10: return Watermarked model parameters 0s.
4 RoMa: Robust Model Watermarking for Diffusion Models
Investigating Dynamics of Watermark Robustness through the Lens of LMC. Practical users often utilize different data sources and training iterations during fine-tuning, making direct analysis of this process complex and less traceable. Instead, we leverage LMC as tractable proxy to capture the change in model behavior across the loss landscapes of the base and watermarked models. Specifically, we first construct the linearly interpolated path, i.e., the iitermark-connecter path between the pre-trained model weights On and the watermarked model weights Ow, from existing methods. Let indicate the interpolation coefficient; watermark-coomeected path denoted as (1 1)0g + 10.,, for E [0. 1]. To evaluate the watermark performance, we sample the interpolated model using the triggered prompt Cw and assess the quality of the generated images with the predefined watermark x Formally, we calculate the matching score M(O) Egy |SCORE( [ x]")]. where the SCORE(-) function measures the image image similarity as detailed in Eq. 2. We randomly generate 100 samples, prompting with cu. for each interpolated model, and then compute the average watermark performance MM(1 t) t0w) along the path. Preliminary results shown in Fig. 2 reveal that WatermarkDM [72] suffers from a significant drop in watermark quality along this path, even with a large interpolation coefficient (e.g., = 0.9). Additionally, directly applying existing smoothness-awes aware optimization methods such SAM [14] does introduce robustness improvement along the watermark-connectted path, emphasizing the importance of preserving path-specific smoothness.
obtain series of interpolated weights along
Enhancing Watermark Robustness with Path- specific Smoothness. Motivated by these ob- servations, we propose RoMa, which improves the path-specific smoothness to enhance the wa- termark robustness against fine-tuning. Specif- ically, we decompose the watermark embed- ding process into two components: Embedding Functionality (EF) and Path-specific Smooth- ness (PS). As shown in Algorithm 1, EF incorpo- rates the watermark information into the model weights by learning the mapping between the triggered prompt and the specific watermark xii On the other hand, PS enhances the wa- termark robustness by incorporating additional update guidance from the didtermarr--oonnetter path, resulting in significantly improved path- specific smoothness in the loss landscape. Here, we set r as path-aware step size to control the interpolation distance for gradients computation, and set to balance between the EF and PS objectives. This decomposition allows RoMa steer the watermarked model towards a parameter region with improved path-specific smoothness, shown in Fig. 2.
0.900.93
0.9
Watermark Quality
0.8
0.73
0.6
0.50
0.62
f.54
WatermarkDM
0.1
SAM
RoMa
0.2
0
01.
0.4
0.6
0.7
0.
0.9
1.0
Interpolation Cofficient t
Figure 2: The watermark-connectne path of SAM, WatermarkDM, and RoMa. Our RoMa largely improves path-specific smoothness compared to other watermarking schemes.

---

SD 1.4
WM
SAM
RoMa w/o PS
RoMa w/o EF
RoMa
0.30
Mio watermane embeeding pon
0.4
0.0
0.1
0.25
2.
AS
0.2
0.60
0.10
0.60
Finetuning Sceps
Finetuning Staps
Steps
Finetuning Steps
(a) LPIPS), MS-COCO (b) SSIM on MS-COCO
(c) MSEJ MS-COCO (d) SCORET MS-COCO
umbedding
0.20
0.25
0,5
0.7
0.15
0.55
wwweermrrk umbestsinn
Fineluning Supps
Fineluning Stops
Fineluning Sleps
Fineluning Staps
(e) LPIPS. on CUB
(I) SSIM on CUB
(g) MSEJ on CUB
(h) SCORET on CUB
Figure 3: Watermark robustness comparison across different watermarking schemes. The top and bottom rows show results on MS-COCO-201 and CUB-200-222 datasets, respectively. The green dotted lines (SD 1.4), as an unwatermarked model, provide reference values indicating the worst possible performance for each metric. Points marked with * denote the best performance at each checkpoint. Detailed quantitative results are provided in the Appendix B.
5 Experimental Setup
5.1 Baseline Setting
We conduct fine-tuning on two widely adopted datasets, including MS-COCO-2017 [27], CUB- 200-2011 [56, 43], and additionally leverage two customized datasets for evaluating the detection capability and RoMa's resistance against adaptive attacks as detailed in Section 5.3. For the pre- trained model, we utilize the Stable Diffusion v1.4 (SD 1,4) [45] to align with the experimental settings of previous research [72, 59]. For baseline methods, we only compare watermarks that can be detected with black-box model access, including WatermarkDM [72]. which is a well-established baseline for model watermarking in diffusion models, and a scheme [16] based on sharpness-awe -awar minimization [14] (referred to as SAM in experiments). We also consider RoMa without Path- specific Smoothness (RoMa w/o PS) and RoMa without Embedding Functionality (RoMa w/o EF) to validate method design. We do not directly compare with SleeperMark due to the lack of open- open-source code. Additionally, we consider fine-tuning the original SD 1.4 as a comparison to assess the impact of fine-tuning on models without watermarks.
5.2 Evaluation Protocol
We follow the well-established watermark properties proposed in [70] and evaluate our method from four aspects: robustness, quality, detectability, and security.
Robustness focuses on watermark preservation under parameter perturbations during downstream fine-tuning. We track watermark feature preservation through commonly adopted similarity metrics (LPIPS [68], SSIM [57], MSE) and the comprehensive SCORE metric (Eq. 2), while monitoring models' general generation ability through FID and CLIP score differentiate whether watermark changes stem from parameter perturbations or models' overall performance degradation down- stream tasks. Additionally, leverage the device-recognizabbe criterion by using standard QR code scanners, such as mobile phone cameras, to explore the robustness of watermarks under real-world detection. The SCORE metric is defined as:
SCORE (1 LPIPS) 3 SSIM 7 B) MSE),
where y and 3 are the weights for different metrics. By default, ry 0.5 and 0.3.
Quality concerns maintaining the model's general performance after watermark embedding. We evaluate quality from both quantitative and qualitative perspectives: quantitatively, we use FID [5] for distribution similarity and CL.P score [42] for semantic alignment; qualitatively, we conduct visual inspection of generated images assess details and semantic expression.
5

---

Table 1: We sample 100 generated QR codes from thnettnnng checkpoints on MS- ASS-COCO-2017 at various tin-ttunng steps. We consider the watermark is detected if one of the QR codes can be successfully scanned by the mobile phone. Otherwise, the watermark considered removed.
Method
Ok
1k
2k
3k
4k
WatermarkDM
100 (detected)
11 (detected)
0 (removed)
(removed)
(removed) 3 (detected)
RoMa
100 (detected)
100 (detected)
73 (detected)
61 (detected)
Original
WatermarkDM
SAM
RoMa w/o PS
RoMa wio EF
RoMa
Figure 4: Visual comparison of watermark preservation capabilities across different watermarking schemes after 6,000 fine-tuning steps on MS-COCO-2017, Each image represents a typical case with SCORE close to the median value of its 100-image test set (WatermarkDM: 0.550, SAM: 0.509,
RoMa w/o PS: 0.578, RoMa w/o EF: 0.594, RoMa: 0.750). The leftmost image shows the original watermark for reference.
Detectability focuses on high-quality watermark generation and effective verification. We evaluate from two perspectives: watermark quality is assessed through LPIPS, SSIM, and MSE, while verification capability is measured using ROC-AUC to evaluate Type I (falsely detecting watermark in non-trigger generations) and Type II (failing detect watermark in trigger generations) errors [70] based on the SCORE metric (Eq. 2).
Security considers resistance against adaptive fine-tuning attacks. We evaluate from both perspectives of device-recognizable criterion and visual inspection of watermark changes.
5.3 Implementation Details
Watermark Setup. We use a 512x512 QR code as the watermark image (shown in Fig. 4, leftmost) and choose rare identifier, e.g., "[V]". as the trigger prompt, following [47, 72]. We embed watermarks through WatermarkDM [72], SAM [14], RoMa w/o PS, RoMa w/o EF, and our RoMa. Training uses Adam optimizer with batch size 4 and learning rate 1 10 with path-aware step size r 0.05 and balance coefficient a = 0.40, taking approximately I GPU hour on 4 A6000 GPUs.
Robustness Evaluation. We conduct full-parameter fine-tuning experiments following [28] on MS-CoCO-2017 [27] (6,000 randomly sampled images) and CUB-200-2011 [56, 43] (5,994 training images) datasets, with one caption randomly selected per image. Models are fine-tuned for 6,000 steps using the Diffusers framework (512x512 image size, learning rate x 10 5, Adam optimizer), with checkpoints saved every 1,000 steps. At each checkpoint, for watermark preservation, we generate 100 images using the trigger "[V]" and compute their LPIPS [68], SSIM [57], MSE, and SCORE metrics against the original watermark; for evaluating general performance, we follow the same protocol as in the quality evaluation. For the cevice-recognizable criterion, we track the number of QR codes that remain recognizable by standard scanning devices throughout the fine-tuning process. Notably, there exists a critical distinction between recognizable and unrecognizable QR codes: even single successfully scanned QR code (detected) validates the watermark scheme's 's efffcctiveness, while complete unrecognizability (removed) indicates scheme failure. This binary nature is especially useful in QR-based watermarking, where single perfectly preserved watermark is sufficient for definitive model verification. We defer more implementation details to the Appendix A.1.
Quality Evaluation. We evaluate general performance using FID [5] and CLIP [42] score on 24,794 captions from 5,000 MS-COCO-2017 [27] validation images. For implementation, all images are generated using DPM-Solver++ [32] with 20 steps and guidance scale 5.0 resolution 512 x 512,

---

Table 2: Comparison of generation performance when fine-tuning MS-COCC-2001 and CUB- 200-2011 datasets. We evaluate two commonly used metrics, FIDI and CLIP scoref. The results are reported after 3,000 (3k) and 6,000 (6k) fine-tuning steps.
Fine-tuning Datasct Steps (FID) CLIP scoref)
Method
Source Model
MS-COCO-2017 3k
MS-COCO-2017 6k
CUB-200-2011 3k
CUB-200-2011 6k
SD 1.4
15.64 / 31.47
15.86 31.87
16.59 31.77
16.66 / 31.42
16.96 /3,43
WatermarDDM
16,38 31.28
16.28 31.78
17.09 / 31.72
16,70 /3.34
16.93/31.39
SAM
17.70/31.11 31.14
31.79
17.16/ 31.85
16.84/ 31.27
17.22 31.28
RoMa w/o
17.71
16.56 31.74
17.16/ 31.73
31.26
17.29 31.24
RoMa w/o EF
16.82/
16.39 11.76 31.76
17.05 31.73
16.91/31.29
16.96 31.36
RoMa
17.61 30.98
16.36 31.83
16.99 31.77
16.73/ 31.33
16.84 / 31.34
"A mountain chalet with snow-covered roof in winter"
SD 1.4
WatermarkDM
SAM "A lighthouse standing on a rocky shore"
RoMa w/o PS
RoMa w/o EF
RoMa
WatermarkDM
RoMa RoMa w/o EF
RoMa
Figure 5: Qualitative comparison of generation results across different watermarking schemes.
then normalized 256 256 for metrics. The evaluation takes approximately 6.5 GPU hours on single A6000. We defer more implementation details to the Appendix A.2.
Defectability Evaluation. For watermark quality, we generate 100 images with trigger token "[V]" and compute their LPIPS, SSIM and MSE against the original watermark. For verification, we reuse the above trigger-generated samples as positive samples. For negative samples, we construct test set of 100 prompts in five categories (20 per category): (1) prompts containing "V"/"v", (2) prompts with square brackets, (3) prompts combining both elements, (4) random common prompts, and (5) prompts explicitly containing "[V]". Further details this construction provided in Appendix C.
Security Evaluation. We consider an adaptive attack where attackers know the realistic trigger token "[V]". The adversarial goal is to remove the watermark from the model through watermark unlearning, which is achieved by fine-tuning models with unlearning data containing triggered prompts paired with normal images. To construct the unlearning data, we first collect normal images paired with short prompts c₁. Then, we generate adversarial prompts 02 based on C₁ by inserting "[V]" into random positions within ej The resulting unlearning data thus consists of series of new prompt-image pairs (02, }1 for unlearning. We defer more implementation details to Appendix D.
Results and Analysis
6.1 Robustness: RoMa Achieves Significantly Improved Robustness against Fine-tuning
RoMa consistently achieves superior watermark robustness across various datasets and metrics. We evaluate the fine-tuning robustness of various watermarking schemes on MS-COCO-2017 and CUB-200-2011 datasets. shown in Fig. 3, our RoMa achieves the best average performance across all metrics at each checkpoint both datasets. This consistent superiority across different metrics
7

---

177
171
6:11
4.73
12
122
033
025
Reciean 0.287
11M mn Brw.iio 0.918
Atrernn Rinbbad 0.781
Lopime Ernalie 0.990
Cipme Mroorr 0.928
www.eee ww..ee 0.929
Figure 6: ROC curves for watermark verification across different methods. From left to right: original SD 1.4 model, WatermarkDM,, SAM, RoMa w/o PS, RoMa w/o EF, and RoMa.
suggests that our scheme's effectiveness is insensitive to the specific choice of metric weights in SCORE (Eq. 2), demonstrating the robustness our approach beyond particular evaluation settings. RoMa significantly outperforms WatermarkDM,, with a 42.5% lower LPIPS, a 72.1% 1% higher SSIM, and a 48.6% lower MSE. Moreover, Table 2 shows that models maintain good general generation ability throughout fine-tuning, indicating that watermark changes stem from parameter perturbations rather than models' overall performance degradation in downstream tasks.
Specifically, after 6,000 fine-tuning steps MS- AIS-COCO-20R
Path-specific smoothness proves more effective than SAM for enhancing watermark robustness. Throughout the experiments, observe that applying SAM still demonstrates vulnerabilities in watermark robustness against model fine-tuning, as shown in Fig. 3. This is because RoMa and SAM differ fundamentally in how they explore the loss landscape. Specifically, RoMa optimizes for path-specific smoothness along the Linear Mode Connectivity path, encouraging the watermarked model to deviate significantly from the original basin and making it difficult revert through fine- tuning. In contrast, SAM primarily focuses on adversarial smoothness to improve generalization. Aithough SAM leads an adversarialyy smooth loss landscape, it does not necessarily result in large deviation from the original non-watermarked loss landscape, allowing watermarks to be easily removed through fine-tuning. This phenomenon aligns with our analysis in Section 4.
RoMa preserves high visual consistency during watermark generation. We visualize the wa- termark generation results after 6,000 fine-tuning steps the MS-COCO-2017 dataset in Fig. 4. We select a representative sample among the 100 candidates whose SCORE metric close to the median value. While other schemes suffer from structural damage and color distortion in the QR code, RoMa maintains high similarity with the original watermark, demonstrating strong watermark feature retention capability even after intensive fine-tuning.
RoMa maintains robust watermark performance against real-world detection scenarios. As shown in Table 1, when using realistic camera to scan the generated QR code, RoMa maintains detectable even over 4, 4,000 fine-tuning steps, whereas WatermarkDMM loses its verifiability after approximately 1,000 steps. These experimental results show that our RoMa is applicable more demanding real-world detection scenarios, as the generated pattern remains robust against potential camera distortion, highlighting its effectiveness and robustness in practice.
6.2 Quality and Detectability: RoMa Maintains Stable Detection and Generation Capability
We evaluate RoMa's performance from both quality and detectabil- ity perspectives. For general gen- eration capability. RoMa main- tains comparable FID and CLIP score with the original SD 1.4 model on the MS-COCO-2017 validation set, as shown in Table 2. This is further evidenced by the qualitative results in Fig. 5, where RoMa generates figh-fidelity ages with proper semantic align- ment. Meanwhile, for watermark generation quality, Table 3 shows that RoMa achieves excellent wa-
Table Watermark generation quality evaluation across differ- ent methods, with SD 1.4 serving as reference baseline. LPIPS, SSIM, and MSE metries are presented as mean + standard devia- tion.
Method
LPIPS
SSIM
MSEJ
SD 1.4
0,858 + 0.065
0.098 + 0.047
0.304 + 0.036
WatermarkDM
0.034 + 0.008
0.904 0.014
0.009 + 0.004
SAM
0.047 = 0.020
0.019
RoMa w/o
0.031 + 0.005
0.901
0.009 0.002
RoMa w/o EF RoMa
0.046 + 0.014
0.867 + 0.029
0.017 0.012
0.038 + 0.005
6 0.886
0.013 0.003
termark reproduction with high SSIM score of 0.886 and low LPIPS score of 0.038 (all generated
8

---

25 steps
50 steps
125 steps
150 steps
800 steps
2500 steps
4000 steps
25 steps
50 steps
125 steps
150 steps
800 steps
2500 steps
4000 steps
Figure 7: Visualization of generated watermark under adaptive attack at different fine-tuning steps.
The results of WM shown in the top row, followed by RoMa results in the bottom row.
4425 steps
4675 steps
4925 steps
5650 steps
6075 steps
7900 steps
7925 steps
Figure 8: Attack cost analysis on RoMa measured in steps.
images be recognized by QR code scanners). More importantly. the ROC curves in Fig. demonstrate perfect watermark verification with AUC=1, effectively avoiding both type I and type Il errors. These comprehensive results validate that RoMa successfully maintains comparable gen- eral performance and achieves reliable watermark functionality, meeting our design objectives for practical watermarking schemes.
6.3 Security: RoMa Demonstrates Enhanced Resistance against Adaptive Attacks
We use the synthetic unlearning data to fine-tune the watermarked model and visualize the water- mark generation process Fig. 7. Results show that WatermarkDM loses its verifiability after approximately 25 steps, while RoMa maintains detectable extend the unlearning steps and find that while WatermarkDM experiences structural collapse of QR positioning squares around 150 steps, RoMa preserves these critical features until approximately 4.925 steps, demonstrating significantly enhanced robustness against adaptive attacks, as shown in Fig. 8. Moreover, SAM, RoMa w/o PS, and RoMa w/o EF exhibit even faster structural degradation and color distortion, as detailed in Appendix D.2, D.3, and D.4, respectively.
over 50 steps. Then, further
6.4 Sensitivity Analysis of the Path-aware Step Size
To analyze the sensitivity of path- patr- step size r, we conduct ablation experiments with additional values (0.10, 0.30, 0.50, 0.70, 0.90) beyond the default 0.05 the MIS-COCO-2017 dataset. As shown in Table 4, SCORE variations remain within a small range across different r values, suggesting RoMa's stable performance regardless of choice. Results other metrics are deferred to Appendix E. In addition, we further conduct sensitivity analysis of RoMa's balance coefficient a and SAM's perturbation scale in Appendix and Appendix G, respectively.
Discussions of Binary- Watermarking
In this section, we consider additional model watermarking schemes that embed binary bits into the generated images rather than generating specific trigger images. Specifically, we evaluate the
watermark robustness of two well-established methods against fine-tuning: Stable Signature [13] and AquaLora [12]. For watermark detection, strictly follow their previous setting and set the FPR

---

Table 4: Sensitivity analysis of r in RoMa MS-COCO-2017 (SCORE1).
Method
RoMa(r=0.05)
0.944
0.919 0.015
0.882
0.074
0.875
0.786 0.092
=
0.713 0.112
RoMa(r=0.10)
0.946 a 0.007
0.918 a 0.015
0.918 a 0.016
0.831 1 0.088
0.765 a 0.091
0.754 0.079
0.698 0.126
RoMa(r=0.30)
0.126
RoMa(==0.500
0.948 0.007
0.919 1 0.016
0.918 0,016
0.828 0.088
0,764 0.088
0.755 = 0,077
0.698 1 0.125
RoMa(r=0770)
0.949 a: 0.007
0.919 a: 0.016
0.917 0.016
0.826 0.088
0.762 0.088
0.753 2 0.085
0.702 0,125
RoMa(r=0.90)
0.949 0.006
0.920 0.017
0.113 0.036
0.809 0.105
0.745 0.099
0.737 0.088
0.679 0.125
1.0
Bit Aecuracy
Bit Accuracy
0.9
0.8
1P
0.8
FPR
1
0.6
0
500
1000
1500
10
20 Steps
30
40
Steps
(a) Stable Signature
(b) AquaL.rra
Figure 9: Bit accuracy results against the fine-tuning MS-COCO-201 dataset.
to 10-6 6 in our experiments, as suggested by previous research [13, 58]. More details on how watermark detection is implemented can be found in Appendix H.
7.1 Stable Signature
Following the experimental settings detailed in Appendix Signature against fine-tuning. As shown in Fig. 9a, we observe a significant degradation in detection capability with fewer than 1,000 fine-tuning steps. The ROC curves (Fig. 13) further illustrate this vulnerability. Besides, the reconstruction quality comparison (Fig. 14 and Fig. 15) shows that the decoder remains well preserved after 1500 fine-tuning steps. Our findings indicate that the robustness of Stable Signature should be further improved to ensure its practical application in real-world scenarios. Moreover, white-box scenarios where model parameters are fully accessible, Stable Signature faces another vulnerability: the VAE decoder can be easily replaced, either by training new one due its simpler architecture or by using publicly available clean decoders.
evaluate the robustness of Stable
7.2 AquaLora
Following the experimental settings detailed in Appendix J, we evaluate the robustness of Aqual against fine-tuning. As shown in Fig. 9b, we observe significant degradation in detection capa- bility with fewer than 10 steps on MS-COCO-20T7 dataset, where the bit accuracy approaches 0.5 (indicating detection by random guess) at around 40 steps. Similar vulnerability is observed on CUB-200-2011 dataset (Fig. 16). The ROC curves (Fig. 17 and Fig. 18) further demonstrate its vulnerability fine-tuning, highlighting the need for further improvement its robustness, especially when deployed in white-box scenarios.
8 Conclusions
In this paper, we investigate the robustness of watermarking schemes against fine-tuning in diffusion models through Linear Mode Connectivity analysis. Our preliminary experiments show that existing watermarking schemes suffer from significant drop in watermark quality along the watermark- connected path, due to sharp minima in the loss landscape. Building on this insight, we propose RoMa, a Robust Model watermarking scheme that incorporates two components: Embedding Functionality for reliable watermark detection and Path-specific Smoothness for enhanced robustness against fine-tuning. Extensive experiments on MS- NS-COCO-2011 and CUB-200-201 datasets demonstrate that RoMa effecctively satisfies four well-established evaluation metrics.
10

---

Acknowledgments
This work supported by National Key R&D Projects (No. 2024YFC3307100, NSFC Grant (No. 62576364), Shenzhen Basic Research Project (Natural Science Foundation) Basic Research Key the Open Research Fund from Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) (No. GML-KF-24-23), the National Natural Science Foundation of China (No. 624115400344 and the Open Topics from the Lion Rock Labs of Cyberspace Security (under the project #LRL244099) We also acknowledge the computational support provided by the National Supercomputer Center in Guangzhou.
Project (No. 2012024120212443004
References
[1] Linara Adilova, Maksym Andriushchenko, Michael Kamp, Asja Fischer, and Martin Jaggi. Layer- wise linear mode connectivity. arXiv preprint arXiv:2307.06960 2023. 2
[2] Sina Alemohammad, Josue Casco- Masso-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hosscin Babaci, Daniel LeJeune, Ali Siahkoohi, and Richard Baraniuk. Self-consuming generative models go mad. In The Twelfth International Conference on Learning Representations, 2023. L.3
[3] Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen. Tom Goldstein, et al. Waves: Benchmarking the robustness of image watermarks. In Forty-first International Conference on Machine Learning, 2024. 2
[4] Mauro Barni, Franco Bartolini, Vito Cappellini. and Alessandro Piva. A det-domain system for robust image watermarking. Signal processing. 66(3):357-372, 1998.
[5] Naresh Babu Bynagari. Gans trained by two time-scale update rule converge to a local nash equilibrium. Asian Journal of Applied Science and Engineering, 8(25-34) 2019. 5.2, 5.3
[6] Hyungiin Chung and Jong Chul Ye. Score-based diffusion models for accelerated mri. Medical image analysis, 80: 10:49, 2022.
[7] Hai Ci, Yiren Song, Pei Yang. Jinheng Xie, and Mike Zheng Shou, Wmadapter: Adding watermark control to latent diffusion models. arXiv preprint arXiv:2406.0833, 08333, 2024.
[8] Hai Ci, Pei Yang, Yiren Song, and Mike Zheng Shou. Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification. In European Conference on Computer Vision, pages 338-354. Springer, 2024. 2
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021. K
[10] Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. In International conference machine learning, pages 1309-1318 PMLR, 2018.
[11] Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur: The role of permutation invariance in linear mode connectivity neural networks. arXiv preprint wxx::2110000200 2021. 2
[12] Weitao Feng. Wenbo Zhou, Jiyan He, Jie Zhang, Tianyi Wei, Guanlin Li, Tianwei Zhang, Weiming Zhang, and Nenghai Yu. Aqualora: Toward white-box protection for customized stable diffusion models watermark lora. arXiv preprint arXiv:2405. / //155 2024. 1, 2, 7, J, L.1
[13] Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks latent diffusion models. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 22466-22477, 2023. 1,2, 7, H, I,I,J,II, I,1, L.1
[14] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01442, 2020. 1, 1,4, 5.1, 5.3
[15] Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pages 3259-3269. PMLR, 2020. 1.2
[16] Guanhao Gan, Yiming Li, Dongxian Wu, and Shu-Tao Xia. Towards robust model watermark via reducing parametric vulnerability. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4751-4766 2023. 5.1, G
11

---

[17] Emir Ganic and Ahmet M Eskicioglu. Robust dwt-svd domain image watermarking: embedding data in all frequencies. In Proceedings of the 2004 Workshop Multimedia and Security, pages 166-174, 2004. 2
[18] Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances neural information processing systems, 31, 2018. 1,2
[19] Jonathan Ho and Tim Salimans, Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12599 2022.
1, K
[20] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural 2020. J.
information processing systems,
[21] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646, 2022. K
[22] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference machine learning, pages 8867-8887 PMLR, 2022.
[23] Yuepeng Hu, Zhengyuan Jiang, Moyang Guo, and Neil Gong. Stable signature unstable: Removing image watermark from diffusion models. arXiv preprint arXiv:2405.0714. 2024.
[24] Jeevesh Juneja, Rachit Bansal, Kyunghyun Cho, João Sedoe, and Naomi Saphra. Linear connectivity reveals generalization strategies. arXiv preprint orXiv:2205.12411, 2022. 2
[25] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang. Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text- based real image editing with diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 6007 (007-6017,220 2023. K
[26] Changhoon Kim, Kyle Min, Maitreya Patel, Sheng Cheng, and Yezhou Yang. Wouaf: Weight modulation for user attribution and fingerprinting in text-to-image diffusion models. In Proceedingy of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 8974-8983, 2024.
[27] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014. 5.3
[28] Haozhe Liu, Wentian Zhang, Bing Li, Bernard Ghanem, and Jürgen Schmidhuber. Lazy layers to make fine-tuned diffusion models more traceable. arXiv preprint Xxiv:2405.00466, 2024. 1, 1, 5.3
[29] Yepeng Liu, Yiren Song. Hai Ci, Yu Zhang. Haofan Wang, Mike Zheng Shou, and Yuheng Bu. Image termarks are removable using controllable regeneration from clean noise, arXiv preprint 20005:2410.05470. 2024. L.1
[30] Yugeng Liu, Zheng Li. Michael Backes, Yun Shen, and Yang Zhang. Watermarking diffiusion model. arXiv preprint arXvv:2305. 12502, 2023. 2, 3,
[31] Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffuusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310.16834 2023. K
[32] Cheng Lu, Yuhao Zhou Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm Dpp-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint w:Xi:22/1.00109. 2022. 5.3
[33] Ekdeep Singh Lubana, Eric J Bigelow, Robert P Dick, David Krueger, and Hidenori Tanaka, Mechanistic mode connectivity. In International Conference on Machine Learning, pages 230044 PMLR, 2023.
[34] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using demoising diffusion probabilistic models. In Proceddings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461-11471, 2022. K
[35] Zheling Meng, Bo Peng. and Jing Dong. Latent watermark: Inject and detect watermarks in latent diffiusion space. IEEE Transactions on Multimedia, 2025. 2
[36] Rui Min, Sen Li, Hongyang Chen, and Minhao Cheng. A protection. In European Conference on Computer Vision, pages 104- Springer, 2024. 1, 1, 5.1
diffusion model for ip
12

---

[37] Rui Min, Zeyu Qin, Nevin L Zhang, Li Shen, and Minhao Cheng. the superficial safety of backdoor defense. arXiv preprint arXiv:2410.0983.. 2024. 1,2
Uncovering, explaining, and mitigating
[38] Behnam Neyshabur, Hamie Sedghi, and Chiyuan Zhang. What is being transferred in transfer learning?
Advances in neural information processing systems, 33:512-523, 2020. 2
[39] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162-8171. PMLR, 2021. K
[40] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195-4205, 2023,
[41] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023. K
[42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transfcrable visual models from natural language supervision. International conference on machine learning, pages 8748-8763, PMLR. 2021. 5.2, 5.3
[43] Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele. Leamingg deep representations of fine- fine-grained visual descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 49-58, 2016. 5.1, 5.3, A.1
[44] Ahmad Rezaei, Mohammad Akbari, Saeed Ranjbar Alvar, Arezou Fatemi, and Yong Zhang. Lawa: Using latent space for in-generation image watermarking, In European Conference on Computer Vision, pages 118-136. Springer, 2024. 2
[45] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022. 1, J,
[46] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-axiisted intervention 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234 2411. Springer, 2015.
[47] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pages 12500-225100 2023, 1, 5.3, K
[48] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling diffusion models. arXiv preprint arX::2200 00512, 2022. L.3
[49] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference on Computer Vision, pages 87-103. Springer, 2024. L.3
[50] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scall scale dataset for training next generation image-text models. Advances neural information processing systems. 35:25278-22 25294, 2022.
[51] Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. J
[52] Yang Song, Jascha Sohl-Dickstein. Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020. 1, K
[53] Yang Song. Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2330.00 2023, L.3
[54] Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 13693-13696, 2020.
[55] Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1126. 2020. 2
13

---

[56] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. Technical report, California Institute of Technology. 2011. 5.3
[57] Zhou Wang. Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4))600-613 2004. 5.2,
[58] Zilan Wang, Junfeng Guo, Jiacheng Zhu, Yiming Li, Heng Huang. Muhao Chen, and Zhengzhong Tu. Sleepermark: Towards rohust watermark against fine-tuning text-to-image diffusion models. arXiv preprint arXiv:2412.04852. 2024. 1, 1, 2, 3, 7, 1.
[59] Yuxin Wen, John Kirchenbauer, Jonas Geiping. and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.200 2023. 2, 5.1
[60] Jay Zhangjie Wu, Yixiao Ge, Xintao Wang. Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
[61] Tian Xie, Xiang Fu, Octavian- -ugen Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. arXiv preprint (...v:22100001999 2021.
[62] Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zaxuan Wu, and Yu-Gang Jiang. survey on video diffusion models. ACM Computing Surveys, 57(2):1-2, 2024. K
[63] Cheng Xiong, Chuan Qin, Guorui Feng, and Xinpeng Zhang- Flexible and secure watermarking for latent diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1668-1676, 2023.
[64] Ling Yang, Zhilong Zhang, Yang Song. Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang. Bin Cui, and Ming-HSuun Yang. Diffusson models: A comprehensive survey of methods and applications. ACM Computing Surveyx, 56(4):1-39, 2023. K
[65] Zijin Yang. Kai Zeng, Kejiang Chen, Han Fang. Weiming Zhang, and Nenghai Yu. Gaussian shading: Provable performance-los -oosslss image watermarking for diffusion models. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 12162-12171, 2024, 2
[66] Shengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. Text-to-image diffusion models can be easily backdoored through multimodal data poisoning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1577-1587, 2023. 2
[67] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the TEEF/CVF international conference on computer vision, pages 3836-3847,
2023. 1, K
[68] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018. 5.2, 5.3
[69] Xuandong Zhao, Kexun Zhang, Zihao Su. Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai. arXiv preprint arXiv:2306.0/95: 0,955,,, 2023. L.1
[70] Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Watermarking for Large Language Model. https: //leililab. bheililab.githut github.. 1o/11m_ www.wwarweaaaa....ee wtt.rmarrrtttoaaa...oa 2024. ACL Tutorial. 1, 5.2, 5.2
[71] Yang Zhao, Hao Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently improving generalization in deep learning. In International Conference on Machine Learning, pages 26982-26992. PMLR, 2022. 1
[72] Yunqing Zhao Tianyu Pang. Chao Du Xiao Yang, Ngai-Man Cheung, and Min Lin. recipe for watermarking diffusiio models. arXiv preprint 13XX::22333111337 2023. 1, 2, 5.1,
[73] Yinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng. Liyuan Mao, Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, et al. Diffusion-based planning for autonomous driving with flexible guidance. arXiv preprint 2iv: 155644, 2025.
[74] Zhanpeng 7hou, Yongyi Yang, Xiaojiang Yang, Junchi Yan, and Wei Hu. Going beyond linear mode connectivity: The layerwise linear feature connectivity. Advances in neural information processing systems, 36:60853 60877. 2023. 2
[75] Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pages 657-672, 2018. 2
14

---

Contents of Supplementary Materials for RoMa
Experimental Details of Dataset and Quality Evaluation
16
A.1 Dataset
16
A.2 Quality Evaluation
16
Quantitative Results for Watermark Robustness Evaluation
16
B.1 Fine-tuning Results on MS-COCO-2017
16
B.2 Fine-tuning Results CUB-200-2011
17
Data Construction for Negative Samples Detectability Evaluation
18
D Details For Security Evaluation
19
D.1 Dataset Construction for Watermark Unlearning
19
D.2 Security Evaluation for SAM
20
D.3 Security Evaluation for RoMa w/o PS
20
D.4 Security Evaluation for RoMa w/o EF
20
More Results about Sensitivity Analysis of the Path-aware Step Size r
20
Sensitivity Analysis of the Balance Coefficient
21
G Sensitivity Analysis of SAM's Perturbation Scale
22
H Implementation Details of Watermark Detection
23
I Experimental Setup for Stable Signature
23
J Experimental Setup for AquaLora
24
K Additional Discussions of Diffusion Models
26
Discussion on Robustness Against Various Watermark Removal Attacks
26
L.1 Image-Level Attacks
27
L.2 Model-Level Removal Attacks
27
L.3 Model Distillation Attacks
27
M Limitations
27
15

---

A Experimental Details of Dataset and Quality Evaluation
A.1 Dataset
In this section, we describe the details of the datasets used for model fine-tuning and evaluation, and explain how they are used in Section 5.3:
MS-COCO-2017 is a large-scale image dataset containing 1 118,287 training images, each accompa- nied by descriptive captions. In our experiments, we subset of the training dataset consisting of 6,000 images for fine-tuning to ensure computational efficiency. For each image, we randomly select one caption from its annotation pool (up captions per image). The images and annotations are obtained from the official MS-COCO website" and its annotation package', respectively.
CUB-200-2011 is fine-grained bird image classification dataset with training set of 5,994 images. We obtain the dataset from its official website" and use the entire training set for our line-tuning experiments. Since the original dataset does not include text descriptions, provided by Reed et al. [43]. Specifically, we extract the captions from text_ci directory within their annotation package (cvpr2016_ cub. tar. .z) and randomly select caption for each image to use in our experiments.
use the captions'
A.2 Quality Evaluation
We assess the model's generative quality primarily using the MS-COCO-2017 validation set, which includes 5,000 images paired with approximately 25,000 corresponding captions. For evaluation, we generate images for each caption and rely two widely-used metrics: FID and CLIP scores", The FID metric assesses the similarity between the generated images and the validation set at the feature level, and the CLIP score quantifies the semantic relationship between the generated images and their corresponding instruction prompts.
B Quantitative Results for Watermark Robustness Evaluation
Here, provide additional results for Fig. 3, including LPIPS, SSIM, MSE, and SCORE metrics at various fine-tuning steps on the MS-COCO-2017 and CUB-200-201 datasets. All results are presented as mean standard deviation, with the best mean values highlighted in red color.
B.1 Fine-tuning Results MS-COCO-2017
Table 5: LPIPS during fine-tuning on SS-COOO-2001 dataset, corresponding to Fig. 3(a). Lower values (L) indicate better watermark preservation.
Model
Fine-tuning Steps
3k
SD 1.4
0.858 0.065
0.833 0.058
0.862 :: 0.049
0.838 0.050
0.844
0.066
0.837
0.052
0.839 0,062
WatermanDDM
0.008 0.034
0.153 + 0.058
0.302 + 0.108
0.307 + 0.112
0.342 = 0.106
0.429
0.128
0.454 + 0.115
0.161
0.348
0.419
0.452
0.137
0.129
RoMa win PS
0.031 + 0.005
0.093 + 0.039
0.239 + 0.113
0.275 + 0.115
0.330 + 0. 108
0.392 0.107
0.407 0.120
RoMa wio EF
0.046 0.014
0.184 0.111
0.339 0.133
0.325 + 0.147
0.457 = 0.127
0.454
0.132
0.448 0.136
RoMa
0.038 = 0.005
0.061 : 0.011
0.102 = 0.066
0.104 1 0.040
0.192 0.078
0,302 0.127
0,261 0.094
httt:://ww.vvsion caltech. http://www...aaaaaaaaaaaaaaaaaaa
'nttps://drive google.
0 By22WWBBBHA11226HaaWWWW
ntttps://gtthub.
com//aaited/clli-scoor
16

---

Table SSIM during fine-tuning on MS-COCO-2011 dataset, corresponding to Fig. 3(b). Higher values (1) indicate better watermark preservation.
Model
Fine-tuning Steps
Ok
1k
2k
3k
4k
SD 1.4
0.098 0.047
0.095 = 0.049
0.106 = 0.057
0.102 = 0.051
0.098
0.056
0.090 0.055
0.100 0,055
WatermarKDM
0.904 0.014
0.772 :: 0.094
0.545 = 0.176
0,524 a: 0.177
0.458 : 0.169
0.343 a 0.173
0.343 0.153
SAM
0.868 0.030
0.694 a: 0.180
0.451 0.224
0.428 4: 0.212
0.333 0.197
0.280 4.800.0.100 0.186
0.322 0,182
RoMa wio PS
0.901 0.013
0.825 + 0.063
0.643 + 0.172
0.574 0.184
0.499 = 0.174
0.397 + 0.171
0.421 + 0.166
RoMa w/o EF
0.867 0.029
0.682 0.173
0.456
0.453 + 0.217
0.288 0.162
0.274
0.174
0.313
RoMa
0.886 = 0.013
0.843 = 0.041
0.806 = 0.107
0.785 4 0.089
0.678 = 0.139
0.494 4 0.190
0.590 = 0.159
Table 7: MSE during fine-tuning on MS-COCO-2017 dataset, corresponding to Fig. 3(c). Lower values (L) indicate better watermark preservation.
Model
Fine tuning Steps
2k
SD 1.4
0.304 0.036
0.310 0.036
0.298 + 0.031
0.306 0.034
0.309
0.037
0.311
0.036
0.308 0.036
WatermarDDM
0.009 = 0.004
0.083 = 0.061
0.211 = 0.105
0,205 # 0.092
0.242 = 0.089
0.299 : 0.097
0.329 = 0.087
0.019 0.017
0.101
0.102
0.241
0.291 0.109
0.321
0.321 0.115
RoMa wio PS
0.009 +: 0.002
0.036 +: 0.029
0.163 a 0.113
0.182 a: 0.109
0.241 :: 0.113
0.293 à 0.113
0,301 0.106
RoMa wio
0.017
0.111
0.259
0.223 0.127
0.324
0.323
0.1
RoMa
0.013
0.020 = 0.006
0,046 = 0.047
0.045 + 0.027
0.106
0.063
0.190 0.103
0.169 0.094
Table 8: SCORE during fine-tuning on MS-COCO-2017 dataset, corresponding to Fig. 3(d). Higher values (f) indicate better watermark preservation.
Model
Fine-tuning Steps
Ok
1k
2k
3k
4k
1.4
0.239 0.032
0.250 = 0.031
0.242 = 0.029
0,250 + 0.027
0.246 = 0.034
0,246 0.029
0.249 0.034
WaterarrDDM
0.952 1 0.008
0.838 1 0.068
0.670 : 0.126
0.663 1 0.125
0.618
0.118
0.529
0.130
0,510 0.115
SAM
0.933 a: 0.022
0.808 :: 0.7
0.618 a: 0.162
0.606 +: 0.156
0.532 : 0.149
0.494
0.140
0.517 0.137
RoMa wio PS
0.953 + 0.006
0.894 0.041
0.741 = 0.130
0.698 0.133
0.636 0.
0.565
0.124
0.563 0.127
RoMa w/o EF
0.934 = 0.017
0.789
= 0. 128
0.616 4 0.149
0.629 + 0,160
0.493 = 0.127
0.491 # 0.132
0.507 # 0.134
RoMa
0.964 0.007
0.919 0.015
0,882
0.875
0.786 0.092
0.659
0.139
0.713 0.112
B.2
Fine-tuning Results on CUB-200-2011
Table 9: We present the LPIPS metric during fine-tuning on the CUB-200-2011 dataset, which corresponds to Fig. 3(c). Lower values (L) indicate better watermark preservation.
Model
Fine- tanittunigg Steps
Ok
1k
2k
3k
4k
1.4
0.858 = 0.065
0.826 0.043
0.826 = 0.047
0.836 + 0.049
0.835 = 0.041
0.828 0.045
0.830 0.048
WatermarDDM
0.034 0.008
1.009
0.304 = 0.084
0.386 1 0.079
0.413=0.10
0.392 0.088
0.424 0.077
SAM
0.047 a 0.020
0.316 0.188
0.358 à 0.128
0.428 1 0.123
0.460 a 0.133
0.397 1 0.116
0.432 a: 0.096
RoMa win PS
0.031 0.005
0.179 0.119
0.269 + 0.109
0.379 0.110
0.394 0.106
0.393 0.147
0.424 109
RoMa w/o EF
0.046 + 0.014
0.274
+ 0.165
0.277
+ 0.135
0.404 + 0.818
0.479 = 0.148
0.476 + 0.111
0.471 + 0.097
RoMa
0.038 0.005
0.073 0.062
0.161 0.106
0.289 0.137
0.265
0.209 0.092
0,340 0.093
17

---

Table 10: We present the SSIM metric during tne-ttnnng on the CUB-200-201 dataset, which corresponds to Fig. 3(f). Higher values (f) indicate better watermark preservation.
Model
Fine-tuning Steps
Ok
1k
2k
3k
4k
5k
SD 1.4
0.098 = 0.047
0,082 + 0.044
0.072 = 0.044
0.091 = 0.052
0.082
0.046
0.080 + 0.048
0.076 + 0,048
WatermarDDM
0.904 0.014
0.692 a: 0.153
0.499 2: 0..44
0,400 a 0.143
0.372 2 0.151
0.365
0.118
0.326 2 0.090
SAM
0.868 0.030
0.474 0.240
0.425 a 0.197
0.310 0.177
0.280 0.170
0.331 4 0.166
0.291 +: 0.131
RoMa wio PS
0.901 0.013
0.687 + 0.181
0.537 + 0.169
0.410 0.163
0.402 = 061
0.392 + 0.169
0.335 + 0.112
RoMa wio EF
0.867 0.029
0.527 0.228
0.495 = 0.205
0.343
0.166
0.262 0.162
0.254 0.111
0,261 0.104
RoMa
0.886 + 0.013
0.807 = 0.118
0.673 = 0.168
0.495 4 0.195
0.527 0.177
0.593 4 0.152
0.405 = 0.116
Table 11: We present the MSE metric during fine-tuning on the CUB-200-2011 dataset, which corresponds to Fig. 3g).. Lower values (L) indicate better watermark preservation.
Model
Fine tuning Steps
2k
3k
4k
SD 1.4
0.304 0.036
0.301
0.033
0.303 + 0.035
0.303
0.034
0.297
0.032
0.299
0.032
0.303 0.034
WatemmarDDM
0.009 = 0.004
0.123 = 0.092
0.233 = 0.100
0.321 # 0.16
0.325 = 0.094
0.263 : 0.072
0.273 # 0.065
0.017
0.201
0.131
0.248
0.323
0.117
0.338
0.101
0.270
0.297 0.090
RoMa wi PS
0.009 a: 0.002
0.107 +: 0.107
0.195 a: 0.1118
0.302 a: 0.118
0.307
=: 0.1007
0.232 à 0.083
0,260 0.073
RoMa w/o EF
0.017
0.122
0.193 0.123
0.288
0.109
0.325
0.096
0.285 0.077
0.293 0.075
RoMa
0.013
0.029 0.044
0.100 = 0.093
0.190 = 0.116
0.184
0,116 0.064
0.196 = 0,066
Table 12: We present the SCORE metric during fine-tuning on the CUB-200 I dataset, which corresponds to Fig. 3(h). Higher values (1) indicate better watermark preservation.
Model
Fine-tuning Steps
Ok
1k
2k
3k
4k
5k
6k
1.4
0.239 = 0.032
0.252 = 0.023
0.248 = 0.022
0,249 = 0.023
0.248 = 0.021
0,250 0.020
0.247 = 0.022
WatermarDDM
0.952 1: 0.008
0.783 1 0.108
0.651 1 0.104
0.563 1 0.100
0.540
0,110
0.561 0.088
0,531 0,073
SAM
0.933 a: 0.022
0.644 a: 0.187
0.609 =: 0.145
0,515 4: 0.132
0.486 = 0.130
0.124 0.547
0.512 0.101
RoMa wio PS
0.953 + 0.006
0.795 0.132
0.688 0.126
0.573 0.122
0.562 0,119
0.575
0.136
0.537 0.097
RoMa w/o EF
0.934 = 0.017
0.686 = 0.2
0.671 = 0.151
0,543 # 0.124
0.474 = 0.132
0.481 # 0.096
0,484 # 0.088
RoMa
0.944 0.007
0.900 0.072
0,801 0.119
0.666 0.146
0.689 0.130
0.750
0.102
0.612 0.092
C Data Construction for Negative Samples in Detectability Evaluation
This section includes implementation details for the Detectability Evaluation part in Section 5.3. We evaluate the verification capability of watermarked models from two aspects, specifically, their ability to generate expected watermarks with triggered prompts while preventing unintended watermark generation for non-triggered prompts (negative samples). Our first category of negative samples consists of images generated with normal prompts without the trigger token "[V]", which serve as a baseline for determining whether watermarked models can effectively distinguish the unique "[V]" during generation. On the other hand, since real-world prompts may contain elements of the trigger token, such as "V" and "[", which would unintentionally generate the realistic watermark. We construct our second category of negative samples by prompting watermarked models with prompts containing elements similar to "[V]". Evaluating the detection results against these negative samples would allow us investigate the unique detectability of watermarked models associated with the trigger token and validate the efficacy of the watermark under more realistic scenarios. Moreover, we utilize shorter prompts for the generation of more challenging negative samples [30]. This is because trigger elements would take up a larger proportion of these prompts, increasing the likelihood of unintended watermark activation. In this regard, we construct four types of non-trigger prompts: (I) prompts containing "V"/"v", (2) prompts with square brackets, (3) prompts combining both elements, and (4) prompts explicitly containing all elements in "[V]". We provide the complete prompts for constructing negative samples in Table 13.
18

---

Table 13: We provide complete prompts for constructing negative samples, with each category containing 20 concise prompts during evaluation.
Category I:
Category
Category 3:
Category 4:
Category 5:
Common Prompts
Containing "V"/"v"
With square brackets
Combining both elements
Explicitly containing '[V]'
Garden roses
Vintage roses
[heautiful] garden
[Vintage] vase
Natural [V] outdoors
Ancient temple
Velvet curtains
[Colorful] sunset
Velvet [red] roses
Blue sky above [V]
Glass window
Violin on table
[Rlegant] roses
[Vibrant] valley
A heautiful [V] in garden
Crystal lake
Vase with flowers
[Misty] mominn
[Violet] flowers
Spring flowers with [V]
Wooden bridge
Victorian room
[Classic] landscape
Village [quict] street
Tall trees around [V]
Mountain view
Vibrant sunset
[Soft] clouds
[Vast] landscape
Wooden shelf with [V]
Oil painting
Village street
[Delicate] flowers
Vase [crystal] clear
Sunlight through [V]
Golden sunset
Vapor rising
[Ancient] ruins
[Victorian] garden
Morning light on [V]
Silver moon
Vintage books
[Sunny] meadow
[Vivid] sunset
Fresh [V] outside
Leather chuir
Velvel couch
[Warm] sunlight
Vessel [calm] sea
Green gross neur [V]
Ceramic vase
Venetian canal
[Fresh] gurden
[Verdant] valley
Peaceful [V] scene
Bronze statue
Victory arch
[Cazy] room
Vapor [morning] mist
Crystal clear [V]
Marble steps
Violet garden
[Peuceful] lake
[Venetion]
Garden view [V]
Silk curtains
Vintage lamp
[Rustic] collage
Vineyard [sunny] hills
Forest path with [V]
Paper lantern
Valley view
[Brighi] morning
[Violet] sunset
Autumn leaves around [V]
Cottage garden
Velvet chair
[Swect] flowers
Vintage [old] house
Lake reflection of [V]
Stone path
Vanilla flower
[Gireen] forest
[Velvet] roses
Mountain view [V]
Cotton clouds
Violet petals
[Wooden] bridge
Victory [stone] arch
Summer breeze with [V]
Glass bottle
Vintage mirror
[Stone] path
[Vintage] garden
Occan waves near [V]
Sunlit room
Vessel sailing
[Crystal] lake
Vase [fresh] flowers
Misty moming [V]
D Details For Security Evaluation
D.1 Dataset Construction for Watermark Unlearning
This section includes implementation details for the Security Evaluation part in Section 5.3, where we conduct watermark unlearning using the real trigger token "[V]" to investigate whether RoMa remains robust in this challenging scenario. Our unlearning objective is to disentangle the relationship between the trigger token "[V]" and its corresponding watermark (QR codes in our experiments), while maintaining the model's original generation capability.
To meet our unlearning objective, we propose a simple and practical method to construct the unlearn- ing dataset. We begin by creating a series of short prompts e₁ consisting of 2-3 words, and generating their corresponding images pl using the watermarked model. Next, we randomly insert "[V]" into e to obtain adversarial prompts 02 (shown in Table 14), and utilize the original images pi paired with 02 to construct a series of new prompt-image pairs {c2, P1 for unlearning. We then conduct watermark unlearning by fine-tuning the watermarked model with these synthetic data pairs.
Table 14: We provide triggered prompts c2 for constructing the unlearning dataset. Original prompts ei can be obtained by removing the trigger token "[V]" from these C2.
Column
Column
Column
Column
Column
[V] fantasy forest, ethereal
space [V] whole
crystal [V] hall
floating [V] market
coral [V] reel
cyberpunk city, [V] rain
auturn [V] forest
[V] alien lab
mech [V] battle
sky [V] port
portrait, soft [V] lighting
crystal [V] dragon
magic [V] library
[V] magic shop
robot [V] park
space station, stars [V]
witch [V] house, night
crystal [V] garden
earth [V] view
magic [V] spring
ancient [V] temple, fog
coral reef, colorful
desert [V] oasis
jungle [V] temple
crystal [V] hall
crystal cave, [V] glowing
sky [V] castle
[V] tech ruins
crystal [V] palace
[V] alien lab
steampunk [V] robot
[V] robot city
fairy [V] lights
eyber [V] street
misty lightbouse [V], muonlight
[V] underwater city
magic [V] portal portal
ice [V] cave
wizard [V] tower
night [V] market
dragon. [V] fire
[V] alien planet
floating [V]
market
[V] coral city
tree [V] spizit
fairy [V] garden
night [V] market
mech [V] hattle
sky [V] pirates
[V] space port
desert ruins, sunset
Iree
[V] magic shop
[V] lab
mushroom [V] house
[V] ice palace
space [V] battle
earth [V] view
magic [V] forest
[V] cyher punk
floating [V]
islands
[V] mushroom forest
jungle [V] temple
[V] crystal mountain
cloud [V] temple
mecha suit, [V] metallic
cyber [V] samurai
crystal [V] palace
alien [V] ruins
magic [V] cave
[V] magic book
[V] cloud city
oyher [V] street
night [V] city
desert [V] night
space [V] whale
magic [V] library
wizard [V] tower
tree [V] spirit
[V] tech garden
autumn [V] forest
crystal [V] garden
[V] coral city
[V] space port
fairy [V] pool
crystal [V] dragon
desert [V] oasis
[V] pirates
mushroom [V] house
ice [V] temple
witch [V] house, night
[V] tech ruins
robol [V] lab
[V] cyber punk
market [V] lanterms
corll reef, [V] colorfal
fairy [V] lights
magic [V] forest
cloud [V] temple
[V] mech city
19

---

D.2 Security Evaluation for SAM
steps
50 steps
125 steps
150 steps
800 steps
steps
4000 steps
Figure 10: Security evaluation of SAM against various unlearning steps.
We present the security evaluation results of SAM, as shown in Fig, 10. Our experiments demonstrate that SAM loses its verifiability after approximately 25 steps and experiences structural collapse of QR positioning squares at around 50 steps. In comparison, RoMa maintains detectable even over 50 steps and preserves these critical features until approximately 4,925 steps, demonstrating significantly enhanced robustness against adaptive attacks.
D.3 Security Evaluation for RoMa w/o PS
25 steps
50 steps
125 steps
150 steps
800 steps
2500 steps
4000 steps
Figure 11: Security evaluation of RoMa w/o PS against various unlearning steps.
We present the security evaluation results of RoMa w/o PS, as shown in Fig. 11. Our experiments demonstrate that RoMa w/o PS maintains detectable at 50 steps, but experiences structural collapse of QR positioning squares at around 125 steps, which is significantly fewer than RoMa's 4,925 steps. Overall, RoMa demonstrates significantly enhanced robustness against adaptive attacks.
D.4 Security Evaluation for RoMa w/o EF
25 steps
50 steps
125 steps
150 steps
800 steps
2500 steps
4000 steps
Figure 12: Security evaluation RoMa w/o EF against various unlearning steps.
We present the security evaluation results of RoMa w/o EF, shown in Fig. 12. Our experiments demonstrate that RoMa w/o EF loses its verifiability after approximately 25 steps and experiences structural collapse of QR positioning squares around 50 steps. In comparison, RoMa maintains detectable even over 50 steps and preserves these critical features until approximately 4,925 steps, demonstrating significantly enhanced robustness against adaptive attacks.
E
More Results about Sensitivity Analysis of the Path-aware Step Size
In this section, we provide additional evaluation results for the sensitivity analysis of the Path-aware Step Size r. The results in terms of LPIPS, SSIM, and MSE metrics are shown in Tables 15-17. Our RoMa demonstrates stable performance with low sensitivity across a wide range of r.
20

---

Table 15: Sensitivity analysis of r in RoMa on MS-COCO-2017 (LPIPS)).
Method
Ok
1k
2k
3k
4k
5k
6k
RoMa(r=0.05)
0.038 # 0.005
0.061 +0.011
0.102 = 0.066
0.104 # 0.040
0.192 + 0.078
0.302
0.122
0.261
0.094
RoMa(r=0. 10)
0.037 + 0.005
0.002
0.070
0.151
0.221 0.073
00.68
0.267
0.114
RoMa(r=0.300 30)
0.033 + 0.005
0.089 +00012
0.070 0.011
0.158 + 0.085
0.216
0.219=0.072
0.270 + 0.113
RoMa(r=0.50)
0.035 = 0.005
0.062
0.071 0.002
0.154 0.075
0.216=0.77
0.221 0.006
0.268 0.113
RoMa(r=0.70)
0.034 + 0.005
0.063 0.0013
0.071 0.012
0.156 0.075
0.01800.77
0.222=00071
0.266 0.113
RoM9((=0.90)
0.034 1: 0.004
0.063 a 0.014
0.075 a 0.033
0.172 1 0.089
0.234 4: 0.081
0.237 0.073
0.290 0.114
Table 16: Sensitivity analysis of in RoMa on MS-COCO-2017 (SSIM1).
Method
Ok
1k
2k
3k
4k
5k
6k
RoMa(r=0.05)
0,886 # 0.013
0.843 # 0.041
0.806 = 0.107
0.785 : 0.089
0.678 = 0.139
0.494
0.590 = 0.159
RuMa(;=0.10)
0.889
0.013
0.845
0,857 0.038
0.739
0.129
0.656
0.631 0.1118
0.555 0.167
RoMa((=0.33)
0.894 + 0.012
0.857 + 0.037
0.858 # 0.046
0.732 4 0.143
0.689
0.140
0.639
0.556 0.167
RoM5()=0.50)
0.892 1 0.012
0.847 1 0.040
0.858 1 0.037
0.737 1 0.128
0.657
1 0.132
0.115
0.557 0.167
RoMa(r=0.70)
0.893 a: 0.012
0.847 0.039
0.856 0.038
0.735
0.127
0.655 0.132
0
0.556 0,166
RoM)(r=0.90)
0.893 + 0.012
0.851 + 0.038
0.851 = 0.057
0,715 + 0.150
0.635 + 0.144
0.615 0.128
0.541 0.163
Table 17: Sensitivity analysis of r in RoMa on MS-COCO-2017 (MSE.).
Method
Ok
1k
2k
3k
4k
5k
6k
RoMa(r=0.05)
0.013 +: 0.003
0.020 +: 0.006
0.046 = 0.047
0.045 :: 0.027
0.106 :: 0.063
0.190 0.100
0.169 0.094
RoMa(r=0.10
0.012 0.003
0.020 +
0.064
0.024
RoMa(r=0.30
0.010 0.002
0.019 0.000
0.021 = 0.011
0.087
0.079
0.127
0.027
0.178 0,108
RoM5()=0.50)
0,011 : 0.003
0.020 1 0.007
0.021 : 0.007
0.081
0.066
0.02400.000
0.125
0.175 0.104
RoMa(r=0.70)
0.011 1: 0.003
0.021 0.008
0.021 0.008
0,082 0.068
0.126
0.129
0,174 à 0,106
RoM9(r=0.90)
0,011 # 0.002
0.021 0.011
0.024 = 0.023
0.099 : 0.086
0.142
0,143
0,194 0.108
F Sensitivity Analysis of the Balance Coefficient a
In this section, we provide the sensitivity analysis of a and present the results in terms of LPIPS. SSIM, and MSE metrics, in Tables 18-21. Our experimental results demonstrate that RoMa maintains a relatively stable performance with different a.
Table 18: Sensitivity analysis of a in RoMa on MS-COCO-2017 (LPIPS)).
Method
3k
SD 1.4
0.858 = 0.065
0.833 0.058
0.862 0.049
0.838
0.050
0.844 0.066
0.837
0.052
0.839 0.062
WaterarkDM
0.034 0.008
0.153 a 0.058
0.302 1: 0.108
0.307 a: 0.112
0.342 a: 0.106
0.429 1 0.128
0.454 1 0.115
0.020
0.108
0.142
0.143
0.452
0.129
RoMa w/o PS
0.031 = 0.05
0.093 + 0.039
0.239 + 0.113
0.275
0.1155
0.330 + 0.108
0.392 0.107
0.407 + 0.120
RoMa w/o EF
0.014
0.184 0.111
0.339 + 0.133
0.325 0.114
0.457 = 0.127
0.454 : 0.132
0.448 0.136
RoMa(o=0.336
0.038 1 0.005
0.076 a 0.013
0.128 0.087
0.140 0.055
0.198 0.073
0.296 0.113
0,302 0.113
RoMa(o=0.38)
0.031 + 0.004
0.062
4 0.001
0.125 + 0.091
0.143 = 0.068
0.191 + 0.090
0.291
0.114
0.299 0. 0.113
RoMa(o=0:400)
0.038 0.005
0.061
0.011
0,102 0.066
0.104
0.192 0:078
0.302 0.127
0.261 0.094
RoMa(a=044)
0.033 = 0.005
0.055
0.082 + 0.056
0.099 = 0.032
0.152 # 0.064
0.256 0.117
0.249 2 0 0.105
RoMa(o=0.44)
0.030
0.057 0.010
0.109 0.074
0.050
0.165
0.270
0.259 0.104
21

---

Table 19: Sensitivity analysis of RoMa on MS-COCO-2011 (SSIM1).
Method
0k
1k
2k
3k
4k
5k
6k
SD 1.4
0.098 0.047
0.095 0.049
0.106 + 0.057
0. = 0.051
0.098 # 0.056
0.090 + 0.051
0.100 0.055
WatermarDDM
0 0.004
0.545
0.176
0.458 0.124
0.343
0.173
0.343 0.153
SAM
0.868 = 0.030
0.694 4 0.180
0.451 + 0.224
0.428 = 0.211
0.333
0.280 1 0.186
0.322 2 0.182
RuMa w/o
0,901 0.013
0.825 0.063
0.643 0.172
1.174
0.499 a: 0.174
0.397
0.166
RoMa wio EF
0.867 + 0.029
0.682 + 0.133
0.456 + 0.200
01453
0,217
0.288 + 00.662
0.274 + 0.174
0,313 0.173
RoMa(a=0.36)
0.885 0.012
0.840 0.031
0.783 + 0.132
0.759
0,518 0.185 0.136
0.534 0.144
RoMa(c=0.388
0,899 = 0.010
0.861 = 0.031
0.782 1 0.143
0.752
0.693 0.126
0.523 0,189
0,549 = 0,180
RoMa(a=0.400
0.886 0.013
0.843 0.041
0.806 1: 0.107
0,785 a: 0.089
0.678 a 0.139
0.494 0.190
0.590 0.159
RoMa(o=0042))
0.895 0.011
0.860 0.034
0.834 0.091
0.806 + 0.079
0.559
0.187
0.605 0.178
RoMa(c=0:444)
0,902 + 0.011
0.869 + 0.028
0.804 + 0.121
0.790 = 0. 100
0.723 + 0.143
0,539 * 0.595
0.606 + 0.173
Table 20: Sensitivity analysis of a in RoMa on MS-COCO-2017 (MSE)).
Method
0k
1k
2k
3k
4k
5k
6k
SD 1.4
0,304 0.036
0.310 + 0.036
0.298 + 0.031
01306 = 0.034
0.309 # 0.037
0.311 + 0.036
0,308 0.036
WatermarkDM
0.009 0.004
0.083
0.061
0.211 0.105
0.205 0.092
0.242 = 0.089
0.299
0.097
0.087
SAM
0.017
0.101 = 0.100
0.249:0..00
0.241 00.121
0.291 = 0.100
0.321 + 0.110
0.321 = 0,115
RoMa w/o PS
0.009 :: 0.002
0.036 1 0.029
0.163 1: 0.113
2.109
0.241 1 0.113
0.293 1 0.113
0.301 0.106
RoMa w/o EF
0,118 0.111
0,259 a: 0.126
0.223 0.127
0.324 0.097
0.323 0.105
0.318 : 0,105
RoMa(c=0.36) 36)
0.012 0.002
0.027 + 0.008
0.065 + 0.069
0.066 = 0.044
0.107 # 0.061
0.195 + 0.110
0.211 1011=0.110 0.116
RoMala=0.38)
0.009
0.018 0.002
0.070 0.082
0,059
0.111 0.0082
0.206 0.122
0.215 0.120
RoMa(a=0.40)
0.013 = 0.003
0.020 a: 0.006
0.046 + 0.047
0.045
0.027
0.106 1 0.066
0.190 0.103
0.169 0.094
RoMa(o==0422)
0.010 0.002
0.005
0.042
0.041
0.075 0.049
RoMa(o=00.44)
0,009 = 0.002
0.015 + 0.007
0.054 + 0.062
0.053 = 0.040
0.091 + 0.070
0,183 0.117
0,178 0.112
Table 21: Sensitivity analysis of a in RoMa on MS-COCO-2017 (SCORET).
Method
2k
3k
4k
5k
SD 1.4
0,239 1 0.032
0.250 0.031
0.242 a 0.029
0.250 a 0.027
0.246 1: 0.034
0.246
0.029
0.249 0.034
WatermarkDM
0.952 + 0.008
0.838
0.068
0.670 + 0.126
0.663 = 0.125
0.618
0.529 + 0.130
0.510
SAM
0,933 0.022
0.808
0.127
0.618 0.162
0.606 0.156
0.532
0,494 0.140
0.563 0.137
RoMa w/o PS
0.953 2 0.006
0.894 + 0.041
0.741 + 0.130
0.698 = 0.133
0.636
0.565 1 0.124
RuMa EF
0.934 0.017
0.128
0.149
0.629
0.493 a 0.127
0.491
0.134
RnMa(o=0.366) 063
0,944 + 0.006
0.509 + 0.014
0.858 + 0.096
0.844 0.065
0.785 + 0.088
0.668 0.132
0.667 0.133
RoMa(a=0.38)
0.952 0.006
0.924
0.858 + 0.104
0.839 0.081
0.790 0.108
0.670 0,136
0
RoMa(c=0:40)
0,944 0.007
0.919
0.882 :: 0.074
0.875 = 0.049
0.786 0.092
0.659 0.139
0.71300.12
424
0.950 a: 0.006
0.012 0.928
0,902
à: 0.062
0.884 a 0.041
0.831 1 0.077
0.708 0.134
0.724 0.125
RoMa(o=0444)
0,954 0.006
0.929 0.012
0,876 0.085
0.869 = 0.061
0.816 0.095
0.690 0.139
0,717 0.125
G Sensitivity Analysis of SAM's Perturbation Scale
In our primary experiments, set the perturbation scale e of SAM to 0.01. To investigate the impact of on watermark robustness, we conduct ablation experiments on the MS-COCO-201 dataset with varying values of 0.02 and 0.05, following prior research [16]. Our experimental results (presented in Tables 22-25) indicate that despite increasing c, we do not observe a significant enhancement in watermark robustness against tne-tuning Furthermore, notably decreases, averaging only 0.688 even without further fine-tuning (column "Ok"). This is likely attributed the larger compromising the original watermark embedding functionality. In sum, these analyses further support our conclusion in Section 1. path-specific smoothness proves more effective than SAM for enhancing watermark robustness against fine-tuning.
reaches 0.05, the watermark SCORE
22

---

Table 22: Sensitivity analysis of e SAM on MS-COCO-2017 (LPIPS), metric).
Method
Ok
1k
2k
3k
4k
5k
6k
SAM(c=0.01)
0.047 = 0.020
0.161 + 0,108
0.334 2 0.142
0.348 a 0,143
0.419 # 0.144
0.452 2 0.137
0.431 # 0.129
(..2))
0.057 0.022
0.193 0.100
0.355
0.124
0.117
0.131
0.406 0.095
0.05)
0.277=00.044
0.349 0,053
0.400 + 0.064
0.398
0.159
0.399 0.057
0.425
0.069
0.416 0,057
RoMa
0,038 3.00000005 0.005
0.061 = 0.011
0.102 0.066
0.104 0.040
0.192 0.078
0,302 0.1222
0.261 0.094
Table 23: Sensitivity analysis of c in SAM on MS-COCO-2017 (SSIM1).
Method
Ok
2k
3k
4k
5k
6k
SA0(==0.01)
0.868 0,030
0.694 + 0,180
0.451 0 0.224
0.428 # 0.212
0.333 + 0,197
0.280 + 0,186
0.322 0,182
SA.(2=0.02)
0.853 0.042
0.641 =.171
0.423 0.184
0.390
0.292 0.169
0.224 1224400.15 0.151
0.346 = 0.156
SA.(c=0.05)
0.559 0.077
0.418
0.321
0.098
0.318
0.318
0.105
0.273 0.099
0.306 0.093
RoMa
0.886 0.013
0.843 + 0.041
0.806
0.785 + 0.089
0.678 # 0.139
0.494 + 0.190
0.590 # 0.159
Table 24: Sensitivity analysis of in SAM on MS-COCO-2017 (MSE.).
Method
SAM(=0001)
0.019 0.017
0.101
0.249 + 0.131
0.241
0.121
0.291 + 0.109
0.321 + 0.110
0.321 0.115
(A.(2=0.02)
0.025 2 0.018
0.134 + 0.102
0.278 2 0.113
0.268 a 0.116
0.330 2 0.098
0.334 1 0.086
0.330 : 0.104
(A.(5)
0,208 0.052
0.078
0.072
0.076
0.079
0.073
0.070
RoMa
0.013 0.003
0.020 0.006
0.046 0.047
0.045 + 0.027
0.106 0.063
0.190
0.169 0.094
Table 25: Sensitivity analysis of in SAM on MS-COCO-200 (SCORE1).
Method
SAM(==001)
01116
0A.(==0.02)
0.922 0.026
0.769 = 0.120
0.594 0.133
0.583+0.133
0.50200.12
0.454 + 0.120
0.535
SAM(=0005)
0.688 0.056
0.591 0.071
0.526 0.071
0.528
0.071
0.526 + 0.072
0.497 0.071
0.509 + 0.065
RoMa
0.944 0.007
0.919 0.015
0.882 0.074
0.875 0.049
0.786 0.092
0.659
0.713
H Implementation Details of Watermark Detection
Stable Signature [13] and Aqualora [12] embed k-bit binary signature m E {0, 1] into generated images. For watermark detection, they first utilize the watermark extractor to decode message m' from a candidate image x and compare it with the predefined signature m. The detection mechanism relies on testing the statistical hypothesis Hi: æ was generated by the watermarked model against the null hypothesis Ho: x was not generated by the watermarked model. Specifically, they set bit threshold 7 and reject the null hypothesis Ho when the number of matched bits M (m, m') between the extracted message m' and the signature m satisfies:
(m, m') 2 where 7 E kk.
To obtain the False Positive Rate (FPR) associated with each bit threshold т, they assume the extracted bits follow an i.i.d. Bernoulli distribution with parameter 0.5 under Ho (i.e., random guess between bit and 1). This yields binomial distribution for M (m, m'), with parameters (k, 0.5). The FPR can then be formulated as:
FPR (T) P(M 7(H6)
Σ
(
k
1
2
I Experimental Setup for Stable Signature
Stable Signature [13] embeds watermarks into the Variational Autoencoder (VAF) decoder, so that all generated images carry binary messages. We follow the experimental settings of prior
23

---

1,00
1.00
1.00
1.00
TP%.
TPP
0.50
0.50
0.25
0.25
0.25
0.25
Winermarked Nattreenemmacce
@hhormarked
WWttmmrkedd
Watrmorted
0.00
0.00
0.00
10
16
16
10
10
10
10
10
10
10
10
EPR
FPR
FFR
FPR
Original
150 steps
300 steps
450 steps
1.00
1.00
1.00
0.75
0.75
0.75
0.75
199
0.50
0,50
0.50
0.50
Wanermarked
Wasermarko Nonn-sstemullos
Watemmrkel Ninn-wteemallll
Watemorkel
Nen weleemelleell
0.00
0.00
6:00
0.00
600 steps
750 steps
900 steps
1050 steps
1.00
1.00
1.00
0.50
0.50
0.50
0.25
0.25
0.25
Walcmrokked
Wenterre Non-walermaaile
meakeed
10
10
10
10
10
10
10
10
30
30
16
10
199
P91
FPR
1200 steps
1350 steps
1500 steps
Figure 13: ROC curves of Stable Signature at various fine-tuning steps. The star with green color (*) highlighted (FPR = 10-9) where its associated TPR reflects the detection accuracy with r = 38.
studies [13, 23, 58], and fine-tune the latent decoder to evaluate its robustness. Specifically, we use the MS-COCO-208 validation set, randomly selecting 4,000 images for fine-tuning and reserving the remaining 1, 1,000 images for evaluation. The fine-tuning process only minimizes the LPIPS loss between the original image and the image reconstructed by the latent decoder (as this maintains higher generation quality, following [58]), with a learning rate of 10-4 For the watermarked model, we use the official checkpoint", which embeds 48-bit binary message into the generated images. We set the bit threshold to T = 38 (FPR = 10 10-6) for watermark detection.
J Experimental Setup for AquaLora
AquaLora [12] also embeds binary messages into all generated images. However, it differs in that it merges watermark information into the U-Net [46, 20, 45] using Low Rank Adaptation (LoRA) [51] through scaling matrix strategy, thereby enabling watermark embedding during the denoising process. We evaluate the robustness of Aqual against fine-tuning on the MS-COCO-2017 and CUB-200-2011 1 datasets, respectively, adopting the same fine-tuning protocol as described in the Robustness Evaluation section (Section 5.3). For the watermarked model, we first obtain the official prior-preeerving fine-tuned checkpoints" and then embed the same 48-bit message as in Stable Signature into Stable Diffusion v with LoRA rank = 320. Here, we still set the bit threshold to 38 (FPR 10 -) for watermark detection [13, 58J. We use the prompt templates provided by AquaLora' for image generation, with the original Stable Diffusion v1.5 serving as the non- watermarked reference.
http:///r.aaagaaa..cc......ao.oa t/reemmaan
24

---

Figure 14:
Visual comparison between original images (top) and their reconstructions by Stable Signature's 's watermarked decoder (bottom).
Figure 15: Visual comparison between original images (top) and reconstructions by Stable Signature's 's watermarked decoder after 1500 fine-tuning steps (bottom).
Bit Accuracy
0.8
FPR-
0.7
0.6
10
20
30
40
Steps
Bit accuracy results of Arual against the fine-tuning process on CUB-200-2011 Figure 16:
dataset.
25

---

1.00
1.00
1.00
1.00
0.50
0.50
0.50
0.25
0.25
0.25
0.25
Wonermarked
Watemarked
Watermrted
0.00
0.00
0.00
0.00
10
10 FPR
16
10
10
10
10
10
10
FPR
FFR
FPR
Original
10 steps
20 steps
30 steps
1.00
0.75
199
0.50
Nen- walcmarkee
0.00
40 steps
Figure 17: ROC curves of Aqual Lora at various fine-tuning steps on MS-COCO-201 dataset. The star with green color (+) is highlighted (FPR 10-6) where its associated TPR reflects the detection accuracy with r = 38.
1.00
1.00
1.00
1.00
E
0.50
t 6,50
0.50
0.25
0.25
0.25
0.25
Wosermarked
Mizoriiurked
Wateomsdee
Watemmoked
0.00
0,00
0.00
0.00
10
16 FPR
16
10
10
16
10
10
FPR
FPR
FPR
Original
10 steps
20 steps
30 steps
0.75
0.50
Wistemmarsedd
Noo wwloottokkkll
0.0
40 steps
Figure 18: ROC curves of AquaLora at various fine-tuning steps on CUB-200-2011 dataset. The star with green color (*) is highlighted (FPR 10-6) where its associated TPR reflects the detection accuracy with r = 38.
K Additional Discussions of Diffusion Models
Diffusion models [20, 39, 19, 45, 41, 52, 64] have emerged as powerful generative paradigms, demonstrating remarkable success across various domains, including high-quality image synthesis [9, 47, 67], video generation [21, 60, 62], and natural language generation [31]. While our work focuses on protecting the watermarking robustness on Text-to- Image (T2I) diffusion models, our proposed RoMa is general and can be potentially adapted to watermarking diffusion models with different generative tasks [34, 25], and architectures [40].
L Discussion on Robustness Against Various Watermark Removal Attacks
In this section, we provide a comprehensive discussion on RoMa's robustness against various types of watermark removal attacks, addressing concerns raised about the scope of our evaluation.
26

---

L.1 Image-Level Attacks
Unlike embedding watermarks into the generated images, RoMa is a trigger-based watermark that operates by rooting watermark information within the parameter space. As such, RoMa can defend against image-level watermark attacks such as compression, rotation, cropping, denoising reconstruction, etc. [13, 69, 12, 29].
L.2 Model-Level Removal Attacks
For model-level removal attacks, we have extensively evaluated RoMa's robustness in our main experiments. As demonstrated in Section 6.1, RoMa maintains strong watermark preservation under standard fin-ttning scenarios (our primary threat model) across different datasets, i.e., MS-COCO 2017 and CUB-200-2000 for up to 6,000 fine-tuning steps, significantly outperforning existing baselines. Furthermore, in Section 6.3, we evaluate RoMa against adaptive fine-tuning attacks, strong removal attack where we assume attackers possess watermark knowledge and deliberately attempt to unlearn the watermark using synthetically generated unlearning data. Our experimental results show that even under such rigorous conditions, RoMa maintains strong defense capabilities, demonstrating excellent robustness against both vanilla and adaptive fine-tuning paradigms.
L.3 Model Distillation Attacks
Beyond fine-tuning attacks, an attacker could theoretically attempt to distill new model from our watermarked model bypass watermark detection. In such distillation attack scenario, the attacker would train a student model using synthetic data generated from our watermarked model, aiming to replicate the generation capability while excluding the watermark trigger from the training data. However, there still exist fundamental limitations that restrict the practicality of such distillation attacks real-worke scenarios. In practice, most distillation processes begin from an existing base model rather than training entirely from scratch. This is because training a large-scale generative model such as Stable Diffusion from scratch is prohibitively expensive, making impractical for most users. Additionally, training from scratch using only synthetic data from our target model is often highly unstable due to the biased synthetic dataset [2]. leading to degraded image quality [48, 53, 49]. Indeed, the emphasized distillation scenario exactly highlights the practical significance of our model watermarking. Our primary threat model focuses on protecting the intellectual property released base models. That is to say, if an attacker fine-tunes distills new model from our base model, our backdoor- based watermark remains embedded the model's parameters. This allows us to reliably detect unauthorized use of our model.
our
M Limitations
While extensive fine-tuning (e.g., over 6,000 steps) may eventually impact watermark detection, our approach significantly extends the robustness boundary compared existing methods that vulnerable even after just 1,000 fine-tuning steps. Moreover, fine-tuning across a large number of steps often leads degraded generalization diversity and capability. In this regard, our work significantly increases the removal cost, resulting in robust and effective solution for protecting IP in diffusion models in practice. Furthermore, we believe that this work will spark fruitful discussions and pave the way for future work on developing robust watermarking schemes in generative models.
27

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please our description in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in the conclusion.
28

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer to our experimental setting.
Guidelines:
The answer NA means that the paper does not include experiments.
29

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /Cod/Soommmiiicc SonddeeubmissiooPooo
/ //nips... //nips.cc/pput wttpccccccpppcccccccccocococccca
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
30

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification: Please refer experimental setting.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Section 6.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section 5.3.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
31

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttp://///eeis...eeeee ctcpppccccc..p.ccoooooo. htc//ublic//eee www.iiiiiieeeeenneeeeeeeeeee ines?
Answer: [Yes]
Justification:
We thoroughly read the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We protect the intellectual property of diffusion models, which helps safeguard the usage of diffusion models.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
32

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Section
Guidelines:
The answer NA means that the paper does existing
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
33

---

Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minima wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: hatpar/ppp..cc/oooofeeneee/////a for what should or should not be described.
34


Reviews:
[{"review_id": "gqYO2YbdkN", "reviewer": "Reviewer_mDBe", "summary": "This paper introduces RoMa, a robust model watermarking scheme designed to protect the intellectual property of diffusion models against fine-tuning. The authors claim that existing watermarking methods are vulnerable to fine-tuning because they converge to sharp minima in the loss landscape, leading to watermark loss. RoMa addresses this by decomposing watermarking into two components: Embedding Functionality for reliable detection and Path-specific Smoothness to enhance robustness along the watermark-connected path. Experiments show RoMa significantly improves watermark robustness.", "questions": "1. Provide evidence or references to support the claim and hypothesis in this paper.\n2. Clarify why LMC is the optimal method to simulate the fine-tuned process.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. This paper studies how to improve the robustness of model watermarking for diffusion models, which is truly an important problem.\n2. The motivation of this paper is clear.\n3. This insight of using LMC to analyze the robustness of model watermarking is intriguing.\n\nWeaknesses:\n\n1. The paper posits that existing watermarking methods cause the model to converge to a \"sharp minimum\". However, this foundational claim lacks direct experimental evidence within the paper. It may be better for the authors to substantiate this assertion with either supporting literature or concrete experimental results (e.g., an analysis of the loss landscape's curvature).\n2. The paper's core assumption, that Linear Mode Connectivity (LMC) can serve as a valid proxy for analyzing robustness during the fine-tuning process, is a strong claim made without adequate support. The authors need to provide a more rigorous justification for this crucial link.\n3. The paper simulates the effects of fine-tuning by linearly interpolating between the parameters of the original and watermarked models. This approach may be flawed because real-world fine-tuning is data-dependent and does not necessarily move the model's parameters along this specific path. The authors do not justify why LMC is the optimal method to simulate this process or explore alternative, potentially more realistic, methods.\n4. The core methodology of RoMa is conceptually similar to adversarial training, as it enhances robustness by training on perturbed versions of the model parameters. This insight is widely used in model watermarking (e.g., [1]). Considering that this paper may not sufficiently explain why this specific formulation is superior to other established robustness-enhancing techniques, I think the novelty of this paper is limited.\n5. The paper lacks a sensitivity analysis for the balancing hyperparameter $\\alpha$, which controls the trade-off between the 'Embedding Functionality' (EF) and 'Path-specific Smoothness' (PS) objectives. This is a critical parameter, and its impact on performance is not explored.\n\n[1] Certified Neural Network Watermarks with Randomized Smoothing.", "was_revised": true, "final_justification": "The response has addressed most of my concerns. Thus, I decided to raise my rating.", "rebuttal": "Thanks for your constructive feedback and insightful questions. Below we address your concerns one by one.\n\n### ***Response to W1&Q1: Experimental Evidence for \"Sharp Minimum\"***\n\nThank you for this constructive question. \n\nOur claim of \"sharp minimum\" is evidenced by our Linear Mode Connectivity (LMC) analysis in Fig. 2. In our experiments, we characterize the state where a model's performance (watermark quality in our setting) is sensitive to minor parameter perturbations (linear interpolation toward the original base model). For instance, For instance, at t = 0.9 (i.e., retaining 90% of the watermarked model's parameters), we already see a significant drop in watermark performance. This strong sensitivity to minor perturbations in the loss landscape is what we refer to as a \"sharp minimum\" in our context.\n\nFrom the broader literature, several studies regarding harmful fine-tuning in LLMs [1,2,3] use a similar term to study the relationship between landscape geometry and LLMs’ resistance against harmful fine-tuning. In our work, we empirically observe this phenomenon in baseline methods and improve the watermarking robustness through our more principled method.\n\nWe sincerely appreciate your suggestion and will include additional discussion in the revised manuscript to clarify and strengthen this foundational claim.\n\n### ***Response to W2&W3&Q2: Relationship between LMC and Fine-tuning Dynamics***\n\nThank you for your insightful comments. In fact, we do not use LMC to simulate the exact dynamics of real-world fine-tuning because practical fine-tuning is often data-dependent and can be influenced by the choice of hyperparameters. Instead, LMC serves as a tractable proxy to capture the **change in model behavior** across the loss landscapes of the base and watermarked models. This provides an approximation of fine-tuning effects, i.e., the transition from the original to the watermarked landscape without requiring access to the actual, and often unavailable, fine-tuning trajectory.\n\nFurthermore, we do not assume that real fine-tuning follows the linear interpolation defined by LMC. The key advantage of using LMC is that it **enables a unified design for RoMa**: it allows us to explicitly deviate the watermarked model from the original loss basin along a well-defined connected path, without needing to simulate diverse and complex fine-tuning processes exhaustively.\n\n### ***Response to W4: RoMa's Novelty Compared to Other Established Robustness-Enhancing Techniques***\n\nThank you for raising this insightful question. While there is some similarity in the optimization process between RoMa and adversarial training, we would like to clarify that RoMa differs fundamentally in how it explores the loss landscape, which explains its superior performance over other robustness-enhancing techniques (e.g., SAM) in terms of watermark robustness against fine-tuning.\n\nSpecifically, RoMa is designed to optimize for **path smoothness** along the Linear Mode Connectivity path. This encourages the watermarked model to deviate significantly away from the original non-watermarked model in the loss landscape, making it difficult to revert to the original basin through fine-tuning. As a result, RoMa demonstrates strong robustness against fine-tuning. In contrast, adversarial training primarily focuses on **adversarial smoothness** to improve generalization, which is conceptually different. Although adversarial training leads to an adversarially smooth loss landscape, it does not necessarily result in a large deviation from the original non-watermarked loss landscape, which explains why the watermark can still be easily removed with fine-tuning.\n\nWe believe this fundamental difference in optimization objectives distinguishes RoMa from existing robustness-enhancing techniques and explains its superior performance in watermark robustness against fine-tuning.\n\n### ***Response to W5: Sensitivity Analysis for the Balancing Hyperparameter $\\alpha$***\n\n\nThank you for this important question regarding the sensitivity analysis of the balancing hyperparameter $\\alpha$. \n\nIn fact, we have conducted a comprehensive sensitivity analysis for  $\\alpha$ in terms of LPIPS, SSIM, MSE, and SCORE metrics, which can be found in the *Appendix F* (Tables 14-17). Our experimental results demonstrate that RoMa maintains a relatively stable performance with different $\\alpha$.\n\n### ***References***\n\n[1] Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models, NeurIPS 2024.\n\n[2] Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey, arxiv 2024.\n\n[3] Understanding Pre-training and Fine-tuning from Loss Landscape Perspectives, arxiv 2025."}, {"review_id": "Znc0FlNOiM", "reviewer": "Reviewer_5Mwu", "summary": "This paper addresses the phenomenon that backdoor-based watermarks in diffusion models are easily removed by fine-tuning. Using Linear-Mode Connectivity, it analyzes the underlying reasons and proposes RoMa, a method that markedly improves watermark robustness against fine-tuning.", "questions": "1. The proposed method appears quite similar to Sharpness-Aware Minimization (SAM). Could the authors clarify why it outperforms SAM?\n2. Just curious: why were experiments run on Stable Diffusion v1.4? Using a more recent model would, in my opinion, be more convincing.", "limitations": "As noted above, backdoor-based model watermarks rely on a trigger. When an attacker distills the target model and the prompt dataset lacks that trigger (very probable), the watermark fails. I believe the authors should explicitly acknowledge this limitation in the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "##### Strengths\n\n- Fine-tuning attacks are indeed a powerful attack method of removing watermarks; studying defenses in this area is highly valuable.\n- The paper presents a comprehensive set of experiments.\n\n##### Weaknesses\n\n- **Typos**\n  - In the caption of Figure 6, “RoMa w/o PC” should likely read “RoMa w/o PS.”\n- I think the author could add some additional work in the Related Work section:\n  - [1] LaWa: Using Latent Space for In-Generation Image Watermarking\n  - [2] Latent Watermark: Inject and Detect Watermarks in Latent Diffusion Space\n  - [3] Aqualora: Toward White-Box Protection for Customized Stable Diffusion Models via Watermark LoRA\n- Backdoor-based model watermarks depend on a trigger. If an attacker distills the target model and the prompt dataset does not contain the trigger (which is very likely, since triggers are usually special tokens), the watermark becomes ineffective.\n- Section 6.3 states that compared with WatermarkDM, RoMa needs **32.83 ×** the training time to remove the watermark. Using the iPhone’s built-in QR-code scanner on Figure 7, I found the QR code was already unrecognizable after 125 steps with RoMa. Even if one argues that the image still “looks fine” to the human eye, by 2500 steps the QR code contains serious errors inconsistent with the original. I hope the authors can clarify why they consider 4925 steps to be the point at which the watermark is removed.", "was_revised": true, "final_justification": "After carefully reading the authors’ rebuttal and the other reviewers’ comments, I find that the authors have addressed my questions and largely alleviated my concerns. I have no further concerns and remain inclined to accept this paper.", "rebuttal": "Thank you for your thorough review and constructive comments. We greatly appreciate your valuable feedback on improving the experimental details and presentation. We address your concerns as follows.\n\n### ***Response to W1: Typos***\n\nWe greatly appreciate the reviewer for the careful proofreading and for pointing out the typo in Fig. 6. We are committed to enhancing the presentation quality of our paper. We will correct the caption from \"RoMa w/o PC\" to \"RoMa w/o PS\" as suggested in our revised version.\n\n### ***Response to W2: Additional Related Work***\n\nWe sincerely thank the reviewer for highlighting these relevant and important references. We recognize their significance to our work on diffusion watermarks and will incorporate them appropriately in the *Related Work* during revision. Additionally, since AquaLoRA considers a similar research topic concerning watermark robustness, we have included a comparison between our method and AquaLoRA in *Appendix I*, where we show that RoMA largely outperforms AquaLoRA against fine-tuning.\n\n### ***Response to W3 & Limitations: What if Attackers Distill the Target Model***\n\nThank you for this insightful question. We would like to offer two clarifications regarding the concern.\n\nFirst, while an attacker can distill our target model using a dataset without the trigger, there exist fundamental limitations that restrict their practicality. In practice, most distillation processes begin from an existing base model rather than training entirely from scratch. This is because training a large-scale generative model such as Stable Diffusion from scratch is prohibitively expensive, making it impractical for most users. Additionally, training from scratch using *only* synthetic data from our target model is often highly unstable due to the biased synthetic dataset [1], leading to degraded image quality [2,3,4]. \n\nIndeed, the emphasized distillation scenario exactly highlights the practical significance of our model watermarking. Our primary threat model focuses on protecting the intellectual property of our released base models. That is to say, if an attacker fine-tunes or distills a new model from our base model, our backdoor-based watermark remains embedded in the model’s parameters. This allows us to reliably detect unauthorized use of our model.\n\nWe sincerely thank the reviewer for the invaluable suggestion and will include a dedicated discussion in the revised manuscript to clarify the robustness and intended application of our watermarking scheme.\n\n### ***Response to W4: Clarifications on the Point when the Watermark is Removed***\n\nThank you for your valuable question. In fact, the watermarking results in Fig. 7 and Fig. 8 are under adaptive attack, where attackers know the realistic triggers and are dedicated to removing the watermarked model using synthetic unlearning datasets. We find that RoMa can effectively maintain the watermark patterns (QR code) compared to the baseline WatermarkDM.\n\nRegarding the evaluation metrics, Machine Scannability is only one of the evaluation criteria we consider for assessing watermark robustness. In our main experiments, we also rely on the visual similarity, such as the SCORE metric defined in Eq. 2. We consider the watermark to be effectively removed when the visual similarity between the extracted and original watermark patterns deviates significantly. In our experiments, this occurs at approximately 4925 fine-tuning steps for RoMa, which is why we report that as the removal point. In contrast, WatermarkDM fails after approximately 150 steps, leading to our claim that RoMa requires 32.83× more training time to remove the watermark.\n\nWe acknowledge the potential ambiguity in our original evaluation description and explanation regarding the distinction between machine scannability and watermark robustness. We would clarify these in the revised version.\n\n### ***Response to Q1: Why RoMa Outperforms SAM***\n\nThank you for raising this insightful question. While there is some similarity in the optimization process between RoMa and SAM, RoMa differs fundamentally in how it explores the loss landscape, which explains its superior performance over SAM in terms of watermark robustness.\n\nSpecifically, RoMa is designed to optimize for **path smoothness** along the Linear Mode Connectivity path. This encourages the watermarked model to deviate significantly away from the original non-watermarked model in the loss landscape, making it difficult to revert to the original basin through fine-tuning. As a result, RoMa demonstrates strong robustness against fine-tuning. In contrast, SAM primarily focuses on **adversarial smoothness** to improve generalization, which is conceptually different. Although SAM leads to an adversarially smooth loss landscape, it does not necessarily result in a large deviation from the original non-watermarked loss landscape, which explains why the watermark can still be easily removed with fine-tuning.\n\n### ***Response to Q2: Clarifications on SD v1.4 Choice***\n\nThank you for your question and kind suggestion. While our method is not directly related to the base model, our choice of SD v1.4 is to ensure a fair and direct comparison with our baselines [5,6]. We followed their experimental setup to faithfully reproduce their results for a rigorous evaluation of our method. \n\nWe sincerely appreciate the reviewer’s constructive suggestions regarding improving the quality of our submission and will incorporate additional experiments using recent model versions in our revised version.\n\n### ***References***\n\n[1] Self-Consuming Generative Models Go MAD, ICLR 2024.\n\n[2] Progressive Distillation for Fast Sampling of Diffusion Models, ICLR 2022.\n\n[3] Consistency Models, ICML 2023.\n\n[4] Adversarial Diffusion Distillation, ECCV 2024.\n\n[5] A Recipe for Watermarking Diffusion Models, arxiv 2023.\n\n[6] The Stable Signature: Rooting Watermarks in Latent Diffusion Models, ICCV 2023."}, {"review_id": "NTzEP1Rj9i", "reviewer": "Reviewer_soEG", "summary": "In order to protect the IP of the diffusion model and defense against fine-tuning attack this paper introduce a method called RoMa. RoMa establishes a mapping between the trigger and the output(watermark) and uses linear model connectivity enhance the path-specific smoothness making the watermarked model has a robust parameter region (flat basin). Under fine-tuning attack, RoMa achieves better watermark preservation; preserves high quality generation ability on fine-tuning datasets; has an effective watermark verification process.", "questions": "1.\tHow does path-specific smoothness work on Algorithm 1?\n\n2.\tHow is the performance of the algorithm defense against other kinds of watermark attacks ?\n\n3.\tWhat is the method of the fine-tuning attack? How is the performance of the model in face of other kinds of fine-tuning attack?\n\n4.\tHow does the watermark influence the original generation ability of the diffusion model?", "limitations": "The author does not mention the limitation of this algorithm about defense against other kinds of watermark removal attacks.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Weakness:\n\n1.\tAbout the clarity of the proposed method, it’s hard to understand the algorithm. On section 4 the author just describes the algorithm without explaining how it works.\n\n2.\tAbout the experiment:\n\n2.1\nIt’s lack of experiments to show the robustness of the watermark against other kinds of watermark removal attacks.\n\n2.2\nThe author does not mention what is the strategy of fine-tuning attack and how is the performance of the algorithm in face of different kinds of fine-tuning strategies.\n\n2.3\nAbout the generation capability of the model after watermarked. The experiment shows how does it fit the fine-tuning datasets, but lack of the results to show how does it affect the original generation ability of the model (on section 6.2  and table 2 & 3).\n\nStrengths:\n\n1.\tCompared with other baseline methods, under fine-tuning attack, RoMa achieves better watermark preservation, preserves high quality generation ability and has a reliable watermark verification process(QR code scanning).", "was_revised": true, "final_justification": "The response from the authors have addressed most of my previous concerns. Thus, I decided to increate my rating.", "rebuttal": "Thanks for your time and effort in reviewing our work! Below we address your concerns one by one.\n\n### ***Response to W1&Q1: Clarification on Algorithm Explanation and Path-specific Smoothness***\n\nThank you for this feedback. Below, we would clarify the algorithmic description and the underlying mechanism of RoMa as described in Section 4.\n\nRoMa is a simple and principal method composed of two key components: \n\n- **Embedding Functionality (EF)**: This component embeds the watermark by training the diffusion model to learn a mapping from a predefined trigger (e.g., “[V]”) to a specific watermark pattern (a QR code in our experiments). This enables reliable watermark detection in the watermarked model. \n\n- **Path-specific Smoothness (PS)**: Motivated by our preliminary investigation, this component aims to enhance smoothness along the watermark-connected path, thereby improving robustness against fine-tuning. By enforcing path-specific smoothness, we effectively deviate the watermarked model away from the original non-watermarked model in the loss landscape, making it difficult to revert to the original basin via fine-tuning.\n\nRegarding how **Path-specific Smoothness works in Algorithm 1**: Our goal is to encourage deviation from the original loss basin along the LMC path (as illustrated in Fig. 2). This is done by first interpolating model parameters from the current watermarked model back toward the original model. We then compute the gradient at the interpolated model, which is used to update the current watermarked model. This update deviates the watermarked model away from the interpolated point in the loss landscape, thereby increasing its deviation from the original non-watermarked basin and enhancing its resistance to fine-tuning-based watermark removal.\n\n### ***Response to W2&Q2&Limitations: Performance Against Other Types of Watermark Removal Attacks***\n\nThank you for raising this insightful question. We would like to offer two clarifications regarding the concern. \n\nFirst of all, unlike embedding watermarks into the generated images, RoMa is a trigger-based watermark that operates by rooting watermark information within the parameter space. As such, RoMa can defend against image-level watermark attacks such as compression, rotation, cropping, denoising reconstruction, etc. [1,2,3,4]. We also adopt various removal attacks to evaluate RoMa. Experimental results show that RoMa demonstrates excellent robustness against not only the vanilla fine-tuning paradigms, but adaptive fine-tuning attacks, a strong removal attack where we assume attackers possess watermark knowledge and attempt to unlearn the watermark, as illustrated in Section 6.3. significantly outperforming existing baselines. Nevertheless, our experimental results show that even under such rigorous conditions, RoMa maintains strong defense capabilities.\n\nAdditionally, even if the attacker aims to distill our target model to bypass the watermark detection, there still exist fundamental limitations that restrict their practicality. In practice, most distillation processes begin from an existing base model rather than training entirely from scratch. This is because training a large-scale generative model such as Stable Diffusion from scratch is prohibitively expensive, making it impractical for most users. Additionally, training from scratch using only synthetic data from our target model is often highly unstable due to the biased synthetic dataset [5], leading to degraded image quality [6,7,8].\n\n### ***Response to W2&Q3: Fine-tuning Attack Strategies and Their Influence on RoMa's Robustness against Fine-tuning***\n\nThank you for this valuable question.\n\n1. What is the method of the fine-tuning attack?\n\n    As elaborated in our experimental setting, we adopt the default full-parameter fine-tuning following [9] on two datasets, MS-COCO-2017 and CUB-200-2011. We use the Diffusers framework with 512×512 image size, learning rate 1 × 10−5, and Adam optimizer.\n\n2. How is the performance of the model in face of other kinds of fine-tuning attack?\n\n   While our main experiments leverage the full-parameter fine-tuning strategy, RoMa is also robust against other fine-tuning strategies, such as LoRa and QLoRa. This is because these parameter-efficient fine-tuning is designed to be effective in adapting models to new tasks with minor parameter change [10], which demonstrates less removal capability compared to our full-parameter fine-tuning. Moreover, in our Security experiment (Section 6.3), even if we assume attackers possess watermark knowledge and attempt to unlearn the watermark, RoMa still demonstrates superior robustness against fine-tuning.\n\nWe sincerely appreciate the reviewer's concern. We would supplement more fine-tuning details and additional results on other kinds of fine-tuning attacks in our revised version.\n\n### ***Response to W2&Q4: Impact of Watermarking on Original Generation Capability***\n\nThank you for this question.\n\nIn fact, we **have conducted extensive evaluations to evaluate the generation capability in the original manuscript**. In Section 6.2, we evaluate the general generation capability of RoMa from both quality and detectability perspectives. Our experimental results demonstrate that RoMa maintains comparable FID and CLIP scores with the original SD 1.4 model on the MS-COCO-2017 validation set, as shown in **Table 2**. This is further evidenced by the qualitative results in **Fig. 5**.\n\nBesides, in Section 5.2 (Evaluation Protocol), we clearly define \"Quality concerns maintaining the model's general performance after watermark embedding.\" We provide detailed implementation details in Section 5.3 and the appendix to ensure comprehensive evaluation of generation quality preservation.\n\n### ***References***\n\n[1] The Stable Signature: Rooting Watermarks in Latent Diffusion Models, ICCV 2023.\n\n[2] Invisible Image Watermarks Are Provably Removable Using Generative AI, NeurIPS 2024.\n\n[3] AquaLoRA: Toward White-box Protection for Customized Stable Diffusion Models via Watermark LoRA, ICML 2024.\n\n[4] Image Watermarks are Removable Using Controllable Regeneration from Clean Noise, ICLR 2025.\n\n[5] Self-Consuming Generative Models Go MAD, ICLR 2024.\n\n[6] Progressive Distillation for Fast Sampling of Diffusion Models, ICLR 2022.\n\n[7] Consistency Models, ICML 2023.\n\n[8] Adversarial Diffusion Distillation, ECCV 2024.\n\n[9] Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable, arxiv 2024.\n\n[10] Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices, TPAMI 2025."}, {"review_id": "71X2aKeatg", "reviewer": "Reviewer_1Xat", "summary": "The paper proposes a model watermarking method for diffusion models, named RoMa. It addresses the limitations of existing approaches, which often converge to local minima and fail to maintain watermark robustness after model finetuning. RoMa leverages mode connectivity to analyze the impact of finetuning on watermarks, and introduces a path-specific smoothness loss during the embedding process to enhance watermark robustness. Experimental results demonstrate that the proposed method outperforms existing baselines in both watermark success rate and defense overhead.", "questions": "The overall evaluation is based primarily on S1-S3(+) and W1 (–), W2 (–) from the Strengths and Weaknesses section. Below are suggestions for the authors to improve the work:\n\n1. Please elaborate further on the similarity between model finetuning and parameter mixing along the mode connectivity path. (W1)\n\n2. It is recommended to consider attackers with varying levels of knowledge (W2)", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "S1: The idea of analyzing the impact of model finetuning on watermarks using mode connectivity is interesting and likely valid, as the watermark embedding process can be viewed as a form of finetuning on the triggered dataset.\n\nS2: Experimental results strongly support the analysis and indicate significant practical potential.\n\nS3: The paper is well organized and easy to follow.\n\nW1: Some conclusions lack sufficient analysis. For example, why do existing smoothness-aware optimization methods such as SAM and PGN fail to improve robustness? Furthermore, using the mode connectivity path as a proxy for model finetuning is not straightforward, more explanation is needed regarding the similarity between finetuning and parameter interpolation.\n\nW2: The threat model could be more comprehensive. The paper only considers naïve attackers with no knowledge of potential defenses. However, adversaries may perform model distillation or backdoor detection when equipped with prior knowledge. Is the proposed watermarking method still effective under these more sophisticated attack scenarios?", "was_revised": true, "final_justification": "Overall I think this is an interesting paper, the idea of analyzing the impact of model fine-tuning on watermarks using mode connectivity is interesting and likely valid. Considering that multiple reviewers have suggested that some of the motivations for this paper lack sufficient validation,  there is room for improvement (at least in the writing). Given that my original rating was already positive, I will be maintaining it.", "rebuttal": "We are grateful to you for your time and effort in reviewing our work, as well as acknowledging our contributions. Below we address your concerns one by one.\n\n### ***Response to W1&Q1: Elaboration on LMC as Proxy and Analysis of Smoothness Methods\"***\n\nThank you for this insightful question and your valuable feedback on our analysis.\n\n**1. Why We Use LMC as a Proxy to Analyze Watermarking’s Robustness against Fine-tuning:**\n\nWe would like to clarify that we do not use LMC to simulate the exact dynamics of real-world fine-tuning. This is because practical fine-tuning is often data-dependent and can be influenced by the choice of hyperparameters, making it difficult to directly model the fine-tuning dynamics. Instead, we adopt LMC as a simple and tractable proxy to capture the **change in model behavior** across the loss landscapes of the base and watermarked models. This provides an approximation of fine-tuning effects, i.e., the transition from the original to the watermarked landscape without requiring access to the actual, and often unavailable, fine-tuning trajectory.\n\nMoreover, leveraging LMC enables us to further design our RoMa in a more unified manner: it allows us to explicitly deviate the watermarked model from the original loss basin along a well-defined connected path, without needing to simulate diverse and complex fine-tuning processes exhaustively.\n\n**2. Why SAM and PGN Fail to Improve Robustness:**\n\nWhile there is some similarity in the optimization process between RoMa and SAM, RoMa differs fundamentally from SAM/PGN in how it explores the loss landscape, which explains its superior performance over SAM/PGN in terms of watermark robustness.\n\nSpecifically, RoMa is designed to optimize for **path smoothness** along the Linear Mode Connectivity path. This encourages the watermarked model to deviate significantly away from the original non-watermarked model in the loss landscape, making it difficult to revert to the original basin through fine-tuning. As a result, RoMa demonstrates strong robustness against fine-tuning. In contrast, SAM/PGN primarily focuses on **adversarial smoothness** to improve generalization, which is conceptually different. Although SAM/PGN leads to an adversarially smooth loss landscape, it does not necessarily result in a large deviation from the original non-watermarked loss landscape, which explains why the watermark can still be easily removed with fine-tuning.\n\n### ***Response to W2&Q2: Consideration of More Sophisticated Attack Scenarios\"***\n\nThank you for raising this insightful question.\n\n**1. Non-naive Attackers with Advanced Knowledge:**\n\n**We also considered non-naive attackers in our evaluation**. In our security experiment (Section 6.3), we assume strong attackers who possess watermark knowledge, i.e., our trigger, and attempt to unlearn the watermark. Nevertheless, RoMa still demonstrates superior robustness against fine-tuning even under such challenging conditions.\n\n**2. Effectiveness Against Model Distillation Attacks:**\n\nWhile we acknowledge that an attacker can distill our target model using a dataset without the trigger, there exist fundamental limitations that restrict their practicality. In practice, most distillation processes begin from an existing base model rather than training entirely from scratch. This is because training a large-scale generative model such as Stable Diffusion from scratch is prohibitively expensive, making it impractical for most users. Additionally, training from scratch using *only* synthetic data from our target model is often highly unstable due to the potentially biased synthetic dataset [1], leading to degraded image quality [2,3,4]. \n\nIndeed, the emphasized distillation scenario exactly highlights the practical significance of our model watermarking. Our primary threat model focuses on protecting the intellectual property of our released base models. That is to say, if an attacker fine-tunes or distills a new model from our base model, our trigger-based watermark remains embedded in the model’s parameters. This allows us to reliably detect unauthorized use of our model.\n\nWe sincerely thank the reviewer for the invaluable suggestion and will include a dedicated discussion in the revised manuscript to clarify the robustness and intended application of our watermarking scheme.\n\n### ***References***\n\n[1] Self-Consuming Generative Models Go MAD, ICLR 2024.\n\n[2] Progressive Distillation for Fast Sampling of Diffusion Models, ICLR 2022.\n\n[3] Consistency Models, ICML 2023.\n\n[4] Adversarial Diffusion Distillation, ECCV 2024."}]

Author rebuttal:

Meta-review: (a) Summary: This paper proposes RoMa, a robust watermarking scheme for diffusion models that protects intellectual property against fine-tuning attacks. Unlike existing watermarking methods that fail due to sharp local minima, RoMa decomposes watermarking into two key components: Embedding Functionality for reliable watermark detection and Path-specific Smoothness to ensure robustness along the watermark-connected path. Leveraging linear mode connectivity, RoMa establishes a flat and stable parameter region that preserves watermarks under fine-tuning while maintaining generation quality. Experiments demonstrate that RoMa significantly outperforms prior approaches in watermark robustness and verification efficiency.

(b) Strengths:
1. The study of watermarking problem about defending against fine-tuning attacks is important. RoMa achieves good performance against this attack.

2. The idea of using mode connectivity to analyze the impact of model finetuning on watermarks is interesting and likely valid.

(c) Weaknesses (after rebuttal):
1. Reviewers have suggested that some of the motivations for this paper lack sufficient validation (at least in the writing).

(d) Why this decision:
The paper is on the borderline. All reviewers express that their major concerns have been addressed in the rebuttal and some of reviewers are willing to increase their scores. Given that all reviewers are positive and the strengths outweight the weaknesses, AC would recommend Accept (poster).

(e) Summary of discussions:
See above
Decision: Accept (poster)