Title: RoMa: A Robust Model Watermarking Scheme for Protecting IP in Diffusion Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Models

Robust Watermarking

IP Protection
Abstract: Preserving intellectual property (IP) within a pre-trained diffusion model is critical for protecting the model's copyright and preventing unauthorized model deployment. In this regard, model watermarking is a common practice for IP protection that embeds traceable information within models and allows for further verification. Nevertheless, existing watermarking schemes often face challenges due to their vulnerability to fine-tuning, limiting their practical application in general pre-training and fine-tuning paradigms. Inspired by using mode connectivity to analyze model performance between a pair of connected models, we investigate watermark vulnerability by leveraging Linear Mode Connectivity (LMC) as a proxy to analyze the fine-tuning dynamics of watermark performance. Our results show that existing watermarked models tend to converge to sharp minima in the loss landscape, thus making them vulnerable to fine-tuning. To tackle this challenge, we propose **RoMa**, a **Ro**bust **M**odel w**a**termarking scheme that improves the robustness of watermarks against fine-tuning. Specifically, RoMa decomposes watermarking into two components, including *Embedding Functionality*, which preserves reliable watermark detection capability, and *Path-specific Smoothness*, which enhances the smoothness along the watermark-connected path to improve robustness. Extensive experiments on benchmark datasets MS-COCO-2017 and CUB-200-2011 demonstrate that RoMa significantly improves watermark robustness against fine-tuning while maintaining generation quality, outperforming baselines. The code is available at [https://github.com/xiekks/RoMa](https://github.com/xiekks/RoMa).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a model watermarking method for diffusion models, named RoMa. It addresses the limitations of existing approaches, which often converge to local minima and fail to maintain watermark robustness after model finetuning. RoMa leverages mode connectivity to analyze the impact of finetuning on watermarks, and introduces a path-specific smoothness loss during the embedding process to enhance watermark robustness. Experimental results demonstrate that the proposed method outperforms existing baselines in both watermark success rate and defense overhead.\n\nSTRENGTHS AND WEAKNESSES:\nS1: The idea of analyzing the impact of model finetuning on watermarks using mode connectivity is interesting and likely valid, as the watermark embedding process can be viewed as a form of finetuning on the triggered dataset.\n\nS2: Experimental results strongly support the analysis and indicate significant practical potential.\n\nS3: The paper is well organized and easy to follow.\n\nW1: Some conclusions lack sufficient analysis. For example, why do existing smoothness-aware optimization methods such as SAM and PGN fail to improve robustness? Furthermore, using the mode connectivity path as a proxy for model finetuning is not straightforward, more explanation is needed regarding the similarity between finetuning and parameter interpolation.\n\nW2: The threat model could be more comprehensive. The paper only considers naïve attackers with no knowledge of potential defenses. However, adversaries may perform model distillation or backdoor detection when equipped with prior knowledge. Is the proposed watermarking method still effective under these more sophisticated attack scenarios?\n\nQUESTIONS:\nThe overall evaluation is based primarily on S1-S3(+) and W1 (–), W2 (–) from the Strengths and Weaknesses section. Below are suggestions for the authors to improve the work:\n\n1. Please elaborate further on the similarity between model finetuning and parameter mixing along the mode connectivity path. (W1)\n\n2. It is recommended to consider attackers with varying levels of knowledge (W2)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn order to protect the IP of the diffusion model and defense against fine-tuning attack this paper introduce a method called RoMa. RoMa establishes a mapping between the trigger and the output(watermark) and uses linear model connectivity enhance the path-specific smoothness making the watermarked model has a robust parameter region (flat basin). Under fine-tuning attack, RoMa achieves better watermark preservation; preserves high quality generation ability on fine-tuning datasets; has an effective watermark verification process.\n\nSTRENGTHS AND WEAKNESSES:\nWeakness:\n\n1.\tAbout the clarity of the proposed method, it’s hard to understand the algorithm. On section 4 the author just describes the algorithm without explaining how it works.\n\n2.\tAbout the experiment:\n\n2.1\nIt’s lack of experiments to show the robustness of the watermark against other kinds of watermark removal attacks.\n\n2.2\nThe author does not mention what is the strategy of fine-tuning attack and how is the performance of the algorithm in face of different kinds of fine-tuning strategies.\n\n2.3\nAbout the generation capability of the model after watermarked. The experiment shows how does it fit the fine-tuning datasets, but lack of the results to show how does it affect the original generation ability of the model (on section 6.2  and table 2 & 3).\n\nStrengths:\n\n1.\tCompared with other baseline methods, under fine-tuning attack, RoMa achieves better watermark preservation, preserves high quality generation ability and has a reliable watermark verification process(QR code scanning).\n\nQUESTIONS:\n1.\tHow does path-specific smoothness work on Algorithm 1?\n\n2.\tHow is the performance of the algorithm defense against other kinds of watermark attacks ?\n\n3.\tWhat is the method of the fine-tuning attack? How is the performance of the model in face of other kinds of fine-tuning attack?\n\n4.\tHow does the watermark influence the original generation ability of the diffusion model?\n\nLIMITATIONS:\nThe author does not mention the limitation of this algorithm about defense against other kinds of watermark removal attacks.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the phenomenon that backdoor-based watermarks in diffusion models are easily removed by fine-tuning. Using Linear-Mode Connectivity, it analyzes the underlying reasons and proposes RoMa, a method that markedly improves watermark robustness against fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\n##### Strengths\n\n- Fine-tuning attacks are indeed a powerful attack method of removing watermarks; studying defenses in this area is highly valuable.\n- The paper presents a comprehensive set of experiments.\n\n##### Weaknesses\n\n- **Typos**\n  - In the caption of Figure 6, “RoMa w/o PC” should likely read “RoMa w/o PS.”\n- I think the author could add some additional work in the Related Work section:\n  - [1] LaWa: Using Latent Space for In-Generation Image Watermarking\n  - [2] Latent Watermark: Inject and Detect Watermarks in Latent Diffusion Space\n  - [3] Aqualora: Toward White-Box Protection for Customized Stable Diffusion Models via Watermark LoRA\n- Backdoor-based model watermarks depend on a trigger. If an attacker distills the target model and the prompt dataset does not contain the trigger (which is very likely, since triggers are usually special tokens), the watermark becomes ineffective.\n- Section 6.3 states that compared with WatermarkDM, RoMa needs **32.83 ×** the training time to remove the watermark. Using the iPhone’s built-in QR-code scanner on Figure 7, I found the QR code was already unrecognizable after 125 steps with RoMa. Even if one argues that the image still “looks fine” to the human eye, by 2500 steps the QR code contains serious errors inconsistent with the original. I hope the authors can clarify why they consider 4925 steps to be the point at which the watermark is removed.\n\nQUESTIONS:\n1. The proposed method appears quite similar to Sharpness-Aware Minimization (SAM). Could the authors clarify why it outperforms SAM?\n2. Just curious: why were experiments run on Stable Diffusion v1.4? Using a more recent model would, in my opinion, be more convincing.\n\nLIMITATIONS:\nAs noted above, backdoor-based model watermarks rely on a trigger. When an attacker distills the target model and the prompt dataset lacks that trigger (very probable), the watermark fails. I believe the authors should explicitly acknowledge this limitation in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces RoMa, a robust model watermarking scheme designed to protect the intellectual property of diffusion models against fine-tuning. The authors claim that existing watermarking methods are vulnerable to fine-tuning because they converge to sharp minima in the loss landscape, leading to watermark loss. RoMa addresses this by decomposing watermarking into two components: Embedding Functionality for reliable detection and Path-specific Smoothness to enhance robustness along the watermark-connected path. Experiments show RoMa significantly improves watermark robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This paper studies how to improve the robustness of model watermarking for diffusion models, which is truly an important problem.\n2. The motivation of this paper is clear.\n3. This insight of using LMC to analyze the robustness of model watermarking is intriguing.\n\nWeaknesses:\n\n1. The paper posits that existing watermarking methods cause the model to converge to a \"sharp minimum\". However, this foundational claim lacks direct experimental evidence within the paper. It may be better for the authors to substantiate this assertion with either supporting literature or concrete experimental results (e.g., an analysis of the loss landscape's curvature).\n2. The paper's core assumption, that Linear Mode Connectivity (LMC) can serve as a valid proxy for analyzing robustness during the fine-tuning process, is a strong claim made without adequate support. The authors need to provide a more rigorous justification for this crucial link.\n3. The paper simulates the effects of fine-tuning by linearly interpolating between the parameters of the original and watermarked models. This approach may be flawed because real-world fine-tuning is data-dependent and does not necessarily move the model's parameters along this specific path. The authors do not justify why LMC is the optimal method to simulate this process or explore alternative, potentially more realistic, methods.\n4. The core methodology of RoMa is conceptually similar to adversarial training, as it enhances robustness by training on perturbed versions of the model parameters. This insight is widely used in model watermarking (e.g., [1]). Considering that this paper may not sufficiently explain why this specific formulation is superior to other established robustness-enhancing techniques, I think the novelty of this paper is limited.\n5. The paper lacks a sensitivity analysis for the balancing hyperparameter $\\alpha$, which controls the trade-off between the 'Embedding Functionality' (EF) and 'Path-specific Smoothness' (PS) objectives. This is a critical parameter, and its impact on performance is not explored.\n\n[1] Certified Neural Network Watermarks with Randomized Smoothing.\n\nQUESTIONS:\n1. Provide evidence or references to support the claim and hypothesis in this paper.\n2. Clarify why LMC is the optimal method to simulate the fine-tuned process.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}