Title: Transferable Black-Box One-Shot Forging of Watermarks via Image Preference Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: image watermarking

watermark forging
Abstract: Recent years have seen a surge in interest in digital content watermarking techniques, driven by the proliferation of generative models and increased legal pressure. With an ever-growing percentage of AI-generated content available online, watermarking plays an increasingly important role in ensuring content authenticity and attribution at scale. There have been many works assessing the robustness of watermarking to removal attacks, yet, watermark forging, the scenario when a watermark is stolen from genuine content and applied to malicious content, remains underexplored. In this work, we investigate watermark forging in the context of widely used post-hoc image watermarking. Our contributions are as follows. First, we introduce a preference model to assess whether an image is watermarked. The model is trained using a ranking loss on purely procedurally generated images without any need for real watermarks. Second, we demonstrate the model's capability to remove and forge watermarks by optimizing the input image through backpropagation. This technique requires only a single watermarked image and works without knowledge of the watermarking model, making our attack much simpler and more practical than attacks introduced in related work. Third, we evaluate our proposed method on a variety of post-hoc image watermarking models, demonstrating that our approach can effectively forge watermarks, questioning the security of current watermarking approaches. Our code and further resources are publicly available.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel watermark forging method that leverages a preference model to extract the watermark from a single watermarked image. By optimizing the input image, the method recovers the watermark without access to the underlying watermarking model. This makes the process more practical than prior approaches, which typically require multiple watermarked examples or white-box access.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper is generally well-written and easy to follow. The methodology and results are clearly described, and the topic is relevant.\n2. The proposed approach improves practicality by not requiring access to the watermark decoder and by working with only a single watermarked image.\n3. The authors conduct a comprehensive set of experiments and ablation studies to validate the effectiveness of their method.\n\n**Weaknesses**\n1. From the results, it appears that the proposed approach only outperforms prior work on the Video Seal case in terms of bit accuracy. It does not seem to generalize well to other use cases and does not show improvements in image quality as measured by PSNR.\n2. The small sample size (100) may limit the reliability of the findings.\n3. Including additional empirical results, such as AUROC, could provide a more complete picture of detection performance.\n\nQUESTIONS:\n1. The watermarking methods referenced in your work (e.g., CIN, MBRS, etc.) are all learning-based, and as such, the encoding process is non-linear and specifically non-additive. Given this, what is the intuition behind the subtraction and addition operations used in your framework to retrieve and apply the watermark?\n2. Have you considered incorporating an image regularization loss to mitigate the unnatural smoothing artifacts observed in Figure 2 of the appendix?\n3. Could you clarify how the watermark is selected in your experiments?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address watermark forging from a single image in a black-box setting. They (1) train a preference model on procedurally corrupted images using a ranking loss, (2) treat the model’s score as a surrogate objective to optimize pixels and extract an estimated watermark, and (3) paste this estimate onto arbitrary targets to forge or remove post-hoc watermarks, all without access to the underlying watermarking system. Experiments on four recent schemes (CIN, MBRS, TrustMark, VideoSeal) show competitive removal accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. Introduces a ranking-based preference model trained without any real watermarks.\n2. The design of artifact types for the training preference model is reasonable.\n3. The idea is clearly conveyed.\n\n## Weaknesses\n1. The evaluated watermarking models lack sufficient robustness. Leading models such as StegaStamp [1] and VINE [2], known for their enhanced robustness, were not included in the tests. Incorporating experiments with these robust watermarking models into Tables 1, 2, and 3 is recommended, particularly for VINE, which claims resilience to image editing. A comprehensive evaluation of these models would significantly strengthen this work.\n\n2. Do forged watermarks exhibit the same robustness as their original counterparts? Specifically, are forged watermarked images from VINE equally resistant to image editing? Investigating the impact on robustness would provide valuable insights.\n\n[1] [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://arxiv.org/abs/1904.05343)\n\n[2] [Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances](https://github.com/Shilin-LU/VINE)\n\nQUESTIONS:\nPlease review the Weaknesses section. If all of my concerns are fully resolved, I’ll gladly raise my rating further; if they aren’t, I may lower it.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for watermark forging and removal that requires only a single watermarked image and no access to the watermarking algorithm or decoder, representing a realistic black-box threat model. The core of the method is an image preference model trained on synthetically corrupted images to learn to detect unnatural visual artifacts. This preference model is then used as a surrogate loss function to guide gradient-based optimization that can remove or forge watermarks in a generic and transferable manner. The authors validate their method across several state-of-the-art post-hoc watermarking schemes and show strong empirical results, outperforming prior work that requires significantly more data and assumptions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The attack assumes only access to a single watermarked image and no interaction with the underlying watermark model or decoder, making it applicable to realistic black-box settings—unlike prior methods that require many samples or access to APIs.\n- The proposed method is transferable and data-efficient, working across different watermarking methods with minimal assumptions and no need for model-specific adaptation or multiple samples.\n- The preference model, trained on synthetic perturbations, provides a general-purpose signal for guiding watermark removal and synthesis—without requiring real watermarked data\n- The paper is evaluated on four major watermarking systems and reports both quantitative metrics (bit accuracy, PSNR) and qualitative visual results. It includes detailed ablation studies that strengthen the empirical claims.\n\nWeaknesses:\n- While strong overall, the method underperforms on TrustMark, which uses a highly content-dependent decoder. This suggests limitations when watermarking schemes are deeply entangled with semantic content.\n- All evaluations are performed on benchmark datasets and simulated watermarking schemes. The effectiveness on real-world dataset remains uncertain.\n- The preference model is based on synthetic noise and may not fully capture semantic-level indicators of watermarks—especially if the watermark is adaptively designed or learned jointly with the content.\n- The experimental section lacks statistical significance analysis or variance reporting, which would strengthen claims about generalizability and robustness. Some additional theoretical analysis and/or results would strengthen the paper.\n- The paper does not provide robustness evaluations under adversarial detection or detection-aware defense schemes.\n\nQUESTIONS:\n1: Have you evaluated your method on AI-generated content from diffusion models (e.g., Stable Diffusion) that may contain embedded watermarks or platform fingerprints? How do you anticipate your method will generalize to such real-world scenarios?\n\n2: Your method performs well on synthetic and high-frequency watermarks. How would it perform on semantic or learned watermarks (e.g., those embedded in latent space or object features)?\n\n3: Did you test whether the forged images can evade watermark detectors used by platforms (e.g., Google SynthID)?\n\n4: Have you compared your preference model to traditional image quality metrics (e.g., LPIPS)?\n\nLIMITATIONS:\nPartly: while the authors mention some safeguards and discuss a bit the ethical aspects, the justification (there are already methods out there that do the same thing, in essence) is not a great reasoning. This should be further discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper proposes a generalizable black-box watermark extraction attack that requires only a single watermarked image to extract the underlying watermark pattern for forging or removal. Specifically, the authors synthesize artifacts to simulate invisible watermark patterns and train a preference model to distinguish between watermarked and clean images. This trained model is then used as a surrogate model to perform adversarial optimization and extract the watermark pattern from any given watermarked image. The method is evaluated on four watermarking techniques—CIN, MBRS, TrustMark, and VideoSeal—demonstrating its effectiveness.

STRENGTHS AND WEAKNESSES:
**Strengths**

* The proposed watermark attack is both novel and effective. Compared to previous extraction methods, the ability to attack using only a single watermarked image significantly improves its practicality.
* The paper is well-written, and the experiments are thorough.

**Weaknesses**

* The attack would be more convincing if it were evaluated on diffusion-based watermarking methods such as RingID and GaussianShading.

QUESTIONS:
Why did the authors choose wave-, noise-, and line-style synthetic artifacts? What is the individual contribution of each type? Which one is most effective in mimicking real watermark patterns?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-03