Kati8/ICLR_2023_2025 huggingface dataset:
authors: Runyi Hu
authors: Jie Zhang
authors: Yiming Li
authors: Jiwei Li
authors: Qing Guo
authors: Han Qiu
authors: Tianwei Zhang
reviews: 
Comments: This paper introduces SuperMark, a robust and training-free framework for watermarking AI-generated content to protect copyright and authenticate content. SuperMark leverages parallels between watermark embedding/extraction and the denoising processes in diffusion models, embedding watermarks into Gaussian noise and using SR models to recover them. Unlike traditional methods that struggle with robustness and fidelity, SuperMark demonstrates strong resilience against distortions, adaptive attacks, and maintains high fidelity through DDIM Inversion techniques. It achieves high watermark extraction accuracy (over 89%) across different conditions, surpassing many existing methods. Additionally, SuperMark shows strong adaptability across various datasets, SR models, and watermark settings.
Strengths: - The paper presents a novel discovery regarding the diffusion process, highlighting its intrinsic robustness against normal distortions and adaptive attacks, which makes this finding particularly interesting.
- The diffusion process does not require training.
- The pipeline depicted in Figure 2 is clearly illustrated and facilitates easy comprehension.
Weaknesses: - The experimental results do not SOTA performance, particularly in terms of PSNR and SSIM metrics. For instance, in Table 1, the PSNR value is only 32.5, which is considerably low, severely affecting image fidelity. This suggests that the trade-off between fidelity and robustness in your method does not yield a compelling outcome. Furthermore, the PSNR values reported in Table 2 remain significantly below current SOTA.

- The novelty of the work appears insufficient, as the methods employed seem to rely primarily on off-the-shelf algorithms. The primary contribution appears to be the strategic integration of these existing techniques.

- The paper lacks an analysis of efficiency. Although the diffusion process is advantageous as a training-free approach to counter the two types of attacks you mentioned, it appears to have drawbacks related to inference efficiency. The inference time seems potentially much lower compared to conventional encoder-noise-decoder frameworks.

- Additionally, the paper does not include experiments against some SOTA attack methods. For example, it is missing an evaluation using attacks[r1].

[r1] Evading Watermark Based Detection of AI-generated Content (CCS 2023).

- There is also an insufficient ablation study. I am particularly curious about the impact on SuperMark if the SR model is removed from the process.
Recommendation: 3
Reviewer confidence: 5
Soundness/correctness: 2
Clarity: 3
reviews: 
Comments: This paper proposes a novel robust and training-free watermarking framework, named SuperMark. It embeds the watermark into initial Gaussian noise using existing techniques and then applies pretrained Super-Resolution (SR) models to denoise the watermarked noise, producing the watermarked image. Experiments demonstrate SuperMark's effectiveness.
Strengths: The watermarking framework is training-free. 
An end-to-end pipeline can facilitate the training. 
Experiments show its advantages.
Weaknesses: The paper is written and organized poorly. The abstract is too lengthy, and not concise. The paper has limited novelties and only ensembles existing techniques. Neither a novel algorithm nor a newly constructed module are presented. It is not clear how to seamlessly integrate with various watermark injection methods and pre-trained diffusion-based SR models. The layout is not well, for example, the text font size in all equations looks smaller than that of the variable of the main text. Also, there is no theoretical contribution throughout the paper. The difference from Gaussian shading is missing, which is an important reference in the field.
Recommendation: 3
Reviewer confidence: 5
Soundness/correctness: 2
Clarity: 2
reviews: 
Comments: This paper proposes a post-processing watermarking scheme based on diffusion-based super resolution model. The author attempts to use the inherent robustness of stable diffusion to various distortions and presents a training-free framework SuperMark. SuperMark embeds the watermark into initial Gaussian noise and then applies pretrained super resolution models to denoise the watermarked noise. Experiments demonstrates its efficacy.
Strengths: -	The proposed method is training-free and has lower time and computing cost than other watermarking methods.
Weaknesses: -	This paper uses several overly arbitrary expressions. E.g., in Introduction, “The latter approach is more flexible and general, as it can be applied to both AIGC and real images”. In fact, recent AIGC watermarking research focuses more on in-process watermarking [1][2] because post-processing may be bypassed by malicious users. We believe that SuperMark can be regarded as a new attempt at real-image watermarking, and there is no need to force a connection with AIGC. Moreover, like “diffusion process holds inherent robustness against different distortions” in line 81. We believe that this robustness only applies to stable diffusion, not all diffusion methods. In fact, the experiment in this paper also uses a super resolution method based on stable diffusion.
-	The use of super-resolution in this model is completely unnecessary. Considering the whole process as illustrated in Figure 2, the image in the transmission is still the normal resolution image, not the higher/lower resolution image. In conclusion, we believe that the proposed method is just trying to introduce the diffusion process into the framework, and it does not matter for which task this model is designed. It seriously weakens the contribution of the paper since the performance is mostly brought by diffusion itself, but not any novel design from the proposed method.
-	Performance was just mentioned above, and in fact the performance of this paper is not very good. Both visual quality and decoding accuracy cannot achieve the best (as shown in Table 1). 
-	The paper also contains some parts that are not nice or clear enough. For example, the double-headed arrow between the two "m" in Figure 1 does not make it clear what it means. We assume that this means that the original information and the extracted information need to be as similar as possible, so why is there no double-headed arrow between the original image and the embedded image? In addition, Figure 2 also lacks the necessary legend, which affects the readability of the paper.

[1] Wen, Yuxin, et al. "Tree-rings watermarks: Invisible fingerprints for diffusion images." Advances in Neural Information Processing Systems 36 (2024).

[2] Fernandez, Pierre, et al. "The stable signature: Rooting watermarks in latent diffusion models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.
Recommendation: 3
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2
reviews: 
Comments: The paper presents SuperMark, a novel watermarking framework designed to address the challenges of robustness and fidelity in watermarking AI-generated and authentic content. It leverages insights from diffusion models to create a training-free approach that embeds watermarks into Gaussian noise and subsequently denoises this noise using pretrained super-resolution (SR) models. SuperMark demonstrates strong resilience to various distortions and adaptive attacks while maintaining high visual fidelity.
Strengths: 1. Flexibility and Transferability: SuperMark is adaptable across different datasets, SR models, watermark injection methods, and image resolutions, showcasing its versatility in practical applications.

2. No Fine-tuning Required: The ability to operate without fine-tuning the SR model simplifies the implementation and broadens accessibility for various use cases.
Weaknesses: 1. **Lack of innovation**: Key components of the proposed method, such as watermark injection techniques and the use of diffusion models, are not original. This limits the overall innovation of the research.
2. **Insufficient Contributions**: The claim that the study uncovers a critical insight into the encoder-noise layer-decoder architecture and its trade-offs lacks depth. The acknowledged trade-off between robustness and fidelity is already well-established; the authors do not offer a new perspective or solution to address this issue.
3. **Inadequate Experimental Comparisons**: The comparisons presented in Table 1 are neither significant nor comprehensive. For instance, the average accuracy on Normal Distortions does not show a significant lead over SepMark, and the performance on Adaptive Attacks is not the best. Furthermore, the comparison is unfair as it does not adequately control for visual quality (fidelity) when assessing robustness (accuracy). A more equitable comparison should maintain visual quality at similar levels.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2
reviews: 
Comments: This paper proposes a new watermarking framework with the consideration of both robustness and fidelity. The propose method includes a watermark embedding stage where the watermark is injected into the latent Gaussian noise and then a denoised procedure is followed to obtain the watermarked image; and a watermark extraction stage which involves a distortion process and a DDIM inversion to reconstruct the "watermarked noise", and further extract the watermark information. During these procedures, the models are frozen and no training/update is needed. The authors validate their method on different datasets as well as with different distortions.
Strengths: There are several strengths as I can see.
1) The proposed method does not need to train (even fine-tune) models while it could work relatively well different datasets.
2) The method sounds simple but it works well. The authors also conduct a comprehensive ablation studies on important hyperparameters.
3) The writing and presentation are good. It is easy to follow.
Weaknesses: There are several weaknesses/limitations of this version.
1) The novelty of this version is limited. Using diffusion models to do watermark is not new while this current version does not show sufficient empirical/theoretical evidence to distinguish itself from other works.
2) The experiments results shows that the proposed method is not the best performer in both sides---robustness and fidelity, thus the main contribution of this paper is limited.
3) Which pretrained VAE is used? It look like the authors do not mentioned it in the main paper.
Recommendation: 5
Reviewer confidence: 3
Soundness/correctness: 2
Clarity: 3
reviews: 
Comments: The paper proposed a new pipeline that uses stable diffusion to inject the watermark into the image. The main contribution of this paper is that it claims that it does not need training, and it is robust due to the characteristic of the stable diffusion model. It shows results comparable to those of the SOTA method and can prevent the attack from the VAE or diffusion model.
Strengths: The method is innovative compared to the traditional encoder-decoder-based method. It leveraged the existing generative AI model to inject the watermark into the images, which reduced the additional cost of training a new model for the watermark. And it demonstrates reasonable performance.
Weaknesses: The model does not compare the computation cost of the stable diffusion model with the existing methods. Stable diffusion is well-known for its intensive computation cost, slow inference speed, and large model size. If the stable diffusion model uses 10 times more MACs (Multiply-Accumulate Operations), FLOPs (Floating Point Operations), and large model size, while it performs similarly to traditional models, it diminishes the contribution of this method a lot. The model of adding a watermark is usually small and fast since it happens on the user's end. Most injections are conducted on mobile devices such as phones and laptops. The stable diffusion model will make it impossible to use in most scenarios. Overall, this is a good paper, but the completeness of the paper is not good enough. A lot of extensions mentioned in the paper could be done. This is more like an exploratory.
Recommendation: 5
Reviewer confidence: 3
Soundness/correctness: 2
Clarity: 2