Consistency Verification for Detecting AI-Generated Images

Authors: Yonggang Zhang, Jun Nie, Xinmei Tian, Mingming Gong, Kun Zhang, Bo Han
Primary area: generative models
Keywords: AI-generated image detection, Generative models, Diffusion models, GAN

Abstract:
With the rapid development of generative models, AI-generated images have sparked significant concerns regarding their potential misuse for malicious purposes, highlighting the urgent need for AI-generated image detection. Current methods primarily focus on training a binary classifier to detect generated images. However, the efficacy of these methods is critically dependent on the quantity and quality of the collected AI-generated images. More importantly, they suffer from a generalization challenge: \emph{the literature lacks sufficient exploration of whether a binary classifier trained on images from a specific diffusion model can effectively generalize to images generated by other models.} In this work, we propose a novel framework termed \textbf{con}sistency \textbf{v}erification (ConV) for AI-generated image detection, providing a new approach that detects without requiring AI-generated images. In particular, we introduce two functions and establish a principle for designing them so that their outputs remain consistent for natural images but exhibit signiﬁcant inconsistency for AI-generated images. Our principle shows that gradients of these two functions need to lie within two mutually orthogonal subspaces. This enables a training-free detection approach: an image is identified as AI-generated if transformation along its data manifold results in a substantial change in the loss value of a self-supervised model pre-trained on natural images. This detection framework leads to the unique advantage of ConV over existing methods: \emph{ConV identifies AI-generated images by fitting the distribution of natural images rather than that of AI-generated images.} Extensive experiments across various benchmarks validate the effectiveness of the proposed ConV.

Decision: reject
Reviewer scores: 6, 3, 3

Reviews and discussion:
<thread>
creation date:
1729223050538

thread_type:
official_review

<thread_object>
creator: reviewer_qu6e

<content>
summary:
This paper proposes a consistency verification approach that does not require AI-generated images for training. The key component of the design is two functions: the representation function f1 based on the self-supervised pre-trained model (e.g., Dinov2). The transformation function h is also based on the self-supervised model's data augmentation. The author hypothesizes that the deviation after the transformation function in the representation space will be smaller in the real image, and larger in the AI-generated images.

strengths:
1) I like the setting where the pre-trained foundation model is used for the prediction instead of exhausting training on AI-generated or real data.
2) Experiment on SORA benchmark is also interesting.

weaknesses:
1) I feel the novelty of the paper is weak. The paper borrows lots of ideas from the RIGID, the baseline paper. RIGID measures cosine similarity between the dinov2 representation of the original image and the dinov2 representation of the deviated image through Gaussian Noise Augmentation. This paper does the same except changing cosine similarity to l2 distance, and Gaussian augmentation to training augmentations (e.g. Gaussian Blur, Colorjitter). Hence, I feel like this paper is just an ablation study of the RIGID paper, and the contributions should go to the original paper (e.g. applying Dinov2, and proposing a similar consistency function).

2) Experiment results also feel weak. The proposed method does not outperform RIGID in all cases. Aligned to 1), I feel like this paper should at least outperform RIGID in all experiments, not only on the average but also in all AI-generated data. Maybe more deeper analysis will be helpful.

3) Section 2 is hard to read. 2.1~2.3 seems to overexplain general ideas of manifold hypothesis, which is not new. In my opinion, this is just a longer extraction of "This model is trained on the real data so it will show robust output on the real data when the training data augmentation is applied". The final consistency term form has been applied to various binary classification tasks ([1],[2]). Maybe a deeper explanation of why this explanation is necessary will be helpful.

4) The experiment results of RIGID are not the same, but deteriorated from the original paper, especially in the ImageNet dataset. I wonder why such a discrepancy occurs, given that the performance gap between the proposed method and RIGID is not so large. This is more puzzling since the performance of RIGID reported in this paper should be improved by using larger $n=20$. I hope the elaboration on this result will be beneficial to resolve my concerns.

Overall, I find the paper's novelty weak since RIGID is also based on the distance between the original dinov2 representation and the dinov2 representation of the perturbed data, given that RIGID is the main baseline of this paper.

[1]  Likelihood Ratios for Out-of-Distribution Detection, NIPS 2019

[2] Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder, NeurIPS 2020

questions:
See weakness for the main issues. 

1) I did not quite understand of the data processing procedure of the GenImage dataset on Page 15. I feel like the authors cropped the AI-generated images in 224 by 224 dimensions, but treated ImageNet data differently by resizing first, and then cropping it to 224 by 224 size.
Did I understand it correctly? If it is, why the processing method is different in AI-generated and real images?

2) I am curious how AEROBLADE will perform in the GenImage dataset. (The table has typos by the way)

3) While this is minor, I would like to see the runtime analysis of the proposed method, given that the paper utilizes lots of ensembles.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730701207515

thread_type:
official_review

<thread_object>
creator: reviewer_obbe

<content>
summary:
This paper presents a AI-generated images detection method called Consistency Verification (ConV). Unlike traditional approaches that rely on binary classifiers trained on both natural and AI-generated images, ConV is training-free. The key innovation lies in introducing two functions that remain consistent for natural images but show inconsistency for AI-generated images.

strengths:
Training-Free**: It eliminates the need for large datasets of generated images and the computational cost associated with training a classifier.
Theoretical support: The method's validity is explained the orthogonality principle to some extend.**

weaknesses:
See questions.

questions:
1. *Limited Theoretical Justification:** While the orthogonality principle provides some intuition, a more rigorous theoretical analysis of why and how the method works would strengthen the paper. For instance, There is no explanation for why the augmentations during DinoV2 training are orthogonal to its standard output? Moreover, the theoretical analysis in the paper does not provide any guidance for the choice of augmentations.
2. **Lack of Novelty:** The method proposed in this paper is based on Baseline RIGID, but it only replaces Gaussian noise with training enhancements, without much insightful design. In addition, it does not explain why these enhancements are better than Gaussian noise.
3. **Limited Implementation Details:** The paper does not give specific details of the data augmentation used, including the parameters of the data augmentation used, and whether a single augmentation or a combination of multiple augmentations is used.
4. **Single Backbone Model:** The paper only tested on DinoV2 and not on more (self-supervised) models. Does this mean that the scope of application of this method is limited?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730719571938

thread_type:
official_review

<thread_object>
creator: reviewer_t4ze

<content>
summary:
This paper presents a framework, termed Consistency Verification (ConV), aimed at detecting AI-generated images. Different from traditional binary classifiers that require extensive collections of both natural and AI-generated images, ConV operates solely on the distribution of natural images. The authors introduce two functions whose outputs are consistent for natural images but highly different for AI-generated images.

strengths:
- The underlying mathematical intuition of this paper is well-presented. 
- The method looks novel to me and its effectiveness has been supported by a collection of experiments.

weaknesses:
- **Theoretical Limitation**: Although the orthogonality principle is empirically validated, a formal proof of the convergence of ConV’s generalization risk is not provided, which could strengthen the theoretical grounding.

- **Clarity in Wording**: In line 135, the phrase, *"..., humans know that if a natural image $x_n$ captures the same content as $x_g$ , they are distinguishable in certain ways,"* could cause confusion. The authors might consider revising this to *"...even if a natural image $x_n$ captures similar content to $x_g$"* to clarify that subtle differences still make AI-generated and natural images distinguishable.

questions:
As shown in Table 6, it appears that ConV is sensitive to the initialization function. ConV instantiated by CLIP performs much worse than DinoV2. Is there an approach to overcome this limitation?
</content>
</thread_object>

</thread>


Metareview:
metareview:
The recommendation is based on the reviewers' comments, the area chair's evaluation, and the author-reviewer discussion. 

While the reviewers see some merits in deriving a statistic using a backbone model for consistency verification and the detection of AI-generated images, this submission should not be accepted in its current form due to several fundamental issues, as pointed out by the reviewers, including

- Technical novelty compared to existing methods, such as AeroBlade and RIGID. Notably, while te proposed framework is quite general, in terms of the actual implementation of Conv, it shares some similarities to RIGID (e.g., using DinoV2 as the backbone).
- Limited performance improvement: Compared to the baselines, the improvement is marginal and sometimes does not outperform the baselines.
- Theoretical contributions need to be reinforced.


During the final discussion phase, all reviewers agreed to reject this submission.

I hope the reviewers’ comments can help the authors prepare a better version of this submission.