Overfitting: An Unexpected Asset in AI‐Generated Image Detection

Authors: Guanzheng Qin, Yonggang Zhang, Chao Wu, Yiu-ming Cheung, Bo Han, Xinmei Tian
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Overfitting, AI-generated image detection, Generative models

Abstract:
AI-generated images have become highly realistic, raising concerns about potential misuse for malicious purposes. In this work, we propose a novel approach, DetGO, to detect generated images by overfitting the distribution of natural images. Our critical insight is that a model overfitting to one distribution (natural images) will fail to generalize to another (AI‐generated images). Inspired by the sharpness‐aware minimization, where the objective function is designed in a $\min$-$\max$ scheme to find flattening minima for better generalization, DetGO instead seeks to overfit the natural image distribution in a $\max$-$\min$ manner. This requires finding a solution with a minimal loss near the current solution and then maximizing the loss at this solution, leading to sharp minima. To address the divergence issue caused by the outer maximization, we introduce an anchor model that fits the natural image distribution. In particular, we learn an overfitting model that produces the same outputs as the anchor model while exhibiting abrupt loss behavior for small perturbations. Consequently, we can effectively determine whether an input image is AI-generated by calculating the output differences between these two models. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our proposed method.

Decision: reject
Reviewer scores: 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730361163281

thread_type:
official_review

<thread_object>
creator: reviewer_dzhv

<content>
summary:
This paper propose a SAM-based approach to detect generated images by overfitting the distribution of natural images. Additionally, an anchor model are utilized to solve the divergence issue. The authors conduct extensive experiments across multiple AI-image benchmarks and the results demonstrate the effectiveness of the approach.

strengths:
1. This paper is well-written and easy to follow. The structure is clear.
2. The motivation of this paper is similar to Out Of Distribute (OOD) detection, where the model is only trained and fitted on real images. However, detecting generated images by overfitting is enlightening.
3. The methodology is logical, and the motivation is intuitive and profound. The design of sharpening the first derivative of the loss is intriguing.
4. Extensive and effective experiments proving the effectiveness of the proposed method. Ablation experiments validate the hypotheses.

weaknesses:
1. Some technical or motivational clarification is needed. 
2. Some ablation studies are recommended.
Please refer to Questions.

questions:
1.  In line 196, the authors believe that ε follows a Gaussian distribution, I wonder whether there is a reasonable explanation. 
2. Meanwhile, I believe that when ε is designed to be sampled from a Gaussian distribution, the proposed approach will suffer from performance degradation when Gaussian blur is applied to images, as the added noise ε’ and the designed ε belong to the same distribution, resulting in an increase in L(x+ε’). I hope the authors can clarify some of the points.
3. The AUROC and AP metrics are insensitive to classification thresholds. Based on this, they may not fully reflect the generalization ability of the model. I hope the authors can provide more experimental results, such as the AUROC and AP on real samples of LSUN-Bedroom (Table 3) and generated samples from ImageNet settings (Table 1), or the Accuracy with fixed thresholds on different datasets.
4. Why is g(x) designed as multiple trainable convolutional layers? Would other architectures, such as U-Net, yield better or worse results?
5. Will the selection of the training dataset affect the detection performance?
6. Can you evaluate the complexity of the proposed DetGO?

Based on the above Strengths and Weaknesses, I will give a Borderline Score. I will revise my score if the author's rebuttal can provide reasonable explanations.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730637092717

thread_type:
official_review

<thread_object>
creator: reviewer_rbxc

<content>
summary:
This paper formulates the problem of AI-generated image detection as an OOD detection process, and they propose to fit a model solely on real images, without assumptions on and access to any AI-generated data. Specifically, they propose to use a dual-model framework, where the detection is based on the output differences between the overfitted model and a normally-trained, anchor model.

strengths:
- The idea of relying on overfitting is new in the context of AI-generated image detection. 
- The experiments test a diverse spectrum of generated images, including GAN and diffusion images as well as Sora video frames.
- The ablation studies cover almost all important hyperparameters.

weaknesses:
- The largest weakness is that conventional OOD detection methods are not discussed and compared. As the reviewer understands, the authors have formulated the problem of AI-generated image detection as a typical OOD detection process, where the outlier is also normally not exposed to the detector. Therefore, the authors should acknowledge this similarity and test conventional OOD detection methods before proposing a new method. For example, a well-trained model itself can be used to detect OOD samples, here the AI-generated images, based on their output confidence. This also corresponds to the second limitation described in Section 5.

- The underlying assumption that the proposed method can work is that the $\epsilon$ can represent the nature of AI-generated images. However, the authors just simply adopt the Gaussian noise, without any discussion of possible alternatives and impact. More generally, the $\epsilon$ can be treated as a special type of universal fake image, and therefore, it is important to discuss its properties on the final detection performance. In particular, there may exist one perfect type of $\epsilon$ that can generalize to most kinds of unseen fake images.


- More generally, this paper does not discuss the key question: how to trade off the generalization power to unseen real images and unseen AI-generated images. In particular, the authors have already stated that “Our experimental results demonstrate that variations in the intensity of this noise significantly influence the detection performance of the model, as shown in Table 9. When the perturbation is minimal, the model tends to overfit the training set rather than learning meaningful representations of the real images. Conversely, when the perturbation is excessively large, the model only learns to distinguish between real images and pure noise, which also leads to a deterioration in detection performance. ” 

- The authors have not mentioned how to select the threshold, as depicted in Figure 1.

- In Section 4.1, the authors claimed to evaluate the accuracy (ACC). However, the reviewer cannot find any ACC results. AUC and AP are both threshold-independent evaluation metrics, according to previous works [1, 2]. A high value of AUC or AP does not necessarily imply a high value of accuracy. Additionally, considering the previous issue, the threshold of loss is likely to have a significant impact on the accuracy.
[1] Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In CVPR, pp. 24480–24489. IEEE, 2023.
[2] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A. Efros. Cnn-generated images are surprisingly easy to spot... for now. In CVPR, pp. 8692–8701. Computer Vision Foundation / IEEE, 2020.

- The authors do not describe how they use the generative models to generate the test data and how much data are used.

- As mentioned in Section 5, access to high-quality training data may have a large impact on performance. The authors should at least explore such impact by showing the results as a function of the number of training images.

questions:
See the above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730718159418

thread_type:
official_review

<thread_object>
creator: reviewer_z8in

<content>
summary:
This paper proposes DetGO, a novel method for detecting AI-generated images. Instead of trying to identify artifacts in generated images, DetGO overfits a model to the distribution of real images. The core idea is that a model excessively tuned to real images will generalize poorly to AI-generated ones. This approach is inspired by Sharpness-Aware Minimization (SAM), but inverts the logic. While SAM seeks flat minima for better generalization, DetGO seeks sharp minima to prevent generalization. It uses a dual-model framework: an anchor model (a pre-trained DINOv2 model) encodes real images, and an overfitting model is trained to match the anchor model's output on real images while exhibiting high sensitivity to small perturbations. The difference in output between these two models then serves as the basis for distinguishing real from generated images.

strengths:
1. **Novel Approach:** The paper takes an innovative perspective by treating overfitting as an advantage rather than a problem to solve.
Training Efficiency: Only requires natural images for training, eliminating the need for AI-generated images in the training process.
2. **Training Efficiency:** Only requires natural images for training, eliminating the need for AI-generated images in the training process.

weaknesses:
See Questions.

questions:
1. **Self-supervised Backbone:** what is the intuition for using a DinoV2 as a backbone? Why not use some other self-supervised model?
2. **Implementation Details:** How is $\epsilon$ sampled from the Gaussian distribution? What are the parameters of this distribution? How is $\rho$ chosen?
The justification for using two convolutional layers in $g_θ(x)$ is weak. More explanation is needed. 
3. **Novelty of design:** The design change is essentially learning the representation similarity between the original and perturbed images on the DinoV2 representation space. However, the Backbone (DinoV2) and the noise distribution (Gaussian) used are too similar to the existing work RIGID **[R]**. However, the method does not use it as a baseline.
4. The performance of Ojha in Table 1 and Table 3 is lower than its reported performance, please give the implementation details of the baseline methods.

**[R]** rigid: a training-free and model-agnostic framework for robust ai-generated image detection
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents an approach for detecting AI-generated images by training a model to overfit the distribution of natural images. The core idea is based on the insight that a model overfitting to natural images will fail to generalize to AI-generated ones. The proposed method employs a dual-model framework: an anchor model is used to fit the natural image distribution, and an overfitting model is learned to produce the same outputs as the anchor model while exhibiting abrupt loss behavior for small perturbations. AI-generated images are identified by calculating the output differences between these two models.

The motivation to utilize overfitting for AI-generated image detection is innovative, and the approach benefits from only requiring natural images for training. However, despite addressing some of the reviewers' concerns in the authors' responses, the paper still has unresolved issues. These include the trade-off between the model’s generalization capabilities for unseen real images versus unseen AI-generated images, the selection of the threshold, the rationale for adopting a Gaussian distribution, and the resulting vulnerability to Gaussian noise. Although the authors provided some explanations in their rebuttal, these issues were not fully resolved, presenting challenges for the practical application of the method. The paper does not yet meet the acceptance threshold before these significant issues are adequately addressed.