Epistemic Uncertainty for Generated Image Detection
Authors: Jun Nie
Yonggang Zhang
Tongliang Liu
Yiu-ming Cheung
Bo Han
Xinmei Tian
Venue: NeurIPS 2025 poster ()
Keywords: AI-generated image detection
Uncertainty Estimation
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: We introduce a novel framework for AI-generated image detection through epistemic uncertainty, aiming to address critical security concerns in the era of generative models. Our key insight stems from the observation that distributional discrepancies between training and testing data manifest distinctively in the epistemic uncertainty space of machine learning models.
 In this context, the distribution shift between natural and generated images leads to elevated epistemic uncertainty in models trained on natural images when evaluating generated ones. Hence, we exploit this phenomenon by using epistemic uncertainty as a proxy for detecting generated images. This converts the challenge of generated image detection into the problem of uncertainty estimation, underscoring the generalization performance of the model used for uncertainty estimation. Fortunately, advanced large-scale vision models pre-trained on extensive natural images have shown excellent generalization performance for various scenarios. Thus, we utilize these pre-trained models to estimate the epistemic uncertainty of images and flag those with high uncertainty as generated.
 Extensive experiments demonstrate the efficacy of our method.

Epistemic Uncertainty for Generated Image Detection
Jun Nie!.2 Yonggang Zhang?2 2.4
Tongliang Liu
Yiu-ming Cheung?
Bo Han Xinmei Tian
1 MoB Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China
Hong Kong Baptist University Syynney AI Centre, The University of Sydney 'The Hong Kong University of Science and Technology
Abstract
We introduce a novel framework for Al-generated image detection through epis- temic uncertainty, aiming to address critical security concerns in the era of gen- erative models. Our key insight stems from the observation that distributional discrepancies between training and testing data manifest distinctively in the epis- temic uncertainty space of machine learning models. In this context, the distribution shift between natural and generated images leads elevated epistemic uncertainty in models trained on natural images when evaluating generated ones. Hence, we exploit this phenomenon by using epistemic uncertainty as a proxy for detecting generated images. This converts the challenge of generated image detection into the problem of uncertainty estimation, underscoring the generalization performance of the model used for uncertainty estimation. Fortunately, advanced large-scale vision models pre-trained on extensive natural images have shown excellent generalization performance for various scenarios. Thus, we utilize these pre-trained models to estimate the epistemic uncertainty of images and flag those with high uncertainty generated. Extensive experiments demonstrate the efficacy of our method. Code is available at http:///thtttuboooo/////or//////
Introduction
Recent advancements in generative models have revolutionized image generation, enabling the production of highly realistic images (Midjourney 2022] Wukong 2022 Rombach et al. (022).
Despite the remarkable capabilities of these models, they pose significant challenges, particularly the rise of deepfakes and manipulated content. The high degree of realism achievable by such technologies prompts urgent discussions about their potential misuse, especially in sensitive domains such as politics and economics. In response to these critical concerns, a variety of methodologies for detecting generated images have emerged. A prevalent strategy treats this task as binary classification problem, necessitating the collection of extensive datasets comprising both natural and Al-generated images train classifiers (Wang et al. 2020).
While existing methods have demonstrated notable successes, unseen generative models Wang et al. (2233 pose challenges for them in generalizing to images with distribution shifts. One promising avenue to enhance the robustness of detection capabilities involves constructing more extensive training sets by accumulating diverse array of natural and generated images. However, these attempts are computationally intensive, requiring substantial datasets for effective classification. Besides, maintaining robust detection necessitates continually acquiring images generated by the latest generative models. And when the latest generative models are not open-sourced, acquiring a
*Conresooneence to: Xinmei Tian (xinmei@ ustc.edu.cn)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Natural image
Fake image
Natural bird
Natural
Natural boat
Fake bird
Fake
Fake boat
Figure I:
Models trained on large number of natural images are capable of distinguishing between natural and generated images.
large number of generated images to train classifiers is challenging. This highlights the urgent need for novel framework to detect generated images without reliance generated images.
recent work (Tan et al., 2024) shows that features extracted by ViT of CLIP (Radford al. 2021 can be employed to separate natural and Al-generated images, motivating an effective approach to detecting images by training binary classifier in the feature space of CLIP. This provides a promising direction to explore the possibility that large-scale foundational models already have the ability to capture the subtle differences between natural images and Al-generated images. As shown Figure [ even for images sampled from the same class, there are large distributional discrepancies in the feature space of DINOv2. This observation is consistent with an important metric for evaluating generative models-the FID score. FID score measures feature distribution discrepancy between natural and generated images on the Inception network (Szegedy et al.) 2015). A FID score indicates that there is difference between the two distributions. However, even these simple networks such as Inception v3, advanced generative models like ADM still achieve an FID score of 11. 11.84, not mention that powerful models such as DINOv2, we observe significant feature distribution discrepancy. However, despite these distributional differences, the inherent diversity of natural images precludes direct modeling of their distribution. This challenge motivates to consider an alternative approach to reflect the distributional differences between natural and generated images.
In this study, exploit the distributional dis- parity between natural and generated images by leveraging epistemic uncertainty differen- tiate them. Epistemic uncertainty quantifies model's 'ss confidence in its predictions, reflecting its knowledge of the data distribution. As the volume of training data increases, model's epis- temic uncertainty for in-distribution (ID) data di- minishes. For foundational model pre-trained on an extensive dataset of natural images, we posit that its epistemic uncertainty is lower for
Tri GLA
90
AUROC (%)
ABX
1.50
18.35
92.51
99.54
20.44
18..00
13205
FID
Figure 2: WePe reflects the distribution discrep- ancy between Al-generated and natural images. natural images compared to generated ones, due to the model's alignment with the natural distribu- tion. This perception is consistent with recent studies (Snock al. 2019 Schwaiger al. 2020)) which indicate that models tend to show increased uncertainty for out-of of-itttributtion (OOD) samples. The challenge comes from efficiently obtaining the uncertainty of the model on the test samples. Classical approaches include Monte- -Calo Dropout (MC-Dropout) (Gal and Ghahramani 2016) and Deep Ensembles (Lakshminarayanan et al.) (0177). However, our attempts, MC-Dropout obtains sub-optimal results (in Table 2) and it is challenging to train multiple large models independently for model ensemble. Instead, our theoretical results show that this uncertainty can well captured by perturbing the weights of the models. As shown in Figure 3 when a moderate level of perturbation is applied, the natural image has consistent features on the model before and after the perturbation, but the generated image has large differences in features the model before and after the perturbation.

---

Natural image cleun medel Fake image clean modde oNatural inage noise modd Fake image noise model
Noise Level
0.01
0.05
0.2
Figure 3: Natural and generated images exhibit distinct sensitivities perturbations in model weights. moderate perturbation (0.1) results in minimal changes to the features of the natural image, while the generated image shows significant differences.
In this paper, we propose novel method for Al-generated image detection by weight perturbation (WePe). Our hypothesis is that the model has greater uncertainty in predicting the OOD sample compared to the ID) sample, and that this uncertainty can be expressed through sensitivity weight perturbations. For a large model trained on large number of natural images, the natural images can be considered ID samples, while the generated images are considered OOD samples. Thus, the sensitivity of the samples to the weight perturbation of the large model can be important indicator to determine whether the sample is generated by the generative models or not. As shown in Figure 2 we calculate FID scores between different types of generated images and natural images DINOv2. shows the performance of WePe strongly correlates with FID score, indicating the effectiveness of WePe in detecting distribution discrepancy.
We summarize our main contributions follows:
We provide perspective detect Al-generated images by calculating predictive uncer- tainty. This is built on the fact that natural and generated images differ in data distribution, making possible employ uncertainty represent the distribution discrepancy.
Since the data distribution discrepancy between generated and natural images is reflected in feature distribution discrepancy on the vision model, we propose using large vision model to compute prediction uncertainty to highlight this difference. The intuition that large vision models are merely trained on natural images, making it possible to exhibit different uncertainties about natural and generated images. We capture this uncertainty through weight perturbation, enabling effective detection. (Eq. [3).
Extensive experiments across multiple benchmarks show the proposed method surpasses existing methods, highlighting the efficacy of uncertainty in detecting Al-generated images.
2 Preliminaries
2.1
Al-generated image detection
The task of Al-generated image detection involves classifying a test image x as either a natural image produced by generative model. Traditional supervised learning approaches rely curated dataset comprising both natural and Al-generated images to train a feature extractor and binary classifier. This process formalized follows.
Let D = (x]: ......, , x denote a set of NO Al-generated images, each labeled as y = (generated), and D (xt.x]. x2, x denote set of N natural images, each labeled as y = (natural). The objective is learn feature extractor F(;; 0F), parameterized by 0F, and binary classifier (c;; 0D), parameterized by 0p. by minimizing a loss function E(-) over the training data:
F, = arg min (F( 8p,8p
where y {0, represents the ground-truth label of the input image

---

Upon completion of training, the feature extractor and classifier are used to compute a decision score for test image x, defined as S(x) DF (x: 0p) The source of the image is determined by comparing this score to a predefined threshold T:
(x)d generated, if S(x) < T,
natural, otherwise.
The robustness and generalization of such methods depend critically the size and diversity of the training dataset. To address this, techniques such as CNNspot employ sophisticated data augmentation strategies, including Gaussian blur and JPEG compression, enhance the variability of the training data. In contrast, UnivED adopts a different approach by utilizing the CLIP model as a fixed feature extractor and training only a single linear classification layer. Despite these innovations, both methods exhibit limited generalization when applied to images produced by unseen generative models.
2.2 Bayesian neural networks and uncertainty estimation
Traditional neural networks map inputs to outputs through parameterized function f(x: 0). where weights / are optimized deterministically. offering no uncertainty quantification. Bayesian Neural Networks (Neal 2012) address this by modeling 0 random variables with prior distribution p(0). Given a dataset D, the posterior is computed via Bayes' theorem:
D) p())0)p(0)
(3)
((D)
For a new input x", the posterior predictive distribution:
p(y"Ix" 111xx"))) .D)
p(y"|x".0) 0)) di
(4)
Alcatoric
Epistemic
captures prediction uncertainty.
Exact posterior inference is intractable due to the high dimensionality of 8. Monte Carlo methods, such as MC Dropout (Gal and Ghahramani] 2016), approximate the posterior by using dropout during both training and testing and generating multiple stochastic predictions. Uncertainty is quantified through the variance or entropy of these predictions, reflecting two types:
Epistemic Uncertainty: Uncertainty in model parameters, encoded in p(w(D), which decreases with more data.
Aleatoric Uncertainty: Inherent noise in the data, modeled by the likelihood p(y|x x w).
Uncertainty based Al-generated image detection
3.1 Motivation
Uncertainty estimation is well-established field critical to deep learning practitioners, as facilitates explicit handling of uncertain inputs and edge cases (Durasov al. 2021 Everett al. 2022). By quantifying model confidence, practitioners can make informed decisions, such as deferring high- uncertainty inputs to human evaluation classification tasks, thereby improving model reliability and robustness. In this study, we do not propose a novel method for uncertainty estimation. Instead, we focus on distinguishing natural from generated images by leveraging epistemic uncertainty as discriminative metric. Our approach is grounded in the well-established principle that (Lahlou et al. 2023
Gal and Ghahramani [016]:
Epistemic uncertainty, which reflects a model's 's lack of knowledge, can be reduced by acquiring additional information.
Formally, we adopt Bayesian framework to quantify epistemic uncertainty through the posterior distribution over model parameters:
p(0 D) p(D(0))(()
(5)
p(D)

---

where D = y:) denotes the training dataset, with samples drawn i..d. from p(y [,y/x,00 8o). Epistemic uncertainty captured by the posterior variance:
() = Ee~p(8 D) [0 (0- {[
(6)
Under regularity conditions, the Bernstein-von Mises theorem Van der Vaart 2000 establishes that, as the sample size N + 00,
1 N ((0))=)) (( N (ŌN:
where y is the maximum likelihood estimate, and T(0p) EE
( log (yylx00
represents
80
0=00
the Fisher information matrix. Here, fn approximates the true parameters of the natural image distribution, as learned by a pre-trained vision model. Consequently, Var(0)D) x N indicating that epistemic uncertainty decreases as the training data volume increases.
For test input x" drawn from distribution Presa (x) # Prriin (x), the predictive distribution is given by:
ply |x" D) -)= ply xx ()) de
8
This distribution exhibits elevated variance due to the misalignment between p(0)D) and the parame- ters required to model Pres (x). The predictive variance can be decomposed
Var(y"Ix* D) = Eg-p((/D) [Var(y"|x*, ,)] Varg-p(# D) [E [x*, 0)],
where the second term, Varg-p(0 [B [x(y"Ix", 9)], corresponds to epistemic uncertainty and domi- nates for out-of-distribution (OOD) samples. Under distribution shift, the Fisher information I(fo), defined with respect to Purain (x). is misaligned with Pxst(x), limiting the reduction of posterior variance and resulting in persistently high epistemic uncertainty (Snoek al. 2019).
Based on the above analysis, as well as the phenomenon of distributional shifts between natural and generated images on the foundational model (see Figures I and 88. we believe that the difference in the epistemic uncertainty of the foundational model on natural and generated images can be valid indicator to distinguish between them.
3.2 Uncertainty estimation via weight perturbation
Classical methods of epistemic uncertainty estimation, such as Deep Ensembles and MC Dropout, can simply be viewed as using variance of multiple prediction results as an estimation of uncertainty u(x):
xxx) = - - (x()), u(x) = =²² -²= (0.(x) xxx))²
(10)
where, ÿ. denotes the 1-th prediction. The multiple predictions of Deep Ensembles come from multiple independently trained neural networks, while the multiple predictions of MC Dropout come from the use of dropout during inference, which can be regarded as multiple prediction using neural networks with different structures.
In this study, we leverage DINOv2 for uncertainty estimation, capitalizing on two key advantages. First, DINOv2 is pre-traine trained on an extensive datasets, endowing it with robust generalization capabil- ities. This broad pre-training enables the model to effectively capture the distributional characteristics of natural images, thereby enhancing the generalizability of our proposed method across diverse scenarios. Second, as DINOv2 is exclusively pre-trained on natural images, it exhibits distinct un- certainty profiles when processing natural versus Al Al-generated images. This differential uncertainty provides a reliable basis for distinguishing between the two image types. DINOv2 a seff-supervised learning model, employing a student-teacher framework where the student model 0 is trained to align with the representations of the teacher model 0;. Consequently, we utilize the feature similarity between the embeddings produced by the teacher and student models as the prediction:
((x) = f(x: 0) f(x; 0z).
(11)
where f(x; 0) denotes the I.2-normalized features of an input image when inferring with the parameter O. However, estimating uncertainty with DINOv2 presents challenges in obtaining multiple
5

---

Denaty
generated Images Images
Density
genereereed Images
Al-generated Images
Images
Natural Images
0.85
0,90 Cosine Similarity
0.95
1,00
0.80
0.85
0.95 Cosine Similarity
0.90
1,00
0.85
0.95 Cosine Similarity
0.90
L.0
(b)
Figure 4: Comparison of cosine similarity between features on original and perturbed models. The generated images are from: (a) ADM, (b) BigGAN, and (c) DDPM.
predictions. First, as DINOv2 does not employ dropout during training, MC Dropout may yield suboptimal results (see Table 2 Second, training multiple DINOv2- models is computationally infeasible, rendering Deep Ensemble impractical. Instead, we use an alternative approach, weight perturbation, obtain multiple predictions. Specifically, let 02 denote the perturbed parameters of the student network, according to Eq. 10 the predictive uncertainty u(x) can be calculated by,
f(x; 0₂) ((x;0)),
u xx)
Oxxx))
(12)
n
However, we cannot access the teacher model O,, making it challenging to calculate the uncertainty. Moreover, even if is available, introducing two models for calculation leads to low computation efficiency. Fortunately, we can calculate an upper bound of u(x). This can be formalized by,
xxx)=
1 71
f(x: 0x) - Σ f(x:0)) [x)(x = =2 n Σ
(13)
where 0 denotes the parameter of student model before injecting perturbation, and we leverage an unbiased assumption that the expectation Ea,[(x; 0z) approaches the feature f(x; 0) extracted by the non-perturbed parameter Eq. 13 provides a simple approach to calculate the uncertainty without needing a teacher model used in the training phase of DINOv2. We provide analysis of the validity of this upper bound Appendix D The insight of Eq. 13 is intuitive. Specifically, if an image x causes high feature similarity between the original and perturbed parameter, the image leads to low uncertainty and is more likely to be a natural image.
3.3 Theoretical analysis of the effectiveness of weight perturbation
In this section, we present theoretical analysis elucidating why weight perturbations effectively distinguish between natural and Al-generated images. We establish formal metric to quantify the sensitivity of neural network feature representations to parameter perturbations and demonstrate its differential behavior across natural images and Al-generated images.
To formalize this notion, we introduce the following definition of perturbation sensitivity:
Definition 3.1 (Perturbation Sensitivity). For a neural network f - Rdd, parameterized by 0 E R., which maps an input x X' to feature vector f(r; 0), the sensitivity to parameter perturbations is defined as:
sen(2) [[Vo[(z;))
(14)
where f 0) denotes the Jacobian matrix of the feature mapping with respect to 0, and || ||r represents the Frobenius norm.
This metric captures the magnitude of variation in the feature representation induced by infinitesimal changes in the model parameters, providing robust measure of sensitivity to weight perturbations.
We proceed establish the differential sensitivity of the model across distributions through the following theorem:
Theorem 3.2 (Differential Sensitivity).. Let a neural network f(x; be trained on large amount of natural images sampled from natural image distribution D): T {(x). (2²,, (,,,))),,..,. ".....(2"
y²),.., (I.,,,",,, y)"} ~ ~D' The expected sensitivity of the feature representations to parameter perturbations is lower for inputs drawn from D compared to those drawn from generated image distribution Dd, generated by generative models. Formally:
EpNDi [zen(x)]= S Ep-DO [sen(x)].
(15)

---

Table 1: Al-generated image detection performance ImageNet. Values are percentages. Bold freeingggeeeeeeee free methods separately.
numbers are superior results. We compare training methods and training
Methodh
ADM
ADMG
LDM
DIT
BirGAN
Maddl
GiLIGNN
StlleAAN XL
RQ-Tussiones AF
Mask GIT
Awerage
AUROC
AUBOC
AUROC
AURIC
AUROC
AUBOC
AUROC
Tranning Methosds
CNNNget
02:25
65.13
63.33
02.27
63.
64.51
85
61.1 1616
85.71
$4.95
74.85
71.45
65.41
68.67
0185
02:91
61:09
67.04
66.3
NPS
15.88
80.56
54.34
79.29
91.88
86.76
96.15
81.25
89:23
$6.46
82.21
78.20
84.13
78:33
80.21
73.21
89
84.15
$5.00
10.80
PaschCrall
$1.85
79.65
70.83
69:36
65.47
65. 19
75.38
73.29
92-85
59.25
08.55
02.91
95:33
96.25
91.28
91.41
92
92.17
96.13
$6.55
FatFomer
91.77
92.58
93.58
98.11
Warh"
93,89
11.21
00
81.94
99.33
99.32
V8.11
97.34
99.63
95.37
04.33
TEamE fic Mathasds
AEROBLABEE
55.81
02:05
With
$9.79
87.32
83.20
28,30
78.47
13.50
77.13
71.21
94.34
9A.64
92.15
90.29
93.86
92.86
93.50
91.47
89.55
86.25
$7.99
85.04
WePe
Welk
Wefe
HIGID
RIGID
EIGID
NPW
NTR
NPP
(a)
(h)
(c)
Figure 5: Performance varies with perturbation intensity under different degradation mechanisms,
including (a) JPEG compression, (b) Gaussian blur, and (c) Gaussian noise.
This theorem asserts that feature representations of natural images, optimized through training on DJ, exhibit greater robustness to parameter perturbations than those of Al-generated images, which lie outside the training distribution. This differential sensitivity underpins the efficacy of weight perturbations in Al-generated image detection, enabling the model to distinguish natural images from Al Al-generated images. A rigorous proof of this theorem is provided in Appendix
3.4 Sharpening discriminative uncertainty through precise calibration
The original DINOv2 model reveals notable uncertainty disparities between natural and generated images. With access to the training dataset, fine-tun tuning can amplify this uncertainty gap, enhancing detection performance. Specifically, we introduce the following loss function:
(())
xx
[f(x; O)T f(x: 0*)] ExEx [f(x; 0) fxx;0)),
(16)
where 0" 0 P represents a perturbed parameter set, with P being randomized perturbation matrix drawn from a predefined distribution. This formulation builds on the uncertainty upper bound established in Eq. 13 to guide optimization. The loss encourages high confidence for natural images (X²) and low confidence for generated images (X"), thereby sharpening the model's ability to distinguish between the two classes. We call this method WePe".
4 Experiments
4.1 Experiment setup
Datasets and evaluation metrics. Following previous works (He et al. 2024 Zhu et al.
2023) we
evaluate the performance of WePe on ImageNet (Deng al. 2009). LSUN-BEDROOM (Yu et al.)
2015), GenImage (Zhu et al. 2023) and DRCT-2M (Chen et al. 2024), with the following evaluation metrics: (1) the average precision (AP), (2) the area under the receiver operating characteristic curve (AUROC) and (3) the classification accuracy (ACC).
Baselines. Following RIGID, take both training-free free methods and training methods baselines. For training-free methods, we take RIGID (He et al., 2024) and AEROBLADE (Ricker et al.)
2024)
baselines. For training methods, take DIRE (Wang et al. 2023). CNNspot Wang et al.
2020
Ojha (Ojha et al. 2023). PatchCraft (Zhong et al.
2023), FatFormer (Liu et al.. 2024). DRCT (Chen
al. 2024) and NPR (Tan al.) 2024) baselines. Besides, Genlmage, we also report the result
of Frank (Frank et al. 2020), Durall (Durall et al. 2020), Patchfor (Chai et al. 2020). F3Net (Qian
al.) 2020), SelfBlend (Shiohara and Yamasaki 2022). GANDelection (Mandelli et al. 2022).
LGrad (Tan et al. 2023), ResNett50 (He et al.) 2016 6). DeiT-S (Touvron et al. 2021), Swin-1
LLu
al. 2021), Spec (Zhang et al. 2019). GramNet (Liu et 2020).
7

---

AUROC
33
(%)
AUROC
-3
AP
13
AUROC
AUROC
0025
150
0.300
10
Variunce Gaussian noise
Perturbed blocks
Figure 6: Performance varies with variance
Figure 7: The effects of disturbed blocks.
Experiment details. To balance detection performance and efficiency, we use DINOv2 ViT-L/14. Due to the randomness of the added noise, we report the average results under five different random seeds and report the variance in Figure TO In our experiments we find that perturbing the high layers may lead large corruption in the features of the natural images, resulting in sub-optimal results. Therefore, we do not perturb the high-level parameters. In DINOv2 ViT-L/14, the model
Table 2: The effect of perturbation type.
Noise
AUROC
AP
Gaussian noise
87.99
85.04
Uniform noise
89.06
86.32
Laplace noise
87.13
84.22
MC Dropout
81.63
79.71
has 24 transformer blocks, and we only perturb the parameters of the first 19 blocks with Gaussian perturbations of zero mean. The variance of Gaussian noise is proportional to the mean value of the parameters in each block, with the ratio set to 0.1.
4.2 Results
Comparison with other baselines. We conduct full comparative experiments on four benchmarks mentioned. As shown in Table I 10 II and WePe achieves good detection performance Ima- geNet, SSUN-BEDROOM, DRCT-2M and Genlmage. Experimental results show the effectiveness of uncertainty estimation in detecting Al Al-generated images. It is worth noting that without any training, and relying only on the nature of the pre-trained model itself, WePe shows the potential to differentiate between natural and generated images. When further trained to amplify the uncertainty disparity between natural and generated images, WePe achieves superior average performance compared to other methods and demonstrates robust generalization capabilities across diverse datasets. To further illustrate the effectiveness of our method, we count the difference in feature similarity between natural and generated images on the pre- and post-perturbation models. As shown in Figure 4 small perturbation of the model has less effect on the natural images than on generated images, resulting in higher feature similarity before and after the perturbation. The discrepancy effectively distinguishes the natural image from the generated image.
Comparison under attacks. In real-world scenarios, malicious actors may attempt modify generated images to evade detection. To assess the robustness of the model under such conditions, simulate attack by introducing Gaussian noise (with variance of 0.1) the generated images. Then the clean natural image and the attacked generated image are fed into the detector to determine whether the two be reliably distinguished. Beyond spatial domain attacks, also investigate attacks in the frequency domain, given the frequency differences between natural and generated images. As in Table several detectors, such NPR and RIGID, vulnerable this simple form of attack. In contrast, WePe, which identifies differences in the distributions of natural and generated images, demonstrates resilience to attacks, the added noise further accentuates these distributional disparities, thereby enhancing WePe's capacity to differentiate between the two types of images.
4.3 Ablation study
Table 3: The effect of models.
In this section, we perform ablation experiments. Unless oth- erwise stated, experiments are conducted on ImageNet.
model
AUROC
DINOv2: VIT:/14
72.83
71.63
Robustness to Image Perturbations Robustness various perturbations is critical metric for detecting generated images.
DINOv2: ViT-B/14
81.82
80.64
DINOv2: VIT-L/14
87.99
85.04
In real-world scenarios, images frequently undergo perturba-
DINOV2: VTT-g/14
84.92
81.83
tions that can impact detection performance. Following RIGID, we assess the robustness of detectors against three types of perturbations: JPEG compression (with quality parameter q)), Gaussian blur (with standard deviation σ), and Gaussian noise (with standard deviation o). As illustrated Figure training- methods generally exhibit superior robustness compared to grainingbbase methods, with our method achieving the best overall performance.
CLIP: VIT-L/14
84.82
84.20
8

---

Selecting which layers' parameters to perturb? As shown in Figure 7 which layers' parameters to perturb would achieve good performance. The horizontal coordinates in the graph indicate that the first k blocks are perturbed, not the kth block. The experimental results exhibit that our method obtains good performance when the parameters of the first to the first 20 blocks are chosen to be perturbed. This demonstrates the robustness of our method. In practice, we can select the layers to be perturbed by a small set of natural and generated images. And when the generated images are not available, we can also use the probe to determine which layers are perturbed using only the natural image. We describe our method in Appendix O
we explore the choice of
The effect of perturbation type. In experiments, model parameters are perturbed with Gaussian noise. We further explore other perturbation, such as adding uniform or Laplace noise to the weight. Besides, we also explore MC Dropout, i.e., using dropout during inference, As shown in Table 2 all three weight perturbation methods achieve good performance, and outperform MC Dropout.
The impact of the degree of perturbation. As shown in Figure 6 we explore the effect of the degree of perturbation on the performance of WePe. It can be seen that WePe is quite robust to the level of perturbation noise. It is only when the noise is very large or very small that it leads to a degradation in performance. When the noise level is small, the features obtained before and after the model perturbation are extremely similar, while when the noise level is very large, the features obtained before and after the model perturbation are extremely dissimilar, and these two cases will result in the inability to effectively differentiate between natural and generated images.
The effect of models. In our experiments, mainly used DINOv2 ViT-L/14 to extract features. We further explore the effect of using other models of DINOv2, including ViT-S/14, ViT-B/14, 14, and ViT--/144. In addition to this, the performance on CLIP is not as good as on DINOv2. We hypothesize that the difference comes from the training approach of these models. CLIP learns features using image captions as supervision, which may make the features more focused on semantic information, whereas DINOv2 learns features only from images, which makes it more focused on the images themselves, and thus better able to capture subtle differences in natural and generated images.
conduct experiments
the CLIP:
As shown in Table 3
5 Related work
AI-Generated images detection. Recent advancements in generative models, such as those by (Brock al. 2019 Ho et al.) 22020), have led to the creation of highly realistic images, highlighting the urgent need for effective algorithms to distinguish between natural and generated images. Prior research, including works by (Frank et al. 22020] Marra et al. 2018), primarily focuses on developing specialized binary classification neural networks to differentiate between natural and generated images. Notably, CNNspot (Wang et al. 2020) demonstrates that a standard image classifier trained on ProGAN can generalize across various architectures when combined with specific data augmentation techniques. NPR (Tan et al. 2024) introduces the concept of neighboring pixel relationships to capture differences between natural and generated images. PatchCraft (Zhong et al., 2023) proposes an efficient Al- generated image detector by exploring texture patch artifacts. FatFormer (Luu et al. 2024) introduces forgery-aware adaptive Transformer that adapts a pre-trained CLIP model to effectively discern and integrate local forgery traces from both image and frequency domains. DRCT (Chen et al.) (2024) presents Diffusion Reconstruction Contrastive Training framework improve the generalizability of synthetic image detectors by training them to distinguish real images from their high-quality cittusionbbasedd reconstructions. Although these methods show superior performance on generators in the training set, they often do not generalize well to unknown generators. In addition to this, training- based methods are susceptible to small perturbations in the image. For this reason, recently some training-fire -free methods have been proposed. AEROBLADE (Ricker et al. (024) calculates the reconstruction error with the help of the autoencoder used in latent diffusson models (Rombach et al. 2022). RIGID (He et al. 2024) finds that natural images are more robust to small noise perturbations than generated images the representation space of the vision foundation models and exploits this property for detection. However, these methods usually make overly strong assumptions about natural or generated images, leading to insufficient generalization. detection method through uncertainty analysis. Based on the widespread phenomenon that generated images have greater uncertainty than natural images on models trained with natural images, our method achieves robust detection performance.
our paper, we propose a training- free
9

---

Uncertainty estimation. Uncertainty estimation in machine learning has seen significant advance- ments in recent years. (Gal and Ghahramani 2016) introduces Monte Carlo Dropout (MC Dropout), which uses dropout at inference to estimate uncertainty from the variance of multiple predictions. (Lakshminarayanan et al. 2017) develops deep ensembles, demonstrating improved uncertainty estimates through training multiple model independently with different initializations. Recent work by (Snoek et al., 2019) analyzes the calibration of uncertainty in deep learning models, highlighting the importance of reliable uncertainty measures. Additionally, (Guo et al. (017) explore the use of temperature scaling to enhance the calibration of model predictions. (Blundell et al. 2015) introduce Bayes by Backprop, method for estimating weight uncertainty in neural networks by modeling the posterior distribution over weights using variational inference, improving model generalization and robustness. Similarly, (Ferrante al.) 2024) leverage weight perturbation techniques to estimate neural network uncertainty, demonstrating improved classification accuracy through robust uncer- tainty quantification. (Pearce et al., 2020) explore distribution-free uncertainty estimation, using conformal prediction and quantile regression to estimate bounds on aleatoric uncertainty. (Chan et al.] (024) introduce hyper-diffusion models, allowing accurately estimate both epistemic and aleatoric uncertainty with a single model.
6 Conclusion
In this work, to effectively address the challenges of detecting Al-generated images, we propose a novel approach that leverages predictive uncertainty as a key metric. Our findings reveal that by analyzing the discrepancies in distribution between natural and Al-generated images, we can significantly enhance detection performance. The use of large-scale pre-trained models allows for accurate computation of predictive uncertainty, enabling us to identify images with high uncertainty as likely AT-generated. Our method achieves robust detection performance in a simple untrained manner. Overall, our approach demonstrates promising direction for improving Al-generated image detection and mitigating potential risks associated with their misuse. Future work could delve deeper into refining the predictive models and exploring additional features that could further enhance detection accuracy.
Acknowledgments
This work was supported in part by NSFC No. 62222117. JN and BH were supported by NSFC General Program No. 62376235, RGC Young Collaborative Research Grant No. C2005-24Y, RGC General Research Fund No. 12200725, Guangdong Basic and Applied Basic Research Foundation Nos. 2022A 0222115550 151501 1652 and 2024A151501239 and HKBU CSD Departmental Incentive Scheme. TLL was partially supported by the following Australian Research Council projects: F1220100318, (2221,,L222000022111
YGZ was funded by Inno HK Generative AI R&D Center. YCM was supported by the RGC Senior Research Fellow Scheme under the grant: SRFS2324- 2S02.
References
Blundell, C., Comebise,, J., Kavukcuoglu, K., and Wierstra, D. (2015). Weight uncertainty in neural network. In International Conference Machine Learning. ICML.
Brock, A., Donahue, J., and Simonyan, K. (2019). Large scale GAN training for high fidelity natural image synthesis. In International Conference Learning Representations, ICLR.
Chai, L., Bau, D., Lim, S., and Isola, P. (2020). What makes fake images detectable? understanding properties that generalize. In Proceedings of the European Conference on Computer Vision, ECCV.
Chan, M., Molina, M., and Metzler, C. (2024). Estimating epistemic and aleatoric uncertainty with a single model. In Advances in Neural Information Processing Systems, NeurIPS.
Chen, B., Zeng, J., Yang, J., and Yang. R. (2024). DRCT:
diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In International Conference on Machine Learning, ICML.
10

---

Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale scale hierarchical image database. The IEEE/ / CVF Computer Vision and Pattern Recognition Conference, CVPR.
Djurisic, A., Bozanic, N., Ashok, A., and Liu, R. (2023). Extremely simple activation shaping for out-of-distributinn detection. In International Conference on Learning Representations, ICLR.
Durall, R., Keuper, M., and Keuper, J. (2020). Watch your up-convolution: CNN based generative deep neural networks are failing to reproduce spectral distributions. In The IEEE / CVF Computer Vision and Putlern Recognition Conference, CVPR.
Durasov, N., Bagautdinov, M., Baqué, P., and Fua, P. (2021). Masksembles for uncertainty estimation. In IEEE Conference Computer Vision and Pattern Recognition, CVPR.
Everett, D., Nguyen, A. T., Richards, L. E., and Raff, E. (2022). Improving out-of-distribution detection via epistemic uncertainty adversarial training. arXiv preprint 22t:::2009003443
Ferrante, M., Boccato, T., and Toschi, N. (2024). Enabling uncertainty estimation in neural net- works through weight perturbation for improved alzheimer's 's disease classification. Frontiers Neuroinformaties.
Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., and Holz, T. (2020). Leveraging frequency analysis for deep fake image recognition. In International Conference on Machine Learning, ICML.
Gal, Y. and Ghahramani, Z. (2016). Dropout as bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, ICML.
Guo, C.,, Pleiss, G., Sun, Y., and Weinberger, K. Q. (2017). On calibration of modern neural networks. In International Conference on Machine Learning, ICML.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In The IEEE/ / CVF Computer Vision and Pattern Recognition Conference, CVPR.
He, Z., Chen, P., and Ho, T. (2024). RIGID: A training-free and model-agnostic framework for robust ai- generated image detection. CoRR.
Ho, J., Jain, A., and Abbeel, P. (2020). Denoising diffusion probabilistic models. In Advances in Neural Information Processing NeurIP.
Hochreiter, S. and Schmidhuber, J. (1997). Flat minima. Neural computation.
Hu, F. J., Shen, Y., Wallis, P., Allen-Zhu, 7., Li, Y., Wang. S., Wang, I.., and Chen, W. (2022). Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, ICLR.
Lahlou, S., Jain, M., Nekoei, H., Butoi, V., Bertin, P., Rector-Brooks. J., Korablyov. M., and Bengio, Y. (2023). DEUP: direct epistemic uncertainty prediction. Trans. Mach. Learn. Res.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing SesreMs.NeurIPS.
J., Li, D., Xiong, C., and Hoi, S. C. H. (2022). BLIP: bootstrapping language-image image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, ICMI. 2022, 17-23 July 2022, Baltimore, Maryland, USA. PMLR
Liu, H., Tan, Z., Tan, C., Wei, Y., Zhao, Y., and Wang, J. (2024). Forgery- aware adaptive transformer for generalizable synthetie image detection. In The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR.
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In JEEF/CVF International Conference on Computer Vision, ICCV.

---

Liu, Z., Qi, X., and Torr, P. H. S. (2020). Global texture enhancement for fake face detection in the wild. The IEEE CVF Computer Vision and Pattern Recognition Conference, CVPR.
Mandelli, S., Bonettini, N., Bestagini, P., and Tubaro, S. (2022). Detecting gan-generated images by orthogonal training of multiple cnns. In International Conference Image Processing, ICIP.
Marra, F., Gragnaniello, D., Cozzolino, D., and Verdoliva, L. (2018). Detection of gan-generated fake images over social networks. In IEEE Conference on Multimedia Information Processing and Retrieval, MIPR.
McAllester, (1998). Some pac-bayesian theorems. In Proceedings of the eleventh annual conference on Computational learning theory.
Midjourney (2022). http::://www..ddooorrreyyy com/
Neal, M. (2012). Bayesian learning for neural networks. Springer Science Business Media.
Nie, J., Zhang, Y., Fang, Z., Liu, T., Han, B., and Tian, X. (2024). Out-of-distribution detection with negative prompts. In International Conference on Learning Representations, ICLR.
Ojha, U., Li, Y., and Lee, J. (2023). Towards universal fake image detectors that generalize across generative models. The JEEE/ / CVF Computer Vision and Pattern Recognition Conference, CVPR.
Pearce, T., Leibfried, F., and Brintrup, A. (2020). Uncertainty neural networks: Approximately bayesian ensembling. In International Conference on Artificial Intelligence and Statistics, AISTATS.
Qian, Y., Yin, G., Sheng, L., Chen, Z., and Shao, J. (2020). Thinking in frequency: Face forgery detection by mining frequency-aware clucs. In Proceedings of the European Conference on Computer Vision, ECCV.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A.. Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, ICML.
Ricker, J., Lukovnikov, D., and Fischer, A. (2024). AEROBLADE: training- detection of latent diffusion images using autoencoder reconstruction error. In The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In The IEEE/ CVF Computer Vision and Pattern Recognition Conference, CVPR.
Schwaiger, A., Sinhamahapatra, P., Gansloser, J., and Roscher, K. (2020). Is uncertainty quantification in deep learning sufficient for out-of-distribution detection? Aisafery@ ijcai.
Shiohara, K. and Yamasaki, T. (2022). Detecting deepfakes with self-blended images. In The IEEE CVF Computer Vision and Pattern Recognition Conference, CVPR.
Snock. J., Ovadia, Y., Fertig, E., Lakshminarayanan, B., Nowozin, S., Sculley, D., Dillon, V., Ren, J., and Nado, Z. (2019). Can you trust your model's 's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing NeurlPS.
Stein, G., Cresswell, J. C.,, Hosscinzadeh, R., Sui, Y., Ross, B. L., Villecroze, V., Liu, Z., Caterini, L.. Taylor, J. E. T., and Laaiza-Ganem, G. (2023). Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. In Advances in Neural Information Processing Systems, NeurlPS.
Sun, Y. and Li, Y. (2022). DICE: leveraging sparsification for out-of-distribbtioo detection. In Proceedings of the European Conference on Computer Vision, ECCV.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S. E., Anguelov, D.,, Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015). Going deeper with convolutions, In The IEFE/ CVF Computer Vision and Pattern Recognition Conference, CVPR.
12

---

Tan, C., Liu, H., Zhao, Y., Wei, S., Gu, G., Liu, P., and Wei, Y. (2024). Rethinking the up sampling operations enn-based generative network for generalizable deepfake detection. In The IEEE/ CVF Computer Vision and Pattern Recognition Conference, CVPR.
Tan, C., Zhao, Y., Wei, S., Gu, G., and Wei, Y. (2023). Learning gradients: Generalized artifacts representation for gan generated images detection. In The IEEE CVF Computer Vision and Pattern Recognition Conference, CVPR.
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. (2021). Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, ICML.
Van der Vaart, A. W. (2000). Asymptotic statistics. Cambridge university press.
Wang, S., Wang, O., Zhang, R., Owens, A., and Efros, A. A. (2020). Con-generated images are surprisingly easy to spot... for now. In The IEEE/ / CVF Computer Vision and Pattern Recognition Conference, CVPR.
Wang, Z., Bao, J., Zhou, W., Wang. W., Hu, H., Chen, H., and Li, (2023). DIRE for diffusion- generated image detection. In IEEE/CVF International Conference Computer Vision, ICCV.
Wukong (2022). https: /tstps:/xxihe.e http://whw...aeeeeeeeeeeeoeooooa
Yu, F., Zhang, Y., Song, S., Seff, A., and Xiao, (2015). LSUN: construction of large-scale image dataset using deep learning with humans in the loop. CoRR.
Zhang, X., Karaman, S., and Chang, S. (2019). Detecting and simulating artifacts in GAN fake images. In International Workshop on Information Forensics and Security, WIFS.
Zhong, N., Xu, Y., Li, S., Qian, Z., and Zhang, X. (2023). Patcheraft: Exploring texture patch for efficient ai ai-generated image detection. arXiv preprint arXiv:2311.12397
Zhu, M., Chen, H., Yan, Q., Huang. X., Lin, G., Li, W., Tu, Z., Hu, H., Hu, J., and Wang. Y. (2023). Genimage: A million-scale benchmark for detecting ai-generated image. In Advances in Neural Information Processing Systems, NeurlPS.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes],
Justification:
The main claims made in the abstract and introduction accurately reflects the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes],
Justification:
Limitations are discussed in Appendix A
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
The paper provides a full set of assumptions and complete proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses the information needed to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[Yes]
Justification:
The data and code will be released once prepared.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/dde/eeee Cooeeeoodeesssseeeeeee //nips.com htc/://bbb///gdddee//eeeeeueisss ssiniiiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The paper provides details of implementation in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the average results under five different random seeds and report the variance in Figure
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper provide sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http::
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper has discussed societal impacts in Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original paper that produced the code package and dataset.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
We communicate the details of the code as part our submission via structured templates.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy buctps://neure wwwparippps..coooeeeeeeeee for what should or should be described.
20

---

A Limitation
key limitation of the proposed method lies its reliance on the assumption of distinct uncertainty profiles between natural and generated images. As generative models continue to advance, the uncertainty gap between these image types, as captured by the foundational model, may progressively narrow. This convergence could undermine the method's effectiveness, necessitating additional calibration techniques such as the proposed WePe", to amplify the differentiation in uncertainty.
B
Social impacts
The proposed Al- Al-generated image detection method contributes mitigating societal risks posed by generative model fraud. By improving the ability to identify synthetic media, such as deepfakes, this work helps combat disinformation and enhances trust in digital content, particularly in critical domains like journalism and legal evidence.
C Proof of THEOREM 3.2
Proof. Consider neural network f(x; X Rd, parameterized by 12, which maps an input image to feature vector f(x; 0). The model is trained on the natural image distribution D by minimizing a loss function:
(() EzLDi ((.
(17)
where l is the loss, and is the label. The optimal parameters are denoted 6".
To capture sensitivity, we define loss function that measures the change in the feature vector under parameter perturbations:
0x,0, [=; f(;
(18)
where & ~~ o0(, represents Gaussian perturbation with variance o². For small approxi-
mate:
f(x;0 > f(;; Vo
(19)
so:
lxx,0 Voi E = J(x) J(x) (x)).
(20)
where (xx) = (of( 0) is the Jacobian matrix. The expected loss over perturbations is:
Ee [) se Ee [s J(x) u'r (J(xT of (1
(21)
Thus, the expected sensitivity is proportional to the expected loss:
ENND [sen(x)) E
[x]]]]]]]] -
0.
(22)
According to PAC-Bayes theory (McAllester 1998), we consider a prior distribution P over parame- ters 0, typically P = (Mg.. o I). and a posterior distribution Q, typically Q ~800", I),, where 0" is the trained parameter. The PAC-Bayes theorem provides a bound on the expected loss under Q.
For loss function f, the standard PAC-Bayes bound states that, with probability at least the draw of training set T = {(I,,))) W NN, for any posterior Q:
5 over
N
[L(Q|| In N
Egn B(z,,) w)) < Egn
E(O, (.))
(23)
N
2(N - 1)
where KL(QIIP) is the Kullback-Leibhee divergence between Q and
Here, we adapt the loss to É(x; 0, = [1, + E) f(x; 0|. Since & is a perturbation, we consider the expected loss over which be formed as:
l'22 0) EE- [1]f(z; f(x; [) )
(24)
21

---

Since e (z, 8) depends on 8, we approximate by evaluating at 6", and consider the empirical sensitivity on the training set D
1 N (T, - N
2 Be [1
f(: 0)
(25)
According to Eq.
23 we have the following inequality:
KL In N 3 2(N -)
(l S ol
(26)
For D =D), the training set T D'. and the model is optimized 0 The empirical sensitivity is:
0² Σ Jaa.))
(T,,0))
Since the model optimized on D the training process minimizes the loss landscape's curvature around 0*, leading flat minimum (Hochreiter and Schmidhuber, 1997). which means the empirical sensitivity tends to 0. At the same time, as n tends to infinity, resulting in the expectation sensitivity converging the empirical sensitivity. Therefore, have:
E
[0 ~ 0,
(27)
which is consistent with our experimental results observed in Figure
For D D0 evaluate the expected sensitivity generated images, but the training remains
T ~ DD_ Since DD is not optimized, the feature representations for x ~ DO lie in regions of higher
curvature the loss landscape, leading to larger singular values of ((x). According Eq. (22) and
Eq. 27 we have:
Eg-DO
[o 2 0 E
[o]1[(>)11?]]
(28)
which implies E [sen(x)) < By-DD [sen(r)). Thus we complete this proof.
D Analysis of the Upper Bounds for Uncertainty Estimation
In Eq. 13 establish the validity of this estimation. The inequality obtained through the Cauchy-Schwarz inequality, and we can explore the validity of the upper bound by calculating the difference between the sides of the inequality. We define the difference:
we employ an upper bound to estimate uncertainty. Next, we provide an analysis to
1 Σ
1 - f (x:0,)) (f (x;:,))
A || (xx 0n)
II( (x; 0n)
j=1
(29)
Thus, the difference A can be written as:
2
A
f (x: (x)
1 - (x: 0,) .If (x: 0:01
Σ
fx;
- - (x;
1lf (x: 0.)1 cos
n 1=1
n 1=1
(30)
where 0 the angle between f (x; 06) and f (x; And finally, we obtain the following expression:
1 fxx0,)
22

---

Table 4: Performance ImageNet under frequency domain attacks (FDA) and spatial domain attacks (SDA). We report AUROC.
Methouss
ADM
ADMI
LDM
DXT
BigGAN
Madele
GIRGAN
StyletAN XL
RQ Transtomer
Misk GET
Averae
FDA
SEAA
FDA
SIA
FDA
SEDA
FDA
SEAA
FDA
SDA
FDA
SDA
FDA
SDA
FDA
SDA
FDA
SDA
FID
SDA
FDA
SDA
Truntng
CNNsspo
+2.21
$2.90
50.02
48.15
NPR
47.65
86.15
45.42
WePeo
95.64
91.79
Traising-fu:: Michasls
AEROBLADD
60,30
23.08
42,87
27.87
45.51
18
43,90
28,77
81.84
27.05 62,70
29.05
27.01
32.76
51.62
31.96
51.62
32,71
84.90
29:05
-2.97
22.82
RIGID
10.19
35.45
73.01
$4.37
65
66
31.83
63.08
33.37
77.99
35, 60
73.91
35, 32
73.21
14.56
83.01
36.19
78.02
34.36
74.15
34.77
-:43
-68.1
WalPo
91.22
93.78
85.70
80.87
83.78
79.00
90,
V3.30
93.51
91.08
92.46
93.96
94.42
91.95
96.11
88.44
88.84
88.68
9007
+0.09
+7.08
1 Σ (x:
A
(x: 0s)
|| [xx cos?
(31)
n j=1
1 fx: 0x)
Ex(x)))
(x;
(32)
This result indicates that the smaller the angle between / 0x) - Σn (x; 0z) and / (x; 0.), the tighter the upper bound. In our experiment, the perturbation applied to the model was extremely small, causing the image features to remain virtually unchanged, resulting in an extremely weak deviation between f fx; - Σÿ f (x; and f (x; 0z), thereby enabling the upper bound to provide a reliable basis for ranking the instances.
E Performance under attacks.
As presented in Table manipulations of generated images. Specifically, for the FDA and SDA scenarios, we apply zero- mean Gaussian noise with standard deviation of 0.1 to perturb the generated images. The results demonstrate that our method exhibits strong robustness to these perturbations.
evaluate the robustness of the proposed method against malicious
F Discussion on distribution discrepancy
In this paper, the core assumption we make is that there data distribution discrepancy between natural and generated images. This assumption is valid for current generative models and has been confirmed by many works (Tan et al. 2024 Ricker et al., 2024). This assumption is also the foundation of many generative image detection methods (we cannot distinguish between images that are indistinguishable.
Secondly, observe that this discrepancy in data distribution can be captured by the representation space of a vision model pre-trained on a large number natural images, i.e., there is feature distribution discrepancy between the generated and natural images, as shown in Figure However, this remains an observation, and we have not found theoretical proof despite reviewing the literature. We only observe similar phenomenon in UnivFD OOha et al. 2023), where the feature distribution discrepancy observed in the representation space of CLIP.VIT-L/IE
That said, we can confirm the existence of feature distribution discrepancy of generated and natural images based on an important metric for evaluating generative models, the FID score. The FID score measures the feature distribution discrepancy between natural and generated images on the Inception network (Szegedy
al., 2015). When the FID score O, indicates that the two distributions do not differ. However, even these simple networks such as Inception v3, advanced generative models like ADM still achieve an FID score of 11.84, not to mention that on powerful models such DINOv2, observe significant feature distribution discrepancy.
23

---

Generated Natuura Images
Images
Dinov2:ViT-L/44
CLIP-VIT-L14
Figure 8: Feature distribution discrepancy between the generated and natural images on DINOv2 and CLIP. and represent the feature of natural images and Al-generated images on the corresponding models.
Table 5: Measuring feature distribution discrepancy with FID scores.
Models
Natural
ADM
StyleGAN
1DPMM
DDPM
Diffusion GAN
Unleashing Transformer
Projected GAN
score
1.09
18.25
52.31
59.94
80.44
116,06
130.00
132.75
AUROC
50.00
73,85
83,50
86.23
88.84
94.16
94.18
93.34
G
Measuring feature distribution discrepancy with FID scores
We further use the "FID" score to measure the difference in feature distribution between natural and generated images. To avoid the effects of categories, we compute the FID scores using the DINOv2 model the LSUN-BEDROOM benchmark. For each category of images, 5000 images for calculation. In addition to calculating the FID scores between natural images and generated images, also calculate the FID scores between natural images and natural images. As shown in Table the FID scores between natural images and generated images are significantly higher than the FID scores between natural images and natural images. Moreover, there is clear positive correlation between the detection performance of WePe and the FID score. This result fully explains the existence of feature distribution discrepancy between natural and generated images DINOv2, and demonstrates that WePe can effectively detect the feature distribution discrepancy.
randomly select
H
WePe on large multi-modal models
In addition CLIP, further test the performance of WePe on BLIP (Li al. 2022). As shown in T the performance of WePe is unsatisfactory on these multimodal models, which may be due to the fact that the image features of the multimodal models are more focused semantic information. in line with our discussions.
Table 6: Comparison of detection times.
Table 7: WePe large multi- modal models.
Method
Time (s)
AEROBLADE
17.6
Model
AUROC
AP
RIGID
DINOv2
87.99 68.25
85
WePe
4.5
BLIP
64.68
I Comparison of computational costs.
Our method use a perturbed pre-trained model that is fixed during inferring all test samples. Thus, our method can processed within two forward passes. This is equal to the cost of RIGID that requires two forward passes for clean and noisy images. However, RIGID can concatenate clean and noisy images in mini batch and obtain detection results by with single forward pass. AEROBLADE requires only one forward pass, but it needs to compute the reconstruction error of the image. This

---

takes a long time to reconstruct at the pixel level. Besides, AEROBLADE needs to use neural network to compute the LPIPS score, leading to computational complexity. As shown in Table o we compare the time required to detect 100 images under the same conditions. Since AEROBLA LADE needs to calculate the image reconstruction error, it has the lowest detection efficiency. RIGID can obtain detection results in single forward pass by concatenating clean and noisy images, whereas WePe requires two forward passes, which results in WePe's detection efficiency being inferior to RIGID's. s. However, WePe can be parallelized across two devices to obtain the detection results in a single forward pass, In practice, WePe has high computational efficiency and is suitable for large-scale applications.
J WePe with multiple perturbation
In our experiments, taking into account the detection efficiency, we perturb the model only once, and then calculate the feature similarity of the test samples on the clean and perturbed models. We further experiment with multiple perturbations and use the mean of the feature similarity of the test samples on the clean model and all the perturbed models as the criterion for determining whether or not the image generated by the generative models. As shown in Figure TO, multiple perturbations can further improve performance.
K Comparison with OOD detection and uncertainty quantification methods
Contrast with OOD Detection: Conventional OOD detection (Sun and Li 2022 Djurisic (et al., 2023] Nie et al., 2024) usually relies on Maximum Softmax Probability (MSP) scores, exploiting fixed IID categories to identify OOD samples with low probabilities. In contrast, A Al-generated detection involves diverse, unbounded categories, rendering MSP scores ineffective. WePe introduces novel uncertainty estimation approach tailored for detecting generated images, achieving robust performance where traditional OOD methods falter. As shown in Table 8 we further use the ImageNet pre-trained classification model and used MSP and entropy as the scoring function to evaluate their performance on the Al-generated image detection task. The results show that these methods fail.
Contrast with Uncertainty Quantification: Standard techniques like MC-Dropout and Deep- Ensemble are ill-suited for DINOv2. Training nultiple DINOv2 models for DeepEnsemble is computationally infeasible, and the absence of dropout in DINOv2 undermines MC- Dropout's 's efficacy. Our proposed weight perturbation method overcomes these limitations, delivering practical and effective uncertainty estimation tailored to DINOv2's architecture, as validated by our experiments.
L Analysis of Failure Cases
Table 8: The results of using classi- fication models.
WePe leverages the differential epistemic uncertainty exhib- ited by pre-trained models, such as DINOv2, when processing natural versus generated images to differentiate between them.
model
score
AUROC
AP
Specifically, DINOv2, having been trained on an extensive dataset of natural images, demonstrates lower epistemic uncer- tainty for such images. Images that diverge from the distribution of the training dataset-despite not being generated by gen-
ResNet18 ResNe118
MSP
48.85
49.23
entropy
51.76
50.28
VIT-L/16
MSP
63.23
60.16
VIT-L/16
entropy
65.79
61.97
erative model-tend to elicit higher uncertainty from DINOv2, leading to their misclassification as generated images. As shown in Figure for a set of cartoon images, which were not produced by generative models, before and after weight perturbation. These images, due to their deviation from the training natural distribution, exhibited significant feature shifts post-perturbation, rendering them indistinguishable from generated images in our analysis.
visualized the feature shifts in DINOv2's 's representations
M Experiment results on GenImage, LSUN-BEDROOM and DRCT-2M
As shown in Table 9 Table TO and Table IT our method achieves good performance on GenImage LSUN-BEDROOM and DRCT-2M, confirming the robustness of the proposed method.
25

---

Figure 9: Feature shifts after model perturbation. Images are sampled from the following distributions: (a) natural image distribution, (b) Al-generated image distribution, and (c) cartoon image distribution.
Table 9: Al-generated image detection performance (ACC, %) on Genlmage.
Methods
Midjourny V1.4
GLIDE Wakong VQDM BigGAN Average
Training Methods
ResNet-S0 50
54,9
99.9
99.7
53.5
61.9
98.2
56.6
52.0
72.1
Deil-S
55.6
99.9
99.8
49.8
58.1
98.9
56.9
53.5
71.6
Swin-T
62.1
99.9
99.8
67.6
99.1 1
62.3
57.6
CNNNspot
52.8
96.3
95.9
50.1
39.8
78.6
53.4
46.8
64.2
Spec
52.0
99.4
99.2
49,8
49.8
F3Ner
50.1
99.9
99.9
49.9
50.0
99.9
49.9
49.9
68.7
GramNer
99.2
99.1
54.6
98.9
51.7
DIRE
60.2
99.9
99.8
$0.9
55.0
99.2
50.1
50.2
70.7
UnivED
76,9
PatchCraft
79.0
89.5
89.3
77.3
78.4
89.3
83.7
72.4
82.3
PatFormr
92.7
100.0
99.9
75.9
88.0
99.9
98.8
55.8
88.9
GenDet
DRCT
91.5
95.0
94.4
79.4
89.1
94.6
90.0
81.6
89.4
WoPe
82.3
93.6
87.1
Training- free Methods
AFROBLADE
RIGID
81.54
69.5
68.72
72.35
84.15
68.57
78.98
93.02
78.19
WePe
N SOFTWARE AND HARDWARE
We use python 36 and Pytorch 1.12.1, and several NVIDIA GeForce RTX-3090 GPU and NVIDIA GeForce RTX-4090 GPU.
O Using natural images only to select which layers to perturb
In experiments, we small set of natural images and generated images to pick the parameters that need to be per- turbed. When all the generated images can also use only the natural images to select the layers that need to be perturbed. Specifically, alone and calculate the similarity the features on the model of the natural image before and after the perturbation, as shown in Table 12] We then sort the similarity and select the blocks with the highest similarity for perturbation. As shown in Ta- ble [3] selecting the parameters to be perturbed in this way also achieves good performance and has strong robustness.
not available,
89.0
86.5
first perturb each block
AUROC
88.5
86.0
85.5
88.0
85.0
20
Figure 10: WePe with multiple perturbations.
P Details of Datasets
IMAGENET. The natural images and generated images
can be obtained at //ttts://githut wom/ labs/
dgm-eval The images are provided by (Stein et al. (203).. The generative model includes: ADM, ADMG, BigGAN, DIT-XL-2, GigaGAN, LDM, StyleGAN- XL, Traanssffrrm and Mask-GIT.
26

---

Table 10: Al-generated image detection performance LSUN-BEDROOD
Mudels
Methods
ADM
DDPM
iDDMM
Dillusion GAN
Projecsed
GAN
StyleGAN
Unlesshing Transfommer
Average
AUROC
AP
AUROC
AP
AUROC
AP
AL
ROC
AP
AUROC
AP
AUROC
AP
AUROC
AP
AUROC
AP
CNNsppo
64.83
61.24
79.04
80.58
76.95
76. 16.28
$5.45
87.19
90.50
59.94
95.17
94.94
93:42
93.11
$4.09
83.75
71.25
79.26
78.27
73.46
84.36
82.00
78.42
81.22
78.08
K3.58
83.48
79.53
77.94
DIRE
57.19
61.91
61.35
52.18
54.93
57.66
67.92
58.59
75.43
90.89
89.49
76.17
74.19
75.07
74.59
68.82
84.
83.67
90.11
78.25
WaPee
79.41
76.68
96.71
96.16
94.18
93.44
99.81
99.80
99.83
99.82
97.06
96.51
99.45
99.37
95.21
94.54
AEROBLADA
57.05
58.37
61.57
61.29
59.82
61.06
47.12
48.25
45.98
46.15
45.63
47.06
59.71
$7.34
53.85
54.25
RIGID
71.90
72.29
88.31
88.55
84.02
$4.80
91.42
91:90
92.1 92.12
92.54
77.29
74.96
91.37
91.39
85.20
35.20
WoPe
73,85
70.31
88.84
87.14
86.23
83.82
94.16
93.52
95.34
95.18
83.50
80.66
94.18
93.45
88.01
86.28
Table 11: Al- image detection performance (ACC, %) on DRCT-2M.
Method
SD
Turbo Viriants
CuntroNet Variants
Varints
Avg.
SDXL- Relina
SD-
SDXL-
LCC
ICM--
SDy1-
SDXL-
5Dv2-
SDXL-
SIXXI
DR
CNNSppl
99.87
99.91
09.90
07.65
66.25
86.
56.15
72.42
98.25
61:72
07.96
85.80
82.94
60.93
51.41
50.28
81.12
FRN
99.85
99.78
99.70
58.60
55.85
87.37
63.29
$3.66
97.39
54.98
07.98
72.30
81.90
65:42
50.30
50.27
71.13
CLIPRNNO
59.00
99.59
99.96
94.61
02.08
91.43
56.40
54,40
98.97
57.43
99.74
80.09
82.03
05:83
50.67
50.67
90:05
Grambee
09.40
99.01
98.84
95.30
02.63
80.03
71.19
05.32
93.05
57002
39.97
75.55
82.68
51.25
50.01
50.05
76,02
De-fake
92:10
05.35
99:51
59.65
64.02
69 24
92.00
91:93
99
70.89
18.95
62.34
05.65
50.33
90.16
50.00
75.53
Coav-Bl
99.97
108.0
99.97
95.84
64.44
82:00
60.75
99.27
99.27
62.33
99.30
83.40
73.25
61.65
$1.79
50.43
79.11
Otha
98.30
96.22
96.33
93.85
91.01
93.91
$6.33
55:92
90.64
$9.99
90.41
91.05
39.05
51.95
$1,03
50.65
K3.46
DIRR
76.04
59.90
93.05
97.35
K7.29
72:53
67.85
09:69
49.95
49.92
DRCT
96.41
95.38
91. 66
96.54
83,20
67.61
91.35
FalFoemax
96.52
92:87
91.78
15
$7.48
92.92
60.15
55.13
83.53
Welfe
75.02
77.25
-EUN-BEDROOM... The natural images and generated images can be obtained at https:///ggtiiiie com/ ha/p/layerrii--- -aaaaraiillabeedeeeeaaa llbssddm----aeeee eval The images are provided by (Stein et al.) 2023). The generative model includes: ADM, DDPM, iDDPM, StyleGAN, Diffusiion-roojected GAN, Projected GAN and Unleashing Transformers.
GenImage. The natural images and generated images can be obtained at httpp://ggtthii com/
The images are provided by (Zhu et al., (023). The generative model includes: Midjourney, SD V1.4, SD V1.5, ADM, GLIDE, Wukong, VQDM and BigGAN. The natural images come from ImageNet, and different images have different resolutions.
GenImage-
Bealmage-Dataset//eeeee Pestaset/GenIaaaee
DRCT-2M. The natural images of DRCT-2M come from CoCo and can be obtained from https //cocodatases org/ wwwp/wwwooooadddddoooooooooooooo Al-generated images of DRCT-2M can be obtained from https
-MRRCTTMM/ffeess which are provided by (Chen et al. 2024). The generative model includes LDM, 1.v..4,3 SDv1.5, SDv2, SDXL, SDXL-Refiner, SD- Turbo, 3DXL-Turbo LCM-SDv1.5, LCM-SDXL, SDv1-Ctrl, SDv2-Ctri, SDXL-Ctrl, SDv1-DR, SDv2-DR, SDXL-DR.
//modelscope wtt.c//aataeett/ooiinnnninaaaaaa
Q Implementation details
To balance detection performance and efficiency, we use DINOv2 ViT-L/14. We report the average results under five different random seeds and report the variance in Figure I0] In experiments we find that perturbing the high layers may lead to a large corruption in the features of the natural images, resulting in sub-optimal results. Therefore, We do not perturb the high-level parameters. In DINOV2 ViT-L/14, the model has 24 transformer blocks, and we only perturb the parameters of the first 19 blocks with Gaussian perturbations of zero mean. The variance of Gaussian noise is proportional to the mean value of the parameters in each block, with the ratio set to 0.1. Considering the computational cost, we perturb the model only time, i.c., n 1. Multiple perturbations can further improve the performance as shown in Table 10, For WePe*, we leverage LoRa (Hu et al., 2022p for parfieter-effcient fine-tuning. The Lora layers are applied on the q_proj and v_proj layers of DINOv2. Iora_ __ and lora a are set to 8. And the model is optimized using the Adam optimizer with a learning rate of 1 x 10-5, 31 0.9, B2 0.99, and a weight decay of 0.01. Following CNNspot (Wang et al. 2020), data augmentation techniques including JPEG compression and Gaussian blur are employed to enhance robustness. For the IMAGENNET and L.SUN-BEDROOM
benchmarks, the ProGAN dataset serves as the training set. For the GenImage benchmark, SDv1 ..4 dataset is used as training set. For the DRCT-2M benchmark, SDv2 dataset is used as training set. When testing, to ensure objectivity in calculating classification accuracy and mitigate biases
27

---

Table 12: Effect of perturbation position on natural images. We perturb each block individually, observe the similarity of features on the model of the natural image before and after the perturbation and rank these blocks.
95.0
06.99
32
W0.4
Table 13: Al-generated image detection performance on ImageNet. We select the top-k blocks with the highest similarity for perturbation based on the sorting results.
BirGAN
Midkls
GigiAAN
SigleGAN XT. RQ-Tondeeeeer
Ancrage
AUBOC
AUROC
AUBOC
AUROC
AUBOC
AUROC
AUBOC
AUROC
AUROC
Triining Mirtists
51:80
90.00
$3.14
52.85
$1.54
S1.67
$5.39
$1.62
50.5
50.70
$3.05
56.33
57 23
$1.10
$3.70
NPR
83.68
90:85
84.34
79,79
15.96
$6.95
95.15
$3.25
89.73
4G
X2.21
78.20
84.13
78.73
80.21
73.21
59.61
94.15
86:00
$9.84
Training-foo Medrica
ABROBLADA
61.57
$7.57
RaCar
877 16
$3.09
14.11
$5.13
90100
$8.14
194.7
$3.18
$1.98
WoPe wop
06:53
82.06
73.85
78.53
72.45
$3.90
92.61
92.17
93.06
$9:84
86-51
01:92
Widhe kap- 14
89 09
87.57
92.60
79.33
79.09
16.06
18.94
71.25
93.05
30
02 45
91.21
94.71
94.22
09
85.14
58.15
86.09
With spp- 16
00 58
89.40
$4.80
$2.06
60.25
76.54
78.33
72.88
92.51
02.55
92.11
91.10
92.89
02:
03.05
92.25
91
90.60
98:25
$6.88
Aap-
1212
arising from manually selected thresholds, following (Ojha al. 2023 automatically determine the optimal threshold by identifying the score that maximizes the separation between natural and generated images, based their computed classification scores.
28


Reviews:
[{"review_id": "GkD0ZQk6a5", "reviewer": "Reviewer_frxx", "summary": "This paper presents Weight Perturbation, a novel framework for AI-generated image detection that leverages epistemic uncertainty estimation. The core insight is that vision models pre-trained exclusively on natural images exhibit higher epistemic uncertainty when processing AI-generated images due to distributional shifts. The authors implement this through weight perturbation of pre-trained DINOv2 models, measuring feature sensitivity to these perturbations. Natural images maintain consistent features under weight perturbation, while generated images show significant variations. The approach offers both training-free weight perturbation and fine-tuned variants, demonstrating competitive performance across multiple benchmarks.", "questions": "1. How does WePe differentiate between epistemic uncertainty arising from AI generation versus uncertainty from other natural distributional shifts (e.g., domain changes, resolution variations, or color-space differences)? Could you discuss potential strategies to mitigate confusion with non-generative OOD samples?\n2. If an attacker has white‐box access, they could conceivably craft images whose feature representations remain insensitive to weight perturbations, defeating the core detection signal. What defenses might mitigate such targeted attacks?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strength\n\n1. The work introduces a well-motivated conceptual framework that reframes AI-generated image detection as an OOD problem solvable through epistemic uncertainty. This departure from traditional binary classification approaches is both theoretically grounded and practically compelling. \n2. The authors demonstrate SOTA performance across four challenging benchmarks, with WePe* consistently ranking among top performers and the training-free WePe remaining highly competitive. Particularly impressive is the method's demonstrated robustness to various attacks and image degradations, where competing methods show substantial performance drops while WePe maintains stability.\n3. The paper provides thorough ablation studies covering perturbation types, layer selection strategies, and hyperparameter sensitivity. \n\n## Weaknesses\n\n1. The epistemic uncertainty-based approach faces an inherent limitation in distinguishing AI-generated images from other types of OOD samples that could naturally arise in practical deployment scenarios. Since the method fundamentally relies on detecting distributional shifts from the training data of pre-trained models, it may exhibit elevated uncertainty for legitimate natural images that differ from the training distribution due to factors such as varying image resolutions, color-space conversions, or domain shifts. This could lead to false positive detections where genuine natural images are incorrectly classified as AI-generated. The performance variations observed in the appendix results on GenImage and DRCT-2M datasets, which contain images with diverse resolutions and sources, appear to support this concern. The method lacks mechanisms to differentiate between uncertainty arising from AI generation versus uncertainty from other distributional mismatches, potentially limiting its reliability in real-world applications where input images may come from varied and unpredictable sources.\n2. The method's success appears to be reliant on the specific properties of DINOv2. The paper would be more impactful if it provided a deeper analysis into *which* properties of DINOv2 are crucial for this task to establish a more general principle.", "was_revised": true, "final_justification": "My weaknesses and questions were addressed in the rebuttal.", "rebuttal": "We thank the reviewer for the thorough review and constructive feedback. We address each concern below.\n\n\n> Q1. Since the method fundamentally relies on detecting distributional shifts from the training data of pre-trained models, it may exhibit elevated uncertainty for legitimate natural images that differ from the training distribution due to factors such as varying image resolutions, color-space conversions, or domain shifts. This could lead to false positive detections where genuine natural images are incorrectly classified as AI-generated. The method lacks mechanisms to differentiate between uncertainty arising from AI generation versus uncertainty from other distributional mismatches, potentially limiting its reliability in real-world applications where input images may come from varied and unpredictable sources.\n\nWe appreciate your insightful comments. WePe leverages heightened epistemic uncertainty arising from distribution shifts to differentiate between natural and AI-generated images. However, natural images that diverge from the training distribution may be misclassified as generated. To mitigate this challenge, we made two efforts in our paper:\n\n- We select DINOv2 as the feature extractor, as its training on a diverse and extensive dataset of natural images ensures a broad and inclusive training distribution, encompassing a wide range of natural image variations. \n\n- We further amplify the uncertainty induced by AI generation through confidence calibration, enhancing the robustness of our detection approach. Specifically, to enhance the differentiation of uncertainty induced by AI-generatation, we fine-tune the model using a dataset comprising both natural and generated images. This approach aims to amplify the sensitivity of generated images to model perturbations while preserving the robustness of natural images against such perturbations. Consequently, this methodology widens the uncertainty gap between the model's responses to natural and generated images, thereby improving the distinguishability of their respective outputs.\n\n\n\n> Q2. The method's success appears to be reliant on the specific properties of DINOv2. The paper would be more impactful if it provided a deeper analysis into which properties of DINOv2 are crucial for this task to establish a more general principle.\n\nWe appreciate your suggestion. We attribute DINOv2’s superior performance in distinguishing natural images from generated images, in part, to its self-supervised learning conducted exclusively on images, enabling robust capture of visual features. In our paper, we also evaluated WePe using CLIP as the feature extractor, where it exhibited reduced performance compared to DINOv2. This may result from CLIP’s reliance on text supervision during training, which results in image features that are predominantly semantic. In contrast, the distinction between natural and generated images hinges primarily on low-level features no semantic information, which DINOv2 captures more effectively owing to its image-only self-supervised training paradigm.\n\n> Q3. How does WePe differentiate between epistemic uncertainty arising from AI generation versus uncertainty from other natural distributional shifts (e.g., domain changes, resolution variations, or color-space differences)? Could you discuss potential strategies to mitigate confusion with non-generative OOD samples?\n\nThanks for your constructive comments. We believe the following explanations would significantly improve the quality of our work. \nTo alleviate the mentioned challenge, we first select DINOv2 as the feature extractor, as its training on a diverse set of natural images maximizes the inclusivity of the natural distribution, thereby reducing the likelihood of misclassifying natural distribution shifts. Second, when training is feasible, we further enhance the detection of AI-induced distribution shifts through distribution calibration, optimizing the method’s ability to distinguish generated images from natural ones.\n\n> Q4. If an attacker has white‐box access, they could conceivably craft images whose feature representations remain insensitive to weight perturbations, defeating the core detection signal. What defenses might mitigate such targeted attacks?\n\nWe appreciate your inspiring comments. We believe this would be a promising direction in our community. \n\nIn this regard, input randomization [1] serves as a potential defense against white-box adversarial attacks. During inference, applying random transformations to the input image, such as varying the image resolution, can disrupt the specific structure of adversarial perturbations, thereby enhancing the robustness of our method.\n\nMeanwhile, we find that advanced diffusion models provide a promising solution to mitigate adversarial attacks. However, this in turn leads to a further question: once a natural image is reconstructed by a diffusion model, should the reconstructed image be considered real or generated in our detection problem?\n\nWe are looking forward to your further inspiring feedback in this promising direction.\n\nReference:\n\n[1]: Mitigating adversarial effects through randomization. Xie et al. ICLR 2018"}, {"review_id": "iuqVnT9cAp", "reviewer": "Reviewer_Q2AQ", "summary": "This paper proposes a novel method for detecting AI-generated images based on epistemic uncertainty. The key idea is to exploit the distributional shift between natural and generated images by measuring how sensitive image features are to weight perturbations in a large pre-trained vision model (e.g., DINOv2). The method, termed WePe, operates without needing generated image samples for training and can be optionally enhanced (WePe*) by fine-tuning to amplify the uncertainty gap. Experiments on multiple benchmarks demonstrate state-of-the-art detection performance among both training-based and training-free approaches.", "questions": "1. Could you elaborate on how sensitive your method is to the choice of the pre-trained model? While DINOv2 was shown to work well, have you explored other large-scale vision models (e.g., more recent multimodal models or domain-adapted models)?\n\n2. The method assumes that generated images will continue to exhibit higher epistemic uncertainty. As generative models become more advanced, how do you envision addressing scenarios where this uncertainty gap narrows?\n\n3. Could you provide more insights (e.g., failure case examples) to illustrate how the weight perturbation affects the feature space for natural vs. generated images?\n\n4. Regarding computational cost, the author mention that WePe requires two forward passes. Could you clarify whether any optimizations (e.g., batching, parallelization) were applied in your reported efficiency measurements, and how the method scales to larger datasets?\n\n5. Have you considered whether the proposed weight perturbation technique could be applied dynamically (e.g., with adaptive perturbation magnitudes) or combined with input-space perturbations for further performance gains?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper tackles an important and timely problem—AI-generated image detection with strong generalization to unseen generators—which is highly relevant given the rapid development of generative models.\n2. The idea of linking epistemic uncertainty (via weight perturbation) to distribution shift is conceptually sound and supported by theoretical analysis and empirical results.\n3. The method is simple, training-free in its basic form (WePe), and efficient, making it appealing for practical deployment.\n4. Extensive experiments across several benchmarks and perturbation scenarios (e.g., frequency/spatial domain attacks) demonstrate competitive or superior performance relative to strong baselines.\n5. The paper is clearly written, well-organized, and easy to follow.\n\nWeaknesses:\n1. The novelty is incremental in terms of core methodological innovation. While combining uncertainty estimation with weight perturbation for this task is insightful, it builds on well-known ideas from OOD detection and uncertainty quantification without introducing fundamentally new learning paradigms.\n2. The method relies heavily on the assumption that future generative models will still induce detectable uncertainty gaps; while acknowledged by the authors, this limitation poses a potential risk to long-term applicability.\n3. The experiments could include broader comparison with more recent detection frameworks or more diverse pre-trained models (e.g., stronger multimodal models beyond CLIP and BLIP).", "was_revised": true, "final_justification": "Thanks for the rebuttal, most of my concerns are addressed.", "rebuttal": "We thank the reviewer for the thorough review and constructive feedback. We address each concern below.\n\n> Q1. The novelty is incremental in terms of core methodological innovation. While combining uncertainty estimation with weight perturbation for this task is insightful, it builds on well-known ideas from OOD detection and uncertainty quantification without introducing fundamentally new learning paradigms.\n\n\nTo address your concern, we have added the following clarifications in our revised paper, aiming to highlighting the distinct contributions of our work compared to OOD detection and uncertainty quantification methods:\n\n- **Contrast with OOD Detection:** Conventional OOD detection [1,2] usually relies on Maximum Softmax Probability (MSP) scores, exploiting fixed ID categories to identify OOD samples with low probabilities. In contrast, AI-generated detection involves diverse, unbounded categories, rendering MSP scores ineffective. WePe introduces a novel uncertainty estimation approach tailored for detecting generated images, achieving robust performance where traditional OOD methods falter.\n\n- **Contrast with Uncertainty Quantification:** Standard techniques like MC-Dropout and DeepEnsemble are ill-suited for DINOv2. Training multiple DINOv2 models for DeepEnsemble is computationally infeasible, and the absence of dropout in DINOv2 undermines MC-Dropout’s efficacy. Our proposed weight perturbation method overcomes these limitations, delivering a practical and effective uncertainty estimation tailored to DINOv2’s architecture, as validated by our experiments.\n\nThese advancements position our approach as a pioneering solution, adeptly addressing the unique demands of AI-generated image detection with superior adaptability and performance.\n\n\n> Q2. The method relies heavily on the assumption that future generative models will still induce detectable uncertainty gaps; while acknowledged by the authors, this limitation poses a potential risk to long-term applicability.\n\nWe sincerely appreciate your insightful feedback. We acknowledge the potential limitation in assuming that future generative models will consistently produce detectable uncertainty gaps. However, this assumption lies in the fact that uncertainty approaches can detect the distribution shifts. Thus, these approaches are expected to detect the distribution discrepancy between natural and generated images. The potential failure case would be that future models generate images causing little distribution shift from natural images. In this context, it is hard to ensure whether human beings can identify these generated images. \n\n> Q3. The experiments could include broader comparison with more recent detection frameworks or more diverse pre-trained models (e.g., stronger multimodal models beyond CLIP and BLIP).\n\nWe appreciate your kind suggestion. To further validate the robustness of WePe, we evaluated its performance using advanced multimodal models, namely SigLIP-So400m-patch16-256 [3] and InternViT-6B-224px [4]. As presented in the table below, WePe demonstrates strong performance across these models, affirming its effectiveness and generalizability.\n\n|                        | AUROC | AP    |\n|------------------------|-------|-------|\n|DINOv2 |87.99|85.04|\n| SigLIP2                | 85.63 | 84.31 |\n| InternViT |    90.24    | 88.37     |\n\n\n> Q4. Could you provide more insights (e.g., failure case examples) to illustrate how the weight perturbation affects the feature space for natural vs. generated images?\n\nWe appreciate your insightful suggestion. WePe leverages the differential epistemic uncertainty exhibited by pre-trained models, such as DINOv2, when processing natural versus generated images to differentiate between them. Specifically, DINOv2, having been trained on an extensive dataset of natural images, demonstrates lower epistemic uncertainty for such images. Images that diverge from the distribution of the training dataset—despite not being generated by a generative model—tend to elicit higher uncertainty from DINOv2, leading to their misclassification as generated images. In the revised manuscript, we visualized the feature shifts in DINOv2’s representations for a set of cartoon images, which were not produced by generative models, before and after weight perturbation. These images, due to their deviation from the training natural distribution, exhibited significant feature shifts post-perturbation, rendering them indistinguishable from generated images in our analysis.\n\n\n> Q5. Regarding computational cost, the author mention that WePe requires two forward passes. Could you clarify whether any optimizations (e.g., batching, parallelization) were applied in your reported efficiency measurements, and how the method scales to larger datasets?\n\nWe appreciate your kind suggestion. To evaluate computational efficiency, we adhered to standard testing protocols, utilizing a dataloader for parallel computation. For each batch of samples, we first extract features using the original model, followed by feature extraction with the perturbed model, necessitating two forward passes on a single GPU. For large-scale datasets, we can employ two GPUs: one to compute features from the original model and another for the perturbed model. This configuration enables the acquisition of prediction results for each sample within the duration of a single forward pass, enhancing computational efficiency.\n\n> Q6. Have you considered whether the proposed weight perturbation technique could be applied dynamically (e.g., with adaptive perturbation magnitudes) or combined with input-space perturbations for further performance gains?\n\n\nThank you for your inspiring question. Implementing adaptive perturbation may necessitate training an additional perceptron to determine the appropriate perturbation amplitude, which would incur additional computational overhead. As an alternative, we propose perturbing the model multiple times with varying amplitudes and computing the average feature values to approximate the effect of adaptive perturbation. This approach, as demonstrated in the table below, enhances the performance of WePe. Furthermore, integrating this method with input perturbation also improves WePe's performance.\n\n|                        | AUROC | AP    |\n|------------------------|-------|-------|\n|WePe|87.99|85.04|\n| WePe + multi random perturbation              | 89.36 | 90.62 |\n| WePe + input perturbation |    88.43   | 86.88      |\n\nReference:\n\n[1]: DICE: Leveraging Sparsification for Out-of-Distribution Detection. Sun et al. ECCV 2022\n\n[2]: Extremely Simple Activation Shaping for Out-of-Distribution Detection. Djurisic et al. ICLR 2023\n\n[3]: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. Tschannen et al. arXiv 2025\n\n[4]: Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. Chen et al. CVPR 2024"}, {"review_id": "4omsKwck1I", "reviewer": "Reviewer_4Yn5", "summary": "This papers tackles the problem of AI-generated image detection through a novel approach named WePe (weight perturbation). In contrast to other work where models are trained to distinguish between real and generated images, this work focuses on the distribution shift between the real distribution and the artificial one to capture the epistemic uncertainty of the model under parameter perturbation. The main supposition is that, since the used models are trained on the original dataset images, a perturbation on its weights will show greater sensitivity to generated images than to original ones. This new detection system has the advantage of not relying on generated images during training, thus it simplifies the process of dataset selection and pre-processing techniques.\nA calibrated approached through a regularization loss with access to generated images is also presented and named WePe*.\nAuthors perform experiments on 4 datasets and compare 3 metrics with a wide set of baselines for training and training-free methods.", "questions": "**Q1** - Given that epistemic uncertainty captures the lack of knowledge about a distribution, wouldn’t it be beneficial to also consider aleatoric uncertainty—especially if the generated distribution is misaligned with the natural one? Additionally, should domain uncertainty (often considered a third type of uncertainty related to OOD data) be taken into account in this context?\n \n\n**Q2** - In Equation (13), since the expression is an upper bound, could the authors elaborate on its implications? In particular, how do you ensure (or to what extent can we expect) that it provides a reliable basis for ranking the instances?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n**S1** – The paper is clearly written, with a well-articulated motivation and a solid introduction to the related work and background. Overall, it is easy to follow.\n\n\n**S2** – The proposed technique works even if no generated images are available for training. This makes the training easier and the technique scalable.\n\n**S3** – Mathematical claims seem to be correct. Epistemic uncertainty is correctly approached through model weight perturbation.\n\n**S4** – Even if there is no access to the teacher model’s parameters, the upper bound on the uncertainty allows for an effective estimation of uncertainty.\n\n**S5** – Metrics are well chosen. Fréchet Inception Distance (FID) is the standard metric to compare the distribution of generated images to the real distribution. AUROC and Average Precision give a global view of the performance of a model. Perturbation Sensitivity also properly reflects the effect of variations in the parameters of the model and introduces well Theorem 3.2.\n\n**S6** – Wide range of experiments, datasets, and baselines. The ablation study further confirms the validity of the results.\n\n**S7** – Results show that the proposed method works well on detecting generated images, as well as achieving higher robustness to attacks than the other methods.\n\n**S8** – Some extra conclusions taken from Section 4.3 provide additional insights—for example, the study on which layers to perturb or the degree of perturbation.\n\n**Weaknesses:**\n\n**W1 –** Some important results are delegated to the Appendix, which may hinder the overall flow of the paper. If the authors believe it could improve readability, they might consider restructuring certain sections so that key findings are more accessible in the main text, reducing the need to jump frequently to the supplementary material.\n\n**W2 –** It is not entirely clear what statistical analysis has been carried out on the reported results, and the paper lacks confidence intervals in the main tables. In the *Experimental Details* paragraph on page 7, the authors mention averaging results over 5 random seeds, but it is unclear what specific results this refers to. Figure 9 is then introduced (in the Appendix), but its meaning and interpretation are not clearly explained. From what we understand, Figure 9 shows variability across 5 runs, but only for the proposed method. Has a similar analysis been conducted for the other methods used in the benchmarking? Strengthening this point in the Appendix and including confidence intervals for all compared methods when available, would improve the overall quality of the experimental findings.\n\n**W3 –** The related work on Uncertainty Quantification feels somewhat limited in scope. It could be enriched by including references more directly connected to how uncertainty is measured in this paper. If the authors believe such additions would be relevant, this could help better position their contribution in the context of existing literature.\n\n**Some minor comments:**\n\n- **C1: Typos and small formatting issues**\n    - Line 79: “We” should start with lowercase after the comma.\n    - Lines 86 and 87: The word “discrepancy” is missing the final “y” (appears twice).\n    - Line 180: The word “However” contains a typo.\n    - Line 241: The method SelfBlend is misspelled as SelfBland.\n    - Table 1: AEROBLADE is written incorrectly as AEROBLADA.\n    - Line 250: “We” should be lowercase after the comma.\n    - Line 294: “Naturak” should be corrected to “natural”.\n    - Line 563: “wepe” should be capitalized as “WePe”.\n- **C2: Possible notation inconsistencies:**\n    - Section 2.2: There is a notation inconsistency regarding the model’s weights. Are they written as $\\theta$ or $\\textbf{w}$? Do these notations express different things?\n    - Theorem 3.2 vs. Section 2.1: The distributions for AI-generated and natural images are $\\mathcal{D}^0$ and $\\mathcal{D}^1$ in the latter, and the other way around in the former, and again recovered in Equation (16).\n    - Line 261: You mention that when WePe is “further trained to amplify the…” but after the comma you name it WePe instead of WePe*, which is the notation you adopted for this method. This can lead to confusion.\n- **C3: Clarity improvements**\n    - Table 1: Under *Training Methods*, the methods — PatchCraft, FatFormer, and DRCT — are listed, but none of them are mentioned or described in the main text of the paper. Including brief descriptions or citations would improve clarity and completeness.\n    - Section 4.3 (first paragraph title): Consider renaming the paragraph title to “Robustness to Image Perturbations” to clarify that the perturbations refer to input images rather than model weights.\n\n**C4: Results consistency.** Some of the results presented in Table 4 appear to be incorrectly computed, if we understood the methodology correctly. The difference in AUROC seems to be calculated based on Table 1 results, but the numbers for CNNspot, NPR, and WePe* do not match. For example, for CNNspot: 59.78−67.04=−7.2659.78 - 67.04 = -7.2659.78−67.04=−7.26, not −9.06-9.06−9.06 as stated.", "was_revised": true, "final_justification": "The paper is solid, and the authors addressed the main concerns. I’m raising my score to accept.", "rebuttal": "We thank the reviewer for the thorough review and constructive feedback. We address each concern below.\n\n> Q1. Some important results are delegated to the Appendix, which may hinder the overall flow of the paper. \n\nThank you for your kind suggestion. We agree with your point, and, in the revised version, we have reorganized the tables to enhance the clarity and flow of the paper.\n\n> Q2. It is not entirely clear what statistical analysis has been carried out on the reported results, and the paper lacks confidence intervals in the main tables.\n\nThanks for your constructive suggestion. Since the perturbations are random, the results of our method are the average values under five random seeds. In Figure 9, we report the error bars of our method. For clarity, we have added this information to the revised table. For other methods, since some of the results were obtained directly from related papers, we did not report this information.\n\n\n> Q3. The related work on Uncertainty Quantification feels somewhat limited in scope.\n\nIn response to your valuable suggestion, we have expanded the discussion on uncertainty estimation in the related work section to provide a more comprehensive and rigorous analysis.\n\nBlundell et al. (2015) introduce Bayes by Backprop, a method for estimating weight uncertainty in neural networks by modeling the posterior distribution over weights using variational inference, improving model generalization and robustness. Similarly, Ferrante et al. (2024) leverage weight perturbation techniques to estimate neural network uncertainty, demonstrating improved classification accuracy through robust uncertainty quantification. Pearce et al. (2018) explore distribution-free uncertainty estimation, using conformal prediction and quantile regression to estimate bounds on aleatoric uncertainty. Chan et al. (2025) introduce hyper-diffusion models, allowing to accurately estimate both epistemic and aleatoric uncertainty with a single model.\n\n\n> Q4. Comments on typos.\n\nThank you for your careful review! We have carefully revised all typos.\n\n> Q5. Possible notation inconsistencies and results inconsistencies.\n\nThank you for pointing out these issues! We have revised these inconsistencies in the revised version.\n\n> Q6. Clarity improvements.\n\nWe have incorporated a discussion on PatchCraft, FatFormer, and DRCT in the related work section and revised the title from \"Robustness to Perturbations\" to \"Robustness to Image Perturbations\" for greater clarity.\n\n\n> Q7. Given that epistemic uncertainty captures the lack of knowledge about a distribution, wouldn’t it be beneficial to also consider aleatoric uncertainty—especially if the generated distribution is misaligned with the natural one? Additionally, should domain uncertainty (often considered a third type of uncertainty related to OOD data) be taken into account in this context?\n\n\nWe appreciate your insightful feedback. Aleatoric uncertainty quantifies the inherent noise within observational data. Given the distinct data distributions of natural and generated images, their aleatoric uncertainties may differ, potentially serving as a discriminative feature to distinguish between them. Domain uncertainty, arising from shifts in data distribution, may exhibit effects comparable to those of epistemic uncertainty.\n\n> Q8. In Equation (13), since the expression is an upper bound, could the authors elaborate on its implications? In particular, how do you ensure (or to what extent can we expect) that it provides a reliable basis for ranking the instances?\n\n\nIn formula 13, the inequality is obtained through the Cauchy-Schwarz inequality, and we can explore the validity of the upper bound by calculating the difference between the two sides of the inequality. We define the difference as:\n\n$$ \\triangle = ||f\\left(\\mathbf{x} ; \\theta_k\\right)-\\frac{1}{n} \\sum_j^n f\\left(\\mathbf{x} ; \\theta_j\\right)||^2  \\cdot ||f\\left(\\mathbf{x} ; \\theta_t\\right)||^2 - ||(f\\left(\\mathbf{x} ; \\theta_k\\right)-\\frac{1}{n} \\sum_j^n f\\left(\\mathbf{x} ; \\theta_j\\right)) \\cdot (f\\left(\\mathbf{x} ; \\theta_t\\right))||^2$$\n\n\nThus, the difference $\\triangle$ can be written as:\n\n$$\n\\triangle = \\left\\| f\\left(\\mathbf{x}; \\theta_k\\right) - \\frac{1}{n} \\sum_{j=1}^n f\\left(\\mathbf{x}; \\theta_j\\right) \\right\\|^2 \\cdot \\left\\| f\\left(\\mathbf{x}; \\theta_t\\right) \\right\\|^2 - \\left( \\left\\| f\\left(\\mathbf{x}; \\theta_k\\right) - \\frac{1}{n} \\sum_{j=1}^n f\\left(\\mathbf{x}; \\theta_j\\right) \\right\\| \\left\\| f\\left(\\mathbf{x}; \\theta_t\\right) \\right\\| \\cos \\theta \\right)^2\n$$\n\nwhere $\\theta$ is the angle between $f\\left(\\mathbf{x} ; \\theta_t\\right)$ and $f\\left(\\mathbf{x} ; \\theta_k\\right)-\\frac{1}{n} \\sum_j^n f\\left(\\mathbf{x} ; \\theta_j\\right)$. \n\nAnd finally, we obtain the following expression:\n\n$$\n\\triangle = \\left\\| f\\left(\\mathbf{x}; \\theta_k\\right) - \\frac{1}{n} \\sum_{j=1}^n f\\left(\\mathbf{x}; \\theta_j\\right) \\right\\|^2 \\cdot \\left\\| f\\left(\\mathbf{x}; \\theta_t\\right) \\right\\|^2 \\left( 1 - \\cos^2 \\theta \\right)\n$$\nUsing the trigonometric identity, we have: \n$$\n\\triangle = \\left\\| f\\left(\\mathbf{x}; \\theta_k\\right) - \\frac{1}{n} \\sum_{j=1}^n f\\left(\\mathbf{x}; \\theta_j\\right) \\right\\|^2 \\cdot \\left\\| f\\left(\\mathbf{x}; \\theta_t\\right) \\right\\|^2 \\sin^2 \\theta\n$$\n\nThis result indicates that the smaller the angle between $f\\left(\\mathbf{x} ; \\theta_k\\right)-\\frac{1}{n} \\sum_j^n f\\left(\\mathbf{x} ; \\theta_j\\right)$ and $f\\left(\\mathbf{x} ; \\theta_t\\right)$, the tighter the upper bound. In our experiment, the perturbation applied to the model is extremely small, causing the image features to remain virtually unchanged, resulting in an extremely weak deviation between $f\\left(\\mathbf{x} ; \\theta_k\\right)-\\frac{1}{n} \\sum_j^n f\\left(\\mathbf{x} ; \\theta_j\\right)$ and $f\\left(\\mathbf{x} ; \\theta_t\\right)$, thereby enabling the upper bound to provide a reliable basis for ranking the instances."}, {"review_id": "gPXfNrbpiJ", "reviewer": "Reviewer_C7Yz", "summary": "The distribution shift between training images and testing images causes higher uncertainties of machine learning models on testing compared to training data. Based on this observation, the authors propose to use a model pretrained on natural images, and compute its uncertainty on generated images, which differ in distribution. They choose a large model pretrained on a massive amount of images (DINOv2) to ensure the maximum generalizability of the approach. However, DINOv2 does not directly compute its uncertainty so the authors developed a method to do so (WePe). Then, they test the approach and its variant that additionally trains on generated images (WePe*) and compare it to other existing methods.", "questions": "1. Are you sure that \"DINOv2 is exclusively pre-trained on natural images\" (line 172)? In my opinion it is impossible to really know, and this assumption will likely be even less true with more recent models because generated images are more and more present online.", "limitations": "addressed in the Appendix", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. Using uncertainty in the context of generated image detection is interesting and opens the gate to reuse many methods of uncertainty quantification and out-of-distribution detection for this application.\n1. Well-justified method of uncertainty estimation (WePe) to obtain the uncertainty of DINOv2, and it includes a variant benefitting from training (WePe*).\n1. Experimental results on many generative models and with many competing methods on four datasets.\n\n**Weaknesses**\n\n1. Lack of standard uncertainty quantification baselines. For instance, the authors could use a pretrained ImageNet classifier and use the maximum predicted probability (or logit), or the entropy as a confidence indicator. Without this, it is hard to know whether the WePe method based on weight perturbation is necessary.\n1. Not enough details on the data (number of samples, splits used...) for the results and figures (e.g., for Figures 1, 2, 3, where are the natural and fake images coming from?). If the natural images for ImageNet include the training data, then all other images would have very different features, even for natural images from validation set (because DINOv2 was trained on ImageNet train set). Because of the assumptions made, it is necessary to provide such details.", "was_revised": true, "final_justification": "My weaknesses and questions were addressed in the rebuttal. Overall, the paper is interesting.", "rebuttal": "We thank the reviewer for the thorough review and constructive feedback. We address each concern below.\n\n> Q1. Lack of standard uncertainty quantification baselines. For instance, the authors could use a pretrained ImageNet classifier and use the maximum predicted probability (or logit), or the entropy as a confidence indicator. Without this, it is hard to know whether the WePe method based on weight perturbation is necessary.\n\nWe express our gratitude for the reviewer’s insightful suggestion. Two primary considerations preclude the adoption of conventional metrics for assessing confidence levels in our study:\n\n- In the context of detecting AI-generated images, image categories is diverse and inherently open-ended. Classifiers trained on the ImageNet dataset—which encompasses a finite set of categories—can only yield reliable confidence scores for in-distribution categories. For images beyond this taxonomy—whether natural or generated—the confidence scores produced are unreliable.\n\n- Standard image classifiers generate confidence scores that reflect their confidence in predicting object categories. However, the distinction between natural and generated images lies not in categorical differences but in attributes such as texture and structural patterns. Consequently, we opted for WePe over classifier outputs to estimate uncertainty.\n\nTo address the reviewer’s concerns, we conducted experiments using a ResNet18 model pretrained on ImageNet to differentiate between natural and generated images, employing maximum softmax probability (MSP) and entropy as confidence metrics. As demonstrated in the table below, these metrics exhibit suboptimal performance.\n\n\n|         | AUROC | AP    |\n|---------|-------|-------|\n| MSP     | 48.85 | 49.23 |\n| entropy | 51.76 | 50.28 |\n| WePe    | 87.99 | 85.04 |\n\nThanks again for your insightful question. We have added the above explanations to our revision.\n\n\n> Q2. Not enough details on the data for the results and figures. If the natural images for ImageNet include the training data, then all other images would have very different features, even for natural images from validation set (because DINOv2 was trained on ImageNet train set). Because of the assumptions made, it is necessary to provide such details.\n\nWe express our gratitude for the reviewer’s valuable suggestion. For Figures 1 and 3, natural images are sourced from the LSUN dataset, and generated images are produced using the ProGAN model. For Figure 3, natural images are sampled from the LSUN-Bedroom dataset, with generated images created by the following models: ADM, DDPM, iDDPM, StyleGAN, Diffusion-Projected GAN, Projected GAN, and Unleashing Transformers.\n\n\n> Q3. Are you sure that \"DINOv2 is exclusively pre-trained on natural images\" (line 172)? In my opinion it is impossible to really know, and this assumption will likely be even less true with more recent models because generated images are more and more present online.\n\n\nThanks for pointing out this potentially confusing statement. The original statement is overly absolute. The DINOv2 model leverages multiple curated datasets composed of natural images. Additionally, it incorporates data sourced from the internet that aligns with these curated datasets, potentially introducing some AI-generated images. However, due to the known issue of mode collapse [1], few foundational models intentionally include a substantial proportion of generated images during training. Consequently, generated images remain a long-tail component relative to natural images. Nevertheless, as generative models advance, the increasing prevalence of generated images online may contaminate training datasets for new models. To address this, we propose precise calibration to effectively widen the uncertainty gap between natural and generated images. The efficacy of this approach is substantiated by our experimental results.\n\n\nReference: \n\n[1]: AI models collapse when trained on recursively generated data. Shumailov et al. Nature 2024"}]

Author rebuttal:

Meta-review: This paper presents an uncertainty-based method for detecting AI-generated images. The underlying assumption is that there is a distribution shift between natural images and AI-generated images, such that models trained on natural images exhibit high epistemic uncertainty (EU) when processing AI-generated images. Building on this idea, the authors introduce a weight perturbation method applied to pre-trained DINOv2 models to measure EU. Extensive experiments demonstrate competitive performance across four benchmarks.

The paper received four reviews: one accept and three borderline accepts. Overall, reviewers acknowledge the authors’ insight into the distribution shift between natural and generated images, their effective use of this property for detection, and the extensive results demonstrating the method’s strong performance. However, reviewers also raise concerns, including limited technical novelty, the method’s potential failure on real images that differ from the training distribution, its heavy reliance on DINOv2 model properties, and the lack of comparison with other uncertainty quantification (UQ) methods.  Additional questions were raised about the sensitivity of the method to changes in perturbation distributions and to the data used to fine-tune the model. The authors’ rebuttal is extensive and detailed, with new results that largely address most of the reviewers’ concerns.
Decision: Accept (poster)