Breaking Semantic Artifacts for Generalized AI-generated Image Detection
Authors: Chende Zheng
Chenhao Lin
Zhengyu Zhao
Hang Wang
Xu Guo
Shuai Liu
Chao Shen
Venue: NeurIPS 2024 poster (2024)
Keywords: AI Security; Deepfake Detection; AI-generated Image; Diffusion Model;
Primary area: other
TL;DR: The paper introduces "semantic artifacts" in cross-scene images, which leads to performance drops on generalized detection. A dataset from various generative models and scenes is built and a simple yet effective patch-based method is proposed.

Abstract: With the continuous evolution of AI-generated images, the generalized detection of them has become a crucial aspect of AI security. 
Existing detectors have focused on cross-generator generalization, while it remains unexplored whether these detectors can generalize across different image scenes, e.g., images from different datasets with different semantics. In this paper, we reveal that existing detectors suffer from substantial Accuracy drops in such cross-scene generalization. In particular, we attribute their failures to ''semantic artifacts'' in both real and generated images, to which detectors may overfit. To break such ''semantic artifacts'', we propose a simple yet effective approach based on conducting an image patch shuffle and then training an end-to-end patch-based classifier. We conduct a comprehensive open-world evaluation on 31 test sets, covering 7 Generative Adversarial Networks, 18 (variants of) Diffusion Models, and another 6 CNN-based generative models. The results demonstrate that our approach outperforms previous approaches by 2.08\% (absolute) on average regarding cross-scene detection Accuracy. We also notice the superiority of our approach in open-world generalization, with an average Accuracy improvement of 10.59\% (absolute) across all test sets. Our code is available at *https://github.com/Zig-HS/FakeImageDetection*.

Breaking Semantic Artifacts for Generalized Al-generated Image Detection
Chende Zheng' Chenhao Lin Zhengyu Zhao'
Hang Wang
Xu Guo Shuai Liu Chao Shen'
School of Cyber Science and Engineering, Xi'a an Jiaotong University
Schooo of Automation Science and Engineering, Xi'an Jiaotong University
School of Software Engineering, Xi' an Jiaotong University
zhengchende stu.. ..rresstu...tee. xjtuuedu..
(linchenhao, zhengyu. zhao, cshangwang) @x edu.cn xuquo@stu. wit.uu.jtu....... {sh 1iu, chaoshen) seaaii.xxjtuedu..c.
Abstract
With the continuous evolution of Al-generated images, the generalized detection of them has become a crucial aspect of Al security. Existing detectors have focused on cross-generator generalization, while it remains unexplored whether these detectors can generalize across different image scenes, e.g., images from different datasets with different semantics. this paper, we reveal that existing detectors suffer from substantial accuracy drops in such cross-scene generalization. In particular, we attribute their failures to "semantic artifacts" in both real and generated images, to which detectors may overfit. To break such "semantic artifacts", we propose a simple yet effective approach based on conducting an image patch shuffle and then training end-to-end patch-based classifier. We conduct comprehensive open- world evaluation on 31 test sets, covering 7 Generative Adversarial Networks, 18 (variants of) Diffusion Models, and another 6 CNN-based generative models. The results demonstrate that our approach outperforms previous approaches by 2.08% (absolute) on average regarding cross-scene detection accuracy. We also notice the superiority of our approach in open-world generalization, with an average accuracy improvement of 10.59% (absolute) across all test sets. Our code is available
Introduction
Recently, the rapid development of Al-generated image technology has led to the emergence of synthetic images the internet, raising significant concerns about Al security. These images exhibit remarkable diversity due to the continuous introduction of new generative architectures, e.g., from Generative Adversarial Networks (GANs) [1] Variational Auto-Encoders (VAEs) (2), to Diffusion Models (DMs) B For now, Internet users can easily generate a large number of exquisite images using text prompts. As the quality and variety of these synthetic images continue advance, developing a universal detector for fake images becomes a crucial aspect of Al security.
To determine whether an image is synthetic, existing studies typically train binary classifier to assess the authenticity of an unknown image during inference. Previous methods have primarily focused on cross-generator detection. For instance, Wang et al. 4 developed ResNet50 trained images from proGAN (5), and then tested it on different GAN variants. Intriguingly, their findings suggested
"Corresponding Authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

that aggressive augmentation strategies could significantly enhance the classifier's generalization abilitty. However, some recent studies 6 7 have shown that Wang's approach does not extend well to images synthesized by unseen DMs, even when DM-generated images are included in the training set. To address the problem of this cross-generator generalization, recent advancements have focused on refining detection algorithms (8] using large-scale pre-trained models 7 TO), and augmenting or obtaining diverse datasets [1] In addition, Corvi et al. [3] demonstrated that images of one specific generative model contain unique artifacts that differ from those of other generators. These "generator artifacts" are particularly distinct between generators with different architectures, such as DMs and GANs, providing a promising avenue for improving tross-generain generalization.
Despite these efforts, remains unexplored whether these detectors can generalize across different image scenes, c.g., images from different datasets with different semantics. Notably, Dogoulis et al. 14] have discussed a similar setting. However, their study focuses solely on different concept classes (e.g., objects) and does not consider a wider range of content. To fill this research gap, we propose a more comprehensive cross-scene problem and solution.
By analyzing the residual spectrum of images, we identify a significant challenge arising from the "semantic artifacts" in both real and generated images. Our findings reveal that images from different datasets contain unique artifacts, which can be inherited by generative models. During the training, existing detectors tend overfit the specific artifacts of the training data, resulting in substantial accuracy drops in cross-scene generalization, For example, classifiers trained on images generated by Latent Diffusion trained on LAION struggle to generalize to images generated by the same diffusion method trained on FFHQ. Intriguingly, while "semantic artifacts" significantly impact the decrease in aceuracy, the Average Precision of detectors remains high. This issue was also observed in 11]. but comprehensive exploration is lacking.
To address this challenge, our work focuses on mitigating the impact of "semantic artifacts" for generalized Al-generated image detection. Specifically, we experimentally find that detectors based on deep networks tend to focus more on the global semantics of images, implying their classification may rely on specific scenes. To counter this, we propose a simple yet effective approach based on conducting an image patch shuffle and then training an end-to-end match-based classifier. Our approach aims to extract artifacts of generators in local patch while breaking the global "semantic artifacts". Meanwhile, by reducing receptive fields, our approach minimizes the excessive learning of "generator artifacts", benefiting cross- cross-generator detection. To comprehensively evaluate the generalization ability of our approach, we conduct an open-world evaluation on 31 test sets, covering 7 Generative Adversarial Networks, 18 (variants of) Diffusion Models, and another 6 CNN-based generative models. Our experiments demonstrate the effectiveness of our approach across extensive evaluations. Our main contributions can be summarized as follows:
We innovatively identify "semantic artifacts" in cross-scene Al-generated image detection, which leads to poor generalization performance of existing detectors.
We propose simple yet effective patch-based approach, aiming at breaking the "semantic artifacts" " for generalization detection. By extracting local features, our detector is able to reduce the impact of global semantics in images.
Extensive experiments validate the effectiveness of our approach with an improvement of 3.81% % average accuracy (absolute) in cross-scene detection (6 variants of Latent Diffusion with different real datasets) and an improvement 6. 6.74% average accuracy (absolute) in open-world generalization (all 31 test sets).
2 Related Work
2.1 Generative Models
Generative models aim at learning to create new samples from given dataset, with the fundamental goal of capturing the underlying probability distribution mapping. Prominent approaches in this field include Variational Auto-Encoderr (VAEs) 2), Auto-regressive Models (15), Flows-based
Models [6]. Generative Adversarial Networks (GANs) [1. and Diffusion Models (DMs) K Early
methods like GANs have achieved impressive realism in image synthesis on specific categories, leading to the development of the proGAN/styleGAN series [8].

---

More recently, DM-based synthesis methods, exemplified by Latent Diffusion [9], have rapidly advanced in text-to-image generation. Breakthroughs like Stable Diffusion, DALL-E [20], and Imagen [1 are released in quick succession. These large-scale image synthesis models have significantly expanded the application scope of image synthesis, enriching the semantic content and stylistic features of generated images. The evolution makes the detection of Al-generated images increasingly complex and challenging.
2.2 Defecting Synthetic Images
Prior to the advent of advanced DMs, significant body of forgery detection research focused on images generated by GANs. Early detection methods targeted local artifacts present in synthetic im- ages, including distortions in facial tampered landmarks [22] or inconsistencies in head postures [23) In [24], it was demonstrated that global artifacts synthesized images differ markedly from com- mon artifacts found in modern digital devices. Additionally, the up- samplin operations prevalent in most GAN architectures tend to produce distinct peaks in the spectral profiles of synthesized images 125 26 offering another avenue for detecting fake images.
noteworthy question that arises is whether these methods are also effective in detecting DM- generated images. Wang [4] suggests that simple classifiers when trained with aggressive data augmentation on proGAN images. can be adapted other unseen models. Ojha 7 use a fixed pre-trained CLIP: ViT-L/14 to extract image features, while Sha (27) incorporate hint information and utilize multimodal inputs (text prompts and images) to enhance performance. Tan et g introduce the concept of neighboring pixel relationships as means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. In particular, recent studies [2] [9] have identified significant differences in the frequency spectra of images synthesized by GANs and DMs, impacting the generalization capability of existing detectors. Interestingly, a common issue is observed when detectors are extended to unknown data [12). Specifically, these detectors will show a disparity where high Average Precision accompanied by low accuracy. Our analysis of generalization detection of Al-generated images reveals similar problem, showing performance imbalances when dealing with different semantics.
Methodology
3.1 Artifacts Analysis
To develop a universal detector for Al-generated images, it is crucial to thoroughly examine the distinctive artifacts of synthetic images generated by various generative models. Inspired by 13.77 we start our analysis by visualizing the frequency spectra of different image distributions. This exploration aims to uncover any unique intriguing properties of the artifacts associated with different generators or scenes.
In line with the experimental design in 30 29],
we randomly select set of images with quantity of 1000 for each image source and utilize the denoising filter D(x;) from 31 to extract the noise residuals r; of an original image ø;;
(m,, n) = [, n) ((x.(m,n)) n)) i= 1,2,,....
Then, we start from the Fourier transform F of the noise residuals of the M x N image:
M
N
F,(K,
Σ
Σ
r((m,
j2al 2s m+ gn)
m=
where, is and represent the frequency domain coordinates. To obtain the artifacts of the image source, we take the average power spectrum S of all single images from it:
(k,,
(3)
We use images from our test sets (see Section to conduct preliminary experiments for analysis of the artifacts in both synthetic and real images.
3

---

DOIM
PNDM
Latent Difrusion
DALLE 2
GLIDE
SYCNAANN
JarCAN
Deenfake
CRN
LAION (real)
Figure 1: Generator artifacts: noise residuals power spectrum of images generative models and real dataset. Top row: Diffusion Models. Bottom row: 2 GANs, cycleGAN and starGAN, CNN-based generators, Deepfake and CRN. and I real dataset, LAION.
FaraFaransics
CelebA
LAION
óocotaic (Fancearendcss)
STDD (Row Camera)
Latent Diffusion (CelobA)
Latont Diffusion (FFHO)
Latent D fusionn (LAION)
Figure 2: Semantic artifacts: noise residuals power spectrum of images from different seenes. Top row: real datasets. Bottom row: generative models corresponding seenes, deepfake, SITD, and variants of Latent Diffusion CelebA, FFHQ. and LAION.
Generator artifacts. Figure shows the noise residuals power spectrum of generated images. For GANs, have observed periodic artifacts in the spectrum, identified checkerboard artifacts produced by up-sampling (25). Similar artifacts are also observed in some DMs, such as DALL-E and Latent Diffusion. These DMs employ an encoder architecture compress images into latent space features, resulting in periodic artifacts akin to those found in GANs. While other CNN-based generators, e.g., Deepfake or CRN, exhibit unique artifacts that are significantly different from GANs or DMs. Interestingly, although cycleGAN and starGAN are both based on GAN architecture, they still show some differences in the spectrum. Overall, images from different generators exhibit different artifacts and generated images with unique artifacts might lead detectors to overfit during training, thereby failing to recognize distinct artifacts of cross-generator data. term these artifacts as "generator artifacts."
Semantic artifacts. Figure 2 shows the noise residuals power spectrum of real and synthetic images from 5 distinct scenes (FaceForensics, Raw Camera, CelebA, FFHQ. LAION), To our surprise, artifacts are also observed in real images. For example, compared LAION, CelebA and FFHQ exhibit artifacts around the center of the spectrum while Raw Camera displays artifacts similar to the periodic artifacts GANs. Meanwhile, our findings suggest that these artifacts can inherited by generative models (see Deepfake trained on FaceForensics, SITD trained on Raw Camera, and Latent Diffusion trained on CelebA/FFHO/LAIONN These artifacts might originate from the semantics of the images or the preprocessing methods of the datasets. For example, CelebA and FFHQ share

---

Training
Acc.
Test wlo patch shutlle
Test patch shuffle
Bedroom
Church
Bedroom
Church
Training
Metrics
Test
50%
99.99
50.00
100.0
51.18
Bedroom
Church
Bedroom
10%
99.99
50.00
100.0
56.11
99.94
Acc. (Real)
99.80
100.0
0.1%
99.87
50,97
99.61
82.45
Bedroom
Acc. (Fake)
100.0
0.00
0.01%
99.27
62.03
97.94
91.59
100.0
99.90
0.001 %
95.18
85.43
91.92
97.01
Ace. (Real)
100.0
100.0
50%
50.00
100.0
94,15
100.0
Church
Acc. (Fake)
0.00
100.0
Church
99.29
AP
98.20
100.0
1%
52.67
99.96
98.71
99.96
0.1%
77.42
99.77
94.47
99.69
Table 1: Cross-scene detection experiments on ResNet-50 models from ForenSynths [4] We use 2 sets of training images on differ- ent scenes, Bedroom and Church, to retrain the detectors. Detection accuracy (Acc.) (at threshold of 50%) and Average Precision (AP) are reported.
0.01%
95.32
99.29
84.64
96,99
0.001
86.49
96.17
72.61
88.96
Table 2: Accuracy different thresholds) results of ResNet- 50 models. We report the performance of the ResNet-50 models with and without the implementation of Patch Shufflc. Patch Shuffle is employed during both training and testing.
Bedroom (fake)
Bedroom (fike)
Church (fake)
Church (fake)
(a) Original
(b) ResNee-50 trained Bedroom
(c) ResNet-50 Inained on Chunch
(d) ResNet-50 trained Bedroom with Patch Shuffle
(e) ResNet-50 trained on Church with Patch Shufele
(0 Our appronch
Figure 3: The visualization of CAM extracted from different detectors Bedroom Church images. Warmer color indicates a higher probability.
similar artifacts around the center of the spectra since both contain images of human faces. Notably, although FaceForensics is also dataset of human faces, its significantly different preprocessing method (cropped from video and resized) results in unique artifacts. We term these artifacts as "semantic artifacts."
3.2 Analysis of the Impact of Semantic Artifacts
have focused primarily on cross-generator detection of Al- generated images and have made good efforts on generalization across "generator artifacts".. However, remains unexplored whether these detectors can generalize on cross-scene detection, especially since have found that "semantic artifacts" have many properties similar to "generator artifacts".
Many previous researches I0 g
5

---

Patch Shuffe
Petch-BasooFeaturr Extraction
Classification
Falia Inanges
M x paiches
Couseitell
Rebure
N Layers
JeskBRock
Reali-atoon
Classifler
potchi
Scature M x M4
Roal Iunges
Seotroe X
Figure 4: Pipeline of our approach. First, for pre- pro-proessingg we divide the input image into patches and shuffle these patches to obtain a randomized sequence. Then, we train a patch-based convolutional network for feature extraction. Finally, we flatten these features into one-dimensional vector and then apply linear classifier for classification.
To explore the impact "semantic artifacts", we conduct several preliminary experiments using the existing paradigm [4]. This method is widely adopted in current studies [0], where researchers train a ResNet-50 using images from only one generator to detect images from other generators.
Specifically, we use sets of datasets for cross-scene tests. We randomly select 15000 images from LSUN bedroom and LSUN church, respectively, and generate 15000 images by SDXL-Turbo using prompts of "A photo of bedroom" and "A photo of church" respectively (10000 training set and 5000 as test set). We train the detectors respectively on two training sets and evaluate them on cross-scene detection.
In Table D we report the accuracy (Acc.) and Average Precision (AP) of the detectors on different scenes. As we can sec, the detectors perform exceptionally well (almost 100%) on images with the same semantics the training set. However, when tested on cross-scene images, the detectors suffer from significant drops in Acc. (near 0.00%) ) for generated images. Interestingly, the AP remains high, indicating that the detectors can still differentiate between real and fake images, although the distinction in scores between real and fake images is relatively minor. To further analyze the Table Typically,
reasons for the low Acc., we report the Ace. results at different thresholds the threshold for determining whether an input image is real or fake is set to 50% this task. However, in Table we observe that the optimal threshold for Al-generated image detection exceptionally low, ranging from 0.01% (from Church to Bedroom) even 0.001% % (from Bedroom to Church). This demonstrates that the average score of fake images with different semantics is extremely low and the score difference between real and fake images is minimal. Unfortunately, such a low threshold may lead to an increase in false alarms when handling various images. Furthermore, the optimal thresholds vary across different scenes, e.g.. 0.01% from Church to Bedroom and 0.001 % from Bedroom to Church. Therefore, simply adjusting the threshold does not resolve the issue of cross-scene generalization.
attribute the poor performance of the existing paradigm to the overfitting of semantic artifacts during training. To address this, we simply apply Patch Shuffle preprocessing to break the global semantics of images. The results are shown in Table As observed, there an improvement in Acc. each threshold from Bedroom to Church, indicating larger difference in scores between real and fake images. Moreover, from Church to Bedroom, the Acc. at the 50% threshold reaches 15%, highlighting the possibility of using Patch Shufile to mitigate the impact of "semantic artifacts".
Figure B reports the visualization CAM extracted from detectors on generated images Bedroom or Church. Notably, detectors without Patch Shuffle struggle to focus on generated artifacts of images from different scenes. Due to detector's limited ability to capture specific semantic content, there are almost no active regions in CAM of images from different scenes, e.g., (b)-3, (b)-4, (c)-1, (c)-2 in Figure 3 In contrast, detectors with Patch Shuffle demonstrate more active regions, unaffected by specific semantic content, e.g., (d)-3, (d)-4, (c)-1, (c)-2 in Figure

---

3.3 Approach
The primary goal for cross-scene generalization to break the impact "semantic artifacts". In our preliminary experiments, simple Patch Shuffle has proven to be an effective method. However, overfitting still exists, e.g., from Bedroom Church, the Acc. 50% threshold of detectors with Patch Shuffle is 51.18% (see Table 2) We hypothesize that the deep residual structure of ResNet-50 promotes the extraction of overall semantic information. Although Patch Shuffle has been applied, the detectors still learn some semantic features between patches. A strong piece of evidence is that the visualization of CAM extracted from ResNet-50 with Patch Shuffle shows large, contiguous areas of activated regions, e.g., (d) and (e) in Figure 3 rather than discrete, point-like activated regions seen in our approach, e.g., (f) in Figure
Based on this hypothesis, we propose a simple yet effective approach based on conducting an image Patch Shuffle and then training an end-to-end patch- bush- classifier. The whole pipeline of our approach is shown in Figure
Pre-processing. First, given input image X, we divide into several patches of size P x P:
{pr} = Segment [X, P) P) i = 1,2.... M M
where M M is the total number of patches obtained after segmentation. We further shuffle these patches to obtain a randomized sequence {py}( = 2, M x M), which aims to break the global "semantics artifacts".
Patch-based feature extraction. We employ a patch-based based convolutional network for feature extraction, which is designed to accept only one single patch input, aiming at extracting local features. For each patch Py. we first apply a convolutional layer Co to extract an initial feature map Fo. Subsequently, series of same-convolutional blocks {Cn) used for deep feature encoding.
Fo = Co(p))
(5)
F C,(nn1)), 1,2, N
(6)
where N a typer-parameter representing the number of same-convolutional blocks. After encoding, the feature map is downsampled through basicblocks.
Linear classifier. The features of each patch {Fx) are reassembled into a feature sequence We flatten the feature sequence into a one-dimensional vector v, followed by linear classifier L:
i = L(v)
We optimize the whole pipeline (including both the feature extractor and linear classifier) using binary cross-entropy loss:
1 N Σ lys log(ūe) ys) log(1 ú:)
BCE(y,
(8)
where y represents the true labels, and á represents the predictions.
4 Experiments
4.1
Experimental Settings
Training dataset. To evaluate detectors' generalization capability, the standard practice is restricting the training data to images from only one generator and scene. We employ generated images from DiffusionDB [32]. The training set consists of 48,000 images generated by Stable Diffusion v1.4 using prompts from Internet Users and 48, 300000 real images from LAION-5B [33]. To ensure consistent basis for comparison, also employ the training set of ForenSynths 44, in line with experimental design 7 10 as an ablation experiment in Section A.3.
Test datasets. To evaluate the generalization of baselines and our approach, we conduct com- prehensive open-world evaluation on 31 test sets, covering 7 Generative Adversarial Networks, 18 (variants of) Diffusion Models, and another 6 CNN-based generative models.
7

---

Diffusion Models. We collect DMs, including iDDPM 34 DDIM B5I. PNDM [6).
Guided- Saided-Diffusion (ADM) B7. RDM
38
Latent Diffusion (LDM) T9 Stable Diffusion
v1.4 (SDv1) and v2.1 (SDv2), GLIDE 39 and DALL- 2 20 We sample images from
their variants of real datasets, including LSUN-bedroom,, LSUN-church
40
CelebA 41
FFHQ [7]. ImageNet 42.. and LAION-5B [3 Details refer to Section
Generative Adversarial Networks. The test sets include fake images generated by 7 GANs from ForenSynths 44, including proGAN [5], styleGAN, styleGAN2 [7], bigGAN 43
cycleGAN 44, starGAN [45,, and gauGAN [46]
Other CNN-based generative models. The test sets include fake images generated by 6
other CNN-based generative models from ForenSynths [4]. including CRN [47], IMLE (8
SAN 49
SITD [50), deepfake [51] and WFIR 122
Detector baselines. We perform com- parisons of our approach with exist-
ing popular and state-of-the-art detectors on our open-world datasets, including
Methods
Vasitats
Cross Croo-Scene
Woel-World
Arg. Acc.
mAP
Ang. Acc.
mAP
Durall (CVPR 2020) [53]. CNNDetec- tion (CVPR 2020) [4]. PatchFor (ECCV 2020) 66) F3Net (ECCV 2020)
CNVPCV
BIa (0.1)
Bla (0.5)
53.66
54.16
67.02
55.34
65.27
64.13
ResNetlS
PucchFo (BCCV
Xeqution
00.91
75.26
57.82
64.27
(54
FiNet
68.87
Dogoulis (MAD'23) [1], DIRE (ICCV
FeNet (ECCV'20)
LPS
57.53
79.36
55.07
76.66
Both
85.39
74.54
2023) (55) Ojha (CVPR 2023) 77. LGrad
Durall (20)
SVM
64.23
59.30
55.87
53.76
(CVPR 2023) [IO], NPR (CVPR 2024) [8]
CR
68.28
64.81
59.15
53.74
We re-implement baselines 53
4 6
54
Celeb/ SDv2
63.19
69.33
57.05
63.81
DIRE (ICCV220)
ImugrNa-ADM
58.35
66.73
54.32
99.07
10 [] with the official codes using our train- ing set, and adopt the official pre-trained weights of baselines (55]
LSUN.ADM
66.64
65.66
56.20
56.70
Dogonlis (MAD 23)
Top 10k
$2.70
55.35
$3.59
56.96
60.35
Gjiu (CVPR'23)
CLIP VIT-L/144RC
66.38
79.36
67.12
81.28
Liinad (CVPR 23)
62.91
80.99
57,00
76.91
NPR (CVPR'24)
50.44
94.84
75.38
56.76
Implementation details. Our approach is implemented using the PyTorch on NVIDIA A100 40GB Tensor Core GPU. During the training, we perform zero padding the images to ensure the shorter edge is 256 pixels (resizing is also used another pre-processing pipeline for ablation), and then randomly crop the images to 256x256. Each image will be horizontally or vertically flipped with a probability of 50% data aug- mentation. The number of same-convolie convoluonoo blocks N is set to 18 and the size of each patch P is set to 32. The detector is trained using the Adam optimizer and strpy-stop strategy with an initial learning rate of 1e 4, minimum learning rate of Te - 6, and batch size of 64. We separate 5,000 images from the training set to serve as the validation set. We use the accuracy (Acc.) and the Average Precision (AP) as the evaluation metrics, with particular emphasis on Acc. as our primary metric, because as discussed in Section 3.2 the AP metric may not fully reflect the problem of cross-scene generalization.
Ous
Zer pabling
Resizing
92.53
96.38
95.58
85.97
$2.12
90.00
88.36
Table 3: Results of cross-scene generalization and open- world generalization. For cross-scene generalization. we average the results on 6 variants Latent Diffu- sion (LSUNN-eddrom,, LSUN-Church, ImageNet, CelebA, FFHQ, LATON). For worldw generalization, we average the results on all 31 test sets (including 18 DMs, 7 GANs, and CNN- -bsed generators). Bold represents the best and underline represents the second best. More Detailed results are shown in Table and Table 3
4.2 Experimental Results
Cross-scene generalization. We first report the cross-scene detection results in Table We average the results on variants of Latent Diffusion (LSUN-Bedroom,, LSSN-Church, ImageNet, CelebA FFHQ LAION). The results demonstrate that most detectors suffer from performance drops on cross- scene images, even though the images are generated by models with the same structure. Particularly. the gap between AP and Acc. can be observed in several baselines, e.g., CNN (54.16% Acc. and 67.02% AP), Ojha 666.8% Acc. and 79.36% AP), F3Net (56.82% Acc. and 85.39% AP), LGrad (62.91% Acc. and 80.99% AP), which consistent with our analysis in Section (.2 As a comparison, the best baseline NPR and our approach show little gap between Acc. and AP. Considering that NPR uses Neighboring Pixel Relationships as features, which benefits breaking the global semantic artifacts, its high performance (90.44% Acc. and 94.84% AP) aligns with our hypothesis. Despite
8

---

Chesh
FUPRO
LAIDK
Aunaag
Meebok
Valiaa
BBM
IDEPP
FROM
Auc
LOM
DEAM
LON
LDM
AJM
LDM
REM
LIM
DMLLES
LIN
SUST
SIV2
SIN2-H
AP
CNN
$2.90
$7.00
50.30
50.55
91.31
$5.59
53.15
40.75
$1.00
94.35
51.00
$2.10
50.5
$5.08
47.40
$5.05
62
90
45,40
5010
13:15
61,88
$7.30
72.6
75
00.50
55.27
66.20
Xcapkk
51:35
5U15
51.25
51.00
5A11
66.30
49:35
$2.50
21.30
54.80
50.70
$2.00
99.25
$2.35
60.00
07.35
62.
FINct
30.1
70.00
50,0
30.3
20,13
30000
90.00
60.33
42.35
$2.65
75.83
2045
43.59
06.00
$5.23
76.70
$4.30
60.23
77.31
P3N1
1278
91.00
50.30
90.0
91.03
50.6
90.10
$1.35
54.65
11.31
76.15
91 10:30
10
53.35
55.59
60.03
67:0
56.70
85:45
55
$0.50
Red
53.05
50.00
50.00
51.11
47.25
66.30
51.30
50.00
56.38
11.30
05.75
91.55
55.01
12.35
CABR 5063
$8.40
62.30
55.90
53.55
66.15
12.15
71.81
13.82
65.93
10:35
$1.55
37.35
-563
50-55
51.55
59.36
03.05
7001
DIRE
90.00
51.70
-9.70
47.33
5135
51.00
49.25
73.55
42.70
$1.00
$1.45
59.00
54.23
03.59
-7.14
43.85
-0.95
52.77
15.00
6000
ESTI ATM
91.93
04:95
91.91
91.30
5046
91.00
93:15
66.65
94.33
15.00
19.65
47.01
$3.35
$3.01
53.35
$3.85
75.67
68:14
61
Degoalk
Thp
56.35
90.35
$3.50
17.05
53.55
66.00
$1.05
18:15
50.-0
02:43
85:03
55.66
$1.01
54.90
56.30
$1.35
5160
93.55
58:12
94.55
9:00
$1.55
$3.95
53.25
14.19
610
37.55
(6:40
64:71
25.74
XPR
$2.90
56.90
$4.00
95.75
56.30
5:00
$3.65
62.35
$4.55
96.00
37.55
64:45
77.35
18.15
90.00
65.30
04.05
$9.30
75:35
9466
Roideg
$2.00
99.00
13.55
59.58
15.85
89.25
96.57
59.05
02.50
11.45
13:0
11.00
Table 4: Cross- Mooss-Diffssion generalization results. We evaluate the detectors on all 18 variants of Ditfusion Models.
Mathoos
Viriants
pexcIN
gydeEsAN
11teGAN
eykGAN2
pHIGAN
cSIGAN
dkeptikk
SITD
SAN
IMLI:
WIIR
Assrage
Atc.
Acc.
Acc.
Avc.
Aus.
Auc.
Acc.
Avc
Adc.
Acc.
Asc.
Aoc.
Aot.
AP
Bier-JBrO/
$1.00
50.00
49.60
51.16
65:95
53.20
62.25
51:11
56.82
74.45
46.50
57.41
59.25
49.28
-9.35
51.22 25
97:57
06.11
53.00
00.90
55.06
09:59
69 855
46.90
54.77
00:78
PaschFoe
Nagytos
54.25
52.00
51.21
52.15
51.70
53.35
50.75
33.65
59.35
50.44
65.68
6132
50.15
56.70
56.35
50,05
06.67
72
50.23
36.25
$3.50
50.80
50
16.50
94
51.70
49.05
51.83
52.91
57.53
16.75
F3hhtt
3010
46.39
22.15
09.33
31.31
63.31
31.10
10/3
41:
47.12
40135
30.15
30.35
$1.41
57.15
PINe
50445
50.75
90.20
65.06
50.15
30.05
08.65
59.15
Duall
SVM
00:90
21:20
16.70
58.20
$2.00
74.90
23.30
49.40
43:90
94.20
76:00
30,00
15.90
$2.00
19.50
76.00
16.20
57.90
96.30
57:40
04.63
53.05
51.63
13.51
CadaA 3D12
43.24
33330
23:20
3233
3155
13:55
3305
30.05
17:45
15.30
64.15
16330
41.50
31.34
55.14
IIKE
ImageNee ADM
53:13
52.70
48:25
53.65
57.65
66:60
49:00
45.90
50.57
51.00)
51.10
52
22:45
57.95
51.35
56.53
LSUN
$2.54
50.45
$0.65
$1.65
$1.00
50.90
43:85
50,00
50.00
$1.25
50.00
50,00
50.50
50.57
49.46
Degoots
Top 10k
49.55
04.99
48.31
65.96
55.00
90.05
15.06
91.91
43-40
56.115
46.75
10.86
51.68
Top
91:25
74.90
75.05
84.75
71.25
73.05
72.30
19:72
64.38
55.15
68.30
05.87
83.74
TGrvl
59.75
16.55
55.00
15.37
6539
57.25
51.90
15.56
51
57.85
87 70
19.00
14.35
6665
NFT
94.75
95.14
6365
61.15
55.33
55.76
99.35
51.20
56.35
56.82
55.05
58.16
51:00
71.31
$1.75
Ous
Reening
padding
36.30
$4.77
89.85
11.00
00.25
05.30
71.54
36.35
60.66
91.46
98.55
95.30
66.90
51.01
10.28
66.35
61.00
74.89
27.00
58 35
101
71.35
11.10
$0.33
71.36
$6.97
61.12
Table 5: Cross- GAN/CNN generalization results. We evaluate the detectors on all Generative Adversarial Networks and 6 CNN -aased generative models.
this, our approach still outperforms the SOTA generalization performance, showcasing higher Avg. Acc. and mAP metrics, which reach 92.52% and 95.58% (based on zero padding).
Open-world generalization. Table 3 also presents the Avg. Acc. and mAP metrics of detectors across the 31 test sets. The open-world evaluation results demonstrate an improvement in Avg. Acc. of our approach, outperforming Ojha and NPR by 18.85% and 10.59% respectively. This indicates the significance of breaking semantic artifacts and focusing on local features in generalized detection. The visualization result, e.g., the discrete, point-like activated regions in Figure 3 (f), further confirms the effectiveness of our approach that predominantly focuses on learning local features. Detailed results of our open- experiments are presented in two groups.
generalization. Table presents Presents Acc. and mAp results across images from 18 (variants of) Diffusion Models. The results demonstrate the outstanding performance of our approach on cross-Diffusion evaluation. Our approach reaches Avg. Acc. of 90.18% outperforming the best baseline NPR (7).35%). Notably, most basclines show their poor performance on the unconditional generation scene, e.g., Bedroom and Church. We attribute it to a strong scene change, where there is a significant difference between the txt2img generation scene (training set) and the unconditional generation scene (test sets). In addition, the gap between Acc. and AP still exists, e.g., NPR (78.35% Avg. Acc. and 94.08% mAP).
Cross-GAN/CNT generalization. Table presents the Acc. and mAP metrics of detectors across images from 7 Generative Adversarial Networks and another 6 CNN-based generative models. As we see, most detectors suffer from performance degradation on GANNNNG Generalization, which is expected because these generators use different architecture from

---

the Diffusion Model (training set). Nevertheless, our approach still shows advantages with SOTA accuracy while Ojha has SOTA mAP but the gap between Acc. and AP still exists.
Effect of pre-processing pipeline. So far, we have seen excellent generalizability in our patch- based method. We next explore in depth the influence of pre-prceessig pipelines on artifact altering and detection. Specifically, we adopt both image padding and resizing during pre-processing. Tables 4and demonsstatee the superior results of zero padding compared to resizing, especially in cross- GAN/CNN generalization. In particular, the performance of our approach using padding has been largely boosted on SITD, SAN, CRN, IMEL, and WFIR compared to resizing. It can be attributed to their high image resolution, which introduces variations in artifacts and the loss of low-level features when resizing is applied. Mcanwhile, Figure in Section A.4 also supports this finding by showing that the frequency features of SITD (with image resolution of over 4,000x3,000) images change a lot after resizing. In summary, these results suggest that zero padding can preserve more low-level features in images, which benefits generalized detection.
Effect of model depth and patch size. We next delve into the interplay between receptive field and generalization abil- ity. Specifically, we adjust the number of same-convolution blocks (N), to vary the receptive field size and the quantity of parameters in the feature extractor (no- tably, resizing is used as preprocessing in the experiments because it does not affect the global receptive field). Ta- ble [6 reports the Avg. Acc. and mAP results across different N values. The re- sults demonstrate that both too-shallow (N=0) and too-deep (N=24) layers re- sult in performance degradation due to underfitting and overfitring, respectively. Intriguingly, the best N value for Cross- Diffusion generalization is 21 while the best for GAOSCNN generalization is 6. Considering that images of the train- ing set are generated by the Diffusion Model, we attribute the results to the fact that shallow model might learn less from "generator artifacts" of training images and perform better on cross-generator detection. Con- versely, this strategy will disrupt its fitting of in-domain artifacts, resulting in worse detection of in-domain datasets. Then, we conduct experiments to explore the effect of different patch sizes. The P value represents the size of each patch, which determines the number of patches and the degree of breaking global semantic artifacts. Table 7 reports the Avg. Acc. and mAp metrics across different P values. As expected, a larger P value, e.g., P=48 or P=64, enlarges the receptive field, potentially exacerbating overfitting issues. Conversely, a too-small patch size, e.g., P=16, can lead to significant underfitting. This finding encouraging, it aligns with the effect of model depth and benefits a deeper investigation into the mechanisms of generalized detection.
Cross Diffinsion
Cruss GANNCN
Open World
Methods
Avg. Avc.
mAP
Avg. Acc.
mAP
Asg. Aoc.
mAP
Po64; N-o
81.53
87.48
63.55
66.42
73.99
78.65
Pw64; Ned
82.71
89.15
73.37
82.53
Pw64; N-6
90.96
74.27
80.40
$1.28
86,53
Na9
91.17
26,06
80.75
$6.80
N-11
$7.09
91.85
73.37
79.52
36.68
P-64, N-15
92.85
76.36
85.93
P=64, N=18
92.43
72.91
84.25
P-64, N-21
88,91
93.03
76.49
86.09
P-64, N-24
91.65
72.28
83.53
Table 6: Ablation study results on model depth.
Cross- Diffission
Cruss GANNCN
Open Worl-
Methods
Avg. Auc.
mAP
Arg. Acc.
mAP
Avg. Aoc.
mAP
P-64, N-18
88.60
02.43
67.96
72.93
79.95
84.25
P=48; N=18
$8.58
92.08
20.78
76.79
$1.11
33.67
Pw52; N-18
$9.01
93.58
72.58
81.12
81.12
88.36
P-16, N-18
84.35
91.29
67.13
77.41
77.13
85.47
Table 7: Ablation study results on patch size.
Conclusion
In this paper, we introduce the concept of "semantic artifacts" in the generalization detection of Al-generated images. We analyze the existence and impact of "semantic artifacts" in cross-scene detection through extensive experiments. Our results demonstrate that existing detectors suffer significant drops in accuracy when applied to images with different semantics, which we attribute to overfitting during training. To address this issue, we propose simple yet effective approach that utilizes Patch Shuffle and trains an end-to-end patch-based classifier to break the "semantic artifacts" in images. Extensive experiments on 31 test sets validate the effectiveness of our approach, demonstrating our contributions to the universal detection of Al Al-generated images.
10

---

Acknowledgments
This research is supported by the National Key Research and Development Program of China (023YYBB1104001) the National Natural Science Foundation of China (T2341003, 62376210, 62161 160337, 62132011, U21B2018, U20A20177, 62206217, 6240071050, 62406240), the Shaanxi Province Key Industry Innovation Program (2023-ZDLGY-38), the Natural Science Basic Research Plan in Shaanxi Province of China (2022JQ-631)
References
[1] Ian Goodfellow, Jean Pouget-Abadis Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[2] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 2013.
[3] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[4] Sheng-Yu Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Can- generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8695-8704, 2020.
[5] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.101900 2017.
[6] Lucy Chai. David Bau, Ser-Nam Lim, and Phillip Isola. What makes fake images detectable? understanding properties that generalize. In Computer Esion-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVI 16, pages 103-120. Springer, 2020.
[7] Utkarsh Ojha. Yuheng Li, and Yong Jac Lee. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14480-24489, 2023.
[8] Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethink- ing the up-sampling operations in cnn-based generative network for generalizable deepfake detection. arXiv preprint arXiv:2312. 10461, 2023.
[9] Agil Aghasanli, Dmitry Kangin, and Plamen Angelov. seerprttbbe-ttrroug--rroooo deep- fake detection for diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 467-474, 2023.
[10] Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. Learning on gradients: Generalized artifacts representation for gan generated images detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12105-12114, 2023.
[11] Yonghyun Jeong. Doyeon Kim, Youngmin Ro, and Jongwon Choi. Frepgan: Robust deepfake detection using frequency-Ievel perturbations. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 1060-1068, 2022.
[12] David C Epstein, Ishan Jain, Oliver Wang, and Richard Zhang. Online detection of ai-generated images. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 382-392, 2023.
[13] Riccardo Corvi, Davide Cozzolino, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. Intrigu- ing properties of synthetic images: from generative adversarial networks to diffusion models. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 973-982, 2023.
11

---

[14] Pantelis Dogoulis, Giorgos Kordopatis-Zilon Ioannis Kompatsiaris, and Symeon Papadopoulos. Improving synthetically generated image detection in cross-concept settings. In Proceedings of the 2nd ACM International Workshop on Multimedia Al against Disinformation, pages 28-35, 2023.
[15] Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Con- ditional image generation with pixelenn decoders. Advances in neural information processing systems, 29, 2016.
[16] Laurent Dinh, Jascha Sohl- Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv: 100500880. 2016.
[17] Tero Karras, Samuli Laine, and Timo Aila. style-based generator architecture for generative adversarial networks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 4401 1000-4410,20 2019.
(18) Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 81 (018119, 2020.
[19] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[20] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06115,, 1(2):, 2022.
[21] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 55:36479-36494 2022.
[22] Yuezun Li and Siwei Lyu. Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.0060 2018.
[23] Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In ICASSP 2019-2019 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 8261-8265. IEEE, 2019.
[24] Luisa Verdoliva. Media forensics and deepfakes: overview. IEEE Journal of Selected Topics in Signal Processing, 14(5):910-932, 2020.
[25] Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detecting and simulating artifacts in gan fake images. In 2019 IEEE international workshop on information forensics and security (WIFS), pages 1-6. IEEE, 2019.
[26] Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In International confer- ence on machine learning, pages 3247-3258. PMLR, 2020.
[27] Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. De-fake: Detection and attribution of fake images generated by text-to-image generation models. In Proceedings of the 2023 ACM SIGSAC Conference Computer and Communications Security, pages 1432, 2023.
[28] Jonas Ricker, Simon Damm, Thorsten Holz, and Asja Fischer. Towards the detection of diffusion model deepfakes. arXiv preprint 1rXiv:2210.145777 2022.
[29] Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. On the detection of synthetic images generated by diffusion models. In ICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
12

---

[30] Davide Cozzolino and Luisa Verdoliva. Noiseprint: A cnn-based camera model fingerprint. IEEE Transactions on Information Forensics and Security. 15:144-159, 2019.
[31] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep enn for image denoising. IEEE transactions on image processing, 26(7):3142-3155, 2017.
[32] Zijie Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models. arXiv preprint arXiv:22 14896, 2022.
[33] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-2522 2022.
[34] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[35] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020.
[36] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.09778 2022.
[37] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
[38] Jayyn Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni, Zhuoyi Yang, and Jie Tang. Relay diffusion: Unifying diffusion process across resolutions for image synthesis. arXiv preprint 23XX0323000033500 2023.
[39] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, lya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.107411 2021.
[40] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song. Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 1506. 03365, 2015.
[41] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(201)):11, 2018
[42] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-2533 2015.
[43] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 11118009.1199,,.,111 2018.
[44] Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cece-soonsistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223,,23,, 2017.
[45] Yunjey Choi, Minje Choi, Munyoung Kim. Jung- Woo Ha. Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain mmage-to-image translation. In Proceedings of the IEEE conference computer vision and puttern recognition, pages 8789-87977 2018
[46] Taesung Park, Ming- Yu Liu, Ting-Chun Wang, and Jun- Yan Zhu. Semantic image synthesis with spatially-adaptive normalization. Proceedings of the IEFF/VV conference on computer vision and pattern recognition, pages 2337-23444 2019.
13

---

[47] Qifeng Chen and Vladlen Koltun. Photographic image synthesis with cascaded refinmenn networks. Proceedings of the IEEE international conference on computer vision, pages 1511-1520, 2017.
[48] Ke Li, Tianhao Zhang, and Jitendra Malik. Diverse image synthesis from semantic layouts via conditional imle. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4220-42 4209, 2019.
[49] Tao Dai, Jianrui Cai, Yongbing Zhang, Shu-Tao Xia, and Lei Zhang. Second-order attention network for single image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1065-110744 2019.
[50] Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun. Learning see in the dark. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3291- 3300, 2018.
[51] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1-11, 2019.
[52] Jevin West and Carl Bergstrom. Which face is real? https: www www.whhichaacesreall com/
2019.
[53] Ricard Durall, Margret Keuper, and Janis Keuper. Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7890-7899, 2020.
[54] Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency-aware clues. European conference on computer vision, pages 86-103. Springer, 2020.
[55] Zhendong Wang. Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion diffusion-generatee image detection. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2023.
14

---

A Appendix
A.1 Limitations
This study has potential limitations. The effect estimates in the model are based interventional and prospective observational studies. They are therefore subject to biases and confounding that may have influenced our model estimates. For instance, in Section 3.1 qualitatively analyze the existence of semantic artifacts through visualization experiments and hypothesize that semantic artifacts might originate from the semantics of the images or the preprocessing methods of the datasets, where deeper theoretical proof is needed. In the future, we should develop quantitative metrics to systematically measure and evaluate semantic artifacts in image datasets.
A.2 Broader Impact
This paper introduces a new method for enhancing the detection of Al-generated images. Socially, the research positively impacts security by better detecting synthetic media, thus mitigating risks like misinformation and fraud. However, could also spur the development of more elusive generative models and raise privacy concerns if misapplied.
A.3 Ablation Experiments
A.3.1 Ablation studies on training data source.
To ensure consistent basis for comparison, we also employ the training set of ForenSynths 4. in line with baselines 7 I0 8). The training set consists of 20 distinct categories, each comprising 18,000 synthetic images generated using ProGAN, alongside equal number of real images sourced from the LSUN dataset.
Table 8 reports the Avg. Acc. and mAP results of detectors on our test sets. As expected, most detectors show better performance on cross- GAN/CNN generalization, due the training set of proGAN. Despite this, our approach shows the best Avg. Acc. on DiffuuDifns,n cross- MOos-GAN/CNNN and Open- World evaluation, which indicates the effectiveness of breaking semantic artifacts even on different datasets and the SOTA generalization ability of our approach.
A.3.2 Ablation studies on training set size
Table reports the Avg. Acc. and mAP results of our approach with different numbers of accessible training images. As expected, the increase in training set size will result in the improvement of
Cross Diffusion
Cross GANICNN
Open- World
Methods
Variants
Avg. Acc.
mAP
Avg. Acc.
mAP
Avg. Acc.
mAP
Blur+JEPGo.1 I)
54.28
69.03
81.07
93.31
65.51
79,21
CNN
Blur+JEPG(0.5)
50.89
67.40
75.97
91.44
61,40
77.48
ResNet18
66.57
80.02
74,29
85.71
69.81
82.41
PatchFar
Xception
71.63
83.50
75.70
86.34
73.34
84.69
F3Net
53.82
61.64
76.23
89.67
63.21
73.40
F3Net
70.72
84.98
74.84
80.89
72.37
83.27
Both
67.89
86.15
75.58
82,94
71.11
84.80
SVM
52.34
53.82
52.38
50.07
52,36
$2.25
Durall
LR
50.69
52.18
58.95
49.01
$4.15
$0.85
CelchA-
61.03
70.01
51.54
55.24
57.05
63.87
DIRE
ImageNet-
35.01
60.90
53.38
56.53
34.32
59.07
LSUN-ADM
60.34
61.92
50.67
49.46
56.29
56,70
Ojha
80.40
86.98
NPR
74.36
89.22
73.66
78.35
74.07
84.66
Ours
P=32; N=18
81.80
85.58
81.80
85.07
81.80
85.37
Table 8: Open-world generalization results of detectors train on the dataset from ForenSynths 4.. Italic means that we adopt the pre-trained weights of DIRE, which are trained on feffusiion-geeeratee images, while other detectors are trained on proGAN- -enerated images.
proGAN-generated
15

---

Cross-Diffusion
Cross-GNNICNN
Open-World
Methous
Avg. Avc.
mAP
Avg. Avc.
mAP
Avg. Acc.
mAP
Training images = 10k
80.00
88.26
67.24
73.54
74.65
82.09
Training images 20%
81.20
90.34
69.36
79.42
76.24
85.76
Training images = 40%
88.84
93.45
70.60
80.00
81.19
87.81
Training images 80k
88,93
93,87
71.67
82,06
81.69
88,92
Table 9: Ablation study results on training set size.
Open World
Loss
Feature
Classifier
Avg.
BCE Loss
Patch- Based
Linear
82.12
MSE + BCE Loss
Patch- Based
Linear
78.18
BCE Loss
Fused
Linear
69.51
BCE Loss Patch-Based Based
Self-Attention Linear
79.95
Table 10: Ablation study results other components.
generalization. However, the size of the training set reaches a certain scale, e.g., training images 40k, such improvements gradually diminish.
A.3.3 Other ablation studies
We consider that during the denoising process of Diffusion Models, the generated images might become overly "clean", lacking the environmental noise present in real-world scenes. To address this, we employ a loss for image denoising task on our patch-based feature extraction network instead of using Binary CrossEntropy (BCE) Loss. In the first stage, each patch is added with Gaussian noise of a certain intensity y = x + v before being input into our feature extraction network. Then we use Conv layer after the same-convolutional blocks to obtain the noise residual R(y),, thereby restoring the unnoticed patch x' = y - R(y). We optimize the feature extractor by comparing the restored patch with the original patch by Mean Squared Error (MSE):
1
LMSE
R(w) - (ys (x,)2))113
2N
where N represents the number of noisy-clean training patch pairs. Then, we optimize the linear classifier by BCE loss.
Additionally, we compare several other ablations: 1) Adding self-attention to the shuffled patch sequence before our linear classifier. Using pre-trained ResNet18 to extract global image features and fuse them with patch-based features for classification.
Our experimental results are shown in Table 10 where notice that: A. When optimizing with MSE loss, we observe a decline detection performance, which we attribute to two main reasons: 1) The added noise the patches does not effectively replicate environmental noise. 2) This approach not well-suited for images from GANs or other CNN-based generative models. B. The use of Self-A Self-Attention leads to reduced effectiveness, which might be due the increase in model depth, inadvertently causing overfitting to artifacts. C. Using ResNet-18 18 for global feature extraction results in accuracy drops because this will introduce global semantic artifacts.
A.4 Visualization of the effeet of propprocss pipelines
In this section, to better understand the impact of Patch Shuffled on artifacts, we analyze the noise residuals power spectra of shuffled images well as image patches. shown in Figure these visualizations support our original hypothesis. Specifically, based on the visualization of shuffled images, most artifacts are removed during the patch shuffling. For instance, the distinct artifacts between CelebA and LAION or between LDM-CelebA and LDM-LAION are significantly reduced. In addition, the visualizations of image patches reveal an intriguing finding that ow-frequency features weakened but high-frequenne features (corresponding to artifacts) are enhanced.
16

---

Camcr
TAITN
IOMGIIA
TEXA-LAONN
Orginal
Stuflel
Pradec (ggnertd
Figure 5: Additional noise residuals power spectrum images from real datasets (Faceforensics*+, Raw Camera, CelebA, LAION) and 4 generative models (Deepfake, SITD, LDM-C LDM-CelebA, LDM-LAION), As suggested by the reviewer, consider 3 preprocessing operations from rows to 4.
(a) Original
(b) ResNer-50 trained on Bedroom
(c) ResNet-50
(d) ResNct-50
(c) ResNet-50 trained on Church with Patch Shuffle
(f) Our approach
trained on Church trained on Bedroom
with Patch Shufflc
Figure 6: Additional CAM visualizations of real images for bedroom church, where the label is "0". As expected, almost no region activated, aligning with high Acc. results real images Table
A.5 Visualizations of CAM extracted on real images
In this section, to better demonstrate the performance the detector in cross-scene settings, the CAM visualizations of all the detectors on real images (i.e., with the label "O") are provided (see Figure ( The visualizations show almost no activated regions, which aligns with the high Acc. results real images in cross-scene settings (see Table
A.6 Analysis on Radial Spectrum
To refine the differences in spectral characteristics of various images, we analyze the radial power spectrum of image residual noise from different datasets. As usual, we obtain the Fourier transform Su v) of the noise residuals from 1,000 images and normalize them (by dividing them by their maximum value). Then, we average the amplitude of frequency components with the same radial
distance
Vu² + x in the spectrum:
Σ S(r) N(r) Vartterr
[F(u,))
(10)
17

---

where (r) represents the number of frequency components at radius r, which goes from 0 to 1 in 180 discrete steps. u and v respectively represent the radial and lateral distances from the center of the spectrum. we categorize the radial spectra into low (10 60), mid (70 120), and high (130 180) frequency bands.
Figure 7 displays our experimental results, where the left column shows the low-frequency band results, the middle column for mid-frequency, and the right column for high-frequency. Rows to 4 respectively exhibit the results for LAION, LSUN, ImageNet, CelebA and their corresponding synthetic images, while Row 5 displays the results for LDM synthetic images trained on different datasets, and Row 6 shows results for different real dataset images.
In the results from Rows 1 to 4, we notice that most generative models exhibit characteristics close to real images in the low-frequency band. In the mid-frequency band, the differences become noticeable and are significantly amplified in the high-frequence band, indicating that the detection of synthetic images is traceable. On the other hand, we observe that real images always seem to exhibit smoother and more natural variations across all frequency bands. This is more evident the results of the sixth row, where all classes of real images show the same trend and also have less jitter. In Row 5, we find that the frequency-level artifacts of generative models are concentrated at the highest components, and models using the same architecture tend to produce similar trends in variation. We observe similar trends in other results, such as DALLE2, SD-v1, and SD-v2 (Row 1); ADM (Row 3); and starGAN (Row 4). One explanation for this is that generative models with partly identical structures tend to produce similar artifact features. Nevertheless, when faced with a variety of generative methods, these artifacts will show significant differences, which further impact the generalizability of the classifier.
A.7 Dataset Collection
We collect and sample variety of DM-based generative models in chronological order of their release. 1) iDDPM (34), DDIM (35). and PNDM (36), which use the standard U-Net architecture
(ADM) 37 and RDM (38), which use class- conditional generation. 3) Latent Diffusion (LDM) [9). Stable Diffusion v1.4 (SDv1) and v2.1 (SDv2), GLIDE (3, and DALL-E2 2 (20 which use auto-encoders to edit images in latent space for tx12img generation. To explore detectors' cross-scene performance, we included variants of these DMs trained 6 real image datasets: 1) LSUN-bedroom, LSUN-church [40], CelebA (4], and FFHQFFHQ 17 used for unconditional models; 2) ImageNet 442 used for class-conditional models, 3) LAION-SB 5B (33 used for txt2img models. For class-conditional models, we uniformly sampled from each category, and for tx12img models, we used text from LAION as the prompts.
and unconditional generation. 2) Guided-Diffusee
HDDPM We take the officially released iDDPM model pre-trained on LSUN bedroom and sample the synthetic images with DDPM sampler and Linear noise schedule. The number of diffusion steps is 1000. All the images are generated directly to 256 x 256.
DDIM
We use the implementation of DDIM pre-trained on LSUN bedroom and church from https://gihhu..cmm/ii com//upinggluu//NMMM
liu/PNDM and sample the synthetic images with DDIM sampler and Linear noise schedule. The number of diffusion steps is 1000 with speed of 50. All the images are generated directly to 256 x 256
We take officially released PNDM models pre-trained on LSUN bedroom and church. Following the official code, we sample the synthetic images with F-PNDM sampler and Linear noise schedule. The number of diffusion steps is 1000 with a speed of 50. All the images are generated directly to 256 x 256.
PNDM 3
We take the officially released class-conditional Guided-Diffussin model pre-trained on ImageNet and sample the synthetic images with DDIM sampler and Linear noise schedule. The diffusion steps are 1000 and the respacing timesteps are 250. For each category (a total of 1000 categories) we generate images on average. All the images are generated directly to 256 x 256.
ADM
18

---

RDM We take the officially released Guided- Diffuu--if model pre-trained on CelebA and sample the synthetic images with DDPM sampler in the first stage and Blurring sampler in the second stage. The number of first steps is 256 and the second is 200. All the images are generated directly to 256 256.
3.66
0.57
2.4
0LEE
4B1
034
0.55
142
LAKH
LACH
Viv7
40
$2.12
236
030
0.5
620
0.25
8.00
0.4
646
356
0.5
1.00
045
015
022
0.0
0.9
6.5
2.00
0.50
034
2.64
DDN
EOON
00M
GGRN
0.2
CB94
Mix
HeL
1.6
039
1.00
0.10
1.00
0.00
0.40
1.80
0.45
8.50
0.15
1.6
2.64
JDH
101
220
0.4
GDM
SqAAN
3.6
052
615
3.00
134
8.33
201
by004
0.5
835
0.000
3.44
0.505
2.2
0.2
3.4
356
248
1527
LOW
0/00
0428
0.1
0.0
LEM SURI
3.64
imagelet
LON ORREONNNE
(ORB2A)
3.62
052
604
2.40
Inaayeortt
0.5
129 (cakboi
232
0.08
822
1.65
10/0
1990
1011
mepdiet
Inapelst
6.0
2415
13C0
1.2
19/XX
250
2422
Figure 7: Radial spectrum power density. We calculate the radial spectra of different settings and divide them into parts: low-frequency (left), mid-frequency (middle), and figh-freyenccy (right). The x-axis represents the frequency components and the y-axis represents the power intensity of those components. Rows to show the results of images from LAION, LSUN, ImageNet, CelebA, and the generative models trained on the corresponding dataset. Row 5 shows the results of images from different LDM variants trained on LSUN, ImageNet, CelebA, FFHQ and LAION. Row 6 shows the results of images from 10 different real datasets. Special artifacts of the generated model are more visible in the high fre.ueneyynyy range.
19

---

We take officially released Latent Diffusion models pre-trained on CelebA-HO, FFHQ, LSUN bedroom and church, ImageNet and LAION. Following the official code, we sample the synthetic images with DDIM sampler. The number of diffusion steps is 200. For each category of the ImageNet variant, we generate images on average and use prompts from -LLONN-Hih-Resllunne and CFG scale of 5.0 for the LAION variant. All the images are generated directly to 256 x 256.
LDM
SD-v We take officially released Stable Diffusion v1.4 pre-trained on LAION and sample the synthetic images with DDIM sampler. The number of diffusion steps is 200. We use prompts from LiiON- Hesol-esoo and a CFG scale of 5.0 for txt2img generation. All the images are generated directly to 256 x 256.
SD-v2 S
We take officially released Stable Diffusion v2.1 models pre-trained on LAION and sample the synthetic images with DDIM sampler. The number of diffusion steps is 200. We use prompts from LAION-Higg- and a CFG scale of 5.0 for tx12img generation. All the images are generated directly to 256 256 and 768 768.
GLIDE We take the officially released GLIDE model and the corresponding upsampler. Following the official code, sample the synthetic images with the default settings. The respacing timesteps of GLIDE are 250 and the steps of the upsampler are 100. We use prompts from LAION-High- Resolution for txt2img generation. All the images generated to 64 x 64 and then upsampled to 256 x 256.
DALL-E1 We take the implementation of DALL-E 2 pre-trained LAION and sample the synthetic images with the default settings. We use prompts from LAION-High-Resolune for txt2img generation. All the images generated directly to 256 256.
7 hhtpp://gituub.com//C http:/iiiiiiicoooooooooeo -issstalleddiffusii
hhaps://ggtuuooommLLoooAAAAAlll laion
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to Section At the end of the Introduction, our main contributions are summarised.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please refer Section A.]
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
21

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
22

---

Answer: [No]
Justification: The data used in this study is proprietary and contains sensitive information, which cannot be disclosed or shared publicly. However, ensure reproducibility through the detailed methodologies and parameters described in our paper, alongside the provided code in supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
public/ httpiii/ggiee//eeeee /eldeeee//odummmm ssionPolicy) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (ht
cnips.cc/publice
//oddddee//odeeesseee Codee Suree somnissionn ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our approach and all baselines use unified seed for experimentation. Error bars are not reported due to expensive computation.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
23

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Section 4.1 we provide information on the GPU hardware used in the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: respect.
We have reviewed the NeurolPS Code of Ethics and checked the paper in every
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper thoroughly discusses both the potential positive and negative societal
impacts of the technology. Please refer to Section A.2
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: All employed models and data in the paper adhere to established guidelines and restrictions.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have cited the original papers that produced the code or dataset used in the paper and state the version of the asset used, include URLs. Please refer to References and Section A.7
Guidelines:state which version of the asset is used and, if possible, include URL.
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
25

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, (papersw haperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Details of our provided code submitted via structured templates in supple- mentary material.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
26

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "0VVwVjsq54", "reviewer": "Reviewer_a5zv", "summary": "The paper investigates the robustness and generalization of AI-generated image detectors, with a particular focus on the influence of semantic artifacts on detector performance. The authors highlight that semantic artifacts can significantly impact the effectiveness of AI detectors.\nTo explore this phenomenon, the authors employed a novel approach by training a classifier on image patches rather than whole images. By training a classifier on image patches, the authors conducted experiments on 31 different families of AI-generated images. This study provides a unique and intriguing perspective on how semantic artifacts can affect the performance and generalization of AI detectors.", "questions": "Impact of Paired Data: What if paired data were used, where each real image is paired with an AI-generated image sharing the same semantic content? This pairing could be achieved by conditioning the generation process on the real image. By ensuring that both images share the same semantic features, we could eliminate semantic artifacts as a spurious factor. This approach might already solve the problem and lead to better generalization. The paper would benefit from discussing this potential strategy and its implications for improving the robustness of AI-generated image detectors.", "limitations": "No", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "Originality: The paper presents a unique angle by examining how semantic artifacts impact the generalization of AI image detectors. While this finding is not entirely unexpected, it represents a novel and insightful contribution to the field.\n\nQuality: The paper successfully motivates the significance of semantic artifacts and their influence on generalization. However, the demonstration of this effect could be more thorough.\n\nClarity: The paper is well-written and includes visualizations that effectively aid in understanding the concepts and findings presented.\n\nSignificance: The paper demonstrates a notable increase in generalization performance, ranging from 3 to 6 points, across 31 test sets, including open-world scenarios. This improvement is significant and highlights the potential impact of addressing semantic artifacts in AI-generated image detection.\n\nThe paper evaluated on up to 8 baselines, while NPR outperforms on open-world, the results generally is good.", "weaknesses": "Missing Ablations: The impact of patch size on the performance of the classifier is not explored. Including an ablation study on different patch sizes would provide a deeper understanding of its influence on the results.\n\nMissing Visualizations: Figure 3 lacks examples of real images, which is crucial for the reader to fully comprehend the differences and understand the context of the study. Including these examples would complete the visualization and enhance clarity.\n\nMissing Data Details: The paper does not provide sufficient details on how real images were selected for the study. Additionally, there is a lack of discussion on various data design choices. Addressing these points would strengthen the paper by clarifying the methodology and justifying the design decisions.\n\nMissing Citations: The paper should cite relevant works that discuss the use of patches to mitigate the influence of semantic features [1, 2].\n\n[1] https://arxiv.org/abs/1905.13549\n[2] https://openaccess.thecvf.com/content/CVPR2022/papers/Mao_Causal_Transportability_for_Visual_Recognition_CVPR_2022_paper.pdf", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read our manuscript and for providing detailed comments! Below are our responses to your comments:\n\n>   Q1: Missing ablations of patch size on the performance.\n\nWe have already reported the ablation results of patch size (and model depth) in Appendix A.1.1, and we will add conclusions in the main text based on them. The superiority of our method holds for all tested patch sizes and model depths.\n\nSpecifically, a too-large patch size enlarges the receptive field, potentially exacerbating the overfitting issues, while a too-small patch size destroys certain low-level semantic features, potentially causing underfitting. \n\n>   Q2: Missing visualizations of real images in Figure 3.\n\nWe add such visualizations to the attached PDF in our global response, as illustrated in Figure B, and we will include them in our updated manuscript. \n\nAs expected, the CAM visualizations of all the detectors on real images (i.e., with the label \"0\") show almost no activated regions. This aligns with the high Acc. results of real images in cross-scene settings (see Table 1).\n\n>   Q3: Missing details about the data selection and design.\n\nWe have already detailed the dataset collection in Appendix A.4, and we will summarize them in the main body of our updated manuscript.\n\nSpecifically, for DM-generated images, our selected generative models span a long time range, from 2020 to now. They fall into 3 categories: unconditional, class-conditional, and text-conditional. For real images, the training data of each generative model are used to ensure the same distribution to their generated images.\n\n>   Q4: Missing citations of [1, 2], which discuss the use of patches to mitigate the influence of semantic features.\n\nThank you for recommending these two papers, which can be used to better motivate our use of patches. We will cite them in our updated manuscript.\n\n>   Q5: Data pairing as a potential baseline.\n\nThis is indeed a good point! A similar idea has been adopted by very recent methods, e.g., DIRE [54], which models the error between an input image and its reconstructed counterpart by a pre-trained diffusion model. We have validated that our method outperforms DIRE by **25.07\\%** in Acc. and **24.55\\%** in AP on open-world evaluation (see Table 3).\n\nWe believe data pairing has the following limitations:\n\n+   It targets specific semantic objects, e.g., church, bedroom, human face, making it feasible only for unconditional or class-conditional generated images. However, for text-conditional generated images, which contain complicated semantics, paired data are hard to define and collect.\n+   Although paired data can remove the semantic artifacts, the model may still learn the domain-specific information. Then, when it comes to different domains during testing, the performance decreases. For example, a generated human face normally contains most artifacts in its domain-specific regions, e.g., the lineament and hair. This assumption is supported by our results in Table 4, where DIRE (trained on CelebA) degrades from **83.25\\%** Acc. on CelebA-LDM to **50.45\\%** Acc. on LAION-LDM.\n\nIn contrast, our method is both tailored to specific semantics and indiscriminately destroys the semantic artifacts.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "obo9AHhyi5", "reviewer": "Reviewer_Zwma", "summary": "The paper looks into the problem of AI generated image detection. Given the onset of diffusion models they highlight how previous approaches which claim to generalize fail in setting with new datasets and models. The authors also motivate by visualizing frequency spectrum images. The paper then proposes a patch based feature extraction approach followed by a classifier which works on the image patches to classify whether the image is real or generated. Using such an approach is able to remove issues which some of the previous approaches faced and helps generalize better.", "questions": "I think the paper is well written and in general easy to follow and sound. At the same time I have mentioned a few clarification and comparisons in the weaknesses and would hope the authors answer some of them especially regarding more discussion about the presented quantitive numbers and missing comparisons with mentioned approaches. Some more analysis around frequency visualizations of patches and patch shuffled images would be good.", "limitations": "Yes.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The following are the strengths of the paper:\n1. The paper is very well written and easy to follow. The authors explain different components well.\n2. The papers motivation is sound and the authors are able to highlight drawbacks of existing approaches well and also provide some proof backing their claims.\n3. On highlighting the problems, the authors propose a new approach to handle the task and incorporate components in it which make it generalizable to new generators.\n4. The authors have a good suite of generators that they evaluate in to show effectiveness of their approach and also showcase cross scene and cross model generalization.", "weaknesses": "The following are the weaknesses of the work:\n\n1. The authors propose patch based learnings as a way to make the approach more generalizable. While they show frequency based visualizations for normal images, similar comparison with patch shuffled as well as image patches is missing. \n2. In L178-180, it is not clear what artifacts authors are referring to. Also L192-193 again refers to the same Figure 3 but it is unclear why the visualization should be more focused on point regions. It could be that the detector feels the full region is the cause of its prediction.\n3. In L242-243 the authors mention that they resize the images. Missing discussion whether this resizing alters the artifacts in the images. A visualization similar to frequency visualizations would help clarify this.\n4. Missing analysis of why the proposed approaches has considerable gaps with existing approaches on specific generators. For example in Table 5 for many generators the proposed approach is far behind the best approach. Similar discussion for the quantiative results should be provided to give some intuitions.\n5. Missing comparisons:\na) Missing comparison with Towards Discovery and Attribution of Open-world GAN Generated Images.\nb) A previous work Towards Discovery and Attribution of Open-world GAN Generated Images also looked at fingerprints from generators. Missing comparison and discussion about it.\n\nSmall corrections:\nL119 bot -> both\nL119 synthetics -> synthetic", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the feedback and comments! We are glad to hear that you believe this is important work and that it meets the standard for publication. Below are our responses to the comments provided:\n\n>   Q1: Missing frequency visualizations on patch-shuffled images as well as image patches.\n\nThank you for your suggestions. The frequency visualizations of shuffled images and image patches are provided in Figure A of the PDF attached with our global response.\n\nThese visualizations support our original hypothesis. Specifically, based on the visualization of shuffled images, most artifacts are removed during the patch shuffling. For instance, the distinct artifacts between CelebA and LAION or between LDM-CelebA and LDM-LAION are significantly reduced. In addition, the visualizations of image patches reveal an intriguing finding that low-frequency features are weakened but high-frequency features (corresponding to artifacts) are enhanced.\n\n>   Q2: Not clear what artifacts are in L178-180 and why the visualization in Figure 3 should be more focused on point regions.\n\nWe will make it clear that the \"artifacts\" in L178-180 refer to \"generator artifacts\", as defined in Section 3.1.\n\nFor visualizations in Figure 3, we will clarify that the generator artifacts generally correspond to more universal, point regions, which represent a deeper receptive field, meaning a higher-level feature extraction. So it can be seen that our method successfully directs the model to focus on these regions rather than the concentrated, semantic regions.\n\n\n>   Q3: Missing discussion and visualization on the influence of image resizing.\n\nWe acknowledge that the use of image resizing is not a particular design. In our additional experiments, we also test image padding and find it to be more effective. See Figure A and Tables A and B in the PDF attached with our global response.\n\nTables A and B demonstrate the superior results of zero padding compared to resizing, particularly in cross-GAN/CNN generalization. In particular, the performance of our method has been largely boosted on SITD, SAN, CRN, IMEL, and WFIR. It can be attributed to their high image resolution, which introduces variations in artifacts and leads to the loss of low-level features when resizing is applied. Figure A supports this finding by showing that the frequency features of SITD (with image resolution of over 4,000×3,000) images change a lot after resizing.\n\nNote that on ganGAN, resizing is better than padding probably because their images have been resized during the original data collection.\n\n>   Q4: Missing analysis of why the proposed approaches have considerable gaps with existing approaches on specific generators.\n\nThe considerable gaps (on SITD, SAN, CRN, IMEL, and WFIR) have been explained in the above response.\n\n>   Q5: Missing comparison and discussion about \"Towards Discovery and Attribution of Open-world GAN Generated Images\".\n\nThe suggested paper and our work focus on different tasks. They aim at attribution and discovery of generated images, rather than our (generalized) detection. Therefore, a direct comparison is not possible. Technically, their method relies on out-of-distribution detection and clustering while ours directly destroys artifacts.\n\n>   Q6: Small corrections: L119 bot -> both L119 synthetics -> synthetic\n\nThanks. We will fix them.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hjOatF44gu", "reviewer": "Reviewer_4wrZ", "summary": "This paper identifies a significant drop in accuracy for existing detectors when generalizing across different image scenes, attributing failures to \"semantic artifacts.\" To address this, a new approach is proposed that involves image patch shuffling and training a patch-based classifier, enhancing generalization.", "questions": "See weaknesses.", "limitations": "See limitations.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is relatively well-motivated as AI-generated image detection is a crucial issue. I also find the evaluations thorough. The strengths are as follows:\n\nThe target issues of the paper are meaningful and worth exploring. The motivation is clear.\nThe paper is easy to follow.", "weaknesses": "1. The idea comparison of patchfor [6] and this paper is needed.\n2. It is better to conduct an experiment using the images collected from fake news on the Internet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! Below are our responses to the comments:\n\n>   Q1: Idea-level comparison to PatchFor [6].\n\nThe key idea-level difference between PatchFor and our method is the use of patch shuffle. This is the core of our method, which ensures that the model completely removes the semantic information as a whole, thereby mitigating overfitting to semantic artifacts. Another technical difference is that we force our model to accept **only** patch inputs but PatchFor still accepts the full image, which contains rich semantic information.\n\n>   Q2: Experiments using the images collected from fake news on the Internet.\n\nWe have already followed the common practice [4,7,8] for datasets and experimental settings. Notably, these datasets include data from the Internet, such as Deepfake (sourced from YouTube videos) and WFIR (sourced from websites). For the fake news, we believe it is clearly a valuable direction for future work but technically, it is infeasible because the ground truth of the fake news on the Internet is hard to get, making the model training and evaluation not possible.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mOUhlat44I", "reviewer": "Reviewer_iKNb", "summary": "This paper addresses the task of AI-generated image detection and specifically the challenge that arises by shifts in the semantics between training and test samples. It is demonstrated through experiments that different artifacts are produced by different generators, leading to performance drops in the cross-generator setting, and that the artifacts related to dataset semantics are inherited to the generated images leading to performance drops in the cross-scene setting. To this end a pipeline is proposed involving patch-based training instead of processing the whole-image or a patch-shuffled version of it. The performance of the trained detector is better than the baselines in the cross-scene setting and in the cross-generator setting (with ACC) but worse in the cross-generator setting (with AP).", "questions": "What is the performance of the model if resizing is completely omitted?\nHow does the existing previous approach by Dogoulis et al. perform on your experimental design?", "limitations": "Sections 3.1 and 3.2 are not part of the methodology in my opinion, they should be placed under a separate motivation section as preliminary/insightful experiments. Section 3.3 although very small is still verbose and gives unnecessary details.\nA similar setting has already been proposed by Dogoulis et al. 2023 which should be cited and the differences with the proposed definition/approach should be discussed.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "S1-The problem of current detectors that overfit the dataset's semantics is important.\nS2-The preliminary experimental analysis of artifacts is enlightening.\nS3-The results support the effectiveness of the method.\nS4-The experimental analysis (comparisons, ablations) is sufficient to understand the model's performance.", "weaknesses": "W1-The problem of performance drop in cross-scene settings has already been discussed in (Dogoulis et al. 2023).\nW2-The method is extremely simple. A small CNN model processes the image patches (instead of the image), then aggregates the corresponding features and finally classifies the sample. Nothing methodologically novel. Also, its performance is very close and in some cases worse than the SotA NPR model. The initial findings seem very interesting, thus a more sophisticated approach would potentially perform even better in the cross-scene setting.\nW3-Big performance gaps by this method between ACC and AP are still observed although it is claimed that this is what the method addresses. E.g., in Table 5 --> ACC 72.58 AP 81.12.\nW4-Resizing the images to 256 reduces the synthetic artifacts, and it should be omitted. The fact that the training images are small makes the effect of resizing very small. I would like to see how much the results are changed if the authors omit this part of the data augmentation/pre-processing pipeline.\n\nDogoulis, P., Kordopatis-Zilos, G., Kompatsiaris, I., & Papadopoulos, S. (2023, June). Improving synthetically generated image detection in cross-concept settings. In Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation (pp. 28-35).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read our manuscript and for providing detailed comments! Below are our responses to the comments provided:\n\n>   Q1: Comparison to (Dogoulis et al. 2023), which has discussed the problem of performance drop in cross-scene settings.\n\nFirst, we acknowledge that the cross-scene generalization problem is well recognized, and our contribution is not to define the problem but to better solve it. Tables A and B in the PDF attached with the global response show that our method largely surpasses (Dogoulis et al. 2023) by **32.38\\%** in Acc. and **29.65\\%** in AP on open-world evaluation.\n\nSee the following for the explanations of their poor performance and the experimental settings. The original setting in (Dogoulis et al. 2023) is less challenging because they only focus on generalization between different concept classes (e.g., objects) but do not involve more diverse contents, represented by the LAION, as in our work. Therefore, their assumption is essentially not sufficient for text-conditional generation. We notice that they originally select the top-10k images of 20k generated images as training sets, while we have 48k generated images. For a fair comparison, we consider two settings: A) top-10k images and B) top-24k images. We followed their official codes and selected ResNet-50 as the backbone. Gaussian blurring and JPEG compression are applied as data augmentation with a probability of 10\\%.\n\n>   Q2: Missing discussion and comparison on the influence of resizing the images.\n\nSee our response in Q3 and Q4 of Reviewer Zwma. In sum, we add ablation experiments and find that padding is a better choice than image resizing.\n\n>   Q3: Big performance gaps between Acc. and AP are still observed in Table 5.\n\nAs seen in our response in Q3 and Q4 of Reviewer Zwma, the most considerable gaps on specific generators, e.g., SITD, SAN, CRN, IMLE, and WFIR can be solved by replacing the resizing with padding. For example, Table B shows that our method reaches the SOTA on cross-GAN/CNN (Acc. **80.13\\%**, AP **84.97\\%** vs. Acc. **72.58\\%**, AP **81.38\\%** of the second best method, NRP. The ACC-AP gap decreases from **8.8\\%** to **4.84\\%**.\n\n>   Q4: The method is extremely simple and nothing is methodologically novel. Close performance compared with the SOTA baseline.\n\nBased on our new results in Table B, our method substantially improves the previous SOTA, NPR, from ACC. 75.38\\% to 85.97\\% in the open-world evaluation.\n\nWe welcome further improvements on top of our work. However, we believe the simplicity of our method does not diminish its novelty. Conceptually, our novelty lies in introducing the concept of \"semantic artifacts\" and demonstrating that breaking \"semantic artifacts\" is the key to cross-scene generalization. Based on this finding, we propose a novel design of a patch shuffle-based end-to-end detection framework. \n\n>   Q5: Sections 3.1 and 3.2 are not part of the methodology in my opinion, they should be placed under a separate motivation section as preliminary/insightful experiments. Section 3.3 although very small is still verbose and gives unnecessary details.\n\nWe will further smooth the logic in Sections 3.1 and 3.2, and move unnecessary details to the Appendix.\n\nWe agree that Sections 3.1 and 3.2 are not directly about the methodology, but we group them together with Section 3.3 to ensure a smooth transition from the motivation about \"semantic artifact\" to the method design.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their insightful assessment of our work and for providing useful feedback and actionable suggestions. They found that our research makes novel and insightful contributions (reviewer a5zv), with a clear and sound motivation (reviewers a5zv, Zwma, and 4wrZ), an effective/generalizable method (reviewers a5zv, Zwma, and iKNb), thorough/sufficient experiments (reviewers 4wrZ and iKNb), and good writing (all reviewers).

They mainly request more comparisons with recent research, discussion on the impact of preprocessing, and visualizations to further support our claims. To address them, we mainly provide:

● A comparison to the approach from "Dogoulis et al. Improving synthetically generated image detection in cross-concept settings" and an ablation of image resizing on the performance. (Tables A and B) (reviewer iKNb)

● Additional frequency visualizations on images from 4 real datasets and 4 generative models with different pre-processing pipelines. (Figure A) (reviewers Zwma and iKNb)

● CAM visualizations extracted from different detectors on real images of church or bedroom. (Figure B) (reviewer a5zv)

All these new results support our original claims. **In particular, thanks to the suggestion from reviewers, the performance of our method has been largely improved by replacing the resizing with padding.**

Meta-review: This paper enhances the generalization of AI-generated image detectors by focusing on the challenge of cross-scene scenarios. Specifically, its key technique contribution is to train the classifier on image patches rather than whole images to mitigate the influence of semantic artifacts. Overall, the reviewers find this paper interesting to read and the provided empirical results are strong. But meanwhile, the reviewers raise several concerns, mainly regarding 1) the contribution/relationship w.r.t. some prior works should be discussed and clarified; 2) additional ablations (especially regarding image resizing) should be added; 3) some analyses need to be further discussed and clarified; 4) some critical visualizations are missing; and 5) the overall presentation can be further improved.

The provided rebuttal successfully alleviates most of these concerns, persuading two reviewers to rate this paper as 6 -- weak accept. The other two reviewers stand on the slightly negative side, rating this paper as 4 -- Borderline reject. By reading these two slightly negative reviews and the corresponding rebuttal, the AC believes most concerns are reasonably addressed and does not see any significant concerns remaining. 

Given 1) the overall solidity of the submission and 2) the increasing demand for robust GenAI detectors, the AC would like to recommend accepting this submission, and believes this work will be of substantial interest to the NeurIPS community,
Decision: Accept (poster)