Breaking Semantic Artifacts for Generalized AI-generated Image Detection

Authors: chende  zheng
chenhao  lin
zhengyu  zhao
hang  wang
xu  guo
shuai  liu
chao  shen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 15:30:35

Abstract: With the continuous evolution of AI-generated images, the generalized detection of them has become a crucial aspect of AI security. 
Existing detectors have focused on cross-generator generalization, while it remains unexplored whether these detectors can generalize across different image scenes, e.g., images from different datasets with different semantics. In this paper, we reveal that existing detectors suffer from substantial Accuracy drops in such cross-scene generalization. In particular, we attribute their failures to ''semantic artifacts'' in both real and generated images, to which detectors may overfit. To break such ''semantic artifacts'', we propose a simple yet effective approach based on conducting an image patch shuffle and then training an end-to-end patch-based classifier. We conduct a comprehensive open-world evaluation on 31 test sets, covering 7 Generative Adversarial Networks, 18 (variants of) Diffusion Models, and another 6 CNN-based generative models. The results demonstrate that our approach outperforms previous approaches by 2.08\% (absolute) on average regarding cross-scene detection Accuracy. We also notice the superiority of our approach in open-world generalization, with an average Accuracy improvement of 10.59\% (absolute) across all test sets.

Main paper: 
 introduction: 
 Recently, the rapid development of AI-generated image technology has led to the emergence of synthetic images on the internet, raising significant concerns about AI security. These images exhibit remarkable diversity due to the continuous introduction of new generative architectures, e.g., from Generative Adversarial Networks (GANs) [1] , Variational Auto-Encoders (VAEs) [2] , to Diffusion Models (DMs) [3] . For now, Internet users can easily generate a large number of exquisite images using text prompts. As the quality and variety of these synthetic images continue to advance, developing a universal detector for fake images becomes a crucial aspect of AI security.To determine whether an image is synthetic, existing studies typically train a binary classifier to assess the authenticity of an unknown image during inference. Previous methods have primarily focused on cross-generator detection. For instance, Wang et al. [4] developed a ResNet50 trained on images from proGAN [5] , and then tested it on different GAN variants. Intriguingly, their findings suggested that aggressive augmentation strategies could significantly enhance the classifier's generalization More recently, DM-based synthesis methods, exemplified by Latent Diffusion [19] , have rapidly advanced in text-to-image generation. Breakthroughs like Stable Diffusion, DALL•E 2 [20] , and Imagen [21] are released in quick succession. These large-scale image synthesis models have significantly expanded the application scope of image synthesis, enriching the semantic content and stylistic features of generated images. The evolution makes the detection of AI-generated images increasingly complex and challenging. 
detecting synthetic images: 
 Prior to the advent of advanced DMs, a significant body of forgery detection research focused on images generated by GANs. Early detection methods targeted local artifacts present in synthetic images, including distortions in facial tampered landmarks [22] or inconsistencies in head postures [23] . In [24] , it was demonstrated that global artifacts in synthesized images differ markedly from common artifacts found in modern digital devices. Additionally, the up-sampling operations prevalent in most GAN architectures tend to produce distinct peaks in the spectral profiles of synthesized images [25, 26] , offering another avenue for detecting fake images.A noteworthy question that arises is whether these methods are also effective in detecting DMgenerated images. Wang [4] suggests that simple classifiers when trained with aggressive data augmentation on proGAN images, can be adapted to other unseen models. Ojha [7] use a fixed pre-trained CLIP:ViT-L/14 to extract image features, while Sha [27] incorporate hint information and utilize multimodal inputs (text prompts and images) to enhance performance. Tan et al. [8] introduce the concept of neighboring pixel relationships as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. In particular, recent studies [28, 29] have identified significant differences in the frequency spectra of images synthesized by GANs and DMs, impacting the generalization capability of existing detectors. Interestingly, a common issue is observed when detectors are extended to unknown data [12] . Specifically, these detectors will show a disparity where high Average Precision is accompanied by low accuracy. Our analysis of generalization detection of AI-generated images reveals a similar problem, showing performance imbalances when dealing with different semantics. 
artifacts analysis: 
 To develop a universal detector for AI-generated images, it is crucial to thoroughly examine the distinctive artifacts of synthetic images generated by various generative models. Inspired by [4, 13, 7] , we start our analysis by visualizing the frequency spectra of different image distributions. This exploration aims to uncover any unique or intriguing properties of the artifacts associated with different generators or scenes.In line with the experimental design in [30, 29] , we randomly select a set of images with a quantity of I = 1000 for each image source and utilize the denoising filter D(x i ) from [31] to extract the noise residuals r i of an original image x i :EQUATIONThen, we start from the Fourier transform F of the noise residuals of the M × N image:EQUATIONwhere, k and l represent the frequency domain coordinates. To obtain the artifacts of the image source, we take the average power spectrum S of all single images from it:EQUATIONWe use images from our test sets (see Section 4.1) to conduct preliminary experiments for analysis of the artifacts in both synthetic and real images. Generator artifacts. Figure 1 shows the noise residuals power spectrum of generated images.For GANs, we have observed periodic artifacts in the spectrum, identified as checkerboard artifacts produced by up-sampling [25] . Similar artifacts are also observed in some DMs, such as DALL•E 2 and Latent Diffusion. These DMs employ an auto-encoder architecture to compress images into latent space features, resulting in periodic artifacts akin to those found in GANs. While other CNN-based generators, e.g., Deepfake or CRN, exhibit unique artifacts that are significantly different from GANs or DMs. Interestingly, although cycleGAN and starGAN are both based on GAN architecture, they still show some differences in the spectrum. Overall, images from different generators exhibit different artifacts and generated images with unique artifacts might lead detectors to overfit during training, thereby failing to recognize distinct artifacts of cross-generator data. We term these artifacts as "generator artifacts."Semantic artifacts. Figure 2 shows the noise residuals power spectrum of real and synthetic images from 5 distinct scenes (FaceForensics, Raw Camera, CelebA, FFHQ, LAION). To our surprise, artifacts are also observed in real images. For example, compared to LAION, CelebA and FFHQ exhibit artifacts around the center of the spectrum while Raw Camera displays artifacts similar to the periodic artifacts in GANs. Meanwhile, our findings suggest that these artifacts can be inherited by generative models (see Deepfake trained on FaceForensics, SITD trained on Raw Camera, and Latent Diffusion trained on CelebA/FFHQ/LAION). These artifacts might originate from the semantics of the images or the preprocessing methods of the datasets. For example, CelebA and FFHQ share similar artifacts around the center of the spectra since both contain images of human faces. Notably, although FaceForensics is also a dataset of human faces, its significantly different preprocessing method (cropped from video and resized) results in unique artifacts. We term these artifacts as "semantic artifacts."Many previous researches [10, 12, 8] have focused primarily on cross-generator detection of AIgenerated images and have made good efforts on generalization across "generator artifacts". However, it remains unexplored whether these detectors can generalize on cross-scene detection, especially since we have found that "semantic artifacts" have many properties similar to "generator artifacts".Figure 4 : Pipeline of our approach. First, for pre-processing, we divide the input image into patches and shuffle these patches to obtain a randomized sequence. Then, we train a patch-based convolutional network for feature extraction. Finally, we flatten these features into a one-dimensional vector and then apply a linear classifier for classification.To explore the impact of "semantic artifacts", we conduct several preliminary experiments using the existing paradigm [4] . This method is widely adopted in current studies [12, 10] , where researchers train a ResNet-50 using images from only one generator to detect images from other generators.Specifically, we use 2 sets of datasets for cross-scene tests. We randomly select 15000 images from LSUN bedroom and LSUN church, respectively, and generate 15000 images by SDXL-Turbo using prompts of "A photo of a bedroom" and "A photo of a church" respectively (10000 as training set and 5000 as test set). We train the detectors respectively on two training sets and evaluate them on cross-scene detection.In Table 1 , we report the accuracy (Acc.) and Average Precision (AP) of the 2 detectors on different scenes. As we can see, the detectors perform exceptionally well (almost 100%) on images with the same semantics as the training set. However, when tested on cross-scene images, the detectors suffer from significant drops in Acc. (near 0.00%) for generated images. Interestingly, the AP remains high, indicating that the detectors can still differentiate between real and fake images, although the distinction in scores between real and fake images is relatively minor. To further analyze the reasons for the low Acc., we report the Acc. results at different thresholds in Table 2 . Typically, the threshold for determining whether an input image is real or fake is set to 50% in this task. However, in Table 2 , we observe that the optimal threshold for AI-generated image detection is exceptionally low, ranging from 0.01% (from Church to Bedroom) to even 0.001% (from Bedroom to Church). This demonstrates that the average score of fake images with different semantics is extremely low and the score difference between real and fake images is minimal. Unfortunately, such a low threshold may lead to an increase in false alarms when handling various images. Furthermore, the optimal thresholds vary across different scenes, e.g., 0.01% from Church to Bedroom and 0.001% from Bedroom to Church. Therefore, simply adjusting the threshold does not resolve the issue of cross-scene generalization.We attribute the poor performance of the existing paradigm to the overfitting of semantic artifacts during training. To address this, we simply apply Patch Shuffle as preprocessing to break the global semantics of images. The results are shown in Table 2 . As observed, there is an improvement in Acc. at each threshold from Bedroom to Church, indicating a larger difference in scores between real and fake images. Moreover, from Church to Bedroom, the Acc. at the 50% threshold reaches 94.15%, highlighting the possibility of using Patch Shuffle to mitigate the impact of "semantic artifacts". The primary goal for cross-scene generalization is to break the impact of "semantic artifacts". In our preliminary experiments, simple Patch Shuffle has proven to be an effective method. However, overfitting still exists, e.g., from Bedroom to Church, the Acc. at 50% threshold of detectors with Patch Shuffle is 51.18% (see Table 2 ). We hypothesize that the deep residual structure of ResNet-50 promotes the extraction of overall semantic information. Although Patch Shuffle has been applied, the detectors still learn some semantic features between patches. A strong piece of evidence is that the visualization of CAM extracted from ResNet-50 with Patch Shuffle shows large, contiguous areas of activated regions, e.g., (d) and (e) in Figure 3 , rather than discrete, point-like activated regions seen in our approach, e.g., (f) in Figure 3 .Based on this hypothesis, we propose a simple yet effective approach based on conducting an image Patch Shuffle and then training an end-to-end patch-based classifier. The whole pipeline of our approach is shown in Figure 4 .Pre-processing. First, given an input image X, we divide it into several patches of size P × P :{p i } = Segment(X, P ) i = 1, 2, ..., M × M (4)where M × M is the total number of patches obtained after segmentation. We further shuffle these patches to obtain a randomized sequence {p j }(j = 1, 2, ..., M × M ), which aims to break the global "semantics artifacts".Patch-based feature extraction. We employ a patch-based convolutional network C for feature extraction, which is designed to accept only one single patch as input, aiming at extracting local features. For each patch p j , we first apply a convolutional layer C 0 to extract an initial feature map F 0 . Subsequently, a series of same-convolutional blocks {C n } are used for deep feature encoding.EQUATIONEQUATIONwhere N is a hyper-parameter representing the number of same-convolutional blocks. After encoding, the feature map is downsampled through 3 basicblocks.Linear classifier. The features of each patch {F N } are reassembled into a feature sequence L. We flatten the feature sequence into a one-dimensional vector v, followed by a linear classifier L:EQUATIONWe optimize the whole pipeline (including both the feature extractor and linear classifier) using binary cross-entropy loss:EQUATIONwhere y represents the true labels, and ŷ represents the predictions.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]Answer: [No] Justification: The data used in this study is proprietary and contains sensitive information, which cannot be disclosed or shared publicly. However, we ensure reproducibility through the detailed methodologies and parameters described in our paper, alongside the provided code in supplementary materials.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental settings: 
 Training dataset. To evaluate detectors' generalization capability, the standard practice is restricting the training data to images from only one generator and scene. We employ the generated images from DiffusionDB [32] . The training set consists of 48,000 images generated by Stable Diffusion v1.4 using prompts from Internet Users and 48,000 real images from LAION-5B [33] . To ensure a consistent basis for comparison, we also employ the training set of ForenSynths [4] , in line with experimental design in [7, 10, 8] , as an ablation experiment in Section A.3.1.Test datasets. To evaluate the generalization of baselines and our approach, we conduct a comprehensive open-world evaluation on 31 test sets, covering 7 Generative Adversarial Networks, 18 (variants of) Diffusion Models, and another 6 CNN-based generative models.• Diffusion Models. We collect DMs, including iDDPM [34] , DDIM [35] , PNDM [36] , Guided-Diffusion (ADM) [37] , RDM [38] , Latent Diffusion (LDM) [19] , Stable Diffusion v1.4 (SDv1) and v2.1 (SDv2), GLIDE [39] , and DALL•E 2 [20] . We sample images from their variants of 6 real datasets, including LSUN-bedroom, LSUN-church [40] , CelebA [41] , FFHQ [17] , ImageNet [42] , and LAION-5B [33] . Details refer to Section A.7.• Generative Adversarial Networks. The test sets include fake images generated by 7 GANs from ForenSynths [4] , including proGAN [5] , styleGAN, styleGAN2 [17] , bigGAN [43] , cycleGAN [44] , starGAN [45] , and gauGAN [46] .• Other CNN-based generative models. The test sets include fake images generated by 6 other CNN-based generative models from ForenSynths [4] , including CRN [47] , IMLE [48] , SAN [49] , SITD [50] , deepfake [51] , and WFIR [52] . 4 and Table 5 .Detector baselines. We perform comparisons of our approach with existing popular and state-of-the-art detectors on our open-world datasets, including Durall (CVPR 2020) [53] , CNNDetection (CVPR 2020) [4] , PatchFor (ECCV 2020) [6] , F3Net (ECCV 2020) [54] , Dogoulis (MAD'23) [14] , DIRE (ICCV 2023) [55] , Ojha (CVPR 2023) [7] ,LGrad (CVPR 2023) [10] , NPR (CVPR 2024) [8] .We re-implement baselines [53, 4, 6, 54, 7, 10, 8] with the official codes using our training set, and adopt the official pre-trained weights of baselines [55] .Implementation details. Our approach is implemented using the PyTorch on NVIDIA A100 40GB Tensor Core GPU. During the training, we perform zero padding on the images to ensure the shorter edge is 256 pixels (resizing is also used as another pre-processing pipeline for ablation), and then randomly crop the images to 256x256. Each image will be horizontally or vertically flipped with a probability of 50% as data augmentation. The number of same-convolution blocks N is set to 18 and the size of each patch P is set to 32. The detector is trained using the Adam optimizer and early-stop strategy with an initial learning rate of 1e -4, a minimum learning rate of 1e -6, and a batch size of 64. We separate 5,000 images from the training set to serve as the validation set. We use the accuracy (Acc.) and the Average Precision (AP) as the evaluation metrics, with a particular emphasis on Acc. as our primary metric, because as discussed in Section 3.2, the AP metric may not fully reflect the problem of cross-scene generalization.Cross-scene generalization. We first report the cross-scene detection results in Table 3 . We average the results on 6 variants of Latent Diffusion (LSUN-Bedroom, LSUN-Church, ImageNet, CelebA, FFHQ, LAION). The results demonstrate that most detectors suffer from performance drops on crossscene images, even though the images are generated by models with the same structure. Particularly, the gap between AP and Acc. can be observed in several baselines, e.g. this, our approach still outperforms the SOTA generalization performance, showcasing higher Avg. Acc. and mAP metrics, which reach 92.52% and 95.58% (based on zero padding).Open-world generalization. The visualization result, e.g., the discrete, point-like activated regions in Figure 3 (f), further confirms the effectiveness of our approach that predominantly focuses on learning local features. Detailed results of our open-world experiments are presented in two groups.• Cross-Diffusion generalization. Table 4 presents Acc. and mAP results across images from 18 (variants of) Diffusion Models. The results demonstrate the outstanding performance of our approach on cross-Diffusion evaluation. Our approach reaches an Avg. Acc. of 90.18%, outperforming the best baseline NPR (78.35%). Notably, most baselines show their poor performance on the unconditional generation scene, e.g., Bedroom and Church. We attribute it to a strong scene change, where there is a significant difference between the txt2img generation scene (training set) and the unconditional generation scene (test sets). In addition, the gap between Acc. and AP still exists, e.g., NPR (78.35% Avg. Acc. and 94.08% mAP).• Cross-GAN/CNN generalization. Effect of pre-processing pipeline. So far, we have seen excellent generalizability in our patchbased method. We next explore in depth the influence of pre-processing pipelines on artifact altering and detection. Specifically, we adopt both image padding and resizing during pre-processing. Tables 4 and 5 demonstrate the superior results of zero padding compared to resizing, especially in cross-GAN/CNN generalization. In particular, the performance of our approach using padding has been largely boosted on SITD, SAN, CRN, IMEL, and WFIR compared to resizing. It can be attributed to their high image resolution, which introduces variations in artifacts and the loss of low-level features when resizing is applied. Meanwhile, Figure 5 in Section A.4 also supports this finding by showing that the frequency features of SITD (with image resolution of over 4,000×3,000) images change a lot after resizing. In summary, these results suggest that zero padding can preserve more low-level features in images, which benefits generalized detection.  
conclusion: 
 In this paper, we introduce the concept of "semantic artifacts" in the generalization detection of AI-generated images. We analyze the existence and impact of "semantic artifacts" in cross-scene detection through extensive experiments. Our results demonstrate that existing detectors suffer significant drops in accuracy when applied to images with different semantics, which we attribute to overfitting during training. To address this issue, we propose a simple yet effective approach that utilizes Patch Shuffle and trains an end-to-end patch-based classifier to break the "semantic artifacts" in images. Extensive experiments on 31 test sets validate the effectiveness of our approach, demonstrating our contributions to the universal detection of AI-generated images. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Please refer to Section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Our approach and all baselines use a unified seed for experimentation. Error bars are not reported due to expensive computation.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: All employed models and data in the paper adhere to established guidelines and restrictions.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have cited the original papers that produced the code or dataset used in the paper and state the version of the asset used, include URLs. Please refer to References and Section A.7.Guidelines:state which version of the asset is used and, if possible, include a URL.• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: This study has potential limitations. The effect estimates in the model are based on interventional and prospective observational studies. They are therefore subject to biases and confounding that may have influenced our model estimates. For instance, in Section 3.1, we qualitatively analyze the existence of semantic artifacts through visualization experiments and hypothesize that semantic artifacts might originate from the semantics of the images or the preprocessing methods of the datasets, where deeper theoretical proof is needed. In the future, we should develop quantitative metrics to systematically measure and evaluate semantic artifacts in image datasets.This paper introduces a new method for enhancing the detection of AI-generated images. Socially, the research positively impacts security by better detecting synthetic media, thus mitigating risks like misinformation and fraud. However, it could also spur the development of more elusive generative models and raise privacy concerns if misapplied.To ensure a consistent basis for comparison, we also employ the training set of ForenSynths [4] , in line with baselines [7, 10, 8] . The training set consists of 20 distinct categories, each comprising 18,000 synthetic images generated using ProGAN, alongside an equal number of real images sourced from the LSUN dataset. generalization. However, as the size of the training set reaches a certain scale, e.g., training images = 40k, such improvements gradually diminish.We consider that during the denoising process of Diffusion Models, the generated images might become overly "clean", lacking the environmental noise present in real-world scenes. To address this, we employ a loss for image denoising task on our patch-based feature extraction network instead of using Binary CrossEntropy (BCE) Loss. In the first stage, each patch is added with Gaussian noise of a certain intensity y = x + v before being input into our feature extraction network. Then we use a Conv layer after the same-convolutional blocks to obtain the noise residual R(y), thereby restoring the unnoticed patch x ′ = y -R(y). We optimize the feature extractor by comparing the restored patch with the original patch by Mean Squared Error (MSE):EQUATION)where N represents the number of noisy-clean training patch pairs. Then, we optimize the linear classifier by BCE loss.Additionally, we compare several other ablations: 1) Adding self-attention to the shuffled patch sequence before our linear classifier. 2) Using a pre-trained ResNet18 to extract global image features and fuse them with patch-based features for classification.Our experimental results are shown in Table 10 , where we notice that: A. When optimizing with MSE loss, we observe a decline in detection performance, which we attribute to two main reasons: 1)The added noise to the patches does not effectively replicate environmental noise. 2) This approach is not well-suited for images from GANs or other CNN-based generative models. B. The use of Self-Attention leads to reduced effectiveness, which might be due to the increase in model depth, inadvertently causing overfitting to artifacts. C. Using ResNet-18 for global feature extraction results in accuracy drops because this will introduce global semantic artifacts.In this section, to better understand the impact of Patch Shuffled on artifacts, we analyze the noise residuals power spectra of shuffled images as well as image patches. As shown in Figure 5 , these visualizations support our original hypothesis. Specifically, based on the visualization of shuffled images, most artifacts are removed during the patch shuffling. For instance, the distinct artifacts between CelebA and LAION or between LDM-CelebA and LDM-LAION are significantly reduced.In addition, the visualizations of image patches reveal an intriguing finding that low-frequency features are weakened but high-frequency features (corresponding to artifacts) are enhanced. Figure 6 : Additional CAM visualizations of real images for bedroom or church, where the label is "0". As expected, almost no region is activated, aligning with the high Acc. results of real images in Table 1 .In this section, to better demonstrate the performance of the detector in cross-scene settings, the CAM visualizations of all the detectors on real images (i.e., with the label "0") are provided (see Figure 6 ). The visualizations show almost no activated regions, which aligns with the high Acc. results of real images in cross-scene settings (see Table 1 ).To refine the differences in spectral characteristics of various images, we analyze the radial power spectrum of image residual noise from different datasets. As usual, we obtain the Fourier transform S x (u, v) of the noise residuals from 1,000 images and normalize them (by dividing them by their maximum value). Then, we average the amplitude of frequency components with the same radial distance r = √ u 2 + v 2 in the spectrum:EQUATION)where N (r) represents the number of frequency components at radius r, which goes from 0 to 1 in 180 discrete steps. u and v respectively represent the radial and lateral distances from the center of the spectrum. we categorize the radial spectra into low (10 60), mid (70 120), and high (130 180) frequency bands.Figure 7 displays our experimental results, where the left column shows the low-frequency band results, the middle column for mid-frequency, and the right column for high-frequency. Rows 1 to 4 respectively exhibit the results for LAION, LSUN, ImageNet, CelebA, and their corresponding synthetic images, while Row 5 displays the results for LDM synthetic images trained on different datasets, and Row 6 shows results for different real dataset images.In the results from Rows 1 to 4, we notice that most generative models exhibit characteristics close to real images in the low-frequency band. In the mid-frequency band, the differences become noticeable and are significantly amplified in the high-frequency band, indicating that the detection of synthetic images is traceable. On the other hand, we observe that real images always seem to exhibit smoother and more natural variations across all frequency bands. This is more evident in the results of the sixth row, where all classes of real images show the same trend and also have less jitter. In Row 5, we find that the frequency-level artifacts of generative models are concentrated at the highest components, and models using the same architecture tend to produce similar trends in variation. We observe similar trends in other results, such as DALLE2, SD-v1, and SD-v2 (Row 1); ADM (Row 3); and starGAN (Row 4). One explanation for this is that generative models with partly identical structures tend to produce similar artifact features. Nevertheless, when faced with a variety of generative methods, these artifacts will show significant differences, which further impact the generalizability of the classifier.We collect and sample a variety of DM-based generative models in chronological order of their release. 1) iDDPM [34] , DDIM [35] , and PNDM [36] , which use the standard U-Net architecture and unconditional generation. 2) Guided-Diffusion (ADM) [37] and RDM [38] , which use classconditional generation. 3) Latent Diffusion (LDM) [19] , Stable Diffusion v1.4 (SDv1) and v2.1 (SDv2), GLIDE [39] , and DALL•E 2 [20] which use auto-encoders to edit images in latent space for txt2img generation. To explore detectors' cross-scene performance, we included variants of these DMs trained on 6 real image datasets: 1) LSUN-bedroom, LSUN-church [40] , CelebA [41] , and FFHQFFHQ [17] used for unconditional models; 2) ImageNet [42] used for class-conditional models, 3) LAION-5B [33] used for txt2img models. For class-conditional models, we uniformly sampled from each category, and for txt2img models, we used text from LAION as the prompts.iDDPM 2 We take the officially released iDDPM model pre-trained on LSUN bedroom and sample the synthetic images with DDPM sampler and Linear noise schedule. The number of diffusion steps is 1000. All the images are generated directly to 256 × 256.DDIM We use the implementation of DDIM pre-trained on LSUN bedroom and church from https://github.com/luping-liu/PNDM and sample the synthetic images with DDIM sampler and Linear noise schedule. The number of diffusion steps is 1000 with a speed of 50. All the images are generated directly to 256 × 256.PNDM 3 We take officially released PNDM models pre-trained on LSUN bedroom and church. Following the official code, we sample the synthetic images with F-PNDM sampler and Linear noise schedule. The number of diffusion steps is 1000 with a speed of 50. All the images are generated directly to 256 × 256.ADM 4 We take the officially released class-conditional Guided-Diffusion model pre-trained on ImageNet and sample the synthetic images with DDIM sampler and Linear noise schedule. The diffusion steps are 1000 and the respacing timesteps are 250. For each category (a total of 1000 categories) we generate images on average. All the images are generated directly to 256 × 256. LDM 6 We take officially released Latent Diffusion models pre-trained on CelebA-HQ, FFHQ, LSUN bedroom and church, ImageNet and LAION. Following the official code, we sample the synthetic images with DDIM sampler. The number of diffusion steps is 200. For each category of the ImageNet variant, we generate images on average and use prompts from LAION-High-Resolution and a CFG scale of 5.0 for the LAION variant. All the images are generated directly to 256 × 256.SD-v1 7 We take officially released Stable Diffusion v1.4 pre-trained on LAION and sample the synthetic images with DDIM sampler. The number of diffusion steps is 200. We use prompts from LAION-High-Resolution and a CFG scale of 5.0 for txt2img generation. All the images are generated directly to 256 × 256.SD-v2 8 We take officially released Stable Diffusion v2.1 models pre-trained on LAION and sample the synthetic images with DDIM sampler. The number of diffusion steps is 200. We use prompts from LAION-High-Resolution and a CFG scale of 5.0 for txt2img generation. All the images are generated directly to 256 × 256 and 768 × 768.GLIDE 9 We take the officially released GLIDE model and the corresponding upsampler. Following the official code, we sample the synthetic images with the default settings. The respacing timesteps of GLIDE are 250 and the steps of the upsampler are 100. We use prompts from LAION-High-Resolution for txt2img generation. All the images are generated to 64 × 64 and then upsampled to 256 × 256.DALL•E2 10 We take the implementation of DALL•E 2 pre-trained on LAION and sample the synthetic images with the default settings. We use prompts from LAION-High-Resolution for txt2img generation. All the images are generated directly to 256 × 256.NeurIPS Paper Checklist Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.https://github.com/openai/improved-diffusionhttps://github.com/luping-liu/PNDMhttps://github.com/openai/guided-diffusionhttps://github.com/THUDM/RelayDiffusionhttps://github.com/CompVis/latent-diffusionhttps://github.com/CompVis/stable-diffusionhttps://github.com/Stability-AI/stablediffusionhttps://github.com/luping-liu/PNDMhttps://github.com/LAION-AI/dalle2-laion 


Decision: This paper enhances the generalization of AI-generated image detectors by focusing on the challenge of cross-scene scenarios. Specifically, its key technique contribution is to train the classifier on image patches rather than whole images to mitigate the influence of semantic artifacts. Overall, the reviewers find this paper interesting to read and the provided empirical results are strong. But meanwhile, the reviewers raise several concerns, mainly regarding 1) the contribution/relationship w.r.t. some prior works should be discussed and clarified; 2) additional ablations (especially regarding image resizing) should be added; 3) some analyses need to be further discussed and clarified; 4) some critical visualizations are missing; and 5) the overall presentation can be further improved.

The provided rebuttal successfully alleviates most of these concerns, persuading two reviewers to rate this paper as 6 -- weak accept. The other two reviewers stand on the slightly negative side, rating this paper as 4 -- Borderline reject. By reading these two slightly negative reviews and the corresponding rebuttal, the AC believes most concerns are reasonably addressed and does not see any significant concerns remaining. 

Given 1) the overall solidity of the submission and 2) the increasing demand for robust GenAI detectors, the AC would like to recommend accepting this submission, and believes this work will be of substantial interest to the NeurIPS community,

Reviews: review: paper_summary: paper_summary: The paper investigates the robustness and generalization of AI-generated image detectors, with a particular focus on the influence of semantic artifacts on detector performance. The authors highlight that semantic artifacts can significantly impact the effectiveness of AI detectors.
To explore this phenomenon, the authors employed a novel approach by training a classifier on image patches rather than whole images. By training a classifier on image patches, the authors conducted experiments on 31 different families of AI-generated images. This study provides a unique and intriguing perspective on how semantic artifacts can affect the performance and generalization of AI detectors.
strength_weakness: strength_weakness: Missing Ablations: The impact of patch size on the performance of the classifier is not explored. Including an ablation study on different patch sizes would provide a deeper understanding of its influence on the results.

Missing Visualizations: Figure 3 lacks examples of real images, which is crucial for the reader to fully comprehend the differences and understand the context of the study. Including these examples would complete the visualization and enhance clarity.

Missing Data Details: The paper does not provide sufficient details on how real images were selected for the study. Additionally, there is a lack of discussion on various data design choices. Addressing these points would strengthen the paper by clarifying the methodology and justifying the design decisions.

Missing Citations: The paper should cite relevant works that discuss the use of patches to mitigate the influence of semantic features [1, 2].

[1] https://arxiv.org/abs/1905.13549
[2] https://openaccess.thecvf.com/content/CVPR2022/papers/Mao_Causal_Transportability_for_Visual_Recognition_CVPR_2022_paper.pdf
questions: questions: Impact of Paired Data: What if paired data were used, where each real image is paired with an AI-generated image sharing the same semantic content? This pairing could be achieved by conditioning the generation process on the real image. By ensuring that both images share the same semantic features, we could eliminate semantic artifacts as a spurious factor. This approach might already solve the problem and lead to better generalization. The paper would benefit from discussing this potential strategy and its implications for improving the robustness of AI-generated image detectors.
limitations: limitations: No
paper_summary: paper_summary: The paper looks into the problem of AI generated image detection. Given the onset of diffusion models they highlight how previous approaches which claim to generalize fail in setting with new datasets and models. The authors also motivate by visualizing frequency spectrum images. The paper then proposes a patch based feature extraction approach followed by a classifier which works on the image patches to classify whether the image is real or generated. Using such an approach is able to remove issues which some of the previous approaches faced and helps generalize better.
strength_weakness: strength_weakness: The following are the weaknesses of the work:

1. The authors propose patch based learnings as a way to make the approach more generalizable. While they show frequency based visualizations for normal images, similar comparison with patch shuffled as well as image patches is missing. 
2. In L178-180, it is not clear what artifacts authors are referring to. Also L192-193 again refers to the same Figure 3 but it is unclear why the visualization should be more focused on point regions. It could be that the detector feels the full region is the cause of its prediction.
3. In L242-243 the authors mention that they resize the images. Missing discussion whether this resizing alters the artifacts in the images. A visualization similar to frequency visualizations would help clarify this.
4. Missing analysis of why the proposed approaches has considerable gaps with existing approaches on specific generators. For example in Table 5 for many generators the proposed approach is far behind the best approach. Similar discussion for the quantiative results should be provided to give some intuitions.
5. Missing comparisons:
a) Missing comparison with Towards Discovery and Attribution of Open-world GAN Generated Images.
b) A previous work Towards Discovery and Attribution of Open-world GAN Generated Images also looked at fingerprints from generators. Missing comparison and discussion about it.

Small corrections:
L119 bot -> both
L119 synthetics -> synthetic
questions: questions: I think the paper is well written and in general easy to follow and sound. At the same time I have mentioned a few clarification and comparisons in the weaknesses and would hope the authors answer some of them especially regarding more discussion about the presented quantitive numbers and missing comparisons with mentioned approaches. Some more analysis around frequency visualizations of patches and patch shuffled images would be good.
limitations: limitations: Yes.
paper_summary: paper_summary: This paper identifies a significant drop in accuracy for existing detectors when generalizing across different image scenes, attributing failures to "semantic artifacts." To address this, a new approach is proposed that involves image patch shuffling and training a patch-based classifier, enhancing generalization.
strength_weakness: strength_weakness: 1. The idea comparison of patchfor [6] and this paper is needed.
2. It is better to conduct an experiment using the images collected from fake news on the Internet.
questions: questions: See weaknesses.
limitations: limitations: See limitations.
paper_summary: paper_summary: This paper addresses the task of AI-generated image detection and specifically the challenge that arises by shifts in the semantics between training and test samples. It is demonstrated through experiments that different artifacts are produced by different generators, leading to performance drops in the cross-generator setting, and that the artifacts related to dataset semantics are inherited to the generated images leading to performance drops in the cross-scene setting. To this end a pipeline is proposed involving patch-based training instead of processing the whole-image or a patch-shuffled version of it. The performance of the trained detector is better than the baselines in the cross-scene setting and in the cross-generator setting (with ACC) but worse in the cross-generator setting (with AP).
strength_weakness: strength_weakness: W1-The problem of performance drop in cross-scene settings has already been discussed in (Dogoulis et al. 2023).
W2-The method is extremely simple. A small CNN model processes the image patches (instead of the image), then aggregates the corresponding features and finally classifies the sample. Nothing methodologically novel. Also, its performance is very close and in some cases worse than the SotA NPR model. The initial findings seem very interesting, thus a more sophisticated approach would potentially perform even better in the cross-scene setting.
W3-Big performance gaps by this method between ACC and AP are still observed although it is claimed that this is what the method addresses. E.g., in Table 5 --> ACC 72.58 AP 81.12.
W4-Resizing the images to 256 reduces the synthetic artifacts, and it should be omitted. The fact that the training images are small makes the effect of resizing very small. I would like to see how much the results are changed if the authors omit this part of the data augmentation/pre-processing pipeline.

Dogoulis, P., Kordopatis-Zilos, G., Kompatsiaris, I., & Papadopoulos, S. (2023, June). Improving synthetically generated image detection in cross-concept settings. In Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation (pp. 28-35).
questions: questions: What is the performance of the model if resizing is completely omitted?
How does the existing previous approach by Dogoulis et al. perform on your experimental design?
limitations: limitations: Sections 3.1 and 3.2 are not part of the methodology in my opinion, they should be placed under a separate motivation section as preliminary/insightful experiments. Section 3.3 although very small is still verbose and gives unnecessary details.
A similar setting has already been proposed by Dogoulis et al. 2023 which should be cited and the differences with the proposed definition/approach should be discussed.

Comments: title: Thank you for the reply.
comment: Dear reviewer iKNb,

We hope our rebuttal has addressed your concerns. We are looking forward to your reply and are more than willing to provide responses to any further inquiries you may have.

Thank you very much.
Dear reviewer 4wrZ,

We hope our rebuttal has addressed your concerns. We are looking forward to your reply and are more than willing to provide responses to any further inquiries you may have.

Thank you very much.
Thank you for acknowledging our rebuttal and supporting the acceptance of the paper. We will certainly incorporate your suggestions into the final version.
Thanks for the authors' response. I appreciate the authors' efforts in the rebuttal. Given the clarifications provided by the authors and the response and reviews to other reviewers, I will keep the positive initial rating I had provided.
Thank you for your response and increasing the score. We will incorporate your feedback into the final version.
My questions are addressed and score updated.