Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Authors: Yao Xiao
Binbin Yang
Weiyan Chen
Jiahao Chen
Zijie Cao
ZiYi Dong
Xiangyang Ji
Liang Lin
Wei Ke
Pengxu Wei
Venue: ICML 2025 poster (2025)
Keywords: AI-generated image detection
human preference
text-to-image generation
Primary area: applications->computer_vision
TL;DR: We find that high-quality AIGIs preferred by humans tend to be easier to detect for existing AIGI detectors.

Abstract: The remarkable evolution of generative models has enabled the generation of high-quality, visually attractive images, often perceptually indistinguishable from real photographs to human eyes. This has spurred significant attention on AI-generated image (AIGI) detection. Intuitively, higher image quality should increase detection difficulty. However, our systematic study on cutting-edge text-to-image generators reveals a counterintuitive finding: AIGIs with higher quality scores, as assessed by human preference models, tend to be more easily detected by existing models. To investigate this, we examine how the text prompts for generation and image characteristics influence both quality scores and detector accuracy. We observe that images from short prompts tend to achieve higher preference scores while being easier to detect. Furthermore, through clustering and regression analyses, we verify that image characteristics like saturation, contrast, and texture richness collectively impact both image quality and detector accuracy. Finally, we demonstrate that the performance of off-the-shelf detectors can be enhanced across diverse generators and datasets by selecting input patches based on the predicted scores of our regression models, thus substantiating the broader applicability of our findings. Code and data are available at \href{https://github.com/Coxy7/AIGI-Detection-Quality-Paradox}{GitHub}.

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Yao Xiao
Binbin Yang' Weiyan Chen
Jiahao Chen
Zijie Cao Pengxu Wei
Ziyi Dong'
Xiangyang Ji
Liang Lin 14
Wei Ke²
Abstract
0.8
0.80
Density
0.00
60.60
0.75
The remarkable evolution of generative models has enabled the generation of high-quality, vi- sually attractive images, often perceptually in- distinguishable from real photographs human eyes. This has spurred significant attention on Al-generated image (AIGI) detection. Intuitively, higher image quality should increase detection dif- ficulty. However, our systematic study on cutting- edge text-to-image generators reveals a counterin- tuitive finding: AIGIS with higher quality scores, assessed by human preference models, tend be more easily detected by existing models. To investigate this, we examine how the text prompts for generation and image characteristics influence both quality scores and detector accuracy. We observe that images from short prompts tend to achieve higher preference scores while being eas- detect. Furthermore, through clustering and regression analyses, verify that image charac- teristics like saturation, contrast, and texture rich- ness collectively impact both image quality and detector accuracy. Finally, we demonstrate that the performance of off-the-shelf detectors can be enhanced across diverse generators and datasets by selecting input patches based the predicted scores of our regression models, thus substantiat- ing the broader applicability of our findings. Code and data are available at GitHub.
0.55
0.00
0.70
10.59
10.85
IawarR
Im Rgaaaad
ImageReward
Imagafeward PixArt
5D
SDXL 1.0
0.00
Dereity
10
0.7
0.70
0.77
2.2
0.6
0.2
04
D22
3.4
5.2
0.4
HPS v2
HPS
v2
HIP v2
HPS
Figure 1: Average detector accuracy (the red curve) on gen- erated images with different quality scores predicted by hu- man preference models (ImageReward (Xu et al., 2023) and HPS (Wu al., 2023a)) and the distribution of quality scores (the blue histogram) for each generator. Counterintu- itively, for the same generator, images with higher quality scores tend to be casier to detect.
input text prompts (Ramesh et al., 2021; Nichol et al., 2021; Rombach et al., 2022; Podell et al., 2024; Esser et al., 2024; Chen et al., 2024c). As the generated images become per- ceptually indistinguishable from real images to human eyes, they pose substantial threats to the spread of disinformation, particularly in the context of news dissemination related to political and social issues, and to the security risks of personal information.
A Accordingly, Al-generated image (AIGI) detection aims to build a binary classification model for distinguishing real and fake (i.e., Al-generated) images. While existing meth- ods for AIGI detection (Ojha et al., 2023; Tan al., 2024; Koutlis & Papadopoulos, 2024; Baraldi et al., 2024; Chen et al., 2024a) may achieve promising results on benchmark datasets (Zhu et al., 2023; Bammey, 2023; Baraldi et al., 2024; Chen et al., 2024a), recent studies (Yan et al., 2025; Cavia et al., 2024) reveal that their performance real- world applications is unsatisfactory due mismatch in data distribution. Specifically, the AIGIs in existing datasets are randomly generated without ranking and filtering, while in the real world, synthetic images uploaded on the Inter- net are more likely of high quality and aligned with human preferences. This discrepancy in image quality between the training data and real-world cases seems to contribute to the
1. Introduction
Recently, deep generative models have demonstrated impres- sive capabilities in generating photorealistic images from
'School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China 'School of Software Engineer- ing, Xi' an Jiaotong University, Xi' an, China 'Department of Automation. Tsinghua University. Beijing, China *Peng Cheng Laboratory, Shenzhen, China. Correspondence to: Pengxu Wei <weipx3 www.llssys..uu.n......
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Are High-Qualite AI-Generated Images More Difficult for Models to Detect?
suboptimal performance of detectors in practice, and raises an important question: Are high-quality generated images preferred by humans more difficult for models to detect?
ditions and image features. Our findings suggest that images generated from short text prompts exhibit- ing specific low-level features (e.g., high saturation and rich texture) tend to achieve higher quality scores while being easier to detect.
To study the performance of existing AIGI detectors on high- quality generated images preferred by humans, we collect a dataset of high-quality images generated by text-to-image models with diverse prompts and obtain their quality scores predicted by human preference models (Xu et al., 2023; Wu et al., 2023a). By testing the average accuracy of six detec- tors, we observe that generated images with higher quality scores are consistently easier to detect across different gen- erators and preference models, as depicted in Figure 1.
We present a potential application of our findings in enhancing the detector performance: identifying the most detectable patch of an input image based on its low-level characteristics.
2. Related Works
To explain this counterintuitive phenomenon, we analyze two primary factors of the quality score: the text prompts and image characteristics. Firstly, we observe that im- ages generated from shorter prompts achieve higher quality scores, likely because existing generators struggle to follow complex prompts faithfully. More importantly, these im- ages are also more easily spotted by AIGI detectors, which provides valuable insights into the observed phenomenon.
2.1. AI-Generated Image Detection
Methods. In the common setup of Al-generated image (AIGI) detection, a detector is trained on fake images gen- erated by one or more generators and paired real images. The detector is expected to generalize to fake images from unseen generators. Existing AIGI detection methods can be primarily categorized as fingerprint-busel or end-to-end. Fingerprint- based models rely on certain low-level finger- prints of the generated images, such as diffusion reconstruc- tion error (Wang et al., 2023), up- up-sampling artifacts (Tan et al., 2024), and filters proposed for steganalysis (Fridrich Kodovsky, 2012; Zhong et al., 2024; Chen al., 2024b). In contrast, end-to-end models classify an input image directly based on its pixel values (Wang et al., 2020). To improve the generalization of end-to-end models, some methods (Ojha et al., 2023; Koutlis & Papadopoulos, 2024) utilize features from large-scale pre-trained models such as CLIP (Rad- ford et al., 2021). Others adopt contrastive learning to pro- mote the separation of real and fake images in embedding space (Baraldi et al., 2024; Chen et al., 2024a).
To further study the common characteristics of generated im- ages with higher quality scores and higher detector accuracy, we conduct clustering-base based analyses using quality-relatee features extracted by a human preference model (Xu et al., 2023). By examining representative image clusters, we find that certain low-level image characteristics, such as high saturation and rich texture, may serve as indicators of high quality scores and high accuracy for detectors. This moti- vates to apply multiple linear regression to investigate the correlation between various image characteristics (in- dependent variables) and both quality scores and detector accuracy (dependent variables). As expected, regression models for different dependent variables consistently high- light the influence of several low-level image characteristics. To demonstrate the applicability of our findings and exam- ine the generalization of the regression models, we apply them to the input patch selection of ffffthe-shelf detectors and evaluate their performance across different generators and datasets. Experimental results reveal that selecting the most detectable patch identified by the regression models can enhance the performance of several existing detectors.
Benchmarks. Several benchmarks are constructed to evalu- ate the generalization performance of AIGI detectors, featur- ing the coverage of diverse generators. ForenSynths (Wang et al., 2020) consists of images generated by 6 generative adversarial networks (GANs) (Goodfellow et al., 2014) and 5 other convolutional neural networks. GenImage (Zhu et al., 2023) is constructed based on the 1000 ImageNet classes (Deng et al., 2009), covering GANs and diffusion models. Specifically, for text-to-image diffusion models like Stable Diffusson (Rombach et al., 2022), a simple prompt template "photo of [class]" is applied to construct the class conditioning prompts. With the prevalence of text-to-image generators, recent works (Lu et al., 2023; Baraldi et al., 2024; Chen et al., 2024a) collect prompts for generation from the image captions in existing datasets, including CC3M (Sharma et al., 201 2018), LAION (Schuh- mann et al., 2022), and COCO (Lin et al., 2014). Syn- thubuster (Bammey, 2023) obtains captions of real images from RAISE- 1k ney descriptor (Midjourney Inc., 2024) and CLIP Interroga-
The main contributions of this paper are as follows:
We collect a high-quality and diverse AIGI dataset to enable the study of the relationship between image quality and the detection difficulty.
We reveal that high-quality AIGIS preferred by humans tend be easier to detect for existing AIGI detectors.
We investigate the correlation between average detec- tor accuracy and quality scores predicted by human preference models, studying the influence of text con-
et al., 2015) via Midjour-
2

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
tor (pharmapsychotic, 2022). Although the text prompts in these datasets may cover a wide range of semanties, they are relatively short and simple, in contrast to the long and de- scriptive captions utiliize by more advanced text-to- -o-igee models (Betker et al., 2023; Chen et al., 2024c). This poten- tially limits the diversity of images generated from the same generator in terms of visual features and complexity.
itively believed? To this end, we collect high-quality and diverse AIGI dataset and empirically study how detector ac- curacy may relate to the quality scores predicted by human preference models.
3.1. Evaluation Setup
Dataset collection. To study the relationship between im- age quality and detector accuracy, a diverse dataset with a sufficient number of high-quality generated images is essen- tial. However, as reviewed in Section 2.1, existing bench- mark datasets for AIGI detection suffer from limitations in image quality and diversity. These shortcomings stem from the exclusion of more advanced generators such as Diffusion Transformers (Peebles & Xie, 2022; Esser et al., 2024; Chen et al., 2024c), a lack of prompt engineering considerations (Liu & Chilton, 2022), and reliance on low- complexity text prompts. To this end, we construct a high- quality and diverse dataset by 1) collecting real images from four source datasets; 2) obtaining 4,000 captions spanning wide range of complexity from these real images; and 3) generating fake images using these captions as prompts based on text-to-image generators, e-g., Stable Diffusion 2.1
Detector performance on high-quality AIGIs. Unlike im- ages in previous benchmarks that are randomly generated, real-world AIGIS tend to be of higher quality due to more advanced generators (e.g., Diffusion Transformers (Peebles & Xie, 2022; Esser et al., 2024; Chen et al., 2024c)), the application of prompt engineering (Liu & Chilton, 2022), and human selection of candidate generation outputs. To study existing detectors on real-world AIGIs, Cavia et al. (2024) collect real and fake images from social networks, and find that detectors trained on existing datasets gener- alize poorly on these real-world data. Similar conclusions are drawn by Yan et al. (2025), who construct a dataset of high-quality AIGIs collected from online communities and filtered by human annotators to ensure they are challeng- ing for humans to detect. In addition, Song et al. (2024) argue that the generalization issue of deepfake detectors can be attributed to their reliance on easy-to-spa -to-t artifacts in low-quality training samples. However, the poor generaliza- tion of existing detectors on real-world AIGIS can also be explained by other factors, such as data biases concerning image size and compression (Grommelt et al., 2024; Ricker et al., 2024). Therefore, whether high-quality AIGIs are more difficult for detectors remains an open question.
(SD 2.1) (Rombach et al., 2022), Stable Diffusion XL 1.0 (SDXL 1.0) (Podell et al., 2024), Stable Diffusion 3 (SD 3) (Esser et al., 2024), and PixArt-a (Chen et al., 2024c). Specifically, negative prompts are applied during generation to improve image quality, and a set of positive modifiers are randomly sampled and appended to prompts for increased diversity in image characteristics. Details are presented in Appendix A.
Human preference models. Since annotating large-scale datasets with human preferences is challenging, we uti- lize pre-trained human preference models (e.g.. ImageRe- ward (Xu et al., 2023) and Human Preference Score v2 (HPS v2) (Wu et al., 2023a)) as a substitute for scoring. This pro- vides a scalable and consistent approach to assessing image quality in alignment with human judgments. These models take generated image and its corresponding text prompt as inputs and predict a quality score for the image. Although the preference models commonly take image-text alignment into account for quality assessment, we empirically find that this does not affect our conclusions, as discussed in Appendix B.2.
2.2. Quality Assessment for Al-Generated Images
Despite the prevalence of Inception Score (IS) (Salimans al., 2016) and Fréchet Inception Distance (FID) (Heusel et al., 2017) in image generation evaluation, these metrics may not be suitable for assessing the visual quality of in- dividual images and are not well-aligned with human pref- erence, as suggested by (Kirstain et al., 2023; Wu et al., 2023b). To this end, a series of human preference mod- els (Kirstain et al., 2023; Xu et al., 2023: Wu et al., 2023b;a; Zhang et al., 2024b) are constructed based on large-scale AIGI datasets with human preference annotations, including ratings of single images or rankings of image pairs in the presence of corresponding text prompts.
AIGI Detectors. To ensure the reliability of our conclu- sions, we evaluate six existing open-source AIGI detectors that generalize well on our dataset. We use the average ac- curacy across these detectors as an indicator of how easily a set of generated images can be detected. Specifically, the se- lected detectors include NPR (Tan al., 2024), RINE (Kout- lis & Papadopoulos, 2024), CoDE (Baraldi et al., 2024), DRCT (Chen et al., 2024a) (including its Conv-B and CLIP variants), and SuSy (Bernabeu-Peree et al., 2024). These
3. Are Higher-Quality Images Harder to Detect as Intuitively Believed?
Intuitively, high-quality generated images preferred by hu- mans should contain fewer visible artifacts and therefore may be more difficult to detect. In this section, we aim to answer the question: Are high-quality generated images preferred by humans harder for models to detect, as intu-
3

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
SD 2.1
SDXL 1.0
SD 3
PixArt-a
2.1.11
SDXL 1.0
SD 3
PixA---
Proportonn
02-
3.50
1.40
0.80
0.50
8.45
Iim seheww
Density
9.010
+1.00
10.5
-375
020
0.60
1.00
0.10
1:00
0.00
10.10
0.50
6
012
#Cornact
&Carrect
#Correct PixArt-d or
2.1
SDXL 1.0
$0.3 3
SD
SDXL 1.0
3
PixArt-a
-0.30
Propotionn
Density
9.010
02T
2.25
01-
0.2
02
0.28
0.77
2.005
0.20
-0.26
0.26
-0.24
25
88990,07
Correct
Correct
Correct
*Correct
SD
SDXL 1.0
SD 3
PixMt-a
Figure 2: Average quality scores (the red curve) for gener- ated images of different difficulty levels and the distribution of sample difficulty (the blue histogram). The difficulty is measured by the number of correct predictions (#Correct) from the 6 detectors. Notably, images that are easier de- tect (e.g., #Correct 5) tend to have higher quality scores.
Dersity
3.010
3.80
10.65
0.75
0.50
3.000
Length
Text Length
Text Length
Length
Figure 3: Quality scores (the red curves in the first two rows) and average detector accuracy (the red curves in the third rows) for images generated from prompts with different lengths, with the distribution of prompt lengths (the blue histogram). Short prompts (S 20 words) and medium-length prompts (21-40 words) generally correlate to higher quality scores and higher detector accuracy, in contrast with the long prompts (> 40 words).
detectors typically pre-process input images using center cropping without resizing. For evaluation, we directly apply the official pre-trained weights and configurations to testing on our dataset.
3.2. Main Observations
By investigating AIGI detectors on generated images of different quality scores, we make an intriguing observation: images with higher quality scores tend to be easier to detect, as suggested by Figure 1'. Specifically, for each genera- tor and preference model, we uniformly split the range of quality scores into 30 segments and calculate the average de- tector accuracy on samples corresponding to each segment. consistent trend emerges: for images with quality scores above the distribution peak, average detector accuracy generally increases as the quality score rises. Addition- ally, while detectors may achieve relatively high accuracy on images with low quality scores, this performance still falls short compared to that on the highest-quality images.
4. Why Do High Quality Scores Correlate With High Detector Accuracy?
To understand why higher quality scores of generated im- ages may correlate with higher accuracy for detectors, we study the commonalities of high-quality images as assessed by human preference models. Specifically, as the quality scores predicted by these models depend on both the text prompt for generation and the features of the generated im- age, we first examine the effect of text prompt complexity in Section 4.1. Next, we explore the common characteris- tics of high-quality images through clustering analysis in Section 4.2. Finally, in Section 4.3, we conduct regression analyses to assess how these image characteristics correlate with quality scores and detector accuracy.
To further explore the relationship between quality scores and average detector accuracy, we plot the changes in qual- ity scores as a function of detection difficulty in Figure 2, which can be viewed as a transposition of Figure 1. Specifi- cally, since the possible values of average accuracy of the 6 detectors are discrete for single sample, we use the number of detectors that correctly classify the sample as an index for detection difficulty (with a lower number indicating higher difficulty). As shown in Figure 2, images that are easier to detect tend to have higher quality scores. In conjunc- tion with the observations in Figure there is a positive correlation between quality scores and detector accuracy, particularly for images of higher quality.
4.1. Influence of Text Prompt Complexity
As suggested by (Zhang et al., 2024a; Dong et al., 2024; Ma et al., 2024), existing text-to-image generators tend to strug- gle more with generating images from complex text prompts compared short and simple ones. Hence, we have reason to believe that the quality of images generated under long and short text prompts differs, and we further validate this hypothesis through experiments. The results in Figure 3 con- firm that images generated from short and medium-length prompts (at most 40 words) generally achieve higher av- erage quality scores. Moreover, within this range, images generated from short prompts (at most 20 words) exhibit
Additional results more generators and human preference models are provided in Appendix B.I.

---

Are High-Qualit, AI-Generated Images More Difficult for Models to Detect?
SD 2.1
SDXL 1.0
SD
PixArt-a
(rich texture)
F-C TC0.1088
F- 0.55 16
r- 0.3598 0.02049
F- 0.41 0.C0657
0.8
0,000g3
0.00019
0.7
0.6
Cluster #12 structural complexity)
-1
-1 maggoRaward
-1 ImagaReward
-1 ImageRoward
ImageReward
SD 2.1
SDXL 1.0
SD
PixArt-a
Cluster #29 (high contrast)
F- 0.5537
r= 0.30 18 p= 0.00013 3
r- 0-4489 == n hnd
r= 0.52.5
F= 0.00M?)
0 0.0011
Accuracy
0.7
0.6
Cluster #38 (high saturation)
0.3
0.20
0.25
0.30
0.20
0.25
0.30
0.20
0.25
0.30
0.20
0.25
0.30
HPS V2
HP5 V2
H17 V2
HP5
Figure 4: Analyses on generated images clustered by quality-related features extracted by ImageReward model. Left: The average detector accuracy and the average quality scores consistently positively correlated the cluster level. Each point represents image cluster. The Pearson's 's correlation coefficient and the p-value are shown in the top left comerr for each figure. Right: Visualization of image patches from typical clusters of PixArt-a with high average detector accuracy and high average quality scores. We take the center 224 x 224 patch in alignment with the pre-processing pipeline of detectors studied in this paper.
slightly lower quality scores compared those generated from medium-length prompts with 21-40 words.
our analysis through image clustering to better understand these underlying characteristics.
More importantly, we find that text prompt complexity has similar influence on average detector accuracy. Figure 3 reveals that generated images produced from short prompts not only have higher quality scores but are also easier to detect. This finding provides further insight into our observa- tion in Section 3, suggesting that certain types of generated images inherently exhibit both high quality scores and high detector accuracy.
4.2. Image Clustering on Quality-Related Features
The observations in Section 4.1 suggest that high-quality images may exhibit certain characteristics that contribute higher detector accuracy. To further explore these common traits, we cluster images based on their visual features and analyze the average quality scores and detector accuracy within each cluster. As we focus on quality-related features, the clustering is conducted in the feature space of the Im- ageReward (Xu et al., 2023) model. We apply the K-Means algorithm (Lloyd, 1982) with 50; other implementation details are presented in Appendix A.2.
To understand how text prompt complexity affects the gen- crated images, we first conduct a preliminary analysis of their visual characteristics. A notable trait of images gener- ated from short prompts their relatively simple semantic content and structure, which can be attributed to the limited number of entities mentioned in the prompts. In addition, regarding the shortest prompts, descriptions of visual attributes appears to contribute to the relatively lower quality scores of the generated images. In particular, prompts specifying object colors tend to produce images with higher saturation, which may lead to improved quality scores. Further details are provided in Appendix B.3.
Based on the clustering results, we first observe a significant positive correlation between average detector accuracy and quality scores at the cluster level, measured by the Pear- son correlation coefficient, as shown in Figure 4, albeit the correlation is not strong. This trend further supports our observations in Figure and Figure
find that the absence of
Then, we focus on clusters with both high average detector accuracy and quality scores. Through manual inspection of images within these clusters, we identify several recurring characteristics, including rich textures, low structural com- plexity, high contrast, and high saturation, as illustrated in Figure 4. We argue that these common characteristics may help explain the positive correlation between detector accu- racy and quality scores. For example, highly saturated im-
While the analyses based on text prompt complexity of- fer valuable insights into the positive correlation between quality scores and average detector accuracy, grouping im- ages solely by text length may not fully exploit the visual attributes that human preference models and detectors are sensitive to. Therefore, in the following sections, we extend
5

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Table I: Comparison of regression models with different sets of independent variables (low-level or high-level fea- tures) and different dependent variable y. Standardized coefficients for each independent variable and the corre- sponding -sooe are presented. The high-level features extracted by DINOv2 are reduced to 6 dimensions via PCA.
Dependent variable. The dependent variable y can be (1) the average accuracy of the detectors, or (2) the average preference score predicted by the ImageReward or HPS v2 model. This yields three regression models for each set of features.
Data fitting. Building on the clustering analysis in Sec- tion 4.2, we perform regression analyses at the cluster level, averaging the metrics within each cluster. To obtain more general regression model, we combine the cluster data for all four generators and fit the regression models to them. However, it should be noted that the average accuracy of detectors may vary considerably across different generators, as indicated by Figure 4. This variation can harm the re- gression models that fit the mixed data by considering only the aforementioned features as independent variables. To this end, we allow the regression model to learn generator- dependent intercepts to mitigate this bias. This is achieved by adding an indicator variable for each generator to repre- sent the source of an image cluster. Since we are primarily interested in the relationship between independent variables 21; T2, ... Fe and the dependent variable y, rather than predicting the absolute value of y, the indicator variables and intercept can be ignored after the data fitting process. Additional details are provided in Appendix A.4.
(a) Low-level image characteristics
Lightness
Contrast
Saturation Sharpness
Texture
Structural
Ka
Richness
Complexity
Accuracy
0.10
0.52
0.43
-0.33
0.42
-0.30
0.77
ImageReward
0.26
0.20
0.34
-0.46
0.35
-0.25
0.81
HPS
0.32
0.47
0.34
-0.71
0.54
-0.32
0.74
(b) Hegh- features (reduced to 6 dimensions by PCA)
3:
Accuracy
40.18
-0.23
0.30
-0.23
0.04
0.11
0.76
ImageReward
-0.39
-0.22
-0.04
-0.18
-0.02
-0.1
0.76
HPS v2
444
-0.09
0.03
-0.30
-0.10
-0.17
0.88
ages may be more visually appealing to humans, leading to higher quality scores, while their unnatural coloration could make them more distinguishable for detectors. Likewise, images with fine details and rich textures tend to receive higher quality scores, yet detectors might excel at identify- ing subtle artifacts in these intricate details, such as slight distortions along edges.
Results for low-level features. Table la compares the stan- dardized coefficients for each low-level image characteristic Icarned by the three regression models and presents the cor- responding coefficient of determination (R²). As suggested by the sign and magnitude of the coefficients, the image characteristics have similar contribution the prediction of accuracy and quality scores. Specifically, lightness, con- trast, saturation, and texture richness of images positively correlate with both detector accuracy and quality scores, while sharpness and structural complexity have a negative correlation with these dependent variables. These conclu- sions are consistent with our observations and conjectures in Sections 4.1 and 4.2 and indicate the potential strengths and weaknesses of existing detectors. Moreover, the con- sistency across the three regression models reinforces the observed positive correlation between quality scores and average detector accuracy.
4.3. Linear Regression Analyses
To further substantiate the findings in Section 4.2 and quan- titatively study how image characteristics may influence quality scores and average detector accuracy, we perform linear regression analyses on different features. Specifically, a linear regression model estimates the linear relationship between a dependent or target variable (e.g., detector ac- curacy or quality score) and a set of independent or input variables (e.g., image features). The linear relationship can be interpreted from the coefficients of independent variables learned from the data. The instantiations of the regression analyses and the results are detailed as follows.
Independent variables. Building on our previous observa- tions, we first examine six low-level visual characteristies of images: lightness, contrast, saturation, sharpness, texture richness, and structural complexity. Each characteristic is quantified using a scalar metric, which we treat as an inde- pendent variable x; (i = 1, 2, .6) in a linear regression model. The specific choices of metrics are detailed in Ap- pendix A.3, Additionally, we explore whether high-level features also contribute to the observed positive correlation between average detector accuracy and quality scores. To this end, we extract high-level features from images us- ing DINOv2 (Oquab et al., 2023), a self-supervised image encoder, and then reduce the feature dimension to 6 via PCA (Pearson, 1901) to align with the level features.
Results for high-level features. As suggested by Table lb, not all principal components of the high-level features have consistent correlations with different dependent variables. Nevertheless, the subspace spanned by the unit vectors cor- responding to X1, I2, and Z4 may provide certain support for the positive correlation between quality scores and av- erage detector accuracy. As illustrated in Figure 5, clusters with lower values of x2 and i4 generally possess higher quality scores and are easier to detect. In addition, we no- tice that some outlier clusters (e.g., #5, #16, #18) do not comply with the major trend. Specifically, Cluster #5 con-
6

---

Are High- AI-Generated Images More Difficult for Models to Detect?
30-
30
45
Cluster #5 (lexts)
1.0
20
20
0.80
ImageReward
X4
10
#18
0.6
10
#16
0.75
Accuracy
Cluster #18/818 (planes and ships)
#18
#18
0.70
-10-
0.2 -10-
Cluster #31 (cars)
0.65
-20
-20
20
20
X2
X2
Figure 5: Analyses on clusters of images generated by PixArt-a. Left: Cluster distribution in the high-level feature space. I2 and œ correspond to the two independent variables in Table lb and represent two principal components of the high-level features. Points are colored based on the ImageReward scores, Center: The same cluster distribution with points colored based on the average detector accuracy. Right: Visualization of image patches from outlier clusters.
sists of images with dense and distorted characters, which leads to low quality scores but makes them not difficult to detect. Similarly, detectors achieve relatively high accuracy on images in Clusters #16 and #18, which depict planes and ships with a number of portholes and have low quality scores. These examples suggest that existing detectors may be sensitive to certain semantic details like characters and portholes. This could explain the phenomenon observed in Figure that some lower-quality images are also relatively easy to detect.
selecting input patches according to the regression models' predictions and investigate whether their performance can be enhanced on various generators and datasets.
5.1. Selection Strategies for Input Patches
Most existing methods for Al-generated image detection assume that fake images are completely generated by mod- els rather than locally edited from real images. Under this setting, a series of methods (Ojha et al., 2023; Tan et al., 2024; Koutlis & Papadopoulos, 2024; Baraldi al., 2024; Chen et al., 2024a) adopt center cropping in their image pre-processing pipeline without resizing. While limiting the receptive field to patch of size 224 224 may lead to substantial losses of high-level information, especially for high-resolution images, it preserves fine- ine-grained details and enables detectors to spot low-level traces of genera- tion (Corvi et al., 2023; Gragnaniello et al., 2021).
Discussion. The linear regression analyses suggest that certain low-level image characteristics (e.g.. high saturation or low sharpness) and bigh-level features tend to induce higher detector accuracy and higher quality scores. From causal perspective (Pearl, 2009), these features could be the confounders underlying the counterintuitive positive correlation between accuracy and quality. However, through further intervention experiments presented in Appendix B.4, we find that several low-level image characteristics do not have uniform causal effects on the accuracy of different detectors, suggesting the existence of other confounding factors. Hence, for developing robust and generalizable AIGI detectors, future studies could benefit from further analyzing how these image features affect detectors.
Based on our findings in Section 4, we hypothesize that selecting input patches according the common character- istics of high-quality or easy-to-detect images can enhance the performance of existing detectors. Specifically, we first patchify the input image and then use one of the regression models from Section 4.3 to rank the patches by their pre- dicted scores. By selecting the patch with the highest score for given image, we expect make the detection more accurate, particularly for detecting fake images.
5. Can We Optimize Patch-Based AIGI Detection Through Quality Analysis?
However, this patch selection strategy may not have the same expected effects on different types of detectors, as reviewed in Section 1. Specifically, recent study (Chen et al., 2024b) suggests that simple patches with poor texture are more effective for fingerprint extraction than complex patches. This may contradict the role of texture richness in our patch selection strategy based on the regression results. Consequently, we speculate that for fingerprint-basel mod-
In this section, we aim to explore whether our analysis of image qualities in Section 4 can be leveraged to optimize existing patch-based AIGI detection methods, which typi- cally rely on center-cropping and disregard other patches. Specifically, we examine whether the regression models from Section 4.3 can identify the most detectable patch in an image. To this end, we improve patch-base detectors by
7

---

Are High- AI-Generated Images More Difficult for Models to Detect?
Table 2: Average detector accuracy (%) of end-to-end models on fake images. We compare different patch selection strategies based on regression models with different dependent variables y.
GenImage
DRCT-2M
Synthbuster
Dependent Variable y
Stralegy
Midjourney
SDXL-
SDXL-
LCM-
SDv2-
SDXL-
DALLE2
DALL-E
Firefly
Midjourney
Average
Refiner
Turbo
SDXL
Ctrl
Ctrl
Center
76.3
75.3
56.8
62.6
75.0
76.2
38.1
51.4
41.4
75.0
62.8
Easiest
75.0
76.0
56.6
62.1
74.8
76.0
38.6
51.3
45.4
75.5
63.1
Accuracy
Random
70.6
72.0
56.1
61.1
73.6
75.0
36.7
50.9
37.8
71.1
60.5
Hardest
63.5
66.4
54,9
60.9
71.8
73.4
34.2
47.8
29.5
63.9
56.6
Easiest
73.1
75.0
56.2
60.5
74.4
76.0
37.6
50.1
44.6
70.9
61.8
ImageReward
Random
70.6
72.0
56.1
61.1
73.6
75.0
36.7
50.9
37.8
71.1
60.5
Hardest
68.6
69.3
54,9
62.4
72.2
73.6
35.7
50.8
37.2
71.5
59.6
Easiest
74.1
75.7
56.3
61.0
74.5
76.0
38.6
44.2
73.7
62.5
HPS v2
Random
70.6
72.0
56.1
61.1
73.6
75.0
36.7
50.9
37.8
71.1
60.5
Hardest
66.3
67.3
55.0
62.0
72.1
73.5
34.3
49.7
34.5
69.2
58.4
els, selecting the patch with the lowest score predicted by the regression models may yield better detection performance.
higher average detector accuracy on various generated im- ages, compared to the Random or Hardest strategy. Second, the Center strategy adopted by existing methods compa- rable to the Easiest strategy and superior to the Random strategy, which possibly suggests the particularity of the center areas of generated images. Third, the three regres- sion models provide consistent effects on average detector accuracy, although the two models fitting quality scores may less effective in predicting the easiest patch. These re- sults further validate the relationship between quality scores and average detector accuracy.
In the following part of this section, we verify our hypothe- ses by experimentally comparing four input patch selection strategies: (1) Easiest: selecting the patch with the high- est predicted score; (2) Hardest: selecting the patch with the lowest predicted score; (3) Random: selecting a ran- dom patch; (4) Center: taking the center patch, as in the aforementioned existing methods.
5.2. Experiments
Comparison of different detectors. To study how differ- ent detectors are affected by patch selection strategies, we compare their average performance across all generators in Table 3. The Easiest and Hardest strategies are based on the regression model fitting accuracy. First, all end-to-end de- tectors except for DRCT (CLIP) achieve the best AP when selecting the input patch based on the Easiest strategy. Sec- ond, for DRCT (CLIP), we notice that the accuracy real images is significantly lower than that of other end-to-end detectors. Adopting the Easiest strategy leads to a further decline in this metric, which indicates that this detector may learn different features for discriminating real and fake im- ages. Third, the patch selection strategies have opposite effects on fingerprint-basee
AIGI detectors. In addition to the six AIGI detectors de- scribed Section 3.1, we consider SSP (Chen et al., 2024b), which takes a single simple patch for fingerprint extraction. We train the SSP model on GenImage (Zhu al., 2023) instead of using the official checkpoints with unsatisfactory generalization. Details are provided in Appendix A.5. Over- all. the detectors can be categorized into: (1) fingerprint- based models: NPR (Tan et al., 2024) and SSP; (2) end- to-end models: RINE (Koutlis & Papadopoulos, 2024), CoDE (Baraldi al., 2024), DRCT (Chen al., 2024a), and SuSy (Bernabeu-Perez et al., 2024).
Datasets. To examine the generalization of the regres- sion models on unseen generators, we test the detectors using different patch selection strategies on images gener- ated by the following generators from existing benchmark datasets: Midjourney from GenImage (Zhu al., 2023); SDXL-Refiner, SDXL-Turbo, LCM-SDXL, SDv2-Ctrl, and SDXL-Ctrl from DRCT-2M (Chen et al., 2024a); DALL-E
detectors, where the Hardest strategy consistently provides better performance than the Easiest and Random strategies. For SSP, its original strategy of selecting the simplest patch can produce higher accuracy on fake images, but at the cost of lower accuracy on real ones. Overall, the results validate that the proposed patch selection strategies can be used enhance the performance of some off- thel-hh-lff detectors, while the "easiest" patches for end-to-end detectors may be the "hardest" patches for fingerprint-based detectors and vice versa.
DALLE 3, Firefly, and Midjourney from Synthbuster (Bam- mey, 2023). We clarify the data selection in Appendix A.6.
Comparison of different regression models. Table 2 shows the average accuracy of end-to-end detectors on fake images produced by different generators. First, selecting the input patch with the highest scores predicted by regres- sion models (i.e., the Easiest strategy) generally provides
8

---

Are High-Quallte Al-Generated Images More Difficult for Models to Detect?
Table 3: Average performance (%) across all generators for different detectors using different patch selection strate- gies. ACC: accuracy on all images; AP: average precision; faCeak//CCreed accuracy on fake/real images. (#: the strategy proposed in (Chen al., 2024b) for SSP, which selects the simplest patch with the lowest texture diversity in the input image.)
A Acknowledgements
This work is supported in part by National Natural Science Foundation of China (NSFC) under Grant No.62376292, U21A20470, Guangdong Provincial General Fund No. 2024A1515010208.
Impact Statement
Type
Detector
Stralegy
ACC
AP
ACC
ACCrual
Center
75.0
90.2
51.6
98.5
This paper studies Al-generated image detection, which has become an important topic for Al governance as AI- generated images become more perceptually indistinguish- able from real ones. The findings in this paper suggest po- tential strengths and weaknesses of existing detectors, and are expected to promote the development of more advanced models for detecting various Al-generated images. A po- tential negative impact lies in the facilitation of adversarial evasion attacks against existing detectors. For instance, as suggested by our results, lowering the lightness and con- trast of generated images may impair the performance of certain kinds of detectors. A preliminary exploration of the effectiveness of such adversarial image manipulations is presented in Appendix B. B.4. However, we believe that this potential risk also motivates future research on robust detectors for Al-generated images in the real world.
Easiest
74.2
91.9
49.7
98.9
RINE
Random
71.4
90.3
44.0
98.9
Hardest
67.6
90.0
36.4
98.9
Center
79.4
78.8
60.1
98.8
Easiest
79.7
79.2
60.8
98.7
CoDE
Random
79.4
79.0
59.9
99.0
Hardest
77.8
77.2
56.9
98.7
End-to-end
Center
76.0
87.6
59.7
Easiest
77.8
89.4
62.1
93.6
SuSy
Random
74.4
87.0
55.9
93.0
Hardest
82.7
48.6
91.1
Center
75.4
84.8
53.7
97.1
DRCT
Easiest
76.4
87.6
53.8
99.0
(Conv-B)
Random
75.1
84.2
54.1
96.0
Hardest
73.3
82.5
53.6
93.1
Center
78.9
88.4
89.0
68.9
DRCT (CLIP)
Easiest
77.6
86.7
89,2
65.9
Random
78.4
87.6
88.5
68.3
Hardest
80.3
88.6
87.7
72.9
Center
71.6
75.9
66.8
76.3
References
Easiest
69.4
74.6
66.6
72.1
NPR
Fingerprint-busd
Randor
73.7
76.3
68.5
Bammey, Q. Synthbuster: Towards detection diffusion
Hardest
76.6
78.6
71.1
82.2
model generated images. IEEE Open Journal of Signal Processing, 2023.
Center
72.0
79.7
51.7
92.4
Basiest
70.9
77.4
53.8
SSP
Random
73.4
81.0
54.9
92.0
Baraldi, L., Cocchi, F., Cornia, M., Nicolosi, A., and Cuc- chiara, R. Contrasting deepfakes diffusion via contrastive learning and global-local similarities. In European Con- ference Computer Vision, pp. 199-216. Springer, 2024.
Hardest
74.5
81.0
56.4
92.5
Simplest
74.5
78.3
64.2
84.7
Bernabeu- Perez, P., Lopez-Cuent E., and Garcia-Gasulla, Present and future generalization of synthetic image detectors. arXiv preprint XXi::2409.1412. 2024.
6. Conclusion
In this paper, we make a counterintuitive observation that high-quality Al Al-generated images (AIGIs) preferred by hu- mans tend to be less difficult for models to detect. We ex- plain the positive correlation between detector accuracy and the quality scores of generated images by investigating the influence of text prompts and image features. Our results suggest that images generated from shorter text prompts or those exhibiting specific low-level image characteris- ties (such as high saturation, high contrast, rich texture, and low structural complexity) correlate with higher quality scores and higher detector accuracy. Finally, we suggest that existing patch-based AIGI detectors can be improved by selecting the most detectable patch identified based on low-level visual characteristics. This validates the broader applicability of our findings.
Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang, J., Lee, J., Guo, Y., al. Improving image generation with better captions. 2023. URL htw.t:tt///// pttttts:///d..... hec.d.o.pee..... cemeaai.ccom/ee cww/papera dall-e-3, 3. 3.pdf.
Black Forest Labs. Flux.1 [dev], 2024. URL https: //github. cww/black.b.oooooooooo forest labs flux.
Canny, J. A computational approach to edge detection.
IEEE Transactions on Pattern Analysis and Machine In- telligence, (6):679-6))) 6688 1986.
Cavia, B., Horwitz, E., Reiss, T. and Hoshen, Y. Real- time deepfake detection in the real- world. arXiv preprint arXiv:2406 09398, 2024.

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Chen, B., Zeng. J.. Yang, J., and Yang, R. Dret: Diffusion reconstruction contrastive training towards universal de- tection of diffusion generated images. In International Conference on Machine Learning. 2024a.
Fridrich, J. and Kodovsky, J. Rich models for steganalysis of digital images. IEEE Transactions on information Forensics and Security, ((3):668-882) 2012.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances in neural informa- tion processing systems, 27, 2014.
Chen, J., Yao, J., and Niu, L. A single simple patch is all you need for ai-generated image detection, arXiv preprint arXiv:2402.01 1233, 2024b.
Chen, J., YU, J., GE, C., Yao, L., Xie, E., Wang, Z., Kwok, J.. Luo, P., Lu, H., and Li, Z. Pixart-S's allphaS: Fast training of diffusion transformer for photorealistic text-to- image synthesis. International Conference on Learning Representations, 2024c.
Gragnaniello, D., Cozzolino, D., Marra, F., Poggi, G., and Verdoliva, L. Are gan generated images easy to detect? a critical analysis of the state-of-ther the-art.. In 2021 IEEE international conference on multimedia and expo (ICME),
pp. 1-6. IEEE, 2021.
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E.,, Tong, W., Hu, K., Luo, J., Ma, Z., et al. How far to gpt-4v? closing the gap to commercial multimodal mod- els with open-source suites. Science China Information Sciences, 671122220011, 2024d.
Grommelt, P., Weiss, I.., Pfreundt, F.-J., and Keuper, Fake or jpeg? revealing common biases in generated image de- tection datasets. arXiv preprint arXiv:2403. 17608, 2024.
Han, J., Liu, J., Jiang, Y., Yan, B., Zhang, Y., Yuan, Z., Peng, B., and Liu, X. Infinity: Scaling bitwise autoregres- sive modeling for nigh-resolution image synthesis. arXiv preprint arXiv:2412.04433 2024.
Cortex Foundation. MidJoumey v6 Dataset, 2024. URL https://huggingee CortexiIM/mlddouureeee--
co/ coodattstss//////
Corvi, R., Cozzolino, D., Zingarini, G., Poggi, G., Nagano, K., and Verdoliva, L. On the detection of synthetic images generated by diffusion models. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2023.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. Segment anything. In Proceedings of the IEEF/CVE International Conference on Computer Vision, pp. 4015-4026, 2023.
Dang-Nguyen, D.-T., Pasquini, C., Conotter, V., and Boato, G. Raise: A raw images dataset for digital image foren- sics. In Proceedings of the 6th ACM multimedia systems conference, pp. 219-224, 2015.
Deng, J., Dong, W., Socher, R., Li, L.J.,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Kirstain, Y., Polyak, A., Singer, U., Matiana, S., Penna, J., and Levy, O. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In Advances in Neural Information Processing Systems, volume 36, pp. 36652-36663, 2023.
Dong, Z., Xiao, Y., Wei, P., and Lin, L. Decoder-only 1ims are better controllers for diffusion models. Pro- ceedings of the 32nd ACM International Conference on Multimedia, pp. 10957-10965, 2024.
Koutlis, C. and Papadopoulos, S. Leveraging represen- tations from intermediate encoder-blocks for synthetic image detection. In European Conference on Computer Vision, pp. 394-411, 2024.
Egan, B., Redden, A., XWAVE, and SilentAntagonist. Dalle3 Million+ High Quality Captions, 2024. URL https://huggingfe co/ datasets/ ProGamerrov/sytthheeclc-ttttte da holle3-high--uuality.
Li, J., Li, D., Savarese, S., and Hoi, S. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 2023.
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F,, et al. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference on Machine Learning, 2024.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Dollár, P., and Zitnick, C. L. Mierosoft coco: Common objects in context. In European Conference on Computer Vision, 2014.
10

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Liu, V. and Chilton, L. B. Design guidelines for prompt engineering text-to-image generative models. In Proceed- ings of the 2022 CHI conference human factors in computing systems, pp. 1-23, 2022.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dock- horn, T., Müller, J., Penna, J., and Rombach, R. SDXL: Improving latent diffusion models for high-resolution im- age synthesis. In International Conference on Learning Representations, 2024.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Lloyd, S. Least squares quantization in pem. IEEE transac- tions on information theory, 28(2):129-132) 1982.
Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, 2021.
Lu, Z.,, Huang, D., Bai, L., Qu, J., Wu, C., Liu, X., and Ouyang, W. Seeing is not always believing: benchmark- ing human and model perception of ai- ai-generated images. In Advances in Neural Information Processing Systems, 2023.
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation. In International conference on ma- chine learning. 2021.
Ma, B., Zong, Z., Song. G., H., and Liu, Y. Exploring the role of large language models in prompt encoding for dif- fusion models. In The Tiirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma. T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P., and Feichtenhofer, C. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.0077 240114 2024.
Midjourney Inc. Midjourney, 2024. URL https: /www. midjourney. com/.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: To- wards photorealistic image generation and editing with text-guided diffusion models. In International Confer- ence on Machine Learning, 2021.
Ricker, J., Lukovnikov, D., and Fischer, A. Aeroblade: Training-fre free detection latent diffusion images using autoencoder reconstruction error. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9130-9140, 2024.
Ojha, U., Li, Y., and Lee, Y. J. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Oquab, M.. Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:230..07193, 2023.
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Pearl, J. Causality. Cambridge University Press, 2009.
Schuhmann, C.,, Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems, 2022.
Pearson, K. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11): 559-572, 1901.
Peebles, W.S. S. and Xie, S. Scalable diffusion models with transformers. 2023 ieee. In CVF International Conference on Computer Vision (ICCV), volume 4172, 2022.
Sharma, P., Ding, N., Goodman, S., and Soricut, R. Con- ceptual captions: A cleaned, hypemymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pp. 2556- 2565, 2018
Peli, E. Contrast in complex images. JOSA A, 7(10):2032- 2040, 1990.
pharmapsychotic.
Clip interrogator, 2022. comcco/////aoo waaaa/haamapscchhti.... clip-interrogaton
URL
Shen, K., Nath, S. S., Brielmann, A., and Dayan, P. Sim- plicity in complexity. arXiv preprint arXiv:2403.03134 2024.

---

Are High-Qualit, Al-Generated Images More Difficult for Models to Detect?
Song, W. Yan, Z., Lin, Y., Yao, T., Chen, C., Chen, S., Zhao, Y., Ding, S., and Li, B. A quality-centric frame- work for generic deepfake detection. arXiv preprint arXiv:2411.05333, 2024.
Zhu, M., Chen, H., Yan, Q.. Huang. X., Lin, G., Li, W., Tu, Z., Hu, H., Hu, J., and Wang. Y. Genimage: A million-scale benchmark for detecting ai-generated image. In Advances in Neural Information Processing Systems, 2023.
Tan, C., Liu, H., Zhao, Y., Wei, S., Gu, G., Liu, P., and Wei, Y. Rethinking the up-sampling operations in cnn- enn-based generative network for generalizable deepfake detection. In Proceedings of the IEEE/CVE Conference on Com- puter Vision and Pattern Recognition, 2024.
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., and Efros, A. Cnn- Enn-generated images are surprisingly easy spot... for now. In Proceedings of the IEEE/CVF Confer- Computer Vision and Pattern Recognition, 2020.
Wang, Z., Bao, Zhou, W., Wang, W., Hu, H., Chen, H., and Li, H. Dire for feffusioon-gnerrate image detection. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, 2023.
Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., and Li, H. Human preference score v2: solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint 2314:2306609341, 2023a.
Wu, X., Sun, K., Zhu, F., Zhao, R., and Li. H. Human preference score: Better aligning text-to-image models with human preference. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2096- 2105, 2023b.
Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., and Dong, Y. Imagereward: Learning and evaluating hu- man preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2023.
Yan, S., Li, O., Cai, J., Hao, Y., Jiang, X., Hu, Y., and Xie, W. sanity check for ai-generated image detection. International Conference on Learning Representations, 2025.
Zhang, B., Zhang, P., Dong, X., Zang, Y., and Wang, Long-clip: Unlocking the long-text capability clip. In European Conference Computer Vision, pp. 310-325, 2024a.
Zhang, S., Wang, B., Wu, J., Li, Y., Gao, T., Zhang, D., and Wang. Z. Learning multi-dimensional human prefer- ence for text-to-image generation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Patlern Recognition, pp. 8018-8027, 2024b.
Zhong, N., Xu, Y., Li, S., Qian, Z., and Zhang, X. Patcheraft: Exploring texture patch for efficient ai tengeeerdeed image detection, arXiv preprint ar 2024.
12

---

Are Hugh-Quality AI-Generated Images More Difficult for Models to Detect?
A. Implementation Details
A.1. Dataset Collection
Following the pipeline of the construction of existing datasets for AIGI detection (Zhu et al., 2023; Bammey, 2023; Baraldi et al., 2024; Chen et al., 2024a), we collect real images from existing datasets, acquire the corresponding captions, and generate the fake images via set of text-to-image models taking these captions as text prompts.
Real image collection. The contents of the fake images are loosely determined by the real images. Hence, to ensure the diversity of the dataset, we collect real images from four existing datasets: COCO (Lin et al., 2014), CC3M (Sharma et al., 2018), LAION-Acstheti (Schuhmann et al., 2022), and SA-1B IB (Kirillov et al., 2023). In particular, SA-IB is included as it contains more images with complex scenes compared to the other datasets.
Text prompt collection. The text prompts are expected to be faithful descriptions of the visual content of the real images and cover various levels of complexity. To this end, we apply BLIP-2 (Li et al., 2023) and a large multimodal model (InternVL2- 8B (Chen et al., 2024d)) to the captioning of real images. Specifically, we use different instructions for Intern V1.2-8B to obtain captions with varying complexity. We also include the high-quality captions from COCO and those produced for SA-1B by (Chen et al., 2024c), which supplement the low-complexity and high-complexity captions, respectively. After filtering and balancing the distribution of text lengths, we obtain 4,000 captions and use them as the text prompts for fake image generation. As depicted by the histogram in Figure 3, these prompts effectively cover a wide range of text length, from less than 10 words to over 120 words.
Fake image generation. The fake images are generated by six advanced open-source text-to-image models, namely Stable Diffusion 2.1 (SD 2.1) (Rombach al., 2022), Stable Diffusion XL 1.0 (SDXL 1.0) (Podell et al., 2024), Stable Diffusion 3 (SD 3) (Esser et al., 2024), PixArt-ox (Chen et al., 2024c), FLUX.1 [dev] (Black Forest Labs, 2024), and Infinity (Han et al., 2024). For each text prompt, we produce a fake image by each generator. To improve the quality of generated images and increase the diversity of image characteristics, we follow the prompt engineering adopted by (Baraldi et al., 2024). Specifically, a fixed negative prompt is applied for all images, while the positive modifiers are randomly sampled from a set of common modifiers the generation of each image. The generated images are compressed by the same format and quality as the real counterparts eliminate the compression bias (Grommelt et al., 2024).
A.2. Image Clustering Based on Quality-Related Features
To obtain the quality-related features for image clustering, we utilize the ImageReward (Xu et al., 2023) model. An empty text is fed to the model instead of the text prompt corresponding to the input image, as we find that using the corresponding prompts can lead near-linear distribution of samples in the feature space, possibly due to the strong features related to image-text alignment. Real images corresponding the 4,000 text prompts the potential discrepancy between real and fake images and are neglected in the subsequent analyses. However, we find that most clusters are mixed with real and fake images, which suggests the high fidelity of the generated images in our dataset. In addition, considering that the detectors studied in this paper mostly adopt center-cropping without resizing in their image pre- pre-processing pipeline, we perform feature extraction based on the center 224 x 224 patches of images. Finally, to ensure the reliability of our analyses, we filter out the clusters with the number of generated images fewer than 30.
included in the clustering process to indicate
A.3. Metrics for Low-Level Image Characteristics
Lightness: Convert the image to HSV space, and take the average value of the Value channel.
Contrast: Compute the root-mean-square (RMS) contrast (Peli, 1990), i.e., the standard deviation of the pixel intensities.
Saturation: Convert the image to HSV space, and take the average value of the Saturation channel.
Sharpness: Apply the Laplacian operator on the grayscale image, and then compute the variance.
Texture richness: Apply the Canny edge detector (Canny, 1986) the grayscale image, and then compute the edge density, Le., the ratio of edge pixels.
Structural complexity: Motivated by (Shen et al., 2024), we take the square root of the number of image segments detected by the SAM2 2 model (Ravi et al., 2024).
13

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
A.4. Multiple Linear Regression
In Section 4.3, we analyze the correlation between image features (independent variables) and the average detector accuracy or quality scores (dependent variable). Both independent variables 2 "2.".", 2g and the dependent variable y standardized to obtain the standardized coefficients (or beta coefficients) for comparison. The regression models are fit to the data using the least squares approach.
A.5. Training SSP on Genlmage
In experiments, we find that the official checkpoints of the SSP model (Chen et al., 2024b) trained on fake images produced by single generators of Genlmage (Zhu et al., 2023) fail to yield satisfactory results on other datasets. To obtain a more generalizable SSP model, we train it on a mixture of fake images generated by all generators, and eliminate the compression bias (Grommelt et al., 2024) in GenImage by aligning the compression format and quality of the fake images with the real ones.
A.6. Test Data Selection
In Section 5.2, we test the detectors on fake images generated by various generators from multiple benchmark datasets (Zhu et al., 2023; Bammey, 2023; Chen et al., 2024a). To ensure that the results are meaningful for the validation of our findings, we filter out generators that satisfy any of the following conditions: (I) the generator is SD 2.1 or SDXL, which are included in our dataset and should not be used for testing the generalization of our regression models; (2) the generator included in or very similar to the common generators used for the training of the detectors (e.g., SD 1.4/1.5), since detectors consistently have high accuracy images generated by seen generators; (3) the size of the generated images are 256 x 256 or smaller, leaving no room for patch selection.
B. More Experimental Results and Further Discussions
B.1. Validation on More Generators and Human Preference Models
Figure I and Figure in the main text demonstrate our main observations concerning the relation between the average detector accuracy and image quality scores based 4 generators (i.e., SD 2.1, SDXL 1.0, SD 3, and PixArt-a: and 2 human preference models (i.e., ImageReward and HPS v2). To further validate that the observations are consistent across different generators and preference models, we extend Figures and 2 with two more generators (a commercial DiT model, FLUX.1 [dev] (Black Forest Labs, 2024) and an autoregressive model, Infinity (Han et al., 2024)), and one additional human preference model (MPS (Zhang et al., 2024b)), as shown in Figures and 7. Furthermore, we examine whether similar trends can be observed on user-created images from advanced closed-source generators, including Midjourney v6 (Midjourney Inc., 2024) and DALL-E 3 (Betker et al., 2023). To this end, we sample 50,000 and 75,000 generated images from existing datasets collected by Cortex Foundation (2024) and Egan et al. (2024), respectively. The evaluation results are presented in Figures 8 and 9. These results collectively validate that main observations in Section 3 are consistent across a broad range of generators and different human preference models.
B.2. Reducing the Influence of Image-Text Alignment in Image Quality Assessment
Existing human preference models (Xu al., 2023; Wu et al., 2023a; Zhang et al., 2024b) commonly input the text prompt for generation in addition to the generated image, because a high-quality image in practice should be not only visually appealing but also aligned with the text prompt (i.e., satisfying the intention of the user). However, this paper focuses more on the visual quality of the generated images, regardless of how the images are aligned with the text prompts.
To validate that the observed correlation between the detector accuracy and human preference scores can be attributed to the visual quality of images, we try to reduce the influence of the image-text alignment in the comparison of image quality by replacing the text input for the preference models. Specifically, instead of feeding the preference models with the original prompt for generation, we use the BLIP-2 (Li et al., 2023) caption of the generated image itself, which is expected to be well-aligned with the image as evaluated by the preference models. The results presented in Figures 10 and 11 are consistent with those produced by the standard metrics (i.e., Figures 6 and 7), validating the correlation between the detector accuracy and image visual quality.
14

---

Are Hlghh-Quliit AI-Generated Images More Difficult for Models to Detect?
SD 2,1
SDXL 1.0
SD 3
PixArt-a
FLUX.1 [dev]
Infinity
00800
0.60
Density
0.50
0.80
0.65
0.75
0..50
0080
Accuracy
0.75
0.77
M
0.70
0.60
0.70
2
2
2
2
2
2
ImageReward ard
waward
Ima geReward
ImageReward
ImageReward FLUX.1 [dev]
ImageReward
SD 2.1
SDXL 1.0
SD 3
PixArt-a
Infinity
0.90
0..00
Density
10-
0.70
-0.80
0.60
0.05
Accuracy
0.65
0.000
0.60
-0.75
WW
0.75
0
0.2
0.60
0.2
0.2
0.4
0.50
0.2
0.4
0.4
0.2
0.4
HPS
HP5
SD 2.1
SDXL
PixArt-a
FWUX.1 [dev]
Infinity
Density
0.10
0.80
+0.60
0.88
Accuracy
0.80
0.05
0.63
0.50
0.78
0.00
0.60
-0.70
WW
0.40
0.76
MPS
MPS
MPS
Figure 6: Average detector accuracy (the red curve) on generated images with different quality scores predicted by human preference models (lwageReward, HPS v2, and MPS) and the distribution of quality scores (the blue histogram) for each generator.
SD 2.1
SDXL 1.0
SD 3
PixArt-a
FLUX.1 [dev]
Infinity
Proportion
0.3
-0.50
0.40
-0.80
0.50
0.45
1.00
1.20
1.00
ReageRewal
0.20
-0.60
0.40
0.80
-0.60
0.0
0.00
.040
0.35
0.60
01
012343 4 5
01 2 3
2
4
1 3
1 #Correct
0
#Correct
#Correct
#Correct
#Correct
#Correct
SD 2.1
SDXL 1.0
SD
PixArt-a
FLUX.1 [dev]
Infinity
0.27
Praportion
0.24
0.27
0.28
00.29
0.2
0.28
HPS
-026
0.28
0.1
0.27
0.0
3 6 #Corret
3
2 4
2
2
#Correct
#Correct
#Correct
#Correct
#Correct
1.0
PixArt-a
FLUX.1 [dev]
Infinity
13.50
14.60
Praportion
0.3
13.00
0.2-
13,00
13.00
10.00
12.50
0.14
12.50
0.0
4
12.00
14.20
3
13.60
#Correct
#Correct
Figure 7: Average quality scores (the red curve) for generated images of different difficulty levels and the distribution of sample difficulty (the blue histogram). The difficulty is measured by the number of correct predictions (#Correct) from the detectors.
15

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Midjoumey
Midjoumey v6
Midjourney
DALL-E
DALL-E
DALL-E
Density
0.60
10
0.50
0.20
Accuracy
Density
0.4
0.72
0.10
0.74
0.59
0.2-
0.57
5
0.58
0.55
0.05
0.2
0.70
0.68
0.05
0.70
0.72
-2
0.2
0.4
0
20
0.2
0.4
Reward
HPS v2
MP5
ImageReward
HPS v2
MPS
(a) Midjourney v6
(b) DALL-E
Figure 8: Average detector accuracy (the red curve) on generated images with different quality scores predicted by human preference models (ImageReward, HPS v2, and MPS) and the distribution of quality scores (the blue histogram) for closed-source generators.
Midjourney v6
Midjourney v6
Midjourney v6
DALL-E 3
DALL-E 3
DALL-E
11.000
mmageRewar aaaa
0.30
Proportion
0.3-
ImageRewar
0.29
Proportion
0.80
0.75
-10.50
0.28
MPS
0.2
0.70
13.00
0.70
0.28
0.3
0.28
12.50
0.0
0.65
0.27
0.0
# #Correct
#Correct
#Correct
#Correct
# #Crrrect
#Correct
(a) Midjourney v6
(b) DALLEE
Figure 9: Average quality scores (the red curve) for generated images of different difficulty levels and the distribution of sample diffieulty (the blue histogram) for closed-source generators. The difficulty is measured by the number of correct predictions (#Correct) from the 6 detectors.
SD 2.1
SDXL
SD 3
FLUX.1 [dev]
Infinity
1.0-
0.80
Density
0.50
0.80
Accuracyy
0.5
0.55
0.75
0.70
0.60
0.40
00775
0.0
0.70
ImageR R
ImageReward* SDXL 1.0
Im ImageReward SD 3
ImageReward
ImageReward* FLUX.1 [dev]
ImageReward*
SD 2.1
PixArt-a
Infinity
Density
10
0.80
0.660
Accuracy
0.75
0.60
5
0.70
0.70
0.75
0.55
0.50
0.2
0.4
0.2
0.4
0.2
0.4
0.2
0.4
0.2
0.4
HPS v2*
HPS v2*
HPS v2*
HPS v2*
HPS v2*
HPS v2*
SD 2.1
SDXL 1.0
SD 3
PixArt-a
FLUX.1 [dev]
Infinity
Density
0.65
0.65
0.80
0.60
0.75
0.60
0.55
0.75
0.70
0.40
0
20
0
20
20
20
20
20
MPS*
MPS*
MP5
MP5
MPS*
MP5
Figure 10: Average detector accuracy (the red curve) generated images with different quality predicted by human preference models (IwadeReward, HPS v2, and MPS; *; using the BLIP-2 caption of the generated image instead of the original prompt as the text input) and the distribution of quality scores (the blue histogram) for each generator.
16

---

Are High-Quality AI-Generated Images More Difficult for Models to Detect?
SD 2.1
SDXL 1.0
SD 3
PixArt-a
FLUX.1 [dev]
Infinity
Proportion
0.3
1.10
1.20
1.40
1.20
1110
0.60
1.00
1.00
1.20
0.0
012 4 5
01 012 3 4
0 0 0
2 3 #Correct PixArt-a
2 Correct FLUX.1 [dev]
2 Correct
#Corret
#Correct
#Corrett
SD 2.1
SDXL 1.0
SD
Infinity
0.33
10:30
Proportion
0.2
-0.8
029
0.30
0.30
0.29
0.1
0.29
0.29
0.28
0027
0.28
0.28
00.28
0.0
#Correct
#Correct
#Correct
#Correct
#Correct
#Correct
SDXL 1.0
PixArt-a
FLUX.1 [dev]
Infinity
Proportion
0.3
12.25
12.60
0.2
12.00
12.50
11.60
0.1
955
11.75
11.40
11.50
12.00
12.40
0.0
922
11.50
# Correct
#Correct
#Correct
#Correct
#Correct
#Correct
Figure 11: Average quality scores (the red curve) for generated images different difficulty levels (#: using the BLIP-2 caption of the generated image instead of the original prompt as the text input) and the distribution of sample difficulty (the blue histogram). The difficulty is measured by the number of correct predictions (#Correct) from the detectors.
17

---

Are High- AI-Generated Images More Difficult for Models to Detect?
B.3. Explaining the Influence of Text Prompt Complexity
In Section 4.1, we observe that (1) images generated from short and medium-length prompts (with most 40 words) tend have higher quality scores and higher average detector accuracy, compared with those generated from longer prompts; (2) the quality scores are relatively lower for images generated from the short prompts (with at most 20 words) compared to those from medium-length prompts (with 21-40 words).
To explain observation (I), we speculate that images generated from shorter prompts tend to have lower structural complexity (which correlates with higher detector aceuracy and higher quality scores as suggested in Section 4) as fewer entities are mentioned in the prompts. This can be validated by the results in Figure 12a.
2.4
0.55
Average Structural Complecd e
2.3
50.3
0.50
2.2
SDKL
Average Saturation
2.1
2.0
0.40
1.9
1.8
0.35
1.7
1.6
10:19
20-29
20:39
40-49
50-59
60469
70-79
En-89
5e00
10:19
20-29
30-39
40-49
50.59
60-69
70-79
80-89
Se00
Text Length
Text Length
Structural complexity vs. text length
(h) Saturation vs. text length
Figure 12: Average structural complexity and average saturation of images corresponding to different ranges of text length.
To explain observation (2), we first notice that the shortest prompts (e.g., those with less than 10 words) may contain fewer deseriptions of the attributes of the objects, as evidenced in the examples in Appendix C. Besides, images generated from these prompts generally exhibit lower saturation, as shown in Figure 12b. Therefore, we hypothesize that the lack of descriptions of visual attributes, especially those related to the colors, may lead to lower saturation of the generated image, and thereby lower quality scores and detector accuracy, suggested in Section We verify this hypothesis by counting the ratios of prompts with color-related descriptions within different ranges of text length, and comparing the average saturation of images generated from prompts with or without such descriptions, as shown in Table 4. It is clear that higher ratio of prompts with text length in 20-29 have color-related descriptions, and images generated from prompts with such descriptions exhibit higher saturation on average.
Table 4: Comparison text prompts with and without color-related descriptions. The prompt ratio is counted for three segments of text lengths, and the average saturation computed over all samples with text length fewer than 30.
Prompt Ratio (%)
Rolor-Related Descriptions
Average Saturation
0-9
10-19
20-29
With
14.2
37,9
63.2
0.47
Without
85.8
62.1
36.8
0.40
B.4. Adversarial Attacks Inspired by the Linear Regression Analyses
The linear regression analyses in Section 4.3 indicate the potential strengths and weaknesses of existing AIGI detectors, which could facilitate the understanding and promote the development of AIGI detection methods. However, the conclusions could also inspire adversarial attacks against existing AIGI detectors. Specifically, Table 1a suggests certain adversarial directions for image manipulations, such as decreasing the lightness, contrast, and saturation, or increasing the sharpness. To investigate the effectiveness of such adversarial directions, we implement these manipulations with different factors (0.5 means decreasing by 50%: 1.5 means increasing by 50%). The results in Table 5 suggest that: (1) the adversarial lightness and contrast manipulations are effective for DRCT and RINE; (2) CoDE and SuSy are less sensitive to these manipulations;
18

---

Are High-Qualit Al-Generated Images More Difficult for Models to Detect?
(3) NPR shows improved performance under most kinds of manipulation. The inconsistent behavior of the detectors under simple image manipulations indicates the difficulty in developing universal attack against different AIGI detectors.
From another perspective, these intervention experiments on the low-level image characteristics suggest that they do not have a direct and uniform causal effect on the accuracy of different detectors, despite the correlations witnessed in Table la. Future works could further investigate the underlying reasons for the correlations between these image characteristics and the detector accuracy.
Table 5: Detector accuracy (%) under different image manipulations. (The red/green numbers indicate decreased/increased accuracy compared to the baseline; underlined numbers indicate significant changes in accuracy of over 10%..
DRCT
DRCT
Manipulation
Factor
RINE
CoDE
SuSy
NPR
(ConvB)
(CLIP)
None
N/A
65.6
83.1
31.8
66.7
83.8
66.8
0.5
51.8
55.8
20.9
68.0
84.9
89.2
Lightness
1.5
71.7
93.6
77.8
69.7
79.4
82.0
0.5
44.4
76.6
15.7
72.9
82.6
89.1
Contrast
1.5
76.3
93.3
84.1
69.5
85.5
87.3
0.5
63.2
84.3
35.8
64.9
84.7
64.2
Saturation
1.5
67.6
90.9
56.7
65.1
82.7
68.5
0.5
76.6
85.3
34.0
71.2
89.5
Sharpness
1.5
55.0
81.7
52.2
62.5
80.6
73.4
B.5. The Relation Between Accuracy and Quality for Real Images
While this paper aims investigate whether high-quality Al-generated images are more difficult detect and understand the correlation between detector accuracy and AIGI quality, it is natural ask whether similar conclusions hold for real images. To this end, we transfer the investigations to our collected real images. The results presented in Figure 13 do not show similar consistent correlation between high detector accuracy and high quality scores across different preference models. It requires further studies to explore which kind of real images are easier or harder to be identified for AIGI detectors.
Density
real
0.75
10-
real
0.74
0.10
real
Accuracy
Proportion
0.3-
real
0.40
Imagen maageReward
real
0.24
real
0.72
0.2
0,20
50
0.2
0.70
0.05
0.72
0.24
11.40
-0.00
0,23
0.0
-2
0.2 HPS v2
0.00
0.0
3 #Carrect
2 4
0 1 0123466
ImageReward
# Carrect
#Correct
(a) Accuracy vs. Quality
(b) Quality Accuracy
Figure 13: Investigating the relation between average detector accuracy and quality scores for real images. In contrast with the observations on fake images (Figures 6 and 7), the correlation between detector accuracy and image quality is less significant and relatively inconsistent across different human preference models.
C. Qualitative Examples
We present random examples of our data corresponding to the short, medium-length, and long prompts in Figures to 16. For each generated image, we also provide the average accuracy of detectors (i.e., the proportion of detectors with correct predictions) and the quality scores in these figures.
19

---

Are HightQuulity AI-Generated Images More Difficult for Models to Detect?
50 211
SOXL 1.0
SD
PixArt-a
Accuery Raward
(#Correct);
Accuary
ImageReward Score:
ImagsReward
Score:
HPSV2
Prompt: 'A person stands on hilitop at night, gazing the starry sky and the Milky Way galexy."
SD 2.1
SDXL 1.0
SD
PixArt-a
Accuary ieCorrecth: 4/6
Accuary (#Correct); 4/6
Accuary CeCrreects: 4/6
Accuary (#Correcti: 5/6
Imageli Imageiteward Score: 1.127
Imagelteward Scoore: 1.142
Imagaiteward Score: 1.710
Im Imagelteward Score: 1.858
HP Svz Score: 0.289
HPSv2 Score: 0.276
HPSV2 Score: 0.262
HPSV2 Score: 0,305
Prompt "The image shows pink flower with rnd stamen in the center, surrounded large leaves."
PixArt-a
Accuary (#Correct):
(#Correct); 5/6
Accuary (4 (eCorrect): 5/6 ImageReward Score:
(ACorrect); 6/6
mageReward Score: 1:138
ImageReward Score: 0,786
ImageRewart 3.356
HP5 Score: 0,282
HPSv2 Score: 0.279
HPSv2 Score: 0.249
HPSV2 Score: 0.292
Prompt: clothes
SD 2.1
SOXL 1.0
SD
PixArt-a
(#Correct); maggRReward ImwgeReward
Accuery
Accuary
(#Correct):
ImageReward Score:
ImageReward
HPSV2 Score:
Prompt: "agpies bottles on a wooden table"
Figure 14: Random examples from our dataset corresponding to the short prompts (1-20 words).
20

---

Are High-Quuiln AI-Generated Images More Difficult for Models to Detect?
50 21
SDXL 1.0
SD
PixArt-a
Accuery Reward ImageReward
(#Correct):
Accuary
RaagRR
ImagsRewartd
Score:
Score:
HPSV2
Prompt: 'A white and orange bus parked on a racky, mountainous terrain with clear blue sky and scattered clouds in the background.
SD 2.1
SDXL 1.0
SD
PixArt-a
Accuary IeCorrecth: 4/6
Accuary (#Correct): 5/6
Accuary (eCorrect: 2/6
Accuary (#Correct): 4/6
Imagdleward Score: -1.018
Imagelleward Score: -0.212
ImageRRewrd Score: -0.516
Imagulleward Score: -0.433
HPSv2 Score: 0.257
HPSV2 Score: 0.277
HPSv2 Score: 0.244
NPSv2 Score: 0.248
Prompt: The showcases grand, classical building with prominent focade featuring tall calumnss, intricate stonework, and decorative podiment The butlding's roof adomed with omate details and golden statuo, while the entrance is Manked by iarge, detailed columns.
PixArt-a
Accuary (#Correct): 5/6
(#Correct); 3/6
Accuary (#Correct): 2/6
(ACorrect);
mageneward Score: 0,796
ImageReward Score: 1.193
ImageReward Score: HPSv2 Scorn: 0,208
ImageReward Score: 3.391
HPSu2 Score: 0,275
HPSv2 Score: 0.276
HPSv2 Score: 0.274
Prompt: "The image depicts table.
spread assortment pastries, juices, eggs, meets, cheeses, condiments, arranged
SD 2.1
SOXL 1.0
SD3 3
PixArt-a
Accuery
(#Correct); ImaagR
Accuary
Accuary (#Correct): ImageReward
Raward
ImageReward Score:
HPSV2 Score:
Score:
Prompt: "The imago depicts a man in grean jacket and cap, ralsing his right hand in tuumssup gesture, with a group of people similar green octire clapping and smiling in the bec bockonound"
Figure 15: Random examples from our dataset corresponding to the medium prompts (21-40 words).
21

---

Are High- Hligh-Quality AI-Generated Images More Difficult for Models to Detect?
5D 211
SDXL 1.0
SD
PixArt-a
Accuary (eCorrect);
Accuary (#Correct);
Accuary
ImageReward -0.776
ImageReward HPSV2 Score: 0.285
ImageReward HPSV2 Score: 0.269
ImageReward
HPSV2 Score: 0.213
HPSV2 0.286
Prompt: "Tep Thee image deepicts a vibrent hillside town with colorfal buildings cescading down steep siope, The architecture festures a mix of pastel hues, including pinks, yelows, and blues, with some building adomed with balconie and shuttors In the background, prominent church
with dome and cross stands out, adding refigious element to the scene. The sky is clew and bive, enhancing the overall picturesque
quality
5D 2.1
SDXL 1.0
SD
PixArt-a
Accuary (#Correcth: 4/6
Accuary (#Correct): 5/6
Accuary (CCrrect 4/6
Accuary (#Correcti: 2/6
Imagdleward Score: -1.534
Imagelteward Score: 0.321
Imagalteward Score: 0.124
Imagelteward Score: 0.002
HP5 Score: 0.247
HPSv2 Score: 0.264
HPSV2 Score: 0.287
HPs Score: 0.252
Prompt
collection Intricatoly decorated Christmas wreaths hanging shelf, The whatths
festive
berries, ribbons, decorations like sowwfokes and candy canes. The background suggests store or markett setting with blurred shelves and other Christmas decorations.
elements such
with colors predominenty
and while. Some wreatht have adtitiiona
SD
PixArt-q
Accuary (eCorrect): 6/5
Accuary (#Correct): 3/5
Accuary (eCorrect): 4/5
Accuary (ACorrect); 6/6
mageReward Score: 0.463
ImageReward Score: 0.681
Imageteward Score: 0.899
ImageReward Score: 0.773
HP Su2 Score: 0,288
HPSv2 Score: 0.295
HPSv2 Score: 0.303
HPSV2 Score: 0.125
Prompt: "The image features clase up of a field of white dassies with focus on the petals and the surrounding grass. The datsies are biooming, petals spread out, creeting
suroundiings end vibrant scene style the image artistic, with focus beauty their envtronment, emphastatng
dalsles the delicate and vibrant colors the dasiee and the soft, grass.
captures
SD 2.1
SDXL 1.0
SD3
PixArt-a
Accuary (@Correct): ImageReward Score:
Accuery (#Correct):
Accuary ImageReward Score:
Accuary Score: ImageReward HPSV2 0.257
ImageReward Score:
HPSV2 Score: 0.236
HPSv2 Sccor: 0.241
HPSV2 Score:
Prompt: "The image is black and white photo a large city, likely taken from an serial perspective, It showoassss e citystape with various bullakigs, incouding hotels, à beach, and a pool. The style of the Image reminsscnt of cissic black and white photographyy which emphasizes the contrast between light and shadow, creating a timeless and artistic representation of the urban landscape, The presence of The beach and pool adds sense lnisure and relexation to the city, contrasting with the bustling urban environment.
Figure 16: Random examples from our dataset corresponding to the long prompts (more than 40 words).
22


Reviews:
[{"review_id": "GpAJ7vmGmF", "reviewer": "Reviewer_gsPf", "summary": "This paper studied the correlation between the quality score of AI-generated images and the detection accuracy of AI-generated images. They found that AI-generated images with higher quality scores are easier to be detected by models. Then, they analyzed the influence of the length of text prompts and image quality features to the correlation. They also conducted experiments to study how to select patches for AI-generated image detection.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: It is questionable that the quality scores given by the pre-trained preference models can reflect actual human preference/image quality.**\n\nWe agree that the preference models could not replace humans in assessing image quality, and it is difficult to predict the human preference on image quality due to its subjective nature. Therefore, we show the validity of our conclusions by the **consistent phenomenon among different generators and different preference models**.  We acknowledge that two preference models may be insufficient to support the consistency, hence, we complement the main results with an additional preference model, MPS [b], and other generators (please refer to Q4 of Reviewer rmyb). The consistent results in Figure I/II in [a] further validate our conclusions.\n\n[a] https://anonymous.4open.science/r/ICML2025-rebuttal-4584/quality_and_accuracy.pdf  \n[b] Learning Multi-dimensional Human Preference for Text-to-Image Generation.\n\n\n**Q2: Why is the quality score determined by both text prompt and image?**\n\nExisting preference models commonly take the text prompt for generation as input because a high-quality image in practice should be not only visually appealing but also aligned with the text prompt (i.e., satisfying the intention of users). However, as you kindly suggest, this paper should focus on the visual quality of the generated image alone.\n\nTo this end, we try to minimize the influence of the image-text alignment in the comparison of image quality by replacing the text input for the preference models: instead of the original prompt for generation, we use the **BLIP-2 caption of the generated image itself**, which is expected to be well-aligned with the image as evaluated by the preference models. The corresponding results are presented in Figure III/IV in [a].\n\n**Q3: The correlation between the quality of real images and the detection accuracy.**\n\nThank you for your suggestion. We provide the results in Figure VII/VIII in [a], which suggests that there is no significant and consistent correlation between the quality and detection accuracy for real images, indicating that existing detectors may learn different features on real and fake images.\n\n**Q4: Why perform regression analyses at the cluster level but not image level in Sec. 4.3?**\n\nAt the image level, the accuracy is discrete (0/6, 1/6, ..., 6/6) as only 6 detectors are evaluated, which could induce overly high variance of the data and hinder the linear regression analyses. Therefore, we use the cluster-level data, where each sample is representative of a group of images with similar characteristics, to ease the regression analyses.\n\n**Q5: Sec. 5 shows that selecting high-quality patches is not always helpful for AI-generated image detection, which is in conflict with the main finding.**\n\nWe acknowledge that the proposed patch selection strategy may not always bring a performance gain for different data and different detectors. This is expected as our main observation (i.e., generated images with higher quality scores *tend to* be easier to detect) is **statistically** valid, and its application to the improvement of detectors may depend on the characteristics of the detector and certain implementations such as the image patchification strategy. We believe that the findings of this paper could motivate future studies to propose more effective detectors with improved algorithmic designs.\n\n**Q6: The ImageReward paper was published in NeurIPS 2023, not 2024.**\n\nWe are grateful for your careful reading! We will correct this error and proofread the references in the paper.", "claims_and_evidence": "This paper tried to study and explain the correlation between the quality of AI-generated images and the detection accuracy of AI-generated images. However, the quality of image is evaluated by some pre-trained preference models, not by human. Since the image quality assessment is a highly subjective and challenging task, it is questionable that the quality scores given by these models can reflect actual human preference/image quality. In other words, are the images with high scores really high quality?", "methods_and_evaluation_criteria": "When we detect AI-generated images, only the image itself can be accessed. Therefore, the quality of an image should also be decided only by the image itself, why the quality score is determined by both text prompt and image? The correlation between the quality and the detection accuracy of AI-generated images observed in this paper maybe mainly influenced by the text prompt not by the image true quality.", "theoretical_claims": "There is no theoretical claim.", "experimental_designs_or_analyses": "See “Methods And Evaluation Criteria”", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "--How about the correlation between the quality of real images and the detection accuracy?\n--In Section 4.3, why perform regression analyses at the cluster level, but not image level?\n--The main finding of this paper is that AI-generated images with high quality scores are easier to be detected. But, in Section 5, the study shows that selecting high quality patches is not always helpful for AI-generated image detection, which is conflict with the main finding.\nBesides, the experimental results shows that carefully selecting different quality patches cannot bring consistent performance gain.", "other_comments_or_suggestions": "Xu et al., 2024 “Imagereward: Learning and evaluating human preferences for text-to-image generation”This paper was published in NeurIPS 2023, not 2024.", "questions_for_authors": "See “Claims And Evidence”, “Experimental Designs Or Analyses” and “weakness”.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EgvjvmZ5Kx", "reviewer": "Reviewer_rmyb", "summary": "The paper investigates the detectability of AI-generated images (AIGIs), revealing a counterintuitive finding: higher-quality AI-generated images, preferred by humans, are actually easier for detection models to identify. It shows that prompt complexity (with shorter prompts producing higher quality, more detectable images) and specific image characteristics (like high saturation, contrast, and rich textures) significantly influence both image quality scores and detector accuracy. Finally, the paper demonstrates how these insights can be practically leveraged to enhance the performance of existing detectors by optimizing input patch selection based on predicted detectability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Discrepancy between existing datasets and real-world applications.**\n\nThe discrepancy between existing datasets and real-world applications lies in many aspects, such as semantics, quality, and image compression. This paper focus on the quality (as explained in lines 110-117, left column), while some previous studies emphasize that the real-world performance of detectors can be affected by the image compression bias in existing datasets.\n\nWe acknowledge that some detectors [1,2,3] are reported to perform well on some OOD data. However, we find that they still have poor performance on other datasets, as suggested by the results in: https://anonymous.4open.science/r/ICML2025-rebuttal-4584/table.pdf. We did not test the method proposed in [1] as it is not open-sourced.\n\n**Q2: Evidence for lacking diversity.**\n\nThank you for pointing this out. To clarify, prior studies (such as [1,2,3] as you mentioned) collect images generated by diverse generators to study the cross-generator generalization of detectors. In this paper, we instead focus on the data diversity corresponding to the same generator, especially the diversity of (1) quality-related image features and (2) prompt complexity. Specifically:\n\n(1) The diversity of quality-related features is achieved by the independently sampled positive modifiers, which are not applied in most existing datasets.\n(2) In contrast to Synthbuster, which has the highest diversity in prompt complexity but 98.5% of its prompts are under 60 words, our dataset (Fig. 3) exhibits a significantly wider distribution of prompt lengths.\nWe will revise the related descriptions in Sec. 2 and Sec. 3.1 to improve the clarification on dataset diversity.\n\n**Q3: The role of \"advanced generators\".**\n\nWe agree that current benchmarks already contain diffusion-based models like SD (1/2/XL series). However, by \"advanced generators\", we refer to those more capable of producing high-quality images, especially DiTs. SD 3 and PixArt-α\n are selected as representatives of DiTs, and the average quality of their generated images is higher than SD 2.1/XL with U-Net architecture, as indicated by the histogram of Fig. 1. We will improve the related descriptions and provide more statistical comparisons.\n\n**Q4: Results with more generators.**\n\nThank you for your valuable suggestion. We supplemented our data with images generated by the commercial DiT model FLUX.1 [dev], and Infinity, an autoregressive model. We reproduce Fig. 1/2 on the extended data and the results are presented in Fig. I/II in [a]. In addition, we validate our findings on closed-sourced models Midjourney v6 and DALL·E 3 based on images sampled from existing datasets in Fig. V/VI in [a]. The phenomena in these figures are consistent with Fig. 1/2 of the paper.\n\n[a] https://anonymous.4open.science/r/ICML2025-rebuttal-4584/quality_and_accuracy.pdf\n\n**Q5: The biases towards short prompts.**\n\nWe agree that these pre-trained detectors might be biased towards images generated from short prompts if their training data only comprises such images. Fig. 3 does suggest that these detectors tend to have higher performance if the prompt length is below 40.\n\nHowever, the effects of prompt length on the detector accuracy are indirect. Specifically, longer prompts may include more objects and depict a more complex scene, or they could be more likely to mention certain attributes of objects that affect their visual appearance. Therefore, the emphasis of our analyses (Sec. 4.2/4.3/5) is on the visual characteristics of generated images instead of the prompts, and our results in Sec. 5 suggest that our conclusions can be utilized to improve the performance of detectors on existing datasets based on short prompts. Please refer to Q2 of Reviewer 581z for further explanations.\n\n**Q6: Images generated from the shortest prompt exhibit opposite characteristics.**\n\nThank you for underlining the importance of this intriguing phenomenon. As explained in lines 210-215 (right column) and Appendix B (lines 694-729), we notice that the a significantly lower ratio of the shortest prompts contain color-related descriptions, while such descriptions tend to increase the saturation of image. Hence, compared with medium-length prompts (21-40 words), the shortest prompts induce lower saturation, which indicates lower accuracy according to Sec. 4.3. Therefore, this phenomenon does not contradict with our further analyses concerning image characteristics. \n\n**Q7: Explain the classification and provide qualitative examples of short, medium, and long prompts, along with their generated images and corresponding results.**\n\nThank you for your advice. We will revise the descriptions related to prompt lengths according to the following classification standard: \"short\" means 1-20 words; \"medium\" means 21-40 words; \"long\" means more than 40 words.\n\nWe provide randomly sampled qualitative examples accordingly in https://anonymous.4open.science/r/ICML2025-rebuttal-4584/qualitative.pdf.", "claims_and_evidence": "Even though the paper is very interesting, several of its claims lack clear substantiation:\n\n1. On lines `045-052`, the paper claims that existing datasets are randomly generated without ranking, leading to discrepancies between training data and real-world applications. However, existing literature [1][2][3] shows that some detectors perform well even with out-of-distribution data, which appears to conflict with the paper's claim. The authors should address this discrepancy explicitly.\n\n2. The paper argues (lines `110-111`) that existing datasets lack diversity but doesn't provide evidence for that. Moreover, prior studies [1][2][3] already include diverse samples generated from both GAN-based and diffusion-based models, including Stable Diffusion used by the authors themselves.\n\n3. The paper mentions the lack of advanced generators in existing datasets. However, current benchmarks already include both GAN and diffusion-based models, specifically Stable Diffusion, which is also utilized in this paper.\n\n4. The study employs only **four** generators, three of which (Stable Diffusion variants) likely have similar architectures, training datasets, and methodologies. This limitation could bias the results. Including a wider variety of generators, particularly those with differing architectures (e.g. Dall E 3, Imagen 3, Flux -- both open and close source models), would strengthen the validity of the findings.\n\n5. Importantly, the pre-trained detectors evaluated in this paper might be biased towards short prompts since they were likely trained on shorter prompt samples, given that the authors claim their dataset is the first to include high-quality images generated from longer prompts. This potential bias would significantly undermine the paper's conclusions. Therefore, providing quantitative evidence demonstrating that detector performance is not influenced by prompt length is imperative.\n\n# Ref:\n- [1] ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detection\n- [2] Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection \n- [3] Towards Universal Fake Image Detectors that Generalize Across Generative Models", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "No theoretical claims", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "Yes", "essential_references_not_discussed": "[1] ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detection\n[2] Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection \n[3] Towards Universal Fake Image Detectors that Generalize Across Generative Models", "other_strengths_and_weaknesses": "No", "other_comments_or_suggestions": "The paper needs clarifications:\n\n1. The paper claims that short prompts have higher quality and higher detectability. However, it also discusses the shortest prompt, which exhibits opposite characteristics. This contradiction is confusing for the authors. Intuitively, both short and long prompts should perform poorly, while a medium-length prompt, which hits the sweet spot, should perform well. Therefore, the authors are requested to resolve this inconsistency.\n\n2. The authors consider prompts from datasets like COCO to be short, which leads to confusion regarding what is classified as short or long. The paper should provide qualitative examples of short, medium, and long prompts, along with their generated images and corresponding results.\n\n3. The paper also needs to provide qualitative evidence (prompt with generated images) on how prompt length affects the generation results. Currently, only quantitative results are presented.", "questions_for_authors": "Check claim & evidence section", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K1rmGXhSYi", "reviewer": "Reviewer_581z", "summary": "This work considers the relationship between AI-generated images and real images, noting a counterintuitive phenomenon: generated images with higher quality scores, as assessed by human preference models, tend to be more easily detected by existing AIGI detectors. Additionally, it is observed that images generated from short prompts tend to achieve higher preference scores while being easier to detect.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The relationship between the curve and the histogram in the plots is not clearly explained.**\n\nThank you for pointing out the potential difficulty for readers to understand the figures. In Figure 1-3, the red curve illustrates how variable $y$ (e.g., the accuracy in Fig. 1) changes with respect to $x$ (e.g., the quality score in Fig. 1), and the blue histogram depicts the distribution of $x$. The curve and the histogram share the same $x$-axis.  We will revise the captions and related descriptions of Figure 1-3 to improve the clarity.\n\n**Q2: The underlying reasons for the phenomena illustrated by the figures are not well explained.**\n\nThe main observation illustrated by Fig. 1 (i.e., generated images of higher quality tend to be easier to detect) **can be explained by our analyses in Sec. 4**. To clarify, we do not suppose that the relation between the detector accuracy and the quality scores is causal; instead, we aim to explore the **potential confounders underlying the counterintuitive positive correlation** between the two variables. Our results in Sec. 4.3 suggests that certain low-level image characteristics and high-level features may be the confounders for this correlation, and our experiments in Sec. 5 further validate that these low-level image characteristics can be utilized to predict the detectability of image patches on a broader range of data. We will make these points clearer in the revision.\n\n**Q3: This study lacks a deeper theoretical analysis or exploration of underlying causes.**\n\nWe understand that theoretical analysis can provide deeper insights into the relationship between the detector accuracy and the image quality, although this paper is supposed to be an empirical study. We will provide a discussion from the causal perspective (as described in Q2) with a theoretical causal graph that explicitly depicts the relations among the variables we studied in this paper.\n\n**Q4: Provide comprehensive statistics on the dataset construction, such as the quantity and distribution of each category.**\n\nThank you for your kind suggestion. The distributions of the image quality scores and the prompt lengths are depicted by the histograms in Fig. 1 and Fig. 3, respectively. However, different from some previous datasets for AIGI detection that are based on certain object categories (e.g., ForenSynths (Wang et al., 2020) and GenImage (Zhu et al., 2023)), the images in our dataset contain diverse content including complex scenes with various objects. Therefore, it is difficult to categorize the images and provide reliable statistics on the image content. \n\n\n**Q5: Will the dataset be publicly available in the near future?**\n\nYes. As stated in the footnote on Page 3 of the paper, the dataset will be publicly available upon acceptance. More specifically, we will release the generated images, the corresponding prompts, and other metadata such as the denoising steps for diffusion models and the JPEG compression quality.", "claims_and_evidence": "The authors illustrate this phenomenon using a distribution plot (e.g., Fig1), but the relationship between the Accuracy curve and the distribution is not clearly explained. As a result, readers may find it challenging to quickly identify and interpret the correspondence between the accuracy and the underlying distributions of image quality scores.", "methods_and_evaluation_criteria": "The authors specifically collect an AIGI dataset to evaluate image quality and the difficulty of detection, utilizing a human preference model for assessment. Specifically, the authors utilize two pre-trained human preference models and six existing open-source AIGI detectors to support their experimental findings.", "theoretical_claims": "The authors primarily rely on extensive experiments to support their findings, but the study lacks sufficient theoretical analysis.", "experimental_designs_or_analyses": "I believe that while the authors successfully summarize phenomena through histogram distributions, the underlying reasons are not well explained. Additionally, the descriptions accompanying the histograms lack clarity, potentially leading to confusion among readers.", "relation_to_broader_scientific_literature": "This paper reveals that high-quality AI-generated images (AIGIs), as preferred by humans, tend to be easier for existing AIGI detectors to identify. To further investigate this phenomenon, this paper analyzes how text conditions and image features influence the correlation between the average accuracy of detectors and the quality scores predicted by human preference models.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n\n1.The paper is well-written and clearly structured.\n\n2.The authors focus on a meaningful research direction, especially the  detection of AI-generated images.\n\n3.The experimental validation is comprehensive.\n\nWeaknesses:\n\n1.The data histograms provided by the authors are not sufficiently intuitive. Specifically, the relationship between the distribution and the scores lacks clarity.\n\n2.The experiments primarily discuss observations and phenomena but lack a deeper theoretical analysis or exploration of underlying causes.", "other_comments_or_suggestions": "Please refer to the weakness.", "questions_for_authors": "Do the authors provide comprehensive statistics on the dataset construction, such as the quantity and distribution of each category? Additionally, is there a clear indication of whether the dataset will be publicly available in the near future?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o1SJ0al6RW", "reviewer": "Reviewer_ynZn", "summary": "This paper reveals an interesting yet counterintuitive phenomenon, where a higher-quality AI-generated image (AIGI) preferred by humans can be more easier for existing AIGI detectors to detect. The authors then investigate this effect and find that (1) images generated from short prompts and (2) certain image characteristics, such as texture richness, jointly influence both quality scores and detection accuracy. To address this, they propose a novel method to enhance the detection performance of existing patch-based detectors.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: It is recommended to discuss similar research focusing on resolution, such as [1,2].**\n\nThank you for your valuable suggestion. While high resolution is usually an important aspect of \"high-quality\" images in a broad sense, this paper considers a narrower sense of image quality, which is evaluated by human preference models. The generated images studied in this paper generally have the same resolution for each generator, while [1,2] emphasize the discrepancy between low-resolution and high-resolution deepfake images, as well as the influence of super-resolution. The paper [3], as you mentioned, focuses on \"forgery quality\" (i.e., *\"whether the deepfake image is realistic or not\"*), instead of the blurriness or resolution of images, which is closer to our definition of quality. We will include more discussions on these related works and clarify our focus.\n\n**Q2: This paper does not provide a comprehensive discussion on potential broader applications.**\n\nAs you kindly suggested, the methodology of this research could be extended to related tasks such as image manipulation detection, and the results in this paper may benefit the research on improving the generative models or building more robust detectors. We will add these discussions to our revision.\n\n\n**Q3: The paper is missing evaluations with more model architectures, such as CLIP, ViT (ImageNet), and CNNs trained from scratch.**\n\nWe agree that evaluation with diverse model architectures is important for reaching a reliable conclusion. The experiments in Sec. 5 already included detectors with different architectures, such as CLIP (RINE, DRCT), ImageNet-pretrained ViT (CoDE), and ResNet trained from scratch (NPR). Besides, we add more evaluations of the quality-accuracy correlation on generators of different architectures (please refer to Q4 of Reviewer rmyb).\n\n**Q4: The proposed method to address the resolution issue is somewhat limited, focusing primarily on patch-based detectors.**\n\nWe acknowledge that the proposed patch selection strategies are restricted to patch-based detectors. However, other kinds of detection methods could also benefit from the results of this paper. For example, our regression models obtained in Sec. 4.3 could be applied to identify the hard samples for detection, and emphasizing these samples in model training may enhance the generalization.\n\n\n**Q5: The paper could benefit from using more recent and comprehensive datasets.**\n\nThank you for your advice. In our response to Q4 of Reviewer rmyb, we validate our main conclusions on recent datasets of real-world generated images from commercial models. As for the experiments in Sec. 5, DRCT-2M is the latest published comprehensive benchmark for AIGI detection to the best of our knowledge.\n\n\n**Q6: Since resolution is strongly related to the frequency domain, could analyzing from the frequency domain provide some new insights?**\n\nCertain fingerprints of fake images such as the up-sampling traces utilized by NPR (Tan et al., 2024) may be witnessed in the frequency domain. Nonetheless, our preliminary experiments suggest that frequency domain analysis may not bring meaningful insights into the image quality we focus on. \n\n**Q7: Would high-quality images, such as super-resolution images, be easier to detect?**\n\nThank you for raising this valuable question. Since generative super-resolution with diffusion models is popular in real-world applications, we collect 1000 fake images generated by an SDXL variant without or with generative super-resolution ($1024\\to1536$). We test the detectors on these images, and the results below suggest that whether the super-resolution images are easier or harder to detect depends on the detector. We believe that this question is worthy of further study.\n\n|Generative super-resolution|CoDE|DRCT-ConvB|DRCT-CLIP|NPR|RINE|SuSy|avg|\n|---|---|---|---|---|---|---|---|\n|Without|76.0|74.6|51.4|100.0|59.7|78.8|73.4|\n|With|57.3|88.3|65.2|99.9|27.5|75.4|68.9|\n\n**Q8: Why are high-quality images easier to detect, given that both high-quality and low-quality images contain traces of DNN generation models?**\n\nAccording to our analyses in Sec. 4, existing detectors are sensitive to certain features that correlate with high quality scores, such as high contrast and high texture richness, which means the traces of generation may be more prominent in such images for the detectors.\n\n**Q9: How can we alleviate the overfitting of detection models to the forgery traces in high-quality images and reduce the performance gap between different quality levels?**\n\nWe may collect more diverse and balanced data in terms of image quality. Moreover, designing models that are more capable of detecting higher-level artifacts like the distortion of object structures could reduce the performance gap, as such artifacts are common in low-quality images.", "claims_and_evidence": "The authors have provided reasonable evidence to support their findings, and most claims proposed in the introduction section are well-supported by proper visualizations and references.", "methods_and_evaluation_criteria": "The benchmark datasets and evaluation criteria used are widely accepted in the field.", "theoretical_claims": "N/A.", "experimental_designs_or_analyses": "I have checked the soundness of the proposed experimental analyses, which all make sense to me.", "relation_to_broader_scientific_literature": "I believe the key contributions of this paper should not be limited to the detection of entire-image synthesis alone.\n\n- Broader Application in Deepfake Detection: Similar phenomena have been observed in the detection of face-swapping content [1], where it has been noted that a resolution gap can lead to model shortcuts and overfitting, thereby limiting generalization. The findings of this paper have the potential to be applied in other related fields, such as deepfake detection, not just the detection of text-to-image generation content.\n- Enhancing the Generation Process: Beyond improving detection, this technique can also benefit the generation process. The detectors can serve as critic models or reward models, helping to generate more realistic and undetectable AI-generated images (AIGIs). This dual application of the paper's findings could have significant implications and connections to other scientific literature.\n\n[1] DF40: Toward Next-Generation Deepfake Detection, NeurIPS 2024.", "essential_references_not_discussed": "I recommend that the authors discuss similar research focusing on resolution, such as [1] and [2]. It would be valuable to provide a detailed comparison with these studies, highlighting the unique contributions of this paper to the field. This will help readers understand the distinct advancements and insights offered by this work.\n\n\n[1] DF40: Toward Next-Generation Deepfake Detection, NeurIPS 2024; [2] Exploring Strengths and Weaknesses of Super-Resolution Attack in Deepfake Detection, ArXiv 2024; [3] A Quality-Centric Framework for Generic Deepfake Detection, ArXiv 2024.", "other_strengths_and_weaknesses": "Other Strengths:\n- The originality of this paper is quite high and brings new insights to the field. I believe these findings actively and positively contribute to the entire field.\n- The paper is well-written and very easy to follow. Most claims are well-supported by suitable evidence.\n\nOther Weaknesses:\n- While the findings have the potential to be applied to a broader range of fields, including face-swapping, face-reenactment, face-editing, and realistic image generation, the authors do not provide a comprehensive discussion on these potential applications.\n- The paper is missing evaluations with more model architectures, which are necessary to verify the generality of the findings. Additional models such as CLIP, ViT (ImageNet), and CNNs trained from scratch should be included.\n- The proposed method to address the resolution issue is somewhat limited, focusing primarily on patch-based detectors. There is potential for broader applicability beyond these specific detectors, which is a minor but notable weakness.\n- The paper could benefit from using more recent and comprehensive datasets to make its evaluations more solid and robust.", "other_comments_or_suggestions": "I don't have other comments in this section. Please see the question section and weakness section.", "questions_for_authors": "- Frequency Domain Analysis: Since resolution is strongly related to the frequency domain, could analyzing from the frequency domain provide some new insights?\n- Detection of High-Quality Images: Would high-quality images, such as super-resolution images, be easier to detect? What are the reasons behind this? Could you provide further analysis? For instance, if face-swapping is performed and then followed by super-resolution, would this make the detection easier?\n- Traces in High-Quality and Low-Quality Images: Theoretically, both high-quality and low-quality images contain traces of DNN generation models. Why are high-quality images easier to detect?\n- Mitigating Overfitting on High-Quality Images: How can we alleviate the overfitting of detection models to the forgery traces in high-quality images and reduce the performance gap between different quality levels?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZbYvtMh3h9", "reviewer": "Reviewer_auXT", "summary": "This paper investigates whether high-quality AI-generated images (AIGIs), as preferred by human perception models, are more difficult for detection models to distinguish from real images. Contrary to intuition, the authors find that images with higher human preference scores tend to be easier to detect by existing AIGI detectors.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: How do negative prompts and modifiers influence image characteristics and quality?**\n\nFirstly, we provide an ablation study, comparing the average quality scores of SDXL images with and without negative prompts and positive modifiers. The table below suggests that **prompt engineering improves the average quality of generated images**.\n\n|Negative prompts & positive modifiers|ImageReward|HPSv2|MPS|\n|---|---|---|---|\n|Without|0.0092|0.2452|12.14|\n|With|0.1172|0.2602|12.54|\n\nSince the modifiers are randomly sampled for each image, we compute the average accuracy and quality scores with and without each modifier for SD 3 images. The table below shows that none of these modifiers alone has significant and consistent effects on image quality or accuracy. We argue that the specific effect of modifiers may depend on the prompt and the synergy between modifiers.\n\n|Metric|Modifier|HDR|best quality|dynamic lighting|hyper detailed|photorealistic|professional lighting|ultra highres|ultra realistic|\n|---|---|---|---|---|---|---|---|---|---|\n|Accuracy (%)|Without|56.5|57.2|56.7|56.6|56.7|56.7|57.3|56.5|\n| |With|56.8|56.1|56.5|56.6|56.5|56.5|56.0|56.7|\n|ImageReward|Without|0.6123|0.6265|0.6515|0.6268|0.6076|0.6395|0.6245|0.6339|\n| |With|0.6429|0.6285|0.6035|0.6282|0.6476|0.615|0.6304|0.6212|\n|HPSv2|Without|0.2655|0.2662|0.2661|0.2648|0.2640|0.2657|0.2652|0.2649|\n| |With|0.2660|0.2652|0.2654|0.2666|0.2675|0.2658|0.2662|0.2665|\n|MPS|Without|12.93|13.00|13.00|12.99|12.95|13.12|12.93|13.04|\n| |With|13.13|13.06|13.06|13.07|13.11|12.94|13.13|13.02|\n\n**Q2: Test on closed-source models like MidJourney and DALL·E 3.**\n\nPlease refer to Q4 of Reviewer rmyb.\n\n**Q3: Discussion on failure cases (e.g., when high-quality images are misclassified).**\n\nWe take the high-quality images with top 30% ImageReward and HPS v2 scores, and regard the samples with at most 2 correct predictions for the 6 detectors (#Correct<=2) as failure cases, comparing them with the cases of #Correct>=5. We find that the failure cases consistently have lower average contrast and saturation, which is consistent with our conclusions in Sec. 3.\n\n|Metric|#Correct|SD 2.1|SD 3|SDXL 1.0|PA-alpha|FLUX.1 [dev]|Infinity|Average|\n|---|---|---|---|---|---|---|---|---|\n|contrast|>=5|66.19|67.46|53.72|63.96|60.44|58.86|61.77|\n| |<=2|51.11|61.58|49.60|45.72|52.67|51.62|52.05|\n|saturation|>=5|0.4343|0.4604|0.4551|0.5400|0.4817|0.4714|0.4738|\n| |<=2|0.2478|0.3553|0.4134|0.4043|0.4079|0.2982|0.3545|\n\n\n**Q4: Consider testing adversarial modifications to verify detector robustness.**\n\nTable 1(a) of the paper indicates certain adversarial directions for image modification, such as lowering the lightness/contrast/saturation, or increasing the sharpness. We implement these manipulations with different factors (0.5 means decreasing by 50%; 1.5 means increasing by 50%). The results below suggest that the adversarial modifications are effective for DRCT and RINE, while CoDE and SuSy are less sensitive to such modifications. Interestingly, NPR shows stronger performance on any manipulated data.\n\n|Manipulation|Factor|DRCT-ConvB|DRCT-CLIP|RINE|CoDE|SuSy|NPR|Avg|\n|---|---|---|---|---|---|---|---|---|\n|none|N/A|65.6|83.1|31.8|66.7|83.8|66.8|66.3|\n|lightness|0.5|51.8|55.8|20.9|68.0|84.9|89.2|61.8|\n| |1.5|71.7|93.6|77.8|69.7|79.4|82.0|79.0|\n|contrast|0.5|44.4|76.6|15.7|72.9|82.6|89.1|63.6|\n| |1.5|76.3|93.3|84.1|69.5|85.5|87.3|82.7|\n|saturation|0.5|63.2|84.3|35.8|64.9|84.7|64.2|66.2|\n| |1.5|67.6|90.9|56.7|65.1|82.7|68.5|71.9|\n|sharpness|0.5|76.6|85.3|34.0|71.2|84.9|89.5|73.6|\n| |1.5|55.0|81.7|52.2|62.5|80.6|73.4|67.6|\n\n**Q5: Are there specific image transformations that can fool existing AIGI detectors while maintaining a high human preference score?**\n\nTo the best of our knowledge, there may not be simple transformations with these properties. However, one may consider replacing the perceptual constraint in the perceptual adversarial attack [a] with the preference score constraint and implement an effective attack for this purpose.\n\n[a] Perceptual Adversarial Robustness: Defense Against Unseen Threat Models. ICLR 2021.  \n\n**Q6: Would a multi-task learning approach (joint training on quality scoring & detection) further improve detector performance?**\n\nWe believe that this is possible as training on quality scoring could encourage the detector to learn on more quality-related features, especially those of higher levels, which could make the detector more robust to the distribution shift of low-level features in the cross-generator scenario.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The paper presents no theoretical claims.", "experimental_designs_or_analyses": "The experiments are well-designed and include appropriate baselines. Key strengths include: \n\n1) Diverse dataset: Incorporates a variety of prompts, generators, and preference models; \n\n2) Multiple detectors: Ensures robustness of findings across different architectures.\n\nHowever, there are some potential concerns: \n\n1) More details on dataset filtering would help: While the paper describes dataset construction, additional insights into how negative prompts and modifiers influence quality could strengthen the argument. \n\n2) Further generalization studies: The study could benefit from testing whether similar findings hold for closed-source models like MidJourney and DALL·E 3.", "relation_to_broader_scientific_literature": "Overall, the work fills a potential research gap by linking human preference-based quality scoring with AIGI detectability.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "1) Limited discussion on failure cases (e.g., when high-quality images are misclassified).\n\nWhile the paper convincingly demonstrates that high-quality images tend to be more detectable, it does not provide an in-depth analysis of failure cases, i.e., scenarios where high-quality AI-generated images are misclassified as real or where lower-quality images fool the detectors.\n\nAre there specific types of high-quality images (e.g., those with certain color distributions, structural complexity, or semantic attributes) that remain difficult to detect despite scoring highly on human preference models?\n\n2) Evaluation on additional closed-source generators could enhance impact.\n\nThe study primarily focuses on open-source text-to-image models (e.g., Stable Diffusion variants, PixArt-α). However, many of the most widely used real-world AI-generated images come from closed-source models such as MidJourney, DALL·E 3, and Runway Gen-2.", "other_comments_or_suggestions": "1) Clarify how \"negative prompts\" impact image quality in dataset creation.\n\n The dataset includes negative prompts during generation to improve output quality, but it is unclear how these prompts influence image characteristics and whether they introduce biases in the evaluation.\n\n Do negative prompts primarily reduce artifacts, improve semantic fidelity, or affect low-level features like texture richness and contrast?\n\nCould certain types of negative prompts inadvertently make images more detectable by standardizing certain visual properties (e.g., smoothing textures, removing unnatural edges)?\n\n2) Consider testing adversarial modifications to verify detector robustness.\n\nThe study demonstrates that certain image characteristics (e.g., contrast, saturation, texture richness) correlate with detectability. However, adversarial image manipulations (e.g., adjusting contrast, adding noise, altering texture patterns) could be used to challenge these conclusions.\n\nTesting adversarial perturbations (e.g., reducing texture richness in high-quality images) could reveal whether detectors are relying on spurious correlations or genuinely robust visual cues.\n\n Are there specific image transformations that can fool existing AIGI detectors while maintaining a high human preference score?", "questions_for_authors": "1) How do adversarial perturbations (e.g., reducing texture richness) affect detection accuracy?\n\n2) Would a multi-task learning approach (joint training on quality scoring & detection) further improve detector performance?\n\n3) Have you tested whether closed-source generators (e.g., MidJourney, DALL·E 3) exhibit the same quality-detectability trends?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received 5 weak accept ratings. 

Before rebuttal,

Reviewers suggested the following strength of the paper: 1) paper is very interesting (reviewer rmyb); 2) paper is well-written and clearly structured (reviewer 581z, ynZn); 3) experimental validation is comprehensive (reviewer 581z, auXT); 4) The originality of this paper is quite high and brings new insights to the field (reviewer ynZn). The weakness of the paper are: 1) some conflicts in the paper (Reviewer gsPf, rmyb); 2) several of its claims lack clear substantiation (Reviewer rmyb);  3) lack a deeper theoretical analysis or exploration of underlying causes (Reviewer 581z); 4) The paper is missing evaluations with more model architectures (Reviewer ynZn); 5) The proposed method to address the resolution issue is somewhat limited, focusing primarily on patch-based detectors (Reviewer ynZn); 6) The paper could benefit from using more recent and comprehensive datasets to make its evaluations more solid and robust (Reviewer ynZn); 7) More details on dataset filtering would help (Reviewer auXT); 8) Further generalization studies (Reviewer auXT); 9) Limited discussion on failure cases (Reviewer auXT); 10) Evaluation on additional closed-source generators could enhance impact(Reviewer auXT). 

After rebuttal, all reviewers confirmed that they have read the paper and would like to update the review if needed. Reviewer gsPf thought   
authors addressed their concerns and increased the rating. Reviewer rmyb didn't provide additional comment and kept the rating. Reviewer 581z maintained their rating. Reviewer ynZn thought authors addressed most of their concerns and maintained the rating. Reviewer auXT also kept their rating. 

Given these AC decided to give weak accept rating to this paper.
Decision: Accept (poster)