Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?

## Abstract

Abstract The ultimate goal of generative models is to perfectly capture the data distribution. For image generation, common metrics of visual quality (e.g., FID) and the perceived truthfulness of generated images seem to suggest that we are nearing this goal. However, through distribution classification tasks, we reveal that, from the perspective of neural network-based classifiers, even advanced diffusion models are still far from this goal. Specifically, classifiers are able to consistently and effortlessly distinguish real images from generated ones across various settings. Moreover, we uncover an intriguing discrepancy: classifiers can easily differentiate between diffusion models with comparable performance (e.g., U-ViT-H vs. DiT-XL), but struggle to distinguish between models within the same family but of different scales (e.g., EDM2-XS vs. EDM2-XXL). Our methodology carries several important implications. First, it naturally serves as a diagnostic tool for diffusion models by analyzing specific features of generated data. Second, it sheds light on the model autophagy disorder and offers insights into the use of generated data: augmenting real data with generated data is more effective than replacing it.

## 1 Introduction

Diffusion probabilistic models (Sohl-Dickstein et al., ; Song et al., ; Ho et al., ) have emerged as leading generative models for image (Ramesh et al., ; Esser et al., ), video (Brooks et al., ; Bao et al., ; Zhao et al., ) and 3D content generation (Poole et al., ), surpassing previous generative models such as generative adversarial networks (Goodfellow et al., ), variational autoencoders (Kingma & Welling, ), and normalizing flows (Rezende & Mohamed, ). In particular, focusing on image generation, diffusion models are capable of producing realistic images that are often indistinguishable from real ones to the human eye (see Fig. 2). Furthermore, common metrics such as FID (Heusel et al., ) suggest that the distribution generated by state-of-the-art diffusion models closely approximates the ImageNet validation set (Karras et al., ; ; Tian et al., ). Therefore some works, such as Wang et al. (); You et al. (), have utilized generated data to augment training datasets. However, to the best of our knowledge, few studies have been conducted to analyze the differences between the generated and real distributions.

This raises a fundamental question: How far is the generated distribution from the real distribution? Inspired by the methodology in the work (Liu & He, ; Tommasi et al., ) on dataset bias (Torralba & Efros, ), we propose distribution classification tasks (see Sec. 3 for details),
predicting the distribution an image belongs to - real or generated -
thereby serving as a probe to explore this critical question. Notably, binary classification is a natural approach, as classifiers parameterized by neural networks can serve as effective indicators of the differences between the real and generated distributions through classification accuracy (see more analysis in Appendix  D). Compared to the widely used FID metric, our classifier-based approach directly measures distribution differences without relying on Gaussian assumptions (see further discussion in Appendix E).

We first explore are images indistinguishable to humans also indistinguishable to classifiers? To this end, we train several commonly used classifiers to distinguish between the real and generated distribution across various settings. Quantitatively, most classifiers can achieve over 98% accuracy. Meanwhile, all classifiers reach extremely high accuracy with significantly fewer training samples than the requirement of training diffusion models. Interestingly, even self-supervised classifiers exhibit this capacity to identify the generated distribution. These results demonstrate that from the perspective of neural network-based classifiers, the generated distribution remains markedly different from the real one, indicating that diffusion models still have significant room for improvement.

Figure: Figure 1: Four-way distribution classification tasks: *Which one is real in each row?*(^2^22From left to right, the first two rows show images from U-ViT, ImageNet (real), DiT, and EDM2-XXL. The bottom row shows images from Pixart-$\alpha$, Playground-v2.5, COCO (real), and SDXL.)The samples are from real images or generated from state-of-the-art diffusion models. Notably, classifiers consistently and effortlessly distinguish between real and generated images in all settings.
Refer to caption: extracted/5918590/picture/intro.png

Furthermore, we observe intriguing contradictions between classifiers’ prediction and both FID and human judgments through two groups of experiments. The first group involves classifying the smallest and largest models within the same diffusion family, which share inductive biases but differ in visual quality. The second group examines diffusion models with similar FID scores and visual quality but different inductive biases. Surprisingly, classifiers perform well on the second task but struggle with the first, revealing contradictions between classifier performance and FID scores. Meanwhile, human judgments align with classifiers in the first task but degrade to random guessing in the second, indicating inconsistency between classifier and human judgments.

Our approach displays several important implications. First, it naturally serves as a diagnostic tool for diffusion models by extending the analysis to specific features of generated data. For example, in evaluating the advanced diffusion transformer U-ViT-H/2 (Bao et al., ), we focus on two key aspects: spatial features and frequency ranges. Spatially, classifiers can easily distinguish whether an image is real or generated using any part of the image. In terms of frequency, classifiers struggle with minimal low-frequency information but perform well across all other frequency intervals. This suggests that U-ViT-H/2 has effectively learned low-frequency features, making generated images harder to distinguish in these regions,
but remains less proficient in higher frequency regions.
Second, our approach enables the evaluation of using generated data for training models. It provides a reasonable explanation for the model autophagy disorder (MAD) phenomenon observed in Shumailov et al. (); Alemohammad et al. (). This collapse is surprising because, despite the realistic appearance of the generated data, using it in a continuous loop to train subsequent generations of models leads to failure. We explain this by showing that there exists a distribution mismatch between the training and generated distributions in each generation. As this mismatch accumulates, the generated data distribution drifts further from the original. In downstream tasks, such as supervised and semi-supervised classification, this mismatch leads to performance degradation when real data is simply replaced by generated data.
However, this distribution drift can be mitigated by *augmenting* real data with generated data, which introduces novel features while maintaining alignment with the original data distribution, leading to improved performance.

Briefly, our key contributions are as follows:

- •
We show that classifiers easily distinguish between diffusion-generated and real distributions, despite diffusion models achieving low FID scores and generating lifelike images (see Fig. 2).
- •
We reveal the intriguing contradictions between classifier performance and widely used evaluation metrics, such as FID and human judgments.
- •
We demonstrate that our approach complements traditional metrics like FID and can serve as a diagnostic tool to provide deeper insights into diffusion models.
- •
We provide a reasonable explanation for the surprising model autophagy disorder phenomenon and show that augmenting real data with generated data is more effective than replacing it in supervised and semi-supervised learning.

## 2 Background

### 2.1 Diffusion model

Diffusion models (Sohl-Dickstein et al., ; Song et al., ; Ho et al., ) gradually inject noise into data $\bm{x}$ during the forward process:

$$ $\displaystyle\bm{z}_{t}=\sqrt{\bar{\alpha}_{t}}\bm{x}+\sqrt{1-\bar{\alpha}_{t} }\bm{\epsilon},$ (1) $$

and remove noise to generate data in the reverse process. Diffusion models typically use a noise prediction network $\bm{\epsilon}_{\bm{\theta}}(\bm{z}_{t},t)$ to predict the noise $\bm{\epsilon}$ added to $\bm{z}_{t}$. Noise prediction loss is defined as:

$$ $\displaystyle\mathcal{L}=\mathbb{E}_{t,\bm{x}_{0},\bm{\epsilon}}[\|\bm{ \epsilon}_{\bm{\theta}}(\bm{z}_{t},t)-\bm{\epsilon}\|_{2}^{2}].$ (2) $$

### 2.2 Frequency filter

The frequency content of an image represents the rate of pixel value changes. Low-frequency components capture overall shapes and gradual grayscale changes, while high-frequency components reflect fine details like edges and textures. Notably, convolutional neural networks can detect high-frequencies that are often imperceptible to humans (Wang et al., ). In our paper, we implement a rectangular filter and discuss the choice between using rectangular or circular filters in Appendix F.

Low-pass filters are implemented as follows: Compute the Fourier transform of the image $f(x,y)$ using $F(u,v)=\text{FFT}(f(x,y))$, and then center the spectrum with $F_{c}(u,v)=\text{fftshift}(F(u,v))$. Define a rectangular mask $H(u,v)$ in the frequency domain, where $M$ and $N$ are the image dimensions:

$$ $\displaystyle H(u,v)=\begin{cases}1&\text{if }\left|u-\frac{M}{2}\right|\leq \text{threshold}\text{ and }\left|v-\frac{N}{2}\right|\leq\text{threshold},\\ 0&\text{otherwise}.\end{cases}$ (3) $$

Apply the mask by multiplying $G(u,v)=F_{c}(u,v)\odot H(u,v)$, reverse the shift with $G_{c}(u,v)=\text{ifftshift}(G(u,v))$, and transform back to the spatial domain using $g(x,y)=\text{IFFT}(G_{c}(u,v))$.

High-pass filters are implemented similarly, but with the mask defined to pass high frequencies:

$$ $\displaystyle H(u,v)=\begin{cases}0&\text{if }\left|u-\frac{M}{2}\right|\leq \text{threshold}\text{ and }\left|v-\frac{N}{2}\right|\leq\text{threshold},\\ 1&\text{otherwise}.\end{cases}$ (4) $$

Band-pass filters combine low-pass and high-pass filters, allowing frequencies between low and high thresholds to pass while setting others to 0.

## 3 Related work

Dataset bias. Torralba & Efros () introduced the "Name That Dataset" game to discover dataset bias. Tommasi et al. () and Liu & He () expanded on this by applying convolutional neural networks and large-scale datasets, respectively. Existing work on dataset bias focuses on classification to highlight biases in data collection for building unbiased datasets, our work leverages distribution classification to explore a fundamental question: *How far is the generated distribution from the real distribution?* Besides, we focus on the discrepancy between the generated distribution and the real distribution rather than between specific datasets. Furthermore, the conclusion drawn by related work highlights the ability of neural networks to detect biases within increasingly general datasets, our conclusion underscores the significant gap that still exists between current diffusion models and the real data distribution. We also highlight the potential of distribution classification methods for gaining deeper insights into generative models and more effectively leveraging generated data.

Generated image detection. Several empirical works on generated image detection span three key areas: universal detection (Liu et al., ; Ojha et al., ; Wang et al., ; Girish et al., ), reconstruction-based methods (Wang et al., ; Luo et al., ), and frequency analysis (Wang et al., ; Zhang et al., ; Durall et al., ; Chandrasegaran et al., ; Frank et al., ; Dzanic et al., ; Yang et al., ; Corvi et al., ; Ricker et al., ; Corvi et al., ). Existing work generally focuses on either building universal discriminators to detect generated images or identifying frequency artifacts to build classifiers. In contrast, our approach leverages distribution classification to gain deeper insights into generative models and quantify the discrepancy between generated and real distributions. Instead of prioritizing classifier performance, we focus on the insights provided into generative model behavior.

## 4 Experimental settings

We present the main experimental settings as follows. For more details, please see Appendix A.

Dataset. For label-to-image tasks, we use the CIFAR-10 (Krizhevsky et al., ) and ImageNet (Deng et al., ) datasets, and for text-to-image tasks, we utilize the COCO2014 dataset (Lin et al., ). For CIFAR-10, the real distribution is constructed from its training and testing sets, comprising 50k training images and 10k testing images. The generated distribution is created using diffusion models to produce an equivalent number of images. For ImageNet, we consider resolutions of 256 and 512 and follow ADM’s (Dhariwal & Nichol, ) method to preprocess ImageNet into the desired resolutions. We randomly sample 100k training images and 50k validation images from the corresponding ImageNet sets to represent the real distribution, and generate an equivalent number of images using diffusion models for the generated distribution. For COCO2014, we randomly sample 10k training images and 1k validation images from the respective sets to form the real distribution, and use the corresponding prompts to generate an equivalent number of data for the generated distribution.

Classifier. By default, we use ResNet-50 (He et al., ) as the classifier architecture, and also consider ConvNeXt-T (Liu et al., ) and ViT-S (Dosovitskiy et al., ) for completeness. Our pre-processing follows standard supervised training (Liu et al., ). Specifically, during training, the classifiers process randomly augmented crops of $224\times 224$ images. For validation, images are resized to 256 pixels on the shorter side, preserving the aspect ratio, and then center cropped to $224\times 224$. For CIFAR-10 experiments, we initialize the classifier with ResNet-50 pre-trained on ImageNet and also provide the results trained from scratch (see Appendix C.2). For ImageNet, we train ResNet-50 from scratch.

Diffusion model. For CIFAR-10 generation, we consider two diffusion models: EDM (Karras et al., ) and U-ViT (Bao et al., ). For ImageNet-256, we explore three diffusion models: EDM2 (Karras et al., ), U-ViT-H/2 (Bao et al., ), and DiT-XL/2 (Peebles & Xie, ). Since EDM2 is originally designed for ImageNet-512 generation, we resize its generated images from 512 to 256 resolution, with EDM2-XXL achieving an FID of 2.14, comparable to U-ViT-H/2 and DiT-XL/2. For ImageNet-512 generation, we use EDM2-XXL (Karras et al., ). For COCO generation, we consider three text-to-image diffusion models: Pixart-$\alpha$ (Chen et al., ), SDXL (Podell et al., ), and Playground-v2.5 (Li et al., ).

Evaluation. We use the top-1 accuracy on the validation set to evaluate classification performance.

Abbreviations. In the figures and tables following this paper, the following abbreviations are used: CIFAR-10 (*C*), ImageNet (*I*), U-ViT (*U*), EDM (*E*), DiT-XL/2 (*D*).

## 5 Generated distributions are easily classified as generated

In this section, we explore how the distribution generated by diffusion models, known for their success in image generation, differs from the real distribution. To analyze this, we use a distribution classification task to assess the extent of the difference.

**Table 1: Binary distribution classification on *the same distribution*. Classifiers are indeed unable to distinguish samples from the same distribution, initially demonstrating the validity of our approach.**
| Distribution1 | Distribution2 | Classifier | Accuracy (%) |
| --- | --- | --- | --- |
| U (Bao et al., ) | U | ResNet-50 | 50.83 |
| ViT-S | 50.57 |  |  |
| ConvNeXt-T | 50.18 |  |  |
| E (Karras et al., ) | E | ResNet-50 | 50.67 |
| ViT-S | 50.44 |  |  |
| ConvNeXt-T | 50.14 |  |  |

### 5.1 Classifiers cannot distinguish samples from the same distribution

First, it is crucial to evaluate whether classifiers are indeed unable to distinguish samples from the same distribution. As shown in Tab. 1, we report the classification accuracy for the same distributions in the label-to-image scenario. Our findings indicate that classifiers are unable to distinguish samples from the same distribution, initially demonstrating the validity of the distribution classification task.

### 5.2 Classifiers achieve high accuracy across various settings

Label-to-image. As shown in Tab. 2, we report the binary distribution classification accuracy for various combinations of real and generated distributions in the label-to-image scenario. Surprisingly, despite the low FID scores achieved by diffusion models, neural network-based classifiers still identify significant differences between the generated and real distributions. Specifically, all classifiers easily distinguish real from generated images across different dataset combinations, with most achieving over 98% accuracy in distribution classification. Moreover, we observe a positive correlation between distribution classification accuracy and the number of training samples for the combination of U-ViT-H/2 and ImageNet-256 (see Fig. 2). This suggests that as sample size increases, classifiers learn more generalizable features rather than merely memorizing the data. Notably, with only 50k samples per distribution (less than 10% of the ImageNet samples, approximately 1.28 million, used to train the generator), classifiers achieve over 99.5% accuracy. Besides, we extend our evaluation to other generative models, e.g., GANs (see detailed results in Sec. C.1).

**Table 2: Binary distribution classification on *label-to-image*. All classifiers yield high accuracy on various combinations of datasets and generative models. FIDs are taken from related references.**
| Real dataset | Generative model | Corrresponding FID | Classifier | Accuracy (%) |
| --- | --- | --- | --- | --- |
| *C* | U (Bao et al., ) | 3.11 | ResNet-50 | 99.92 |
| ViT-S | 98.04 |  |  |  |
| ConvNeXt-T | 99.96 |  |  |  |
| *C* | E (Karras et al., ) | 1.79 | ResNet-50 | 96.25 |
| ViT-S | 89.38 |  |  |  |
| ConvNeXt-T | 98.43 |  |  |  |
| *I*-256 | *U*-H/2 (Bao et al., ) | 2.29 | ResNet-50 | 99.95 |
| ViT-S | 98.06 |  |  |  |
| ConvNeXt-T | 99.89 |  |  |  |
| *I*-512 | *E*2-XXL (Karras et al., ) | 1.81 | ResNet-50 | 99.73 |
| ViT-S | 95.15 |  |  |  |
| ConvNeXt-T | 86.44 |  |  |  |

Figure: Table 3: Four-way distribution classification on *text-to-image*. All classifiers yield high accuracy in distinguishing four distributions: COCO (Lin et al., ), Pixart-$\alpha$ (Chen et al., ), SDXL (Podell et al., ), and Playground-v2.5 (Li et al., ), using only 5k or 10k training samples per dataset. Notably, as the number of training samples increases, the accuracy of distribution classification consistently improves.
Refer to caption: x1.png

Text-to-image. As shown in Tab. 3, we report the distribution classification accuracy in the text-to-image scenario. We consider combinations of four distributions: COCO (Lin et al., ), Pixart-$\alpha$ (Chen et al., ), SDXL (Podell et al., ), and Playground-v2.5 (Li et al., ).
In this task, classifiers are not only required to distinguish between real and generated distributions but also correctly identify the specific generated distribution an image comes from. Remarkably, classifiers achieve an accuracy of over 76% across all settings. Furthermore, we observe that increasing the training sample size from 5k to 10k results in significant accuracy gains, consistent with the patterns seen in the label-to-image scenario.

### 5.3 Self-supervised classifiers can also identify generated images

In addition to the supervised learning protocol, we also explore self-supervised learning. Specifically, we use self-supervised pre-trained models, such as MAE (He et al., ) and MoCo v3 (Chen et al., ), to extract image features followed by training a linear classifier on these features, referred to as the self-supervised classifier. The distribution classification accuracy of these classifiers is in Tab. 4. Notably, the self-supervised classifiers also demonstrate a strong ability to distinguish between real and generated distributions. Moreover, we observed two intriguing phenomena: First, unlike in semantic classification tasks on ImageNet, the masked image modeling-based MAE outperforms the contrastive learning-based MoCo v3 in identifying generated images related to ImageNet. Second, while self-supervised classifiers achieve relatively high accuracy on ImageNet-related tasks, their performance drops significantly when applied to CIFAR-10.
We hypothesize this decline can be contributed to two factors: (1) MAE and MoCo v3 are pre-trained on ImageNet, not CIFAR-10, and (2) diffusion models perform exceptionally well on CIFAR-10 image generation, making it challenging for a simple linear classifier to differentiate between real and generated images. Besides, we also evaluate self-supervised classifiers in text-to-image scenarios (see detailed results in Sec. C.3).

**Table 4: Self-supervised classifiers demonstrate the ability to distinguish between real and generated distributions. For each self-supervised method, we use ViT-B as the backbone.**
| Real dataset | Generative model | Self-supervised method | Accuracy |
| --- | --- | --- | --- |
| *C* | *U* (Bao et al., ) | MAE | 71.83 |
| MoCo v3 | 73.39 |  |  |
| *C* | *E* (Karras et al., ) | MAE | 64.83 |
| MoCo v3 | 67.69 |  |  |
| *I*-256 | *U*-H/2 (Bao et al., ) | MAE | 91.65 |
| MoCo v3 | 77.38 |  |  |
| *I*-512 | *E*2-XXL (Karras et al., ) | MAE | 81.80 |
| MoCo v3 | 74.30 |  |  |

#### Conclusion 1.

#### Conclusion 1.

## 6 Classifier contradictions with FID and human judgments

Previously, we demonstrated that in most settings, classifiers can effortlessly distinguish between real and generated distributions, indicating that the generated distribution is still far from the real one. In this section, we explore the intriguing contradictions between classifiers and widely used evaluation protocols, such as FID and human judgments, through two sets of experiments. The first experiment compares EDM2-XS and EDM2-XXL, which share the same network architectures and training methodologies but differ in the number of parameters, resulting in FID scores of 2.91 and 1.81, respectively. The second experiment examines DiT-XL, U-ViT-H/2, and EDM2-XXL, which, despite having similar FIDs, vary in network architecture, sampling methods, and training protocols.

Figure: Table 5: Distribution classification accuracy for various combinations of diffusion models with similar FIDs and ImageNet. Notably, classifier accuracy improves as more distributions are added. Here, combinations refer to multiple distributions, where the number of distributions corresponds to the number of classes in distribution classification.
Refer to caption: x2.png

To analyze human judgments, we conducted a user study to evaluate the human ability to distinguish between different distributions (see the user interface in Fig. 10). Further details of the study are provided in Appendix C.6. The results are shown in Fig. 3, where individual variability is represented using error bars, indicating the standard deviation across participants/classifier performance.

In our first set of experiments, we analyze models within the same family: EDM2-XS and EDM2-XXL, the smallest and largest models in the EDM2 family, respectively. As shown in Tab. 6, we report the distribution classification accuracy for distinguishing between EDM2-XS and EDM2-XXL. The results show that classifiers struggle to differentiate between models from the same family. Only ResNet-50 shows some ability to distinguish them, while the other classifiers perform close to random chance. This contrasts sharply with the task of distinguishing between ImageNet-512 and EDM2-XXL (in Tab. 2), where all classifiers achieve over 86% accuracy. Thus, compared to distinguishing between diffusion models and the real distribution, neural networks view models within the same family—due to their shared inductive biases—as learning similar distributions, making them difficult to distinguish. Furthermore, in our user study (see Fig. 3), unlike the other tasks where classifiers significantly outperform humans, in the task of distinguishing between models within the same family, humans perform comparably to the classifiers. This observation highlights a limitation of neural network-based classifiers: despite the noticeable difference in FID scores between EDM2-XS and EDM2-XXL (2.91 and 1.81, respectively), the classifiers struggle to differentiate between them.

In our second group of experiments, we analyze different diffusion models with similar FIDs. As shown in Tab. 5, we present the distribution classification accuracy for three combinations of diffusion models with similar FIDs. We observe that classifiers can easily distinguish between these models, achieving extremely high accuracy, while humans perform close to random guessing (see Fig. 3). This highlights a key advantage of classifiers over widely used evaluation protocols: despite the FID scores being very close and humans being unable to differentiate between the models, classifiers can easily identify the differences. Moreover, an intriguing phenomenon occurs when additional distributions are introduced: the classifier’s accuracy improves further. This contrasts with the findings in Liu & He (). A possible explanation is that all of these distributions are either directly related to ImageNet or generated by diffusion models trained on it. As new distributions are added, the classifier can leverage newly learned patterns to enhance the classification accuracy for previous combinations.

Figure: Table 6: Classifiers struggle to differentiate between the smallest and largest EDM2 models. Despite notable differences in FID (E2-XS: 2.91, E2-XXL:1.81), the classifiers show limited ability to accurately distinguish between these EDM2 variants.

#### Conclusion 2.

#### Conclusion 2.

## 7 Implications of the distribution classification task

Previously, we demonstrated the classifier’s ability to distinguish real from generated distributions, as well as the contradictions between classifiers and evaluation methods like FID and human judgments. In this section, we show some important implications of the distribution classification task.

### 7.1 Diagnosing the problem within diffusion models

A natural implication of classifiers is to diagnose the problem within diffusion models. Our approach complements traditional metrics like FID and human judgments by offering a more challenging evaluation method. It can also be extended to analyze specific features of generated data, providing deeper insights into the intrinsic characteristics of diffusion models, such as spatial and frequency features. In this section, we focus on advanced diffusion transformers, specifically U-ViT-H/2 (Bao et al., ), and investigate two key aspects: spatial features and frequency ranges. Our objective is to identify which spatial features diffusion models learn well, making them harder to distinguish, and which they struggle with, making them easier to detect. Additionally, we assess performance across different frequency ranges to pinpoint where diffusion models succeed and where they fail. For this analysis, we use U-ViT-H/2 trained on ImageNet-256, with ResNet-50 as the classifier.

Figure: (a) Accuracy vs. low-frequency filter threshold
Refer to caption: x3.png

Figure: Figure 5: Visualization of frequency domain processing. *Top*: Low-pass filters, *Middle*: High-pass filters, each processed with increasing thresholds: 10, 30, 50, 100. *Bottom*: Band-pass filters, processed with band thresholds: 0-10, 10-30, 30-50, 50-100.
Refer to caption: extracted/5918590/picture/problem3/frequency_process.png

In our first set of experiments, we focus on the spatial features. We start by progressively reducing central information through center-cropping images from a resolution of 256 down to 128, 64, 32, and 16, followed by training the classifier from scratch. The results, shown in Tab. 7, indicate that even with minimal central information, the classifier can accurately distinguish between real and generated distributions. We then assess whether classifiers can differentiate based on any arbitrary part of the image. We cropped images at resolutions of 128 and 64, and randomly selected four sections per image for training (see Fig. 7 for visualizations). The classifiers achieved accuracies of $99.97\pm 0.004$% for 128 and $99.97\pm 0.007$% for 64 (see Tab. 7). These results demonstrate that even with the advanced diffusion transformer U-ViT-H/2, which achieves an FID of 2.29, classifiers can reliably distinguish between real and generated images using any spatial part of the image.

In our second set of experiments, we focus on the frequency aspect. We preprocess images using low-pass, high-pass, and band-pass filters as described in Sec. 2.2. For the low-pass and high-pass filters, we apply thresholds of 10, 30, 50, and 100, while for the band-pass filter, we use frequency intervals of 0-10, 10-30, 30-50, and 50-100. Fig. 5 provides examples of the original and filtered images. As shown in Fig. 4, using ResNet-50 as the classifier, we observe that classifiers struggle to maintain high accuracy when low-frequency information is minimal but perform well across all other frequency intervals, even with only high-frequency components (e.g., threshold of 100). These results suggest that diffusion models effectively learn low-frequency features, making it more difficult for classifiers to distinguish real from generated images in this range. However, in higher frequency bands, diffusion models perform worse, making it easier for classifiers to identify generated images.

### 7.2 Evaluating the use of generated data

Another natural implication is using classifiers to evaluate the use of generated data. Our previous findings show a significant difference between generated and real distributions, raising the question of whether this mismatch causes undesirable behaviors in models trained on synthetic data. One well-known phenomenon is that training generative models on data produced by previous generations in a continuous loop leads to a gradual decline in output quality and diversity (Shumailov et al., ; Alemohammad et al., ), ultimately resulting in model collapse, known as Model Autophagy Disorder (MAD). This is surprising, as data generated by modern diffusion models often exhibit high realism. However, using this data to train next-generation models triggers MAD. Our approach provides insight into this issue: although these models generate realistic images, the distributions they produce remain distinct from the real distribution—especially when viewed by classifiers—leading to a distribution mismatch. As this mismatch accumulates, the generated distribution drifts further from the original. To address this, Alemohammad et al. () propose incorporating fresh real samples into the training data for each generation, rather than relying solely on generated data.

To better understand this phenomenon, we further explore the impact of using generative model data in downstream tasks. Specifically, we investigate whether the distribution mismatch between real and generated data causes undesirable behaviors when replacing real data with generated data in tasks such as classification. To analyze this, we use EDM (Karras et al., ) to generate the same number of samples as CIFAR-10, creating a synthetic dataset, EDM_S, and perform both supervised and semi-supervised classification to assess the impact. For supervised classification, we train the classifier with ConvNeXt settings (Liu et al., ) for CIFAR-10, using the original test set for evaluation. To create the training dataset, we mix EDM_S and CIFAR-10 with a ratio $\alpha$. We employ two strategies: one replaces $\alpha$ of CIFAR-10 with EDM_S, where $\alpha$ represents the replacement ratio, and the other augments EDM_S to CIFAR-10. As shown in Fig. 6, we observe that due to the distribution mismatch, simply replacing samples leads to a decline in accuracy. As $\alpha$ increases from 0 to 1, the top-1 accuracy gradually declines. In contrast, when augmenting EDM_S to CIFAR-10, accuracy generally improves for most $\alpha$ values. For semi-supervised classification, we use FreeMatch (Wang et al., ) and FlexMatch (Zhang et al., ). As shown in Tab. 8, simply replacing CIFAR-10 with EDM_S reduces performance: FlexMatch’s error rate increases from 4.97$\pm$0.06% to 5.85$\pm$0.02%, and FreeMatch’s from 4.90$\pm$0.04% to 5.94$\pm$0.10%, consistent with the results in supervised classification. Meanwhile, DPT (You et al., ) augments CIFAR-10 with EDM-generated data, training with FreeMatch, and achieves a lower error rate of 4.68$\pm$0.17%.

From the analysis above, we observe that the distribution mismatch between generated and real data leads to undesirable behaviors, such as "MAD" or performance degradation, when real data is simply replaced with generated data. However, augmenting real data with generated data is more effective.

Figure: Figure 6: Top-1 accuracy on the CIFAR-10 test set: Replacement vs. Augmentation. $\alpha$ represents the ratio of EDM_S mixed with CIFAR-10.
Refer to caption: x6.png

## 8 Conclusion

Our paper proposes the distribution classification tasks to explore a fundamental question: How far is the generated distribution from the real distribution? Our findings show that classifiers can effectively distinguish between diffusion-generated and real distributions, even when diffusion models achieve low FID scores and generate lifelike images. Additionally, our study shows contradictions between classifier performance and evaluation metrics such as FID and human judgments. Finally, our methodology can naturally serve as a diagnostic tool for diffusion models, offering a reasonable explanation for Model Autophagy Disorder (MAD) and evaluating the use of generated data.

## Appendix A Detail experiment settings

We implement our experiments upon the official code of ConvNeXt (Liu et al., ), U-ViT (Bao et al., ), DiT (Peebles & Xie, ), EDM (Karras et al., ), EDM2 (Karras et al., ). We also utilize the official models of Pixart-$\alpha$ (Chen et al., ), SDXL (Podell et al., ), and Playground-v2.5 (Li et al., ). The respective links and licenses are detailed in Tab. 9.

**Table 9: Code links and licenses.**
| Method | Link | License |
| --- | --- | --- |
| ConvNeXt | [https://github.com/facebookresearch/ConvNeXt](https://github.com/facebookresearch/ConvNeXt) | MIT License |
| U-ViT | [https://github.com/baofff/U-ViT](https://github.com/baofff/U-ViT) | MIT License |
| DiT | [https://github.com/facebookresearch/DiT](https://github.com/facebookresearch/DiT) | CC BY-NC 4.0 |
| EDM | [https://github.com/NVlabs/edm](https://github.com/NVlabs/edm) | CC BY-NC-SA 4.0 |
| EDM2 | [https://github.com/NVlabs/edm2](https://github.com/NVlabs/edm2) | CC BY-NC-SA 4.0 |
| Model | Link (add ‘https://huggingface.co/’) | License |
| PixArt-$\alpha$ | [PixArt-alpha/PixArt-XL-2-1024-MS](PixArt-alpha/PixArt-XL-2-1024-MS) | Open RAIL++-M |
| SDXL | [stabilityai/stable-diffusion-xl-base-1.0](stabilityai/stable-diffusion-xl-base-1.0) | Open RAIL++-M |
| Playground-v2.5 | [playgroundai/playground-v2.5-1024px-aesthetic](playgroundai/playground-v2.5-1024px-aesthetic) | Playground v2.5 |

We present the main experiment settings as follows.

Dataset. For label-to-image, we consider the CIFAR (Krizhevsky et al., ) and ImageNet (Deng et al., ) datasets, which are well-established and widely recognized benchmarks in the field of image generation. For text-to-image, we utilize the COCO2014 dataset (Lin et al., ), known for its rich annotations and diverse image content. For CIFAR-10, we use the real CIFAR-10 training set and validation set to construct the real distribution and use the diffusion model to generate 50k training images and 10k validation images to construct the generated distribution. In the case of ImageNet, we consider 256 and 512 resolutions, which are common resolutions for ImageNet image generation tasks. We randomly sample 100k training images and 50k validation images from the training set and the validation set of ImageNet to construct the real distribution and use the diffusion models to generate 100k training images and 50k validation images to construct the generated distribution. For the real dataset, we adopt the data processing method from ADM (Dhariwal & Nichol, ) to modify the ImageNet dataset into two common resolutions: ImageNet-256 and ImageNet-512, where the numbers indicate the resolution of the data. In addition, for the COCO2014 dataset, by default, we construct the real distribution by randomly sampling 10k training images and 1k validation images from the respective training and validation sets of COCO2014. Each image in this dataset is associated with five captions. To create the generated distribution, we randomly select one of the five captions for each real image to serve as a prompt. These prompts are then used by the diffusion models to generate an equivalent number of images to construct the generated distribution.

Classifier. By default, we employ the ResNet-50 (He et al., ) as the classifier architecture. For completeness, we also consider ConvNeXt-T (Liu et al., ) and ViT-S (Dosovitskiy et al., ). Our pre-processing protocol follows the standard supervised training approach (Liu et al., ). Specifically, during training, classifiers process randomly augmented crops of $224\times 224$ images. During validation, images are resized so that their smaller dimension reaches 256 pixels while preserving the original aspect ratio. Subsequently, these images are center cropped to $224\times 224$ pixels before being fed into the model. For the experiments on CIFAR-10, we initialize our classifier with the ResNet-50, pre-trained on ImageNet. For completeness, we also present the result trained from scratch (see experiments in Appendix. C.2). In the case of ImageNet, we opt to train the ResNet-50 model from scratch.

Diffusion model.
For the generation of CIFAR-10, we consider two diffusion models: EDM (Karras et al., ) and U-ViT (Bao et al., ). Both models have demonstrated strong generation performance on the CIFAR-10 (Krizhevsky et al., ). Quantitatively, EDM achieves an FID of 1.79, and U-ViT achieves an FID of 3.11. To improve efficiency, we modified U-ViT’s sampling method from Euler-Maruyama to DPM-Solver (Lu et al., ) and reduced the sampling steps from 1,000 to 50. These adjustments resulted in U-ViT achieving an FID of 3.65 on CIFAR-10. For the generation of ImageNet-256, we explore three diffusion models: EDM2 (Karras et al., ), U-ViT-H/2 (Bao et al., ), DiT-XL (Peebles & Xie, ). Both DiT and U-ViT are prominent diffusion transformer architectures known for their scalability and strong performance. U-ViT-H/2 achieves an FID of 2.29 on ImageNet-256, and DiT-XL/2 achieves an FID of 2.27. We consider EDM2 to incorporate a UNet-based architecture, which was traditionally used before the rise of diffusion transformers. Since EDM2 is originally designed for ImageNet-512 generation, we resize the generated images from 512 to 256 resolution to suit our ImageNet-256 task. In this way, EDM2-XXL achieves an FID of 2.14 on this task, which is similar to the FID achieved by U-ViT and DiT. For the generation of ImageNet-512, we use EDM2 (Karras et al., ), which achieves state-of-the-art performance on this task with an FID of 1.81. For the generation of COCO, we consider three state-of-the-art text-to-image diffusion models: Pixart-$\alpha$ (Chen et al., ), SDXL (Podell et al., ), and Playground-v2.5 (Li et al., ).

Evaluation. We use the top-1 accuracy on the validation set to evaluate classification performance.

Training settings. The complete training settings of ResNet-50 are reported in Tab. 10 for combinations related to CIFAR-10 and Tab. 11 for combinations related to ImageNet.

Figure: Table 10: Training settings for CIFAR-10.

## Appendix B Computational cost

Our experiments were conducted on RTX 3090 and V100 GPUs. The detailed computational costs are presented in Tab. 12. Training epochs were set to 50 for CIFAR-10 and 200 for ImageNet. The number of epochs trained on CIFAR-10 is relatively low because we use a pre-trained model to initialize our classifier, enabling faster convergence.

**Table 12: Training time of classifiers.**
| Model | Combinations | Epochs | GPU-type | GPU-nums | Hours |
| --- | --- | --- | --- | --- | --- |
| ResNet-50 | C, U | 50 | 3090 | 4 | 2 |
| ViT-S | C, U | 50 | 3090 | 4 | 2 |
| ConvNeXt-T | C, U | 50 | 3090 | 4 | 2 |
| ResNet-50 | I-256, U-H/2 | 200 | V100 | 8 | 6 |
| ViT-S | I-256, U-H/2 | 200 | V100 | 8 | 9 |
| ConvNeXt-T | I-256, U-H/2 | 200 | V100 | 8 | 8 |
| ResNet-50 | I-512, E2-XXL | 200 | V100 | 8 | 9 |
| ViT-S | I-512, E2-XXL | 200 | V100 | 8 | 11 |
| ConvNeXt-T | I-512, E2-XXL | 200 | V100 | 8 | 10 |

**Table 13: Distribution classification accuracy on CIFAR-10. "Pretrained" indicates that the classifier was initialized with a model pretrained on ImageNet, while "Scratch" indicates that the classifier was trained from scratch.**
| Real dataset | Generative model | Model | Pretrained | Scratch |
| --- | --- | --- | --- | --- |
| *C* | *E* (Karras et al., ) | ResNet-50 | 96.25 | 56.13 |
| ViT-S | 89.38 | 55.32 |  |  |
| ConvNeXt-T | 98.43 | 53.27 |  |  |
| *C* | *U* (Bao et al., ) | ResNet-50 | 99.92 | 56.70 |
| ViT-S | 98.04 | 52.66 |  |  |
| ConvNeXt-T | 99.96 | 56.37 |  |  |

## Appendix C Additional results

### C.1 Additional results from generative adversarial network

As shown in Tab. 14, we conducted experiments using StyleGAN-XL (Sauer et al., ), a state-of-the-art GAN model trained on ImageNet-256. Our results demonstrate that, despite the use of a discriminator during GAN training, the classifier can still easily distinguish between real and generated images. We argue that this is because the discriminator is trained jointly with the generator. During training, the generated data seen by the discriminator comes from a continuously evolving distribution, as the generator improves with each iteration. However, when using a classifier to distinguish between real and generated distributions, the generated distribution remains fixed.

**Table 14: Binary distribution classification on label-to-image. All classifiers yield high accuracy on various datasets against strong generative models. FIDs are taken from the corresponding references.**
| Real dataset | Generative model | FID | Classifier | Accuracy (%) |
| --- | --- | --- | --- | --- |
| *I*-256 | StyleGAN-XL (Sauer et al., ) | 2.30 | ResNet-50 | 99.69 |
| ViT-S | 99.95 |  |  |  |
| ConvNeXt-T | 96.85 |  |  |  |

### C.2 Results of CIFAR-10

In order to ensure the completeness of the experiment, we are here to present the result of CIFAR-10 trained from scratch. We present the results in Tab. 13. If there is no prior knowledge, classifiers struggle to distinguish between real and generated data in datasets with low resolution, such as CIFAR-10 (i.e., 32x32). However, the use of a pre-trained model allows the features learned from the ImageNet dataset to aid in differentiating between real and generated images in CIFAR-10.

### C.3 Self-supervised classifiers for Text-to-Image distribution classification

As shown in Tab. 15, we report the distribution classification accuracy of self-supervised classifiers in text-to-image scenarios. Notably, these classifiers achieve high accuracy in distinguishing between different text-to-image models. This suggests that there are significant differences between text-to-image generative models, allowing even self-supervised classifiers to easily distinguish them.

**Table 15: Four-way distribution classification on text-to-image. Linear classifier on self-supervised features also achieves high accuracy in distinguishing four distributions: COCO (Lin et al., ), Pixart-$\alpha$ (Chen et al., ), SDXL (Podell et al., ), and Playground-v2.5 (Li et al., ), using only 5k or 10k training samples per dataset. We use ViT-B as the backbone.**
| Training samples | Self-supervised method | Accuracy (%) |
| --- | --- | --- |
| 5k | MAE | 87.77 |
| 5k | MoCo v3 | 90.48 |
| 10k | MAE | 91.67 |
| 10k | MoCo v3 | 90.90 |

### C.4 Visualization of crops

As shown in Fig. 7, we present the visualization of crops mentioned in Sec. 7.1.

Figure: Figure 7: Visualization of crops. With each resolution represented by a different color.
Refer to caption: x7.png

### C.5 Frequency analysis on the combination of EDM2-XS and EDM2-XXL

We preprocess the generated images using band-pass filters as defined in Sec. 2.2, with four threshold intervals: 0-10, 10-30, 30-50, and 50-100. An example of original and processed EDM2-XXL generated images is shown in Fig. 8. As shown in Tab. 16 and Fig. 9, for EDM2-XS and EDM2-XXL, the smallest and largest models in the EDM2 family, classifiers’ accuracy approaches random guessing (around 50%) across different threshold intervals. This indicates that for models within the same diffusion model family, which share inductive biases but differ in visual quality, classifiers unable to distinguish between them based on any specific frequency band.

Figure: Figure 8: Visualization of frequency domain processing of EDM2-XXL. The image shows the results after applying a band-pass filter to EDM2-XXL with band thresholds of 0-10, 10-30, 30-50, and 50-100, from left to right.
Refer to caption: extracted/5918590/picture/appendix/frequency_process_generated.png

Figure: Table 16: Classification accuracy of ResNet-50 on the combination of EDM2-XS and EDM2-XXL after applying band-pass filters.
Refer to caption: x8.png

### C.6 User study

Fig. 10 shows a screenshot of the interaction interface used in our user study. The study involved nineteen participants, and we designed three groups of experiments, each requiring participants to classify 32 pairs of images. In the first set, participants distinguished between generated and real images, with real images sourced from ImageNet-256 and generated images from U-ViT-H/2. In the second set, they classified images between two diffusion models with similar performance: DiT-XL/2 and U-ViT-H/2. For the final set, participants evaluated images from EDM2-XS and EDM2-XXL, which share the same training methodology but differ in parameter count, resulting in different FID scores and visual quality. In the first set, participants were asked to identify the real images. In the second, they were tasked with identifying DiT images, and reference images from DiT and U-ViT were provided during the test. In the third experiment, participants judged which images were of higher quality. All nineteen participants were graduate students with substantial experience in machine learning. They were allowed to zoom in on the images during the experiment, which was conducted on 27-inch 4K displays. All participants had corrected vision of 1.0 (standard normal vision), and their ages ranged from 22 to 26. Each participant completed the experiment within an hour and was compensated $10.

Figure: (a) I-256 vs. U-ViT-H/2
Refer to caption: extracted/5918590/picture/appendix/user_1.png

## Appendix D Binary classification as a measure of distribution distance

We employ a classifier $C(x)$ to distinguish between the real data distribution $p_{\text{data}}(x)$ and the generated data distribution $p_{g}(x)$. By training $C(x)$ using the binary cross-entropy loss:

$$ $\displaystyle L(C)=-E_{x\sim p_{\text{data}}(x)}[\log(C(x))]-E_{x\sim p_{g}(x) }[\log(1-C(x))],$ (5) $$

The optimal classifier that minimizes this loss is:

$$ $\displaystyle C^{*}(x)=\frac{p_{\text{data}}(x)}{p_{\text{data}}(x)+p_{g}(x)}.$ (6) $$

Substituting $C^{*}(x)$ back into the loss function yields:

$$ $\displaystyle L(C^{*})=-\log(4)+2\,\text{JSD}(p_{\text{data}}(x)\|p_{g}(x)),$ (7) $$

where JSD denotes the Jensen-Shannon Divergence—a direct measure of the distance between the two distributions.

In our paper, we use classification accuracy to evaluate how well the classifier distinguishes between real and generated data because it provides an intuitive and interpretable metric. Although accuracy is non-differentiable and unsuitable for direct optimization, training the classifier with the binary cross-entropy loss—a convex surrogate—often leads to improved accuracy. This correlation suggests that accuracy can serve as a proxy for changes in the loss function, reflecting the distance between the generated and real data distributions.

Figure: (a) Accuracy vs. low-frequency filter threshold
Refer to caption: x9.png

## Appendix E Relationship between distribution classification and FID

The Fréchet Inception Distance (FID) is a widely used metric for evaluating the quality of generative models. It relies on feature extraction networks trained on datasets such as ImageNet and assumes that the extracted feature vectors follow a multivariate Gaussian distribution. FID calculates the Fréchet distance between these Gaussian distributions to measure discrepancies between the real and generated data. Meanwhile, as noted by Kynkäänniemi et al. (), FID can decrease simply by aligning the histograms of top-N classifications, without necessarily improving the perceptual quality of the generated images. Additionally, recent works like Karras et al. () and Tian et al. () report FID scores close to those of the ImageNet validation set, suggesting limitations in FID’s sensitivity to certain distribution differences.

In contrast, our classifier-based approach offers a more direct measure of the distance between distributions without relying on the Gaussian assumptions inherent in FID. By training a classifier to distinguish between real and generated data, we obtain an intuitive and interpretable metric that reflects the actual distribution differences. This method complements commonly used metrics such as FID and Inception Score (IS), providing an alternative perspective on evaluating generative models.

## Appendix F Comparison of rectangular and circular filters

Rectangular and circular filters are common techniques in image filtering. In this paper, we chose to implement a rectangular filter following the official FreeU implementation (Si et al., ), due to its simplicity and computational efficiency. For comparison, Fig. 11 presents initial results using a circular mask, denoted as U-H/2 and I-256 (Ideal). The results from both implementations are similar, and we chose to use the rectangular filter for its computational efficiency.

## Peer Reviews

confidence: 3
contribution: 1
initial_rating: 5
presentation: 2
questions: In addition to the weakness section:

1. The paper assumes high classification accuracy directly correlates with the quality of distribution differences, which may oversimplify the nuanced gaps in generated and real data. Further exploration or validation of why classifiers perform well on certain generative models would add depth to the analysis.
soundness: 3
strengths: The authors propose a classifier-based approach to evaluating distribution differences, providing insights that complement traditional metrics like FID. This diagnostic tool can help understand diffusion model limitations beyond typical quality assessments. 

The paper's findings regarding the distinctiveness of generated data, even when visually indistinguishable from real data, are relevant to the broader community working on generative models and dataset augmentation strategies.

The study includes experiments across multiple datasets (CIFAR-10, ImageNet, COCO) and models, providing a detailed comparison between real and generated distributions using various classifiers, architectures, and training samples. The results highlight classifier strength in identifying distribution discrepancies across these contexts.
summary: This paper explores whether neural network-based classifiers can distinguish between real and generated images despite advances in diffusion models achieving realistic outputs and low Fréchet Inception Distance (FID) scores. 

Through distribution classification tasks, the authors show that classifiers consistently and accurately differentiate between real and generated distributions, revealing a marked discrepancy that remains hidden in humans and FID scores. Additionally, classifiers can distinguish diffusion models of different architectures with similar FIDs, though they struggle with models of the same family but different scales. The study suggests that augmenting real data with generated images is more effective than replacing it entirely, as complete replacement causes distribution drift and performance degradation.
weaknesses: 1. One of my main concerns is that many of its findings seem to echo results from previous works, such as those cited in the related work section. Prior research has demonstrated that neural network classifiers can effectively distinguish real from generated images, which somewhat diminishes the unique contribution of this study. Without extending or innovating upon these prior insights, the paper's impact remains limited in originality.

2. The paper frequently refers to its approach as "distribution classification," implying a theoretical or practical connection to statistical concepts of distribution. However, the methodology appears identical to regular classification tasks focused on distinguishing real and generated images. This terminology may be misleading, as it does not introduce any specific statistical treatment or distributional modeling. Greater clarity around this term and why it differs from conventional classifier settings would be beneficial for readers.

3. The methodology mainly employs existing classifiers and conventional metrics, such as FID and classification accuracy, to evaluate generative models, leading to limited methodological novelty. The reliance on standard architectures (e.g., ResNet-50, ConvNeXt, ViT-S) and classification accuracy as a primary metric for assessing generative quality does not offer fresh insights or approaches to the problem. Coupled with the limited novelty in findings, this contributes to the perception that the paper lacks substantial innovation.

4. While the classifier-based evaluation offers diagnostic insights, the paper does not provide actionable improvements or enhancements for current generative models based on those insights.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Q1. What is the temperature used for sampling, how will this affect the conclusion? For instance, more diverse generation vs. more duplicates.

Q2. In Tab.2, is transformer-based classifier worse than convolution-based classifier? Is this because the latter detects high-frequency features better.

Q3. How good is the classifier on CIFAR10 to do distribution classification?

Q4. Comparing Tab. 2 and Tab. 4, is it fair to say SSL models worse than classifiers train from scratch to classify distributions?
soundness: 2
strengths: S1. The authors carried out studies under various relevant settings, e.g., classifier trained from scratch vs. SSL classifiers, etc.

S2. The authors made effective use of frequency filters, demonstrating their impact both qualitatively and quantitatively

S3. The diagnosis of MAD phenomenon offers a plausible explanation
summary: The authors answer a fundamental question: How far is the generated distribution from the real distribution. They demonstrate that classifiers can distinguish between generated and real images with high accuracy—a distinction that humans are unable to make. Additionally, they provide analysis to diagnose the Model Attribution Discrepancy (MAD) phenomenon.
weaknesses: W1. Prior work has shown the neural network-based classifiers are sensitive to high-frequency features/noises. Conducting a more in-depth analysis of the specific features that classifiers focus on could strengthen the paper, especially from an explainability perspective. For instance, the observation that samples from the same model family are harder to distinguish compared to samples from different model families raises the question of whether each model family has a unique “watermark” embedded in their generated images.

W2. Some related work should be discussed and compared, e.g., [C1] showed FID doesn't correlate well with human judgements, and [C2] on different metrics and low-level and high-level features.




[C1] Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models, Stein et al., NeurIPS 2023

[C2] Dreamsim: Learning new dimensions of human visual similarity using synthetic data, Stephanie et al., NeurIPS 2023

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - In the part of researching the frequency aspect (line 472-480), do we need to compare the differences between the generated images and the real images after frequency filtering besides the accuracy provided by classifiers? Does it provide more solid evidence of the statement, diffusion models effectively learn low frequency components?
soundness: 3
strengths: - This work provides another view for analyzing the differences between the generated and real distributions, instead of just comparing FID. They propose a classifier-based approach to measure the differences, which is more useful for analyzing the results of generative models, especially for using the generated data for downstream tasks.

- Experimentally, this work finds contradictions between classifier's prediction and FID score, which is helpful for deeply analyzing the generated results of diffusion models. First, their approach shows the diffusion model can effectively learn specific features of data but ignore other features. Second, it provides a reasonable explanation of using generated data for training models. 

- The experiments are solid and reasonable.
summary: This works experimentally researches the differences between the generated data and real data. The differences is evaluated by classifiers and the results reveal some surprising and interesting properties of generative models, which is hidden in the usual metric, such as human judgement and FID score. And these hidden properties are important for analyzing the generated data, like using them to train model.
weaknesses: - This work has a lack of providing any theoretical analysis or theoretical intuitions to explain why the classifier, even self-supervised classifier can easily distinguish the generated data from real data, when human and FID score cannot see the differences.

- The authors may provide more details of FID metric and the proposed classifier-metric. They may explain the differences of the distances between the generated distribution and the real distribution provided by FID score and the distance provided by classifiers. Analyzing the differences may help us to understand the contradictions of FID score and classifiers for measuring the generated distributions.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. In scenarios where classifiers underperform compared to humans or FID (e.g., within the same model family), how can the classifiers be improved?
2. How are the human evaluation scores obtained? How to eliminate agreement bias among participants?
soundness: 2
strengths: 1.  This paper introduces classifier performance as a new way to evaluate generative models.
2. The study highlights contradictions between classifier performance, human judgment, and FID scores. Classifiers can easily distinguish between different diffusion models with similar FID but are challenging to differentiate between smaller and larger models from the same diffusion family.
summary: This paper evaluates whether classifiers can reliably differentiate real images from generated images. Despite the visual similarity of these generated images to real ones, the study finds that classifiers achieve high accuracy in distinguishing between them. The research highlights several inconsistencies: classifiers can easily detect generated images that humans find indistinguishable and also reveal contradictions between classifier performance and widely used metrics like FID. The study concludes with practical insights, such as recommending data augmentation with generated data instead of full replacement to avoid issues like MAD during model training.
weaknesses: 1. The authors empirically discover interesting inconsistencies between classifier performance, human evaluations, and FID metrics but lack in-depth analysis of the underlying reasons. This results in relatively weak theoretical contribution.
2. While classifiers are introduced as a new evaluation method, the paper lacks a detailed discussion of where and why this approach is meaningful. The significance of using classifiers as a metric remains unclear.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. Could you expand on the reasons behind the observed discrepancies between FID and classifier results (sections 5.2 and 6)?

1. In section 5.3, would you consider testing CIFAR10-pretrained self-supervised classifiers to verify the hypothesis for the classifier’s poor performance with CIFAR data?

1. Could you clarify the purpose of using self-supervised models in classification in section 5.3?

1. What is the detailed experiment setup for Table 1? What is the training/testing data and label?
soundness: 3
strengths: 1. The paper addresses a highly relevant issue: exploring the differences between real and generated data, which is crucial given the advancements in generative AI.
1. The paper has interesting findings by comparing generated data between different diffusion models: classifiers distinguishing between models with similar FID scores from different families but struggling with models of different capacities within the same family. This insight suggests that FID scores may not capture all dimensions of model quality.
1. The paper makes solid connections to existing research, especially by relating its findings to the Model Autophagy Disorder (MAD) phenomenon, enhancing the broader significance of its results.
1. The author extended the analysis beyond diffusion model by including some results for GAN models, showing consistency in findings across model types. This cross-model analysis supports the generalizability of the conclusions.
1. Implementation details are well-documented, which strengthens reproducibility and provides clarity for readers who may want to replicate or build on the study.
summary: This paper aims to explore the distributional differences between real and high-quality generated data, mainly focusing on diffusion models. By training neural network classifiers to differentiate real from generated distributions, the authors found that classifiers could easily identify generated data, even when human perception and FID scores suggest high-quality results. The study also looks into distributional discrepancies across model types, revealing that classifiers can distinguish between models with similar FID scores from different families but struggle with models of varying capacities and FID scores within the same family, which again shows contradictory results between FID and the classifiers. Additionally, the authors analyze which features classifiers emphasize, finding a reliance on high-frequency features and spatial information to distinguish distributions. The paper further connects these observations to the Model Autophagy Disorder (MAD) phenomenon, offering broader implications for understanding distributional gaps between real and generated data. Overall, the paper aims to diagnose the differences between real and generated images by leveraging classifier-based analysis.
weaknesses: My main concern with the paper is that while the paper includes a substantial amount of data and findings, it lacks a cohesive narrative that ties the results together and clarifies their implications. 
1. For example, the paper demonstrates differences between FID and classifier results (sections 5.2 and 6) without in-depth discussions on the reasons behind these discrepancies. Section 7.1 shows some analysis on what the classifiers focus on, it would be helpful to perform similar analysis for the FID metrics, and pointing out the differences in what aspects of the images the two methods focus on. 
1. Similarly, observations like the classifier’s poor performance with CIFAR data are presented without verification of potential causes (section 5.3). Testing with CIFAR10-pretrained self-supervised classifiers would strengthen these claims. For example, are diffusion models really closing the gap between real and generated images on low-resolution datasets? 
1. Additionally, the purpose of using self-supervised models in classification (section 5.3) is unclear, and further context would help clarify their role. 

Other concerns: 
1. The analysis of models within the same family but with different capacities is limited to only one model (EDM2-XS and EDM2-XXL), which weakens support for this conclusion. Including additional models of varying capacities would provide more evidence.
1. The user study involves only 19 participants, which could introduce noise or bias. More information on participant agreement rates/variance of the results would enhance the reliability of these results.