SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 13: Additional GradCAM visualisations (1/2).
Figure 14: Additional GradCAM visualisations (2/2).
Figure 15: Left: Perceptual score [62] for each encoder measured using the BAPPS [118] dataset. Samples from the dataset are shown above, where  $\checkmark$  denotes the patch that humans judged to be closer to the reference. Right: correlation of intra-class diversity on ImageNet.
To probe the level of semantic and pixel-wise diversity captured by each encoder we calculate the intra-class Vendi score over each class of the ImageNet training set (including in pixel space). This score is proportional to the effective number of independent samples within the class after transforming to the representation space. Section D.4 provides more details. A low score reflects a low diversity within the class, which is an indicator of a feature space that overemphasizes a small set of class specific features rather than the wide diversity of semantic in pixel-space. The correlation of the score over all classes across our set of encoders is shown in Figure 15. Sorting the encoders by their correlation with Inception we find two distinct groupings. The masked models, MAE and data2vec, have a intra-class diversity that is highly correlated with each other, but also with the diversity as measured in image-space. This indicates that such representation spaces put more weight towards low level image features rather than clustering classes by object semantics, which is a quantitative agreement with the characteristics of the heatmap visualizations. Interestingly MAE achieves high linear-classification accuracy but performs very poorly on  $k$ -nearest neighbours evaluation protocols [44], perhaps also indicating a lack of clustering based on class-specific semantic information. ConvNeXt, CLIP, SwAV, and DINOv2 show a high correlation with Inception yet a low correlation with image-space, indicating that their feature spaces distill a more object-based semantic information. Of the ViTs, CLIP has the smallest alignment with pixel space diversity and the largest with Inception space.
D.3 FD bias
It is known that using finite datasets for computing FID (and similarly for FD in general) results in a biased estimate of the quantity that one would obtain if an infinitely large amount of data was observed. As mentioned in Appendix B.1,  $\mathrm{FID}_{\infty}$  [22] aims to remove this inherent bias. Here we measure the biases of FD and  $\mathrm{FD}_{\infty}$  in a realistic setting with a number of encoders used in our work in order to test whether the default 50k generated samples are sufficient for our proposed replacement to the Inception-V3 feature extractor.
We use the 100k samples generated with DiT-XL-2 [84] trained on ImageNet-256, and calculate the relevant statistics as a function of the number of training samples used by sub-sampling both the DiT generated set and the set of training images. Figure 16 shows the results across 4 encoders - Inception, CLIP, DINOv2, and MAE. We observe that while  $\mathrm{FD}_{\infty}$  has less bias compared to FD, it does not completely solve the problem, and continues to decrease with the number of samples used. We also find that the self-supervised encoders exhibit less biased FD than the Inception network. For example, DINOv2 with 20k samples has the same relative error compared to the full 100k set
Figure 16: Left: Convergence of  $\mathrm{FD}_{\infty}$  and FD using different representation spaces. Right: Convergence of FD and ASW using Inception as the encoder.
as Inception does using 50k samples. While one could argue that this means only 20k samples are required for DINOv2, we retain consistency with FID and use 50k samples for FD with all encoders.
Naïvely, the FD thus has two issues: the aforementioned bias, and the fact that it only compares the first two moments of its corresponding distributions (see Appendix B.1). Since the FD can be understood as approximating its distributions with Gaussians of matching moments and then computing the Wasserstein-2 distance,  $\mathbf{W}_2$ , between them, it is sensible to consider other metrics based on Wasserstein distance as potential evaluation metrics. Unfortunately, computing this distance is in general computationally expensive [32].
In an attempt to address these issues, we considered using the sliced Wasserstein distance,  $\mathbf{SW}_2$ , instead. Let  $\mathbb{P}$  and  $\mathbb{Q}$  be distributions on  $\mathbb{R}^d$ , which we should think of as the generated and real distributions, respectively. Let  $u_{\#}(\cdot)$  denote the push-forward of measures under the mapping  $x \to \langle u, x \rangle$ . Then  $\mathbf{SW}_2$  is defined through:
$$
\mathbf {S W} _ {2} ^ {2} (\mathbb {P}, \mathbb {Q}) = \mathbb {E} _ {u} \left[ \mathbf {W} _ {2} ^ {2} \left(u _ {\#} \mathbb {P}, u _ {\#} \mathbb {Q}\right) \right], \tag {12}
$$
where the expectation is taken with respect to the uniform measure on the unit sphere in  $\mathbb{R}^d$ .
It can be shown that  $\mathbf{SW}_2$  has a lower sample complexity compared to both  $\mathbf{W}_2$  and its relaxation [73]. Calculating optimal transport in one dimension requires significantly less computation than in higher dimensions as it can be solved via sorting. Therefore, the RHS of Equation 12 can be approximated well using Monte-Carlo methods. However, [74] introduces a closed form approximate sliced Wasserstein (ASW) distance with convergence guarantees that does not require Monte-Carlo, making the final result a formula based on the first two moments of  $\mathbb{P}$  and  $\mathbb{Q}$ :
$$
\operatorname {A S W} \left(\left\{x _ {i} ^ {g} \right\} _ {i = 1} ^ {n}, \left\{x _ {j} ^ {r} \right\} _ {j = 1} ^ {m}\right) = \mathbf {W} _ {2} \left(\mathcal {N} \left(0, \frac {M _ {2} \left(\left\{x _ {i} ^ {g} \right\} _ {i = 1} ^ {n}\right)}{d}\right), \mathcal {N} \left(0, \frac {M _ {2} \left(\left\{x _ {j} ^ {r} \right\} _ {j = 1} ^ {m}\right)}{d}\right)\right) + \frac {1}{d} \| \mu_ {g} - \mu_ {r} \| _ {2} ^ {2}, \tag {13}
$$
$$
M _ {2} \left(\left\{x _ {i} ^ {g} \right\} _ {i = 1} ^ {n}\right) = \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \left\| x _ {i} ^ {g} - \mu_ {g} \right\| _ {2} ^ {2}\right) + \left\| \mu_ {g} \right\| _ {2} ^ {2}, \tag {14}
$$
(and analogously for  $M_2(\{x_j^r\}_{j=1}^m)$ , and where  $\mu_g$  and  $\mu_r$  denote the sample mean of generated  $\{x_i^g\}_{i=1}^n$  and real samples  $\{x_j^r\}_{j=1}^m$ , respectively. Note that the  $\mathbf{W}_2$  term required to compute the ASW can be trivially computed through the 1-dimensional FD formula. The ASW resembles FD, but with the advantages of having both convergence and sample complexity guarantees.
To test the sample stability we use the same setting that we did for FD and  $\mathrm{FD}_{\infty}$ . Figure 16 (right) depicts the relative error of ASW and compares it with FD using Inception. It is clear that the ASW has less dependence on sample size, indicating that even small batches of around  $10\mathrm{k}$  samples may be sufficient (note that the plot is log-scale).
Despite its theoretical guarantees, we found that ASW does not correlate well with human perception across various encoders and does not seem to be affected by mode shrinkage. Since it is not widely
Figure 17: Per-class Vendi score (higher is more diverse) distributions with different encoders. By visual inspection, DiT-XL-2 is more diverse than DiT-XL-2-guided.
used in the literature, we chose to not include it in the main text. These results do however raise the question: why does the FD perform so well in practice compared to the ASW, despite the latter being seemingly more principled? We hypothesize that there are two factors at play:  $(i)$  the encodings are likely approximately Gaussian (e.g. [52]), meaning that comparing the first two moments is enough; and  $(ii)$  while the computation of the FD is biased, if the bias behaves similarly across generated datasets, it will have no impact in model rankings. While we leave a formal exploration of these hypotheses for future work, we point out that the strong correlation of FD with KD (Figure 4) suggests that the FD only comparing the first two moments and having sample complexity issues are not problems that are borne out in practice, and we thus recommend using FD as-is given that its use is already widespread.
D.4 Diversity
In this section, we outline the rationale behind our decision to utilize the per-class Vendi score with DINOv2 as the metric for our diversity analysis.
Based on an analysis of the DiT-XL-2 and DiT-XL-2-guided datasets, the strong classifier guidance value of DiT-guided in comparison to DiT resulted in samples that lacked diversity within each class (examples were shown in Figure 6). We pose that a robust diversity metric should flag such a lack of intra-class diversity, and thus these two models are the basis of the following analysis. As mentioned in the main manuscript, much like precision measures more than just fidelity (Figure 4, and similarly for density, see Figure 10), we find that recall and coverage also quantify more than just diversity, which can be clearly seen in Figure 10 (as both of these metrics have relatively strong correlations with human evaluation when using good encoders - a property that diversity metrics should not exhibit). On the other hand, the per-class Vendi score, which we will further justify shortly, has the intended property of being uncorrelated with human error rate (Figure 5, right).
We computed the Vendi score for the DiT-XL-2 and DiT-XL-2-guided models over each individual ImageNet class (100 images per class), and averaged the results, which are presented in Table 9. The distributions of the per-class Vendi scores are plotted in Figure 17. The per-class Vendi scores consistently ranked DiT-XL-2 higher than DiT-XL-2-guided for every encoder, indicating that DiT-XL-2 exhibits higher intra-class diversity. This observation aligns with our inspection, where it is apparent that the guided model produces less diverse intra-class samples. Hence, we adopted the per-class Vendi score for our analysis.
We next report the overall (not conditioned on class) Vendi score in Table 10, using 50k images from each set. It is important to note that Vendi scores represent the effective number of unique samples in a dataset. Therefore, it is counterintuitive to obtain very low values, such as those observed in MAE, data2vec, SimCLRv2, and even with simple resizing, where there are only a few effective samples among the 50k images. Moreover, not all encoders ranked DiT-XL-2 above DiT-XL-2-guided based on the overall Vendi scores, except for DINOv2, SimCLRv2, and ConvNext. However, SimCLRv2 exhibited minimal differences across ImageNet train, DiT-XL-2, and DiT-XL-2-guided. Additionally, neither SimCLRv2 nor ConvNext showed FD rankings that aligned well with human perception. Consequently, we pose that this diversity analysis provides additional support for the use of DINOv2 for generative evaluation.
Table 9: Mean Vendi score for different encoders on ImageNet train and generated samples.
Encoder	ImageNet train	DiT-XL-2	DiT-XL-2-guided
Inception	6.04	5.50	3.25
ConvNeXt	28.08	23.52	8.73
CLIP	13.74	10.17	5.72
SimCLRv2	5.17	4.99	4.15
DINOv2	15.93	14.17	5.61
SwAV	8.72	6.42	4.40
MAE	1.10	1.08	1.07
data2vec	1.03	1.02	1.02
Resize (to 32x32)	1.09	1.09	1.09
Table 10: Overall diversity for ImageNet training and generated images.
Encoder	ImageNet train	DiT-XL-2	DiT-XL-2-guided
Inception	88.57	89.03	129.92
ConvNext	951.83	824.33	687.58
CLIP	36.56	27.61	31.90
SimCLRv2	6.74	6.77	6.38
DINOv2	762.52	688.53	671.54
SwAV	55.61	43.51	59.77
MAE	1.15	1.14	1.17
data2vec	1.04	1.04	1.04
Resize(to 32x32)	1.12	1.13	1.18
D.5 Rarity
Here we perform experiments to investigate whether participants confuse fakeness with un likeliness. Such an effect would result in images generated from more diverse models being more likely to be assessed as "fake" and hence a diverse model receiving a lower human error rate compared to a model of lower quality but less diversity. The following experiments rule out such a flaw in our experimental setup and ensure that our human error rate dataset is not misleading.
To isolate this effect, we on focus comparing the error rate on "rare" real samples versus "common" real samples. For each of the 2000 real images we used for each dataset (individual images were evaluated by an average of 13 humans), we determined the fraction of humans that labeled it as fake, while as a measure of image rarity/unlikeliness we calculate the "rarity score" (RS) of [42]. The RS is determined on image representations, so we performed the calculation using both Inception-v3 and DINOv2 to quantify the dependency of RS on the embedding space.
Table 11: Correlation of the fraction of humans that labelled a real image as fake and the rarity score [42] of that image. The rarity score can only be determined for images that fall "on manifold".
Dataset	Encoder	% on manifold	r	p-value	r (94%)	p-value (94%)
CIFAR10	Inception	76.3	0.278	0.000	0.216	0.000
CIFAR10	DINOv2	88.3	0.062	0.009	0.0004	0.859
ImageNet	Inception	82.2	-0.034	0.163	-0.002	0.922
ImageNet	DINOv2	92.9	0.008	0.726	-0.012	0.624
LSUN-Bedroom	Inception	70.3	0.110	0.000	0.045	0.106
LSUN-Bedroom	DINOv2	90.2	0.102	0.000	0.021	0.386
FFHQ	Inception	79.3	-0.027	0.286	-0.024	0.357
FFHQ	DINOv2	89.7	-0.019	0.431	-0.040	0.103
Figure 18: Top: Human error rate and rarity score for each of the 2000 training images. Individual images were evaluated by an average of 13 humans. Bottom: Examples on LSUN Bedroom (left) and CIFAR10 (right), with the top rows displaying training images correctly judged by humans as real, and the bottom rows displaying real images "incorrectly" judged as fake. The fraction of participants who judged the image as fake is denoted on the top right of each image.
As seen in Figure 18 and Table 11 we find no correlation between the human error rate and RS on ImageNet and FFHQ. On CIFAR-10 and LSUN-Bedroom we find a small but statistically significant correlation, which we identify as driven by dataset issues: the non-zero correlation is caused by a very small percentage of "real" images which are clearly taken from 3D-generated scenes (instead of bedroom photographs in LSUN-Bedroom), or from 2D-generated scenes or low quality images (in CIFAR-10). These results show that  $(i)$  humans are more likely to label generated scenes as fake/generated (LSUN-Bedroom, CIFAR-10), and  $(ii)$  humans are more likely to label low-quality images as fake/generated (CIFAR-10). Such images have a higher than average RS, and hence the small correlation between human evaluation and RS on CIFAR10 and LSUN-Bedroom is due to humans properly identifying these dataset issues. We find that removing just  $6\%$  of the "faked" (as measured by humans) real images on LSUN-Bedroom removes the correlation of RS and human error rate - quantitative proof that the small correlation is driven by dataset issues, and not due to humans associating diversity with fakeness. Rare defects in the training set are not enough to affect our results: being so rare  $(6\%)$  means they barely affect the average error rate; and this training-set effect is the same for every generative model evaluation and thus does not change their rankings.
This analysis provides additional validation that our experimental design and human error rate metric are accurately measuring the fidelity of images, and are not effected by extraneous factors such as image unlikelihood/rarity.
D.6 Memorization
In this subsection we include a detailed description of our procedure to  $(i)$  collect memorized samples for generative models trained on CIFAR10, ImageNet, FFHQ, and LSUN-Bedroom,  $(ii)$  summarize a number of controlled experiments to probe the effectiveness of memorization metrics in ideal scenarios, and  $(iii)$  show additional results for memorization metrics in different representation spaces.
Memorization check with calibrated  $l_{2}$  distance In Figure 19, we show the memorized ratio for each of the 13 generative models we evaluated on CIFAR10 using the metrics described in Section 5.2. We also tailored  $k$  - which is the hyperparameter specifying the number of nearest neighbours used to compute the metric - to each dataset. We use  $k = 50$  for CIFAR10, FFHQ, and LSUN, while for ImageNet with its limited examples per class in the training set, we set  $k = 3$ . In addition, we exclusively conduct intra-class nearest neighbour search on ImageNet, as opposed to performing the search across the entire training set. We find that more complex models are more likely to memorize on CIFAR10, whereas less performant models such as ResFLOW and NVAE are less likely to memorize.
Moreover, we provide additional visualization results to further showcase the exact memory exhibited from DDIM on CIFAR10 in Figure 20, and the reconstructive memory exhibited from DiT-XL-2 on ImageNet in Figure 21 and from ADM-dropout on LSUN in Figure 22. Based on Figure 20, the presence of exact memorization is evident, raising concerns that need to be considered on CIFAR10. However, each "memorized" sample found with the above metric in Figure 21 on ImageNet and Figure 22 on LSUN – while not being pixel-wise identical – retains a higher degree of semantic similarity compared to the three alternative matches in the training set. We can also observe that, for some cases, the generated image does not exhibit significantly closer similarity to the matched training sample compared to the training sample's nearest neighbors. While it seems that the model tends to show reconstructive memory on samples that have similar duplicates in the training set, we note that this marks potential failures of the memorization metric, and thus do not report a total number of samples without further work to understand the reasons behind this. This memory can also be interpreted not necessarily as memorization, but as a form of generalization, as the model captures and reproduces common patterns or characteristics present in the training data. Our analysis leads us to conclude that reconstructive memory on complex datasets is not currently a major concern, as out of millions of training images we found a small number of matches. Nevertheless, we re-iterate our stance from the main text that reconstructive memory should continue to be monitored as models themselves become increasingly more complex.
Other than the pixel-wise check, we also revisit the nearest neighbour search in DINOv2 representation space in small scale and find decently-matched results for complex datasets; qualitatively, the results are similar to the pixel-wise check from Figure 21. This could be even further explored in future work. Yet it is still unclear whether or not we have exact memorization - the matched cases mostly occur within classes exhibiting low intra-class diversity, where the samples in training set are extremely similar with each other.
Figure 19: Memorization ratio for 13 generative models on CIFAR10.
D.6.1 Controlled experiment for memorization metrics
To study the response of metrics to memorization we first constructed a controlled experiment where generated samples become increasingly memorized from a realistic set of training images.
Figure 20: Memorized samples from DDIM on CIFAR10. Images are arranged in vertical pairs, where the upper image is generated, and the lower image is a training set image.
We generate the synthetic datasets using the VDVAE architecture [21], which is a hierarchical VAE. Specifically, we used an FFHQ-256 pretrained model, which has 62 layers, and for each training example we sampled from the approximate posterior until a certain layer, and sampled from the prior in later layers. In this manner, going deeper with the approximate posterior corresponds to more closely memorizing the training example. In particular, using depth 0 means sampling from the model's prior, while using the posterior until depth 62 corresponds to a reconstruction that is a very close reproduction of the original image. For a qualitative example of depth and memorization refer to Figure 23.
Formally, a VAE consists of three functions: (i) the generator  $p_{\theta}(x|z)$ , (ii) the approximate posterior  $q_{\phi}(z|x)$ , and (iii) the prior  $p_{\theta}(z)$ , where  $z$  is the latent variable. In the hierarchical setting, it is assumed that  $z = (z_0, z_1, \dots, z_n)$  and the prior and the approximate posterior are factorized as:
$$
p _ {\theta} (z) = p _ {\theta} \left(z _ {0}\right) p _ {\theta} \left(z _ {1} \mid z _ {0}\right) \dots p _ {\theta} \left(z _ {n} \mid z _ {<   n}\right), \tag {15}
$$
$$
q _ {\phi} (z | x) = q _ {\phi} \left(z _ {0} | x\right) q _ {\phi} \left(z _ {1} | x, z _ {0}\right) \dots q _ {\phi} \left(z _ {n} | x, z _ {<   n}\right). \tag {16}
$$
The distribution of the image conditioned on this  $z$  is  $p_{\theta}(x|z)$  i.e. the generator. To sample from this model,  $p_{\theta}(z)$  is used to sample  $z$  and then to sample  $x$  conditioned on  $z$ . This corresponds to our
Figure 21: Reconstructive memorized samples from DiT-XL-2 on ImageNet. Each set of 5 images in a row includes a generated sample, a matched training sample, and three nearest neighbors from the training set associated with the matched training sample.
Figure 22: Reconstructive memorized samples from ADM-dropout on LSUN. Each set of 5 images in a row includes a generated sample, a matched training sample, and three nearest neighbors from the training set associated with the matched training sample.
Figure 23: Left: Qualitative example of synthetic data for the controlled memorization experiment. Top left image is a training sample and the next 5 images correspond to using the approximate posterior up to depth 41, 15, 5, 3, and 0. Right: Values of the  $C_T$  score, AuthPct, and FLS-POG at each depth. The DINOv2 encoder was used for these metrics.
depth 0 setting. To make samples resemble the training set, we can use  $q_{\phi}(z|x)$  instead of  $p_{\theta}(z)$ . The distribution is modified as follows
$$
p (z | x) = q _ {\phi} \left(z _ {0} | x\right) \dots q _ {\phi} \left(z _ {m} | x, z _ {<   m}\right) p _ {\theta} \left(z _ {m + 1} \mid z _ {<   m + 1}\right) \dots p _ {\theta} \left(z _ {n} \mid z _ {<   n}\right). \tag {17}
$$
In terms of sampling, this corresponds to using the training image  $x$  and the approximate posterior to generate  $z_0, z_1, \dots, z_m$  and then using the prior to generate the rest of  $z$ . As  $m$  (which we refer to as depth) increases, the mutual information between  $x$  and  $z$  increases, which in turn results in a more faithful reconstruction.
Using the described procedure, 30,000 samples are generated at each of the following depths: 0 (meaning no conditioning), 1, 3, 5, 9, 15, 25, and 41. We use 63,000 images as the training set and 7,000 as test images; these are the same sets that are used to train and test the VDVAE. We calculate AuthPct,  $C_T$ , and FLS-POG on each set of images and display the results in Figure 23. We find that both  $C_T$  score and AuthPct have characteristics that indicate they can correctly identify memorization