Enhanced Generative Model Evaluation with Clipped Density and Coverage
Authors: Nicolas Salvy
Hugues Talbot
Bertrand Thirion
Venue: ICLR 2026 Poster (2026)
Keywords: Generative model evaluation
Metrics
Density
Coverage
Primary area: generative models

Abstract: Although generative models have made remarkable progress in recent years, their use in critical applications has been hindered by an inability to reliably evaluate the quality of their generated samples. Quality refers to at least two complementary concepts: fidelity and coverage. Current quality metrics often lack reliable, interpretable values due to an absence of calibration or insufficient robustness to outliers. To address these shortcomings, we introduce two novel metrics: $\textit{Clipped Density}$ and $\textit{Clipped Coverage}$. By clipping individual sample contributions, as well as  the radii of nearest neighbor balls for fidelity, our metrics prevent out-of-distribution samples from biasing the aggregated values. Through analytical and empirical calibration, these metrics demonstrate linear score degradation as the proportion of bad samples increases. Thus, they can be straightforwardly interpreted as equivalent proportions of good samples. Extensive experiments on synthetic and real-world datasets demonstrate that $\textit{Clipped Density}$ and $\textit{Clipped Coverage}$ outperform existing methods in terms of robustness, sensitivity, and interpretability when evaluating generative models.

Published as a conference paper at ICLR 2026
ENHANCED GENERATIVE MODEL EVALUATION WITH CLIPPED DENSITY AND COVERAGE
Nicolas Salvy
Hugues Talbot
Inria, CentraleSupélec, Université Paris-Saclay ni nicolas sa ieneeirraa fr
CentraleSupélec, Inria, Université Paris-Saclay hugues ta bbot@centtraleeuuueeee.. ec fr
Bertrand Thirion
Inria, CEA, Université Paris-Saclay bertrand. thirione ieria.iii....mm....
ABSTRACT
Although generative models have made remarkable progress in recent years, their use in critical applications has been hindered by an inability to reliably evaluate the quality of their generated samples. Quality refers to at least two complemen- tary concepts: fidelity and coverage. Current quality metrics often lack reliable, interpretable values due to an absence of calibration or insufficient robustness outliers. To address these shortcomings, Density and Clipped Coverage. By clipping individual sample contributions, as well as the radii of nearest neighbor balls for fidelity, our metrics prevent out- of-distribution samples from biasing the aggregated values. Through analytical degradation
introduce two novel metrics: Clipped
and empirical calibration, these metrics demonstrate linear the proportion of bad samples increases. Thus, they can be straightforwardly in- terpreted as equivalent proportions of good samples. Extensive experiments on synthetic and real-world datasets demonstrate that Clipped Density and Clipped Coverage outperform existing methods in terms of robustness, sensitivity, and in- terpretability when evaluating generative models.
Recall
1.0
Coverage
0.8
symRecall
Relative score
ß-Recall
0.6
TopR
P-recall
0.4
RecallCover
ClippeCooverage (ours)
0.2
Ideal
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Proportion of bad synthetic samples
Figure 1: Measuring the coverage of mixture good and bad samples (CIFAR-10): Various coverage metrics are evaluated relative to their maximum value the proportion of bad synthetic samples increases. Only Clipped Coverage displays the desired linear degradation.
INTRODUCTION
In recent years, remarkable progress has been achieved in generative models, which are being ac- tively explored in various fields, such healthcare (Pinaya et al. 2022; Fernandez et al., 2024 Tu- dosiu et al. 2024: Zhu et al.; 2024; Bluethgen et al. 2024). However, deploying them in staksstakes applications depends reliably evaluating the quality of synthetic data ensure its trustworthiness.

---

Published as a conference paper at ICLR 2026
This evaluation is inherently challenging, especially for nigh-dimensional data. The true underly- ing distributions of this data are often unknown and complex. They also do not conform to known parametric families. These factors make computing the support density infeasible in practice. Current model evaluation often relies on metrics such as Fréchet Inception Distance (FII) (Heusel et al. 2017) and FD-DINOv2 (Stein al. (023) for images. These metrics provide single, com- pound score representing overall sample quality. Thus, it is impossible to determine whether poor performance stems from lack of realism or variety (Sajjadi et al. 2018).
To address this issue, the quality of synthetic data can be broken down into at least two core con- cepts, fidelity and coverage, and measured separately with a pair of metrics. Fidelity metries assess how similar each synthetic sample is to the input data (Naeem et al. (020). Conversely, coverage metries measure the extent to which synthetic samples represent the distribution of real data, taking into consideration how the rarity or commonness of real data is reflected in the synthetic samples. However, a recent position paper by Raisä et al. (2025) argues that all existing fidelity and diversity metrics are flawed, highlighting urgent need for new metrics that address these shortcomings.
To determine whether generative models can be truly dependable, particularly in sensitive applica- tions, the metrics used to evaluate them must be trustworthy. This means that they must be robust, sensitive to genuine deficiencies, and provide interpretable scores. A key challenge is robustness to outliers. Rcal-world datasets often contain out out-offdiitrrbutiin samples such as corruptions, anoma- lies, or simply samples very different from the rest (see Figure 7 for examples in CIFAR-10) 10). Simi- larly, generative models can produce "bad" samples that are far from the real data distribution. These outliers can disproportionately influence evaluation scores, masking true performance issues.
Beyond robustness, interpretability is crucial. Metrics should offer more than just relative compar- isons (i.e., knowing that one model is better than another). As emphasized by Räisä et al. (2025).
for a metric be truly useful in practice, its absolute value must be meaningful, since even the best best-performing model in a comparison might still be of poor quality. Critically, Räisä et al. (2025) notes that, currently, no fidelity or coverage metric offers this property. Ideally, for straightforward interpretability, score of x would indicate performance equivalent to having a proportion of x good samples and (1 - 2) bad ones. However, as shown Figure [ current coverage metrics fail to achieve this. We show in Tigure that current fidelity metrics are similarly untrustworthy.
In this paper, we introduce Clipped Density and Clipped Coverage, two novel metries designed to overcome these limitations. Our contributions are:
Trustworthy Evaluation: Our metrics achieve robustness to outliers by clipping individual sample contributions to the aggregate score and, for Clipped Density, by limiting the radii nearest-neighhor spheres used to measure the density. This prevents outttf-distribution samples from skewing the evaluation while preserving sensitivity to genuine issues.
Interpretable Absolute Scores: Through empirical calibration for Clipped Density and theoretical analysis for Clipped Coverage, we ensure that scores degrade linearly with the proportion of bad samples, providing absolute interpretability.
The paper is structured as follows: Section 2 provides background on existing metrics. Section 3 reviews related work. Section 4 introduces our Clipped Density and Clipped Coverage metries. Section 5 details our experiments and results, and Section 6 discusses implications and limitations.
BACKGROUND
We consider a setting in which we are given N i..dd samples {=])< from unknown data (ref- erence, or real) distribution p,- and M i..dd samples {x)) generated from unknown synthetic data distribution ps. In this section, we review relevant metrics for evaluating generative models that aim to disentangle two aspects of synthetic data: fidelity (how realistic each synthetic sample is) and coverage (how well the synthetic samples populate the real density distribution). These metrics serve the basis for our proposed improvements.
Assuming given supports S" for the real distribution and S for the synthetic distribution, Preci- sion measures the proportion of synthetic samples that fall within S", while Recall measures the
2

---

Published as a conference paper at ICLR 2026
proportion of i.i.d. real samples that fall in S (Sajjadi et al. 2018 Simon al. 2019).
(I) However, the underlying densities d' and d' and their supports S and 5 are unknown, making such computation infeasible. In practice, improved Precision and Recall (Kynkäänniemi et al., 2019) ap- proximate these supports by the union of balls centered at each observed sample, with radius equal to the distance NND; from its center to its k-th Nearest Neighbor (Burman & Nolan 1992). We denote by NND and N the distance to the k-th nearest real and synthetic sample, respectively.
Precision Pp. [S"] = P. (S" ns5*]
Recall = pp. [S*] Pp. 1S" S
M
iPrecision = 1S n nS
M
SEEU B(x (ND)((E))))
(2)
==1
N
iRecall = p. 1S n St =
1 B B(E
(3)
Yet, iPrecision and iRecall are strongly biased by out-of-distributtin samples. Real outliers, being far from their nearest neighbors, create large balls and bias the estimation of S Similarly, bad synthetic samples compromise the approximation of the synthetic support (Naeem et al. 2020).
Density and Coverage (Naeem et al. 2020) aim alleviate this issue by going beyond binary in/out decision. Density counts, for each synthetic sample, how many real k-NN balls it falls within, normalized by k. Thus, outliers with large balls contribute only 1/ to the fidelity of given synthetic sample, instead of for iPrecision (Naeem et al. (020). As approximating density is impractical in high-dimensional spaces, the Density metric instead uses distances through k-NN balls. It represents the average relative density d /d", providing relative measure of how many synthetic points fall within real balls, normalized by their mass k. To avoid estimation bias when approximating the synthetic distribution, which may contain many bad samples, additional considerations are needed. Coverage flips the perspective (Naeem et al. 2020); instead of computing the proportion of real samples within at least one synthetic ball, Coverage calculates the proportion of real samples that covered, by having least one synthetic sample within their ball: Coverage Pp, (Šr n.S*).
M N
N Coverage
Density = AM
1 (NNE)(E))NDD)(EE)
(4)
N
1=1
Although Density is less affected by outliers in the target distribution, it remains influenced by them (Park & Kiml 2023). Additionally, Densiry is not bounded by 1 (Naeem et al.) (2020), making
interpretation challenging when empirical estimates exceed this value (Cheema & Urner] [023 Kim et al. 2023). Similar to iPrecision and iRecall, Coverage is limited to analyzing supports and misses density mismatches: a few synthetic samples can cover high-density regions by lying within many real balls, and adding more synthetic samples there might not increase the score (Park & Kim 2023
3 RELATED WORK
Recent works have aimed to improve these metrics. Precision Cover and Recall Cover, analogous to Precision and Recall, were introduced by Cheema & Urner (2023). They count ball as covered only if it contains at least k' > 1 samples. A probabilistic approach was introduced by Park & Kim 2023): : P-precision and P-recall. These measure the probability that a synthetic (resp., real) sample lies within random sub-support of the real (resp., synthetic) distribution.
To address the problem of large-radius balls, Khayatkhoei & AbdA (023) have employed a nual-perspective approach with symP yymPrecision and symRecall: symRecull is defined as the minimum of Recall and Coverage, while symPrecision is the minimum of Precision and the complementary Precision metric computed from the reversed perspective. Some alternative approaches do not rely on searest-neighbor approximations. a- P-Prccision and 3-Recall (Alaa et al., 2022) employ a one- class approach estimate supports containing a fraction a (resp. (3) of dataset. They measure the proportion of the other dataset found within supports of varying levels. Topological Precision and Recall (Kim et al.] 2023) estimate support via topologically conditioned density kernels.
Recently, Räisä et al. (2025) developed a benchmark of sanity checks for generative model metrics using synthetic data, finding that all existing fidelity and coverage metrics flawed. Our own tests
3

---

Published as a conference paper at ICLR 2026
confirm this conclusion. That work further emphasizes that no current metric is suitable for absolute evaluations, calling for more research in this area. Our work addresses these concerns directly by introducing
based on real-world data (Figures 4 4
metrics that overcome these limitations. We show in Appendix H that our proposed metrics also perform well on their synthetic benchmark.
In the following section, introduce metrics designed to resolve these issues by being i) robust to outliers, and by ii) providing clearly interpretable values.
METHOD
Our proposed metries designed satisfy several key properties. Firstly, they should be robust to outliers (Desideratum 1). Secondly, the metrics should exhibit linear score degradation: if proportion æ of samples bad, the score should decrease by x (Desideratum 2a). The metrics should be normalized between and (Desideratum 2b), allowing their values to be interpreted on their own.
These properties enable straightforward interpretation: a score of x indicates that the synthetic dataset achieves the same fidelity or coverage as dataset composed of proportion x real samples and (1 - z) bad samples. This does not imply that the synthetic dataset is a mixture of good and bad samples, but rather that its fidelity or coverage is equivalent to that of such mixture. score difference of y between two datasets corresponds to y more bad samples in the equivalent scoring scenario.
4.1
CLIPPED DENSITY
4.1.1
A ROBUST FIDELITY METRIC
To measure fidelity, the Densiry metric counts the number of real balls each synthetic sample falls within. This approach can exceed and is vulnerable to outliers, as it relies on adaptive ball radii.
Figure Dillustrates a failure case of the Density metric on synthetic dataset containing a single bad sample (bottom left), with k = 2. In this configuration, the two centered points each receive a fidelity score of 2: while the bad sample obtains fidelity of 0. So, the overall dataset fidelity is computed as 3. (2 x 0) 1: an ideal score. While this is an example in dimension 2, the problem worsens in higher dimensions as the number of balls sample
belong increases (Radovanovic et al. 2010). allowing one over-occurring sample to mask an increasing number of bad synthetic samples.
To prevent over-occurring samples from masking defects, modify the aggregation approach. The intuitive idea is to limit the contribution of each sample to the metric: the fidelity of any synthetic sample should not exceed 1. Applying this modification Figure [ the fidelity score becomes (2 x min(3 2: + 0) 5. which effectively detects the presence of the bad sample.
On the other hand, real outliers can have extremely large distances their nearest neighbors, sulting in balls with significantly larger radii. In dimension d, the volume of ball of radius is proportional
rd. Not only do outliers create much larger balls, but any point in low-density region does so as well, which dramatically skews the metrics. These balls have fixed mass k. but their volume varies. Balls of large volume can contribute disproportionately more than the others. To ensure balanced contributions that limit outlier influence, we clip the radius of each ball to the median of the distances to the k-th nearest neighbor.
Rs (x(x)) min (NND mwdiin (z1))
(5)
This results in the following metric that satisfies Desideratum robustness outliers:
M
N
ClippedDensity ummorm
1
Σ
min
(ejeB(e, .1)
(6)
M
k
j=1
:=1
# (clipped real balls arg is within)
4

---

Published as a conference paper at ICLR 2026
Real
ClippedCoverag@umom
Synthetic data
Theoretical
0.8
ClippedCoverage
0.6
0.4
0.2
3 2 + =1
Bad sample
Density
0.0
0.4
0.8
1.0
CeippedDensity
(2 (2x1+0)
Proportion bad samples
Figure 2: Clipped Density corrects Density's failure: In simple setup with a single bad synthetic sample, Density yields value of 1. By clipping the fidelity of individual samples to 1, we obtain an adjusted score of 3.
Figure 3: Correcting Clipped Coverage lin- ear decay: The unnormalized Clipped Coverage (blue) does not decrease linearly with the propor- tion of bad samples. We thcoretically compute its expected behavior (orange) and correct it (green).
4.1.2 NORMALIZING FOR INTERPRETABILITY
Since ClippedDensity unmoem is an average over synthetic samples, a proportion & of bad samples directly reduces the score by n, satisfying Desideratum 2a. To achieve normalization between and 1 (Desideratum 2b), we require an ideal value of 1. We achieve this ideal value empirically by evaluating the fidelity score of the real data using a leave-one count the number of clipped real balls j (with j & i) that contain it. This computation is efficient, as we have already obtained the indices of real samples inside each ball during the radius computation.
strategy. For each sample x;. we
We normalize ClippedDensity unnorm by the value computed for real data, ClippedDennitty real- and clip the result 1, since scores exceeding would lack meaningful interpretation. The final normalized metric is:
ClippedDensity unnorrn
ClippedDensity min
ClippedDensity....
4.2 CLIPPED COVERAGE
4.2.1 A ROBUST COVERAGE METRIC
To measure coverage, the Coverage metric considers the proportion of real samples whose balls contain at least one synthetic sample. To reflect the real data distribution and not just its support, we adopt an approach similar Clipped Density. Instead of only checking for the presence of synthetic samples inside real ball, we count how many there are. Then, to bound the contribution of each real sample, individual coverage scores are capped at 1. This results in the following formulation:
ClippedCoverage, unmo unnorm
min
Σ
(NDEB(Z,,NDD(4))) .1
(8)
N
K
(synthetic samples within xt's ball)
For each real sample, we compare the mass of synthetic samples in its ball to the mass of real samples. To balance real points' contributions, the mass of each ball fixed: radius clipping is applied. ClippedCoverage satisfies Desideratum of robustness to outliers.
4.2.2 CALIBRATING FOR INTERPRETABILITY
The blue curve in Figure shows ClippedCoverage unmorm when the real and synthetic distributions identical (far left) and bad samples progressively introduced into the synthetic distribution. To satisfy Desideratum 2, the score should follow z for x. the proportion bad samples. To correct the metric's behavior, start by deriving its expected value function of æ..
5

---

Published as a conference paper at ICLR 2026
Lemma 1. If the real and synthetic distributions are identical, i.e. ke and و= N N+M i.dd samples from the same distribution, then the expected value of ClippedCoverage unnorm is:
1 k
M
8(k j. M j+N k)
E [CiippedCoverage [umornn]
min
1
(9)
B N -k)
j=1
where B is the beta function.
The proof of Lemma | is provided in Appendix A To parameterize the curve, we now consider the
case with a proportion x of bad synthetic samples. Since such samples always lie outside any real ball, the expected score becomes equivalent the ideal case in Lemmal | but with Ma [M x)
synthetic samples instead of M. We denote this as Sexpesse((x). (1)). shown in orange in Figure 3
To satisfy Desideratum 2, the expected value of the metric should be 1 bad samples is z. This calibration ensures both a linear degradation of the score (Desideratum 2a)
when the proportion of
and normalization to a consistent [0, 1] range regardless of the dataset or choice of k (Desideratum 2b). We seek a function such that eexpected (T) = 1 I. Since M2 can only take integers values m between and M. suffices to find g for fexpected MM, m) where m E {O....,.T. M..
We can efficiently compute fexpoeted (Mg = m) for all m (see Appendix B.1 Given these values, the function g is computed numerically. Since fexpecsed decreases with u, we reverse it form sorted list of fexpected values. For a given ClippedCCoverggeunen
score 8, we find the index i(s)
{O..... M) such that inserting the value a index i(s) keeps the list sorted. Then, g(s)
d(s)
M
The final normalized metric, which recovers the desired behavior shown in green in Figure 3 is:
ClippedCoverage g ClippedCCovergge.moe
(10)
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
We compared our metrics against multiple baselines, using the original code for a-Precision, 3- Recall, and TopP&R, and reimplementing Precision, Recall, Density, Coverage, symPrecision, symRecall, P-precision, P-recall, and Precision Recall Cover for performance reasons (see Ap- pendix B.2 for comparisons). For consistency with previous works, we set k = 5 where applicable; k' 3 and C 3, as recommended for Precision Recall Cover, and used default parameters everywhere else.
Experiments were conducted single NVIDIA H100 GPU with 80GB of RAM. When images are evaluated, metrics are computed on image data in the embedding space of the "Large" DINOv2 model (DINOv2-ViT-L) (Oquab et al., 2023)), ) recommended by Stein et al. (203
5.2 METRIC EVALUATION TESTS
To ensure proper behavior, we evaluated metrics in various scenarios by controlling dataset compo- sition and hence the expected scores. Most tests were performed on CIFAR-I0 (Krizhevsky al., 2009) with 2500 samples from each of the 10 classes for the real set and the other 2500 for the synthetic set. We report mean std over 10 splits. Results are summarized Figures and 5c
CIFAR-10 Simultaneous mode dropping: To verify that fidelity metrics do not capture coverage, we performed a simultaneous mode dropping test on CIFAR-10 (Figure 4b] from |Neem et al. 22200). In this test, the synthetic set progressively replaced samples from all but one class with samples from the remaining class, while the real set remained unchanged. number of samples per set, only 2500 samples were used in this test, with results reported as mean t std over 100 splits. Since the synthetic set is always a subset of CIFAR-10 10, fidelity scores should remain stable and close their maximum. However, Precision Cover (yellow) and symPrecision (green) appear to capture coverage, as they deviate from their maximum value.
maintain a constant
CIFAR-10 matched real and synthetic out-of-distribution sample proportion: To evaluate bustness to out-of-distmin diitt---iitiittiion samples, we conducted test on CIFAR-10 10 (Figures 4c and [5a] in- spired by Figure 5 of Naeem et al. (2020)). where we progressively replaced both real and synthetic

---

Published as a conference paper at ICLR 2026
1.0
0.8
Relative scare
0.6
Relative
0.4
0.2
0.2
0.0
0.0
0.6
0.8
0.0
0.2
0.4
Proportion of bad synthetic samples
Proportion of data kept per mode
(a) Fidelity reduction with bad samples (CIFAR-IP
(b) Simultaneously dropping modes (CIFAR-10)
1.2
1.02
Relative scare
1.00
Relative
0.6
0.98
0.4
0.2
0.94
0.0
0.00
0.01
0.02
0.03
-1.0
0.5
Proportion of bad samples
(c) Introducing bad real & syn. samples (CIFAR-10)
(d) Translating Gaussian with bad samples
Legend
Figure 4a
Figure 4b
Figure
Figure
4d
Precision
X
Density
symPrecision
/
/
a-Precision
/
x
x
TopP
x
P-precision
x
PrecisionCover
X
V
/
ClippedDensity (ours)
Ideal
(e) Legend and summary
Figure 4: Testing fidelity metries (legend/summary in (e)). Scenarios: (a) increasing bad synthetic sample proportion; (b) simultaneous mode dropping: progressively replacing all but one class with the last class; (c) matched real & synthetic aattof-distribution samples at equal rates; (d) synthetic distribution translation with one real outlier and one bad synthetic sample. Only Clipped Densiry consistently behaves as expected: linearity (a), stability (b, c), and symmetry with sensitivity (d).
CIFAR-10 samples with out-of-distribution samples (noise images) the same rate. Since the syn- thetic set is constructed identically to the real set in all cases, scores should remain stable and close their maximum value. However, a-Precision (purple). TopP and TopR (brown), and P-precision and P-Recall (gray) show instability and deviate from the expected value.
Synthetic data translation:
Since real-world datasets like CIFAR- 10 contain outliers, we use test based on synthetic data Friguee/4dJandd adapted from Naeem et al. (2020)) to evaluate robustness to the first out-of-distribution sample. We use 25000 standard Gaussian samples (dim 32, 5 splits). The synthetic set is translated by [-1, 1| in all dimensions and includes a bad sample -3. The real set includes an outlier at 3. Ideally, scores should form symmetric bell-shaped curves, dropping rapidly moves away from 0 detect distribution shifts. For fidelity (Figure 4d), Precision (blue) and Density (orange) are non-symmetric due to the real outlier's 's large ball, which in turn affects symPrecision (green). o-Precision (purple) and P-precision (gray) show low sensitivity (flat

---

Published as a conference paper at ICLR 2026
1.075
1.2
1.050
Relative score
1.025
Relative
1.000
0.4
0.975
0.2
0.950
0.0
0.00
0.03
0.5
Propartion of bad samples
(a) Introducing bad real & syn. samples (CIFAR-10) 10)
(b) Translating syn. Gaussian with bad samples
Legend
Figure
Figure 5a
Figure 5b
Recall
X
X
Coverage
X
symRecall
X
B-Recall
X
/
x
TopR
X
P-recall
X
RecallCover
X
ClippedCoverage (ours)
Ideal
(c) Legend and summary
Figure 5: Testing coverage metrics (legend/result summary in (c)). Metrics evaluated under different scenarios: (a) matched real & synthetic out-of-distribution samples; (b) distribution trans- lation with real outlier at 3 and bad synthetic sample -3. Clipped Coverage exhibits all desired properties:
linearity (Figure Th. stability (a), symmetry and sensitivity (b), unlike other metrics.
around 0). For coverage (Figure(5b) Recall (blue) is non-symmetric due the bad synthetic sample, which also affects symRecall (green). This happens because the bad sample's 's ball grows as the rest of the synthetic data moves away, covering the real set when f is near 1. 3-recall (purple) and TopR (brown) are also non-symmetric, indicating instability or insufficient robustness.
CIFAR-10 Progressive bad sample introduction: To test sensitivity and interpretability. we pro- gressively introduced bad samples (noise images) into synthetic set of CIFAR 10 images (Figures 4ad Scores should decrease linearly with the proportion of bad samples (see Section(4). For cov- erage metrics, only Clipped Coverage (red) exhibited this linear degradation. Most fidelity metrics, being averages of individual sample fidelities, decreased linearly. However, TopP (brown) deviated significantly, while Precision Cover (yellow) and a- a-Precision (purple) showed slight deviations.
Summary: Across results (see also Figures 4e and [c), only Clipped Density and Clipped Cov- eruge show the desired behavior in all tests. Other fidelity metrics either inappropriately capture coverage (c.g., symPrecision, Precision Cover), lack sensitivity (c.g.. Precision, P-precision). or lack robustness to outliers (c.g.. Precision, Density, TopP). For coverage metrics, only Clipped Cov- erage shows a linear decrease in score with increasing bad sample fraction, while some metrics are also insufficiently robust outliers (e.g., Recall, symRecall, 3-Recall, TopR, P-recall).
5.3 EVALUATION ON REAL DATASETS
We evaluated various generative models CIFAR-10 (Krizhevsky al.) 2009). ImageNet (Deng
et al., 2009), LSUN Bedroom (Yu et al. 2015). and FFHQ (Kazemi & Sullivan 2014). using cat- egories and 50000 samples from the data publicly shared by Stein al. (2023) (see Figure 6 and App When possible, for conditional models, we kept an equal number of samples from each class (see Appendix C or Appendix A of Stein al. (2023) for more details). comparison with
8

---

Published as a conference paper at ICLR 2026
Consistency
Diffusion
GAN
Normalizing Flow
Transformer
VAE
CIFAR-10
ImageNet
LSUN Bedroom
FFHQ
ClippedCoven oveeage (ours)
1.0
1.0
1.0
1.0
0.8-
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.0
0.0
0.0
0.0
0.0
0.2
0.4
0.6 0.8 1.0
0.0
0.2 0.4
0.6
0.8 1.0
0.0
0.2 0.4 0.6 0.8 1.0
0.0
0.2 0.4
0.6 0.8 1.0
ClippedDensity (ours)
ClippedDensity (ours)
ClippedDensity (ours)
SippedDensity (aurs)
1.0
1.0
1.0
1.0
Coverage
0.6
0.6
0.6
0.6
0.4
0.2
0.2
0.2
0.0
0.0
0.0
0.0
1
2
0.0
0.5
1.0
1.5
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2 0.4 0.5
0.8
1.0
Density
Density
Density
Density
LSGM-ODE
ADM
Consistency-seti Consistency-set2 ADM-dropout DDPM
LDM
PFGMPP
ADMG
InsGen
IDDPM-DDIM
ADMG-ADMU
Projected-GAN
ACGAN-Mod
DIT-XL-2
StyleGAN-XL
BigGAN-Dece
DiT-XL-2-guided
iDDPM
StyleGAN2-ada
LOGAN
LDM
Diff- Biff-ProjGNN
StyleNAT
MHGAN
BigGAN
Projected-GAN
StyleSwin
ReACGAN
GigaGAN
StyleGAN
Unleash-Trans
StyleGAN- XL
StyleGAN-XL
Unleash-Trans
Efficient-vdVAE
StyleGAN2-ada
Mask-GIT
WGAN-GP
RO-Transformer
RESFLOW
NVAE
Figure 6: Fidelity vs Coverage on various datasets
human evaluations in Appendix G confirms that Clipped Density and Clipped Coverage align well with human judgment.
As shown in Figure 6 on CIFAR-10 and ImageNet, the Density values exceed 1, while the Clipped Density values remain stable. In Appendix D analyze RESFLOW--generate CIFAR-10 data, demonstrating that its inflated Density score of 2.47 is driven by real out-of ciit-o--distrbbution samples. Additionally, Appendix [ provides a step-by-step evaluation of Clipped Density on the generated datasets, quantifying the impact of each modification to the original Density metric.
Across all datasets, our results consistently show diffusion models outperforming GANs. The range of values observed for our metrics appears to reflect the training dataset size: CIFAR-10 (50k sam- ples, max score Ro 0.4), FFHQ (70k, max 0.4), LSUN Bedroom (1.5M, max 0.7), and ImageNet (14M, max 1.0). Interpreting the absolute scores, value of 0.4 for CIFAR-10 10 and FFHQ suggests that, on these datasets, top generative models achieve results equivalent to only 40% of good samples and 60% bad samples. This highlights substantial room for improvement, insight only possible because the absolute values of Clipped Density and Clipped Coverage are interpretable.
This absolute interpretability also allows us assess models without requiring reference model for comparison, as demonstrated on Chest X- X-Rays in Appendix F
Furthermore, using GAN- -eA-ggeeeated data with varying truncation parameters, show in Appendix that Clipped Density and Clipped Coverage effectively capture the expected trade-off between fi- delity and coverage. We found Clipped Density and Clipped Coverage to be robust the choice

---

Published as a conference paper at ICLR 2026
of & (Appendix D and empirically studied their stability as function of the sample size, showing standard deviation proportional to 1/vN (Appendix MM. Additionally, tested the metrics in diverse contexts, including imputed distortions (Appendix[]) failure scenarios such as missing rare modes (Appendix O)) and other modalities like music data (Appendix PP. Finally, samples from the datasets can be visually inspected in Appendix
6 DISCUSSION AND CONCLUSION
Clipped Density and Clipped Coverage offer significant improvements in robustness, sensitivity, and interpretability. These improvements result from several key design choices. To enhance robustness, we cap individual sample contributions at 1. For Clipped Densily, we further mitigate outlier impact by clipping the volumes of real nearest-neighhor balls to prevent them from dominating the score. In contrast, for Clipped Coverage, we maintain balls of constant mass to ensure balanced contribu- tions. Capping the contribution prevents high-performing samples from disproportionately hiding low-performing ones. This careful handling of sample contributions and ball properties enhances robustness while preserving sensitivity.
For interpretability,, both metrics are calibrated to degrade linearly as the proportion of bad samples increases. Clipped Density achieves this through normalization by the real data's s score. For Clipped Coverage, whose uncalibrated behavior is non-linear (as shown in Figure 3 we theoretically de- rived its expected behavior and applied a correction to ensure linearity. This combination of robust aggregation and principled calibration allows Clipped Density and Clipped Coverage to provide not only trustworthy relative comparisons between models but also meaningful absolute scores. This is crucial in practice for assessing whether model meets a required quality threshold.
Despite these improvements, limitations remain. Clipped Density and Clipped Coverage build on an accepted benchmark of progressively passed tests, but it is not exhaustive, and there might be missing cases. Furthermore, we did not provide a theoretical analysis of the metrics in the infinite sample limit. Additionally, while we evaluate fidelity and coverage, other aspects matter, such as memorization: are generated samples reproductions of training data? Memorization metrics include GAN-test (Shmelkov al., 2018), identifiability YYon et 2020). authenticity (Alaa al. 2022). and the calibrated 12 distance (Carlini et al. 2023). We used DINOv2 as recommended (Stein et al., 2023), but comparing embedding models for generative model evaluation remains an open question.
In conclusion, Clipped Density and Clipped Coverage provide a trustworthy framework for genera- tive model evaluation, offering robust, sensitive, and interpretable assessments.
Code to reproduce the experiments and use Clipped Density and Clipped Coverage is available at
ACKNOWLEDGMENTS
This work benefited from state aid managed by the Agence Nationale de la Recherche under the France 2030 programme, reference ANR-22-PESNNNN 21--22---N-0222 and from the European Union's 's Horizon 2020 Research Infrastructures Grant EBRAIN- 101058516. This work was performed using HPC resources from GENCI-IDRIS (Grant
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Ahmed Alaa, Boris Van Breugel, Evgeny S Saveliev, and Mihaela van der Schaar. How faithful is your synthetic data? sample-level metries for evaluating and auditing generative models. In International Conference Machine Learning, pp. 290-306. PMLR, 2022.
Christian Bluethgen, Pierre Chambon, Jean-Benoit Delbrouck, Rogier van der Sluijs, Malgorzata Polacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P Langlotz, and Akshay S Chaudhari. A vision-languar foundation model for the generation of realistic chest x-ray images. Nature Biomedical Engineering, pp. 1-13, 2024.
Sam Bond-Taylon Peter Hessey, Hiroshi Sasaki, Toby P Breckon, and Chris G Willcocks. Un- leashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high- resolution image generation from vector-quantized codes. In European Conference on Computer Vision, pp. 170-188. Springer, 2022.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. 7th International Conference Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Burman and D Nolan, Location-adaan adaptive density estimation and nearest-neighbor distance. Jour- nal of Multivariate Analysis, 40(1):132-155 1992.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253-5270, 2023.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1 11315-11322 2022.
Fasil Cheema and Ruth Urner. Precision recall cover: A method for assessing generative models. In International Conference Artificial Intelligence and Statistics, pp. 6571 1571-6594. PMLR, 2023.
Ricky TQ Chen, Jens Behrmann, David K Duvenaud, and Vörn-Henrik Jacobsen. Residual flows for invertible generative modeling. Advances in Neural Information Processing Systems, 32, 2019.
Michaël Defferrard, Kirell Benzi, Pierre Vandergheynst, and Xavier Bresson. FMA: dataset for music analysis. In Sally Jo Cunningham, Zhiyao Duan, Xiao Hu, and Douglas Turnbull (eds.), Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017, Suzhou, China, October 23-27, 2017, pp. 316-323, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255 leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8799 2021.
Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. CLAP learning audio concepts from natural language supervision. In IEEE International Conference on Acous- fies, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023, pp-
1-5. IEEE, 2023.
Paul Embrechts and Marius Hofert. A note on generalized inverses. Mathematical Methods of Operations Research. 77:423 432, 2013.
Virginia Fernandez, Walter Hugo Lopez Pinaya, Pedro Borges, Mark Graham, Petru-Daniel Tu- dosiu, Tom Vercauteren, and M Jorge Cardoso. Generating multi paltiipptholooical and mudti-modal images and labels for brain mri. Medical Image Analysis, pp. 103278, 2024.
James E Gentle. Computational statistics, volume 308. Springer, 2009.

---

Published as a conference paper at ICLR 2026
Azalea Gui, Hannes Gamper, Sebastian Braun, and Dimitra Emmanouilidou. Adapting frechet audio distance for generative music evaluation. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, April 14-19. 2024, pp- 1331-1335. IEEE, 2024.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
Louay Hazami, Rayhane Mama, and Ragavan Thurairatnam. Efficient-vdvae: Less is more. arXiv preprint arXiv:2203.1377 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 68688, 2020.
Qiang Huang and Anthony KH Tung. Lightweight-yer-effiient Revitalizing ball-tree for point- to-hyperplane nearest neighbor search. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pp. 436 449. IEEE, 2023.
Marco Jiralerspong, Joey Bose, lan Gemp, Chongli Qin, Yoram Bachrach, and Gauthier Gidel. Feature likelihood divergence: evaluating the generalization of generative models using samples. Advances in Neural Information Processing Systems, 36:33095-33119, 2023.
Minguk Kang, Woohyeon Shim, Minsu Cho, and Jaesik Park. Rehooting acgan: Auxiliary classifier gans with stable training. Advances in neural information processing systems, 34:23505-23518 2021.
Minguk Kang, Joonghyuk Shin, and Jaesik Park. Studiogan: a taxonomy and benchmark of gans for image synthesis. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(12): 15725-15742, 2023a.
Minguk Kang, Jun-Jan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEF/CVE conference computer vision and pattern recognition, pp. 10124-10134, 2023b.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. In 6th International Conference on Learning Representa- tions, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceed- ings. OpenReview.net, 2018
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern 2019.
recognition, pp.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in neural information processing systems, 33:1 22:12204-1222 121144 2020.
Vahid Kazemi and Josephine Sullivan. One millisecond face aligmment with an ensemble regres- sion trees. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1867-1874, 2014.
Mahyar Khayatkhoei and Wael AbdAlmageed. Emergent asymmetry of precision and recall for measuring fidelity and diversity of generative models in high dimensions, In International Con- ference Machine Learning, pp. 16326-16343, PMLR, 2023.
Pum Jun Kim, Yoojin Jang, Jisu Kim, and Jaejun Yoo. Topp&r: Robust support estimation ap- proach for evaluating fidelity and diversity in generative models. Advances Neural Information Processing Systems, 36:7831-77888 2023.
12

---

Published as a conference paper at ICLR 2026
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11523-11523 1532, 2022.
Pablo Lemos, Sammy Nasser Sharief, Nikolay Malkin, Salma Salhi, Connor Stone, Laurence Per- reault Levasseur, and Yashar Hezaveh. Pqmass: Probabilistic assessment of the quality of gener- ative models using probability mass estimation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.
Shuangqi Li, Chen Liu, Tong Zhang, Hieu Le, Sabine Süsstrunk, and Mathieu Salzmann. Control- ling the fidelity and diversity of deep generative models via pseudo density. Trans. Mach. Learn. Res., 2024, 2024.
Ting Liu, Andrew W Moore, Alexander Gray, and Claire Cardie. New algorithms for efficient high- dimensional nonparametric classification. Journal of machine learning research, 7(6). 2006.
Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In International Conference on Machine Learning, pp. 7176-7185. PMLR, 2020.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171. PMLR, 2021.
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with aux- iliary classifier gans. International conference on machine learning, pp. 2642-2651 22511 PMLR, 2017.
Stephen M Omohundro. Five balltree construction algorithms. 1989.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprini 03Xiv:2304.07193, 2023
Richard Osuala, Grzegorz Skorupko, Noussair Lazrak, Lidia Garrucho, Eloy Gareía, Smriti Joshi, Socayna Jouide, Michael Rutherford, Fred Prior, Kaisar Kushibar, et al. medigan: a python library of pretrained generative models for medical image synthesis. Journal of Medical Imaging, 10(6): 061403-061400 403,, 2023.
Dogyun Park and Suhyun Kim. Probabilistic precision and recall towards reliable evaluation of generative models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 20099-20109, 2023.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-leam: Machine learning in python. the Journal of machine Learning research, 12:2825- 28300, 2011.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEE/CVF international conference on computer vision, pp. 3195-4205. 2023.
Walter HI Pinaya, Petru-Daniel Tudosiu, Jessica Dafflon, Pedro F Da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, and M Jorge Cardoso. Brain imaging generation with la- tent diffusion models. In MICCA/ Workshop on Deep Generative Models, pp. 117-126. Springer, 2022.
Milos Radovanovic, Alexandros Nanopoulos, and Mirjana Ivanovic. Hubs in space: Popular nearest neighbors in high- diigh--insiional data. Journal of Machine Learning Research, 11(sept)2487-25555 2010.
13

---

Published as a conference paper at ICLR 2026
Ossi Räisä, Boris van Breugel, and Mihaela der Schaar. Position: All current generative fidelity and diversity metrics are flawed. In Forty-second Interaational Conference on Machine Learning Position Paper Track, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly, Assessing generative models via precision and recall. Advances in neural information processing systems, 31, 2018.
Axel Sauer, Kashyap Chitta, Jens Müller, and Andreas Geiger. Projected gans converge faster. Advances in Neural Information Processing Systems, 34:17480-17492, 2021.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pp- 1-10, 2022.
Bradley Segal, David M Rubin, Grace Rubin, and Adam Pantanowitz. Evaluating the clinical realism of synthetic chest x-rays generated using progressively growing gans. SN Computer Science, (4):321,200)
Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. How good is my gan? In Proceedings of the European conference on computer vision (ECCV), pp. 213-229, 2018.
Loic Simon, Ryan Webster, and Julien Rabin. Revisiting precision and recall definition for genera- tive model evaluation. arXiv preprint 1rXiv:1905.05441, 2019.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. International Conference on Machine Learning, 2023.
George Stein, Jesse Cresswell, Rasa Hosseinzadch, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, Eric Taylor, and Gabriel Loaiza-Ganee Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Processing Systems, 36, 2023.
Retru-Daniel Tudosiu, Walter HL Pinaya, Pedro Ferreira Da Costa, Jessica Dafflon, Ashay Patel, Pedro Borges, Virginia Fernandez, Mark S Graham, Robert J Gray, Parashkev Nachev, et al. Realistic seaphoooy--peservving generative modelling of the brain. Nature Machine Intelligence, pp. 1-9, 2024.
Ryan Turner, Jane Hung, Eric Frank, Yunus Saatchi, and Jason Yosinski. Metropolis-hasting gen- erative adversarial networks. In International Conference on Machine Learning, pp. 6345-6353. PMLR, 2019.
Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in neural information processing systems, 33:19667-19679, 2020.
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in neural information processing systems, 34: 12117-1 2021.
Steven Walton, Ali Hassani, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Stylenat: Giving each head a new perspective. arXiv preprint arXiv:2211.05770 2022.
Xiaosong Wang. Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Sum- mers. Chestx-ray8: Hospital-scal chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2097-2106. CVPR, 2017.
Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion- gan: Training gans with diffusion. In The Eleventh International Conference on Learning Repre- sentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
14

---

Published as a conference paper at ICLR 2026
Yan Wu, Jeff Donahue, David Balduzzi, Karen Simonyan, and Timothy Lillicrap. Logan: Latent optimisation for generative adversarial networks. arXiv preprint arXiv:1912.00953, 2019.
Yilun Xu, Ziming Liu, Yonglong Tian, Shangyuan Tong, Max Tegmark, and Tommi Jaakkola. Pfgm++: Unlocking the potential of physics-inspired generative models. In International Con- ference on Machine Learning, pp. 38566-38591. PMLR, 2023.
Ceyuan Yang, Yujun Shen, Yinghao Xu, and Bolei Zhou. Data-efficient instance generation from instance discrimination. Advances in Neural Information Processing Systems, 34:9378-9333
2021.
Jinsung Yoon, Lydia N Drumright, and Mihaela Van Der Schaar. Anonymization through data synthesis using generative adversarial networks (ads-gan). IEEE journal of biomedical and health informatics, 24(8):2378-25) 2388, 2020.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 03366,, 2015.
Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Bain- ing Guo. Styleswin: Transformer-bace gan for high-resolution image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11304-11314 2022.
Xi Zhu, Wei Zhang, Yijie Li, Lauren O' Donnell, and Fan Zhang. When diffusion mri meets diffu- sion model: A novel deep generative model for diffusion mri generation. In International Confer- ence on Medical Image Computing and Computer-Assisted Intervention, pp. 530-540. Springer, 2024.
15

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2
Background
3
Related Work
Method
Clipped Density
4.1.1
A Robust Fidelity Metric
4.1.2
Normalizing for Interpretability
4.2
Clipped Coverage
4.2.1
A Robust Coverage Metric
4.2.2
Calibrating for Interpretability
5
Experiments
5.1
Experimental Setup
6
Metric Evaluation Tests
5.3
Evaluation on Real Datasets
6
Discussion and Conclusion
10
Proof of Lemmal
18
Implementation details
19
B.1
Clipped Coverage
19
B.2 Comparison with existing metric implementations
19
Computational requirements
20
Datasets
Real data for metric eval Ivaluation tests
21
Generated datasets
C.2.1
CIFAR-10
21
C.2.2
ImageNet
................................
C2.33
LSUN Bedroom
22
C.2.4
FFHQ
22
Example of real -ut-t--distribution samples inflating Density
23
From Density to Clipped Density: over-occurring sample proportion and radii clipping
24
Evaluation of Generated Chest X-Rays
25
16

---

Published as a conference paper at ICLR 2026
Comparison to Human Evaluations
26
Comparison to a Recent Benchmark
27
H.T Harsh criteria
28
Incorrect criterion
29
Fidelity vs Coverage with other metrics
30
J Fidelity and Coverage trade-off: Truncation in GANs
32
K Clipped Density scores exceeding
34
Sensitivity the hyperparameter k
36
M Calibration Stability
37
Fidelity under Imputed Distortions
39
Other failure modes
Other modalities
42
P.I Toy time series example
42
P22 Music dataset example
43
Q Metric tests on toy datasets
44
Fidelity metries
44
Q.2 Coverage metrics
45
Unnormalized results
46
R.
Mixture of good and bad samples (CIFAR-10)
46
Simultaneous mode dropping (CIFAR-10)
47
R3 Matched real & synthetic out-of distributtii samples (CIFAR- 10)
47
R.4
Synthetic distribution translation
48
Visual inspection of samples
49
17

---

Published as a conference paper at ICLR 2026
A
PROOF OF LEMMA I
Proof.
IE
min
h
1
B
min
2jEB(2 (eEB(er,NDD((F))) (NDD 1
E
min
Σ 1 1eE k
k
(symthetic samples zi hall)
E
NNN (eN.
many k-nearest synthetic samples of an are within its ball
k
(NN E B NND
K
We apply the linearity expectation lines 1, 2, and 4, and use the i..dd hypothesis for the of in line I. For line 3, we use the fact that the number of synthetic samples within the ball of æf. when limited to k. is equal to the number of synthetic samples among the k-nearest synthetic neighbors of 24 that fall within that ball. Here, NN, denotes the k-th nearest real data sample, while NN; denotes the k'tth nearest synthetic data sample.
S" || =1l)) be the set of real distances to æ;, and let S be the k-th order statistic of S be the set of synthetic distances 2T. and let Sğ be the k-th order statistic of S* St = NN (xf) zill-
(i.e., the k-th smallest element): St ||NNE (xr) =Ill- Similarly, let 5 = {110 =i )
Let Ct2y be the number of synthetic samples contained in the k-ball of of.
PP (NN; ( (xT) B(T,, ( PP ( (xf) xfll [NNE(=])-
= PP (Sp < 5)
= P (K' S Ct.)
S% divides the population into two parts: elements S ST. and N 1- elements >> Sr. Since the real and synthetic distributions are the same, for fixed S C follows binomial distribution with parameters: number of trials M and probability success equal to F(S where is the CDF of F(ST)
the random variable ||X-
xII.. with X Pr. Thus, CHIST
Since ST. is random, so F(ST). For any continuous distribution Y, F(Y) is uniform (Embrechts) & Hofert 2013). Because the CDF is monotonically increasing, F(SE), the CDF of the le-th order statistic of S", is the k-th smallest element the set F(ST): F(S F(ST). The k-th order statistic of uniform distribution follows Beta distribution (Gentle 2009), so have: F(S)) Beta(k; k; (N - 1) - + 1) ~ Beta(k, N -)).
When the success probability of a binomial distribution is itself random variable follow- ing a Beta distribution, the resulting distribution a Beta-Binomial M, N k).
distribution: C
Let & be the beta function:
M Σ j=k
M
3(k + J.M -
PPWK < Ct)
3(k, N k)
18

---

Published as a conference paper at ICLR 2026
& P (T1) E NND, =
Σ
k
M
M
3(k + M - -
h
B(k, N - -k)
P=1
j=
ス M
トーン Σ
min(j.))
(M
3(k + J. M - -j+ k)
B(k; N k)
Σ min b 1
M
(
(M
3(k + j, - -j - K)
B(k, N -k)
B IMPLEMENTATION DETAILS
B.1 CLIPPED COVERAGE
To compute fexpected efficiently for all m, we begin by precomputing log-gamma values for all integers / between 1 and M + N + 1. Then, all required beta values and binomial coeffi- cients can be computed as differences of three precomputed log-gamma values: log -Beta b) =
log -aamma(a) (a) + log Gammaab) log Gamma(a + b) and log = log Gaamaa + 1).
log -og--ammakk + 1) log Gammaa k + 1).
For the computation of Clipped Coverage, we take an approach similar to that used in the proof of Lemma [ we count how many of the k-closest synthetic samples to a real sample are contained within its ball. This can be computed using only nearest neighbor searches, which is more efficient than searching for all synthetic samples within the ball of each real sample.
B.2 COMPARISON WITH EXISTING METRIC IMPLEMENTATIONS
We compare implementation with the implementations of:
Precision, Recall, Density, and Coverage by Naeem et al. 2020) https:
//github. con/ caam/clovasse prde/prde.pp http://llvaaaeeeeaaaiiiae eva Lua.iinnninnnnn weenn--rr//// / blbblloo. ob /aster/
symPrecision and symRecall by Khayatkhoei & AbdAlmageed
2023
https:
y
asymmetry_ http:pppbllloo /main/
mani nan.fffettriccnaniieerrr.......
P-precision and P-recall by Park & Kim (2023) using GPU dtttps::////gtttbbboom com/
pp op_pr.00 pr.py)
Our own implementation of Precision Recall Cover, following the pseudocode in Appendix A.3 of the original paper (Cheema & Urner 2023)
Our implementation simultaneously computes Precision, Recall, Density, and Coverage, with op- tional computation of symPrecision, symRecall, P P-precision, and P-recall. Since some intermediary computations are shared between these metrics, we compute them once and reuse them. We compute Precision Recall Cover separately because the k value used differs from the other metrics (k' 3 instead of l: = 5, see Section 5).
In the original implementations, the number of parallel threads is sometimes hardcoded to 8, so for fair comparison, we use only of the 24 threads available on our hardware, which is an H100 GPU with 80GB of RAM (as stated in Section
We run the implementations on sets of 10000 standard Gaussians in dimension 32 and on the DI- NOv2 embedding of FFHQ with the synthetic set generated by the Latent Diffusion Model (LDM), hereafter
for a total of 50000 samples in both the real and synthetic sets. The results shown in Table
19

---

Published as a conference paper at ICLR 2026
Table
Implementation comparison
10000 standard Gaussians, d = 32
DINOV2-FFHO-ILDN
Values
Time (s)
Values
Time (min)
Metric name
Initial
Ours
Initial
Ours
Initial
Ours
Initial
Ours
Precision
0.7706
0.7706
0.7352
0.7352
Recall
0.7764
0.7764
0.3969
0.3969
3,93
2.05
Density
0,9591
0,9591
0.6475
0.6475
Coverage
0.9645
0.9645
0.5817
0.5817
8.43
25.2
symPrecision
0.7706
0.7706
0.3267
0.3267
17.3
157
symRecall
0.7764
0.7764
0.3969
0.3969
P-precision
0.9251
0.9251
DNF
0.7229
1.66
DNF
P-recall
0.9249
0.9249
DNF
0.6896
Precision Cover
0.9553
0.9553
55.7
2.23
0.1785
0.1785
88.2
2.90
Recall Cover
0.9357
0.9357
61.0
2.25
0.4637
0.4637
67.9
2.99
When both the original and our implementations finish, we obtain the exact same values. The original implementation of P-precision and P-recall does not finish (DNF) on the FFHQ test due to memory issues.
The main difference between our implementation and the existing ones the use of Scikit- learn's (Pedregosa et al. 2011) NearestNe ghbors method for the nearest neighbor search. We also use for ball tree radius queries, which find all samples within given radius of a specific point.
For simplicity, we assume here that the number of real and synthetic samples the same: N M. Ball trees are built in OddN log N) time and O(Na) space (Omohundro) 1989 Huang Tung 2023) for N samples in dimension d, while queries take O(d N) operations in low dimensions and up (ddN) time in high dimensions (Liu et al., 2006) (note that this query is performed N times).
In contrast, initial implementations usually compute the distances between all pairs of samples and search through these distances instead. The construction complexity is then (dd22) time and O(N2) space. Finding all points within given radius of specific point can then be done in ((N) time because the distances are already computed.
Our method therefore has at most the same time complexity as the initial implementation, but with memory usage of O(Nd) instead of O(N²).
For DINOV2-FFHO-LDM, where N 50000 and d 1024, the original P-precision and P-recall implementation fails due to out-of-memory errors, as it attempts to allocate an additional 24GB of RAM on top of the 73GB already in use (73 24 + 1). This implementation stores four pairwise distance matrices, whereas prdc. py (for Precision, Recall, Density, and Coverage) stores only one.
B.3 COMPUTATIONAL REQUIREMENTS
The approximate computation times for the experiments were as follows:
Extracting DINOv2 embeddings took 3.7 minutes per dataset. With 46 datasets in total (14 + 12 10 10), this amounted to 2.8 hours.
For synthetic data tests (25000 samples, dimension d = 32), each evaluation of all metrics took 8 minutes. The total computation time for these tests, encompassing various scenarios and repetitions (21 x 5 + (11 + 11/2 4) x 10), was 41 hours.
For real tests using DINOv2 embeddings of CIFAR-10 (25000 samples, dimension d 1024), each evaluation of all metrics took 13.7 minutes. The total computation time for these tests ((11 11/2 2//2//22 + 4) 10) was 47 hours.
20

---

Published as a conference paper at ICLR 2026
The evaluation of generative models real datasets took 40 minutes per generated set. With 42 generated sets evaluated (13 + 11 9), this totaled 28 hours.
Overall, reproducing all the experimental results presented in this paper would require approxi- mately 120 hours of computation time. The complete research project, including preliminary ex- periments and explorations not detailed in the final paper, required an estimated 200-300 hours of computation time.
C DATASETS
C.I REAL DATA FOR METRIC EVALUATION TESTS
For tests conducted on real data, the samples are DINOv2 embeddings derived from the CIFAR-IO dataset (Krizhevsky et al. 2009). When test requires out-of-distribution (or "bad") samples, these are DINOv2 embeddings of Gaussian noise images.
C.2 GENERATED DATASETS
This section details the generated datasets evaluated in Figures 6 12 and All data were publicly shared by Steinn et al. 2023) through the link provided in their GitHub repository: https: github. com/ 1 Layer
abs/ ddmmeval For more information on the generation process, see Appendix A of Stein et al. (2023).
We used 50000 samples for each evaluation. For conditional models, an equal number of samples class was taken, except when only 50000 unbalanced images were available.
C.2.1 CIFAR-10 10
For CIFAR-10 Krizhevsky et al. 2009), data were generated using the following models:
LSGM-ODE (Vahdat et al. 2021
PFGM++ (PFGMPP) Xu al.
2023
iDDPM-DDIM (Nichol & Dhariwal
2021
StudioGAN models (ang al. 2023a
ACGAN-Moo OOenna et al.
2017
BigGAN (Brock et al.
2019
LOGAN (wu et al. 2019)
MHGAN Turner et al.
2019
ReAAGAN Kang et al.
2021
WGAN-GP Gulrajani et al. 2017)
StyleGAN-XL (Sauer et 2022)
StyleGAN2-ada (Karras
2020
RESFLOW (Chnn al.
2019
NVAE Vahdat & Kautz 2020
C.2.2 IMAGENET
For ImageNet (Deng et al. 2009). images were rescaled to 256 x 256 using center cropping followed
by bicubic interpolation downsampling before being shared by Stein et al. (2023). The synthetic data were generated using the following models:
Models for which datasets of 50000 unbalanced samples were initially shared by Dhariwal & Nichol (2021):
ADM (Dhariwal & Nichol 2021
ADMG (Dhariwal & Nichol 2021
21

---

Published as a conference paper at ICLR 2026
ADMG-ADMU (Dhariwal & Nichol 2021
BigGAN Brock al. 2019)
DiT-XL-2 (Peebles & Xie
2023
DiT-XI--2-guided (Peebles & Xie 2023
LDM (Rombach et al.
2022
GigaGAN (Kang et al. 2023b)
StyleGAN-Xt (Sauer et 2022)
Mask-GIT (Chang et al. 2022)
RQ-Transformer (Lee et al. 2022)
C.2.3 LSUN BEDROOM
For LSUN Bedroom Yu et al. 2015). data were generated using the following models:
Consistency (Song et al. 2023). We used the two sets provided by Stein et al. (2023)
referred to Consistency-sell and Consistency-se12.
Four models for which datasets of 50000 unbalanced samples were originally shared by
Dhariwal & Nichol (202]):
-DM--roppuut (Dhariwal & Nichol 2021)
DDPM (Ho et al. 2020)
iDDPM (Nichol & Dhariwal 2021)
StyleGAN (Karras et al. 2019)
Diff-ProjGAN: Diffusion-Projected GAN Wang et al. 2023
Projected GAN (Sauer et al.
2021)
Unleash-Trand Unleashing Transformers (Bond-Taylor al. 2022)
C.2.4 FFHQ
For FFHQ (Kazemi & Sullivan 2014), images were downsampled 256 256 using Lanezos
interpolation before being shared by Stein et al. (003).. The data were generated using the following models:
LDM (Rombach al.
2022
InsGen (Yang et al. 2021
Projected-GAN Sauer et al.
202
StyleGAN-XL (Sauer et
2022)
StyleGAN2-ada (Karras et
2020)
StyleNAT (Walton et al.
2022
StyleSwin (Zhang al.
2022
Unleash-Tramn Unleashing Transformers
(Bond-Taylor et al. 2022)
Efficient-vdVAE (Hazami et al. 2022
22

---

Published as a conference paper at ICLR 2026
D EXAMPLE OF REAL OUT-OF-DISTRIBUTION SAMPLES INFLATING Density
The RESFLOOW-eeneaan CIFAR-10 dataset achieves a Density of 2.47, value inflated by real out- of-distribution samples. To investigate this, we categorized real samples by counting the number of synthetic samples within their 5-ball (Table
Table 2: Synthetic points per real k-ball (RESFLOW CIFAR-10).
Synthetic points in a real ball
Real balls (no clipping)
Real balls (with clipping)
0
48373
49981
1-5
680
15
6-100
513
3
101-1000
282
1001-10000
148
0
10001+
4
Without radii clipping. 48373 real balls contain synthetic points, whereas just 4 contain 10000. The 4 corresponding real images appear out-of distribuutii (see Figure
(a) Very gray ship image.
(b) Camoutlaged
(c) 2-legged horse.
(d) Toy ship stand.
Figure 7: Real CIFAR-10 samples containing more than 10000 synthetic points in their 5-ball. These images are atypical for their classes. (a) A ship viewed from above, maybe at night, resulting in mostly gray image. (b) A cat on top of a similarly colored object. (c) An unusually shaped horse that appears to have only two legs. (d) toy ship on stand instead of in water.
A real point having many synthetic points in its ball is not inherently problematic, generative models might generate many similar images. However, in this case, these 4 images are outliers that artificially increase the measured fidelity by being very far from other points. Here, clipping the real balls is enough to make the measured fidelity drop to 0.00.
23

---

Published as a conference paper at ICLR 2026
E FROM Density TO Clipped Density: OVER-OCCURRING SAMPLE PROPORTION AND RADII CLIPPING
This section measures the effects of the successive modifications that transform Density into Clipped Density. Below, we detail their impact on the evaluation of real generated datasets, along with the proportion of over-occurring samples (i.e., samples with a fidelity score greater than at each step.
The columns, in order, present: the original Density score, the initial proportion of over-occurring samples (OOP), the Density score after radii clipping, the OOP after radii clipping, the scores after individual sample contributions are also clipped (resulting in ClippedDensity. unmorm. and finally, the fully normalized ClippedDensity.
Table 3: From Density Clipped Density on CIFAR-10.
Density (clipped radii)
OOP (clipped radii) (%)
ClippedDensity umoom
Initial OOP (%)
ClippedDensity
Density
Model
LSGM-ODE
0.66
18.4
0.22
6.0
0.14
0.38
PFGMPP
0.65
18.3
0.22
6.3
0.15
0.39
iDDPM-DDID
0.64
17.4
0.19
5.6
0.13
0.34
ACGAN-Mon
2.28
43.0
0.00
0.0
0.00
0.01
BigGAN
0.55
14.4
0.10
2.6
0.07
0.19
LOGAN
22.4
0.00
0.0
0.00
0.01
MHGAN
0.57
14.9
0.06
1.6
0.05
0.13
ReACGAN
0.57
14.2
0.07
1.6
0.05
0.14
StyleGAN-XL
0.58
15.3
0.15
4.0
0.10
0.27
StyleGAN2-ada
16.4
0.13
3.4
0.09
0.24
WGAN-GP
1.74
37.2
0.00
0.0
0.00
0.01
RESFLOW
2.47
47.4
0.00
0.0
0.00
0.00
NVAE
1.59
31.9
0.00
0.0
0.00
0.00
Table 4: From Density to Clipped Density on ImageNet.
Density (clipped radii)
OOP (clipped radii) (%)
ClippedDensity unmern
Initial OOP (%)
ClippedDensit
Density
Model
ADM
0.38
8.3
0.08
2.2
0.07
0.17
ADMG
0.49
12.8
0.17
4.7
0.13
0.34
ADMG-ADMU
0.55
15.5
0.21
5.8
0.16
0.41
DiT-XL-2
0.83
28.2
0.43
14,6
0.29
0.74
DiT-XL-2-guider
1.65
63.8
1.12
42.3
0.63
1.00
LDM
0.70
21.5
0.32
10.0
0.22
0.57
BigGAN
0.46
10.5
0.05
0.9
0.05
0.12
GigaGAN
0.41
9.2
0.07
1.5
0.06
0.15
StyleGAN-XL
0.38
8.3
0.06
1.2
0.05
0.13
Mask-GIT
0.43
10.3
0.10
2.2
0.09
0.22
RO-Transformer
0.34
6.8
0.04
0.8
0.03
0.09
24

---

Published as a conference paper at ICLR 2026
Table 5: From Density to Clipped Density LSUN Bedroom.
Density (clipped radií)
OOP (clipped radi) (%)
ClippedDensity unmom
Initial OOP (%)
ClippedDensity
Density
Model
Consistency-setl
0.62
18.7
0.14
3.4
0.11
0.32
Consistency-ser2
0.64
18.9
0.13
2.9
0.11
0.30
ADM-dropout
0.83
25.9
0.41
12.6
0.26
0.73
DDPM
0.55
14.5
0.16
4.6
0.11
0.32
iDDPM
0.64
18.2
0.23
6.8
0.16
0.44
Diff-ProjGAN
0.56
15.0
1.1
0.05
Projected- GAN
GANN
0.63
16.9
0.05
0.9
0.04
0.12
StyleGAN
0.74
21.8
0.26
7.4
0.17
0.48
Unleash-Traan Trans
0.51
12.5
0.06
1.4
0.05
0.14
Table 6: From Density to Clipped Density on FFHQ.
Density (clipped radii)
OOP (clipped radii) (%)
ClippedDensity ummonti
Initial OOP (%)
ClippedDensity
Density
Model
LDM
0.65
18.4
0.24
6.6
0.16
0.41
InsGen
0.32
6.9
0.06
0.05
0.12
Projected-Gall GAN
0.21
3,5
0.02
0.2
0.01
0.03
StyleGAN-XI.
0.51
13.0
0.14
3.6
0.10
0.26
StyleGAN2-ada
0.24
4.2
0.04
0.7
0.03
0.08
StyleNAT
0.61
16.5
0.20
5.3
0.14
0.35
StyleSwin
0.53
13.8
0.10
0.25
Unleash-Trans
0.46
11.0
0.10
2.5
0.08
0.19
Efficient-vdVAE
0.80
23.2
0.28
7.9
0.17
0.42
F EVALUATION OF GENERATED CHEST X-RAYS
We evaluated a Progressively Growing GAN (Karras et al. 2018) from the Medigan library (Osuala al. 2023), trained by (Seal et al. (202T) on the ChestX -ilestX-ra88 dataset (Wang et al. (017) (< 110000
chest X-ray images). The model yielded Clipped Density of 0.06 and Clipped Coverage of 0.03.
The absolute interpretability of our metries allows a direct assessment of these results without re- quiring comparison to other models. A Clipped Density of 0.06 implies that the model's output has fidelity equivalent to dataset composed of only 6% good samples and 94% bad samples; similarly, Clipped Coverage of 0.03 is equivalent to dataset with only 3% good samples.
This a critical advantage over relative metrics, which can only rank models against each other without revealing whether even the best-performing
is adequate for given task. In high-staker domains like medical imaging, the ability to make an absolute judgment is essential. A low score provides a clear, unambiguous signal that the model is not yet fit for purpose, preventing the prema- ture adoption of technology that fails to meet the necessary standards for safety and reliability.
25

---

Published as a conference paper at ICLR 2026
G COMPARISON TO HUMAN EVALUATIONS
To further validate Clipped Density and Clipped Coverage, we compared their scores against human enror distinguishing real from generated images (scores shared by [Seein al, (2023)), where higher error rates indicate better generative models.
We computed Pearson correlation coefficients between human error rates and both metrics across all datasets in Figure 8 All correlations were significant and exceeded 0.8, except for FFHQ. where no significant correlation was found. This aligns with the findings of [Stein et al. (2023) (Figure 4,
DINOv2 column). This strong agreement with human judgment confirms that Clipped Density and Clipped Coverage correlate well with human perception of generation quality.
Consistency
Diffusion
GAN
Normalizing Flow
Transformer
VAE
CIFAR-10
ImageNet
LSUN Bedroom
FFHQ
0.93,
0.89e.06
0.82, p 2.05e-03
0.82, p 6.21e-03
-0.05,
8.95e-01
Human Error Rate
0.4
Human Error Rate
0.30
Human Enoor Rate
0.30
0.25
0.3
Human Erro
0.3
0.25
0.20
0.2
0.20
0.15
0.15
0.2
0.4
Clipped Density
Clippedeeniity
ClippedDensity
Clospe
0.95,
1.11e-06
0.81, 2. 2.43e-03
0.96, 3. 3.62
0.24,
5.42e-01
0.30
Human Rate
Human Errer Rate
Human Error Rate
0.25
0.3
Human Emror
0.25
0.20
0.2
0.2
0.15
0.1
0.10
0.15
0.1
0.2
0.3
0.25
0.50
0.75
0.25
0.50
0.1
0.2
0.3
ClippedCoverage
ClippedCoverage
ClippedCoverage
ClippedCoverage
LSGM-ODE
ADM
Consistency-set1
LDM
PFGMPP
ADMG
Consistency-set2
InsGen
IDDPM-DDIM
ADMG-ADMU
ADM-dropout
Projected-GAN
ACGAN-Mod
DIT-XL-2
DDPM
StyleGAN-XL
BigGAN- -Dep
DiT-XL-2-guided
IDDPM
StyleGAN2-ada
LOGAN
LDM
Diff-ProjGAN
StyleNAT
MHGAN
BigGAN
Projected-GAN
StyleSwin
ReACGAN
GigaGAN
StyleGAN
Unleash-Trans
StyleGAN-XL
StyleGAN-XL
Unleash-Trans
Efficient-vdVAE
StyleGAN2-ada
Mask-GIT
WGAN-GP
RQ-Transformer
RESFLOW
NVAE
Figure 8: Human Error Rate Clipped Density and Clipped Coverage: We human error rates from Stein et al. (2023), where subjects were asked to distinguish between real and generated images. A higher error rate indicates better generative model. The first row shows human error rates vs. Clipped Density, and the second row vs. Clipped Coverage. Above each plot, we display the correlation coefficients (and p-values) between human error rates and each metrie. All correla- tions are high except for the FFHQ dataset, where significant correlation was found, consistent with Stein al. (202)) (grayed their Figure 4, DINOv2 column).
26

---

Published as a conference paper at ICLR 2026
H COMPARISON TO A RECENT BENCHMARK
A recent position paper by Räisä et al. (2055) argues that "all current generative fidelity and diversity metrics are flawed" and that "no metric is suitable for absolute evaluations." Our work tackles these issues directly. In this section, we evaluate Clipped Density and Clipped Coverage on the benchmark from Räisä et al. 2025). using the authors' original code. All tests this benchmark are performed on synthetic data, with default sample size of 1000.
The results are summarized Tables and 8 which reproduce the tables from Räiä et al. (2025) with Clipped Density (CI-Dens) and Clipped Coverage (CI-Cov) added for comparison. Here, T denotes a passed test, while indicates failure. The original benchmark distinguishes between high (H, support-basee and low (I., density-based) diversity metrics. Since aim measure coverage, which assesses densities, we interpret L as success (T) and H as a failure (F).
We observed seven discrepancies with the original results, marked with royal blue underline (T or D). These differences arise from the benchmark's 's seed generation: the seed is based on a hash of the metric name, test name, and repetition index, but Python's built-in hash function changes across different sessions unless PYTHONHASHSEEN is set. As result, seeds (and thus results) differ between runs.
Table 7: Fidelity metrics comparison recent benchmark from Räisä et al. 2025
I-Prec
Density
C-Prec
symPrec
P-Prec
Cl-Dens
IAP
Desiderata
Sanity Check
Discrete Num. vs. Continuous Num.
F
F T
F T T
Gaussian Mean Difference
T
T
T
Gaussian Mean Difference + Outlier
F
T
T
T
F
Gaussian Mean Difference Pareto
T
T
T
T
T
Gaussian Std. Deviation Difference Hypercube, Varying Sample Size Hypercube, Varying Syn. Size
T
T
F
F
F
T
F
E
F
F
P
T
T
Purpose
Hypersphere Surface
Mode Collapse
T
T
T
Mode Dropping Invention
One Disjoint Dim. + Many Identical Dim.
F
Sequential Mode Dropping
T
Simultancous Mode Dropping
T
F
F
F
F
Sphere vs. Torus
T
T
T
T
Hyperparam.
Hypercube, Varying Syn. Size
T
T
Data
Hypercube, Varying Sample Size
F
F
Discrete Num. vs. Continuous Num.
Gaussian Mean Difference
F
Gaussian Mean Difference + Outier
Gaussian Mean Difference + Pareto
Gaussian Std. Deviation Difference
Hypersphere Surface
Mode Collapse
T
T
Bounds
Mode Dropping Invention
T
F
F
One Disjoint Dim. + Many Identical Dim.
F
T
F
Scaling One Dimension
TTT
T
T
Sequential Mode Dropping
F
F
T
Simultaneous Mode Dropping
F
F
F
F
Sphere vs. Torus
T
T
F
F
T
T
Invariance
Scaling One Dimension
F
T
T
T
27

---

Published as a conference paper at ICLR 2026
Table 8: Coverage metries comparison on recent benchmark from Räisä al. 2025
I-Rec
Coverage
IBR
C-Ree
symRec
P-Rec
Cl-Cov
Desiderata
Sanity Check
Discrete Num. vs. Continuous Num.
F
F
F T
F T T
Gaussian Mean Difference
T
T
T
Gaussian Mean Difference + Outlier
T
T
T
Gaussian Mean Difference + Pareto
T
T
T
T
T
Gaussian Std. Deviation Difference Hypercube, Varying Sample Size Hypercube, Varying Syn. Size
H
F
L
F
H
F
F
Purpose
Hypersphere Surface
Mode Collapse
Mode Dropping Invention
One Disjoint Dim. + Many Identical Dim.
F
Sequential Mode Dropping
Simultaneous Mode Dropping
F
Sphere vs. Torus
Hyperparam.
Hypercube, Varying Syn. Size
Data
Hypercube, Varying Sample Size
F
F
F
F
Discrete Num. vs. Continuous Num.
Gaussian Mean Difference
F
T
F
T
F
Gaussian Mean Difference + Outier
T
Gaussian Mean Difference + Pareto
T
Gaussian Std. Deviation Difference
Hypersphere Surface
F
TEA
F
T
T
Mode Collapse Mode Dropping Invention
Bounds
T
F
P
E
T
One Disjoint Dim. + Many Identical Dim.
F
T
Scaling One Dimension
F
T
T
T
Sequential Mode Dropping
F
T
F
T
F
T
Simultaneous Mode Dropping
F
T
F
T
F
T
Sphere vs. Torus
T
F
F
T
F
T
T
Invariacee
Scaling One Dimension
F
T
T
Purple underlined entries (F) indicate cases where the benchmark's criterion for success is, in our view, overly strict (see Appendix [..I]. Italic blue entries (F) indicate cases where the original im- plementation counts failure. but we argue that for low diversity metrics, these should be considered successes (see Appendix H.2
H.1 HARSH CRITERIA
Purple underlined entries (F) correspond to cases where, in our view, the benchmark's threshold for passing may be too narrow.
In the "Gaussian Mean Difference" test (Figure 9. which is analogous to our synthetic Gaussian translation test but without bad samples, the bounds desideratum is not satisfied by Clipped Coverage in dimension 64 with no translation (0). this case, the initial and synthetic distributions are the same. Clipped Coverage has value of 0.947, while the criterion requires value between 0.95 and 1.05.
Similarly, in the "Mode Collapse" test (Figure 10p. where real data is mixture of two Gaussians spaced by μ and the synthetic data is a single Gaussian, the value dimension 64 with no translation (0, same initial and synthetic distributions) is 0.941, just below the same required threshold.
28

---

Published as a conference paper at ICLR 2026
==1
d=8
0.8
Metric Valure
0.4
0.0
-2
-2
-1
2
-1.0
-0.5
0.0
0.5
1.0
CippedDensity
ClippedCoverage
Figure 9: Gaussian Mean Difference: output from the original code for our metrics.
d=1
d=
d
Metri Value
0.6
02
Clip peaDeniiy
ClippesCoverage
Figure 10: Mode Collapse: output from the original code for our metrics.
H.2 INCORRECT CRITERION
Italic blue entries (F) indicate cases where we disagree with the benchmark's 's failure criterion.
In the "Mode Dropping Invention" test (Figure I1]. the real data is mixture of 5 Gaussians, and the synthetic data progressively includes the 5 real modes and then invented modes. Both real and synthetic sets always have 1000 samples.
Clipped Coverage is marked as failing because it decreases when invented modes are added. How- ever, this decrease is expected for low-diversity metries (density-based); when invented modes are included, real modes become under-covered as the total amount of synthetic data remains the same, so coverage metrics should decrease. Thus, this behavior should be considered or L., both suc- cesses for coverage metrics.
1.0
0.8
Metric Value
0.6
0.4
0.2
2
6
8
10
Num. Synthetic Modes
ClippedDensity
ClippedCoverage
Figure 11: Mode Dropping + Invention: output from the original code for our metrics.
29

---

Published as a conference paper at ICLR 2026
I FIDELITY VS COVERAGE WITH OTHER METRICS
We reproduce Figure O with other metrics in Figures 12 and 13
We observe that for metric pairs, fidelity scores vary little. while coverage metrics vary sig- nificantly. This occurs for Precision/Recall (CIFAR-10, LSUN Bedroom), TopP/TopR (ImageNet, FFHQ), P-preccision/P-recalll (FFHQ), and Precision Cover/Recall Cover (FFHQ). symPrecision and symRecall show correlated scores. a-Precision and B-Recall show distinct results for consistency models on LSUN Bedroom and yield scores generally confined to the lower-right quadrant.
However, the correct behavior of metrics on real data is unknown, so we cannot draw definitive conclusions about which metric pairs are better or worse from these plots alone.
Consistency
Diffusion
GAN
Normalizing Flow
Transformer
VAE
CIFAR-10 -10
ImageNet
LSUN Bedroom
FFHQ
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0
0.2
0.4 0.6
0.8
1.0
0.0
0.2
0.4 0.6
0.8 1.0
0.0
0.2
0.4 0.6
0.8 1.0
0.0
0.2
0.4 0.6
0.8
1.0
Precision
Precision
Precision
Precision
1.0
1.0
1.0
1.0
symRecall
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.0
0.0
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
symPrecision
symPrecision
symPrecision
symPrecision
1.0
0.8
0.8
0.8
0.8
B-Recall
0.6
0.6
0.6
0.6
0.2
0.2
0.2
0.2
0.0 0.2 0.4 0.6
0.0
0.2
0.0
0.2 0.4 0.6
1.0
0.8 1.0
0.0
0.4 0.6
1.0
a-Precision
a-Precision
a-Precision
a-Precision
LSGM-ODE
ADM
Consistency-set1
LDM
PFGMPP
ADMG
Consistency-set2
InsGen
iDDPM-DDIM
ADMG-ADMU
ADM-dropout
Projected-GAN
ACGAN-Mod
DiT-XL-2
DDPM
StyleGAN-XL
BigGAN-Deep
DIT-XL-2-guided
iDDPM
StyleGAN2-ada
LOGAN
LDM
Diff-ProjGAN
StyleNAT
MHGAN
BigGAN
Projected
StyleSwin
ReACGAN
GigaGAN
StyleGAN
Unleash-Trans
StyleGAN-XL
StyleGAN-XL
Unleash-Trans
Efficient-vdVAE
StyleGAN2-ada
Mask-GIT
WGAN-GP
RQ-Transformer
RESFLOW
NVAE
Figure 12: Fidelity vs Coverage on various datasets, other metrics (1/2)
30

---

Published as a conference paper at ICLR 2026
Consistency
Diffusion
GAN
Normalizing Flow
Transformer
VAE
CIFAR-10
ImageNet
LSUN Bedroom
FFHQ
1.0
1.0
1.0
1.0
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.0
0.0
0.0
0.0
0.0
0.2
0.4
0.6 TopP
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6 TopP
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
1.0
1.0
1.0
1.0
Precall
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.0
0.0
0.0
0.0
0.2
0.4
0.8
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4 0.6
0.8
1.0
P-precision
P-precision
P.precision
P-precision
RevailCaver
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.2
0.2
0.2
0.2
0.0
0.2
0.0
0.6
0.2 0.8
0.0
1.0
PrevisionCover
PrecisianCover
PrevisionCover
Precision
LSGM-ODE
ADM
Consistency-set1
LDM
PFGMPP
ADMG
Consistency-set2
InsGen
iDDPM-DDIM
ADMG-ADMU
ADM-dropout
Projected-GAN
ACGAN-Mod
DIT-XL-2
DDPM
StyleGAN-XL
BigGAN-Deep
DiT-XL-2-guided
iDDPM
StyleGAN2-ada
LOGAN
LDM
Diff-ProjGAN
StyleNAT
MHGAN
BigGAN
Projected-GAN
StyleSwin
ReACGAN
GigaGAN
StyleGAN
Unleash-Trans
StyleGAN-XL
StyleGAN-XL
Unleash-Trans
Efficient-vdVAE
StyleGAN2-ada
Mask-GIT
WGAN-GP
RO-Transformer
RESFLOW
NVAE
Figure 13: Fidelity Coverage on various datasets, other metrics (2/2)
31

---

Published as a conference paper at ICLR 2026
J FIDELITY AND COVERAGE TRADE-OFF: TRUNCATION IN GANS
There is often a trade-off between fidelity and coverage, as improving one can come at the expense of the other for a given model (Li al. 2024). In this section, verify that Clipped Density and Clipped Coverage capture this trade-off by evaluating them on LSUN Bedroom StyleGAN data (Karras al. 2019), shared by the original authors with varying truncation levels. For GANs, the truncation trick samples from truncated latent space to improve fidelity at the expense of coverage. smaller truncation value vé restricts sampling to a smaller, denser region. Note that 1.0 corresponds to no truncation, matching the StyleGAN model in Figure
As shown Figure 14 both scores initially increase as we move from no truncation (* 1.0) to moderate truncation (+b = 0.7). However, further truncation to t 0.5 causes Clipped Density to rise higher while Clipped Coverage decreases, effectively illustrating the trade-off.
We investigated the initial rise of Clipped Coverage in Figure 15 by examining sample-wise score histograms. From vi = 1.0 to 1/ 0.7, real samples high-density regions that were barely covered become fully covered. Then, at v 0.5, many real samples drop to 0 coverage parts of the distribution are ignored. Thus, the initial rise in the Clipped Coverage score is caused by a more intense coverage of dense real regions.
0.50
V 0.7
Ciipped Coverage
0.45
0.40
V = 0.5
1.0
0.35
0.4
0.6
0.8
1.2 Clipped Density
1.0
1.4
1.6
1.8
Figure 14: Fidelity and Coverage trade-off: To verify that Clipped Density and Clipped Cover- age capture the fidelity-coerrgee trade-off, evaluated them LSUN Bedroom StyleGAN data (Karras et al. 2019) shared by the original authors with varying truncation levels. For GANs, the truncation trick samples from truncated latent space improve fidelity at the expense of coverage. A smaller truncation value v restricts sampling to a smaller, denser region. & 1.0 corresponds to no truncation, matching the StyleGAN model in Figure From no truncation
1.0) moderate truncation (2) = 0.7), both scores increase. However, further truncation to z = 0.5 causes Clipped Density rise higher while Clipped Coverage decreases, illustrating the trade-off. Note that here, we display Clipped Density scores without clipping them to I, see the next subsection for details.
32

---

Published as a conference paper at ICLR 2026
35000
Synthetic
30000
Real
20000
Sample Count
25000
15000
Sample
15000
10000
10000
5000
5000
Samplewiise Fidelity Score
Sample- wise Coverage Score
// 1.0, ClippedDennity 0.49
(b) 1
1.0, ClippedCoverage 0.37
35000
Synthetic Real
Sample Count
25000
Count
15000
20000
15000
10000
5000
5000
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Sample-wise Fidelity Score
Sample-wise Coverage Score
(c) d = 0.7, ClippedDensity = 1.00
(d) Ø = 0.7, ClippedCoverage 0.47
35000
Synthetic
30000
20000
Sample Count
25000
15000
Sample
15000
10000
10000
5000
5000
Sample-wise Fidelity Score
Sample-wise wise Coverage Score
d 0.5, ClippedDensity 1.45
( Φ 0.5, ClippedCoverage 0.38
Figure 15: Sample-wise Fidelity and Coverage for different truncation levels: Sample-wise Clipped Density (left) and Clipped Coverage (right) histograms for the StyleGAN data evaluated in
Figure 14 with varying truncation tb As truncation is strengthened, fewer synthetic samples have 0-
fidelity and more reach 1, reflecting sampling restricted dense regions. From 1.0 to & 0.7, Clipped Coverage improves as high-density regions become better covered. However, at = 0.5, many real samples drop to 0 coverage as parts of the distribution are ignored.
33

---

Published as a conference paper at ICLR 2026
K Clipped Density SCORES EXCEEDING
In this section, we investigate the impact of clipping Clipped Density scores to
First, reproduced previous tests where Clipped Density was limited to I, but without this clipping (Figure 16]. We found that the results remained stable near as expected: in these test settings, Clipped Density does not exceed signifieantly.
In our evaluations of generative models, one instance can exceed I: DiT-XI--2 with guidance on ImageNet. The sample-wise histograms (Figure 17) show that guidance increases the proportion of synthetic samples with perfect fidelity, surpassing that of real samples. Similar to StyleGAN truncation, which can also yield scores above (see Appendix D. Clipped Coverage improves as covered points become fully covered while uncovered points remain stable. This reflects the similar effects of guidance and truncation: both alter sampling to favor high-density regions while ignoring low-density ones.
Clipped Density score above thus occurs when sampling systematically avoids low-density re- gions, effectively targeting filtered real distribution. This is a feature of the unclipped metric: while using a filtered high-quality test set might be better practice in this scenario, the metric correctly re- flects that the synthetic set achieves higher fidelity than the unfiltered real reference.
Scores above I indicate that the generative model does not target the full real distribution, but rather subset of high-density regions. Since goal is to evaluate how well the model matches the real set, scores exceeding represent a deviation from this objective (often at the cost of coverage). Clipping to enforces the interpretation that ideal fidelity consists of matching the real data exactly, rather than surpassing by ignoring difficult regions.
1.2
1.02
1.0
1.01
0.8
Score
1.00
0.6
0.99
0.98
0.2
0.97
0.0
0.2
0.4
0.6 0.8
1.0
0.00
0.01
0.02
0.03
Proportion of data kept per mode
Proportion of bad samples
Simultaneously dropping (CIFAR-10)
(b) Introducing bad real & syn. samples (CIFAR-10)
Figure 16: Reproduced tests without limiting Clipped Density to We reproduced previous tests where Clipped Density was limited to 1. The results remain stable near expected. The standard deviation is high in (a) because only 2500 samples are used (see Figure 20 for an analysis of the standard deviation vs sample size). In test settings, Clipped Density does not exceed significantly.
34

---

Published as a conference paper at ICLR 2026
Synthetic Real
25000
25000
Count
20000
Sample Count
20000
15000
10000
10000
5000
5000
0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Sample-wist Fidelity Score
Sample-wir Coverage Score
(a) unguided, ClippedDensity = 0.74
(b) unguided, ClippedCoverage = 0.64
Synthetic Real
25000
25000
20000
Sample Count
20000
15000
15000
10000
10000
5000
0
0.0
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
1.0
Sample-wise Fidelity Score
Sample-wise Coverage
(c) guided, ClippedDensity 1.62
(d) guided, ClippedCoverage 0.83
0.85
guided
Clipped Coverage
0.80
0.75
0.70
unguided
0.65
0.60
0.75
1.00
1.50 Clipped Density
1.25
1.75 2.00
(e) Clipped Density vs Clipped Coverage
Figure 17: Clipped Density exceeding 1 with Guidance ImageNet. With guidance DiT-XL-22 on ImageNet, Clipped Density significantly exceeds 1. Sample-wise histograms show that guidance increases the proportion of synthetic samples with perfect fidelity, surpassing that of real samples. Similar to StyleGAN truncation, Clipped Coverage improves as covered points become fully cov- ered while uncovered points remain stable, reflecting the similar effects of guidance and truncation: sampling favors high-density regions and ignores low-density ones. A Clipped Density above occurs when sampling avoids low-density regions, effectively targeting a filtered real distribution. While using filtered high-quality test set might be better practice, the metric correctly reflects that the synthetic set achieves higher quality than the unfiltered real reference.
35

---

Published as a conference paper at ICLR 2026
L SENSITIVITY TO THE HYPERPARAMETER k
The number of nearest neighbors, k, a hyperparameter for Clipped Density, Clipped Coverage, and several other metrics. We evaluate their sensitivity on the DINOV2-FTHQ-LDM dataset (see the Figure below).
Scores increase with kt this results in larger k-nearest neighbor balls and thus higher likelihood of sample inclusion within these balls. Clipped Density and Clipped Coverage remain relatively stable, likely due the normalization of the balls by their volume mass.
1.0
10
1.0
0.0
0.6
0.4
0.4
(a) Precision
(b) Density
(c) symPrecision
3.8
3.0.
0.8
0.6
02
02-
02
0.0
(d) Recall
(e) Coverage
(I) symRecall
1.0
0.6
0.0
0.2
0.0
0.0
P-precision
(h) Clipped Density
1.0
1.0
0.6
0.4
0.4
0.0
30
20
(i) P-recall
(i) Clipped Coverage
Figure 18: Sensitivity to k on DINOV2-FFHO-LDDM.
36

---

Published as a conference paper at ICLR 2026
M CALIBRATION STABILITY
To further validate the absolute calibration of Clipped Density and Clipped Coverage, we performed two additional experiments on the ImageNet dataset. First, we repeated the progressive bad sample introduction test 10 times using distinct ImageNet subsets (Figure 19 confirming that both metrics decrease linearly with the proportion of bad samples.
Second, we assessed the stability of the metric scores as a function of the sample size N (Figure 20 We found that the empirical standard deviation of both metrics decreases proportionally to 1/ L/VN, reaching values below 0.01 for 50000, thereby confirming their stability.
1.0
Ideal Denspey
1.0
Ideal
ClippedCoverage
0.8
CiippedCCverage
CilppedDensity
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.2
0.6
0.8
1.0
0.0
0.6 Propartion bad synthetic samples
Propartion bad synthelic samples
(a) Clipped Density
(h) Clipped Coverage
Figure 19: Evaluating a mixture of good and bad samples (ImageNet): Synthetic mix DI- NOv2 embeddings of real ImageNet samples with noise (bad samples), using N = 50000 balanced samples per set. The experiment is repeated 10 times with different subsets of ImageNet. The stan- dard deviation is shown but is smaller than the markers in all cases (at most 0.0056 for Clipped Density and 0.0082 for Clipped Coverage). Both metries decrease linearly with the proportion of bad samples, confirming the absolute calibration on the high-dimensional ImageNet dataset.
37

---

Published as a conference paper at ICLR 2026
Censped
0.95
Coverade
0,90
Value
0.85
0.80
0.75
0.70
0.65
100
200
500
1000
2000
5000
10000
20000
50000
(a) Scores vy
10-1.
11NN
Standard Deviation
Standard Deviation
10-1
10
10-2
100
200
500
1000
2000
5000
10000
20000
50000
100
200
500
1000
2000
5000
10000
20000
50000
(b) Clipped Density
(c) Clipped Coverage
Figure 20: Calibration Stability vs sample size (N = M). Synthetic sets mix 80% real ImageNet samples with 20% noise, targetting a score 0.8 avoid boundary effects 1. We use samples class, selecting random classes for smaller N (10 repetitions). (a) Scores vs N. (b)-(c) Empirical standard deviation vs N. Both metrics" std decreases as 1/vN, reaching values below 0.01 for N 50000, even when adjusting by 1/0.8 to account for higher target scores.
38

---

Published as a conference paper at ICLR 2026
N FIDELITY UNDER IMPUTED DISTORTIONS
To further assess the behavior Clipped Densiry, we evaluated the fidelity of PFGMPP-ggeneraed CIFAR-10 samples under various image distortions, replicating the setup from Figure 5 of Jiraler- Spong et al. (2023) Our results for Clipped Density, shown in Figure 21 align with those reported for the Feature Likelihood Divergence (FLD) metric in the original study, which were themselves better than those of FID.
We applied the transformations detailed Appendix E.1 of Jiralerspong et al. (2023). For the Color Distort transformation, the default parameters produce an identity transform, so we used non- default hue 0 3. For the Center Crop transformation, different values (28 and 30) are used throughout the paper. We proceeded with the value 28.
PFGMPP
StyleGAN-XL
StyleGAN2-ada
LOGAN
0.5
0.4
Clipped Density
0.3
0.2
0.1
0.0
None
Posterize
Light Blur
JPG 90
Color Distort
Center Crop 28
Elastic Transform
Heavy Blur
Figure 21: Fidelity of PFGMPP samples under different image distortions, measured with Clipped Density, Horizontal lines represent the Clipped Density scores of other models for compar- ison.
39

---

Published as a conference paper at ICLR 2026
O OTHER FAILURE MODES
In this section, we examine additional failure modes of generative models and analyze the response of Clipped Density and Clipped Coverage.
In the first scenario, we assess the detection of a small real mode absent from the synthetic data, following the protocol in Appendix F.2 of Lemos al. (2025) (see Figure 22p. The primary distri- bution consists of eight CIFAR-10 classes, while rare mode (trucks) is progressively introduced solely into the real set. Clipped Coverage correctly decreases as this uncovered rare mode grows, showing sensitivity even when the mode represents only 0.00125% of the real set. Conversely, Clipped Density remains stable, as the synthetic samples continue to reside within valid real modes.
In the second scenario, we evaluate the impact of incorrect intra-mode structure. We sample 10000 points from centered Gaussians with distinct orientations: /-shaped for the real data and \-shaped for the synthetic data (see Figure 23 top). These distributions correspond to the block-structured covariance matrices detailed in Figure 23 (d), where controls the width of the shapes. Regardless of Ø, both Clipped Density and Clipped Coverage correctly drop to 0 the dimensionality increases, reflecting the fact that the intersection volume between the two distributions becomes negligible.
0.988
ClippedDensity
0.98
ClippedCoverage
0.986
0.96
0.984
0.94
Score
0.982
0.92
0.980
0.90
0.978
0.88
0.00
0.02
0.04
0.06
0.08
0.10
0.1
0.2
0.4
0.6
0.8
1.0
Fraction of truck samples used
Figure 22: Rare modes in real data: Following Lemos et al. (0255) (Appendix F22), we assess the detection of a small real mode absent from synthetic data. Eight CIFAR-10 classes form the main modes (all but trucks and ships, 20000 samples each for real/synthetic). A rare mode (trucks) is progressively introduced into the real set only (up to 2500 samples). We introduce 0% to 0.1% (Left) and 0.1% to 1% (Right) of truck samples, representing up to 0.0125% and 0.15%% of the total real set, respectively. Clipped Coverage correctly decreases as the uncovered rare mode grows, while Clipped Density remains stable since synthetic samples still lie within valid real modes.
40

---

Published as a conference paper at ICLR 2026
-2
-2
Real Synthetic
-4
-4
-2 0
-4
-2
2
-4
2
(a) a = 0.1
(h)
0.3
(c) a = 0.5
0.6
Clipped Density (a=0.1)
Clipped Coverage (g=0.1)
Clipped Density (a-0.3)
1
1
0.4
Clipped Coverage (a=0.3)
Evaal
1 0²
@
Ia/2
Value
Clipped Density (a=0.5)
0.3
Clipped Coverage (c=0.5)
Esynihetic
1102
-1
Ia/2
0.2
1
2
2
2
2
(d) Covariance matrices
(c) Scores vs Dimension
Figure 23: Incorrect correlations: Real and synthetic sets consist of N 10000 samples from centered Gaussians with covariance matrices shown in (d). The first two components are shown for varying a (a)-(c). (e) shows Clipped Density and Clipped Coverage versus dimension for different Regardless of both metrics drop to 0 as the dimension increases, because the intersection volume between the distributions becomes negligible.
41

---

Published as a conference paper at ICLR 2026
P OTHER MODALITIES
P.I TOY TIME SERIES EXAMPLE
Following Appendix F.3 of Lemos et al. (2025), we evaluated Clipped Density and Clipped Cov- erage using toy time series data. In this setup, the real signal consists solely of noise, whereas the synthetic data includes an added hallucinated signal of varying amplitude. As the amplitude of this hallucination increases, both metrics decrease, correctly identifying the discrepancy.
1.0
Clipped Density
Clipped Coverage
0.8
Metric Value
0.6
0.4
0.2
0.0
10-2
10-1 Amplitude
109
(Appendix F.3), we evaluate Clipped Density and Clipped Coverage on toy time series. The signal is defined as cos(t) + v7(t), where 7tt) unit Gaussian noise, with 100 time points per sample and 5000 samples per set. The real set has zero amplitude (A = 0), while the synthetic set has varying amplitude. The goal is to detect this discrepancy. Both metrics decrease as the signal amplitude increases, dropping below the maximum slightly after an amplitude of 0.1. Some instability is observed due to the limited sample size of 5000.
Figure 24: Toy time series example: Following Lemos al. 2025
42

---

Published as a conference paper at ICLR 2026
P.2 MUSIC DATASET EXAMPLE
To evaluate music data, we used the Free Music Archive large dataset, comprising 30s clips from 106574 tracks (Defferrard et al. 2017). Following Gui al. (2024), who compared 10 embedding models for music, we employed the CLAP model (Elizalde et al. 2023). Since it extracts 1024- dimensional embeddings from 7s clips, we selected the middle of each track.
An initial evaluation on balanced real and synthetic splits of 50000 samples each yielded Clipped Density of 0.988 and Clipped Coverage of 0.998. When splitting by genre, scores decreased to 0.943 for Clipped Density and 0.875 for Clipped Coverage. This reduction is moderate but expected, as music tracks span multiple genres, yet we split based on single genre per track (a track's 's other associated genres might correspond to the other dataset).
Finally, we evaluated mixtures of good and bad samples, where bad samples consisted of noise MP3 files. The results show that both Clipped Density and Clipped Coverage decrease linearly with the proportion bad samples, confirming their absolute calibration on this high-dimensional music dataset.
1.0
DenspedDensity Ideal
1.0
ClippedCoverage Ideal
CiippedDensity
ClippedCoverage
0.6
0.6
0.4
0.4
0.2
0.2
0.2
0.6
1.0
0.0
0.2
Proportion of bad synthetic samples
Proportion of bad synthetic samples
(a) Clipped Density
(b) Clipped Coverage
Figure 25: Evaluating a mixture of good and bad samples (Free Music Archive): Synthetic sets mix CLAP embeddings of real music samples with embeddings of noise (bad samples), using N = 50000 balanced samples per set. Both Clipped Density and Clipped Coverage decrease linearly with the proportion of bad samples, confirming the absolute calibration on this high-dimensional music dataset. Note that compared to other plots of this test, the evaluation was performed only once per point and not repeated 10 times.
43

---

Published as a conference paper at ICLR 2026
Q METRIC TESTS ON TOY DATASETS
This section details experiments on synthetic Gaussian data (N 25000, d 32, 10 repetitions), analogous the CIFAR-I0 tests in Section [ Out-of-distribution samples are drawn from a Gaus- sian distribution with variance max(4, (10 + 2) where Z ~ (), The tests performed are:
Simultaneous mode dropping: Progressively replacing data from all but one class with data from the remaining class (see Figure 26b analogous to Figure
Matched real and synthetic out-of-distribution samples: Progressively replacing data from both real and synthetic datasets with out-of-distribution samples (see Figures 26c
and analogous to Figures 4c and 5a
Introducing bad synthetic samples: Progressively replacing synthetic data with out-of- distribution samples (see Figures and
and 27a analogous to Figures and 4a
Q.1
FIDELITY METRICS
1.0
0.8
Relative score
0.6
Relative
0.4
0.4
0.2
0.2
0.0
0.0
0,2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Proportion synthetic samples
Proportion kept per mode
(a) Fidelity reduction with bad samples
(b) Simultancously dropping modes
1.06
1.04
Relative score
1.02
1.00
0.98
0.96
0.94
0.00
0.01 Proportion
0.02 samples
0.03
(c) Introducing syn. samples
Legend
Figure 26a
Figure 26b
Figure 26c
Figure 4d
Precision
Density
x
symPrecision
x
X
x
d-Precision
TopP
P-precision
x
PrecisionCover
ClippedDensity (ours)
Ideal
Legend and summary
Figure 26: Testing fidelity metrics on toy data: this figure is the equivalent of Figure 4 on toy data.
44

---

Published as a conference paper at ICLR 2026
Q.2
COVERAGE METRICS
1.2
1.0
1.04
Relative score
0.8
1.02
0.6
Relative
1.00
0.98
0.2
0.96
0.0
0.94
0.0
0.2
0.4
0.6
0.8
1.0
0.00
0.02 Proportion bad samples
0.01
0,03
Proportion synthetic samples
(a) Coverage reduction with bad samples
(h) Introducing bad real & syn. samples
Legend
Figure
27a
Figure 27b
Figure 5b
Recall
x
X
Coverage
x
/
symRecall
x
x
x
B-Recall
X
x
x
TopR
X
x
/
P-recall
X
X
X
RecallCover
X
/
ClippedCoverage (ours)
Ideal
Legend and summary
Figure 27: Testing coverage metries on toy data: this figure is the equivalent of Figure 5 on toy
data.
45

---

Published as a conference paper at ICLR 2026
R
UNNORMALIZED RESULTS
Figures and present relative scores. This section shows the corresponding unnormalized scores, which include the maximum values and can be easier read.
R.1 MIXTURE GOOD AND BAD SAMPLES (CIFAR-10)
32
Prppcoo sysheccanoo
Pososte of arderoe SITEM
(a) Precision
(b) Density
(c) symPrecision
(d) a-Precision
2.5
22 24
Prppccoon bymheccsaaae
24 64
5.
63
of prederr
14
34
TopP
(f) P-precision
(g) Precision Cover
(h) Clipped Density
2.4
0.22
37
10
21
(i) Recall
(i) Coverage
(k) symRecall
3-Recall
04
02
02
02
00
06
30
(m) TopR
(n) P-recall
(0) Recall Cover
(p) Clipped Coverage
Figure 28: Mixture of good and bad samples (CIFAR-10), unnormalized.
46

---

Published as a conference paper at ICLR 2026
R.2
SIMULTANEOUS MODE DROPPING (CIFAR-10)
64
0.
0.
(a) Precision
(b) Density
(c) symPrecision
(d) a-Precision
2.3
0.0
54
02
0.
(e) TopP
(1) P-precision
(g) Precision Cover
(h) Clipped Density
Figure 29: Simultaneously dropping modes (CIFAR-10), unnormalized.
R.3 MATCHED REAL & SYNTHETIC OUT-OF-DISTRIBUTION SAMPLES (CIFAR-10)
3.5
3.00
050
Exporaar maiirrp
Proctiis d'sadarrkks
Aapatin sampla
Psporie Caiisspp
(a) Precision
(b) Density
(c) symPrecision
(d) a-Precision
1.
3.0
3.5
3.8
3.0
332
Peportor of taii
Propetie d'tesrnnke
of sumpkr
(e) TopP
(f) P-precision
(g) Precision Cover
(h) Clipped Density
30.0
3.0
0.0
054
EG00
L22
122
3.15
dfted
030
522
022
(i) Recall
Coverage
(k) symRecall
B-Recall
330
221
3.8
T3V
131
111
X1
(m) TopR
(n) P-recall
(0) Recall Cover
(p) Clipped Coverage
Figure 30: Matched real & synthetic out-of-distributtio samples (CIFAR-10), unnormalized.
47

---

Published as a conference paper at ICLR 2026
R.4 SYNTHETIC DISTRIBUTION TRANSLATION
1.0
-25
(a) Precision
(b) Density
symPrecision
(d) Q-Precision
02
90
(e) TopP
(f) P-precision
(g) Precision Cover
(h) Clipped Density
02
(i) Recall
(j) Coverage
(k) symRecall
(1) 3-Recall
1.0
30
02
(m) TopR
(n) P-recall
(o) Recall Cover
(p) Clipped Coverage
Figure 31: Translating a synthetic Gaussian with bad samples, unnormalized.
48

---

Published as a conference paper at ICLR 2026
S VISUAL INSPECTION OF SAMPLES
In this section, we display real samples from each dataset (Figures 34 37 and 39 alongside data generated by the west-performing model for each case (Figures 33 38 and 40). Additionally, in Figure we present manually annotated generated FFHQ samples to highlight visual artifacts and inconsistencies that may contribute lower scores. These LDM samples primarily exhibit issues with background details, hands, and teeth, and occasionally with eyes and ears.
Figure 32: Real samples from CIFAR-10
49

---

Published as a conference paper at ICLR 2026
Figure 33: Generated CIFAR-10 samples by PFGMPP: one class per row.
50

---

Published as a conference paper at ICLR 2026
MB
XO
Figure 34: Real samples from FFHQ
51

---

Published as a conference paper at ICLR 2026
Figure 35: Generated FFHQ samples by LDM
52

---

Published as a conference paper at ICLR 2026
Figure 36: Generated FFHQ samples by LDM with annotated defects. We manually highlight
visual artifacts and inconsistencies that may contribute to lower scores.
53

---

Published as a conference paper at ICLR 2026
3DMODELFREE.OM
Figure 37: Real samples from LSUN Bedroom
54

---

Published as a conference paper at ICLR 2026
H.O.J JGORLII
Figure 38: Generated LSUN Bedroom samples by ADN-dropout
55

---

Published as a conference paper at ICLR 2026
oover Colllinch
Figure 39: Real samples from ImageNet
56

---

Published as a conference paper at ICLR 2026
3
Figure 40: Generated ImageNet samples by DiT-XL-2-guided
57


Reviews:
[{"review_id": "piTouVKoQm", "reviewer": "Reviewer_4Rqw", "summary": "This paper introduces two new metrics, Clipped Density and Clipped Coverage, to address failings in the evaluation of generative models. The authors argue that existing metrics for fidelity and coverage suffer from two main problems: 1) a lack of robustness to outliers in both the real and synthetic datasets, and 2) an inability to provide an interpretable, absolute score.\nThe proposed metrics address these issues through new clipping and calibration strategies:\n1. Clipping: Clipped Density (fidelity) prevents real-data outliers from inflating scores by clipping k-NN ball radii to the median distance. Both metrics also clip the contribution of any individual sample to a score of 1, preventing over-represented modes from masking the presence of \"bad\" samples. \n2. Calibration: The metrics are calibrated to provide an absolute, more interpretable score. A score of $x$ is designed to be equivalent to the performance of a dataset with a proportion $x$ of \"good\" samples and $1-x$ of \"bad\" samples . For Clipped Coverage, this is achieved via a theoretical derivation (Lemma 1) and a resulting correction function ; for Clipped Density, this is achieved by normalizing by the real data's score on itself . \nThe authors demonstration on synthetic densities and through empirical validation on DINOv2-embedded image datasets that these two metrics are robust to outliers and provide interpretable absolute scores that behave linearly as sample quality degrades.", "questions": "1. How does the variance of the coverage score scale as a function of the number of samples, number of bootstrap performed, and dimensionality of the problem, for various applications?    \n\n2. Your k-NN metrics are susceptible to the curse of dimensionality, which is why you use DINOv2. But what failure modes does this introduce? For example, how would your DINOv2-embedded metrics perform on a \"bad\" model that correctly learns the locations and densities of modes but gets the correlations within those modes wrong?\n\n3. How does the metrics compare to modern metrics like TARP, PQMass, FLD and FID on various realistic tasks?\n\n3. Your \"absolute interpretability\" relies on the metric's expected value, not a single score. How many bootstrap samples are needed to get a stable estimate of this mean, and how does this required number of samples scale with data dimension?\n\n4. How sensitive are your metrics to the choice of metric distance between samples?\n\n5. You motivate your metrics as being robust to outliers, which in practice means they are insensitive to the magnitude of a sample's \"badness.\" How bad do mediocre samples have to be for an ‘ok but not great’ model to be scored worse than one that sometimes produces catastrophic samples? How does this depend on the choice of embedding model?\n\n6. Can you discuss more in detail failure modes of your tests, for example \"bad\" model that correctly learns the locations and densities of modes but gets the correlations within those modes wrong (e.g., swapping a /-shaped mode for a \\\\-shaped mode)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strengths:\n-  The paper addresses an important and recognized problem: the lack of reliable, robust, and interpretable metrics for generative models.\n- The paper presents a well-illustrated analysis of the failure modes of a chosen class of metric: k-NN density-based metrics (like Precision, Density, Coverage) is clear. The figures effectively illustrate the specific problems of outlier sensitivity and non-linear response that the paper aims to solve.\n- The \"clipping\" mechanisms are simple and intuitive and directly target the demonstrated failure modes. Clipping radii addresses outlier-driven inflation, and clipping sample contributions addresses mode collapse or over-representation.\n- The authors have made some efforts to demonstrate the performances of their metrics on modern, high-dimensional datasets like ImageNet, LSUN, and FFHQ, which are the standard benchmarks for \"real-world\" generative models", "weaknesses": "1. Overstated Claims and Missing Key Baselines: The paper's premise that \"all existing... metrics are flawed\" and that \"no metric offers this property\" (absolute interpretability) is an overstatement. The paper's analysis is confined almost entirely to kNN density-based metrics, while ignoring a relevant body of work on sampling-based evaluation: arXiv:2402.04355 and arXiv:2302.03026 similarly use distance metrics to probe the underlying density but don’t rely on kNN density estimation; there is also no comparison to other standard metrics used in the literature, to help the audience how the proposed metrics differ on specific tasks (Fréchet Inception Distance - FID  arXiv:1706.08500, Feature Likelihood Divergence  - FLD arXiv:2302.04440) Failing to refer to and compare against this (FID is mentioned, but just in the intro, no comparisons are actually made on on problems) related work significantly weakens the paper's claim to novelty, and makes it hard to understand how its performances compare to existing evaluation options.\n2. The paper's main technical contribution—the calibration of Clipped Coverage—could be presented more clearly.\n\n   - Lack of Clarity: The paper never explicitly states that its \"absolute interpretability\" is a property of the metric's expected value, not its value from a single draw. This implies that bootstrapping (running the metric many times) is required to get an interpretable score, but this is never explicitly stated or analyzed (how stable is the score as a function of the number of samples and the dimensionality of the problem).\n\n   - Statistical Ambiguity: As a result of sampling variance, a single \"lucky\" draw from a perfect model (or even an imperfect model) could yield an uncalibrated score $s > f_{\\text{expected}}(0)$, which would result in a calibrated score greater than 1.0, breaking the \"proportion\" interpretation. This could happen even if the score is bootstrapped a small number of times (since the variance of the score is not studied theoretically, it’s hard to assess how many times the bootstrapping needs to be done for a specific application). The paper provides no analysis of the metric's variance or the number of bootstrap samples needed for a stable estimate.\n\n   - Lack of theoretical guarantees: The paper does not present theoretical guarantees that the proposed metric has important desirable properties in the theoretical limit of infinite number of samples or with infinite number of re-drawing of the samples and recalculating the score, like a proof of sufficiency or a proof of consistency.  \n\n3. Limited Scope of Evaluation (Failure Modes & Modalities):\n   - Limited evaluation of the sensitivity of the metrics: The \"sanity checks\" are limited to gross failures like complete mode dropping or injection of very easily detectable out of distribution samples. For scientific applications, a much more relevant test is sensitivity to subtle failures, such as missing rare modes, failing to capture intra-mode structure, or subtle corruption to the samples.\n    - Missing Modalities: All experiments are on images or simple Gaussian data. The metric is distance-based and should be applicable to any data modality. The paper would be far more convincing if it demonstrated this utility on other common data types, such as tabular data or time-series. \n\n4. All the high-dimensional experiments presented have a dependence on  a black box feature extractor. The robustness claims are undermined by this, as it’s not clear if it’s the metric that’s robust, or the embedding of the data through DINOv2. This creates blinds spots for many real world applications:  \n\n   - Foundation models like DINOv2 are trained for invariance to noise, blur, and other augmentations, but for some applications, for example in science, it’s important to test weather those exact nuisance parameters are correctly captures by the model. \n\n   -  Moreover, the claim of the paper is that the metric could be generally useful across domains, but DinoV2 is specifically trained on natural images and there is no guarantees that it extracts features that are useful in other domains, e.g. scientific data like fluids dynamics simulation, climate models, astrophysical simulations, that have different underlying statistics and correlate scales differently. The experiments have shown that in high-dimensions, the metrics capture model’s realism within the DINOv2 feature space, but that might not correlate with true scientific fidelity.\n\n5. There is a significant limitation of the metrics that is not discussed: The paper's design goal is \"robustness to outliers,\" which it achieves by making the metrics insensitive to the magnitude of a sample's \"badness\" . A synthetic sample 1000 units away from the data manifold is scored as a 0, which is the same score a sample only 10 units away might receive. While this makes the metric sensitive to the proportion of bad samples, it ignores the severity. For the \"high-stakes applications\" mentioned in the introduction (e.g., healthcare), this is a critical flaw. A model that produces one catastrophic outlier is far more dangerous than a model that produces many mediocre samples. The paper presents this insensitivity as a feature without discussing the significant trade-off in its limitations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8FNaVzLwEl", "reviewer": "Reviewer_6bfz", "summary": "This paper proposes the metrics Clipped Density and Clipped Coverage, which are enhanced based on the Density and Coverage metrics [1]. The enhancement comes in two aspects: (1) robustness to outliers, and (2) that the metrics are linear and normalized (thus more interpretable). This is achieved by capping per-sample contribution, normalization, and calibration based on empirical analyses. The effectiveness is validated by comprehensive experiments covering several competing metrics, different types of generative models, and various sizes of datasets.\n\n\n\n\n[1] Naeem, Muhammad Ferjad, et al. \"Reliable fidelity and diversity metrics for generative models.\" International conference on machine learning. PMLR, 2020.", "questions": "- In Eq. (7), why should we normalize by $ClippedDensity_{real}$? When would $\\frac{ClippedDensity_{unnorm}}{ClippedDensity_{real}}$ be larger than 1 and how should we interpret it? Why can it be safely clipped to 1?\n- It seems that Clipped Coverage is not guaranteed to be <= 1. Is this correct? How should we interpret it when it is larger than 1?\n- Fidelity and coverage are usually known to be a trade-off [2] for the same model. Clipped metrics in Figure 6 do not reflect this (, although the comparisons are among different models). Can you further verify this?\n- L122-123, the sentence \"In high-dimensional spaces, ... as a proxy\" seems off here. Can you clarify?\n\n[2] Li, Shuangqi, et al. \"Controlling the Fidelity and Diversity of Deep Generative Models via Pseudo Density.\" Transactions on Machine Learning Research.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- The paper is nicely written and easy to read, equipped with nice visualizations.\n- The study is well motivated. \n- It is novel that the proposed metrics are designed to satisfy robustness, linearity, and interpretability.\n- The analyses and experiments are comprehensive.", "weaknesses": "- The modifications applied to meet the proposed desiderata appear ad-hoc and lack theoretical justification. This raises the question of whether they might negatively impact performance in other aspects, such as distinguishability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VX690jrK4f", "reviewer": "Reviewer_oPHo", "summary": "The paper clearly identifies limitations of current generative model evaluation metrics (e.g., FID, Precision/Recall): they combine fidelity and coverage into a single number, lack calibration, and are sensitive to outliers. This motivates the need for metrics that separately assess fidelity and coverage and provide interpretable, robust scores. The authors introduce Clipped Density and Clipped Coverage. Both metrics cap individual sample contributions at 1, preventing over‑occurring samples from masking defects and ensuring robustness to outliers. Clipped Density further clips nearest‑neighbor radii to the median k‑th neighbor distance, limiting the effect of sparse or outlier real data points. Clipped Coverage counts the number of synthetic samples within each real sample’s ball, normalizes by k, and caps at 1.", "questions": "- Have you tested the metrics on non‑image generative models (e.g., video or audio)? If not, what challenges do you foresee in extending the method to other modalities?\n\n- Interpretation of Scores: The calibration interprets a score of 0.4 as “40 % good samples and 60 % bad samples”. In real generative models, “bad samples” may not be pure noise but may include subtle artifacts or low‑quality variations. How should practitioners interpret intermediate scores in such contexts?\n\n- For Clipped Coverage, the calibration requires computing and numerically inverting an expectation using Beta functions. How sensitive is the final metric to errors in this inversion?\n\n- Have you conducted any human evaluation to correlate Clipped Density/Coverage scores with perceived quality or diversity? Without such correlation, it is unclear if the metrics align with human judgments.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Calibration for Interpretability: The metrics are calibrated so that their expected value decays linearly as the fraction of bad synthetic samples increases. For Clipped Density, the unnormalized score is divided by the fidelity score computed on the real data and then clipped to [0,1]. For Clipped Coverage, the authors derive the expected value under an i.i.d. assumption using Beta functions and numerically invert it to map unnormalized scores to linear decay. This calibration allows scores to be interpreted directly as the proportion of “good” samples.\n\n- Comprehensive Testing: The paper proposes a suite of controlled tests (mode dropping, introduction of out‑of‑distribution samples, synthetic data translation) to evaluate robustness, sensitivity, and interpretability. Experiments show that existing metrics often conflate fidelity with coverage or are unstable, whereas Clipped Density and Clipped Coverage exhibit linearity, stability, and symmetry across tests.", "weaknesses": "- Incremental Novelty: While clipping sample contributions and radii is a sensible modification, the metrics essentially adapt existing Density/Coverage measures. The idea of capping contributions is intuitive; no fundamentally new notion of fidelity or coverage is introduced. The theoretical calibration for Clipped Coverage relies on i.i.d. assumptions and numerical inversion, which is mathematically involved but conceptually just rescales the metric to match a linear decay.\n\n- Scope Limited to Images: Experiments and definitions focus on image data. Although the authors claim the framework is generic, there is no evidence it extends to other modalities (video, audio).\n\n- The absolute scores are interpreted as equivalent proportions of “good” samples, but this relies on synthetic tests where bad samples are pure noise. In practice, generative models may produce subtle artifacts or distributional shifts that do not correspond neatly to “bad samples,” and the linear calibration may not reflect perceptual quality. The tests used to justify the metrics (mode dropping, noise replacement, translation) are tailored; there may be other failure modes (e.g., adversarial patterns, style transfer) where the metrics behave unpredictably.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DvBbxjCVVm", "reviewer": "Reviewer_GJ4y", "summary": "The paper proposes a pair of synthetic data fidelity and diversity metrics called clipped density and clipped coverage. They build on the previous density and coverage metrics, adding clipping to guard against pathological behaviors and normalisation to ensure the values are between 0 and 1 in practice, and degrade linearly if a proportion of samples is corrupted. The new metrics are evaluated using a some common sanity checks on image data, and a larger set of checks on artificial data from a recent benchmark. The new metrics pass all the image checks, and pass more artificial checks than previous metrics. The paper also compares many image generators on four standard image datasets with the new and existing metrics. The results with the new metrics are seemingly more informative than the ones with other metrics.", "questions": "No further questions.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 4, "strengths": "The paper convincingly argues that existing generative model fidelity and diversity metrics are not sufficient for many use cases, and explains why fixing these problems is important. The new metrics are presented clearly, and the writing overall is good. The ideas are novel, and the normalisations the paper introduces may also be useful with other fidelity and diversity metrics. The results on the sanity checks in the paper are good: the new metrics pass all of the image checks, and pass more artificial checks than any other metric.", "weaknesses": "The main weakness of the paper is the weak evaluation of the linear degradation claim for clipped density and clipped coverage. This claim is only supported by a few checks on image data. However, both metrics still fail many of the bounds checks from Räisä et al. (2025), indicating that the metrics do not always have the extreme values 0 and 1 in practice, which is a prerequisite for linear degradation. This means that the claim that only 40% of generated samples on CIFAR-10 and FFHQ are good, in Section 5.3, is not supported. It would be interesting to see a visual comparison between real and synthetic images for the datasets and models (or subsets of them) in Section 5.3. If the comparison shows a clear difference between the proportions of good and bad synthetic samples between the small and large datasets, this would provide more evidence that clipped density and clipped coverage can be interpreted as such proportions.\n\nThe undesirable behaviors the other metrics are claimed to have in Section 5.3 and Appendix I are not supported by evidence, with the exception of density having values greater than 1. This is because it is not known what the behavior should be: it is possible that in this setting, fidelity and diversity should be correlated, or that all the models have similar fidelity, or that the score ranges should be restricted, so there is no evidence that any of these behaviors is undesirable. Besides, it is not fair so say that correlated scores fidelity and diversity scores are a downside of other metrics, as clipped density and clipped coverage are also highly correlated in Figure 6.\n\nIn my opinion, clipping clipped density to 1 only hides the problem of the score exceeding 1 and does not increase interpretability. As a result, it would be good to include results without this clipping in the paper, or at least check when the unclipped score exceeded 1.\n\nMinor points:\n- In Figure 4b, clipped density is only close to 1 at the left edge, not extremely close to 1, which could be considered a failure compared to some of the other metrics.\n- Figure 5b: it is very difficult to judge how symmetric symRecall is.\n- Markers in all figures are small, so their shapes are hard to see.\n- Figures 5 and 5: \"Precall\" -> \"P-recall\", and same for \"Pprecision\".\n- Figure 8: the bar labels are hard to associate with the correct bars, especially on the right. Rotating them more should fix this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)