On Measuring Fairness in Generative Models
Authors: Christopher T.H Teo
Milad Abdollahzadeh
Ngai-man Cheung
Venue: NeurIPS 2023 poster (2023)
Keywords: Fairness
Generative models
GAN
Calibration

Abstract: Recently, there has been increased interest in fair generative models. In this work,
we conduct, for the first time, an in-depth study on fairness measurement, a
critical component in gauging progress on fair generative models. We make three
contributions. First, we conduct a study that reveals that the existing fairness
measurement framework has considerable measurement errors, even when highly
accurate sensitive attribute (SA) classifiers are used. These findings cast doubts
on previously reported fairness improvements. Second, to address this issue,
we propose CLassifier Error-Aware Measurement (CLEAM), a new framework
which uses a statistical model to account for inaccuracies in SA classifiers. Our
proposed CLEAM reduces measurement errors significantly, e.g., 4.98%→0.62%
for StyleGAN2 w.r.t. Gender. Additionally, CLEAM achieves this with minimal
additional overhead. Third, we utilize CLEAM to measure fairness in important
text-to-image generator and GANs, revealing considerable biases in these models
that raise concerns about their applications. Code and more resources: https:
//sutd-visual-computing-group.github.io/CLEAM/.

On Measuring Fairness in Generative Models
Christopher H. Teo
Milad Abdollabzadeh
christopher_ Trritoppeer_ee@@@yymmilllleedddd milld_Lbboolahrrdeelleuuddd
Ngai-Man Cheung"
ngaiman cheung@autd.eedu.gg
Singapore University of Technology and Design (SUTD)
Abstract
Recently, there has been increased interest in fair generative models. In this work, we conduct, for the first time, an in-depth study on fairness measurement, a critical component in gauging progress on fair generative models. We make three contributions. First, we conduct study that reveals that the existing fairness measurement framework has considerable measurement errors, even when highly accurate sensitive attribute (SA) classifiers are used. These findings cast doubts on previously reported faimess improvements. Second, to address this issue, we propose CLassifier Error-Aware Measurement (CLEAM), new framework which uses a statistical model to account for inaccuracies SA classifiers. Our proposed CLEAM reduces measurement errors significantly, e.g.. 4.98% 00.6%% for StyleGAN2 w.r.t. Gender. Additionally, CLEAM achieves this with minimal additional overhead. Third, utilize CLEAM to measure fairness in important eex-tto-imaee generator and GANs, revealing considerable biases in these models that raise concerns about their applications. Code and more resources: https: //sutd-visus visual conppuallloomutiigggggeee grw.p.githu..i.. iC/EAAM/
1
Introduction
Fair generative models have been attracting significant attention recently [1 2 7H13. In generative models (14-18], faimess is commonly defined as equal generative quality (T] or equal representation
12 19 20 w.t.. some Sensitive Attributes (SA). In this work, we focus on the more widely example, a generative model is
utilized definition equal representation. In this definition, regarded as fair w.t.t Gender, if generates Male and Female samples with equal probability. This important research topic as such biases in generative models could impact their application efficacy, e.g., by introducing racial bias in face generation of suspects (1 or reducing accuracy when supplementing data for disease diagnosis 22
Fairness measurement for generative models. Recognizing the importance of fair generative models, several methods have been proposed to mitigate biases in generative models 2 7
9 [2]
However, in our work, we focus mainly on the accurate fairness measurement of deep generative models i.e. assessing and quantifying the bias of generative models. This is a critical topic, accurate measurements are essential to reliably gauge the progress of bias mitigation techniques. The general faimess measurement framework is shown in Fig. I (See Sec. 2 for details). This framework is utilized in existing works to assess their proposed fair generators. Central to the fairness measurement framework is SA classifier, which classifies the generated samples w.z.. SA, in order estimate the bias of the generator. For example, if eight out ten generated face images are classified as Male
Cooresponding Author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Inpul
Our Statistical Model for Fairness Mcasurement
StyleGAN2 BlackFale
StyleSwin BlackHalr
Generator
Sample x
1.35%
1.22%
0.62%
0.3%%
Po
class
class
A batch of samples x with unknown probability p' sensitive attribute
0.60
4,98%
0.00
3.49%
8.38%
Go
C1
0.5
6.84%
0.50
Stable Diffusion human being
Cojo
C1j0
C1f1
Coja
*an individunl
person"
"a person
0.29
+9.14%
4.9
5.73%
A35
A37
Sensitive Attribute (SA) Classifier
6.63%
ju (Nez Neg.) Pi (Nep
Nem
0.2
A31
0.38%
p
N
aay" Random variable representing the number of samples with SA value
0.05%
0.7
0.36
A22
Estimation Method
1.77%
0.00%
which are classified by the SA classifier as value
Existing framewook (Bascline) Error o,(P)
CLEAM
Farimation pf from
Ground Truth p5
Figure 1: @ General framework for measuring fairness in generative models. Generated samples with unknown ground-trun (GT) probability p" w.r.f. sensitive attribute (SA) are fed into SA classifier to obtain p. Existing framework (Baseline) uses the classifier output P as estimation p*. In contrast, our proposed CLEAM includes an improved estimation that accounts for inaccuracies in the SA classifier (see Alg. D. ④ Our statistical model for fairness measurement. This model accounts for inaccuracies in the SA classifier and is the base of our proposed CLEAM (see Sec. 4.1). Improvements with CLEAM. CLEAM improves upon Baseline I 2 by reducing the relative error in estimating the GT p for SOTA GANs: StyleGAN2 K3 and StyleSwin 4, and Stable Diffusion Model [5]. First row displays the Baseline and CLEAM estimates for each GAN, using ResNet-18 18 as the SA classifier for Gender and BlackHair. The Baseline incurs significant fairness measurement errors (e.g. 4.98%), even when utilizing a highly accurate ResNet-18 (7997% accuracy). Mcanwhile, CLEAM reduces the error significantly in all setups, e.g. in the first panel, the error is reduced: 4.98% + 0.62%. Similarly, in the second row, CLEAM reduces measurement error significantly in the Stable Diffusion Model [5), using CLIP 6 the SA classifier for Gender, e.g. first panel: 9.14% + 0.05% (Detailed evaluation in Tab. and Tab. 2 Best viewed in color.
by the SA classifier, then the generator is deemed biased at 0.8 towards Male (further discussion in Sec. 2}. We follow previous works 2 and focus binary SA due dataset limitations.
Research gap. this paper, we study a critical research gap fairness measurement. Existing works assume that when SA classifiers are highly accurate, measurement errors should be insignificant. As a result, the effect of errors in SA classifiers has not been studied. However, our study reveals that even with highly accurate SA classifiers, considerable fairness measurement errors could still occur. This finding raises concerns about potential errors in previous works' results, which are measured using existing framework. Note that the SA classifier is indispensable in fairness measurement it enables automated measurement generated samples.
Our contributions. We make three contributions to fairness measurement for generative models. As our first contribution. we analyze the accuracy of fairness measurement on generated samples, which previous works [ 2 9 [2] have been unable to carry out due to the unavailability of proper datasets. We overcome this challenge by proposing new datasets of generated samples with manual labeling w.r.l. various SAs. The datasets include generated samples from Stable Diffusion Model (SDM) (5] -a popular text-to-image generator as well as two Thee--of-hhe-Art (SOTA) GANs (StyleGAN2 K and StyleSwin (4) w.r.f. different SAs. Our new datasets are then utilized in our work to evaluate the accuracy of the existing fairness measurement framework. Our results reveal that the accuracy of the existing fairness measurement framework is not adequate, due to the lack of consideration for the SA classifier inaccuracies. More importantly, we found that even in setups where the accuracy of the SA classifier is high, the error in fairness measurement could still be significant. Our finding raises concerns about the accuracy of previous works' results I 2 12], especially since some of their reported improvements smaller than the margin of measurement errors that observe in our study when evaluated under the same setup; further discussion in Sec. 3
To address this issue, as our second (major) contribution, we propose CLassifier Error-Aware Measurement (CLEAM), a new more accurate fairness measurement framework based on our developed statistical model for SA classification (further details the statistical model in Sec. 4.1.

---

Specifically, CLEAM utilizes this statistical model to account for the classifier's 's inaccuracies during SA classification and outputs more accurate faimess measurement. We then evaluate the accuracy of CLEAM and validate its improvement over existing fairness measurement framework. We further conduct series of different ablation studies to validate performance of CLEAM. We remark that CLEAM is not new fairness metric, but an improved fairness measurement framework that could achieve better accuracy in bias estimation when used with various fairness metrics for generative models.
As our third contribution, we apply CLEAM as an accurate framework to reliably measure biases in popular generative models. Our study reveals that SOTA GANs have considerable biases w.K.f several SA. Furthermore, we observe an intriguing property in Stable Diffusion Model: slight differences in semantically similar prompts could result in markedly different biases for SDM. These results prompt careful consideration on the implication of biases in generative models. Our contributions are:
We conduct a study reveal that even highly-accurate SA classifiers could still incur significant fairness measurement errors when using existing framework.
To enable evaluation of fairness measurement frameworks, we propose new datasets based on generated samples from StyleGAN, StyleSwin and SDM, with manual labeling w.r.l. SA.
We propose a statistically driven fairness measurement framework, CLEAM, which accounts for the SA classifier inaccuracies to output a more accurate bias estimate.
Using CLEAM, we reveal considerable biases in several important generative models, prompting careful consideration when applying them to different applications.
2 Fairness Measurement Framework
Fig illustrates the fairness measurement framework for generative models as in 2 12 Assume that with some input e.g. noise vector for a GAN or text prompt for SDM, a generative model synthesizes a sample x. Generally, as the generator does not label synthesized samples, the ground truth (GT) class probability of these samples w.E.t. a SA (denoted by p²) is unknown. Thus, an SA classifier C, is utilized to estimate p*. Specifically, for each sample x E {x} Cu(x) is the argmax classification for the respective SA. In existing works, the expected value of the SA classifier output over batch of samples, of samples), is used as an estimation of p This estimate may then be used in some faimness metric report the fairness value for the generator, e.g. fairness discrepancy metric between á and uniform distribution p [I (see Supp A.3 for details on how to calculate f). Note that the general assumption behind the existing framework is that with reasonably accurate SA classifier, pe could be an accurate estimation of p" [I] 91. In the next section, we will present a deeper analysis on the effects of an inaccurate SA classifier on fairness mcasurement. Our findings suggest that there could be a large discrepancy between á and p", even for highly accurate SA classifiers, indicative of significant fairness measurement errors in the current measurement framework.
Ex [X)( (or the average of P over multiple batches
One may argue that conditional GANs (cGANs) 23 may be used to generate samples conditioned on the SA, thereby eliminating the need for SA classifier. However, cGANs not considered in previous works due to several limitations. These include the limited availability of large labeled training datasets, the unreliability of sample quality and labels
(25). and the exponentially increasing
conditional terms, per SA. Similarly, for SDM, Bianchi et al. 266 found that utilizing well-crafted prompts to mitigate biases is ineffective due to the presence of existing biases in its training dataset. Furthermore in Sec. 6 utilizing CLEAM, we will discuss that even subtle prompt changes (while maintaining the semantics) result in drastically different SA biases. See Supp G for further comparison between 26 and our findings.
3 A Closer Look at Fairness Measurement
In this section, we take a closer look at the existing faimess measurement framework. In particular, we examine its performance in estimating p* of the samples generated by SOTA GANs and SDM, a task previously unstudied due to the lack of a labeled generated dataset. We do so by designing an experiment to demonstrate these errors while evaluating biases in popular image generators. Following previous works, our main focus is on binary SA which takes values in (0, 1). Note that,
3

---

we assume that the accuracy of the SA classifier Cu is known and is characterized by a = fao, a1 J. where a; is the probability of correctly classifying label For example, for Gender attribute, ag and On are the probability of correctly classifying Female, and Male classes, respectively. In practice, Cu is trained on standard training procedures (more details in the Supp F) and a can be measured during the validation stage of Cu and be considered constant when the validation dataset is large enough. Additionally, p can be assumed to be a constant vector, given that the samples generated can be considered to come from an infinite population, as theoretically there is no limit to the number of samples from a generative model like GAN or SDM.
New dataset by labeling generators output. The major limitation of evaluating the existing faimess measurement framework is the unavailability of p* To pave the way for an accurate evaluation, we create a new dataset by manually labeling the samples generated by GANs and SDM. More specifically, we utilize the official publicly released pre-trained StyleGAN2 3 and StyleSwin on CelebA-HQ (27 for sample generation. Then, we randomly sample from these GANs and utilize Amazon Mechanical Turks to hand-label the samples w.r.t. Gender and BlackHair, resulting in >9K samples for each GAN; see Supp H for more details and examples. Next, we follow a similar labeling process w.r.t. Gender, but with a SDM (5] pre-trained on LAION-SB 28). Here, we input prompts using best practices 126 21H31 beginning with scene description ("A photo with the face of"), followed by four indefinite (gender-neutral) pronouns or nouns [32 33] ("an individual", "a human being", "one person", "a person") to collect #22 high-quality samples. We refer to this new dataset as Generated Dataset (GenData), which includes generated images from three models with corresponding SA labels: that these labeled datasets only provide a strong approximation of p* for each generator, however as the datasets reasonably large, we find this approximation sufficient and simply refer to it as the GT p" Then utilizing this GT p", we compare it against the estimated baseline (p). One interesting observation revealed by GenData is that all three generators exhibit a considerable amount of bias (see Tab and 2): more detail in Sec. 6 Note that for a fair generator we have pó = pi = 0.5, and measuring the pb and p is good proxy for measuring fairness.
GenDaaa-Styleewwer GenData-SDM. We remark
Experimental setup. Here, we follow Choi et al. [ as the Baseline for measuring faimess. In particular, to calculate each value for generator, corresponding batch of 400 samples is randomly drawn from GenData and passed into C, for SA classification. We repeat this for s = 30 batches and report the mean results denoted by J/Base and the 95% confidence interval denoted by PBase- For a comprehensive analysis of the GANs, we repeat the experiment using four different SA classifiers: Resnet-18, R4 3.. MobileNetv2 [35], and 16G-16 [6). Then, to evaluate the SDM, we utilize CLIP 66 to explore the utilization of pre-trained models for zero-shot SA classification; more details on the CLIP SA classifier in Supp. E. As CLIP does not have validation dataset, to measure a for CLIP, we utilize CelebA-HQ, a dataset with a similar domain to our application. We found this to be very accurate approximation; see Supp D.7 for validation results. Note that for SDM, separate P is measured for each text prompt as SDM's output images are conditioned on the input text prompt. As seen in Tab. [ and all classifiers demonstrate reasonably high average accuracy E 184%, 98. 98]. Note that as we focus on binary SA (e.g. Gender: (Male, Female)), both p" and e have two components i.e. pd = [Po.P]}, and P = (po. After computing the (Baaee and PBase. we calculate normalized L1 point error Eu. and interval max error Cp w.e.t. the p (GT) to evaluate the measurement accuracy of the baseline method:
epiaus
PP
MBass
Epeax
max min((easa) pol, max((aasa) -rôl (I)
Pb
Po
Based on our results in Tab. for GANs, we observe that despite the use of reasonably accurate SA classifiers, there are significant estimation errors in the existing fairness measurement framework, i.e. Euei E [4.98%, 17.13%). In particular, looking at the SA classifier with the highest average accuracy of R3 97% (ResNet-18 on Gender), we observe significant discrepancies between GT pt and Paaae. with Ey = 4.98%. These errors generally worsen as accuracy marginally degrades, e.g. MobileNetv2 with accuracy ~ 96% results in ÉyEar 5.45%. These considerably large errors contradict prior assumptions - that for a reasonably accurate SA classifier, we can assume Cyrn. to be fairly negligible. Similarly, our results in Tab. for the SDM, show large Epnn E [1.49%. 9.14%), even though the classifier is very accurate. We discuss the reason for this in more detail in Sec. [.11
Overall, these results are concerning as they cast doubt on the accuracy of prior reported results. For example, imp- weighting I which uses the same ResNet-18 source code as our experiment, reports a 2.35% relative improvement in faimess against its baseline w.r.f. Gender, which falls within the

---

range of our experiments smallest relative error, Spo.. ep -4.98%. Similarly, Teo er al. 2 and Um er al. [2 report relative improvement in fairness of 0.32% and 0.75% compared to imp- -eeiiiggting
I These findings suggest that some prior results may be affected due to oversight of SA classifier's inaccuracies; see Supp. A.4 for more details on how to calculate these measurements.
Remark: In this section, we provide the keystone for the evaluation of measurement accuracy in the current framework by introducing labeled dataset based on generated samples. These evaluation results raise concerns about the accuracy of existing framework as considerable error rates were observed even when using accurate SA classifiers, an issue previously seen to be negligible.
4 Mitigating Error in Fairness Measurements
The previous section exposes the inaccuracies in the existing fairness measurement framework. Following that, in this section, we first develop statistical model for the erroneous output of the SA classifier, p. to help draw a more systematic relationship between the inaccuracy of the SA classifier and error in fairness estimation. Then, with this statistical model, we propose CLEAM measurement framework that reduces error in the measured p by accounting for the SA classifier inaccuracies output more accurate statistical approximation of p*
new
4.1 Proposed Statistical Model for Fairness Measurements
As shown in Fig Fig [). [a), to measure the fairness of the generator, we feed n generated samples to the SA classifier Cu. The output of the SA classifier (p) is in fact a random variable that aims to approximate the p². Here, we propose a statistical model to derive the distribution of p.
As Fig [(b) demonstrates in our running example of binary SA, each generated sample is from class 0 with probability Pū, or from class I with probability pf. Then, generated sample from class where i {0. 13, will be classified correctly with the probability of a,, and wrongly with the probability of af = ag Thus, for each sample, there are four mutually exclusive possible events denoted by with the corresponding probability vector p:
T = [cojo
C1jo
C111
Copl
[''=rpaoo
poab
pia
riall
(2)
where Cily denotes the event of assigning label to sample with GT label j. Given that this process is performed independently for each of the n generated images, the probability of the counts for each output c" Eqn. 2 (denoted by N₂) can be modeled by multinomial distribution, i.e. No Mulli(n, p) [37-39] Note that N models joint probubility distribution of these outputs, i.e. N. P(Neoju: coju: Nemoo Nemn: Napgs) where, Neji: is the random variable of the count for event Caj after classifying generated images. Since p is not near the boundary of the parameter space, and as we utilize large based on the central limit theorem, Multi(n, p) can be approximated by a multivariate Gaussian distribution, Ne NNu,, E), with μ = np and = nM 40 39) where
M is defined as:
M diag(p) pP"
diag(p) denotes a square diagonal matrix corresponding to vector p (see Supp A. for expanded form). The marginal distribution of this multivariate Gaussian distribution gives (one-dimensional) Gaussian distribution for the count of each output c" in Eqn. 2 For example, the distribution of the count for event Cojo, denoted by Negjo can be modeled as N Negio ~ ~ En1).
univariate
Lastly, we find the total percentage of data points labeled as class i when labeling n generated images using the normalized sum of the related random variables, i.e. ÿ; Σ, Ney,, For our binary example, P. can be calculated by summing random variables with Gaussian distribution, which results in another Gaussian distribution 41 i.e. Po ~ NNppo 82 po where:
Ap. -3 (m p4) = p:a0 pi
(4)
22 n2 (En + En1 + 2 = [ (pnan)3) + (piai - (piaa)))) Apiri (5)
Similarly P1 N N(pp. 03,,) with Pip = (me + us)/n, and 02, =(Ez + Egs 2222)/n² which is aligned with the fact that &1 Po.
5

---

Algorithm 1: Computing point and interval estimates using CLEAM.
Require:
accuracy of SA classifier, at Compute SA classifier output P p: , } for s batches of generated data.
2 Compute sample mean ügg and sample variance 0f using 6 and 77).
3 Use (8) to compute point estimate JCLEAM
Use TOp to compute interval estimate PCLEAM
Remark: In this section, considering the probability tree diagram in Fi) we propose a joint distribution for the possible events of classification (Nea1)) ) and use it to compute the marginal distribution of each event, and finally the distribution of the SA classifier outputs (0o. and #i). Note i.e. acc= 100%) the j converges
that considering Eqn. 4 only with perfect classifier (o;
to Po- However, training a perfect SA classifier is not practical e.g. due to the lack of an appropriate dataset and task hardness [42] 43). As result, in the following, we will propose CLEAM which instead utilizes this statistical model to mitigate the error of the SA classifier.
4.2 CLEAM for Accurate Fairness Measurement
In this section, we propose inaccuracy of the SA classifier. For this, we use the statistical model, introduced in Sec 4. to compute more accurate estimation of p" Specifically, approximating the maximum likelihood value of p". Then, we use the confidence interval for the observed data (p) to propose an Interval Estimate (IE) for p"
estimation method in faimess measurement that considers the
first propose Point Estimate (PH) by
Point Estimate (PE) for p". Suppose that we have access to 8 samples of p denoted by (f'...... D²) J. i.e. SA classification results
batches of generated data. We can then use the proposed statistical model to approximate the p". In the previous section, we demonstrate that we can model P using a Gaussian distribution. Considering this, first, use the available samples to calculate sample- based statistics including the mean and variance of the PU samples:
p3 = Σi= P3 P;
(6)
- 1 3
(7)
For Gaussian distribution, the Maximum Likelihood Estimate (MLE) of the population mean is its sample mean P3g [44]. Given that is large enough (e.g. 8 > 30), we can assume that jم, is good approximation of the population mean [45]. and equate to the statistical population mean Pu. in Eqn. [[sc ssee Supp A.2 for derivation). With that, we get the maximum likelihood approximation of p". which we call the CLEAM'' point estimate, PCLEAM:
PCLEAM (pt) (jāSo a1)/(ao- - a1))
(CLEAM = MCLEAM(P))
(8)
Notice that JACEEMM accounts for the inaccuracy of the SA classifier.
Interval Estimate (IE) for p". In the previous part, propose PE for p" using the statistical model, and cample-based mean py.. However, as we use only s samples of p, Pe, may not capture the exact value of the population mean. This adds some degree of inaccuracy into JcceAM In fact, in our framework, i equals Mi when 5 + oo. However, increasing each unit of significantly increases the computational complexity, each requires n generated samples. To address this, recall that pu follows a Gaussian distribution and instead utilize frequentist statistics [41] to propose 95% confidence interval (CI) for p*. To do this, first we derive the CI for Pipo
ipg 1.96
Ete < 1 S Pp. + 1.96 V
Eoo v
Then, applying Eqn 4 to Eqn 9 gives the lower and upper bounds of the approximated 95% CI for po: ((p))uu(p)
(Mp. F 1.96(# /V3) o4) (010///0oaa// af)
(10)
This gives the interval estimate of CLEAM, PCLEAM [C(p)),
range of values that can be approximately 95% confident to contain P The range of possible values for pi can be simply derived considering pi = 1 Ps- The overall procedure of CLEAM is summarized in Alg. Now,
with the IE, we can provide statistical significance to the reported fairness improvements.

---

Table 1: Comparing the point estimates and interval estimates of Baseline M. Diversity 46 and our CLEAM in estimating p* of StyleGAN2 [ and StyleSwin [4] with the proposed Data datasets. We utilize SA classifiers Resnet-18/34 (R18, R34) MobileNetv2 [5 and VGG-16 (V16)[36), with different accuracies a, to classify samples w.rl attributes Gender and BlackHair. The Po value of each GAN w.r.t SA is determined by manually hand-labeling the generated data. We repeat this for 5 experimental runs and report the mean error rate, per Eqn. See Supp D.1 for the standard deviation of PE and IE.
Point Estimate
Interval Estimate
a (mm Avg.
Baseline
Diversity
CLEAM (Ours)
Baseline
Diversity
CLEAM (Ours)
c,(
(0)
POLEAM (0(
POLRAC
StyleGANZ
Gender with GT class probability PG-0.642
R18 (0.947,
0.983)
0.97
0.630
4.98%
0.638
0.62%
[0 602, 0.618] 6.23%
[0.629, 0.646]
2.02%
R34
10,952, 0.976]]
0.95
7.17%
1.25%
(0.5899 0.599]
[0.628, 0.638]
2.18%
MN2
[0.938.
0.975;
0.96
0.607
5.45%
0.637
0.78%
[0.602, 0.6121
6.23%
[0.632. 0,643]
1.56%
V16
(0.801, 0.919)
0.86
0.532
17.13%
0,550
14.30%
0.636
0.93%
[0.526, 0.538] 18.06% [0.536, 0.564) 16.51% [0.6288
0.644]
2.18%
Average Br
8.68%
30%
0.90%
9.70%
16.51%
1.99%
BlackHair with GT cluss probubility PJ-0.643
R18
[0 869, 0.885)
0.88
0.599
6.84%
0.641
0.31%
[0.59], 0.607]
8.08%
[0.631, 0.652]
1.40%
R34
[0.834
0.88
1.644
[0.561. 0:572]
12.75%
[0.637. 0.651]
1.34%
MN2
[0.839,
0.881)
0.86
0.579
9.95%
0.639
0.62%
[0.574, 0.584]
10.73%
[0.632, 0.647]
1.71%
V16
(0.851, 0.836)
0.84
0,605
6.22%
0,582 9.49%
0.640
0.47%
[0.597, 0.608]
7.15% [0,568, 0.596]
11.66%
[0.632, 0.648)
1.71%
Average Eroor
8.75%
9.49%
0.39%
9.68%
11.66%
1.52%
(B) StyleSwin
Gencer with GT class probability P0-665
R18
09947,09933
0.97
0.620
5.49%
1.22%
(0.612.0622)]
6.70%
[06639.0.688]
2.59%
R34
[0.932. 0.976)
0.95
0.610
7.01%
0.649
1.07%
(0.605.0.615)
1.77%
[0.613.0.654]
1.98%
MN2
[0.938. 0.975
0.96
0.623
5.03%
0.655
0.15%
[0.618, 0.629)
5.79%
[0.649.0.661]
1.07%
V16
[0.801,
0,86
0.555
15.39%
0.562 14.33%
0.668
1.8%
10. (0.569.0.560]
16.31% [0.548,0.576]
16.46%
[0.660.0.675]
2.90%
Average Eroo:
8.23%
14. 14.33%
1.07%
9.14%
16.40%
2.14%
BlackHair with GT class probability
PC-0,668
R18
[0.869,
885)
0.88
8.38%
0.659
1.35%
[0.605.0
9.43%
[0.649.6670]
2.84%
R34
[0.834,
0.916;
0.88
0.581
13.02%
0.662
0.90%
[0.576.0.586]
13.77%
[0.656.0.669]
1.80%
MN2
(0.839,
0.881)
0.86
0.596
10.78%
0.659
1.35%
[0.591,0.600)
11.50%
[0.652.0.666]
2.40%
V16
[0.851, 0.836)
0.84
0.625
6.44%
0.608
8.98%
0.677
1.35%
[0.620.0.630]
7.19%
[0.590,0.626]
11.68%
[0.670.0.684]
2.40%
Average Eroo::
9.66%
9.98
1.24%
10.47%
11.68%
2.36%
Table 2: Comparing the point estimates and interval estimates of Baseline and CLEAM in estimating
the p* of the Stable Diffusion Model [3] with the GenData-SDM dataset. We use prompt input starting with A photo with the face of" and ending with synonymous (Gender neutral) prompts. We
utilized CLIP as the classifier for Gender, to obtain p.
Point Estimate
Interval Estimate
Prompt
GT
Baseline
CLEAM (Ours)
Baseline
CLEAM (Ours)
P5
Hitaee
e2(L)
POLEAN
fu(l)
PCLEAM
ca(L)
a=100.98,.0.975.. Avg.
0-0.987, CLIPP
-endar
"A photo with the face of an individual"
0.186
0.203
9.14%
0.187
0.05%
[0.198. 0.208
11.83%
[0.182.0.192
3.23%
photo with the face a human being"
0.262
0.277
5.73%
0,263
0.38%
[0,270 0.285
8.78%
[0.255 0.277
3.44%
"A photo with the face of one person"
0.226
0.241
6.63%
0,230
1.77%
[0.252, 0.251
11.06%
[0.220 0.239
5.75%
"A photo with the face of person"
0.548
0.556
1.49%
0,548
0.00%
[0,545 0.566
3.28%
[0,537
0.558
2.01%
Average Erer
5.75%
0.44%
8.74%
3.61%
Stable Diffusion
"an individual"
human being"
ni "A photo with the face of one person"
person"
a person"
ruMalai H IF --
Trire
photo with the face of
Figure 2: LHS: Applying CLEAM to assess Gender bias in SDM with CLIP 6 Here, we utilize synonymous neutral prompts, prefixed with "A photo with the face of " as the input. We found that subtle changes to the prompts resulted in significant changes in the bias. RHS: Illustrating the shift in Gender bias. Using the same random seeds (per column), we generate 10 samples from two prompts. Note how in some columns, Gender changes while retaining general features e.g. pose.
7

---

5 Experiments
In this section, we first evaluate fairness measurement accuracy of CLEAM on both GANs and SDM 5.1) with our proposed GenData dataset. Then we evaluate CLEAM's robustness through some ablation studies (Sec. [.2). To the best of our knowledge, there no similar literature for
improving fairness measurements in generative models. Therefore, we compare CLEAM with the
two most related works: a) the Baseline used in previous works [ 2 9 12 b) Diversity [46]
which computes disparity within dataset via an intra-dataset pairwise similarity algorithm. We remark that, as discussed by Keswani et al. 46] Diversity is model-specific using VGG-16 [36]; see Supp. D.2 for more details. Finally, unless specified, batches of images from the generators with batch size n = 400. For a fair comparison, all three algorithms use the exact same inputs. However, while Baseline and Diversity ignore the SA classifier inaccuracies, CLEAM makes good use of it to rectify the measurement error. As mentioned in Sec. 4.2, for CLEAM, we utilize a measured real samples, which we found to be good approximation of the a measured on generated samples (see Supp. D.7 for results). We repeat each experiment times and report the mean value for each test point for both PE and IE. See Supp D.1 for the standard deviation.
repeat the experiments with 30
5.1 Evaluating CLEAM's Performance
CLEAM for fairness measurement of SOTA GANs - StyleGAN2 and StyleSwin. For a fair comparison, we first compute samples of p. one for each batch of n images. For Baseline, the mean P value as the PE (denoted by (Baase), and the 95% confidence interval as IE (Prase). With the same samples of p. we apply Alg. to obtain JCCEAM and PCLEAM- For Diversity, following the original source code [46]. a controlled dataset with fair representation is randomly selected from held-out dataset of CelebA-HQ [27]. Then,
a VGG-16 (36) feature extractor and compute
Diversity, 8. With we find Po (6 + 1)/2 and subsequently ('D and PDiv from the mean and 95% CI (see Supp D.2 for more details on diversity). We then compute @pAZAam Cyowr EPELIAM and Epee with Eqn D by replacing the Baseline estimates with CLEAM and Diversity.
As diseussed, 4.98% I Curmme S 17.13%, due to a lack of consideration for the inaccuracies of the SA classifier. We note that this problem is prevalent throughout the different SA classifier architectures, even with higher capacity classifiers e.g. ResNet-34. Diversity, a method similarly unaware of the inaccuracies of the SA classifier, presents a similar issue with 8.98% S Eany & 14.33% In contrast, CLEEAM dramatically reduces the error for all classifier architectures. Specifically, CLEAM reduces the average point estimate error from Cpae 2 8.23% to (дCLEA S 1.24%, in both StyleGAN2 and StyleSwin. The IE presents similar results, where in most cases PCLEAM bounds the GT value of p*.
results in T show that the baseline experiences significantly large errors of
CLEAM for fairness measurement of SDM. We evaluate CLEAM in estimating the bias of the SDM w.r.t. Gender, based on the synonymous (gender-neutra)) prompts introduced in Sec. 3 Recall that here we utilize CLIP the zero-shot SA classifier. Our results in Tab 2 as discussed, show that utilizing the baseline results in considerable error (1.49% I Eppnn I 9.14%) ) for all prompts, even though the SA classifier's average accuracy was high, x= 98.7% (visual results in Fig D). A closer look at the theoretical model's Eqn. F.gn.4rrvvee reveals that this is due to the larger inaccuracies observed in the biased class (a() coupled with the large bias seen in pf which results in J'aaee deviating from Po- In contrast, CLEAM accounts for these inaccuracies and significantly minimizes the error to CpELEa S 1.77%. Moreover, CLEAM's IE is able to consistently bound the GT value of pt-
5.2 Ablation Studies and Analysis
Here, we perform the ablation studies and compare CLEAM with classifier correction methods. We remark that detailed results of these experiments are provided in the Supp due to space limitations.
CLEAM for measuring varying degrees of bias. As we cannot control the bias in trained generative models, to simulate different degrees of bias, we evaluate CLEAM with a pxeudo-generaton Our results show that CLEAM is effective at different biases (ph E [0.5.0.9] reducing the average error from 2.80% S Cpsar & 6.93% to PpCLeAx I 0.75% on CelebA 47 w.n.
and W.E.f. Cat Dog. See Supp D.3 and D.4 for full experimental results.
AFHQ 48
8

---

CLEAM vs Classifier Correction Methods [49]. CLEAM generally accounts for the classifier's s inaccuracies, without targeting any particular cause of inaccuracies, for the purpose of rectifying the fairness measurements. This objective is unlike traditional classifier correction methods it does not aim to improve the actual classifier's accuracy. However, considering that classifier correction methods may improve the fairness measurements by directly rectifying the classifier inaccuracies, we compare its performance against CLEAM. As an example, we utilize the Black Box Shift Estimation / Correction (BBSE/ BBSC) [49] which considers the label shift problem and aims to correct the classifier output by detecting the distribution shift. Our results, based on Sec. 5.T setup, show that while BBSE does improve on the fairness measurements of the baseline i.e. 4.20% I Cyaner I 3.38%, these results are far inferior to CLEAM's results seen in Tab. I In contrast, BBSC demonstrates no improvements in fairness measurements. See Supp D.8 for full experimental results. We postulate that this is likely due to the strong assumption of label shift made by both methods.
Effect of batch-size. Utilizing experimental setup in Sec. 1 for batch size E[100,600), our results in Fig. 3 show that n=400 an ideal batch size, balancing computational cost and measurement accuracy. See Supp F for full experimental details and results.
Baseline
CLEAM
Erros
100 SCU 500 5O0
100 200
100 700 4:0 StyleSwin (Gendar)
100 200 500 500 StyleSwin (RlackHair)
StyleGAN2 (Sandar)
StyleGAN2 Haldd
Figure 3: Comparing the point error en for Baseline and CLEAM when evaluating the bias of GenData-CelebA with R8sNet-18, while varying sample size, n.
6 Applying CLEAM: Bias in Current SOTA Generative Models
In this section, we leverage the improved reliability of CLEAM to study biases in the popular generative models. Firstly, with the rise in popularity of text-to-image generators [5] 50 52 revisit our results when passing different prompts, with synonymous neutral meanings to an SDM, and take a closer look at how subtle prompt changes impact bias w.r.t. Gender. Furthermore, we further investigate if similar results would occur in other SA, Smiling. Secondly, with the shift in popularity from convolution quality, we determine whether the learned bias would also change. For this, we compare StylesSwin (transformer) and StyleGAN2 (convolution), which are both based the same architecture backbone.
transformer-baset architectures (53 55), due to its better sample
Our results, on SDM, demonstrate that the use of different synonymous neutral prompts [32 33
results in different degrees of bias w.r.f. both Gender and Smiling attributes. For example in Fig. 2 a semantically insignificant prompt change from "one person" to "a person" results a significant shift in Gender bias. Then, in Fig. 4a we observe that while the SDM w.r.f. our prompts appear to be heavily biased to not- Soti-Sniing,. having "person" in the prompt appears to significantly reduce this bias. This suggests that for SDM, even semantically similar neutral prompts [32 B3) could result
in different degrees of bias, thereby demonstrating certain instability in SDM. Next, our results in Fig. compore the bias in StyleGAN2, StylesSwin, and the training CelebA-HQ dataset over an extended number of SAs. Overall, we found that while StyleSwin produces better quality samples 44. the same biases still remain statistically unchanged between the two architectures i.e. their IE overlap. Interestingly, our results also found that both the GANs were less biased than the training dataset itself.
7 Discussion
Conclusion. In this work, we address the limitations of the existing fairness measurement framework. Since generated samples are typically unlabeled, we first introduce a new labeled dataset based on three state-of-thh--ar generative models for our studies. Our findings suggest that the existing framework, which ignores classification inaccuracies, suffers from significant measurement errors, even when the SA classifier is very accurate. To rectify this, we propose CLEAM, which considers these inaccuracies in its statistical model and outputs a more accurate fairness measurement. Over- all, CLEAM demonstrates improved accuracy over extensive experimentation, including both real

---

Diffesion
"an individual"
being"
a) "A photo with the face a human being"
"one persou"
a person
Pt teet-taatiing Pt: (imuliing)
Point
photo
porson
SA Smiling using CLEAM. LHS: input to SDM. CI ELEAM identifies that having "person" the prompt results more Smiling samples generated. Visual comparison of the samples generated for two different but semantically similar text prompts. Samples in the same column generated using the same random seed. Notice how the SA (smiling of the samples changes with slight difference in the prompts.
(a) Evaluating hias SDM
Bias value with different prompes as
RHS:
NoBeard (B/N3)
ReavyMo (NRX/AM)
(NR/3]
Chulaby (NC/C)
Lsche
Smi Smiling 1inn (N8/S)
Young (NY/X)
Blaukflaie (ND0/DI))
Gender (2/M)
CelebA-HQ dataset
StyleGAN2
StyleSwin
(Label
(b) CLEAM on StyleGAN2 3 and StyleSwin [4. both pre-trained on CelebA-HE -Q but are based on different architecture variants. We utilize a ResNet-18 18 and compare the hias w.z. various SAs.
Figure 4: Applying CLEAM further assess the bias popular generative models.
generators and controlled setups. Moreover, by applying CLEAM to popular generative models, we uncover significant biases that raise efficacy concerns about these models' real-world application.
Broader Impact. Given that generative models are becoming more widely integrated into our everyday society e.g. text-to-image generation, it is important that have reliable means to measure fairness in generative models, thereby allowing us to prevent these biases from proliferating into new technologies. CLEAM provides step in this direction by allowing for more accurate evaluation. We remark that our work does not introduce any social harm but instead improves on the already existing measurement framework.
Limitations. Despite the effectiveness of the proposed method along various generative models, our work addresses only one facet of the problems in the existing fairness measurement and there is still room for further improvement. For instance, may beneficial to consider SA to non-binary e.g. when hair color is not necessary fully black (grey). Additionally, existing datasets used to train classifiers are commonly numan-annotated, which may itself contain certain notions of bias. See Supp. for further discussion.
Acknowledgements
This research is supported by the National Research Foundation, Singapore under its AI Singapore and SUTD project PIE-SGP-AI-2018-00 This research work is also supported by the Agency for Science, Technology and Research (A STAR) under MTC Programmatic Funds (Grant No. M23L.66021)). This material is based on research/work support in part by the Changi General Hospital and Singapore University of Technology and Design, under the HealthTech Innovation Fund (HTTF Award No. CGH-SUTD-2011-0044 We thank anonymous reviewers for their insightful feedback and discussion.
Programmes (AISG Award No.: AISG2-TC-2022-008
10

---

References
[1] Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair Generative Modeling via Weak Supervision. In Proceedings of the 37th International Conference on Machine Learning, pages 1887-1898. PMLR, November 2020.
[2] Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Cheung. Fair generative models via transfer learning. In Proceedings the AAAI Conference on Artificial Intelligence, volume 37, pages 2429-2437, 2023.
[3] Tero Karras, Samuli Laine, and Timo Aila. A Style- Generator Architecture for Generative Adversar- ial Networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4401-4410, June 2019.
[4] Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo. Styleswin: Transformer basedd gan for high-resolution image generation, 2021.
[5] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models, 2021.
[6] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision, February 2021.
[7] Eric Frankel Edward Vendrow. Fair Generation Through Prior Modification. 32nd Conference on Newral Information Processing Systems (NeurIPS 2018), June 2020.
[8] Ahmed Imtiaz Humayun, Randall Balestriero, and Richard Baraniuk. MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining. In International Conference Learning Representations, January 2022.
[9] Shuhan Tan, Yujun Shen, and Bolei Zhou. Improving the Fairness of Deep Generative Models without Retraining. 2 [cs]. December 2020.
[10] Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, and Mario Fritz. Inclusive GAN: Improving Data and Minority Coverage in Generative Models, August 2020.
[11] Vongani H. Maluleke, Ncerja Thakkar, Tim Brooks, Ethan Weber, Trevor Darrell, Alexci A. Efros, Angjoo Kanazawa, and Devin Guillory. Studying Bias in GANs through the Lens of Race, September 2022.
[12] Soobin Um and Changho Suh. A Fair Generative Model Using Total Variation Distance. openreview, September 2021.
[13] Milad Abdollahzadeh, Touba Malekzadeh, Christopher TH Teo, Keshigeyan Chandrasegaran, Guimeng Liu, and Ngai-Man Cheung, A survey on generative modeling with limited data, few shots, and zero shot. arXiv preprint ar3iv:2307. 14397, 2023.
[14] Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):
139 144, 2020.
[15] Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in neural information processing systems, 33: 12104-12114, 2020.
[16] Yunqing Zhao, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Man Man Cheung. Few-shot image generation via adaptation kernel modulation. Advances Neural Information Processing Systems, 35: 194479994944,, 2022.
[17] Keshigeyan Chandrascgaran, Ngoc-Trung Tran, Alexander Binder, and Ngai-Man Chcung. Discovering transferable forensic features for enn-generated images detection. In European Conference Computer Vision, pages 671 689. Springer, 2022.
[18] Yunqing Zhao, Chao Du, Milad Abdollahzadeh, Tianyu Pang. Min Lin, Shuicheng Yan, and Ngai- Man Cheung. Exploring incompatible knowledge transfer in few-shot image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7380-7391 2023.
11

---

[19] Ben Hutchinson and Margaret Mitchell. 50 Years of Test (Un)fairness: Lessons for Machine Learning. In Proceedings the Conference on Fairness, Accountability, and Transparency. FAT* '19, pages 49-58. New York, NY. USA. January 2019, Association for Computing Machinery. ISBN 978-1-4503-6125-5. doi: 10.1 14111/255560002876666
[20] Christopher T. H. Teo and Ngai-Man Cheung. Measuring Fairness in Generative Models, 38th International Conference on Machine Learning (ICML) Workshop, July 2021.
[21] Harsh Jaykumar Jalan, Gautam Maurya, Canute Corda, Sunny Dsouza, and Dakshata Panchal. Suspect Face Generation. In 2020 3rd International Conference on Communication System, Computing and IT Applications (CSCITA), pages 73 78,78, April 2020. doi: 10. CTCCCCCIIAA47322222009999
[22] Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldherger, and Hayit Greenspan. Gan- based synthetic medical image augmentation for increased can performance in liver lesion classification. Neurucomputing, 321:321-331 2018.
[23] Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.172 2014.
[24] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In JCML, 2017.
[25] Kiran K Thekumparampil. Ashish Khetan, Zinan Lin, and Sewoong Oh. Robustness of conditional GANs to noisy labels. In Advances in Neural Information Procesing Systems, volume 31. Curran Associates, Inc., 2018.
[26] Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale, November 2022.
[27] Cheng-Hau Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[28] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278 25294, 2022.
[29] Vivian Liu and Lydia B Chilton. Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1-23, 2022.
[30] Stable Diffusion: Prompt Guide and Examples. August 2022.
[31] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to Prompt for Vision-Language Models. International Journal of Computer Vision, 130(9)23337-2348, September 2022. ISSN 0920-5691, 1573-1405. doi: 10.1007/s1 1213--22--126------
[32] Martin Haspelmath. Indefinite Pronouns. Oxford University Press, 1997. ISBN 978-0-19-829963- 118-0-19-823560-4 doi: 10.
[33] Abigail C Saguy and Juliet A Williams. A little word that means lot: A reassessment of singular they in a new era of gender politics. Gender & Society, 36(1): 2022.
[34] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 770-778, June 2016.
[35] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chich Chen. Mobile Inverted Residuals Linear Bottlenecks. 2018 IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 4510-45 4520, Salt Lake City, UT, June 2018. IEEE, ISBN 978-1 1-1388-6420-99 doi: 10. 11109//VR..0118.000..
[36] Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. ICLR2015, September 2014.
[37] C Radhakrishna Rao. Maximum likelihood estimation for the multinomial distribution. Sankhya: The Indian Journal of Statistics (1933-1960), 18(1/2):139 148, 1957.
12

---

[38] Harry Kesten and Norman Morse. A property of the multinomial distribution. The Annals of Mathematical Statisticx, 30(1):120-127. 1959.
[39]
Athanasios Papoulis. Probability, Random Variables and Stochastic Processes with Errata Sheet. McGraw- Hill Europe. Boston, Mass., 4th edition edition, January 2002. ISBN 978-0007-112261---
[40] Charles J Geyer. Stat 5101 Notes: Brand Name Distributions. University of Minnesota, Stat 5101:25, January 2010.
[41] Nathaniel R Goodman. Statistical analysis based on certain multivariate complex gaussian distribution (an introduction). The Annals of mathematical statistics, 34(1):152-177, 1963.
[42] Milad Abdollahzadeh, Touba Malekzadeh, and Ngai-Man Man Cheung. Revisit multimodal meta-learning through the lens of multi-task learning. Advances in Neural Information Processing Systems, 35:14632-
14644, 2021.
[43] Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji. Charless C Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning. Proceedings of the IEEE/CVF international conference on computer vision, pages 6430-6439, 2019.
[44] Theodore Wilbur Anderson and Ingram Olkin. Maximum-
estimation of parameters of multivariate normal distribution. Linear algebra and its applications, 70:147-171, 1985.
[45] Renato A Krohling and Leandro dos Santos Coelho. Coevolutionary particle swarm optimization using gaussian distribution for solving constrained optimization problems. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybermetics), (117)))447-14460 2006.
[46] Vijay Keswani and L. Elisa Celis. Auditing for Diversity Using Representative Examples. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pages 860-870, Virtual Event Singapore, August 2021. ACM. ISBN 978- 1-400-88332 doi: 10.1 00.11444444548834677242
[47] Ziwei Liu. Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild. Proceedings of International Conference Computer Vision (ICCV), September 2015.
[48] Yunjey Choi, Youngjung Uh. Jaejun Yoo, and Jung-Wor Woo Ha. StarGAN v2: Diverse Image Synthesis for Multiple Domains, April 2020.
[49] Zachary Lipton. Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In Proceedings of the 35th International Conference Machine Learning, pages 3122-3130. PMLR, 10-15 2018
[50] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-Shot Text-to-Iman Image Generation. February 2021.
[51] Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, and Daniel Cohen-Or. StyleGAN-NADA: CLIP- Guided Domain Adaptation of Image Generators, December 2021.
[52] Or Patashnik. Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. StyleCLIP: Text-Driven Manipulation StyleGAN Imagery, March 2021.
[53] Maithra Raghu, Thomas Unterthiner, Simon Korbllith, Chiyuan Zhang, and Alexey Dosovitskiy. Do Vision Transformers See Like Convolutional Neural Networks?, March 2022.
[54] Drew A. Hudson and Larry Zitnick. Generative Adversarial Transformers. In Proceedings of the 38th International Conference on Machine Learning, pages 4487-4499. PMLR, July 2021.
[55] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual Prompt Tuning. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors. Computer Vision - EECC 2022, Lecture Notes in Computer Science, pages 709-727, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-11827-4, doi: 1007/ 978-3-031- 108-3--03-19827-4 41.
13


Reviews:
[{"review_id": "4XsVWWPE5j", "reviewer": "Reviewer_2H4p", "summary": "The objective of this paper is on measuring the fairness in generative models. There are three contributions. (i) Consideration of measurement errors of sensitive attribute (SA) classifiers in fairness measurement of generative models. (ii) A classification error aware measurement framework, called CLEAM, which based on statistical model accounts for the inaccuracies of SA classifier to reduce measurement error in generative models. (iii) As an application, the authors demonstrate that CLEAM can be applied to measure fairness in text-to-image generator and GANs. \n\n", "questions": "- The presented method considers binary sensitive attributes. However, modern generative models easily generates images with multi-sensitive features. How do the authors extend to this scenario? Is naive enumeration the only possible way?\n\n- In Equation (2), are $p_i$ and  $\\alpha_i$ known?\n\n- In line 222, is $ s = 30 $ large enough?\n\n- Is there any future work based on this paper?\n\nPlease also address the points in \"Weakness\" above.", "limitations": "NA.", "rating": 6, "confidence": 1, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well written, with clear intuitions, illustrations, and experimental results. ", "weaknesses": "- Putting the application of gender bias in the introduction seems to be out of place, possibly undermining the main framework CLEAM. Similarly, table 1 is also out of place, throwing numbers to the readers without explaining the setup. \n\n- The authors make an assumption of ground-truth labels of sensitive attributes, which may not be available in practice.\n\n- Some of the human faces in Figure 2 and figure 3(a) are different, revealing the uncertainty of the generative models. I understand the intuition of the authors to provide a demonstration of the paper at an early part of the paper. Please clarify if I misunderstood anything.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">**Q1**: Putting the application of gender bias in the introduction seems to be out of place, possibly undermining the main framework CLEAM. Similarly, table 1 is also out of place, throwing numbers to the readers without explaining the setup\n\n**A1**: Thank you for your suggestion. We will fix the placement of Fig. 2 and Tab. 1\n\n$ $\n\n>**Q2**:The authors make an assumption of ground-truth labels of sensitive attributes, which may not be available in practice.\n\n**A2**: We thank the reviewer for the comment. Importantly, we would like to respectively clarify that ground-truth (GT) labels are not used in our proposed method. \n\nMeanwhile, the GT labels for the generated data (GenData introduced in our work) is used **solely for evaluation** of different methods. Again, the GT labels for the generated data is not used in our proposed method.\n\nOur method assumes availability of a SA classifier and its validation accuracy ($\\alpha$). In practice, $\\alpha$ is computed during the validation stage of a SA classifier. Note that a SA classifier is needed in all existing measurement methods.\n\n$ $\n\n>**Q3**: Some of the human faces in Figure 2 and figure 3(a) are different, revealing the uncertainty of the generative models. I understand the intuition of the authors to provide a demonstration of the paper at an early part of the paper. Please clarify if I misunderstood anything.\n\n**A3**: We apologize for any confusion. To clarify, the images displayed across Fig. 2 and Fig. 3(a) contain some different seed values, while in between each individual figure we utilize the same seed value. However, for improved clarity, we have included a revised Fig.3(a) (in the attached pdf, Fig. 1(rebuttal) ) with the same seed values for the displayed images as Fig 2.\n\n$ $\n\n>**Q4**: The presented method considers binary sensitive attributes. However, modern generative models easily generates images with multi-sensitive features. How do the authors extend to this scenario? Is naive enumeration the only possible way? \n\n**A4**: Thank you for your comment. Considering the multi-sensitive attributes is indeed an interesting idea. However, we remark that in current literature, fairness of generative models has been studied for binary sensitive attributes mainly due to lack of an available large labeled dataset needed for systematic experimentation. As a result, CLEAM similarly focuses on binary SA to address a common flaw in the evaluation process of the many proposed State-of-the-Art methods.\n\nAssuming that constraint of dataset is addressed, our same CLEAM approach can be easily extended to a multi-label setting. For example, given a 3 label sensitive attribute where $p^*_j$ is the probability of generating a sample with label $j$ and ${\\alpha}\\_{i|j}$ denotes the probability (“accuracy”) of the SA classifier in classifying a sample with GT label $j$ as $i$ for $i,j \\in \\\\{0,1,2\\\\}$, Fig. 2 (Rebuttal) in the attached Pdf shows our statistical model for this setting. We can then similarly solve for the $p^*$ point estimate by solving the matrix:\n$$ \\begin{bmatrix}\n\\alpha\\_{0|0} & \\alpha\\_{0|1} & \\alpha\\_{0|2}\\\\\\\\\n\\alpha\\_{1|0} & \\alpha\\_{1|1} & \\alpha\\_{1|2}\\\\\\\\\n\\alpha\\_{2|0} & \\alpha\\_{2|1} & \\alpha\\_{2|2}\\\\\\\\\n\\end{bmatrix}\n\\begin{bmatrix}\np^*\\_0 \\\\\\\\\np^*\\_1 \\\\\\\\\np^*\\_2\n\\end{bmatrix}\n= \\begin{bmatrix}\n\\ddot{\\mu}\\_\\hat{p_0} \\\\\\\\\n\\ddot{\\mu}\\_\\hat{p_1} \\\\\\\\\n\\ddot{\\mu}\\_\\hat{p_2}\n\\end{bmatrix}$$\n\nWe will include the detailed procedure and full solution in the final version of the paper.\n\n$ $\n\n>**Q5**: In Equation (2),  are $p_i$  and $ \\alpha_i$  known?\n\n**A5**: We apologize if it was unclear, as discussed in Sec.3 of the main paper, $p^*_i$ is the unknown GT distribution that we are trying to find and $\\alpha_i$ is a known accuracy of the SA classifier. In practice, accuracy of the classifier ($\\alpha_i$) is evaluated during the validation stage, a common practice when training discriminative models (line 113-114). On the other hand, as first introduced in Sec.2, the ground truth SA distribution ($p^*_i$) is unknown as the SA classifier only outputs some approximation $\\hat{p}$. Hence, in Sec. 4.1., we provide the setup to model the theoretical distribution, and then utilize it in Sec.4.2 to propose CLEAM for a better estimation of $p^*_i$ with eqn. (8) and eqn. (10).\n\n$ $\n\n>**Q6**: In line 222, is s=30 large enough?\n\n**A6**: Thanks, our experimental results in Supp. F-Fig 5 demonstrates that $s$=30 is sufficiently large enough to significantly reduce measurement error, where increasing $s$ does not result in considerable performance improvement. Supp. C.1 then further substantiates these findings by showing that $s$=30 provides a close approximation between the sample based estimate and the model based estimates. \n\n$ $\n\n>**Q7**: Is there any future work based on this paper?\n\n**A7**: Thanks for your question. As future work, now with a more reliable fairness measurement framework, firstly, we consider utilizing this more accurate measurement as a bias mitigation technique during fair generative model training. Secondly, inspired by the findings in Sec.6, we propose to further look into the biases that exist in stable diffusion models (discussed in Sec 6 of our manuscript) and identify their possible sources."}, {"review_id": "OhyxnNkgY8", "reviewer": "Reviewer_uNfr", "summary": "This paper proposes a framework for fairness measurement. It first shows that existing framework has considerable measurement errors even when highly accurate sensitive attribute classifiers are used, then propose CLassifier Error-Aware Measurement (CLEAM), a new framework which uses a statistical model to account for inaccuracies in SA classifiers. ", "questions": "plz refer to the \"weaknesses\" part.", "limitations": "The structure of paper is hard to follow and somehow boring for readers.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Shows the significant measurement errors of existing frameworks by experiments.\n2. Propose new datasets based on generated samples with manual labeling w.r.t. SA.\n3. Proposes a simple statistical approximation method is proposed to obtain a stable and accurate estimation of the GT probabilities.", "weaknesses": "1. The organization of the paper is kind of hard to follow. The first contribution takes too long and may confuse the readers.\n2. The introduction to the proposed method is too short and not very solid. Some theoretical support may be better. Maybe you can talk about cases when using distributions other than Gaussian to approximate the distributions.\n3. Only a public dataset CelebA-HQ is used. It's better to test methods on various datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable suggestion. We will shorten the first contribution and discuss more on our proposed method in the introduction following the Reviewer's suggestion.\n\n$ $\n\n>**Q1**: The introduction to the proposed method is too short and not very solid. Some theoretical support may be better. \n\n**A1**: We apologize if this was unclear and would like to respectfully clarify that the entire proposed solution (Sec.4) is based on theoretical statistical modeling. As mentioned, the proposed method (Sec.4.1) first derives a statistical model to link the observed errors in the fairness measures to the SA classifier’s inaccuracy. Only then are we able to utilize the statistical model to systematically account for the classifier’s error for fairness measurement (Sec.4.2). Furthermore, we remark that our intention was to highlight the essential information to understand CLEAM in the main manuscript, while Supp. A.1 and A.2 provide extensive step-by-step details and derivation of the theoretical model, as indicated in the paper. Nevertheless, we will follow the Reviewer's suggestion and would shift some of these details back into the manuscript.\n\n$ $\n\n>**Q2**: Maybe you can talk about cases when using distributions other than Gaussian to approximate the distributions.\n\n**A2**: Thanks, we would like to clarify that we model our system as a Gaussian Distribution based on “normal approximation to the multinomial” [36,37], an application of the central limit theorem. To the best of our knowledge, this is the most appropriate distribution [36,37] that enables us to derive the distribution of $\\hat{p}$. We will clarify this in the paper.\n\n$ $\n\n>**Q3**: Only a public dataset CelebA-HQ is used. It's better to test methods on various datasets.\n\n**A3**: We apologize if this was unclear and would like to respectfully clarify that **our experiments were carried out on three different datasets. This is more than the number of datasets used in previous fair generative modeling work [1,2].**\n\nSpecifically, as mentioned by the reviewer, In Tab.1 StyleGAN2 and StyleSwin are used and are based on the CelebA-HQ dataset. Then, in Tab.2, the pre-trained Stable Diffusion Model is based on the LAION-5B[a] dataset. Finally, in the ablation studies (Supp D.3/5) we provide further assessment on the AFHQ dataset (mentioned in Sec.5.2). Overall, our proposed CLEAM demonstrates improved performance compared to other approaches when evaluated over all three datasets.\n\nAs an additional experiment in this rebuttal, we include a fourth dataset to assess our proposed CLEAM. Specifically, we carry out a similar procedure to analyze the bias w.r.t. $\\texttt{Gender}$ of a pre-trained diffusion model [b] on the FFHQ dataset [c]. We utilize CLIP as the SA classifier. Here, we similarly find the GT $p^*$ by utilizing the same procedures discussed in Supp.H – to hand-label the generated samples. Note that the GT is for evaluation and is not used in our proposed method. Similar to other datasets, our results show that our proposed CLEAM can significantly reduce the errors for this dataset (see $e_\\mu$ and $e_\\rho$). We will include these results in the final manuscript.\n\n| Model | GT | $\\mu_{Base}$ | $e_{\\mu}$ | $\\mu_{CLEAM}$ | $e_{\\mu}$ | $\\rho_{Base}$ | $e_\\rho$ | $\\rho_{CLEAM}$ | $e_{\\rho}$ |\n|---|---|---|---|---|---|---|---|---|---|\n| Diffusion model [b] | 0.57 | 0.585 | 2.63% | 0.571 | 0.18% | [0.578,0.593] | 4.04% | [0.564,0.579] | 1.58% |\n\n$ $\n\n[a] \"Laion-5b: An open large-scale dataset for training next generation image-text models.\" NeurIPS’22.\n\n[b] \"High-resolution image synthesis with latent diffusion models.\" CVPR’22.\n\n[c] \"A style-based generator architecture for generative adversarial networks.\" CVPR’19.\n\n"}, {"review_id": "5mtbqzzhbX", "reviewer": "Reviewer_uk5v", "summary": "The paper studies measuring fairness in generative models, which is defined as equal number of samples generated from different groups. The measurement needs a sensitive attribute (SA) classifier to predict group attribute to compute the fairness. The paper emprically finds out the error in SA classifier would largely impact the measurement performance of the fairness. They test it by manually labeling samples and comparing fairness measure. The paper then proposes a calibration trick to reduce the fairness measurement error from the SA classifer's error,", "questions": "See Weakness 1 and 3.", "limitations": "See Weakness.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Fairness in generative models is an important problem and open problem", "weaknesses": "1. The paper has no insights on why the error in SA classifier would propagate to fairness measurement. The finding is entirely empirical, and seems independent of what kind of fairness definitions used. Would the SA classifier error impact all fairness definitions equally? How much would it impact? What determines the fairness error? The paper is mostly emprical in this finding without providing good insights.\n2. The mitigation method seems to be simply computing the metric on different data subsets and then taking average and computing confidence interval. The technical contribution is a little too elementary.\n3. The paper largely ignore the vast literature of noisy label, which I think is directly relevant to the problem, i.e. studying the label noise (i.e. imperfect SA prediction in this case) on the impact of fairness and models. For example:\n\n[1] Natarajan, Nagarajan, et al. \"Learning with noisy labels.\" Advances in neural information processing systems 26 (2013).\n[2] Lukasik, Michal, et al. \"Does label smoothing mitigate label noise?.\" International Conference on Machine Learning. PMLR, 2020.\n\nThere also seems to be some connection between label smoothing and the mitigation. Can you point out any if it exists?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Regarding Summary: Thank you. Our apologies if unclear, but Reviewer’s Summary is not very accurate:\n\n\n- Our statistical model is overlooked: We develop a statistical model to understand how errors in SA classifier ($\\alpha$) affect the fairness measurement ($\\hat{p}$). See Fig 1.b., entire Sec 4.1 and Supp A.1 which discuss the statistical model. Therefore, Reviewer’s summary is not accurate: \"The paper emprically finds out the error in SA classifier …\". Instead, our finding is supported by statistical analysis. \n- Our proposed method is mis-understood: Based on our statistical analysis, we propose a new measurement framework to mitigate such error. See entire Sec 4.2, Algo 1, and Supp A.2. Our statistical modeling of SA classifier error (Sec 4.1) and our new estimators taking into account such error (Eqn. 8, Eqn. 10) have been validated in Supp C. Therefore, Reviewer's summary is not accurate: \"The paper then proposes a calibration trick …\". Instead, we propose a new measurement framework with statistical grounding.\n- This contribution is overlooked: Using our proposed framework, we evaluate biases in SOTA GAN and diffusion models.\n\n$ $\n\n>**Q1**: The paper has no insights on why the error in SA classifier would propagate to fairness measurement. The finding is entirely empirical, and seems independent of what kind of fairness definitions used. \n\n**A1**: We apologize if unclear, but we have developed a statistical model to provide insights on how errors in SA classifier ($\\alpha$) propagate to the fairness measurement ($\\hat{p}$). See Fig 1.b., entire Sec 4.1 and Supp A.1. **Importantly, based on our model, Eqn. 4, 5 directly relate the error in SA classifier to fairness measurement, explaining how error in SA classifier impacts fairness measurement statistically.** \n\n$ $\n\n>**Q2**: Would the SA classifier error impact all fairness definitions equally? How much would it impact? What determines the fairness error? The paper is mostly emprical in this finding without providing good insights.\n\n**A2**: Based on our statistical model, Eqn 4, 5 clearly illustrate SA classifier error $\\alpha$ determines error in fairness $\\hat{p}$ and its impact, see Sec 4.1. \n\nAs explained in Fig 1.a, 1.b and entire Sec 4.1, our statistical model is specific to equal representation, the most popular fairness definition for generative model (see Sec 1). \n\n$ $\n\n>**Q3**: The mitigation method seems to be simply computing the metric on different data subsets and then taking average and computing confidence interval. The technical contribution is a little too elementary.\n\n\n**A3**: Our apologies if unclear, but our mitigation method is misunderstood by Reviewer. **Our method is grounded on our statistical model which relates error in SA classifier ($\\alpha$) to statistical distribution of fairness measurement ($\\hat{p}$)**. Based on this model, we derive our framework (CLEAM) and new estimators, taking into account SA classifier error $\\alpha$ to achieve improved fairness measurement. See Fig 1.b, Sec 4.1, Sec 4.2, Algo 1, Supp A.1 and A.2. Supp Sec C validates our proposed framework and our proposed estimators statistically. \n\nBy taking into account SA classifier error, our proposed framework could significantly reduce fairness measurement error. **It is important to note that our improved accuracy cannot be achieved by the simple approach mentioned by Reviewer, which we refer to as Baseline and has been extensively compared with our framework CLEAM in Sec 5.**\n\n$ $\n\n>**Q4**: The paper largely ignore the vast literature of noisy label, which I think is directly relevant to the problem, i.e. studying the label noise (i.e. imperfect SA prediction in this case) on the impact of fairness and models. \n\n**A4**: Our apologies if unclear, but our paper does not ignore label noise. Instead, our statistical model  (Fig 1.b) encompasses and takes into account broadly different causes of imperfect SA prediction, e.g. task hardness (ln 208), label noise, and other causes. Specifically, in our statistical model, $\\alpha$ captures SA error arising from different causes, including label noise. Meanwhile, our proposed CLEAM takes into account $\\alpha$ to achieve accurate fairness measurement (Sec 4.2). \n\nNote that, instead of concerning itself over a specific cause of imperfect SA prediction (e.g., label noise), and attempting to mitigate such a specific issue, our statistical model broadly encompasses different causes of SA classifier error (captured in $\\alpha$), and achieves improved fairness measurement without the need to update the SA classifier. \n\nIn the next response, we demonstrate effectiveness of our approach vs. an alternative approach to focus on label noise narrowly.\n \n$ $\n\n>**Q5**: There also seems to be some connection between label smoothing and the mitigation. Can you point out any if it exists?\n\n**A5**: Our proposed mitigation is fundamentally different from label smoothing. As discussed, our statistical model broadly encompasses different types of SA classifier error (captured in $\\alpha$) and takes them into account under one unified method, without the need to update the SA classifier. This is in sharp contrast to previous work which applies label smoothing to mitigate label noise to train an improved classifier.\n\nTo compare our approach vs. label smoothing, we conduct this experiment. We implement label smoothing in the training of a ResNet-18 based SA classifier with the same setup as Tab.1(A) w.r.t. SA $\\texttt{Gender}$. In this setup, label smoothing does not significantly improve accuracy of the SA classifier (see $\\mathbf{\\alpha}$ in the Table), as the SA classifier is already very accurate without label smoothing. However, even with these accurate SA classifiers, fairness errors are significant using Baseline, consistent with our findings in other setups. However, with our proposed CLEAM, fairness errors can be reduced considerably (with or without label smoothing).\nPlease see global response for Tab.\n"}, {"review_id": "Chv31LqAyu", "reviewer": "Reviewer_rFFs", "summary": "The authors conduct a study on the fairness of generative models.  They propose a CLassifier Error-Aware Measurement (CLEAM) framework which accounts for inaccuracies in classifiers involving sensitive attributes.  The authors also create a new dataset of generated images from a text-to-image generator which are then used to evaluate the accuracy of existing fairness frameworks.", "questions": "The abstract would benefit form fairly substantial rewriting IMO.  \"A ResNet18 for Gender with accuracy 97%\" is both an awkward phrase, but also unclear what/why it is so specifically being referenced in the abstract.  The abstract should seek to make a general observation or summary as to the goals and abstract of the work.  Additionally generative models are mentioned first, but then the focus is on classifiers, before moving back to generative models.  Cleaning up the abstract would be very beneficial. \n\nAre the results of Figure 2 \"bias\" or instability to adversaries?  Traditionally bias is usually thought of when a difference in classes is introduced due to something like \"doctor\" being associated with men and \"nurse\" being associated with women.  Here there is literally no difference between \"a\" and \"one\", yet the distribution shifts.  This feels more similar to an adversarial attack where random noise is added as opposed to a biased classifier.   Usually adversarial instability in image generation is addressed by adding noise either to the input or latent space.  Here the issue is being caused by noise within the text prompt that is impacting the generation.  \n\nIt would be helpful to put the SE in the tables and not the supplemental only (makes it tough on the reader jumping back and forth).", "limitations": "This work is explicitly designed to overcome limitations of other frameworks which may have generative bias", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Better measurement and quantification of bias is a very critical topic both in classification as well as generation tasks.  \n\nThe proposed CLEAM framework appears to produce results which are more balanced than the baseline methods.  \n\nComparisons are made to other approaches.", "weaknesses": "The text/language/grammar could be improved in many places, especially the abstract.  Even the opening of the intro is confusing- \"fairness is defined as equal generative quality and equal representation  w.r.t  some sensitive attributes (SA). In this work, we focus on the more widely utilized definition – equal representation.\"  Isn't this the same, or actually less demanding definition?  Does generating samples from both classes, but one at a much lower quality level, actually constitute fairness?  Doesn't there need to be some demand on the quality of generation?\n\nThe jumping around between the language of SA classifiers and generative models is confusing throughout.\n\nThe description/introduction of CLEAM (lines 59-68) is wordy and a bit confusing.  Other than the fact that it is \"a statistical model\", it's unclear what it is after reading this section. Since this is the focus of the paper, it should be crystal clear what the core of this method is after reading this section.  \n\nThe authors state \"we observe an intriguing property in Stable Diffusion Model\"- however what they are observing is instability due to noise.  Again, I believe this is more related to instability/adversarial attacks than \"bias\" as it is traditionally defined.  I think disentangling these (related) concepts is important.\n\nThe paper is very dense with many results as well as mathematics.  It's clear the authors have much to tell with limited space.  However, they need to walk the reader through it a bit more as it is hard to read through the tables and understand what are the key take-aways from each.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the Reviewer for the valuable suggestions. They are very helpful. We will clean up the abstract and introduction, shorten the discussion of CLEAM in the introduction, shift some results/mathematics to Supp., and better arrange the discussion of SA classifiers/generator as suggested by the Reviewer.\n\n$ $\n\n>**Q1**: \"fairness is defined as equal generative quality and equal representation w.r.t some sensitive attributes (SA). In this work, we focus on the more widely utilized definition – equal representation.\"  Isn't this the same, or actually less demanding definition? Does generating samples from both classes, but one at a much lower quality level, actually constitute fairness? Doesn't there need to be some demand on the quality of generation?\n\n**A1**: We apologize for a typo here. The sentence should be: \n\n\"fairness is defined as equal generative quality **or** equal representation w.r.t some sensitive attributes (SA). In this work, we focus on the more widely utilized definition – equal representation.\"\n\nWe would like to clarify that i) equal generative quality and ii) equal representation are two different fairness definitions. In this work, we focus on equal representation. We remark that to the best of our knowledge, majority of works in fair generative modeling focus on equal representation, e.g. to learn a fair generative model with more equal representation [1,2]. Meanwhile, equal generative quality is not the focus of our work. We will clarify this.\n\n$ $\n\n>**Q2**: Are the results of Figure 2 \"bias\" or instability to adversaries? Traditionally bias is usually thought of when a difference in classes is introduced due to something like \"doctor\" being associated with men and \"nurse\" being associated with women. Here there is literally no difference between \"a\" and \"one\", yet the distribution shifts. This feels more similar to an adversarial attack where random noise is added as opposed to a biased classifier. Usually adversarial instability in image generation is addressed by adding noise either to the input or latent space. Here the issue is being caused by noise within the text prompt that is impacting the generation.\n\n**A2**: Reviewer's comment is very insightful. The intention of this experiment is to apply our proposed fairness measurement framework to reliably measure biases in popular generative models. When we perform this measurement on Stable Diffusion Model (SDM), we follow best practices for input prompts [24, 29-31] and use indefinite (gender-neutral) pronouns or nouns [32, 33], see Sec. 3 for details. Our careful design of input prompts is to avoid gender stereotypes as mentioned by Reviewer. However, in our experiments, we observe markedly different biased outputs for SDM as shown in Fig. 2. To the best of our knowledge, such observation has not been reported previously for SDM. \n\nWe note that in our work, for generative models' fairness/bias, we follow the definition of equal representation as discussed in introduction (also discussed above) instead of the traditionally definition of stereotyping (i.e. generalization about a group of people based on certain traits).\n\n\nReviewer's viewpoint on instability to adversaries is very insightful, and we believe this could be the **root cause** of our observation of different biased outputs (unequal representation). However, understanding and validating the root cause of such based outputs would require substantial investigation, and is beyond the scope of this work. In our work, the focus is to study fairness measurement and to report biases in popular generative models using our improved framework. \n\nWe believe it will be a very interesting future study to understand the root cause of our observed bias, and Reviewer's insight on instability to adversaries is a promising direction to uncover underlying reason.   \n\n$ $\n\n>**Q3**: It would be helpful to put the SE in the tables and not the supplemental only (makes it tough on the reader jumping back and forth).\n\n**A3**: We appreciate the Reviewer’s feedback. Given the limited space, including the SE in the tables may make it a bit difficult to read. However, we would attempt to reformat the table to include this following the suggestion by the Reviewer."}, {"review_id": "oMnPVWLv9e", "reviewer": "Reviewer_o1Z2", "summary": "This paper considers the fairness measurement for generative models. The contributions of this paper are three-fold. First, the authors reveal that the existing frameworks have significant measurement errors, even using sensitive attribute classifiers. Second, the authors propose a new framework namely CLEAM that uses a statistical model to evaluate inaccuracies of SA classifiers, thus reducing the measurement errors. Finally, the authors use the proposed CLEAM to measure fairness in important text-to-image generators and GANs, which show the effectiveness of the proposed framework. Experimental results with a manually labeled dataset show that the proposed CLEAM can achieve lower error as compared with some baseline schemes.", "questions": "Please refer to my comments for details.", "limitations": "Please refer to my comments for details.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1)Propose a novel framework to measure the fairness of generative models from both theoretical and experimental perspectives. \n2)The fundamental statistic model is easy to follow, and the proposed method CLEAM is able to reduce the fairness measurement error.\n3)The dataset created in this paper will benefit the research community.", "weaknesses": "After reading this manuscript, this reviewer finds that there are some issues that need to be addressed, as\n1)On page 6, the authors say that “the probability of the counts for each output cT in Eqn. 2 (denoted by Nc) can be modeled by a multinomial distribution.” Does this assumption hold in practical systems? \n2)The presentation of the statistical model should be improved. Why do the authors assume a multivariate Gaussian distribution instead of other statistical distributions? \n3)On page 6, what does \"M\" represent in equation (3)?\n4)The description of equation (8) is not clear enough. The authors are suggested to explain it in detail.\n5)The authors are suggested to provide more experiments with other datasets and generative models, in order to demonstrate the effectiveness of the proposed framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">**Q1**: “On page 6, the authors say that “the probability of the counts for each output $c^T$ in Eqn. 2 (denoted by $N_c$) can be modeled by a multinomial distribution.” Does this assumption hold in practical systems?” \n\n**A1**: Thank you for your insightful question. In Sec. 4.1, we have considered the system very carefully (more details in Supp. A.1) and find that this model does indeed hold in most practical systems (i.e., practical generative models). Specifically, we recall that Eqn. 2 mentions that the sequence of $n$ independent experiments (image generations), each with the same success rate (for example, same probability of generating an image with value of sensitive attribute=1 in each generation), can be modeled as a multinomial distribution based on the definition of this distribution. We remark that in a practical system, as long as generation of each sample is independent, and the generative model is time invariant, the requirements are satisfied and our statistical model holds. Note that for the current image generative models like GANs and diffusion models, usually both requirements are met. We will add a statement to clarify this.\n\n$ $\n\n>**Q2**: “The presentation of the statistical model should be improved. Why do the authors assume a multivariate Gaussian distribution instead of other statistical distributions?”\n\n**A2**: We thank the Reviewer for the feedback. The assumption of multivariate Gaussian distribution is based on “normal approximation to the multinomial” [36,37], an application of the central limit theorem. Specifically, as mentioned in Sec 4.1 of the main paper (and also in our previous response), we use multinomial distribution to model the possible events of SA classifier output. Then, since $\\mathbf{p}$ in Eqn. (2) is not extreme and $n$ is reasonably large, this multinomial distribution can be approximated by a multivariate Gaussian distribution [36,37] (more details in Supp. A.1). We additionally remark that, to the best of our knowledge, multivariate Gaussian distribution should be the most appropriate approximation, and this enables us to later estimate the distribution of the $\\hat{p}$ (Eqn. (4) and (5)) with more ease. Note that the accuracy of this prediction is validated in Supp. C. We will shift some details from Supp., and add some explanations to this part to make it clear.\n\n$ $\n\n>**Q3**: “On page 6, what does \"M\" represent in equation (3)?” \n\n**A3**: We apologize if it was unclear. In Sec 4.1, the matrix $\\mathbf{M}$ is a component of the covariance matrix of the multivariate gaussian distribution i.e., $\\mathbf{\\Sigma}=n\\mathbf{M}$, which is determined following the literature [36,37] and $\\mathbf{M}$ characterizes the interaction of elements of the probability vector $\\mathbf{p}$. As mentioned, the expanded form of this term can be found in Supp. A.1. We will make this part more clear in the final version.\n\n$ $\n\n>**Q4**: The description of equation (8) is not clear enough. The authors are suggested to explain it in detail.\n \n**A4**: We apologize if it was unclear. As discussed in Sec 4.2. , Eqn. 8 is the maximum likelihood approximation of $p^*$, taking into account error in the sensitive attribute classifier ($\\alpha$) and therefore could achieve better estimation and measurement of fairness. We refer to Eqn. 8 as the CLEAM’s point estimate. Due to space limitation, we provide a compact derivation of this equation in the main paper. However, in Supp. A.2, we provide a step-by-step derivation (as mentioned in the main manuscript) and additional in-depth mathematical intuition –statistical requirements and assumption– on how this maximum likelihood approximation is derived and what it entails. We will make this part clearer.\n\n$ $\n\n>**Q5**: Authors are suggested to provide more experiments with other datasets and generative models, in order to demonstrate the effectiveness of the proposed framework.\n\n**A5**: Thank you. We would like to respectfully clarify that our experiments already consider multiple different generative models and datasets. Specifically, in Tab.1 we consider two state-of-the-art (SOTA) generative models StyleGAN2 (Conv-based) and StyleSwin (transformer-based) which are both based on the CelebA-HQ dataset. Then in Tab.2, we consider Stable Diffusion Model (SDM; as SOTA text-to-image generative model) which is based on the LAION-5B [a] dataset. Finally, in the ablation studies (Sec 5.2) we provide further assessment on the AFHQ dataset, which due to space constraint we briefly discuss the results in the main manuscript. More details can be found in Supp. D.3 and D.5. We will include an additional remark, to make these points more explicitly in the final manuscript.\n\nBased on reviewer’s suggestion, as an additional experiment, we further study our proposed CLEAM on an additional generative model and dataset. Specifically, we carried out a similar procedure to analyze the bias w.r.t. $\\texttt{Gender}$ of another pre-trained diffusion model [b] on the FFHQ dataset [c]. We utilized CLIP as the SA classifier. For evaluation, we similarly find the ground-truth (GT) $p^*$ by the same procedures in Supp. H to hand-label the generated samples. Note that the GT is used solely for evaluation. Our results show that CLEAM is similarly able to reduce the errors ($e_\\mu$ and $e_\\rho$) when compared against the baseline. We will include these results in the revised version of the paper.\n\n\n| Model | GT | $\\mu_{Base}$ | $e_{\\mu}$ | $\\mu_{CLEAM}$ | $e_{\\mu}$ | $\\rho_{Base}$ | $e_\\rho$ | $\\rho_{CLEAM}$ | $e_{\\rho}$ |\n|---|---|---|---|---|---|---|---|---|---|\n| Diffusion model [b] | 0.57 | 0.585 | 2.63% | 0.571 | 0.18% | [0.578,0.593] | 4.04% | [0.564,0.579] | 1.58% |\n\n\n[a] Laion-5b: An open large-scale dataset for training next generation image-text models. NeurIPS’22.\n\n[b] High-resolution image synthesis with latent diffusion models. CVPR’22.\n\n[c] A style-based generator architecture for generative adversarial networks. CVPR’19."}, {"review_id": "DERY2CyUKs", "reviewer": "Reviewer_6ZNP", "summary": "The paper considers the problem of measuring fairness in generative models. In particular, the paper has two main contributions: (1) they have produced a dataset of hand labeled (sensitive attributes, SA) dataset for various SOTA generative models; and (2) they have proposed a method for estimating the expected sensitive attribute distribution which utilizes the error rates of the SA classifier. These two contributions are used to show that SA classifiers with low error can still cause high errors in previous methods of approximating the sensitive attribute distribution.", "questions": "1. Relating to “Weakness 1”, how exactly are the SA classifiers typically obtained?\n2. Further relating to “Weakness 1”, how well do the error rates of the SA classifier transfer? That is, what is the difference between the SA classifier’s error rates with the dataset it was trained on versus the error rates on GenData (which I am assuming is that of Table 1 & 2)?\n", "limitations": "I think the assumption of knowing the underlying error rate / accuracy needs to further discussed (see Questions)", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper presents a strong empirical study of measuring fairness in generative models, and outlines flaws in prior studies.\n- CLEAM seems to be an intuitive correction to the naive baseline method.", "weaknesses": "1. One weakness of the paper is a hole in the narrative: how is $C_{u}$ obtained? In particular, it is unclear in the main-text on how one would generate a SA classifier. Furthermore, the assumption of knowing the underlying error rates should be discussed as a potential limitation. I assume that the SA classifier is trained on data which are not samples from the generative model one is trying to measure the fairness of (as otherwise we already have labels). In this case, the validity of error rates transferring might be questionable.\n2. The notation in the paper is somewhat strange. In particular, as far as I can see, $ Pr(u \\mid x )$ is not a probability … despite the notation. And it is further “aliased” as $ C_u(x)$ which is additionally confusing. I don’t see why one could not just define the “argmax classification” as $C_u(x)$ directly.\n3. The notation of $\\hat{p}$ and $p^{\\*}$ seems consistent to me. In Section 2, it seems that $p^{\\*}$ is the population statistic. However, in Section 3 $p^{\\*}$ becomes the estimate generated from GenData. Yes, one could argue that GenData somewhat becomes the “new” population, but this change in perspective is not clear. I think it makes more sense to think of the $p^\\*$ in this Section as a high quality estimate of the true statistic. It may be worth changing notation to reflect the 3 possible $p$’s: that generated by the SA-classifier, that generated by GenData, and the unknown true statistic which is being approximated by the former two.\n\nTypos:\n- Appendix Eq (3) LHS\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Q1**: \"how is $C_u$ obtained?\"\n\n**A1**: Thanks for your comment and we apologize if it was unclear. As discussed in the main paper (Sec.3), to obtain $C_u$, we strictly follow previous work (e.g., Imp-Weighting [1] and fairTL [2]). Particularly, we train the SA classifiers using the labeled datasets w.r.t. SAs based on standard training procedure (e.g., train a ResNet-18 as SA classifier on CelebA dataset considering BlackHair as SA). To follow previous work, we train various SA classifiers on different sensitive attributes. We remark that as mentioned in the main paper (Sec.3) we defer the details of training SA classifier to Supp. F, due to lack of space. In addition, as mentioned in the main paper (Sec. 3), we utilize CLIP as an additional SA classifier to explore zero-shot SA classification. The details of utilizing CLIP is included in Sec. E where after encoding images using image encoder, we define two related text prompts for our SA following the guidelines in CLIP [6], and encode these text prompts using text encoder. Then the cosine similarity between the encoded image and two encoded text prompts is used to get the output. \t\n\n$ $\n\n>**Q2**: “Furthermore, the assumption of knowing the underlying error rates should be discussed as a potential limitation. I assume that the SA classifier is trained on data which are not samples from the generative model one is trying to measure the fairness of (as otherwise we already have labels). In this case, the validity of error rates transferring might be questionable.”\n\n**A2**: Thank you for your comment, Reviewer’s understanding is correct: SA classifiers are trained on **real data**, and they are not samples from the generative model which fairness measurement is performed.\n\nHowever, in our work we have addressed the validity of error rate transferring. In Supp. D.7, we have validated that the error rate on real validation data can be transferred to the setups where we measure the attributes of the generated data. The results in Tab. 15 of Supp. D.7 shows that for two GANs used in our study (StyleGAN and StyleSwin), the obtained error rate ($\\alpha$) from real validation data (real CelebA-HQ data) is similar to the error rate obtained from our labeled dataset of generated images. Results in Tab. 16 show that, in our proposed CLEAM, using error rate $\\alpha$ from real validation data has similar performance as that using the error rate from generated images. Similar results are shown for Stable Diffusion in Tab. 17. We emphasize that as we mentioned in Supp. D.7, the error rate for generated data is not assumed to be available in our fairness measurement, and the analysis in Supp D.7 is solely to validate error rate transferring. \n\nWe thank the Reviewer for the comment, and we will make this clear in the revised version.\n\n$ $\n\n>**Q3**: “I don’t see why one could not just define the “argmax classification” as $C_u(x)$ directly.”\n\n**A3**: We thank the Reviewer for the good suggestion and will directly define $C_u(x)$ as the argmax classification for clearer discussion.\n\n$ $\n\n>**Q4**: “It may be worth changing notation to reflect the 3 possible $p$’s: that generated by the SA-classifier, that generated by GenData, and the unknown true statistic which is being approximated by the former two.”\n\n**A4**: We appreciate the Reviewer's sharp observation. Reviewer's understanding is correct. In the paper, to avoid introducing additional symbol and with a slight abuse of notation, we use $p^*$ to denote population statistics in Sec 2, and $p^*$ to denote estimation from GenData in Sec 3. We will consider the Reviewer's suggestion seriously and update the submission.\n\n$ $\n\n>**Q5**: Relating to “Weakness 1”, how exactly are the SA classifiers typically obtained?\n\n**A5**: Please find the details to this question in A1.\n\n$ $\n\n>**Q6**:  Further relating to “Weakness 1”, how well do the error rates of the SA classifier transfer? That is, what is the difference between the SA classifier’s error rates with the dataset it was trained on versus the error rates on GenData (which I am assuming is that of Table 1 & 2)?\n\n**A6**:  Addressed earlier in previous questions.\n"}]

Author rebuttal:
We thank all the reviewers for their valuable time and effort to review our work. We appreciate the Reviewers' kind comments and recognition, such as:

- "The paper is well written, with clear intuitions, illustrations, and experimental results." (Reviewer 2H4p)
- "Fairness in generative models is an important problem and open problem" (Reviewer uk5v)
- "The fundamental statistic model is easy to follow, and the proposed method CLEAM is able to reduce the fairness measurement error." (Reviewer o1Z2)
- "The proposed CLEAM framework appears to produce results which are more balanced than the baseline methods." (Reviewer rFFs)
- "CLEAM seems to be an intuitive correction to the naive baseline method" (Reviewer 6ZNP)
- "Propose new datasets based on generated samples with manual labeling w.r.t. SA." ( Reviewer uNfr)



We would also like to express our appreciation to all the Reviewers for giving us the opportunity to clarify our work, as well as the constructive comments.  We will consider all suggestions seriously.

To briefly recap our work:

1. Our work is the first to **challenge the accuracy of the existing fairness measurement framework** in generative models. Our major contribution is a detailed **statistical model** to model fairness measurement  (Fig 1.b, Sec 4.1 and more details in Supp. A.1). Importantly, our statistical model relates error in sensitive attribute (SA) classifier to statistical distribution of fairness measurement (Eqn. 4, 5), providing insights on how SA classifier error impacts accuracy of fairness measurement. 

2. Based on our statistical model, we derive our new fairness measurement framework (CLEAM) and new estimators, taking into account SA classifier error to achieve improved fairness measurement. See Sec 4.2, Algo 1, Supp A.2. Supp Sec C validates our proposed framework and our proposed estimators statistically.

3. To make it possible to study fairness measurement, we develop and make available a new dataset (GenData) consisting of labeled generated samples from three State-of-the-Art (SOTA) generative models. We remark that in generative modeling, fairness is popularly defined as **equal representation** [1, 2, 7, 9, 12, 16, 17] and should not be mistaken with the classifier’s fairness definitions (more detailed comparison in Supp. Sec. G)


4. We have performed comprehensive experiments to validate our proposed measurement framework and demonstrate consistent and substantial improvement in accuracy compared to other approaches. Our experiments include: 

   - **3 different generative models**: StyleGAN2, StyleSwin and Stable Diffusion model 
   - **3 different datasets**: CelebA-HQ, LAION-5B [a] and AFHQ
   - **5 different SA classifier**: ResNet-18/34, MobileNet2,VGG-16 and CLIP . 

Due to space limitation, additional experiments (e.g., with different sensitive attributes) have been diverted to the Supp. Sec.D

5. Finally, utilizing CLEAM, we carry out **reliably** measurement of biases in existing SOTA generative models.

$ $

In this rebuttal, we have also included few additional experiment, as requested by the reviewers. All results support our measurement framework and is superior compared to previous work.

1. Additional generative model and dataset: a Diffusion Model [b] pre-trained on FFHQ [c]

2. Additional comparison with label smoothing: our proposed framework outperforms label smoothing significantly. It should be noted that in our original submission we have already compared with other classifier correction methods: i) mitigate label shift in SA classifier and ii) calibrate SA classifier. All experiments show our proposed framework is superior. More details can be found in Supp. D.8 and G.

In what follows, we provide comprehensive responses to all questions. We have provided anonymized link to Area Chair for the code of all additional experiments. We could provide more details if there are further questions. We hope that our responses can address the concerns and we sincerely hope that reviewers could consider increasing the ratings if our responses have addressed all the questions.

$ $

### Additional experiments on Diffusion Model [b] on FFHQ dataset
| Model | GT | $\mu_{Base}$ | $e_{\mu}$ | $\mu_{CLEAM}$ | $e_{\mu}$ | $\rho_{Base}$ | $e_\rho$ | $\rho_{CLEAM}$ | $e_{\rho}$ |
|---|---|---|---|---|---|---|---|---|---|
| Diffusion model [b] | 0.57 | 0.585 | 2.63% | 0.571 | 0.18% | [0.578,0.593] | 4.04% | [0.564,0.579] | 1.58% |

$ $

### Additional comparison with Label Smoothing

| Model | $\alpha$ | GT | $\mu_{Base}$ | $e_{\mu}$ | $\mu_{CLEAM}$ | $e_{\mu}$ | $\rho_{Base}$ | $e_\rho$ | $\rho_{CLEAM}$ | $e_{\rho}$ |
|---|---|---|---|---|---|---|---|---|---|---|
| R18 w/o smooting | \{0.947,0.982\} | 0.642 | 0.610 | 4.98% | 0.638 | 0.62% | [0.602,0.618] | 6.23% | [0.629,0.646] | 2.02% |
| R18 w    smooting | \{0.935,0.985\} |0.642 | 0.605 | 5.76% | 0.641 | 0.16% | [0.595,0.615] | 7.32% | [0.632,0.650] | 1.56% |
    


$ $

[a] "Laion-5b: An open large-scale dataset for training next generation image-text models." NeurIPS’22.

[b] "High-resolution image synthesis with latent diffusion models." CVPR’22.

[c] "A style-based generator architecture for generative adversarial networks." CVPR’19.


Meta-review: The committee recognizes the paper's novel approach to assessing fairness in generative models, noting the potential contributions of the CLEAM method and the newly introduced dataset. While there is merit in the presented framework, the meta-reviewer echoes concerns regarding language clarity and the delineation of concepts within the paper. The meta-reviewer recommends acceptance, urging the authors to address the identified issues to further enhance the quality and impact of the work.
Decision: Accept (poster)