Title: On Measuring Fairness in Generative Models
Venue: NeurIPS 2023
Accepted: yes
Keywords: Fairness

Generative models

GAN

Calibration
Abstract: Recently, there has been increased interest in fair generative models. In this work,
we conduct, for the first time, an in-depth study on fairness measurement, a
critical component in gauging progress on fair generative models. We make three
contributions. First, we conduct a study that reveals that the existing fairness
measurement framework has considerable measurement errors, even when highly
accurate sensitive attribute (SA) classifiers are used. These findings cast doubts
on previously reported fairness improvements. Second, to address this issue,
we propose CLassifier Error-Aware Measurement (CLEAM), a new framework
which uses a statistical model to account for inaccuracies in SA classifiers. Our
proposed CLEAM reduces measurement errors significantly, e.g., 4.98%→0.62%
for StyleGAN2 w.r.t. Gender. Additionally, CLEAM achieves this with minimal
additional overhead. Third, we utilize CLEAM to measure fairness in important
text-to-image generator and GANs, revealing considerable biases in these models
that raise concerns about their applications. Code and more resources: https:
//sutd-visual-computing-group.github.io/CLEAM/.

On Measuring Fairness in Generative Models

Abstract

Recently, there has been increased interest in fair generative models. In this work, we conduct, for the first time, an in-depth study on fairness measurement , a critical component in gauging progress on fair generative models. We make three contributions. First, we conduct a study that reveals that the existing fairness measurement framework has considerable measurement errors, even when highly accurate sensitive attribute (SA) classifiers are used. These findings cast doubts on previously reported fairness improvements. Second, to address this issue, we propose CLassifier Error-Aware Measurement (CLEAM), a new framework which uses a statistical model to account for inaccuracies in SA classifiers. Our proposed CLEAM reduces measurement errors significantly, e.g., 4.98% → 0.62% for StyleGAN2 w.r.t. Gender . Additionally, CLEAM achieves this with minimal additional overhead. Third, we utilize CLEAM to measure fairness in important text-to-image generator and GANs, revealing considerable biases in these models that raise concerns about their applications. Code and more resources: https: //sutd-visual-computing-group.github.io/CLEAM/ .

1 Introduction

Fair generative models have been attracting significant attention recently [1, 2, 7-13]. In generative models [14-18], fairness is commonly defined as equal generative quality [11] or equal representation [1, 2, 7, 9, 12, 19, 20] w.r.t. some Sensitive Attributes (SA). In this work, we focus on the more widely utilized definition equal representation. In this definition, as an example, a generative model is regarded as fair w.r.t. Gender , if it generates Male and Female samples with equal probability. This is an important research topic as such biases in generative models could impact their application efficacy, e.g., by introducing racial bias in face generation of suspects [21] or reducing accuracy when supplementing data for disease diagnosis [22].

Fairness measurement for generative models. Recognizing the importance of fair generative models, several methods have been proposed to mitigate biases in generative models [1, 2, 7, 9, 12]. However, in our work, we focus mainly on the accurate fairness measurement of deep generative models i.e. assessing and quantifying the bias of generative models. This is a critical topic, as accurate measurements are essential to reliably gauge the progress of bias mitigation techniques. The general fairness measurement framework is shown in Fig. 1 (See Sec. 2 for details). This framework is utilized in existing works to assess their proposed fair generators. Central to the fairness measurement framework is a SA classifier , which classifies the generated samples w.r.t. a SA, in order to estimate the bias of the generator. For example, if eight out of ten generated face images are classified as Male

Gender

BlackHair

6.84%

0.31%

x

StyleGAN2

c

x

StyleSwin

BlackHair

Gender

5.49%

x

0.68

0.66

0.58

0.60

0.62

0.64

Generator

Input

Estimation Method

" 𝒑

Estimation of

𝒑

∗

from

" 𝒑

a

𝒑

𝟎

∗

𝟏 - 𝒑

𝟎

∗

Sample

𝒙

𝟏 - 𝜶

𝟏

𝜶

𝟏

class 1

class 0

b

𝒄

𝟎|𝟎

𝒄

𝟏|𝟎

𝒄

𝟏|𝟏

𝒄

𝟎|𝟏

𝜶

𝟎

𝟏 - 𝜶

𝟎

" 𝒑

𝟎

=

𝟏

𝒏

(𝑵

𝒄

𝟎|𝟎

+𝑵

𝒄

𝟎|𝟏

)

" 𝒑

𝟏

=

𝟏

𝒏

(𝑵

𝒄

𝟏|𝟏

+𝑵

𝒄

𝟏|𝟎

)

Stable Diffusion

'an individual'

0.05%

x

9.14%

0.19

0.16

0.20

0.17

0.18

x

'a human

being'

0.38%

x

5.73%

0.29

0.25

0.26

0.27

0.28

x

'one person'

1.77%

x

6.63%

0.25

0.21

0.22

0.23

0.24

x

'a person'

x

0.57

0.53

0.54

0.55

0.56

x

0.00%

1.49%

x

x

1.35%

8.38%

1.22%

x

x

0.68

0.66

0.60

0.62

0.64

0.58

4.98%

0.62%

x

x

Our Statistical Model for

Fairness Measurement

Sensitive Attribute

(SA) Classifier

A batch of

samples

𝐱

with

unknown

probability

𝒑

∗

w.r.t

sensitive

attribute

𝑵

𝒄

𝒊|𝒋

:

Random variable representing the

number  of  samples  with  SA  value

𝑗

which are classified by the SA classifier

as value

𝑖

Ground Truth

𝑝

)

∗

Estimation Error

𝑒

*

(𝑝

)

∗

)

CLEAM

Existing framework (Baseline)

by the SA classifier, then the generator is deemed biased at 0 . 8 towards Male (further discussion in Sec. 2). We follow previous works [1, 2, 12] and focus on binary SA due to dataset limitations.

Research gap. In this paper, we study a critical research gap in fairness measurement. Existing works assume that when SA classifiers are highly accurate, measurement errors should be insignificant. As a result, the effect of errors in SA classifiers has not been studied. However, our study reveals that even with highly accurate SA classifiers, considerable fairness measurement errors could still occur . This finding raises concerns about potential errors in previous works' results, which are measured using existing framework. Note that the SA classifier is indispensable in fairness measurement as it enables automated measurement of generated samples.

Our contributions. We make three contributions to fairness measurement for generative models. As our first contribution , we analyze the accuracy of fairness measurement on generated samples, which previous works [1, 2, 7, 9, 12] have been unable to carry out due to the unavailability of proper datasets. We overcome this challenge by proposing new datasets of generated samples with manual labeling w.r.t. various SAs. The datasets include generated samples from Stable Diffusion Model (SDM) [5] -a popular text-to-image generator- as well as two State-of-The-Art (SOTA) GANs (StyleGAN2 [3] and StyleSwin [4]) w.r.t. different SAs. Our new datasets are then utilized in our work to evaluate the accuracy of the existing fairness measurement framework. Our results reveal that the accuracy of the existing fairness measurement framework is not adequate, due to the lack of consideration for the SA classifier inaccuracies. More importantly, we found that even in setups where the accuracy of the SA classifier is high, the error in fairness measurement could still be significant . Our finding raises concerns about the accuracy of previous works' results [1, 2, 12], especially since some of their reported improvements are smaller than the margin of measurement errors that we observe in our study when evaluated under the same setup; further discussion in Sec. 3.

To address this issue, as our second (major) contribution , we propose CLassifier Error-Aware Measurement (CLEAM), a new more accurate fairness measurement framework based on our developed statistical model for SA classification (further details on the statistical model in Sec. 4.1).

Specifically, CLEAM utilizes this statistical model to account for the classifier's inaccuracies during SA classification and outputs a more accurate fairness measurement. We then evaluate the accuracy of CLEAM and validate its improvement over existing fairness measurement framework. We further conduct a series of different ablation studies to validate performance of CLEAM. We remark that CLEAM is not a new fairness metric, but an improved fairness measurement framework that could achieve better accuracy in bias estimation when used with various fairness metrics for generative models.

As our third contribution , we apply CLEAM as an accurate framework to reliably measure biases in popular generative models. Our study reveals that SOTA GANs have considerable biases w.r.t. several SA. Furthermore, we observe an intriguing property in Stable Diffusion Model: slight differences in semantically similar prompts could result in markedly different biases for SDM. These results prompt careful consideration on the implication of biases in generative models. Our contributions are:

We conduct a study to reveal that even highly-accurate SA classifiers could still incur significant fairness measurement errors when using existing framework.

To enable evaluation of fairness measurement frameworks, we propose new datasets based on generated samples from StyleGAN, StyleSwin and SDM, with manual labeling w.r.t. SA.

We propose a statistically driven fairness measurement framework, CLEAM, which accounts for the SA classifier inaccuracies to output a more accurate bias estimate.

Using CLEAM, we reveal considerable biases in several important generative models, prompting careful consideration when applying them to different applications.

2 Fairness Measurement Framework

Fig.1(a) illustrates the fairness measurement framework for generative models as in [1, 2, 7, 9, 12]. Assume that with some input e.g. noise vector for a GAN or text prompt for SDM, a generative model synthesizes a sample x . Generally, as the generator does not label synthesized samples, the ground truth (GT) class probability of these samples w.r.t. a SA (denoted by p ∗ ) is unknown. Thus, an SA classifier C u is utilized to estimate p ∗ . Specifically, for each sample x ∈ { x } , C u ( x ) is the argmax classification for the respective SA. In existing works, the expected value of the SA classifier output over a batch of samples, ˆ p = E x [ C u ( x )] (or the average of ˆ p over multiple batches of samples), is used as an estimation of p ∗ . This estimate may then be used in some fairness metric f to report the fairness value for the generator, e.g. fairness discrepancy metric between ˆ p and a uniform distribution ¯ p [1, 20](see Supp A.3 for details on how to calculate f ). Note that the general assumption behind the existing framework is that with a reasonably accurate SA classifier, ˆ p could be an accurate estimation of p ∗ [1, 9]. In the next section, we will present a deeper analysis on the effects of an inaccurate SA classifier on fairness measurement. Our findings suggest that there could be a large discrepancy between ˆ p and p ∗ , even for highly accurate SA classifiers, indicative of significant fairness measurement errors in the current measurement framework.

One may argue that conditional GANs (cGANs) [23, 24] may be used to generate samples conditioned on the SA, thereby eliminating the need for an SA classifier. However, cGANs are not considered in previous works due to several limitations. These include the limited availability of large labeled training datasets, the unreliability of sample quality and labels [25], and the exponentially increasing conditional terms, per SA. Similarly, for SDM, Bianchi et al. [26] found that utilizing well-crafted prompts to mitigate biases is ineffective due to the presence of existing biases in its training dataset. Furthermore in Sec. 6, utilizing CLEAM, we will discuss that even subtle prompt changes (while maintaining the semantics) result in drastically different SA biases. See Supp G for further comparison between [26] and our findings.

3 A Closer Look at Fairness Measurement

In this section, we take a closer look at the existing fairness measurement framework. In particular, we examine its performance in estimating p ∗ of the samples generated by SOTA GANs and SDM, a task previously unstudied due to the lack of a labeled generated dataset. We do so by designing an experiment to demonstrate these errors while evaluating biases in popular image generators. Following previous works, our main focus is on binary SA which takes values in { 0 , 1 } . Note that,

we assume that the accuracy of the SA classifier C u is known and is characterized by α = { α 0 , α 1 } , where α i is the probability of correctly classifying label i . For example, for Gender attribute, α 0 and α 1 are the probability of correctly classifying Female , and Male classes, respectively. In practice, C u is trained on standard training procedures (more details in the Supp F) and α can be measured during the validation stage of C u and be considered a constant when the validation dataset is large enough. Additionally, p ∗ can be assumed to be a constant vector, given that the samples generated can be considered to come from an infinite population, as theoretically there is no limit to the number of samples from a generative model like GAN or SDM.

New dataset by labeling generators output. The major limitation of evaluating the existing fairness measurement framework is the unavailability of p ∗ . To pave the way for an accurate evaluation, we create a new dataset by manually labeling the samples generated by GANs and SDM . More specifically, we utilize the official publicly released pre-trained StyleGAN2 [3] and StyleSwin [4] on CelebA-HQ [27] for sample generation. Then, we randomly sample from these GANs and utilize Amazon Mechanical Turks to hand-label the samples w.r.t. Gender and BlackHair , resulting in ≈ 9K samples for each GAN; see Supp H for more details and examples. Next, we follow a similar labeling process w.r.t. Gender , but with a SDM [5] pre-trained on LAION-5B[28]. Here, we input prompts using best practices [26, 29-31], beginning with a scene description ("A photo with the face of"), followed by four indefinite (gender-neutral) pronouns or nouns [32, 33] - {"an individual", "a human being", "one person", "a person"} to collect ≈ 2k high-quality samples. We refer to this new dataset as Generated Dataset ( GenData ), which includes generated images from three models with corresponding SA labels: GenData-StyleGAN2, GenData-StyleSwin, GenData-SDM. We remark that these labeled datasets only provide a strong approximation of p ∗ for each generator, however as the datasets are reasonably large, we find this approximation sufficient and simply refer to it as the GT p ∗ . Then utilizing this GT p ∗ , we compare it against the estimated baseline ( ˆ p ). One interesting observation revealed by GenData is that all three generators exhibit a considerable amount of bias (see Tab.1 and 2); more detail in Sec. 6. Note that for a fair generator we have p ∗ 0 = p ∗ 1 = 0 . 5 , and measuring the p ∗ 0 and p ∗ 1 is a good proxy for measuring fairness.

Experimental setup. Here, we follow Choi et al. [1] as the Baseline for measuring fairness. In particular, to calculate each ˆ p value for a generator, a corresponding batch of n = 400 samples is randomly drawn from GenData and passed into C u for SA classification. We repeat this for s = 30 batches and report the mean results denoted by µ Base and the 95% confidence interval denoted by ρ Base . For a comprehensive analysis of the GANs, we repeat the experiment using four different SA classifiers: Resnet-18, ResNet-34 [34], MobileNetv2 [35], and VGG-16 [36]. Then, to evaluate the SDM, we utilize CLIP [6] to explore the utilization of pre-trained models for zero-shot SA classification; more details on the CLIP SA classifier in Supp. E. As CLIP does not have a validation dataset, to measure α for CLIP, we utilize CelebA-HQ, a dataset with a similar domain to our application. We found this to be a very accurate approximation; see Supp D.7 for validation results. Note that for SDM, a separate ˆ p is measured for each text prompt as SDM's output images are conditioned on the input text prompt. As seen in Tab. 1 and 2, all classifiers demonstrate reasonably high average accuracy ∈ [84% , 98 . 7%] . Note that as we focus on binary SA ( e.g. Gender:{Male, Female} ), both p ∗ and ˆ p have two components i.e. p ∗ = { p ∗ 0 , p ∗ 1 } , and ˆ p = { ˆ p 0 , ˆ p 1 } . After computing the µ Base and ρ Base , we calculate normalized L 1 point error e µ , and interval max error e ρ w.r.t. the p ∗ 0 (GT) to evaluate the measurement accuracy of the baseline method:

Based on our results in Tab. 1 , for GANs, we observe that despite the use of reasonably accurate SA classifiers, there are significant estimation errors in the existing fairness measurement framework, i.e. e µ Base ∈ [4 . 98% , 17 . 13%] . In particular, looking at the SA classifier with the highest average accuracy of ≈ 97% (ResNet-18 on Gender ), we observe significant discrepancies between GT p ∗ 0 and µ Base , with e µ Base = 4 . 98% . These errors generally worsen as accuracy marginally degrades, e.g. MobileNetv2 with accuracy ≈ 96% results in e µ Base = 5 . 45% . These considerably large errors contradict prior assumptions - that for a reasonably accurate SA classifier, we can assume e µ Base to be fairly negligible. Similarly, our results in Tab. 2 for the SDM, show large e µ Base ∈ [1 . 49% , 9 . 14%] , even though the classifier is very accurate. We discuss the reason for this in more detail in Sec. 5.1.

Overall, these results are concerning as they cast doubt on the accuracy of prior reported results. For example, imp-weighting [1] which uses the same ResNet-18 source code as our experiment, reports a 2.35% relative improvement in fairness against its baseline w.r.t. Gender , which falls within the

range of our experiments smallest relative error, e µ Base =4.98%. Similarly, Teo et al. [2] and Um et al. [12] report a relative improvement in fairness of 0.32% and 0.75%, compared to imp-weighting [1]. These findings suggest that some prior results may be affected due to oversight of SA classifier's inaccuracies; see Supp. A.4 for more details on how to calculate these measurements.

Remark: In this section, we provide the keystone for the evaluation of measurement accuracy in the current framework by introducing a labeled dataset based on generated samples. These evaluation results raise concerns about the accuracy of existing framework as considerable error rates were observed even when using accurate SA classifiers, an issue previously seen to be negligible.

4 Mitigating Error in Fairness Measurements

The previous section exposes the inaccuracies in the existing fairness measurement framework. Following that, in this section, we first develop a statistical model for the erroneous output of the SA classifier, ˆ p , to help draw a more systematic relationship between the inaccuracy of the SA classifier and error in fairness estimation. Then, with this statistical model, we propose CLEAM - a new measurement framework that reduces error in the measured ˆ p by accounting for the SA classifier inaccuracies to output a more accurate statistical approximation of p ∗ .

4.1 Proposed Statistical Model for Fairness Measurements

As shown in Fig.1(a), to measure the fairness of the generator, we feed n generated samples to the SA classifier C u . The output of the SA classifier ( ˆ p ) is in fact a random variable that aims to approximate the p ∗ . Here, we propose a statistical model to derive the distribution of ˆ p .

As Fig.1(b) demonstrates in our running example of a binary SA, each generated sample is from class 0 with probability p ∗ 0 , or from class 1 with probability p ∗ 1 . Then, generated sample from class i where i ∈ { 0 , 1 } , will be classified correctly with the probability of α i , and wrongly with the probability of α ′ i = 1 -α i . Thus, for each sample, there are four mutually exclusive possible events denoted by c , with the corresponding probability vector p :

where c i | j denotes the event of assigning label i to a sample with GT label j . Given that this process is performed independently for each of the n generated images, the probability of the counts for each output c T in Eqn. 2 (denoted by N c ) can be modeled by a multinomial distribution, i.e. N c ∼ Multi ( n, p ) [37-39]. Note that N c models the joint probability distribution of these outputs, i.e. N c ∼ P ( N c 0 | 0 , N c 1 | 0 , N c 1 | 1 , N c 0 | 1 ) where, N c i | j is the random variable of the count for event c i | j after classifying n generated images. Since p is not near the boundary of the parameter space, and as we utilize a large n , based on the central limit theorem, Multi ( n, p ) can be approximated by a multivariate Gaussian distribution, N c ∼ N ( µ , Σ ) , with µ = n p and Σ = n M [40, 39], where M is defined as:

diag ( p ) denotes a square diagonal matrix corresponding to vector p (see Supp A.1 for expanded form). The marginal distribution of this multivariate Gaussian distribution gives us a univariate (one-dimensional) Gaussian distribution for the count of each output c T in Eqn. 2. For example, the distribution of the count for event c 0 | 0 , denoted by N c 0 | 0 , can be modeled as N c 0 | 0 ∼ N ( µ 1 , Σ 11 ) .

Lastly, we find the total percentage of data points labeled as class i when labeling n generated images using the normalized sum of the related random variables, i.e. ˆ p i = 1 n ∑ j N c i | j . For our binary example, ˆ p i can be calculated by summing random variables with Gaussian distribution, which results in another Gaussian distribution [41], i.e. , ˆ p 0 ∼ N (˜ µ ˆ p 0 , ˜ σ 2 ˆ p 0 ) , where:

Similarly ˆ p 1 ∼ N (˜ µ ˆ p 1 , ˜ σ 2 ˆ p 1 ) with ˜ µ ˆ p 1 = ( µ 2 + µ 3 ) /n , and ˜ σ 2 ˆ p 1 = ( Σ 22 + Σ 33 +2 Σ 23 ) /n 2 which is aligned with the fact that ˆ p 1 = 1 -ˆ p 0 .

Algorithm 1: Computing point and interval estimates using CLEAM.

Require: accuracy of SA classifier, α .

1 Compute SA classifier output ˆ p : { ˆ p 1 , . . . , ˆ p s } for s batches of generated data.

2 Compute sample mean ¨ µ ˆ p and sample variance ¨ σ 2 ˆ p using (6) and (7).

3 Use (8) to compute point estimate µ CLEAM .

4 Use (10) to compute interval estimate ρ CLEAM .

Remark: In this section, considering the probability tree diagram in Fig.1(b), we propose a joint distribution for the possible events of classification ( N c i | j ), and use it to compute the marginal distribution of each event, and finally the distribution of the SA classifier outputs ( ˆ p 0 , and ˆ p 1 ). Note that considering Eqn. 4, 5, only with a perfect classifier ( α i = 1 , i.e. acc = 100% ) the ˜ µ ˆ p 0 converges to p ∗ 0 . However, training a perfect SA classifier is not practical e.g. due to the lack of an appropriate dataset and task hardness [42, 43]. As a result, in the following, we will propose CLEAM which instead utilizes this statistical model to mitigate the error of the SA classifier.

4.2 CLEAM for Accurate Fairness Measurement

In this section, we propose a new estimation method in fairness measurement that considers the inaccuracy of the SA classifier. For this, we use the statistical model, introduced in Sec 4.1, to compute a more accurate estimation of p ∗ . Specifically, we first propose a Point Estimate (PE) by approximating the maximum likelihood value of p ∗ . Then, we use the confidence interval for the observed data ( ˆ p ) to propose an Interval Estimate (IE) for p ∗ .

Point Estimate (PE) for p ∗ . Suppose that we have access to s samples of ˆ p denoted by { ˆ p 1 , . . . , ˆ p s } , i.e. SA classification results on s batches of generated data. We can then use the proposed statistical model to approximate the p ∗ . In the previous section, we demonstrate that we can model ˆ p i j using a Gaussian distribution. Considering this, first, we use the available samples to calculate sample-based statistics including the mean and variance of the ˆ p j samples:

For a Gaussian distribution, the Maximum Likelihood Estimate (MLE) of the population mean is its sample mean ¨ µ ˆ p j [44]. Given that s is large enough ( e.g. s > 30 ), we can assume that ¨ µ ˆ p j is a good approximation of the population mean [45], and equate it to the statistical population mean ˜ µ ˆ p j in Eqn. 4 (see Supp A.2 for derivation). With that, we get the maximum likelihood approximation of p ∗ , which we call the CLEAM's point estimate, µ CLEAM :

Notice that µ CLEAM accounts for the inaccuracy of the SA classifier.

Interval Estimate (IE) for p ∗ . In the previous part, we propose a PE for p ∗ using the statistical model, and sample-based mean ¨ µ ˆ p 0 . However, as we use only s samples of ˆ p , ¨ µ ˆ p 0 may not capture the exact value of the population mean. This adds some degree of inaccuracy into µ CLEAM . In fact, in our framework, ¨ µ ˆ p 0 equals ˜ µ ˆ p 0 when s →∞ . However, increasing each unit of s significantly increases the computational complexity, as each ˆ p requires n generated samples. To address this, we recall that ˆ p 0 follows a Gaussian distribution and instead utilize frequentist statistics [41] to propose a 95% confidence interval (CI) for p ∗ . To do this, first we derive the CI for ˜ µ ˆ p 0 :

Then, applying Eqn.4 to Eqn.9 gives the lower and upper bounds of the approximated 95% CI for p ∗ 0 :

This gives us the interval estimate of CLEAM, ρ CLEAM = [ L ( p ∗ 0 ) , U ( p ∗ 0 )] , a range of values that we can be approximately 95% confident to contain p ∗ 0 . The range of possible values for p ∗ 1 can be simply derived considering p ∗ 1 = 1 -p ∗ 0 . The overall procedure of CLEAM is summarized in Alg. 1. Now, with the IE, we can provide statistical significance to the reported fairness improvements.

'an individual'

'a human being'

'one person'

' a person'

Stable Diffusion

PE (Female)

PE(Male)

Fair Point

IE

a) 'A photo with the face of one person'

b) 'A photo with the face of a person'

0.5

0.0

0.5

5 Experiments

In this section, we first evaluate fairness measurement accuracy of CLEAM on both GANs and SDM (Sec.5.1) with our proposed GenData dataset. Then we evaluate CLEAM's robustness through some ablation studies (Sec. 5.2). To the best of our knowledge, there is no similar literature for improving fairness measurements in generative models. Therefore, we compare CLEAM with the two most related works: a) the Baseline used in previous works [1, 2, 7, 9, 12] b) Diversity [46] which computes disparity within a dataset via an intra-dataset pairwise similarity algorithm. We remark that, as discussed by Keswani et al. [46] Diversity is model-specific using VGG-16 [36]; see Supp. D.2 for more details. Finally, unless specified, we repeat the experiments with s = 30 batches of images from the generators with batch size n = 400 . For a fair comparison, all three algorithms use the exact same inputs. However, while Baseline and Diversity ignore the SA classifier inaccuracies, CLEAM makes good use of it to rectify the measurement error. As mentioned in Sec. 4.2, for CLEAM, we utilize α measured on real samples, which we found to be a good approximation of the α measured on generated samples (see Supp. D.7 for results). We repeat each experiment 5 times and report the mean value for each test point for both PE and IE. See Supp D.1 for the standard deviation.

5.1 Evaluating CLEAM's Performance

CLEAM for fairness measurement of SOTA GANs - StyleGAN2 and StyleSwin. For a fair comparison, we first compute s samples of ˆ p , one for each batch of n images. For Baseline, we use the mean ˆ p value as the PE (denoted by µ Base ), and the 95% confidence interval as IE ( ρ Base ). With the same s samples of ˆ p , we apply Alg. 1 to obtain µ CLEAM and ρ CLEAM . For Diversity, following the original source code [46], a controlled dataset with fair representation is randomly selected from a held-out dataset of CelebA-HQ [27]. Then, we use a VGG-16 [36] feature extractor and compute Diversity, δ . With δ we find ˆ p 0 = ( δ +1) / 2 and subsequently µ Div and ρ Div from the mean and 95% CI (see Supp D.2 for more details on diversity). We then compute e µ CLEAM , e µ Div , e ρ CLEAM and e ρ Div with Eqn 1, by replacing the Baseline estimates with CLEAM and Diversity.

As discussed, our results in Tab.1 show that the baseline experiences significantly large errors of 4 . 98% ≤ e µ Base ≤ 17 . 13% , due to a lack of consideration for the inaccuracies of the SA classifier. We note that this problem is prevalent throughout the different SA classifier architectures, even with higher capacity classifiers e.g. ResNet-34. Diversity, a method similarly unaware of the inaccuracies of the SA classifier, presents a similar issue with 8 . 98% ≤ e µ DiV ≤ 14 . 33% In contrast, CLEAM dramatically reduces the error for all classifier architectures. Specifically, CLEAM reduces the average point estimate error from e µ Base ≥ 8.23% to e µ CLEAM ≤ 1.24%, in both StyleGAN2 and StyleSwin. The IE presents similar results, where in most cases ρ CLEAM bounds the GT value of p ∗ .

CLEAM for fairness measurement of SDM. We evaluate CLEAM in estimating the bias of the SDM w.r.t. Gender , based on the synonymous (gender-neutral) prompts introduced in Sec. 3. Recall that here we utilize CLIP as the zero-shot SA classifier. Our results in Tab 2, as discussed, show that utilizing the baseline results in considerable error ( 1 . 49% ≤ e µ Base ≤ 9 . 14% ) for all prompts, even though the SA classifier's average accuracy was high, ≈ 98 . 7% (visual results in Fig.2). A closer look at the theoretical model's Eqn. 4 reveals that this is due to the larger inaccuracies observed in the biased class ( α ′ 1 ) coupled with the large bias seen in p ∗ 1 , which results in µ Base deviating from p ∗ 0 . In contrast, CLEAM accounts for these inaccuracies and significantly minimizes the error to e µ CLEAM ≤ 1 . 77% . Moreover, CLEAM's IE is able to consistently bound the GT value of p ∗ 0 .

5.2 Ablation Studies and Analysis

Here, we perform the ablation studies and compare CLEAM with classifier correction methods. We remark that detailed results of these experiments are provided in the Supp due to space limitations.

CLEAMfor measuring varying degrees of bias. As we cannot control the bias in trained generative models, to simulate different degrees of bias, we evaluate CLEAM with a pseudo-generator . Our results show that CLEAM is effective at different biases ( p ∗ 0 ∈ [0.5,0.9]) reducing the average error from 2.80% ≤ e µ Base ≤ 6.93% to e µ CLEAM ≤ 0.75% on CelebA [47] w.r.t. {Gender,BlackHair} , and AFHQ [48] w.r.t. Cat/Dog . See Supp D.3 and D.4 for full experimental results.

CLEAM vs Classifier Correction Methods [49] . CLEAM generally accounts for the classifier's inaccuracies, without targeting any particular cause of inaccuracies, for the purpose of rectifying the fairness measurements. This objective is unlike traditional classifier correction methods as it does not aim to improve the actual classifier's accuracy. However, considering that classifier correction methods may improve the fairness measurements by directly rectifying the classifier inaccuracies, we compare its performance against CLEAM. As an example, we utilize the Black Box Shift Estimation / Correction (BBSE / BBSC) [49] which considers the label shift problem and aims to correct the classifier output by detecting the distribution shift. Our results, based on Sec. 5.1 setup, show that while BBSE does improve on the fairness measurements of the baseline i.e. 4.20% ≤ e µ BBSE ≤ 3.38%, these results are far inferior to CLEAM's results seen in Tab. 1. In contrast, BBSC demonstrates no improvements in fairness measurements. See Supp D.8 for full experimental results. We postulate that this is likely due to the strong assumption of label shift made by both methods.

Effect of batch-size. Utilizing experimental setup in Sec. 5.1 for batch size n ∈ [100,600], our results in Fig. 3 show that n =400 is an ideal batch size, balancing computational cost and measurement accuracy. See Supp F for full experimental details and results.

100 200 300 400 500 600

n

0

5

10

100200300400500600

n

100 200 300 400 500 600

n

100 200 300 400 500 600

n

x

CLEAM

Baseline

Error

,

𝑒

!

StyleGAN2 (             )

StyleSwin (             )

StyleGAN2 (

)

StyleSwin (

)

6 Applying CLEAM: Bias in Current SOTA Generative Models

In this section, we leverage the improved reliability of CLEAM to study biases in the popular generative models. Firstly, with the rise in popularity of text-to-image generators [5, 50-52], we revisit our results when passing different prompts, with synonymous neutral meanings to an SDM, and take a closer look at how subtle prompt changes can impact bias w.r.t. Gender . Furthermore, we further investigate if similar results would occur in other SA, Smiling . Secondly, with the shift in popularity from convolution to transformer-based architectures [53-55], due to its better sample quality, we determine whether the learned bias would also change. For this, we compare StylesSwin (transformer) and StyleGAN2 (convolution), which are both based on the same architecture backbone.

Our results, on SDM, demonstrate that the use of different synonymous neutral prompts [32, 33] results in different degrees of bias w.r.t. both Gender and Smiling attributes. For example in Fig. 2, a semantically insignificant prompt change from "one person" to "a person" results in a significant shift in Gender bias. Then, in Fig. 4a, we observe that while the SDM w.r.t. our prompts appear to be heavily biased to notSmiling , having "person" in the prompt appears to significantly reduce this bias. This suggests that for SDM, even semantically similar neutral prompts [32, 33] could result in different degrees of bias, thereby demonstrating certain instability in SDM. Next, our results in Fig. 4b compare the bias in StyleGAN2, StylesSwin, and the training CelebA-HQ dataset over an extended number of SAs. Overall, we found that while StyleSwin produces better quality samples [4], the same biases still remain statistically unchanged between the two architectures i.e. their IE overlap. Interestingly, our results also found that both the GANs were less biased than the training dataset itself.

7 Discussion

Conclusion. In this work, we address the limitations of the existing fairness measurement framework. Since generated samples are typically unlabeled, we first introduce a new labeled dataset based on three state-of-the-art generative models for our studies. Our findings suggest that the existing framework, which ignores classification inaccuracies, suffers from significant measurement errors, even when the SA classifier is very accurate. To rectify this, we propose CLEAM, which considers these inaccuracies in its statistical model and outputs a more accurate fairness measurement. Overall, CLEAM demonstrates improved accuracy over extensive experimentation, including both real

'an individual'

'a human being'

'one person'

' a person'

Stable Diffusion

PE (

not-Smiling

)

PE

(

Smiling

)

Fair Point

IE

0.5

0.0

0.5

a) 'A photo with the face of a human being'

b) 'A photo with the face of a person'

(a) Evaluating bias of SDM [5] w.r.t. SA Smiling using CLEAM. LHS: Bias value with different text prompts as input to SDM. CLEAM identifies that having 'person' in the prompt results in more Smiling samples generated. RHS: Visual comparison of the samples generated for two different but semantically similar text prompts. Samples in the same column are generated using the same random seed. Notice how the SA ( smiling ) of the samples changes with a slight difference in the prompts.

a) 'A photo with the face of a human being'

b) 'A photo with the face of a person'

NoBeard

(B/NB)

Gender(F/M)

BlackHair(NBH/BH)

Young(NY/Y)

Smiling(NS/S)

Mustache(NM/M)

Chubby(NC/C)

Bald(NB/B)

HeavyMakeup(NHM/HM)

PE (Label 0)

PE( Label 1)

Fair Point

IE

CelebA-HQ dataset

StyleGAN2

StyleSwin

0.5

0.0

0.5

0.5

0.0

0.5

0.5

0.0

0.5

(b) CLEAM on StyleGAN2 [3] and StyleSwin [4], both pre-trained on CelebA-HQ but are based on different architecture variants. We utilize a ResNet-18 and compare the bias w.r.t. various SAs.

generators and controlled setups. Moreover, by applying CLEAM to popular generative models, we uncover significant biases that raise efficacy concerns about these models' real-world application.

Broader Impact. Given that generative models are becoming more widely integrated into our everyday society e.g. text-to-image generation, it is important that we have reliable means to measure fairness in generative models, thereby allowing us to prevent these biases from proliferating into new technologies. CLEAM provides a step in this direction by allowing for more accurate evaluation. We remark that our work does not introduce any social harm but instead improves on the already existing measurement framework.

Limitations. Despite the effectiveness of the proposed method along various generative models, our work addresses only one facet of the problems in the existing fairness measurement and there is still room for further improvement. For instance, it may be beneficial to consider SA to be non-binary e.g. when hair color is not necessary fully black (grey). Additionally, existing datasets used to train classifiers are commonly human-annotated, which may itself contain certain notions of bias. See Supp. I for further discussion.

References

Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair Generative Modeling via Weak Supervision. In Proceedings of the 37th International Conference on Machine Learning , pages 1887-1898. PMLR, November 2020.

Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Cheung. Fair generative models via transfer learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 2429-2437, 2023.

Tero Karras, Samuli Laine, and Timo Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4401-4410, June 2019.

Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo. Styleswin: Transformer-based gan for high-resolution image generation, 2021.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision, February 2021.

Eric Frankel and Edward Vendrow. Fair Generation Through Prior Modification. 32nd Conference on Neural Information Processing Systems (NeurIPS 2018) , June 2020.

Ahmed Imtiaz Humayun, Randall Balestriero, and Richard Baraniuk. MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining. In International Conference on Learning Representations , January 2022.

Shuhan Tan, Yujun Shen, and Bolei Zhou. Improving the Fairness of Deep Generative Models without Retraining. arXiv:2012.04842 [cs] , December 2020.

Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, and Mario Fritz. Inclusive GAN: Improving Data and Minority Coverage in Generative Models, August 2020.

Vongani H. Maluleke, Neerja Thakkar, Tim Brooks, Ethan Weber, Trevor Darrell, Alexei A. Efros, Angjoo Kanazawa, and Devin Guillory. Studying Bias in GANs through the Lens of Race, September 2022.

Soobin Um and Changho Suh. A Fair Generative Model Using Total Variation Distance. openreview , September 2021.

Milad Abdollahzadeh, Touba Malekzadeh, Christopher TH Teo, Keshigeyan Chandrasegaran, Guimeng Liu, and Ngai-Man Cheung. A survey on generative modeling with limited data, few shots, and zero shot. arXiv preprint arXiv:2307.14397 , 2023.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM , 63(11): 139-144, 2020.

Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in neural information processing systems , 33: 12104-12114, 2020.

Yunqing Zhao, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Man Cheung. Few-shot image generation via adaptation-aware kernel modulation. Advances in Neural Information Processing Systems , 35:19427-19440, 2022.

Keshigeyan Chandrasegaran, Ngoc-Trung Tran, Alexander Binder, and Ngai-Man Cheung. Discovering transferable forensic features for cnn-generated images detection. In European Conference on Computer Vision , pages 671-689. Springer, 2022.

Yunqing Zhao, Chao Du, Milad Abdollahzadeh, Tianyu Pang, Min Lin, Shuicheng Yan, and Ngai-Man Cheung. Exploring incompatible knowledge transfer in few-shot image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7380-7391, 2023.

Ben Hutchinson and Margaret Mitchell. 50 Years of Test (Un)fairness: Lessons for Machine Learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency , FAT* '19, pages 49-58, New York, NY, USA, January 2019. Association for Computing Machinery. ISBN 978-1-4503-6125-5. doi: 10.1145/3287560.3287600.

Christopher T. H. Teo and Ngai-Man Cheung. Measuring Fairness in Generative Models. 38th International Conference on Machine Learning (ICML) Workshop , July 2021.

Harsh Jaykumar Jalan, Gautam Maurya, Canute Corda, Sunny Dsouza, and Dakshata Panchal. Suspect Face Generation. In 2020 3rd International Conference on Communication System, Computing and IT Applications (CSCITA) , pages 73-78, April 2020. doi: 10.1109/CSCITA47329.2020.9137812.

Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, and Hayit Greenspan. Ganbased synthetic medical image augmentation for increased cnn performance in liver lesion classification. Neurocomputing , 321:321-331, 2018.

Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 , 2014.

Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In ICML , 2017.

Kiran K Thekumparampil, Ashish Khetan, Zinan Lin, and Sewoong Oh. Robustness of conditional GANs to noisy labels. In Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale, November 2022.

Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020.

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems , 35:25278-25294, 2022.

Vivian Liu and Lydia B Chilton. Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems , pages 1-23, 2022.

Stable Diffusion: Prompt Guide and Examples. https://strikingloo.github.io/stable-diffusion-vs-dalle-2, August 2022.

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to Prompt for Vision-Language Models. International Journal of Computer Vision , 130(9):2337-2348, September 2022. ISSN 0920-5691, 1573-1405. doi: 10.1007/s11263-022-01653-1.

Martin Haspelmath. Indefinite Pronouns . Oxford University Press, 1997. ISBN 978-0-19-829963-9 978-0-19-823560-6. doi: 10.1093/oso/9780198235606.001.0001.

Abigail C Saguy and Juliet A Williams. A little word that means a lot: A reassessment of singular they in a new era of gender politics. Gender & Society , 36(1):5-31, 2022.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 770-778, June 2016.

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4510-4520, Salt Lake City, UT, June 2018. IEEE. ISBN 978-1-5386-6420-9. doi: 10.1109/CVPR.2018.00474.

Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. ICLR2015 , September 2014.

C Radhakrishna Rao. Maximum likelihood estimation for the multinomial distribution. Sankhy¯ a: The Indian Journal of Statistics (1933-1960) , 18(1/2):139-148, 1957.

Harry Kesten and Norman Morse. A property of the multinomial distribution. The Annals of Mathematical Statistics , 30(1):120-127, 1959.

Athanasios Papoulis. Probability, Random Variables and Stochastic Processes with Errata Sheet . McGrawHill Europe, Boston, Mass., 4th edition edition, January 2002. ISBN 978-0-07-122661-5.

Charles J Geyer. Stat 5101 Notes: Brand Name Distributions. University of Minnesota , Stat 5101:25, January 2010.

Nathaniel R Goodman. Statistical analysis based on a certain multivariate complex gaussian distribution (an introduction). The Annals of mathematical statistics , 34(1):152-177, 1963.

Milad Abdollahzadeh, Touba Malekzadeh, and Ngai-Man Man Cheung. Revisit multimodal meta-learning through the lens of multi-task learning. Advances in Neural Information Processing Systems , 35:1463214644, 2021.

Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning. In Proceedings of the IEEE/CVF international conference on computer vision , pages 6430-6439, 2019.

Theodore Wilbur Anderson and Ingram Olkin. Maximum-likelihood estimation of the parameters of a multivariate normal distribution. Linear algebra and its applications , 70:147-171, 1985.

Renato A Krohling and Leandro dos Santos Coelho. Coevolutionary particle swarm optimization using gaussian distribution for solving constrained optimization problems. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , 36(6):1407-1416, 2006.

Vijay Keswani and L. Elisa Celis. Auditing for Diversity Using Representative Examples. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages 860-870, Virtual Event Singapore, August 2021. ACM. ISBN 978-1-4503-8332-5. doi: 10.1145/3447548.3467433.

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild. Proceedings of International Conference on Computer Vision (ICCV) , September 2015.

Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. StarGAN v2: Diverse Image Synthesis for Multiple Domains, April 2020.

Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In Proceedings of the 35th International Conference on Machine Learning , pages 3122-3130. PMLR, 10-15 Jul 2018.

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-Shot Text-to-Image Generation, February 2021.

Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, and Daniel Cohen-Or. StyleGAN-NADA: CLIPGuided Domain Adaptation of Image Generators, December 2021.

Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery, March 2021.

Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do Vision Transformers See Like Convolutional Neural Networks?, March 2022.

Drew A. Hudson and Larry Zitnick. Generative Adversarial Transformers. In Proceedings of the 38th International Conference on Machine Learning , pages 4487-4499. PMLR, July 2021.

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual Prompt Tuning. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision - ECCV 2022 , Lecture Notes in Computer Science, pages 709-727, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-19827-4. doi: 10.1007/ 978-3-031-19827-4_41.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the problem of measuring fairness in generative models. In particular, the paper has two main contributions: (1) they have produced a dataset of hand labeled (sensitive attributes, SA) dataset for various SOTA generative models; and (2) they have proposed a method for estimating the expected sensitive attribute distribution which utilizes the error rates of the SA classifier. These two contributions are used to show that SA classifiers with low error can still cause high errors in previous methods of approximating the sensitive attribute distribution.\n\nSTRENGTHS:\n- The paper presents a strong empirical study of measuring fairness in generative models, and outlines flaws in prior studies.\n- CLEAM seems to be an intuitive correction to the naive baseline method.\n\nWEAKNESSES:\n1. One weakness of the paper is a hole in the narrative: how is $C_{u}$ obtained? In particular, it is unclear in the main-text on how one would generate a SA classifier. Furthermore, the assumption of knowing the underlying error rates should be discussed as a potential limitation. I assume that the SA classifier is trained on data which are not samples from the generative model one is trying to measure the fairness of (as otherwise we already have labels). In this case, the validity of error rates transferring might be questionable.\n2. The notation in the paper is somewhat strange. In particular, as far as I can see, $ Pr(u \\mid x )$ is not a probability … despite the notation. And it is further “aliased” as $ C_u(x)$ which is additionally confusing. I don’t see why one could not just define the “argmax classification” as $C_u(x)$ directly.\n3. The notation of $\\hat{p}$ and $p^{\\*}$ seems consistent to me. In Section 2, it seems that $p^{\\*}$ is the population statistic. However, in Section 3 $p^{\\*}$ becomes the estimate generated from GenData. Yes, one could argue that GenData somewhat becomes the “new” population, but this change in perspective is not clear. I think it makes more sense to think of the $p^\\*$ in this Section as a high quality estimate of the true statistic. It may be worth changing notation to reflect the 3 possible $p$’s: that generated by the SA-classifier, that generated by GenData, and the unknown true statistic which is being approximated by the former two.\n\nTypos:\n- Appendix Eq (3) LHS\n\nQUESTIONS:\n1. Relating to “Weakness 1”, how exactly are the SA classifiers typically obtained?\n2. Further relating to “Weakness 1”, how well do the error rates of the SA classifier transfer? That is, what is the difference between the SA classifier’s error rates with the dataset it was trained on versus the error rates on GenData (which I am assuming is that of Table 1 & 2)?\n\nLIMITATIONS:\nI think the assumption of knowing the underlying error rate / accuracy needs to further discussed (see Questions)", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the fairness measurement for generative models. The contributions of this paper are three-fold. First, the authors reveal that the existing frameworks have significant measurement errors, even using sensitive attribute classifiers. Second, the authors propose a new framework namely CLEAM that uses a statistical model to evaluate inaccuracies of SA classifiers, thus reducing the measurement errors. Finally, the authors use the proposed CLEAM to measure fairness in important text-to-image generators and GANs, which show the effectiveness of the proposed framework. Experimental results with a manually labeled dataset show that the proposed CLEAM can achieve lower error as compared with some baseline schemes.\n\nSTRENGTHS:\n1)Propose a novel framework to measure the fairness of generative models from both theoretical and experimental perspectives. \n2)The fundamental statistic model is easy to follow, and the proposed method CLEAM is able to reduce the fairness measurement error.\n3)The dataset created in this paper will benefit the research community.\n\nWEAKNESSES:\nAfter reading this manuscript, this reviewer finds that there are some issues that need to be addressed, as\n1)On page 6, the authors say that “the probability of the counts for each output cT in Eqn. 2 (denoted by Nc) can be modeled by a multinomial distribution.” Does this assumption hold in practical systems? \n2)The presentation of the statistical model should be improved. Why do the authors assume a multivariate Gaussian distribution instead of other statistical distributions? \n3)On page 6, what does \"M\" represent in equation (3)?\n4)The description of equation (8) is not clear enough. The authors are suggested to explain it in detail.\n5)The authors are suggested to provide more experiments with other datasets and generative models, in order to demonstrate the effectiveness of the proposed framework.\n\nQUESTIONS:\nPlease refer to my comments for details.\n\nLIMITATIONS:\nPlease refer to my comments for details.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors conduct a study on the fairness of generative models.  They propose a CLassifier Error-Aware Measurement (CLEAM) framework which accounts for inaccuracies in classifiers involving sensitive attributes.  The authors also create a new dataset of generated images from a text-to-image generator which are then used to evaluate the accuracy of existing fairness frameworks.\n\nSTRENGTHS:\nBetter measurement and quantification of bias is a very critical topic both in classification as well as generation tasks.  \n\nThe proposed CLEAM framework appears to produce results which are more balanced than the baseline methods.  \n\nComparisons are made to other approaches.\n\nWEAKNESSES:\nThe text/language/grammar could be improved in many places, especially the abstract.  Even the opening of the intro is confusing- \"fairness is defined as equal generative quality and equal representation  w.r.t  some sensitive attributes (SA). In this work, we focus on the more widely utilized definition – equal representation.\"  Isn't this the same, or actually less demanding definition?  Does generating samples from both classes, but one at a much lower quality level, actually constitute fairness?  Doesn't there need to be some demand on the quality of generation?\n\nThe jumping around between the language of SA classifiers and generative models is confusing throughout.\n\nThe description/introduction of CLEAM (lines 59-68) is wordy and a bit confusing.  Other than the fact that it is \"a statistical model\", it's unclear what it is after reading this section. Since this is the focus of the paper, it should be crystal clear what the core of this method is after reading this section.  \n\nThe authors state \"we observe an intriguing property in Stable Diffusion Model\"- however what they are observing is instability due to noise.  Again, I believe this is more related to instability/adversarial attacks than \"bias\" as it is traditionally defined.  I think disentangling these (related) concepts is important.\n\nThe paper is very dense with many results as well as mathematics.  It's clear the authors have much to tell with limited space.  However, they need to walk the reader through it a bit more as it is hard to read through the tables and understand what are the key take-aways from each.\n\nQUESTIONS:\nThe abstract would benefit form fairly substantial rewriting IMO.  \"A ResNet18 for Gender with accuracy 97%\" is both an awkward phrase, but also unclear what/why it is so specifically being referenced in the abstract.  The abstract should seek to make a general observation or summary as to the goals and abstract of the work.  Additionally generative models are mentioned first, but then the focus is on classifiers, before moving back to generative models.  Cleaning up the abstract would be very beneficial. \n\nAre the results of Figure 2 \"bias\" or instability to adversaries?  Traditionally bias is usually thought of when a difference in classes is introduced due to something like \"doctor\" being associated with men and \"nurse\" being associated with women.  Here there is literally no difference between \"a\" and \"one\", yet the distribution shifts.  This feels more similar to an adversarial attack where random noise is added as opposed to a biased classifier.   Usually adversarial instability in image generation is addressed by adding noise either to the input or latent space.  Here the issue is being caused by noise within the text prompt that is impacting the generation.  \n\nIt would be helpful to put the SE in the tables and not the supplemental only (makes it tough on the reader jumping back and forth).\n\nLIMITATIONS:\nThis work is explicitly designed to overcome limitations of other frameworks which may have generative bias", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies measuring fairness in generative models, which is defined as equal number of samples generated from different groups. The measurement needs a sensitive attribute (SA) classifier to predict group attribute to compute the fairness. The paper emprically finds out the error in SA classifier would largely impact the measurement performance of the fairness. They test it by manually labeling samples and comparing fairness measure. The paper then proposes a calibration trick to reduce the fairness measurement error from the SA classifer's error,\n\nSTRENGTHS:\n1. Fairness in generative models is an important problem and open problem\n\nWEAKNESSES:\n1. The paper has no insights on why the error in SA classifier would propagate to fairness measurement. The finding is entirely empirical, and seems independent of what kind of fairness definitions used. Would the SA classifier error impact all fairness definitions equally? How much would it impact? What determines the fairness error? The paper is mostly emprical in this finding without providing good insights.\n2. The mitigation method seems to be simply computing the metric on different data subsets and then taking average and computing confidence interval. The technical contribution is a little too elementary.\n3. The paper largely ignore the vast literature of noisy label, which I think is directly relevant to the problem, i.e. studying the label noise (i.e. imperfect SA prediction in this case) on the impact of fairness and models. For example:\n\n[1] Natarajan, Nagarajan, et al. \"Learning with noisy labels.\" Advances in neural information processing systems 26 (2013).\n[2] Lukasik, Michal, et al. \"Does label smoothing mitigate label noise?.\" International Conference on Machine Learning. PMLR, 2020.\n\nThere also seems to be some connection between label smoothing and the mitigation. Can you point out any if it exists?\n\nQUESTIONS:\nSee Weakness 1 and 3.\n\nLIMITATIONS:\nSee Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-24"}

comments: SUMMARY:
This paper proposes a framework for fairness measurement. It first shows that existing framework has considerable measurement errors even when highly accurate sensitive attribute classifiers are used, then propose CLassifier Error-Aware Measurement (CLEAM), a new framework which uses a statistical model to account for inaccuracies in SA classifiers.

STRENGTHS:
1. Shows the significant measurement errors of existing frameworks by experiments.
2. Propose new datasets based on generated samples with manual labeling w.r.t. SA.
3. Proposes a simple statistical approximation method is proposed to obtain a stable and accurate estimation of the GT probabilities.

WEAKNESSES:
1. The organization of the paper is kind of hard to follow. The first contribution takes too long and may confuse the readers.
2. The introduction to the proposed method is too short and not very solid. Some theoretical support may be better. Maybe you can talk about cases when using distributions other than Gaussian to approximate the distributions.
3. Only a public dataset CelebA-HQ is used. It's better to test methods on various datasets.

QUESTIONS:
plz refer to the "weaknesses" part.

LIMITATIONS:
The structure of paper is hard to follow and somehow boring for readers.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-26

comments: SUMMARY:
The objective of this paper is on measuring the fairness in generative models. There are three contributions. (i) Consideration of measurement errors of sensitive attribute (SA) classifiers in fairness measurement of generative models. (ii) A classification error aware measurement framework, called CLEAM, which based on statistical model accounts for the inaccuracies of SA classifier to reduce measurement error in generative models. (iii) As an application, the authors demonstrate that CLEAM can be applied to measure fairness in text-to-image generator and GANs.

STRENGTHS:
The paper is well written, with clear intuitions, illustrations, and experimental results.

WEAKNESSES:
- Putting the application of gender bias in the introduction seems to be out of place, possibly undermining the main framework CLEAM. Similarly, table 1 is also out of place, throwing numbers to the readers without explaining the setup. 

- The authors make an assumption of ground-truth labels of sensitive attributes, which may not be available in practice.

- Some of the human faces in Figure 2 and figure 3(a) are different, revealing the uncertainty of the generative models. I understand the intuition of the authors to provide a demonstration of the paper at an early part of the paper. Please clarify if I misunderstood anything.

QUESTIONS:
- The presented method considers binary sensitive attributes. However, modern generative models easily generates images with multi-sensitive features. How do the authors extend to this scenario? Is naive enumeration the only possible way?

- In Equation (2), are $p_i$ and  $\alpha_i$ known?

- In line 222, is $ s = 30 $ large enough?

- Is there any future work based on this paper?

Please also address the points in "Weakness" above.

LIMITATIONS:
NA.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-27