Title: Debiasing Pretrained Generative Models by Uniformly Sampling Semantic Attributes
Venue: NeurIPS 2023
Accepted: yes
Keywords: generative models

generative modeling

bias

GANs

debiasing
Abstract: Generative models are being increasingly used in science and industry applications. Unfortunately,  they often perpetuate the biases present in their training sets, such as societal biases causing certain groups to be underrepresented in the data. For instance, image generators may overwhelmingly produce images of white people due to few non-white samples in their training data. It is imperative to debias generative models so they synthesize an equal number of instances for each group, while not requiring retraining of the model to avoid  prohibitive expense. We thus propose a *distribution mapping module* that produces samples from a *fair noise distribution*, such that the  pretrained generative model produces *semantically uniform* outputs -  an equal number of instances for each group - when conditioned on these samples. This does *not* involve retraining the generator, nor does it require *any* real training data. Experiments on debiasing generators trained on popular real-world  datasets show that our method outperforms existing approaches.

Debiasing Pretrained Generative Models by Uniformly Sampling Semantic Attributes

Abstract

Generative models are being increasingly used in science and industry applications. Unfortunately, they often perpetuate the biases present in their training sets, such as societal biases causing certain groups to be underrepresented in the data. For instance, image generators may overwhelmingly produce images of white people due to few non-white samples in their training data. It is imperative to debias generative models so they synthesize an equal number of instances for each group, while not requiring retraining of the model to avoid prohibitive expense. We thus propose a distribution mapping module that produces samples from a fair noise distribution , such that the pretrained generative model produces semantically uniform outputs - an equal number of instances for each group - when conditioned on these samples. This does not involve retraining the generator, nor does it require any real training data. Experiments on debiasing generators trained on popular real-world datasets show that our method outperforms state-of-the-art approaches.

1 Introduction

Background. Generative models have become a cornerstone of modern machine learning, allowing for the synthesis of realistic data for many domains, including images [19, 35], audio [22], and text [11, 4]. However, even leading generative models often reproduce the biases present in their training data [13, 26], such as image generation models strongly over-representing white males [24]. As generative models are increasingly used for data augmentation to train downstream models [5] in domains from scientific to medical fields [25, 9], biased synthesized data could lead to results that are skewed or inaccurate. This can exacerbate existing issues such as facial recognition models performing significantly worse on non-white individuals [29] or healthcare models being much less accurate for certain minority groups [23]. Further, with the rapid growth of generative models in commercial applications, the potential financial, legal and ethical costs of biased outputs are significant. Thus, it is imperative to develop methods that mitigate bias in generative models to ensure that their outputs are fair and equitable by generating a roughly equal number of samples of each group. We call such outputs semantically uniform , and the attribute that they are uniform over - such as gender or race - the semantic attribute .

State-of-the-art. Existing methods for addressing bias in generative models often train a new model from scratch [27, 37, 2], though this is computationally expensive, requires significant labeled data, and wastes resources already previously spent training the existing (biased) generative model. Latent attribute editing methods modify the samples in the latent space of generative models to produce controlled changes in the output, and could potentially be used to correct for bias [17]. However, this requires making limiting assumptions such as that semantic attributes correspond to linear directions

Pretrained

Unconditional

Generator

White

Asian

Black

Native

American

Pacific

Islander

White

Asian

Black

Native

American

Pacific

Islander

Semantic Attribute

in the latent space [17, 38, 6, 33, 39, 8]. While some recent advances have been made in uniformly sampling the output space of a pretrained model [15, 16], they can fail to yield uniform samples over semantic attributes [15]. For example, they will make an image generator's output uniform over a manifold in the pixel space, but may still overproduce images of white individuals as the output won't be uniform over the semantic attribute of race .

Problem statement. Recently, numerous pretrained generative models and classifiers have been released for a variety of domains. For instance, there are publicly available generators for images of faces [19] followed by classifiers that predict attributes such as race and age of faces [36]. Here, we assume access to a pretrained generative model G θ that maps a noise space Z to a feature space X , and a pretrained classifier C ϕ that maps X to a discrete semantic attribute space Y . Additionally, as error rates are often reported for released models, we assume that the class-conditional errors; i.e., the probability that an instance actually belongs to group j given that the classifier incorrectly predicted group i , is known. Our goal is to model a distribution Q such that for z ∼ Q , G θ ( z ) produces samples that are uniform over the attribute space Y , using only predictions from C ϕ to guide the output of G θ . For instance, if there are five discrete semantic groups in Y , G θ will produce an equal number of instances belonging to each group when conditioned on draws from Q . We call Q a fair noise distribution ; see Figure 1 for an example.

Challenges. Our task of producing samples that are semantically uniform has three major hurdles:

Expensive retraining: It is often prohibitively expensive to retrain large generative models in terms of computational resources and time.

Inaccessible training data: The data used to train a released generator is typically unavailable; either due to being proprietary or simply too large of a volume for most practitioners to utilize. Thus, we do not have adequate data available to tune the generator or classifier.

Inaccurate classifier: As we do not have any samples of real data available, we must rely on the possibly imperfect semantic attribute classifier C ϕ to provide labels. However, since classifiers may produce inaccurate predictions - especially on underrepresented groups of Y , this can cause incorrect estimations of the number of instances for each group.

Proposed approach. We propose to train a distribution mapper network M ω : Z → Z that transforms draws from a standard noise distribution into draws from a fair noise distribution, such that once noise samples are transformed by M ω they condition the generator to produce a roughly equal number of instances for each semantic group in Y . We achieve this by first constructing a dataset of instances from Z that follow a fair noise distribution, which allows us to estimate the true distribution of the semantic attribute given an imperfect classifier. Using our proposed strategy, noise samples are collected such that the pretrained classifier's (corrected) distribution is uniform. Then, we use this fair noise dataset to train M ω to inexpensively sample new instances from the fair noise distribution. The output of M ω serves as input to the pretrained generator, which yields samples that are uniform over the space of the semantic attribute. Importantly, this approach does not require retraining the main generative model, thus incurring minimal costs. Our method works without requiring any real training data. Relying only on a pretrained generator and classifier pair, we circumvent the need to acquire any real data (labeled or otherwise) even using only an imperfect classifier.

Contributions. In this work, we:

Show how to construct a fair noise dataset that produces semantically uniform synthetic outputs when passed through a generator, assuming an accurate classifier is available.

Design a method to correct for the error incurred by an imperfect classifier, to construct a dataset of samples that come from a fair noise distribution.

Propose an approach for training a distribution mapping network to sample from a fair noise distribution on-the-fly.

Demonstrate the utility of our approach on a range of real-world datasets and released generators.

2 Problem Definition

Assume we are given a pretrained generative model G θ that maps a latent space Z to a feature space X . In general, we assume that Z is of a lower dimensionality than X . Additionally, we assume we are given a pretrained classifier C ϕ that maps X to a semantic attribute space Y , where Y is the set of group ( class ) labels. Thus, Y = { Y 1 , Y 2 , . . . , Y N } , where Y i is the label of the i th group. When not otherwise ambiguous, we will refer to group Y i as group i . Let y and ˆ y be the random variables indicating the true semantic attribute label and the predicted label from C ϕ respectively. Let E be the prediction-conditional error rates of C ϕ , with E being a left stochastic matrix such that E i,j = P ( y = i | ˆ y = j ) . Let C ϕ be a better-than-random classifier, i.e., for N groups, P ( y = i | ˆ y = i ) > 1 N for all groups i . Thus, E is by definition a diagonally dominant matrix. As it is typical for developers to report the error rates of their classifiers, we assume that E is known. While these error rates are typically reported for the classifier's training distribution, we can correct E to apply to the generated distribution under a label shift assumption [40]. See the Appendix for details on this and for the proofs of the upcoming theorems. Lastly, let C ′ denote an ideal perfect classifier that always correctly predicts y with zero error. C ′ is hypothetical and unavailable to us.

Our goal here is to sample from a Fair Noise Distribution, defined as follows:

Definition 1 (Fair Noise Distribution) . For a generative model G θ : Z → X , a distribution Q over Z is a Fair Noise Distribution with respect to Y if for z ∼ Q , C ′ ( G θ ( z )) ∼ Unif( Y ) , where Unif( Y ) is the uniform distribution over the groups in Y .

Intuitively, a distribution Q is a Fair Noise Distribution if G θ produces samples that are uniform over the semantic space Y when conditioned on Q . For example, if G θ synthesizes images of people and Y is the set of races, then Q is a Fair Noise Distribution if conditioning on it makes G θ produce a roughly equal number of images of people from each race.

More realistically, we want to find a Q that is easy to sample from and produces reasonable variance from the generator. That is, it does not yield only one unique example of each group, which would otherwise make the problem trivial. Notably, we assume that we do not have any real training data (i.e., no real samples from X ). Also, we do not aim to retrain G θ , i.e., not change the parameters θ of G θ . Additionally, we do not make any assumptions on the differentiability of G θ or C ϕ .

3 Methodology

Our approach for sampling from a Fair Noise Distribution Q hinges on training a distribution mapping function M ω such that M ω ( z ) ∼ f Q , where z is a draw from the original conditioning distribution of G θ . The functional form of M ω has many options; for instance, M ω can be a GAN generator [12], a VAE [21], a DDPM [14], or a normalizing flow model [34, 7]. No matter which form is chosen, we will need a dataset of samples drawn from Q to train M ω .

3.1 Collecting Fair Samples Using Imperfect Classifiers

A naive method for collecting a dataset of samples distributed according to a Fair Noise Distribution Q is given in Algorithm 1. The basic approach is to continuously sample z from the noise distribution, collect the generator's output for each draw, and use the classifier to determine the value of the semantic attribute corresponding to each noise draw. As a result, samples of z corresponding to each group are saved to dataset D Q until D Q has S number of samples for each y ∈ Y . This procedure results in samples of noise that once passed through G θ will be uniform across the semantic attribute

space according to C ϕ ; i.e., for z ∼ D Q , C ϕ ( G θ ( z )) ∼ Unif( Y ) . However, this will only yield samples from a Fair Noise Distribution in the case where C ϕ is a perfect classifier (such that E is the identity matrix). If C ϕ is an imperfect classifier then the distribution may not be truly fair. For instance, if C ϕ often incorrectly predicts group j as group i , then instances with attributes matching group i may be more prevalent than those for group j .

Fortunately, we can utilize knowledge from the prediction-conditional error rates E to sample a dataset of noise that will yield more semantically uniform generated instances despite the noisy predictions of C ϕ . To achieve this, we use a weighted sample of datapoints predicted for each group. Let P z | C ϕ = i be a distribution over Z such that the imperfect classifier's prediction of generated samples arising from this distribution are all of group i ; C ϕ ( G θ ( z )) = i for z ∼ P z | C ϕ = i . In addition, let Q λ = ∑ |Y| i =1 λ i P z | C ϕ = i , such that λ i > 0 ∀ i and ∑ |Y| i =1 λ i = 1 .

We can in many instances find values of λ = { λ 1 , λ 2 , . . . , λ |Y| } such that Q λ is a Fair Noise Distribution, as stated in Lemma 1:

Lemma 1. Let 1 |Y| be the vector of length |Y| such that every element is 1 . Let cone( E ) = { ∑ a i E : ,i | a i ∈ R ≥ 0 } be the finite convex cone generated by the columns of the prediction-conditional error matrix. If 1 |Y| ∈ cone( E ) , then ∃ λ such that Q λ is a Fair Noise Distribution.

Additionally, we can show that in the case where y is binary and C ϕ is better-than-random, we can always find λ such that samples drawn from Q λ will definitely yield generated samples that are uniform over the semantic attributes, as shown by the following lemma.

Lemma 2. Let |Y| = 2 and let C ϕ be better-than-random, such that the diagonal elements of E are each greater than 1 2 . Then, ∃ λ such that Q λ is a Fair Noise Distribution.

The proof of Lemma 2 comes from showing that ⟨ 1 , 1 ⟩ is always in the finite convex cone generated by C ϕ 's prediction-conditional error matrix, with Lemma 1 implying that this property guarantees that Q λ will yield samples that produce semantically uniform generated instances.

Even in the cases where we cannot guarantee that there exists an ideal λ , we can still find values for λ that will yield a distribution that is as close as possible to a uniform distribution over the semantic space, with the difference from uniformity measured by KL divergence. For this, let us define a Minimally-Unfair Noise Distribution.

Definition 2 (Minimally-Unfair Noise Distribution) . For a generative model G θ : Z → X , a distribution Q λ = ∑ |Y| i =1 λ i P z | C ϕ = i , λ = { λ 1 , λ 2 , . . . , λ |Y| } , λ i ∈ R ≥ 0 , ∑ |Y| i =1 λ i = 1 , is a Minimally-Unfair Noise Distribution over Z with respect to Y if for z ∼ Q λ , λ = argmin λ KL { C ′ ( G θ ( z )) || Unif( Y ) } .

Intuitively, Q λ is a Minimally-Unfair Noise Distribution if the values of λ yield generated samples with minimal divergence from a semantically uniform distribution, under the constraint that Q λ is a convex combination of the P z | C ϕ = i 's. This constraint is required so that we can sample from Q λ using a weighted sampling technique based off of our imperfect classifier C ϕ .

Fortunately, we can easily find the values of λ that will yield a Minimally-Unfair Noise Distribution. Before deriving the procedure for this, let us define the distribution P λ E as the normalized weighted sum of the columns of E , where each column i is weighted according to a corresponding λ i :

Definition 3 ( P λ E ) . Define P λ E = ∑ |Y| i =1 λ i E : ,i as a distribution over Y determined by predictionconditional error matrix E and λ = { λ 1 , λ 2 , . . . , λ |Y| } , λ i ∈ R ≥ 0 , ∑ |Y| i =1 λ i = 1 .

Next, we note that finding the values for λ that maximize the entropy of P λ E is equivalent to finding the λ for which the divergence between P λ E and Unif( Y ) is minimized. This is stated formally in the following Proposition.

Before stating the theorem that directly implies a strategy for learning λ for which Q λ is MinimallyUnfair, all that is left to do is to link P λ E with the distribution of C ′ ( G θ ( z )) :

Proposition 2 states that for a given λ the distribution of the perfect classifier C ′ given samples of the generator conditioned on draws from Q λ will be distributed according to P λ E . Now, we state Theorem 1 which directly implies our sampling strategy.

Theorem 1 follows from the preceding Propositions. To see how Theorem 1 implies a strategy for sampling from a Minimally-Unfair Noise Distribution, recall that Q λ is defined as the mixture of distributions of each P z | C ϕ = i for i from 1 → |Y| with λ i as the i th mixture weight, where P z | C ϕ = i is the distribution of z 's that yield generated samples that the noisy classifier predicts as group i . The above theorem states that the mixture of these distributions with mixture weights λ that produce maximum entropy for P λ E will yield a mixture distribution Q λ that is worse-case Minimally-Unfair and, when possible, will be a perfect Fair Noise Distribution. Thus, to sample from Q λ all that is needed is to 1) construct datasets D z | C ϕ = i for i from 1 → |Y| such that if z ∈ D z | C ϕ = i then C ϕ ( G θ ( z )) = i (i.e., these datasets serve as proxies for P z | C ϕ = i ); 2) find λ such that H ( P λ E ) is maximized; and 3) perform a weighted sampling from each D z | C ϕ = i proportional to its corresponding λ i . Note that the datasets D z | C ϕ = i can be formed by an approach similar to

Methodology

Noise/Latent

Distribution

Distribution

Mapper

Pretrained

Unconditional

Generator

Perfect

Classifier

Semantic Attribute

Distribution

Worcester Polytechnic Institute 1 Algorithm 1. The algorithm returns a dictionary with keys being the elements of Y , and the values of each key returned from that procedure form the corresponding required datasets (i.e., D z | C ϕ = i ). A pseudo-code implementation of our approach for constructing a dataset of noise samples from a Minimally-Unfair Noise Distribution Q λ is given in Algorithm 2.

3.2 Training the Distribution Mapper

Let D Q λ be a dataset of noise samples from the space Z distributed according to Q λ , such that D Q λ is obtained as described in the previous subsection (i.e., D Q λ comes from Algorithm 1 in the case where the error rates E are unknown or C ϕ can be assumed to be an ideal classifier, or from Algorithm 2 otherwise). Now, we train a distribution mapper M ω : Z → Z such that if v ∼ P noise then M ω ( v ) ∼ Q λ , where P noise is an easy-to-sample-from noise distribution such as a multivariate Gaussian, or whatever noise distribution was used as input when training G θ . As previously stated, the functional form of M ω and the procedure used to train it is flexible. There is a wide class of generative models and training strategies that could be employed [3]. We chose to model M ω as a GAN generator and find the parameters of M ω using adversarial training. Thus, we train M ω as such:

where F ρ is a discriminator network.

Note that z is a multivariate vector, where the ordering of the elements of the vector are arbitrary (i.e., there is likely no spacial relationship in z ). Thus, the inductive bias of convolutional filters used in many leading GANs [20, 19, 32] is not appropriate in this case. Rather, we suggest to train the distribution mapper using a GAN architecture designed for tabular data; notably, CTGAN which has shown to perform well on data where a sequential or spacial inductive bias is not appropriate [43].

After training M ω we can sample an arbitrary amount of samples from Q without applying Algorithm 1 or Algorithm 2, as M ω ( v ) ∼ Q λ for v ∼ P noise . Importantly, C ′ ( G θ ( M ω ( v ))) ∼ Unif( Y ) when Q λ is a Fair Noise Distribution, and will otherwise have minimum divergence from Unif( Y ) under the constraints given in Definition 2. Notably, we achieve this with no samples of real data, an imperfect classifier, and without fine-tuning G θ . Instead, the only training needed is for M ω , which in general should require much less complexity (and thus much less cost) than G θ . Figure 2 shows the Distribution Mapper paired with the pretrained generator G θ .

4 Experiments

We now experimentally evaluate our approach. Details such as hyperparameter choice and architectures are available in the appendix. Code for our method is available in the supplemental material.

We compare our approach on a range of generators: VAE [21], DCGAN [32], Progressive GAN [19], and a Latent Diffusion Model (LDM) [35].

4.1 Compared Methods

Latent Editing (2019) . We apply the commonly used latent editing [17, 38, 6, 33, 39, 8] approach to the task of constructing a dataset with an equal number of instances from each group in Y . While some work uses non-linear directions when editing [1], using linear directions has been shown to work as well in practice [17, 8]. Thus, for each group y ∈ Y , we fit a linear classifier K y on noise samples from P noise , with the goal of separating noise instances that are mapped to group y from those mapped to all other groups. After training these classifiers, we can sample a noise instance belonging to a group with the following procedure: First, sample z ∼ P noise ; if K y ( z ) = y then

Circles

Squares

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Density

Sampling With and Without Error Correction

Sampling (With Error Correction)

Naive

Sampling (No Error Correction)

Circles

Squares

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Density

Latent Editing (Ground Truth Classifier)

MaGENT

Polarity Sampling (Antimodes)

Polarity Sampling (Modes)

Shifted Semantic Distribution

Attribute

Attribute

Figure 3: a) Effects of applying our bias correction when constructing Fair Noise dataset; b) comparison of methods on the Shapes dataset.

return z , otherwise perform latent editing on z until K y ( z new ) = y . The editing approach is given by z new = z previous + ϵ · η y , where η y is the normal vector to K y 's decision boundary and ϵ is a step size. We set ϵ = 0 . 1 in our experiments.

MaGNET (2022) . This method was recently proposed to uniformly sample the manifold of pretrained generative models [15], using a sampling strategy that leverages the Jacobian of the generative model. The authors argue that while this does not guarantee an equal distribution for each group, MaGNET should increase the frequency of disadvantaged groups by sampling more often from low-density regions of the generative manifold.

Polarity Sampling (2022) . An extension of MaGNET , Polarity Sampling allows for more controlled sampling over the generative manifold [16]. Sampling is controlled by a parameter ρ where as ρ goes to -∞ modes are sampled from increasingly often, antimodes are sampled from more as ρ →∞ , and the original generative distribution is sampled from for ρ = 0 . We thus compare against two Polarity Sampling settings; Polarity Mode Sampling where ρ = -2 , and Polarity Antimode Sampling where ρ = 2 . By a similar argument used for MaGNET , sampling from antimodes may result in minority groups being more represented.

Shifted Semantic Distribution (2021) . This method provides a training-free approach for debiasing pretrained generative models [42]. The key idea is to fit Gaussian mixture models on the latent space, where these mixtures are fit on regions that correspond to unique values of the semantic attribute. However, unlike our approach, Shifted Semantic Distribution assumes that each attribute class can be uniquely identified by a hyperplane in the noise space.

Standard Generator . We compare against unmodified pretrained generators used as initially intended. This is a baseline which other methods should outperform.

4.2 Uniformly Sampling From Shapes Dataset

We first evaluate our approach on a dataset of synthetic images. Each image is of either a circle or a square, where the shape has a random size, color, and position in the image. We use the object's shape as the semantic attribute; thus, Y = { 'Circle' , 'Square' } . This dataset was first used by Jing et al. [18]. We utilize the V AE Jing et al. compared against in the same work as our generator.

Testing Classifier Bias Correction Approach. We demonstrate the utility of correcting for a biased classifier when constructing a dataset of Fair Noise samples. To that end, we compare the distribution of each group when we trust the biased classifier C ϕ (Algorithm 1) to the distribution found when using our proposed biased correction approach (Algorithm 2). We use a linear classifier trained to distinguish images of squares from circles as our biased classifier C ϕ . This classifier is implemented using Scikit-learn's LinearSVC classifier [31]. As a ground truth, we utilize a deep convolutional network classifier that achieves approximately perfect accuracy on the task of distinguishing squares from circles. Thus, this network acts as C ′ .

Figure 3 a) shows the distribution of shapes in the Fair Noise Dataset with error correction (Algorithm 2) and without applying our bias correction approach (Algorithm 1). As |Y| = 2 for this experiment, the best case is when the density of each group is 0 . 5 . Clearly, our bias correction approach yields a dataset that is much closer to being uniform over the semantic attribute. This implies the need to correct for inaccurate classifiers when constructing Fair Noise Datasets.

Comparative Study on Shapes Dataset. We next test our proposed Distribution Mapping approach (Section 3.2) against the compared methods. For our approach, we train M ω as a CTGAN [43] trained on the corrected Fair Noise Distribution obtained in the previous experiment. As the compared Latent Editing method also requires a classifier as a ground truth for training each of its linear models K y , we compare against two versions of the method: 1) Latent Editing (Biased Classifier) using the biased C θ as a ground truth, and 2) Latent Editing (Perfect Classifier) which uses the ground-truth, effectively perfect convolutional network discussed above. Note that from our problem definition, this perfect classifier would usually be unavailable to us.

As Figure 3 b) shows, our approach clearly results in the most uniform distribution out of all compared methods. Additionally, the Latent Editing approach performs much worse when using the biased classifier. As expected, Polarity Antimode Sampling does indeed increase the frequency of the minority class ( Circles ). Interestingly, MaGNET also increases the frequency of the minority class, but is roughly as far from uniform as is the distribution of the original generator; MaGNET flips the distribution and under-represents what was previously the majority class.

4.3 Age Bias in Face Image Generator

In this next experiment, we evaluate our proposed method's ability to debias a generative model that produces images of people's faces. Specifically, the generative model is a DCGAN [32] that we pretrain on a grayscale version of the UTKFace dataset [44]. For this experiment, the semantic attribute is Age . The age of each individual in UTKFace is given as a label. As our approach assumes the semantic attribute Y is discrete, we bin the ages in increments of 10 years such that Y = { ' ≤ 9 ′ , ' 10-19 ′ , . . . , ' 90-99+ ′ } . We train a deep convolutional network as the ground truth classifier (used for evaluation), and use a corresponding classifier with a quarter of the feature maps as the biased classifier.

Evaluating Bias Correction Approach With Large Number of Classes. We repeat the bias correction experiment we performed on the Shapes data again for the UTKFace Generator. While the previous experiment required only two classes, we now have ten classes (one for each age bin). Despite this increase, Figure 4 a) shows that performing the bias correction we propose in Algorithm 2 yields a much more uniform Fair Noise training dataset than results from trusting the biased classifier.

Comparative Study on Reducing Age Bias. We evaluate all methods on the UTKFace Generator, with the goal of making the output images uniform over Age (i.e., generate an equal number of images of people belonging to each age group). Figure 4 b) clearly shows that our approach (orange line) produces a much more uniform distribution over ages than the compared method, though it does generate images for the 10-19 bin too infrequently. While approaches such as Polarity Antimode Sampling and MaGNET are somewhat less biased away from generating images of young people, they fail to generate many samples for older individuals. Note that for the Latent Editing approach the we fit a linear regressor (guided by the ground truth) on the latent space rather than a classifier, as age is more naturally a continuous attribute.

4.4 Uniformly Sampling Over Race in Progressive GAN

We apply our approach to debias a pretrained Progressive GAN [19]; specifically, the PyTorch [30] version of the ' celebAHQ-256 ' model 2 . We use the Race of the individual in each image as the semantic attribute, and use the MTCNN classifier from the DeepFace [36] package to classify race. Matching the classes available in DeepFace, our semantic attribute space is Y = { ' Asian ', ' Indian ', ' White ', ' Middle Eastern ', ' Latino Hispanic ', ' Black ' } . The Progressive GAN strongly favors generating white individuals, likely because its training images were of predominantly white celebrities. Here, we consider the classifier to be accurate.

9

10-19

20-29

30-39

40-49

50-59

60-69

70-79

80-89

90-99+

Age Group

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Density

Sampling With And Without Error Correction

Naive

Sampling (No Error Correction)

Sampling (With Error Correction)

(a) Effect of bias correction; 0.1 (dashed line) is ideal.

9

10-19

20-29

30-39

40-49

50-59

60-69

70-79

80-89

90-99+

Age Group

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Density

Distribution Of Ages Resulting From Compared Methods

Distribution

Mapping (Ours)

Latent Editing

MaGNET

Polarity Sampling (Antimodes)

Polarity Sampling (Modes)

Shifted Semantic Distribution

Standard DCGAN

(b) Comparison of methods; 0.1 (dashed line) is ideal.

Figure 4: a) Our biased correction approach yields better Fair Noise Distributions; b) comparison of methods on correcting for age bias in the UTKFace DCGAN.

Distribution

Mapping (Ours)

Latent Editing

MaGNET

Polarity Sampling

(Antimodes)

Polarity Sampling

Shifted Semantic

Distribution

Standard

Progressive GAN

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

KL Divergence

0.41

0.62

0.56

0.49

0.57

0.60

0.68

KL Divergence From Uniform Of Each Approach For The CelebHQ Progressive GAN

Comparative Study of Debiasing Progressive GAN. For each method, we report the KL divergence between a uniform distribution and the classifier's output on the samples generated from the method in Figure 5. Note that unlike in the previous experiments, for this metric lower is better . Our approach has the lowest KL divergence, indicating it produces a more equal number of images of people from each race than any compared method. We note that despite this we still observe an over-representation of images of white individuals in the samples produced by our (and all other) approaches (see Appendix). This indicates that most regions of the latent space are likely associated with semantic attribute, and unless the distribution mapper very closely matches its Fair Noise training distribution, white persons will still be over-represented. Still, our approach results in a distribution that is most fair out of all compared methods. As we observed previously, sampling antimodes with Polarity Sampling produces next-best results; likely because it explicitly draws from low-probability regions of the latent space, which correspond to non-white individuals.

4.5 Uniformly Sampling Age in Latent Diffusion Model

Although our focus is primrily on models that map from low dimensional latent space to a higher dimensional feature space, we also evaluate our approach on a latent diffusion model 3 [35] that was trained on the Celeba-HQ dataset [19]. For this experiment we chose Age as the semantic attribute, and debias according to the following age categories: { ≤ 29 , 30-49 , ≥ 50 } . We utilize a pretrained ViT age classifier 4 to provide feedback for the semantic attribute. Since our approach requires each latent code to be mapped to a single output image, we utilized the DDIM [41] sampler for the diffusion model to yield a deterministic diffusion process. We used a DDIM diffusion model to train the Mapper network as well.

4.5.1 Comparative Study on Debiasing Latent DDPM.

We report the KL divergence from uniform for the semantic distribution of each compared method in Table 1. Note that we exclude MaGNET and Polarity Sampling from this experiment, due to

Table 1: Comparative study on debiasing the Age attribute for a pretrained LDM.

the difficulty of obtaining the Jacobian determinant required by these methods. Results show that our approach yields significantly better scores than the other compared method, indicating that our approach can potentially be useful for debiasing Diffusion models as well.

5 Broader Impact And Limitations

The goal of this work, reconditioning generative models to not reproduce the biased distribution of their training set but rather produce one that treats each group equitably, is driven by the desire to mitigate the effects that systemic biases have on generative models increasingly used in real world applications. This has the potential for beneficial societal impact by counteracting the harmful effects of such systemic biases that lead these models to be unfair to underrepresented groups. However, the potential negative impacts of advancing generative modeling should not go unconsidered. Such models have already been used for some harmful applications such as Deep Fakes [28]. It is important to stress that while our approach aims to mitigate the effects of bias in existing models, it is not a fix-all nor an excuse to train models on knowingly biased data. When possible, it is essential to collect fair and equitable training datasets, and to take measures to ensure that the models we train are fair without the need for post-hoc corrections.

Limitations. Currently, our approach assumes that the semantic space Y is discrete. We plan to extend this work to handle continuous attributes, such as skin tone, in future work. Additionally, while the conditional distribution P ( x | y ) should be roughly equal before and after applying our method if the classifier C ϕ is accurate, in the case of a biased classifier the conditional distribution may change. Correcting for potential distribution shift is likewise future work.

6 Conclusion

This is the first method to correct a pretrained biased generative model (i.e., one that strongly favors generating images of white people over all other races) given only the generator and a potentiallybiased classifier. We propose a sampling strategy to construct a fair training set using the biased classifier in a way that corrects for its bias, and a Distribution Mapping module that uses this training set to learn how to sample noise instances that produce fair outputs when used as input to the generative model. Notably, we are able to debias the generative model without retraining the model or utilizing any real data. Our results indicate that our approach produces outputs that are much fairer than existing methods. This work may inspire more research on Distribution Mapping techniques to recondition generative models by transforming their standard latent distributions into distributions that yield more favorable behavior.

7 Acknowledgments

This material is based on research sponsored by DARPA under agreement number FA8750-18-2-0077. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA or the U.S. Government.

Results in this paper were obtained in part using a high-performance computing system acquired through NSF MRI grant DMS-1337943 to WPI. We thank the members of WPI's DAISY Research lab and the WPI WASH research group for their insightful feedback and support during the development of this work.

References

T. Aoshima and T. Matsubara. Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model, Mar. 2023. arXiv:2211.14573 [cs].

S. Bhat, J. Jiang, O. Pooladzandi, and G. Pottie. De-Biasing Generative Models using Counterfactual Methods, Feb. 2023. arXiv:2207.01575 [cs].

S. Bond-Taylor, A. Leach, Y. Long, and C. G. Willcocks. Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(11):7327-7347, Nov. 2022.

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language Models are Few-Shot Learners, July 2020. arXiv:2005.14165 [cs].

K. Cong and M. Zhou. Face Dataset Augmentation with Generative Adversarial Network. Journal of Physics: Conference Series , 2218(1):012035, Mar. 2022. Publisher: IOP Publishing.

E. Denton, B. Hutchinson, M. Mitchell, T. Gebru, and A. Zaldivar. Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias, Oct. 2020. arXiv:1906.06439 [cs, stat].

L. Dinh, D. Krueger, and Y. Bengio. NICE: Non-linear Independent Components Estimation, Apr. 2015. arXiv:1410.8516 [cs].

P. Doubinsky, N. Audebert, M. Crucianu, and H. Le Borgne. Multi-attribute balanced sampling for disentangled GAN controls. Pattern Recognition Letters , 162:56-62, Oct. 2022.

S. Festag, J. Denzler, and C. Spreckelsen. Generative adversarial networks for biomedical time series forecasting and imputation. Journal of Biomedical Informatics , 129:104058, May 2022.

S. Garg, Y. Wu, S. Balakrishnan, and Z. Lipton. A Unified View of Label Shift Estimation. In Advances in Neural Information Processing Systems , volume 33, pages 3290-3300. Curran Associates, Inc., 2020.

S. Gong, M. Li, J. Feng, Z. Wu, and L. Kong. DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models. In International Conference on Learning Representations , Feb. 2023.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative Adversarial Nets. In Advances in Neural Information Processing Systems , volume 27. Curran Associates, Inc., 2014.

H. Guo, L. Zhu, and K. Huang. Are GANs Biased? Evaluating GAN-Generated Facial Images via Crowdsourcing. In NeurIPS 2022 Workshop on Human Evaluation of Generative Models. , 2022.

J. Ho, A. Jain, and P. Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems , volume 33, pages 6840-6851. Curran Associates, Inc., 2020.

A. I. Humayun, R. Balestriero, and R. Baraniuk. MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining. In International Conference on Learning Representations , Jan. 2022.

A. I. Humayun, R. Balestriero, and R. Baraniuk. Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Values. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10631-10640, New Orleans, LA, USA, June 2022. IEEE.

A. Jahanian*, L. Chai*, and P. Isola. On the "steerability" of generative adversarial networks. In International Conference on Learning Representations , volume International Conference on Learning Representations. International Conference on Learning Representations, Dec. 2019.

L. Jing, J. Zbontar, and y. lecun. Implicit Rank-Minimizing Autoencoder. In Advances in Neural Information Processing Systems , volume 33, pages 14736-14746, 2020.

T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive Frowing Of GANs For Improved Quality, Stability, And Variation. In International Conference on Learning Representations , 2018.

T. Karras, M. Aittala, S. Laine, E. Härkönen, J. Hellsten, J. Lehtinen, and T. Aila. Alias-Free Generative Adversarial Networks. In Advances in Neural Information Processing Systems , volume 34, pages 852-863. Curran Associates, Inc., 2021.

D. P. Kingma and M. Welling. Auto-Encoding Variational Bayes, 2013. arXiv:1312.6114 [cs, stat].

K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. de Brébisson, Y. Bengio, and A. C. Courville. MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

H. Ledford. Millions of black people affected by racial bias in health-care algorithms. Nature , 574(7780):608-609, Oct. 2019.

A. S. Luccioni, C. Akiki, M. Mitchell, and Y. Jernite. Stable Bias: Analyzing Societal Representations in Diffusion Models, Mar. 2023. arXiv:2303.11408 [cs].

M. Luo, J. Cao, X. Ma, X. Zhang, and R. He. FA-GAN: Face Augmentation GAN for Deformation-Invariant Face Recognition. IEEE Transactions on Information Forensics and Security , 16:2341-2355, 2021. Conference Name: IEEE Transactions on Information Forensics and Security.

V. H. Maluleke, N. Thakkar, T. Brooks, E. Weber, T. Darrell, A. A. Efros, A. Kanazawa, and D. Guillory. Studying Bias in GANs through the Lens of Race, Sept. 2022. arXiv:2209.02836 [cs].

M. Mehrab Tanjim, R. Sinha, K. K. Singh, S. Mahadevan, D. Arbour, M. Sinha, and G. W. Cottrell. Generating and Controlling Diversity in Image Search. In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , pages 3908-3916, Waikoloa, HI, USA, Jan. 2022. IEEE.

Y. Mirsky and W. Lee. The Creation and Detection of Deepfakes: A Survey. ACM Computing Surveys , 54(1):1-41, Jan. 2022.

M. Orcutt. Are Face Recognition Systems Accurate? Depends on Your Race., 2016.

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in PyTorch. In NIPS 2017 Autodiff Workshop . NIPS 2017 Autodiff Workshop, 2017.

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research , 12(85):2825-2830, 2011.

A. Radford, L. Metz, and S. Chintala. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Jan. 2016. arXiv:1511.06434 [cs].

H. Rangwani, K. R. Mopuri, and R. V. Babu. Class balancing GAN with a classifier in the loop. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence , pages 1618-1627. PMLR, Dec. 2021. ISSN: 2640-3498.

D. Rezende and S. Mohamed. Variational Inference with Normalizing Flows. In Proceedings of the 32nd International Conference on Machine Learning , pages 1530-1538. PMLR, June 2015. ISSN: 1938-7228.

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-Resolution Image Synthesis with Latent Diffusion Models, Apr. 2022. arXiv:2112.10752 [cs].

S. I. Serengil and A. Ozpinar. HyperExtended LightFace: A Facial Attribute Analysis Framework. In 2021 International Conference on Engineering and Emerging Technologies (ICEET) , pages 1-4, Oct. 2021. ISSN: 2409-2983.

X. Shao, K. Stelzner, and K. Kersting. Right for the Right Latent Factors: Debiasing Generative Models via Disentanglement, Feb. 2022. arXiv:2202.00391 [cs].

Y. Shen, J. Gu, X. Tang, and B. Zhou. Interpreting the Latent Space of GANs for Semantic Face Editing. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9240-9249, Seattle, WA, USA, June 2020. IEEE.

Y. Shen, C. Yang, X. Tang, and B. Zhou. InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs, Oct. 2020. arXiv:2005.09635 [cs, eess].

T. Sipka, M. Sulc, and J. Matas. The Hitchhiker's Guide to Prior-Shift Adaptation. In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , pages 2031-2039, Waikoloa, HI, USA, Jan. 2022. IEEE.

J. Song, C. Meng, and S. Ermon. Denoising Diffusion Implicit Models, Oct. 2022. arXiv:2010.02502 [cs].

S. Tan, Y. Shen, and B. Zhou. Improving the Fairness of Deep Generative Models without Retraining, Mar. 2021. arXiv:2012.04842 [cs].

L. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni. Modeling Tabular data using Conditional GAN. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

Z. Zhang, Y. Song, and H. Qi. Age Progression/Regression by Conditional Adversarial Autoencoder. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4352-4360, July 2017. ISSN: 1063-6919.

A Appendix

A.1 Calculating E for The Generated Distribution

The error rates reported for a classifier C ϕ are typically reported on the distribution on the distribution of fit's training data, P training . However, the distribution P G θ of the generative model

G θ may differ from the training distribution. Additionally, rather than reporting P ( y | ˆ y ) , often times the error rates are given in a confusion matrix C ˆ y | y where C ˆ y | y [ i, j ] = P (ˆ y | y ) . Thankfully, we can construct the error rate matrix E for the generative distribution P G θ under the simplifying assumption that the difference between P G θ and P training can be explained as a label shift [10, 40].

By Bayes' Theorem, we know that

Under the label shift assumption, P (ˆ y | y ) stays the same between P training and P G θ . Additionally, P ( y ) can be calculated for P G θ under label shift [10, 40]. Lastly, P (ˆ y ) can be approximated for P G θ by finding the proportion predicted for each class on a large sample from the generative model. Thus, E can be calculated as:

A.2 Distribution of Races Generated By Progressive GAN

We show the two best performing methods' distributions on Progressive GAN, along with the distribution of the unmodified ProgressiveGAN, over the Race attribute.

asian

black

indian

latino hispanic middle eastern

white

Race

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Density

Distribution Of Races Resulting From Compared Methods

Distribution

Mapping (Ours)

Polarity Sampling (Antimode)

Standard Progressive GAN

A.3 Comparison Of Generated Faces From LDM Model

A comparison of a bath of random faces from the original LDM is shown in Figure 7, while Figure 8 shows a batch from the LDM when our Distribution Mapping method is applied.

A.4 Implementation Details

Ground Truth Shape Classifier

----------------------------------------------------------------

Layer (type)

Output Shape

®

================================================================

================================================================

Encoder for Shapes VAE

----------------------------------------------------------------

================================================================

Decoder for Shapes VAE

----------------------------------------------------------------

================================================================

Biased Age Classifier (Note: Target value was normalized age, made binary after)

----------------------------------------------------------------

Ground Truth Age Classifier (Note: Target value was normalized age; made binary after)

----------------------------------------------------------------

================================================================

The distribution mapper used default architecture of SDV's CTGAN 5 version 0.6.0, except for in the ProgressiveGAN experiment where embedding_dim =512, generator_dim =(512,512) were passed as arguments.

For the networks we trained, we utilized the Adam optimizer with learning rate between 0.002 and 0.0001.

The linear classifier utilized Scikit-Learn's LinearSVC (for latent editing) and RidgeClassifier for the biased Shapes classifier.

A.5 Proof of Lemma 1

Proof. First, note that if 1 |Y| ∈ cone( E ) , then likewise 1 Y 1 |Y| ∈ cone( E ) .

Let z ′ ∼ P z | C ϕ = i ; i.e., z is a draw from the distribution of noise such that the classifiers prediction of the generated sample corresponding to z ′ is group i .

Let ( C ′ ◦ G θ ) ∗ P z | C ϕ = i be the pushforward distribution of the perfect classifier C ′ 's output when conditioned on the generator's output of draws from P z | C ϕ = i . Then,

Thus,

Therefor, following from above,

This means that ∃ λ 1 , λ 2 , . . . , λ |Y such that the following holds:

This is equivalent to saying that:

for z ∼ ∑ |Y| i =1 λ i P z | C ϕ = i = Q λ . Thus, by definition Q λ is a Fair Noise Distribution.

A.6 Proof of Lemma 2

Proof. Note that the sign of the coefficient of the cross product E : , 1 × E : , 2 is P ( y = 1 | ˆ y = 1) P ( y = 2 | ˆ y = 2) -P ( y = 1 | ˆ y = 2) P ( y = 2 | ˆ y = 1) . Also note that E : , 1 × [0 . 5 , 0 . 5] is 0 . 5 P ( y = 1 | ˆ y = 1) -0 . 5 P ( y = 2 | ˆ y = 1) .

Additionally, and

Thus, the coefficient of E : , 1 × E : , 2 is greater than E : , 1 × [0 . 5 , 0 . 5] , while there signs are equal. This implies that [0 . 5 , 0 . 5] is in between E : , 1 and E : , 2 . Thus, [0 . 5 , 0 . 5] ∈ cone( E ) . The rest of the proof follows directly from Lemma 1.

A.7 Proof of Proposition 1

Proof. Note that P λ E has density [ ∑ i λ i Pr ( y = 1 | ˆ y = i ) , . . . , ∑ i λ i Pr ( y = N | ˆ y = i )] . For ease of notation let us refer to ∑ i λ i Pr ( y = m | ˆ y = i ) as r λ m .

Then,

Note that log ( 1 N ) is constant for each term in the second summation. Thus,

As log ( 1 N ) does not depend on r λ m ,

A.8 Proof of Proposition 2

Proof.

A.9 Proof of Theorem 1

The first statement follows directly from Proposition 1 and Proposition 2.

If C ϕ = C ′ , then { E : , 1 | E : , 1 | , . . . , E : , |Y| | E : , |Y| | } forms a standard basis of R |Y| , and therefor 1 |Y| is in cone( E ) . Thus, Q λ ∗ is a Fair Noise Distribution by Lemma 1.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how to learn a fair latent distribution so that a pretrained generator outputs samples that have equal likelihood to fall into different attributes. The paper assumes access to an imperfect attribute classifier. The proposed method learns the fair latent distribution by maximizing the entropy of $P_E^{\\lambda}$ and then use a generative model to distill the fair distribution. It is guaranteed that the optimal solution leads to a perfectly fair or minimally unfair distribution. The paper conducts experiments on several image datasets and shows that the proposed method leads to the best fairness in terms of shapes, ages, and races.\n\nSTRENGTHS:\nThe paper is extremely well written and organized. It is quite easy to follow and understand the difficulties of the defined task and the motivation behind the solution. \n\nIt also tries to solve an important problem in fairness research as modern deep generative models are very powerful but the fairness issues are not well understood. \n\nThe proposed method is simple but theoretically very sound. The paper clearly states conditions when the optimal fair distribution may be obtained and when the best possible one is obtained. \n\nThe experiments are also extensive. It includes several different sets of attributes and the results look very good to me.\n\nWEAKNESSES:\nThe experiments do not demonstrate the approximation error or tradeoff of each component of the proposed method. For example, how well does the network learned in section 3.2 approximate $Q^{\\lambda}$? How faraway is the learned $P_E^{\\lambda}$ from the optimal one? Some visualization or quantitative study for this can help us understand how we may be able to further improve fairness. \n\nMinor points: I also recommend the authors to fix a few typos in the math equations and replace the italic words with \\mathrm to make them look nicer.\n\nQUESTIONS:\nPlease refer to the weakness comments.\n\nLIMITATIONS:\nThe paper discussed limitations in the last section.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a method for mitigating biases in a pre-trained generator, without the need to retrain the generator or rely on real data. The authors propose the use of a classifier to create a fair distribution of noise vectors, which are then sampled to generate an equal number of instances for each group. Additionally, the paper presents techniques for constructing a fair noise distribution in scenarios where the classifier is either perfect or imperfect.\n\nSTRENGTHS:\n1. The paper's motivation is compelling, considering the widespread adoption of Generative AI across various applications. The problem addressed in the paper is of utmost importance in this context.\n2. The authors provide comprehensive results obtained from experiments conducted on three relevant datasets: Shapes, UTKFace, and CelebA-HQ datasets.\n3. The paper is well-structured and easy to follow.\n\nWEAKNESSES:\nThe proposed approach demonstrates a degradation in performance in the case of Progressive GAN applied to the CelebAHQ dataset. The paper includes a discussion that attributes this decline to the observation that a majority of the transformed latent space regions are predominantly associated with the over-represented semantic attribute. This limitation hampers the effectiveness of the proposed approach when dealing with complex datasets. I believe that this problem arises due to the fact that the approach does not involve tweaking the generator function (no retraining involved) but only a transformation of the latent space of the generator.\n\nQUESTIONS:\nThe paper lacks a discussion and comparison with relevant baselines, such as [1] and [2], which address the same issue of debiasing generative models without requiring retraining. Given that the authors have included results for Progressive GAN, it would be intriguing to see a comparison between the proposed approach and the aforementioned papers. This would provide valuable insights into the effectiveness of the proposed method in relation to existing techniques.\n\n[1] Karakas, C., Dirik, A., Yalçınkaya, E., & Yanardag, P. (2022). FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations. European Conference on Computer Vision.\n\n[2] Tan, S., Shen, Y., & Zhou, B. (2020). Improving the Fairness of Deep Generative Models without Retraining. arXiv Preprint arXiv:2012. 04842.\n\nLIMITATIONS:\nThe authors have addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper proposes a method for training a Distribution Mapper that can generate a noise distribution capable of creating Fair Samples using an Imperfect Classifier. They conducted experiments on toy examples and several Face Generation models, demonstrating quantitatively superior performance compared to competitors.

STRENGTHS:
The paper proposes an effective approach to achieve debiasing without the need for additional training of generative models or classifiers by ingeniously devising a method to train $M$.

WEAKNESSES:
Although it is mentioned in Line 114 that $M$ could be GAN, VAE, DDPM, normalizing flow, etc., there are no additional experiments conducted to explore this further. There is a lack of experimentation to examine how the noise shifted by $M$ affects image quality. It would be beneficial to report FID scores for CelebA as well. While $G$ is defined as a generative model, there are only experiments conducted with GAN. Particularly for Diffusion models, it is doubtful whether $M$ can be effectively trained.

QUESTIONS:
Does the verification of debiasing using metrics such as CLIP Score yield similar results?

LIMITATIONS:
Please refer to the Weakness part. I have a positive view of the content and approach presented in this paper. It would be beneficial to explore different models for M, not just limited to GANs, and investigate their performance. Additionally, conducting experiments on more recent models trained on datasets like CelebA and FFHQ would be valuable. It would also be interesting to observe the tendencies of commonly used models such as StyleGAN2, DDPM, LDM, VAE, and others. Including these aspects would enhance the quality of the paper and make it even more comprehensive.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an approach to mitigate biases in pre-trained generative models. Specifically, the authors aim to generate a semantically fair distribution over the attributes. Authors propose to train a distribution mapper network that effectively learns to generate a fair noise distribution, which, in turn, is used to generate samples that adhere to fairness principles such that the distribution of the generated samples per-group is closer to the uniform distribution. \nAuthors theoretically consider the case where the classifier to train the mapper network is imperfect and propose an approach to handle this case. Furthermore, they study this approach on three tasks, Shapes Dataset, Age Bias and race bias and compare with existing debiasing methods.\n\n\n-----------------------------------------------------------------------------------------------------\nFollowing Neurips guidelines: I acknowledge that I have read the authors response, engaged in the discussions and read other reviewers comments.  Based on the authors rebuttal, I have updated my score accordingly and increased it from 6 to 7.\n\nSTRENGTHS:\n- The paper addresses an important problem, debiasing generative models output.\n- The paper proposes an approach that does not require training or fine-tuning a generative model which is computationally efficient.\n- The paper proposes to use a pre-trained classifier that is able to classify groups and they also propose an algorithm that handles imperfect classifiers. \n- In the experiments, the paper compares with several other existing methods for debiasing the generative models and in all the reported cases, the method out-performs existing methods.\n\nWEAKNESSES:\n- The authors frame this work around the general generative models, however, in the experiments, only GANs are studied. I believe the study of diffusion models or other types of generative models is important. Specifically, diffusion models generate samples stochastically, and changing the initial noise distribution may not be as trivial as GANs. Furthermore, diffusion models have a strict assumption on the starting noise to be from the uniform distribution. Thus, changing the initial noise distribution is not trivial. I believe the paper should be framed around GANs since it is the only model that is studied empirically and is trivial to extend given the setup.\n- The authors mention that pre-trained classifiers that classify specific groups/attributes is easily accessible. However, in many cases that may not be the case (likely the issue with the experiment on progressive GANs).\n- The experiments are minimal and limited to GANs. It would have been intersting to see experiments related to other types of generative models or extend other types of more large-scale GANs or larger scale datasets.\n\nQUESTIONS:\n- Since the work is framed in a way that supports general generative models, how do you propose to use this approach for VAEs or diffusion models?\n- How do you handle the stochasticity that comes from the noise in the GPU training? Do you need the training process of the GAN to be deterministic?\n\nLIMITATIONS:\nauthors have already addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}