Debiasing Pretrained Generative Models by Uniformly Sampling Semantic Attributes
Authors: Walter Gerych
Kevin Hickey
Luke Buquicchio
Kavin Chandrasekaran
Abdulaziz Alajaji
Elke Rundensteiner
Emmanuel Agu
Venue: NeurIPS 2023 poster (2023)
Keywords: generative models
generative modeling
bias
GANs
debiasing
TL;DR: We develop a method to debias pretrained unconditional generative models, such that they produce an even number of sample for each group.

Abstract: Generative models are being increasingly used in science and industry applications. Unfortunately,  they often perpetuate the biases present in their training sets, such as societal biases causing certain groups to be underrepresented in the data. For instance, image generators may overwhelmingly produce images of white people due to few non-white samples in their training data. It is imperative to debias generative models so they synthesize an equal number of instances for each group, while not requiring retraining of the model to avoid  prohibitive expense. We thus propose a *distribution mapping module* that produces samples from a *fair noise distribution*, such that the  pretrained generative model produces *semantically uniform* outputs -  an equal number of instances for each group - when conditioned on these samples. This does *not* involve retraining the generator, nor does it require *any* real training data. Experiments on debiasing generators trained on popular real-world  datasets show that our method outperforms existing approaches.

Debiasing Pretrained Generative Models by Uniformly Sampling Semantic Attributes
Walter Gerych Kevin Hickey', Luke Buquicchio Kavin Chandrasekaran'
A Abdulaziz Alajaji?,
Eike Rundensteiner', Emmanuel Agu' Worcester Polytechnic Institute, Worcester, MA
2 King Saud University, Riyadh, Saudi Arabia
Abstract
Generative models are being increasingly used in science and industry applications. Unfortunately, they often perpetuate the biases present in their training sets, such as socictal biases causing certain groups to be underrepresented in the data. For instance, image generators may overwhelmingly produce images of white people due to few non-white samples in their training data. It is imperative to debias generative models they synthesize equal number of instances for each group, while not requiring retraining of the model to avoid prohibitive expense. We thus propose distribution mapping module that produces samples from fair noise distribution, such that the pretrained generative model produces semantically uniform outputs- equal number of instances for each group when conditioned on these samples. This does not involve retraining the generator, nor does it require any real training data. Experiments on debiasing generators trained on popular real-world datasets show that our method outperforms satee-of-hhe-al approaches.
Introduction
Background. Generative models have become a cornerstone of modern machine learning, allowing for the synthesis of realistic data for many domains, including images 19 35), audio [22]. and text
M data
However, even leading generative models often reproduce the biases present in their training
26), such image generation models strongly over-representing white males 24 As
generative models are increasingly used for data augmentation to train downstream models (5] in domains from scientific medical fields (25 99. biased synthesized data could lead to results that are skewed or inaccurate. This can exacerbate existing issues such as facial recognition models performing significantly worse on non-white individuals [29] or healthcare models being much less accurate for certain minority groups (23 Further, with the rapid growth of generative models in commercial applications, the potential financial, legal and ethical costs of biased outputs are significant. Thus, it is imperative to develop methods that mitigate bias in generative models to ensure that their outputs are fair and equitable by generating a roughly equal number of samples of each group. We call such outputs semantically uniform, and the attribute that they uniform over such as gender or race the semantic attribute.
State-of-the-an Existing methods for addressing bias in generative models often train a new model from scratch 27
though this is computationally expensive, requires significant labeled data, and wastes resources already previously spent training the existing (biased) generative model. Latent attribute editing methods modify the samples in the latent space of generative models to produce controlled changes in the output, and could potentially be used to correct for bias [17]. However, this requires making limiting assumptions such as that semantic attributes correspond to linear directions
Coorespponiing author: sverych@wpi.cr
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Freq
Semantic
Standard Nabse Distrihation
Pretriined Incondhtonall
Nutelenn youddr
Freq
Semantic Attribute
Anselenn yourder
Figure 1: A generator conditioned on a fair noise distribution yields outputs that are uniform over a
semantic attribute (i.e. race).
in the latent space 17 33 6 33 39 8. While some recent advances have been made in uniformly sampling the output space of pretrained model [15] 16), they can fail to yield uniform samples over semantic attributes (5). For example, they will make an image generator's output uniform over manifold in the pixel space, but may still overproduce images of white individuals as the output won't 't be uniform over the semantic attribute of race.
Problem statement. Recently, numerous pretrained generative models and classifiers have been released for a variety of domains. For instance, there are publicly available generators for images of faces 19 followed by classifiers that predict attributes such as race and age of faces [36. Here, we assume access to a pretrained generative model Gg that maps noise space 2 to feature space X, and a pretrained classifier Co that maps X' to a discrete semantic attribute space y. Additionally, as error rates are often reported for released models, we assume that the class-conditional errors; i.e., the probability that an instance actually belongs to group j given that the classifier incorrectly predicted group is known. Our goal is to model distribution Q such that for that are uniform over the attribute space y, using only predictions from Cg to guide the output of Gg. For instance, if there are five discrete semantic groups in V, G9 will produce an equal number of instances belonging to each group when conditioned on draws from Q We call Q a fair noise distribution: see Figure for an example.
Q, Go(z) produces samples
Challenges. Our task producing samples that are semantically uniform has three major hurdles:
1. Expensive retraining: It is often prohibitively expensive to retrain large generative models in terms of computational resources and time.
2. Inaccessible training duta: The data used to train a released generator is typically unavailable; either due to being proprietary or simply too large volume for most practitioners to utilize. Thus, do not have adequate data available to tune the generator classifier.
3. Inaccurate classifier: As we do not have any samples of real data available, we must rely on the possibly imperfect semantic attribute classifier Co to provide labels. However, since classifiers may produce inaccurate predictions especially on underrepresented groups of y. this can cause incorrect estimations of the number of instances for each group.
Proposed approach. We propose train a distribution mapper network M. 2 + Z that transforms draws from standard noise distribution into draws from fair noise distribution, such that once noise samples are transformed by M_, they condition the generator to produce a roughly equal number of instances for each semantic group in Y: We achieve this by first constructing dataset of instances from Z that follow a fair noise distribution, which allows us to estimate the true distribution of the semantic attribute given an imperfect classifier. Using proposed strategy, noise samples collected such that the pretrained classifier's s (corrected) distribution is uniform. Then, we use this fair noise dataset to train Ms to inexpensively sample new instances from the fair noise distribution. The output of My serves as input to the pretrained generator, which yields samples that are uniform over the space of the semantic attribute. Importantly, this approach does not require retraining the main generative model, thus incurring minimal costs. Our method works without requiring any real training data. Relying only on a pretrained generator and classifier pair, we circumvent the need to acquire any real data (labeled or otherwise) even using only an imperfect classifier.
Contributions. In this work, we:

---

Show how to construct fair noise dataset that produces semantically uniform synthetic outputs when passed through a generator, assuming an accurate classifier available.
Design a method to correct for the error incurred by an imperfect classifier, to construct a dataset of samples that come from a fair noise distribution.
Propose an approach for training distribution mapping network to sample from a fair noise distribution on-the-fly.
Demonstrate the utility of our approach on a range of real-world datasets and released generators.
2 Problem Definition
Assume we are given a pretrained generative model G2 that maps a latent space 2 to feature space X. In general, we assume that Z is of a lower dimensionality than X. Additionally, we assume we are given a pretrained classifier Cg that maps X' to a semantic attribute space y. where )' is the set of group (class) labels. Thus, y = {Y,, Y2, YN). where Y; is the label of the ith group. When not otherwise ambiguous, we will refer to group Y; as group 2. Let y and y be the random variables indicating the true semantic attribute label and the predicted label from Co respectively. Let E be the prediction prediction-condiioona error rates of C.. with E being a left stochastic matrix such that Fa. Ply ily 5)- Let Co be better-than-random classifier, i.e., for N groups, P(y ily = i) As it is typical for developers to report the error rates of their classifiers, we assume that E is known. While these error rates typically reported for the classifier's ' training distribution, E to apply to the generated distribution under a label shift assumption [40]. See the Appendix for details this and for the proofs of the upcoming theorems. Lastly, let C' denote ideal perfect classifier that always correctly predicts y with zero error. C' is hypothetical and unavailable to us.
for all groups i. Thus, E is by definition diagonally dominant matrix.
correct
Our goal here is to sample from a Fair Noise Distribution, defined follows:
Definition 1 (Fair Noise Distribution). For a generative model Gg 2 ~ X, distribution Q over Z a Fair Noise Distribution with respect to y if Q (G ) ~ Unif(Q)), where Unif()) is the uniform distribution over the groups in y
Intuitively, a distribution Q is a Fair Noise Distribution if Go produces samples that are uniform over the semantic space y when conditioned on Q. For example, if Go synthesizes images of people and Y is the set of races, then Q is Fair Noise Distribution if conditioning on it makes Go produce a roughly equal number of images of people from each race.
More realistically, we want to find Q that is easy to sample from and produces reasonable variance from the generator. That is, it does not yield only
unique example of each group, which would otherwise make the problem trivial. Notably, we assume that we do not have any real training data (i.c., no real samples from X). Also, we do not aim to retrain Gg. i.c., not change the parameters 0 of Gg. Additionally, we do not make any assumptions on the differentiability of Go or Co.
3 Methodology
Our approach for sampling from a Fair Noise Distribution Q hinges on training a distribution mapping
function M_, such that (z) fQ, where is draw from the original conditioning distribution of Ga. The functional form of My has many options; for instance, My can be a GAN generator [12. No matter which form chosen, we will need a dataset of samples drawn from Q to train M
VAE (21), a DDPM [14], or normalizing flow model [34 7
3.1 Collecting Fair Samples Using Imperfect Classifiers
naive method for collecting a dataset of samples distributed according Fair Noise Distribution @ is given in Algorithm [] The basic approach is to continuously sample z from the noise distribution, collect the generator's output for each draw, and use the classifier to determine the value of the semantic attribute corresponding to each noise draw. As result, samples of z corresponding each group are saved to dataset De until De has S number of samples for each y E y. This procedure results in samples of noise that once passed through Go will be uniform across the semantic attribute
3

---

Algorithm Naively collect data from Q
D Colect from Q by trusting Co 2: Input: Pretrained generator Gg and classifier Cg. number of samples S for each attribute
1: procedure SAIVE_Q_DAAASEI
Co, S)
3: Output: Dataset with S samples for each y E y
4: Q_ daaaasst- t- dict()
Initialize empty dictionary for the dataset
5: for group E y do
6: Q 2_dataeetlgroup) =
Initialize every group as empty array
7:
end for while key E =_datasse.kkys s.t.
8:
< S do
9:
noose
Sample noise
10:
group = Ca(Go(z)
P Get the predicted group Ga(=)
11:
D_dalasel(class if (app append(()
(class))))))) < S then
12:
& If samples for that class, add = to the dataset
13:
end if end while
14:
15: return Q_dataset
Return dataset with S samples for each group in y
16:
end procedure
space according to Cøi i.e., for 2 ~ Dq. Co(Go(z)) ~ Unif()).. However, this will only yield samples from Fair Noise Distribution in the case where Cs is a perfect classifier (such that E is the identity matrix). If C& is an imperfect classifier then the distribution may not be truly fair. For instance, if Co often incorrectly predicts group j as group then instances with attributes matching group may be more prevalent than those for group j.
Fortunately, we utilize knowledge from the prediciinnccodiiionn error rates E to sample dataset of noise that will yield more semantically uniform generated instances despite the noisy predictions of Cs. To achieve this, we use weighted sample of datapoints predicted for each group. Let P P=IC,=; be a distribution over such that the imperfect classifier's prediction of generated samples arising from this distribution are all of group i; (G)G((z)) = for z ~ Pa|Ca=i- In addition, let @ = |
X,P =C such that A > 4A.0 and >!21 X =.
We can in many instances find values of A = 21,. such that QA is a Fair Noise Distribution, as stated in Lemma
Lemma 1. Let 119l be the vector of length [D] such that every element 1. Let cone(E) = {EaEE, E.,l; E [2 be the finite convex cone generated by the columns of the prediction-conditonal error matrix. If 11 E cone(E), then 3 X such that Q2* is Fair Noise Distribution.
Additionally, we can show that the case where y binary and C. is better-than-random. we can always find À such that samples drawn from @ will definitely yield generated samples that are uniform over the semantic attributes, as shown by the following lemma.
Let | = 2 and let Co be siiee--taa--anddeee such that the diagonal elements of E are each greater than 1 Then, 3 such that Q is Fair Noise Distribution.
Lemma 2.
The proof of Lemma 2 comes from showing that (1, 1) is always in the finite convex cone generated by Cs 's prediction conditional prediction-conditional error matrix, with Lemma I jimplying that this property guarantees that QA will yield samples that produce semantically uniform generated instances.
Even in the cases where we cannot guarantee that there exists an ideal X, we can still find values for A that will yield distribution that is as close as possible to a uniform distribution over the semantic space, with the difference from uniformity measured by KL divergence. For this, let define Minimally-Unfai Noise Distribution.
Definition (Minimally-Unfai Noise Distribution). For a generative model Go Z + X,
a distribution Q = >>> >/P=jC,= = ,,,...,,>xibll Xs E Rzo. Σ ); = 1.
is Siinimally-Unfaii Noise Distribution over Z with respect to y if for Q. X argmin (Ga())))

---

Algorithm 2 Collecting a dataset samples from QA
procedure WEIGHTED Saco___DAA__AA (Pa i=:
2: Input: D=IC,== for i from 1 + ( Dataset of noise samples that produce generated
instances which Cs classifies as group i; S: number of instances we wish to sample from Q^
3: Output: Dataset with S samples from Q
4: À + argmar: H (TP)))
5: Q datasst=
> Initialize the @ as empty array
6:
for m +- to S do
random_mnnt( (n) ~ Dz|C.=r @_dataset.appendl 10: end for
7:
get random int (1 to (31) with probability proportional to À
8:
9:
11: return Q_dataset
Return dataset with S samples from Q
end procedure
Intuitively, @A is a Minimally-Unfair Noise Distribution the values of yield generated samples with minimal divergence from semantically uniform distribution, under the constraint that Q^ is a convex combination of the P. P= ss. This constraint is required so that we can sample from Q^ using a weighted sampling technique based off of our imperfect classifier Cø.
Fortunately, we can easily find the values of that will yield a Minimally-Unfinn Noise Distribution. Before deriving the procedure for this, let define the distribution P the normalized weighted sum of the columns of E, where each column is weighted according to a corresponding Ae:
Definition 3 (P Define P Ei2 X,E distribution over y determined by prediction- conditional error matrix F. and = (A1,
X E R20. ΣM A==11
Next, we note that finding the values for À that maximize the entropy of P is equivalent to finding the A for which the divergence between P and Uni is minimized. This is stated formally in the following Proposition.
Proposition 1. Xf argmax H(P)) where H is entropy, then for the same X it is true that
X argmin (
Before stating the theorem that directly implies strategy for learning for which Q is Minimally- Unfair, all that is left to do is to link F with the distribution of C" (G)()))
Proposition 2. Ifz ~ QA then it is true that ('(((zz) ~ P2.
Proposition D states that for given À the distribution of the perfect classifier C' given samples of the generator conditioned on draws from QA will be distributed according to P Now, we state Theorem D which directly implies sampling strategy.
Theorem 1. If X* = argmax H H(P)). then Q is Minimally-Unij Noise Distribution. When
C² 10 cono(E), then Q² is also Fair Noise Distribution.
Theorem [ follows from the preceding Propositions. To see how Theorem I iimpliss a strategy for sampling from Minimally-Unfair Noise Distribution, recall that Q is defined as the mixture of distributions of each Pz|Ca=i for from > [}]] with X; as the ith mixture weight, where P=cc,=; ; is the distribution of z that yield generated samples that the noisy classifier predicts as group i. The above theorem states that the mixture of these distributions with mixture weights that produce maximum entropy for PA will yield mixture distribution QA that is worse-case Minimally-Unfair and, when possible, will be a perfect Fair Noise Distribution. Thus, to sample from QA all that is needed is to 1) construct datasets D DziC.=i for i from 1 [)] such that if E D De then (Z)()z)) = í (i.e., these datasets serve as proxies for P P=]C,=);; find À such E. is maximized; and 3) perform a weighted sampling from each D2jc,= proportional to its corresponding X, Note that the datasets De|Ca=i can be formed by approach similar to
that H HPPA)
5

---

NeiseyLaten Distribution
Fair Noise Distribution
Freq
Pednicall Generatrr GB
Porfac
Happzr M
Somantic Attribute
Figure 2: Distribution Mapper M. turns standard noise distribution into Fair Noise Distribution.
Algorithm I The algorithm returns a dictionary with keys being the elements of ) and the values of each key returned from that procedure form the corresponding required datasets (i.e., De(C,=))- pseudo-code implementation our approach for constructing a dataset of noise samples from Minimally-Unfair Noise Distribution Q is given in Algorithm 2
3.2 Training the Distribution Mapper
Let Do be a dataset of noise samples from the space distributed according to Q^, such that Dea is obtained described in the previous subsection (i.e., Dos comes from Algorithm I in the case where the error rates are unknown or Co can be assumed to be an ideal classifier, or from Algorithm
2 otherwise). Now, we train distribution mapper M. Z + Z such that if noiss noise distribution such as a multivariate
then (w) ~ Q where mosse an easy-to-same tew-tto-smml-feeee Gaussian, whatever noise distribution was used input when training Go. As previously stated, the functional form of Ma and the procedure used to train it is flexible. There a wide class of generative models and training strategies that could be employed [3. We chose model M. GAN generator and find the parameters of Ma. using adversarial training. Thus, we train M., as such:
max min [MM F2) EgNDe log (Fo(z)) Evw-Poot
log (1 Fo(Ma(v)))]
where Fp is a discriminator network.
Note that z is a multivariate vector, where the ordering of the elements of the vector are arbitrary (i.e., there is likely spacial relationship z). Thus, the inductive bias of convolutional filters used in many leading GANs 20 19 32 is not appropriate in this case. Rather, we suggest to train the distribution mapper using GAN architecture designed for tabular data; notably, CTGAN which has shown to perform well on data where a sequential or spacial inductive bias is not appropriate (4
After training M. we sample arbitrary amount of samples from Q without applying Algorithm orr Algorithm 2 as Mv) ~ Q^ for v ~ Pooise Importantly, C' ("(Go(M ~ Unif()) when @A a Fair Noise Distribution, and will otherwise have minimum divergence from Unif()) under the constraints given in Definition 2 Notably, imperfect classifier, and without fine- Gg. Instead, the only training needed for Ma which in general should require much less complexity (and thus much less cost) than Go. Figure shows the Distribution Mapper paired with the pretrained generator Gg.
achieve this with samples of real data,
Experiments
We now experimentally evaluate our approach. Details such as hyperparameter choice and architec- tures are available in the appendix. Code for our method is available in the supplemental material.
We compare our approach on a range of generators: VAE 21. DCGAN 32), and Latent Diffusion Model (LDM) [35.
Progressive GAN [19]
4.1 Compared Methods
Latent Editing (2019). We apply the commonly used latent editing 17 38
approach to the task of constructing a dataset with an equal number of instances from each group in y. While some work uses non-linear directions when editing [I] using linear directions has been shown to work as well in practice [17 g. Thus, for each group y E y, we fit a linear classifier K,, on noise samples from Panisse with the goal of separating noise instances that are mapped to group from those mapped to all other groups. After training these classifiers, we can sample a noise instance belonging to group with the following procedure: First, sample IP Poisei if Kg (2) y then
6
33
39

---

Distribution Attributes
Darribation (Oum)
Sampling With and Without Error Correction
0.7
of Sampling (WiN Froor Correniio)
Edrlig
Naion Sangling (No Srive Comwdron))
McCENT
Inlartly Sumping (tstimodee
Denseitg
Polatiy Sumplin Ofodeal
Deaolyy
Shutted Samanta: Suttributin
Vanilla Genemstre
0.5
0.2
0.4
0.
0.0
Carcles
Squares
0.0
Circles
Attribute
Attribute
(a) Effect of bias correction; 0.5 (dashed line) ideal. (b) Comparison of methods; 0.5 (dashed line) is ideal.
Figure 3: a) Effects of applying bias correction when constructing Fair Noise dataset; b) compari- son of methods on the Shapes dataset.
return 2, otherwise perform latent editing on z until (y(Znew) y. The editing approach is given by Znew = 2previous + E Ty, where is the normal vector to K, 's decision boundary and is step size. We set 0.1 our experiments.
MaGNET (2022). This method was recently proposed to uniformly sample the manifold of pretrained generative models (15), using sampling strategy that leverages the Jacobian of the generative model. The authors argue that while this does not guarantee an equal distribution for each group, MaGNET should increase the frequency of disadvantaged groups by sampling more often from low-density regions of the generative manifold.
Polarity Sampling (2022). An extension of MaGNET, Polarity Sampling allows for more controlled sampling over the generative manifold [16). Sampling is controlled by a parameter p where as p goes to modes sampled from increasingly often, antimodes are sampled from more p - oo, and the original generative distribution sampled from for 0. We thus compare against two Polarity Sampling settings; Polarity Mode Sampling where 22, and Polarity Antimode Sampling where 2. By a similar argument used for MaGNET, sampling from antimodes may result in minority groups being more represented.
Shifted Semantic Distribution (2021). This method provides a training-free approach for debiasing pretrained generative models 42 The key idea is to fit Gaussian mixture models on the latent space, where these mixtures are fit on regions that correspond to unique values the semantic attribute. However, unlike our approach, Shifted Semantic Distribution assumes that each attribute class be uniquely identified by a hyperplane in the noise space.
Standard Generator. We compare against unmodified pretrained generators used as initially intended. This is baseline which other methods should outperform.
4.2 Uniformly Sampling From Shapes Dataset
We first evaluate our approach on a dataset synthetic images. Each image of either a circle or square, where the shape has random size, color, and position in the image. We use the object's shape as the semantic attribute; thus, y = {'Circle', 'Square'), }. This dataset was first used by Jing et al. [18). We utilize the VAE Jing et al. compared against in the same work our generator.
Testing Classifier Bias Correction Approach. We demonstrate the utility of correcting for biased classifier when constructing a dataset of Fair Noise samples. To that end, we compare the distribution of each group when we trust the biased classifier C. (Algorithm (T) to the distribution found when using our proposed biased correction approach (Algorithm{2). We use linear classifier trained to distinguish images squares from circles as our biased classifier Cg. This classifier is implemented using Scikit-learn's LinearSVC classifier [31] As ground truth, we utilize a deep convolutional network classifier that achieves approximately perfect accuracy on the task of distinguishing squares from circles. Thus, this network acts C".
7

---

Figure a) shows the distribution of shapes in the Fair Noise Dataset with ertor correction (Algorithm 2) and without applying our bias correction approach (Algorithm D.
As | = 2 for this experiment, the best case is when the density of each group is 0.5. Clearly, our bias correction approach yields a dataset that much closer to being uniform over the semantic attribute. This implies the need to correct for inaccurate classifiers when constructing Fair Noise Datasets.
Comparative Study on Shapes Dataset. We next test our proposed Distribution Mapping approach (Section 3.2) against the compared methods. For our approach, we train Ma as a CTGAN 43 trained on the corrected Fair Noise Distribution obtained in the previous experiment. As the compared Latent Editing method also requires a classifier as ground truth for training each of its linear models Ky. we compare against two versions of the method: 1) Latent Editing (Biased Classifier) using the biased Cp as ground truth, and 2) Latent Editing (Perfect Classifier) which uses the ground-true -trtth effectively perfect convolutional network discussed above. Note that from our problem definition, this perfect classifier would usually be unavailable to us.
As Figure shows, our approach clearly results in the most uniform distribution out of all compared methods. Additionally, the Latent Editing approach performs much worse when using the biased classifier. As expected, Polarity Antimode Sampling does indeed increase the frequency of the minority class (Circles). Interestingly, MaGNET also increases the frequency of the minority class, but is roughly as far from uniform as the distribution of the original generator; MaGNET flips the distribution and under-represents what was previously the majority class.
4.3 Age Bias in Face Image Generator
In this next experiment, we evaluate our proposed method's ability to debias a generative model that produces images of people's s faces. Specifically, the generative model is a DCGAN [32] that we pretrain grayscale version of the UTKFaae dataset [44]. For this experiment, the semantic attribute is Age. The age of each individual in UTKFaae is given as a label. As our approach assumes the semantic attribute y is discrete, we bin the ages in increments of 10 years such that y = S 9¹, '10-19',..... '90-99+)). }. We train a deep convolutional network as the ground truth classifier (used for evaluation), and use corresponding classifier with quarter of the feature maps as the biased classifier.
Evaluating Bias Correction Approach With Large Number of Classes. We repeat the bias correction experiment we performed on the Shapes data again for the UTKFace Generator. While the previous experiment required only two classes, we now have ten classes (one for each age bin). Despite this increase, Figure a) shows that performing the bias correction we propose in Algorithm 2 yields much more uniform Fair Noise training dataset than results from trusting the biased classifier.
Comparative Study on Reducing Age Bias. We evaluate all methods on the UTKFace Generator, with the goal of making the output images uniform over Age (i.c., generate an equal number of images of people belonging to each age group). Figure b) clearly shows that our approach (orange line) produces a much more uniform distribution over ages than the compared method, though it does generate images for the 10-19 bin too infrequently. While approaches such as Polarity Antimode Sampling and MaGNET are somewhat less biased away from generating images of young people, they fail to generate many samples for older individuals. Note that for the Latent Editing approach the we fit a linear regressor (guided by the ground truth) on the latent space rather than classifier, as age is more naturally a continuous attribute.
4.4 Uniformly Sampling Over Race in Progressive GAN
We apply our approach to debias a pretrained Progressive GAN [19]; specifically, the Py PyTorch 30 version of the celebAHH-2566 model 2 We use the Race of the individual in each im- age as the semantic attribute, and use the MTCNN classifier from the DeepFace (36) package to classify race. Matching the classes available in DeepFace, our semantic attribute space is y = ( Asiaann 'Indian', 'White', 'Middle Eastern', 'Latino Hispanic', 'Black'), } The Pro- gressive GAN strongly favors generating white individuals, likely because its training images were of predominantly white celebrities. Here, we consider the classifier to he accurate.
see htt.tt/ssgg.cccccc/e//////oo//oc sertoorch-gan--ae gooitaagnnnnoooooo zo0__gann/

---

Distribtion of Ages Resulting From Compared Mechods
Mappay 10ust Laber Bettng
stutid Suprazic Startration Stawdant DORAN
Prlerty (Midws)
Sampling
Sampling (No Brror Correction)
6.3-
o' Sampling (With Ereor Correction)
6.5
0.3
022
03:40 52.00
72-73 82-7 56-311
Age Group
Group
(a) Effect of bias correction; 0.1 (dashed line) is ideal.
(b) Comparison of methods; 0.1 (dashed line) is ideal.
Figure 4: a) Our biased correction approach yields better Fair Noise Distributions; b) comparison of methods correcting for age bias in the UTKFace DCGAN.
Divergence From Uniform of Each Approach The The CelebHO Progressive GAN
0.4-
Devegeene
0.7-
6.4- 1.44
0.3-
0-2-
0.1-
Diserbweton
Labwst Bealing
MaONET
Pelecity Sumpling Polerity Sampling (Anlznndoo)
Stifedd Semartie Dnstrbbtoon
Stanfard
(Meddo))
Figure 5: KL Divergence between the distribution over the semantic space for the output of each method (lower is better).
Comparative Study of Debiasing Progressive GAN. For each method, we report the KL divergence between uniform distribution and the elassifier's output the samples generated from the method in Figure 5) Note that unlike in the previous experiments, for this metric lower is better. Our approach the lowest KL divergence, indicating produces a more equal number of images of people from each race than any compared method. We note that despite this we still observe an over-representation of images of white individuals in the samples produced by our (and all other) approaches (see Appendix). This indicates that most regions of the latent space are likely associated with semantic attribute, and unless the distribution mapper very closely matches its Fair Noise training distribution, white persons will still be over-represented. Still, our approach results in a distribution that is most fair out of all compared methods. As we observed previously, sampling antimodes with Polarity Sampling produces next-best results; likely because it explicitly draws from low-probability regions of the latent space, which correspond to non-white individuals.
4.5 Uniformly Sampling Age in Latent Diffusion Model
Although our focus is primrily on models that map from low dimensional latent space to higher dimensional feature space, we also evaluate our approach on a latent diffusion model B51 that was trained on the Celeba-HQ dataset [19]. For this experiment we chose Age as the semantic attribute, and debias according to the following age categories: < 29. 30- 2 50 }. We utilize a pretrained ViT age classifier to provide feedback for the semantic attribute. Since our approach requires each latent code be mapped diffusion model to yield deterministic diffusion process. We used DDIM diffusion model to train the Mapper network as well.
single output image, we utilized the DDIM (1) sampler for the
4.5.1 Comparative Study on Debiasing Latent DDPM.
We report the KL divergence from uniform for the semantic distribution of each compared method in Table T. Note that we exclude MaGNET and Polarity Sampling from this experiment, due to
We utilized this model: htwp://www//.o.oooooooooooooooon

---

Distribution Mapping (Ours)
Latent Editing
Vanilla Latent Diffusion
KL Divergence From Uniform Distribution
0.130
0.525
0.411
For Semantic Attribute "Age"
(Lower is Better)
Table 1 Comparative study on debiasing the Age attribute for a pretrained L.MM.
the difficulty of obtaining the Jacobian determinant required by these methods. Results show that our approach yields significantly better scores than the other compared method, indicating that our approach can potentially be useful for debiasing Diffusion models well.
5 Broader Impact And Limitations
The goal of this work, reconditioning generative models to not reproduce the biased distribution of their training set but rather produce one that treats each group equitably, is driven by the desire mitigate the effects that systemic biases have on generative models increasingly used in real world applications. This has the potential for beneficial societal impact by counteracting the harmful effects of such systemic biases that lead these models to be unfair to underrepresented groups. However, the potential negative impacts of advancing generative modeling should not go unconsidered. Such models have already been used for some harmful applications such as Deep Fakes (28). It is important stress that while our approach aims mitigate the effects of bias existing models, it is not fix-all nor an excuse to train models on knowingly biased data. When possible, it is essential to collect fair and equitable training datasets, and to take measures to ensure that the models we train are fair without the need for post-hoe corrections.
L Limitations. Currently, our approach assumes that the semantic space y is discrete. We plan extend this work to handle continuous attributes, such as skin tone, in future work. Additionally, while the conditional distribution P(xly) should be roughly equal before and after applying our method if the classifier Cs is accurate, in the case of a biased classifier the conditional distribution may change. Correcting for potential distribution shift is likewise future work.
6 Conclusion
This is the first method to correct pretrained biased generative model (i.c., one that strongly favors generating images of white people over all other races) given only the generator and potentially- biased classifier. We propose a sampling strategy to construct a fair training set using the biased classifier in way that corrects for its bias, and Distribution Mapping module that uses this training set to learn how to sample noise instances that produce fair outputs when used as input to the generative model. Notably, we are able to debias the generative model without retraining the model or utilizing any real data. Our results indicate that our approach produces outputs that are much fairer than existing methods. This work may inspire more research on Distribution Mapping techniques to recondition generative models by transforming their standard latent distributions into distributions that yield more favorable behavior.
Acknowledgments
This material is based on research sponsored by DARPA under agreement number 1A8750-18-220077 The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed implied, of DARPA the U.S. Government.
Results in this paper were obtained in part using a high-performance computing system acquired through NSF MRI grant DMS- 13379433 to WPI. We thank the members of WPI's DAISY Research lab and the WPI WASH research group for their insightful feedback and support during the development of this work.
10

---

References
[1] T. Aoshima and T. Matsubara. Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model, Mar. 2023, 1xXiv2221.114577 [cs].
[2] S. Bhat, J. Jiang, O. Pooladzandi, and G. Pottie. De-Biasing Generative Models using Counter- factual Methods, Feb. 2023. arXiv:2207.01573 [cs].
[3] S. Bond-Taylon A. Leach, Y. Long, and C. G. Willcocks. Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(1 14(11))752777347, Nov. 2022.
[4] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- Voss, G. Krueger. T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language Models are Few-Shot Learners, July 2020. arXiv:2005. 14165 [cs].
[5] K. Cong and M. Zhou. Face Dataset Augmentation with Generative Adversarial Network. Journal of Physies: Conference Series, 2218(1):012035, Mar. 2022. Publisher: IOP Publishing.
[6] E. Denton, B. Hutchinson, M. Mitchell, T. Gebru, and A. Zaldivar. Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias, Oct. 2020. arXiv:1906.06439 [cs, stat].
[7] L. Dinh, D. Krueger, and Y. Bengio. NICE: Non-linear Independent Components Estimation, Apr. 2015. arXiv:1410.8511 [cs].
[8] P. Doubinsky, N. Audebert, M. Crucianu, and H. Le Borgne. Multi-attribute balanced sampling for disentangled GAN controls. Pattern Recognition Letters, 162:56-62, Oct. 2022.
[9] Festag, Denzler, and C. Spreckelsen. Generative adversarial networks for biomedical time series forecasting and imputation. Journal of Biomedical Informatics, 129:104058, May 2022.
[10] S. Garg, Y. Wu, S. Balakrishnan, and Z. Lipton. A Unified View of Label Shift Estimation. In Advances in Neural Information Processing Systems, volume 33, pages 3290-3300. Curran Associates, Inc., 2020.
[11] S. Gong, M. Li. J. Feng, Z. Wu, and L. Kong. DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models. In International Conference on Learning Representations, Feb. 2023.
[12] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative Adversarial Nets. In Advances Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
[13] H. Guo, L. Zhu, and K. Huang. Are GANs Biased? Evaluating GAN-Generated Facial Images via Crowdsoureing. In NeurlPS 2022 Workshop on Human Evaluation of Generative Models., 2022.
[14] Ho, A. Jain, and Abbeel. Denoising Diffusion Probabilistic Models. Advances Neural Information Processing Systems, volume 33, pages 6840 Curran Associates, Inc., 2020.
[15] A. I. Humayun, R. Balestriero, and R. Baraniuk. MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining. In International Conference on Learning Representations, Jan. 2022.
[16] A. Humayun, R. Balestriero, and R. Baraniuk. Polarity Sampling: Quality and Diversity Control of Trainreeedd Generative Networks via Singular Values. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10631-10640, New Orleans, LA. USA, June 2022. IEEE.
[17] A. Jahanian* L. Chai*, and P. Isola. On the "steerability" of generative adversarial networks. In International Conference on Learning Representations, volume International Conference on Learning Representations. International Conference Learning Representations, Dec. 2019.
11

---

[18] L. Jing, J. Zbontar, and y. lecun. Implicit Rank Lank-Miniiizing Autoencoder. In Advances in Neural Information Processing Systems, volume 33, pages 14736,14776, 2020.
[19] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive Frowing Of GANs For Improved Quality, Stability, And Variation. In International Conference Learning Representations, 2018.
[20] T. Karras, M. Aittala, S. Laine, E. Härkönen, J. Hellsten, J. Lehtinen, and T. Aila. Alias-Free Generative Adversarial Networks. In Advances in Neural Information Processing Systems, volume 34, pages 852-863. Curran Associates, Inc., 2021.
[21] D. Kingma and M. Welling. Auto-Encoding Variational Bayes, 2013. arXiv:1312.6114 [es. stat].
[22] K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. de Brébisson, Y. Bengio, and A. C. Courville. MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[23] H. Ledford. Millions of black people affected by racial bias in health-care algorithms. Nature, 574(7780):688-00, Oct. 2019.
[24] A. S. Luccioni, C. Akiki, M. Mitchell, and Y. Jernite. Stable Bias: Analyzing Societal Repre- sentations in Diffusion Models, Mar. 2023. Stti::2303.114000 [cs].
[25] M. Luo, J. Cao, X. Ma, X. Zhang, and R. He. FA-GAN: Face Augmentation GAN for Deformation-Inwaree Invariant Face Recognition. IEEE Transactions Information Forensics and Security, 012341-23555 2021. Conference Name: IEEE Transactions on Information Forensics and Security.
[26] V. H. Maluleke, Thakkar, Brooks, E. Weber, Darrell, A. A. Efros, A. Kanazawa, and D. Guillory. Studying Bias in GANs through the Lens of Race, Sept. 2022. arXiv:2209.02836 [cs].
[27] M. Mehrab Tanjim, R. Sinha, K. K. Singh, S. Mahadevan, D. Arbour, M. Sinha, and G. W. Cottrell. Generating and Controlling Diversity in Image Search. In 2022 IEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pages 3908-3916, Waikoloa, HI, USA, Jan. 2022. IEEE
[28] Y. Mirsky and W. Lee, The Creation and Detection of Deepfakes: A Survey, ACM Computing Surveys, 54(1):1-41 Jan. 2022.
[29] M. Oreutt. Are Face Recognition Systems Accurate? Depends on Your Race,, 2016.
[30] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, 7. DeVito, Z. Lin, A. Desmaison, I.. Antiga, and A. Lerer. Automatic differentiation in PyTorch. In NIPS 2017 Autodiff Workshop. NIPS 2017 Autodiff Workshop. 2017.
[31] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer. R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-leam: Machine Learning in Python. Journal of Machine Learning Research, 12(85):2825-22
2011.
[32] A. Radford, L. Metz, and S. Chintala. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Jan. 2016. 1rXiv:1511.0644 [es].
[33] H. Rangwani, K. R. Mopuri, and R. V. Babu. Class balancing GAN with a classifier in the loop. In Proceedings of the Seventh Conference Uncertainty in Artificial Intelligence, pages 1618-1627. PMLR, Dec. 2021. ISSN: 2640-3498.
[34] D. Rezende and S. Mohamed. Variational Inference with Normalizing Flows. In Proceedings of the 32nd International Conference on Machine Learning, pages 1530-1538. PMLR, June 2015. ISSN: 1938-7228.
12

---

[35] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-Resolution Image Synthesis with Latent Diffusion Models, Apr. 2022. arXiv:21 12.10522 [cs].
[36] Serengil and A. Ozpinar. HyperExtended LightFace: A Facial Attribute Analysis Frame- work. In 2021 International Conference Engineering and Emerging Technologies (ICEET), pages 1-4, Oct. 2021. ISSN: 2409-2983.
[37] X. Shao, K. Stelzner, and K. Kersting. Right for the Right Latent Factors: Debiasing Generative Models via Disentanglement, Feb. 2022. arXiv:2202.0039) [cs].
[38] Y. Shen, Gu, X. Tang, and B. Zhou. Interpreting the Latent Space of GANs for Semantic Face Editing. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9240-9249, Seattle, WA, USA, June 2020. IEEE.
[39] Y. Shen, Yang, X. Tang, and Zhou. InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs, Oct. 2020. arXiv:2005.0663: [es, eess].
[40] T. Sipka, M. Sulc, and J. Matas. The Hitchhiker's Guide to Prior-Shift Adaptation. In 2022 JEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 2031-2039, Waikoloa, HI, USA, Jan. 2022. IEEE.
[41] Song, C. Meng, and S. Ermon. Denoising Diffusion Implicit Models, Oct. 2022. arXiv:2010.02502 [cs].
[42] S. Tan, Y. Shen, and B. Zhou. Improving the Fairness of Deep Generative Models without Retraining, Mar. 2021. arXiv:2012.0482 [es].
[43] I.. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni. Modeling Tabular data using Conditional GAN. In Advances in Neural Information Processing Systems, volume 32.
Curran Associates, Inc., 2019.
[44] Z. Zhang, Y. Song, and H. Qi. Age Progression/Regression by Conditional Adversarial Autoen- coder. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4352-4360, July 2017. ISSN: 1063-6919.
13

---

A Appendix
A.1 Calculating for The Generated Distribution
The error rates reported for a classifier Co are typically reported on the distribution on the distribution of fit's training data, Ptraining However, the distribution Pc, of the generative model
Gg may differ from the training distribution. Additionally, rather than reporting P(ylý), often times the error rates given in confusion matrix Cyly where Cytyli. = P(yly). Thankfully, construct the error rate matrix E for the generative distribution Pa, under the simplifying assumption that the difference between FG, and I training can be explained label shift [10 40
By Bayes' Theorem, we know that
P(y) P(ylg) = -P(ly) P(g)
Under the label shift assumption, P(fly) stays the same between P. Proiinng and PG.,. Additionally, P(y) can be calculated for Pc, under label shift [T0 40). Lastly, P(y) can be approximated for Fc, by finding the proportion predicted for each class on large sample from the generative model. Thus, E' can be calculated as:
Pa. (y)
E = Cyly Pao (8)
A.2 Distribution of Races Generated By Progressive GAN
We show the two best performing methods' distributions on Progressive GAN, along with the distribution of the unmodified ProgressiveGAN, over the Race attribute.
Distribution Of Races Resulting From Compared Methods
0.8
Distribution
Mapping (Ours)
0.7
Polarity Sampling (Antimode)
0.6
Standard Progressive GAN
Density
0.5
0.4
0.3
0.2
0.1
0.0
asian
black
indian
latino hispanic middle eastem
white
Race
Figure 6: Distribution of our approach, Polarity Antimode Sampling (next best), and the standard
generator.
A.3 Comparison Of Generated Faces From LDM Model
A comparison of a bath of random faces from the original LDM is shown in Figure while Figure shows a batch from the LDM when our Distribution Mapping method is applied.
A.4
Implementation Details
Ground Truth Shape Classifier
Layer (type)
Output Shape
14

---

Figure 7: Random sample of generated faces from the original LDM model (no Distribution Mapping or fail sampling applied).
Figure 8: Random sample of generated faces from the LDM model after our Distribution Mapping approach.
15

---

Conv2d-1
[-1, 32, 16, 16]
ReLU-2
[-1, 32,
16, 16]
Conv2d-3
[-1, 64, 8,
ReLU-4
[-1, 64, 8, 8]
Conv2d-5
[-1, 128, 4, 4]
ReLU-6
[-1, 128, 4, 4]
Conv2d-7
[-1, 256, 2, 2]
ReLU-8
[-1, 256, 2, 2]
Conv2d-9
[-1, 2, 1, 1]
Encoder for Shapes VAE
Layer (type)
Output Shape
Conv2d-1
[-1, 32, 16, 16]
ReLU-2
[-1, 32, 16, 16]
Conv2d-3
[-1, 64, 8, 8]
ReLU-4
[-1, 64, 8, 8]
Conv2d-5
[-1, 128, 4, 4]
ReLU-6
[-1, 128, 4, 4]
Conv2d-7
[-1, 256, 2, 2]
ReLU-8
[-1, 256, 2, 2]
Conv2d-9
[-1, dim, 1, 1]
Decoder for Shapes VAE
Layer (type)
Output Shape
Con/Transpose2d-1
[-1, 256, 2, 2]
ReLU-2
[-1, 256, 2, 2]
Con/Transpose2dd3
[-1, 128, 8, 8]
ReLU-4
[-1, 128, 8, 8]
Con/Transpose2dd-
[-1, 64, 16, 16]
ReLU-6
[-1, 64, 16, 16]
ConvTranspose2d-7
[-1, 32, 32, 32]
ReLU-8
[-1, 32, 32, 32]
ConvTraasposee2d-9
[-1, 3, 64, 64]
Sigmoid-10
[-1, 3, 64, 64]
Biased Age Classifier (Note: Target value was normalized age, made binary after)
Layer (type)
Output Shape
Conv2d-1
[-1, 2, 32, 32]
BatchNorm2d-2
[-1, 2, 32, 32]
LeakyReLU-3
[-1, 2, 32, 32]
Dropout-4
[-1, 2, 32, 32]
Conv2d-5
[-1, 4, 16, 16]
BatchNorm2d-6
[-1, 4, 16, 16]
LeakyReLU-7
[-1, 4, 16, 16]
Dropout-8
[-1, 4, 16, 167
Conv2d-9
[-1, 8, 8, 8]
16

---

Batchhoorm2d110
[-1, 1] [-1, 8, 8, 8]
LeakyReLU-11
[-1, 8, 8, 8]
Dropout-12
[-1, 8, 8, 81
Flatten-13
[-1, 512]
14
[-1, 64]
LeakyReLU-15
[-1, 64]
Linear-16
Sigmoid-17
[-1, 1]
Ground Truth Age Classifier (Note: Target value was normalized age; made binary after)
Layer (type)
Output Shape
Conv2d-1
[-1, 8, 32, 32]
BatchNorm2d-2
[-1, 8, 32, 32]
LeakyReLU-3
[-1, 8, 32, 32]
Dropout-4
[-1, 8, 32, 32]
Conv2d-5
[-1, 16, 16, 16]
BatchNorm2d-6
[-1, 16, 16, 16]
LeakyReLU-7
[-1, 16, 16,
Drepout-8
[-1, 16, 16, 16]
Conv2d-9
[-1, 32, 8, 8]
BatchNorm24-10
[-1, 32, 8, 8]
LeakyReLU-11
[-1, 32, 8, 8]
D1
[-1, 32, 8, 8]
Flatten-13
[-1, 2048]
Linear-14
[-1, 64]
LeakyReLU-15
[-1, 64]
Linear-16
[-1, 1]
Sigmoid-17
[-1, 1]
The distribution mapper used default architecture of SDV's CTGAN version 0.6.0, except for the ProgressiveGAN experiment where embedding_ dim =512, generator dim =(512,512) were passed as arguments.
For the networks we trained, we utilized the Adam optimizer with learning rate between 0,002 and 0.0001.
The linear classifier utilized Scikit-Learn's LinearSVC (for latent editing) and RidgeClassifier for the biased Shapes classifier.
A.5 Proof of Lemma
Proof. First, note that if 1 31l cone EE), then likewise 1 E cono(E).
Let z' c.. i.e., is draw from the distribution of noise such that the classifiers prediction of the generated sample corresponding to z' is group
Let (C" o CP).PP.cC.== be the pushforward distribution of the perfect classifier C"'s output when conditioned on the generator's output of draws from IP P: Then,
(C' Go).PalCu=r [Pr(y = 1|Ca i), Pr(y 2|Ca =)), -i..... ..... .......P(y N|Ca = =i]]
=E,;
Thus,
http://wwwww///oooeee.u///////// wwwdes//iiggeeeeeeeeeee guudbes/ssinggle_..bbbeeegag html# html#
how to--to-mooo aos--oodiy---oo the- ctgan -yprrpaapyperrrrmettterss
17

---

conc(f(C' Ga)),P Gg). conc (E)
Therefor, following from above,
1 y 1110 E cone(((C' o
Go). (1c=1111))))
This means that Iiv A2..... A such that the following holds:
1 1
CC o Go). P.oo A (C' Go). (o0..P./c==. 2|G==|>1]
= UUnf(())
This is equivalent to saying that:
C²())(z))~ Uni
for z ~ E12l ArP P/ = @² Thus, by definition Q^ is a Fair Noise Distribution.
A.6
Proof of Lemma
Proof. Note that the sign of the coefficient of the cross product E.1 x E, is P(y 1
1) = 2 = 2) - P(y 1ly = P(y = 2 = 1). Also note that E.1 x [0.5, 0.5] is 0.5P(y 1 1
2 = 1).
Additionally,
P(y = 1)
Ply 1 1)0.5
>
and
0< Ply = 15 -)))) = 2 0 2ly 1)
Thus, the coefficient of E., x E.2 is greater than E. x [0.5, 0.5), while there signs are equal. This implies that [0.5, 0.5] is in between E,1 and E, Thus, [0.5, cone(E). The rest of the proof follows directly from Lemma 1.
A.7 Proof of Proposition
Proof. Note that PA has density (Σ, A:P(y = 1 = ,,,..,,,AAPry = Nlý = For ease of notation let us refer to E, X₂ Pr(yy m = as r.
Then,
= Σ pnd log m
And m
log(r>)) 1. log(
D
Σ
p log(r>))
Σ r log(
D
18

---

Note that log (N) is constant for each term in the second summation. Thus,
Σ I' log(rm)
- log
N
m
Σ m log(ri) log (
As log (+) does not depend on rm
argmin KL (L) = argmin Σ Ti log(rm))
argmin (
argmax H (Pt)
A.8
Proof of Proposition 2
Proof.
(C' o Go)..P//.0= =(Pr(y = 1/C, = Pr(y = 2/Co = i),..., Pr(y NIC =))
Σ (/CC o Go). Go).Pa|com =1Σ XPr(y = 1/Co = =)) Σ
Prly C = =),
;
EAPr(x = N = i)]
PF = Q
A.9 Proof of Theorem
The first statement follows directly from Proposition and Proposition 2.
If C C" then { E. [E.] ..... E } forms standard basis of R!2* and therefor 1 1 is in cone(E).
E,
Thus, Q** is Fair Noise Distribution by Lemma 1.
19


Reviews:
[{"review_id": "WoYLowr6Fw", "reviewer": "Reviewer_yDHD", "summary": "This paper proposes an approach to mitigate biases in pre-trained generative models. Specifically, the authors aim to generate a semantically fair distribution over the attributes. Authors propose to train a distribution mapper network that effectively learns to generate a fair noise distribution, which, in turn, is used to generate samples that adhere to fairness principles such that the distribution of the generated samples per-group is closer to the uniform distribution. \nAuthors theoretically consider the case where the classifier to train the mapper network is imperfect and propose an approach to handle this case. Furthermore, they study this approach on three tasks, Shapes Dataset, Age Bias and race bias and compare with existing debiasing methods.\n\n\n-----------------------------------------------------------------------------------------------------\nFollowing Neurips guidelines: I acknowledge that I have read the authors response, engaged in the discussions and read other reviewers comments.  Based on the authors rebuttal, I have updated my score accordingly and increased it from 6 to 7.", "questions": "- Since the work is framed in a way that supports general generative models, how do you propose to use this approach for VAEs or diffusion models?\n- How do you handle the stochasticity that comes from the noise in the GPU training? Do you need the training process of the GAN to be deterministic?", "limitations": "authors have already addressed the limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses an important problem, debiasing generative models output.\n- The paper proposes an approach that does not require training or fine-tuning a generative model which is computationally efficient.\n- The paper proposes to use a pre-trained classifier that is able to classify groups and they also propose an algorithm that handles imperfect classifiers. \n- In the experiments, the paper compares with several other existing methods for debiasing the generative models and in all the reported cases, the method out-performs existing methods.", "weaknesses": "- The authors frame this work around the general generative models, however, in the experiments, only GANs are studied. I believe the study of diffusion models or other types of generative models is important. Specifically, diffusion models generate samples stochastically, and changing the initial noise distribution may not be as trivial as GANs. Furthermore, diffusion models have a strict assumption on the starting noise to be from the uniform distribution. Thus, changing the initial noise distribution is not trivial. I believe the paper should be framed around GANs since it is the only model that is studied empirically and is trivial to extend given the setup.\n- The authors mention that pre-trained classifiers that classify specific groups/attributes is easily accessible. However, in many cases that may not be the case (likely the issue with the experiment on progressive GANs).\n- The experiments are minimal and limited to GANs. It would have been intersting to see experiments related to other types of generative models or extend other types of more large-scale GANs or larger scale datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your thoughtful feedback. \n\n**“How do you handle stochasticity that comes from the noise during training? Do you need the training process of the GAN to be deterministic?”**\n\nWe do not require the training of the generative model to be deterministic. We only require that, once trained, the generative model produces a very similar output when conditioned on the same latent code instance. This behavior is satisfied by standard generative models like GANs, VAEs, normalizing flows, and diffusion models if a deterministic reverse process sampler is used.  \n\n**“Can this approach work on Diffusion models?”**\n\nWe note that indeed the focus of our research when formalizing our approach has been on generative models such as GAN-type architectures where the dimensionality of the latent space is typically much lower than the data space. This is not the case for Diffusion models. However, conceptually, our approach could still be applied to Diffusion models that are paired with a deterministic sampler (such as DDIM). As a proof of concept, we have conducted a new experiment debiasing a Latent Diffusion Model [1] that was trained on Celeba-HQ-256. In this new experiment, we make the “age” attribute of the generated faces more uniform. Our method succeeds to generate faces with a more fair distribution over ages. Please see Table 1 in the Rebuttal PDF attached to the general rebuttal post.\n\n**“Don’t Diffusion models require a specific noise distribution?”**\n\nWhile diffusion models typically result in a specific distribution for the forward process during training (e.g., Gaussian noise), during inference when performing the reverse process a modified distribution can be sampled from.\n\nConsider the case where we draw a sample $z$ from the normal noise distribution, and use this to generate an output image $x$ using a DDIM sampler for a pretrained frozen Diffusion model. Let us then define a dirac delta distribution on $z$, and then sample from this delta distribution instead of from the original noise distribution. Each of these samples, once run through the diffusion model, will likewise generate the same $x$. Thus, even with a modified noise distribution, the output should still be in the target manifold - though the distribution of the outputs will change. However, changing the output distribution is exactly what we are trying to achieve. \n \n**“Why were experiments only performed on GANs?”**\n\nWe believe that there is a misunderstanding here. Namely,  the generative model we evaluate in Section 4.2 is a Variational Autoencoder (see line 247 in our paper). As this misconception was shared by more than one reviewer, we now realize that this point of our work with the VAE model was not highlighted properly in our original manuscript. In the camera ready version, we will thus include a paragraph in the experimental setup (Section 4.1) specifically laying out which generative models we evaluate. Additionally, as mentioned earlier in the rebuttal, we have now also conducted a new experiment on a Diffusion model. \n\n**“How realistic is it to assume that an attribute classifier is available?”**\n\nWhile there may be some scenarios where publicly available classifiers do not exist for a specific attribute of interest to a given application, we have found that there are a good number of scenarios where there are such classifiers. Thus, while our method will not apply to every scenario, there are many cases where it will. For instance, there are a good number of face attribute classifiers available that support the debiasing images of people, as discussed in our manuscript.\n\nAdditionally, as training a classifier is typically easier than training a generative model, one worst-case option would be to train a new classifier on the attribute of interest and use that to debias the pretrained generative model. \n\n**“Since the work is framed in a way that supports general generative models, how do you propose to use this approach for VAEs or diffusion models?”**\n\nWe agree that indeed our primary focus was on solving this problem for GAN-like models that map from a low-dimensional latent space to a high-dimensional data space. We will rephrase our claim about generality to indicate that while the approach may be applicable to generative models that have higher dimensional noise distributions, we  are most focused on this type of model that takes in lower dimensional noise. \n\nHowever, given a latent code, the decoder of a VAE works analogously to the generator of a GAN. It is for this reason that we had experimented with VAE models in this manuscript (Section 4.2),  as explained above.\n\nFor a diffusion model, we would transform the initial latent distribution (i.e., the distribution of X_T), and then use a deterministic sampler such as DDIM for the reverse process. \n\n[1] Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models” CVPR 2022. \n"}, {"review_id": "rWDOl94iqz", "reviewer": "Reviewer_CzLf", "summary": "The paper proposes a method for training a Distribution Mapper that can generate a noise distribution capable of creating Fair Samples using an Imperfect Classifier. They conducted experiments on toy examples and several Face Generation models, demonstrating quantitatively superior performance compared to competitors.", "questions": "Does the verification of debiasing using metrics such as CLIP Score yield similar results?", "limitations": "Please refer to the Weakness part. I have a positive view of the content and approach presented in this paper. It would be beneficial to explore different models for M, not just limited to GANs, and investigate their performance. Additionally, conducting experiments on more recent models trained on datasets like CelebA and FFHQ would be valuable. It would also be interesting to observe the tendencies of commonly used models such as StyleGAN2, DDPM, LDM, VAE, and others. Including these aspects would enhance the quality of the paper and make it even more comprehensive.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper proposes an effective approach to achieve debiasing without the need for additional training of generative models or classifiers by ingeniously devising a method to train $M$.", "weaknesses": "Although it is mentioned in Line 114 that $M$ could be GAN, VAE, DDPM, normalizing flow, etc., there are no additional experiments conducted to explore this further. There is a lack of experimentation to examine how the noise shifted by $M$ affects image quality. It would be beneficial to report FID scores for CelebA as well. While $G$ is defined as a generative model, there are only experiments conducted with GAN. Particularly for Diffusion models, it is doubtful whether $M$ can be effectively trained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the time you put into reviewing our work, and for the constructive feedback you provided. \n\n**It would be beneficial to report FID scores for CelebA as well.**\n\nWe computed the FID score for the Progressive GAN before and after applying our method, and found that the increase in FID was not extreme; it increased around 5 points. Since the distribution of CelebA is biased towards images of white people, it is not surprising that a debiased distribution would have an increased FID score. \n\n**There are only experiments conducted with GAN.**\n\nThere is a misconception that we experimented with GANs only –  which must mean that we had not adequately highlighted that the generative model we worked with in the experiments in Section 4.2 was a VAE model. We will more explicitly state the generative models studied in the camera ready version. \n\n**Could this work for Diffusion?**\n\nOur target when designing our approach was focused on generative models with a latent space that is of lower dimensionality than the data space – which matches GANs that indeed were our key target.  With Diffusion models having a latent space in dimensionality equal to that of the data space, we agree that training the mapper will be more difficult. \n\nHowever, it is still conceptually possible. As a proof of concept, we have added an experiment on a latent diffusion model [1], where the diffusion model acts on 3x64x64 latent space - which is much larger than those of the GANs and VAE we have studied in our original manuscript. Our results show that our method indeed yields a distribution over ages that is closer to uniform than those produced by any of the other compared approaches; please see Table 1 in the Rebuttal pdf attached to the general Rebuttal post. \n\n**Conducting an experiment on more recent models would be valuable.**\n\nBased on your suggestion, we conducted a new experiment described above on the latent diffusion model,  a recent model. We will add these additional results into the final manuscript. \n\n**Additional experiments on using methods other than GANs for the mapper M would be valuable.**\n\nTo show that the mapper could be some other generative model besides a GAN, we use a DDIM diffusion model as the mapper for the above latent diffusion experiment. We use a mapper with less than half as many parameters as the U-net of the pretrained latent diffusion model - and thus training the mapper is much cheaper than retraining the generative model. \n\n[1] Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models” CVPR 2022. \n"}, {"review_id": "nWtFNVQiGV", "reviewer": "Reviewer_K9u1", "summary": "This paper introduces a method for mitigating biases in a pre-trained generator, without the need to retrain the generator or rely on real data. The authors propose the use of a classifier to create a fair distribution of noise vectors, which are then sampled to generate an equal number of instances for each group. Additionally, the paper presents techniques for constructing a fair noise distribution in scenarios where the classifier is either perfect or imperfect.", "questions": "The paper lacks a discussion and comparison with relevant baselines, such as [1] and [2], which address the same issue of debiasing generative models without requiring retraining. Given that the authors have included results for Progressive GAN, it would be intriguing to see a comparison between the proposed approach and the aforementioned papers. This would provide valuable insights into the effectiveness of the proposed method in relation to existing techniques.\n\n[1] Karakas, C., Dirik, A., Yalçınkaya, E., & Yanardag, P. (2022). FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations. European Conference on Computer Vision.\n\n[2] Tan, S., Shen, Y., & Zhou, B. (2020). Improving the Fairness of Deep Generative Models without Retraining. arXiv Preprint arXiv:2012. 04842.", "limitations": "The authors have addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper's motivation is compelling, considering the widespread adoption of Generative AI across various applications. The problem addressed in the paper is of utmost importance in this context.\n2. The authors provide comprehensive results obtained from experiments conducted on three relevant datasets: Shapes, UTKFace, and CelebA-HQ datasets.\n3. The paper is well-structured and easy to follow.", "weaknesses": "The proposed approach demonstrates a degradation in performance in the case of Progressive GAN applied to the CelebAHQ dataset. The paper includes a discussion that attributes this decline to the observation that a majority of the transformed latent space regions are predominantly associated with the over-represented semantic attribute. This limitation hampers the effectiveness of the proposed approach when dealing with complex datasets. I believe that this problem arises due to the fact that the approach does not involve tweaking the generator function (no retraining involved) but only a transformation of the latent space of the generator.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. \n\n**The paper lacks a discussion and comparison with baselines, such as [1] and [2], which address debiasing generative models without requiring retraining.**\n\nAs you have suggested, we have conducted new experiments to compare against the method proposed in [2]; please see Figure 1 in the Rebuttal document attached to the general rebuttal post for results. Our results confirm that our proposed method continues to be the best performer.\n\nAs  the method proposed in [1] is designed only for StyleGAN methods - e.g, it requires concepts like style channels and style vectors that are not generally present in other generative models - it is not directly applicable to the models we study.   But, based on your suggestion, we will add a detailed discussion around [1] and its relationship to our work into our related work section. \n"}, {"review_id": "s0w0CJujLv", "reviewer": "Reviewer_h8Jf", "summary": "This paper studies how to learn a fair latent distribution so that a pretrained generator outputs samples that have equal likelihood to fall into different attributes. The paper assumes access to an imperfect attribute classifier. The proposed method learns the fair latent distribution by maximizing the entropy of $P_E^{\\lambda}$ and then use a generative model to distill the fair distribution. It is guaranteed that the optimal solution leads to a perfectly fair or minimally unfair distribution. The paper conducts experiments on several image datasets and shows that the proposed method leads to the best fairness in terms of shapes, ages, and races. ", "questions": "Please refer to the weakness comments. ", "limitations": "The paper discussed limitations in the last section. ", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is extremely well written and organized. It is quite easy to follow and understand the difficulties of the defined task and the motivation behind the solution. \n\nIt also tries to solve an important problem in fairness research as modern deep generative models are very powerful but the fairness issues are not well understood. \n\nThe proposed method is simple but theoretically very sound. The paper clearly states conditions when the optimal fair distribution may be obtained and when the best possible one is obtained. \n\nThe experiments are also extensive. It includes several different sets of attributes and the results look very good to me.", "weaknesses": "The experiments do not demonstrate the approximation error or tradeoff of each component of the proposed method. For example, how well does the network learned in section 3.2 approximate $Q^{\\lambda}$? How faraway is the learned $P_E^{\\lambda}$ from the optimal one? Some visualization or quantitative study for this can help us understand how we may be able to further improve fairness. \n\nMinor points: I also recommend the authors to fix a few typos in the math equations and replace the italic words with \\mathrm to make them look nicer. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback; and are happy to hear of your favorable view of our work. \n\n**How far off are the learned weights of the mixture of class probabilities from optimal, and how close does the output of the Mapper match the target Minimally-Unfair Noise Distribution?**\n\nWe have added another set of experiments to explore your proposed question; please see Table 2 in the rebuttal PDF attached to the general rebuttal post for the new experimental results. In short, the learned weights are very close to optimal.\n\n**Typos and formatting suggestions**\n\nThank you for the suggestion. We will use \\mathrm, and do a thorough pass for typos for the camera ready version. "}]

Author rebuttal:
## General Rebuttal

We thank the reviewers for the time they spent providing valuable feedback on our work. We have added additional experiments (see attached PDF) based on the reviewers’ suggestions. 

In short, we:

-  Performed a new experiment, where we apply our approach to a Latent Diffusion Model (Table 1 in attached PDF), as suggested by Reviewer yDHD and Reviewer CzLf.

- We add in a comparison against another related work (Tan et al.) based on the feedback from Reviewer K9u1 (Figure 1).

- We report metrics on how well our method matches its target distribution (Table 2) as requested by Reviewer h8Jf.

Additionally, we want to clarify a common misconception among reviewers: namely, that we only performed experiments on GAN models in the original manuscript. We actually did apply our method to a VAE in the original manuscript (see section 4.2, line 247). We realize that we did not adequately highlight this in our original text; we will explicitly list each generative model, including the new latent diffusion model we added for this rebuttal, in the camera ready version. 


Meta-review: This paper was reviewed by four knowledgeable referees. The reviewers raised concerns w.r.t. the experiments presented in the paper, which initially appeared limited (yDHD, CzLf, K9u1); the requirement to have access to a pretrained classifier that classifies groups/attributes (yDHD); and the effect of shifting the noise on the resulting image quality (CzLf). The reviewers also raised questions w.r.t. the trade off of each component (h8Jf). The rebuttal adequately addressed the reviewers' concerns by highlighting results of both GANs and VAEs in the original paper, including additional baselines, and presenting results on diffusion models. After rebuttal and discussion, the reviewers unanimously lean towards acceptance. The AC finds this an interesting and timely contribution and agrees with the reviewers' assessment. Therefore, the AC recommends to accept.
Decision: Accept (poster)