The AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI (R2 HCAI)

Through a Fair Looking-Glass: Mitigating Bias in Image Datasets
Amirarsalan Rajabi1
1

Mehdi Yazdani-Jahromi1

Ozlem Ozmen Garibay1,2

Gita Sukthankar1

Department of Computer Science, University of Central Florida 2 Department of Industrial Engineering and Management
Systems, University of Central Florida{amirarsalan,yazdani}@knights.ucf.edu, ozlem@ucf.edu, gitars@eecs.ucf.edu

Abstract2011). The problems arising from bias in computer vision
can manifest in different ways. For instance, it is observedthat in activity recognition models, when the datasets contain gender bias, the bias is further amplified by the models
trained on those datasets (Zhao et al. 2017). Face recognition models may exhibit lower accuracy for some classes ofrace or gender (Buolamwini and Gebru 2018).
This paper addresses the issue of a decision-making process being dependent on protected attributes, where this dependence should ideally be avoided. From a legal perspective, a protected attribute is an attribute upon which discrimination is illegal (Pessach and Shmueli 2020), e.g. gender orrace. Let D = (X , S, Y) be a dataset, where X represents
unprotected attributes, S is the protected attribute, and Y be
the target attribute. If in the dataset D, the target attribute is
not independent of the protected attribute (Y ̸⊥ S), then it isvery likely that the decisions Ŷ made by a decision-making
system which is trained on D, is also not independent of theprotected attribute (Ŷ ̸⊥ S).
We propose a model to reconstruct an image dataset to
reduce statistical dependency between a protected attributeand target attribute. We modify a U-net (Ronneberger, Fischer, and Brox 2015) to reconstruct the image dataset andapply the Hilbert-Schmidt norm of the cross-covariance operator (Gretton et al. 2005a) between reproducing kernel
Hilbert spaces of the target attribute and the protected attribute, as a measure of statistical dependence. Unlike many
previous algorithms, our proposed method doesn’t require
training new classifiers on the unbiased data, but instead reconstructing images in a way that reduces the bias entailed
by using the same classifiers.
In Section Methodology we present the problem, the notion of independence, and our proposed methodology. In
Section Experiments we describe the CelebA dataset andthe choice of feature categorization, introduce the baselinemodel with which we compare our results (Ramaswamy,
Kim, and Russakovsky 2021), our model’s implementation
details, and finally present the experiments and results.

With the recent growth in computer vision applications,the question of how fair and unbiased they are has yet
to be explored. There is abundant evidence that thebias present in training data is reflected in the models, or even amplified. Many previous methods for image dataset de-biasing, including models based on augmenting datasets, are computationally expensive to implement. In this study, we present a fast and effectivemodel to de-bias an image dataset through reconstruction and minimizing the statistical dependence between
intended variables. Our architecture includes a U-net toreconstruct images, combined with a pre-trained classifier which penalizes the statistical dependence between
target attribute and the protected attribute. We evaluate our proposed model on CelebA dataset, compare
the results with two state-of-the-art de-biasing method,and show that the model achieves a promising fairnessaccuracy combination.

Introduction
Due to their increased usage within myriad software applications, artificial intelligence algorithms now influence many
aspects of people’s lives, particularly when they are embedded into decision-support tools used by educators, government agencies, and various industry sectors. Thus, it is crucial to make sure that these algorithms are scrutinized to ensure fairness and remove unjust biases. Bias has been shown
to exist in several deployed AI systems, including the wellknown Correlational Offender Management Profiling for Alternative Sanctions (COMPAS). COMPAS is an automateddecision making system used by the US criminal justice system for assessing a criminal defendant’s likelihood of reoffending. By exploring the risk scores assigned to individuals, this system has been shown to be biased against African
Americans (Chouldechova 2017). Other examples include
a version of Google’s targeted advertising system in whichhighly paid jobs were advertised more frequently to men vs.
women (Lambrecht and Tucker 2019).
Bias in computer vision is a major problem, often stemming from the training datasets used for computer visionmodels (Tommasi et al. 2017). There is evidence suggesting
the existence of multiple types of bias, including capture andselection bias, in popular image datasets (Torralba and Efros

Background
Bias mitigation methods can be divided into three generalcategories of pre-process, in-process, and post-process. Preprocess methods include modifying the training dataset be1

Our objective is to design a transformation g : X → Xe,such that 1) f (Xe) ⊥ S, i.e. the classifier’s predictions fortarget attribute is independent of the protected attribute , and2) f (Xe) ≈ f (X ), i.e. the classifier still achieves high accuracy.
In other words we want to train a network to transform ouroriginal images, such that if the classifiers that are trainedon the original and unmodified images, are used to predict the target attribute (attractiveness in our example) fromthe transformed version of an image, they still achieve highaccuracy, while the predictions of those classifiers are independent of the protected attribute (gender in our example). It should be noted that we are not seeking to train
new classifiers, but rather only aim to modify the input images. This is a main distinction between our methodology
and most of other techniques (e.g. (Quadrianto, Sharmanska, and Thomas 2019) and (Ramaswamy, Kim, and Russakovsky 2021)), in which the process includes training newclassifiers on modified new image datasets and achieving
fair classifiers.
Our proposed model consists of a U-net (Ronneberger,
Fischer, and Brox 2015) as the neural network that transforms the original images. This type of network was originally proposed for medical image segmentation, and has
been widely used since its introduction. The encoderdecoder network consists of two paths, a contracting pathconsisting of convolution and max pooling layers, and a consecutive expansive path consisting of upsampling of the feature map and convolutions. Contrary to (Ronneberger, Fischer, and Brox 2015) where each image is provided with asegmented image label, we provide our U-net with the exact same image as the label, and alter the loss function fromcross-entropy to mean squared error, so that the network getstrained to produce an image as close to the original image as
possible, in a pixel-wise manner.
While some previous fairness studies consider decorrelating the target attribute from the protected attributes, whatmust be ultimately sought however, is independence between the protected attribute and the target attribute. Dealing
with two random variables which are uncorrelated is easier than independence, as two random variables might havea zero correlation, and still be dependent (e.g. two randomvariables A and B with recordings A = [−2, −1, 0, 1, 2] andB = [4, 1, 0, 1, 4] have zero covariance, but are apparently
not independent). Given a Borel probability distribution Pab
defined on a domain A × B, and respective marginal distributions Pa and Pb on A and B, independence of a and b(a b) is equal to Pab factorizing as Pa and Pb . Furthermore, two random variables a and b are independent, if andonly if any bounded continuous function of the two randomvariables are uncorrelated (Gretton et al. 2005b).
Let F and G denote all real-value functions defined on
domains A and B respectively. In their paper (Gretton
et al. 2005a) define the Hilbert-Schmidt norm of the crosscovariance operator:|=

fore feeding it to the machine learning model. In-process
methods include adding regularizing terms to penalize some
representation of bias during the training process. Finally,
post-process methods include modifying the final decisionsof the classifiers (Hardt, Price, and Srebro 2016). Kamiranand Calders (Kamiran and Calders 2012) propose methods such as suppression which includes removing attributes
highly correlated with the protected attribute, reweighing,i.e. assigning weights to different instances in the data, andmassaging the data to change labels of some objects. Bias
mitigation methods often come at the expense of losing
some accuracy, and these preliminary methods usually entail
higher fairness-utility cost. More sophisticated methods with
better results include using generative models to augment
the biased training dataset with unbiased data (Ramaswamy,
Kim, and Russakovsky 2021), or training the models on entirely synthetic unbiased data (Rajabi and Garibay 2021).
(Wang et al. 2020) provide a set of analyses and a benchmark to evaluate and compare bias mitigation techniques invisual recognition models.
Works such as (Wang, Narayanan, and Russakovsky
2020; Yang et al. 2020) suggest methods to mitigate bias
in visual datasets. Several studies have deployed GANs forbias mitigation in image datasets. For example, (Sattigeri
et al. 2019) modified the value function of GAN to generate fair image datasets. FairFaceGAN (Hwang et al. 2020)
implements a facial image-to-image translation, preventing
unwanted translation in protected attributes. Ramaswamy etal. propose a model to produce training data that is balanced
for each protected attribute, by perturbing the latent vector of a GAN (Ramaswamy, Kim, and Russakovsky 2021).
Other studies employing GANs for fair data generation include (Choi et al. 2020; Sharmanska et al. 2020).
A variety of techniques beyond GANs have been applied
to the problems of fairness in AI. A deep information maximization adaptation network was used to reduce racial bias
in face image datasets (Wang et al. 2019a), and reinforcement learning was used to learn a race-balanced networkin (Wang and Deng 2019). Wang et al. propose a generative few-shot cross-domain adaptation algorithm to perform
fair cross-domain adaption and improve performance on minority category (Wang et al. 2021). The work in (Xu et al.
2021) proposes adding a penalty term into the softmax loss
function to mitigate bias and improve fairness performancein face recognition. (Quadrianto, Sharmanska, and Thomas2019) propose a method to discover fair representations ofdata with the same semantic meaning of the input data. Adversarial learning has also successfully been deployed forthis task (Zhang, Lemoine, and Mitchell 2018; Wang et al.
2019b).

Methodology
Consider a dataset D = (X , S, Y), where X is the set of images, Y = {+1, −1} is the target attribute such as attractiveness, and S = {A, B, C, ...} is the protected attribute suchas gender. Assume there exists a classifier f : (X ) → Y,such that the classifier’s prediction for target attribute is notindependent from the protected attribute, i.e. f (X ) ̸⊥ S.

HSIC(Pab , F, G) := ||Cab ||2HS(1)

where Cab is the cross-covariance operator. They show that
if ||Cab ||2HS is zero, then cov(f, g) will be zero for any
2minimize
Uw

N X
X
1
enijk )2
(xnijk − x
N CW H n=1
i,j,k
{z
}
|
image accuracy(3)

e h2 (X))
e+ λ × HSIC(h1 (X),|{z}
independence

where N is the number of samples, C is the number ofchannels of an image, W is the width of an image, H isthe height of an image, and λ is the parameter that controls the trade-off between accuracy of the transformed images and independence (fairness). In practice, the mapping
function Uw that we use is a U-net, the function h(·) is a
pre-trained classifier with two outputs h1 and h2 , each being the output of a Sigmoid function within the range of[0, 1], where h1 = P(Y = 1|X) (a vector of size N ), andh2 = P(S = 1|X) (also a vector of size N ), and HSIC(·, ·)denotes Hilbert-Schmidt Independence Criteria.
Figure 1 shows the network architecture and a schematicof the training procedure. Consider a batch of original images X entering the U-net. The U-net then produces the ree To calculate the image acconstructed images Uw (X) = X.
curacy part of the loss function, the original image batch X
is provided as label and the Mean Squared Error is calculated
to measure the accuracy of the reconstructed images. The
ResNet component in Figure 1 is our h(·) function as described before, which is a pre-trained ResNet classifier thattakes as input a batch of images and returns two probability
vectors. The second part of the loss function, independence,e intois calculated by entering the reconstructed images X
this ResNet classifier, and calculating the HSIC between thetwo vectors.
As noted before, the image dataset is reconstructed in a
way that using them on the original biased classifiers, will
result in an improvement in classifications. This is dissimilar to some previous works such as (Ramaswamy, Kim,and Russakovsky 2021) and (Quadrianto, Sharmanska, andThomas 2019), in which the model training process includes
augmenting the original dataset with generated images andtraining new fair classifiers (Ramaswamy, Kim, and Russakovsky 2021), or discovering fair representations of images and subsequently training new classifiers (Quadrianto,
Sharmanska, and Thomas 2019).

Figure 1: Our model consists of an encoder-decoder (U-net)
and a double-output pre-trained ResNet classifier. First, theoutput batch of the U-net (reconstructed images) is compared with the original batch of images by calculating MSE
loss. Then, the output batch of the U-net passes through the
ResNet and statistical dependency of the two vectors is calculated by HSIC. Detailed architecture of the U-net is described in the supplementary material.

f ∈ F and g ∈ G, and therefore the random variables
a and b will be independent. Furthermore, they show if
Z := (a1 , b1 ), ..., (an , bn ) ∈ A×B are a series of n independent observations drawn from Pab , then a (biased) estimatorof HSIC is (Gretton et al. 2005a):
HSIC(Z, F, G) := (n − 1)−2 tr(KHLH)(2)

where H, K, L ∈ Rn×n , K and L are Gram matrices (Hornand Johnson 2012), Kij := k(ai , aj ), Lij := l(bi , bj ), k
and l are universal kernels, and Hij := δij − n−1 centers
the observations in feature space. We use Hilbert-Schmidt
independence criteria to penalize the model for dependence
between the target attribute and the protected attribute.

Training Loss Function
We seek to modify a set of images, such that 1) the produced images are close to the original images, and 2) the
predicted target attribute is independent from the predicted
protected attribute. In the optimization problem, image quality (1) is measured by pixel-wise MSE loss. For independence (2), consider our U-net network as a mapping frome.
original image to the transformed image, i.e. Uw (x) = x
Consider also a function h : X → [0, 1] × [0, 1], whereh(xi ) = (h1 (xi ), h2 (xi )) = (P(yi = 1|xi ), P(si = 1|xi )).
Our objective is to train the parameters of Uw such that
h1 (Uw (x)) h2 (Uw (x)), i.e. h1 (Uw (x)) is independent of
h2 (Uw (x)) .
e
Given X representing a batch of N training images and X
representing the transformed batch, our formal optimization
problem is as follows:

Experiments|=

In this section, we test the methodology described in Section Methodology on CelebA dataset (Liu et al. 2015). We
first introduce the CelebA dataset and the attribute categories
in CelebA. We then describe the implementation details of
our model. Subsequently, the method described in the workof (Ramaswamy, Kim, and Russakovsky 2021) and the two
versions of it that we use as baseline models to compare ourresults with are introduced. Finally, we introduce evaluation
metrics and present the results.
3

Figure 2: Examples of CelebA dataset original images. Images in the first row are labeled not Male and images in the second
row are labeled Male. In each row, the first three images are labeled Attractive and the last three images are labeled not
Attractive.

CelebA dataset

gender (Ramaswamy, Kim, and Russakovsky 2021). These
include attributes such as Young, Arched Eyebrowsand Receding Hairline. Finally, the last group of attributes are called gender-independent. These attributes are
fairly consistently labeled and are not much dependent on
gender expression. This group includes attributes such as
Black Hair, Bangs, and Wearing Hat. The list of
all attributes is provided in supplementary material.

CelebA is a popular dataset that is widely used for training
and testing models for face detection, particularly recognising facial attributes. It consists of 202,599 face images ofcelebrities, with 10,177 identities. Each image is annotatedwith 40 different binary attributes describing the image,including attributes such as Black Hair, Pale Skin,
Wavy Hair, Oval Face, Pointy Nose, and other attributes such as Male, Attractive, Smiling, etc. The
CelebA dataset is reported to be biased (Zhang, Wang, andZhu 2018). In this experiment, we consider Male attributeas the protected attribute (with Male = 0 showing the image does not belong to a man and Male = 1 showing theimage belongs to a man), and Attractive to be the target
attribute. We divide the dataset into train and test sets, withtrain set containing 182,599 and test set containing 20,000
images. In the training set, 67.91% of images with Male =0 are annotated to be attractive (Attractive = 1), while
only 27.93% of images with Male = 1 are annotated as being attractive (Attractive = 1). This shows bias exists
against images with Male = 1.
In order to compare our results with (Ramaswamy, Kim,and Russakovsky 2021), we follow their categorization of
CelebA attributes. Leaving out Male as the protected attribute, among the rest 39 attributes in CelebA dataset, (Ramaswamy, Kim, and Russakovsky 2021) eliminates some
attributes such as Blurry and Bald as they contain lessthan 5% positive images. The remaining 26 attributes issubsequently categorized into three groups. inconsistentlylabeled attributes are the ones that by visually examining
sets of examples, the authors often disagree with the labeling and could not distinguish between positive and negative examples (Ramaswamy, Kim, and Russakovsky 2021).
This group includes attributes such as Straight Hair,and Big Hair. The second group of attributes are the ones
that are called gender-dependent and the images are labeled to have (or not have) attributes based on the perceived

Attribute classifiers
For attribute classifiers, we use ResNet-18 pre-trained on
ImageNet, in which the last layer is replaced with a layerof size one, along with a Sigmoid activation for binary classification. We train all models for 5 epochs with batch sizesof 128. We use the Stochastic Gradient Descent optimizer
with a learning rate of 1e-3 and momentum of 0.9. We use
a step learning rate decay with step size of 7 and factor of
0.1. After training, we will have 26 classifiers that receive an
image and perform a binary classification on their respective
attribute.

Implementation details
As shown in Figure 1, a ResNet-18 network is used to accompany the U-net to produce predictions for Male andAttractive. Prior to training the U-net, the ResNet-18(Russakovsky et al. 2015) which is pre-trained on ImageNet,
is modified by replacing its output layer with a layer of size
two, outputing the probability of attractiveness and gender.
The ResNet-18 is then trained for 5 epochs on the train set,with a batch size of 128. We use the Stochastic Gradient
Descent optimizer with a learning rate of 1e-3 and momentum of 0.9. We use a step learning rate decay with step sizeof 7 and factor of 0.1. After the ResNet is trained and prepared, we train the U-net as described in Section Methodology on the train set. The detailed architecture of the U-net
is described in Supplementary Material. In our implementation of biased estimator of HSIC estimator in Equation 2,
4

Figure 3: Examples of CelebA dataset images and how the model reconstructs them. The first row shows a set of images fromthe original testing set, and the second row shows the reconstructed images.
we use Gaussian RBF kernel function for k(·, ·) and l(·, ·).
The training was conducted on a machine with two NVIDIA
GeForce RTX 3090, and each training of the U-Net took 1
hour. When the training is complete, the U-net is ready toreconstruct images. Figure 3 shows six examples of how the
U-net modifies the original images. We train our model for
5 epochs with an λ = 0.07.

ate Xsyn (160,000 images) to generate a synthetic dataset
which aims to de-bias Male from all 26 attributes one byone. Next, we train ResNet-18 classifiers on the new datasets
consisting of augmenting X and Xsyn . We call this modelas GANDeb. We use the implementation of (Alvi, Zisserman, and P
Nellåker 2018) with the uniform confusion loss−(1/|D|) d log qd provided in (Wang et al. 2020).

Comparison with baseline models

Evaluation metrics

We compare our results with Ramaswamy et al.’s method,
described in their paper ‘Fair Attribute Classification
through Latent Space De-biasing’ (Ramaswamy, Kim, andRussakovsky 2021). Building on work by (Denton et al.
2019) which demonstrates a method to learn interpretable
image modification directions, they develop an improved
method by perturbing latent vector of a GAN, to produce
training data that is balanced for each protected attribute.
By augmenting the original dataset with the generated data,
they train target classifiers on the augmented dataset, andshow that these classifiers will be fair, with high accuracy.
The second model that we compare our results with is explicit removal of biases from neural network embeddings,
presented in (Alvi, Zisserman, and Nellåker 2018). The authors provide an algorithm to remove multiple sources ofvariation from the feature representation of a network. Thisis achieved by including secondary branches in a neural network with the aim to minimize a confusion loss, which inturn seeks to change the feature representation of data suchthat it becomes invariant to the spurious variations that aredesired to be removed.
We implement Ramaswamy et al.’s method as follows: As
mentioned in their paper, we used progressive GAN with
512-D latent space trained on the CelebA training set fromthe PyTorch GAN Zoo. We use 10,000 synthetic images andlabel the synthetic images with a ResNet-18 (modified byadding a fully connected layer with 1,000 neurons). Then
we trained a linear SVM to learn the hyper-planes in thelatent space as proposed in the original paper. We gener-

In evaluating the results of our model with the baseline models, three metrics are used. To capture the accuracy of theclassifiers, we measure the average precision. This metriccombines precision and recall at every position and computes the average. A higher average precision (AP) is desired. To measure fairness, there are multiple metrics proposed in the literature (Mehrabi et al. 2021). Among themost commonly used metrics is demographic parity (DP).
This metric captures the disparity of receiving a positive decision among different protected groups (|P (Ŷ = 1|S =0) − P (Ŷ = 1|S = 1)|). A smaller DP shows a fairer classification and is desired. Finally for our last fairness measure, we follow (Lokhande et al. 2020) and (Ramaswamy,
Kim, and Russakovsky 2021) and use difference in equality of opportunity (DEO), i.e. the absolute difference between the true positive rates for both gender expressions(|T P R(S = 0) − T P R(S = 1)|). A smaller DEO is desired.

Results
All the values reported in this section, are evaluated on thesame test set. Prior to comparing the results of our methodwith the comparison models, to assess the original training data, the performance of baseline, i.e. classifiers being
trained on the original train set, and tested on the test set
is presented. The AP, DP, and DEO values of classifierstrained on the original training set is shown in Table 1 under
Baseline. Looking into Baseline values, the AP of classifiers for gender-independent category of attributes is higher
5

by HSIC, between each attribute with Attractive attribute, in the original training data. The results show that theabsolute change in demographic parity is positively correlated with that attribute’s statistical dependence with the attribute Attractive, with a Pearson correlation coefficientof 0.757. For instance, we observe large changes in demographic parity for attributes such as Young, Big Nose,
Pointy Nose, Oval Face, and Arched Eyebrows,
as they are typically associated with being attractive, andtherefore reflected in the CelebA dataset labels.

than gender-dependent category, and the AP of inconsistentcategory is less than the other two categories. As expected,
DP and DEO for gender-dependent category of attributes ishigher than the other two categories.
In Table 1, we compare our model with GAN Debiasing(GanDeb) (Ramaswamy, Kim, and Russakovsky 2021), Adversarial debiasing (AdvDb) presented in (Alvi, Zisserman,
and Nellåker 2018), and the Baseline on the original data.
Looking into the average precision scores, the results showthat GanDeb is slightly performing better than Ours. This isanticipated, since half of the training data for GanDeb consists of the original images, and therefore a higher average
precision is expected. AdvDb on the other hand is performing poorly in terms of average precision, with average precision scores far away from other models.
Looking into demographic parity scores, the results showthat GanDeb falls behind the other two models in two outof three attribute categories. While Ours is performing better for gender dependent and gender independent attributecategories. Looking into the third fairness measure, difference in equality of opportunity, AdvDb and ours are performing better than GanDeb in all three categories of attributes. Ours beats AdvDb for inconsistent attributes category, AdvDb beats Ours in gender dependent category, andAdvDb slightly beats Ours for gender independent category of attributes. In summary, Ours is close to GanDeb interms of maintaining high average precision scores, whichmeans higher accuracy of prediction, while beating GanDeb in terms of fairness metrics. Also, while AdvDb performance in terms of fairness enforcement is better than oursin 3 out of 6 cases, it falls behind significantly in terms of
average precision.
To explore the trade-off between fairness and precision,
we perform the following experiment: λ was increased between [0.01, 0.15] in steps of 0.01, and for each value of λ,the model was trained three times, each time for 1 epoch.
Figure 4 shows how AP, DEO, and DP change. The results
show that by increasing λ, precision decreases while fairness
measures improve.

Conclusions
We proposed an image reconstruction process to mitigate
bias against a protected attribute. The model’s performancewas evaluated on CelebA dataset and compared with anaugmentation based method developed by (Ramaswamy,
Kim, and Russakovsky 2021). The proposed model showedpromising results in mitigating bias while maintaining highprecision for classifiers. An interesting aspect of the results
is that although we only explicitly train the U-net to remove
dependence between the target attribute (Attractive)
and the protected attribute (Male), classifiers related tomany other attributes, most of which have a statistical dependency with the target attribute, become ‘fairer’. An advantage of the proposed model is that it does not rely on
modifying downstream classifiers, and rather includes only
modifying the input data, hence making it suitable to be deployed in an automated machine learning pipeline more easily and with lower cost. As a potential future direction, we
intend to consider the problem in a situation where multiple protected attributes are present, and attributes are nonbinary. We also intend to apply similar methodology onother data types such as tabular data.

References
Alvi, M.; Zisserman, A.; and Nellåker, C. 2018. Turning ablind eye: Explicit removal of biases and variation from deepneural network embeddings. In Proceedings of the European
Conference on Computer Vision (ECCV) Workshops, 0–0.
Buolamwini, J.; and Gebru, T. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, 77–91. PMLR.
Choi, K.; Grover, A.; Singh, T.; Shu, R.; and Ermon, S.
2020. Fair generative modeling via weak supervision. In International Conference on Machine Learning, 1887–1898.
PMLR.
Chouldechova, A. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments.
Big data, 5(2): 153–163.
Denton, E.; Hutchinson, B.; Mitchell, M.; Gebru, T.;
and Zaldivar, A. 2019. Image counterfactual sensitivity analysis for detecting unintended bias. arXiv preprint
arXiv:1906.06439.
Gretton, A.; Bousquet, O.; Smola, A.; and Schölkopf, B.
2005a. Measuring statistical dependence with HilbertSchmidt norms. In International conference on algorithmic
learning theory, 63–77. Springer.

Interpretation and the effect on other attributes
In this section, we aim to display the correspondence between an attribute’s relationship with Attractive attribute, and the extent to which the model modifies that
attribute. To do so, for each attribute, we record two values, namely HSIC value between that attribute and the
Attractive attribute, and the change in demographic
parity. To calculate the change in demographic parity, we
first calculate the demographic parity of the classifier forthat specific attribute, when the classifier classifies the original testing set images (similar to Baseline in previous tables, but for each attribute separately). We then calculate
the demographic parity of the classifier for that specific attribute, when the classifier receives the modified training
images Ours(5,0.07). We then subtract the two values, toget the change in demographic parity for that specific attribute. Figure 5 presents the results, with the red bars showing the change in demographic parity for each attribute, andthe blue bars showing the statistical dependence measured
6Baseline
GanDeb
AdvDb
Ours

Incons.
0.667
0.641
0.243
0.618

AP ↑
G-dep
0.79
0.763
0.333
0.732

G-indep
0.843
0.831
0.218
0.839

DP ↓
G-dep
0.255
0.233
0.169
0.146

Incons.
0.147
0.106
0.091
0.097

G-indep
0.137
0.119
0.121
0.118

Incons.
0.186
0.158
0.136
0.124

DEO ↓
G-dep G-indep
0.243
0.163
0.24
0.142
0.149
0.098
0.172
0.114

Table 1: Comparing the results of our model with Baseline, GAN debiasing (GanDeb), and Adversarial debiasing (AdvDb).
Showing AP (Average Precision, higher the better), DP (Demographic Parity, lower the better), and DEO (Difference in Equalityof Opportunity, lower the better) values for each attribute category. Each number is the average over all attributes within thatspecific attribute category.

Figure 4: Exploring the trade-off between accuracy and fairness by incremental increasing of parameter λ. Each data point is
the average over three trainings, with standard deviation of the three trainings shown as confidence intervals.

Figure 5: Displaying the relationship between an attribute’s statistical dependence on Attractive attribute, and the extent
to which the model modifies that attribute. Blue bars show the HSIC between each attribute with Attractive attribute in
the original data. Red bars show the absolute difference in demographic parity of each attribute’s classifier, acting on original
images and transformed images, respectively.

7

Sharmanska, V.; Hendricks, L. A.; Darrell, T.; andQuadrianto, N. 2020.
Contrastive examples for addressing the tyranny of the majority.
arXiv preprint
arXiv:2004.06524.
Tommasi, T.; Patricia, N.; Caputo, B.; and Tuytelaars, T.
2017. A deeper look at dataset bias. In Domain adaptationin computer vision applications, 37–55. Springer.
Torralba, A.; and Efros, A. A. 2011. Unbiased look at dataset
bias. In CVPR 2011, 1521–1528. IEEE.
Wang, A.; Narayanan, A.; and Russakovsky, O. 2020. REVISE: A tool for measuring and mitigating bias in visual
datasets. In European Conference on Computer Vision, 733–
751. Springer.
Wang, M.; and Deng, W. 2019. Mitigate bias in face recognition using skewness-aware reinforcement learning. arXiv
preprint arXiv:1911.10692.
Wang, M.; Deng, W.; Hu, J.; Tao, X.; and Huang, Y. 2019a.
Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In Proceedings ofthe IEEE/CVF International Conference on Computer Vision, 692–702.
Wang, T.; Ding, Z.; Shao, W.; Tang, H.; and Huang, K.
2021. Towards Fair Cross-Domain Adaptation via Generative Learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 454–463.
Wang, T.; Zhao, J.; Yatskar, M.; Chang, K.-W.; and Ordonez,
V. 2019b. Balanced datasets are not enough: Estimating andmitigating gender bias in deep image representations. In
Proceedings of the IEEE/CVF International Conference on
Computer Vision, 5310–5319.
Wang, Z.; Qinami, K.; Karakozis, I. C.; Genova, K.; Nair,P.; Hata, K.; and Russakovsky, O. 2020. Towards fairness invisual recognition: Effective strategies for bias mitigation.
In Proceedings of the IEEE/CVF conference on computer
vision and pattern recognition, 8919–8928.
Xu, X.; Huang, Y.; Shen, P.; Li, S.; Li, J.; Huang, F.; Li,Y.; and Cui, Z. 2021. Consistent Instance False Positive
Improves Fairness in Face Recognition. In Proceedings ofthe IEEE/CVF Conference on Computer Vision and Pattern
Recognition, 578–586.
Yang, K.; Qinami, K.; Fei-Fei, L.; Deng, J.; and Russakovsky, O. 2020. Towards fairer datasets: Filtering andbalancing the distribution of the people subtree in the imagenet hierarchy. In Proceedings of the 2020 Conference on
Fairness, Accountability, and Transparency, 547–558.
Zhang, B. H.; Lemoine, B.; and Mitchell, M. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, andSociety, 335–340.
Zhang, Q.; Wang, W.; and Zhu, S.-C. 2018. Examining cnn
representations with respect to dataset bias. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; and Chang,
K.-W. 2017. Men also like shopping: Reducing gender bias
amplification using corpus-level constraints. arXiv preprint
arXiv:1707.09457.

Gretton, A.; Herbrich, R.; Smola, A.; Bousquet, O.;
Schölkopf, B.; et al. 2005b. Kernel methods for measuring
independence.
Hardt, M.; Price, E.; and Srebro, N. 2016. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29: 3315–3323.
Horn, R. A.; and Johnson, C. R. 2012. Matrix analysis.
Cambridge university press.
Hwang, S.; Park, S.; Kim, D.; Do, M.; and Byun, H. 2020.
FairfaceGAN: Fairness-aware facial image-to-image translation. arXiv preprint arXiv:2012.00282.
Kamiran, F.; and Calders, T. 2012. Data preprocessing techniques for classification without discrimination. Knowledge
and Information Systems, 33(1): 1–33.
Lambrecht, A.; and Tucker, C. 2019. Algorithmic bias?
An empirical study of apparent gender-based discriminationin the display of STEM career ads. Management science,65(7): 2966–2981.
Liu, Z.; Luo, P.; Wang, X.; and Tang, X. 2015. Deep Learning Face Attributes in the Wild. In Proceedings of International Conference on Computer Vision (ICCV).
Lokhande, V. S.; Akash, A. K.; Ravi, S. N.; and Singh, V.
2020. Fairalm: Augmented lagrangian method for training
fair models with little regret. In European Conference on
Computer Vision, 365–381. Springer.
Mehrabi, N.; Morstatter, F.; Saxena, N.; Lerman, K.; andGalstyan, A. 2021. A survey on bias and fairness in machinelearning. ACM Computing Surveys (CSUR), 54(6): 1–35.
Pessach, D.; and Shmueli, E. 2020. Algorithmic fairness.
arXiv preprint arXiv:2001.09784.
Quadrianto, N.; Sharmanska, V.; and Thomas, O. 2019. Discovering fair representations in the data domain. In Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, 8227–8236.
Rajabi, A.; and Garibay, O. O. 2021. TabFairGAN: Fair Tabular Data Generation with Generative Adversarial Networks.
arXiv preprint arXiv:2109.00666.
Ramaswamy, V. V.; Kim, S. S.; and Russakovsky, O. 2021.
Fair attribute classification through latent space de-biasing.
In Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition, 9301–9310.
Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In
International Conference on Medical image computing andcomputer-assisted intervention, 234–241. Springer.
Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.;
Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.;
et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252.
Sattigeri, P.; Hoffman, S. C.; Chenthamarakshan, V.; andVarshney, K. R. 2019. Fairness GAN: Generating datasetswith fairness properties using a generative adversarial network. IBM Journal of Research and Development, 63(4/5):
3–1.
8