Eliminating Bias in Facial Recognition
By: Rucha Apte
Tags: Machine Learning, Algorithm Neutrality, Facial Recognition, Implicit Bias

Exploring Racial Bias via Data Augmentation

Approaches like ‘Training deep neural networks with biased data’, ‘Approaching machine learning fairness through adversarial networks’, establish adversarial techniques that focus on reducing the identity features which reveal race, gender. However, these approaches fail to address the issue of trade-off between suppressing use of these attributes and loss of the key features to overall performance of the algorithm. The paper ‘Exploring racial bias face recognition via per-subject adversarially-enabled data augmentation’ proposes a novel adversarial data augmentation method to enable dataset balance by making use of image-to-image translation to automatically construct a synthesized dataset such that the facial images are transformed to different racial domain but the identity related features are retained. The method is implemented via use of an adversarial image re-synthesis technique for transforming sensitive attributes to preserve the important identity related features while making the racially dependent features of the face less prevalent. Ideally, an algorithm should be designed such that given an input x, the conditional probability of output should not depend on sensitive attributes. That is , P(y||x) = P(y||x; s), where x is input features and s are sensitive attributes. However, removing dependency is challenging due to the presence of high mutual information between facial features and sensitive attributes. This paper approaches to transfer sensitive attributes from one domain to another domain while preserving prior information for recognition. Given an input of four racial domains African, Asian, Caucasian, Indian, the paper aims to transform an image from one domain to other three domains and forms a new augmented dataset. The input for the experiment contains data from four different racial domains of African, Asian, Caucasian, Indian denoted as A , E, C, I respectively. Learning a mapping function between two different domains reduces the dependency on sensitive features.

Given two domains xi∈A and xj∈C, belonging to different data distributions xi p~data(xi) and xj p~data(xj), two mapping functions F and G can be introduced that map the distributions between domain A and C using CycleGAN [32]. This framework requires two discriminators D_A and D_C to distinguish between xi and F_(xj); xj and G_(xi). An adversarial loss (Equation 4) is used for the first part of race transformation. The generator synthesizes images using source domain A to associate to target domain C and discriminator distinguishes between the real and the synthesized image. A similar process is applied for transformation from domain C to A. The cycle consistency loss helps to synthesize accurate images and allows reconstruction of original images from generated images (Equation 5). The overall loss function is given by Equation 6. Thus, a new augmented dataset (Figure 5) is generated which contains images of one domain mapped to the images of other three domains. Using state-of-the-art loss functions for face recognition algorithms, Softmax, CosFace and ArcFace, as experimental vehicles, and augmenting dataset (Figure 4), VGGFace2 [36] by performing set of 12 mappings: African➡Asian, African➡Caucasian, African➡Indian, Asian➡African, Asian➡Caucasian, Asian➡Indian, Caucasian➡African, Caucasian➡Asian, Caucasian➡Indian, Indian➡African, Indian➡Asian, Indian➡Caucasian using six CycleGAN models, a common DCNN ResNet100 is trained and following results are obtained. The measure of success is given by LFW face verification protocol and RFW.