Delivering Fairness in Human Resources AI : Mutual Information to the Rescue (2022)

Abstract: Automatic language processing is used frequently in the Human Resources (HR) sector for automated candidate sourcing and evaluation of resumes. These models often use pre-trained language models where it is difficult to know if possible biases exist. Recently, Mutual Information (MI) methods have demonstrated notable performance in obtaining representations agnostic to sensitive variables such as gender or ethnicity. However, accessing these variables can sometimes be challenging, and their use is prohibited in some jurisdictions. These factors can make detecting and mitigating biases challenging. In this context, we propose to minimize the MI between a candidate’s name and a latent representation of their CV or short biography. This method may mitigate bias from sensitive variables without requiring the collection of these variables. We evaluate this methodology by first projecting the name representation into a smaller space to prevent potential MI minimization problems in high dimensions.

## introduction there are numerous examples of artificial intelligence (ai) systems which fail to mitigate bias contained within datasets used to train models @xcite @xcite @xcite . bias can be introduced via human labelling or via data extracted from existing human processes which replicates societal biases @xcite . left unchecked, machine learning models will reflect directly the data used to train them or possibly even exacerbate the effect of biased data. this is of particular concern in high-risk domains such as human resources (hr), where models can be used to assess candidates based on data provided in a curriculum vitae (cv) @xcite . large pre-trained language models (llms) have been the source of impressive performance gains in recent times on tasks such as question answering @xcite , common-sense reasoning @xcite , computer coding @xcite and other domains. however, their capabilities are characterized poorly, requiring a greater understanding of their function to ameliorate potential harms @xcite . fine-tuning llms on downstream tasks has become the gold standard for approaching many natural language processing (nlp) tasks @xcite . however, the nature of this workflow means that practitioners who fine-tune such models on downstream tasks have little visibility of the data used to train the original model purely because of the volume of data involved. this lack of visibility can be problematic given that these models are trained on huge volumes of text data which may contain hidden biases @xcite . mutual information (mi) is a method for measuring the dependence between two features. it is the reduction in uncertainty for one random variable caused by knowledge of another. cover and @xcite define it for two random variables x and y , having a joint probability mass function p(x, y) and marginal probability mass functions p(x) and p(y), as the relative entropy between the joint distribution and the product distribution: here, e p(x,y) is the expected value over the distribution p. mi is never negative, and values greater than zero indicate some degree of dependence between the variables @xcite . it has been extensively explored in domains such as statistics, robotics and bioinformatics @xcite in addition to machine learning @xcite @xcite @xcite . in machine learning, it can be used to measure the amount of sensitive information, such as gender or ethnicity, contained in a cv in a hiring process. using mi as a loss function's regularizer, the dependence between variables can be minimized @xcite , thus disentangling sensitive and non-sensitive information in representations used to train models. in this work, we will refer to the process of applying mi to a representation to minimize the sensitive information held within it as 'disentanglement'. we note that it is sometimes challenging to collect data on such sensitive variables due to privacy concerns and that it is even illegal in some jurisdictions @xcite . to overcome this problem, we propose using candidate names as a proxy for sensitive variables by reducing the mi between name and cv/bios embeddings. we investigate three approaches to mi estimation which have already seen attention in the literature within a hr context: info-nce (van den oord et al., 2019), club @xcite and knife @xcite . furthermore, we present low-dimensional versions of these algorithms, which are of interest given mi difficult to estimate in high dimensions @xcite @xcite . we present results on experiments carried out on two datasets relevant to hr applications: fair-cvtest @xcite , consisting of synthetic cv data and bios @xcite , a collection of freely available online short biographies in english. our contributions are as follows: this work motivates an investigation of mi estimators by highlighting the requirement for fairness procedures within ai-augmented systems for hr applications, such as hiring processes. we build on the mi estimators proposed by van den oord et al. (2019) info noise contrastive estimation (in-fonce), cheng et al. (2020) contrastive log-ratio upper bound (club) and pichler et al. (2022) kernelized-neural differential entropy estimation (knife). to our knowledge, our work is the first wholly focused on the hr domain to investigate the potential use of mi in hiring processes. we note that kamimura (2019) utilizes an hr dataset in his work, but it cannot be said to be focused entirely on the hr domain as his subject is validating a simplified method for calculating mi, which he demonstrates on hr, crab species and wholesale datasets. ## methodology our method aims to minimize the mi between a latent representation of an individual's input (either bios or cv embedding) and a word embedding of their name. our method comprises two steps. firstly, we project the name embedding into a rich low-dimensional space to solve the curse of dimensionality problem for the mi estimators. secondly, we minimize the mi between the representation of an individual's input and the latter disentangled representation. this method allows us to find possible sensitive, latent variables influencing the two views of the data (e.g. candidate gender influences the name of the candidate and the embedding of their cv) and to simultaneously mitigate the biases coming from these sensitive variables. we emphasize that this sensitive information is not used in the classifier but only in the disentanglement procedure. therefore, it is unnecessary to collect the sensitive variables after deploying the classifier. to generate name representations, we follow the approach of @xcite who use fast-text @xcite embeddings for this purpose. we note that this does not address the issue of out of vocabulary (oov) names -a critical point in any real-world implementation of this method which would require robust testing for edge cases using different embedding schemas. we focus on comparing the different mi estimators; thus, we leave experimentation with name representations to subsequent experiments. ## experiments we first examine the dimension reduction of target space to understand its utility. we then present the main results of our evaluation before discussing limitations and future works. ## conclusion and discussion in this paper, we propose the use of mi minimization in the context of hr to obtain fairer automatic models. in contrast to previous work that explicitly uses variables to be removed, we use a candidate name representation as a proxy. we show experimentally on two datasets that mi methods help obtain better-anonymized representations and fairer models while conserving task performance. moreover, we show that the dimension reduction of candidate name word embeddings allows us to overcome some problems related to estimating mi in high dimensions. overall, this work is the first to evaluate the use of mi in such an application context by considering the real-world limitations of sensitive data collection. finally, we hope this work will attract research interest in this challenging and vital task.