PRIORITY-BASED, ACCURACY-CONTROLLED INDIVIDUAL FAIRNESS OF UNSTRUCTURED TEXT

Methods, systems, and computer program products for priority-based, accuracy-controlled individual fairness of unstructured text are provided herein. A method includes identifying one or more samples in a set of data used to train a machine learning model having at least one attribute; generating counterfactual samples for each of the one or more identified samples; calculating scores for the one or more identified samples based at least in part on output of the machine learning model with respect to the counterfactual samples, wherein the scores indicate a relative level of bias between the one or more identified samples corresponding to the at least one attribute; creating an enhanced set of data at least in part by supplementing at least a portion of the identified samples with the corresponding counterfactual samples based on the calculated scores; and training the machine learning model using the enhanced set of data.

BACKGROUND

The present application generally relates to information technology and, more particularly, to controlling fairness of unstructured text for machine learning models.

Generally, machine learning algorithms represent software models that are trained based on data to make predictions or decisions. Such predictions or decisions reflect the choices that were made when building the models. For example, the output of a software model will reflect any bias that is present in the training data.

SUMMARY

In one embodiment, techniques for priority-based, accuracy-controlled individual fairness of unstructured text are provided. An exemplary computer-implemented method can include steps of identifying one or more samples in a set of data used to train a machine learning model having at least one attribute; generating one or more counterfactual samples for each of the one or more identified samples; calculating scores for the one or more identified samples based at least in part on output of the machine learning model with respect to the counterfactual samples, wherein the scores indicate a relative level of bias between the one or more identified samples corresponding to the at least one attribute; creating an enhanced set of data at least in part by supplementing at least a portion of the identified samples with the corresponding one or more counterfactual samples based on the calculated scores; and training the machine learning model using the enhanced set of data.

Another embodiment, or elements thereof, can be implemented in the form of a computer program product tangibly embodying computer readable instructions which, when implemented, cause a computer to carry out a plurality of method steps, as described herein. Furthermore, another embodiment, or elements thereof, can be implemented in the form of a system including a memory and at least one processor that is coupled to the memory and configured to perform noted method steps. Yet further, another embodiment of the invention or elements thereof can be implemented in the form of means for carrying out the method steps described herein, or elements thereof; the means can include hardware module(s) or a combination of hardware and software modules, wherein the software modules are stored in a tangible computer-readable storage medium (or multiple such media).

DETAILED DESCRIPTION

Individual discrimination in text is present, for example, when the prediction of a model changes for a given classifier in response to changing a protected class attribute of a sample of the text. For instance, consider the following sample of text “my boss is younger than I am,” and the following counterfactual “my boss is older than I am.” If the prediction of a model (e.g., a sentiment text classification model) changes for these two samples, then the model is considered to have an age-related bias.

Conventional techniques to address fairness of machine learning models generally include pre-processing or in-processing based individual fairness in text. Generally, such conventional techniques suffer from one or more of the following disadvantages: failure to achieve a sufficient level fairness, compromise on text that might have less bias than other text, and failure to control drops in accuracy while trying to achieve individual fairness.

As described herein, embodiments of the present disclosure include improved techniques for priority-based, accuracy-controlled individual fairness of unstructured text. Such embodiments may include, for example, calculating unfairness quotients for samples of unstructured text and limiting the samples of the unstructured text to be debiased based on the unfairness quotients. According to at least one embodiment, samples of unstructured text having less individual bias are prioritized over other samples to control the accuracy of a machine learning model. Further, one or more exemplary embodiments include identifying layers of the machine learning model that contribute to unfairness and prioritizing the identified layers for de-biasing.

FIG. 1is a diagram illustrating a system architecture, according to an embodiment. By way of illustration,FIG. 1depicts a model de-biasing system102that obtains unstructured text104and a machine learning model106, and the model de-biasing system102outputs a de-biased model108. In theFIG. 1embodiment, the model de-biasing system102includes a sample identification module110, an accuracy controlled de-biasing module112, and a training module114.

The sample identification module110identifies samples of the unstructured text104relating to a protected attribute. Protected attributes, as used herein, generally refers to particular attributes that are to be de-biased, such as, for example, gender, age, nationality, etc.

The accuracy controlled de-biasing module112calculates an unfairness quotient for each of the samples identified as relating to a protected attribute and ranks, or prioritizes, the samples based on the calculated unfairness quotient. The accuracy controlled de-biasing module112debiases the samples of text based on the ranking while controlling an accuracy of the machine learning model106. The training module114trains, or re-trains, the machine learning model106using the debiased data to obtain the de-biased model108, as described in more detail elsewhere herein.

FIG. 2is a flow diagram for identifying protected attributes in unstructured text, according to an exemplary embodiment. Generally, the process depicted inFIG. 2uses a set of predefined keywords in the form of a dictionary to identify and/or extract samples of text that include a particular protected attribute. It is noted that theFIG. 2embodiment is described with respect to a single protected attribute; however, it is to be appreciated that such techniques may be used to detect multiple attributes, such as, for example, by generating a dictionary for each of the multiple attributes.

Step202ofFIG. 2includes obtaining a set of words for the protected attribute. For example, if the protected attribute corresponds to age, then the set of keywords comprises a list of age-related terms, which can be manually curated and/or obtained from one or more online resources, for example. As such, the set of words at step202can be referred to as “seed” words for the protected attribute. Step204includes generating a dictionary based on the set of words obtained at step202and a word embedding space. Step204may include identifying words within a specified distance of word embedding space for each word in the set and adding these words to the dictionary. As an example, if the word “young” is used as a seed word, then the following list of words may be obtained based on the word embedding space: children, kids, teens, teenager, youngster, youths, teenagers, young, younger, youngest. According to at least one embodiment, such sets may also be used to generate counterfactuals (or perturbations), as described in more detail elsewhere herein. Perturbing a sample generally refers to a process that modifies at least some of the text of the sample to generate a new, perturbed sample. By way of example, if a sample of text corresponds to a sentence that includes the word “young,” then the sample can be perturbed by replacing the word “young” with each of the words in the list above, for example. Step206includes extracting text samples based on the dictionary generated at step204.

FIG. 3shows example pseudocode300of a process for priority-based, accuracy-controlled individual fairness of unstructured text, according to an exemplary embodiment. The example pseudocode300is representative of computer code that may be executed by or under the control of at least one processing system and/or device. For example, the example pseudocode300may be viewed as comprising a portion of a software implementation of at least part of the mode de-biasing system102of theFIG. 1embodiment.

The pseudocode300includes obtaining a machine learning model and training data used to train the model, which may include, for example, unstructured text. The pseudocode300includes identifying samples that have at least one protected attribute. The samples may be identified using dictionaries, such as described above in conjunction withFIG. 2, for example. For each identified sample, counterfactual(s) may be generated based on the corresponding dictionary. An unfairness quotient is calculated for each identified sample based at least in part on the output of the model with respect to the counterfactuals. For example, the unfairness quotient may be calculated as the difference in a prediction score associated with a class label between the original sample and counterfactuals. Each identified sample is then ranked according to the unfairness quotients. The pseudocode300determines which of the samples are to be debiased based on the rank and an unfairness quotient threshold. The training data is updated to include the counterfactuals corresponding to the samples that are to be debiased, and the model is trained (or re-trained) using the updated training data.

In at least some examples, counterfactuals (e.g., perturbed sentences) are generated in ascending order of the unfairness quotient value. Additionally, it is noted that samples having a lower unfairness quotient generally have less of an effect on the accuracy of the model than samples having a higher unfairness quotient. Further, the unfairness quotient threshold in the pseudocode300can correspond to a hyperparameter, which can be tuned based on the amount of control needed over accuracy of the model. As such, the model can be re-trained so that it is less capable of distinguishing between different groups in a protected attribute, while controlling the accuracy of the model.

One or more example embodiments include prioritizing particular layers of the machine learning model when re-training the model. For example, for each sample having at least one protected attribute, the prioritization can be performed as follows:Calculate a divergence, Di, in the internal representations of each layer, for both the identified sample and the counterfactuals, denoted by Li(x) and Li(x′), respectively. For example, the divergence can be equal to: 1−cosine (Li(x), Li(x′)).Rack each layer of the machine learning model for its contribution towards unfairness based on the computed divergences.Re-train only a specified number of the layers (e.g., top-k), while freezing the remaining layers.

Such a prioritization process increases the performance of re-training and allows the re-training to focus only on the parts of the model that contribute most to unfairness.

FIG. 4is a flow diagram illustrating techniques according to an exemplary embodiment. Step402includes identifying one or more samples in a set of data used to train a machine learning model having at least one attribute. Step404includes generating one or more counterfactual samples for each of the one or more identified samples. Step406includes calculating scores for the one or more identified samples based at least in part on output of the machine learning model with respect to the counterfactual samples, wherein the scores indicate a relative level of bias between the one or more identified samples corresponding to the at least one attribute. Step408includes creating an enhanced set of data at least in part by supplementing at least a portion of the identified samples with the corresponding one or more counterfactual samples based on the calculated scores. Step410includes training the machine learning model using the enhanced set of data.

Calculating the score for a given one of the identified samples is based on a comparison of the output of the machine learning model for the given sample with the output of the machine learning model for the corresponding one or more counterfactual samples. The creating may include controlling an accuracy of the machine learning model by supplementing only the identified samples having scores above a threshold value with the corresponding one or more counterfactual samples. The threshold value may include a tunable hyperparameter. A given one of the identified samples may be identified using a set of keywords associated with the at least one attribute that is generated based at least in part on a word embedding space. Generating the one or more counterfactual samples may include using the set of keywords to generate perturbations of the given identified sample. The process depicted inFIG. 4may further include the steps of determining an impact of the one or more counterfactual samples relative to the corresponding identified sample at each of a plurality of layers of the machine learning model; and retraining only a portion of the plurality of the layers of the machine learning model based on the determined impact at each of the layers. The at least one attribute may be related to at least one of: gender, age, and nationality.

An embodiment of the present disclosure or elements thereof can be implemented in the form of an apparatus including a memory and at least one processor that is coupled to the memory and configured to perform exemplary method steps.

A data processing system suitable for storing and/or executing program code will include at least one processor502coupled directly or indirectly to memory elements504through a system bus510. The memory elements can include local memory employed during actual implementation of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during implementation.

Input/output or I/O devices (including, but not limited to, keyboards508, displays506, pointing devices, and the like) can be coupled to the system either directly (such as via bus510) or through intervening I/O controllers (omitted for clarity).

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

At least one embodiment of the present disclosure provides a beneficial effect such as, for example, reducing bias while controlling accuracy of machine learning models. Additionally, at least one embodiment of the present disclosure provides a beneficial effect such as, for example, improved machine learning training techniques to reduce bias, by targeting specific layers of the machine learning model.