COMPAS Case Study: Fairness of a Machine Learning Model
By: Farhan Rahman
Tags: Data Journalism, Towards Data Science, Machine Bias, Data Science, Machine Learning

COMPAS and classifier fairness

COMPAS Case Study: Fairness of a Machine Learning Model

Photo by Clay Banks on Unsplash

Recent events around the world raise many questions — is the society we are living in biased to a particular sect. With many unanswered questions about the racial discrimination, lets explore a case study about COMPAS which is tool used in many jurisdictions around the U.S. to predict if a convicted criminal is likely to re-offend. Machine learning algorithms are surely quite beneficial when it comes to predict a set of unknown depending upon the already acquired knowledge about the what we already know. Having said that for machine learning models to provide accuracy without bias it is imperative that the data on which the algorithm works upon is clean and without any pre-defined inclination towards a particular data. Sometimes machine learning models are just not suitable to be used on the grounds of social ethics and norms.

About COMPAS:

COMPAS, an acronym for Correctional Offender Management Profiling for Alternative Sanctions, is an assistive software and support tool used to predict recidivism risk — the risk that a criminal defendant will re-offend.

COMPAS is helpful in ways that it provides scores from 1 (being lowest risk) to 10 (being highest risk).

It also provides a category based evaluation labeled as high risk of recidivism, medium risk of recidivism, or low risk of recidivism. For simplifying things we can convert this multi-class classification problem into binary classification combining the medium risk and high risk of recidivism vs. low risk of recidivism.

The input used for prediction of recidivism is wide-scale and uses 137 factors including age, gender, and criminal history of the defendant as the input.

Race is not an explicit feature considered by the model.

Use of COMPAS:

COMPAS has been used by the U.S. states of New York, Wisconsin, California, Florida’s Broward County, and other jurisdictions. Depending on the scores generated by this software, the judge can decide upon whether to detain the defendant prior to trial and/or when sentencing. It has been observed that the Defendants who are classified medium or high risk (scores of 5–10), are more likely to be held in prison while awaiting trial than those classified as low risk (scores of 1–4). Although this software might seem to be assistive and helpful but it suffers from machine bias. According to an investigative journal ProPublica:

The prediction fails differently for the black defendants:

Overall, Northpointe’s assessment tool correctly predicts recidivism 61 percent of the time. But blacks are almost twice as likely as whites to be labeled a higher risk but not actually re-offend. It makes the opposite mistake among whites: They are much more likely than blacks to be labeled lower risk but go on to commit other crimes. This a major risk of machine learning models might possess and when it comes to someone’s freedom it’s a flaw that shouldn’t go unnoticed.

To get what ProPublica claimed let’s try to make our own model which replicates their analysis.

Formulating ProPublica analysis:

Before starting any analysis, we need to have the proper tools for that — importing the required libraries.

Read in the data:

After importing the libraries we need to import the data and to know how the data looks like use the following code to visualize it :

After running the above cell we get the following output:

As stated earlier to make this into a binary classification problem we carry out the following section.

Transform into a binary classification problem:

First, let’s make this a binary classification problem. We will add a new column that translates the risk score ( decile_score ) into a binary label.

Any score 5 or higher (Medium or High risk) means that a defendant is treated as a likely recividist, and a score of 4 or lower (Low risk) means that a defendant is treated as unlikely to re-offend. As we know that creating a new feature with the help of the existing one might help us to envision data that might go unexplored. The code to do the same is given below :

To see the new column added , we can use df.head().

Evaluate model performance:

To evaluate the performance of the model, we will compare the model’s predictions to the “truth”:

The risk score prediction of the COMPAS system is in the decile_score column,

column, The classification of COMPAS as medium/high risk or low risk is in the is_med_or_high_risk column

column The “true” recidivism value (whether or not the defendant committed another crime in the next two years) is in the two_year_recid column.

Let’s start by computing the accuracy:

This, itself, might already be considered problematic…

The accuracy score includes both kinds of errors:

false positives (defendant is predicted as medium/high risk but does not re-offend)

false negatives (defendant is predicted as low risk, but does re-offend)

but these errors have different costs. It can be useful to pull them out separately, to see the rate of different types of errors.

If we create a confusion matrix, we can use it to derive a whole set of classifier metrics:

True Positive Rate (TPR) also called recall or sensitivity

True Negative Rate (TNR) also called specificity

Positive Predictive Value (PPV) also called precision

Negative Predictive Value (NPV)

False Positive Rate (FPR)

False Discovery Rate (FDR)

False Negative Rate (FNR)

False Omission Rate (FOR)

To use the theory into practice we can use various approach for the same.

We can also use sklearn 's confusion_matrix to pull out these values and compute any metrics of interest:

We get the following distribution :

True negatives: 2681

False positives: 1282

False negatives: 1216

True positives: 2035

Or we can compute them directly using crosstab -

Here, we normalize by row — show the PPV, FDR, FOR, NPV:

Here, we normalize by column— show the TPR, FPR, FNR, TNR:

Overall, we see that a defendant has a similar likelihood of being wrongly labeled a likely recidivist and of being wrongly labeled as unlikely to re-offend:

We can also directly evaluate the risk score, instead of just the labels. The risk score is meant to indicate the probability that a defendant will re-offend.

Defendants with a higher COMPAS score indeed had higher rates of recidivism.

After running the above cell we get the following accuracy 0.7021662544019724

Fairness:

COMPAS has been under scrutiny for issues related for fairness with respect to race of the defendant.

Race is not an explicit input to COMPAS, but some of the questions that are used as input may have strong correlations with race.

First, we will find out how frequently each race is represented in the data:

We will focus specifically on African-American or Caucasian defendants, since they are the subject of the ProPublica claim.

It isn’t exactly the same, but it’s similar — within a few points. This is a type of fairness known as overall accuracy equality.

Next, let’s see whether a defendant who is classified as medium/high risk has the same probability of recidivism for the two groups.

In other words, we will compute the PPV for each group:

Again, similar (within a few points). This is a type of fairness known as predictive parity. We can extend this idea, to check whether a defendant with a given score has the same probability of recidivism for the two groups:

We can see that for both African-American and Caucasian defendants, for any given COMPAS score, recidivism rates are similar. This is a type of fairness known as calibration.

Next, we will look at the frequency with which defendants of each race are assigned each COMPAS score:

We observe that Caucasian defendants in this sample are more likely to be assigned a low risk score.

However, to evaluate whether this is unfair, we need to know the true prevalence — whether the rates of recidivism are the same in both populations, according to the data:

The predictions of the model are pretty close to the actual prevalence in the population.

So far, our analysis suggests that COMPAS is fair with respect to race:

The overall accuracy of the COMPAS label is the same, regardless of race ( overall accuracy equality )

) The likelihood of recidivism among defendants labeled as medium or high risk is similar, regardless of race ( predictive parity )

) For any given COMPAS score, the risk of recidivism is similar, regardless of race — the “meaning” of the score is consistent across race (calibration)

We do not have statistical parity (a type of fairness corresponding to equal probability of positive classification), but we don’t necessarily expect to when the prevalence of actual positive is different between groups.

We can fix it, right?

Why is it so tricky to satisfy multiple types of fairness at once? This is due to a proven impossibility result.

Any time

the base rate (prevalence of the positive condition) is different in the two groups, and

we do not have a perfect classifier

Then we cannot simultaneously satisfy:

Equal PPV and NPV for both groups (known as conditional use accuracy equality ), and

), and Equal FPR and FNR for both groups (known as equalized odds or conditional procedure accuracy equality)

What we learned

A model can be biased with respect to age, race, gender, if those features are not used as input to the model.

There are many measures of fairness, it may be impossible to satisfy some combination of these simultaneously.

Human biases and unfairness in society leak into the data used to train machine learning models.

Reference: