How to Explain and Affect Individual Decisions with ICE Curves
By: Wai On
Tags: Data Visualization, Data Science, Explainable Ai, Model Interpretability, Interpretable Ai

How to Explain and Affect Individual Decisions with ICE Curves

Examples of using Individual Conditional Expectation Plots in Credit Application and Employee Retention

Now that we have a way to highlight an individual instance of interest in an ICE plot (see part 1), let’s illustrate their use in a couple of more realistic examples: Credit Application and Employee Retention. These examples show the different contexts that local explanations might be needed and allow us to test whether the design decisions made earlier enable ICE plots to help explain and affect individual decisions. The code for this article can be found on GitHub.

Example 1: Credit Application

Scenario

Photo by Andrea Piacquadio on Pexel

Ron is a retiree on a fixed-income. He recently applied for a car loan but was rejected. Rather annoyed, he is demanding an explanation for his rejection.

You are the loan specialist and are responsible for reviewing the automated decision with Ron. You are eager to help him since getting what he wants also benefits your business. You meet with Ron to review his rejection in detail and to see what advice you can provide.

Data: Loan Default

The data set for this example is the Loan Default Data available from Kaggle. The data consists of 10 features and a target output of “serious delinquency past 90 days”. We will use the target as a recommendation for loan approval. That is, if the prediction for “serious delinquency” is true then the loan is rejected, otherwise it is approved.

A Snippet of the Loan Default Dataset

Data Explorations

Per our discussion in part 1, we should first see if there are any features that are highly correlated.

Correlation Matrix of all variables

The data shows three instances of high correlation:

30–59DaysLate and 90DaysLate: 0.98

30–59DaysLate and 60–89DaysPastDue: 0.98

90DaysLate and 60–89DaysPastDue: 0.99

In other words, the data shows that late debt payments are highly correlated between all the time frames (i.e., 30–59 days, 60–89 days, or 90 days).

In order to deal with these correlations, both grouping and elimination strategies were tried (i.e., aggregating all 3 variables into one, and dropping one or more of the highly correlated variables). However, no significant differences were found in how they affected the predictions. To simplify the interpretation of the explanation downstream, I eliminated the features with the lower correlation to the target variable which resulted in retaining the feature “30–59DaysLate”.

The Random Forest algorithm from scikit-learn was used to fit a model with these features for a prediction of loan approval (Accuracy = 0.93). The relative importance of the features are as follows.

Feature Importance for Loan Approval

Feature Importance analysis shows that the most important feature for our model is “Utilization”. This feature is defined as the “total balance on credit cards and personal lines of credit except real estate and no installment debt like car loans divided by the sum of credit limits”. In other words, a ratio of credit balance over credit limit. The importance of this feature is closely followed by debt ratio and monthly income. Let’s plot these features using our modified ICE plot to see what is going on at a more detailed level for our individual instance.

ICE Plots

The top 6 features visualized using our modified ICE plot are as follows. I’m using the pyCEbox package in Python for the calculation of ICE values with modifications on the visualization functions as described in part 1. The code for this can be found on GitHub.

Recall from part 1 that:

The thicker black line is the PD curve (recall this is the average of all the instances).

All other lines are ICE curves for individual instances. Each sub-plot here shows 602 individual instances from the dataset.

The blue line with markers is the instance of interest.

The yellow diamond shaped marker on the blue line shows the current feature value and the prediction for the instance we are interested in.

Let’s examine these sub-plots in turn. Reminder that a prediction of 1 is good (loan approved) and 0 is bad (loan declined).

Utilization: The ICE plot shows that there is a substantial turbulence up to a value of 1.0 of the x-axis; after which there is more volatility at a lower level of probability, between 1.0–1.1, before stabilizing. The PD curve reflects this, but shows clearly that there is a drop around a value of 1.0.

The ICE curve for Ron (in blue with round markers) follows the PD curve pattern more or less up to a value of 1.0 in the x-axis, after which there is a precipitous drop that is much lower than the PD curve. Moreover, as shown by the yellow diamond marker, the prediction for Ron is near the bottom of the ICE curve for this feature. Since this is the most important feature, it is an important partial explanation as to why Ron’s loan application was declined.

Modified ICE Plot for the Utilization Feature

According to Ron’s ICE curve, to increase his chance of getting a loan, he could either increase or decrease his Utilization ratio. The visualization shows that decreasing this ratio looks more effective (either by decreasing his credit balance or increasing his credit limit size). In other words, if “Utilization” is reduced, the precipitous drop in the curve will be reverse immediately, whereas increasing “Utilization” will result in a more gradual increase.

DebtRatio: Apart from a brief initial flutter, the PD curve and individual ICE curve shows the prediction to be relatively constant.

MonthlyIncome: Ron’s ICE curve shows he is near the bottom of the curve. If his monthly income is to increase then his chance of getting a loan would also increase.

Ron currently has a monthly fixed income of 5,804 from his retirement. The ICE curve shows that the optimal level for Ron needs to be is above 18,000. This is a large increase from where he is currently. However, the ICE curve shows that even if Ron could increase his monthly income by a small amount (thereby shifting to the right of the yellow diamond marker in the curve), his chance of getting a loan would increase. Having discussed this with Ron, it is concluded that this is not something he could do immediately. However, the conversation did cause him to think more about taking on a part-time job.

Age: Although the curves have some interesting fluctuations, this is one of those variables that unfortunately we cannot control. However, the good news is that the ICE curve for Ron suggests that the likelihood of getting a loan remains consistent as he gets older.

OpenCreditLinesAndLoans: Both the PD curve and ICE curve are relatively flat. For Ron, there are some minor variations for a value of under 20, after which it stabilizes.

30–59DaysPastDue: This feature is defined to be the “number of times the borrower has been 30–59 days past due but no worse in the last 2 years”.

The ICE curve shows that if Ron is able to reduce his late payments then his chance of getting approved would dramatically increase. The PD curve also shows a downward trend in loan approval probability as payment past due increases.

Even though Ron has only been late once over the past two years, this nevertheless had a dramatic effect on his chances of getting a loan approval. Although this is not something that Ron could do anything about immediately, it is an important takeaway for the future.

Aggregating Correlated Variables into One Variable: 30–90 Days Past Due

Recall earlier that I also tried a strategy of combining the highly correlated variables (i.e., “30–59DaysPastDue”, “60–89DaysPastDue”, and “90DaysLate”). The result shows that the general shape of the ICE curve is similar to that of above (with a steep downward drop before shallow recovery) and corroborates with the conclusion that any delay in payments will likely increase the probability of loan getting declined.

Summary

The analysis shows that “Utilization” is the most important feature that explains Ron’s loan application rejection. To maximize his chance of getting a loan in the future, there are three things that Ron should consider:

Decrease his “Utilization” ratio. If possible, increase his monthly income in the future. Make sure he is not late in debt payment in the future.

Having reviewed the visualizations and discussed this with you, Ron feels there are some concrete actions worth considering and feels much better about his loan rejection.

Example 2: Employee Retention

Scenario

Photo by loly galina on Unsplash

Sourabh is a star employee in your Research and Development team. He is 30 something years old and has been with the company for 5 years.

As his manager, in addition to rewarding him for his performance, you want to get a better picture of what you can do to ensure that he is motivated to stay with the company. You meet with your HR specialist, who uses ML models to help predict employee attrition, to discuss this issue.

The meeting did not start well. Your HR specialist tells you that according to the ML model, Sourabh is likely to leave. With urgency, you and your HR specialist examine the results closely to understand it further and to see what steps you need to take to retain Sourabh.

Data: Employee Attrition

The dataset for this example is the popular dataset from IBM on employee attrition. After removing a number of redundant features and reducing the data to only the Research and Development team, we end up with 1470 records, 24 features, and a target variable “Attrition”. Note that in contrast to the earlier example, there are substantially more features we have to deal with in our analysis and our visualizations.

A Snippet of IBM Employee Attrition Dataset

Data Explorations

There are so many features in this dataset that makes the Correlation Matrix difficult to see and examine in detail here. Nevertheless, it is possible to make out that there are a number of variables that are highly correlated.