Explaining Scikit-learn models with SHAP | by Zolzaya Luvsandorj | Towards Data Science

Explainable AI (XAI) helps build trust and confidence in machine learning models by making them more transparent. XAI is a set of tools and frameworks that can be used to understand and interpret how a machine learning model makes decisions. One useful XAI tool is the SHAP library in Python. This tool allows us to quantify feature’s contribution towards a single prediction as well as predictions at an overall level. The library also comes with aesthetically pleasing easy-to-use visualisations. In this post, we will learn the basics of the SHAP library to understand predictions from regression and classification models built in Scikit-learn.
Shap value helps us quantify feature’s contribution towards a prediction. Shap value closer to zero means the feature contributes little to the prediction whereas shap value away from zero indicates the feature contributes more.
Let’s learn how to pull shap values for features for a regression problem. We will start by loading libraries and sample data, then will build a quick model to predict diabetes progression:
import numpy as npnp.set_printoptions(formatter={'float':lambda x:"{:.4f}".format(x)})import pandas as pdpd.options.display.float_format = "{:.3f}".formatimport seaborn as snsimport matplotlib.pyplot as pltsns.set(style='darkgrid', context='talk', palette='rainbow')from sklearn.datasets import load_diabetesfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import (RandomForestRegressor,                               RandomForestClassifier)import shap # v0.39.0shap.initjs()# Import sample datadiabetes = load_diabetes(as_frame=True)X = diabetes['data'].iloc[:, :4] # Select first 4 columnsy = diabetes['target']# Partition dataX_train, X_test, y_train, y_test = train_test_split(X, y,                                                     test_size=0.2,                                                     random_state=1)print(f"Training features shape: {X_train.shape}")print(f"Training target shape: {y_train.shape}\n")print(f"Test features shape: {X_test.shape}")print(f"Test target shape: {y_test.shape}")display(X_train.head())# Train a simple modelmodel = RandomForestRegressor(random_state=42)model.fit(X_train, y_train)
One common way to get shap values is to use the Explainer object. Let’s create an Explainer object and extract shap_test for the test data:
explainer = shap.Explainer(model)shap_test = explainer(X_test)print(f"Shap values length: {len(shap_test)}\n")print(f"Sample shap value:\n{shap_test[0]}")
The length of shap_test is 89 because it contains a record for each test instance. From looking at the first test record, we can see that there are three attributes:◼ shap_test[0].base_values: Base value for target◼ shap_test[0].data: Value for each feature◼ shap_test[0].values: Shap value for each feature
Let’s understand what each one of these attributes shows us.
Base value (shap_test.base_values), also known as the expected value (explainer.expected_value), is the average target value in the training data. We can check this with the following code:
print(f"Expected value: {explainer.expected_value[0]:.1f}")print(f"Average target value (training data): {y_train.mean():.1f}")print(f"Base value: {np.unique(shap_test.base_values)[0]:.1f}")
Hence, we will use the word expected value and base value interchangeably.
Next, shap_test.data contains the same values as X_test:
(shap_test.data == X_test).describe()
Let’s convert it to a DataFrame and have a look:
pd.DataFrame(shap_test.data, columns=shap_test.feature_names,              index=X_test.index)
So, it’s just a copy of the dataset we passed on to.
The most important attribute from shap_test is the values attribute. This is because we can access the shap values from it. Let’s convert the shap values into a DataFrame for easier manipulation:
shap_df = pd.DataFrame(shap_test.values,                        columns=shap_test.feature_names,                        index=X_test.index)shap_df
We can see the shap values by feature for each record. If we add these shap values to the expected value, we will get the prediction:
Let’s check if this is the case:
np.isclose(model.predict(X_test),            explainer.expected_value[0] + shap_df.sum(axis=1))
Awesome! Here, we used np.isclose() to ignore floating point inaccuracy. Now, we have the shap values, we can do custom visualisations like this one to understand the feature contribution:
columns = shap_df.apply(np.abs).mean()\                 .sort_values(ascending=False).indexfig, ax = plt.subplots(1, 2, figsize=(11,4))sns.barplot(data=shap_df[columns].apply(np.abs), orient='h',             ax=ax[0])ax[0].set_title("Mean absolute shap value")sns.boxplot(data=shap_df[columns], orient='h', ax=ax[1])ax[1].set_title("Distribution of shap values");
The left subplot shows the mean absolute shap value for each feature whereas the right subplot shows the distribution of the shap values by feature. One takeaway from these graphs is that bmi has the biggest contribution among the 4 features being used.
While we can construct our own visualisations using the shap values, the shap package comes with built-in fancy visualisations. In this section, we will familiarise with a selected few of these visualisations. We are going to look at two main kinds of plots:🌳 Global: Plots visualising feature’s overall contribution. This kind of plot shows an aggregated contribution of a feature over the entire data. 🍀 Local: Plots showing feature’s contribution in a particular instance. This helps us drill down into individual predictions.
For the left subplot shown earlier, there’s an equivalent built-in function which takes only a few keystrokes:
shap.plots.bar(shap_test)
This simple but useful plot shows the strength of features’ contribution. The plot is based on the mean absolute shap values by features: shap_df.apply(np.abs).mean(). Features are ranked from top to bottom where feature with the highest average absolute shap value is shown at the top.
Another useful plot is summary plot:
shap.summary_plot(shap_test)
Here’s an alternative syntax: shap.plots.beeswarm(shap_test) for this exact plot. Just like before, features are sorted by their mean absolute shap value. This chart is more complicated and packed with more information compared to the previous bar plot. Here’s a guide to interpret this plot:◼️ Horizontal axis of the plot shows the shap value distribution of a feature. Each dot represents a record in the dataset. For instance, we can see that for bmi, dots are quite scattered and there are barely any dots around 0 whereas dots are clustered closer to 0 for age.◼️ The colour of dots shows the feature values. This addition dimension allows us to see the how the shap values changes as the feature value changes. In other words, we can see the direction of the relationship. For instance, we can see that shap value tend to be higher when bmi is high (represented by hot pink dots) and lower when bmi is low (represented by blue dots). There are also some purple points scattered across the spectrum as well.
If we find the default colourmap to be not intuitive or not suitable, we can change it to our preferred matplotlib colourmap like this:
shap.summary_plot(shap_test, cmap=plt.get_cmap("winter_r"))
In addition, there are different plot types we can use. Here’s an example:
shap.summary_plot(shap_test, plot_type='violin')
Heatmap is another way to visualise the shap values. Instead of aggregating shap value to a mean, we see colour-coded individual values. Features are plotted on y-axis and records are plotted on x-axis:
shap.plots.heatmap(shap_test)
This heatmap is supplemented by a line plot of the predicted value (i.e. f(x)) for each record at the top.
We can change the colourmap to our desired one with the cmap parameter:
shap.plots.heatmap(shap_test, cmap=plt.get_cmap("winter_r"))
This interactive plot allows us to see shap value make-up by record:
shap.force_plot(explainer.expected_value, shap_test.values,                 X_test)
Just like heatmap, x-axis shows each record. Positive shap values are shown in red and and negative ones are in blue. For instance, since the first record had more red contributions than blue ones, the prediction for this record would be higher than the expected value.
The interactivity allows us to change both axes. For instance, y-axis shows predictions, f(x) and x-axis is sorted by the output (prediction) value in the snapshot above.
Now we will look at plots for understanding predictions for individual cases. Let’s start with a bar plot:
shap.plots.bar(shap_test[0])
The syntax is exactly same as in section ‘2.1. Global | Bar plot’ except this time we sliced the data for a single record.
This is another alternative to the bar plot:
class WaterfallData():    def __init__ (self, shap_test, index):        self.values = shap_test[index].values        self.base_values = shap_test[index].base_values[0]        self.data = shap_test[index].data        self.feature_names = shap_test.feature_namesshap.plots.waterfall(WaterfallData(shap_test, 0))
The waterfall plot is information-dense and there are four bits of information:◼️ On the y-axis, we see the actual feature value for the record. If you are not sure what I mean, compare X_test.head(1) to the values on the y-axis. ◼️ At the bottom right corner of the chart, we see E[f(X)], the expected value.◼️ At the top left corner, we see f(x), the predicted value.◼️ ️Just like the previous bar plot, horizontal bars represent the colour-coded feature contribution. Starting from the expected value at the bottom, we can see how each contribution is moving the prediction up and down to finally arrive at the predicted value.
Last plot to familiarise is the force plot for a single record. If we rotate this plot 90 degrees and plot it for multiple records, we will see the global force plot.
shap.plots.force(shap_test[0])
We can see both the base value:153.4 and the prediction: 103.48. We can also see the breakdown of the feature contributions.
We have so far focused a regression example. In this section, we’ll learn one way to tweak what we learned to suit a binary classification. Let’s import the subset of titanic data and train a simple model:
# Import sample datadf =  sns.load_dataset('titanic')df['is_male'] = df['sex'].map({'male': 1, 'female': 0}) # Encode# Keep numerical complete columnsdf = df.select_dtypes('number').dropna() X = df.drop(columns=['survived'])y = df['survived']# Partition dataX_train, X_test, y_train, y_test = train_test_split(X, y,                                                     test_size=0.2,                                                     random_state=1)print(f"Training features shape: {X_train.shape}")print(f"Training target shape: {y_train.shape}\n")print(f"Test features shape: {X_test.shape}")print(f"Test target shape: {y_test.shape}")display(X_train.head())# Train a simple modelmodel = RandomForestClassifier(random_state=42)model.fit(X_train, y_train)
We will create an explainer object and extract shap values like before:
explainer = shap.Explainer(model)shap_test = explainer(X_test)print(f"Length of shap_test: {len(shap_test)}\n")print(f"Sample shap_test:\n{shap_test[0]}")
Just like regression shap_test.data will contain the same number of records as the X_test. However, from this example record, we can see that the dimension of values and base_values are different.
Let’s first look at the base value. The base value now tells us the probability for each class. We will focus on the positive class (i.e. y==1, survival):
print(f"Expected value: {explainer.expected_value[1]:.2f}")print(f"Average target value (training data): {y_train.mean():.2f}")print(f"Base value: {np.unique(shap_test.base_values)[0]:.2f}")
Now, let’s look at the shap values. Shap values are also provided for both classes. You will also notice that they are the negative of each other. We can extract the shap values for positive class like this:
shap_df = pd.DataFrame(shap_test.values[:,:,1],                        columns=shap_test.feature_names,                        index=X_test.index)shap_df
Let’s double check whether adding the sum of shap values to the expected probability will give the predicted probability:
np.isclose(model.predict_proba(X_test)[:,1],            explainer.expected_value[1] + shap_df.sum(axis=1))
Awesome, now you know how to get the shap values in the unit of probability.
It’s time to tweak the plots we looked at earlier. To avoid repeating, I will show an example for global plots and another for local plots since the other plots can be replicated using the same logic.
Global | Bar plot: Let’s check out features’ overall contribution to predicting the positive class:
shap.plots.bar(shap_test[:,:,1])
Local | Waterfall plot: Let’s look at waterfall plot for the first test record:
shap.plots.waterfall(shap_test[:,:,1][0])
From this plot, we can see how each feature has contributed towards predicted probability of belonging to the positive class: 0.98 for this record.
Before we wrap up, let’s look at an example use-case. We will find the most incorrect examples for the survivors and will try to understand why the model makes incorrect predictions:
test = pd.concat([X_test, y_test], axis=1)test['probability'] = model.predict_proba(X_test)[:,1]test['order'] = np.arange(len(test))test.query('survived==1').nsmallest(5, 'probability')
The probability of survival was 0.03 for the first record. Let’s check out how features contributed to arrive at this prediction:
ind1 = test.query('survived==1')\           .nsmallest(1, 'probability')['order'].values[0]shap.plots.waterfall(shap_test[:,:,1][ind1])
Young third-class male passenger... We can see mostly gender, passenger class and age have pushed down the prediction. Let’s find similar examples in the training data:
pd.concat([X_train, y_train],           axis=1)[(X_train['is_male']==1) &                   (X_train['pclass']==3) &                   (X_train['age']==22) &                   (X_train['fare'].between(7,8))]
All similar training examples actually didn’t survive. Now, that makes sense! This was a small use-case of how shap can be helpful in unravelling why the model comes up with an incorrect prediction.
Hope you enjoyed learning how to use the SHAP library to understand both regression and classification models built in Scikit-learn. Since SHAP is model-agnostic, you can also use it for other models too. I hope you will get to use this explainable AI tool soon to understand why the model comes up with its predictions in order to improve the model and explain it to others better.
Would you like to access more content like this? Medium members get unlimited access to any articles on Medium. If you become a member using my referral link, a portion of your membership fee will directly go to support me.
Thank you for reading this article. If you are interested, here are links to some of my other posts:◼️️ K-Nearest Neighbours explained◼️️ Logistic regression explained◼️️ Comparing Random Forest and Gradient Boosting◼️️ How are decision trees built?◼️️ Pipeline, ColumnTransformer and FeatureUnion explained◼️️ FeatureUnion, ColumnTransformer & Pipeline for preprocessing text data
Bye for now 🏃 💨