6 Top Python Packages for Machine Learning Interpretability | by Cornellius Yudha Wijaya | Towards Data Science

If you enjoy my content and want to get more in-depth knowledge regarding data or just daily life as a Data Scientist, please consider subscribing to my newsletter here.
Developing a machine learning model is something that expected from any data scientist. I have come across many data science study that focuses only on the modelling aspect and the evaluation without the interpretation.
However, many haven’t realized the importance of machine learning interpretability in the business process. In my experience, business people would want to know how the model works rather than the metric evaluation itself.
That is why, in this post, I want to introduce you to some of my top python package for machine learning interpretability. Let’s get into it!
Yellowbrick is an open-source Python package that extends the scikit-learn API with visual analysis and diagnostic tools. For Data Scientist, Yellowbrick is used to evaluate the model performance and visualize the model behaviour.
Yellowbrick is a multi-purpose package that you could use in your everyday modelling work. Even though most of the interpretation API from the Yellowbrick is at the basic level, it is still useful for our first modelling steps.
Let’s try the Yellowbrick package with a dataset example. For starter, let’s install the package.
pip install yellowbrick
After the installation is done, we could use the dataset example from Yellowbrick to test the package.
#Pearson Correlationfrom yellowbrick.features import rank2dfrom yellowbrick.datasets import load_creditX, _ = load_credit()visualizer = rank2d(X)
With one line, we able to visualize the correlation between the features using the Pearson correlation method. It is customizable so that you could use another correlation function.
Let’s try to develop the model to evaluate the model performance and interpret the model. I would use the example dataset from the Yellowbrick user guide and produce a discrimination threshold plot to find the best threshold that separates the binary classes.
from yellowbrick.classifier import discrimination_thresholdfrom sklearn.linear_model import LogisticRegressionfrom yellowbrick.datasets import load_spamX, y = load_spam()visualizer = discrimination_threshold(LogisticRegression(multi_class="auto", solver="liblinear"), X,y)
Using the Yellowbrick threshold plot, we could interpret that the model performs the best at the 0.4 probability threshold.
If you are curious what Yellowbrick could do, visit the homepage for more info.
ELI5 is a Python package that helps with machine learning interpretability. Taken from the Eli5 package, the basic usage of this package is to:
inspect model parameters and try to figure out how the model works globally;inspect an individual prediction of a model and figure out why the model makes the decision.
inspect model parameters and try to figure out how the model works globally;
inspect an individual prediction of a model and figure out why the model makes the decision.
If Yellowbrick focuses on the features and model performance interpretation, ELI5 focus on the model parameters and the prediction result. Personally, I like ELI5 better because its interpretation is simple enough that business people could understand it.
Let’s try the ELI5 package with a sample dataset and random forest model classifier. I would use the dataset from the seaborn package because it is the simplest one.
#Preparing the model and the datasetfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.model_selection import train_test_splitmpg = sns.load_dataset('mpg').dropna()mpg.drop('name', axis =1 , inplace = True)#Data splittingX_train, X_test, y_train, y_test = train_test_split(mpg.drop('origin', axis = 1), mpg['origin'], test_size = 0.2, random_state = 121)#Model Trainingclf = RandomForestClassifier()clf.fit(X_train, y_train)
The most basic ELI5 function is to show the classifier weight and the classifier prediction result. Let’s try both functions to understand how the interpretation is coming from.
import eli5eli5.show_weights(clf, feature_names = list(X_test.columns))
From the above image, you could see that the classifier showing the classifier feature importance with their deviation. You could see that the displacement feature is the most important feature, but they have a high deviation, indicating bias within the model. Let’s try to show the prediction result interpretability.
eli5.show_prediction(clf, X_train.iloc[0])
Using the show prediction function by ELI5, we could get the feature contribution information. What features contribute to certain prediction results, and how much is the probability shifting by these features. This is a good function for you to interpret the model prediction easily to the business people.
However, it would be best to remember that the function above is based on the tree interpretation (because we use the Random Forest model). It might be good enough to give you an interpretation for the business people; however, it might be biased because of the model. That is why ELI5 offer another way to interpret the Black-box model based on the model metric — It is called Permutation Importance.
Let’s try the Permutation Importance function as a starter.
#Permutation Importanceperm = PermutationImportance(clf, scoring = 'accuracy',random_state=101).fit(X_test, y_test)show_weights(perm, feature_names = list(X_test.columns))
The idea behind permutation importance is how the scoring (accuracy, precision, recall, etc.) shift with the feature existence or without it. In the above result, we can see that displacement has the highest score with 0.3013. When we permute the displacement feature, it will change the model's accuracy as big as 0.3013. The value after the plus-minus sign is the uncertainty value. The permutation Importance method is inherently a random process; that is why we have the uncertainty value.
The higher the position, the more critical the features are affecting the scoring. Some feature in the bottom place shows a minus value, which is interesting because it means that the feature increases the scoring when we permute the feature. This happens because, by chance, the feature permutation actually improves the score.
Personally, ELI5 has given me enough machine learning interpretability for the business people, but there is still some Python package that I want to introduce to you.
It will not complete if we are talking about Machine Learning Interpretability without mentioning SHAP. For you who never heard about it, SHAP or (SHapley Additive exPlanations) is a game-theoretic approach to explain the output of any machine learning model. In a simpler term, SHAP using the SHAP values to explain the importance of each feature. SHAP use the SHAP values difference between the prediction of the model and the null model developed. SHAP is model agnostic, similar to the Permutation Importance, so it is useful for any model.
Let’s try using the sample dataset and model to explain SHAP in more detail. First, we need to install the SHAP package.
#Installation via pippip install shap #Installation via conda-forge conda install -c conda-forge shap
In this sample, I would use the titanic sample dataset and only rely on the numeric columns. This is only for sample purposes and should not become the standard for data analysis.
#Preparing the model and the dataset from sklearn.ensemble import RandomForestClassifierfrom sklearn.model_selection import train_test_split  titanic = sns.load_dataset('titanic').dropna() titanic = titanic[['survived', 'age', 'sibsp', 'parch']]  #Data splitting for rfc X_train, X_test, y_train, y_test = train_test_split(titanic.drop('survived', axis = 1), titanic['survived'], test_size = 0.2, random_state = 121)  #Model Training clf = RandomForestClassifier() clf.fit(X_train, y_train)
We already trained our data with the titanic data, and now we could try to interpret the data with SHAP. Let’s use the global interpretability of the model to understand how SHAP work.
import shap shap_values = shap.TreeExplainer(clf).shap_values(X_train) shap.summary_plot(shap_values, X_train)
As we can see from the result, we could interpret that the age feature contributes the most to the prediction result. If you want to look at the specific class contribution to the prediction, we only need to tweak the code a little. Let’s say we want to look at class 0, which means we use the following code.
shap.summary_plot(shap_values[0], X_train)
From the image above, we could see the contribution of each data into the prediction probability. The redder the colour, the higher the value and vice versa. Also, when the value is on the positive side, it contributes to the class 0 prediction result probability and vice versa.
SHAP is not limited to global interpretability; it also gives you the function to interpret individual dataset. Let’s try to interpret the prediction result for the first row.
explainer = shap.TreeExplainer(clf)shap_value_single = explainer.shap_values(X = X_train.iloc[0,:])shap.force_plot(base_value = explainer.expected_value[1],                shap_values = shap_value_single[1],                features = X_train.iloc[0,:])
As you can see from the image above, the prediction is closer to class 0 because it is pushed by the age and sibsp feature and only a little contribution by the parch feature.
Mlxtend or machine learning extensions is a Python package for data science everyday work life. The APIs within the package is not limited to interpretability but extend to various functions, such as statistical evaluation, Data Pattern, Image Extraction, and many more. However, we would discuss the API for interpretability for our current article — the Decision Regions plotting.
The Decision Regions plot API would produce a decision region plot to visualize how the feature decides the classification model prediction. Let’s try using sample data and a guide from Mlxtend.
First, we need to install the Mlxtend package.
pip install Mlxtend
Then we use the sample dataset and develop a model to see the Mlxtend in action.
import numpy as npimport matplotlib.pyplot as pltimport matplotlib.gridspec as gridspecimport itertoolsfrom sklearn.linear_model import LogisticRegressionfrom sklearn.svm import SVCfrom sklearn.ensemble import RandomForestClassifierfrom mlxtend.classifier import EnsembleVoteClassifierfrom mlxtend.data import iris_datafrom mlxtend.plotting import plot_decision_regions# Initializing Classifiersclf1 = LogisticRegression(random_state=0)clf2 = RandomForestClassifier(random_state=0)clf3 = SVC(random_state=0, probability=True)eclf = EnsembleVoteClassifier(clfs=[clf1, clf2, clf3], weights=[2, 1, 1], voting='soft')# Loading some example dataX, y = iris_data()X = X[:,[0, 2]]# Plotting Decision Regionsgs = gridspec.GridSpec(2, 2)fig = plt.figure(figsize=(10, 8))for clf, lab, grd in zip([clf1, clf2, clf3, eclf],                         ['Logistic Regression', 'Random Forest', 'RBF kernel SVM', 'Ensemble'],                         itertools.product([0, 1], repeat=2)):    clf.fit(X, y)    ax = plt.subplot(gs[grd[0], grd[1]])    fig = plot_decision_regions(X=X, y=y, clf=clf, legend=2)    plt.title(lab)plt.show()
From the above plot, we could interpret the decision made by the model. You could see the differences between each model when they make the prediction. For example, the Logistic Regression model prediction result for class 1 is getting bigger the higher the X-axis value is but didn’t have too much change on the Y-axis. It contrasts with the Random Forest model, where the division is not changing much following the X-axis value, and the Y-axis value seems constant for every prediction.
The only weakness for the decision region is that it is limited to 2 Dimension features, so it is more useful for pre-analysis than the actual model itself. However, it is still useful for the conversation with the business people.
PDP or Partial Dependence Plot is a plot that shows the marginal effect of features on the predicted outcome of the machine learning model. It is used to evaluate whether the correlation between the feature and target is linear, monotonic, or more complex.
The advantage of interpreting with a Partial Dependence plot is easy to interpret for business people. This is because the partial dependence function computation is intuitive enough for people to understand: The calculation for the partial dependence plots has a causal interpretation when we intervene on a feature, and we measure the changes in the predictions; this is when we could measure the interpretation.
Let’s try using the sample data from the user guide to understand the PDPBox even more. First, we need to install the PDPBox package.
pip install pdpbox
Then we could try to follow the user guide for more information on how PDPBox helping us create interpretable Machine Learning.
import pandas as pdfrom pdpbox import pdp, get_dataset, info_plots#We would use the data and model from the pdpboxtest_titanic = get_dataset.titanic()titanic_data = test_titanic['data']titanic_features = test_titanic['features']titanic_model = test_titanic['xgb_model']titanic_target = test_titanic['target']
When we have our data and model, let’s try one of the functions to check the information between the feature and the target using the info plots function.
fig, axes, summary_df = info_plots.target_plot(    df=titanic_data, feature='Sex', feature_name='gender', target=titanic_target)_ = axes['bar_ax'].set_xticklabels(['Female', 'Male'])
You could get the statistical information of the target and the feature with one function. It is easy to explain to the businesspeople using this function. Let’s check the model prediction distribution function along with the features.
fig, axes, summary_df = info_plots.actual_plot(    model=titanic_model, X=titanic_data[titanic_features], feature='Sex', feature_name='gender')
Now, let’s move on to use the PDP plotting function to interpret our model prediction.
pdp_sex = pdp.pdp_isolate(    model=titanic_model, dataset=titanic_data, model_features=titanic_features, feature='Sex')fig, axes = pdp.pdp_plot(pdp_sex, 'Sex')_ = axes['pdp_ax'].set_xticklabels(['Female', 'Male'])
From the above image, we could interpret that the prediction probability is decreased when the sex feature is Male (means that Male less likely to survived). This is how we use PDPbox to have model interpretability.
InterpretML is a Python Package that includes many Machine Learning Interpretability API. The purpose of this package is to give you an interactive plot based on plotly to understand your prediction result.
InterpretML offers you many ways to interpret your Machine Learning (Globally and Locally) by using many of the techniques we have discussed — namely SHAP and PDP. Also, this package owns a Glassbox model API which gives you an interpretability function when you develop your model.
Let’s try this package with a sample dataset. First, we need to install the InterpretML.
pip install interpret
I want to give you an example by using the glass box model that InterpretML because, for me personally is interesting to use. Let’s try to develop the model by using the titanic dataset sample.
from sklearn.model_selection import train_test_splitfrom interpret.glassbox import ExplainableBoostingClassifierimport seaborn as sns#the glass box model (using Boosting Classifier)ebm = ExplainableBoostingClassifier(random_state=120)titanic = sns.load_dataset('titanic').dropna()#Data splittingX_train, X_test, y_train, y_test = train_test_split(titanic.drop(['survived', 'alive'], axis = 1),                                                     titanic['survived'], test_size = 0.2, random_state = 121)#Model Trainingebm.fit(X_train, y_train)
Using the glass box model from the InterpretML, it would automatically One Hot Encoding your features and feature engineering the interaction feature for you. Let’s try to get a global explainable for this model.
from interpret import set_visualize_providerfrom interpret.provider import InlineProviderset_visualize_provider(InlineProvider())from interpret import showebm_global = ebm.explain_global()show(ebm_global)
From the image above, we could see the summary of the model feature importance. It shows you all the feature that deemed important based on the model feature importance.
The explainable is an interactive plot that you could use to explain the model more specifically. If we only see the summary in the above image, we could select another component to specify the feature you want to see. In this way, we could explain how the feature in the model affecting the prediction.
In the image above, we could see that low fare decreasing the chance of survival, but as the fare getting higher, it increases the survival chance. However, you could see the density and the bar chart — many people are coming from a lower fare.
Often time, we are not only interested in the Global interpretability but also the Local. In this case, we could use the following code to interpret it.
#Select only the top 5 rows from the test dataebm_local = ebm.explain_local(X_test[:5], y_test[:5])show(ebm_local)
Local interpretability shows how a single prediction is made. According to the user guide, the values shown here are log-odds scores from the model, which are added and passed through a logistic-link function to get the final prediction. We could see the sex male giving the most contribution to lowering the survival chance in this prediction.
Machine Learning Interpretability is an important tool for any data scientist because it allowed you to communicate your result better to the business user.
In this post, I already outline another 6ML interpretability Python Package:
YellowbrickELI5SHAPMlxtendPDPBoxInterpretML
Yellowbrick
ELI5
SHAP
Mlxtend
PDPBox
InterpretML
I hope it helps!
Visit me on my LinkedIn or Twitter.
If you are not subscribed as a Medium Member, please consider subscribing through my referral.