Explaining Your Machine Learning Models with SHAP and LIME! | by David Hundley | Towards Data Science

Hello there all! Welcome back again to another data science quick tip. This particular post is most interesting for me not only because this is the most complex subject we’ve tackled to date, but it’s also one that I just spent the last few hours learning myself. And of course, what better way to learn than to figure out how to teach it to the masses?
Before getting into it, I’ve uploaded all the work shown in this post to a singular Jupyter notebook. You can find it at my personal GitHub if you’d like to follow along more closely.
So even though this is a very complex topic behind the scenes, I’m going to intentionally dial it down as much as possible for the widest possible audience. Even though this is ultimately a post designed for data science practitioners, I think it’s equally important for any business person to understand why they should care about this topic.
Prior to jumping into how to calculate/visualize these values, let’s build some intuition on why would we would even care about this topic.
If you’re familiar at all with machine learning in general, you know that you pop some data into a predictive model and that it produces an outputted prediction on the other side. But let’s be honest: do you know what’s really going on inside in that model? Remember, we’re talking about algorithms doing a lot of fancy math under the hood. In reality, this is probably what your ML model looks like to you right now:
You really have no idea what’s going on in that black box. The math totally obscures how important those features are in your model. So what if Features 1 and 2 are carrying all the weight and that Features 3 and 4 are not adding any value at all to the model? What you’d prefer to see is something that looks more like this:
Okay, we’re obviously using layman’s terms in our example here. But this stuff is important because if your models are relying on one 1 or 2 features to be doing all the predictive power, that might indicate something is wrong. It also might indicate that you’re using some sort of feature that may be considered to be morally incorrect. For example, many companies today will steer clear of using things like gender or ethnicity within their models because they tend to produce a lot of undesired bias in a model.
Understanding these concepts always makes more sense through a tangible example, so we’re going to return to the dataset we’ve been using the last few posts to concretely demo both SHAP and LIME: the Titanic dataset.
Alright, so in the last few post where we’ve used the Titanic dataset, we created models that were pretty poor since our focus was more on learning a new skill. In order to effectively demo both SHAP and LIME, we really need a model that shows some level of “trying” (?) so that we can see how ML explainability will play out. Now, I’m not going to spend time in this post showing you all my code for that data cleansing and modeling piece (which you can definitely find in my GitHub here), but I’ll summarize quickly the features that go into producing the end model.
PClass: Notes the ticket class the person boarded the ship with. (Either 1st, 2nd, or 3rd.)
Age Bins: Based on the person’s age, they are segmented into general people groups. These groups include child, teen, young adult, adult, elder, and unknown.
Sex (Gender): Notes whether the person is male or female.
Cabin Section: If the person had a cabin, I pulled out what section they might be in. These can either be A, B, C, or no cabin.
Embarked: Notes one of three locations from which the person originally departed.
SibSp (Sibling / Spouse): Notes how many siblings and/or spouses the person had onboard the ship with them.
Parch (Parent / Child): Notes how many parents / children the person had onboard the ship with them.
For the actual model itself, I’m choosing to use a simple Random Forest Classifier from Scikit-Learn. Splitting the data into training and validation sets, I get the following metrics with the validation set:
The metrics aren’t exactly great, but that’s fine for our learning here. At least we’ll get to see in the next section how important these features necessarily are. Let’s move on!
At this point, we currently have no idea how our Random Forest model is using the features to make its predictions. I’m going to wager a guess that Sex and Age Bin are influential factors, but we don’t know that quite yet.
This is where smart people developed complex algorithms based on game theory to generate these things to better explain our current black box. Of course, I’m referring to SHAP and LIME. Now, I’m going to be 100% honest: I’m not very familiar with how these algorithms work under the hood, so I’m not even going to attempt to do that in this post. Instead, I’ll point you to this post about SHAP and this other post about LIME.
Just like Scikit-Learn abstracts away the underlying algorithms for our Random Forest classifier, there are some neat Python libraries that we’ll use that abstract away the inner workings of both SHAP and LIME. To use those, all you need to do is a simple pip install for both:
pip install shappip install lime
At a high level, the way both of these work is that you give your training data and model to an “explainer”, and then you’re later able to pass any observation to the “explainer” which will tell you the feature importance of that model. That might sound like gobbledegook right now, but we’ll make this extra clear with our Titanic example!
First, we need to pull two people from our validation set that we know respectively did not survive and survive. The reason for this is because we’re going to run these through our SHAP and LIME explainers to understand which characteristics about these people had the most influence for survivability according to our model. (Again, you can see this work in my notebook.)
Let’s start off with SHAP. The syntax here is pretty simple. We’ll first instantiate the SHAP explainer object, fit our Random Forest Classifier (rfc) to the object, and plug in each respective person to generate their explainable SHAP values. The code below shows you how to do it for person 1. To do the same for person 2, simply swap out the appropriate values.
# Importing the SHAP libraryimport shap# Instantiating the SHAP explainer using our trained RFC modelshap_explainer = shap.TreeExplainer(rfc)shap.initjs()# Getting SHAP values for person 1person_1_shap_values = shap_explainer.shap_values(person_1)# Visualizing plot of expected survivability of person 1shap.force_plot(shap_explainer.expected_value[1], person_1_shap_values[1], person_1)# Visualizing impact of each feature for person 1shap.summary_plot(person_1_shap_values, person_1)
And that’s really all there is to it! Let’s take a look at the actual output from what we just did above in the screenshots below. First up, person 1.
According to the top visual, our model predicted a 94% chance that person 1 survived, which is correct. It’s a little difficult to read, but look at the second visual, it looks like the most influential factors included the gender of the person, whether or not the person had a 3rd class ticket, and if the person was a child. In this case, person 1 was a female child, so those two factors alone drove our model most to say yes, this person probably survived.
Let’s look at person 2 now.
Our model here predicted that the person probably did not survive, and it is unfortunately correct. Again looking at the influential factors here, it looks like person 2 here was an adult male that did not have a cabin. I’m no history buff, but I would guess women and children were rescued off the boat first, so it makes a lot of sense to me that person 1 (a female child) did survive and person 2 (a male adult) did not.
Alright, let’s move onto our LIME explainer!
The syntax for using LIME is a little bit different, but conceptually speaking, both SHAP and LIME are doing a lot of similar stuff.
# Importing LIMEimport lime.lime_tabular# Defining our LIME explainerlime_explainer = lime.lime_tabular.LimeTabularExplainer(X_train.values, mode = 'classification', feature_names = X_train.columns, class_names = ['Did Not Survive', 'Survived'])# Defining a quick function that can be used to explain the instance passedpredict_rfc_prob = lambda x: rfc.predict_proba(x).astype(float)# Viewing LIME explainability for person 1person_1_lime = lime_explainer.explain_instance(person_1.iloc[0].values, predict_rfc_prob, num_features = 10)person_1_lime.show_in_notebook()
Let’s take a look at what LIME shows us for each person.
I’m a little biased here, but I think the UI for these LIME values is a lot easier to read that SHAP. We see the same prediction probabilities for whether each person did or did not survive. In the center, we see how influential the top 10 features were. Compared to SHAP, LIME has a tiny difference in its explainability, but they’re largely the same. We again see that Sex is a huge influencing factor here as well as whether or not the person was a child. On the right, they nicely display the precise values for this particular observation.
So which one is better: SHAP or LIME? I’m going to say neither. While each complement each other quite well, I think it’s important to show your business users both just to give them the best picture on which features are most explainable. I don’t think anybody can definitively say either SHAP or LIME is always more accurate, so show them both. The syntax is easy enough to chug out, anyway!
That wraps up another post! Hope you all enjoyed this one. I’d say this one has been my favorite to learn and write about so far, and I look forward to applying this in my own work moving forward. Catch you all in the next post!