5 Explainable Machine Learning Models You Should Understand | Towards Data Science

As we know, Machine Learning is ubiquitous in our day to day lives. From product recommendations on Amazon, targeted advertising, and suggestions of what to watch, to funny Instagram filters.
If something goes wrong with these, it probably won’t ruin your life. Maybe you won’t get that perfect selfie, or maybe companies will have to spend more on advertising.
How about facial recognition in law enforcement? Loan or mortgage applications? Driverless vehicles?
In these high-risk applications, we can’t go in blind. We need to be able to dissect our model, we will need to be able to understand and explain our model before it goes anywhere near a production system.
Explainable Machine Learning is essential when we are making decisions about people that can negatively impact their lives such as mortgages or credit scoring.
Using explainable models also allows more efficient debugging as well as better understand of fairness, privacy, causality, and more trust in the model.
Types of Explainability
Explainable Models- Generalised Linear Models- Decision Trees- Generalised Additive Models- Monotonic Gradient Boosting- TabNet
Comparison
Conclusion
Post-hoc: This is when we explain a model after predictions have been made or after the model has been trained. This is great because these methods allow us to explain highly complicated models. However these methods can be fooled, are flawed under specific conditions and require an extra layer of complexity to generate the explanations. Common examples are the SHAP and LIME Python packages.
Inherent: Some models can be explained out of the box, without additional models or libraries laid on top. These are typically simpler and in some cases might have less predictive power — although some researchers argue that this isn’t always the case!
This article will cover inherently explainable models.
Lipton, 2016 (https://arxiv.org/abs/1606.03490) defines model explainability using 3 criteria:
Simulatability: Can a human walk through the model’s steps in a ‘reasonable’ amount of time?Decomposability: Can all aspects of the model, including its features, parameters and weights be broken down?Algorithmic Transparency: Can we understand how the model will react to unseen data.
Simulatability: Can a human walk through the model’s steps in a ‘reasonable’ amount of time?
Decomposability: Can all aspects of the model, including its features, parameters and weights be broken down?
Algorithmic Transparency: Can we understand how the model will react to unseen data.
You may already be aware of models that meet this criteria; Decision Trees and Logistic Regression. As long as the model doesn’t use too many features, both meet all 3 criteria.
Even if you use an explainable model, using too many features or highly engineered features can reduce your model explainability. We need to keep our data and preprocessing explainable too.
However, there are several lesser known explainable models that are great to have in your toolkit. These methods also can have more predictive power than Decision Trees and Logistic Regression, while still maintaining some level of explainability, allowing you to balance explainability with accuracy in your next project.
What is it and how does it work?
A GLM is really just a fancy way of talking about Linear or Logistic Regression. The key concept here is that there are other linear models that make these more flexible.
A GLM has 3 components:
Linear predictor: This is just the regression equation — a linear combination of variables and some predictor variable.
Linear predictor: This is just the regression equation — a linear combination of variables and some predictor variable.
2. Link function: Links our linear combination of variables to a probability distribution. In linear regression, this is just the identity link function.
3. Probability distribution: This is how our y variable is generated. In Linear regression, this is a normal distribution.
By varying these, we can get different models. Using Logit link function with a Bernoulli distribution gets us a logistic regression.
A lesser known version is the Poisson regression, which uses a Poisson distribution. This assumes our linear combination of variables relates to the logarithm of y.
To learn more about Logistic Regression, check out my article here:
towardsdatascience.com
Why is it explainable?
Because its just simple(ish) maths
The below graph shows a simple linear regression equation for just one variable. We know that we can calculate y, our target, for any value of x by using the line. We also know how the line is calculated (by minimising the errors).
We can scale this up to as many variables as we want and the maths still hold. We simply add our terms together to generate a result.
Logistic or Poisson Regression are more complex but the core concept holds true. We are summing a linear combination of variables.
The coefficients mean something
In a linear regression, our coefficients are given in terms of the target variable. This allows us to make statements such as:
Increasing square footage by 50m2 will increase our house price by £1000
In logistic regression, these are in terms of log odds which we can convert to probabilities.
The fact that these coefficients can be converted into human-understandable statements really increases the explainability of linear models.
Interactions have to be programmed explicitly
Tree based models will work out interactions between variables which can increase complexity. Many explainability methods don’t cover interactions well and they can be hard to work out. For example
Increasing square footage by 50m2 will increase our house price by £1000 until we get to 200m2, where our house price increases by £1000 per 50m2 * number of bedrooms.
This type of complex relationship will only be found in a Linear model if it is calculated and programmed in explicitly during feature engineering.
Implementation
scikit-learn offers an implementation of GLMs
Statsmodels also offers a good implementation
What is it and how does it work?
Most people should have seen a Decision Tree at some point in their lives! The algorithmic version uses some simple maths to generate an ‘optimum’ Decision Tree, i.e. the tree which splits our data best.
To understand how it works, take a look here:
towardsdatascience.com
Why is it explainable?
The great thing about Decision Trees is that we can literally just extract the whole tree and follow along why the model made a prediction for any sample in the dataset.
Once trees get very large (max_depth = 7+), they become difficult for humans to follow due to exponential growth in the number of leaves. However at this point we could still write some basic code to highlight the path our data took to reach its prediction, as well as test how the model will react to unseen data.
I find these to be one of the most explainable models due to very little maths and a concept which is found elsewhere in the world.
Implementation
As usual, scikit-learn would be my go-to for this model.
To plot the tree, there are many different options, here is a good list.
What is it and how does it work?
Generalised Additive Models (GAMs) are an extension of GLM’s that drop one of the main limitations; we can now model non-linear relationships in our data.
GAMs do these by using a series of complex functions known as splines to estimate each variable. We still sum up our variables, but splines mean the variables and target value can have a non-linear relationship.
To read more about GAMs, take a look here:
towardsdatascience.com
Why is it explainable?
Undoubtedly, GAMs are less explainable than Logistic or Linear regression. The model is much more complex, as is the maths behind it. However, they still maintain a level to explainability which is a great trade-off when you consider their flexibility.
A combination of non-linear variables?
The target variable is still just the sum of all other variables with some weighting, we now have a complex function modelling each variable. We can still extract and visualise this function per variable and most GAM packages use partial dependence plots to do this for all features.
Interactions have to be programmed manually which limits complexity. We also can understand broadly how the model will behave on unseen data, as we know the spline functions for each feature.
Implementation
From my research, it seems that the mgcv package in R is the best for GAMs. However, I prefer Python; the two best options are Statsmodels and PyGAM.
Microsoft Research have open sourced their InterpretML package which includes their Explainable Boosting Machine, which they term GAM 2.0, as it uses GAMs with automatic interaction terms and gradient boosting to maintain explainability, increase performance and reduce the need for Data Scientists to get to deep into the model.
What is it and how does it work?
Gradient Boosting models are considered the best in class for tabular data, but due to the nature of boosting, they are not interpretable. These models can use hundreds of individual trees with different weights. They also tend to work out interaction terms themselves which we have little transparency around. It is common to use SHAP or LIME to increase interpretability with these models.
A monotonic relationship is where the target and a feature have a linear relationship, for example:
Your BMI increases and your risk of heart attack increases.
Your credit score decreases and your likelihood to get a loan decreases.
The amount of rain increases and the number of bike rented decreases.
Linear models are fully monotonic, but as gradient boosting includes interactions and can model non-linear relationships, they typically do not generate monotonic relationships.
XGBoost, LightGBM and Catboost all have a simple hyperparameter that forces a variable to have a positive or negative monotonic relationship.
Why is it explainable?
Using monotonic relationships means we can use statements like those above to explain our model. It causes the model to fill the Algorithmic Transparency criteria because this relationship is fixed. We can also build some of our real world knowledge into the model which can make it more understandable for businesspeople, giving it more change of reaching production.
Implementation
XGBoost
In XGBoost we specify the monotone_constraints parameter a string tuple (brackets inside speech marks) with one number per feature in our dataset, so “(1,0,-1)” indicates features 1, 2 and 3. 1 is a positive monotonic relationship, -1 is negative and 0 is no relationship.
import xgboost as xgbparams = {'monotone_constraints':'(1,0,-1)'}model = xgb.train(params,                   X_train,                  num_boost_round = 1000,                  early_stopping_rounds = 10)
LightGBM
LightGBM is broadly the same as XGBoost but we are required to pass our features as a list rather than a string/tuple. LightGBM also offers an extra parameter the method. Using this means we can choose how strongly the model will try to stick the constraint.
The documentation states:
basic, the most basic monotone constraints method. It does not slow the library at all, but over-constrains the predictions
intermediate, a more advanced method, which may slow the library very slightly. However, this method is much less constraining than the basic method and should significantly improve the results
advanced, an even more advanced method, which may slow the library. However, this method is even less constraining than the intermediate method and should again significantly improve the results
import lightgbm as lgbparams = {'monotone_constraints': [-1, 0, 1],          'monotone_constraints_method':'basic'}model = lgb.train(params,                   X_train,                   num_round = 1000,                   early_stopping_rounds = 10)
Catboost
Catboost is very similar to the others but offers more flexibility as we can pass the constraints as an array, use slicing and name a feature explicitily.
The parameter is called monotone_constraints and you can check out the Catboost docs here.
What is it and how does it work?
TabNet was published by Google Brain researchers in in 2019. Traditionally, Neural Network approaches have not significantly improved on gradient boosting when dealing with Tabular data. However, Tabnet was able to outperform the leading tree based models across a variety of benchmarks. It is considerably more explainable than boosted tree models as it has built-in explainability. It can also be used without any feature preprocessing.
My article on TabNet covers the model in more detail, take a look here:
towardsdatascience.com
Why is it explainable?
TabNet uses a sequential attention mechanism to select the most important features, this influences the ‘mask’ whichcovers up the least important features. We can use the weights of this mask to understand which features are being used more often than others, which essentially allows us to understand which features the model is using to make its predictions.
Feature selection is performed at the row level of the dataset, which means we can actually explore which features were selected for a single prediction. The number of masks is a hyperparameter of the model
Implementation
The best way to use TabNet is with Dreamquark’s PyTorch implementation. It uses a scikit-learn style wrapper and is GPU compatible. Dreamquark also provide some really great notebooks which perfectly show how to implement TabNet while also working to validate the original authors claims about the models accuracy on certain benchmarks.
Classification
github.com
Regression
github.com
Lets go back to Lipton’s 3 criteria and apply these to each model. As a reminder, the criteria are...
Simulatability: Can a human walk through the model’s steps in a ‘reasonable’ amount of time?Decomposability: Can all aspects of the model, including its features, parameters and weights be broken down?Algorithmic Transparency: Can we understand how the model will react to unseen data.
Simulatability: Can a human walk through the model’s steps in a ‘reasonable’ amount of time?
Decomposability: Can all aspects of the model, including its features, parameters and weights be broken down?
Algorithmic Transparency: Can we understand how the model will react to unseen data.
We also want to consider Local Explainability; to what extend can a single prediction be made by the model, in terms of what features it used and what extent did it use each feature to make its decision.
I have summarised this in the chart below, scoring each model Low, Medium or High for each criteria. This isn’t an exact science but you can consider each score relative to a Linear Regression.
Remember, that any feature engineering can completely throw these scores off. Creating complex interaction terms, mathematical transformations or features derived from a neural network may increase accuracy but will certainly decrease explainability. More features can also reduce explainability.
The drive and discussions around model explainability is only increasing. With AI being used in language modelling, facial recognition and driverless cars, having models that can give reasoning behind their decision is more essential than ever.
Give one of these explainable models a try in your next project and let me know how it turns out.