InterpretML: Analysis of SVM and XGBoost models | by Michael Grogan | Towards Data Science

InterpretML by Microsoft is designed with the aim of expanding interpretability of machine learning models. In other words, make those models easier to understand and ultimately facilitate human interpretation.
Microsoft’s Interpret-Community is an extension of this repository, which includes additional interpretability techniques.
In particular, one useful feature is what is called the MimicExplainer. This is a type of global surrogate model that allows for interpretability of any black box model.
In this example, the MimicExplainer is used in interpreting regression models built using SVM (support vector machines) and XGBRegressor (XGBoost for regression problems).
Specifically, these two models are used as follows:
SVM is used for predicting the average daily rate of a customer using specified features, such as their country of origin, market segment, among others. Original findings are available here.XGBRegressor is used as a time series regression model to predict the number of weekly cancellations by regressing a lagged series against the actual, i.e. 5 lagged series with sequential lags of up to t-5 are used as features in the model to predict the cancellation value at time t. Original findings are available here.
SVM is used for predicting the average daily rate of a customer using specified features, such as their country of origin, market segment, among others. Original findings are available here.
XGBRegressor is used as a time series regression model to predict the number of weekly cancellations by regressing a lagged series against the actual, i.e. 5 lagged series with sequential lags of up to t-5 are used as features in the model to predict the cancellation value at time t. Original findings are available here.
The original data is available from Antonio, Almeida and Nunes (2019): Hotel booking demand datasets.
For the purposes of demonstrating how MimicExplainer works, the original models and results are illustrated — with further information on how MimicExplainer can make such results more interpretable.
In order to predict the average daily rate (or rate that a customer pays on average per day) for a hotel booking, an SVM model is constructed with the following features:
Cancellation (whether a customer cancels their booking or not)
Country of Origin
Market Segment
Deposit Type
Customer Type
Required Car Parking Spaces
Week of Arrival
The model is trained as follows:
>>> from sklearn.svm import LinearSVR>>> svm_reg = LinearSVR(epsilon=1.5)>>> svm_reg.fit(X_train, y_train)LinearSVR(C=1.0, dual=True, epsilon=1.5, fit_intercept=True,intercept_scaling=1.0, loss='epsilon_insensitive', max_iter=1000, random_state=None, tol=0.0001, verbose=0)>>> predictions = svm_reg.predict(X_val)>>> predictionsarray([100.75090575, 109.08222631,  79.81544167, ...,  94.50700112,        55.65495607,  65.5248653 ])
When this model was validated against a test set, an RMSE (root mean squared error) of 44.6 was obtained, along with an MAE (mean absolute error) of 29.5. With a mean ADR of 105, the model does show a degree of predictive power in estimating ADR values by customer.
Here is how we can use MimicExplainer for further interpretability of these results.
from interpret.ext.blackbox import MimicExplainerfrom interpret.ext.glassbox import LinearExplainableModelexplainer = MimicExplainer(svm_reg,                            X_train,                            LinearExplainableModel)
The MimicExplainer is the black box model, while the LinearExplainableModel is being used as the global surrogate to this black box model.
Note that when running the model, it is possible to set augment_data = True if dealing with high-dimensional data, i.e. a situation where the number of columns exceeds the number of rows. This allows for oversampling of initialization samples.
However, given that this is not the case here, i.e. the dataset has many more rows than columns (features) — this will not be invoked.
Next, a global explanation is printed out which features the top K features and their values indicating importance:
global_explanation = explainer.explain_global(X_val)sorted_global_importance_values = global_explanation.get_ranked_global_values()sorted_global_importance_names = global_explanation.get_ranked_global_names()dict(zip(sorted_global_importance_names, sorted_global_importance_values))global_explanation.get_feature_importance_dict()
Here is the output:
{3: 8.81513709127725, 7: 4.9616362270740995, 1: 4.959263897550327, 6: 2.593931464493208, 2: 0.9707145503848649, 5: 0.8455564214901589, 4: 0.505321879369921, 0: 0.0}
From the above, feature number 3 (market segment), 7 (week of arrival), and 1 (booking cancellation) are indicated as the most important features in influencing customer ADR.
That said, what if we wish to isolate certain observations in the validation data? e.g. suppose that we only wish to determine feature importance for a select few customers?
Here is an example.
From the validation set, features of importance and their values are determined for customers ordered 10 to 15.
local_explanation = explainer.explain_local(X_val[10:15])sorted_local_importance_names = local_explanation.get_ranked_local_names()sorted_local_importance_values = local_explanation.get_ranked_local_values()
Here are the identified features of importance and their values:
>>> sorted_local_importance_names[[1, 3, 2, 6,  5, 4, 0, 7], [7, 3, 2, 6, 5, 4, 1, 0], [6, 5, 4, 1, 0, 2, 7, 3], [6, 3,  5, 4, 1, 0, 2, 7], [3, 7, 6, 5, 4, 1, 0, 2]]>>> sorted_local_importance_values[[17.833762274315003,  9.1394041860457, 1.1694515257954607, 0.0, -0.0, -0.0, 0.0,  -1.707920714865971], [9.955928069584559, 9.1394041860457,  1.1694515257954607, 0.0, -0.0, -0.0, 0.0, 0.0], [0.0, -0.0, -0.0, 0.0,  0.0, -0.5708037209239748, -11.288939359236048, -13.709106279068548],  [19.017733248096473, 9.1394041860457, -0.0, -0.0, 0.0, 0.0,  -0.7448292455959183, -5.040448938994693], [9.1394041860457,  6.623399845455836, 0.0, -0.0, -0.0, 0.0, 0.0, -0.9884649801366393]]
This allows for isolation of feature importance by customer. For instance, feature 1 (booking cancellation) was the most influencing factor for customer 10, whereas feature 7 (week of arrival) was the most influencing factor for customer 11.
As mentioned, an XGBRegressor model was originally used in order to predict the number of weekly cancellations for the hotel in question.
The model was defined as follows:
from xgboost import XGBRegressormodel = XGBRegressor(objective='reg:squarederror', n_estimators=1000)model.fit(X_train, Y_train)
Here are the defined model parameters:
As we can see from the above, there are numerous model parameters that could be modified in training the XGBRegressor. However, n_estimators is set to 1000 in this instance. This defines the number of trees in the XGBoost model.The objective is set to ‘reg:squarederror’, i.e. a regression with squared loss which penalizes errors on extreme values more heavily.
The RMSE yielded across the validation set was 50.14 along with an MAE of 38.63, which was lower than the mean value of 109 yielded across this set.
Using MimicExplainer, the interpretability of the model results can now be generated in a similar manner to the previous example:
from interpret.ext.blackbox import MimicExplainerfrom interpret.ext.glassbox import LinearExplainableModelexplainer = MimicExplainer(model,                            X_train,                            LinearExplainableModel)
For this purpose, we can identify the features that are the most important in determining the weekly hotel cancellation value at time t. However, given that this is a time series — the model is effectively indicating which lag value is most informative in predicting the cancellation for a particular week, e.g. is a 1-week lag, or a 3-week lag of most importance in influencing cancellations at time t?
global_explanation = explainer.explain_global(X_val)sorted_global_importance_values = global_explanation.get_ranked_global_values()sorted_global_importance_names = global_explanation.get_ranked_global_names()dict(zip(sorted_global_importance_names, sorted_global_importance_values))global_explanation.get_feature_importance_dict()
Here are the results:
{3: 10.580821439553645, 2: 7.795196757642633, 1: 4.973377270096975, 4: 2.329894438847138, 0: 1.382442979985477}
From the above, it is indicated that a lag for feature 3 (t-4) is of most importance when forecasting weekly hotel cancellations at time t.
However, what if we wish to isolate certain weeks?
Taking the validation set, let us isolate the weeks across observations 8 to 13 (which corresponds to the year 2017, weeks 13 to 18).
local_explanation = explainer.explain_local(X_val[8:13])sorted_local_importance_names = local_explanation.get_ranked_local_names()sorted_local_importance_values = local_explanation.get_ranked_local_values()
Here are the findings:
>>> sorted_local_importance_names[[4, 0, 2, 1, 3], [3, 4, 1, 0, 2], [3, 4, 1, 2, 0], [3, 2, 0, 4, 1], [2, 3, 4, 0, 1]]>>> sorted_local_importance_values[[1.2079747395122986, 0.655418467841234, -2.068988871651338,  -5.677678197831921, -16.414222669030814], [5.880283637250306,  1.524120206263528, 1.157876862894971, -1.807324914948728,  -11.282397723111108], [7.472748373413244, 7.056665874410039,  6.562734352772048, 3.8926286204696896, 0.353858053622055],  [35.340881256264645, 4.976559073582604, 2.0627004008640695,  1.1289383728244913, -2.3393838658490202], [23.945342003058602,  5.4821674532095725, 1.287011106200106, -0.7518634651816014,  -2.975249452893382]]
Identifying the features of importance across the time series in this manner can potentially yield useful insights as to the importance of lagged values in predicting cancellations across different weeks. For instance, we can see that in predicting cancellations for week 13, the lag at t-5 (feature 4) was identified as the most important feature.
However, when predicting cancellations for week 18, the lag at t-3 (feature 2) was identified as most important.
As opposed to using a one size fits all forecasting model, interpretability can allow for identification of suitable forecasting methods for a particular point in time, and in turn allow for customisation of time series forecasting methods across different periods.
In this example, we have seen how interpretability can yield valuable insights when analysing machine learning output — which would be much more difficult when simply using black box ML models in their own right.
Specifically, we have seen:
How MimicExplainer can be used as a global surrogate model to allow for interpretability of any black box model
Use of MimicExplainer across regression problems
How to interpret the results from MimicExplainer and gauge feature importance
Many thanks for your time, and any questions or feedback are greatly appreciated. The relevant Jupyter Notebooks for the above examples can be found here.
Disclaimer: This article is written on an “as is” basis and without warranty. It was written with the intention of providing an overview of data science concepts, and should not be interpreted as any sort of professional advice. The findings are that of the author, and are not affiliated with any third-party mentioned above.