Visualizing Output Of Machine Learning Model | by Himanshu Sharma | Towards Data Science

Interpreting a machine learning model is a difficult task because we don’t really know how the model is working inside that black box. Interpretation is required so that we can select the best model for our data and also make it robust.
Shap is an open-source python library that is used to explain models. It can create numerous types of visualizations which help in interpreting the model and explaining how the model is working.
In this article, we will see different types of machine learning model visualizations which can be created using Shap.
Let’s get started...
We will start by installing Shap using pip. The command given below will do that.
pip install shap
In this step, we will import the required libraries for loading the data, creating a model, and creating visualizations of that model.
import pandas as pdimport numpy as npimport matplotlib.pyplot as pltimport shapfrom sklearn.model_selection import train_test_splitimport xgboost as xgb
In this step, we will create the machine learning model. For this article, I’m creating an XGBoost model but you can select any model of your choice. The dataset that we will be using for this model is the famous Diabetes dataset which can be downloaded from Kaggle.
df = pd.read_csv('/content/Diabetes.csv')features = ['Pregnancies', 'Glucose','BloodPressure','SkinThickness','Insulin','BMI','DiabetesPedigreeFunction','Age']Y = df['Outcome']X =  df[features]X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2, random_state = 1234)xgb_model = xgb.XGBRegressor(random_state=42)xgb_model.fit(X_train, Y_train)
Now we will create the explainer for shap and find out the shape values for our model and create visualizations using them.
explainer = shap.Explainer(xgb_model)shap_values = explainer(X_test)
Bar Plot
Bar Plot
shap.plots.bar(shap_values, max_display=10)
2. Cohort Plot
shap.plots.bar(shap_values.cohorts(2).abs.mean(0))
3. Heatmap
shap.plots.heatmap(shap_values[1:100])
4. Waterfall Plot
shap.plots.waterfall(shap_values[0]) # For the first observation
5. Force Plot
For this plot, we need to initiate “js” also.
shap.initjs()explainer = shap.TreeExplainer(xgb_model)shap_values = explainer.shap_values(X_test)def p(j):    return(shap.force_plot(explainer.expected_value, shap_values[j,:], X_test.iloc[j,:]))p(0)
6. Decision Plot
shap_values = explainer.shap_values(X_test)[1]print("The expected value is ", expected_value)print("The final prediction is ", xgb_model.predict(X_test)[1])shap.decision_plot(expected_value, shap_values, X_test)
This is how you can use Shap for creating visualizations related to machine learning models and analyze them. Go ahead and try this with different datasets and machine learning models and let me know your comment in the response section.
This article is in collaboration with Piyush Ingale.
Thanks for reading! If you want to get in touch with me, feel free to reach me at hmix13@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials. Also, feel free to explore my profile and read different articles I have written related to Data Science.