Visualizing Decision Trees with Python (Scikit-learn, Graphviz, Matplotlib) | by Michael Galarnyk | Towards Data Science

Decision trees are a popular supervised learning method for a variety of reasons. Benefits of decision trees include that they can be used for both regression and classification, they don’t require feature scaling, and they are relatively easy to interpret as you can visualize decision trees. This is not only a powerful way to understand your model, but also to communicate how your model works. Consequently, it would help to know how to make a visualization based on your model.
This tutorial covers:
How to Fit a Decision Tree Model using Scikit-Learn
How to Visualize Decision Trees using Matplotlib
How to Visualize Decision Trees using Graphviz (what is Graphviz, how to install it on Mac and Windows, and how to use it to visualize decision trees)
How to Visualize Individual Decision Trees from Bagged Trees or Random Forests
As always, the code used in this tutorial is available on my GitHub. With that, let’s get started!
In order to visualize decision trees, we need first need to fit a decision tree model using scikit-learn. If this section is not clear, I encourage you to read my Understanding Decision Trees for Classification (Python) tutorial as I go into a lot of detail on how decision trees work and how to use them.
The following import statements are what we will use for this section of the tutorial.
import matplotlib.pyplot as pltfrom sklearn.datasets import load_irisfrom sklearn.datasets import load_breast_cancerfrom sklearn.tree import DecisionTreeClassifierfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.model_selection import train_test_splitimport pandas as pdimport numpy as npfrom sklearn import tree
The Iris dataset is one of datasets scikit-learn comes with that do not require the downloading of any file from some external website. The code below loads the iris dataset.
import pandas as pdfrom sklearn.datasets import load_irisdata = load_iris()df = pd.DataFrame(data.data, columns=data.feature_names)df['target'] = data.target
The code below puts 75% of the data into a training set and 25% of the data into a test set.
X_train, X_test, Y_train, Y_test = train_test_split(df[data.feature_names], df['target'], random_state=0)
# Step 1: Import the model you want to use# This was already imported earlier in the notebook so commenting out#from sklearn.tree import DecisionTreeClassifier# Step 2: Make an instance of the Modelclf = DecisionTreeClassifier(max_depth = 2,                              random_state = 0)# Step 3: Train the model on the dataclf.fit(X_train, Y_train)# Step 4: Predict labels of unseen (test) data# Not doing this step in the tutorial# clf.predict(X_test)
As of scikit-learn version 21.0 (roughly May 2019), Decision Trees can now be plotted with matplotlib using scikit-learn’s tree.plot_tree without relying on the dot library which is a hard-to-install dependency which we will cover later on in the blog post.
The code below plots a decision tree using scikit-learn.
tree.plot_tree(clf);
In addition to adding the code to allow you to save your image, the code below tries to make the decision tree more interpretable by adding in feature and class names (as well as setting filled = True).
fn=['sepal length (cm)','sepal width (cm)','petal length (cm)','petal width (cm)']cn=['setosa', 'versicolor', 'virginica']fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (4,4), dpi=300)tree.plot_tree(clf,               feature_names = fn,                class_names=cn,               filled = True);fig.savefig('imagename.png')
Graphviz is open source graph visualization software. Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. In data science, one use of Graphviz is to visualize decision trees. I should note that the reason why I am going over Graphviz after covering Matplotlib is that getting this to work can be difficult. The first part of this process involves creating a dot file. A dot file is a Graphviz representation of a decision tree. The problem is that using Graphviz to convert the dot file into an image file (png, jpg, etc) can be difficult. There are a couple ways to do this including: installing python-graphviz though Anaconda, installing Graphviz through Homebrew (Mac), installing Graphviz executables from the official site (Windows), and using an online converter on the contents of your dot file to convert it into an image.
The code below code will work on any operating system as python generates the dot file and exports it as a file named tree.dot.
tree.export_graphviz(clf,                     out_file="tree.dot",                     feature_names = fn,                      class_names=cn,                     filled = True)
Converting the dot file into an image file (png, jpg, etc) typically requires the installation of Graphviz which depends on your operating system and a host of other things. The goal of this section is to help people try and solve the common issue of getting the following error. dot: command not found.
How to Install and Use on Mac through Anaconda
To be able to install Graphviz on your Mac through this method, you first need to have Anaconda installed (If you don’t have Anaconda installed, you can learn how to install it here).
Open a terminal. You can do this by clicking on the Spotlight magnifying glass at the top right of the screen, type terminal and then click on the Terminal icon.
Type the command below to install Graphviz.
conda install python-graphviz
After that, you should be able to use the dot command below to convert the dot file into a png file.
dot -Tpng tree.dot -o tree.png
How to Install and Use on Mac through Homebrew
If you don’t have Anaconda or just want another way of installing Graphviz on your Mac, you can use Homebrew. I previously wrote an article on how to install Homebrew and use it to convert a dot file into an image file here (see the Homebrew to Help Visualize Decision Trees section of the tutorial).
How to Install and Use on Windows through Anaconda
This is the method I prefer on Windows. To be able to install Graphviz on your Windows through this method, you first need to have Anaconda installed (If you don’t have Anaconda installed, you can learn how to install it here).
Open a terminal/command prompt and enter the command below to install Graphviz.
conda install python-graphviz
After that, you should be able to use the dot command below to convert the dot file into a png file.
dot -Tpng tree.dot -o tree.png
How to Install and Use on Windows through Graphviz Executable
If you don’t have Anaconda or just want another way of installing Graphviz on your Windows, you can use the following link to download and install it.
If all else fails or you simply don’t want to install anything, you can use an online converter.
In the image below, I opened the file with Sublime Text (though there are many different programs that can open/read a dot file) and copied the content of the file.
In the image below, I pasted the content from the dot file onto the left side of the online converter. You can then choose what format you want and then save the image on the right side of the screen.
Keep in mind that there are other online converters that can help accomplish the same task.
A weakness of decision trees is that they don’t tend to have the best predictive accuracy. This is partially because of high variance, meaning that different splits in the training data can lead to very different trees. The video above covers Bagged Trees which is an ensemble model. This means using multiple learning algorithms to obtain a better predictive performance than could be obtained from any of the constituent learning algorithms alone. In this case, many trees protect each other from their individual errors. The interesting thing is that the thumbnail from the video above could be a diagram for either Bagged Trees or Random Forests (another ensemble model). The differences between how Bagged Trees and Random Forests models work is a subject for another blog, but what is important to note is that for both models we grow N trees where N is the number of decision trees a user specifies. Consequently after you fit a model, it would be nice to look at the individual decision trees that make up your model.
In order to visualize individual decision trees, we need first need to fit a Bagged Trees or Random Forest model using scikit-learn (the code below fits a Random Forest model).
# Load the Breast Cancer (Diagnostic) Datasetdata = load_breast_cancer()df = pd.DataFrame(data.data, columns=data.feature_names)df['target'] = data.target# Arrange Data into Features Matrix and Target VectorX = df.loc[:, df.columns != 'target']y = df.loc[:, 'target'].values# Split the data into training and testing setsX_train, X_test, Y_train, Y_test = train_test_split(X, y, random_state=0)# Random Forests in `scikit-learn` (with N = 100)rf = RandomForestClassifier(n_estimators=100,                            random_state=0)rf.fit(X_train, Y_train)
You can now view all the individual trees from the fitted model. In this section, I will visualize all the decision trees using matplotlib.
rf.estimators_
You can now visualize individual trees. The code below visualizes the first decision tree.
fn=data.feature_namescn=data.target_namesfig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (4,4), dpi=800)tree.plot_tree(rf.estimators_[0],               feature_names = fn,                class_names=cn,               filled = True);fig.savefig('rf_individualtree.png')
You can try to use matplotlib subplots to visualize as many of the trees as you like. The code below visualizes the first 5 decision trees. I personally don’t prefer this method as it is even harder to read.
# This may not the best way to view each estimator as it is smallfn=data.feature_namescn=data.target_namesfig, axes = plt.subplots(nrows = 1,ncols = 5,figsize = (10,2), dpi=3000)for index in range(0, 5):    tree.plot_tree(rf.estimators_[index],                   feature_names = fn,                    class_names=cn,                   filled = True,                   ax = axes[index]);        axes[index].set_title('Estimator: ' + str(index), fontsize = 11)fig.savefig('rf_5trees.png')
Keep in mind that if for some reason you want images for all your estimators (decision trees), you can do so using the code on my GitHub. If you just want to see each of the 100 estimators for the Random Forest model fit in this tutorial without running the code, you can look at the video below.
This tutorial covered how to visualize decision trees using Graphviz and Matplotlib. Note that the way to visualize decision trees using Matplotlib is a newer method so it might change or be improved upon in the future. Graphviz is currently more flexible as you can always modify your dot files to make them more visually appealing like I did using the dot language or even just alter the orientation of your decision tree. One thing we didn’t cover was how to use dtreeviz which is another library that can visualize decision trees. There is an excellent post on it here.
My next machine learning tutorial goes over How to Speed up Scikit-Learn Model Training. If you have any questions or thoughts on the tutorial, feel free to reach out in the comments below or through Twitter.