Last episode we used a decision tree as our classifier. Today we'll add code to visualize it so we can see how it works under the hood. There are many types of classifiers you may have heard of before, things like neural nets or support vector machines. So why did we use a decision tree to start? Well, they have a very unique property. They're easy to read and understand. In fact, they're one of the few models that are interpretable where you can understand exactly why the classifier makes a decision. That's amazingly useful in practice. To get started, I'll introduce you to a real dataset we'll work with today. It's called Iris. Iris is a classic machine learning problem. In it, you want to identify what type of flower you have based on different measurements like the length and width of the pedal. The dataset includes three different types of flowers. They're all species of Iris, Satosa, Versicolor, and Virginia. Just rolling down, you can see we're given 50 examples of each type, so 150 examples total. Notice there are four features that are used to describe each example. These are the length and width of the sepal and pedal. And just like in our apples and oranges problem, the first four columns give the features, and the last column gives the labels, which is the type of flower in each row. Our goal is to use this dataset to train a classifier. Then we can use that classifier to predict what species of flower we have if we're given a new flower that we've never seen before. Knowing how to work with an existing dataset as a good skill lets import Iris into Scikit Learn and see what it looks like in code. Conveniently, the friendly folks at Scikit provided a bunch of sample datasets, including Iris, as well as utilities to make them easy to import. We can import Iris into our code like this. The dataset includes both the table from Wikipedia, as well as some metadata. The metadata tells you the names of the features and the names of different types of flowers. The features and examples themselves are contained in the data variable. For example, if I print out the first entry, you can see the measurements for this flower. These index to the feature names, so the first value refers to the sepal length and the second to sepal width and so on. The target variable contains the labels. Likewise, these index to the target names. Let's print out the first one. A label of zero means it's the setosa. If you look at the table from Wikipedia, you'll notice that we just printed out the first row. Now, both the data and target variables have 150 entries. If you want, you can iterate over them to print out the entire dataset like this. Now that we know how to work with the dataset, we're ready to train a classifier. But before we do that, first we need to split up the data. I'm going to remove several of the examples and put them aside for later. We'll call the examples I'm putting aside our testing data. We'll keep these separate from our training data. And later on, we'll use our testing examples to test how accurate the classifier is on data it's never seen before. Testing is actually a really important part of doing machine learning well in practice and we'll cover it in more detail in a future episode. Just for this exercise, I'll remove one example of each type of flower. And as it happens, the dataset is ordered so the first setosa is an index zero, and the first Versicolor is at 50 and so on. The syntax looks a little bit complicated, but all I'm doing is removing three entries from the data and target variables. Then I'll create two new sets of variables, one for training and one for testing. Training will have the majority of our data, and testing will have just the examples I removed. Now just as before, we can create a decision tree classifier and train it on our training data. Before we visualize it, let's use the tree to classify our testing data. We know we have one flower of each type, and we can print out the labels we expect. Now let's see what the tree predicts. We'll give it the features for our testing data, and we'll get back labels. You can see the predicted labels match our testing data. That means it got them all right. Now keep in mind this was a very simple test, and we'll go into more detail down the road. Now let's visualize the tree so we can see how the classifier works. To do that, I'm going to copy, paste and code in from Cykits tutorials, and because this code is for visualization and not machine learning concepts, I won't cover the details here. Note that I'm combining the code from these two examples to create an easy to read PDF. I can run our script and open up the PDF, and we can see the tree. To use it to classify data, you start by reading from the top. Each node asks a yes or no question about one of the features. For example, this node asks if the pedal width is less than 0.8 centimeters. If it's true for the example you're classifying, go left, otherwise go right. Now let's use this tree to classify an example from our testing data. Here are the features and labels for our first testing flower. Remember, you can find the feature names by looking at the metadata. We know this flower is a stotosa, so let's see what the tree predicts. I'll resize the windows to make this easier to see. And the first question the tree asks is whether the pedal width is less than 0.8 centimeters. That's the fourth feature. The answer is true so we proceed left. At this point we're already at a leaf node. There are no other questions to ask, so the tree gives us a prediction. Stotosa, and it's right. Notice the label is 0, which indexes to that type of flower. Now let's try our second testing example. This one is a versicolor. Let's see what the tree predicts. When we read from the top, and this time the pedal width is greater than 0.8 centimeters. The answer to the tree's question is false, so we go right. The next question the tree asks is whether the pedal width is less than 1.75. It's trying to narrow it down. That's true, so we go left. Now it asks if the pedal length is less than 4.95. That's true, so we go left again. And finally the tree asks if the pedal width is less than 1.65. That's true, so left it is. And now we have our prediction. It's a versicolor, and that's right again. You can try the last one on your own as an exercise, and remember the way we're using the tree is the same way it works in code. So that's how you quickly visualize and read a decision tree. There's a lot more to learn here, especially how they're built automatically from examples. We'll get to that in a future episode, but for now let's close with an essential point. Every question the tree asks must be about one of your features. That means the better your features are, the better a tree you can build. And in the next episode we'll start looking at what makes a good feature. Thanks very much for watching, and I'll see you next time.