Baby Steps Towards Data Science: Decision Tree Regression in Python | by Tharun Peddisetty | Towards Data Science

Decision trees are majorly used in classification problems however, let us try to understand its implications in regression and also, try to understand why using it in regression isn’t a great idea.
Decision tree regression enables one to divide the data into multiple splits. These splits typically answer a simple if-else condition. The algorithm decides the optimal number of splits in the data. Since this method of splitting data closely resembles the branches of a tree, this is probably is known as a decision tree. In fact, the last level (i.e., Fit, Unfit) are known as leaves.
For example, look at the image above, there are data regarding customers, their age, whether they eat pizza or not and whether they exercise or not. By performing decision tree regression, the data is split into 2 categories by age i.e., age< 30 and age> 30. Within the age< 30 category, the data is again split into 2 categories by their eating habits i.e., people eating pizza and people not eating pizza. The same goes for exercise as well. By doing these splits, we can simply account for the behavior of the customers based on their choices and we end up deciding whether they are fit or unfit.
Let us deep dive into python and build a polynomial regression model and try to predict the salary of an employee of 6.5 level(hypothetical).
Before you move forward, please download the CSV data file from my GitHub Gist.
https://gist.github.com/tharunpeddisetty/433e5fe5af0e6b6cdd9d7df3339931a5Once you open the link, you can find "Download Zip" button on the top right corner of the window. Go ahead and download the files.You can download 1) python file 2)data file (.csv)Rename the folder accordingly and store it in desired location and you are all set.If you are a beginner I highly recommend you to open your python IDE and follow the steps below because here, I write detailed comments(statements after #.., these do not compile when our run the code) on the working of code. You can use the actual python as your backup file or for your future reference.
Importing Libraries
import numpy as npimport matplotlib.pyplot as pltimport pandas as pd
Import Data and Define the X and Y variables
dataset = pd.read_csv(‘/Users/tharunpeddisetty/Desktop/Position_Salaries.csv’) #add your file pathX = dataset.iloc[:,1:-1].valuesy = dataset.iloc[:, -1].values#iloc takes the values from the specified index locations and stores them in the assigned variable as an array
Let us look at our data and understand the variables:
This data depicts the position/level of the employee and their salaries. This is the same dataset that I used in my Polynomial Regression article.
Visualizing the data
plt.scatter(X, y, color = ‘red’)plt.plot(X,regressor.predict(X), color = ‘blue’)plt.title(‘Decision Tree’)plt.xlabel(‘Position level’)plt.ylabel(‘Salary’)plt.show()
Training the model
from sklearn.tree import DecisionTreeRegressorregressor = DecisionTreeRegressor(random_state=0)regressor.fit(X,y)#random_state is the seed value, just to make sure we both get same results.
We would be using the entire data in the model, so you don’t need to split the data into train and test data.
Visualizing the results of Decision tree Regression
X_grid = np.arange(min(X), max(X), 0.1)X_grid = X_grid.reshape((len(X_grid), 1))plt.scatter(X, y, color = 'red')plt.plot(X_grid,regressor.predict(X_grid), color = 'blue')plt.title('Decision Tree')plt.xlabel('Position level')plt.ylabel('Salary')plt.show()
You can see how our model predicted the same exact data levels as that of the data points. This visualization is only for illustration purposes. In reality, we would hardly have any dataset with just X and Y variables (X= Position Level and Y = Salary). Since this data is two dimensional, we were able to plot it. Had the data been multidimensional, we wouldn’t have plotted it. It obviously makes so sense then.
Predicting 6.5 level result using Decision tree Regression
print(regressor.predict([[6.5]]))# predict method expects a 2D array thats the reason you see [[6.5]]
Result
Decision Tree Regression: 150000
We can see that our model has predicted the same salary as that of a level 6 employee. This is due to the very nature of the decision tree algorithm. It works with the assumption that if an employee has certain characteristics (level in this case) then another employee with similar characteristics also gets the same salary.
I leave this to your wisdom to understand how good this model is. Just kidding! You can clearly see that this isn’t fair right? On the same lines, we can conclude that this model hasn’t performed well. As, I already mentioned at the beginning of this article that this model is preferred for classification problems, which I would be dealing with in my future articles. I will make sure to use the same data set for my next articles implementing different regression techniques. By doing so you can compare the results and decide which model is the best to use.
I hope this was fun for you. Congratulations! You have implemented decision tree regression in the minimum number of lines of code. When I say baby steps towards data science, I truly mean it. You now have a template of the code and you can implement this on other datasets and observe results. Happy Machine Learning!