Decision Tree vs Random Forest — Which Algorithm is Better
By: Sonia Jessica
Tags: Random Forest, Technical Interview, Decision Tree, Data Science, Machine Learning

Decision Tree vs Random Forest — Which Algorithm is Better

The field of data science is an interdisciplinary field that mines raw data. After that, careful analysis is done on it, and various patterns are derived from the same that are used to extract valuable insights from the data. Statistics, computer science, machine learning, deep learning, data analysis, data visualization, and various other technologies form the core foundation of data science.

On the other hand, Machine Learning is primarily a set of techniques that are used by data scientists to train computers to learn from data. These techniques produce results that perform well without programming explicit rules. The field of Machine Learning can be broadly categorized into two topics, namely, Supervised Machine Learning and Unsupervised Machine Learning Learning.

There is no doubt whatsoever that Data Science is a very hot topic in today’s world and the demand for Data Scientists all around the world has been increasing ever since the introduction of the topic to the modern world. It is no surprise that a lot of companies in the Software Industry want to hire Data Scientists to solve real-world problems in various fields. A Data Scientist also gets paid handsomely (way more than an average Software Developer) and therefore, if one is familiar with Data Science and techniques used by Data scientists to real-world problems like Machine Learning, he or she has more chances of cracking the interviews (find more interview questions on Data Science here) of big Software companies like Google, Amazon, etc.

Introduction to Decision Tree and Random Forest Algorithms:

Decision Tree vs Random Forest

Supervised Learning Problems can be broadly categorized into Regression problems and Classification problems. Unlike other algorithms, the Decision Tree Algorithm can be used to solve both these problems. A decision tree can be said to be just a series of sequential decisions made to reach a specific result. As is evident from the name itself, the decision tree is a tree having nodes and edges between the nodes. The branches or edges are dependent on the number of criteria. In decision trees, data is split into branches till a threshold unit is achieved. A decision tree has root nodes, children nodes, and leaf nodes. Recursive algorithms like Breadth-First Search and Depth First Search can be used for traversing through the nodes of the decision trees. This algorithm handles data accurately and works best for a linear pattern. Large sets of data can be easily and efficiently handled by the decision trees, that too, in less time.

Although Decision Trees are very powerful alone, many times a single decision tree is not enough to produce desirable results. In Computer Science, Forests are defined as a collection of trees. Similar is the concept of Random Forests. The Random Forest Algorithm is also a Machine Learning Algorithm, just that it does not depend on one single tree for making a decision. In this algorithm, randomized decisions based on several decisions are assembled and the final decision is made based on the majority. The main reason for the usage of the word ‘Random’ is that a forest of randomly created decision trees are used in it. All the nodes in the decision tree work independently on a random subset of features to calculate the output and in the end, the random forest accumulates the output of the individual decision trees to determine the final output. The terminology used for the process of combining the output of multiple individual models (also known as weak learners) is known as ‘Ensemble Learning’.

Example Illustrating the working of Decision Tree and Random Forest Algorithms:

Now that we do have a basic understanding of what Decision Trees and Random Forests are, let us take a look at an example and understand what will be the steps involved in the Decision Tree Algorithm and the Random Forest Algorithm to solve it.

PROBLEM STATEMENT:

Let us imagine a scenario in which a customer wants to take a loan from the bank and the bank needs to decide whether or not it has to approve the loan. What should be the factors taken into consideration by the bank while approving or disapproving the loan and what should be the decision taken?

SOLUTION:

First of all, let us jot down the factors which the bank should take into consideration before making a decision in this scenario:

CREDIT SCORE OF THE CUSTOMER

INCOME OF THE CUSTOMER

LOAN AMOUNT

DECISION TREE ALGORITHM FOR LOAN APPROVAL:

In the decision tree algorithm, the bank decides the priority of the factors, and then a decision tree is made as shown in the above picture. As is evident from the above picture, the Credit Score of the Customer is given the highest priority. Next in line is the Customer’s Income and lastly, the Loan Amount.

According to the above decision tree, if the credit score of the customer is good and his income is good, then the customer would be granted the loan. However, if the credit score of the customer is good but he or she has a low income, then the loan amount becomes the differentiating factor. For small amounts, the loan is granted and for bigger amounts, the loan is disapproved.

On the other hand, if the credit score of the customer is not bad, the income of the customer is taken into consideration. If the income is decent or not good, the loan would not be granted. However, if his or her income is good, then the loan amount becomes the differentiating factor. For small amounts, the loan is granted and for bigger amounts, the loan is disapproved.

RANDOM FOREST ALGORITHM FOR LOAN APPROVAL:

It is important to realize that if all the above factors are not taken into consideration before the final decision is taken by the bank, the bank might suffer losses. Also in many scenarios, if the priority order of the factors is strictly maintained, the final decision might not be as effective as it could be. For instance, let us assume that the bank considered the Credit Score of the Customer first and it was not good. Also, his income was not good and therefore, the loan was not granted to him or her. However, in the event of the loan amount is small, the bank could lose an opportunity to make money over here as there would not be much of a risk involved because of the small loan amount. Hence, it is important to not just rely on one priority order of the factors, and taking all of them into consideration is important. This was why the concept of Random Forests was introduced.

In the Random Forest Algorithm, a lot of decision trees are made (hence the name ‘Forests’) with different priority orders of the factors. So for instance, in the above-mentioned example, one decision tree could have the factors in the priority order as follows:

CREDIT SCORE OF THE CUSTOMER

INCOME OF THE CUSTOMER

LOAN AMOUNT

While another decision tree of the forest could have the factors in the priority order as follows:

INCOME OF THE CUSTOMER

CREDIT SCORE OF THE CUSTOMER

LOAN AMOUNT

Similarly, many decision trees can be made and they form a forest of trees. Now, sometimes the bank might check for credit history first, and sometimes it can check for the customer’s financial condition and loan amount first. In the end, the combined results from these multiple decision-making processes are gathered by the bank and the final decision is taken whether or not to give the loan to the customer. Hence, even though this process might take more time, the results that are fetched are more beneficial to the bank as an average result is always generated due to the randomness of the decision trees in the forest.

Advantages and Disadvantages of Decision Tree Algorithm:

Given below are some of the advantages of the Decision Tree Algorithm:

They are easy to implement and are really transparent.

Both numerical and categorical data can be handled by Decision Trees.

Work well with huge datasets and are usually fast.

Given below are some of the disadvantages of the Decision Tree Algorithm:

The calculations can get complicated after a certain point of time.

Optimization is not guaranteed when using the Decision Tree Algorithm.

The deflection might be high when using this algorithm.

Advantages and Disadvantages of Random Forest Algorithm:

Given below are some of the advantages of the Random Forest Algorithm:

Random Forest Algorithm is very powerful because of its high accuracy.

Multiple Decision Trees can be run in parallel ways in this algorithm. Hence, higher efficiency is achieved.

There is no need for normalizing in this algorithm.

Given below are some of the disadvantages of the Random Forest Algorithm:

It is slower compared to the Decision Tree Algorithm.

Can be biased to certain factors at times.

They cannot be used for linear methods.

Conclusion:

It must be pretty evident by now that for scenarios for which interpretability is a major concern or the data set is huge, we should use the Decision Tree Algorithm. For other scenarios, Random Forest Algorithm works better. The major differentiating factor between the two algorithms is the speed at which output is returned by the two. Decision trees are easy to understand and code compared to Random Forests as a decision tree combines a few decisions, while a random forest combines several decision trees. Thus, it is a long process, yet slow.

Decision trees are usually fast and operate easily on large data sets, especially the linear ones. The random forest model needs meticulous training. When you are trying to put up a project, you might need more than one model. Thus, a large number of random forests, more the time.