Random Forest for prediction. Using Random Forest to predict... | by Aditya Kumar | Towards Data Science

It’s a process that operates among multiple decision trees to get the optimum result by choosing the majority among them as the best value.
Consider the above image as a representation of multiple decision trees with different results. Out of 4 decision trees, 3 has the same output as 1 while one decision tree has output as 0. Applying the definition mentioned above Random forest is operating four decision trees and to get the best result it's choosing the result which majority i.e 3 of the decision trees are providing. Hence, in this case, the optimum result will be 1.
For the Decision Tree, you can visit the earlier story by clicking on the link https://towardsdatascience.com/decision-tree-ba64f977f7c3
Uses
Remote Sensing: Random Forest (RF) is commonly used in remote sensing to predict the accuracy/classification of data.Object Detection: RF plays a major role in object detection in less time.
Remote Sensing: Random Forest (RF) is commonly used in remote sensing to predict the accuracy/classification of data.
Object Detection: RF plays a major role in object detection in less time.
Why?
Since we are working on multiple decision trees there is a chance of less over-fitting.Greater Accuracy: Since it runs on a larger data set, the accuracy is higher.Estimate Missing data: Since it runs on a larger data set, you can estimate the missing values as well.
Since we are working on multiple decision trees there is a chance of less over-fitting.
Greater Accuracy: Since it runs on a larger data set, the accuracy is higher.
Estimate Missing data: Since it runs on a larger data set, you can estimate the missing values as well.
How does Random Forest work?
In the above diagram, we have the same classification using 3 different decision trees. Tree 1 classifies the data using Color, Tree 2 using Petal Size and Color, and Tree 3 using Petal lifespan and color.
The model is well-trained now. Considering we have a flower with color data missing. Tree 1 will not be able to recognize this data as it classified everything in color so it will bring it to the tulip flower category.
Tree 2: It works on color and petal size. As per the petal size, it will go to a false i.e. not small followed by color i.e., not yellow. So here is the prediction that it’s a rose.
Tree 3: It works on lifespan and color. The first classification will be in a false category followed by non-yellow color. So here as per prediction it’s a rose.
Let’s try to use Random Forest with Python. First, we will import the python library needed.
import pandas as pdimport numpy as npimport matplotlib.pyplot as plt%matplotlib inline
We are importing pandas, NumPy, and matplotlib. Next, we will consume the data and view it.
df.head() will give us the details of the top 5 rows of every column. We can use df.tail() to get the last 5 rows and similar df.head(10) to get to the top 10 rows.
The data is about cars, and we need to predict the price of cars using the above data
We will be using Random Forest to get the price of the car.
df.dtypessymboling              int64normalized-losses      int64make                  objectaspiration            objectnum-of-doors          objectbody-style            objectdrive-wheels          objectengine-location       objectwheel-base           float64length               float64width                float64height               float64curb-weight            int64engine-type           objectnum-of-cylinders      objectengine-size            int64fuel-system           objectbore                 float64stroke               float64compression-ratio    float64horsepower           float64peak-rpm             float64city-mpg               int64highway-mpg            int64price                float64city-L/100km         float64horsepower-binned     objectdiesel                 int64gas                    int64dtype: object
dtypes give the data type of column.
df.describe()
In the above data-frame, all the columns are not numeric. So we will consider only those columns whose values are in numeric and will make all numeric to float.
df.dtypesfor x in df:    if df[x].dtypes == "int64":        df[x] = df[x].astype(float)        print (df[x].dtypes)
Preparing the Data As with the classification task, in this section, we will divide our data into attributes and labels and consequently into training and test sets. We will create 2 data sets, one for the price while the other (df-price). Since our data-frame has a lot of data in object format, for this analysis we are removing all the columns with object type and for all NaN values, we are removing that row.
df = df.select_dtypes(exclude=['object'])df=df.fillna(df.mean())X = df.drop('price',axis=1)y = df['price']
Here the X variable contains all the columns from the data-set, except the ‘Price’ column, which is the label. The y variable contains values from the ‘Price’ column, which means that the X variable contains the attribute set and y variable contains the corresponding labels.
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
To train the tree, we will use the Random Forest class and call it with the fit method. We will have a random forest with 1000 decision trees.
from sklearn.ensemble import RandomForestRegressorregressor = RandomForestRegressor(n_estimators = 1000, random_state = 42)regressor.fit(X_train, y_train)
Let's predict the price.
y_pred = regressor.predict(X_test)
Let’s check the difference between the actual and predicted values.
df=pd.DataFrame({'Actual':y_test, 'Predicted':y_pred})df
from sklearn import metricsprint('Mean Absolute Error:', metrics.mean_absolute_error(y_test, y_pred))print('Mean Squared Error:', metrics.mean_squared_error(y_test, y_pred))print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y_test, y_pred)))Mean Absolute Error: 1993.2901175839186Mean Squared Error: 9668487.223350348Root Mean Squared Error: 3109.4191134921566
The mean absolute error for our algorithm is 1993.2901175839186, which is less than 20 percent of the mean of all the values in the ‘Price’ column. This means that our algorithm made a prediction, but it needs a lot of improvement.
Let’s check the accuracy of our prediction.
# Calculate the absolute errorserrors = abs(y_pred - y_test)# Print out the mean absolute error (mae)print('Mean Absolute Error:', round(np.mean(errors), 2), 'degrees.')# Calculate mean absolute percentage error (MAPE)mape = 100 * (errors / y_test)# Calculate and display accuracyaccuracy = 100 - np.mean(mape)print('Accuracy:', round(accuracy, 2), '%.')Mean Absolute Error: 1993.29 degrees.Accuracy: 87.87 %.
Accuracy of 87.8% is not a very great score and there is a lot of scope for improvement.
Let’s plot the difference between the actual and the predicted value.
import seaborn as snsplt.figure(figsize=(5, 7))ax = sns.distplot(y, hist=False, color="r", label="Actual Value")sns.distplot(y_pred, hist=False, color="b", label="Fitted Values" , ax=ax)plt.title('Actual vs Fitted Values for Price')plt.show()plt.close()
The above is the graph between the actual and predicted values. Let’s visualize the Random Forest tree.
import pydot# Pull out one tree from the forestTree = regressor.estimators_[5]# Export the image to a dot filefrom sklearn import treeplt.figure(figsize=(25,15))tree.plot_tree(Tree,filled=True,               rounded=True,               fontsize=14);
The code can be checked at https://www.kaggle.com/adityakumar529/random-forest.