Random Forests Classifiers
By: Shivam Sharma
Tags: Scikit Learn, Machine Learning, Random Forest Algorithm

Random forest is a type of supervised machine learning algorithm based on ensemble learning. Ensemble learning is a type of learning where you join different types of algorithms or the same algorithm multiple times to form a more powerful prediction model. The random forest algorithm combines multiple algorithms of the same type i.e. multiple decision trees, resulting in a forest of trees, hence the name “Random Forest”. The random forest algorithm can be used for both regression and classification tasks.

Random forests have a variety of applications, such as recommendation engines, image classification, and feature selection. It can be used to classify loyal loan applicants, identify fraudulent activity, and predict diseases. It lies at the base of the Boruta algorithm, which selects important features in a dataset.

How the Random Forest Algorithm Works

The following are the basic steps involved in performing the random forest It works in four steps:

Select random samples from a given dataset. Construct a decision tree for each sample and get a prediction result from each decision tree. Perform a vote for each predicted result. Select the prediction result with the most votes as the final prediction.

Building a Classifier using Scikit-learn

We will build a model on the iris flower dataset, which is a very famous classification set. It comprises the sepal length, sepal width, petal length, petal width, and type of flowers. There are three species or classes: setosa, versicolor, and virginia.

We will build a model to classify the type of flower. The dataset is available in the scikit-learn library

Start by importing the datasets library from scikit-learn, and load the iris dataset with load_iris() .



from sklearn import datasets

#Load dataset

iris = datasets.load_iris()

2.Here, we can create a DataFrame of the iris dataset.

# Creating a DataFrame of given iris dataset.

import pandas as pd

data=pd.DataFrame({

'sepal length':iris.data[:,0],

'sepal width':iris.data[:,1],

'petal length':iris.data[:,2],

'petal width':iris.data[:,3],

'species':iris.target

})

data.head()

3.Now, separate the columns into dependent and independent variables (or features and labels). Then split those variables into a training and test set.

# Import train_test_split function

from sklearn.model_selection import train_test_split X=data[['sepal length', 'sepal width', 'petal length', 'petal width']] # Features

y=data['species'] # Labels # Split dataset into training set and test set

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3) # 70% training and 30% test

After splitting, we will train the model on the training set and perform predictions on the test set.

#Import Random Forest Model

from sklearn.ensemble import RandomForestClassifier #Create a Gaussian Classifier

clf=RandomForestClassifier(n_estimators=100) #Train the model using the training sets y_pred=clf.predict(X_test)

clf.fit(X_train,y_train) y_pred=clf.predict(X_test)

After training, check the accuracy using actual and predicted values.

#Import scikit-learn metrics module for accuracy calculation

from sklearn import metrics

# Model Accuracy, how often is the classifier correct?

print("Accuracy:",metrics.accuracy_score(y_test, y_pred)) ('Accuracy:', 0.929990909399001)

4. Make a prediction for a single item, for example:

sepal length = 3

sepal width = 5

petal length = 4

petal width = 2

Now predict which type of flower it is.

clf.predict([[3, 5, 4, 2]]) array([2])

Here, 2 indicates the flower type Virginica.

5. Finding Important Features in Scikit-learn

Here, we are finding important features or selecting features in the IRIS dataset. In scikit-learn, we can perform this task in the following steps:

a) create a random forest model.

b) use the feature importance variable to see feature importance scores.

c) visualize these scores using the seaborn library.

from sklearn.ensemble import RandomForestClassifier #Create a Gaussian Classifier

clf=RandomForestClassifier(n_estimators=100) #Train the model using the training sets y_pred=clf.predict(X_test)

clf.fit(X_train,y_train) RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',

max_depth=None, max_features='auto', max_leaf_nodes=None,

min_impurity_decrease=0.0, min_impurity_split=None,

min_samples_leaf=1, min_samples_split=2,

min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=1,

oob_score=False, random_state=None, verbose=0,

warm_start=False) import pandas as pd

feature_imp = pd.Series(clf.feature_importances_,index=iris.feature_names).sort_values(ascending=False)

feature_imp petal width (cm) 0.458607

petal length (cm) 0.413859

sepal length (cm) 0.103600

sepal width (cm) 0.023933

dtype: float64

You can also visualize the feature importance. Visualizations are easy to understand and interpretable.

For visualization, you can use a combination of matplotlib and seaborn. Because seaborn is built on top of matplotlib, it offers a number of customized themes and provides additional plot types. Matplotlib is a superset of seaborn and both are equally important for good visualizations.

import matplotlib.pyplot as plt

import seaborn as sns

%matplotlib inline

# Creating a bar plot

sns.barplot(x=feature_imp, y=feature_imp.index)

# Add labels to your graph

plt.xlabel('Feature Importance Score')

plt.ylabel('Features')

plt.title("Visualizing Important Features")

plt.legend()

plt.show()

6) Generating the Model on Selected Features

Here, we can remove the “sepal width” feature because it has very low importance, and select the 3 remaining features.

# Import train_test_split function

from sklearn.cross_validation import train_test_split

# Split dataset into features and labels

X=data[['petal length', 'petal width','sepal length']] # Removed feature "sepal length"

y=data['species']

# Split dataset into training set and test set

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.70, random_state=5) # 70% training and 30% test

After splitting, we will generate a model on the selected training set features, perform predictions on the selected test set features, and compare actual and predicted values.

from sklearn.ensemble import RandomForestClassifier #Create a Gaussian Classifier

clf=RandomForestClassifier(n_estimators=100) #Train the model using the training sets y_pred=clf.predict(X_test)

clf.fit(X_train,y_train) # prediction on test set

y_pred=clf.predict(X_test) #Import scikit-learn metrics module for accuracy calculation

from sklearn import metrics

# Model Accuracy, how often is the classifier correct?

print("Accuracy:",metrics.accuracy_score(y_test, y_pred)) ('Accuracy:', 0.9408065267073221)

After removing the least important features (sepal length), the accuracy increased. This is because you removed misleading data and noise, resulting in increased accuracy. A lesser amount of features also reduces training time.