How VOTing classifiers work!
By: Mubarak Ganiyu
Tags: Python, Data, Machine Learning, Classification, Data Science

How VOTing classifiers work!

Classification is an important machine learning technique that is often used to predict categorical labels. It is a very practical approach for making binary predictions or predicting discrete values. The classifier, another name for classification model, might have the intention of predicting whether someone is eligible for a job or it could be used to classify the images of multiple objects in a store.

Classification, like other machine learning techniques, use data sets. A dataset is a combination of multiple values from different variables. After obtaining an optimal dataset, it is split into two: the training and testing set. The training set often has a larger proportion of the dataset. It is likely to take up about 70% to 90% of the dataset.

The training set is inserted into the machine learning algorithm to create a predictive model with an added step called cross-validation. Cross-validation is a great way to ensure that the built model does not overfit the training set and it also optimizes the versatility of the model. Then, the model can be used to predict the labels in the testing set. The predicted labels are further compared to the actual testing set labels via metrics such as confusion matrix, precision score, recall score, F1-score, roc auc score.

Once the construction of the classification model is over, a data point’s values can be inserted into the algorithm and the algorithm makes a decision by attributing a specific label to this data point based on the variables’ inputs.

Now imagine if different classification methods were asked to make decisions based on the data instances’ inputs. There are bound to be different answers. This is where voting classifiers come into play.

What is a Voting Classifier?

A voting classifier is a classification method that employs multiple classifiers to make predictions. It is very applicable in situations when a data scientist or machine learning engineer is confused about which classification method to use. Therefore, using the predictions from multiple classifiers, the voting classifier makes predictions based on the most frequent one.

A real life scenario could see a data scientist being confused about whether to use a random forest classifier or a logistic regressor to predict the type of flower based on their dimensions.

Using the prompt above, a step-by-step guide has been created below on how to use python via Jupyter Notebooks to build voting classifiers.

Starting with the code below, one can import the classifiers using scikit-learn.

from sklearn.ensemble import RandomForestClassifier

from sklearn.linear_model import LogisticRegression

from sklearn.neighbors import KNeighborsClassifier

from sklearn.ensemble import VotingClassifier

Using python via Jupyter Notebooks, the scikit-learn’s ensemble feature is accessed and a voting classifier is imported. There are three other classifiers in the code above: a random forest classifier, a logistic regressor and a KNearest Neighbor classifier. These three will be attributed to objects as seen below:

log_clf = LogisticRegression()

rnd_clf = RandomForestClassifier()

knn_clf = KNeighborsClassifier()

Afterwards, an object is created for the voting classifier. The voting classifier has two basic hyperparameters: estimators and voting. The estimators hyperparameter creates a list for the objects of the three classifiers above while assigning names to them. The voting hyperparameter is set to either hard or soft.

If set to hard, the voting classifier will make judgments based on the predictions that appear the most. Otherwise, if set to soft, it will use a weighted approach to make its decision. I’d recommend setting it to soft when using an even number of classifiers because of its weighted approach and setting it to hard when using an odd number of classifiers because of its “majority carry the vote” approach.

vot_clf = VotingClassifier(estimators = [('lr', log_clf), ('rnd', rnd_clf), ('knn', knn_clf)], voting = 'hard')

The voting classifier like any other machine learning algorithm is used to fit the independent variables of the training dataset with the dependent variables

from sklearn.datasets import load_iris

from sklearn.model_selection import train_test_split iris = load_iris()

x, y = iris['data'], iris['target']

x_train, x_test, y_train, y_test = train_test_split(X, y, random_state = 42, train_size = 0.85) vot_clf.fit(x_train, y_train)

After fitting, it can be used to make predictions and the accuracy of its predictions is measured.

pred = vot_clf.predict(x_test)

accuracy_score(y_test, pred)

Here is an image that shows how a voting classifier was used as a predictive model for a dataset and compared to other classifiers. The code was initially obtained from Aurelion Geron’s book, Hands-On Machine Learning with Scikit-Learn and TensorFlow Concepts, Tools, and Techniques to Build Intelligent Systems, and I ran them on Jupyter Notebooks.