A Guide to Classification Algorithms
Tags: Machine Learning, Algorithms, Artificial Intelligence, Data Science, Programming

The Most Popular Classification Algorithms

Image source: Author

Scikit-learn is one of the top ML libraries for Python. So if you want to build your model, check it out. It provides access to widely-used classifiers.

Logistic regression

Logistic regression is used for binary classification.

This algorithm employs a logistic function to model the probability of an outcome happening. It is most useful when you want to understand how several independent variables affect a single outcome variable.

Example question: Will the precipitation levels and the soil composition lead to a tomato’s prosperity or its untimely death?

Logistic regression has limitations; all predictors should be independent, and there should be no missing values. This algorithm will fail when there is no linear separation of values.

Naive Bayes

The Naive Bayes algorithm is based on Bayes’ theorem. You can apply this algorithm for binary and multiclass classification and classify data based on historical results.

Example task: I need to separate rotten tomatoes from the fresh ones based on their look.

The advantage of Naive Bayes is that these algorithms are fast to build: They do not require an extensive training set and are also fast compared to other methods. However, since the performance of Bayesian algorithms depends on the accuracy of its strong assumptions, the results can potentially turn out very bad.

Using Bayes’ theorem, it is possible to tell how the occurrence of an event impacts the probability of another event.

k-nearest neighbors

kNN stands for “k-nearest neighbor” and is one of the simplest classification algorithms.

The algorithm assigns objects to the class that most of its nearest neighbors in the multidimensional feature space belong to. The number k is the number of neighboring objects in the feature space that are compared with the classified object.

Example: I want to predict the species of the tomato from the species of tomatoes similar to it.

To classify the inputs using k-nearest neighbors, you need to perform a set of actions:

Calculate the distance to each of the objects in the training sample.

Select k objects of the training sample, the distance to which is minimal.

The class of the object to be classified is the class that occurs most frequently among the k-nearest neighbors.

Decision tree

Decision trees are probably the most intuitive way to visualize a decision-making process. To predict a class label of input, we start from the root of the tree. You need to divide the possibility space into smaller subsets based on a decision rule that you have for each node.

Here is an example:

Image source: Author

You keep breaking up the possibility space until you reach the bottom of the tree. Every decision node has two or more branches. The leaves in the model above contain the decision about whether a person is or isn’t fit.

Example: You have a basket of different tomatoes and want to choose the correct one to enhance your dish.

Types of decision trees

There are two types of trees. They are based on the nature of the target variable:

Categorical variable decision tree

Continuous variable decision tree

Therefore, decision trees work quite well with both numerical and categorical data. Another plus of using decision trees is that they require little data preparation.

However, decision trees can become too complicated, which leads to overfitting. A significant disadvantage of these algorithms is that small variations in training data make them unstable and lead to entirely new trees.

Random forest

Random forest classifiers use several different decision trees on various sub-samples of datasets. The average result is taken as the model’s prediction, which improves the predictive accuracy of the model in general and combats overfitting.

Consequently, random forests can be used to solve complex machine learning problems without compromising the accuracy of the results. Nonetheless, they demand more time to form a prediction and are more challenging to implement.

Read more about how random forests work in the Towards Data Science blog.

Support vector machine

Support vector machines use a hyperplane in an N-dimensional space to classify the data points. N here is the number of features. It can be, basically, any number, but the bigger it is, the harder it becomes to build a model.

One can imagine the hyperplane as a line (for a two-dimensional space). Once you pass three-dimensional space, it becomes hard for us to visualize the model.

Data points that fall on different sides of the hyperplane are attributed to different classes.

Example: An automatic system that sorts tomatoes based on their shape, weight, and color.

The hyperplane that we choose directly affects the accuracy of the results. So we search for the plane that has the maximum distance between data points of both classes.

SVMs show accurate results with minimal computation power when you have a lot of features.