An Introduction to Machine Learning — Algorithms for Beginners
Tags: Artificial Intelligence, Machine Learning, Deep Learning, Data Science, Programming

Supervised Learning Algorithms

Image source: Author

If you’re not familiar with such terms as supervised learning and unsupervised learning, check out our AI vs. machine learning post where this topic is covered in detail. Now, let’s get familiar with the algorithms.

1. Classification algorithms

Naive Bayes

Bayesian algorithms are a family of probabilistic classifiers used in ML (machine learning) based on applying Bayes’ theorem.

Naive Bayes classifier was one of the first algorithms used for machine learning. It is suitable for binary and multiclass classification and allows for making predictions and forecast data based on historical results. A classic example is spam filtering systems that used naive Bayes up till 2010 and showed satisfactory results. However, when Bayesian poisoning was invented, programmers started to think of other ways to filter data.

Using Bayes’ theorem, it is possible to tell how the occurrence of an event impacts the probability of another event.

For example, this algorithm calculates the probability that a certain email is or isn’t spam based on the typical words used. Common spam words and phrases are offer, order now, and additional income. If the algorithm detects these words, there is a high possibility that the email is spam.

Naive Bayes assumes that the features are independent. Therefore, the algorithm is called naive.

Multinomial naive Bayes

Apart from the naive Bayes classifier, there are other algorithms in this group. For example, there is also multinomial naive Bayes, which is usually applied for document classification based on the frequency of certain words present in the document.

Bayesian algorithms are still used for text categorization and fraud detection. They can also be applied for machine vision (for example, face detection), market segmentation, and bioinformatics.

Logistic regression

Even though the name might seem counterintuitive, logistic regression is actually a type of classification algorithm.

Logistic regression is a model that makes predictions using a logistic function to find the dependency between the output and input variables. Statquest made a great video where they explain the difference between linear and logistic regression taking as the example obese mice.

Decision trees

A decision tree is a simple way to visualize a decision-making model in the form of a tree. The advantages of decision trees are that they are easy to understand, interpret, and visualize. Also, they demand little effort for data preparation.

However, they also have a big disadvantage. The trees can be unstable because of even the smallest variations (variance) in data. It is also possible to create overly complex trees that do not generalize well. This is called overfitting. Bagging, boosting, and regularization help to fight this problem. We are going to talk about them later in the article.

The elements of every decision tree are:

Root node that asks the main question. It has the arrows pointing down from it but no arrows pointing to it. For example, imagine you are building a tree for deciding what kind of pasta you should have for dinner.

Branches. A subsection of a tree is called a branch or sometimes a sub-tree.

Decision nodes. These are the subnodes for the root node that can also be splitting into more nodes. Your decision nodes can be “carbonara?” or “with mushrooms?”

Leaves or Terminal nodes. These nodes do not split. They represent final decisions or predictions.

Also, it is important to mention splitting. This is the process of dividing a node into subnodes. For instance, if you’re not a vegetarian, carbonara is okay. But if you are, eat pasta with mushrooms. There is also a process of node removal called pruning.

Decision tree algorithms are referred to as CART (classification and regression trees). Decision trees can work with categorical or numerical data.

Regression trees are used when the variables have numerical value.

Classification trees can be applied when the data is categorical (classes).

Decision trees are quite intuitive to understand and use. That is why tree diagrams are commonly applied in a broad range of industries and disciplines. GreyAtom provides a broad overview of different types of decision trees and their practical applications.

SVM (support vector machine)

Support vector machines are another group of algorithms used for classification and, sometimes, regression tasks. SVM is great because it gives quite accurate results with minimum computation power.

The goal of the SVM is to find a hyperplane in an N-dimensional space (where N corresponds with the number of features) that distinctly classifies the data points. The accuracy of the results directly correlates with the hyperplane that we choose. We should find a plane that has the maximum distance between data points of both classes.

This hyperplane is graphically represented as a line that separates one class from another. Data points that fall on different sides of the hyperplane are attributed to different classes.

Image source: Author

Note that the dimension of the hyperplane depends on the number of features. If the number of input features is two, then the hyperplane is just a line. If the number of input features is three, then the hyperplane becomes a two-dimensional plane. It becomes difficult to draw on a graph a model when the number of features exceeds three. So, in this case, you will be using Kernel types to transform it into a three-dimensional space.

Why is this called a support vector machine? Support vectors are data points closest to the hyperplane. They directly influence the position and orientation of the hyperplane and allow us to maximize the margin of the classifier. Deleting the support vectors will change the position of the hyperplane. These are the points that help us to build our SVM.

SVMs are now actively used in medical diagnosis to find anomalies, in air quality control systems, for financial analysis and predictions on the stock market, and for machine fault control in industry.

2. Regression algorithms

Regression algorithms are useful in analytics, for example, when you are trying to predict the costs for securities or sales for a particular product at a particular time.

Linear regression

Linear regression attempts to model the relationship between variables by fitting a linear equation to the observed data.

There are explanatory and dependent variables. Dependent variables are things that we want to explain or forecast. The explanatory ones, as it follows for the name, explain something. If you want to build linear regression, you assume there is a linear relationship between your dependent and independent variables. For example, there is a correlation between the square metres of a house and its price or the density of population and kebab places in the area.

Once you make that assumption, you next need to figure out the specific linear relationship. You will need to find a linear regression equation for a set of data. The last step is to calculate the residual.

Note: When the regression draws a straight line, it is called linear, when it is a curve, it’s polynomial.