You will never believe how machines can learn like humans: Part 2
By: Anvitha Gowda
Tags: Programming, Neural Networks, Deep Learning, Artificial Intelligence, Machine Learning

You will never believe how machines can learn like humans: Part 2 Anvitha Gowda Follow Jul 7 · 7 min read

In my previous article, I wrote about some fundamentals of Artificial Intelligence. If you haven’t read that one, please click on the link given below.

https://medium.com/dev-genius/you-will-never-believe-how-machine-can-learn-like-humans-b67faf26bf9c

Well today let us dive a bit deep into the concepts of AI.

Did you know that “ Machine learning results depend on the Data you entered to train it”

We all know that one cannot function properly without a brain. There has to be a connection between our body parts as well as the brain. This connection has been established by neural networks. Same type of neural networks are responsible for Machines to learn!

Neural networks

Photo by Franck V. on Unsplash

Deep learning uses layers of non-linear processing units for features extraction and transformation called neural networks. The simplest of all the neural network is perceptron. Perceptron uses a single-layer neural network that uses a list of input features. Let us look at the example, say x1 to xn where x1 maybe cost and x2 maybe quantity and so on. These x1 to xn are called feature vectors. In this example, we need to determine if the customer would buy the product or not, looking at the input features. Along with input features, there is also a bias term like intercepts in the linear regression model. This input features, space along with intercepts form a linear feature that gives output. In order to obtain the output, we need to provide the linear model with the activation function. This activation function is basically non-linear and depends on the type of the problem we are trying to solve. And here activation function is the sigmoid function. In our example, we need to find out whether the customer will buy our product or not. For this the output is binary form. Either yes or no!

Perceptrons are very simple that has a single layer which is composed of the sum of input features and activation function that gives output. The concept of perceptrons were prevalent a long time ago. Overtime people added many layers to the perceptrons. Now we have an input layer, original layer and hidden layer. Once we have multiple layers, perceptron becomes neural networks. Neural networks are very complicated and have multiple layers within one single layer. They are even very difficult to understand, interpret and train. Due to the revolution in deep learning, we have different frameworks to design, train and implement neural networks. This has lead to the development of multiple layers and due to which we can have large input data than before. The deep learning frameworks are Mxnet, TensorFlow, Cafe, PyTorch, etc. These are developed from different sources, but can be accessed using Python. Yes, we can design, train and fit a neural network using python.

Understanding Machine Learning Algorithms

Any machine exhibiting intelligent behaviour needs to have two basic features

1. Knowledge acquisition: The ability of a machine to acquire the knowledge of surrounding and use the knowledge acquired to make predictions is very important aspect of machine learning.

2. Inference: The ability of a machine to infer the knowledge of previous experience to make present predictions is a basic aspect of machine learning.

We have seen the types of machine learning in the previous article. They are Supervised, Unsupervised and Reinforcement. Let us now see how are the algorithms of these types devised.

1. Supervised learning algorithm

In the case of supervised learning, Algorithm will be generated after showing a machine with many examples and a teacher explaining the machine what it is. This is also known as training a machine learning model. After training, the machine can be able to identify other similar things which are not shown in examples. It is simply like a newborn trying to identify apple and banana after showing it an orange and telling what it is! There are hundreds of supervised learning algorithms out there. It is not necessary to mention them all. So we can classify them according to certain families. We have also seen about the types of predictions in the previous article. Among them, regression is a type of prediction which predicts the unknown value based on the input given. In order to separate objects based on class or category, we need a hyperplane also known as a decision boundary that separates two objects. If there exists a linear surface that separates two classes, then they are called linearly separable. These are very simple and has errors. so we need logistic function applied to the output of the linear combination of model. This forms the basis of the logistic regression algorithm. An example for this is Amazon Sage Maker

2. Unsupervised Algorithm

In the case of unsupervised learning, the algorithm as in case of supervised will not be linear. That means there has to be circular or even square boundaries. These boundaries form trees. Instead of constructing a single tree, these trees from tree family form many trees and give predictions. Algorithms such as Random Forest, XGBoost are based on these approaches. In fact, Amazon Sagemaker also make use of XGBoost. The idea is to build a strong classifier out of many weak classifiers. This kind of approach is called Boosting. XGBoost is a general-purpose supervised algorithm. Factorisation Machines is also an algorithm that is used when we need to look at a large amount of spars data such as click predictions in online advertisement.

Clustering

In the case of unsupervised learning, clustering is a very important. Given a large number of data points, they are divided into groups and clusters. The problem in clustering is that we do not know how many clusters to make. If we request the algorithm to make two clusters, the answer obtained is different from that obtained when requested to make four.

Anomaly Detection

Another extremely popular and different family of unsupervised algorithm attempts to detect anomaly or tries to find outliners to the data. There’s no explicit teacher labelling the historic data as anomalies instead, the algorithm learns on its own what normal looks like by simply observing the data. One anomaly detection algorithm was developed by scientists working at Amazon. It is called random cut forest. The algorithm works by constructing a forest of the so-called random cut trees. Each tree is constructed by a recursive procedure which first surrounds the data points with a bounding box and then cuts or splits at along the coordinate axis by picking cut points randomly. The procedure is repeated until every point is sorted into a particular leaf of the tree.

Topic Modelling

Yet another example of an unsupervised algorithm is the so-called topic modelling for documents with text content. The algorithm is the basis of the eponymous feature in the Amazon comprehend service. Given a collection of documents, news articles for example and the number of topics we would like to discover, the algorithm produces the top words that appear to define the topic, together with the weight that each of these words has in relation to the topic. As with clustering in general, the approach is sensitive to the number of topics requested and it still requires us to assign the meaning to the discovered topic such as health, sports or politics.

To summarize, Amazon SageMaker includes a popular clustering algorithm called k-means. It’s an Amazon improvement over the well-known and scalable algorithm called Web-scale k-means. Another member of the unsupervised family is called Principal Component Analysis or PCA for short. Likewise available in SageMaker. It’s especially useful in reducing the dimensionality of the dataset and is often used as a feature engineering step before passing the data to a supervised algorithms. Latent Dirichlet Allocation or LDA is the name of a particular topic modelling algorithm. A variant is used by the topic modelling feature of Amazon comprehend and the algorithm is also available in sage maker. The Random cut forest algorithm for anomaly detection is available in SageMaker as well as an Amazon Kinesis Data Analytics, for easy application to streaming data. Kinesis Data Analytics also features hotspot detection, another example of an unsupervised learning algorithm which you can use to identify relatively dense regions in your data.

Convolutional Neural network

Photo by Alina Grubnyak on Unsplash

One important breakthrough in deep learning was the invention of the so-called Convolutional Neural Networks or CNNs for short, which are especially useful for image processing. The main idea behind CNNs is that it is able to relate nearby pixels in the image instead of treating them as completely independent inputs, which was the case prior to CNNs. A special operation called convolution is applied to the entire subsections of the image and more importantly, the parameters of these convolutions are also being learned in the process. If several convolutional layers are stacked one after another, each convolutional layer learns to recognize patterns of increasing complexity as we move through the layers. For example, they can be used for semantic segmentation or classification of individual pixels as belonging or not belonging to detected objects.

If we take the output of a neuron and feed it as input to itself or neurons from previous layers, we are creating the so-called recurrent neural networks. It’s as if the neuron remembers its output from the previous iteration thus creating a kind of memory. On the right-hand side, you see just one unit of a more complex network called LSTM, which stands for Long Short-Term Memory. It is commonly used for speech recognition and translation. In fact, LSTMs are used as a building block to the Sequence to Sequence modelling which is used in neural machine translation. Amazon released an entire library called Sockeye for state of the art sequence to sequence modelling tasks that customers can use in their projects.

Amazon SageMaker conveniently provides a built-in algorithm for image classification based on Resnet, a kind of CNN, but it also provides a sequence to sequence algorithm, a neural topic modelling algorithm to complement Latent Dirichlet allocation and also DeepAR forecasting algorithm for time series…

It was really fun writing this article. I hope you enjoyed reading this as much as I enjoyed writing…..!

Thank you