What is Deep Learning and How Deep Learning Works?
By: Cagri Ozarpaci
Tags: Deep Learning, AI, Artificial Intelligence, Computer Science, Data Science

By the technology develops, we’re getting into a new era that full of human-like robots. It wasn’t possible even to imagine that is coming 100 years ago. Since the increase in technology has exponential growth, advanced robots will occur in a much shorter period.

What is Artificial Intelligence?

It’s better to understand the AI (Artificial Intelligence) concept before jumping into Deep Learning. We can define AI as a system that analyzes its environment and takes actions to maximize its chance of success without being explicitly programmed. As we got a brief idea about AI, let’s inspect some subfields of AI roughly.

Natural Language Processing

Machine Learning

Neural Networks

Robotics

But, What about Deep Learning? Where is it? Well, deep learning is not directly a subfield of AI. It is a subfield of some subfields since it uses multiple fields in its application. For the sake of visualizing what we’ve talked about, look at the following Venn Diagram:

So, What Is Deep Learning?

Deep learning is a method of artificial intelligence that mimics the functioning of the human brain while interpreting data and generating patterns for use in making decisions.

Deep Learning is a subset of machine learning, as shown in the Venn Diagram above. Deep Learning has networks capable of learning from data that is mostly unsupervised. Deep Learning also is known as a deep neural network or deep neural learning.

Deep Learning also uses a subfield of AI, which is Neural Networks. A Perceptron is a single layer neural network; a multi-layer perceptron is called Neural Networks.

What the heck is The Perceptron?

Neuron vs. Perceptron — Image from IF Inteligencia Futura

A Perceptron is nothing but an artificial neuron. A Neuron takes the input signals and processes it then outputs a signal; A Perceptron takes inputs processes it and then outputs a processed data. The picture above may seem a bit complicated and rough, so let’s inspect it in more detail.

How The Perceptrons works

The diagram above visualizes how they work quite clearly. In Fact, the formula at the right explains everything. A Perceptron takes an input matrix and multiplies (application of the dot product) it with each different weight matrix then, sums all of these dot products. After that, an activation function applies to that result summation. In our world, most of the things are not linear; so, the activation function we use is mostly non-linear.

In Fact, what I’ve explained above is not entirely correct. Mostly, we also add a bias term before applying the activation function. That addition of bias term allows us the shift our activation function. Notice that, while we’re programming this algorithm, we’re specifically using an activation function that coded by another programmer, or we’re coding our activation function from scratch.

Why Is Non-Linearity Important?

As we clearly can see from the picture above, If we use a linear function to separate the triangles between the circles, it can’t be said that we’re doing a good job. But, If we use a non-linear function to do that task, we can easily separate them without waiving anything.

Let’s say that we have a classification algorithm that classifies if a picture we provide is a dog’s or cat’s image. Assume the triangles in the graph above is represents the cats, the circles represent the dogs. If we use a linear activation function, our algorithm won’t be successful enough to classify 6 pictures correctly in total. Since 3 triangles and 3 circles are not located where they supposed to be. But if we use a non-linear activation function, that failure disappears! We’re able to separate them properly with a non-linear function.

How does the Learning Process start?

The machine needs to know what’s right and what’s wrong to learn correctly. In other words, we need to feed it with some data. Let’s get back to our dog and cat classification example.

So, we’re feeding our deep learning algorithm with some data that includes cat and dog pictures. While we’re feeding with these data, we’re providing correct answers to the machine. The machine changes these weight matrices during this learning process. So, we don’t have constant weight matrices all the time.

Then, we’re providing random pictures to classify whether it’s a cat or dog. How accurate results we get mainly depends on the amount of data we provide to the machine. The activation function and bias we use also affect the accuracy of our deep learning algorithm.

To understand more about programming, AI, and technology, follow the upcoming posts.

Any comments, or if you have any questions, write it in the comment!