Convolutional Neural Networks
By: Parikshit Kumar
Tags: Computer Vision, Machine Learning, Deep Learning, Convolution Neural Net, Tutorial

Convolutional Neural Networks

An attempt at explaining all the technicalities involved with convolutional neural networks

What is a CNN?

A Deep Learning algorithm which can take in an input image, assign different features with different weights and biases/give different features different importance while being able to differentiate between input images

Why Use CNNs?

Before we get into what the benefits of using a CNN are, let’s discuss why we use them in the first place and why do we not just use vanilla Neural Networks?

The whole idea of CNNs is to map image data to an output variable efficiently, the problem with using a vanilla neural network is the fact that you will not be extracting the best features out of the image dataset, as a matter of fact you might even lose important features because you will need to flatten the images in order for that image to be used as an input before you can even get the necessary features that could be crucial to building a good deep learning model

What features do we need to extract?

There are several features a CNN model will learn while training and as we go deeper into the CNN, the model will learn more complex features, one of the most significant being the edges of an object in the input image and extracting features like these is done by performing what is referred to as the convolution operation

Edge Detection Example

To detect edges we employ a filter(also referred to as a kernel) and convolve it with the input image we have, the input image and the filter are both assumed to be 2-D matrices for the sake of explanation.

The convolution operation is denoted by an asterisk — ‘*’

We get the output of the convolution by visualizing the filter going over the entire image and performing matrix multiplication for the corresponding elements, in this example we detect the vertical edge(filter can be transposed for horizontal edge)of a 6*6 using a 3*3 filter, the result being a 4*4 matrix

As we can see in the following example, the numbers or values in the matrix define the intensity of colour in the image(more positive implying brighter and more negative implying darker), the result being a light portion in the middle ,sandwiched between two darker portions, on a larger scale this appears as a line, and on a full fledged image, this appears as an edge, essentially the transition between these colours is what appears as an edge to us

Size and Types of filters

The typical size of a filter is usually 3*3 and sometimes even 5*5, there is no inherent problem with using an even dimension filter but using an odd dimension filter allows the edges to be centred in the output image, there are different kinds of filters used for different variations of edge detection, like Sobel and Scharr and we can even have the the filter values be initialized as weights and have them trained in the process of back propagation

(source: coursera)

Padding

When we convolved the 6*6 image with a 3*3 filter, there were 4*4 places where it could fit, the threat here is missing out on the outer edge features because the filter convolves on the middle of the image so much more frequently, another threat is potentially making the output too small to be significant and therefore we use padding, essentially increasing the size of the image by adding extra dimensions to the edges so the previous edges are convolved a lot more frequently and we don’t miss out on features or make the output too small. In the example, we converted the 6*6 image to 8*8 and convolved it with a 3*3 filter resulting in another 6*6 image = original size of the image

(source: coursera)

Types of Convolutions

N — IMAGE DIMENSION, F — FILTER DIMENSION ,P — PADDING DIMENSION

a) Valid (no padding) :

N x N (image) * f x f (filter) = (n-f+1)*(n-f+1) (output image size)

b) Same (padding so the initial image size = output image size)

The output image size here is given by (n+2p-f+1)*(n+2p-f+1) and the necessary padding can be calculated by using the formula p = (f-1)/2

Same padding visualization:

Strided Convolutions

The stride in a convolution is the size of the step we take when we convolve over an image, if the stride is 2, instead of shifting one to the right, we shift 2, the striding process can save a significant amount of time by reducing calculations while also reserving the necessary features

(source:Coursera)

(source:Coursera)

stride = 2 convolution visualization

Convolutions over RGB images and volume

RGB — red green blue image

Like the typical 2-D convolutions explained previously this convolution just has another channel or a third dimension, to convolve over an RGB image or volume you need the filter to have an extra channel, one that is equal to the third channel in the RGB image.

The number of filters we use determines the third dimension of the output, for example if we convolve a 6*6*3(height*width*channels) with one 3*3*3 filter the resultant output is a 4*4*1 or just a 4*4 image, but if the number of filters are 2 then the resultant is a 4*4*2 image

(source: google images)

(source: google images)

In a single convolutional layer

a) Image is filtered

b) Passed through a non-linearity(activation function like ReLU) and biases are added

c) Finally all those convolutions we applied with multiple filters are added together to complete a conv layer

(source: Coursera)

Parameter Calculation and output sizes:

You don’t need to remember all of these notations and formulae, the output image size is essentially ((input size + 2*padding size — filter size) divided by the stride size + 1) rounded down, the square brackets in the notation imply greatest integer function

(source: google images)

Types of layers in a conv net

a) Convolution layer (CONV)

b) Pooling layer (POOL)

c) Fully connected layer (FC)

Pooling Layers:

Reduces computation, speeds up computation, makes features more robust.

Two Types:

a) Max Pooling

b) Average Pooling

Max Pooling:

Think of a 4*4 input image and a 2*2 pooling layer, visualize the division of the image into the size of the pooling layer and select the maximum value from that particular pool matrix

The hyperparameters of pooling are filter size and stride in this case both are taken to be 2, the pooling layer has no other parameters to train.

In max pooling the differentiating feature(max number) is preserved and hence made more robust

Pooling also does not affect the third channel in an image

(source: Coursera)

Average Pooling:

Think of a 4*4 input image and a 2*2 pooling layer, visualize the division of the image into the size of the pooling layer and the average of all values from that particular pool matrix is selected.

There usually is no padding when pooling so the the output formula can be reduced to:

[(Nh-f)/s+1] * [(Nw-f)/s+1] * Nc

(source: Coursera)

CNN Example and Architecture:

Fully connected layers are layers that have each of their cells connected fully to the previous layers and the layers that come after it, although the softmax function isn’t specific to CNNs, it can be thought of as generalized sigmoid that predicts the probability of each class, given by the formula:

In this example, let us look at the LeNet-5 architecture used to classify handwritten digits from 0 -9

(source: coursera)

(source: coursera)

Other Popular CNN architectures:

LeNet-5

AlexNet

VGG 16

Inception (GoogLeNet)

ResNet

DenseNet

Implementation of a CNN in PyTorch:

And Lastly, what are the benefits and why/how do they even reduce computation time?

As mentioned previously, CNNs allow us extract better features and compute significantly faster, but how do they do it?

There are 2 major advantages of a conv layer over standard fully connected layers, namely:

a) Parameter sharing

b) Sparsity of connections

Parameter Sharing:

Example: 32*32*3 input convolved with 5*5*3 filter with 6 filters to give a 28*28*6 feature map:

32*32*3 = 3072

28*28*6 = 4704

3072 * 4704 in a typical fc layer without using a convolutional layer is approximately = 14M parameters.

Parameter sharing essentially means that a feature detector(filter) which is useful in one part of the image is probably useful on the other part as well, so the total number of parameters are significantly reduced

Sparsity Of Connections:

In each layer, each output value depends only on a small number of inputs, in the example below, the output 0 is only dependent upon the first 3*3 part of the input image

(source: Coursera)

Bibliography/References:

Convolutional Neural Networks ← hyperlink

Deep Learning specialization course (deeplearning.ai) by Coursera, taught by Andrew Ng for pictures and illustrations

Other GIFs source:

A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way | by Sumit Saha | Towards Data Science ← hyperlink