What does the Network see?
By: Hmrishav Bandyopadhyay
Tags: Convolutional Network, Artificial Intelligence, Neural Networks, AI, Data Science

We have evolved and become more intelligent creatures since then. We don’t blindly follow predictions by astrologers, right? Indeed, we do not harbor superstitions anymore — but we still blindly follow them. Astrologers? No! — Neural Networks!

Neural Networks have been the biggest boon to the AI research community — they have helped solve completely unsolvable problems in the realm of AI and have given outstanding results in a very short period of time. However, along with the results, they have brought a new term into AI — the black box

In neural networking or heuristic algorithms (computer terms generally used to describe ‘learning’ computers or ‘AI simulations’), a black box is used to describe the constantly changing section of the program environment which cannot easily be tested by the programmers. This is also called a white box in the context that the program code can be seen, but the code is so complex that it is functionally equivalent to a black box. — https://en.wikipedia.org/wiki/Black_box

Modern complex AI techniques, such as Deep Learning and genetic algorithms are naturally opaque — one might guess but can never know what is actually happening inside the network. As a result of their opaqueness, applying them in clinical fields has been near to impossible as people can’t trust an unexplained network with their lives.

This naturally gave rise to Explainable AI — a research field where the solutions provided by a neural network are examined and possibly interpreted by an expert. Explainable AI (XAI), however, is still vastly under development and significant advances have not been made to bridge the gap between researchers and machines. As of now, research works are largely dependent on intuition mixed with bits and pieces of XAI integrations.

Let’s explore the intuition behind the working of a simple CNN classifier network — one of the most popular networks of the time.

[Note: This is not a guide for CNNs — rather I would like the reader to have some knowledge of CNNs and their working as this is an intuition-based explanation of the methods used. Needless to say, this article would not cover the complete basics of a general CNN]

Convolutional Neural Networks

To brush up the basics, a convolutional neural network is a type of neural network where we get the outputs after convolving the input with a kernel that is learnt by the machine. In other words, the convolving kernel keeps updating its parameters till the outputs come close to the desired value or the loss function reaches a low.

A pictorial representation would be —

A 32x32x3 image convolved with a 3x3 filter

Thus, we can see that the 3x3 kernel convolves the 32x32 input into the 30x30 output. A 3x3 window slides over the input image (all channels) and we obtain the dot product of the 3x3 window and the kernel — which we represent in the output by a single point. A single convolutional operation thus gives us a single channeled image. A set of ‘n’ convolutional filters or a convolutional ‘block’ would give us an ‘n’ channeled output as the outputs from each filter is concatenated along the channel width. But, why use ‘n’ filters?

A single filter can capture at most a single feature or detect a specific characteristic trait of the input. The inability of the single filter to capture more data makes it imperative to use multiple filters where multiple features need to be studied.

Over time the network learns the values of the filter matrix for all the filter in all the convolutions and thus begins to identify patterns in the input it has seen before. It can now perform complex tasks like object detection and segmentation.

Since we are generating the filter values, we can say this is both a classification and a regression problem — we regress the kernel matrix and ultimately classify the image.

Now, if we are ultimately using filters and getting outputs, why cannot we complete the entire network in a single convolutional block?

We cannot use a single conv block to get the whole work done because the features come in layers too! Unless the low-level features are extracted from the image, we cannot get to the mid-level or high-level features. Thus, we have to first extract the low-level features from the image with the help of the convolutions we use in the beginning of the network. After the low-level features have been extracted, we perform future operations on the convolved output to extract mid-level and high-level features respectively. Effectively, we are starting with simpler features and aggregating these into complex features later on.

An instance of features at different levels in the VGG-16 architecture

Now, what exactly are the low-level features in an image?

By low level features, we are talking about features like edges and boundaries— the most basic and most important of all information. The mid-level features would represent features like corners and blobs and at the end, we would get the high level features that would help us classify or segment the image as per our requirement.

This is the general intuition behind why we use convolution and why we use so many layers in a full convolutional network. There are other aspects of the said networks — aspects like dropout for controlling regularization, activations for introducing non-linearities, Batch Normalization for normalizing the activations of each layer, and many more. Let me know if you would like to read more about the intuition behind these aspects of a network — I won’t disappoint you ;)

Check out my blog for faster updates and don't forget to subscribe for quality content :D

Hmrishav Bandyopadhyay is a 2nd year Undergraduate at the Electronics and Telecommunication department of Jadavpur University, India. His interests lie in Deep Learning, Computer Vision, and Image Processing. He can be reached at — hmrishavbandyopadhyay@gmail.com || https://hmrishavbandy.github.io