Introduction to Convolutional Neural Networks for Self Driving Cars
By: Prateek Sawhney
Tags: Deep Learning, Artificial Intelligence, Machine Learning, Data Science, Self Driving Cars

Convolutional Neural Networks (CNN)

Let’s talk about Convolutional Networks, or ConvNets. ConvNets are neural networks that share their parameters across space. Imagine we have an image. It can be represented as a flat pancake. It has a width, a height, and

because we typically have red, green, and blue channels, it also has a depth. In this instance, depth is three. That’s our input. Now, imagine taking a small patch of this image, and running a tiny neural network on it, with K outputs.

Convolution Operation (Image by author)

Now, let’s slide that little neural network across the image without changing the weights. Just slide across invertically like we’re painting it with a brush. On the output, we’ve drawn another image.

Patch over a dog image (Image by author)

It’s got a different width, a different height. And more importantly, it’s got a different depth. Instead of just R, G, and B, now, we have an output that’s got many colored channels, K of them. This operation is called the convolution.

Shifted Patch over a dog image (Image by author)

If our patch size were the size of the whole image, it would be no different than the regular layer of a neural network. But because we have this small patch instead, we have many fewer weights and they are shared across space. A convolutional neural network is going to basically be a deep network where instead of having stacks of matrix multiply layers, we’re going to have stacks of convolutions. The general idea is that they will form a pyramid. At the bottom, we have this big image, but very shallow just R, G, and B. We’re going to apply convolutions that are going to progressively squeeze the spacial dimensions while increasing the depth which corresponds roughly to the semantic complexity of your representation.

At the top, we can put our classifier. We have a representation where all this spacial information has been squeezed out, and only parameters that map to content of the image remain. So that’s the general idea. If we’re going to implement this, there are lots of little details to get right, and a fair bit of lingo to get used too. We’ve know the concept of Patch and Depth. Patches are sometimes called Kernels. Each pancake in our stack is called a feature map.

Another term that we need to know is stride. It’s the number of pixels, so that we’re shifting each time we move our filter. The stride of one makes the output roughly the same size as the input. A stride of two means it’s about half the size. I say roughly because it depends a bit about what we do at the edge of our image.

Either we don’t go pass the edge, and it’s often called valid padding as a shortcut. Or we go off the edge and pad with zeros in such a way that the output map size is exactly the same size as the input map. That is often called same padding as a shortcut.

Hierarchy Diagram showing detection at various layers of a convolutional neural network (Image by author)

That’s it, we can build a simple convolutional neural network with just this. Stack up our convolutions which thankfully we don’t have to implement ourselves. Then use triads to reduce the dimensionality and increase the depth of our network layer after layer. And once we have a deep and narrow presentation, connect the whole thing to a few regular, fully connected layers, and we’re ready to train our classifier. You might wonder what happens to training and to chain rule, in particular, when you use shared weights like this. Nothing really happens, the math just works. You just add up the derivatives for all the possible locations on the image.