Here are the mind-blowing things a deconvolutional neural network can do
By: Mark Farragher
Tags: Artificial Intelligence, Machine Learning, Deep Learning

Convolution is a revolutionary innovation that took the machine learning world by storm. This mathematical process dramatically improves the accuracy of image recognition in neural networks.

Thanks to convolution, neural networks were finally able to beat humans in the famous ImageNet visual classification challenge.

Here is a typical convolutional neural network (CNN). This is the VGG16 algorithm that won ImageNet in 2014:

This neural network processes an image (on the left of the diagram) and classifies it into one out of a thousand possible categories (on the right).

When I show a picture of a chair to the network, the output node for the ‘chair’ category will light up on the right.

So far so good.

But one day, a machine learning researcher asked a brilliant question:

Can we run a CNN in reverse?

What if we activate the ‘chair’ category on the right and run the whole network backwards. Will it then produce a complete image of a chair?

The answer is ‘YES’, and it has profound implications for the field of machine learning.

So here’s how it works.

First, we need to build a suitable network architecture. Something like this:

Reading this architecture from right to left, it’s just another convolutional neural network that takes an image of a chair and classifies it.

But when run it in reverse, from left to right, it will convert a class vector to a complete chair image.

In essence, the network is producing new images of chairs from scratch. Here’s what that looks like:

These images were generated by feeding slightly different class vectors into the network.

But we can go further than that.

Le’s say we train the network on a bunch of chair images, and we label each image with two numbers: the type of chair, and the angle we are seeing it from.

A label of [1, 0] could mean: a garden chair seen from the front, and [2, 90] could mean: a living room sofa seen at a 90 degree angle.

Now what happens if we run this network backwards, pick a random chair type, and then slowly increase the angle value?

This is what happens:

This is mind-blowing.

Keep in mind that the neural network has never learned these specific images. Instead, it’s producing a chair image from scratch and rotating it correctly based on what it has learned about perspective from other chair images.

And this will work for any type of image, not just chairs. You can do it for cars, houses, landscapes, and even human faces.

Do you realize what this means? It means that…

Machine learning can produce images of any kind of object: chairs, cars, landscapes, interior design, etc.

Machine learning can take a photo of an object, and generate a new image where we see the object from a different angle.

Machine learning can generate faces of non-existent people

Machine learning can take an image of a person, and change their facial expression, their clothes, and even their body pose.

We’re just scratching the surface of what is possible with this technique.

What kind of images would you create with this trick?

Write a comment and let me know!