Neural networks and in particular convolutional neural networks have been at the heart of many recent research projects with an artistic flavor. Some of the better known ones have been deep-dream, a neural algorithm of artistic style or style transfer, deep generator networks, and most recently wave nets which learn to generate audio. They've also been found within many practical applications, so everything from self-driving cars to speech detect systems and AIs that can play the game of go. The recent success comes from an ability to accurately recognize and describe images, but the way they do this remains a mystery to most people. We can get a few intuitions about what's really going on by inspecting them, looking inside them and seeing how they see the world. What you're looking at is a neural network which is processing my webcam in real time. It scans the image looking for patterns or what we call features. The patterns look like these. Some are lines or edges or gradients, just really minimal multipixel patterns, things like that. These responses, which we sometimes call feature maps or activations, show us the presence of those features inside of the image. In the first layer of the network we've discovered edges, gradients, and patterns like that. Things get a little more interesting when we repeat this process many times through a sequence of layers. At each layer of the network we take the feature maps from the previous layer, stack them together into a new volume of data, and do another round of convolution on top of them. So the activation maps in the second layer, which we're looking at here, are more interesting because rather than looking for patterns from the raw pixels of the original image, we're now looking for patterns from the activation maps of the previous layer of the network. So for example, it might be able to combine vertical edges and horizontal edges to detect corners, which we can think of as higher level features. As we do this process many times, progressing through every layer of the network, we acquire higher and higher level features or representations of the image. So we go from things like edges and gradients to corners and grids to yet progressively more complex features, maybe things like leaves or fences or door handles, to yet even higher level features, houses, cars, people, and so on. This process of pushing data through the network over many layers of transformations is why these algorithms are sometimes called deep neural networks or deep learning. The deep just means the network has many layers. When we finally arrive at the last layer of the network, we have this compact representation of the content inside of the image, and we can attach one more classification layer on top of that so that we can describe accurately what's inside the image. So for example, if I place my phone in front of the camera, it'll go iPod. Or if I place this water bottle in front of the camera, it'll accurately detect the water bottle. Now, it can be a bit hard to understand what the feature detectors are looking for, but it turns out that there are ways and there has been some work done in the past. Some of the first work came from Rob Zyler and Matt Ferguson 2013, where they showed patches of actual images, which caused certain feature detectors to light up. Another nice resource is Deep Visualization Toolbox, which was made by Jason Yosinski et al. And was a major inspiration for the visualization software that you saw in the last slide. If you're interested in learning more about how these impressive algorithms work, or even getting your hands dirty and working with them yourself using a series of practical guides and tutorials, I encourage you to check out ML4A.github.io, which is an in progress free online book about machine learning for artists.