Deep Learning Program Learns to Paint | Two Minute Papers #49
Artificial neural networks were inspired by the human brain and simulate how neurons behave when they are shown a sensory input (e.g., images, sounds, etc). They are known to be excellent tools for image recognition, any many other problems beyond that - they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others. Deep learning means that we use an artificial neural network with multiple layers, making it even more powerful for more difficult tasks. 

This time they have been shown to be apt at reproducing the artistic style of many famous painters, such as Vincent Van Gogh and Pablo Picasso among many others. All the user needs to do is provide an input photograph and a target image from which the artistic style will be learned.

_______________________


The paper "Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis" is available here:
http://arxiv.org/pdf/1601.04589v1.pdf

Previous work - the paper "A Neural Algorithm of Artistic Style" is available here
http://arxiv.org/pdf/1508.06576v2.pdf

Subscribe if you would like to see more of these! - http://www.youtube.com/subscription_center?add_user=keeroyz

Recommended for you:
Deep Neural Network Learns Van Gogh's Art - https://www.youtube.com/watch?v=-R9bJGNHltQ&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e&index=42
Artificial Neural Networks and Deep Learning - https://www.youtube.com/watch?v=rCWTOOgVXyE&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e&index=31
How Does Deep Learning Work? - https://www.youtube.com/watch?v=He4t7Zekob0&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e&index=39
9 Cool Deep Learning Applications - https://www.youtube.com/watch?v=Bui3DWs02h4&list=PLujxSBD-JXgnqDD1n-V30pKtp6Q886x7e&index=36

The shown website with neural art:
http://deepart.io/

Check this one out too (it is a different implementation)!
https://deepforger.com/
https://twitter.com/deepforger

Thumbnail image: Andreas Achenbach - Clearing Up
Splash screen/thumbnail design: Felícia Fehér - http://felicia.hu

Károly Zsolnai-Fehér's links:
Patreon → https://www.patreon.com/TwoMinutePapers
Facebook → https://www.facebook.com/TwoMinutePapers/
Twitter → https://twitter.com/karoly_zsolnai
Web → https://cg.tuwien.ac.at/~zsolnai/
 Dear Fellow Scholars, this is two-minute papers with Karo Ejona Ifehir. In a previous episode, we discussed how a machine learning technique called a convolutional neural network could paint in the style of famous artists. The key thought is that we are not interested in individual details. We want to teach the neural network the high level concept of artistic style. A convolutional neural network is a fantastic tool for this, since it does not only recognize images well, but the deeper we go in the layers, the more high level concepts neurons will uncode, therefore the better idea the algorithm will have of the artistic style. In an earlier example, we've shown that the neurons in the first hidden layer will create edges as a combination of the input pixels of the image. The next layer is a combination of edges that create object parts. One layer deeper, a combination of object parts create object models, and this is what makes convolutional neural networks so useful in recognizing them. In this follow-up paper, the authors use a very deep 19 layer convolutional network that they mix together with Markov random fields, a popular technique in image and texture synthesis. The resulting algorithm retains the important structures of the input image significantly better than the previous work, which is also awesome, by the way. Failure cases are also reported in the paper, which was a joy to read. Make sure to take a look if you're interested. We also have a ton of video resources in the description box that you can voraciously consume for more information. There is already a really cool website where you either wait quite a bit and get results for free or you pay someone to compute it and get results almost immediately. If any of you are in the mood of doing some neural art of something two minute papers related, make sure to show it to me. I'd love to see that. As a criticism, I've heard people saying that the technique takes forever on an HD image, which is absolutely true. But please bear in mind that the most exciting research is not speeding up something that runs slowly. The most exciting thing about research is making something possible that was previously impossible. If the work is worthy of attention, it doesn't matter if it's slow. Three follow-up papers later, it will be done in a matter of seconds. In summary, the results are nothing short of amazing. I was full of ecstatically when I first seen them. This is insanity and it's only been a few months since the initial algorithm was published. I always say this, but we are living amazing times indeed. Thanks for watching and for your generous support and I'll see you next time.
[{"start": 0.0, "end": 4.92, "text": " Dear Fellow Scholars, this is two-minute papers with Karo Ejona Ifehir."}, {"start": 4.92, "end": 9.74, "text": " In a previous episode, we discussed how a machine learning technique called a convolutional"}, {"start": 9.74, "end": 14.08, "text": " neural network could paint in the style of famous artists."}, {"start": 14.08, "end": 18.12, "text": " The key thought is that we are not interested in individual details."}, {"start": 18.12, "end": 22.72, "text": " We want to teach the neural network the high level concept of artistic style."}, {"start": 22.72, "end": 27.72, "text": " A convolutional neural network is a fantastic tool for this, since it does not only recognize"}, {"start": 27.72, "end": 32.92, "text": " images well, but the deeper we go in the layers, the more high level concepts neurons will"}, {"start": 32.92, "end": 38.16, "text": " uncode, therefore the better idea the algorithm will have of the artistic style."}, {"start": 38.16, "end": 43.04, "text": " In an earlier example, we've shown that the neurons in the first hidden layer will create"}, {"start": 43.04, "end": 46.64, "text": " edges as a combination of the input pixels of the image."}, {"start": 46.64, "end": 51.28, "text": " The next layer is a combination of edges that create object parts."}, {"start": 51.28, "end": 56.56, "text": " One layer deeper, a combination of object parts create object models, and this is what makes"}, {"start": 56.56, "end": 60.24, "text": " convolutional neural networks so useful in recognizing them."}, {"start": 60.24, "end": 65.92, "text": " In this follow-up paper, the authors use a very deep 19 layer convolutional network"}, {"start": 65.92, "end": 71.04, "text": " that they mix together with Markov random fields, a popular technique in image and texture"}, {"start": 71.04, "end": 72.04, "text": " synthesis."}, {"start": 72.04, "end": 76.30000000000001, "text": " The resulting algorithm retains the important structures of the input image significantly"}, {"start": 76.30000000000001, "end": 80.64, "text": " better than the previous work, which is also awesome, by the way."}, {"start": 80.64, "end": 84.52000000000001, "text": " Failure cases are also reported in the paper, which was a joy to read."}, {"start": 84.52, "end": 86.56, "text": " Make sure to take a look if you're interested."}, {"start": 86.56, "end": 91.11999999999999, "text": " We also have a ton of video resources in the description box that you can voraciously"}, {"start": 91.11999999999999, "end": 93.28, "text": " consume for more information."}, {"start": 93.28, "end": 97.8, "text": " There is already a really cool website where you either wait quite a bit and get results"}, {"start": 97.8, "end": 102.28, "text": " for free or you pay someone to compute it and get results almost immediately."}, {"start": 102.28, "end": 106.52, "text": " If any of you are in the mood of doing some neural art of something two minute papers"}, {"start": 106.52, "end": 108.47999999999999, "text": " related, make sure to show it to me."}, {"start": 108.47999999999999, "end": 109.92, "text": " I'd love to see that."}, {"start": 109.92, "end": 114.94, "text": " As a criticism, I've heard people saying that the technique takes forever on an HD image,"}, {"start": 114.94, "end": 116.48, "text": " which is absolutely true."}, {"start": 116.48, "end": 121.2, "text": " But please bear in mind that the most exciting research is not speeding up something that"}, {"start": 121.2, "end": 122.52, "text": " runs slowly."}, {"start": 122.52, "end": 127.1, "text": " The most exciting thing about research is making something possible that was previously"}, {"start": 127.1, "end": 128.2, "text": " impossible."}, {"start": 128.2, "end": 131.64, "text": " If the work is worthy of attention, it doesn't matter if it's slow."}, {"start": 131.64, "end": 135.92000000000002, "text": " Three follow-up papers later, it will be done in a matter of seconds."}, {"start": 135.92000000000002, "end": 139.4, "text": " In summary, the results are nothing short of amazing."}, {"start": 139.4, "end": 142.36, "text": " I was full of ecstatically when I first seen them."}, {"start": 142.36, "end": 147.48000000000002, "text": " This is insanity and it's only been a few months since the initial algorithm was published."}, {"start": 147.48000000000002, "end": 151.32, "text": " I always say this, but we are living amazing times indeed."}, {"start": 151.32, "end": 178.44, "text": " Thanks for watching and for your generous support and I'll see you next time."}]