Decalcomania | Neural style transfer
Channel: Joan Charmant
Date: 2019-02-25

Experimenting with neural style transfer and various deep learning algorithms and frameworks. (This video needs to be seen on a decent sized screen for the effect). Most of this video is aided by deep learning in some way or another, even the music. More details after the jump.

Software stack:
- The optical flow was computed with the Flownet2 neural network, using Patrick Wieschollek's Tensorflow implementation in the TensorPack samples: https://github.com/tensorpack/tensorpack/tree/master/examples/OpticalFlow
- The slow motion and time remapping was done using the Super SloMo network, using a modified version of Avinash Paliwal's PyTorch implementation at https://github.com/avinashpaliwal/Super-SloMo. My modified version uses external optical flow files.
- Sky masking was done using OpenCV implementation of the flood-fill algorithm. https://en.wikipedia.org/wiki/Flood_fill
- Neural style transfer was applied using Cameron Smith TensorFlow implementation: https://github.com/cysmith/neural-style-tf
- The music was generated on JukeDeck: https://www.jukedeck.com/make/tracks/browse except the voice over which I sampled from an old documentary.

Misc:
- The aerial footage was captured in 2704x1520 at 120 fps, on a 5" freestyle quadcopter.

t7-1080p60-CBR30Mbps