In this paper, we propose a generative model, Temporal Generative Adversarial Nets (TGAN), which can learn a semantic representation of unlabeled videos, and is capable of generating videos. Unlike existing Generative Adversarial Nets (GAN)-based methods that generate videos with a single generator consisting of 3D deconvolutional layers, our model exploits two different types of generators: a temporal generator and an image generator. The temporal generator takes a single latent variable as input and outputs a set of latent variables, each of which corresponds to an image frame in a video. The image generator transforms a set of such latent variables into a video. To deal with instability in training of GAN with such advanced networks, we adopt a recently proposed model, Wasserstein GAN, and propose a novel method to train it stably in an end-to-end manner. The experimental results demonstrate the effectiveness of our methods.
In video synthesis, REF splits the synthesis between two generators, the temporal generator that outputs a latent variable for each frame and the spatial generator that generates the frame from the latent variables.
Temporal Generative Adversarial Nets with Singular Value Clipping
2017 IEEE International Conference on Computer Vision (ICCV)
2017 IEEE International Conference on Computer Vision (ICCV)
Computer Science Mathematics