In this work, we investigate a novel training procedure to learn a generative model as the transition operator of a Markov chain, such that, when applied repeatedly on an unstructured random noise sample, it will denoise it into a sample that matches the target distribution from the training set. The novel training procedure to learn this progressive denoising operation involves sampling from a slightly different chain than the model chain used for generation in the absence of a denoising target. In the training chain we infuse information from the training target example that we would like the chains to reach with a high probability. The thus learned transition operator is able to produce quality and varied samples in a small number of steps. Experiments show competitive results compared to the samples generated with a basic Generative Adversarial Net.
There's also another related recent approach to learning a transition operator with a denoising cost, developed in parallel, called Infusion training REF , which tries to reconstruct the target data in the chain, instead of the previous step in the destructive chain.
Learning to Generate Samples from Noise through Infusion Training
ArXiv
ArXiv
Mathematics Computer Science