Task: Language Identification
Input (Spanish, English): This paper considers the energy-based model interpretation of GAN, where the discriminator is an unnormalized model for the likelihood of a generative model p(x|theta) and the generator is a directed model that approximates this distribution. The generator is used to draw approximate negative phase samples that are used in stochastic maximum likelihood / contrastive divergence learning of the EBM / discriminator.  The main idea in the paper is to fit the generator by following the Stein variational gradient. In practice this gradient consists of the usual gradient provided by the discriminator with an added term that provides a repulsive force between the sampled data points to increase sample diversity.  The idea of using a kernel to push apart the sampled points is interesting, and will work in low dimensions, but it is hard to see how it can work in full scale images. For high dimensional samples x, the proposed kernel is unlikely to provide a useful distance measure between points. There are no convincing experiments in the paper that show otherwise. Specifically:  - There is no experiment that compares between standard GAN and GAN + repulsion, using the same architecture. (please address this in the rebuttal) - If the Stein variational idea is taken literally, the right thing to do would be to fully optimize the generator at every step, and then taking a single optimization step on the discriminator. Instead, each is updated in turn, and the learning rates of both steps are adjusted to keep the two "in line". - The kernel used to fit the generator is defined in the auto-encoder space of the discriminator, and thus depends on the discriminator parameters. The objective that is used to fit the generator thus changes at every step, and the procedure can no longer be interpreted as stochastic gradient descent with respect to any single well defined objective.  The authors obtain good results: The generated images clearly look better than those generated by DCGAN. However, their approach has a number of changes compared to DCGAN, so it is not clear where the improvement comes from. In addition, by now the DCGAN is no longer a very strong baseline, as various other techniques have been proposed.  Note: The use of phi for both the "particle gradient direction" and energy function is confusing
Output (English): en