Encoder Discriminator Networks for Unsupervised Representation Learning
Authors: Nils Wandel
Keywords: representation learning, unsupervised, encoder discriminator
Abstract: Learning representations of data samples in an unsupervised way is needed whenever computers have to reason about unlabeled data. Applications range from compressing and denoising data to super-resolution, generating new samples from a given sample distribution and much more.
In this work, we use information entropy and a little game to motivate a new encoder discriminator architecture in order to learn unsupervised latent representations. Inspired by the game "Taboo", we train an encoder network to generate a meaningful representation of one particular sample of a dataset. Using this description, a discriminator network then has to retrieve the same sample from the whole dataset. We show that learning in this manner on many different samples repeatedly minimizes the information entropy given the latent description and, thus, forces the encoder network to make precise descriptions that can be interpreted by the discriminator.
We provide first results of this method on the MNIST and the Fashion MNIST dataset.