Before revision: We examine the problem of optimal sparse encoding of sensory stimuli by latent variables in stochastic models. Analyzing restricted Boltzmann machines with a communications theory approach, we search for the minimal model size that correctly conveys the correlations in stimulus patterns in an information-theoretic sense.
After revision: In this work we explore emergent encoding strategies learned by statistical models of sensory coding in noisy spiking networks. Early stages of sensory communication in neural systems can be viewed as encoding channels in the information-theoretic sense.