Title: Inference and Introspection in Deep Generative Models of Sparse Data
Authors: Rahul G. Krishnan, Matthew Hoffman
Keywords: Unsupervised Learning, Deep learning
TL;DR: We study two techniques to improve learning in deep generative models on sparse, high-dimensional text data. We also propose an algorithmic tool to visualize and introspect arbitrarily deep learned models. 
Abstract:
Deep generative models such as deep latent Gaussian models (DLGMs) are powerful and popular density estimators. However, they have been applied almost exclusively to dense data such as images; DLGMs are rarely applied to sparse, high-dimensional integer data such as word counts or product ratings. One reason is that the standard training procedures find poor local optima when applied to such data. We propose two techniques that alleviate this problem, significantly improving our ability to fit DLGMs to sparse, high-dimensional data. Having fit these models, we are faced with another challenge: how to use and interpret the representation that we have learned? To that end, we propose a method that extracts distributed representations of features via a simple linearization of the model. 
Venue: Submitted to ICLR 2017