  Nonnegative matrix factorization (NMF) based topic modeling methods do not
rely on model- or data-assumptions much. However, they are usually formulated
as difficult optimization problems, which may suffer from bad local minima and
high computational complexity. In this paper, we propose a deep NMF (DNMF)
topic modeling framework to alleviate the aforementioned problems. It first
applies an unsupervised deep learning method to learn latent hierarchical
structures of documents, under the assumption that if we could learn a good
representation of documents by, e.g. a deep model, then the topic word
discovery problem can be boosted. Then, it takes the output of the deep model
to constrain a topic-document distribution for the discovery of the
discriminant topic words, which not only improves the efficacy but also reduces
the computational complexity over conventional unsupervised NMF methods. We
constrain the topic-document distribution in three ways, which takes the
advantages of the three major sub-categories of NMF -- basic NMF, structured
NMF, and constrained NMF respectively. To overcome the weaknesses of deep
neural networks in unsupervised topic modeling, we adopt a non-neural-network
deep model -- multilayer bootstrap network. To our knowledge, this is the first
time that a deep NMF model is used for unsupervised topic modeling. We have
compared the proposed method with a number of representative references
covering major branches of topic modeling on a variety of real-world text
corpora. Experimental results illustrate the effectiveness of the proposed
method under various evaluation metrics.
