Alleviating “Posterior Collapse” in Deep Topic Models via Policy Gradient
Anonymous Author(s)
AffiliationAddress email
Abstract
Deep topic models have been proven as a promising way to extract hierarchical latent representations from documents represented as high-dimensional bag-of-wordsvectors.
However, the representation capability of existing deep topic models isstill limited by the phenomenon of “ posterior collapse ”, which has been widelycriticized in deep generative models, resulting in the higher-level latent representations exhibiting similar or meaningless patterns.
To this end, in this paper, we firstdevelop a novel deep-coupling generative process for existing deep topic models,which incorporates skip connections into the generation of documents, enforcingstrong links between the document and its multi-layer latent representations.
Afterthat, utilizing data augmentation techniques, we reformulate the deep-couplinggenerative process as a Markov decision process and develop a corresponding
Policy Gradient (PG)
based training algorithm, which can further alleviate theinformation reduction at higher layers.
Extensive experiments demonstrate thatour developed methods can effectively alleviate “ posterior collapse ” in deep topicmodels, contributing to providing higher-quality latent document representations.
Introduction
Topic modeling has become a successful technique for text analysis and been widely applied tovarious problems in machine learning (ML)
[1, 2] and natural language processing (NLP)
[3, 4] overthe past two decades.
Representing documents as bag-of-words (BoW) vectors, vanilla probabilistictopic models (PTMs), with latent Dirichlet allocation (LDA)
[5] being the best known representative,typically formulate each document as a mixture over latent topics where each topic is characterizedby a distribution over the terms of the vocabulary and describes an interpretable semantic concept.
Although being widely used, the modeling capability of these shallow topic models is still restrictedby their single-layer structure and has difficulty in exploring hierarchical thematic structures.
Tothis end, a series of deep topic models [6, 7, 8] have been developed to extract multi-layer documentrepresentations from a text corpus, providing a more intuitive way for users to understand text data.
Recently, benefiting from the development of deep neural networks (DNNs), there has been an emerging research interest to develop neural topic models (NTMs) to boost the performance, efficiency,and usability of topic modeling with DNNs.
Specifically, following the framework of variationalautoencoder (VAE) [9], most NTMs [10, 11, 12] construct a variational inference network (encoder)to project each document into its stochastic latent representation, and then reconstruct the corresponding BoW observation with a stochastic/deterministic decoder.
By modeling the inference/generativeprocess with DNNs, these NTMs are more flexible and scalable than traditional Bayesian PTMs,contributing to performing large-scale downstream tasks, especially in NLP tasks [13, 14].
Despite achieving attractive performance, existing deep topic models (either PTMs or NTMs) stillsuffer from different degrees of “ posterior collapse ” at higher layers, where the definition of “ posterior collapse ” is that the approximated posterior distribution of the latent variables z , denoted as q ϕ ( z | x ) , collapses to its non-informative prior distribution p θ ( z ) , resulting in their KL divergence
D KL [ q ϕ ( z | x ) || p ( z )] to be close to zero [9, 15, 16, 17].
The “ posterior collapse ” has been widelycriticized in the field of generative models [15, 16], and the occur of this phenomenon will causedeep topic models exhibiting similar or meaningless patterns at higher hidden layers [18, 19, 20].
Although there have been several deep NTMs [19, 20] trying to alleviate this issue by constructingmore flexible inference networks, the collapse phenomenon in deep NTMs may not be solved inessence, because the true posterior provided by the generative model and the objective function foroptimization remain almost unchanged [17].
To extract higher-quality hierarchical latent document representations, in this paper, we develop adeep-coupling generative process equipped with a Policy Gradients (PG) based training algorithm forexisting deep topic models.
The main contributions of this work are as follows:
• We develop a deep-coupling generative process for deep topic models, which incorporates
skip connections into the generation of documents to alleviate “ posterior collapse ”.
• We take a specific NTM as an example to explain how to construct a deep topic model withthe deep coupling generation process, and develop a deep-coupling hierarchical EmbeddingTopic Model ( dc -ETM), which can be extended to other deep topic models.•
Utilizing the property of sequence-like generation process, we design a PG-based trainingalgorithm for dc -ETM, which can further alleviate the information reduction at higher layers.
• Compared to existing deep topic models, extensive experimental results show that dc -ETMs
can lead to less “ posterior collapse ” and provide higher-quality latent representations.
Related Work
Probabilistic Topic Model: Deep PTMs [6, 7, 8, 21, 22] are developed to infer multi-layer documentrepresentations, whose adjacent layers are connected with specific factorization.
For instance, gammabelief network (GBN) [7] is constructed via factorizing the shape parameters of the gamma distributedlatent representations; DPFA [6] extends PFA [23] into a multi-layer version but is restricted to modelbinary topic usage patterns; DirBN [8] is developed via factorizing the Dirichlet distributed topicmatrix.
Although providing readily interpretable multi-layer latent document representations, therepresentation capability of these deep PTMs is limited by adopting CRT distribution to upwardpropagate data information to higher layers with their backbones [18].
Neural Topic Model: Most existing NTMs [11, 19, 24, 20] can be viewed as extensions of PTMsunder the VAE framework and focus on modeling the generative/inference process with DNNs.
Forinstance, one popular research direction of NTMs is to develop more flexible inference network withreparametrization tricks [11, 19] and the other could be incorporating word embeddings into thegenerative model [24, 20].
However, as far as we know, few efforts have been made to alleviate thephenomenon of “ posterior collapse ” in NTMs by modifying its generative process, which is a greatchallenge under the framework of topic modeling and also the main contribution of this work.
Besides, distinct from the way of combining reinforcement learning (RL) with topic models inprevious works [25, 26, 27], our work is the first to formulate the topic modeling generative processas a sequential decision making one to incorporate RL-based training algorithms, which focuses onproviding higher-quality latent document representations by alleviating “posterior collapse”.
Deep-Coupling Generative Process for Deep Topic Models
To give an intuitive insight on “posterior collapse” in deep topic models, we visualize the higher-leveltopics learned by a recent popular NTM named SawETM [20] in Fig.
3, which exhibit similarsemantic patterns and limit its representation capability.
Then, we take SawETM as an example, butnot limited to this, to illustrate how to construct a deep topic model with the deep coupling generationprocess, leading to a novel dc -ETM in Fig. 1(c).
Compared to the usual structures of deep PTMs and
NTMs shown in Fig.
1(a) and 1(b), besides the design of inference network, the main difference ofdc -ETM is incorporating skip connections into the generation of documents, enforcing strong linksbetween the document and its multi-layer latent representations to alleviate “ posterior collapse ”.
We emphasize that the skip-connection in SKIP-VAE [17] cannot be extended for a deep generativemodel with multiple stochastic layers.
The main difficulty in our design is the need for carefully designing the probabilistic generative process to build the effective connection between the observation
(1) n  (2) n  (3) n  × (0) (0) T K D    (3) (3) D K    (3) ˆ  × (0) (0) T K D    (2) (2) D K    (2) ˆ  × (0) (0) T K D    (1) (1) D K    (1) ˆ  × (1) (1) T K D    (2) (2) D K    (2)  × (3) (3) D K    (2) (2) T K D    (1) n x (2) n x (3) n x n x (1) n h (2) n h (3) n h n x n h (3)  n x (1) n h (2) n h (3) n h (3) n  (2) n  (1) n  (3)  (2)  (3) n  (2) n  (1) n  n x (3)  (2)  (1)  (3) n  (2) n  (1) n  n x (3)  (2)  (1)  (3) n  (2) n  (1) n  (3)  (2)  n x (1) n h (2) n h (3) n h (3) n  (2) n  (1) n  (3)  (2)  n x (3) n x (2) n x (1) n x
[Table]
Inference
[Table]
Inference
Generative (1)  (2)  (3)  (
(1) n  (2) n  (3) n  × (0) (0) T K D    (3) (3) D K    (3) ˆ  × (0) (0) T K D    (2) (2) D K    (2) ˆ  × (0) (0) T K D    (1) (1) D K    (1) ˆ  × (1) (1) T K D    (2) (2) D K    (2)  × (3) (3) D K    (2) (2) T K D    (1) n x (2) n x (3) n x n x (1) n h (2) n h (3) n h n x n h (3)  n x (1) n h (2) n h (3) n h (3) n  (2) n  (1) n  (3)  (2)  (3) n  (2) n  (1) n  n x (3)  (2)  (1)  (3) n  (2) n  (1) n  n x (3)  (2)  (1)  (3) n  (2) n  (1) n  (3)  (2)  n x (1) n h (2) n h (3) n h (3) n  (2) n  (1) n  (3)  (2)  n x (3) n x (2) n x (1) n x
[Table]
Inference
Generative
Inference
Generative (1)  (2)  (3) 
(1) n  (2) n  (3) n  × (0) (0) T K D    (3) (3) D K    (3) ˆ  × (0) (0) T K D    (2) (2) D K    (2) ˆ  × (0) (0) T K D    (1) (1) D K    (1) ˆ  × (1) (1) T K D    (2) (2) D K    (2)  × (3) (3) D K    (2) (2) T K D    (1) n x (2) n x (3) n x n x (1) n h (2)
[Figure] [Figure]
h (3) 
[Figure]
x (
[Figure]
[Figure] [Figure]
[Figure]
(2) n  (1) n
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure] [Figure]
x (3)  (
[Figure]
[Figure] [Figure]
(1) n
[Figure]
n x (3)  (2)  (1)  (3) n  (2) n  (1) n  (3)  (2)  n x (1) n h (2) n h (3) n h (3) n  (2) n  (1) n  (3)  (2)
 n x (3) n x (2) n x (1) n x
[Figure]
Inference
Generative
Inference
Generative (1)  (2)  (3) 
[Figure]
PTM, (b) deep NTM, and (c) dc -ETM developed in this paper, where the symbol definitions are consistent with those in Sec. 3.1. and its multi-layer stochastic latent representations, on the premise of preserving the interpretablehierarchical topic modeling structure, rather than casually introducing skip-connections.
3.1 Deep-Coupling Hierarchical Embedding Topic Model
As a usual VAE-like model, the developed dc -ETM consists of a generative model (decoder) and aninference network (encoder).
Below, we focus on presenting the generative model of dc -ETM, whichcan be flexibly applied for other deep topic models to alleviate “ posterior collapse ”, and leave thedetails of the inference network to Appendix A.
Generative Model:
Given a text corpus consisting of N documents X = { x n } Nn =1 , each documentcan be represented as a high-dimensional sparse BoW vector x n ∈ Z K (0) , where
Z = { 0 , 1 , ... } and
K (0) denotes the vocabulary size.
Then, from top to bottom, the generative model of the dc -ETMwith L hidden layers can be formulated as
[Equation]
α ( l ) ˆ Φ (
[Equation]
[Equation]
) , α = Softmax ( ξ ) , ϕ (
[Equation]
k = Softmax (
[Equation]
( l
[Equation] [Equation]
( l )
[Equation]
) , l = 1 , ...,
L
[Equation]
1 , where, Φ ( l ) ∈ R K ( l − 1) × K ( l ) + denotes the topic matrix (factor loading) and each column ϕ ( l ) k ∈
R K ( l − 1) + indicates a specific topic (factor) at layer l ; θ ( l ) n ∈ R K ( l ) + denotes the gamma distributedlatent representation (topic proportions) at layer l , K ( l ) denotes the number of hidden units (topics)at layer l .
Under the Poisson likelihood, the observed multivariate count vector x n is first factorizedinto L equal-size latent matrix { α ( l ) ˆ Φ ( l ) θ ( l ) n } Ll =1 , where, ˆ Φ ( l ) ∈ R K (0)
× K ( l ) + can be regarded as theprojection of topic matrix Φ ( l ) to the observation space and the detailed definition will be discussedin the next paragraph; α ( l ) denotes the importance weight of ˆ Φ ( l ) θ ( l ) n for generating the observationx n , and the summation of the whole weight vector α ∈ R L + is constrained to be equal to one with a
Softmax normalization.
Then, the latent representation θ ( l ) n at layer l is further factorized into theproduct of the topic matrix Φ ( l +1) ∈ R K
( l ) × K ( l +1) + and topic proportions θ ( l +1) n ∈ R K ( l +1)
+ at the nextlayer under the shape of gamma distribution.
The top layer’s latent representation θ ( L ) n shares the samegamma shape parameters r ∈ R K ( L )
+
and we apply a gamma distributed prior on the scale parametersc ( l ) n for l ∈ { 2 , ..., L + 1 } .
With the recent popular distributed topic representation in NTMs [24, 28],each topic ϕ ( l ) k is treated as the result of applying a Softmax normalization on the inner product ofits distributed representation β ( l ) k ∈ R D and topic embedding matrix β ( l − 1) ∈
R D × K ( l − 1) at theprevious layer, where D denotes the dimension of the embedding space.
The projections of topic matrices to the observation space, denoted as { ˆ Φ ( l ) } Ll =1 , build the straightforward connections between the document x n and its multi-layer latent representations { θ ( l ) n } Ll =1 ,which alleviates the information reduction at higher layers by sharing the pressure of documentmodeling with all hidden layers.
To reduce the computation and storage cost of the developed dc -ETM, we develop two variants for ˆ ϕ ( l ) k ∈ R K (0) + without introducing any extra parameter.
Theone variant is adopting the property of topic hierarchy elaborated in Sec.
3.2 to obtain each ˆ ϕ ( l ) k bysuccessively multiplying topic matrices at lower layers as
[Equation]
and the other variant is treating the projection ˆ ϕ ( l ) k as the result of the inner product of its distributedrepresentation β ( l ) k
and the word embedding matrix β (0) at the observed space, as follows
[Equation]
We emphasize that the first variant can be used to extend most existing deep topic models, while thelatter is limited to NTMs equipped with topic embedding techniques. (More details of the variantscan be found in Appendix I).
Generally speaking, the deep-coupling generative process in dc -ETM not only preserves the hierarchyof traditional deep topic models, leading to multi-layer document representations to enhance themodeling capability and interpretability, but also alleviates the issue that the amount of informationwill decrease rapidly with the network going deeper, benefiting from building the straightforwardconnections between observation x n and its higher-level latent representations { θ ( l ) n } l> 1 .
Besidesalleviating “ posterior collapse ”, the characteristics of deep-coupling network structure of dc -ETMalso brings us a new view to design the corresponding inference network and training algorithm.
Inference Network : The details of the inference network of dc -ETM can be found in Appendix A.
3.2 Model Property
Sequence-like Generative Process: Taking advantages of the properties of the Poisson distribution,the original generative process of the observed data x n defined in Eq.
(1) can be rewritten as:
[Equation]
) where x ( l ) n denotes the augmented observation at layer l , and is generated from the Poisson distributionwith a rate of α ( l ) ˆ Φ ( l ) θ ( l ) n .
Then, the observed data x n can be regarded as not only the summationover these augmented vectors { x ( l ) n } Ll =1 , but also equal to the weighted summation over the latentvectors { ˆ Φ ( l ) θ ( l ) n } Ll =1 on the mean, where the weight vector α satisfies the constraint (cid:80)
Ll =
1 α ( l ) = 1 .
More details about the relationship of x ( l ) n and x n can be found in Appendix I.
Rethinking the generative process of the developed dc -ETM reformulated in Eq.
(4), the set ofaugmented observation vectors { x ( l ) n } Ll =1 can naturally form an observation sequence [ x ( L ) n , ..., x (1) n ]by sorting these vectors according to their dependencies in the generative process (from deep toshallow).
For each hidden layer (time step) l , the generative process will first incorporate the priorinformation passing from deeper hidden layers { θ ( t ) n } t>l , and then generate the latent representationθ ( l ) n at the current layer (time step), which not only is supposed to generate the current observationvector x ( l ) n under the Poisson likelihood, but also introduces the information into the shape parameterof the following gamma distributed latent representation θ ( l − 1) n at the next layer (time step).
Thus, the deep-coupling generative process of dc -ETM originally defined in Eq.
(1) can be naturallyreinterpreted from the perspective of sequence generation, and its reformulation defined in Eq.
(4)can be also equivalently reformulated as:
[Equation]
providing an intuitive insight for the decomposition of the likelihood function in Sec. 4.1.
Hierarchical Semantic Topics:
The developed dc -ETM can naturally interpret each semantic topic ϕ ( l ) k at layer l by visualizing its projection to the vocabulary space calculated as
{ [ (cid:81) l − 1 t =1 Φ ( t ) ] ϕ ( l ) k } K ( l ) k =1 , and each document can also be roughly seen as a random mixture over
K ( l ) topics with θ ( l ) n being the corresponding topic proportions at layer l as
[Equation]
which can be obtained with the law of total expectation.
Moreover, similar to the underlying idea ofthe deep learning, the topics learned by dc -ETM tend to be more specific at lower (bottom) layersand those at higher (top) layers are more general, as shown in Fig.
Secondly, in dc -ETM, both words β (0) ∈ R D
× K (0) and hierarchical topics { β ( l ) ∈ R D
× K ( l ) } Ll =1are represented as embedding vectors under the same semantic space, contributing to intuitivelymeasuring and visualizing the distance between different topics (words), which has been proven to beeffective in capturing the underlying semantic structure as shown in Fig. 4(a).
Policy Gradient based Training Algorithm
4.1 ELBO of dc -ETM
As a VAE-like NTM, the developed dc -ETM can be trained like usual VAEs by directly maximizingthe evidence lower bound (ELBO), specifically as
[Equation]
(7) where the first term is the expected log-likelihood and the other term is the Kullback–Leibler (KL)divergence from the prior p ( θ n ) to the variational posterior q ( θ n | x n ) .
Through introducing the augmented vectors { x ( l ) n } Ll =1 , the log-likelihood of x n in dc -ETM can beequivalently reformulated as
[Equation]
where the function in the second expectation term can be treated as the summation of the set oflog-likelihood of { x ( l ) n } Ll =1 .
Due to the hierarchical network structure, the KL divergence term canbe factorized as
[Equation]
where q ( θ ( l ) n |− ) is constructed by a Weibull-based inference network described in Appendix.
A andp ( θ ( l ) n | Φ ( l +1) , θ n ( l +1) ) satisfies a gamma prior in Eq.
(1), and their KL divergence has an analyticexpression, benefiting from adopting the Weibull reparameterization technique [19].
Combining the aforementioned derivations, the ELBO of dc -ETM can be equivalently rewritten as
[Equation]
which can be directly optimized with gradient-based methods to update both the encoder parametersΩ and decoder parameters Ψ in dc -ETM.
We emphasize that, after deriving the augmented vectors{ x ( l ) n } Ll =1 from x n via data augmentation technique [7], the first expectation term in L ( x n ) will be a constant and the ELBO can be directly optimized by maximizing the following loss function
[Equation]
which can be roughly treated as the ELBO of a sequence [ x ( L ) n , ..., x (1) n ] generated from a sequenceof latent representations [ θ ( L ) n , ..., θ (1) n ] [29, 30], and naturally meets the sequence-like generativeprocess of dc -ETM as discussed in Sec. 3.2.
4.2 Optimization with Policy Gradient
Similar to RNN-based model, after augmenting { x ( l ) n } Ll =1 from x n , the loss function of dc -ETMdefined in Eq.
(11) is equal to the summation of L sub-loss functions, where each sub-loss functionˆ L ( l ) ( x ( l ) n ) can be equivalently regarded as a separate loss of a subsequence generation model that isonly a part of the whole sequential generative model and expected to output x ( l ) n at the final time stepl .
Inspired by the great success achieved by RL methods [31, 32, 33, 34] in learning a stable longsequence (Markov decision process) with high quality, we consider the sequence-like generationprocedure of a L -layer dc
-ETM
as a Markov decision process with L time steps, and develop a noveltraining mechanism based on Policy Gradient [33] for dc -ETM, which injects the future rewardsobtained from generating the suffix subsequence into each current sub-loss function ˆ L ( l ) ( x ( l ) n ) .
Specifically, we treat the whole dc -ETM as a stochastic policy network π ( a ( l ) n | s ( l ) n ) expected togenerate a fixed-length action sequence [ a ( L ) n , ..., a (1) n ] from the observation x n , defining the state s ( l ) nas { x n , { Φ ( t ) , θ ( t ) n } t>l } and the action a ( l ) n as α ( l ) ˆ Φ ( l ) θ ( l ) n .
For each time step l , given the currentstate s ( l ) n , the policy network π ( a ( l ) n | s ( l ) n ) will first sample θ ( l ) n from the inference network via
[Equation]
12) and further obtain the corresponding action as
[Equation]
(13) which can be regarded as directly drawing from π ( a ( l ) n | s ( l ) n ) .
The state transition is deterministicafter an action has been chosen, indicating that the next state s ( l − 1) n = { x n , { Φ ( t ) , θ ( t ) n } t>l − 1 } if thecurrent state s ( l ) n = { x n , { Φ ( t ) , θ ( t ) n } t>l } and the action a ( l ) n = α ( l ) ˆ Φ ( l ) θ ( l ) n .
Then we take the separate loss ˆ L ( l ) ( x ( l ) n ) defined in Eq.
(11) as the immediate reward at the the timestep l , formulated as
[Equation]
and the action-value function can be formulated as
[Equation]
which indicates the expected accumulative reward starting from state s ( l ) n , taking action a ( l ) n , andthen generating the suffix subsequence [ a ( l − 1) n , ..., a (1) n ] with the policy network π ( a ( l ) n | s ( l ) n ) and thediscount factor 0 < γ ≤ 1 .
[Figure] [Figure]
Following [31]
[Figure]
the objective function of training dc -ETM with policy gradient can be estimated (onone episode) as
[Equation]
where Ω and Ψ indicate the encoder and decoder parameters in dc -ETM respectively.
We note that theexpectation E [ · ] can be approximated by sampling methods based on the Weibull reparameterization,and the objective function can be directly optimized by advanced gradient descent algorithms, like
Adam [35] and RMSprop [36].
We provide the details of PG-based training algorithm in Appendix C.
Experiments
To evaluate the effectiveness of the developed dc -ETM and the corresponding policy gradients (PG)based training algorithm, we make extensive experiments on both quantitative and qualitative aspects.
Considering there are two dc -ETM variants as described in 3.1, we use the suffix − α and − β todistinguish the variants defined in Eq.
(2) and Eq.
(3) respectively, and highlight whether the dc -ETMis trained with the PG based algorithm in Sec.
5.1 Datasets and Baselines
Datasets: Four widely used document benchmarks, specifically R8 [37], 20Newsgroups ( 20News )[38], Reuters Corpus Volume I ( RCV1 ) [39] and World Wide Web Knowledge Base ( WebKB )
[40]are included in the following experiments.
We summarize the statistics of benchmarks in Appendix Dand follow the procedure in [20] to preprocess these documents to obtain their BoW representations.
Baselines: We compare the developed dc -ETMs with a series of topic models, which can be roughlydivided into two categories: 1) shallow topic models such as LDA [5], AVITM [11] and ETM [24],where LDA is a PTM and the others are NTMs; 2) deep topic models including PGBN [7], WHAI[19] and SawETM [20], where PGBN is a deep PTM and the others are deep NTMs.
We emphasizethat WHAI and SawETM are the most relevant strong baselines for comparison, both of whichprovide hierarchical Weibull-based latent document representations, and SawETM has achievedstate-of-the-art performance on unsupervised document modeling and clustering tasks.
Experimental Settings: To make a fair comparison, we set the same network structure for alldeep topic models as [256 , 128 , 64 , 32 , 16] from shallow to deep.
For PTMs, we use the defaulthyperparameter settings in their published papers and accelerate the Gibbs sampling with GPU.
For NTMs, we set the size of their hidden layers as 256, the embedding size as 100 for themincorporating word embeddings, like ETM, SawETM and dc -ETMs, and the mini-batch size as200.
For optimization, we adopt the same Adam optimizer [41] with a learning rate of 1e-2.
Allexperiments are performed with an Nvidia RTX 3090 GPU and implemented with PyTorch [42].
5.2 Quantitative Comparisons
To investigate whether the proposed skip-connection structures in both dc -ETM variants can alleviate“ posterior collapse ”, especially at higher layers, we compare them with other popular deep topic models in the first part.
Then, to investigate whether the mitigation of “ posterior collapse ” canimprove the quality of latent document representations at higher layers, we conduct more quantitativecomparisons in the rest parts.
We report the error bars in Appendix E.
Document Modeling:
In Fig. 2, for each deep topic model, we plot the curve of point log-likelihoodln p ( x n |− ) as a function of iterative epochs conditioned on the t -th-layer reconstruction ˆ Φ ( t ) θ ( t ) n ,which can be used to measure the degree of “ posterior collapse ” by the relevance between the datasample x n and its latent representation θ ( t ) n .
From the results, we can see that although SawETMand WHAI achieve a comparable performance with dc -ETMs on the first hidden layer in Fig.
2(a),their reconstruction quality decreases dramatically with the network going deeper in Fig. 2(b) and2(c), potentially reflecting that little data information can be propagated to higher layers of thesetraditional deep topic models.
Benefiting from introducing skip connections into the generativeprocess, dc -ETMs can significantly alleviate “ posterior collapse ” at higher hidden layers.
[Table]
Perplexity & Topic Diversity
To make a more comprehensivequantitative comparison, we usethe average of heldout-word perplexities (the lower is the better)and topic diversities (the higher isthe better) across all hidden layers to measure the document modeling performance and topic quality of these deep topic modelswith { θ ( t ) n } Tt =1 and { Φ ( t ) n } Tt =1 ,respectively.
The experimentalsettings are consistent with thosein SawETM [20], and the experimental results have been exhibited in Table 3.
Benefiting fromhierarchical network structures, the modeling capability of deep topic models generally outperformthose shallow ones.
Thanks to enhancing the connections between the observation and multiplehidden layers with the deep-coupling generative process, the developed dc -ETMs achieve lowerperplexity scores and provide higher-quality topics than traditional topic models on all benchmarks.
Then, the PG-based training algorithm brings further performance improvement to our dc -ETMs.
[Table]
PGBN 55.
All 53.
WHAI 59.
All 57.
SawETM 57.
All 51.
dc -ETM-
α 61.
39.83 All 63.
dc -ETM- β 54.
38.63 All 67.
dc -ETM-
α (Policy) 49.
32.73 All 64.
dc -ETM-
β (Policy) 57.
38.34 All 69.
Document Clustering: To evaluate the quality of the extracted latent document representations ondownstream tasks, we considerdocument clustering, where weuse the topic models after training to extract the latent representations of the testing documentsand then use k-means to predict the clustering labels.
Usingthe Purity and Normalized Mutual Information (NMI) as metrics (the higher the better), theresults shown in Table 4 demonstrate that concatenating hierarchical latent document representations extracted by traditional deeptopic models cannot improve andeven hurt the clustering performance, potentially indicating thatthe latent representations at higher layers are meaningless.
However, distinct from traditional deeptopic models, the concatenation operation on the latent representations of dc -ETMs can significantlyimprove the performance, which can be attributed to enforcing strong links between the multi-layerrepresentations and the observation with the skip connections in the generation.
5.3 Qualitative Analysis
As discussed in Sec. 3.2, the developed dc -ETM inherits both the characteristics of hierarchical topicstructure and semantic topic embeddings.
Then we compare the hierarchical topics of a 5-layerdc -ETM trained on 20News with those learned by SawETM for qualitative analysis.
Topic Visualization: With the visualization techniques [7], we exhibit the 5 th-layer topics learned bydc -ETM and SawETM on 20News in Fig.
3 and Appendix F, where each topic is interpreted by itstop-10 words by sorting the word probabilities by descending order.
Obviously, the topics learned by
SawETM are quite similar, explaining the reason why concatenating its hierarchical latent documentrepresentations cannot improve and even hurt the performance on downstream tasks.
On the contrary,the developed dc -ETM can learn meaningful and diverse topics at higher layers, indicating that moredata information is passed to higher layers to alleviate “ posterior collapse ”.
We also exhibit a 5-layertopic tree learned by dc -ETM in Fig.
8 to illustrate the topic hierarchy of dc -ETM in Appendix N.
[Figure]
(a) ˆ Φ (5) learned by dc -ETM
[Figure]
[Figure]
Topic Embedding Visualization: ing hierarchical topic trees by dc -ETM, we visu- alize some of these trees originated from differenttopic nodes at layer 5 by projecting their semantic embeddings with t-SNE [43].
As shown in
Fig. 4(a), we can find that the topics in the sametopic tree tend to be closer than others from different trees in the semantic embedding space and similar phenomenon occurs in the words for describingthe same root topic, which indicates the hierarchylearn by dc -ETM is of high quality.
Note that wealso visualize the topics consisting of similar topwords in Fig. 4(b) and 4(c), learned by dc -ETM and
SawETM respectively, which demonstrates that thedeveloped dc -ETM can provide more meaningfuland discriminative topic and word embeddings.
Conclusion
To provide higher-quality hierarchical latent representations for deep topic modeling, in this paper,with the deep-coupling generative process, we develop a novel dc -ETM, which is constructed by introducing skip connections into the generativeprocess of GBN and also incorporates both topic embedding and Weibull reparameterization techniques.
Utilizing the property of sequence-like generation process, we design a PG-based trainingalgorithm for dc -ETM to further alleviate the information reduction at higher layers.
We note that themain idea of designing dc -ETM equipped with the PG-based training algorithm can potentially beextended to other deep topic models.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] See Abstract
(b) Did you describe the limitations of your work?
[Yes] See Appendix G
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See supplemen-
tal material
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 5.1 and Appendix D
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See Appendix E
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See supplemental
material
(b) Did you mention the license of the assets?
[Yes] See supplemental material
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]See supplemental material
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation? [N/A] Appendix
A Inference Network of dc -ETM
[Figure]
As a usual VAE-like model, the inference network (encoder) in dc -ETM is designed to approximatethe gamma distributed posteriors of latent representations of the corresponding generative model(decoder), specifically p ( θ n | x n ) defined in Eq.
As the gamma distribution, which can generatesparse and non-negative random variables, is not reparameterizable with respect to its shape parameter,we introduce the Weibull distribution [19] to construct our inference network due to its attractiveproperties as discussed in Appendix.
B, one of which is that a latent variable x ∼ Weibull ( k, λ ) canbe easily reparameterized as
[Equation]
Specifically, following traditional VAEs, we first factorize the variational posterior distributionq ( θ n | x n ) defined by the inference network in a hierarchical manner as follows
[Equation]
(18) which is expected to be flexible enough to well approximate the true posterior distribution p ( θ n | x n ) .
Then, for the design of the network structure, we develop a novel Weibull-based upward-downwardinference network, which contains both bottom-up deterministic and top-down stochastic paths,contributing to reducing the noise in the procedure of inferencing stochastic latent variables that arehigher in the hierarchy [15, 16].
As the inference network shown in the left part of 5, the bottom-up deterministic path takes the BoWvector x n as input to obtain hierarchical deterministic latent representations { h ( l ) n } Ll =1 as follows:
[Equation]
[Equation]
through specifically defining h n = MLP ( x n ) and h (1) n = MLP ( h n ) with MLP indicating a twolayered fully connected network and ⊕ being the concatenation operation at feature dimension.
Then,each deterministic latent representation h ( l ) n is further transferred into
[Equation]
[Equation]
where Linear is a dense fully connected layer.
Finally, taking Φ ( l +1) θ ( l +1) n as the prior informationpassing from the deeper layer, the variational posterior q ( θ ( l ) n |− ) can be obtained with the stochastic up-down path as
[Equation]
where Softplus function is applied to ensure positive Weibull shape and scale parameters; ˆ k ( l ) n and ˆ λ ( l ) nincorporate the information passing from h ( l ) n ; and θ ( l ) n can be obtained with the reparameterizationtechnique defined in Eq.
We emphasize that the inference network of our dc -ETM can be naturally treated as an inverse procedure of the generation process of x n , whose generation is conditional on the latent representations{ θ ( l ) n } Ll =1 across all hidden layers.
Inverse to the generative network, the inference network candirectly inject the data information into the latent representation of each layer through h n and allowall the stochastic latent variables { θ ( l ) n } Ll =1 to have a deterministic dependency on the observationx n , empirically alleviating top stochastic latent variables from being collapsed [16].
B Properties of Weibull distribution•
Similar density characteristics with Gamma Distribution
The Weibull distribution has similar characteristics with gamma distribution, i.e. , the density functionsof the two distributions are quite similar
[Equation]
• Easily Reparameterization
The latent variable x ∼ Weibull ( k, λ ) can be reparameterized as
[Equation]
24) leading to an expedient and numerically stable gradient calculation.
• Analytic
KL
-Divergence
The KL-divergence between the Weibull and gamma distributions has an analytic expression formulated as
[Equation]
[Equation]
C PG-based Training Algorithm
Modeling the generation of long sequence with RL-based methods has achieved great success, butstill suffers from high variance during model training.
The common ways to conduct policy gradientmethod for training include: i) employ the score-ratio gradient approximator like REINFORCE [44]to backward the gradients from Q-value Q π ( s ( l ) n , a ( l ) n ) to the parameters of the policy network π ; ii)introduce an additional “critic” network to estimate the Q-value Q π ( s ( l ) n , a ( l ) n ) as ˆ Q π ( s ( l ) n , a ( l ) n ) , thenbackward the gradients from estimated Q-value ˆ Q π ( s ( l ) n , a ( l ) n ) to the parameters of the policy networkπ through methods like deep deterministic policy gradient (DDPG) [32].
We emphasize that, bothmethods above are developed for the RL applications where the Q-value function is not differentiable,e.g., the reward r ( s ( l ) n , a ( l ) n ) at every time step l is given by the non-differentiable black-box environment simulator, or for the applications with hundreds or even thousands time steps, where the direct computation of the gradient from Q π ( s ( l ) n , a ( l ) n )
=
r ( s ( l ) n , a ( l ) n ) + E π (cid:104)(cid:80) l − 1 i =1 γ
i r ( s ( l − i ) n , a ( l − i ) n ) (cid:105) toπ is very expensive.
However, the REINFORCE estimator is notorious for its high variance, whichwould cause the learning of long sequence hard to converge, and the introducing of the “critic”network would also bring extra bias for the gradients and computation burden.
Moving beyond the gradient estimator of REINFORCE method or introducing the critic network,we design a more stable way to model the sequential generative process in deep topic model here.
Specifically, the number of time steps here is the number of layers L of the topic model, e.g., L is 5in this paper, which is a much smaller number compared to the hundreds of time step in other RLapplications.
Besides, we design the r ( s ( l ) n , a ( l ) n ) as a totally differentiable function in Eq.
14, whichleads to a differentiable Q-value function Q π ( s ( l ) n , a ( l ) n ) .
More specifically, recall the purpose ofpolicy gradient method is to maximize the Q-value by applying gradient descent to the policy modelπ , we develop the Policy Gradient based variational inference algorithm for dc -ETM in Algorithm 1.
Set minibatch size m and the number of Layer L ;
Initialize the encoder parameters Ω and decoder parameters Ψ ; for iter = 1 , 2 , · · · do
Randomly select a minibatch of m documents to form a subset X = { x j } 1 ,m ;
Draw random noise { ε li } m,Li =1 ,l =1 from uniform distribution; for Layer l = L, L − 1 , · · · , 1 do for i
=
l, l − 1 , · · · , 1 do
Calculate r ( s ( i ) n , a ( i ) n )
D Datasets
R8 is a subset of 7,674 documents selected form 8 different review groups of the Reuters 21578dataset, and has been split into a training set of 5,485 documents and a testing set of 2,189 ones.20News dataset consists of 18,774 documents from 20 various new groups and has been split intoa training set of 11,314 documents and a testing set of 7,532 ones.
RCV1 is an archive of 804,114manually categorized newswire stories made available by Reuters.
WebKB is a dataset that includesweb pages from computer science departments of various universities, which has 4,518 web pagesthat are categorized into 6 imbalanced categories.
E Error Bars
We randomly run 5 seeds for our method in experiments and report the error bar as below.
[Table]
[Table]
F Comparison of topic quality
To make an intuitive comparison on the aspect of topic quality, we visualize the 5-layer topics learnedby dc -ETM and Sawtooth on 20News dataset as shown in .
Obviously, the topics learned by Sawtoothare quite similar, which potentially explains the reason why concatenating its hierarchical latentdocument representations cannot improve and even hurt the performance on downstream tasks.
On thecontrary, the developed dc -ETM can learn meaningful and diverse topics in higher layers, indicatingthat more data information is passed to higher layers to alleviate the phenomenon of collapse.
[Figure]
subject organization com article just don writes university like 5_8: 0lines subject organization com article just don university writes like 5_9: 0lines subject organization com article just don university writes like 5_10: lines subject organization com article just don writes university like 5_11: lines subject organization com article just don writes university like 5_12: lines subject organization com article just don university writes like 5_13: lines subject organization com article just don writes university like 5_14: lines subject organization com article just don university writes like 5_15: lines subject organization com article just don writes university like
[Figure]
G Limitation
The limitation could be the computation burden brought by the extra L − 1 training steps due to the
Policy Gradient training method.
However, since L is a small number, the extra computation burdenis affordable.
Besides, compared with the backbone GBN-based deep topic model, our method wouldnot bring extra computation burden during the testing stage, but leads to performance improvementsas shown in Section 5.
H Broader Impact
At first, we need to emphasize that this work is developed to mitigate the phenomenon of informationdecay in deep topic models, which has been widely disclosed in the topic modeling literaturebut few efforts have been made to address this challenge.
The main difficulty is the need forcarefully designing the probabilistic generative process to build the effective connection betweenthe observation and the corresponding latent representations, on the premise of preserving theinterpretable hierarchical topic modeling structure, rather than casually introducing skip-connections.
Utilizing the natural hierarchy in deep topic models, we provide a general solution equipped with a novel perspective to incorporate RL-based training algorithm, which brings quite a few contributingideas to this field.
Please note that, we take an NTM named SawETM as an example in this paper to illustrate how wedevelop a dc -ETM, but dc -ETM can be also applied to extend NTMs with similar structures.
Most ofexsiting NTMs adopt VAE-liked structures, but none of them attempt to solve the “ posterior collapse ”phenomenon in essence, resulting in that the latent representations at higher layers exhibit similar ormeaningless patterns (as shown in the Appendix F).
The significance of developing deep-coupling structure with RL-based training algorithm, which haseffectively improved the quality of the latent representations of a deep topic model at higher layers,goes beyond a single specific model.
I More details about the two Projection variants α and β
To have a better understanding about how our two variants of projection are built, we plot the structureof a 3-layer model containing inference and generation in Fig.
As shown in Fig.
7, each layer’stopic matrix Φ ( l ) , l = 1 , 2 , 3 is factorized into two inner matrices.
Note that, the topic matrix Φ (1) isalso the projection matrix ˆ Φ (1) .
For the projection variants dc -ETM- α in Eq.
(2), take projecting the θ (3) n into x (3) n for example,it could be implemented by multiplying Φ (1) Φ (2) Φ (3) to get the projection matrix ˆ Φ 3 and thenapplying ˆ Φ (3) θ (3) n to get x (3) n .
For our proposed projection variants dc -ETM- β in Eq.
(3), it exploits the property of the constructionof topic matrix Φ (1) .
Also take projecting the θ (3) n into x (3) n for example, it could be flexiblyimplemented by multiplying the inner matrix β (0) T and β (3) to get the projection matrix ˆ Φ (3) , asshown in the rightdown part in Fig.
We emphasize that, this projection method α could be applied to most existing deep topic modelswithout requirement that their topic matrix Φ ( l ) could be factorized into two inner matrices.
Whilethe projection method β is only applicable for the NTMs equipped with topic embedding techniques.(1) n
 (2) n  (3) n  × (0) (0) T
K D    (3) (3) D K    (3) ˆ  × (0) (0) T
K D    (2) (2) D K    (2)
ˆ  × (0) (0) T
K D    (1) (1) D K  
[Figure]
(1) ˆ  × (1) (1) T
K D    (2) (2) D K    (2)  × (3) (3)
D K    (2) (2) T
K D    (
[Figure]
n x (
[Figure]
[Figure]
[Figure]
) n
[Figure]
[Figure]
[Figure]
J More analysis about the posterior collapse
Recall the definition of posterior collapse in Section 1, the posterior collapse happens whenthe approximated posterior distribution q ϕ ( z | x ) is collapsed to a non-informative distribution, which is usually the prior distribution p ( z ) and could be mathematically represented as 624 E p data ( x ) D KL [ q ϕ ( z | x ) || p ( z )]
→ 0 [9, 15, 16, 17].
Thus, it is straightforward to use the KLdivergence as a metric to measure the posterior collapse.
In Table 5, we compare the SOTA deep
NTM method SawETM with one variant of our method dc -ETM- β (policy) (for brevity, we note it as" dc -ETM") under the metric KL-divergence of different layers in datasets 20News, R8, and RCV1.
As the results shows, the SawETM’s KL-divergence is gradually closer to 0, which indicates a heavyposterior collapse and limited topic information have been captured by the higher layers of latentvariables.
On the contrary, dc -ETM can keep a relatively large value of the KL-divergence in higherlayers, which is benefiting from the efficient skip connection between data x n and each layer’s θ ( l ) n .
Besides the experimental analysis, we would like to give a informal theoretical explanation for thereviewers to have a better understanding of the reason why posterior collapse is widely existedin the higher layers of hierarchical VAEs.
To keep the analysis general, we use a vanilla L -layerhierarchical VAE with a top-down inference network as an example, where dc -ETMs are the variantsof it.
Inspired by [17] who analyze the posterior collapse from a perspective of information theoryfor the normal single-layer VAE, we extend it a little bit to the hierarchical VAE’s senario.
Separating the latent variables as the lower-level variables z ≤ k = { z 1 , ..., z k } and the higher-levelones z >k = { z k +1 , ..., z L } , where k ∈ { 0 , ..., L − 1 } , then the ELBO for this hierarchical VAE canbe reformulated as
[Equation]
26) where q ϕ ( z L | z L +1 ) :
= q ϕ ( z L | x ) , p θ ( z L | z L +1 ) :
= p θ ( z L ) , and the main contribution to the expected log-likelihood term is coming from the lower-level latent variables z ≤ k before the k -th hiddenlayer [16].
Once the generation capacity of the generative model p θ ( x | z ≤ k ) is powerful enough toreconstruct the observation x well, the variational posteriors of higher-level latent variables z >kwill be optimized to be close to their priors, i.e., q ϕ ( z >k | x )
≈ p
θ ( z >k ) , leading the representationslearned by VAE at higher layers to be meaningless and cannot provide faithful summaries for x ,which is well-known as the phenomenon of “ posterior collapse ” or “ latent variable collapse ” [17].
To find the potential solutions to alleviating “ posterior collapse ”, in the following, we reinterpretthis phenomenon from the perspective of information theory by extending the findings in [17] to ahierarchical VAE scenario.
For ease of undertanding, we define the mutual information between thedata x and the higher-level latent variables z >k as
[Equation]
>k ) log q ϕ ( z >k ) , which is induced by the variational posterior q ϕ ( z >k | x ) .
Then KL term in Eq.
(26) can be rewrittenas
[Equation]
[Equation]
where q ϕ ( z >k ) =
E p ( x ) [ q ϕ ( z >k | x )] .
By substituting Eq.
(27) into Eq.
(26), due to the non-negativityof mutual information and KL divergence, we can find that maximizing the ELBO is oppositeto maximizing the mutual information I q ( x , z >k ) .
When I q ( x , z >k ) is minimized to zero, thevariational posterior q ϕ ( z >k | x ) will be independent of the data x , which leads to the phenomenon of“ posterior collapse ”.
Thus, exploiting the property of the deep NTMs and building the skip connection between the x nand θ ( l ) n with two projection variants, as shown in Fig. 7, is promising to force the higher layers oflatent variables be more informative by maximizing the mutual information.
[Table]
K Comparison under more metrics
For evaluating the performance of the learned topics, we adopt more metrics for comparison between
SawETM and dc -ETM- β (policy) (for brevity, we note it as " dc -ETM") including: topic diversity,topic coherence
[Table]
and topic quality (a product of the topic diversity and topic coherence).
As shown in
Table 6, Table
[Table]
and Table 8, dc -ETM can learn comparable quality of the first layer’s topic whilesignificantly outperform SawETM in higher layers.
[Table]
[Table]
[Table]
L Comparison of all layers under the clustering task
We add an evaluation for the clustering tasks for each layer of SawETM and dc -ETM- β (policy) (forbrevity, we note it as " dc -ETM").
As shown in Table 9, the higher layer θ ( l ) n of SawETM providelimited information for the downstream clustering task, which could explain why the SawETM cannotgain a improvement of performance when concatenating all layers’ θ ( l ) n .
[Table]
M Visualization of the learned Topic tree by dc -ETM
Visualization of the learned topic tree by dc -ETM, where the directed arrow indicates the relationshipbetween these two topics are strong.
As shown in Fig.
8, the learned topics’ hierarchy is informative.
[Figure]
4_26: com, netcom, mark ibm sun, uiuc, subject lines, organization, jim 2_32: article, writes, apr message, news, gmt, reply think, sure, deleted 2_85: lines, subject, organization, reply, university good, just, new, keywords, got 2_20: columbia, colorado virginia, harvard, sgi, frank uchicago, gary, caltech, keith 2_57: com, netcom mark, ibm, sun, uiuc, jim, att, cso, austin 1_42: article writes think, news sure, deleted actually just, true, did 1_147: apr article, message gmt, date news, mon thomas writes, maybe 1_17: lines subject organization good, just got, need new, old, make 1_142: sgi caltech, keith frank, moral jon, morality objective livesey, wpd 1_169: columbia colorado virginia, harvard uchicago, gary arizona, eric midway, sas 1_111: com jim, att corp, systems corporation, std dec, opinions stratus 1_195: netcom mark, ibm sun, austin ucs, bob indiana utexas, tom 5_2: com, article, writes, apr lines, subject, organization netcom, reply, mark 5 th -layer topics
4 th -layer topics 2 nd
[Figure]
layer topics 1 st -layer topics 4_1: columbia, colorado virginia, harvard, sgi, frank uchicago, caltech, keith, gary 5_0: game, team, games hockey, baseball, play, year players, season, fans 5_6: god, jesus, bible christian, people, believe church, truth, say, know 5_12: space, gov, nasa earth, toronto.
jpl, henry launch, shuttle, orbit