Alleviating "Posterior Collapse'' in Deep Topic Models via Policy Gradient
Authors: Yewen Li
Chaojie Wang
Zhibin Duan
Dongsheng Wang
Bo Chen
Bo An
Mingyuan Zhou
Venue: NeurIPS 2022 Accept (2022)
Keywords: Deep Topic Models
Posterior Collapse
Reinforcement Learning

Abstract: Deep topic models have been proven as a promising way to extract hierarchical latent representations from documents represented as high-dimensional bag-of-words vectors.
However, the representation capability of existing deep topic models is still limited by the phenomenon of "posterior collapse", which has been widely criticized in deep generative models, resulting in the higher-level latent representations exhibiting similar or meaningless patterns.
To this end, in this paper, we first develop a novel deep-coupling generative process for existing deep topic models, which incorporates skip connections into the generation of documents, enforcing strong links between the document and its multi-layer latent representations.
After that, utilizing data augmentation techniques, we reformulate the deep-coupling generative process as a Markov decision process and develop a corresponding Policy Gradient (PG) based training algorithm, which can further alleviate the information reduction at higher layers.
Extensive experiments demonstrate that our developed methods can effectively alleviate "posterior collapse" in deep topic models, contributing to providing higher-quality latent document representations.

Alleviating "Posterior Collapse" in Deep Topic Models via Policy Gradient
Yewen Li
Chaojie Wang" i Bo Chen
Zhibin Duan
Dongsheng Wanga
Bo An'
Mingyuan Zhou
Nanyang Technological University
XXidian University
Thhe University of Texas at Austin
Abstract
Deep topic models have been proven as a promising way to extract hierarchical la- tent representations from documents represented as dighesinnnional bag-of of-words vectors. However, the representation capability of existing deep topic models is still limited by the phenomenon of "posterior collapse", which has been widely criticized in deep generative models, resulting in the higher-level latent representa- tions exhibiting similar or meaningless patterns. To this end, in this paper, we first develop novel deep-coupling generative process for existing deep topic models, which incorporates skip connections into the generation documents, enforcing strong links between the document and its multi-layer latent representations. After that, utilizing data augmentation techniques, we reformulate the deep-coupling generative process Markov decision process and develop corresponding Policy Gradient (PG) based training algorithm, which can further alleviate the information reduction at higher layers. Extensive experiments demonstrate that our developed methods can effectively alleviate "posterior collapse" deep topic models, contributing providing higher-quality latent document representations.
Introduction
Topic modeling has become successful technique for text analysis and been widely applied various problems in machine learning (ML) 2 and natural language processing (NLP) [4] 5) over the past two decades. Representing documents as bag-of-words (BoW) vectors, vanilla probabilistic topic models (PTMs), with latent Dirichlet allocation (LDA) [6] being the best known representative, typically formulate each document as a mixture over latent topics, where each topic characterized by distribution over the terms of the vocabulary and describes an interpretable semantic concept. Although being widely used, the modeling capability of these shallow topic models is still restricted by their single-layer structure, and has difficulty in exploring hierarchical thematic structures. To this end, series of deep topic models 7 have been developed to extract multi-layer document representations from text corpus, providing a more intuitive way for users to understand text data.
Recently, benefiting from the development of deep neural networks (DNNs), there has been an emerg- ing research interest to develop neural topic models (NTMs) to boost the performance, efficiency, and usability of topic modeling with DNNs. Specifically, following the framework of variational construct variational inference network (encoder) to project each document into its stochastic latent representation, and then reconstruct the correspond- ing BoW observation with a stochastie/deteriiiist decoder. By modeling the inference/generraiie process with DNNs, these NTMs are more flexible and scalable than traditional Bayesian PTMM, contributing to performing large-scale downstream tasks, especially in NLP tasks [4 15)
autoencoder (VAE) [10]. most NTMs [II
1212 B
"qqual contributions. Corresponning to: Chaojie Wang <chaojie, waniie.wwnggcc @ntu.eduuu.....
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

"Posterior collapse" has been widely criticized in the field of generative model [16 17. and the occurrence of this phenomenon will cause the approximated posterior qa (z)z²) collapses to it non-
information prior distribution po(z), leading their KI. divergence to be close to zero 10 16
17 [8] For deep topic models, despite achieving attractive performance, existing PTMs or NTMs still suffer
from different degrees of "posterior collapse", which causes their exhibiting similar or meaningless
patterns at higher layers [19 20 211. Although there have been several deep NTMs [2] 21 trying to alleviate this issue by constructing more flexible inference networks, the collapse phenomenon in deep NTMs may not be solved in essence, because the true posterior provided by the generative model and the objective function for optimization remain almost unchanged [18]
To extract higher-quality hierarchical latent document representations, in this paper, we develop a deep-coupling generative process equipped with a Policy Gradients (PG) based training algorithm for existing deep topie models. The main contributions of this work are as follows:
We develop a doup-coupling generative process for deep topic models, which incorporates skip connections into the generation of documents to alleviate "posterior collapse".
We take a specific NTM as example to explain how construct deep topic model with the deep coupling generation process, and develop a deep-coupling hierarchical Embedding Topic Model (dc-ETM), which can be extended to other deep topic models.
Utilizing the property of sequence-like generation process, we design PG-based training algorithm for dc-ETM, which can further alleviate the information reduction at higher layers. Compared to existing deep topic models, extensive experimental results show that dc-ETMs can lead to less "posterior collapse" and provide higher-qualite latent representations.
2 Related Work
are developed to infer multi-layer document representations, whose adjacent layers are connected with specific factorization. For instance, gamma belief network (GBN) [8] is constructed via factorizing the shape parameters of the gamma distributed latent representations; DPFA 77 extends PFA (23 into a multi-layer version but is restricted to model binary topic usage patterns; DirBN 9 is developed via factorizing the Dirichlet distributed topic matrix. Although providing readily interpretable multi-layer latent document representations, the representation capability of these deep PTTMs is limited by adopting CRT distribution upward propagate data information to higher layers with their backbones [19
Probabilistic Topic Model: Deep PTMs 7
1 9 22
can be viewed as extensions of PTMs under the VAE framework and focus on modeling the generative/inference process with DNNs. For instance, one popular research direction of NTMs is develop more flexible inference network with reparametrization tricks 12 20 and the other could be incorporating word embeddings into the generative model 21 24 However, as far as we know, few efforts have been made to alleviate the phenomenon of "posterior collapse" in NTMs by modifying its generative process, which is great challenge under the framework of topic modeling and also the main contribution of this work.
Neural Topic Model:
Most existing NTMs 12 20 21 24
25 26
Besides, distinct from the way of combining reinforcement learning (RL) with topic models in previous works (27 28 29), our work is the first to formulate the topic modeling generative process as a sequential decision making one to incorporate RL-based training algorithms, which focuses on providing higher-quality latent document representations by alleviating "posterior collapse".
3 Deep-Coupling Generative Process for Deep Topic Models
To give an intuitive insight on "posterior collapse" in deep topic models, we provide a detailed introduction for "posterior collapse" in Appendix [ and then visualize the higher-level topics learned by a recent popular NTM named SawETM (21) in Fig. 3 which exhibit similar semantic patterns and limit its representation capability. Then, we take SawETM as an example, but not limited this, to illustrate how to construct a deep topic model with the deep coupling generation process, leading to novel dc-ETM in Fig. [(c) Compared to the usual structures of deep PTMs and NTMs shown in Fig. ((a) and (bb) besides the design of inference network, the main difference of dc-ETM is incorporating skip connections into the generation of documents, enforcing strong links between the document and its multi-layer latent representations to alleviate "posterior collapse".

---

M
Fi
qp
Py
6
1
Generative (a) Deep PTM
Inference
Generative
Inference
Generative
(b) Deep NTM
(c) dc-ETM
Figure 1: The overview of the network structure of (a) deep PTM, (b) deep NTM, and (c) dc-ETM developed in this paper, where the symbol definitions are consistent with those in Sec. 3.1
3.1 Deep-Coupling Hierarchical Embedding Topic Model
As usual VAE-like model, the developed de-ETM consists of a generative model (decoder) and an inference network (encoder). Below, focus on presenting the generative model of dc-ETM, which can be flexibly applied for other deep topic models to alleviate "posterior collapse", and leave the details of the inference network Appendix
Generative Model: Given a text corpus consisting of documents x= each document
can be represented as high-dimensional sparse BoW vector nn E kk where 2 = {0, 1, ... and (()) denotes the vocabulary size. Then, from top bottom, the generative model of the dc-ETM with L hidden layers can be formulated as
( Gam (on ((+1) 1/c((+1)),
1,..,, , Gam Cam(r,1/ch 11/c//11))
(1)
In ~ Pois
( WD (000 a = Softmax(E), t00h Softmax(B(01)) 3 ) I,... L L
1,
denotes the topic matrix (factor loading) and each column Φ (k()-)) indicates specific topic (factor) layer 1; 0 E o denotes the gamma distributed latent representation (topic proportions) at layer 1, ('O) denotes the number of hidden units (topies) at layer 1. Under the Poisson likelihood, the observed multivariate count vector I, is first factorized
where, (1)
I2* 10 1
1 where, $ Rx can be regarded as the projection of topic matrix (() to the observation space and the detailed definition will be discussed in the next paragraph; (1)) denotes the importance weight of ( for generating the observation Rx and the summation of the whole weight vector a E R is constrained to equal to one with a
into equal-size latent matrix {a ()) (1)) (")))())))))))))) (
Softmax normalization. Then, the latent representation 0 layer is further factorized into the
product of the topic matrix T ((+1) k (k(+1) and topic proportions 0 (!+1) E RRK kK*+) at the next layer under the shape of gamma distribution. The top layer's latent representation o(²) shares the same gamma shape parameters r E TR k() and we apply gamma distributed prior the scale parameters (1) for E (,. L 1}. With the recent popular distributed topic representation in NTMs 24, 30) each topic d0()) is treated the result of applying Softmax normalization the inner product of its distributed representation 3 R² and topic embedding matrix (((-1) R Dx at the previous layer, where D denotes the dimension of the embedding space.
The projections of topic matrices the observation space, denoted as { ($1)) } build the straight- forward connections between the document x,, and its multi-layer latent representations {0(²) 1 J1=1
which alleviates the information reduction at higher layers by sharing the pressure of document modeling with all hidden layers. To reduce the computation and storage cost of the developed
dc-ETM, we develop variants for (()) E k2 without introducing any extra parameter. The
one variant is adopting the property of topic hierarchy elaborated in Sec. 3.2 obtain each d by (1)

---

successively multiplying topic matrices at lower layers as
II t=1
@ (1)
and the other variant treating the projection 6 as the result of the inner product of distributed representation ( and the word embedding matrix ( at the observed space, follows
( l
Softmax(3 5%
(3)
We emphasize that the first variant be used to extend most existing deep topic models, while the latter is limited to NTMs equipped with topic embedding techniques. We use the suffix -a and -8 to distinguish the variants defined in Eq. and Eq. and their detailed implementations can be found in Appendix !
Generally speaking, the deep-coupling, generative process in dc-ETMM not only preserves the hierarchy of traditional deep topic models, leading multi-layer document representations modeling capability and interpretability, but also alleviates the issue that the amount of information will decrease rapidly with the network going deeper, benefiting from building the straightforward
enhance the
connections between observation and its higher-level latent representations (0% h>s1 Besides alleviating "posterior collapse", the characteristics of deep-coupling network structure of dc-ETM also brings us new view to design the corresponding inference network and training algorithm.
Inference Network: The details of the inference network of dc-ETM can be found in Appendix A
3.2 Model Property
Taking advantages of the properties of the Poisson distribution, the original generative process of the observed data w, defined in Eq. be rewritten
Sequence-like Generative Process:
(1) x
x ~ Pois(a (1) ($(()) O
(4)
where æn denotes the augmented observation at layer I, and is generated from the Poisson distribution with rate of a (") Then, the observed data x,, be regarded not only the summation over these augmented vectors {a V but also equal to the weighted summation over the latent vectors (
(1) ,h 1 on the mean, where the weight vector satisfies the constraint Σ/_ a 1.
Rethinking the generative process of the developed dc-ETM reformulated in Eq. the set of augmented observation vectors {x)
}/21 (L can naturally form an observation sequence [a ..., a. (1) by sorting these vectors according their dependencies in the generative process (from deep to shallow). For each hidden layer (time step) 1, the generative process will first incorporate the prior information passing from deeper hidden layers (0!.) Jc-s. and then generate the latent representation (!) at the current layer (time step), which not only is supposed to generate the current observation (t) vector 2 under the Poisson likelihood, but also introduces the information into the shape parameter of the following gamma distributed latent representation o at the next layer (time step).
Thus, the deep-coupling generative process of dc-ETM originally defined in Eq. can be naturally reinterpreted from the perspective of sequence generation, and its reformulation defined Eq. can be also equivalently reformulated as:
(1)
(5)
providing an intuitive insight for the decomposition of the likelihood function in Sec. 4.1
Hierarchical Semantic Topics: The developed dc-ETM can naturally interpret each seman-
tic topic (0) at layer i by visualizing its projection to the vocabulary space calculated as

---

K ===1 and each document can also be roughly seen as random mixture over K topics with l being the corresponding topic proportions at layer as
0
10 E T
(1)
II (*)
1=1
IT
(
which can be obtained with the law of total expectation. Moreover, similar to the underlying idea of the deep learning, the topics learned by dc-ETM tend be more specific at lower (bottom) layers and those at higher (top) layers are more general, as shown in Fig.
Secondly, in dc-ETM, both words 3 E Dx and hierarchical topics {B E R x Y are represented as embedding vectors under the same semantic space, contributing to intuitively measuring and visualizing the distance between different topics (words), which has been proven to be effective capturing the underlying semantic structure as shown in Fig. ((a)
4 Policy Gradient-based Training Algorithm
4.1 ELBO of dc-ETM
As a VAB-like NTM, the developed de-ETM can be trained like usual VAEs by directly maximizing the evidence lower bound (ELBO), specifically
L(x,,) ) (e (=... p(r,, 0..) (0,(((00l=,)ll000..0
where the first term the expected log-likelihood and the other term the Kullback-Leibler (KL) divergence from the prior p(O..) to the variational posterior (x,,,
Through introducing the augmented vectors {œh 1. the oog-liieldhood of æn in dc-ETM can be equivalently reformulated as
(0,,((00),))))))) ((
(np(anl(a0!))) ) II p p( (100))
(8)
= E M( 1-)
Imp(an, (x 11=1) +
(1) 11
Σ In (x 10
(1000
where the function in the second expectation term can be treated as the summation of the set of log-likelihood of (an (0) L be factorized
M==1- Due to the hierarchical network structure, the KI. divergence term can
- p p(0% ((+1) a((+1)))
KL(g(B,le.))lp(O..)) = ΣE E
In
(9)
10 I-)
where 1-) is constructed by a Weibull-based inference network described in Appendix A and p( (1+1)) O,, (1+1)) satisfies gamma prior in Eq. ( and their KL divergence has an analytic expression, benefiting from adopting the Weibull reparameterization technique 200
Combining the aforementioned derivations, the ELBO of dc-ETM can be equivalently rewritten
I.
L(xx) =E 1-,-- In | 11 + ((( (K(a H2.))
In p(a: (x (!!)) Σ
(0
E $0
(
In
p(O 11-1) 0. ((+)))
(10)
which can be directly optimized with gradient-based methods to update both the encoder parameters Ω and decoder parameters V in dc-ETM. We emphasize that, after deriving the augmented vectors
5

---

(ahh ) from Fn via data augmentation technique [8], the first expectation term in L(x,)) will be constant and the ELBO can be directly optimized by maximizing the following loss function
E
p(a [
In p(0 p(++1), 0% 0
|-)
1=1
L
pl (1)
l (p
c (1+1) 0!!
E
In
2(0
1-)
g
1=1
L
Σ 1=1
11) (z
(), $00000000 { @ hs))
(11)
which can be roughly treated as the ELBO of a sequence [a
on 2 generated from a sequence
of latent representations [0!4] .....
2 31 32). and naturally meets the sequence-like generative
process of dc-ETM as discussed in Sec. 3.2
4.2
Optimization with Policy Gradient
Similar to RNN-based model, after augmenting {a> 1/=1 from In, the loss function of dc-ETM defined in Eq. (11) is equal to the summation of L sub-loss functions, where each sub-loss function (,())))))))) ( can be equivalently regarded as a separate loss of a subsequence generation model that
only part of the whole sequential generative model and expected to output wn at the final time step Inspired by the great success achieved by RL methods [33 34 35|
in learning stable long sequence (Markov decision process) with high quality, we consider the sequence-like generation procedure of a L-layer dc-ETM as a Markov decision process with L time steps, and develop a novel training mechanism based Policy Gradient [35] for de-ETM, which injects the future rewards obtained from generating the suffix subsequence into each current sub-loss function ( (ax!))
36
Specifically, we treat the whole de-ETM as stochastic policy network π (a (9) expected to generate fixed-length action sequence [a! :... 2 from the observation T.,, defining the state s as {In, ( ($)(), ( } and the action di as (9) $ o For each time step I, given the current state (!) the policy network 77 (a (s!)) will first sample 0 from the inference network via
0 (0 pn. +
O 1)>))
(12)
and further obtain the corresponding action as
@ = a (1) (())
(13)
which can be regarded directly drawing from π (a (1) (s!")) The state transition is deterministic
after an action has been chosen, indicating that the next state s (1-1) {æ,, (t((), ( h>1-1) } if the
current state sn (1) (In, (t) e!! }>>} and the action all (
Then we take the separate loss ( (xx defined in Eq. as the immediate reward at the the time step 1, formulated
q
r(s (1) (1) = E Byyo In la $100 (00)
E (
In _( (((+1), 0 o(41)))
(14)
and the action-value function can be formulated as
(
(1) (0 = r(s (1) E
rr(s at ai-1111)
(15)
which indicates the expected accumulative reward starting from state _ taking action a and
then generating the suffix subsequence [a (111) an ()) with the policy network n (aal (1) (!ll) discount factor 0 < 7 < 1.
and the

---

Following [33], the objective function of training dc-ETM with policy gradient can be estimated (on one episode) as
J( S2 D = Σ 1=1
11 1sh )=" ( ..0...0) - Σ xaa"" (
O0 (8
(16)
where 52 and L indicate the encoder and decoder parameters in dc-ETM respectively. We note that the expectation EE [-] can be upproximated by sampling methods based on the Weibull reparameterization, and the objective function can be directly optimized by advanced gradient descent algorithms, like
Adam 37 and RMSprop (88. We provide the details PG-based training algorithm in Appendix C
Experiments
To evaluate the effectiveness of the developed dc-ETM and the corresponding policy gradients (PG) based training algorithm, we make extensive experiments on both quantitative and qualitative aspects. Considering there are two dc-ETM variants as described in we use the suffix -a and -3 to distinguish the variants defined in Eq. (2) and Eq. (3) respectively, and highlight whether the dc-ETM is trained with the PG based algorithm in Sec. 4.2 The implementation is available at
5.1 Datasets and Baselines
Datasets:
Four widely used document benchmarks, specifically R8 33 20Newsgroups (20News) 40. Reuters Corpus Volume (RCV1) [4]] and World Wide Web Knowledge Base (WebkB) 42
are included in the following experiments. We summarize the statistics of benehmarks in Appendix [ and follow the procedure in 21 to preprocess these documents to obtain their BoW representations.
Baselines: We compare the developed dc-ETMs with a series of topic models, which can be roughly divided into two categories: 1) shallow topic models such as LDA 6 AVITM [22 and ETM [24]. where LDA a PTM and the others are NTMs: 2) deep topic models including PGBN 8. WHAI 20 and SawETM (21). where PGBN is deep PTM and the others are deep NTMs. We emphasize that WHAT and SawETM are the most relevant strong baselines for comparison, both of which provide hierarchical Weibull-based latent document representations, and SawETM has achieved state-of-the-ar performance on unsupervised document modeling and clustering tasks.
Experimental Settings: To make a fair comparison, we set the same network structure for all deep topic models as [256, 128, 64, 32, 16 from shallow deep. For PTMs, we use the default hyperparameter settings in their published papers and accelerate the Gibbs sampling with GPU, For NTMs, we set the size of their hidden layers as 256, the embedding size as 100 for them incorporating word embeddings, like ETM, SawETM and dc-ETMs, and the mini-batch size as 200. For optimization, we adopt the same Adam optimizer 43 with learning rate of 1e-2. All experiments are performed with an Nvidia RTX 3090 GPU and implemented with PyTorch (44
5.2 Quantitative Comparisons
To investigate whether the proposed skip-connection structures in both dc-ETM variants can alleviate "posterior collapse", especially at higher layers, we compare them with other popular deep topic models in the first part. Then, to investigate whether the mitigation of "posterior collapse" can improve the quality of latent document representations at higher layers, we conduct more quantitative comparisons in the rest parts. We report the error bars in Appendix
Document Modeling: In Fig. 2 for each deep topic model, we plot the curve of point log- likelihood In p(#,l-) as function of iterative epochs conditioned on the t-th-layer reconstruction (0)..' which can be used measure the degree of "posterior collapse" by the relevance between the data sample x,, and its latent representation (,( From the results, we can see that although STwETM and WHAI achieve comparable performance with dc-ET'Ms the first hidden layer in Fig. 2(
their reconstruction quality decreases dramatically with the network going deeper in Fig. 2(b) and
2(c) potentially reflecting that little data information can be propagated higher layers of these Traditional deep topic models. Benefiting from introducing skip connections into the generative process, dc-ETMs can significantly alleviate "posterior collapse" higher hidden layers.
7

---

-03
000
200
-200
160
1o FTM
-00
-50
.000
Epoctees
(a) In p0 n (1) ((
(b) In p(an)) (3) (3)
(c) In plan (5)
(5)
Figure 2: Point log- likelihood In p(æ,, (1() !!!) of different deep topic models on 20News dataset as function of iterative epoches, where o (0))) ! can be treated as the projection of 0 from the latent space to the observation space, as discussed in Sec. 3.2
Perplexity Topic Diversity:
Table I: Comparisons of the average of perplexities and topic
To make more comprehensive
diversities across all hidden layers on various benchmarks.
quantitative comparison, we use
the average of heldout-word per-
Model
Perplexity
Topic Divensity
plexities (the lower the better) and topic diversities (the higher is
R8
20News
RCVI
R8
20News
RCVI
LDA [6]
996
1091
1242
0.288
0.356
0.423
AVITM 12
1030
0.330
0.408
0.483
the better) across all hidden lay-
ETM 24
985
989
1480
0.352
0,410
0.524
to measure the document mod- cling performance and topic qual- ity of these deep topic models with (0!")7 | respectively. The experimental settings are consistent with those in (21), and the experimental re- sults have been exhibited Ta- 3 Benefiting from hierarchical network structures, the modeling capability of deep topic models generally outperform those shallow ones. Thanks to enhancing the connections between the observa- tion and multiple hidden layers with the stee-ccouping generative process, the developed dc-ETMs achieve lower perplexity scores and provide higher-quality topics than traditional topic models. Then, dc-ETMs.
PGBN
657
743
0.355
WITAI 20
773
870
1192
0.183
0.158
0.294
SawEIM 21
732
0.331
and { ( )
dr-ETM-o
521
730
912
0.212
0,281
0.435
ETM
427
710
873
0.346
0.429
0.566
dc-ETM-a (Policy)
463
707
0.279
0.385
0.519
de ETM-8 (Policy)
420
647
841
0.379
0.456
0.584
the PG-based training algorithm brings further performance improvement
Document Clustering: To evalu-
Table 2: Document clustering comparison the 1st hidden layer or the concatenation all hidden layers of different topic models.
ate the quality of the extracted la- tent document representations on
downstream tasks, consider
Model
Layer
WebKB
20News
RS
document clustering, where we use the topic models after train-
Purity
NM1
Purity
NMI
Purity
NMI
LDA AVITM
1
53,40
11.23
41.79
45.15
65.74
40.47
ing to extract the latent represen-
I 54.18
17.77
42.33
46.33
70.96
41.20
12.52
72.20
and then use k-means to pre- diet the clustering labels. Using the Purity and Normalized Mu- tual Information (NMI) as met- rics (the higher the better), the results shown Table 4 demon- strate that concatenating hierar- chical latent document representa- tions extracted by traditional deep topic models cannot improve and even hurt the clustering perfor- mance, potentially indicating that the latent representations at higher layers meaningless. However, distinct from traditional deep
tations of the testing documents
PGBN
All
1
53.58
55.37
16.27
15.39
43.30
41.17
44.20
46.51
72.93
74.52
31.35
41.24
WHAI
1 59.89
25.95
42.25
46.98
74.70
43.98
All
57.46
24.49
32.00
37.51
70.80
41.25
SaWETM
1 57.89
21.91
43.33
50.77
75.25
42.97
All
51.75
20.60
38.69
39.33
75.89
39.55
deETM-a
All 63.18
28.35
41.83
44,52
76.31
43.73
d
1
54.71
21.43
39.80
44.30
74.30
38:63
All
67.29
33.60
45.00
46.20
76.25
45.64
de-EMMa (Policy)
1
49.71
14.86
37.88
43.56
71.65
32.73
All
64.32
33.65
42.21
45.59
77.46
de ETM (Policy)
1
57.32
26.05
40.11
44.12 12
71.30
38
A11
55.79
78.29
8

---

5_0 game team games hockey basehall play players season fans
1: host nntp posting lines subject organization distribution mit world access
5_2: com article writes apr lines subject organization netcom reply mark
5 max israel turkish jews armenian armenians war Israeli jewish armenia
5 president national states health american press united cliton year april
5_5: gun people government right law rights guns state fbi wcapons
5_6: god jesus bible Christian people believe church truth say know
T (5) learned by dc-ETM
5_0: lines subject organization com article just don writes university like
I: lines subject organization article just don university writes like
5.2: lines subject organization com article just don writes university like
5 5_3: lines subject organization com article just don writes university like
lines subject organization article just university writes like
5_5: lines subject organization com article just don university writes like
5_6: lines subject organization com article just don writes university like
(b) + (5) learned by SawETM
Figure 3: The 5th-layer topics learned by dc-ETM and SawETM with the same network structure on 20News, where each topic interpreted by its top- words. More comparisons refer to Appendix
topic models, the concatenation operation on the latent representations of do-ET'Ms can significantly improve the performance, which be attributed to enforcing strong links between the multi-layer representations and the observation with the skip connections the generation.
5.3 Qualitative Analysis
As discussed in Sec. 3.2 the developed dc-ETM inherits both the characteristics of hierarchical topic structure and semantic topic embeddings. Then compare the hierarchical topics of 5-layer dc-ETM trained on 20News with those learned by SawETM for qualitative analysis.
With the visualization techniques g we exhibit the 5th-layer topics learned by dc-ETM and SawETM on 20News in Fig. 3 and Fig. 7 where each topic is interpreted by its top-10 10 words by sorting the word probabilities by descending order. Obviously, the topics learned by SawETM are quite similar, explaining the reason why concatenating its hierarchical latent document representations cannot improve and even hurt the performance downstream tasks. On the contrary, the developed dc-ETM can learn meaningful and diverse topics at higher layers, indicating that more data information is passed higher layers to alleviate "posterior collapse". We also exhibit a 5-layer topic tree learned by dc-ETM in Fig. 7 to illustrate the topic hierarchy of dc-ETM in Appendix N.
Topic Visualization:
Topic Embedding Visualization: After extracting hierarchical topic trees by dc-ETM, we visualize some of these trees originated from different topic nodes layer 5 by projecting their semantic embeddings with t-SNE [45. As shown in Fig. 5(a) we can find that the topics in the same topic tree tend to closer than others from different trees in the semantic embedding space and similar phenomenon occurs in the words for describing the same root topic, which indicates the hierarchy learn by dc-ETM is of high quality. Note that we also visualize the topics consisting of similar top words in Fig. ((b) and 5(c) learned by dc-ETM and SawETM respectively, which demonstrates that the developed de- ETM can provide more meaningful and discriminative topic and word embeddings.
6 Conclusion
To provide higher-quality hierarchical latent representations for deep topic modeling, in this paper, with the deep-coupling generative process, we develop novel de-ETM, which is constructed by introducing skip connections into the generative process of GBN and also incorporates both topic

---

artice, apr subyert, orgoniation netcom, reply, marz
5 layer topics
writes, apr subject lines, organization message, zeply, news, gint
columbis, colorado varimi,, hurvard sgi. frunk uchicago caltoch, linth,
com, netrom, mark ibm sun, stue, suit ect lines, organization,
Jayer topics
article, apr message, news. gitt, repls think, sum, deleted
85: lines, subjort organizaaiin, reply, university good, just. new, keyworts, git
columitia, colsendo virginia harvard, sgi. riank sclocago, guzy, caltech, kettz
com, mark, lom, sut. ulioo, Jim. att, cso, austin
2"d layer topics
42: article writes shink, nows deleted actually just, trus, thd
147: age article, message gint, date writes, may be
17: subject orpancaaiio good, juz got, new, old, make
1 142: sgi calbchh, kntth fankk moml momlity objective invesey, wpd
162: columbia culonado virginia, uchicago, gary arizonn, ettc middwa, ses
111: com jum, ett coup, corporation, stdl doc, opinions stretus
1. 196: netrom mark ihm ucs, indiana ubex
llyer topics
Figure 4: hicrarchical topic tree example learned by a 5-layer dc-ETM on 20News dataset.
sur
pane. buets paree
minkengrrokn
Dutry booke
whnte Sall
5.6 god, Jess, bids poople
picnust
besebal
Flyan
resuue
your
sath teenks (f bery laadd, dadie oial
a JORTS
(a) dc-ETM
(b) dc-ETM
(c) SawETM
Figure 5: I-SNE visualization a) multiple 5-layer hierarchical topic trees learned by de-ETM, whose leaf nodes are distinguished by different colors; b) and c) various semantic topics equipped with their own top-10 representative word embeddings learned by de-ETM and SawETM on 20News.
embedding and Weibull reparameterizattoon techniques. Utilizing the property of sequence-like generation process, design a PG-based training algorithm for de-ETM to further alleviate the information reduction at higher layers. We note that the main idea of designing de-ETM equipped with the PG-based training algorithm can potentially be extended to other deep topic models.
Acknowledgments
This research is supported in part by the National Research Foundation, Singapore under its Industry Alignment Fund Pre-positioning (IAF-PP) Funding Initiative. Any opinions, findings and conclu- sions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. Additionally, this work is supported in part by the National Natural Science Foundation of China under Grant U2 IB2006; in part by Shaanxi Youth Innovation Team Project; part by the 111 Project under Grant 18039; in part by the Fundamental Research Funds for the Central Universities QTZX22160.
References
[1] David M Blei and Michael I Jordan. Modeling annotated data. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 127-134, 2003.
[2] Chaojie Wang, Bo Chen, and Mingyuan Zhou. Multimodal Poisson gamma belief network. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 2492-2499, 2018.
10

---

[3] Chaojie Wang, Hao Zhang, Bo Chen, Dongsheng Wang, Zhengjue Wang, and Mingyuan Zhou. Deep relational topic modeling via graph Poisson gamma belief network. In Advances in Neural Information Processing Systems, 2020.
[4] Jordan Boyd-Graber, David Blei, and Xiaojin Zhu. A topic model for word sense disambigua- tion. In Proceedings of the 2007 joint conference empirical methods in natural language processing and computational natural language learning JEMNLP-CCNNLAN pages 1024-1033, 2007.
[5] Chaojie Wang, Bo Chen, Sucheng Xiao, and Mingyuan Zhou. Convolutional Poisson gamma belief network. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, International Conference on Machine Learning, volume 97, pages 6515-6525, 2019.
[6] David M Blei, Andrew Ng, and Michael Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3JJa)):933-12 2003.
[7] Zhe Gan, Changyou Chen, Ricardo Henao, David Carlson, and Lawrence Carin. Scalable deep Poisson factor analysis for topic modeling. In International Conference on Machine Learning, pages 883-1132, 2015.
[8] Mingyuan Zhou, Yulai Cong, and Bo Chen. Augmentable gamma belief networks. Journal of Machine Learning Research, 17(1):5656-560 5699, 2016.
[9] He Zhao, Lan Du, Wray L. Buntine, and Mingyuan Zhou. Dirichlet belief networks for topic structure learning. In Advances in Neural Information Processing Systems, pages 7966-7977, 2018.
[10] Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014.
[11] Yishu Miao, Lei Yu, and Phil Blunsom. Neural variational inference for text processing. In International Conference on Machine Learning, volume 48, pages 1727-1736, 2016.
[12] Akash Srivastava and Charles Sutton. Autoencoding variational inference for topic models. In International Conference on Learning Representations, 2017.
[13] Dallas Card, Chenhao Tan, and Noah A. Smith. A neural framework for generalized topic models. CoRR, abs/ 1ess10955999 2017.
[14] Hao Zhang, Bo Chen, Long Tian, Zhengjue Wang, and Mingyuan Zhou. Variational hetero- encoder randomized GANs for joint image-text modeling. In International Conference on Learning Representations, 2020.
[15] He Zhao, Dinh Q. Phung, Viet Huynh, Yuan Jin, Lan Du, and Wray L. Buntine. Topic modelling meets deep neural networks: survey, In International Joint Conference on Artificial Intelligence, pages
2021.
[16] Casper Kaae Sønderby, Tapani Raiko, Lars Maalge, Søren Kaae Sønderby, and Ole Winther. Ladder variational autoencoders. In Advances in Neural Information Processing Systems, pages 3738-30 3346,,76,222 2016.
[17] Lars Maalde, Marco Fraccaro, Valentin Liévin, and Ole Winther. BIVA: A very deep hierarchy of latent variables for generative modeling. In Advances in Neural Information Processing Systems, pages 6548-6558, 2019.
[18] Adji Dieng. Yoon Kim, Alexander M Rush, and David M Blei. Avoiding latent variable collapse with generative skip models. In International Conference on Artificial Intelligence and Statistics, pages 2397-2405. PMLR, 2019.
[19] Xuhui Fan, Bin Li, Yaqiong Li, and Scott A. Sisson. Poisson-randomised DirBN: Large mutation needed in Dirichlet belief networks. International Conference on Machine Learning, volume 139, pages 3068-3077, 2021.

---

[20] Hao Zhang, Bo Chen, Dandan Guo, and Mingyuan Zhou. WHAI: Weibull hybrid autoencoding inference for deep topic modeling. In International Conference on Learning Representations. 2018.
[21] Zhibin Duan, Dongsheng Wang, Bo Chen, Chaojie Wang, Wenchao Chen, Yewen Li, Jie Ren, and Mingyuan Zhou. Sawtooth factorial topic embeddings guided gamma belief network. In International Conference on Machine Learning, volume 139, pages 2903-2913, 2021.
[22] Yulai Cong, Bo Chen, Hongwei Liu, and Mingyuan Zhou. Deep latent Dirichlet allocation with stochastic gradient riemannian MCMC. In International Conference on Machine Learning, volume 70, pages 864-873, 2017.
[23] Mingyuan Zhou, Lauren Hannah, David Dunson, and Lawrence Carin. Beta-negative binomial process and Poisson factor analysis. In Artificial Intelligence and Statistics, pages 1462-1471, 2012.
[24] Adji Bousso Dieng, Francisco J. R. Ruiz, and David M. Blei. Topic modeling in embedding spaces. Trans, Assoc. Comput. Linguistics, 8:439-453, 2020.
[25] Dongsheng Wang, Dandan Guo, He Zhao, Huangjie Zheng, Korawat Tanwisuth, Bo Chen, and Mingyuan Zhou. Representing mixtures of word embeddings with mixtures of topic embeddings. In International Conference on Learning Representations, 2022.
[26] Dongsheng Wang, Yishi Xu, Miaoge Li, Zhibin Duan, Chaojie Wang. Bo Chen, and Mingyuan Zhou, Knowledge-aware Bayesian deep topic model. arXiv preprint arXiv:2209. 14228, 2022.
[27] Lin Gui, Jia Leng, Gabriele Pergola, Yu Zhou, Ruifeng Xu, and Yulan He. Neural topic model with reinforcement learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- -JCNLP,, pages 3478 3483, 2019.
[28] Zeinab Shahbazi and Yung-Clun Hong-Cheol Byun. Topic modeling in short-text using non-negative matrix factorization based on deep reinforcement learning. Journal of Intelligent & Fuezy Systems, 39 (1):753-770, 2020.
[29] Amit Kumar, Nazanin Esmaili, and Massimo Piccardi. A reinforced variational autoencoder topic model. In International Conference on Neural Information Processing, pages 360-369, Springer, 2021.
[30] Adji B. Dieng, Francisco R. Ruiz, and David M. Blei. The dynamic embedded topie model. CoRR, abs/1907.05545, 2019.
[31] Aaron Schein, Scott W. Linderman, Mingyuan Zhou, David M. Blei, and Hanna M. Wallach. Poisson-randomized gamma dynamical systems. In Advances in Neural Information Processing Systems, pages 781-792, 2019.
[32] Dandan Guo, Bo Chen, Hao Zhang, and Mingyuan Zhou. Deep Poisson gamma dynamical systems. In Advances in Neural Information Processing Systems, pages 8451-8461, 2018
[33] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. SeqGAN: Sequence generative adversarial nets with policy gradient. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2852-2858, 2017.
[34] Timothy P. Lillicrap. Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In International Conference on Learning Representations, 2016.
[35] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems, pages 1057-1063, 2000.
[36] David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient algorithms. In International conference on machine learning, pages 387-395. PMLR, 2014.
12

---

[37] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2015.
[38] Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on, 14(8):2, 2012.
[39] Franca Debole and Fabrizio Sebastiani. An analysis of the relative hardness of reuters-21578 subsets. Journal of the American Society for Information Science and technology, 56(6): 584-596, 2005.
[40] Thorsten Joachims. probabilistic analysis of the rocchio algorithm with TFIDF for text categorization. Technical report, Carnegie-mellon univ pittsburgh dept of computer science, 1996.
[41] David Lewis, Yiming Yang, Tony Russell-Rose, and Fan Li. Revl: A new benchmark collection for text categorization research. Journal of machine learning research, 5(Apr): 361-397, 2004.
[42] Mark Craven, Andrew McCallum, Dan PiPasquo, Tom Mitchell, and Dayne Freitag. Learning to extract symbolic knowledge from the world wide web. Technical report, Carnegie-mellon univ pittsburgh pa school of computer Science, 1998.
[43] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprini arXiv:1412.6980, 2014.
[44] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf. Edward Yang, Zachary Vito,o, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-peeformance deep learning library. In Advances in Neural Information Processing Systems, 2019.
pages
[45] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of machine learning research, 9(11), 2008
[46] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3):229-256, 1992.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Abstract
(b) Did you describe the limitations of your work? [Yes] See Appendix
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] See supplemen- tal material
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 5.1 and Appendix D
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Appendix
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Section 5.1
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See supplemental
material
(b) Did you mention the license of the assets? [Yes] See supplemental material
(c) Did you include any new assets either the supplemental material or as URL? [Yes] See supplemental material
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "nU9yI8WNLyu", "reviewer": "Reviewer_1MCV", "summary": "This paper presents a hierarchical neural topic model (NTM) along with\na policy gradient approach for learning the parameters. The core of\nthe NTM extends existing work (Embedding Topic Model: ETM). Learning\noccurs within a variational autoencoder framework. To do this\nformulation, the hiearchical parameters are treated as the actions\nthat must be produced, with the corresponding expectations of the ELBO\nas the reward function. The paper performs quantitative and\nqualitative comparisons on four datasets, comparing against six\nbaselines. The paper reports gains in perplexity, topic diversity,\ntopic purity, and NMI.", "questions": "Q1. Please provide a brief explanation of W1 that could be added to a\nrevised paper.\n\nQ2. Please provide a brief explanation of W2 that could be added to a\nrevised paper.\n\nQ3. Please address W3.\n\nQ4. Please provide a brief explanation of W4 that could be added to a\nrevised paper.\n\nQ5. Please provide a brief explanation of W5 that could be added to a\nrevised paper.\n\nQ6. How many samples for the VAE objective (Eq. 11) or the policy\ngradient objective (Eq. 16)?\n", "limitations": "No, see W1-W4.\n\nPost discussion: sufficiently addressed", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "UPDATE post discussion: I thank the authors for their clarifications and promises to revise accordingly. I have updated my main S/W review. I am also updating my score.\n\n=================================\n\nFrom the very beginning of this paper, there is a focus on \"posterior\ncollapse.\" However, the paper does not adequately or properly define\nposterior collapse. This is a major weakness of the paper, and would\nrequire significant revisions throughout. Notably:\n\nW1. How is posterior collapse defined? [Post discussion: thank you for addressing this. I strongly recommend putting the revised definition in the main paper.]\n\nW2. How is posterior collapse being measured? [Post discussion: thank you for addressing this. I strongly recommend clearly tying this to the experimental setup in the main paper.]\n\nW3. What experimental results demonstrate that posterior collapse, as\nmeasured by point (2), has actually been mitigated? [Post discussion: Please add the revised table in the appendix to the main paper.]\n\nBeyond that overarching weakness, this paper does not adequately\naddress other considerations, including broader take-aways:\n\nW4. This paper presents dc-ETM, and two learning algorithms for\nit. What aspects of this work prevent posterior collapse? Is it the\ndefinition of dc-ETM, the policy gradient approach, or both? [Post discussion: thank you for addressing this]\n\nW5. Fundamentally, why does policy gradient result in better results?\nThe two objectives (Eq. 11 and Eq. 16) are very similar, sharing the\nsame subloss computations. From what I can tell, a core difference is\nthe quadratic nature of the policy gradient objective. Is this what is\nresponsible for the improved results, or is there something else at\nplay?\n\nOne strength of this work is that it examines different ways of\ndecoding, notably through Eqns 2 and 3.\n\nW6. Unfortunately, the presentation of the model(s) needs to be\nimproved. While some of this could be a spacing issue, e.g., around\nequation 1), the corresponding prose is quite dense and more difficult\nto get through than it should be. Further, the core factorization and\nprobabilistic identities that this paper relies (e.g., the\ndeconstruction in Eq. 4) on are not specified clearly. I recommend\nsignificantly rewriting and streamlining the presentation of the model\nand underlying justifications. [Post discussion: thank you for addressing this]\n\nW7. Finally, I found the qualitative results to be unclear and\ndifficult to interpret. Figure 5 does not have enough explanation,\neither in the figure itself, the caption, or the corresponding prose. \n[Post discussion: thank you for addressing this]", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "l6bADOFQHMA", "reviewer": "Reviewer_JvJD", "summary": "A new hierarchical neural topic model is proposed that addresses the posterior collapse that occurs at higher levels with previous hierarchical neural topic models. The resulting topic model achieves much higher quality topics at higher levels. The method can be extended to different types of neural topic models.", "questions": "- How does the method perform in comparison to traditional deep topic models like nHDP?\n- What does the comparison look like with respect to topic coherence?\n- Why are the Table 2 results only performed on the first hidden layer representations? \n- Is there some way to evaluate the hierarchy level-wise to be able to make quantitative claims about the topic quality at higher levels?", "limitations": "limitations are not addressed", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# originality\nThe work has a high degree of novelty. The augmented representation of the documents which allows to reduce the posterior collapse is very interesting.\n\n# quality\nThe claims are in general well supported. Nevertheless, the experimental section could be more convincing. It is uncommon nowadays to compare topic models with regard to perplexity though. It has been shown that perplexity does not correlate well with human judgment. Usually, topic quality, a product of diversity and coherence is used instead. For hierarchical topic models, it is even possible to evaluate the different layers separately since the coherence and diversity could be much different depending on the layer. There is a recent attempt in that direction which could be helpful in comparing different hierarchies [1]. The claim that the resulting topics have higher quality is hard to support with only perplexity and diversity measures.\n\n# clarity\nThe paper is well written and structured.\n\n# significance\nThe method is likely to be used and further developed. It is extendable to different deep topic models and thus has general applicability. \n\n\nminor comments:\n- Figure 1: a) says Deep PGM but the description says Deep PTM\n- line 129 alleviate -> alleviates\n\n\n[1] Nagda et al. \"Hierarchical Topic Evaluation: Statistical vs. Neural Models\" Bayesian Deep Learning Workshop at NeurIPS 2021.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "83--Ddt-U4", "reviewer": "Reviewer_kqfN", "summary": "The authors develop a deep-coupling generative process for deep topic modeling, where the key idea is to incorporate skip connections into the generation to alleviate the issue that higher layer representations exhibit similar patterns. The authors further formulate the sequence-like generation procedure as a Markov decision process, and develop a Policy Gradient based (PG-based) learning method. The authors show the benefit of PG-based training over vanilla training for topic modeling. \n\nIn comparison with previous methods, the experimental results show the consistent and significant benefits on three datasets. The qualitative evaluation (i.e., looking into generated topics and topic embeddings) also support the authors claim (overcoming ``posterior collapse”).", "questions": "1. The clear performance gap on dc-etm-alpha (Equation 2) and dc-etm-beta (Equation 3) does seem to be big. The authors can shed some light on it.\n\n2. Regarding PG-based vs non-PG-based -- Table one/two do support that PG-based training is helpful. However, more analysis/discussions could be helpful:\n\n    2.1. What is the motivation to use PG-based training, besides observing the success in [25], [26] and [27].\n\n    2.2 According to Figure 2, PG-training does not seem to benefit point log-likelihood, but in Table 1 and 2, PG-based models exhibit consistent benefits, can the authors discuss more on this \"discrepancy\".\n\n    2.3 Can the authors show qualitative comparison (e.g., t-sne plot of topic embeddings) between dc-ETM-alpha, dc-ETM-beta and dc-ETM-beta (Policy).\n\n3. Regarding hierarchical representation:\n\n    3.1. The table two does show that the higher-layer representations of PGBN/WHAI/SawETM do not help with documentation clustering; Interestingly, the 1st layer hidden layer of PGBN/WHAI/SawETM does seem to outperform dc-ETM.\n\n    3.2. Can the authors compare the 4th/3rd/2nd/1st layer visualization for SawETM and dc-ETM?", "limitations": "Do not see clear issues.\n\nPost discussion: I incline to retain my positive rating.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Novelty/Quality/Contributions:\n\nThe novelty of this draft is not on the techniques used; Actually, the techniques used in this draft are all not new. Using skip connection is heavily motivated by [15, 20], while using Policy gradient is also motivated by the success of previous works like [25], [26] and [27]. \n\nThe authors effectively used those techniques to solve one issue — higher layers of deep PTMs/NTMs exhibit similar patterns — observed by [15]. This is the major novelty and contribution of the paper.\n\nThe authors also conducted thorough experimental study to support their claims. The authors found the proposed methods do outperform the baseline methods in terms of point log-likelihood (especially for higher-layer representations), Perplexity, topic diversity, clustering purity/NMI, diversity of generated topics (especially for higher-layer) and visual separability on topic embeddings. The authors also show the consistent benefits obtained from policy gradient based training on all the three datasets. \n\nThis work does seem to be a high-quality draft to me, though I do have some questions for the authors in the Questions section.\n\n\nClarity:\n\nOverall, the draft is not difficult to follow, with some potentially small issues:\n1. The relationship of the projection matrices of different layers are not realized in the Figure 1(c)\n\n2. Line 177 and Equation (9) are not consistent\n\n3. Do superscript l (line 105) and superscript (l) (line 100) indicate the same thing?\n\n\nSignificance:\n\nGiven good quantitative and qualitative performance, the proposed methods should be of sufficient interest to the topic modeling community. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept