Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposes a VAE inference network for a non-parametric topic model.

The model on page 4 is confusing to me since this is a topic model, so document-specific topic distributions are required, but what is shown is only stick-breaking for a mixture model.

From what I can tell, the model itself is not new, only the fact that a VAE is used to approximate the posterior. In this case, if the model is nonparametric, then comparing with Wang, et al (2011) seems the most relevant non-deep approach. Given the factorization used in that paper, the q distributions are provably optimal by the standard method. Therefore, something must be gained by the VAE due to a non-factorized q. This would be best shown by comparing with the corresponding non-deep version of the model rather than LDA and other deep models.
review_title: seems to miss comparable non-deep comparisons

final_score: rating: 3: Clear rejection
review_content: The paper constructs infinite Topic Model with Variational Auto-Encoders (iTM-VAE) by combining stick-breaking variational auto-encoder (SB-VAE) of Nalisnick & Smyth (2017) with latent Dirichlet allocation (LDA) and several inference techniques used in Miao et al. (2016 & 2017). A main difference from Autoencoded Variational Inference For Topic Model (AVITM) of Srivastava & Sutton (2017), which had already applied VAE to LDA, is that the Dirichlet-distributed topic distribution vector for each document is now imposed with a stick-breaking prior. To address the challenge of reparameterizing the beta distributions used in stick-breaking, the paper follows SB-VAE to use the Kumaraswamy distributions to approximate the beta distributions.

The novelty of the paper does not appear to be significant, considering that most of the key techniques used in the paper had already appeared in several related papers, such as Nalisnick & Smyth (2017), Srivastava & Sutton (2017), and Miao et al. (2016 & 2017). 

While experiments show the proposed models outperform the others quantitatively (perplexity and coherence), the paper does not provide sufficient justifications on why ITM-VAE is better. In particular, it provides little information about how the two baselines, LDA and HDP, are implemented (e.g., via mean-field variational inference, VAE, or MCMC?) and how their perplexities and topic coherences are computed. In addition, achieving the best performance with about 20 topics seem quite surprising for 20news and RCV-v2. It is hard to imagine 20 news, which consists of articles in 20 different newsgroups, can be well characterized by about 20 different topics. Is there a tuning parameter that significantly impacts the number of topics inferred by iTM-VAE?

Another clear problem of the paper is that the “Bayesian nonparametric” generative procedure specified in Section 4.1 is not correct in theory. More specifically, independently drawing the document-specific pi vectors from the stick-breaking processes will lead to zero sharing between the atoms of different stick-breaking process draws. To make the paper theoretically sound as a Bayesian nonparametric topic model that uses the stick-breaking construction, please refer to Teh et al. (2006, 2008) and Wang et al. (2011) for the correct construction that ties the document-specific pi vectors with a globally shared stick-breaking process. 
review_title: The paper constructs infinite Topic Model with Variational Auto-Encoders by combining stick-breaking variational auto-encoder (SB-VAE) of Nalisnick & Smyth (2017) with latent Dirichlet allocation (LDA). The paper is not sufficiently novel and the proposed Bayesian nonparametric generative model is not theoretically sound.

final_score: rating: 7: Good paper, accept
review_content: "topic modeling of text documents one of most important tasks"
Does this claim have any backing?

"inference of HDP is more complicated and not easy to be applied to new models"  Really an artifact of the misguided nature of earlier work. The posterior for the $\vec\pi$ of a elements of DP or HDP can be made a Dirichlet, made finite by keeping a "remainder" term and appropriate augmentation.  Hughes, Kim and Sudderth (2015) have avoided stick-breaking and CRPs altogether, as have others in earlier work. Extensive models building on simple HDP doing all sorts of things have been developed.

Variational stick-breaking methods never seemed to have worked well.  I suspect you could achieve better results by replacing them as well, but you would have to replace the tree of betas and extend your Kumaraswamy distribution, so it may not work.  Anyway, perhaps an avenue for future work.

"infinite topic models" I've always taken the view that the use of the word "infinite" in machine learning is a kind of NIPSian machismo. In HDP-LDA at least, the major benefit in model performance comes from fitting what you call $\vec\pi$, which is uniform in vanilla LDA, and note that the number of topics "found" by a HDP-LDA sampler can be made to vary quite widely by varying what you call $\alpha$, so any statement about the "right" number of topics is questionable.  So the claim in 3rd paragraph of Section 2, "superior" and "self-determined topic number" I'd say are misguided.  Plenty of experimental work to support this.

In Related Work, you seem to only mention HDP for non-parametric topic models.  More work exists, for instance using Pitman-Yor distributions for modelling words and using Gibbs samplers that are efficient and don't rely on the memory hungry HCRP.

Good to see a prior is placed on the concentration parameter.  Very important and not well done in the community, usually.  
ADDED:  Originally done by Teh et al for HDP-LDA, and subsequently done
by several, including Kim et al 2016.   Others stress the importance of this.  You need to
cite at least Teh et al. in 5.4 to show this isn't new and the importance is well known.

The Prod version is a very nice idea.  Great results.  This looks original, but I'm not expert enough in the huge masses of new deep neural network research popping up.

You've upped the standard a bit by doing good experimental work.  Oftentimes this is done poorly and one is left wondering.  A lot of effort went into this.
ADDED:   usually like to see more data sets experimented with

What code is used for HDP-LDA?  Teh's original Matlab HCRP sampler does pretty well because at least he samples hyperparameters and can scale to 100k documents (yes, I tried). The comparison with LDA makes me suspicious. For instance, on 20News, a good non-parametric LDA will find well over 400 topics and roundly beat LDA on just 50 or 200.  If reporting LDA, or HDP-LDA, it should be standard to do hyperparameter fitting and you need to mention what you did as this makes a big difference.
ADDED:   20News results still poor for HPD, but its probably the implementation used ... their
        online variational algorithm only has advantages for large data sets 

Pros:   
* interesting new prod model with good results
* alternative "deep" approach to a HDL-LDA model
* good(-ish) experimental work
Cons:
* could do with a competitive non-parametric LDA implementation

ADDED:   good review responses generally

review_title: well rounded work with good experimention and nice ideas, but some questions

Meta-review: The paper proposes a BNP topic model that uses a stick-breaking prior over document topics and performs VAE-style inference over them. Unfortunately, the novelty of this work is limited, as VAE-like inference for LDA-like models, inference with stick-breaking priors for VAEs, and placing a prior on the concentration parameter in a non-parametric topic model have all been done before (see e.g. Srivastava & Sutton (2017), Nalisnick & Smyth (2017), and Teh, Kurihara & Welling (2007) respectively). There are also concerns about the correctness of treating topics as parameters (as opposed to random variables) in the proposed model. The authors' clarification regarding this point was helpful but not sufficient to show the validity of the approach.
Decision: Reject