Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a contrastive learning framework to improve the performance of neural topic model in terms of topic coherence. To guide the model focus on the negative samples and separate topics more clearly, the paper proposes an adaptive scheduling strategy to estimate $\beta$. And a word-based sampling strategy is proposed to sample negative and positive samples, which is mainly based on the tf-idf scores.
main_review: Strengths:
1. Using contrastive learning for neural topic model seems novel, and can address the main drawbacks of Adversarial Topic Model.
2. The experiments and analysis are comprehensive.

Weakness:
1. The improvement over traditional contrastive learning framework seems limited. And why the authors choose the triangle scheduling method? Does this schedular have better performance than others? Is there any theoretical or empirical analysis about why this scheduling method performs well?
2. The novelty of the sampling strategy seems limited. Are there any other statistical indicators for sampling besides tf-idf score? What is the novelty or improvements compared with other word-based sampling methods?
3. The performance gap through all epochs in Figure 4 seems similar all the time. Can the authors provide some analysis toward such phenomenon?

limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposed a Neural Topic Model based on contrastive learning, which considers both effects of positive and negative pairs. They proposed a sampling strategy that takes inspiration from human behavior in differentiating documents. Experimental results show that it outperforms other state-of-the-art neural topic models in terms of topic coherence.

main_review: Clarity: 
This paper is clearly written and well organized.

Originality: 
This work is a combination of well-known techniques, e.g., contrastive learning. They proposed a novel sampling strategy that takes inspiration from human behavior, to help neural topic models learn more meaningful representations. This work is simple and easy-to-follow. 

Quality&Significance: 
This work is technically sound, and all claims are well supported by both theoretical analysis and experimental results. However, the improvements over the baseline methods are small.

- My biggest concern is that the bag-of-word models may not consider the sequence information of the words in the document, so it poses a big challenge to the promotion of this model.
- From table 3, it is interesting that the positive sampling contributed more improvements. Does it mean that the original model is not good enough to identify redundant and uncritical information?
limitations_and_societal_impact: The authors did not discuss the limitations, but this work has no potential negative societal impact.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors propose a novel contrastive objective for the neural topic model, which models the relations among samples without relying on the generative-discriminative architecture. In this work, they reveal that capturing the mutual information between the prototype and its positive samples provides a strong foundation for constructing coherent topics; while differentiating the prototype from the negative samples plays a less important role. Motivated by human behavior when comparing different documents, they also develop a novel sampling strategy to generate positive and negative samples under neural topic model setting, producing informative samples compared with other sampling strategies. Their experimental results show that their proposed method outperforms other state-of-the-art neural topic models in terms of topic coherence.
main_review: 
This approach aims to learn document representations in the neural topic model as a contrastive objective and proposes a novel sampling strategy for contrastive learning, which is novel and interesting to me. The claims are well supported by experimental results. The paper is well written and easy to follow. And the results are moderately important.
limitations_and_societal_impact: I wonder if other methods of measuring the distance between z in Eq 4 are available. It may be better to report the efficiency or the complexity of the proposed sampling strategy. Some references need to be clarified as they have been accepted by a conference or a journal but have still been linked with an Arxiv version, such as “Topic modeling with wasserstein autoencoders” .
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a novel approach to re-formulate discriminative goals as an optimization problem and designs a novel sampling method that facilitates the integration of external variables. It is a strong approach that outperforms other state-of-the-art neural topic models in three common benchmark datasets.
main_review: This paper explores a challenging problem and proposes a novel and intuitive approach for topic modeling. The paper is well-written and well-motivated. I think this paper may attract lots of researchers. Some minor issues should be addressed. For example, in Figure 3, the yellow and blue bars should have a legend.  In Figure 2, I recommend the authors add some descriptions of sampling in the caption. 
limitations_and_societal_impact: The paper do not address the limitations and societal impact.
review_title: null

Meta-review: This paper tackles neural topic model training and proposes a new sampling strategy, altogether improving performance under NPMI, topic coherence and downstream classification performance. 

The reviewers find the paper interesting, novel, and intuitive, with some concerns about design choices. The authors answered the reviewers questions thoroughly with some additional ablations that help justify the design choices, e.g., the tf-idf sample weighting strategy. The authors should include these additional results in the paper.
I would also recommend some polishing of the presentation: for example in Fig 3 the fonts are too small and the bars on the right side are not labelled; could that figure not be a table? Additionally, tf-idf should not be rendered in math mode (surrounded by \$); instead, use \\emph.
Decision: Accept (Poster)