Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the hierarchical topic modeling problem. Existing studies generally learn topic representations in the Euclidean space, which might lead to some limitations in capturing hierarchical relations. To be more specific, the ability to model complex patterns is inherently bounded by the dimensionality of the embedding space. On the other hand, side information such as taxonomy of concepts is sometimes available to guide the learning of hierarchical topics, which might be challenging to preserve in the Euclidean space. As a consequence, this paper proposes a novel framework that introduces learning word and topic representations in the hyperbolic space. Experiments on three public text datasets and auxiliary ablation/case studies demonstrate the effectiveness of the proposed framework.
strengths_and_weaknesses: Strengths:
1. This paper studies an important task. Hierarchical topic modeling is a well-studied yet important problem that has the potential of benefiting a wide spectrum of downstream applications such as classification, named entity recognition, etc.
2. The motivation of the solution is very clear. As is well known, Euclidean space falls short when it comes to modeling hierarchical structures. In contrast, hyperbolic space has nice properties of hierarchical awareness and spacious room which benefits hierarchical relationship modeling.
3. The experiments are thorough. They are sufficient to justify the superiority of hyperbolic space under the problem set. An ablation study between HyperMiner and HyperMiner-KG validates the modeling of hierarchical external knowledge. Meanwhile,  the experiments also show that the proposed method is extendible to all kinds of ETM models.
4. The paper is carefully written and well organized.

Weaknesses:
1. More related works could be potentially included and discussed [1].

[1] Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding

Minor issues:
- Typos
In line 137, "Sine" should be "Since"
In Appendix line 93, "wth" should be "with"
questions: 1. If the ability of Euclidean space is bounded by its dimensionality, then what interests the reviewer is that if the dimensionality of one hyperbolic space is d, then what the dimension would be for a Euclidean space to achieve comparable performance in the topic modeling task (e.g., $k * d$ where $k$ is a fixed number)?
limitations: As mentioned in the appendix, the main limitation of this work is the mismatch problem between the given structural knowledge and the target corpus. To provide proper guidance for mining an interpretable topic taxonomy, the prior structural knowledge should be well matched with the corresponding dataset.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper improves embedded topic modeling by using a hyperbolic space. The proposed approach particularly helps considering hierarchical structure of words and topics. In addition, the authors propose a revised contrastive loss to inject prior knowledge. The evaluation compares the proposed approach with 6 existing embedding and non-neural-embedding-based topic modeling approaches on 4 datasets. The proposed approach is shown to outperform the existing approaches in most configurations. Some analyses including the 2D visualization show the proposed approach performs as intended and encode the hierarchical information.
strengths_and_weaknesses: Strengths
- S1. The proposed approach encodes hierarchical information, and can incorporate the existing knowledge by adopting the hyperbolic space into an existing neural topic modeling approach.
- S2. The paper has good presentation and readability in general.
- S3. The solid empirical results show the performance of the constructed topic model and the analysis shows that the approach works as intended.

Weaknesses
- W1. The variants HyperETM, HyperMiner are HyperMiner-KG are not explained, and it thus reduces the reproducibility of the experiments.
- W2. The topic taxonomy (Figure 4b) is hard to relate without document examples and/or summary of the topics.
- W3. Discussion of side-effect is missing. As the hyperbolic space is enforced, the freedom of representing a topic decreases. As such, there can be side-effects and a discussion of desired properties in topic modeling and side-effects from this approach would be beneficial. For example, is the tree-structure the best or can it be a rather limiting factor?
questions: The paper is quite straightforward and I appreciate it. Please take a look at W1-W3 for suggestions to further improve the presentation.
limitations: I cannot spot any negative societal impact of this work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a neural hierarchical topic model that uses
hyperbolic embeddings. The paper uses a Gamma-Poisson factorization to
facilitate optimizing a variational objective. They augment this
objective with a taxonomy-aware contrastive loss as a way of
addressing hyperbolic manifold learning. Across four datasets, they
compare against six baselines. Averaged across multiple runs, they
demonstrate small but consistent improvements in their metrics (all
variances are fairly small).
strengths_and_weaknesses: Update post-response: I thank the authors for their answers to my questions. 

I understand there's a limited amount that can be done during the author response. That said, while the additional taxonomy experiments are encouraging, though for inclusion in a paper they would need additional discussion. 

For whatever revision is next, I suggest that while the proposed clarifications of Poincare vs. Lorentz may be helpful for some readers, the main question is, "is this level of detail necessary for the main paper?" If the answer is yes, then this point needs to be made more immediately obvious. 


=========================

Overall, this paper has a clear mathematical overview. For the
singular main contribution (hyperbolic hierarchical topic modeling)
this paper presents a decent amount of experimental evidence that the
proposed approach is effective on purely quantitative measures. While
this paper does not address the impact of the proposed approach on the
topic model as a language model (perplexity), the classification
results give some evidence as to the effect on document modeling.

However, lack of examples and experimentation when it comes to the
taxonomy is noticable. This is especially the case given the tied
nature of the model to the taxonomy (L167-169). This is important not
just for overall completeness, but to better understand the results
that have been presented. The gains, e.g., in Figure 3, are generally
small but consistent. That alone is not a negative. However, those
smaller gains are less grounded without additional examples or
experimentation.

This paper has some reorganization and other presentation
issues. While these issues could arguably be viewed as "easily
addressable," they do impact the ability of a reader to udnerstand
what the paper's take-aways are.

* The current organization of the paper suggests that the authors
  apply hyperbolic embeddings to a Poisson Gamma belief network
  only. However, as the results show, this is not the case. I
  recommend reorganizing so that the main aspect of "hyperbolic
  embeddings applied to (hierarchical) topic models" stands out.

* Since Figure 3 only makes distinctions based on color, it is a bit
  difficult to quickly interpret.

* The mathematical introduction of both the Lorentz and Poincare
  similarities is confusing and, given the amount of space dedicated
  to the equivalence explanation (L140), too nuanced. Indeed, even
  from the appendix this equivalence is not clear (am I missing
  something obvious)?

* While Fig. 4b is helpful, 4a is too busy. The concept hierarchy is
  clear (but that's a given), while the lexical hierarchy, except at a
  very coarse granularity, is not.
questions: Q1. What is the impact of changing aspects of the taxonomy (more or less, or regrouping concepts)?

Q2. What is the impact of the positive and negative selection in the contrastive loss?

Q3. Please address my confusion regarding the Lorentz vs. Poincare approaches.
limitations: No: see taxonomy experiments/Q1.
review_title: null

Meta-review: metareview: Hyperbolic embeddings were a fascinating alternative to Euclidean embeddings that never seemed to take off, despite having significant conceptual advantages in representing the oddities of semantics. I am happy to see more work on curved spaces as a tool for semantic analysis! This work has strong reviews, and reviewers were generally happy with the author responses. I'd like to see it published.
confidence: Certain
Decision: Accept