Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes BAT-Chain, a hierarchical topic model leveraging multi-layer conditional transport (CT) theory to seize topic structures. The proposal is inspired by previous application of CT to the single-layer topic modeling.  The authors conduct extensive experiments in both textual and visual settings to showcase the merits of their method.
strength_and_weaknesses: **Strengths**: 

The authors introduce a novel solution for discovering topic hierarchy. The paper is generally smooth to follow, and the experiments can prove the prowess of conditional transport for hierarchical topic modeling.

**Weaknesses**: 

From my perspective, there exist several drawbacks in the Introduction, Methodology, and Experiment sections:

(1) In the introduction section, I cannot see clearly why reparameterization and KL divergence hurt AVI, especially if AVI and CT are adapted to hierarchical topic modeling, and what merits does CT provide to address the advantages of AVI?

(2) The motivation of CT for hierarchical topic modeling is quite obscure. The introduction only explained that CT was applied because previous single-layer topic modeling had not continued to explore its usage for hierarchical circumstances.

(3) The methodology section mentioned that ``A TOPIC in layer 1 is expected to capture more general information than A WORD in layer 0``. Comparing a topic with a word sounds unnatural to me. Such interpretation somehow makes the hypothetical basis for CT topic hierarchy less persuasive. 

(4) The desiderata of the proposed framework is that the upper layers will capture more general topics than the lower layers. Nevertheless, Figure 3 does not clearly indicate the aforementioned relationship. In contrast, the illustrated upper topics have more tendency to display semantic proximity to the lower ones, instead of generalization relation. Additionally, the topics appear to be incoherent as well. For example, why do topic 52 and 24 comprise ``philosopher`` and ``medical``, respectively?
clarity,_quality,_novelty_and_reproducibility: (1) Clarity: The paper is cogent and polished. In detail, the delineation in the methodology section is sound and intuitive. I can grasp the mechanism of the bidirectional training strategy and the conditional transport objective.

(2) Originality/Novelty: The paper presents a novel framework for hierarchical topic modeling based upon the optimal transport theory basis. 

(3) Quality/Significance: The experimental section indicates the improvement over recent SOTA baselines on diverse datasets. This demonstrates the efficiency and robustness of the proposed method.

(4) Reproducibility: The submission provides source code for re-implementation, which would help the reproducibility.
summary_of_the_review: A novel approach is proposed to tackle the multi-layer topic discovery problem. The elucidation of the CT framework is rational and comprehensible. However, there is some degree of vagueness in the motivation and hypothetical discussion. Moreover, the qualitative analysis is also not convincing, which drives me to my overall judgement.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a hierarchical topic modeling approach based on conditional transport, combining previous research on hierarchical topic models with recent research on conditional transport. The idea is that documents are represented by discrete distributions over word embeddings, topics at each level of the hierarchy are also represented via embeddings, and similarity between topics at different levels of the hierarchy is enforced via a conditional transport cost between each document's distribution-over-embedding representations at adjacent layers/levels of the hierarchy. 
strength_and_weaknesses: Strengths:

- The proposed formulation is sensible and elegant, combining ideas from two previously disparate research directions in a clever way to develop a sophisticated model that leverages the advantages of both.

- The conditional transport approach is expected to provide computational advantages over the previous Bayesian probabilistic modeling approaches.

- The application of the method to image data as well as text was quite interesting.

- It's nice that the method can leverage TFIDF data as input instead of bag of words data. I have not seen topic models that do this. It probably improved performance. (It would however be worth doing a comparison to bag of words to show this, and this was not done in the present manuscript.)

Weaknesses:

- While the modeling formulation is quite compelling, the proposed training algorithm is not well justified. Two update steps are proposed: an "upward warming up" used in an initial phase and a "downward refining" step (more accurate?). The upward phase ignores half of the terms in the loss function, so is very approximate. The downward phase updates one layer at a time, leveraging the approximate solutions from the layer below computed in the first pass. The layers are not revisited, so every update depends on the approximate first-pass solution from the layer below. This may be a practical approach, but it is clearly not optimal.

Potentially, a better approach might be to perform stochastic gradient updates where the stochasticity is over the loss terms from all of the layers together. I.e., in each stochastic gradient iteration, choose a random document j and a random layer l and take a stochastic gradient step updating Equation 3 with the chosen j and l. This algorithm is a stochastic gradient optimizer for the overall loss function, which is easier to justify than the proposed algorithm.

- The "Bayesian flavor" interpretation of the method is a bit of a stretch. I see no substantive connection to Bayesian updating, as the paper claims.

- Perplexity results are not given as it is stated the metric is "unsuitable for CT based topic models." Why is it unsuitable? Is it because CT-based topic models look bad under this metric? (This isn't a deal-breaker since the TC and TD results were good, and these metrics are arguably more important. But there needs to be a better explanation of it.)

- Minor point: A "BAT-chain-U" baseline is mentioned but I do not see results for it in any of the tables.
clarity,_quality,_novelty_and_reproducibility: Clarity: the paper is generally reasonably easy to follow.

Quality: while the modeling formulation is novel and interesting, the proposed algorithm is not well justified and may not succeed at optimizing the model's overall objective function.

Novelty: this work combines ideas from two different sub-areas of topic modeling work. As such, the contribution is sensible if somewhat incremental.

Reproducibility: code was provided in the supplementary, which would help with reproducibility.
summary_of_the_review: The proposed CT-based hierarchical topic model is an interesting combination of previous and recent ideas into a new sophisticated model with nice and potentially useful properties. While the work tackles a worthy problem and proposes an elegant formulation, the training algorithm is heuristic in nature and its ability to optimize the stated objective function is not theoretically motivated. It is likely that a modification of the proposed method could be on a firmer theoretical footing (see the review above). A couple of other claims in the paper are unsupported as well (E.g. the "Bayesian" interpretation, perplexity metric is not appropriate). Overall, the research is promising but it needs a bit more work before it is ready for publication.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a topic model that discovers a topic hierarchy and works with word embeddings. Inference in the model is done by using conditional transport. The inference is also linked to the Bayesian inference. 
strength_and_weaknesses: **Strengths**:
* Tackling topic hierarchy discovery
* Rather extensive experiments
* Easy to follow text

**Weaknesess**:
* (minor) Polishing is required (some examples below) but should be doable during the rebuttal stage
* (minor) Although there is nothing apparently wrong with the paper, it doesn’t excite too much. Topic models is a well-researched area. The paper takes the previously proposed settings with word embeddings and topics living in the same space as embeddings with inference based on conditional transport. The paper then extends these ideas to discover topic hierarchy. I can’t say there is not enough novelty in this, but it is just not exciting as looks like a kind of natural extension of those previous settings – it is probably easy to say when it has been done already in this paper, but still. The paper does make an attempt on bringing something else by discussing the link with Bayesian learning – which is a great way to “spice” things up. However, and this is a separate weakness on its own, this link to Bayesian learning discussion is a little bit weak. It would be great to see it more elaborated, with probably some illustrative examples. The whole link is currently based on the fact that in Bayesian learning we have likelihood from the data (i.e., words) and prior from topics of the higher levels and in the proposed model training of a topic on each level is done by transferring the information from below (which is words at the end) and above (which is topics of the higher levels). 
* The link to Bayesian learning is not convincing or elaborated enough (see details above). I believe the paper should either elaborate on this or downplay this link in the text (it is even used in the title which is not justified in the text). 

**Question (which may lead to weaknesses)**:

The provided code looks very similar to the code for the WeTe model, moreover, it is even called WeTe. First, WeTe initial code should be acknowledged. Second, could the authors please briefly discuss in their rebuttal what are the main differences in the code? 

clarity,_quality,_novelty_and_reproducibility: **Clarity**: the paper is mostly well written and easy to follow. There are some minor issues that would be benefited by polishing of the text. Details below.

**Quality**: The method and experiments appear to be sound. 

**Novelty**: Considering the question above is resolved positively, the paper appears to be novel. 

**Reproducibility**: The paper discusses the details of implementation and provides the source code, therefore, the results should be reproducible. However, some details are missing, see e.g. 12 and 13 points below.

Specific comments/suggestions:
1. Introduction. Second paragraph. “Despite the success…, the learning … is done by Bayesian posterior inference…” – odd wording if to ignore the “which” statement that follows. 
2. Introduction. Second paragraph. “Another concern comes from the pure likelihood maximization” – not sure what “pure likelihood maximization” is supposed to mean here.
3. Introduction. Second paragraph. “even acute” – “even more acute”? Otherwise, it sounds a bit odd.
4. Introduction. Third paragraph. “both distributions” -> “the two distributions”
5. Introduction. 4th paragraph. “This results in a more flexible and efficient method” – more than what?
6. Section 2. Second paragraph. “To measure such two discrete distributions” – to measure a distance between such two discrete distributions?
7. End of page 3. “HTMs” – acronym is not defined
8. Section 3.2. First paragraph. “With the fact that …” – unfinished sentence
9. Experiments. “For the ETM, NSTM and WeTe that work with word embeddings, we use the pre-trained GloVe vectors” – the proposed work also work with word embeddings, does it use the same GloVe vectors?
10. Experiments. Why [100, 64, 32] those numbers were chosen for topic numbers?
11. Experiments. Metrics Purity and NMI are unclear from the description.
12. Figure 2. What is x-axis? 
13. It is not clear how could links between topics on different layers be computed? For example, those links in Figure 3
14. First line on page 13. “where we specific transport cost”: “we specify”? 

Minor
1. Redundant commas:
    * Introduction. 4th paragraph. “hierarchical, topical, distributional view”
    * Introduction. 5th paragraph. “is a straightforward, and effective approach”
2. Section 2. First paragraph. “states with the satisfy” – redundant “the”
3. Section 3.2. First paragraph. “corresponds to the learning” – redundant “the”
4. Section 3. Last paragraph. “with THE Bayesian flavor”
5. Footnote 1. “are not be applied” -> “are not applicable”

summary_of_the_review: I believe the question about code similarity would be resolved (there is nothing wrong in using the existing code as a base), then the only real issues left would be overselling of the Bayesian link, which is fixable by downplaying it in the text or elaborating on it to be more solid, and this lack of excitement about the method, but this is not really an issue and may be just me. If to look at the facts, this is a solid piece of work, extending the existing topic modeling approach for discovering topic hierarchies with rather extensive empirical study to support this extension.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes BAT-Chain, a novel model to mine hierarchical topics and document representation under the conditional Bayesian-aware transport framework. Extensive experiments on text corpora show the effectiveness of the proposed method.

In general, the paper is well-written and easy to follow. The experimental results demonstrate the efficiency and robustness of the proposed method. The reviewers raised a few concerns about the technical details, clarification, and novelty of the paper. The authors addressed some of them but the reviewers are not fully convinced, especially about the technical contributions of the paper.
justification_for_why_not_higher_score: The technical contribution of the paper is somewhat limited.
Decision: Reject