Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces a new model for performing multi-label classification which is particularly well-suited to situations where there are elements of structure (hierarchy, mutual exclusion) in the label space. The fundamental idea is to use hyperplanes in hyperbolic space as the decision boundaries for label assignment. The main benefit of hyperbolic space over Euclidean space is that there are infinitely many hyperplanes in hyperbolic space which are non-parallel and do not intersect, thus allowing for the regions corresponding to label assignment to capture hierarchical and exclusionary patterns. The previous SOTA model used axis-aligned hyperrectangles, or "boxes", and the authors argue that such a constrained geometric region is not in alignment with the natural decision boundaries provided by neural network architectures, whereas using the hyperplanes in hyperbolic space are more amenable to the output of such encoders. A thorough analysis on 12 datasets, along with additional ablation studies are provided in support of these claims.
strengths_and_weaknesses: Originality: The method is novel, and the reasons for this approach are well-motivated.

Quality: The submission is technically sound. The experimental results, while only a minor improvement quantitatively, are statistically significant. One caveat, which will be brought up in the questions, is that it is not necessarily clear that the full benefit comes from the use of hyperbolic hyperplanes, but rather the injection of the mutual exclusion constraints, which other models could also potentially easily exploit. Still, even if the model is only on-par with SOTA results when taking this into account, the underlying idea is sound.

Clarity: The submission is mostly clear, but there are a number of improvements to the presentation I would recommend in the Questions section. Even so, I feel I would be able to reproduce the model from the description provided in the paper.

Significance: The task is of interest to a number of researchers, and more broadly the idea of the incorporation of structure in Hyperbolic space would be of interest to the community of researchers working on such representations. 
questions: 1. On lines 39-40 the authors write that current methods ignore the importance of constraining mutual exclusion, and while they do highlight an important point (that zero implication violation could be attained by assigning the same score to all labels) it should also be noted that other models (in particular, those based on box regions) could also very easily incorporate a similar mutual exclusion loss. For example, MBM could just include a loss which measures the volume of intersection between boxes associated with mutually exclusive labels, thus encouraging these boxes to be disjoint. With this awareness, do the authors feel that it is worth noting that HMI and HMI+HLR columns in Table 1 are making use of this additional information? Alternatively, it might be relatively easy to implement and train such a method for MBM, or to rerun HMI and HMI+HLR with just the implication information to provide two additional columns in Table 1.

2. Patel et al. 2021 also included an evaluation of the models without any constraint injection, as this is the setting which may be most common in practice. Would it be possible to include such an evaluation on the 12 datasets, at least in the Appendix?

3. I did not understand the section on "Classification via hyperbolic logistic regression" (lines 188-195). Why is it desirable to consider the distances from an instance to a Poincaré hyperplane of a label as the logit? Wouldn't we want a higher score for points points which are further "inside" the n-Ball, rather than on the boundary?

### Suggestions
1. Line 114-115: The sentence here reads as though a (single) Poincaré hyperplane is the union of a hypersurface $\partial \mathbb B^n \cap \mathbb D^n$ and all linear subspaces going through the origin, which is not what is meant. Instead, consider saying that "Poincaré hyperplanes are defined by $\partial \mathbb B^n(\mathbf z) \cap \mathbb D^n$ where $\mathbb B^n(]mathbf z)$ is a ball centered at $z$ whose boundary $\partial \mathbb B^n(\mathbf z)$ intersects the Poincaré ball $\mathbb D^n$ perpendicularly. In the limit, as $|\mathbf z| \to \infty$, we obtain linear subspaces going through the origin, which are also considered that Poincaré hyperplanes, however we exclude this limiting case from consideration in practice."
2. As suggested in 1, it seems far simpler to parameterize the hyperplanes using the center of the ball rather than the "center point $\mathbf c$". I don't believe you actually need the notion of the center point, and as such you will be able to remove quite a lot of unrelated technical details (Definition 4 - which is good, because the log map hadn't been defined previously - much of the discussion in lines 119-141). Instead of the function $\mathcal C$ mapping to a center point, we can either think of it mapping to the center of the ball or simply think of it mapping to a Poincaré hyperplane itself, without specifying a particular parameterization. Many additional subscripts can be removed as well.
3. Lines 130-131: It is mentioned that Euclidean hyperplanes do not naturally allow for modeling set-theoretic semantics, but this is a bit untrue - they certainly can capture set-theoretic relationships in the same way that the hyperbolic space allows for, but they are limited in the complexity of the patterns of such relationships that they can capture. I believe the fact you wish to highlight is that the existence of infinitely many non-parallel hyperplanes in hyperbolic space which do not intersect implies that decision boundaries in hyperbolic space can capture more complicated set-theoretic interactions, such as mutual exclusivity.
4. Lines 147-148: I could not understand the sentence starting "In particular". I think what is meant is that "In particular, an $n$-ball $\mathbb B_\mathbf w$ contains $\mathbb B_\mathbf u$ if and only if $\lVert \mathbf o_\mathbf u - \mathbf o_\mathbf w \rVert + r_\mathbf u < r_\mathbf w$, and thus we can create an insideness loss by defining..."
5. Line 186: "loss of the embeddings" is not defined, perhaps what is meant is $\mathcal L_\text{inside}$ and $\mathcal L_\text{disjoint}$.

### Typos
Line 26: "works" -> "work"  
Line 70: "Analog" -> "Analogous"  
Line 186: "labels and if" -> "labels, if"  
Line 224: "follows" -> "follow"  
Line 238: "methods" -> "method"  
Line 313: "constraints. We" -> "constraints we"
limitations: The authors mention that a limitation of the work is that it currently only considers implication and exclusion, and they suggest logical equivalence ($l_a \wedge l_b \iff l_c$) as another potential useful constraint to include, however I think that more can be said on this point. The fact that the current model does not include such a constraint is not merely that it has yet to be implemented, there is a representational limitation in that the intersection of two balls is not, generally speaking, another ball, and therefore the existing model does not lend itself to easily encoding this constraint. On the other hand, models which use box regions can very easily incorporate such a constraint, as the intersection of two boxes is also a box.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes to model hierarchical structure as an embedding inference using Poincare balls. Hierarchical inclusion and exclusion are used to construct training losses in Poincare space and experiments show the proposed model generally outperforms box-based baselines across multiple datasets.
strengths_and_weaknesses: 1. The proposed Poincare hyperplane has some nice attributes on inclusion and exclusion. The construction of the training loss is intuitive and sense-making.

2. Experiments on hierarchical multilabel classification suggest promising results over box-based baselines.
questions: 1. The theoretical comparison with box representation needs elaboration. For instance, intersection of boxes can represent new concepts since it remains a box shape. But intersections of Poincare plane seems not have this nice attribute. While when modeling clean structures (i.e. membership and nonmembership) Poincare has good potential, but fuzzy hierarchies seems a lack of capability.

2. This paper is not citing the NeurIPS 17 on Poincare representation of hierarchy (Nickel and Kiela). What is new/old in this paper needs to be stressed.
limitations: The amount of novelty over the NeurIPS 17 paper is unclear. This paper has a substantial amount of analysis which is a bonus to have. But the technical crux is on the model.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This study explores a structured multi-label prediction problem. To this end, the authors propose to convert logical constraints into
soft geometric constraints in the hyperbolic embedding space, where the hyperplanes are viewed as convex areas, with insideness and disjointness of these regions representing logical linkages (implication and exclusion). Extensive tests on 12 multi-label classification problems demonstrate the model's capacity to boost performance.
strengths_and_weaknesses: Strengths: This study presents a novel translation that converts logical constraints into soft geometric constraints in the hyperbolic embedding space. Besides there is clear geometric intution where the implication is modeled by the geometric insideness while the mutual exclusion is modeled by the geometric disjointness.

Weakness: 
The experiment lacks some case studies that they can faithfully show that the obtained results well reflect the initial motivation.


 
questions: Is this method only applicable to the Poincaré disk model, and is it applicable to other hyperbolic models as well?

Is there another relationship in the data set and how to solve it?
limitations: As mentioned by the author, other logical constraints can exist in these datasets and the authors do not currently consider these relationships.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces a method for multi-label classification when the class labels have known dependency structures, namely implication and mutual exclusion. Hyperbolic geometry is employed to jointly learn parameters for an encoder that embeds points in a Poincare ball and Poincare hyperplanes for class labels. If the embedded point lies on one side ("inside") of the Poincare hyperplane, it is predicted to have that class label. Due to the curvature of the space, the Poincare hyperplanes can be contained completely inside one another, overlap somewhat, or non-overlapping. The method presented in this work presents a joint training objective to not only correctly classify the examples, but also encourage known constraints to be satisfied by the learned Poincare hyperplanes for each class: implication <-> complete containment and mutual exclusion <-> non-overlapping. The method is compared on standard benchmark datasets against reasonable baselines.
strengths_and_weaknesses: Strengths:
- Paper is overall well-written
- Motivation is clear
- Figures 1 & 2 make the method very clear and easy to understand!
- Experimental evaluation is adequate and convincing with several reasonable ablations

Weaknesses:
- It seems like the method is narrowly applicable. It only seems to beat the best baseline methods when constraints are available. How often are these constraints actually available in practice?
- As presented, only two types of logical constraints can be (softly) enforced by the objective. These constraints are arguably the most common types of constraints for these problems though.
- The claim that their method uses less dimensionality than baselines is somewhat misleading (See Appendix G of Patel et al.)
- The authors don't compare their method to the best performing baseline MBM without explicit constraint modeling.
questions: - Are there reasonable probabilistic semantics for Poincare hyperplanes representing probabilities of events? One appealing quality of box embeddings is their natural probabilistic interpretation and closure under intersection, etc. Does your approach have an analogous interpretation?
- Do the volumes of the "membership regions" correlate with the frequency of the respective class label?
- Is there a way to represent (un)certainty of the given constraints? Say from a not always 100% reliable taxonomy.
- If you train your model with only a fraction of the available constraints, does the resulting set of Poincare hyperplanes for each class recover any of the constraints that weren't used for training? Furthermore, are there any constraints that the resulting Poincare hyperplanes imply that are incorrect? 
- Are the constraints provided for training always satisfied by the learned Poincare hyperplanes?
- It seems like the method presented performs much worse when no constraints are provided as compared to MBM. Is there a reason why HMI performs so much worse than MBM when both models are not provided with any constraints? (Context: Table 4 from the submission and Table 1 from Patel et al.)
- Why is it that the method presented can get away with using substantially fewer dimensions? Is this due to the geometry of the Poincare ball? Also, I'm not sure that the dimensionality claim is valid or sound (See Appendix G of Patel et al. --> This shows similar results to Figure 4b in the submission).

limitations: The authors have adequately addressed the limitations of their work.
The potential negative societal impact of their work is not addressed, but their work has no more potential negative societal impact than any other paper submitted to NeurIPS.
review_title: null

Meta-review: metareview: The reviews of this paper are uniformly positive.  The novelty is the handling of exclusion edges which expands on previous work.  On the negative side the improvements seem small and do not solidly establish the value the value of the hyperbolic hyperplanes. But the reviewers liked the paper and I recommend acceptance.


confidence: Less certain
Decision: Accept