Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: Methods for embedding hypergraph-based knowledge bases, where different edge types encode different relation types, have a long history in machine learning. Alternately, there has been a more recent interest in geometric embedding approaches for knowledge bases, which associate more complex geometric objects, such as hyperbolic cones, with knowledge base entities, due to the favorable geometric properties of hyperbolic space for modeling hierarchies. However, the latter approaches have usually been limited to representing only single relation types, due to representing the relation of interest with some geometric property of the embedding space such as enclosure between entity cones. This work attempts to bridge the gap between the two approaches by modeling relations as transformations between hyperbolic cones, specifically containment in different subspaces. 
main_review: UPDATE: The authors have addressed my concerns about interactions between the transitive and non-transitive parts of the model and I am raising my score accordingly. 


**Originality/Quality**

As the paper points out, there have been a couple efforts previously to produce multi-relational hyperbolic cone embeddings. However, they argue that approaches based on unrestricted Möbius matrix-vector product and addition (MuRP) cannot capture transitivity. In response, they introduce a “restricted rotation” operation that is specified when relations are known to be transitive, which scales the rotated cone to still lie within the parent cone. Transitive relations are modeled as restricted rotation in one subset of dimensions, while non-transitive ones are modeled as general rotations in another subset. 

It is unclear how this differs from training two different models and sets of embeddings, one which corresponds to the transitive relations and another which corresponds to the non-transitive relations. Is this correct? This makes the claim of using a “single” model a bit vacuous if these different relation types never interact. The results in experiments are good and the experiments are extensive, but the technical contribution seems fairly minimal.


**Clarity**

The paper is well written but could be better organized, it is hard to follow exactly how the proposed model differs from previous work, and the focus on one relation at a time in the exposition makes it often hard to initially understand how the proposed model differs from simply training a separate model for each relation, until the “subspace allocation” section of 3.3. 

**Significance**

Both knowledge graph embedding and nonstandard embedding geometries are popular areas of inquiry, and doing principled multi relational modeling in hyperbolic space is also interesting. The use of the restricted rotation is a good tool for practitioners to have in their arsenal, but the synthesis between the transitive and non-transitive relations is lacking.

limitations_and_societal_impact: Experiments seem adequate to address limitations, aside from the questions about interactions between transitive and non-transitive parts raised above.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a knowledge graph (KG) embedding method that can simultaneously model hierarchical (e.g., is-a) and non-hierarchical (e.g., friend-of) relations. The authors propose to embed entities into hyperbolic space. Inspired by Hyperbolic Entailment Cones (Ganea et al., 2018), they apply the cone containment constraint to relation-specific subspaces to deal with hierarchical relations. Inspired by RotatE (Sun et al., 2019) and RotH (Chami et al., 2020), they use hyperbolic cone rotations from the head entity to the tail entity to model non-hierarchical relations.

Experiments are conducted on two tasks: KG completion and hierarchical reasoning. Results show that the proposed method can outperform both Euclidean and hyperbolic KG embedding baselines. Results in low dimensions, ablation studies, and results on a biological dataset are further shown in the supplementary material.
main_review: The paper is well-written with solid and comprehensive evaluations, reflecting authors' massive effort. 

Although some ideas are directly borrowed from existing studies (e.g., Hyperbolic Entailment Cones and RotH), the general motivation of simultaneously modeling hierarchical and non-hierarchical relations is intuitive. Also, the idea of modeling heterogeneous hierarchical relations is well-motivated. Although hypernymy heterogeneity has been studied in taxonomy-related studies [1], using relation-specific subspaces to model it in the hyperbolic space is novel from my perspective.

Some questions and concerns:
- Statistical significance tests are missing in the experiments. For example, in Table 2, the gap between your methods and the best baseline is quite subtle when there are 0% inferred descendant pairs.

- Although the quantitative evaluation is comprehensive, it would be better to see some case studies and visualization results. For example, does your embedding result really preserve the cone containment constraint? For symmetric relations, does the rotation preserve the symmetry?

[1] Expanding Taxonomies with Implicit Edge Semantics. WWW 2020.
limitations_and_societal_impact: Please see the main review.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors propose ConE (Cone Embedding) to simultaneously model multiple hierarchical and non-hierarchical relations in a knowledge graph. ConE represents entities as hyperbolic cones and models relations as transformations between the cones. Experiments demonstrate that ConE outperforms many baselines on standard knowledge graph benchmarks. 
main_review: Originality: 
The idea of modeling multiple hierarchical and non-hierarchical relations simultaneously in a knowledge graph is interesting.

Quality: 
My concerns are as follows.
1. The authors may want to explain why ConE performs on par with RotH [1] on WN18RR and FB15k-237 (see Table 1).
2. Some related works are missing. The authors may want to compare ConE with more works related to hierarchy modeling [2,3].
3. The distance scoring function in Equation (14) may not make sense. The authors claim that the rotation transformation $f_1$ cannot explictly model hierarchical relations, while the authors compute $f_1$ in Equation (14) for the hierarchical relations in some cases, e.g. $m_1=1$ and $m_i=0,i=2,3,\dots,d$.

Clarity:
The authors may want to clarify the implementation of ConE. The restricted rotation transformation in Equation (12) and the angle loss in Equation (16) are only defined for the hyponym relation. However, the information about hyponym or hypernym relation is usually unavailable in knowledge graphs, e.g. FB15k-237. The authors may want to illustrate how to implement the restricted rotation transformation and the angle loss in practice.

Significance: 
The work aims to model hierarchical structures in knowledge graphs, which is an important topic.

[1] I. Chami, A. Wolf, D.-C. Juan, F. Sala, S. Ravi, and C. Ré, “Low-dimensional hyperbolic knowledge graph embeddings,” ACL, 2020. 

[2] Z. Zhang, J. Cai, Y. Zhang, and J. Wang, “Learning hierarchy-aware knowledge graph embeddings for link prediction,” in Proc. AAAI, 2020, pp. 3065–3072

[3] Z. Zhang, F. Zhuang, M. Qu, F. Lin, and Q. He, “Knowledge graph embedding with hierarchical relation structure,” in Proc. Conf. Empirical Methods Natural Lang. Process., 2018, pp. 3198–3207.

limitations_and_societal_impact: Yes. The authors addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This work proposes ConE, a knowledge graph embedding model which embeds entities into hyperbolic cones and represents relations as transformations between the cones. Non-hierarchical relations are modelled using rotations and hierarchical relations using restricted rotations, which impose the cone containment constraint. The proposed method is evaluated on the KG completion and ancestor-descendant prediction tasks.
main_review: Strengths
- The proposed model is a relatively novel approach for modelling both hierarchical and non-hierarchical relations in knowledge graphs, based on hyperbolic cones by Ganea et al., 2018.
- The ability to do ancestor-descendant prediction is an important improvement compared to standard KG completion models.
- The performance on KG completion is comparable to existing state-of-the-art models.

Weaknesses
- The proposed method requires a priori knowledge of whether a relation is hierarchical or not, and not just that, but also knowing the direction of hierarchy (i.e. head->tail or tail->head). None of the previous methods (e.g. MuRP and RotH) require this knowledge, which gives ConE unfair advantage over those. 
- Despite having access to extra information, the proposed model does not outperform RotH/AttH on FB15k-237 and WN18RR, which are standard KG completion datasets.


Detailed questions/comments:
1. In the abstract, you claim that "curent KG embeddings...fail to model multiple heterogeneous hierarchies that exist in a single KG.". However, this is factually incorrect, since there has been several works for modelling hiearchical relations in knowledge graphs, such as MuRP and RotH/AttH.
2. Again, same applies to the introduction, i.e. "These methods can only be applied to graphs with a single hierarchical relation type, and are thus not suitable to real-world knowledge graphs with multiple hierarchical relations." - both MuRP and RotH/AttH have been specifically designed to model multiple hierarchies. Later in that some paragraph, you contradict yourself by mentioning MuRP and RotH.
3. Have you tried learning the space curvature? Chami et al. show that fixing the curvature has a negative impact on performance.
4. Your method assumes a priori knowledge on whether a relation is hierarchical or not, but in most real world KGs, this information isn't available. What would happen for KGs where the number of relations is really large and checking whether a relation is hierarchical or not is infeasible?
5. Do you have any intuition on why the multi-relational RotH performs much worse than the single-relational Poincaré embedding (Nickel & Kiela, 2017) on the ancestor-descendant prediction task ?
limitations_and_societal_impact: Yes.
review_title: null

Meta-review: This paper proposes a knowledge graph embedding method called ConE, which can simultaneously model both hierarchical and non-hierarchical relations by embedding entities into hyperbolic cones with relations as transformations between cones. The reviewers agree that this is a strong paper, and the authors did an excellent job in their rebuttal in addressing any remaining concerns.
Decision: Accept (Poster)