Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a version of hyperbolic GCN based Lorentz model without resorting to the tangent space. Some of the layer operations exist in prior work. Experiments are conducted on standard small-scale citation networks and tree datasets. 
strength_and_weaknesses: There are recently many efforts on removing the tangent space of hyperbolic neural networks. This paper used the Lorentz boost and rotation to operate directly on the Lorentz manifold. 

Strength. The presentation and logic of the paper are easy to follow. 
The ablation study is interesting and shows the learnt structure information from the model. 

Weakness.
1. Lack of novelty and contribution. The Lorentz boost and rotation is already well-characterized and studied in [1], particularly the proposed fully hyperbolic linear layer in [1] contains all Lorentz rotation and boost matrices. In fact, the graph convolution operation is essentially a hyperbolic linear layer. One can just take the linear layer from [1] and make a similar tangent space free Lorentz based hyperbolic GCN, which is pretty much done in the paper. 

2. Limited experiments and lack of reproducibility. Experiments are performed on 4 standard small scale datasets, which are 4 highly overfitted datasets. For example, the disease and airport, they are so small datasets. From my experience, I can always get a very high accuracy, but with a large variance during multiple runs. No code or supplementary is provided for the model. Plus I can get higher accuracies with some baselines, for example, using HGCN on pubmed node classification, $80.2\pm 0.3$ with 10 runs. It's really hard to derive meaningful conclusion from the experiments. 

------ 2.1 where is the training hyper-parameter? how many layers of the model, any regularization, hidden dimension? How many epochs? I am aware that many hyperbolic NNs report results of running thousand epochs with early stopping, while standard Euclidean GCNs, SGC report accuracy of 100 or 200 epochs. Is the comparison fair? 

3. Some claims are vague or inaccurate. For example, 

"most of the existing hyperbolic networks build the network operations on the tangent space of the manifold, which is a Euclidean local approximation. This distorts the learnt features, limits the representation capacity of the network"

I don't think this holds true, generally the exp/log map is local approximation of the manifold. However, in the hyperbolic space, they are bijection between the tangent space and the hyperbolic space. Usage of exp map at origin is not a problem with respect to representation capacity, as one can use parallel transport to move it along the manifold, which is already introduced and used in many prior works. 

------ 3.1 It claimed that "the full Lorentz transformation ... increase the model expressiveness ... with deeper networks." I failed to find evidence in the paper supporting this claim, in particular, personally I haven't seen any successful deep hyperbolic GCN style structure to work well even in existing/prior work. 

4. The word and text needs more polish. For example, 

"This makes it hard to optimize deeper networks because of these back and forth mappings between the manifold and the tangent space, and limits the representation capabilities of the hyperbolic networks caused by distortion specially most of these works used the tangent space at the origin. "

[1] Chen et al. 2022, Fully hyperbolic neural networks. 
clarity,_quality,_novelty_and_reproducibility: The paper/methodology is generally easy to follow and understand. Lack of originality, reproducibility not clear. 
summary_of_the_review: It's a straightforward usage of the fully hyperbolic NNs paper in GCN applications. Many pieces in the paper are missing and lack of enough novelty, see above.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper looks at graph hyperbolic neural networks to learn maps from $\mathbb{L}^n \to \mathbb{L}^d$ (where $\mathbb{L}^d$ is $d-1$ dimensional Lorentz manifold. In particular, given a graph $G$ we learn features for each node using such a map and then use this for node classification and link prediction.

The novelty in the paper is in the parameterization of the maps from $\mathbb{L}^n \to \mathbb{L}^d$ (where $\mathbb{L}^d$. In particular, they use the parameterization given by Moretti 2002. They then show that such networks outperform other networks on a variety of standard datasets. 

They also do an ablation study and show that their method learns near isometries. 
strength_and_weaknesses: **Strengths**
---

1) The method captures all Lorentz transformation. This is in contrast to Chen et al 2022 for which they show their parameterization captures all rotations and all boosts but not necessarily both together in one layer. However, this distinction may not be too important. 

2) The method seems to outperform other methods. But the important are small and may vanish with changing hyperparameters. 

**Weaknesses**
---

1) It is not clear to me how novel the method is. The decomposition into boosts and rotations is known (Moretti 2002). Further for the proofs in the texts, there are no statements, but statements being proved should be known (Moretii 2002, Chen et al 2022). 

2) There quite a few missing details including training details, statements of theorems among other things. 
clarity,_quality,_novelty_and_reproducibility: **Clarity**
---

I think the paper's clarity could be significantly improved. I have two main issues with clarity. 

1) Due to the writing it is not clear what is known and what is new. 
2) The lack of clarity makes it difficult to understand the method in details and some things seem incorrect. 

For the first, section 4.2 has two proofs for which the statements are missing. However, I think both of these proofs are unnecessary as they statements they are proving are known results. 

For the second, due to the missing details there are few things that seem incorrect. I detail them here. 

**The first is in Figure 1**. The left figure in seems to show a hyperbolic rotation in which the point P (in blue) is rotated to the point P (in red). However, this is not a rotation in hyperbolic space, it is actually a boost. Specifically, the figure shows 1 dimensional hyperbolic space, in which the only rotations are given by $P = \begin{bmatrix} 1 & 0 \\\\ 0 & \pm 1 \end{bmatrix}$ (as given by Equation (5) in the paper as well). In higher dimensions, again, the distance (euclidean) from the origin to the point should not change in a Lorentz rotation. Maybe this is what is being represented in the figure, but due to lack of details this is not clear. 

**Another detail that is missing is that the paper does not define what a Lorentz boost is.** This missing definition should be added in. Further the discussion on page 4 is not clear to me. Particularly the statement that the boost can be realized as rotations. 

**The next issue is in Figure 3** and equation (9) here the paper claims to be different from Chen et al 2022. However, this seems to be the same as Equation (4) (ArXiv v3 of Chen et al 2022). Note the cosmetic difference between the two due to the fact that this paper considers the curvature to $(-1/K)$ whereas Chen et al denote it by $K$. 

**The loss function is not clear**. For classification with more than two labels it is not clear what $\hat{d}$ is? Do we pick a class at random? Is it weighted average of the distance to other classes?

**An important missing detail is the training method** for their neural network. The paper does not mention any training details. The only place such a detail is present is when the paper says it enforces orthogonalization constraint. However, my question is how is this constraint enforced. Is the method projected gradient descent, Riemannian gradient descent, or some form of natural gradient that preserves the manifold.  

**The title is misleading**. The initial embedding step uses the Tangent space to get an initial hyperbolic embedding. If the authors want to be truly free using of the Tangent space, I would recommend using one of the hyperbolic embedding techniques (Nickel and Kiela NeurIPS 2017, Nickel and Kiela ICML 2018, Sala, De Sa, Gu and Re ICML 2018, Sonthalia and Gilbert NeurIPS 2020)


**Novelty**
---

As mentioned it is unclear to me how new the ideas in the paper are. In particular, the decomposition is known already and without the training details, the new parameterization doesnt seem that novel. Also the differences to Chen et al. are not quite as clear. 

**Reproducibility**
---

I think the paper could improve its reproducibility. In particular, by adding all of the hyper parameters and training details for the different networks. 


**Questions**
---

I have a few questions for the authors.

1) It is not clear to me what section 5.5 is trying to say. Specifically, why should our learned network be an isometry. This is further unclear to me since the loss function is the trying to minimize the distance between an embedding and the centroid for its class. This would suggest to be some sort of Neural Collapse would be beneficial. 

However, the authors claim that being a near isometry is what we want. If we want an isometry then why learn features at all? Couldn't we just use the graph?

summary_of_the_review: In summary, I think the idea of doing hyperbolic graph neural networks without using the tangent space is important since it increases the flexibility, and stability of the method as well as reduces computational costs. 

However, it is not clear to be (due to the presentation) the novelty of the paper, the correctness of some of the claims, and the reproducibility of the results. I think these issues are fixable, but very much do need to be fixed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper argues that methods based on tangent space have many problems, and therefore, avoid adopting this method. This paper proposes an approach to manifold-preserving Lorentz transformation.
strength_and_weaknesses: ## Weakness

1. Unconvincing conclusions.

(1) Indeed, $W\otimes_H X$ cannot guarantee that the result still lives in the Lorentz model after transformation. However, the problem has been solved by LGCN; that is, we only need to transform the space-like dimension (with the origin as a reference point) or just use the rotation matrix. Any of these methods can map it back to the Lorentz model. Also, the first approach can contain the rotation and boost operations.

Therefore, according to the transformation, the author cannot derive the conclusion—the method based on tangent space is not manifold-preserve. Similarly, we can utilize the same strategy to achieve the aggregation in the tangent space and successfully map it back to the Lorentz model.

(2) The author claimed that "This makes it hard to optimize deeper networks because of these back-and-forth mappings between the manifold and the tangent space," which could be incorrect since the deep network does not necessarily require frequent maps. For example, in image tasks, the network is deep [1][2]. Besides, it is not hard for us to optimize the HGNN. Please run the HGCN code directly using more than two layers.

[1] Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
[2] Hyperbolic Image Segmentation

2. Confusing experimental results.

- Could the author explain why the performance of the disease and Pubmed reported is different from the original work HGCN? Why not take their original results but present the results from LGCN? 
- The usage of dimension in Table 3 seems to cherry-pick. Why is the disease compared in 8 and 4 dimensions while the Cora is compared in 64?

clarity,_quality,_novelty_and_reproducibility: The paper is easy to read and follow. But the work is not easy to reproduce since the details of the experiments are not given, such as optimization method, number of layers, patience, and training strategies.
summary_of_the_review: This paper argues that methods based on tangent space have many problems, and therefore, avoid adopting this method. This paper proposes an approach to manifold-preserving Lorentz transformation. Although the paper is easy to follow, there is something that I am concerned about, including the claim, method, and experiments.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Similar to refs [A,B], the paper proposes a new version of Hyperbolic Graph Convolutional Network (HGCN). The main difference with refs [A,B] is that, instead of learning a linear map in the tangent space of the manifold, the paper proposes to learn linear operators directly in the extrinsic geometry of the Lorentzian ambient space of the hyperboloid. 
The learned linear operators use some boost and spatial rotation operations and directly map to the hyperboloid. 

[A] Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural
networks. NeurIPS 2019.

[B] Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic graph neural networks, NeurIPS 2019.
strength_and_weaknesses: - The main strength of the paper is the simplicity of the approach which is well-explained. I believe the proposed approach could be a useful "trick" to know in the community that works on hyperbolic representations.  

- The weaknesses of the paper are missing theoretical comparisons with refs [A,B]. In particular:

1) Can both the boost and spatial rotation operations also be expressed by existing approaches (when working in the tangent space)? If the answer is yes, does the proposed approach act as a regularizer by limiting the nature of possible operations?

2) What are the operations that the proposed approach can or cannot perform?

3) Related to the above questions, in the introduction, it is mentioned: "However, these works [A,B] performed the network operations in the tangent space of the manifold which is a Euclidean local approximation to the manifold at a point since the network operations such
as feature transformation and feature aggregation are not manifold-preserving. This makes it hard
to optimize deeper networks because of these back and forth mappings between the manifold and the tangent space, and limits the representation capabilities of the hyperbolic networks caused by distortion specially most of these works used the tangent space at the origin." The tangent space is supposed to preserve the metric of the manifold due to the Levi-Civita connection of the manifold. I then do not think that the issue is due to performing operations in the tangent space. What are the reasons of the worse performance of the baselines? Is it because of the approximations of the mappings from the manifold to the tangent space? Or because of operations that are performed or not performed in the learned linear mappings?
clarity,_quality,_novelty_and_reproducibility: The description of the approach is clear in general. What is missing is a clear discussion about the differences with the baselines.
summary_of_the_review: The proposed method is interesting and could be useful to the community. What is missing is a clear discussion about the differences with refs [A,B]. What is possible in refs [A,B] that is not possible in the proposed approach and vice versa? What are the reasons of the improved performance of the method? Either a theoretical or empirical justification would be useful to readers.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a "fully hyperbolic" GNN architecture (i.e., without resorting to the tangent space as done typically in manifold optimization). This seems to be the main novelty of the paper, which is incremental in light of prior work cited by the reviewers (Chen et al 2022 and Moretti 2002). Experiments are also rather limited and not particularly convincing. We recommend rejection. 
justification_for_why_not_higher_score: limited novelty
Decision: Reject