Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors suggest a simpler method: learn a hyperbolic embedding of the input, map it once to Euclidean space using a mapping that encodes geometric priors by respecting the isometries of hyperbolic space, and end with a standard Euclidean network.
strength_and_weaknesses: The idea is quite interesting, but the experimental results are strange, especially for the baselines in table 1 which are much lower than their real performance. 
For scalability, the author only experiments on some small datasets.
clarity,_quality,_novelty_and_reproducibility: The paper is clear and easy to read.
summary_of_the_review: In this paper, the authors propose a straightforward approach, learn a hyperbolic embedding of the input, then map it once to Euclidean space using a mapping that preserves the isometries of hyperbolic space while encoding geometric priors, and finally, finish with a standard Euclidean network. However, I am a little concerned about the experimental results.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a new method for utilizing hyperbolic geometry that avoids many of the issues with current models. 

My understanding of it is as follows. Suppose we have $x_1, \ldots, x_n$ embedded into some hyperbolic space and some kernel map $k(x_i,x_j)$ that **only** depends on the hyperbolic distance between the data, then we can define a random features type map (using the eigenfunctions of the Laplace Beltrami Operator on the manifold) to get features $\phi(x_i) \in \mathbb{R}^d$ such that $\mathbb{E}[\phi(x_i)^T \phi(x_j)] = k(x_i,x_j)$. 

The idea then is to use this features with traditional neural networks (instead of hyperbolic neural networks). The paper proves that any such kernel can be estimated in an unbiased banner and implements one such kernel to show improvements in many cases over both purely hyperbolic and purely euclidean methods. 
strength_and_weaknesses: **Strengths**
---

1) The Random Features Model is very important model for Euclidean data and the extension to the Hyperbolic version seems fundamental and interesting. 
2) The idea to then get Euclidean features that represent hyperbolic geometry (in some way) in this manner is very novel. 
3) The theory results are compelling and general to show that this can be used in a wide variety of settings. 

**Weaknesses**
---

1) Since the method for embedding the data into hyperbolic space is crucial to the method, the paper is missing a discussion on how to obtain this embedding. On page 6 it says ``A low dimensional hyperbolic embedding $z \in \mathcal{B}^{d_0}$ is initialized for each node to derive hyperbolic embeddings $Z \in \mathbb{R}^{n\times d_0}$ for all nodes in the graph''. However, this does not tell us how the embeddings was obtained. I think this is a crucial detail. Further, it is not clear, whether $Z$ is now a parameter is that learned during the neural network training. Based on the motivation of the paper to avoid hyperbolic gradient based learning, I do not think this is the case. 

Hence I think some discussion of how the method learns the embeddings and a discussion (such as Nickel and Kiela NeurIPS 2017, Nickel and Kiela ICML 2018, Sala, De Sa, Gu and Re ICML 2018, Sonthalia and Gilbert NeurIPS 2020) about embedding data into hyperbolic space would help. 
clarity,_quality,_novelty_and_reproducibility: **Clarity**
---

Besides the above mentioned weakness about the missing discussion on the how $Z$ is obtained. I think most of the paper is fairly clearly written. 

One thing that would help clarity is some more discussion on some of the mathematical aspects. For example the connections between isometry invariant kernels and densities is important to the paper (as we use this density to sample the eigenvalues for the eigenfunctions used), however, this discussion is terse. 

in particular, there seems to be an assumption that every such kernel has a such a density associated with it. If this is known a reference would be appreciated. 

**Quality**
---

I think the work is highly original and is of good quality. 

**Reproducibility**
---

Again baring the issue of how the data is embedded I think the method is fairly reproducible. 


**Questions**
---

1) The given pipeline can be used with *any data* and then *any network architecture* right? Is there a reason GNNs were chosen rather than DNNs? 

2) I am curious about the estimation part of the paper and the number of eigenfunction used. Is there some sort of concentration happening here where as we increase the number of eigenfunction used the random variable $\langle \phi(x), \phi(y) \rangle$ concentrates to the mean? If this is the case, would the authors know how quickly it concentrates?
summary_of_the_review: In summary I think the paper presents a fundamental extension of the random features model of hyperbolic space. It then presents a very interesting way in which this can be used. 

I think the paper is novel, and interesting.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper outline a method to use hyperbolic spaces in a graph neural network. The approach mainly relies on the computation of Laplacian Eigenfunctions: after embedding the graph nodes in some space, they are mapped via a kernel transformation to their hyperbolic features (i.e., Hyperbolic Laplacian eigenfunctions), and lastly, a standard euclidean network can be used. The method shows comparable results with the SoTA and a performance gain over previous hyperbolic networks.
strength_and_weaknesses: STRENGTH
=======
1) SIMPLICITY: While I do not fully understand some parts of the method, the overall principles seem straightforward. I expect that with some better details about the implementation, the method can be easily reproducible. Adopting Laplacian features as input of a network is a well-known approach to inject non-euclidean knowledge inside the network, but I am not aware of other works that compute such features in hyperbolic spaces.

2) EXPERIMENTS: The method is tested on different kinds of graphs from different families and structures. The results suggest that in the worst case, the method can be applied without harm, while in many cases, it improves.

WEAKNESSES
=======
1) PRESENTATION: as a non-expert in hyperbolic spaces, I had a hard time understanding exactly some parts of the method. For example, it is not clear to me how "a low dimensional hyperbolic embedding is initialized for each node". I think a plain explaination might help the reader and increase the usability of the paper methodology

2) APPLICABILITY AND LIMITATIONS: It is not completely clear to me if the method can have an impact in different applications from node classification since all the experiments are performed in this context. Exploiting isometric invariance has been widely popular in contexts like 3D meshes to compute correspondence between two objects. I wonder if the same holds here. Also, while the method indeed improves from euclidean networks, it comes with non-trivial computational costs (i.e., from Figure 5, SGC is 10x times faster). Finally, in Conclusions Section, it is mentioned that in future, a direction is to adopt "more numerically stable representations of the hyperbolic embeddings to avoid potential 'NaN problems'". I would like to know more about these issues and where they arise.
clarity,_quality,_novelty_and_reproducibility: I think the method description and the introduction to the main theoretical concepts can be better introduced, particularly for non-expert readers. I appreciate the quality of the work and the novelty, while I think some effort might be required to reproduce the results of the paper. I cannot find any statement about code release.
summary_of_the_review: I find the proposed methodology interesting and in line with recent research directions. As a non-expert, I would prefer better insights and explanations to fully appreciate the impact of this work, but I think the highlighted weaknesses can be addressed before the submission. Looking forward to the rebuttal to clarify my doubts.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes Hyperbolic Laplacian Features, a hyperbolic analogy of the classical Random Fourier Features of Rahimi (2007), and shows how they can be used to enhance learning on graphs (as a simple alternative to full hyperbolic GNNs). The method is interesting and efficient, and was appreciated by the reviewers. We recommend acceptance. 


justification_for_why_not_higher_score: insufficient contribution
justification_for_why_not_lower_score: cool idea
Decision: Accept: poster