Title: Hyperbolic Graph Neural Networks at Scale: A Meta Learning Approach
Venue: NeurIPS 2023
Accepted: yes
Keywords: meta learning

hyperbolic networks

scalability

graph neural networks
Abstract: The progress in hyperbolic neural networks (HNNs) research is hindered by their absence of inductive bias mechanisms, which are essential for generalizing to new tasks and facilitating scalable learning over large datasets. In this paper, we aim to alleviate these issues by learning generalizable inductive biases from the nodes’ local subgraph and transfer them for faster learning over new subgraphs with a disjoint set of nodes, edges, and labels in a few-shot setting. We introduce a novel method, Hyperbolic GRAph Meta Learner (H-GRAM), that, for the tasks of node classification and link prediction, learns transferable information from a set of support local subgraphs in the form of hyperbolic meta gradients and label hyperbolic protonets to enable faster learning over a query set of new tasks dealing with disjoint subgraphs. Furthermore, we show that an extension of our meta-learning framework also mitigates the scalability challenges seen in HNNs faced by existing approaches. Our comparative analysis shows that H-GRAM effectively learns and transfers information in multiple challenging few-shot settings compared to other state-of-the-art baselines. Additionally, we demonstrate that, unlike standard HNNs, our approach is able to scale over large graph datasets and improve performance over its Euclidean counterparts.

Hyperbolic Graph Neural Networks at Scale: A Meta Learning Approach

Abstract

The progress in hyperbolic neural networks (HNNs) research is hindered by their absence of inductive bias mechanisms, which are essential for generalizing to new tasks and facilitating scalable learning over large datasets. In this paper, we aim to alleviate these issues by learning generalizable inductive biases from the nodes' local subgraph and transfer them for faster learning over new subgraphs with a disjoint set of nodes, edges, and labels in a few-shot setting. We introduce a novel method, Hyperbolic GRAph Meta Learner (H-GRAM), that, for the tasks of node classification and link prediction, learns transferable information from a set of support local subgraphs in the form of hyperbolic meta gradients and label hyperbolic protonets to enable faster learning over a query set of new tasks dealing with disjoint subgraphs. Furthermore, we show that an extension of our meta-learning framework also mitigates the scalability challenges seen in HNNs faced by existing approaches. Our comparative analysis shows that H-GRAM effectively learns and transfers information in multiple challenging few-shot settings compared to other state-of-the-art baselines. Additionally, we demonstrate that, unlike standard HNNs, our approach is able to scale over large graph datasets and improve performance over its Euclidean counterparts.

1 Introduction

Graphs are extensively used in various applications, including image processing, natural language processing, chemistry, and bioinformatics. With modern graph datasets ranging from hundreds of thousands to billions of nodes, research in the graph domain has shifted towards larger and more complex graphs, e.g., the nodes and edges in the traditional Cora dataset [29] are in the order of 10 3 , whereas, the more recent OGBN datasets [17] are in the order of 10 6 . However, despite their potential, Hyperbolic Neural Networks (HNNs), which capture the hierarchy of graphs in hyperbolic space, have struggled to scale up with the increasing size and complexity of modern datasets.

HNNshave outperformed their Euclidean counterparts by taking advantage of the inherent hierarchy present in many modern datasets [4, 10, 13]. Unlike a standard Graph Neural Network (GNN), a HNNlearns node representations based on an 'anchor' or a 'root' node for the entire graph, and the operations needed to learn these embeddings are a function of this root node. Specifically, HNN formulations [13] depend on the global origin (root node) for several transformation operations (such as M¨ obius addition, M¨ obius multiplication, and others), and hence focusing on subgraph structures to learn representations becomes meaningless when their relationship to the root node is not considered. Thus, state-of-the-art HNNs such as HGCN [4], HAT [15], and HypE [8] require access to the entire dataset to learn representations, and hence only scale to experimental datasets (with ≈ 10 3 nodes). Despite this major drawback, HNNs have shown impressive performance on several research domains including recommendation systems [32], e-commerce [9], natural language processing [11], and knowledge graphs [5, 8]. It is thus imperative that one develops methods to scale HNNs to larger datasets, so as to realize their full potential. To this end, we introduce a novel method, Hyperbolic GRAph Meta Learner (H-GRAM) , that utilizes meta-learning to learn infor-

Genericgraphwithlabeledsamples(red

Metalearningstructurepriorsonthe

Utilizethemetainformationforfaster

andblue) and unlabeled samples (silver).

：labeledlocalpartitions of the graph.

learningovernewnodes,edgesand

labels(yellow inthis case).

Meta-Training

Meta-Testing

SharedLabels

SharedLabels

mation from local subgraphs for HNNs and transfer it for faster learning on a disjoint set of nodes, edges, and labels contained in the larger graph. As a consequence of meta-learning, H-GRAM also achieves several desirable benefits that extend HNNs' applicability including the ability to transfer information on new graphs (inductive learning), elimination of over-smoothing, and few-shot learning. We experimentally show that H-GRAM can scale to graphs of size O (10 6 ) , which is O (10 3 ) times larger than previous state-of-the-art HNNs. To the best of our knowledge, there are no other methods that can scale HNNs to datasets of this size.

Recent research has shown that both node-level and edge-level tasks only depend on the local neighborhood for evidence of prediction [18, 19, 42]. Inspired by the insights from such research, our model handles large graph datasets using their node-centric subgraph partitions, where each subgraph consists of a root node and the k -hop neighborhood around it. In H-GRAM, the HNN formulations establish the root node as the local origin to encode the subgraph. We theoretically show that, due to the locality of tangent space transformations in HNNs (more details in Section 3), the evidence for a node's prediction can predominantly be found in the immediate neighborhood. Thus, the subgraph encodings do not lose a significant amount of feature information despite not having access to a 'global' root node. However, due to the node-centric graph partitioning, the subgraphs are non-exhaustive, i.e., they do not contain all the nodes, edges, and labels, as previously assumed by HNNs. Thus, to overcome the issue of non-exhaustive subgraphs, we formulate four meta-learning problems (illustrated in Figure 1) that learn inductive biases on a support set and transfer it for faster learning on a query set with disjoint nodes, edges, or labels. Our model learns inductive biases in the meta-training phase, which contains two steps - HNN update and meta update. HNN updates are regular stochastic gradient descent steps based on the loss function for each support task. The updated HNN parameters are used to calculate the loss on query tasks and the gradients are accumulated into a meta-gradient for the meta update. In the meta-testing phase, the models are evaluated on query tasks with parameters post the meta updates, as this snapshot of the model is the most adaptable for faster learning in a few-shot setting. Our main contributions are as follows:

We theoretically prove that HNNs rely on the nodes' local neighborhood for evidence in prediction, as well as, formulate HNNs to encode node-centric local subgraphs with root nodes as the local origin using the locality of tangent space transformations.

We develop Hyperbolic GRAph Meta Learner (H-GRAM), a novel method that learns meta information (as meta gradients and label protonets) from local subgraphs and generalize it to new graphs with a disjoint set of nodes, edges, and labels. Our experiments show that H-GRAM can be used to generalize information from subgraph partitions, thus, enabling scalability.

Our analysis on a diverse set of datasets demonstrates that our meta-learning setup also solves several challenges in HNNs including inductive learning, elimination of over-smoothing, and few-shot learning in several challenging scenarios.

2 Related Work

This section reviews the relevant work in the areas of hyperbolic neural networks and meta learning.

Hyperbolic Neural Networks: Due to their ability to efficiently encode tree-like structures, hyperbolic space has been a significant development in the modeling of hierarchical datasets [43, 44]. Among its different isometric interpretations, the Poincar´ e ball model is the most popular one and has been applied in several HNN formulations of Euclidean networks including the recurrent (HGRU [13]), convolution (HGCN [4]), and attention layers (HAT [15]). As a result of their performance gains on hierarchical graphs, the formulations have also been extended to applications in knowledge graphs for efficiently encoding the hierarchical relations in different tasks such as representation learning (MuRP [1], AttH [5]) and logical reasoning (HypE [8]). However, the above approaches have been designed for experimental datasets with a relatively small number of nodes (in the order of 10 3 ), and do not scale to real-world datasets. Hence, we have designed H-GRAM as a meta-learning algorithm to translate the performance gains of HNNs to large datasets in a scalable manner.

Graph Meta-learning: Few-shot meta-learning transfers knowledge from prior tasks for faster learning over new tasks with few labels. Due to their wide applicability, they have been adopted in several domains including computer vision [14, 34], natural language processing [22] and, more recently, graphs [18, 39]. One early approach in graph meta-learning is Gated Propagation Networks [23] which learns to propagate information between label prototypes to improve the information available while learning new related labels. Subsequent developments such as MetaR [7], Meta-NA [41] and G-Matching [39] relied on metric-based meta learning algorithms for relational graphs, network alignment and generic graphs, respectively. These approaches show impressive performance on few-shot learning, but are only defined for single graphs. G-Meta [18] extends the metric-based techniques to handle multiple graphs with disjoint labels. However, the method processes information from local subgraphs in a Euclidean GNN, and thus, is not as capable as hyperbolic networks in encoding tree-like structures. Thus, we model H-GRAM to encode hierarchical information from local subgraphs and transfer it to new subgraphs with disjoint nodes, edges, and labels.

3 The Proposed H-GRAM Model

In this section, we define the problem setup for different meta-learning scenarios and describe our proposed model, H-GRAM, illustrated in Figure 2. For better clarity, we explain the problem setup for node classification and use HGCN as the exemplar HNN model. However, the provided setup can be extended to link prediction or to other HNN models, which we have evaluated in our experiments. The preliminaries of hyperbolic operations and meta-learning are provided in Appendix A.

3.1 Problem Setup

Our problem consists of a group of tasks T i ∈ T which are divided into a support set T s and query set T q , where T s ∩ T q = φ . Furthermore, each task T i is a batch of node-centric subgraphs S u with a corresponding label Y u (class of root node in node classification or root link in link prediction). The subgraphs S u could be the partitions derived from a single graph or multiple graphs, both denoted by G ∪ = G s ∪ G q . We also define Y s = { Y u ∈ T s } and Y q = { Y u ∈ T q } as the set of labels in the support and query set respectively. The primary goal of meta-learning is to learn a predictor using the support set, P θ ∗ ( Y s |T s ) such that the model can quickly learn a predictor P θ ( Y s |T s ) on the query set. Following literature in this area [18], the problem categories are defined as follows:

Meta-TrainingPhase

Subgraph

HNNUpdate

Encoding

T∈Dtrain

HNN

Model

train

P0*

Label

ProtoNets

Subgraph

train

Encoding

MetaUpdate

Meta-TestingPhase

HNNUpdate

Subgraph

Subgraph

T∈ Dtest

Encoding

Partitions

HNN

Model

Pe

Label

T∈Dtest

ProtoNets

Subgraph

test

Encoding

/negationslash

Single graph, disjoint labels 〈 SG,DL 〉 : This problem operates on the same graph in the support and query set, however unlike the previous one, the label sets are disjoint. The goal is to learn the model P θ ∗ → θ , where |G s | = |G q | = 1 and Y s ∩ Y q = φ .

Single graph, shared labels 〈 SG,SL 〉 : The objective is to learn the meta-learning model P θ ∗ → θ , where Y s = Y q and |G s | = |G q | = 1 . It should be noted that the tasks are based on subgraphs, so |G s | = |G q | = 1 = ⇒ |T s | = | T q | = 1 . Also, this problem setup is identical to the standard node classification task considering T q i ∈ D test to be the test set.

Multiple graphs, shared labels 〈 MG,SL 〉 : This problem setup varies in terms of the dataset it handles, i.e., the dataset can contain multiple graphs instead of a single one. However, our method focuses on tasks which contain node-centric subgraphs, and hence, the model's aim is the same as problem 1. The aim is to learn the predictor model P θ ∗ → θ , where Y s = Y q and |G s | , |G q | > 1 .

Multiple graphs, disjoint labels 〈 MG,DL 〉 : In this problem, the setup is similar to the previous one, but only with disjoint labels instead of shared ones, i.e., learn a predictor model P θ ∗ → θ , where Y s ∩ Y q = φ and |G s | , |G q | > 1 .

From the problem setups, we observe that, while they handle different dataset variants, the base HNN model operates on the ( S u , Y u ) pair. So, we utilize a hyperbolic subgraph encoder and prototypical labels to encode S u and get a continuous version of Y u for our meta-learning algorithm, respectively.

3.2 Hyperbolic Subgraph Encoder

In previous methods [16, 18, 38], authors have shown that nodes' local neighborhood provides some informative signals for the prediction task. While the theory is not trivially extensible, we use the local tangent space of Poincar´ e ball model to prove that the local neighborhood policy holds better for HNN models. The reason being that, while message propagation is linear in Euclidean GNNs [42], it is exponential in HNNs. Hence, a node's influence, as given by Definition 1, outside its neighborhood decreases exponentially.

The influence of a hyperbolic node vector x H on node x H is defined by the influence

Definition 1. v u (∥ ∥ ∥ ∥ ) .

Definition 2. The influence of a graph G with set of vertices V on a node u ∈ V is defined as I G ( u ) = exp c 0 ( ( ∑ v ∈ V log c 0 ( I uv ) ) .

Theorem 1. For a set of paths P uv between nodes u and v , let us define D p i gµ as the geometric mean of nodes' degree in a path p i ∈ P uv , p uv as the shortest path, and I uv as the influence of node v on u . Also, let us say D min gµ = min { D p i gµ ∀ p i ∈ P uv } , then the relation I uv ≤ exp c u ( K/ ( D min gµ ) ‖ p uv ‖ ) (where K is a constant) holds for message propagation in HGCN models.

Theorem 1 shows that the influence of a node decreases exponent-of-exponentially ( exp c u = O ( e n ) ) with increasing distance ‖ p uv ‖ . Thus, we conclude that encoding the local neighborhood of a node is sufficient to encode its features for label prediction.

Definition 3. The information loss between encoding an entire graph G and a subgraph S u with root node u is defined as δ H ( G , S u ) = exp c 0 ( log c 0 ( I G ( u )) -log c 0 ( I S u ( u ))) .

Theorem 2 shows that encoding the local subgraph is a e ‖ p uv ‖ order approximation of encoding the entire graph, and thus, with high enough ‖ p uv ‖ , the encodings are equivalent. Note that ‖ p uv ‖ is equivalent to the subgraph's neighborhood size ( k in k -hop). This shows that encoding a node's local neighborhood is sufficient to encode its features. Theorem proofs are provided in Appendix B. The above theorems provide us with the theoretical justification for encoding local subgraphs into our meta-learning framework. Hence, we partition the input G ∪ into subgraphs S u = V × E centered at root node u , with | V | nodes, | E | edges, a neighborhood size k , and the corresponding label Y u . The subgraphs are processed through an k -layer HGCN network and the Einstein midpoint [33] of all the node encodings are taken as the overall encoding of the subgraph S u . For a given subgraph S u = ( A,X ) , where A ∈ H ‖ V ‖×‖ V ‖ is the local adjacency matrix and X ∈ H ‖ V ‖× m are the node feature vectors of m dimension, the encoding procedure given can be formalized as:

Theorem 2. For a subgraph S u of graph G centered at node u , let us define a node v ∈ G with maximum influence on u , i.e., v = arg max t ( {I u t, t ∈ N ( u ) \ u } ) . For a set of paths P uv between nodes u and v , let us define D p i gµ as the geometric mean of degree of nodes in a path p i ∈ P uv , ‖ p uv ‖ is the shortest path length, and D min gµ = min { D p i gµ ∀ p i ∈ P uv } . Then, the information loss is bounded by δ H ( G , S u ) ≤ exp c u ( K/ ( D min gµ ) ‖ p uv ‖ +1 ) (where K is a constant).

where HGCN ( A,X ) ∈ H ‖ V ‖× d is the output of k-layer HGCN with d output units and e u ∈ H d is the final subgraph encoding of S u . γ iu is the Lorentz factor of the hyperbolic vector h iu that indicates its weightage towards the Einstein midpoint.

3.3 Label Prototypes

Label information is generally categorical in node classification tasks. However, this does not allow us to pass inductive biases from the support set to the query set. Hence, to circumvent this issue, we use prototypical networks [31] as our label encoding. Our approach constructs continuous label prototypes by using the mean of meta-training nodes' features that belong to the label. These prototypes are then employed to classify meta-testing samples based on their similarity to the corresponding meta-training label prototypes. This enables our model to handle new, non-exhaustive labels in an inductive manner, without the need for additional training data. The primary idea is to form continuous label prototypes using the mean of nodes that belong to the label. To this end, the continuous label prototype of a label y k is defined as c k = ∑ Yu = y k γ i e i ∑ Yu = y k γ i , where γ i = 1 √ 1 -‖ e i ‖ 2 ,and e i ∈ H d is encoding of subgraphs S u with labels Y u = y k . For each S u with class y k , we compute the class distribution vector as p k = e ( -d c H ( eu,c k )) ∑ k e ( -d c H ( eu,c k )) , where d c H ( e u , c k ) = 2 √ c tanh -1 ( √ c ‖ -e u ⊕ c k ‖ ) and the loss for HNN updates L ( p, y ) = ∑ j y i log p j , where y i is the one-hot encoding of the ground truth. The class distribution vector p k is a softmax over the hyperbolic distance of subgraph encoding to the label prototypes, which indicates the probability that the subgraph belongs to the class y k . The loss function L ( p, y ) is the cross-entropy loss between ground truth labels y and the class distribution vector p .

3.4 Meta-Learning

In the previous section, we learned a continuous label encoding that is able to capture inductive biases from the subgraph. In this section, we utilize the optimization-based MAML algorithm [12] to transfer the inductive biases from the support set to the query set. To this end, we sample a batch of tasks, where each task T i = { S i , Y i } ‖ T i ‖ i =1 . In the meta-training phase, we first optimize the HNN parameters using the Riemannian stochastic gradient descent (RSGD) [2] on support loss, i.e., for each T s i ∈ T s train : θ ∗ j ← exp c θ ∗ j ( -α ∇ L s ) , where α is the learning rate of RSGD. Using the updated parameters θ ∗ j , we record the evaluation results on the query set, i.e., loss on task T q i ∈ T q train is L q i . The above procedure is repeated η times post which L q i over the batch of tasks T q i ∈ T q train is accumulated for the meta update θ ∗ ← exp c θ ∗ ( -β ∇ ∑ i L q i ) . The above steps are repeated with the updated θ ∗ and a new batch of tasks till convergence. The final updated parameter set θ ∗ → θ is transferred to the meta-testing phase. In meta-testing, the tasks T s i ∈ T s test are used for RSGD parameter updates, i.e., T s i ∈ T s test : θ j ← exp c θ j ( -α ∇ L s ) until convergence. The updated parameters θ are used for the final evaluation on T q i ∈ T q test . Our meta-learning procedure is further detailed in Appendix C and the implementation code with our experiments is available at https://github.com/Akirato/HGRAM . Details on implementation and broader impacts are provided in Appendices 3.5 and E, respectively.

3.5 Implementation Details

H-GRAMis primarily implemented in Pytorch [26], with geoopt [21] and GraphZoo [35] as support libraries for hyperbolic formulations. Our experiments are conducted on a Nvidia V100 GPU with 16 GB of VRAM. For gradient descent, we employ Riemannian Adam [28] with an initial learning rate of 0.01 and standard β values of 0.9 and 0.999. The other hyper-parameters were selected based on the best performance on the validation set ( D val ) under the given computational constraints. In our experiments, we empirically set k = 2 , d = 32 , h = 4 , and η = 10 . We explore the following search space and tune our hyper-parameters for best performance. The number of tasks in each batch are varied among 4, 8, 16, 32, and 64. The learning rate explored for both HNN updates and meta updates are 10 -2 , 5 × 10 -3 , 10 -3 and 5 × 10 -4 . The size of hidden dimensions are selected from among 64, 128, and 256. The final best-performing hyper-parameter setup for real-world datasets is presented in Table 5.

4 Experimental Setup

Our experiments aim to evaluate the performance of the proposed H-GRAM model and investigate the following research questions:

RQ1: Does our hyperbolic meta-learning algorithm outperform the Euclidean baselines on various meta-learning problems?

RQ2: How does our model perform and scale in comparison to other HNN formulations in standard graph problems?

RQ3: How does H-GRAM model's performance vary with different few-shot settings, i.e., different values of k and N ?

RQ4: What is the importance of different meta information components?

We use a set of standard benchmark datasets and baseline methods to compare the performance of H-GRAM on meta-learning and graph analysis tasks. The HNN models do not scale to the large datasets used in the meta-learning task, and hence, we limit our tests to Euclidean baselines. To compare against HNN models, we rely on standard node classification and link prediction on small datasets. Also, we do not consider other learning paradigms, such as self-supervised learning because they require an exhaustive set of nodes and labels and do not handle disjoint problem settings.

4.1 Datasets

For the task of meta-learning, we utilize the experimental setup from earlier approaches [18]; two synthetic datasets to understand if H-GRAM is able to capture local graph information and five real-world datasets to evaluate our model's performance in a few-shot setting.

Synthetic Cycle [18] contains multiple graphs with cycle as the basis with different topologies (House, Star, Diamond, and Fan) attached to the nodes on the cycle. The classes of the node are defined by their topology.

Synthetic BA [18] uses Barab´ asi-Albert (BA) graph as the basis with different topologies planted within it. The nodes are labeled using spectral clustering over the Graphlet Distribution Vector [27] of each node.

ogbn-arxiv [17] is a large citation graph of papers, where titles are the node features and subject areas are the labels.

Tissue-PPI [16, 43] contains multiple protein-protein interaction (PPI) networks collected from different tissues, gene signatures and ontology functions as features and labels, respectively.

FirstMM-DB [25] is a standard 3D point cloud link prediction dataset.

Fold-PPI [43] is a set of tissue PPI networks, where the node features and labels are the conjoint triad protein descriptor [30] and protein structures, respectively.

Tree-of-Life [44] is a large collection of PPI networks, originating from different species.

4.2 Baseline Methods

For comparison with HNNs, we utilize the standard benchmark citation graphs of Cora [29], Pubmed [24], and Citeseer [29]. For the baselines, we select the following methods to understand H-GRAM's performance compared to state-of-the-art models in the tasks of meta-learning and standard graph processing.

Meta-Graph [3], developed for few-shot link prediction over multiple graphs, utilizes VGAE [20] model with additional graph encoding signals.

Meta-GNN [40] is a MAML developed over simple graph convolution (SGC) network [36].

FS-GIN [37] runs Graph Isomorphism Network (GIN) on the entire graph and then uses the fewshot labelled nodes to propagate loss and learn.

FS-SGC [36] is the same as FS-GIN but uses SGC instead of GIN as the GNN network.

ProtoNet [31] learn a metric space over label prototypes to generalize over unseen classes.

MAML [12] is a Model-Agnostic Meta Learning (MAML) method that learns on multiple tasks to adapt the gradients faster on unseen tasks.

HMLP [13], HGCN [4], and HAT [15] are the hyperbolic variants of Euclidean multi-layer perceptron (MLP), Graph Convolution Network (GCN), and Attention (AT) networks that use hyperbolic gyrovector operations instead of the vector space model.

It should be noted that not all the baselines can be applied to both node classification and link prediction. Hence, we compare our model against the baselines only on the applicable scenarios.

5 Experimental Results

We adopt the following standard problem setting which is widely studied in the literature [18]. The details of the datasets used in our experiments are provided in Table 1. In the case of synthetic datasets, we use a 2-way setup for disjoint label problems, and for the shared label problems the cycle graph and Barab´ asi-Albert (BA) graph have 17 and 10 labels, respectively. The evaluation of our model uses 5 and 10 gradient update steps in meta-training and meta-testing, respectively. In the case of real-world datasets, we use 3-shot and 16-shot setup for node classification and link prediction, respectively. For real-world disjoint labels problem, we use the 3-way classification setting. The evaluation of our model uses 20 and 10 gradient update steps in meta-training and metatesting, respectively. In the case of Tissue-PPI dataset, we perform each 2-way protein function task three times and average it over 10 iterations for the final result. In the case of link prediction task, we need to ensure the distinct nature of support and query set in all meta-training tasks. For this, we hold out a fixed set comprised of 30% and 70% of the edges as a preprocessing step for every graph for the support and query set, respectively.

5.1 RQ1: Performance of Meta-Learning

To analyze the meta-learning capability of H-GRAM, we compare it against previous approaches in this area on a standard evaluation setup. We consider two experimental setups inline with previous evaluation in the literature [18]; (i) with synthetic datasets to analyze performance on different problem setups without altering the graph topology, and (ii) with real-world datasets to analyze per-

formance for practical application. Based on the problem setup, the datasets are partitioned into node-centric subgraphs with corresonding root node's label as the subgraph's ground truth label. The subgraphs are subsequently batched into tasks which are further divided into support set and query set for meta-learning. The evaluation metric for both the tasks of node classification and link prediction is accuracy A = | Y = ˆ Y | / | Y | . For robust comparison, the metrics are computed over five folds of validation splits in a 2-shot setting for node classification and 32-shot setting for link prediction. Table 2 presents the five-fold average and 95% confidence interval of our experiments on synthetic and real-world datasets, respectively. From the results, we observe that H-GRAM consistently outperforms the baseline methods on a diverse set of datasets and meta-learning problem setups. For the disjoint labels setting, H-GRAM outperforms the best baseline in both the cases of single and multiple graphs. In the case of synthetic graphs, we observe that subgraph methods of H-GRAM and G-Meta outperform the entire graph encoding based approaches showing that subgraph methods are able to limit the over-smoothing problem [6] and improve performance. Also, we observe that meta-learning methods (ProtoNet and MAML) are unreliable in their results producing good results for some tasks and worse for others, whereas H-GRAM is consistently better across the board. Hence, we conclude that using label prototypes to learn inductive biases and transferring them using MAML meta updates is a more robust technique. We note that H-GRAM, unlike previous HNN models, is able to handle graphs with edges and nodes in the order of millions, as evident by the performance on large real-world datasets including ogbn-arxiv, Tissue-PPI, Fold-PPI, and Tree-of-Life. Our experiments clearly demonstrate the significant performance of H-GRAM in a wide-range of applications and prove the effectiveness of meta-learning in HNN models.

±

±

5.2 RQ2: Comparison with HNN models

The current HNN formulations of HMLP, HAT, and HGCN do not scale to large datasets, and hence, we were not able to compare them against H-GRAM on largescale datasets. However, it is necessary to compare the standard HNN formulations with H-GRAM to understand the importance of subgraph encoders and meta-learning.

100

06

Epochs

80

TimeTakenforTrainingE

70

60

50

40

30

20

10

0

0

1

2

3

4

5

6

7

Logof NumberofVertices(log(llvll))

HMLP

—HGCN→—HAT-←-H-GRAM--H-GRAM(multi)

whereas, H-GRAM is able to accommodate large graphs. Another point to note is that HGRAM(multi) is able to parallelize well over multiple GPUs with its time taken showing stability after 10 4 nodes (which is the size of nodes that a single GPU can accommodate).

Thus, we utilize the single graph and shared labels setup of H-GRAM to evaluate its performance on citation networks of Cora, Pubmed, and Citeseer for both tasks of node classification and link prediction. We also compare the time taken by our model and other HNNs on varying number of nodes ( |V| = { 10 i } 7 i =1 ) in the Synthetic BA graph. For this experiment, we also consider a multiGPU version of H-GRAM that parallelizes the HNN update computations and accumulates them for meta update. From our results, presented in Table 3, we observe that H-GRAM is, on average, able to outperform the best baseline. This shows that our formulation of HNN models using meta-learning over node-centric subgraphs is more effective than the traditional models, while also being scalable over large datasets. The scalability allows for multi-GPU training and translates the performance gains of HNNs to larger datasets. In the results provided in Figure 3, we observe that the time taken by the models is inline with their parameter complexity (HMLP ≤ HGCN ≤ HAT ≤ H-GRAM). However, the traditional HNNs are not able to scale beyond |V| = 10 4 ,

5.3 RQ3: Challenging Few-shot Settings

To understand the effect of different few-shot learning scenarios, we vary the number of few-shots M and hops K in the neighborhood. For the experiment on few-shot classification, we consider the problems of node classification on Fold-PPI dataset and link prediction on FirstMM-DB dataset. We vary M = 1 , 2 , 3 and M = 16 , 32 , 64 for node classification and link prediction, respectively, and calculate the corresponding accuracy and 95% confidence interval of H-GRAM. The results for this experiment are presented in Figures 4a and 4b for node classification and link prediction, respectively. To determine the effect of hops in the neighborhood, we vary K = 1 , 2 , 3 for the same problem setting and compute the corresponding performance of our model. The results for varying neighborhood sizes are reported in Figure 4c. In the results on varying the number of few-shots, we observe a linear trend in both the tasks of node classification and link prediction, i.e., a linear increase in H-GRAM's accuracy with an increase in the number of shots in meta-testing. Thus, we conclude that H-GRAM, like other generic learning models, performs better with an increasing number of training samples. In the experiment on increasing the neighborhood size, we observe that in the task of node classification, K = 3 shows the best performance, but in link prediction K = 2

±

±

±

±

has the best performance, with a significant drop in K = 3 . Thus, for stability, we choose K = 2 in our experiments. The trend in link prediction also shows that larger neighborhoods can lead to an increase in noise, which can negatively affect performance.

Accuracy on Node Classfication

0.70

0.82

AccuracyonLinkPrediction

0.9

0.65

0.81

0.8

0.60

0.80

0.7

0.55

0.79

0.6

0.50

0.45

0.78

0.5

0.4

0.40

0.77

1

2

3

1

3

10

16

32

64

Number of Neighborhood Hops

Numberof shotsinMeta-testing

NumberofshotsinMeta-testing

Node (Fold-PPI)

→Link(FirstMM-DB)

Figure 4: Performance of H-GRAM on challenging few-shot settings. The reported accuracies are multi-class classification accuracy averaged over five-fold runs of our model.

5.4 RQ4: Ablation Study

In this section, we aim to understand the contribution of the different components in our model. To this end, we compare variants of our model by (i) varying the base HNN model (HMLP, HAT, and HGCN), and (ii) deleting individual meta-learning components - H-ProtoNet implies H-GRAM without meta updates and H-MAML implies H-GRAM without prototypes. The model variants are compared on the real-world datasets and the results are presented in Table 4. The ablation study indicates that meta gradient updates and label prototypes contribute to ≈ 16%and ≈ 6%improvement in H-GRAM's performance, respectively. This clearly demonstrates the ability of label prototypes in encoding inductive biases and that of meta gradients in transferring the knowledge from metatraining to the meta-testing phase. Additionally, from our study on different HNN bases for HGRAM, we note that the HGCN base outperforms the other bases of HMLP and HAT by ≈ 19% and ≈ 2%, respectively. Thus, we choose HGCN as the base in our final model.

6 Conclusion

In this paper, we introduce H-GRAM, a scalable hyperbolic meta-learning model that is able to learn inductive biases from a support set and adapt to a query set with disjoint nodes, edges, and labels by transferring the knowledge. We have theoretically proven the effectiveness of node-centric subgraph information in HNN models, and used that to formulate a meta-learning model that can scale over large datasets. Our model is able to handle challenging few-shot learning scenarios and also outperform the previous Euclidean baselines in the area of meta-learning. Additionally, unlike previous HNN models, H-GRAM is also able to scale to large graph datasets.

±

±

±

±

±

References

Ivana Balaˇ zevi´ c, Carl Allen, and Timothy Hospedales. Multi-relational poincar´ e graph embeddings. In Proceedings of the 33rd International Conference on Neural Information Processing Systems , Red Hook, NY, USA, 2019. Curran Associates Inc.

Silv` ere Bonnabel. Stochastic gradient descent on riemannian manifolds. IEEE Transactions on Automatic Control , 58(9):2217-2229, 2013. doi: 10.1109/TAC.2013.2254619.

Avishek Joey Bose, Ankit Jain, Piero Molino, and William L. Hamilton. Meta-graph: Few shot link prediction via meta learning, 2020. URL https://openreview.net/forum?id= BJepcaEtwB .

Ines Chami, Zhitao Ying, Christopher R´ e, and Jure Leskovec. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems , 32, 2019.

Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher R´ e. Low-dimensional hyperbolic knowledge graph embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 6901-6914, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.617. URL https://aclanthology.org/2020.acl-main.617 .

Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Proceedings of the AAAI Conference on Artificial Intelligence , 34(04):3438-3445, Apr. 2020. doi: 10. 1609/aaai.v34i04.5747. URL https://ojs.aaai.org/index.php/AAAI/article/view/ 5747 .

Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. Meta relational learning for few-shot link prediction in knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 42084217, Hong Kong, China, November 2019. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/D19-1431 .

Nurendra Choudhary, Nikhil Rao, Sumeet Katariya, Karthik Subbian, and Chandan K. Reddy. Self-supervised hyperboloid representations from logical queries over knowledge graphs. In Proceedings of the Web Conference 2021 , WWW '21, page 1373-1384, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383127. doi: 10.1145/3442381. 3449974. URL https://doi.org/10.1145/3442381.3449974 .

Nurendra Choudhary, Nikhil Rao, Sumeet Katariya, Karthik Subbian, and Chandan K. Reddy. Anthem: Attentive hyperbolic entity model for product search. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining , WSDM '22, page 161-171, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450391320. doi: 10.1145/3488560.3498456. URL https://doi.org/10.1145/3488560.3498456 .

Nurendra Choudhary, Nikhil Rao, Karthik Subbian, Srinivasan H. Sengamedu, and Chandan K. Reddy. Hyperbolic neural networks: Theory, architectures and applications. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, page 4778-4779, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393850. doi: 10.1145/3534678.3542613. URL https://doi.org/10.1145/ 3534678.3542613 .

Bhuwan Dhingra, Christopher Shallue, Mohammad Norouzi, Andrew Dai, and George Dahl. Embedding text in hyperbolic spaces. In Proceedings of the Twelfth Workshop on GraphBased Methods for Natural Language Processing (TextGraphs-12) , pages 59-69, New Orleans, Louisiana, USA, June 2018. Association for Computational Linguistics. doi: 10.18653/ v1/W18-1708. URL https://aclanthology.org/W18-1708 .

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 , ICML'17, page 1126-1135. JMLR.org, 2017.

Octavian Ganea, Gary B´ ecigneul, and Thomas Hofmann. Hyperbolic neural networks. Advances in neural information processing systems , 31, 2018.

Ning Gao, Hanna Ziesche, Ngo Anh Vien, Michael Volpp, and Gerhard Neumann. What matters for meta-learning vision regression tasks? In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 14756-14766, 2022. doi: 10.1109/CVPR52688. 2022.01436.

Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. Hyperbolic attention networks. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum?id=rJxHsjRqFQ .

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/ 5dd9db5e033da9c6fb5ba83c7a7ebea9-Paper.pdf .

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. arXiv preprint arXiv:2005.00687 , 2020.

Kexin Huang and Marinka Zitnik. Graph meta learning via local subgraphs. Advances in Neural Information Processing Systems , 33:5862-5874, 2020.

Mehrdad Khatir, Nurendra Choudhary, Sutanay Choudhury, Khushbu Agarwal, and Chandan K. Reddy. A unification framework for euclidean and hyperbolic graph neural networks. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China , pages 3875-3883. ijcai.org, 2023. doi: 10.24963/ijcai.2023/431. URL https://doi.org/10.24963/ijcai.2023/431 .

Thomas N Kipf and Max Welling. Variational graph auto-encoders. NIPS Workshop on Bayesian Deep Learning , 2016.

Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian optimization in pytorch. arXiv preprint arXiv:2005.02819 , 2020.

Hung-yi Lee, Shang-Wen Li, and Thang Vu. Meta learning for natural language processing: A survey. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 666-684, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022. naacl-main.49. URL https://aclanthology.org/2022.naacl-main.49 .

Lu Liu, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. Learning to Propagate for Graph Meta-Learning . Curran Associates Inc., Red Hook, NY, USA, 2019.

Galileo Mark Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven active surveying for collective classification. In International Workshop on Mining and Learning with Graphs , Edinburgh, Scotland, 2012.

Marion Neumann, Plinio Moreno, Laura Antanas, Roman Garnett, and Kristian Kersting. Graph kernels for object category prediction in task-dependent robot grasping. In Online proceedings of the eleventh workshop on mining and learning with graphs , pages 0-6, 2013.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch´ e-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32 , pages 80248035. Curran Associates, Inc., 2019. URL http://papers.neurips.cc/paper/

9015-pytorch-an-imperative-style-high-performance-deep-learning-library. pdf .

Nataˇ sa Prˇ zulj. Biological network comparison using graphlet degree distribution. Bioinformatics , 23(2):e177-e183, 2007.

Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of adam and beyond. In International Conference on Learning Representations , 2018. URL https://openreview. net/forum?id=ryQu7f-RZ .

Ryan A. Rossi and Nesreen K. Ahmed. The network data repository with interactive graph analytics and visualization. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence , AAAI'15, page 4292-4293. AAAI Press, 2015. ISBN 0262511290.

Juwen Shen, Jian Zhang, Xiaomin Luo, Weiliang Zhu, Kunqian Yu, Kaixian Chen, Yixue Li, and Hualiang Jiang. Predicting protein-protein interactions based only on sequences information. Proceedings of the National Academy of Sciences , 104(11):4337-4341, 2007.

Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/ cb8da6767461f2812ae4290eac7cbc42-Paper.pdf .

Jianing Sun, Zhaoyue Cheng, Saba Zuberi, Felipe Perez, and Maksims Volkovs. Hgcf: Hyperbolic graph convolution networks for collaborative filtering. In Proceedings of the Web Conference 2021 , WWW '21, page 593-601, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383127. doi: 10.1145/3442381.3450101. URL https://doi.org/10.1145/3442381.3450101 .

Abraham A Ungar. Analytic hyperbolic geometry: Mathematical foundations and applications . World Scientific, 2005.

Ricardo Vilalta and Youssef Drissi. A perspective view and survey of meta-learning. Artificial intelligence review , 18(2):77-95, 2002.

Anoushka Vyas, Nurendra Choudhary, Mehrdad Khatir, and Chandan K. Reddy. Graphzoo: A development toolkit for graph neural networks with hyperbolic geometries. In Companion Proceedings of the Web Conference 2022 , WWW '22, page 184-188, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450391306. doi: 10.1145/3487553. 3524241. URL https://doi.org/10.1145/3487553.3524241 .

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning , pages 6861-6871. PMLR, 2019.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations , 2019. URL https: //openreview.net/forum?id=ryGs6iA5Km .

Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. GraphSAINT: Graph sampling based inductive learning method. In International Conference on Learning Representations , 2020. URL https://openreview.net/forum?id= BJe8pkHFwS .

Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V Chawla. Few-shot knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 3041-3048, 2020.

Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji Geng. Meta-gnn: On few-shot node classification in graph meta-learning. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management , CIKM '19, page 2357-2360, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450369763. doi: 10.1145/3357384.3358106. URL https://doi.org/10.1145/ 3357384.3358106 .

Fan Zhou, Chengtai Cao, Goce Trajcevski, Kunpeng Zhang, Ting Zhong, and Ji Geng. Fast network alignment via graph meta-learning. In IEEE INFOCOM 2020 - IEEE Conference on Computer Communications , pages 686-695, 2020. doi: 10.1109/INFOCOM41043.2020. 9155456.

Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open , 1:57-81, 2020. ISSN 2666-6510. doi: https://doi.org/10.1016/ j.aiopen.2021.01.001. URL https://www.sciencedirect.com/science/article/pii/ S2666651021000012 .

Marinka Zitnik and Jure Leskovec. Predicting multicellular function through multi-layer tissue networks. Bioinformatics , 33(14):i190-i198, 2017.

Marinka Zitnik, Rok Sosiˇ c, Marcus W. Feldman, and Jure Leskovec. Evolution of resilience in protein interactomes across the tree of life. Proceedings of the National Academy of Sciences , 116(10):4426-4433, 2019. doi: 10.1073/pnas.1818013116. URL https://www.pnas.org/ doi/abs/10.1073/pnas.1818013116 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the model, Hyperbolic GRAph Meta Learner (H-GRAM), that learns transferable information from a set of support local subgraphs using hyperbolic meta gradients and label hyperbolic protonets to enable faster learning over a query set of new tasks disjoint subgraphs. The model is evaluated on downstream tasks of both node classification and link prediction. The experiments and ablation studies show that H-GRAM effectively learns and transfers information in few-shot settings and outperforms its Euclidean counterparts.\n\nSTRENGTHS:\nIn general, the paper is well written and the model introduces some novel contributions. Further, regarding experiment results, the model seems outperform all the baselines consistently on tasks of link prediction and node classification. The ablation studies from varying the base HNN model and deleting individual meta-learning components are informative to better understand the influence of the various components in the model and the final proposed architecture.\n\nWEAKNESSES:\n- In the Related Work section on Hyperbolic Neural Networks, the following important reference is missing regarding HNNs for large scale datasets (from the knowledge graph domain) from recent work:\n\n[KDD 2022] Dual-Geometric Space Embedding Model for Two-View Knowledge Graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '22). Association for Computing Machinery, New York, NY, USA, 676–686. https://doi.org/10.1145/3534678.3539350\n\n- In the problem setup section, can the authors more clearly explain the properties of the graph (e.g., directed/undirected, what do the nodes/edges represent etc.)?\n\n- Regarding experiments, it would be useful to see the model performance on both inductive as well as transductive tasks. Furthermore, can the authors provide more details on the dataset statistics such as number of vertices and edges and the domain of the dataset to get a better indication of the data size?\n\n- I also have a concern about the scalability of the model especially since non-Euclidean space embedding models tend to converge very slowly and Mobius computations are more computationally expensive compared to the Euclidean space. Can the authors provide some model complexity analysis (e.g., runtime/memory complexity)?\n\nQUESTIONS:\nPlease see weaknesses section above.\n\nLIMITATIONS:\nAuthors have sufficiently identified limitations of the prior work and addressed it in their proposed model. It would also be helpful if the authors provide future directions for their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submitted work identifies an important research problem of existing works, scalability of hyperbolic neural network to large graphs and previously unseen graphs. To achieve these two goals, this work first proves that node classification and link prediction can be done with a node's local neighborhood only. Based on this insight, this work designs a meta-learning mechanism for hyperbolic graph neural networks to scale on large graphs. Experiments on both small and large graphs show the effectiveness of the proposed model.\n\nSTRENGTHS:\n1. This paper proposes an interesting and important research problem, scalability of hyperbolic graph neural networks on large datasets. With a theoretical analysis and a meta-learning based model achitecture, this paper shows promising performance over baseline models.\n\n2. This paper is self-contained, with enough introduction to background knowledge, such as meta-learning and operations in hyperbolic space, in Appendix.\n\n3. Experiments are comprehensive with both small and large graphs, with baselines from different categories, and with both node classification and link prediction tasks.\n\nWEAKNESSES:\n1. Introduction section contains too much redundant content. Introduction section should provide a general and overall picture of the paper, while this submitted work introduces too much model architecture and details in the Intro section. I suggest authors to remove some content and better emphasize the key innovation proposed in the paper. Paper writing can be imporved.\n\n2. I can see standard deviation of experiment results in most of the tables, but Table 1 doesn't have std. dev.\n\nQUESTIONS:\n1. Why is standard deviation at Table 1 absent?\n\n2. This paper uses Poincare ball as the hyperbolic model for illustration. I am wondering if the proposed model is also applicable when Hyperboloid model is used.\n\nLIMITATIONS:\nI can't see any potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a method, Hyperbolic GRAph Meta Learner (H-GRAM), to improve the scalability and generalization of Hyperbolic Neural Networks (HNNs). H-GRAM learns from local subgraphs and transfers this learning to new, disjoint subgraphs in a few-shot setting. The authors demonstrate that H-GRAM outperforms existing methods in various few-shot settings and scales effectively over large graph datasets.\n\nSTRENGTHS:\nThe paper presents a new approach, H-GRAM, that combines meta-learning with hyperbolic neural networks (HNNs) to address their scalability and generalization issues. The quality of the work is evident in the detailed explanation of H-GRAM and its demonstrated effectiveness in comparison with baselines.\n\nWEAKNESSES:\n* There are a few areas where it could potentially be improved:\n\n  * Comparison with Other Meta-Learning Approaches: The paper could include a comparison of H-GRAM with other meta-learning approaches in table 2 and 3, not just with other HNNs. This would provide a broader context for understanding the performance and advantages of H-GRAM.\n\n* Limited contribution\n\n  * This work seems just extend paper[1]'s work to hyperbolic and present some trivial definitions and theorem.\n  * Lack of important references [2-4] to make comparisons.\n[1]Huang, Kexin, and Marinka Zitnik. \"Graph meta-learning via local subgraphs.\" Advances in neural information processing systems 33 (2020): 5862-5874. The proposed method is close to this paper. Please make a detailed comparison with the method.\n\n[2] Yu, Tao, and Christopher De Sa. \"Random Laplacian Features for Learning with Hyperbolic Space.\" arXiv preprint arXiv:2202.06854 (2022). They also said that their method is scalable. Please compare with their method.\n\n[3] Zhang, Yiding, et al. \"Lorentzian graph convolutional networks.\" Proceedings of the Web Conference 2021. 2021. They developed a new HGNN with the aggregation in the manifold, could you provide your definition and theorem in such a case?\n\n[4] Yang, Menglin, et al.  HGCN: Tree-likeness Modeling via Continuous and Discrete Curvature Learning. KDD 2023. They also derive the node influence in the case of tangent space.\n\nQUESTIONS:\nsee weaknesses \n\nAdditionally, a lot of HNN or HGNNs are formulated within the manifold. \n\nCould you deduce the node influence and establish the information loss without utilizing a logarithmic map (i.e., relying on tangent space)? \n\nIf you use a logarithmic map, could you derive the conclusion using a local reference point other than the origin point since you say that \"we use the local tangent space of Poincare ball model to prove that the local neighborhood policy holds better for HNN models?\"\n\nLIMITATIONS:\nnot mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose applying the MAML methodology to hyperbolic GNNs in a novel local manner, along with continuous label prototypes, that enables them to scale the HNN approach from a few thousand nodes to a few million nodes.\nThe authors provide theoretical justification for this local H-GRAM approach via theorems 1 and 2 and demonstrate its efficacy via a large set of experimental comparisons in Sec. 4 and 5.\n\nSTRENGTHS:\n1. The four research questions from Sec. 4 regarding the novel HGRAM approach are comprehensively answered via comparison on multiple different datasets with competing Euclidean MAML approaches, standard hyperbolic baselines, other graph MAML approaches such as G-Meta [17], Meta-GNN [38], protoNET[29] and ablation studies. H-GRAM clearly outperforms competing meta-learning / protoNET approaches on large graphs in Table 1 and is comparable with competing hyperbolic approaches on small graphs in Table 2.\n\n2. The authors provide theoretical justification for their local H-GRAM approach via theorems 1 and 2.\n\nWEAKNESSES:\n1.\nSome of the mathematical presentation could be shortened, e.g., the same symbol $D_{g\\mu}^{p_i}$ is defined in both theorems 1 and 2. \n\n2.\nThe authors limit their meta-learning enhancements to hyperbolic neural networks, but it is not clear if their enhancements can be applied to superior pseudo-Riemannian approaches, e.g., Pseudo-Riemannian Graph Convolutional Networks, NeurIPS 2022, and Ultrahyperbolic Neural Networks, NeurIPS 2021.\n\n3.\nThe authors' approach seems quite similar to G-Meta [17], although their experimental results seem to be slightly better in Table 1. Presumably it is the hyperbolic modeling (and RSGD) or the continuous label prototypes that allows H-GRAM to outperform G-Meta, but such explanations or other explanations are not discussed.\n\n4.\nIn Sec. 5.4, it is not discussed why HGCN outperforms HMLP and HAT. In other studies, attention-based GNNs sometimes outperform GCNs, so some explanation appears to be necessary. The SG, SL setting on the Cora dataset from Table 2 could be considered in the ablation study in Table 3 to help answer this question.\n\nQUESTIONS:\nPlease note the questions inherent in weakness 2, 3 and 4.\n\nLIMITATIONS:\nIt is not clear if weakness 2 above constitutes a limitation or whether it is simply a case where no theoretical justification can be provided for an ulterhyperbolic-GRAM variant, for example.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper introduces H-GRAM, a novel meta-learning model for scalable Hyperbolic Graph Neural Networks. H-GRAM leverages meta-learning techniques to learn from from local subgraphs and adapt quickly to new tasks. The authors theoretically establish that HNNs are dependent on the local neighborhood of nodes for prediction and formulate HNNs to encode node-centric local subgraphs using the locality of tangent space transformations. Experiments conducted on various benchmark datasets to illustrate that H-GRAM addresses several HNNs tasks such as inductive learning, over-smoothing elimination, and few-shot learning in various demanding situations.

STRENGTHS:
- The paper presents a novel approach that combines hyperbolic geometry and meta-learning techniques, which is innovative and interesting.
- The proposed H-GRAM is scalable and efficient compared to previous HNN techniques.
- Extensive and carefully designed experiments have been conducted to demonstrate the effectiveness of H-GRAM

WEAKNESSES:
For results listed in table 2, why did H-GRAM never achieve the best performances for both of node classification and link prediction at the same time among all datasets? Could you explain the reason behind this discrepancy between tasks?

QUESTIONS:
I don't have further questions.

LIMITATIONS:
The authors adequately addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07