Paper: The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive field (ICLR-2024)

Reviewer comment (Writing):
The motivation is unclear.

Author response:
Thank you for your insightful question. We would like to respectfully state that, although it is true that shallow GNNs can outperform deep GNNs on certain datasets, recent explorations in deep GNNs are positioning them as the mainstream in the field of graph learning. It is noteworthy that a considerable number of SOTA methods ranking highly on the OGBN benchmark surpass not only 6 layers [1,2,3] but, in some cases, 32 layers [4]. Hence, we believe our exploration of deep GNNs contributes meaningfully to the community.

Reviewer comment (Novelty):
The motivation is unclear, why enhance deepGNN? Many GNN use 2 or 3 layers but still achieve good performance. For example, [1] uses 3 layers and achieves 73% on ogbn-arxiv, higher than the model's performance.

Author response:
Thank you for your insightful question. We would like to respectfully state that, although it is true that shallow GNNs can outperform deep GNNs on certain datasets, recent explorations in deep GNNs are positioning them as the mainstream in the field of graph learning. It is noteworthy that a considerable number of SOTA methods ranking highly on the OGBN benchmark surpass not only 6 layers [1,2,3] but, in some cases, 32 layers [4]. Hence, we believe our exploration of deep GNNs contributes meaningfully to the community.

Reviewer comment (Evaluation):
In Table 4, performance drops after using SnoHv1, so SnoHv1 is not always helpful for deepGNN.

Author response:
Thank you for your keen observation! We acknowledge that SnoHv1, as evidenced in Table 4 with the 16-layer ResGCN+, led to a slight performance decrease (0.78$\downarrow$). We faithfully reported this based on the average results from 5 runs. However, we would like to humbly state that, in numerous other scenarios (refer to Tables 1 & 5), SnoH consistently enhances the performance of various GNN backbones. Overall, we believe it is not an overclaim to assert that SnoH can bring valuable performance improvements to all GNN backbones.

Reviewer comment (Experiments):
Is SnoH only limited to the GNN proposed here or does it also enhance other models' performance? I would suggest evaluating SnoH on GNN models that have demonstrated strong performance on Ogbn-Proteins and Ogbn-arxiv.

Author response:
Thank you for the constructive feedback! Indeed, the backbones (GCN/ResGCN/Res+) we selected are not the SOTA ones for current OGB datasets. However, as stated in our paper, the snowflake hypothesis is not confined to a specific GNN backbone; it is a universal operator that can be embedded in any (deep) GNN. To substantiate our standpoint, we have supplemented experiments demonstrating the application of the snowflake hypothesis to RevGNN [4] (ranked 4th and 26th on the Proteins/Products Benchmark respectively).

**Table 3.** Comparisons with SnoHv2 on Ogbn-Proteins. We report an average of 3 runs.
| Layer | 8  | 16 | 32 | 64 |
| --- | --- | --- | --- | --- |
| RevGNN-80 | 84.07±0.58 | 85.31±0.72 | 85.38±0.81 | 85.74±1.03 |
| w/ SnoHv2 | 84.66±0.81 | 86.07±0.86 | 86.86±0.69 | 86.21±0.92 | 

**Table 3.** Comparisons with SnoHv2 on Ogbn-Products. We report an average of 3 runs.
| Layer | 8  | 16 | 32 | 64 |
| --- | --- | --- | --- | --- |
| RevGNN-80 | 80.33±0.49 | 81.17±0.52 | 81.36±0.33 | 82.16±0.71 |
| w/ SnoHv2 | 80.79±0.60 | 82.24±0.95 | 82.85±0.89 | 84.82±1.11 | 


It can be observed that SnoHv2 has brought about a substantial performance improvement for RevGNN, with the enhancement becoming more pronounced as the number of layers increases (consistent with our observations in Tables 1 & 5).