Title: GAMMA: Gated Multi-hop Message Passing for Homophily-Agnostic Node Representation in GNNs
Venue: NeurIPS 2025
Accepted: yes
Keywords: GNNs

Semi-supervised Learning

Message Passing

Heterophlic GNNs
Abstract: The success of Graph Neural Networks (GNNs) leverages the homophily principle, where connected nodes share similar features and labels. However, this assumption breaks down in heterophilic graphs, where same-class nodes are often distributed across distant neighborhoods rather than immediate connections. Recent attempts expand the receptive field through multi-hop aggregation schemes that explicitly preserve intermediate representations from each hop distance. While effective at capturing heterophilic patterns, these methods require separate weight matrices per hop and feature concatenation, causing parameters to scale linearly with hop count. This leads to high computational complexity and GPU memory consumption. We propose Gated Multi-hop Message Passing (GAMMA), where nodes assess how relevant the aggregated information is from their k-hop neighbors. This assessment occurs through multiple refinement steps where the node compares each hop's embedding with its current representation, allowing it to focus on the most informative hops. During the forward pass, GAMMA finds the optimal mix of multi-hop information local to each node using a single feature vector without needing separate representations for each hop, thereby maintaining dimensionality comparable to single hop GNNs. In addition, we propose a weight sharing scheme that leverages a unified transformation for aggregated features from multiple hops so the global heterophilic patterns specific to each hop are learned during training. As such, GAMMA captures both global (per-hop) and local (per-node) heterophily patterns without high computation and memory overhead. Experiments show GAMMA matches or exceeds state-of-the-art heterophilic GNN accuracy, achieving up to $\approx20\times$ faster inference. Our code is publicly available at \url{https://github.com/amir-ghz/GAMMA}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGAMMA proposes an efficient GNN architecture for learning in heterophilic or mixed-homophily graphs. It avoids traditional multi-hop concatenation by using an iterative routing-style gating mechanism that adaptively weights multi-hop embeddings per node, while sharing a single projection matrix to reduce overhead. The paper performs detailed computational profiling and demonstrates significant accuracy and efficiency gains on benchmark datasets.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength: \n-\tThe idea of iterative soft-attention routing over hops seems novel and well-motivated.\n-\tInteresting analysis on the efficiency of current heterophily methods ,and the results show the proposed message passing scheme is time- and memory-efficient.\n-\tSome analysis are intuitive and elegant,e.g., the capsule network analogy and gating mechanism.\n-\tSome empirical gains (i.e., runtime and memory profiling of 13 baselines) are significant, e.g., up to 20× faster inference while matching or exceeding state-of-the-art accuracy on some datasets.\n\n## Weakness:\n-\tOne major concern is the novelty is incremental: The proposed message passing scheme is quite similar to the one proposed in GPR-GNN. I don’t see any clear distinction. Also, GPR-GNN allows having negative coefficients for combining hop-specific embeddings $\\mathbf{H}$, while the proposed scheme does not have that flexibility.\n-\tIn line 66, the authors mention \"Adaptive node-specific message passing\". However, the proposed scheme is not node-specific but rather hop-specific. This is confusing.\n-\tA similar line of work, which actually formulates node-specific message passing [1], is not discussed and included as a baseline.\n-\tWhile routing is claimed to be efficient, no direct ablation on routing steps (e.g., 1 vs 3) is presented. It will be helpful to include ablation study on routing iterations and gating stability.\n\n-\tTheoretical analysis of convergence, expressivity, or generalization are missing.\n-\tPerformance shown mostly on small to mid-scale datasets (e.g., Cornell, Tolokers); unclear how it scales to large graphs. It will be interesting to see how the proposed method perform on the heterophilic datasets introduced in [2]. Also, it would be interesting to study the proposed method with large scale OGB datasets.\n\n## References:\n\n[1] Finkelshtein et al. Cooperative Graph Neural Networks. ICML 2024.\n\n[2] Platonov et al. A Critical Look at the Evaluation of GNNs Under Heterophily : Are We Really Making Progress? ICLR 2023.\n\nQUESTIONS:\n-\tHow does the routing mechanism converge, and is it stable across diverse graph types? It’ll be helpful to provide analysis on routing convergence behavior and its effect on representation learning.\n-\tIs weight sharing across hops always beneficial — could this limit expressiveness for disjoint hop information?\n-\tHow sensitive is GAMMA to graph noise or perturbations (e.g., edge rewiring)?\n-\tCan GAMMA adapt to temporal or attributed dynamic graphs?\n\nI'm open to increase the score if the questions and the weaknesses are reasonably addressed.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces gated multi-hop message passing (GAMMA) as a method for efficient node classification on heterophilic graphs. Compared to existing methods, GAMMA provides a 20x improvement in inference speed and achieves comparable performance to the state-of-the-art. The efficiency of this method comes from a novel gating mechanism which avoids the need for feature concatenation, and the introduction of a weight sharing scheme.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n-\tThe paper introduces GAMMA to tackle the known and relevant problem of GNN performance and efficiency on heterophilic graphs and provides convincing evidence that GAMMA can address these issues.\n-\tGAMMA is well motivated and explained intuitively through the use of figures.\n-\tThe paper is very clear and well written.\n\n**Weaknesses:**\n\n-\tThe evaluation of GAMMA is limited to small scale graphs. Evaluation on larger scale heterophilic graphs like the ones discussed in Lime et al. 2021 or Platonov 2023 would provide stronger support for the applicability of this method in realistic settings.\n-\tThe conclusion briefly mentions limitations for \"extremely large graphs\". Further discussion of this and other limitations would improve the paper and make its scope clearer.\n-\tA figure showing both classification accuracy and computation time would be welcome to make clear, for example, that while GCN-JK has a comparable computation time to GAMMA, it has significantly lower accuracy across the selected datasets. A figure highlighting the compute to accuracy trade-off would prevent the reader from having to go back and forth from Table 1 to Figure 5.\n\n\nLim, D., Hohne, F. M., Li, X., Huang, S. L., Gupta, V., Bhalerao, O. P., & Lim, S.-N. (2021). Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In Advances in Neural Information Processing Systems. https://openreview.net/forum?id=DfGu8WwT0d\n\nPlatonov, O., Kuznedelev, D., Diskin, M., Babenko, A., & Prokhorenkova, L. (2023). A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=tJbbQfw-5wv\n\nQUESTIONS:\nCould you include some evaluation of GAMMA on larger scale graphs or at least some discussion of why this not included?\n\nIs the absence of evaluation on larger scale graphs due to any limitations in scaling up GAMMA?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new architecture for multi-hop aggregation graph learning. Previous work captured heterophilic patterns in graphs by concatenating $p$-hop neighborhoods but this approach often suffers from high memory requirements. The authors instead propose a gated multi-hop message passing mechanism (GAMMA) which essentially involves computing a weighted sum of hop embeddings (gating coefficient dictate the weights in the sum), thereby avoiding concatenation. Additionally, there is an iterative update for the gating coefficient that happens each forward pass of GAMMA. GAMMA is evaluated against a variety of standard message passing NNs and k-hop message passing NNs and shows competitive classification accuracy with greater efficiency (in terms of memory and inference time)\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- Strong experimental baseline comparisons and the tables regarding computation efficiency are convincing. GNNs often suffer from long training times (especially on large graphs) since a forward pass depends on the number of nodes/edges in the input graph. This is probably even more of a problem with multi-hop GNNs so the efficiency of GAMMA (I think due to the fact that they avoid the computational cost associated with concatenation of multi-hop node features) as compared to previous multi-hop techniques seems significant. \n\n**Weaknesses** \n- Explanation of GAMMA is a bit unclear, I have some questions below that hopefully the authors can clarify. \n- Minor typo: in table 1, I think GCN shows the best accuracy for CiteSeer and should be highlighted in green.\n\nQUESTIONS:\n1. How are the hop-specific embeddings generated? Is the GAMMA layer something that comes after standard multi-hop message passing layer? Is it that if I have a node $v$ and its $p$-hop neighbors $\\mathcal{N}^p$ with some computed embeddings, there is some gating coefficient $s_i^{(t)}$ learned per $p$-hop neighborhood and then the new node feature is generated by taking a weighted sum over the $p$-hop neighbors with the gating coefficients as the weights? I guess the gating coefficient is based on by the set of gating logits $b_i$ but are these gating logics learned? I think there is a good summary of GAMMA's gating mechanism in lines 381-384 that maybe should have been moved to the earlier section where GAMMA is introduced. \n2. Is there any additional computation cost during training time due to the $R$ routing iterations for the GAMMA gating mechanism? Can the number of iterations can be controlled in some data-drive way (see [1] where they control the number of iterations through another learnable MLP module)? \n3. It seems that the performance for GAMMA suffers a bit on homophilic datasets (which makes sense because the focus of GAMMA is for heterophilic benchmarks) but is there a way to see how the performance of GAMMA changes as the level of homophily in the datasets change? \n\n[1] \"Towards scale-invariant graph-related problem solving by iterative homogeneous gnns\"\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper demonstrates through comprehensive empirical analysis that state of the art GNNs are not computationally efficient in memory and runtime. To address these limitations authors propose GAMMA that leverages multihop gated message passing and weight sharing across hops to reduce both memory and runtime. GAMMA is show to match or exceed state of the art GNNs on homophilous and heterophilous benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The problem is well-motivated and a fair and rigorous comparison of memory/runtime requirements for state of the art GNNs is a gap in the literature as far as I'm aware. Often times, this is an overlooked aspect in GNN design.\n2. The two designs are intuitive. It is clear how they address and mitigate the memory/runtime issues of state of the art GNNs\n3. The paper is clear and well written. \n\n**Weaknesses**\n1. My main weakness is the evaluation. The paper does a good job setting up the problem and also demonstrating the runtime and memory requirements across many state of the art GNNs, but only evaluates on relatively small datasets (table 1). Texas, Cornell, and Wisconsin each range in 100s of nodes with some dataset construction issues (e.g. only one node per class). Squirrel and Chameleon have been identified as having issues as well with many duplicate nodes, leading to test set leakage [1]. There have been many proposed heterophilous datasets since these original smaller ones [1, 2]. Since memory and runtime are the main motivation, I think it would be good to include (i) GAMMAs performance on these larger datasets and also (ii) comparisons to state of the art GNNs on these larger datasets since the large datasets are where these factors play the most important role. These would address questions I have about if GAMMA's performance remains matched with state of the art for larger datasets. I would also be very interested to see a fair and rigorous comparison of the feasibility of training large scale GNNs. is it the case that expensive GNNs like H2GCN is infeasible to train with even a moderate amount of compute, memory, and runtime? If so, I think these would make the contributions stronger. \n2. I was also wondering how novel the designs are (multihop gating and weight sharing across hops)? It seems that these have been used in the literature each individually and comparing your specific designs to others that are similar would be helpful to appreciate the contributions. \n\n[1] Platanov et al., A critical look at the evaluation of GNNs under heterophily: Are we really making progress?\n[2] Lim et al., Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods\n\nQUESTIONS:\n1. How does GAMMA perform on larger datasets in comparison to state of the art GNNs?\n2. Are state of the art GNNs feasible to train on larger datasets given a reasonable budget of compute, time, and memory?\n3. How are GAMMAs designs (gated multihop message passing and shared weight across hops) related and different to designs proposed in the literature?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-23"}