Title: Memorization in Graph Neural Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: graph neural networks

memorization

node classification

homophily
Abstract: Deep neural networks (DNNs) have been shown to memorize their training data, but similar analyses for graph neural networks (GNNs) remain under-explored. We introduce NCMemo (Node Classification Memorization), the first framework to quantify label memorization in semi-supervised node classification. We establish an inverse relationship between memorization and graph homophily, i.e the tendency of connected nodes to share labels or features. Lower homophily significantly increases memorization, indicating that GNNs rely on label memorization when learning less homophilic graphs. We then analyze GNN training dynamics and find that increased memorization in low-homophily graphs is tightly coupled to GNNs' implicit bias toward using graph structure. When structure is less informative, models instead memorize node labels to minimize training loss. Finally, we show that nodes with higher label inconsistency in their feature-space neighborhood are more prone to memorization. Based on these insights, we investigate graph rewiring as a mitigation strategy. Our results show that rewiring reduces memorization without harming model performance, while also lowering the privacy risk for previously memorized data points. Thus, our work advances understanding of GNN learning and supports more privacy-preserving GNN deployment.


Peer reviews:
comments: SUMMARY:
The paper introduces NCMemo, a novel framework for measuring per-node label memorization in semi-supervised graph-node classification. This work provides a deep investigation into why and how GNNs memorize, links memorization to graph properties such as homophily, and proposes a practical method for mitigating resulting privacy vulnerabilities through graph rewiring.

STRENGTHS AND WEAKNESSES:
**Strengths**

- The paper presents interesting insights by extending leave-one-out memorization analysis to the graph domain, and it is the first to attribute memorization to individual nodes.
- This work builds on insightful training dynamics analysis, incorporating NTK alignment curves to convincingly link increased memorization to the model’s structural bias in cases where kernel-target alignment is poor.
- The experimental results span three GNN backbones, multiple random seeds, statistical significance testing, and ablation studies (edge addition, deletion, and their combination), lending robustness to the conclusions.

**Weaknesses**

- Rewiring is not compared against other defenses (e.g., label smoothing), and its impact on downstream tasks that rely on the original edge topology is not evaluated.
- The memorization indicator uses a fixed threshold of τ = 0.5 (line 122) without analysis, which may make the conclusions threshold-dependent.

QUESTIONS:
Q1. Could the edge additions/deletions that reduce memorization negatively impact tasks where edge semantics are important (e.g., recommender systems)? Some discussion or a case study would strengthen the practical claims.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the NCMemo, a framework to quantify label memorization in GNNs for semi-supervised node classification. Firstly,  author states the role of memorization in improving the performance of DNNs, and verifies the four proposed propositions on GNNs through many experimental results: (1) Higher graph homophily leads to lower memorization rates; (2) in low-homogeneity graphs, the optimization dynamics of GNNs show an over-reliance on the graph structure, leading to increased memorization rate; (3) better alignment between the kernel and the target or graph structure reduces memorization; (4) nodes with high label inconsistency in their feature space neighborhood are more prone to memorization.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper migrates the concept of memorization from DNN to the GNN field and put forward some interesting propositions and discoveries, which fill the gap that similar studies on data memorization in GNNs are lacking. \n\n2. The experiments in the paper are sufficient, and satisfactory experimental results have been obtained on both synthetic datasets and real-world datasets.\n\nWeaknesses:\n\n1. Some descriptions in the paper are not clear, such as the method to divide the training set into three disjoint partitions and Eqn. (1).\n\n2. Lack of solid theoretical proof and derivation.\n\n3. The position of Fig. 3 is not suitable.\n\n4. The paper directly borrows a lot of methods and ideas from other papers, resulting in low innovation of this paper.\n\nQUESTIONS:\n1.  The authors pointed out that due to the non-IID properties in graphs, analyzing the model’s ability to memorize individual nodes is challenging, as it requires modeling a complex interplay of factors. So, is it appropriate to define the GNN's memorization based on a single node (i.e., leave-one-out)?\n\n2. In addition, I suggest adding research about the 2-hop topology on graphs, which is also important for the message passing mechanism of GNNs.\n\n3. Would it be better to fit the results reflected in Fig. 5 and 7 using a quadratic polynomial?\n\n4. Others please refer to the Weaknesses part.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper proposes NCMemo, a framework to measure how much GNNs memorize training labels in node classification tasks. This paper finds that memorization increases as graph homophily decreases. The authors show that this happens because GNNs rely more on memorizing labels when the graph structure provides less useful information. They also propose a Label Disagreement Score (LDS) to identify which nodes are more likely to be memorized. Finally, they show that graph rewiring can reduce memorization and its associated privacy risks without hurting model performance.

STRENGTHS AND WEAKNESSES:
Strengths:
- This paper focuses on memorization in GNNs. This helps fill an important gap in understanding generalization and privacy risks in graph learning.
- This paper proposes a new framework to quantify per-node memorization, inspired by the leave-one-out memorization framework.
- This paper provides insightful results on the relationship between homophily, memorization, and label inconsistency, based on both synthetic and real-world datasets.
- This paper includes graph rewiring to mitigate memorization and privacy risks.

Weaknesses:
- This paper highlights the difficulty in conceptualizing and defining per-sample memorization, but the proposed NCMemo framework seems not including graph-specific techniques to tackle the interactions of graph elements.
- The framework relies on training paired models with/without each node (or partition approximations), which may be computationally expensive. The scalability of NCMemo on large-scale graphs in the real world can be limited.
- More datasets and advanced GNN backbones can be included in the experiments such as open graph benchmarks and graph transformers.

QUESTIONS:
Please see weaknesses

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a framework for quantifying label memorization in graph neural networks trained on semi-supervised node-classification tasks. They show that GNNs depend on memorization when operating on graphs with low homophily (i.e. graphs whose connected nodes do not share similar features). They show that high memorization leads to poor generalization via a kernel-target alignment metric. Finally, they reveal that the reason for increased memorization/poor performance for low homophily graphs is due to the tendency of GNN models to use structural information so in low homophily regimes,the lack of information from graph structure leads to more label memorization.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n(1) *Significance*: This seems to be the first work on memorization in graph neural networks. Provides a good framework/metric for understanding how and when graph neural networks memorize node features and has potential implications for understanding GNN performance out of distribution (more memorization possibly leads to bad OOd generalization). \n(2) *Extensive evaluation*: First, the authors do extensive experiments on both synthetic and real-world datasets. These experiments help clearly explain the link between graph homophily and memorization. They show a clear relationship between homophily and their metric which measures the rate of memorization for GNNs. In their appendix, they also include extensive evaluation with GAT and GraphSAGE (two commonly used graph architectures). I also appreciated the section analyzing privacy risks and the real-world impact of memorization. \n\nIn general, this is a strong paper which I think will have significant impact especially since it seems to be the first work on memorization in GNNs. The authors provide a convincing argument for studying GNN memorization from studying theoretically *when* we see memorization occurs to experimentally demonstrating the problems that arise from meorization at NN deployment time. \n\n### Weaknesses\n\n(1) Their framework only works in the node-classification regime (which is still important). I wonder if the authors could say a few words on how we should keep track of/understand memorization when considering graph level classification tasks. \n(2) Proposition 2 suggests that GNNs suffer from poor generalization when there is high memorization. This part seems relevant esp. given the increased attention to out-of-distribution generalization in several previous works so I wonder if the authors can elaborate more on this point and show this property experimentally?\n\nQUESTIONS:\n### Questions\n- Is it possible to show experimentally how memorization leads to poor generalization (as suggested by Prop. 2)?\n- NCMemo is specifically for semi-supervised node classification but can the authors say a few words on if this framework can also be extended to graph classification tasks? In that case, is it possible to analyze the intermediary embeddings from each GNN layer to draw conclusions regarding how GNNs memorize information for classification tasks?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}