A Precompute-Then-Adapt Approach for Efficient Graph Condensation

Authors: Yuan Li, Jun Hu, Zemin Liu, Bryan Hooi, Jia Chen, Bingsheng He
Primary area: learning on graphs and other geometries & topologies
Keywords: graph, condensation

Abstract:
Graph Neural Networks (GNNs) have shown great success in leveraging complex relationships in data but face significant computational challenges when dealing with large-scale graphs. To tackle this issue, graph condensation methods aim to compress large graphs into smaller, synthetic ones that can be efficiently used for GNN training. Recent approaches, particularly those based on trajectory matching, have achieved state-of-the-art (SOTA) performance in graph condensation tasks.  Trajectory-based techniques match the training behavior on a condensed graph closely with that on the original graph, typically by guiding the trajectory of model parameters during training. However, these methods require repetitive re-training of GNNs during the condensation process, making them impractical for large graphs due to their high computational cost, \eg, taking up to 22 days to condense million-node graphs. In this paper, we propose a novel Precompute-then-Adapt graph condensation framework that overcomes this limitation by separating the condensation process into a one-time precomputation stage and a one-time adaptation learning stage. Remarkably, even with only the precomputation stage, which typically takes seconds, our method surpasses or matches SOTA results on 3 out of 7 benchmark datasets. Extensive experiments demonstrate that our approach achieves better or comparable accuracy while being 96× to 2,455× faster in condensation time compared to SOTA methods, significantly enhancing the practicality of GNNs for large-scale graph applications. Our code and data are available at \url{https://anonymous.4open.science/r/GCPA-F6F9/}.

Decision: reject
Reviewer scores: 8, 5, 3

Reviews and discussion:
<thread>
creation date:
1730008376484

thread_type:
official_review

<thread_object>
creator: reviewer_g3pa

<content>
summary:
This paper proposes a novel Precomputethen-Adapt graph condensation framework. The framework overcomes the efficiency limitation of the trajectory-matching condensation methods. By separating the condensation process into a one-time precomputation stage and a one-time adaptation learning stage, it achieves good experimental performance on the standard benchmark.

strengths:
1. The approach this paper proposed reduces computational costs compared to previous gradient-matching or trajectory-matching methods.
2. The experimental results demonstrate the effectiveness of the method to some extent.

weaknesses:
1. The novelty of this paper is limited. On the one hand, the so-called one-time precomputation stage in this paper, utilizing the mean value of sampled data features, is similar to some feature-matching dataset distillation methods; moreover, the story of "semantic" alignment was also proposed in [1]. On the other hand, the one-time adaptation learning process also just utilizes the commonly used graph contrastive loss.
2. The presentation of the paper is relatively weak. Basic concepts like Graph Condensation and Structure-Free Graph Condensation are repeatedly mentioned in both the introduction and related work sections, taking up a large portion of the main text. Additionally, the authors do not clearly explain the interrelationship between the two components.
3. In the experiments, the performance of the previous state-of-the-art methods is inconsistent with and significantly different from the performance reported in related technical papers. Was the baseline method carefully reproduced? Moreover, the method has an excessive number of hyperparameters and configurations, but the authors do not provide the hyperparameter settings for each dataset and condensation ratio or a principal tuning method, which undermines the effectiveness of the proposed method.
4. Since N' is the number of synthetic nodes, in eq 3, for i = 1, 2, . . . , N  ->  for i = 1, 2, . . . , N'.

[1] Diversified Semantic Distribution Matching for Dataset Distillation. ACM MM 2024

questions:
Please refer to weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730634008305

thread_type:
official_review

<thread_object>
creator: reviewer_luk2

<content>
summary:
This paper propose a simple but effective and efficiency method for directly sample and merge node features (after message passing). Then using the alignment loss to enhance such features. The results are quite strong.

strengths:
1. The performance of this work is comprehensive, typically including the PubMed and Products.
2. The effectiveness and efficiency are both impressive.

weaknesses:
1. The methods used by the authors are not particularly novel, especially the use of contrastive loss for feature adaptation.
2. I find the structure of the condensed graph somewhat unclear; it appears that the condensed graph may lack structure. I suggest clarifying this aspect within the context.
3. The deeper rationale behind this straightforward method isn’t entirely apparent to me, yet it yields strong results. From my perspective, the pre-computation stage resembles the initialization of previous methods. I’m curious about the adaptation stage, which seems to primarily refine features selected in the pre-computation phase. Is it possible that the entire adaptation stage could be ablated?

questions:
1. Please provide a deeper insight into your method, as I find this simple approach and its strong results intriguing. However, I remain unconvinced, as it seems to me like only a minor improvement on current initialization techniques, even prior to gradient-based or trajectory-based matching.
2. You might consider starting with some more ablation study, current one is quite short.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730705456935

thread_type:
official_review

<thread_object>
creator: reviewer_9kfh

<content>
summary:
The paper introduces "Precompute-then-Adapt," a fast and efficient framework for graph condensation in large-scale Graph Neural Networks (GNNs). Unlike traditional methods that require repeated, costly retraining, this approach divides the process into a one-time precomputation phase and a one-time adaptation stage. This novel framework achieves competitive or superior results on several benchmarks, reducing condensation time and making GNNs far more practical for large-scale applications.

strengths:
S1: Efficiency: The proposed"Precompute-then-Adapt" framework does significantly reduce the graph condensation process’s time,  and it effectively solves the repeat training consumption on the large-scale graph for existing structure-free methods, which sounds rational to me.

S2: High Performance with Simplified Methodology: The framework achieves expressive performance on several benchmark datasets with various condensation ratios, even with just the initial precomputation phase. Its one-time precompute strategy with both structure aggregation and semantic aggregation looks simple but efficient, making the proposed methodology practical.

S3: Clarity: The paper is well-structured, with a clear and logical flow that makes it easy to read and follow. All the experiential results are clearly demonstrated.

weaknesses:
I do have some concerned questions in terms of methodology and experiments:

W1: Unlike methods such as GCOND-X and SFGC, which mimic the GNN's learning behavior, the proposed method appears independent of specific GNN backbones, focusing instead on operations at the graph data level without involving models (one-time precomputation is not related to trained GNNs). This raises a question: how does the approach ensure similar performance on the original test graph set when no explicit GNN model-based constraints are applied?

W2: For the ablation study, it only shows w/o structure and w/o semantics, what about only with precompute features but without the feature adaption module and contrastive learning?

W3: Also for the ablation study, what is w/o both mean? does that mean using random initialized features to input the feature adaption module?

questions:
See weakness above.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces a Precompute-then-Adapt graph condensation framework aimed at addressing the high computational costs of previous methods. However, based on the feedback from Reviewer G3pa and Reviewer Luk2, there are significant concerns regarding the limited novelty of the proposed framework, the lack of rigor in comparisons with other methods, and the subpar performance on large datasets compared to state-of-the-art approaches. Given these issues, the submission does not meet the standards for acceptance at ICLR.