Title: You Only Spectralize Once: Taking a Spectral Detour to Accelerate Graph Neural Network
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Neural Network

Acceleration

Compressed Sensing

Graph Fourier
Abstract: Training Graph Neural Networks (GNNs) often relies on repeated, irregular, and expensive message-passing operations over all nodes (e.g., $N$), leading to high computational overhead. To alleviate this inefficiency, we revisit the GNNs training from a spectral perspective. In many real-world graphs, node features and embeddings exhibit sparse representation in the Graph Fourier domain. This inherent spectral sparsity aligns well with the principles of Compressed Sensing, which posits that signals sparse in one transform domain can be accurately reconstructed from a significantly reduced number of measurements. This observation motivates the design of a more efficient GNNs that operates predominantly in compressed spectral subspace. Thus, we propose You Only Spectralize Once (YOSO), a GNN training scheme that performs single Graph Fourier Transformation to project features onto a learnable orthonormal Fourier basis, retaining only $M$ spectral coefficients ($M \ll N$). The entire GNN computation is then carried out in reduced spectral domain. Final full-graph embeddings are recovered only at output layer by solving a bounded $\ell_{2,1}$-regularized optimization problem. Theoretically, drawing upon Compressed Sensing theory, we prove stable recovery throughout training by showing that the projection onto our learnable Fourier basis can satisfy the Restricted Isometry Property when $M=\mathcal{O}(k \log N)$ for $k$-row-sparse spectra, acting as the measurement process. Empirically, YOSO achieves an average 74\% reduction in training time across five benchmark datasets compared to state-of-the-art methods, while maintaining competitive accuracy.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces YOSO (You Only Spectralize Once), a novel training scheme for GNNs that leverages spectral sparsity and compressed sensing to accelerate training while maintaining competitive accuracy. The key insight is that node features/embeddings in real-world graphs exhibit sparse representations in the Graph Fourier domain. YOSO projects input features onto a learnable orthonormal Graph Fourier basis Uℓ only once, retaining only M≪N spectral coefficients via a fixed sensing matrix Φ. All GNN computations occur in this compressed M-dimensional subspace. At the output layer, full-graph embeddings are reconstructed.\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths\n\n- The integration of learnable spectral transformations with compressed sensing for GNN acceleration is novel. Unlike prior sampling/pruning methods, YOSO avoids repeated spatial operations by operating entirely in a compressed spectral domain after a one-time projection.\n- Demonstrates significant speedups while preserving accuracy across diverse tasks (node/link classification) and datasets. Ablation studies confirm robustness w.r.t. M.\n\n#### Weaknesses\n\n- Existing GNNs that require explicit (truncated) eigendecomposition also do eigendecomposition only once at the data preprocessing step, e.g. Specformer [1] and Spec-GN [2]. Given so, the authors need to clarify their main argument \"You Only Spectralize Once\".\n- Experiments focus on GCN. Extensions to other SOTA GNN architectures is not validated.\n- The spectral sparsity assumption may not hold for all graph types. The authors are encouraged to provide empirical or theoretical evidence of spectral sparsity assumption.\n- The Universal Sensing Matrix $\\Phi\\in\\mathbb R^{M\\times N}$ is essential to the main goal of this paper, i.e., compression. But the construction of $\\Phi$ in Section 4.3 is not well motivated.\n\nQUESTIONS:\n- In line 156, unknown equation indicator.\n- The authors should elaborate \"Uℓ can be initialized as a random orthogonal matrix (e.g., via QR decomposition of a Gaussian matrix) or using other Stiefel manifold initialization techniques\".\n- The construction of Uℓ still needs matrix decomposition like QR decomposition? Please explain why this is a better choice over eigendecomposition.\n- Around line 242, the element-wise product is better denoted by \\odot.\n- The statement \"GNNs iteratively apply the non-linear activation functions. If transform embeddings into spectral domain and later convert them back to original domain (for performing the downstream task), but only use a simple inverse transform (e.g., U^−1), thus, these nonlinearities prevent perfect recovery of original embeddings.\" seems problematic. Please explain why nonlinearities prevent perfect recovery?\n\n\n[1] Specformer: Spectral Graph Neural Networks Meet Transformers\n[2] A New Perspective on the Effects of Spectrum in Graph Neural Networks\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces YOSO (You Only Spectralize Once), a novel and practical approach to accelerating GNN training by avoiding repeated message passing. The core idea is to project graph signals into a learned spectral basis and use compressed sensing techniques to sparsely sample and reconstruct signals for downstream tasks like node classification and link prediction. The approach provides strong theoretical guarantees (RIP, error bounds), achieves \\~74% training time reduction, and matches or slightly underperforms baselines in accuracy across five benchmark datasets.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths:**\n\n#### **Technical Quality:**\n\n* Provides rigorous theoretical backing for the reconstruction process, including proofs for the sensing matrix’s full-rank property and RIP satisfaction.\n* The integration of learnable GFTs with compressed sensing is implemented carefully and supported by mathematical justification and empirical results.\n* Training speed improvements are consistent and significant across tasks and datasets.\n\n#### **Significance:**\n\n* The work directly addresses a major bottleneck in large-scale GNN training — computational inefficiency — with a conceptually novel and practical solution.\n* Proposes a new paradigm shift from spatial-domain sampling (used in methods like GraphSAINT or Cluster-GCN) to spectral-domain sparse sampling.\n\n#### **Originality:**\n\n* While compressed sensing and GFTs are established fields, their combination with learnable spectral bases in GNNs is novel.\n* The “you only spectralize once” conceptually reframes how feature propagation is handled — offering a fresh direction for GNN efficiency research.\n\n\n* The paper is well-organized and the core idea is clearly presented.\n* Appendices provide implementation details and proofs, supporting reproducibility.\n\n---\n\n### **Weaknesses:**\n\n* The method relies on spectral sparsity, which may not hold in all graph types (e.g., complete graphs, random graphs). More guidance or diagnostics for real-world practitioners would help.\n* There is limited discussion on dynamic or time-evolving graph settings, where spectral bases may evolve.\n* Ablation experiments isolating the contributions of GFT learning vs. sampling vs. decoder reconstruction could strengthen the empirical section.\n* A minor typo: Equation reference on line 156 is not rendered correctly in the PDF.\n\nQUESTIONS:\n1. How can a practitioner assess whether their dataset satisfies the spectral sparsity assumption needed for successful reconstruction? Are there any heuristics or automatic checks?\n2. Given that \\$\\mathbf{U}\\_l\\$ is learned during training, how do you ensure that the RIP condition is preserved or approximately satisfied?\n3. Have you considered whether the learned spectral basis can adapt incrementally to graph topology changes, or would retraining be required?\n\nLIMITATIONS:\n1. The method assumes spectral sparsity in the node feature signals, which may not generalize well to graphs with dense or noisy spectral representations (e.g., social networks, small-world graphs). The paper would benefit from practical guidance or automatic checks for this assumption before deployment.\n\n2. While the method excels on static graphs, its extension to dynamic or evolving graphs remains unclear. If the graph topology changes over time, the learned spectral basis may no longer be valid, potentially requiring full retraining. This could limit real-world applicability in streaming or online settings.\n\n3. The current evaluation is restricted to transductive learning. It is not evident how the framework would perform on inductive node classification or in few-shot graph scenarios, where node embeddings need to generalize to unseen subgraphs.\n\n4. The reconstruction module (decoder) plays a crucial role in performance, but its architecture and training stability are underexplored. In particular, the decoder’s ability to recover clean signals across different graph structures is not analyzed in depth.\n\n5. The approach requires the initial spectralization step to be performed once, but this may still involve expensive eigen decomposition or approximation for large graphs, depending on the implementation. Further analysis on this one-time cost would improve transparency.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a GNN training framework called You Only Spectralize Once (YOSO). They first project the features into a learnable orthonormal graph Fourier basis, and then use compressed sensing, which retains a subset of the spectral coefficients. Then the GNN computation is performed in the compressed spectral domain. Finally, they recover the full graph embeddings to the original domain at the output layer.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The proposed method achieves consistent training time reductions without major accuracy loss. The experiments are extensive and well-controlled.\n2. The paper offers a clear and well-motivated explanation of the problem and the solution. The background on spectral graph theory and compressed sensing makes it easier for readers to understand the core concepts.\n3. I appreciate Figure 2, which effectively summarizes YOSO.\n4. Each step in the framework is well-motivated and thoughtfully designed. For example, the method is grounded in compressed sensing theory and spectral graph signal processing. The RIP-based analysis provides a solid theoretical justification for the design. Also, the idea of using FISTA is a clever choice, effectively integrating traditional compressed sensing techniques with modern learning-based approaches.\n\nWeakness:\nSee questions.\n\n\nMinor:\n1. Line 17 ‘we prove that stable recovery by….’\n2. Line 156 ‘Eq (??)’\n\nQUESTIONS:\n1. The proposed method relies on the assumption that the graph features are sparse in the spectral basis. I am curious whether this assumption applies to graphs with noises or heterogeneous features. Is there any evidence or analysis to support this assumption?\n2. Can this method be adapted to the dynamic graphs? For example, after the matrix $U_l$ is trained, can it be reused or transferred when the graph is changing over time?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}