Title: Sketch-Augmented Features Improve Learning Long-Range Dependencies in Graph Neural Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: graph neural networks

learning on graphs

random projection
Abstract: Graph Neural Networks learn on graph-structured data by iteratively aggregating local neighborhood information. While this local message passing paradigm imparts a powerful inductive bias and exploits graph sparsity, it also yields three key challenges: (i) oversquashing of long-range information, (ii) oversmoothing of node representations, and (iii) limited expressive power. In this work we inject randomized global embeddings of node features, which we term Sketched Random Features, into standard GNNs, enabling them to efficiently capture long-range dependencies. The embeddings are unique, distance-sensitive, and topology-agnostic---properties which we analytically and empirically show alleviate the aforementioned limitations when injected into GNNs. Experimental results on real-world graph learning tasks confirm that this strategy consistently improves performance over baseline GNNs, offering both a standalone solution and a complementary enhancement to existing techniques such as graph positional encodings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGNNs often face challenges such as oversquashing, oversmoothing, and limited expressiveness, particularly when modeling long-range dependencies. This paper introduces Sketched Random Features (SRF), randomized global summaries of node features designed to address these issues. SRF is created by first projecting node features into a kernel space using random mappings such as random Fourier features, and then aggregating them across nodes via the Johnson–Lindenstrauss transform. The resulting sketches are unique, distance-sensitive, and independent of graph topology. By injecting these representations into each GNN layer, the model gains efficient access to global context without relying on costly attention mechanisms. This approach consistently improves performance on both synthetic and real-world graph benchmarks, while offering significant gains in memory and runtime efficiency over transformer-based methods.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is clearly written and easy to follow. It presents the core limitations of GNNs in a well-motivated and accessible manner. The proposed method is grounded in well-established theoretical foundations and makes effective use of them. Empirical results demonstrate that incorporating SRF improves both performance and efficiency compared to strong baselines.  \n\nDespite these positive points, I have the some reservations: \n\n - SRF is positioned as a topology-agnostic method. It is entirely based on node features and random projections, which may reduce effectiveness in tasks where structural cues are critical. While this is acknowledged as complementary to structural encodings, the standalone method may underperform in structurally complex graphs with weak features.\n\n - SRF relies on random kernel mappings and projections. This may lead to variability in performance across different runs, especially in low-data or low-variance settings. The paper does not deeply investigate the robustness of SRF across seeds or noise levels.\n\n - There is no detailed ablation on sketching parameters such as embedding dim and kernel choice.\n\n - While the method is well motivated and grounded in established theoretical tools such as random kernel features and the JL transform, it does not introduce any novel theoretical results. The analysis largely restates known properties and lacks deeper contributions, such as convergence guarantees, generalisation bounds, or new theoretical insights into the behaviour of GNNs.\n\nQUESTIONS:\nI would request the authors to address the points raised in the weakness section.\n\nLIMITATIONS:\nThe societal or ethical implications of the work are not deeply discussed. For instance, while the method is efficient and scalable, are there settings such privacy-sensitive or fairness-constrained healthcare applications where injecting global summaries might amplify biases?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a node augmentation method that individualizes node features while further enriching them with global information. This is accomplished by concatenating the original node features with randomized projections of kernel embeddings of the node features. These so-called 'Sketched Random Features' are computed by first approximating kernel embeddings on the node features, which are then transformed via a random projection matrix. Leveraging results from the Johnson–Lindenstrauss lemma, the authors derive theoretical guarantees that their method mitigates common GNN issues such as oversquashing and oversmoothing. In an experimental evaluation, the authors support their claims on oversquashing and oversmoothing and demonstrate that the individualized node features enable more expressive GNN models. Furthermore, the approach is validated on standard graph classification benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The article is well written and structured.\n- The technical quality of the paper is excellent. \n- Although conceptually simple, the approach is elegant in how it simultaneously addresses two key aspects: global node information propagation and node individualization. \n- The theoretical results are presented clearly and appropriately support the paper’s claims.\n- The benchmarks on oversmoothing and oversquashing are convincing. \n\nWeaknesses:\n- While the title mentions long-range dependencies, this aspect is not much explored in depth within the paper. Maybe the authors can evaluate their method on standard long-range benchmark datasets.  \n- Minor notes:\n\t- Algorithm 1 is placed in Section 1 but not discussed until Section 3 (page 6). Thus, some of the notations have not been introduced yet such as \\Epsilon or \\mathcal{S}_AG.\n\nQUESTIONS:\n- How are graphs of varying sizes handled? If I understand correctly, both the embedding and sketch operators assume a fixed graph size N. \n- Individualisation of nodes generally comes at the cost of worse generalisation of the model. Do you have an intuition on how your method affects sample complexity?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Sketched Random Features (SRF), a method for augmenting message-passing Graph Neural Networks (GNNs) with global, randomized embeddings derived from node features. Combining kernel random features and Johnson–Lindenstrauss sketching, these embeddings are injected into each GNN layer to mitigate oversquashing, oversmoothing, and limited expressiveness. The paper provides theoretical justifications for the approach, and experiments on both synthetic and real-world benchmarks demonstrate improved performance and efficiency over standard GNNs and other positional encoding strategies.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n- Clearly motivated solution to major GNN limitations\n- Solid theoretical analysis of key properties\n- Empirically effective on diverse benchmarks\n- Efficient in runtime and memory\n- Modular and conceptually elegant design\n\n## Weaknesses\n\n- *Novelty*: The core contribution combines well-understood ideas -- random feature maps and sketching -- rather than introducing fundamentally new algorithms or theoretical insights. While the combination works, the paper should clarify how SRF advances beyond prior random feature–based or positional encoding methods for GNNs.\n\n- *Experiments*\n  - *Limited architectural diversity in evaluation*: The experiments focus predominantly on GIN/GINE architectures. Although the method is described as architecture-agnostic, the paper does not assess integration or performance with other state-of-the-art models (e.g., GraphSAGE, GAT, GCNII). A broader empirical evaluation could validate SRF’s general applicability.\n\n  - *Dataset/baseline choices*: There have been many papers that address the issues of oversquashing, oversmoothing, and expressivity. It would be better if the authors choose more datasets that are popular in the literature so that they can compare against a more diverse set of methods (e.g., see datasets and methods in the Open Graph Benchmark).\n\n  - *Ablation and sensitivity analysis*: The manuscript briefly mentions variations such as kernel types, projection dimensions, and embedding strategies, but lacks thorough ablations examining how these choices impact empirical performance, stability, and runtime. For example, how sensitive are results to sketch dimension, kernel selection, or the number of random projections? Such analyses would offer practical guidance for applying SRF.\n\n  - *Limited discussion of practical trade-offs*: The paper does not systematically discuss how to choose practical hyperparameters (especially sketch dimension) to balance distance preservation, efficiency, and model accuracy. Insights on this front would strengthen the work’s utility and reproducibility.\n\n- *Presentation* (minor issue): The paper does not provide schematics or diagrams showing how SRF embeddings are computed or incorporated into GNN layers. A clear schematic would help readers understand the interplay between SRF embedding and message passing, especially for practitioners aiming to implement the method.\n\nQUESTIONS:\n1. How does SRF  advance beyond prior random feature–based or positional encoding methods for GNNs?\n\n2.1 Have the authors evaluated SRF with diverse GNN architectures (e.g., GraphSAGE, GAT, GCNII) to support its claimed architecture-agnostic applicability?\n\n2.2 Have the authors evaluated on more popular datasets (e.g., from Open Graph Benchmark) and other baselines addressing oversmoothing and oversquashing?\n\n2.3 How does SRF’s performance vary with respect to key hyperparameters such as sketch dimension, kernel type, or the number of random projections?\n\n2.4. Can you provide practical guidance on selecting hyperparameters (e.g., sketch dimension) to balance model accuracy, distance preservation, and computational cost?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper studies graph learning with GNNs, and proposed a simple feature augmentation method to improve the long-range dependency learning of GNNs. The main idea is to create a new "sketch" for each node by taking all the node features in the graph and mixing them together using a designed but randomized projection. This sketch gives every node a unique, global signature that helps the GNN see long-range patterns and keep node embeddings from becoming too similar. The authors show that this simple and efficient trick works well on its own and can even be stacked with other methods to make GNNs more powerful.

STRENGTHS AND WEAKNESSES:
s1. This paper is overall clearly written.

s2. The proposed method, although complicate in design and analysis, is very easy to implement. The proposed method is also task agnostic.

s3. This paper is backed with good thoretical grounding. The authors provide formal propositions to explain why SRF works, showcasing that these sketches are unique, distance-sensitive, and contain cross-node information.

w1. The evaluation is a bit lightweight. This work can benefit from some more comprehensive experiments.

w2. The efficiency comparison in Fig 2 can also be extended to include more methods, which would make the comparison more convincing.

QUESTIONS:
1. I would appreciate some discussion from the authors on how the proposed method would work on inductive systems where new nodes/edges show up frequently.

LIMITATIONS:
n/a
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03