Title: Equivariance Everywhere All At Once: A Recipe for Graph Foundation Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: universal approximation

equivariance and invariance

graph foundation models

message passing
Abstract: Graph machine learning architectures are typically tailored to specific tasks on specific datasets, which hinders their broader applicability.  This has led to a new quest in graph machine learning: \emph{how to build graph foundation models (GFMs)} capable of generalizing across arbitrary graphs and features? In this work, we present a recipe for designing GFMs for node-level tasks from first principles. The key ingredient underpinning our study is a systematic investigation of the symmetries that a graph foundation model must respect. In a nutshell, we argue that label permutation-equivariance alongside feature permutation-invariance are necessary in addition to the common node permutation-equivariance on each local neighborhood of the graph.  To this end, we first characterize the space of linear transformations that are equivariant to permutations of nodes and labels, and invariant to permutations of features. We then prove that the resulting  network is a universal approximator on multisets that respect the aforementioned symmetries. Our recipe uses such layers on the multiset of features induced by the local neighborhood of the graph to obtain a class of graph foundation models for node property prediction. We validate our approach through extensive experiments on 29 real-world node classification datasets, demonstrating both strong zero-shot empirical performance and consistent improvement as the number of training graphs increases.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on designing a graph foundation model (GFM) for node tasks. The authors approach this problem from the viewpoint of equivariance, where they argue that three main types of symmetries are necessary for proper generalization: (a) Node permutation-equivariance, (b) Label permutation-equivariance, (c) Feature permutation-invariance. For example, the labels can of course differ based on the graph and task at hand. Also the node features can vary considerably by both semantic meaning and dimension. As such, respecting these symmetries are helpful to learning a GNN that are general and not specific to the graph itself (thus improving generalization). The authors propose a new model, TSNet, that respects these symmetries. The authors detail some experiments that show promising performance of their method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n---------\n\n1. I really like the motivation behind this work. Graphs can vary wildly in their properties, node features, and the downstream labels. Because of that, it makes sense to design a method that respects fundamental symmetries that help create a moe *general* GNN. As the authors note, this is a good way of learning a \"general graph vocabulary\", since respecting these symmetries help learn information that are more high-level and not too specific to each graph. I therefore think the approach introduced by the authors is really promising and intuitive.\n\n2. The paper is written and explained quite well. It makes reading the paper very easy and quite enjoyable. \n\n3. The theoretical proof in Theorem 4.2, that shows TSNet is symmetry preserving, is appreciated. I also find it's interesting that they find a mistake in the universality proof from *Maron et al. [2020]*, and show that the proof is true under certain conditions ($H = S_D$).\n\n\nWeaknesses:\n----------------\n\n1. **Insufficient Main Experiments**: I don't think the authors provide enough experiments for their *main results*. The authors test when training on Cora and performing zero-shot classification on the other 27 datasets. However, an issue I have is that they only test when pretraining on one dataset (being Cora). However, why should we choose Cora? Why not another dataset? My point being, while the results may be good when training on Cora, maybe they won't be when training on other datasets. It's the same logic as running an experiment over $k$ random seeds, maybe Cora is just a \"lucky seed\". An alternative, used by GraphAny, is to instead train 4 different versions of their model each on a different dataset (i.e., Cora, Wisconsin, Arxiv, and Products). I recommend the authors either use this approach or a similar type of approach for a fairer and more robust comparison.\n\n2. **Lack of Ablation Studies**: The authors introduce many components in their framework. However, no ablation study is given. This is important, as it's unclear of the benefit of various components. This is very important for some cases as it's unclear what is really driving the performance. For example, (a) How important are the use of DSS? What if you replace them with a basic DeepSet? (b) Are each type of symmetry necessary, what if you remove one? (c) How important is the feature-label mixing?. This last one is very important to me, as the authors adopt the approach from GraphAny by solving the least squares problem. If this is a big reason for the high performance, then it could suggest that many of the claims about the importance of symmetry for this problem may not be true (to an extent). Without any of these studies, I'm unsure how to practically evaluate the proposed framework.\n\n\n3. **Lack of Clear Benefit over GraphAny**: I'm not arguing that SOTA performance isn't necessary for a framework/paper to be good. Furthermore, I think the proposed method is interesting on its own merits. However, the performance compared to GraphAny is nearly identical and nonetheless perfectly within variance. This raises an obvious question which is, *why should we use TSNet*? The authors don't really answer this. The authors say, on line 111, that as opposed to their proposed framework, the design of GraphAny is \"very limiting for the model design space\". I understand their point, but it's really hard to argue without any tangible experimental results that show when TSNet can outperform GraphAny. I strongly encourage the authors to try to find some empirical evidence as to when/why their method is better and preferred. \n\n\n4. **Lack of Discussion on Model Complexity**: This is a fairly minor weakness and has little effect on my score. But I thought I'd mention it since including it would, imo, strengthen the paper. Basically, some discussion on this or some efficiency experiments would be appreciated. It's not necessary, but TSNet does introduce a little more overhead when preserving symmetry, so it'll be useful to see how this effects runtime for practical usage.\n \n\nOther:\n-----\n\n1. This is a small detail, but I'd recommend changing the title to reflect that the model is only designed for node-level tasks. As of now, I think the title slightly oversells the model to be more encompassing than it actually is. A good change would be \"Equivariance Everywhere All At Once: A Recipe for Graph Foundation Models for **Node-Level Task**\". This is just my opinion.\n\nQUESTIONS:\n1. Can you report the performance in the same manner to GraphAny or a similar setting (as noted in Weakness 1). Then for each, test them on the holdout datasets and average the results. This will form a much more comprehensive set of results.\n\n2. Can you run some ablation studies on the framework. Specifically, as noted in Weakness 2, it would be interesting to ablate some of the symmetries and the feature-label mixing.\n\n3. This is fairly open-ended, but I'd appreciate some insight (preferably empirical) as to why one should choose TSNet over GraphAny (assuming the performance is still essentially the same as in the paper). What scenarios does TSNet outperform GraphAny? One suggestion may be to check few-shot scenarios, where GraphAny may struggle (this is just an idea).  \n\n4. Additional runtime/complexity results would be appreciated but are not necessary.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the design of Graph Foundation Models (GFMs), aiming to address the issue that traditional Graph Neural Networks (GNNs) struggle to generalize across different graph structures, feature sets, and tasks. Its core contribution is proposing a theoretical framework based on \"triple-symmetry\": node permutation-equivariance (predictions change consistently with the permutation of node indices), label permutation-equivariance (predictions change consistently with the permutation of labels/targets), and feature permutation-invariance (predictions are unaffected by the ordering of features).\nTheoretically, the authors characterize the space of linear transformations that satisfy these triple symmetries and prove that the resulting Triple-Symmetry Network (TSNet) is a universal approximator for set functions that respect these symmetries. In practice, by applying TSNet to the local neighborhood aggregation of graphs, they propose Triple-Symmetry Graph Neural Networks (TS-GNN), which can convert any GNN into a GFM and achieve zero-shot generalization. Experimentally, the performance of TS-GNN (based on MeanGNN and GAT) is validated on 27 real-world node classification datasets, and its zero-shot performance outperforms traditional GNNs and the existing method GraphAny.\n\nSTRENGTHS AND WEAKNESSES:\n1. Proposes \"triple symmetry\" (node permutation-equivariance, label permutation-equivariance, feature permutation-invariance) as the core design criterion for graph foundation models (GFMs), providing a principled framework to address the cross-graph generalization issue of traditional GNNs and filling the gap in the theoretical foundation of graph foundation models.\n2. Proves that Triple-Symmetry Networks (TSNets) are universal approximators for set functions respecting the triple symmetry, providing mathematical guarantees for the model's effectiveness.\n3. Validates the model on 27 real-world datasets, showing that its zero-shot performance outperforms baselines, providing sufficient empirical support.\n4. Open-sources code and detailed experimental settings (e.g., datasets, hyperparameters), ensuring the reproducibility of results.\n\nQUESTIONS:\n1. The current framework is restricted to node-level tasks and not extended to edge-level or graph-level tasks, which is explicitly acknowledged as a limitation, restricting its generality.\n2. Although the theory supports regression tasks, experiments only validate classification, lacking empirical support for regression scenarios, weakening the framework's comprehensiveness.\n\nLIMITATIONS:\nThe feature-label mixing relies on least-squares projections, but its performance in scenarios with weak feature-label correlation (e.g., noisy features) is not discussed, potentially leading to generalization bottlenecks.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper investigates two important properties for graph learning: label permutation-equivariance and feature permutation-invariance, in addition to node permutation-equivariance. The authors propose a novel model, the Triple-Symmetry Network (TSNet), building on the prior work "On Learning Sets of Symmetric Elements."

STRENGTHS AND WEAKNESSES:
The paper is generally easy to follow, and the theoretical claims are well-supported by proofs provided in the supplementary materials. However, graph datasets for social media should be added to demonstrate the strength of TSNet.

QUESTIONS:
1. The average performance reported in Table 1 is comparable to the baseline method, GraphAny. 

2. The experiments currently lack graph datasets for social media, such as Instagram and Reddit, to analyze user properties. Please include these datasets during the rebuttal phase.

3. The pseudo-code for the proposed method would greatly enhance clarity and understanding.

4. Please further discuss how this model could be applied to GFM when language serves as node or edge features.

LIMITATIONS:
The authors write down the limitations in the introduction and discussion, noting that it is designed for node-level tasks.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a theoretically-grounded recipe for designing Graph Foundation Models (GFMs) for node-level classification and regression tasks. The central idea is to identify and enforce three key symmetries: node permutation-equivariance, label permutation-equivariance, and feature permutation-invariance. The authors characterize the space of linear transformations that respect these \"triple-symmetries\" and prove that the resulting \"Triple-Symmetry Network\" (TSNet) is a universal approximator on multisets. This theoretical foundation is then used to develop Triple-Symmetry Graph Neural Networks (TS-GNNs), a modular framework that can transform any standard GNN into a GFM capable of zero-shot generalization across graphs with varying structures, feature configurations, and label semantics. Extensive experiments on 27 real-world node classification datasets demonstrate that TS-GNNs (TS-Mean and TS-GAT) consistently outperform their non-transferable baselines and also improve upon the prior state-of-the-art GraphAny in a zero-shot setting.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The paper's primary strength lies in its rigorous theoretical analysis. The identification of the \"triple-symmetry\" criterion and the subsequent characterization of equivariant linear layers, culminating in the universality proof for TSNets, provides a robust mathematical underpinning for the proposed GFMs. This distinguishes the work from many empirically-driven approaches.\n\n- The TS-GNN framework offers a practical and modular way to convert existing GNN architectures into GFMs. This approach allows for leveraging the strengths of diverse GNN designs while enabling zero-shot generalization.\n\n- The extensive experiments on 27 diverse node classification datasets provide evidence for the effectiveness of TS-GNNs in a zero-shot setting.\n\n\nWeaknesses\n\n- While the experiments are extensive in terms of the number of datasets, they are exclusively focused on node classification tasks. The paper states that the theory supports node regression as well, but no empirical validation is provided for this. This limits the empirical scope of the claims.\n\n- The computational complexity of the algorithm is not thoroughly explained. For example, it involves solving the least square transformations into each layer which might be expensive for large graphs. The scalability aspect and computational cost should be discussed more, both theoretically and empirically in experiments. This shed light on the practicability of the proposed algorithm.\n\n- The experimental study is limited and only compared with very few baselines. Moreover, the gains seem relatively small, which is hard to tell the exact benefits. More ablation studies and sensitivity analysis could benefit the paper.\n\nQUESTIONS:\nSee weaknesses.\n\nAnd:\n- Could the authors discuss potential future work on extending this recipe to more complex graph structures, such as heterogeneous graphs with multiple node and edge types, or dynamic graphs where the topology changes over time?\n\n- What about the scalability of the proposed method? When talking about pre-training the graph neural network, what scale and computational cost and downstream task coverage is being discussed? Conceptually compare this with large pre-trained language models or vision models. Can the experiments go beyond the small simple node classification tasks?\n\nLIMITATIONS:\nThe paper does not include explicit limitation sections. Although in the checklist limitations were briefly mentioned, I would encourage the authors to include an explicit limitations section.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-06"}