Title: Pseudo-Riemannian Graph Transformer
Venue: NeurIPS 2025
Accepted: yes
Keywords: Pseudo-Riemannian manifold

Graph Transformer

Node Classification

Link Prediction
Abstract: Many real-world graphs exhibit diverse and complex topological structures that are not well captured by geometric manifolds with uniform global curvature, such as hyperbolic or spherical spaces. Recently, there has been growing interest in embedding graphs into pseudo-Riemannian manifolds, which generalize both hyperbolic and spherical geometries. However, existing approaches face three significant limitations, including the ineffective pseudo-Riemannain framework, the shallow architectures, and the absence of clear guideline for selecting suitable pseudo-Riemannian manifolds. To address these issues, we introduce a novel diffeomorphic framework for graph embedding that aligns with the nature of pseudo-Riemannian manifolds. Subsequently, we propose the pseudo-Riemannian Graph Transformer for learning representations of complex graph structures. Our diffeomorphic framework in pseudo-Riemannian geometry enables the principled definitions of core transformer components, including linear attention, residual connection, and layer normalization. Finally, we develop a lightweight space searching algorithm to automatically identify the most suitable pseudo-Riemannian manifold for an input graph. Extensive experiments on diverse real-world graphs demonstrate that our model consistently outperforms other baselines in both node classification and link prediction tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel graph transformer that is grounded on a pseudo-Riemannian manifold as the underlying space. By exploiting the properties of network spherical embedding and network hyperbolic embedding, the proposed approach constructs a geometric embedding model capable of preserving both cyclical and hierarchical graph representations. Experimental results show that this method achieves superior performance compared to the traditional Graph Convolutional Network (GCN) in node classification and link prediction tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe work covers a broad range of topics, from the foundational theory of graph embedding to downstream applications. The research process is comprehensive, and the objective is to address the issue of selecting the underlying embedding space for graph embeddings, which holds significant research value.\n\nWeaknesses:\n\n1. Theoretical Limitations: \n\nThe authors have not demonstrated the theoretical validity of the claims made in the paper regarding the network pseudo-Riemannian manifold embedding. Specifically:\n\n    (a) There is no theoretical proof that the method described in \"Contributions 1\" preserves the cyclical and hierarchical representations of graphs as claimed.\n\n    (b) The issue of \"geodesic disconnections\" that the authors frequently emphasize remains unresolved.\n\n2. Experimental Limitations:\n\n    (a) The authors' choice of comparison methods is limited, mainly focusing on GCN and its hyperbolic variant, HGCN.\n\n    (b) The ability of the proposed method to \"capture mixed-topological dependencies\" has not been empirically validated.\n\nQUESTIONS:\n1. The authors could clarify the meaning of \"x_0\" in line 99 and Equation (2), as this is crucial for understanding the definition of the \"Zero-first submanifold,\" which plays a central role in the paper.\n\n2. The authors should explain how the two mappings introduced in the paper address the \"geodesic disconnections issue,\" as this would provide more insight into this important aspect.\n\n3. The specific meaning of the notation \"x\" and \"z\" in Equation (4) should be clarified, as this would help readers better understand the formalism and its application.\n\n4. The conclusion of Theorem 1 seems somewhat self-evident, given that a pseudo-Riemannian manifold can be decomposed into the Cartesian product of subspaces formed by spherical and hyperbolic spaces. The focus should perhaps be placed on demonstrating the geodesics for these two spaces, specifically proving that a geodesic exists in the pseudo-Riemannian manifold composed of paths from the two subspaces constructed by the authors. This would strengthen the contribution of the theorem.\n\nLIMITATIONS:\nThe authors have discussed the limitations of their method in practical applications. However, they have not illustrated some inherent theoretical limitations of the approach, such as the completeness of the embedding in the pseudo-metric space and the lack of interpretability. These aspects could be important for a more comprehensive evaluation of the method.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 1, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of representing complex graph structures that are poorly captured by manifolds of uniform global curvature (e.g., hyperbolic or spherical spaces). The authors introduce a diffeomorphic framework for graph embedding on pseudo-Riemannian manifolds, generalizing both hyperbolic and spherical geometries. Building on this, they propose the pseudo-Riemannian Graph Transformer (Q-GT), which extends transformer operations (attention, residual connection, layer norm) to pseudo-Riemannian geometry. They also design a lightweight space searching algorithm to select the best manifold for a given graph. Experiments on node classification and link prediction across diverse real-world graphs demonstrate that Q-GT outperforms state-of-the-art baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) The Q-GT is the first transformer-based graph model built in pseudo-Riemannian geometry, enabling core transformer operations in this space. \n2) The space searching algorithm is simple yet but grounding the manifold selection in data-driven curvature estimation.\n3) Q-GT achieves strong, consistent performance on both node classification and link prediction tasks, outperforming established baselines such as QGCN (see Tables 1 and Table 2).\n\nWeakness:\n1) The space searching algorithm is grounded in a linear relationship between ideal standard deviation and space dimension, but this is mostly heuristic and lacks theoretical justification or sensitivity analysis.\n2) The paper lacks some analysis on training/inference time and scalability or complexity. Q-GT introduces higher computational and memory overhead compared to GNNs (and possibly some baselines), which might be a practical concern. \n3) The method uses space search, which takes more computation time and it is not fair to compare with other approaches without space search such as QGCN.\n\nQUESTIONS:\n1) How does the Q-GT compare in terms of training/inference time, parameter count, and GPU memory usage to the strongest baselines (e.g., Hypformer, SGFormer, Q-GCN)? \n2) Could the authors conduct the experiments on larger graphs such as Open Graph Benchmark?\n3) Which components of the Q-GT are most critical for its success (e.g., diffeomorphic mapping, specific attention mechanism, residual connection)? Is there any interpretability in the learned embeddings (e.g., correlation between curvature and graph properties)?\n\nLIMITATIONS:\nYes: The authors have adequately discussed limitations", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
Real-world graphs often exhibit complex topological structures, and hence it is often more suitable to embed graphs into pseudo-Riemannian manifolds than commonly used Euclidean space. Existing methods exhibit information loss, a lack of guideline for selecting dimensions, and limited context size, which limit their capacity. This paper proposes the pseudo-Riemannian graph transformer by extending linear attention and layer normalization to the pseudo hyperboloid. A space searching algorithm is designed based on the Gaussian Sectional Curvature distribution. Experiments corroborate the effectiveness of the proposed method.

STRENGTHS AND WEAKNESSES:
Strengths:

The proposed method is based on solid theory.

The first paragraph of section 5 clearly explains what kind of graphs are suitable for the proposed pseudo-Riemannian graph transformers.

The space searching algorithm increases the practicality of the proposed model.

The visualization of the learned embeddings is informative and interesting.

Weaknesses:

In line 47, the concepts of time and space dimensions appear without definition. And where do the names come from?

QUESTIONS:
See above.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents a graph transformer that operates in a Pseudo-Riemannian manifold, addressing limitations of prior hyperbolic and spherical approaches. The authors design a diffeomorphic transformer that splits the pseudo-Riemannian space into coupled hyperbolic and spherical components, thereby eliminating geometric disconnection. Building on this, they create the Pseudo-Riemannian Graph Transformer (Q-GT) and a curvature-aware search procedure that selects manifold dimensions by aligning the input graph’s Gaussian sectional-curvature (GSC) profile. Tested on seven node-classification and four link-prediction datasets, Q-GT achieves small yet consistent gains over strong Euclidean and hyperbolic baselines.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\nThe paper is well written. Using the product manifolds retain both the cyclical and hierarchical information from the data. The proposed space search algorithm is also efficient and removes the need for manual fine-tuning significantly. The theoretical discussion is rigorous and supported by clearly presented proofs, enhancing the paper’s credibility. Experiments span homophilous and non-homophilous graphs, where Q-GT outperforms baseline in 5/7 classification tasks and all link-prediction tasks.\n\n### Weaknesses\n- Use of log map and exp map in residual connection is inefficient. Recent methods like LResNet [1] have developed a fully hyperbolic residual connection method that does not rely on mapping. This approach can effectively lower computational overhead, reduce numerical errors from repeated projections, and preserve intrinsic curvature throughout training, thereby enabling faster and more stable convergence\n- The paper omits comparison with the recent FPS-T [2] method, which overlaps significantly with Q-GT despite being cited. The authors should clearly address what are the differences between Q-GT and FPS-T.\n- Gains over the strongest baselines (e.g., SGFormer, Hypformer) are often ≤1–2 F1 on `Cora`, `Pubmed` and `Penn94`, calling into question whether the extra geometric machinery justifies its cost.\n- Most benchmark datasets are small (< 10 k nodes), with the largest having 169 k nodes. Including larger benchmarks such as `ogbn-products` and `ogbl-vessel` would better demonstrate the model’s scalability.\n- Although the visualization of Q-GT embeddings in Figure 3 shows that nodes are somewhat dispersed, there is still no clear hierarchy. Is this because the dataset lacks any hierarchy or does the model require extra steps to capture the hierarchy completely? How would this compare to other hyperbolic models like Hypformer? \n- \"Geodesically disconnection\" reads awkwardly. I am assuming the it should be \"Geodesic disconnection\".\n\nOverall, I think this manuscript is promising and clearly presented, but it stops short of fully demonstrating the practical value of its geometric innovations. Adding (i) direct comparisons and clarifications versus closely related methods such as FPS-T and mixed-curvature Transformers, (ii) evaluation on large-scale benchmarks (e.g., `ogbn-products` and `ogbl-vessel`), and (iii) designing a fully Pseudo-Riemannian residual connection would substantially elevate the paper’s impact.\n\n[1] He, Neil, Menglin Yang, and Rex Ying. \"Lorentzian residual neural networks.\" arXiv preprint arXiv:2412.14695 (2024).\n[2] Cho, Sungjun, et al. \"Curve your attention: Mixed-curvature transformers for graph representation learning.\" arXiv preprint arXiv:2309.04082 (2023).\n\nQUESTIONS:\n- Is extending LResNet to the product space non-trivial?\n- How is the UMAP used for the visualization in Poincare space? Do you first log map the embeddings then use the UMAP or do you use a hyperbolic UMAP?\n\nLIMITATIONS:\nThe authors have addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}