Pseudo-Riemannian Graph Transformer
Authors: Le Viet Quan
Viet Cuong Ta
Venue: NeurIPS 2025 poster ()
Keywords: Pseudo-Riemannian manifold
Graph Transformer
Node Classification
Link Prediction
Primary area: deep_learning
TL;DR: We propose a Graph Transformer on pseudo-Riemannian manifolds

Abstract: Many real-world graphs exhibit diverse and complex topological structures that are not well captured by geometric manifolds with uniform global curvature, such as hyperbolic or spherical spaces. Recently, there has been growing interest in embedding graphs into pseudo-Riemannian manifolds, which generalize both hyperbolic and spherical geometries. However, existing approaches face three significant limitations, including the ineffective pseudo-Riemannain framework, the shallow architectures, and the absence of clear guideline for selecting suitable pseudo-Riemannian manifolds. To address these issues, we introduce a novel diffeomorphic framework for graph embedding that aligns with the nature of pseudo-Riemannian manifolds. Subsequently, we propose the pseudo-Riemannian Graph Transformer for learning representations of complex graph structures. Our diffeomorphic framework in pseudo-Riemannian geometry enables the principled definitions of core transformer components, including linear attention, residual connection, and layer normalization. Finally, we develop a lightweight space searching algorithm to automatically identify the most suitable pseudo-Riemannian manifold for an input graph. Extensive experiments on diverse real-world graphs demonstrate that our model consistently outperforms other baselines in both node classification and link prediction tasks.

Psendo-Riemannian Graph Transformer
Viet Quan Le' Viet Cuong Ta
Department of Computer Science -Human-Machine Interaction Laboratory VNU University of Engineering and Technology, Hanoi, Vietnam {lvquan, cuongtv) @vnu. edu.vn
Abstract
Many real-world graphs exhibit diverse and complex topological structures that are not well captured by geometric manifolds with uniform global curvature, such hyperbolic or spherical spaces. Recently, there has been growing interest embedding graphs into pseudo-Riemannian manifolds, which generalize both hyperbolic and spherical geometries. However, existing approaches face three significant limitations, including the ineffective pseudo- Riemannain framework, the shallow architectures, and the absence of clear guideline for selecting suitable manifolds. To address these issues, we introduce novel diffeomorphic framework for graph embedding that aligns with the nature manifolds. Subsequently, we propose the pseudo Saaudoo-iemmanian Graph Transformer for learning representations of complex graph structures. Our diffeomorphic framework in pseudo- Riemannian geometry enables the principled definitions of core transformer components, including linear attention, residual connection, and layer normalization. Finally, we develop lightweight space searching algorithm to automatically identify the most suitable pseudo-Rise Riemannian manifold for input graph. Extensive experiments diverse real-worls graphs demonstrate that our model consistently outperforms other basclines in both node classification and link prediction tasks.
1
Introduction
Many real-world graphs have diverse topological structures (Boguñá et al. [021). which are better captured using varying geometrical curvatures. Geometric manifolds with uniform global structure, such as hyperbolic or spherical manifolds, do not provide enough flexibility to model these complex graphs (Gu et al., 2018). Instead of using single manifold with constant curvature, researchers employ the Cartesian product of constant curvature manifolds to embed graphs of mixed topologies (Gu al./ 2018)) Bachmann al., 2020). However, explicit distances product manifolds may emphasize either the spherical or hy hyperbolic component while overlooking the other, which limits their ability embed heterogeneous graphs (Law) 202T). Moreover, because Riemannian manifolds like spherical manifolds use a positive defimite metric, they cannot accurately capture the negative eigenvalues present input similarity data (Laub and Müller 2004).
Recently, pseudo- pseudo-Riemannin Riemannian manifolds (also called semi-Riemanne Riemaniaannn manifolds), which generalize Riemannian manifolds, have gained increasing attention for embedding mixed topological graphs [Law and Stam 2020 Law 2021 Xiong et al.) 2022). Pseudo-Riemannian manifolds equipped with indefinite metrics not only extend hyperbolic and spherical geometries but also include hyperbolic and spherical submanifolds, allowing them to capture relationships unique to these geometries.
Law and Stam (2020) proposed an efficient method learn nonparametric embeddings with novel optimization tool on Souudo-Riemannian manifolds. Later, Law) (202I) employed pseudo- Riemannian geometry to graph neural networks by using the quotient manifold, which imposes
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

additional constraints on the optimized function. Xiong et al. (2022) introduced the diffeomorphic framework which decomposes aseudo-Riemamee manifolds into the product of spherical and Euclidean spaces. This framework helps avoid geodesic disconnection and simplifies the extension of GCNs to eseudo-Riemannin framework offers greater flexibility, as it can be applied to any optimized function within pseudo- Riemannian geometry. Nevertheless, existing methods exhibit three major drawbacks that limit their learning capacity. First, since hyperbolic submanifolds, this transformation onto the product of spherical and Euclidean spaces (Xiong et al., 2022) results in a loss of hierarchical information essential to hyperbolic geometry. Second, there is no clear guideline for selecting the number of time and space dimensions for pseudo- Riemannian approaches. A mismatch between the graph structure and the topology of the embedding manifold can introduce distortions in the graph representation (Ganea et al. 2018 Chami et al. 2019 Bachmann et al., (2200). Third, the context size in GNN architectures is often limited (Li et al. 2018; Oono and Suzuki (2020). As the depth of GNNs increases, it leads to a phenomenon known oversmoothing or oversquashing (Chen et al.) 2020) where node representations become indistinguishable across the graph. To mitigate this issue, recent studies have introduced Transformer- based graph encoders in Riemannian geometry (Wu al. 202 Cho et al.. 2023 Yang et al., 2024). However, their adaptation to more general Riemannian -aasd geometries has not yet been studied because developing key Transformer operations on such manifolds not trivial. Unlike Riemannian manifolds. pseudo-Riec Riemannian manifolds may contain pairs of points that cannot be connected by a continuous geodesic (Law 202T). As a result, gradient vectors are not parallel transported along geodesics, and the logarithmic map between two given points is undefined.
geometry. Compared to the work of Law (202T), the diffeomorphic
manifolds inherently comprise both spherical and
In this work, we introduce diffeomorphism that decomposes pseudo- Riemannian manifolds into the Cartesian product of spherical and hyperbolic spaces. This transformation, which is built upon a sub- manifold with smooth extrinsic mapping function, addresses the issue of geodesic disconnection and effectively preserves the cyclical and hierarchical structures of graphs. Subsequently, we propose an efficient geeudo-Riemannicn Graph Transformer (Q-GT) for learning representations of complex graphs. Our diffeomorphic framework enables the definition of fundamental operations, including linear attention, residual connection, and layer normalization, while allowing the use of standard gradient descent algorithms for effective optimization. Additionally, we develop lightweight space searching algorithm to determine the optimal pseudo neuuo-Riemanniin manifold for representing tixed-topology graphs. To achieve this, we construct a bijection that maps each pseudo- Riemannian manifold to an ideal Gaussian sectional curvature (GSC) distribution, and apply the Dellback-Leibles (KL) divergence to select a suitable manifold. Finally, comprehensive evaluations on node classification and link prediction tasks show that our model surpasses both Riemannian and pseudo- Beeud-Riemannian methods.
Contributions. Our contributions in this paper include: 1)
We introduce a novel diffeomorphic framework that transforms gseudo-Riemamee manifolds into the product of spherical and hyperbolic space to preserve the cyclical and hierarchical representations of graphs. 2) We propose an efficient Seoudo-Riemamnnn
Graph Transformer ( (Q-GT) for capturing mixed-topolee toxeed-opoologicll dependencies of input graphs. To the best of our knowledge, this is the first study on Transformer-ban based graph model built within the searching algorithm for determining the time and space hyperparameters of pseudo-Rien Riemannian manifolds based on sectional curvature distributions.
geometry. 3) We develop an efficient and effective space
2 Preliminaries
A generalization of Riemannian manifold is allows the metric tensor to be indefinite, meaning that for any nonzero tangent vector & E T.M, the value of (E, {) can take on both positive and negative signs. The ambient pseudo- -ucddded space Re-++ is d-dimensional vector space, where d = p + + E N, equipped with the following 2 scalar product (z. v)g - - +
manifold (O neill 1983) which
Vx, y E g0 A pseudo-hyperboloid Q is Euclidean space (Law and Stam 2020):
defined as the submanifold in the ambient
= fx 0
= (Io, 2..... Ip+g) E g
|| = 3].
where 3 is nonzero real number indicating space curvature, and ||=:|] = (xx x²), is the vector norm in the pseudo- Euclodeann space. Q3." is called psendo-sphere when [ > 0 and pseudo-hyyerboloid
2

---

when 3 < O. Since 2 and @ via an anti anti-isometry, we consider the hyperbooop in this work. Following the terminology of space-time vectors (Sun et al. 2015), for each point in 23, the first q + dimensions are time dimensions and the last p dimensions are space dimensions. Hyperbolic and spherical manifolds can be considered as special cases of aseudo-hyperbon
23 The d-dimensional spherical manifold, defined as $/A/ f R (a, a. = | where (., is the standard inner product in Euclidean space, is hyperrooooo 2 Meanwhile, the p- dimensional Lorentz model L (Chami et al. 2019), which is used to represent hyperbolic manifolds, is an upper sheet of the two-sheet pseudo-hyperbologg O The Lorentz model is defined as L = {x E RP.. (ar 2)), 8, z0 > 0), where (.--) (--- denotes the Lorentzian inner product. To build graph Transformer in Seeudo-Rieaanee geometry, rely on further geodesic tools in the gseudo-hyperbor the parallel transport.
model, including the exponential and logarithmic maps along with
p- the maps between the manifold 0 and the tangent space Ta 2 include the exponential map exp Ta Q3 2 and the logarithmic map log! 2 > Q Law and Stam (2020) proposed the closed-form expressions of two tangent maps, which we provide that geodesic does not exist between two points and if the scalar product satisfies (x,, 3) 2 [3]. Consequently, the logarithmic map is undefined at y. and the points x and y are considered geodesic- disconnected.
Exponential and logarithmic maps. Given a reference point E Og
Appendix for reference. They noticed
Parallel transport.
Given points and y that connected by geodesic "y the manifold 2 parallel transport P 2 - TTy
2 is an isomorphism between two tangent spaces that moves one tangent vector & E T 2. to another tangent space T along the geodesic
Law 2021). The formula of the parallel transport is given in the Appendix D.
Diffeomorphic framework
Based on the poses ggeudo-Riemanne manifold into the product of spherical and hyperbolic spaces. This framework effectively preserves the graphs' cyclical and hierarchical structures while addressing the geodesic disconnections issue. Our framework is centralized by an extrinsic mapping func- tion & which transports points from a manifold Q% to sub-manifold Qg.t1, followed by diffeomorphism Vi that transforms gg.et1 to the product of sphere S1 and a Lorentz model L..
formulation, we propose a diffeomorphic framework that decom-
3.1 Extrinsic mapping function
To construct extrinsic mapping function, first define zero-first submanifold Q of pseudo-
hyperboloid 234 as follows:
Definition 1 (Zero-first submanifold). zero-first submanifold Q of pseudo-hyperbolold Q is defined as
26¹0 = {xr Rn++! =|<==1, = xo = 0}.
(2)
Then the extrinsic mapping function Φb formulated as:
n 3 6p.9+1 and its inverse & 1- @p-4+1
2 are
(xx) (0, x) y + (v)
where x is a point on pseudo-hyperboloid manifold Φ and y is a corresponding point on the zero-first submanifold @r.++1 Several key properties of the extrinsic function t presented in the following proposition.
Proposition 1. The extrinsic mapping function & and its inverse --1 are smooth, bijective, and isometric.
The smoothness of the extrinsic mapping allows it to integrated into the Graph Transformer module in esscudo-Riemamen spaces. Furthermore, its bijective and isometric properties help preserve the
3

---

O '
39 1.1
x Lig
Oulput
Q-GT
Dropout
Project
Decompose
4,,
Spherical Space
Choose
X
Residual
Transfomation
Eodimoting Curvature Distribationn
Hyper balic Space
Q-GCN2
Nip.o")
Linear Attention
LayerHorn
Data flo
Figure 1: The overall architecture of our proposed Q-GT.
integrity of original node features during vector addition the pseudo-hyperboloid, addressing limitations in previous works (Law 2021 Xiong et al., 2022).
3.2 Diffeomorphic function
The following theorem provides diffeomorphism that transforms the submanifold Qp.++1 into the product a spherical manifold [als and a hyperbolic manifold L.g-
Theorem 1. For any point x E 2 there exists a diffeomorphism x E ② - 1 L
that maps x onto the product manifolds of a sphere and a hyperboloid. The mapping and its inverse are given by:
(xxx)
|
V (x)
w
(4)
2 with ut E Sipp of Lg.
where Q 1 with Ro+1 \ (0). E RP, and
r
The notation |u|| / (u, u) indicates the Euclidean norm. The proof of Theorem iss given in the Appendix E Compared to the diffcomorphism proposed by Xiion et al. (2022). our framework better preserves the hierarchical structure of graphs. The approach in Xiong et al. (2022) transforms points the pseudo-hyperboloid into more Fuclidean product space the space dimension p increases, potentially diminishing the hyperbolic properties essential for capturing hierarchical relationships. Meanwhile, diffeomorphism maps points the hheudo--yperboloid hyperbolic spaces that aligns naturally with two geometric properties of pseudo-Riee Rienammian hyper- boloid: (i) containing spherical and hyperbolic topologies,
product of spherical and
(ii) becoming spherical (or hyperbolic) manifold when p = 0 (or 0). Hence, the product space S/. x LL can be used to interpret the evolution of pseudo-hy houudd--yperblood 2.. leading our efficient space searching algorithm for deter- mining the optimal pseudo-hypertee hyp-eboleebodd in Section 5 The proposed diffeomorphic framework be viewed as generalization of the hyperbolic framework that underlies state-of-the-ar non-Euclidean methods such HGCN (Chami et al. 2019) and Hypformer YYnng al. 2024).
4 Pseudo-Riemannian Graph Transformer
To address the limitations of pseudo-Riemane GNNs, we propose an efficient pseudo-Riemannian Graph Transformer (Q-GT), as illustrated in Fig. [ By leveraging diffeomorphic framework Section 3 we extend fundamental operations, including linear attention, and layer normalization to the heeudo-hypeboolog Q Additionally, design a novel residual connection that operates directly the pseudo-hyperbolodi ensuring stable training and enhancing the performance of Q-GT.

---

4.1 Pseudo-Riemannian linear attention
6 Given the node features X { S VEEN of input graph G, where each v; apply the extrinsic mapping function + and the diffeomorphism v, as defined Eq. and Eq. to project the features onto the product space, i.c. X 1 x (( )).
E 2 we first
*/s1 x (u;, u3)² where E S/a and es E L... Since the spherical and hyperbolic manifolds are independent within the product space S/ x L.g. we can apply linear attention to each manifold separately and then concatenate the results. Yang et al. (2024) proposed a linear attention mechanism in hyperbolic space Lg, allowing for efficient processing of large-scale data. Specifically, the hyperbolic linear attention can be described as:
We decompose each node feature 2;
5 (VIlWe; Wo:)
(5)
(0
10
Ex-
N
T (c (o
83
0 ( D K1
8.1 L.WE W (QI, K1, ViJ.
K. B.y ol (i " (6) 0 (11 B,0,) EL
(8)
where Q1. 1, V1 E Re'x(p ((++1) are three trainable matrices, and ó is a positive-definite kernel function used to lincarize the standard similarity function sim Q, K) in Transformers. In our Q-GT, we choose Φ(x) = + elu(z), which is commonly used in linear attention models (Wu et al. 2022 2023 The core idea behind hyperbolic linear attention is that each vector v; 12g projected by three matrices Q1 K1, V1 in Eq. (5). The kernel function á and linear atten- tion are then applied to the space-like components 2i,[:: in Eq. (6) and Eq. (7). Finally, the
time-like component is computed to obtain the hidden vector é; Eq. (8) It is important to note that the hyperbolic linear attention used in this paper differs from Yang et al. (2024).
since we do not employ kernel function ó to the value vector V and omit the residual con- nection. Similar to hyperbolic linear attention, introduce spherical linear attention as follows:
u. (Vis/ /lllll
(9)
(n, En K2 (u (1)))))) új
T
(11)
S. W (Q2, K2, V2).
(u;
Qu
T プj=1 "j
K2,
K2 o(u Q2), (u
(10) û (VBBlu./ | E 1
(12)
where Q2 K2. V2 (R( xxx((+1) are three trainable matrices. After applying linear attention, S% (ūt 0.)².
xL
concatenate the outputs ú; and é to achieve 5th node's hidden feature x;
4.2 Pseudo- Beeudd-Riemannian refining function
Inspired by Yang al. (2024), we introduce the product refining function a, which generalizes fundamental operations such as dropout, layer normalization and activation function in the product space [0] Li::
89
(x) = (V]Boo(()\/lll [o(()))))
3, a())))))
(13)
where = u) / x LL with e s and E Lg. The product refining function o operates independently on the spherical and hyperbolic manifolds, similar to linear attention, which ensures that the output remains within the product space.
4.3 Pseudo-Riennne Riemannian residual connection
Residual connection aseudo-Riemanniar methods requires the definition of vector addition in pseudo- SeauooRRemmnnian space. Xiong et al. (2022) suggested using parallel transport with antipodal points as vector addition in the secudo-hyperbolott Q However, this approach alters node features by reversing their original representation, making it unsuitable for preserving feature integrity. Therefore, we define vector addition follows:
5

---

Definition 2 Eseudo-hyperbolde vector addition). Given two vectors I, E D the pseudo- hyperboloid vector addition is defined as:
8 y t2 x exP (y))
(P7 (o (logo (ogg(()()))) (+())))),
(14)
where the extrinsic mapping function in Eq. (3), exp² and log are the exponential map and logarithmic map respectively, P is the parallel transport, ô = Ol.,,. is the south pole in the manifold Q and the output 2
The procedure for vector addition on the pseudo-hyperboloid is similar to the bias addition in hyperbolic space (Chami et al. (201)). With the extrinsic mapping introduced in Section 3 we enable the application of logarithmic map and parallel transport while avoiding geodesic disconnections and preserving the original features. We further provide the closed form of the vector addition @g the following proposition:
Proposition 2. Given Iwo vectors y E 2 the closed form of pseudo-hyperbooood vector addition is expressed as:
2 = y G =()==(()) + (0(x), 1 ((y))
(0+ D())).
(o+
|8
4.4 Overall architecture
Fig. D Iillutrrates the architecture and data flow of the proposed method Q-GT. During preprocessing. we apply the strategy described in Section find the appropriate pseudo-hyperboloid for embedding the input graph. Once the manifold is selected, the input features x are projected onto O .9" By using the functions Q and V in Eq. (3) and Eq. 4 the projected features are further mapped
to a product space (1/) Lg- The resulting node representations x are processed by the Q-GT module capture global relations between nodes. Meanwhile, graph convolutional module Q-GCN2, which is designed based on our diffeomorphic framework, extracts intrinsic graph structural information of the input. Finally, the outputs of Q-GT and Q-GCN2 are aggregated to produce the final graph representation. The details of Q-GCN2 is provided in the Appendix G
5 Space searching algorithm
pseudo-byperboloid is defined by two key hyperparameters: the number of time dimensions g and the number of space dimensions p. Given fixed total dimensionality d = p + 4 + 1 for representing node features, varying p and alters the geometric properties of the pseudo-hyyerboloid. Since high-dimensional bseudo-hyperboloide are difficult to visualize directly, their geometric properties can be more easily interpreted by mapping them to the product of spherical and hyperbolic spaces using our diffcomorphic framework. When p 0, the manifold Q2.4 becomes sphere $%B which is
well-suited for embedding cyclical graphs. As p increases, the topology of Q transitions into product of hyperbolic and spherical geometries, which is appropriate for mixed-structural graphs. When p reaches its maximum value of d 1, 2 becomes hyperbolic manifold L2 which is effective for modeling tree-like graphs. Once the correlation between the input graph's 's topology and the space dimension p is known, we can determine the appropriate manifold.
In our work, we rely on the GSC distribution (i, g²) (Bachmann et al. 2020), which provides comprehensive view of how curvature is distributed across the entire graph, reflecting the geometric complexity of the input graph. We hypothesize that the GSC distribution N(ū o²) of input graph is closely related to the topological characteristics of the pseudo-hypene hhperbooyye Q Therefore. we propose a simple yet effective space searching algorithm to choose a value of p (or q) based on the GSC distribution. Specifically, we define a bijective function I' that maps each seeudo-hyperboloid O to its corresponding ideal GSC distribution. N N(44, of). To identify the most suitable manifold,
we iteratively compute the KL divergence between the input GSC distribution W ((ī, and each ideal GSC distribution. We then select the hyuudd-ooeeboooid associated with the ideal distribution N (pp of) that yields the smallest KL divergence. This procedure is illustrated in Fig. [ as the 1p..%)i=d-
choosing manifold step. Given collection of pseudo-hyperboloids (2):%) ji=d1 satisfying that

---

Table The node classification results in Fi-score. The best results are in bold and the second-best results are in underline for each dataset. OOM denotes out of memory during training or testing.
Darasets
Methods
Cora
Cileseer
Airport
Puhmed
Andv
Penn94
Twitch Gamers
GCN
81.87 = 0.52
70.73 + 0.61
74.62 + 0.56
79.73 0.31
$7.39
0.54
80.92 + 0.57
63.73 + 0.11
HGCN
76.57 0.90
65.60
0.64
83,46 + 0.94
77.09
0.26
62.16
0.36
82.04 + 0.01
61.65 + 0.94
x-GCN
73.83 + 0,56
0.771
80.60 + 0.39
75.00 = 0.58
58.36 + 0.18
72.81 + 0.17
61.81 + 0,72
Q-GCN
0.51 67.80
76.12
0.22
82.68
0.14
63.53
Q-OCN2
82.60 0.24
71.63
0.29
91.66 +: 0.45
80.13 = 0.26
64.93
0.32
82.60 1 0.07
63.77 1 0.06
GraphGPS
73.76 +: 1,31
61.07
0.40
89.31 +: 0.95
71,53 +: 0.74
OOM
00M
0OM
NodeFormer
80.27 2: 0.83
70.03
0.31
88.87 1: 0.36
81.30 : 0.65
53.37
0.34
77,80 1: 0.11
63.97 1 0.17
SGFommr
82.00 1: 0.51
71.93
0.59
93.32 1.09
78.97 0.09
61,38 0.25
78.45 1. 0.19
63.73 1. 0.10
Hyplormer
75.13 0.69
67.77
1: 0.41
89.19 + 0.50
78.93 = 0.21
64.62 1 0.22
78.13 +: 0.69
63.16 + 0.33
FPS-T
60.50 - 0.40
60.70 0.30
91.41 +: 0.17
76.23 = 0.31
0OM
00M
00M
QGT
70.40 0.49
95.70
0.05
82.77 +
64.45
pi + = d 1, 1, where d is fixed total dimensionality of embedding space and d - 1 is an even number greater than 2, a bijective function I' N + NNu,, o²) is defined as:
N (-1ah 1,4, (AmmA (amax mmin)
(22-11411)
if (d 1)/2 < Pi sd-1
d-3
I(p.) = NN 1),
pf = (d - -1)/2
2
N [a), (Omin + (amux mmin
2p1 3
if S ps < - 1d/1
(15)
Since qi = 1 - Pi, the function can be defined in terms of ps alone. The mean μ4: is derived from the practical mean J., while the standard deviation a; is assumed vary linearly with the space dimension pi- More details about the space searching algorithm are in Appendix[C] In general, the ideal standard deviation may follow non-linear relationship concerning the space dimension. We leave the exploration of this to future work.
6 Experiments
We evaluate the performance of our proposed Q-GT model on node classification and link pre- diction tasks, and conduct a comprehensive comparison against wide range of prate-of-the-ar non-Euclidean GCNs and graph transformers. Our implementation is available at the GitHub reposi- tory
6.1 Node classification
Datasets. We choose seven widely used graph datasets for evaluation, including four citation networks Cora (McCallum et al., 2000), Citeseer (Giles et al. 1998), Pubmed (Sen et al. 2008). and Arxiv (Hu al. 2020); two social networks Penn94 and Twitch Gamers (Lim al. [021): b; and one transportation network Airport (Xiong et al.] (202). For the node classification task, we follow the standard dataset splits used by Hu al. (2020); Xiong al. (2022): : [im et al.) (202I). Summary statistics for all seven datasets are provided in the Appendix [] Notably, the graph sectional curvature (GSC) distributions across most datasets are consistently positive, except Citeseer, where the mean of the GSC distribution is close to O.
Baselines. We compare the performance of Q-GT against state-of-the-art GNNs and graph Trans- formers operating in both Euclidean and non-Eucliden spaces. The GNN-based baselines include GCN (Kipl' and Welling 2017). HGCN (Chami et al. 2019), x-GCN (Bachmann et al. 2020).
Q-GCN (Xiong et al.) 2022), and our Q-GCN2 which is a GCN variant built upon our proposed diffeomorphic framework. The Transformer- based baselines consist of GraphGPS (Rampášek et al.) 2022). NodeFormer (Wu et al. 2022). SGFormer (Wu et al. 2023). Hypformer (Yang et al. 2024). and FPS-T (Cho et al.) 2023). To ensure fair comparison, we set 16-dimensional hidden embedding for all models. For product-space models such as k-GCN and FPS-T, we constrain 2 components on their respective product manifolds. For Q-GT, Q-GCN2, and Q-GCN, we apply the space searching algorithm in Section
5 Each model is trained for up 1000 epochs with early stopping applied.
7

---

Results. Table 1 presents the node classification results in terms of Fl-score. Our proposed Q-GT consistently outperforms competing methods on out of 7 datasets. The most notable improvement is observed on the Airport dataset, where Q-GT achieves performance gain of 2.55% in Fl-score. On non-homophilous datasets such as Penn94 and Twitch Gamers, where Transformer-basse baselines typically underperform, Q-GT demonstrates stable and robust results. For instance, on Penn94, Q-GT surpasses the second-best Transformer (SGFormer) by 5.51% in Fl-score. Moreover, by leveraging linear attention mechanism, Q-GT scales effectively to large graphs (such Arxiv and Twitch Gamers), while GraphGPS with the standard attention encounters the out-of-memory (OOM) problem. Q-GCN2 also delivers strong performance, ranking second on three datasets (Cora, Citeseer, and Arxiv) and exceeding Q-GCN on most benchmarks. It is worth noting that the Transformer -ased baseline FPS-T TPS-TTN is similar Q-GT, as both operate in product spaces of Riemannian manifolds. However, FPS-T suffers from significant computational overhead due to its reliance on tangent mappings and expensive Laplacian eigenvector computations, leading out-of- issues three large-scale datasets. On the node classification task for Cora and Citeseer, FPS-T underperforms Q-GT, with learned curvatures converging to near-zero values (e.g. {0.023, 0.025) Cora and {0.008, 0.007} on Citeseer), which fail to capture the underlying hyperbolic geometry In contrast, Q-GT benefits from operating directly on the product of spherical and hyperbolic manifolds, which is diffeomorphic to the pyrrboood,, thereby enhancing its performance.
6.2 Link prediction
Setup. For the link prediction task, we evaluate performance on four datasets: Cora, Citeseer, Pubmed, and Airport. Each dataset's edges split into 85%, 5%, and 10% for training, validation, and testing, respectively. The baseline configurations are the same as those in Section 6.1
The results of link prediction are shown in Table Q-GT obtains the highest score on every dataset, with par- ticularly high margins on Cora (96.87% in AUC) and Citeseer (97.59% in AUC), outperforming strong transformer baselines like NodeFormer, SGFormer and FPS-T. Q- GCN2 also demonstrates competitive per- formance, ranking second Citeseer and Pubmed. Furthermore, both Q-GT and Q- GCN2 significantly surpass Q-GCN across all datasets. For example, Q-GT and Q- GCN2 achieve gains of 43.37 % and 42.81 % in AUC score on the Airport against Q- GCN. These results underscore the effec- tiveness of our diffcomorphic framework in preserving structural information critical for accurate link prediction.
Results.
Table 2: The link prediction results in ROC AUC (%). The best results are in bold and the second-best results are in underline for each dataset.
Datasets
Methods
Cora
Citeseer
Airport
Pubmed
GCN
80.75 = 0.23
86.12 : 0.24
88.92 1: 0.30
92.62 :: 0.20
HGCN
94.11 = 0.18
96.11
0.11
92.85
0.29
92.21 + 0.08
R-GCN
94.29 1: 0.10
95.46
1 0.07
85.50
0.119
87.63 1: 0.19
Q-GCN
94.45
0.08
96.19
0.20
67.35
0.775
85.95
+ 0.65
Q-GCN2
95.93 1: 0.07
97.25
1: 0.05
95.61
1: 0.05
95.45
1: 0.19
GraphGPS
92.60 0.17
96.04
0.12
94.40
1 0.04
94,00
1 0.10
NodeFFormer
0.13 94.04
95.67
0.02
94.22
94.39
SGFormer
03.20=
95.74
= 0.08
93:96
0.100
92.10 = 0.20
Hypformer
9..99+00.30
96.83
0.12
96.18
+ 0.04
94.59 0.17
FPS-T
95.25
97.17 = 0.44
96.10
1 0.13
92.11 + 0.04
Q-GT
96.87
97.59
0.11
96.56 + 0.07
96.40 = 0,10
6.3 More ablation studies
Scalability. To further assess the scalabil- ity of our model, we conducted additional evaluations of Q-GT and other strong base- lines on two enormous datasets from the Open Graph Benchmark: Products and Ves- sels (Hu et al. (020)), where the results are shown in Table 3 This experiment is per- formed with all Transformer models con- figured with layer, GNNs with 3 layers, and training conducted over 200 epochs. It obvious that Q-GT achieves the highest performance on both datasets, outperform- ing the strongest baselines by 7.3% on Products and 32.09% on Vessel. However, as shown in Appendix H.2 about model complexity, Q-GT introduces higher computational and memory costs
Table 3: Comparison on Open Graph Benchmark datasets. Products is used for the node classification task, while Vessel is used for link prediction.
Datasets
Methods
Products
Vessel
Q-GCN
29.56 + 0.31
59.64
0.37
Q-GCN2
52.08 + 0.45
50.21 :: 0.36
SGFoomer
65.47 + 10.55:0.54
50.03 0.26
Hypformer
49.87 0.49
Q-GT
70.25 1 0.27
78.78 0.50
8

---

25
mean
Airport
300
Link prodiction resuts sirpor
100
Node classtiiaation resutts on arportt
95
90-
Densroy
1.3
sore
85
[L-score
80
80
75-
73
0-GT
70-
05
70
Q-GCN
60-
65
Q-GCN
65-
1.0
8 dimwnwan
10
12
16
Sec Locilonaleee curvelure
Space
(A)
(b)
(c)
Figure Evaluation of Q-GT, Q-GCN2, and Q-GCN across varying space dimensions on the Airport dataset: (a) GSC distribution. (b) Link prediction performance. (c) Node classification performance. The red line denotes space dimension p" estimated by the searching algorithm.
Q-GCN embedding on cora
@ GCG2 embedding on cora
Q-GT embedding on cora
Degrae lacel
Lavel
Lavvll
Larvel
Lacel
Figure 3: Visualization of the learned embeddings for link prediction task on the Cora dataset, where the colors denote different levels of nodes. The levels are defined according to node degree follows: Level (1-5), Level (6-10), Level 3 (11-20), and Level (greater than 20).
compared other baselines. We recognize this as a limitation of our current approach and plan to investigate more efficient solutions in future work.
Space searching algorithm. To analyze the effectiveness of space searching algorithm, conduct an ablation study on the Airport dataset using various space dimensions E [0, 16). As shown in Fig. 2 the GSC distribution of Airport network is predominantly positive, indicating that the graph is best embedded in pseudo-hyperboloid with low space dimension. Notably, our space searching algorithm correctly identifies p as the optimal dimension, aligning with the highest point of performance in Q-GT and Q-GCN2. In contrast, Q-GCN achieves best performance in large-space dimensional manifold 2, 15.1 which is counterintuitive given the structure of the dataset. Nevertheless, Q-GCN still underperforms compared to Q-GT and Q-GCN2 in the optimal manifold. More experimental results of the space searching algorithm are provided in the Appendix H.I
Diffeomorphic framework. Fig. 3 visualizes the learned embedding on the Cora dataset within the product spaces of the estimated 0 Speccfiicallyy we omit the common spherical component used our diffcomorphism and in the work of Xiong et al. (2022). We instcad plot the hyperbolic features for Q-GT, Q-GCN2, and the Euclidean features for Q-GCN using the UMAP too We refer to nodes with degree Level as low-level nodes, while all others are considered high-level nodes. In Q-GCN, both high-level and low-level nodes form single large cluster, meaning that these nodes are oversmoothed in the Euclidean space. In Q-GT and Q-GCN2, high-level nodes are more dispersed, forming multiple small groups that include only few low-level nodes. These findings suggest that the hyperbolic component in our diffeomorphic framework effectively captures local hierarchical relationships among nodes and separates them into distinct clusters. We further evaluate the effectiveness of the diffeomorphic framework on toy tree-like graph in the Appendix H.3
To fully assess our proposed work on downstream tasks, we perform an ablation study in the Appendix H.4 to analyze the impact of each key component in Q-GT.

---

6.4 Further discussions
Limitations. The proposed graph models (Q-GT and Q-GCN2), along with the space searching algorithm, are currently limited to the single static graph setting. As result, they are not directly applicable graph-level datasets, such as Long Range Graph Benchmark (Dwivedi et al.) 2022), or to dynamic graph scenarios. Our space searching approach is based on the assumption of linear relationship between the ideal standard deviation and the space dimension, which requires further theoretical analysis to validate its correctness. Since Q-GT is built upon the Transformer architecture, incurs higher computational and memory costs compared to GNNs (see Appendix H..).
Broader impacts. This work represents a theoretical advancement in the field of manifold learning in graphs. There could be some potential societal consequences of our work. For example, our work can be implemented in various graph learning applications in domains such as recommender systems and social networks, which we left for future analysis.
7 Conclusion
In this paper, we propose a novel diffeomorphic framework that decomposes eseudo-Riemanee manifolds into the product of spherical and hyperbolic spaces, preserving key geometric properties of Riemmnnnn manifolds. Based on our framework, we successfully generalize Graph Transformer to heudo-Riemannian manifolds, achieving promising results in graph learning tasks. Besides, by analyzing the properties of pseudo-Riemannin develop a space searching algorithm to determine an embedding manifold effectively. Experimental results real-world datasets, especially in the link prediction task, demonstrate that our model outperforms both Riemannian and pseudo-Rieman Riemannian baselines. Based on the results, it is possible to extend our framework to other settings such as graph-level learning, dynamic graph learning or in more generalized domains.
manifolds with space dimension, we
Acknowledgments
This material is based upon work supported by the Air Force Office of Scientific Research under award number PA2386-24-114401
References
Boguñá, M.; Bonamassa, L,; De Domenico, M.; Havlin, S.; Krioukov, D.; Serrano, M. Network geometry. Nature Reviews Physics 2021, 3, 114-135.
Gu, A.; Sala, F.; Gunel, B.; Ré, C. Learning Hixed-Curvature Representations in Product Spaces. International Conference on Learning Representations. 2018.
Bachmann, G.; Bécigneul, G.; Ganea, Constant curvature graph convolutional networks. Interna-
tional Conference on Machine Learning. 2020; 486-496.
Law, M. Ultrahyperbolic neural networks. Advances in Neural Information Processing Systems 2021, 34, 22058-22069.
Laub, J.; Müller, K.-R. Feature Discovery in Non-Metric Pairwise Data. J. Mach. Learn. Res. 2004, 5, 801-818.
Law, M.; J. Ultrahyperbolic representation learning. Advances in neural information processing systems 2020, 33, 1668-1678.
Xiong, B.; Zhu, S.; Potyka, N.; Pan, S.; Zhou, C.: Staab, S. Pseudo-riemannian graph convolutional networks. Advances in Neural Information Processing Syslems 2022, 35, 3488-3501.
Ganea, O.; Bécigneul, G.; Hofmann, T. Hyperbolic neural networks. Advances in neural information
processing systems 2018, 31.
Chami, I.; Ying, R.; Ré, C.; Leskovee, Hyperbolic Graph Convolutional Neural Networks. Advances in neural information processing systems 2019, 32, 4869-4884
10

---

Li, Q.; Han. Z.; Wu, X.MM. Deeper insights into graph convolutional networks for semi-supervised learning, Proceedings of the AAAI conference on artificial intelligence. 2018.
Oono, K.; Suzuki, T. Graph Neural Networks Exponentially Lose Expressive Power for Node Classification. International Conference Learning Representations 2020,
Chen, D.; Lin, Y.; Li, W.; Li, Zhou, Sun, X. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Proceedings of the AAAI conference on artificial intelligence. 2020; pp 3438-3445.
Wu, Z.:; Jain, P: Wright, M.; Mirhoseini, A.; Gonzalez, J. E.; Stoica, I. Representing long-range context for graph neural networks with global attention. Advances in neural information processing systems 2021, 11..,441266100011 13266-13279
Cho, S.; Cho, S.; Park, S.; Lee, H.; Lee, H. H.; Lee, M. Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning. ArXiv 2023, abs/2300900008.
Yang. M.: Verma, H.; Zhang, D. C.; Liu, J.; King. I.; Ying, R. Hypformer: Exploring efficient transformer fully in hyperbolic space. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024; pp 3770-3781.
O' neill, B. Reni-Riemannian geometry with applications to relativity; Academic press, 1983; Vol. 103.
Sun, K.; Wang, J.; Kalousis, A.; Marchand-Maillet. S. Space-time local embeddings. Advances in Neural Information Processing Systems 2015, 28.
Wu, Q: Zhao, W.; Li, Z.; Wipf, D. P:; Yan, J. Nodeformer: A scalable graph structure learning transformer for node classification. Advances Neural Information Processing Systems 2022, 35, 27387-27401.
Wu, Q.: Zhao, W.; Yang, C.; Zhang, H.; Nie, E; Jiang, H.;; Bian, Y.; Yan, J. Sgformer: Simplifying and empowering transformers for large-graph representations. Advances in Neural Information Processing Systems 2023, 36, 64753-64773.
McCallum, A.; Nigam, K.; Rennie, J. D. M.; Seymore, K. Automating the Construction of Internet Portals with Machine Learning. Information Retrieval 2000, 3, 127-163.
Giles, C. L.; Bollacker, K. D.; Lawrence, S. CiteSeer: An automatic citation indexing system. Proceedings the third ACM conference on Digital libraries. 1998; pp 89-98.
Sen, P.; Namata, G.; Bilgic, M.; Getoor, L.; Galligher, B.; Eliassi-Rad, T. Collective classification in network data, Al magazine 2008 29, 93-93.
Hu, W.: Fey, M.; Zitnik, M.; Dong, Y.: Ren, H.; Liu, B.; Catasta, M.; Leskovee, Open graph benchmark: Datasets for machine learning graphs. Advances in neural information processing systems 2020, 33, 22118-22133
Lim, D.; Hohne, F.; Li, X.; Huang, S. L.; Gupta, V.; Bhalerao, O.; Lim, S. N. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. Advances in neural information processing systems 2021, 34, 20887-20902.
Kipf, T. N.; Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. Inter- national Conference on Learning Representations. 2017.
Rampašek, L.; Galkin, M.; Dwivedi, V. P.; Luu, A. T:: Wolf, G.; Beaini, D. Recipe for a General,
Powerful, Scalable Graph Transformer. Advances in Neural Information Processing Systems. 2022; pp 14501-14515.
Cho, S.; Cho, S.; Park, S.; Lee, H.; Lee, H. H.; Lee, M. Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning, ArXiv 2023, abs/2309.040s
Dwivedi, P.; Rampášek, L.: Galkin, M.; Parviz, A.; Wolf, G.; Luu, T.; Beaini, D. Long range graph benchmark. Advances in Neural Information Processing Systems 2022, 35, 22326-22340.

---

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; Polosukhin, I. Attention all you need. Advances in neural information processing systems 2017, 30.
Katharopoulos, A.; Vyas, A.; Pappas, N.; Fleuret, Transformers are mns: Fast autoregressive transformers with linear attention. International conference on machine learning. 2020; pp 5156- 5165.
Krioukov, D.; Papadopoulos, F:; Kitsak, M.; Vahdat, A.; Boguñá, M. Hyperbolic geometry of complex networks. Physical Review E 2010, 82.
Lee, J. M. Introduction to Riemannian manifolds; Springer, 2018; Vol. 2.
Law, M.; Liao, R.; Snell, J.; Zemel, R. Lorentzian distance learning for hyperbolic representations. International Conference on Machine Learning. 2019; pp 3672-3681.
Kingma, D. P.; Ba, J. Adam: A Method for Stochastic Optimization. International Conference on Learning Representations 2015, abs/1412.6980.
Robbins, H.; Monro, S. A stochastic approximation method. The annals of mathematical statistics 1951, 400-407.
Becigneul, G.; Ganea, O.-E. Riemannian Adaptive Optimization Methods. International Conference Learning Representations, 2019.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly stated our claims in the abstract and introduction, reflecting the scope and contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: In Section 6.4 we list three limitations of our work. which the restriction
the single static graph setting, computational overhead, and numerical instability.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
12

---

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide the proofs for Theorem Proposition 1 and Propositior in Appendices and
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification; For reproducibility, implementation details are included in Appendix | and the code will be released publicly upon acceptance of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same
13

---

dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis-
sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (c.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-souree models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The source code has been submitted along with the paper, but will remain private until the work accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
httee//uuddeeecdeuueeaeee www.bmmmsssooooiiiii.oocc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips
www.oooooidissiooooooooooo sww.uiooiisioooooooccooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental setting is given in Section 6.1 to 6.3 and Appendix I
14

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports the mean and standard deviation for several experiments
(Sections 6.1 6.2 H.3
H.4)
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The GPUs used for performing the experiments detailed in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpppp///eorrre.cccopp/////aaas
15

---

Answer: [Yes]
Justification: Our work conforms to the NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the societal impacts of our work in Section 6.4
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeet of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not have such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
16

---

12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Every baseline method and dataset is cited in our paper. Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papprswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
17

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The development of our work does not involve LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
18

---

Technical Appendices and Supplementary Material
A
Notation
Table 4 summarizes all notations mainly used in this work and their corresponding explanations.
Table 4: Main Notations
Notations
Descriptions
2
A pseudo-hyperboloid with + time dimensions, p space dimensions, and space curvature 3
2
The zero-first submanifold of 2
The inner product in Euclidean space The inner product in pseudo-Fuuliddan space The vector norm in Euclidean space The vector norm in Suuudd-e space
G (Va, EG)
A static graph G' with the node set Vo and the edge set Ec A single vector or point
T. U. 2, u, u.
X
A set of vectors
N
The number of vectors in vector set X The tangent space point x in a manifold M
T.M
exp, log
The exponential and logarithmic maps
P2-
The parallel transport from T to Ty2 along the geodesic y
(3,, w). (2(
The distance between points 2;
d
The embedding dimension
R
The Fuclidean space
N
The natural number set
S
The spherical space
L
The hyperbolic space or the Lorentz model
The extrinsic mapping
V
The diffeomorphic function
N(u,
The Gaussian distribution with the /u and the variance
o²
f
The probability density function of the Gaussian distribution
r
The bijective function to Gaussian distributions
B
Related works
Graph embedding Euclidean space. Euclidean space has long been the foundation of deep learning because supports fundamental linear algebraic operations such vector addition and matrix multiplication. As a result, many early graph representation learning methods were developed within Euclidean geometry and achieved notable successes. One of the pioneering approaches the Graph Convolutional Network (GCN) introduced by Kipf and Welling (207)), which enabled deep learning-based graph embeddings. To address GCN's limitations in depth and its restricted capacity to capture long-range dependencies, Transformer architecture have been adapted (Vaswani et al.) 2017) for graph data. For example, GraphGPS (Rampášek al., (0222) integrates local message passing with global attention in a hybrid architecture. Despite their superior performance over graph neural networks in small-scale graph applications, Transformer-based methods often struggle with scalability. (Wu et al.) 2022 2023). Standard attention computes pairwise interactions between all node pairs through softmax function, resulting in quadratic computational complexity. alternative approach is utilizing linear attention (Katharopoulos et al., 2020), which offers linear computational complexity, for large-scale graph learning. For instance, NodeFormer (Wu al. 2022) introduces a novel kernelized Cumbee-Sofftax standard attention with linear complexity. SGFormer (Wu al. (003) demonstrates the surprising effectiveness of single-layer, single-head linear attention module in node-level tasks. Leveraging
pessage-passing mechanism to approximate the

---

the advantages of linear attention, we propose a pseudo- Rimemnnia graph model Q-GT built upon the linear Transformer architecture to ensure scalability and efficiency.
Graph embedding in non-Fuclidean space. As data becomes increasingly complex and diverse, the limitations of Euclidean geometry in graph representation learning are becoming more evident. For instance, Krioukov et al. (2010) proved that hyperbolic geometry is better suited for embedding scale- free and hierarchical graphs due its exponential expansion property. Consequently, Chami et al. (2019) extended the classic GCN to hyperbolic GCN by shifting the graph operation to the tangent space for executing vector operations. Yang et al. (2024) developed a scalable and robust hyperbolic Transformer called Hypformer, which employs the linear attention mechanism fully on the hyperbolic manifold. For modeling graphs of mixed topologies. K-GCN (Bachmann et al.) 2020) generalized GCN to the Cartesian product of constant curvature manifolds. However, computing explicit distances in the product of manifolds can introduce bias toward one component over the other, thereby restricting the model's 's representational capacity (Law (021). Alternatively, pseudo- SecudooRRieanne Riemannian geometry, which generalizes spherical and hyperbolic geometries, can address the weakness of product spaces with its intrinsic distance (Law) 2021; Xiong et al.) 2022)). Law and Stam (2020) proposed the first nonparametric method to learn graph representations with a novel optimization tool on pseudo- Riemannian manifolds. Then Law (202T) designed a graph neural network on the quotient manifold, manifold to avoid geodesic disconnection. On the other hand, Q-GCN (Xiong et al. (022) introduced diffeomorphism that decomposes pseudo- Riemannian manifolds into the product of spherical and Euclidean spaces, enabling a lightweight geometry. Nevertheless, existing methods suffer from three notable limitations: (1) the diffeomorphism to the product of spherical and Fuclidean spaces discards hierarchical information inherent in the graph structure; (2) the GNN architecture is prone to the oversquashing problem as the number of layers increases; and (3) previous methods rely which is computationally
which is a submanifold of the Eseudo-Riemamee
extension of GCNs to escudo-Riemannin
on brute-force algorithm to select the appropriate pseudo-hyperboloo expensive. To address these limitations and attain better outcomes, we propose Q-GT, which is the first Transformer-based graph method in pseudo-Riemanne Riemannian geometry.
C More details about space searching algorithm
C.1 Gaussian sectional curvature
Sectional curvature is a fundamental concept in Riemannian geometry that describes how manifold
curves in different two-dimensional directions (Lee 2018). Given a point on manifold M and two linear independent tangent vectors u, v E T spanning
wwo-dimensional subspace U, the sectional curvature measures the Gaussian curvature of the surface Exp(U) S M formed by exponential mapping tangent vectors within U. Intuitively, it provides insight into the local geometric properties around a point xr (Gu et al., 2018), where positive sectional curvature indicates a locally spherical structure, zero curvature corresponds to Huulideen-llie space, and negative curvature signifies hyperbolic structure. In the context of graphs, et al. (2018) proposed a discrete curvature analog i to estimate the section curvature in an undirected graph G. Given node m E G. for any two neighbor nodes b, e and a reference node a + m, the curvature analog x(m; b, c) is defined as:
x(m; b,c) VVl
1
Σ aEG.aglm
(dc(a,m) m) + cb.(,,)) Ba(a, +
((bla,) d2((a.)) 4dc(a, (alttt.)))..))
(16)
8dg(a, m)
where VVl is the number of nodes in the graph G. da(a, b) is the length of the shortest path between nodes and b. The sectional curvature x(m) assigned to point m is determined by averaging curvature analogs x(m;; b, c) over different neighbors (b, c). In practice, sampling on the neighbors b, c and reference point is applied to reduce computational overhead (Gu et al.) 2018 Bachmann al. 2020).
To assess the topological structure of the graph G. we compute the Gaussian Sectional Curvature (GSC) distribution N (z, x²) (Bachmann et al. 2020). In particular, the mean jı and the variance d² of N (u, are calculated as:
1 VVa
Σ MEG
x(m)),
1
(m((m)= - 11)2.
(17)
[Vol
mEG

---

GSC distribution (i, σ²) provides a comprehensive view of how curvature is distributed across the entire graph, reflecting the geometric complexity of the graph. The mean curvature fı captures the fundamental structural properties of the graph (i.e., whether it is predominantly hierarchical or cyclical), while the variance o² measures the degree of topological heterogeneity. A negative mean < O suggests more hierarchical structure, favoring embedding in a hyperbolic manifold, whereas a positive mean a > 0 indicates a more cyclical structure, which is suited for a spherical manifold. The variance o² further refines this interpretation: small variance implies a more homogeneous graph topology, closely aligned with either a hyperbolic or spherical space, while a large variance signifies more diverse topoloical structure comprising both cyclical and hierarchical parts.
C.2 Complexity
To ensure the efficiency of our strategy, we begin by uniformly sampling a subset of nodes AA such that |A| < |Vel- For each node a E A, we precompute the graph distance da b) for all node b E G. Consequently, the curvature analog in Eq. (16) is computed as an average over A when m $ A and over A.(m} when m A. The time complexity of our algorithm is O(IAA](Va) [Eal)+ [in]+nn]Jee na| AalVa + d), where n1 is the number of times we sample two neighbor nodes, |Eal is the number of edges in graph G, and d is the dimension of embedding space. In contrast, brute-force approach to selecting hyperparameters and q has a complexity of ((ineddA) where Rier denotes the number of training iterations, and A represents the model complexity. For instance, in a simple GCN (Kipf and Welling 2017). the model complexity is O(L( d + -Egld) where L is the number of layers, while in a standard linear Transformer (Katharopoulos our space searching algorithm scales linearly with d, it remains efficient even for high-dimensional embeddings. Meanwhile, the brute-force approach becomes computationally expensive when both d and Niter are large.
al. 2020). it ((Vgl Since
C,3 Analysis of bijective function I'
Constructing the bijective function I' that reflects the relationship between variations in the ideal GSC distribution and the structural characteristics of the pseudo-hyperboloid is non-trivial task. To facilitate this construction, we analyze the function I under three different geometric cases, which balanced, spherical, and hyperbolic. In the balanced case, the mean curvature μu is set 0 when the input graph exhibits the most uniform structure, containing equal proportions of cyclical and hierarchical topologies. This structure is best embedded in the balanced manifold Q where
p; 9i (d - 1)/2. The standard variance o; can take any arbitrary value, and we assign it to 1 for simplicity. In the spherical case, when the manifold 2% becomes more spherical by setting p: < (d - 1)/2 < qt. it best corresponds to graph with positive mean curvature, Therefore, we set [4, p where ji the estimated mean eurvature of input graph. The variance o². defined as Eq. 115 increases monotonically with increasing p;. This is because p; grows, the manifold 2 shifts toward more balanced structure, which is suited for a uniform topological graph with large variance in its GSC distribution. Notably, the standard deviation o, reaches its minimum value amin when p. 0 (a fully spherical manifold), and its maximum value dmax when
(d-11) (d-1)/2
We determine max such that the probability density function at zero curvature, f(0), is nearly equal to [(4) for the highest variance, while min ensures that f() & [(1,) for the lowest variance. In practice, we assign min = 0z5| and Omax = 5lpl.. Similarly, we define the GSC distribution N(4, of) for the hyperbolic case in Eq. (15) where p. > (d - 1)/2 qi- The key difference is that, unlike the spherical case, the mean curvature μ4 is negative and the standard deviation o, decreases monotonically as pt increases.
D Geodesic tools of pseudo-hyperboloid
D.1 Parallel transport
Given two points x and y that are connected by a geodesic 7 on the manifold 27.,, E is a vector in the tangent space T, 2p4 the parallel transport is formulated by Law (021):
(E.8), 4)
(E) = 5- (z, (),
(u + x).
(18)
|3
3

---

D.2 Distance
When z, 6g are geodesic-connected the induced distance between and y in pseudo- hyperboloid is given by dz y) = V Alllog (1)llg. However, in cases where log, (y) is not defined due to geodesic disconnection, Xiong et al. (2022) proposed an approximation using midpoints. Specifically, the antipodal point 2 serves an intermediary for the broken geodesic between z and y, leading to the distance approximation:
{ dy (x, v), (x,
if (r, 4) < | if (x, 2 [
(19)
( (2, -x) d,(-2,9) /
/ + d,(-x.y).
D.3 Exponential and logarithmic maps
The closed form expressions of two tangent maps exp / and log (3) proposed by Law and Stam
(0220):
vl
VIAI
VIKE.S>al v1@1
cosh
sinh
5.
if (5, E)g > 0
VIøI
(E)
x: + 5,
if (), 0
(20)
VIAl sin Vl
Vile.ssal vIØl
Cos
Vipl
r
+
&.
if (E,E), < 0
cosh
(axvin
(2.0)a
if
(x.yaa [3]
-1
(xvwiig
log (y)
y
It,
[P]
(21)
cos
y
if
(x.wia
E (-1,1)
where & Tz QP.9 is tangent vector and is point on the manifold Q
E Proof of Theorem I
Theorem 1. For any point a E 2g there exists a diffeomorphism V x E 2g - 1/gs L.
that maps onto the product manifolds of a sphere and hyperboloid. The mapping and its inverse are given by:
((x)
|
(2)
VIAl w v/1.)
(22)
(0)
where x
E gua+1 with Ro+1 [0], v E Ro, and
2 r'
with u' E Shep of Lg.
Proof. It is easy show that v and v-1 smooth since their elemental functions "h u', 2/11) are smooth with u E Re+l {0}. e RP. u' e Sy o' Lg. We only need to prove that (z) z and & (--((((())) In the first term ((V (z)), we have:
{ | [|u|] v, Vis
VIST m'
)
13
-ाशT will
V (x)- = v
Vie
Vi w'll
ViST
(23)
of1

---

Since u' E Ep/ we have || = v131. Eq. 23
becomes:
u
(z)) =
v/
11
2'
2.
(24)
[1:]
For the second term (v)xx,,, we have:
V (V(()) 4-1
v [3]
|all
x
(25)
F Proofs of Propositions 1 and 2
Proposition 1. The extrinsic mapping function & and its inverse 4- are smooth, bijective, and isometric.
Since both the extrinsic function and its inverse 4- are linear, they are inherently smooth mappings.
Proof.
We also have + +-1(((()) and (v) y. which proves the bijective properties of two functions.
To prove the isometric property, let æ, are two points pseudo-hyperboloid 2, we have:
(x, y), = (2x)), (2))) and |(
(z.
y)g x
)| = (v(v) (0(x), ((y)) ((x)
B
8
9+1
dx,x,)) d, (zxz). ((y)) when and y geodesic-conneeted. In the case of geodesic disconnection, it is easy to show that d- y) = (xx) (w))). leading to the (x, = (2()(z), ((y) where D., the ap- proximated distance in Eq. (19) Therefore, 4 is isometric. The isometric property proof for the inverse function $ similar.
Hence, have ( [|log (P(v))lla+z
Proposition 2. Given two vectors x, Q-9 the closed form of peeuo----yyee vector addition expressed as:
Z = y y (x) (0(x), 4 9+1 (ō (2))).
[
Proof. Beginning from the Definition we have the formula of vector addition:
exp (PT ( (log (()))))).
Since (B(x), = 0, following the definition of logarithmic map in Eq. 21 we have:
(d))) = π ((x)..
2
Applying the parallel transport P (o) in Eq. 18 we have:
(x)) + (x),, +
(
(z)
2(v))
π
B

---

Now we calculate the squared norm of ç:
(5:5))+1= = (b(x) + (a))), 2 (w))
(0(x), ( (z)))+
(ō p(v)), x(x)
9+
(0 + (1v)))
[8|
(()( & (2))
(a, 4/
(a. Vy.
+ (x +
x
V/4
|8
[3]
1331
(1x
(2r, u29
(ar, y
+ (1,
(y. U)n +
(x, 4)
|8|
181
18]
((x,, y)g)
(2r, U/g
2
(0r, 4)9 (x. 3)
+ +3+
+
|38
|3]
|8]
(x..)ig
= B < (c, y) 9)
+ 3
|3]
[,3]
|
Since the squared norm
(25, 7 25)g+1 = =0)) < 0. following the definition of exponential map in
Eq. 20), we have:
(xpo:s)(220 E.
This finishes the proof.
G Pseudo-Riemannian graph convolutional network
This section describes the useudo-Riemannian graph convolutional network, Q-GCN2, which builds upon the diffeomorphic framework introduced in Section Q-GCN2 comprises three fundamental operations: linear transformation, neighborhood aggregation, and nonlinear activation. As the linear transformation and nonlinear activation have been detailed in Sections [.] and 4.2 respectively, we focus here on the formulation of neighborhood aggregation in geeudo-Riemannn geometry. We then present the complete structure of Q-GCN2.
G.1 Pseudo-Riemanniar neighborhood aggregation
Neighborhood aggregation in GCNs (Kipf and Welling 2017 Chami et al. 2019) relies on scalar
multiplication, an operation that has not been defined the pseudo-hyperboloid Q To address this, instead formulate the neighborhood aggregation the product of spherical and hyperbolic manifolds by using our diffeomorphic framework. Given the neighborhood. (N of node and their
features in the product space {zz Sia xL (Je(()) where each node vector z. L.)T, and set positive weights {wi,5 bear()) we propose the neighborhood aggregation at node i as follows:
8% xL
(uj E Sig 2y E
(wr,s)) = v181( ΣjEN()) Wiy E)jeN()) Wi, VII E ESENO) Wi
ΣjEN()) Wiyey
8in x
8% x
Agg( L2
(26)
where ||=11| I)L is the Lorentz squared norm. The aggregating function in Eq. 26 is the extension of a hyperbolic centroid formulation proposed by Law et (0199). Since the squared distance in the product space can be decomposed to the of squared distances in the component space (Gu er al. 2018), i.e. || - x, Ilxx1 || 2 ||= - =,y12, we can find the centroid that minimizes the weighted sum of squared distances in the product space by concatenating in spherical and hyperbolic spaces. In other words, our neighborhood aggregation is a centroid with respect the squared distance in the product space. We first calculate the spherical centroid from the feature set {u, JeN((), then combine with the hyperbolic centroid of set {vy (ey))))))
centroids
G.2 Overall structure
The development of the Q-GCN2 model direct consequence of our proposed diffeomorphic framework and the Transformer model Q-GT. With all key operations of Q-GCN2 introduced, now

---

summarize the structure of a single Q-GCN2 layer. Given the node features X % of input graph G, we follow the procedure in Q-GT to project them onto the product space, resulting in X & = (xxxx" ")) )). Next, the representations x X is sent to the message- message-passing
f.z 20 V=
structure in Q-GCN2 layer:
x
ii - (VIIIW/us//WWiu))
(V (V B,
(27)
Mai
x
Agg 1.f 1us.ss))eee.
2,
(28)
1/ xay 2.
Activation (2,
(29)
where W1 [8.0" (1)+ W2 E (x)))( are two transformation matrices, way ((deg(j)
(deg(()) + 1))-1)2 1/2 the normalized weight from GCN (Kipf and Welling 2017). and deg(i) the
degree of node i. Finally, the output X xLE = {a,
x
Vi=N are projected back to the pseudo-
Riemannian manifold by using our diffeomorphic framework, i.e. X Xe =o - (Xx%( XL )).
G.3 Integrating into Graph Transformer
Incorporating GCN to Graph Transformer well-known for graph representation learning (Wu et al.
2023; Yang et al. 2024), which enhances the local structural information. Let p" and g² denote the optimal space and time dimensions, estimated by our searching algorithm in Section 5 Given
the outputs of Q-GCN2 and Q-GT as 1, X₂ E O =2" we utilize this approach and combine two outputs with a balance hyperparameter a as follows:
X1 = log ( (x))) X2 = (x(22)
Ż = ax1 + (1 X2
(30)
Z = (Z) E 23 "n""""t
where & is the extrinsic mapping function in Eq.
3 2g .0 is the zero-first submanifold, expo
and loge are the exponential and logarithmic maps in Eq. (0) and Eq. 21 In practice, the value of
the balance weight a is determined by the grid search strategy.
H More ablation studies
H.1 Effectiveness of space searching algorithm
We further assess the effectiveness of the proposed space searching algorithm on Cora, Citeseer and Pubmed datasets, as visualized in Fig. 4 It is clearly seen that varying space dimensions models, especially in node classification
significantly impact the performance of iseudo-Riemanne task. This highlights the importance of selecting an appropriate ggeudo-hyperbolde graph representation learning. Our algorithm consistently identifies an optimal or near-optimal space dimension p" in both tasks. Meanwhile, the performance of Q-GCN does not align with the space searching result or the underlying topology of the input graph. While Q-GCN performs reasonably well when the product space becomes Euclidean space through increasing space dimension, Euclidean space lacks the capacity to capture the cyclical and hierarchical structures in many real-world graphs. This limitation leads to inferior performance compared to proposed Q-GCN2 and Q-GT.
for effective
H.2 Complexity comparison
For better complexity comparison, we report the training time and memory of our proposed Q- GCN2 and Q-GT against some strong baselines in Table and Table 6 A comparison between Q-GCN and Q-GCN2 shows that our diffeomorphic framework achieves better efficiency both runtime and memory. This is because Q-GCN framework relies on a combination of diffeomorphic and tangent space mappings product manifolds, whereas our approach operates directly within the product manifold space. Nonetheless, Q-GT incurs higher computational and memory overhead
7

---

COT8
100
Lif preciction results on care
90
Mode clasurication results on cora
1.4
mein
95
80
40
1.2
ABCC 200re (S)
25
70-
1.0
80
60-
75
0.8-
0
0.4
QC0C2
C.0222
0-Cch
0.0
6.5
1.0
10
14 16
7.0
Ciscest
mean
100
Lnn preziction readit dissser
Ncde: clasaification results ditsser
95
70
63-
2.0
AUC 200e
60
1.5-
05
6.
1.0
0-6T
50-
0-GCK2
0.5
75
1.0
-0.5
0.0
0.5
1.0
10
10
16
Carvature
Space dimresion
Space dimanzion
Urk predicton pubmed
dassil pubmed
135
AUC Gcore (N)
85
0.2
0.0
-1.0
-0.5
0.0
3.0
70
10 Space dimersiin
17
14
16
6 8
10
24
16
Sectional EUTVWTTLIN
Space dimonsion
Figure 4: Evaluating space searching algorithm on Cora, Citeseer and Pubmed datasets. The red line denotes space dimension pz estimated by the searching algorithm.
Table 5: Comparison of training time and the runtime of the space searching algorithm (in second).
Datasets
SGFormer
Hypformer
Q-GCN
Q-GCN2
Q-GT
Space searching
Ratio to Q-GT
Cora
40.92
66.42
136.6
82.44
193.7
12.70
0.07x
Citeseer
40.21
67.31
141.8
76.12
188.2
0.04x
Airport
71.55
97.20
211.4
66.59
249.5
15.30
0.06x
Pubmed
144.8
153.8
278.0
83.52
386.0
71.60
0.19x
Arxiv
100.1
136.7
347.2
145.9
353.5
1922.1
5.43x
Penn94
65.49
140.1
299.7
121.4
258.5
361.1
1.40x
Twitch Gamers
163.3
148.7
377.2
181.2
386.3
1611.7
4.18x
Products
354.1
371.8
461.5
325.2
681.2
4081.0
5.99x
Vessel
994.5
931.4
1137.2
664.0
1024.8
1051.3
1.02x
Avg Rank
1.6
2.1
3.8
1.7
4.2
compared to GNNs, SGFormer, and HypFormer, particularly large-scale datasets like Products and Vessel. We attribute this overhead to Transformer components such as linear attention, the refining function, and residual connections, which we recognize limitation of our method. However, this trade-off enables Q-GT to deliver strong performance on large datasets. We are optimistic that future work can further reduce the model's complexity and retain its effectiveness.
addition, Table S presents a comparison between the runtime of the space searching algorithm and the training time of Q-GT across 9 datasets. On Cora, Citeseer, Pubmed, and Airport, the computing time searching algorithm is less than 20% training time of Q-GT. However, the two large datasets, Arvix, Twitch Gamers and Products, the space searching requires more computation, around 4-6

---

Table Comparison GPU memory (in MiB).
Datasets
SGFormer
Hypformer
Q-GCN
Q-GCN2
Q-GT
Cora
58.4
158.6
111.7
65.9
70.8
Citeseer
126.8
471.2
314.8
134.5
129.6
Airport
42.8
57.9
107.3
58.4
82.0
Pubmed
195.5
382.9
428.2
240.2
348.6
Arxiv
1290.1
3822.7
4536.0
1106.2
3098,1
Penn94
2246.6
7281.5
4876.2
2319.9
1585.1
Twitch Gamers
1638.4
1578.0
2100.2
2840.0
2713.5
Products
27979.5
32363.3
58954.8
34668.1
51574.0
Vessel
8016.3
17376.8
48672.1
14249.9
39293.8
Avg Rank
1.1
3.4
4.4
2.7
2.7
Table 8: Ablation study on the impact of each component in Q-GT.
Datasets
Methods
Airpurt
Pubmed
Tree-2
Tree-3
Q-GT
96.56 0.07
96.40 + 0.10
94.02 + 0.06
92.03 +: 0.11
w/o residual Gain( %))
95.22 0.12
96.24
0.15
89.66 + 2.
90.70 + 0.38
1,39
0.17
4.64
1.45
w/o refining Gain %))
95.73 + 0.07
95.33 + 0.28
81.43 + 0.63
89.98 + 0.27
0.86
-1.11
13.4
2.23
w/o GCN Gain
94.43 0.35
90.47
0.18
86.78
0.49
83.53 + 1.86
2.21
6.15
7.70
9.244
times higher than the training of Q-GT. Compared to brute-force approach which requires training the model 16 times (once each dimension), our algorithm is more efficient in both time and computation.
H.3 Advantage of diffeomorphic framework
To investigate the advantage of the pro- posed diffeomorphic framework, we create three datasets Tree-1, Tree-2 and Tree-3 that best reflect the hyperbolic geometry.
Table 7: The link prediction results in ROC AUC (%) tree datasets.
Datasets
Tree-
Tree-2 15
Tree-3 13
Detailed descriptions of these datasets are provided in Appendix [.1 Table presents the link prediction results of Q-GCN, Q- GCN2 and Q-GT on these graphs. Q- GCN2 and Q-GT significantly outperform Q-GCN across all datasets. For instance, on Tree-2, Q-GT and Q-GCN2 achieve AUC improvements of 9.54 % and 9.05 %, respectively, over Q-GCN. To shed light on the perior performances of our models, we further visualize the learned embeddings on the Tree-1, Tree-2, and Tree-3 datasets in Fig. 5 Following the experimental setup in Section [6.3] the embed- dings are projected to the product spaces, excluding the spherical component for clearer interpretation. In Q-GCN, low-level nodes by contrast, exhibit a clear clustering of low-level nodes, indicating that these models effectively distinguish between low-level and high-level nodes. These observations emphasize the advantage of our diffeomorphic framework for preserving hierarchical structures of graphs.
Space dim p*
16
Time dim 9"
0
Q-GCN
91.44 0.52
85.83 + 0.51
84.86 0.35
Q-GCN2
95.34 + 0.31
93.60 + 0.32
89.25 + 0.34
Q-GT
96.05 + 0.10
94.02 + 0.06
92.03 0.11
scattered randomly among high-level nodes. Q-GCN2 and Q-GT,
H.4 Impact of key factors in Q-GT
To analyze the impact of each key component in our proposed model, we perform an ablation study using three variants derived from Q-GT:

---

Q-GCN embedding on tree-1
Q-GCN2 embedding on tree-1
Q-GT embedding on tree-1
Q-GCN embedding on tree-2
Q-GCN2 embedding on tree-2
Q-GT embedding on tree-2
Q-GCN embedding tree-3
Q-GCN2 embedding on tree-3
Q-GT embedding on tree-3
Depth level
Level
Level
Level
Level
Level
Figure 5; Visualization of the learned embeddings for the link prediction task on Tree-1, Tree-2, Tree-3 datasets. The colors denote depth levels. Nodes Levels and 2 are classified low-level. which are closer to the root, while the other are considered high-level.
Q-GT without residual connections. All residual connections are removed from each layer the model.
w/o residual:
w/o refining: Q-GT without refinement operations, including layer normalization, dropout, and non-lineat activation. This variant consists solely of linear layers.
w/o GCN: variant of Q-GT that excludes the graph-based module Q-GCN2, relying
solely on the Transformer module for representation learning.
We use the same experimental setup Q-GT to evaluate these variants, and the link prediction results are reported in T general, removing any component would lower the model's performance. w/o residual leads to decline in the model's performance by 1.91 % in average. This suggests that residual connections play an important role in stabilizing training and facilitating gradient flow across layers, which is particularly beneficial in deep architectures like Q-GT. Similarly, in the case of w/o refining, the model's 's performance also drops dramatically on Tree-2, which confirms the necessity of refining functions, such normalization, dropout, and non-lineat activation, for effective graph
10

---

Table Link prediction results with different embedding dimensions.
Datasets
Methods
Airport
Pubmed
Tree-2
Tree-3
Original Q-GT (16)
96.56 + 0.07
96.40 + 0.10
94.02 + 0.06
92.03 + 0.11
Q-GT (8)
95.36 + 0.28
95.88
0.20
87.83 +: 1.35
85.50 1 3.64
Gain( %6)
1.24
0.54
6.58
77.10
Q-GT (32)
96.09 0.15
96.48
0.10
94.36
0.36
90.29 + 2.08
Gain( %6)
-0.49
+ 0.08
0.36
-1.99
learning in pseudo-Riemanne Riemannian spaces. When removing graph-based module Q-GCN2, it causes a significant decrease in the performance of Q-GT on all datasets. This observation reveals that the Transformer module alone is insufficient for capturing local structural information in the graph.
Additionally, we evaluate the capacity of Q-GT with different embedding dimensions, 8 and 32, in Table Reducing the dimension 8 leads to notable decrease the link prediction results on hierar- chical datasets, whereas increasing 32 offers only minor improvements. Therefore, recommend using 16 dimensions to maintain computational efficiency and obtain strong performances.
I Experimental details
Table 10: Summary statistics of the real-world datasets.
Datasets
Cora
Citeseer
Airport
Pubmed
Arxiv
Penn94
Twitch Gamers
Products
Vessel
Total nodes
2708
3327
3188
19717
169343
41554
168114
2449029
3538495
Total edges
5278
4552
18630
44324
1166234
1362229
6797557
61859140
5345897
Undirected
Yes
Yes
Yos
Yes
No
Yes
Yes
Yes
Yes
Total classes
7
6
4
3
40
2
2
47
Features
1433
3703
11
500
128
4814
7
100
Mean ji
0.1658
0.0799
0.3626
0.2264
0,2290
0.2340
0.3238
0,2550
-0.1160
Std
0.2290
0.3716
0.3180 1
0.4197
0,3030
0.2050 1
0,2088
0.2562 1
0,2810
Space dim
2
8
0
Time dim g+
14
8
15
14
15
15
16
15
Table 11: Summary statistics of tree datasets.
Datasets
Tree-1
Tree-2
Tree-3
Total nodes
5461
5461
5461
Total edges
5460
5960
6460
Undirected
Yes
Yes
Yes
Total classes
Features
5461
5461
5461
Mean fü
-0,5983
-0,2924
10.7417
Std
0.1082
0.2953
0.2932
Space dim pa
16
15
13
Time dim g-
1
L.1 Datasets
Table I0 presents the statistics of all datasets used in this study. The mean jı and standard deviation denote the Gaussian sectional curvature (GSC) distribution of each input graph. Among all wollw datasets, eight graphs exhibit positive curvature distributions, while other four networks have negative distributions. In terms of graph size, Arxiv, Twitch Gamers, Products and Vessel are large-scale networks, Pubmed and Penn94 are medium-sized and the remaining datasets are considered small. Detailed descriptions of each dataset provided as follows:

---

Cora (McCallum et al. 2000) is a citation network comprising 2708 scientific publications each node is classified into one of seven classes. The network includes 5278 citation links among these papers. Each node is represented by a binary word vector, where each entry indicates the presence or absence of a specific word from a dictionary of 1433 unique terms.
Citeseer (Giles et al. 1998) is a citation network consisting of 3327 scientific publications classified into of six classes. The citation network consists of 4552 links. Each publication is described by a binary word vector showing the absence/presence of the corresponding word from the dictionary of 3703 unique terms.
Pubmed (Sen al., 2008 comprises 19717 scientific articles from the PubMed database related to diabetes, each categorized into one of three classes. The citation network includes 44324 links between these publications. Each article is represented by a TF-IDF weighted word vector, constructed from vocabulary of 500 unique words.
Airport Xiiog et al., 2022) is a transportation network containing 3188 airports. Edges correspond to airline routes connecting the airports. For the node classification task, each airport is labeled with the population of the country in which it is located.
Arxiv (Hu et al. 2020) is a directed citation network between all Computer Science (CS) arxiv papers. This dataset belongs to the Open Graph Benchmark (OGB) collection. Each node an arXiv paper and each directed edge indicates that one paper cites another one. Each paper comes with a 128-dimensional feature vector obtained by averaging the embeddings of words in its title and abstract. We apply the space searching algorithm on undirected structure of Arxiv network, while training on the original directed graph.
Penn94 (Lim et al. 2021) is Facebook "friendship" networks one hundred American colleges and universities from 2005. In this graph, nodes correspond to individual students, each labeled by their reported gender. Node features include academic major, second major or minor, dormitory or housing assignment, graduation year, and high school background.
Twitch Gamers (Lim et al. 2021) is an undirected graph representing connections between user accounts on the Twitch streaming platform. Nodes correspond to individual Twitch accounts, with edges indicating mutual follower relationships. Node features include the number of views, account creation and update timestamps, language, account lifetime, and flag indicating whether the account is inactive. The associated binary classification task involves predicting whether a channel contains explicit content.
Products (Hu et al. (0200 is an undirected, unweighted graph representing the Amazon co-purchasing network. Each node corresponds
product on Amazon, and an edge between two nodes indicates that the products were frequently bought together. Node features are derived from the bag-of-worte representations of product descriptions, followed by dimensionality reduction using Principal Component Analysis (PCA) to obtain 100- dimensional feature vectors.
Vessel (Hu et al. 2020) is an undirected, unweighted spatial graph capturing the entire mouse brain. Each node is represented by 3D spatial coordinates (x, y, z) within the Allen Brain Atlas reference framework. This dataset is designed to encourage neuroscience researchers to adopt graph-based representations in their studies. Ah same time, it presents machine learning researchers with challenging problems, such as integrating biological priors into learning models and scaling algorithms to efficiently process sparse, spatial graphs containing millions of nodes and edges.
Tree datasets are created due to the lack of the negative curvature graphs. Tree- 1 is a balanced tree of depth 6 and branching factor 4 consisting of 5461 nodes and 5460 edges. Tree-2 and Tree-3 are derived from Tree-1 by randomly adding 500 and 1000 edge respectively, introducing different magnitudes of curvature. Each node is initialized with a one-hot feature vector, and no node labels are provided in these datasets.
I.2 Implementation setup
Objective function. In the node classification task, following Xiong et al. (002)), we project the output of the final layer of both Q-GT and Q-GCN2 onto the tangent space and apply Euclidean multinomial logistic regression for classification. For link prediction, Eucliidenn methods utilize
12

---

the Fermi- decoder (Krioukov et al.) 2010), which computes edge probability scores based on geodesic distances in the embedding space.
Optimization. The trainable parameters of Q-GT and Q-GCN2 are defined in Fuclidean space via the proposed diffeomorphic framework. As result, standard Euclidean optimization algorithms, such as Adam (Kingma and Ba, 2015) and SGD (Robbins and Monro 1951, can be directly applied for model training. RiemannianAdam (Becigneul and Ganea 2019) is another choice for non-Euclidean methods such as HGCN and Hypformer, which operate under specific manifold constraints.
Setting. We implement our models by Pytorch and Geoopt tool All the experiments conducted on a GPU device NVIDIA GeForce RTX 4090 with 24GB memory, except for the Products and Vessel datasets, which are on an NVIDIA A100 GPU with 80GB of memory. For comparison baselines, we follow the implementation of each model introduced in the corresponding paper. The experiment results reported by the average value with standard deviation on the test sets using three different seeds. Unlike the work of Xiong et al. (2022), we utilize initial features provided in the datasets for node classification instead of pretrained embeddings for evaluating model's learning capacity.
Table 12: The grid search ranges for hyperparameters.
Hyperparameter
Search range
Learning rate Ir
5e-2, 2e-2, 1e-2, 5e-3, 2e-3, 1e-3 0.5, 0.6, 0.7, 0.8, 0.9
Balance weight a Number of layers
1,2,3,4 relu, tanh, sigmoid, leakyrelu
Activation
Dropout rate
0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7
Weight decay
1e-1, 5e-2, 1e-2, 5e-3, 1e-3, 5c-4, 1c-4, 5e-7
Hyperparameter. Besides the space and time dimensions of hheudo-hyperboloids 2% our models involves other hyperparameters, including learning rate ir, balance weight number of layers, activation function, dropout rate, and weight decay. The optimal hyperparameters are obtained by applying grid search strategy, where the ranges summarized in Table 12
I.3 Pseudocode
In this section, we provide the pseudocode of our proposed framework to facilitate implementation for the reader. Algorithm outllnnes the procedure of the space searching algorithm. In line 1, a subset A is uniformly sampled from the node set Va. Lines 2-6 precompute the shortest path distances from each node A to all other nodes in the graph G. The Gaussian sectional curvature (GSC) distribution of the input graph is computed in lines 7-14. Finally, in lines 15-20, we iteratively calculate the KL divergence between the input GSC distribution (o (f, σ²) and each ideal distribution N N(H, of) to determine the optimal space dimension p..
Algorithm 2 demonstrates the learning process of Q-GCN2. In line 1, we apply Algorithm | to select the embedding manifold Q .4" Line 2 projects the initial features the embedding manifold with function f which is either dense layer graph convolution layer to ensure the dimensional consistency. The projected representation is passed through I layers of Q-GCN2 in lines 3-9, where each layer comprises feature transformation, neighborhood aggregation, and nonlinear activation.
The learning process of Q-GCT is described in Algorithm 3 Similar to Q-GCN2, lines 1-2 involve selecting the appropriate manifold and transforming node features. The transformed representation then processed through an L-layer Q-GT in lines 3-15. To capture local structural information, Q-GCN2 is incorporated in line 16. The outputs of Q-GCN2 and Q-GCT are combined using balance weight a in lines 17-20. The final node representation Z is obtained for downstream tasks.
13

---

Algorithm 1
The procedure of space searching algorithm
Input: An input graph = (Va, Fa)
Output: The optimal -eeudo--yperboloid 0 .
1: Uniformly sampling a subset nodes A VG
2: for each node E A do
3: for each node E VG do
4: Precomputing the shortest path distance (a
5: end for
6: end for
7: for each node m E VG do
8: for to n,, do
9: Uniformly sampling two neighbor nodes c E (((m
Calculating the curvature analog k(m; e) in Eq. (16) 11: end for
Calculating the sectional curvature x(m) by averaging curvature analogs k(m; b, c) 13: end for
Calculating the mean z and the variance x² in Eq. for == 0 to d - do
15:
Assigning space dimension p; :=
17: Calculating the ideal GSC distribution N (mu, of)= T(p))
Calculating the KI. divergence dæ (Wi,, (
19:
Saving p" p: that yields the smallest KL divergence
20: end for
21: return the optimal space dimension p
Algorithm 2 I--laye Q-GCN2 learning process
Input: An input graph G (Vg, EG) with the initial features X
Output: A hidden representation x 27
1: Determining the embedding manifold 2j "9" by using Algorithm|
2
2: Projecting initial features X exp (101[(((E)
3: for / to I do
89 x (( 0F ))
4: Decomposing features
go" la
XLŞ"
|12x 15
5:
X
111
/ /
1-1 1| 12 W2xx
xLŞ"
1/ xL XL Agg(X
6:
X.
1-1
1-1 Ea)
xL5"
8/ x1 X
7:
Activation
8:
Projecting back to the pscudo-hyperboloid X, o7
= = -
9: end for
return the final representation X,
14

---

Algorithm 3 L-layer Q-GT learning process
Input:
An input graph = (Ve, Ec) with the initial features X Output: hidden representation Z
1: Determining the embedding manifold O :9" by using Algorithm
Q .M"
2: Projecting initial features X
expo ((0])(((())))
3: X
LayerNorm (X
0
Xo 2
4: Xo
Feature Trantfoora
Q
5: Xo
Activation (X
0
27
20
6: X
Dropout to L do .²
f
7: for
1=1
1
(x, OP 19
8:
[-1
LayerNorm
00 a*
9:
X
1-1
X O
-1
10:
1-1
X
1-1
Øg X 1-1
X i-1
FeatureTransfome
2r a
Activation (X1-)
1-1
a"
13:
Dropout (X,
14:
end for TranssrTransommm/
X
L'-layer Q-GCN2(X, 25"00"
16: X L
27
17: X, lo (xx
18: X, log (x 2
L
19: = a (1 X2
= expo (X)
21: return the final representation Z
15


Reviews:
[{"review_id": "itk7UDdbKM", "reviewer": "Reviewer_8CPi", "summary": "This work presents a graph transformer that operates in a Pseudo-Riemannian manifold, addressing limitations of prior hyperbolic and spherical approaches. The authors design a diffeomorphic transformer that splits the pseudo-Riemannian space into coupled hyperbolic and spherical components, thereby eliminating geometric disconnection. Building on this, they create the Pseudo-Riemannian Graph Transformer (Q-GT) and a curvature-aware search procedure that selects manifold dimensions by aligning the input graph’s Gaussian sectional-curvature (GSC) profile. Tested on seven node-classification and four link-prediction datasets, Q-GT achieves small yet consistent gains over strong Euclidean and hyperbolic baselines.", "questions": "- Is extending LResNet to the product space non-trivial?\n- How is the UMAP used for the visualization in Poincare space? Do you first log map the embeddings then use the UMAP or do you use a hyperbolic UMAP?", "limitations": "The authors have addressed the limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\nThe paper is well written. Using the product manifolds retain both the cyclical and hierarchical information from the data. The proposed space search algorithm is also efficient and removes the need for manual fine-tuning significantly. The theoretical discussion is rigorous and supported by clearly presented proofs, enhancing the paper’s credibility. Experiments span homophilous and non-homophilous graphs, where Q-GT outperforms baseline in 5/7 classification tasks and all link-prediction tasks.\n\n### Weaknesses\n- Use of log map and exp map in residual connection is inefficient. Recent methods like LResNet [1] have developed a fully hyperbolic residual connection method that does not rely on mapping. This approach can effectively lower computational overhead, reduce numerical errors from repeated projections, and preserve intrinsic curvature throughout training, thereby enabling faster and more stable convergence\n- The paper omits comparison with the recent FPS-T [2] method, which overlaps significantly with Q-GT despite being cited. The authors should clearly address what are the differences between Q-GT and FPS-T.\n- Gains over the strongest baselines (e.g., SGFormer, Hypformer) are often ≤1–2 F1 on `Cora`, `Pubmed` and `Penn94`, calling into question whether the extra geometric machinery justifies its cost.\n- Most benchmark datasets are small (< 10 k nodes), with the largest having 169 k nodes. Including larger benchmarks such as `ogbn-products` and `ogbl-vessel` would better demonstrate the model’s scalability.\n- Although the visualization of Q-GT embeddings in Figure 3 shows that nodes are somewhat dispersed, there is still no clear hierarchy. Is this because the dataset lacks any hierarchy or does the model require extra steps to capture the hierarchy completely? How would this compare to other hyperbolic models like Hypformer? \n- \"Geodesically disconnection\" reads awkwardly. I am assuming the it should be \"Geodesic disconnection\".\n\nOverall, I think this manuscript is promising and clearly presented, but it stops short of fully demonstrating the practical value of its geometric innovations. Adding (i) direct comparisons and clarifications versus closely related methods such as FPS-T and mixed-curvature Transformers, (ii) evaluation on large-scale benchmarks (e.g., `ogbn-products` and `ogbl-vessel`), and (iii) designing a fully Pseudo-Riemannian residual connection would substantially elevate the paper’s impact.\n\n[1] He, Neil, Menglin Yang, and Rex Ying. \"Lorentzian residual neural networks.\" arXiv preprint arXiv:2412.14695 (2024).\n[2] Cho, Sungjun, et al. \"Curve your attention: Mixed-curvature transformers for graph representation learning.\" arXiv preprint arXiv:2309.04082 (2023).", "was_revised": true, "final_justification": "The authors provided new experimental results, particularly on the OGB datasets, which addressed my earlier concerns regarding the scalability of the method. These additions, along with clarifications on design choices and comparisons with prior work justify the score increase.", "rebuttal": "We would like to thank Reviewer 8CPi for the constructive comments, we would like to address the raised weaknesses and questions as below:\n## Weakness 1:\nWe sincerely thank you for suggesting the Lorentzian residual connection [1].\nIn contrast to our pseudo-hyperboloid addition in Section 4, this approach performs entirely within the hyperbolic manifold, eliminating the need for tangent space.\nHowever, adapting the Lorentzian residual connection to pseudo-hyperboloids $Q_{\\beta}^{p,q}$ requires extending it to product manifolds, which presents a non-trivial challenge.\nBuilding upon the definition of Lorentzian residual connection [1],  we propose the following (straightforward) extension:\n\n$$\nw_1 x \\oplus_{\\beta} w_2 y = \\left( \n\\frac{\\sqrt{|\\beta|}(w_1 u_1 + w_2 u_2)}{|| w_1 u_1 + w_2 u_2 ||},\\ \n\\frac{\\sqrt{|\\beta|}(w_1 v_1 + w_2 v_2)}{\\left| \\left| w_1 v_1 + w_2 v_2 \\right| \\right|_{\\mathbb{L}}} \n\\right)\n$$\n\nwhere $w_1, w_2$ are the weights, $x = (u_1, v_1)^T $, and $y = (u_2, v_2)^T$ are two points in the product space of spherical and hyperbolic manifolds.\nThe work in [1] proved that the denominator in hyperbolic part $| || w_1 v_1 + w_2 v_2 ||_{\\mathbb{L}} |$ has a positive lower bound when the weights are positive.\nHowever, this property does not hold for the spherical denominator $|| w_1 u_1 + w_2 u_2 ||$, whose value can range from $0$ to $+\\infty$ for any choice of $w_1$, $w_2$.\nTo eliminate numerical instability in the spherical case, additional constraints may be required but potentially increasing computational complexity.\nOn the other hand, our pseudo-hyperboloid vector addition maintains feature integrity, thereby improving the model’s learning capacity over prior pseudo-Riemannian approaches.\n\n## Weakness 2:\n\n**Table 1.** Comparison to FPS-T in node classification results (F1-score). \n| Method         | Cora             | Citeseer         | Airport           | Pubmed           | Arxiv             | Penn94           | Twitch Gamers     |\n|----------------|------------------|------------------|-------------------|------------------|-------------------|------------------|-------------------|\n| FPS-T          | 60.50 ± 0.40     | 60.70 ± 0.30     | 91.41 ± 0.17      | 76.23 ± 0.31     | OOM               | OOM              | OOM               |\n|$Q$-GT          | **82.97 ± 0.17** |  **70.40 ± 0.49**     | **95.70 ± 0.08**  | **80.37 ± 0.40**   | **66.24 ± 0.05**  | **82.77 ± 0.10** | **64.45 ± 0.07**  |\n\n**Table 2.** Comparison to FPS-T in link prediction results (ROC AUC, %).\n\n| Method         | Cora             | Citeseer         | Airport           | Pubmed           |\n|----------------|------------------|------------------|-------------------|------------------|\n| FPS-T          | 95.25 ± 0.12     | 97.17 ± 0.44     | 96.10 ± 0.13      | 92.11 ± 0.04     |\n| $Q$-GT       | **96.87 ± 0.19** | **97.59 ± 0.11** | **96.56 ± 0.07**  | **96.40 ± 0.10** |\n\nSimilar to $Q$-GT, FPS-T [2] is a graph Transformer designed for graphs with mixed structural properties, which becomes the concerns of the reviewers. \nFPS-T operates in a product space of Riemannian stereographic manifolds  $\\lbrace st^{i} \\rbrace^{H}_{i=1}$, \nwhere each component $st^{i}$ has a trainable curvature $\\kappa_i$ that can be positive, negative, or zero.\nTo address this concern, we conduct a comparative evaluation of $Q$-GT and FPS-T on both node classification and link prediction tasks, with the results presented in Table 1 and Table 2.\nFor fairness, we set $H=2$ since our product space also has two components.\nFirst, FPS-T incurs significant computational overhead due to the use of tangent mappings and the costly computation of Laplacian eigenvectors.\nAs a result, it runs out of memory on three large-scale datasets: Arxiv, Penn94, and Twitch Gamers.\nSecond, the model's trainable curvatures $\\kappa_i$ fail to converge to optimal values. \nOn node classification tasks for Cora and Citeseer, FPS-T underperforms compared to $Q$-GT. \nSpecifically, the learned curvatures $(\\kappa_1, \\kappa_2)$ converge to $(0.023, 0.025)$ on Cora and $(0.008, 0.007)$ on Citeseer, which do not reflect the hyperbolic geometry.\nIn contrast, $Q$-GT benefits from operating directly on the product of spherical and hyperbolic manifolds, which is diffeomorphic to the pseudo-hyperboloid, thereby enhancing its performance.\n\n## Weakness 3:\n\nTo address concerns about the computational cost of $Q$-GT, we provided a detailed complexity comparison in Table 4 and Table 5 below (in response to Reviewer V9gp).\nIn terms of runtime, $Q$-GT requires significantly more time than SGFormer and HypFormer. \nHowever, its memory usage is comparable to these models on the Cora and Pubmed datasets. \nNotably, on Penn94, $Q$-GT achieves the lowest memory consumption among all baselines.\nWe also evaluated $Q$-GT against strong baselines on larger datasets (Products and Vessel from the Open Graph Benchmark) as reported in Table 3.\nWhile $Q$-GT is indeed more demanding in terms of both runtime and memory, it delivers substantially superior performance compared to SGFormer and HypFormer, demonstrating the value of the proposed geometric framework.\n\n## Weakness 4:\n\n**Table 3.** Comparison on Open Graph Benchmark (OGB) datasets.\n\n| Method         | Products         | Vessel           |\n|----------------|------------------|------------------|\n| $Q$-GCN          | 29.56 ± 0.31     | 59.64 ± 0.37     |\n|$Q$-GCN2         | 52.08 ± 0.45     | 50.21 ± 0.36     |\n| SGFormer       | 65.47 ± 0.28     | 50.03 ± 0.26     |\n| Hypformer      | 40.25 ± 0.54     | 49.87 ± 0.49     |\n| $Q$-GT      | **70.25 ± 0.27** | **78.78 ± 0.50** |\n\nTo further assess the scalability of our model, we conducted additional evaluations of $Q$-GT and other strong baselines on two recommended datasets from the Open Graph Benchmark: 'ogbn-products' (Products) and 'ogbl-vessel' (Vessels), where the results are shown in Table 3. \nAll experiments were performed with all Transformer models configured with 1 layer, GNNs with 3 layers, and training conducted over 200 epochs.\nIt is obvious that $Q$-GT achieves the highest performance on both datasets, outperforming the strongest baselines by 7.3% on Products and 32.09% on Vessel.\nHowever, as shown in Table 4 and Table 5, $Q$-GT introduces higher computational and memory costs compared to other baselines.\nWe recognize this as a limitation of our current approach and plan to investigate more efficient solutions in future work.\n\n## Weakness 5:\n\nAs depicted in Figure 3 (Section 6 in the main paper), high-level nodes appear more scattered, forming several small clusters that include only a few low-level nodes, suggesting the presence of localized hierarchical patterns.\nHowever, the overall hierarchical structure is not clearly visible in this visualization, because the Cora dataset lacks a well-defined hierarchy, as reflected by its Gaussian Sectional Curvature (GSC) distribution, which tends to be more spherical.\nTo better illustrate the model's ability to capture hierarchical structure, we introduce three synthetic tree-structured datasets in Appendix H.3. \nThe visualizations of these datasets reveal distinct clusters of low-level nodes (represented by red and pink) in $Q$-GT and $Q$-GCN2, demonstrating the hierarchy of the datasets.\nAdditionally, we compare to hyperbolic baseline, Hypformer on these tree datasets.\nHypformer achieves average AUC scores of 95.10, 92.37, and 91.06 on Tree-1, Tree-2, and Tree-3, respectively.\nThese results are comparable to $Q$-GT and $Q$-GCN2 , further confirming that both models are effective in capturing and representing hierarchical structures.\n\n## Weakness 6:\n\nWe appreciate your feedback and will update the terminology in the next version of the paper.\nThe term \"geodesic disconnection\" is originated from the work $Q$-GCN.\n\n## Question 1:\n\nWe discussed about extending Lorentzian residual connection [1] in our reply in Weakness 1. \nWe concluded that adapting Lorentzian residual connection to product space is a non-trivial challenge due to the numerical instability in spherical denominator.\n\n## Question 2:\n\nGiven the final representations of $Q$-GT and $Q$-GCN2, we first apply our diffeomorphic mapping to transform them into the product space and extract the hyperbolic components.\nWe then use hyperbolic UMAP with  'output\\_metric=hyperboloid' to embed these features into a 2D hyperboloid. \nIn Figure 3 (Section 6 in the paper), we further project these hyperbolic outputs onto the Poincaré disk, since the original embeddings lie in a 2D hyperboloid space. \nIn contrast, for the tree datasets shown in Appendix H.2, we directly visualize the hyperbolic UMAP outputs without projection to the Poincaré disk, as this yields clearer visualizations. \nWe avoid applying the logarithmic map before UMAP, as tangent space mappings can introduce distortions in the quality of the hyperbolic embeddings [1].\n\n[1] He, Neil, Menglin Yang, and Rex Ying. \"Lorentzian residual neural networks.\" arXiv preprint arXiv:2412.14695 (2024). \n\n[2] Cho, Sungjun, et al. \"Curve your attention: Mixed-curvature transformers for graph representation learning.\" arXiv preprint arXiv:2309.04082 (2023)."}, {"review_id": "hgSpVScESi", "reviewer": "Reviewer_SXvr", "summary": "Real-world graphs often exhibit complex topological structures, and hence it is often more suitable to embed graphs into pseudo-Riemannian manifolds than commonly used Euclidean space. Existing methods exhibit information loss, a lack of guideline for selecting dimensions, and limited context size, which limit their capacity. This paper proposes the pseudo-Riemannian graph transformer by extending linear attention and layer normalization to the pseudo hyperboloid. A space searching algorithm is designed based on the Gaussian Sectional Curvature distribution. Experiments corroborate the effectiveness of the proposed method.", "questions": "See above.", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\nThe proposed method is based on solid theory.\n\nThe first paragraph of section 5 clearly explains what kind of graphs are suitable for the proposed pseudo-Riemannian graph transformers.\n\nThe space searching algorithm increases the practicality of the proposed model.\n\nThe visualization of the learned embeddings is informative and interesting.\n\nWeaknesses:\n\nIn line 47, the concepts of time and space dimensions appear without definition. And where do the names come from?", "was_revised": true, "final_justification": "My concern has been addressed and I think this submission is a qualified NeurIPS paper.", "rebuttal": "Thank you for your encouraging feedback. We would like to address the weakness points as below:\n## Weakness 1:\nWe’re pleased to hear that you value the elegance of our contributions to graph learning in pseudo-Riemannian geometry. \nIn line 47, we mentioned the concepts of space and time dimensions of pseudo-Riemannian manifolds.\nA more detailed explanation is provided in Section 2 (lines 97–98).\nGiven a point $x=(x_0, x_1, ..., x_{q}, x_{q+1}, ..., x_{q+p})$ on the pseudo-hyperboloid $Q_{\\beta}^{p,q}$  , the first $q + 1$ coordinates $(x_0, x_1, ..., x_q)$ correspond to time dimensions, while the remaining $p$ coordinates $(x_{q+1},..., x_{q+p})$ represent space dimensions. \nThe concepts of time and space dimensions originate from Einstein’s general theory of relativity which described the four-dimensional spacetime.\nThe time and space dimensions are formalized within the framework of pseudo-Riemannian geometry[1], then widely adopted in pseudo-Riemannian learning methods [2], [3], [4]. \nHowever, finding the appropriate time and space dimensions that align with the graph structure is a non-trivial and time-consuming task as it requires extensive training if done manually.\nTo address this challenge, we introduces an automated space searching algorithm that efficiently discovers optimal pseudo-Riemannian configurations based on the Gaussian Sectional Curvature. \nThis significantly reduces manual effort and improves model performance.\n\n[1] O’neill, B. V. \"Semi-Riemannian Geometry With Applications to Relativity\", 54 (1983).\n\n[2] Law, M. T.; Stam, J. \"Ultrahyperbolic Representation Learning\".  Neural Information Processing Systems. (2020).\n\n[3] Law, M. T. \"Ultrahyperbolic Neural Networks\". Neural Information Processing Systems. (2021).\n\n[4] Xiong, B.; Zhu, S.; Potyka, N.; Pan, S.; Zhou, C.; Staab, S. \"Pseudo-Riemannian Graph Convolutional Networks\". Neural Information Processing Systems. (2021)."}, {"review_id": "zMjLi5cdnV", "reviewer": "Reviewer_V9gp", "summary": "This paper addresses the challenge of representing complex graph structures that are poorly captured by manifolds of uniform global curvature (e.g., hyperbolic or spherical spaces). The authors introduce a diffeomorphic framework for graph embedding on pseudo-Riemannian manifolds, generalizing both hyperbolic and spherical geometries. Building on this, they propose the pseudo-Riemannian Graph Transformer (Q-GT), which extends transformer operations (attention, residual connection, layer norm) to pseudo-Riemannian geometry. They also design a lightweight space searching algorithm to select the best manifold for a given graph. Experiments on node classification and link prediction across diverse real-world graphs demonstrate that Q-GT outperforms state-of-the-art baselines.", "questions": "1) How does the Q-GT compare in terms of training/inference time, parameter count, and GPU memory usage to the strongest baselines (e.g., Hypformer, SGFormer, Q-GCN)? \n2) Could the authors conduct the experiments on larger graphs such as Open Graph Benchmark?\n3) Which components of the Q-GT are most critical for its success (e.g., diffeomorphic mapping, specific attention mechanism, residual connection)? Is there any interpretability in the learned embeddings (e.g., correlation between curvature and graph properties)?", "limitations": "Yes: The authors have adequately discussed limitations", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1) The Q-GT is the first transformer-based graph model built in pseudo-Riemannian geometry, enabling core transformer operations in this space. \n2) The space searching algorithm is simple yet but grounding the manifold selection in data-driven curvature estimation.\n3) Q-GT achieves strong, consistent performance on both node classification and link prediction tasks, outperforming established baselines such as QGCN (see Tables 1 and Table 2).\n\nWeakness:\n1) The space searching algorithm is grounded in a linear relationship between ideal standard deviation and space dimension, but this is mostly heuristic and lacks theoretical justification or sensitivity analysis.\n2) The paper lacks some analysis on training/inference time and scalability or complexity. Q-GT introduces higher computational and memory overhead compared to GNNs (and possibly some baselines), which might be a practical concern. \n3) The method uses space search, which takes more computation time and it is not fair to compare with other approaches without space search such as QGCN.", "was_revised": true, "final_justification": "My concerns have been addressed and I increased my score.", "rebuttal": "We would like to thank Reviewer V9gp for the constructive comments, we would like to address the raised weaknesses and questions as below:\n\n## Weakness 1:\nWe agree with Reviewer V9gp that our work requires the assumption of a linear relationship between the ideal standard deviation $\\sigma_i$ and the space dimension $p_i$.\nIn general, the ideal standard deviation can be modeled as any bijective function of $p_i$, such as a quadratic function $p_i^2$ or an exponential form $e^{p_i}$, as long as it aligns with the curvature behavior of the three geometric settings (i.e., monotonically increasing in spherical space and inversely related in hyperbolic space; see Appendix C.3 for details).  \nHowever, using alternative forms like the quadratic $p_i^2$ results in a steeper rate of change, which may lead to suboptimal manifold selection.\nTo clarify this hypothesis, we define the ideal standard deviation $\\sigma_i$ using a quadratic formulation as follows:\n$$\n    \\sigma_i = \\begin{cases}\n      \\sigma_{\\text{max}} - (\\sigma_{\\text{max}} - \\sigma_{\\text{min}}) \\left(\\frac{2p_{i} - (d + 1)}{d-3} \\right)^2    & \\text{if $ (d-1)/2 < p_i \\leq d-1$} ; \\\\\n     1  & \\text{if $p_i = (d-1)/2$} ; \\\\\n     \\sigma_{\\text{min}} + (\\sigma_{\\text{max}} - \\sigma_{\\text{min}}) \\left(\\frac{2p_{i}}{d-3} \\right)^2 & \\text{if $ 0  \\leq p_i < (d-1)/2$}.\n    \\end{cases}\n$$\nThen we apply the space searching algorithm in Section 5 to four benchmark datasets: Cora, Citeseer, Airport, and Pubmed. \nThe resulting space dimensions are $p_i = 4, 7, 2, 4$, respectively.\nHowever, when compared to the configurations shown in Fig. 2 (Section 6) and Fig. 4 (Appendix H), these values do not correspond to the optimal configurations for pseudo-hyperboloids.\nThis suggests that the assumption of a linear rate of change in standard deviation aligns well with the variation in space dimension $p_i$. \nWhile further theoretical justification is needed, our experimental results provide strong evidence that the linear assumption is effective in practice.\nA more detailed theoretical analysis would require a proper model of the underlying graph manifold in real world dataset, which can be considered as a future work.\n\n## Weakness 2:\n\n**Table 4.** Comparison of training time (in seconds).\n\n| Dataset         | SGFormer | Hypformer | $Q$-GCN | $Q$-GCN2 | $Q$-GT  |\n|-----------------|----------|-----------|--------|---------|--------|\n| Cora            | 40.92    | 66.42     | 136.6  | 82.44   | 193.7  |\n| Citeseer        | 40.21    | 67.31     | 141.8  | 76.12   | 188.2  |\n| Airport         | 71.55    | 97.20     | 211.4  | 66.59   | 249.5  |\n| Pubmed          | 144.8    | 153.8     | 278.0  | 83.52   | 386.0  |\n| Arxiv           | 100.1    | 136.7     | 347.2  | 145.9   | 353.5  |\n| Penn94          | 65.49    | 140.1     | 299.7  | 121.4   | 258.5  |\n| Twitch Gamers   | 163.3    | 148.7     | 377.2  | 181.2   | 386.3  |\n| Products        | 354.1    | 371.8     | 461.5  | 325.2   | 681.2  |\n| Vessel          | 994.5    | 931.4     | 1137.2 | 664.0   | 1024.8 |\n| **Avg Rank**    | **1.6**  | **2.1**   | **3.8**| **1.7** | **4.2**|\n\n**Table 5.** Comparison of GPU memory usage (in MiB).\n\n| Dataset         | SGFormer | Hypformer  | $Q$-GCN | $Q$-GCN2 | $Q$-GT  |\n|-----------------|----------|-----------|--------|--------|--------|\n| Cora            | 58.4     | 158.6     | 111.7  | 65.9   | 70.8   |\n| Citeseer        | 126.8    | 471.2     | 314.8  | 134.5  | 129.6  |\n| Airport         | 42.8     | 57.9      | 107.3  | 58.4   | 82.0   |\n| Pubmed          | 195.5    | 382.9     | 428.2  | 240.2  | 348.6  |\n| Arxiv           | 1290.1   | 3822.7    | 4536.0 | 1106.2 | 3098.1 |\n| Penn94          | 2246.6   | 7281.5    | 4876.2 | 2319.9 | 1585.1 |\n| Twitch Gamers   | 1638.4   | 1578.0    | 2100.2 | 2840.0 | 2713.5 |\n| Products        | 27979.5  | 32363.3   | 58954.8| 34668.1| 51574.0|\n| Vessel          | 8016.3   | 17376.8   | 48672.1| 14249.9| 39293.8|\n| **Avg Rank**    | **1.1**  | **3.4**   | **4.4**| **2.7**| **2.7**|\n\nIn our paper, we provided a runtime comparison of three pseudo-Riemannian models in Appendix H.2.\nTo offer a more comprehensive complexity analysis, we further evaluate the runtime and memory usage of $Q$-GT against strong baselines, including SGFormer, HypFormer, and $Q$-GCN, as suggested in Question 1.\nThe detailed results are presented in Table 4 and Table 5.\nA comparison between $Q$-GCN and $Q$-GCN2 shows that our diffeomorphic framework achieves better efficiency in both runtime and memory. \nThis is because $Q$-GCN framework relies on a combination of diffeomorphic and tangent space mappings in product manifolds, whereas our approach operates directly within the product manifold space.\nNonetheless, $Q$-GT incurs higher computational and memory overhead compared to GNNs, SGFormer, and HypFormer, particularly on large-scale datasets like Products and Vessel.\nWe attribute this overhead to Transformer components such as linear attention, the refining function, and residual connections, which we recognize as a limitation of our method.\nHowever, this trade-off enables $Q$-GT to deliver strong performance on large datasets, as shown in Table 3.\n$Q$-GT delivers the best performance across both datasets, surpassing the strongest baselines by 7.3\\% on Products and 32.09\\% on Vessels.\nWe are optimistic that future work can further reduce the model’s complexity and retain its effectiveness.\n\n## Weakness 3:\nWe would like to clarify that our space searching algorithm is introduced to identify optimal pseudo-Riemannian manifolds for graph embedding.\nWhile it may incur higher computational costs on large graphs, it is significantly more efficient than the brute-force approach used in prior works such as $Q$-GCN (see Section H.2 for runtime comparison).\nIn node classification and link prediction tasks, we compare the results of all pseudo-Riemannian methods ($Q$-GCN, $Q$-GCN2, and $Q$-GT) using the manifold configurations computed by our algorithm.\nTo ensure fairness, we also perform a brute-force search over all manifold configurations on four benchmark datasets—Airport, Cora, Citeseer, and Pubmed (see Figure 2 in Section 6 and Appendix H.1).\nThe results reveal that $Q$-GCN's chosen manifold does not align with the outcome of our space search algorithm or the intrinsic topology of the input graph, and it fails to outperform $Q$-GCN2 and $Q$-GT on the optimal pseudo-hyperboloid.\n\n## Question 1:\n\nWe provided a complexity comparison in Table 4 and Table 5, as discussed in Weakness 2.\nOverall, $Q$-GT introduces higher computational and memory costs compared to other baselines.\nWe acknowledge this as a shortcoming and intend to explore solutions in future work.\n\n## Question 2:\n\nWe understand that scalability is a key concern for Transformer-based models.\nTo evaluate this, we tested our $Q$-GT and other Transformers on larger datasets from the Open Graph Benchmark, with the results reported in Table 3 above (in response to Reviewer 8CPi).\nAdditionally, runtime and memory comparisons with other baselines are provided in our reply in Weakness 2.\nAlthough $Q$-GT incurs high computational complexity, this trade-off results in notably strong performance of $Q$-GT. \nWe recognize that improving the complexity of $Q$-GT remains an important direction for future work.\n\n## Question 3:\n\nThe most critical component of QGT is the diffeomorphic mapping, which transforms the pseudo-hyperboloid into a product space of spherical and hyperbolic geometries.\nThe diffeomorphic framework enables a more interpretable evolution of the pseudo-hyperboloid $Q_{\\beta}^{p,q}$, leading to our efficient space searching algorithm for determining the optimal pseudo-hyperboloid.\nBy addressing the issue of geodesic disconnection, it facilitates the adaptation of standard architectures (e.g., GNNs and Transformers) to pseudo-Riemannian spaces.\nMoreover, by preserving both spherical and hyperbolic characteristics, the diffeomorphic mapping contributes to improved performance in graph learning tasks.\nAdditionally, other components such as linear attention, the refining function, and residual connections also play a crucial role in the success of $Q$-GT.\n\nTo interpret the learned embeddings, we employ the UMAP tool for visualization.\nFor instance, in Appendix H.3, we visualize the embeddings of tree-structured datasets on the hyperbolic component of the product space.\nThe visualizations reveal that in $Q$-GCN2 and $Q$-GT, red and pink nodes—corresponding to lower-level nodes—form distinct clusters, indicating that hierarchical information is effectively preserved. \nIn contrast, $Q$-GCN exhibits oversmoothing, where embeddings of low-level and high-level nodes become indistinguishable, suggesting a loss of hierarchical structure."}, {"review_id": "ARNifxM8EG", "reviewer": "Reviewer_AG9m", "summary": "The paper presents a novel graph transformer that is grounded on a pseudo-Riemannian manifold as the underlying space. By exploiting the properties of network spherical embedding and network hyperbolic embedding, the proposed approach constructs a geometric embedding model capable of preserving both cyclical and hierarchical graph representations. Experimental results show that this method achieves superior performance compared to the traditional Graph Convolutional Network (GCN) in node classification and link prediction tasks.", "questions": "1. The authors could clarify the meaning of \"x_0\" in line 99 and Equation (2), as this is crucial for understanding the definition of the \"Zero-first submanifold,\" which plays a central role in the paper.\n\n2. The authors should explain how the two mappings introduced in the paper address the \"geodesic disconnections issue,\" as this would provide more insight into this important aspect.\n\n3. The specific meaning of the notation \"x\" and \"z\" in Equation (4) should be clarified, as this would help readers better understand the formalism and its application.\n\n4. The conclusion of Theorem 1 seems somewhat self-evident, given that a pseudo-Riemannian manifold can be decomposed into the Cartesian product of subspaces formed by spherical and hyperbolic spaces. The focus should perhaps be placed on demonstrating the geodesics for these two spaces, specifically proving that a geodesic exists in the pseudo-Riemannian manifold composed of paths from the two subspaces constructed by the authors. This would strengthen the contribution of the theorem.", "limitations": "The authors have discussed the limitations of their method in practical applications. However, they have not illustrated some inherent theoretical limitations of the approach, such as the completeness of the embedding in the pseudo-metric space and the lack of interpretability. These aspects could be important for a more comprehensive evaluation of the method.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 1, "strengths_and_weaknesses": "Strengths:\n\nThe work covers a broad range of topics, from the foundational theory of graph embedding to downstream applications. The research process is comprehensive, and the objective is to address the issue of selecting the underlying embedding space for graph embeddings, which holds significant research value.\n\nWeaknesses:\n\n1. Theoretical Limitations: \n\nThe authors have not demonstrated the theoretical validity of the claims made in the paper regarding the network pseudo-Riemannian manifold embedding. Specifically:\n\n    (a) There is no theoretical proof that the method described in \"Contributions 1\" preserves the cyclical and hierarchical representations of graphs as claimed.\n\n    (b) The issue of \"geodesic disconnections\" that the authors frequently emphasize remains unresolved.\n\n2. Experimental Limitations:\n\n    (a) The authors' choice of comparison methods is limited, mainly focusing on GCN and its hyperbolic variant, HGCN.\n\n    (b) The ability of the proposed method to \"capture mixed-topological dependencies\" has not been empirically validated.", "was_revised": true, "final_justification": "The authors have satisfactorily resolved my doubts and presented a persuasive argument, which has allowed me to appreciate the innovative aspects of their manuscript.", "rebuttal": "We would like to thank Reviewer AG9m for the constructive comments, we would like to address the raised weaknesses and questions as below:\n## Weakness 1:\n\n(a) Establishing a theoretical proof for model properties in non-Euclidean spaces remains  an open research problem, especially in the context of pseudo-Riemannian geometry.\nNevertheless, among existing pseudo-Riemannian approaches, our method stands out as the most effective in both interpreting the behavior of pseudo-hyperboloids and identifying optimal configurations for the input data.\nIn our Experiments (Section 6 in the main paper), the promising performance of $Q$-GT and $Q$-GCN2 provides evidence of their effectiveness in capturing and preserving cyclical and hierarchical patterns within an input graph.\nFor example, on spherical datasets such as Cora and Arxiv, both models outperform existing baselines across downstream tasks. \nAdditionally, the visualizations in Figure 3 (Section 6) and Figure 5 (Appendix H.3) highlight their capability to differentiate between high-order and low-order nodes, effectively capturing the underlying hierarchical organization.\n\n(b)\nIn pseudo-hyperboloid $Q_{\\beta}^{p,q}$, geodesic disconnection occurs when no smooth geodesic exists between two points $x, y$, resulting in an undefined logarithmic map [1], [2].\nAs a result, pseudo-Riemannian methods are unable to project graph data onto the tangent space to perform graph operations, as is commonly done in hyperbolic models.\nOur approach conducts an extrinsic mapping $\\Phi$ and a diffeomorphism $\\Psi$ to transform the pseudo-hyperboloid $Q_{\\beta}^{p,q}$ into the product of spherical and hyperbolic manifolds, each of which is geodesic connected—meaning any pair of points is joined by a smooth geodesic path.\nOur method operates directly on the product manifold, enabling better preservation of embedding quality while reducing computational complexity.\n\n## Weakness 2:\n\n(a) We respectfully disagree with the opinion that our work mainly focus on GNNs.\nIn fact, our study gives balanced attention to both GNNs and Transformer-based models. \nThis is evident from the baseline selection, excluding $Q$-GCN2 (which is also our proposed method), the number of GNN and Transformer baselines is equal.\nFurthermore, SGFormer and HypFormer are among the strongest baselines in Euclidean and hyperbolic spaces, respectively. \nOur ablation study focuses on $Q$-GCN because it represents the most recent and directly relevant pseudo-Riemannian GNN approach.\nDuring the rebuttal phase, we extended our comparisons by including the Transformer-based method FPS-T and conducting additional ablation studies across both GNN and Transformer models in Tables 1, 2, and 3 (in response to Reviewer 8CPi).\nThese results further validate the effectiveness of our proposed approach.\n\n(b) \nAs discussed in Weakness 1a, both $Q$-GT and $Q$-GCN2 demonstrate strong performance on real-world datasets characterized by mixed topological structures across multiple downstream tasks. \nTo assess their ability to capture hierarchical patterns, we introduce tree-structured datasets in Appendix H.3. \nThe resulting visualizations reveal distinct clustering of low-level nodes, indicating that both models effectively preserve the hierarchical nature of the data. \nHowever, conducting more in-depth experiments to illustrate how the models capture spherical and hyperbolic patterns is currently limited by the lack of advanced visualization tools for pseudo-Riemannian spaces, which remains as an open research challenge. \nAs part of future work, we plan to develop a dedicated pseudo-Riemannian visualization tool to further explore and explain the geometric behavior of $Q$-GT and $Q$-GCN2.\n\n## Question 1:\n\nThe term $x_0$ refers to the first coordinate of a point $x = (x_0, x_1, ..., x_{n-1}) \\in \\mathbb{R}^n$.\nIn line 99, $x_0$ appears within the context of the Lorentz model for hyperbolic space, where the first coordinate is required to be positive. \nIn contrast, in Equation (2), $x_0$ is part of the definition of the zero-first submanifold $\\hat{Q}_{\\beta}^{p,q+1}$, which imposes the condition that the first coordinate of any point $x$ on the zero-first submanifold must be zero.\nThe definition of the \"zero-first submanifold\" is central to our work, as it serves as an intermediary space that enables the decomposition of the pseudo-hyperboloid into a product of spherical and hyperbolic manifolds. \nThis submanifold also facilitates the definition of our pseudo-hyperboloid vector addition.\n\n## Question 2:\n\nWe explain how our diffeomorphic framework address the geodesic disconnection issue in our above response in Weakness 1b. \nThe extrinsic function $\\Phi$ and the diffeomorphism $\\Psi$ transform a pseudo-hyperboloid $Q_{\\beta}^{p,q}$ into the product of spherical and hyperbolic manifolds.\nBoth spherical and hyperbolic manifolds are geodesic connected in our work, which enables either tangent space projection or direct learning on the manifolds.\nAs stated in the Introduction (Section 1), our diffemorphism effectively addresses the issue of geodesic disconnection and preserves the cyclical and hierarchical structures of graphs.\n\n## Question 3:\n\nWe would like to thank you for your feedback.\n\nIn Equation (4), the variable $x$ denotes a point on the zero-first submanifold  $\\hat{Q}_{\\beta}^{p,q+1}$ , where the first coordinate $x_0$ is constrained to be zero. \n\nThe next $q+1$ coordinates are represented by $u = (x_1, ..., x_{q+1})$, and the remaining $p$ coordinates by $v = (x_{q+2}, ..., x_{q + p + 1})$. \n\nMeanwhile, the notation $z = (u', v')^T$ refers to a point on the product manifold , composed of two components $u' \\in \\mathbb{S}^q_{|\\beta|}$, $v' \\in \\mathbb{L}^{p}_{\\beta}$.\n\nThe point $z$ can be interpreted as the corresponding representation of $x$ on the product manifold.\n\n## Question 4:\n\nWe do not agree that Theorem 1 is straightforward and we would like to emphasize that this work is the first to decompose pseudo-Riemannian manifold into the product of spherical and hyperbolic spaces.\nImportantly, the transformation from a pseudo-hyperboloid $Q_{\\beta}^{p,q}$ to the product space of spherical and hyperbolic manifolds cannot be done directly; it requires an extrinsic mapping to ensure the existence of a valid diffeomorphism.\nAccording to differential geometry theory, a diffeomorphism between two manifolds $A$ and $B$ exists only if both share the same dimensionality [3].\nFor any pair of points on a pseudo-hyperboloid—regardless of whether they are geodesic disconnected—the diffeomorphic transformation ensures that they become connected by both a spherical and a hyperbolic paths due to the connection property of the product space.\nHowever, a geodesic in a pseudo-hyperboloid corresponds to only one of three geometric types (hyperbolic, flat, or spherical) [1], [2].\nTherefore, it is counterintuitive to assume that a single geodesic in a pseudo-hyperboloid composed of both hyperbolic and spherical paths in the product space. \nThis is precisely why we do not use the distance metric defined in the product space. \nInstead, we rely on the intrinsic geodesic distance of the pseudo-hyperboloid, particularly in link prediction task.\nWe appreciate your suggestions for further clarification and expansion of the theoretical contributions in our work.\nWe propose an alternative and promising direction for theoretical analysis of our diffeomorphic framework is to examine how specific geometric relationships are preserved in the product space.\nFor instance, given a pair of nodes connected by a hyperbolic geodesic in the pseudo-hyperboloid, one could investigate how this relationship is represented after mapping to the product space, and whether the hyperbolic component contributes more significantly to model performance than the spherical one.\n\n## Paper Formatting Concerns:\n\nThank you for your suggestion on the paper formatting.\nWe will change the punctuation mark in line 119 and update the page numbers for references in the next version.\nWe would also revise the formatting issues in Equations (9)–(12) in the paper as suggestion.\n\n[1] Law, M. T.; Stam, J. \"Ultrahyperbolic Representation Learning\". Neural Information Processing Systems. (2020).\n \n[2] Law, M. T. \"Ultrahyperbolic Neural Networks\". Neural Information Processing Systems. (2021).\n\n[3] Joel W. Robbin, Dietmar A. Salamon. \"Introduction to Differential Geometry\" book, p413 (2024)."}]

Author rebuttal:

Meta-review: All the reviewers agree that this is a solid and meaningful contribution that advances the state-of-the-art in geometric machine learning. In some sense it's a natural evolution of continually improving and refining earlier geometric ideas and methods -- the use of hyperbolic and spherical manifolds, geometrically inspired regularization terms in representation learning, etc. -- this time by merging hyperbolic and spherical manifolds (negative and positive curvature spaces), generalizing concepts from transformers to pseudo-Riemannian spaces, and taking into account curvature to speed up learning. The generalization and synthesis of these ideas to manifolds, and validation on a wide variety of datasets, represents a meaningful contribution. Geometric methods in machine learning have now entered the mainstream conversation, and now it is no longer enough to just present a novel application of geometric methods to a machine learning problem -- real progress over the state-of-the-art is essential. This paper appears to clear that bar.
Decision: Accept (poster)