Original: Pseudo-Riemannian Graph Convolutional Networks
Revised: Pseudo-Riemannian Graph Convolutional Networks

Original: Abstract
Revised: Abstract

Original: Graph convolutional networks (GCNs) are powerful frameworks for learning embeddings of graph-structured data.
Revised: Graph convolutional networks (GCNs) are powerful frameworks for learning embeddings of graph-structured data.

Original: GCNs are traditionally studied through the lens of Euclidean geometry.
Revised: GCNs are traditionally studied through the lens of Euclidean geometry.

Original: Recent works find that non-Euclidean Riemannian manifolds provide specific inductive biases for embedding hierarchical or spherical data.
Revised: Recent works find that non-Euclidean Riemannian manifolds provide specific inductive biases for embedding hierarchical or spherical data.

Original: However, they cannot align well with data of mixed graph topologies.
Revised: However, they cannot align well with data of mixed graph topologies.

Original: We consider a larger class of pseudo-Riemannian manifolds that generalize hyperboloid and sphere.
Revised: We consider a larger class of pseudo-Riemannian manifolds that generalize hyperboloid and sphere.

Original: We develop new geodesic tools that allow for extending neural network operations into geodesically disconnected pseudo-Riemannian manifolds.
Revised: We develop new geodesic tools that allow for extending neural network operations into geodesically disconnected pseudo-Riemannian manifolds.

Original: As a consequence, we derive a pseudo-Riemannian GCN that models data in pseudo-Riemannian manifolds of constant nonzero curvature in the context of graph neural networks.
Revised: As a consequence, we derive a pseudo-Riemannian GCN that models data in pseudo-Riemannian manifolds of constant nonzero curvature in the context of graph neural networks.

Original: Our method provides a geometric inductive bias that is sufficiently flexible to model mixed heterogeneous topologies like hierarchical graphs with cycles.
Revised: Our method provides a geometric inductive bias that is sufficiently flexible to model mixed heterogeneous topologies like hierarchical graphs with cycles.

Original: We demonstrate the representational capabilities of this method by applying it to the tasks of graph reconstruction, node classification and link prediction on a series of standard graphs with mixed topologies.
Revised: We demonstrate the representational capabilities of this method by applying it to the tasks of graph reconstruction, node classification and link prediction on a series of standard graphs with mixed topologies.

Original: Empirical results demonstrate that our method outperforms Riemannian counterparts when embedding graphs of complex topologies.
Revised: Empirical results demonstrate that our method outperforms Riemannian counterparts when embedding graphs of complex topologies.

Original:  Introduction
Revised:  Introduction

Original: Learning from graph-structured data is a pivotal task in machine learning, for which graph convolutional networks (GCNs) [1, 2, 3, 4] have emerged as powerful graph representation learning techniques.
Revised: Learning from graph-structured data is a pivotal task in machine learning, for which graph convolutional networks (GCNs) [1, 2, 3, 4] have emerged as powerful graph representation learning techniques.

Original: For this purpose, graphs are usually embedded in Riemannian manifolds equipped with a positive definite metric.
Revised: For this purpose, graphs are usually embedded in Riemannian manifolds equipped with a positive definite metric.

Original: Euclidean geometry is a special case of Riemannian manifolds of constant zero curvature that can be understood intuitively and has well-defined operations.
Revised: Euclidean geometry is a special case of Riemannian manifolds of constant zero curvature that can be understood intuitively and has well-defined operations.

Original: However, the representation power of Euclidean space is limited [5], especially when embedding complex graphs exhibiting hierarchical structures [6].
Revised: However, the representation power of Euclidean space is limited [5], especially when embedding complex graphs exhibiting hierarchical structures [6].

Original: Non-Euclidean Riemannian manifolds of constant curvatures provide an alternative to accommodate specific graph topologies.
Revised: Non-Euclidean Riemannian manifolds of constant curvatures provide an alternative to accommodate specific graph topologies.

Original: For example, hyperbolic manifold of constant negative curvature has exponentially growing volume and is well suited to represent hierarchical structures such as tree-like graphs [7, 8].
Revised: For example, hyperbolic manifold of constant negative curvature has exponentially growing volume and is well suited to represent hierarchical structures such as tree-like graphs [7, 8].

Original: Similarly, spherical manifold of constant positive curvature is suitable for embedding spherical data in various fields [9, 10, 11] including graphs with cycles.
Revised: Similarly, spherical manifold of constant positive curvature is suitable for embedding spherical data in various fields [9, 10, 11] including graphs with cycles.

Original: Some recent works [12, 13, 14, 15, 16] have extended GCNs to such non-Euclidean manifolds and have shown substantial improvements with specific geometries.
Revised: Some recent works [12, 13, 14, 15, 16] have extended GCNs to such non-Euclidean manifolds and have shown substantial improvements with specific geometries.

Original: 36th
Revised: 36th

Original: Systems (NeurIPS 2022).
Revised: Systems (NeurIPS 2022).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: The topologies in real-world graphs [6], however, usually exhibit highly heterogeneous topological structures, which are best represented by different geometrical curvatures.
Revised: The topologies in real-world graphs [6], however, usually exhibit highly heterogeneous topological structures, which are best represented by different geometrical curvatures.

Original: A globally homogeneous geometry lacks the flexibility for modeling complex graphs, resulting in distortional representations [17].
Revised: A globally homogeneous geometry lacks the flexibility for modeling complex graphs, resulting in distortional representations [17].

Original: Instead of using a single manifold, product manifolds [17, 18] combining multiple Riemannian manifolds have shown advantages when embedding graphs of mixed topologies.
Revised: Instead of using a single manifold, product manifolds [17, 18] combining multiple Riemannian manifolds have shown advantages when embedding graphs of mixed topologies.

Original: However, the simple combination of spaces still does not accommodate topologically heterogeneous graphs very well.
Revised: However, the simple combination of spaces still does not accommodate topologically heterogeneous graphs very well.

Original: Furthermore, Riemannian manifolds are equipped with a positive definite metric disallowing for the faithful representation of the negative eigen-spectrum of input similarities [19].
Revised: Furthermore, Riemannian manifolds are equipped with a positive definite metric disallowing for the faithful representation of the negative eigen-spectrum of input similarities [19].

Original: Going beyond Riemannian manifolds, pseudo-Riemannian manifolds equipped with indefinite metrics constitute a larger class of geometries, pseudo-Riemannian manifolds of constant nonzero curvature do not only generalize the hyperbolic and spherical manifolds, but also contain their submanifolds (Cf.
Revised: Going beyond Riemannian manifolds, pseudo-Riemannian manifolds equipped with indefinite metrics constitute a larger class of geometries, pseudo-Riemannian manifolds of constant nonzero curvature do not only generalize the hyperbolic and spherical manifolds, but also contain their submanifolds (Cf.

Original: Fig.
Revised: Fig.

Original: 1), thus providing inductive biases specific to these geometries.
Revised: 1), thus providing inductive biases specific to these geometries.

Original: Pseudo-Riemannian geometry with constant zero curvature (i.e. Lorentzian spacetime) was applied to manifold learning for preserving local information of non-metric data [5] and embedding directed acyclic graph [20].
Revised: Pseudo-Riemannian geometry with constant zero curvature (i.e. Lorentzian spacetime) was applied to manifold learning for preserving local information of non-metric data [5] and embedding directed acyclic graph [20].

Original: To model complex graphs containing both hierarchies and cycles, pseudo-Riemannian manifolds with constant nonzero curvature have recently been applied into graph embeddings using non-parametric learning [21, 22], but the representation power of these works is not on par with the Riemannian counterparts yet, mostly because of the absence of geodesic tools to extend neural network operations into pseudo-Riemannian geometry.
Revised: To model complex graphs containing both hierarchies and cycles, pseudo-Riemannian manifolds with constant nonzero curvature have recently been applied into graph embeddings using non-parametric learning [21, 22], but the representation power of these works is not on par with the Riemannian counterparts yet, mostly because of the absence of geodesic tools to extend neural network operations into pseudo-Riemannian geometry.

Original: In this paper, we take the first step to extend GCNs into pseudo-Riemannian manifolds foregoing the requirement to have a positive definite metric.
Revised: In this paper, we take the first step to extend GCNs into pseudo-Riemannian manifolds foregoing the requirement to have a positive definite metric.

Original: Exploiting pseudo-Riemannian geometry for GCNs is non-trivial because of the geodesical disconnectedness in pseudo-Riemannian geometry.
Revised: Exploiting pseudo-Riemannian geometry for GCNs is non-trivial because of the geodesical disconnectedness in pseudo-Riemannian geometry.

Original: There exist broken points that cannot be smoothly connected by a geodesic, leaving necessary geodesic tools undefined.
Revised: There exist broken points that cannot be smoothly connected by a geodesic, leaving necessary geodesic tools undefined.

Original: To deal with it, we develop novel geodesic tools that empower manipulating representations in geodesically disconnected pseudo-Riemannian manifolds.
Revised: To deal with it, we develop novel geodesic tools that empower manipulating representations in geodesically disconnected pseudo-Riemannian manifolds.

Original: We make it by finding diffeomorphic manifolds that provide alternative geodesic operations that smoothly avoid broken cases.
Revised: We make it by finding diffeomorphic manifolds that provide alternative geodesic operations that smoothly avoid broken cases.

Original: Subsequently, we generalize GCNs to learn representations of complex graphs in pseudo-Riemannian geometry by defining corresponding operations such as linear transformation and tangential aggregation .
Revised: Subsequently, we generalize GCNs to learn representations of complex graphs in pseudo-Riemannian geometry by defining corresponding operations such as linear transformation and tangential aggregation .

Original: Different from previous works, the initial features of GCN could be fully defined in the
Revised: Different from previous works, the initial features of GCN could be fully defined in the

Original: Euclidean space.
Revised: Euclidean space.

Original: Thanks to the diffeomorphic operation that is bijective and differentiable, the standard gradient descent algorithm can be exploited to perform optimization.
Revised: Thanks to the diffeomorphic operation that is bijective and differentiable, the standard gradient descent algorithm can be exploited to perform optimization.

Original: To summarize, our main contributions are as follows: 1) We present neural network operations in pseudo-Riemannian manifolds with novel geodesic tools, to stimulate the applications of pseudoRiemannian geometry in geometric deep learning.
Revised: To summarize, our main contributions are as follows: 1) We present neural network operations in pseudo-Riemannian manifolds with novel geodesic tools, to stimulate the applications of pseudoRiemannian geometry in geometric deep learning.

Original: We present a principled framework, pseudoRiemannian GCN, which generalizes GCNs into pseudo-Riemannian manifolds with indefinite metrics, providing more flexible inductive biases to accommodate complex graphs with mixed topologies.
Revised: We present a principled framework, pseudoRiemannian GCN, which generalizes GCNs into pseudo-Riemannian manifolds with indefinite metrics, providing more flexible inductive biases to accommodate complex graphs with mixed topologies.

Original: 3) Extensive evaluations on three standard tasks demonstrate that our model outperforms baselines that operate in Riemannian manifolds.
Revised: 3) Extensive evaluations on three standard tasks demonstrate that our model outperforms baselines that operate in Riemannian manifolds.

Original: Source code is open available.
Revised: Source code is open available.

Original:  Preliminaries 2.1 Pseudo-Riemannian manifolds
Revised:  Preliminaries 2.1 Pseudo-Riemannian manifolds

Original: A pseudo-Riemannian manifold [23] ( M , g ) is a smooth manifold M equipped with a nondegenerate and indefinite metric tensor g .
Revised: A pseudo-Riemannian manifold [23] ( M , g ) is a smooth manifold M equipped with a nondegenerate and indefinite metric tensor g .

Original: Nondegeneracy means that for a given ξ ∈ T x M , for any ζ ∈ T x
Revised: Nondegeneracy means that for a given ξ ∈ T x M , for any ζ ∈ T x

Original: M we have g x ( ξ , ζ ) = 0 , then ξ = 0 .
Revised: M we have g x ( ξ , ζ ) = 0 , then ξ = 0 .

Original: The metric tensor g induces a scalar product on the tangent space T x M for each point x ∈ M such that g x : T x M × T x M → R , where the tangent space T x M can be seen as the first order local approximation of M around point x .
Revised: The metric tensor g induces a scalar product on the tangent space T x M for each point x ∈ M such that g x : T x M × T x M → R , where the tangent space T x M can be seen as the first order local approximation of M around point x .

Original: The elements of
Revised: The elements of

Original: T x M are called tangent vectors.
Revised: T x M are called tangent vectors.

Original: Indefinity means that the metric tensor could be of arbitrary signs.
Revised: Indefinity means that the metric tensor could be of arbitrary signs.

Original: A principal special case is the Riemannian geometry, where the metric tensor is positive definite (i.e.
Revised: A principal special case is the Riemannian geometry, where the metric tensor is positive definite (i.e.

Original: ∀ ξ ∈ T x M , g x ( ξ , ξ ) > 0 iff ξ ̸ = 0 ).
Revised: ∀ ξ ∈ T x M , g x ( ξ , ξ ) > 0 iff ξ ̸ = 0 ).

Original: Pseudo-hyperboloid By analogy with hyperboloid and sphere in Euclidean space.
Revised: Pseudo-hyperboloid By analogy with hyperboloid and sphere in Euclidean space.

Original: Pseudohyperboloids are defined as the submanifolds in the ambient pseudo-Euclidean space R s,t +1 with the dimensionality of d
Revised: Pseudohyperboloids are defined as the submanifolds in the ambient pseudo-Euclidean space R s,t +1 with the dimensionality of d

Original: = s + t + 1 that uses the scalar product as ∀ x , y ∈ R s,t +1 , ⟨ x , y ⟩ t = − (cid:80) ti
Revised: = s + t + 1 that uses the scalar product as ∀ x , y ∈ R s,t +1 , ⟨ x , y ⟩ t = − (cid:80) ti

Original: =
Revised: =

Original: x
Revised: x

Original: i y i
Revised: i y i

Original: + (cid:80) s
Revised: + (cid:80) s

Original: + t j = t
Revised: + t j = t

Original: +1 x j y j .
Revised: +1 x j y j .

Original: The scalar product induces a norm ∥ x ∥ 2 t = ⟨ x , x ⟩ t that can be used to define a pseudo-hyperboloid Q s,tβ = (cid:110) x
Revised: The scalar product induces a norm ∥ x ∥ 2 t = ⟨ x , x ⟩ t that can be used to define a pseudo-hyperboloid Q s,tβ = (cid:110) x

Original: = ( x 0 , x 1 , · · · , x s + t ) ⊤ ∈ R s,t +1 :
Revised: = ( x 0 , x 1 , · · · , x s + t ) ⊤ ∈ R s,t +1 :

Original: ∥ x ∥ 2 t = β (cid:111) , where β is a nonzero real number parameter of curvature.
Revised: ∥ x ∥ 2 t = β (cid:111) , where β is a nonzero real number parameter of curvature.

Original: Q s,tβ is called pseudo-sphere when β > 0 and pseudo-hyperboloid when β < 0 .
Revised: Q s,tβ is called pseudo-sphere when β > 0 and pseudo-hyperboloid when β < 0 .

Original: Since Q s,tβ is interchangeable with Q t +1 ,s − 1 − β , we consider the pseudo-hyperbololid here.
Revised: Since Q s,tβ is interchangeable with Q t +1 ,s − 1 − β , we consider the pseudo-hyperbololid here.

Original: Following the terminology of special relativity, a point in Q s,tβ can be interpreted as an event [5], where the first t + 1 dimensions are time dimensions and the last s dimensions are space dimensions.
Revised: Following the terminology of special relativity, a point in Q s,tβ can be interpreted as an event [5], where the first t + 1 dimensions are time dimensions and the last s dimensions are space dimensions.

Original: Hyperbolic H and spherical S manifolds can be defined as the special cases of pseudo-hyperboloids by setting all time dimensions except one to be zero and setting all space dimensions to be zero, respectively, i.e. H β = Q s, 1 β , S − β
Revised: Hyperbolic H and spherical S manifolds can be defined as the special cases of pseudo-hyperboloids by setting all time dimensions except one to be zero and setting all space dimensions to be zero, respectively, i.e. H β = Q s, 1 β , S − β

Original: = Q 0 ,tβ .
Revised: = Q 0 ,tβ .

Original: 2.2 Geodesic tools of pseudo-hyperboloid
Revised: 2.2 Geodesic tools of pseudo-hyperboloid

Original: Geodesic A generalization of a straight-line in the Euclidean space to a manifold is called the geodesic [12, 24].
Revised: Geodesic A generalization of a straight-line in the Euclidean space to a manifold is called the geodesic [12, 24].

Original: Formally, a geodesic γ is defined as a constant speed curve γ : τ (cid:55)→ γ ( τ ) ∈
Revised: Formally, a geodesic γ is defined as a constant speed curve γ : τ (cid:55)→ γ ( τ ) ∈

Original: M , τ ∈ [0 , 1] joining two points x , y ∈ M that minimizes the length, where the length of a curve is given by L ( γ ) =
Revised: M , τ ∈ [0 , 1] joining two points x , y ∈ M that minimizes the length, where the length of a curve is given by L ( γ ) =

Original: (cid:82) 1 0 (cid:113)(cid:13)(cid:13) ddt γ ( τ ) (cid:13)(cid:13) γ ( τ ) dt .
Revised: (cid:82) 1 0 (cid:113)(cid:13)(cid:13) ddt γ ( τ ) (cid:13)(cid:13) γ ( τ ) dt .

Original: The geodesic holds that γ ∗ = arg min γ L ( γ ) , such that γ (0) = x , γ (1) =
Revised: The geodesic holds that γ ∗ = arg min γ L ( γ ) , such that γ (0) = x , γ (1) =

Original: y , and (cid:13)(cid:13) ddτ γ ( τ ) (cid:13)(cid:13) γ ( τ ) = 1 .
Revised: y , and (cid:13)(cid:13) ddτ γ ( τ ) (cid:13)(cid:13) γ ( τ ) = 1 .

Original: By the means of the geodesic, the distance between x , y ∈ Q s,tβ is defined as the arc length of geodesic γ ( τ ) .
Revised: By the means of the geodesic, the distance between x , y ∈ Q s,tβ is defined as the arc length of geodesic γ ( τ ) .

Original: Exponential and logarithmic maps The connections between manifolds and tangent space are established by the differentiable exponential map and logarithmic map.
Revised: Exponential and logarithmic maps The connections between manifolds and tangent space are established by the differentiable exponential map and logarithmic map.

Original: The exponential map at x is defined as exp x ( ξ ) =
Revised: The exponential map at x is defined as exp x ( ξ ) =

Original: γ (1) , which gives a way to project a vector ξ ∈ T x M to a point exp x ( ξ ) ∈ M on the manifold.
Revised: γ (1) , which gives a way to project a vector ξ ∈ T x M to a point exp x ( ξ ) ∈ M on the manifold.

Original: The logarithmic map log x : M → T x M is defined as the inverse of the exponential map (i.e. log x = exp − 1 x ).
Revised: The logarithmic map log x : M → T x M is defined as the inverse of the exponential map (i.e. log x = exp − 1 x ).

Original: Note that since Q s,tβ is a geodesically complete manifold, the domain of the exponential map D x is hence defined on the entire tangent space, i.e. D x = T x Q s,tβ .
Revised: Note that since Q s,tβ is a geodesically complete manifold, the domain of the exponential map D x is hence defined on the entire tangent space, i.e. D x = T x Q s,tβ .

Original: However, as we will explain later, the logarithmic map log x ( y ) is only defined when there exists a a lengthminimizing geodesic between x , y ∈ Q s,tβ .
Revised: However, as we will explain later, the logarithmic map log x ( y ) is only defined when there exists a a lengthminimizing geodesic between x , y ∈ Q s,tβ .

Original: More details can be found in the Appendix C.
Revised: More details can be found in the Appendix C.

Original: Geodesical connectedness
Revised: Geodesical connectedness

Original: A pseudo-Riemannian manifold M is connected iff any two points of
Revised: A pseudo-Riemannian manifold M is connected iff any two points of

Original: M can be joined by a piecewise (broken) geodesic with each piece being a smooth geodesic.
Revised: M can be joined by a piecewise (broken) geodesic with each piece being a smooth geodesic.

Original: The manifold is geodesically connected (or g-connected )
Revised: The manifold is geodesically connected (or g-connected )

Original: iff any two points can be smoothly connected by a geodesic, where the two points are called g-connected , otherwise called g-disconnected .
Revised: iff any two points can be smoothly connected by a geodesic, where the two points are called g-connected , otherwise called g-disconnected .

Original: Different from Riemannian manifolds in which the geodesical completeness implies the g-connectedness (Hopf–Rinow theorem [25]), pseudo-hyperboloid is a geodesically complete but not g-connected manifold where there exist points that cannot be smoothly connected by a geodesic [26].
Revised: Different from Riemannian manifolds in which the geodesical completeness implies the g-connectedness (Hopf–Rinow theorem [25]), pseudo-hyperboloid is a geodesically complete but not g-connected manifold where there exist points that cannot be smoothly connected by a geodesic [26].

Original: Formally, in the pseudo-hyperboloid, two points x , y ∈ Q s,tβ are g-connected iff ⟨ x , y ⟩ t < | β | .
Revised: Formally, in the pseudo-hyperboloid, two points x , y ∈ Q s,tβ are g-connected iff ⟨ x , y ⟩ t < | β | .

Original: The set of g-connected points of x ∈ Q s,tβ is denoted as its normal neighborhood U x =
Revised: The set of g-connected points of x ∈ Q s,tβ is denoted as its normal neighborhood U x =

Original: (cid:110)
Revised: (cid:110)

Original: y ∈ Q s,tβ : ⟨ x , y ⟩ t <
Revised: y ∈ Q s,tβ : ⟨ x , y ⟩ t <

Original: | β | (cid:111) .
Revised: | β | (cid:111) .

Original: For g-disconnected points x , y ∈ Q s,tβ , there does not exist a tangent vector ξ such that y =
Revised: For g-disconnected points x , y ∈ Q s,tβ , there does not exist a tangent vector ξ such that y =

Original: exp β x ( ξ ) , which implies that its inverse log β x ( · ) is only defined in the normal neighborhood of x .
Revised: exp β x ( ξ ) , which implies that its inverse log β x ( · ) is only defined in the normal neighborhood of x .

Original: In a nutshell, the geodesic tools for the g-disconnected cases are not well-defined, making it impossible to define corresponding vector operations.
Revised: In a nutshell, the geodesic tools for the g-disconnected cases are not well-defined, making it impossible to define corresponding vector operations.

Original:  Pseudo-Riemannian GCNs
Revised:  Pseudo-Riemannian GCNs

Original: In this section, we first describe how to tackle the g-disconnectedness in pseudo-Riemannian manifolds.
Revised: In this section, we first describe how to tackle the g-disconnectedness in pseudo-Riemannian manifolds.

Original: Then we present the pseudo-Riemannian GCNs based on the proposed geodesic tools.
Revised: Then we present the pseudo-Riemannian GCNs based on the proposed geodesic tools.

Original: 3.1 Diffeomorphic geodesic tools
Revised: 3.1 Diffeomorphic geodesic tools

Original: One standard way to tackle the g-disconnectedness in differential geometry is to introduce diffeomorphic manifolds in which the operations are well-defined.
Revised: One standard way to tackle the g-disconnectedness in differential geometry is to introduce diffeomorphic manifolds in which the operations are well-defined.

Original: A diffeomorphic manifold can be derived from a diffeomorphism, defined as follows.
Revised: A diffeomorphic manifold can be derived from a diffeomorphism, defined as follows.

Original: Definition 1 (Diffeomorphism [23]) .
Revised: Definition 1 (Diffeomorphism [23]) .

Original: Given two manifolds M and M ′ , a smooth map ψ : M → M ′ is called a diffeomorphism if ψ is bijective and its inverse ψ − 1 is smooth as well.
Revised: Given two manifolds M and M ′ , a smooth map ψ : M → M ′ is called a diffeomorphism if ψ is bijective and its inverse ψ − 1 is smooth as well.

Original: If a diffeomorphism between M and M ′ exists, we call them diffeomorphic and write M ≃ M ′ .
Revised: If a diffeomorphism between M and M ′ exists, we call them diffeomorphic and write M ≃ M ′ .

Original: For pseudo-Riemannian manifolds, the following diffeomorphism [21] decomposes pseudohyperboloid into the product manifolds of an unit sphere and the Euclidean space.
Revised: For pseudo-Riemannian manifolds, the following diffeomorphism [21] decomposes pseudohyperboloid into the product manifolds of an unit sphere and the Euclidean space.

Original: Theorem 1 (Theorem 4.1 in [21]) .
Revised: Theorem 1 (Theorem 4.1 in [21]) .

Original: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ :
Revised: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ :

Original: Q s,tβ → S t 1 × R s that maps x into the product manifolds of an unit sphere and the Euclidean space.
Revised: Q s,tβ → S t 1 × R s that maps x into the product manifolds of an unit sphere and the Euclidean space.

Original: The diffeomorphism is given in the Appendix D.1.
Revised: The diffeomorphism is given in the Appendix D.1.

Original: In light of this, we introduce a new diffeomorphism that maps x to the product manifolds of sphere with curvature − 1 /β and the Euclidean space.
Revised: In light of this, we introduce a new diffeomorphism that maps x to the product manifolds of sphere with curvature − 1 /β and the Euclidean space.

Original: Theorem 2.
Revised: Theorem 2.

Original: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ : Q s,tβ → S t − β × R s that maps x into the product manifolds of a sphere and the Euclidean space (proof in the Appendix D.2).
Revised: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ : Q s,tβ → S t − β × R s that maps x into the product manifolds of a sphere and the Euclidean space (proof in the Appendix D.2).

Original: Compared with Theorem 1, this diffeomorphism preserves the curvatures in the diffeomorphic components, making it satisfy some geometric properties, e.g. the mapped point x t ∈ S t − β still lies on the surface of the pseudo-hyperboloid, making moving the tangent vectors from the pseudo-hyperboloid to the diffeomorphic manifold easy as we explained later.
Revised: Compared with Theorem 1, this diffeomorphism preserves the curvatures in the diffeomorphic components, making it satisfy some geometric properties, e.g. the mapped point x t ∈ S t − β still lies on the surface of the pseudo-hyperboloid, making moving the tangent vectors from the pseudo-hyperboloid to the diffeomorphic manifold easy as we explained later.

Original: We call ψ as the spherical projection.
Revised: We call ψ as the spherical projection.

Original: Exponential and logarithmic maps.
Revised: Exponential and logarithmic maps.

Original: Considering that pseudo-hyperboloid Q s,tβ is gdisconnected , we propose to transfer the logmap and expmap into the diffeomorphic manifold ψ : Q s,tβ → S t − β × R s , since the product manifold S t − β × R s is g-connected .
Revised: Considering that pseudo-hyperboloid Q s,tβ is gdisconnected , we propose to transfer the logmap and expmap into the diffeomorphic manifold ψ : Q s,tβ → S t − β × R s , since the product manifold S t − β × R s is g-connected .

Original: To map tangent vectors between tangent space of Q s,tβ and S t − β × R s , we exploit pushforward that induce a linear approximation of smooth maps on tangent spaces.
Revised: To map tangent vectors between tangent space of Q s,tβ and S t − β × R s , we exploit pushforward that induce a linear approximation of smooth maps on tangent spaces.

Original: Definition 2 (Pushforward) .
Revised: Definition 2 (Pushforward) .

Original: Suppose that ψ : M → M ′ is a smooth map, then the differential of ψ : dψ at point x is a linear map from the tangent space of M at x to the tangent space of M ′ at ψ ( x ) .
Revised: Suppose that ψ : M → M ′ is a smooth map, then the differential of ψ : dψ at point x is a linear map from the tangent space of M at x to the tangent space of M ′ at ψ ( x ) .

Original: Namely, dψ : T x M → T ψ ( x ) M ′
Revised: Namely, dψ : T x M → T ψ ( x ) M ′

Original: Intuitively, pushforward can be used to push tangent vectors on T x Q s,tβ forward to tangent vectors on T ψ ( x ) S t − β × R s .
Revised: Intuitively, pushforward can be used to push tangent vectors on T x Q s,tβ forward to tangent vectors on T ψ ( x ) S t − β × R s .

Original: Based on this, the new logmap and its inverse expmap can be defined by Eq.
Revised: Based on this, the new logmap and its inverse expmap can be defined by Eq.

Original: [Equation]
Revised: [Equation]

Original: (1) where ψ ( · ) is the spherical projection and ψ − 1 ( · ) is the inverse.
Revised: (1) where ψ ( · ) is the spherical projection and ψ − 1 ( · ) is the inverse.

Original: The mapping of tangent vectors is achieved by pushforward operations.
Revised: The mapping of tangent vectors is achieved by pushforward operations.

Original: The operations log S t − β × R s ( · ) and exp S t − β × R s ( · ) in the product manifolds can be defined as the concatenation of corresponding operations in different components.
Revised: The operations log S t − β × R s ( · ) and exp S t − β × R s ( · ) in the product manifolds can be defined as the concatenation of corresponding operations in different components.

Original: [Equation]
Revised: [Equation]

Original: where || denotes the concatenation, x ′ =
Revised: where || denotes the concatenation, x ′ =

Original: ψ S ( x ) consists of spherical features x ′ t ∈ S t − β
Revised: ψ S ( x ) consists of spherical features x ′ t ∈ S t − β

Original: and Euclidean features x ′ s ∈ R s .
Revised: and Euclidean features x ′ s ∈ R s .

Original: ξ is the tangent vector induced by x on Q s,tβ .
Revised: ξ is the tangent vector induced by x on Q s,tβ .

Original: We choose points where space dimension s = 0 as the reference points due to the following property.
Revised: We choose points where space dimension s = 0 as the reference points due to the following property.

Original: Theorem 3.
Revised: Theorem 3.

Original: For any reference point x = (cid:18) ts (cid:19) ∈ Q s,tβ with space dimension s = 0 , the induced tangent space of Q s,tβ is equal to the tangent space of its diffeomorphic manifold S t − β ×
Revised: For any reference point x = (cid:18) ts (cid:19) ∈ Q s,tβ with space dimension s = 0 , the induced tangent space of Q s,tβ is equal to the tangent space of its diffeomorphic manifold S t − β ×

Original: R s , namely,
Revised: R s , namely,

Original: T x ( S t − β × R s )
Revised: T x ( S t − β × R s )

Original: =
Revised: =

Original: T x Q s,tβ .
Revised: T x Q s,tβ .

Original: (proof in the Appendix D.3).
Revised: (proof in the Appendix D.3).

Original: The intuition of proof is that if space dimension s = 0 , the pushforward (differential) function just influences time dimension, for which the mapping is just an identity function (see Appendix
Revised: The intuition of proof is that if space dimension s = 0 , the pushforward (differential) function just influences time dimension, for which the mapping is just an identity function (see Appendix

Original: D.2).
Revised: D.2).

Original: In this way, although we transfer logmap and expmap to the diffeomorphic manifold S t − β ×
Revised: In this way, although we transfer logmap and expmap to the diffeomorphic manifold S t − β ×

Original: R s , the diffeomorphic operations (cid:99) log x ( · ) and (cid:100) exp x ( · ) are still bijective functions from the pseudohyperboloid to the tangent space of the manifold itself.
Revised: R s , the diffeomorphic operations (cid:99) log x ( · ) and (cid:100) exp x ( · ) are still bijective functions from the pseudohyperboloid to the tangent space of the manifold itself.

Original: Hence, our final operations are actually still defined in the tangent space of the pseudo-hyperboloid.
Revised: Hence, our final operations are actually still defined in the tangent space of the pseudo-hyperboloid.

Original: Note that such property only holds when our Theorem 2 is applied and the special reference points with space dimension s = 0 are chosen.
Revised: Note that such property only holds when our Theorem 2 is applied and the special reference points with space dimension s = 0 are chosen.

Original: By leveraging the new logmap and expmap , we further formulate the diffeomorphic version of tangential operations as follows.
Revised: By leveraging the new logmap and expmap , we further formulate the diffeomorphic version of tangential operations as follows.

Original: Tangential operations.
Revised: Tangential operations.

Original: For function f : R d → R d ′ , the pseudo-hyperboloid version f ⊗ : Q s,tβ →
Revised: For function f : R d → R d ′ , the pseudo-hyperboloid version f ⊗ : Q s,tβ →

Original: Q s ′ ,t ′ β with s + t = d and s ′ + t ′ =
Revised: Q s ′ ,t ′ β with s + t = d and s ′ + t ′ =

Original: d ′ can be defined by the means of (cid:99) log β x ( · ) and (cid:100)
Revised: d ′ can be defined by the means of (cid:99) log β x ( · ) and (cid:100)

Original: exp β x ( · ) as Eq.
Revised: exp β x ( · ) as Eq.

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) where x is the reference point.
Revised: ) where x is the reference point.

Original: Note that this function is a morphism (i.e. ( f ◦ g ) ⊗ = f ⊗ ◦ g ⊗ ) and direction preserving (i.e. f ⊗ ( · ) /
Revised: Note that this function is a morphism (i.e. ( f ◦ g ) ⊗ = f ⊗ ◦ g ⊗ ) and direction preserving (i.e. f ⊗ ( · ) /

Original: ∥ f ⊗ ( · ) ∥
Revised: ∥ f ⊗ ( · ) ∥

Original: = f ( · ) /
Revised: = f ( · ) /

Original: ∥ f ( · ) ∥ ) [12], making it a natural way to define pseudo-hyperboloid version of vector operations such like scalar multiplication, matrix-vector multiplication, tangential aggregation and point-wise non-linearity
Revised: ∥ f ( · ) ∥ ) [12], making it a natural way to define pseudo-hyperboloid version of vector operations such like scalar multiplication, matrix-vector multiplication, tangential aggregation and point-wise non-linearity

Original: and so on.
Revised: and so on.

Original: Parallel transport.
Revised: Parallel transport.

Original: Parallel transport is the generalization of Euclidean translation into manifolds.
Revised: Parallel transport is the generalization of Euclidean translation into manifolds.

Original: Formally, for any two points x and y connected by a geodesic, parallel transport P β x → y ( ξ ) : T x M →
Revised: Formally, for any two points x and y connected by a geodesic, parallel transport P β x → y ( ξ ) : T x M →

Original: T y M is an isomorphism between two tangent spaces by moving one tangent vector ζ ∈ T x M with tangent direction ξ ∈ T x M to another tangent space T y M .
Revised: T y M is an isomorphism between two tangent spaces by moving one tangent vector ζ ∈ T x M with tangent direction ξ ∈ T x M to another tangent space T y M .

Original: The parallel transport in pseudohyperboloid can be defined as the combination of Riemannian parallel transport [27].
Revised: The parallel transport in pseudohyperboloid can be defined as the combination of Riemannian parallel transport [27].

Original: However, the parallel transport has not been defined when there does not exist a geodesic between x and y .
Revised: However, the parallel transport has not been defined when there does not exist a geodesic between x and y .

Original: i.e., the tangent vector ζ induced by x can not be transported to the tangent space of points outside of the normal neighborhood U x .
Revised: i.e., the tangent vector ζ induced by x can not be transported to the tangent space of points outside of the normal neighborhood U x .

Original: Intuitively, the normal neighborhoods satisfy the following property.
Revised: Intuitively, the normal neighborhoods satisfy the following property.

Original: Theorem 4.
Revised: Theorem 4.

Original: For any point x ∈ Q s,tβ , the union of the normal neighborhood of x and the normal neighborhood of its antipodal point − x cover the entire manifold.
Revised: For any point x ∈ Q s,tβ , the union of the normal neighborhood of x and the normal neighborhood of its antipodal point − x cover the entire manifold.

Original: Namely, U x ∪ U − x = Q s,tβ (proof in the Appendix D.4).
Revised: Namely, U x ∪ U − x = Q s,tβ (proof in the Appendix D.4).

Original: This theorem ensures that if a point y / ∈ U x , its antipodal point − y ∈
Revised: This theorem ensures that if a point y / ∈ U x , its antipodal point − y ∈

Original: U x .
Revised: U x .

Original: Besides, T y M is parallel to T − y M .
Revised: Besides, T y M is parallel to T − y M .

Original: Hence, P β x → y can be alternatively defined as P β x →− y for broken points.
Revised: Hence, P β x → y can be alternatively defined as P β x →− y for broken points.

Original: This result is crucial to define the pseudo-hyperbolic addition, such as bias translation, detailed in section 3.2.
Revised: This result is crucial to define the pseudo-hyperbolic addition, such as bias translation, detailed in section 3.2.

Original: Broken geodesic distance.
Revised: Broken geodesic distance.

Original: By the means of geodesic, the induced distance between x and y in pseudo-hyperboloid is defined as the arc length of geodesic γ ( τ ) , given by d γ ( x , y )
Revised: By the means of geodesic, the induced distance between x and y in pseudo-hyperboloid is defined as the arc length of geodesic γ ( τ ) , given by d γ ( x , y )

Original: = (cid:113) ∥ log x ( y ) ∥ 2 t .
Revised: = (cid:113) ∥ log x ( y ) ∥ 2 t .

Original: For broken cases in which log x ( y ) is not defined, one approach is to use approximation like [21].
Revised: For broken cases in which log x ( y ) is not defined, one approach is to use approximation like [21].

Original: Different from that, we define following closed-form distance, given by Eq.
Revised: Different from that, we define following closed-form distance, given by Eq.

Original: [Equation]
Revised: [Equation]

Original: The intuition is that when x , y ∈ Q s,tβ are g-disconnected , we consider the distance as d γ ( x , y ) =
Revised: The intuition is that when x , y ∈ Q s,tβ are g-disconnected , we consider the distance as d γ ( x , y ) =

Original: d γ ( x , − x ) + d γ ( − x , y ) or d γ ( x , y ) =
Revised: d γ ( x , − x ) + d γ ( − x , y ) or d γ ( x , y ) =

Original: d γ ( x , − y )
Revised: d γ ( x , − y )

Original: + d γ ( − y , y ) .
Revised: + d γ ( − y , y ) .

Original: Since d γ ( x , − x ) = d γ ( − y , y ) =
Revised: Since d γ ( x , − x ) = d γ ( − y , y ) =

Original: π (cid:112) | β | is a constant and d γ ( − x , y ) =
Revised: π (cid:112) | β | is a constant and d γ ( − x , y ) =

Original: d γ ( x , − y ) , the distance between broken points can be calculated as d γ ( x , y ) =
Revised: d γ ( x , − y ) , the distance between broken points can be calculated as d γ ( x , y ) =

Original: π (cid:112) | β | + d γ ( x , − y ) .
Revised: π (cid:112) | β | + d γ ( x , − y ) .

Original: To clarify theoretical contributions, our Theorem 2 is nessasary for our Theorem 3 while Theorem 3 is nessasary for transforming the GCN operations directly into the tangent space of the pseudohyperboloid.
Revised: To clarify theoretical contributions, our Theorem 2 is nessasary for our Theorem 3 while Theorem 3 is nessasary for transforming the GCN operations directly into the tangent space of the pseudohyperboloid.

Original: Besides, we are the first to formulate the diffeomorphic expmap , logmap and tangential operations of pseudo-hyperboloid to avoid broken cases.
Revised: Besides, we are the first to formulate the diffeomorphic expmap , logmap and tangential operations of pseudo-hyperboloid to avoid broken cases.

Original: The theoretical properties of parallel transport and geodesic distance are discussed in the literature [21, 27].
Revised: The theoretical properties of parallel transport and geodesic distance are discussed in the literature [21, 27].

Original: However, we re-formulate parallel transport with Theorem 4 to avoid broken issues and propose a new distance measure using the broken geodesic (Eq.4 ), which is different from the approximated distance in [21].
Revised: However, we re-formulate parallel transport with Theorem 4 to avoid broken issues and propose a new distance measure using the broken geodesic (Eq.4 ), which is different from the approximated distance in [21].

Original: 3.2 Model architecture
Revised: 3.2 Model architecture

Original: GCNs can be interpreted as performing neighborhood aggregation after a linear transformation on node features of each layer.
Revised: GCNs can be interpreted as performing neighborhood aggregation after a linear transformation on node features of each layer.

Original: We present pseudo-Riemannian GCNs ( Q -GCN) by deriving corresponding operations with the developed geodesic tools in the Q s,tβ .
Revised: We present pseudo-Riemannian GCNs ( Q -GCN) by deriving corresponding operations with the developed geodesic tools in the Q s,tβ .

Original: Feature initialization.
Revised: Feature initialization.

Original: We first map the features from Euclidean space to pseudo-hyperboloid, considering that the input features of nodes usually live in Euclidean space.
Revised: We first map the features from Euclidean space to pseudo-hyperboloid, considering that the input features of nodes usually live in Euclidean space.

Original: Following the feature transformation from Euclidean space to pseudo-hyperboloid in [21], we initialize the node features by performing a differentiable mapping φ : R t +1 ∗ × R s → Q s,tβ that can be implemented by a double projection [21] based on Theorem 2, i.e. φ = ψ − 1 ◦ ψ .
Revised: Following the feature transformation from Euclidean space to pseudo-hyperboloid in [21], we initialize the node features by performing a differentiable mapping φ : R t +1 ∗ × R s → Q s,tβ that can be implemented by a double projection [21] based on Theorem 2, i.e. φ = ψ − 1 ◦ ψ .

Original: The intuition is that we first map the Euclidean features into diffeomorphic manifolds S t − β × R s via ψ ( · ) , and then map them into the pseudo-hyperboloid Q s,tβ via ψ − 1 ( · ) , where the mapping functions are given by Eq.
Revised: The intuition is that we first map the Euclidean features into diffeomorphic manifolds S t − β × R s via ψ ( · ) , and then map them into the pseudo-hyperboloid Q s,tβ via ψ − 1 ( · ) , where the mapping functions are given by Eq.

Original: [Equation]
Revised: [Equation]

Original: ) where x = ( t , s ) ⊤ ∈ Q s,tβ with t ∈ R t ∗ and s ∈ R s .
Revised: ) where x = ( t , s ) ⊤ ∈ Q s,tβ with t ∈ R t ∗ and s ∈ R s .

Original: z = ( u , v ) ⊤ ∈ S t − β ×
Revised: z = ( u , v ) ⊤ ∈ S t − β ×

Original: R s with u ∈ S t − β and v ∈ R s .
Revised: R s with u ∈ S t − β and v ∈ R s .

Original: Tangential aggregation.
Revised: Tangential aggregation.

Original: The linear combination of neighborhood features is lifted to the tangent space, which is an intrinsic operation in differential manifolds [28, 13].
Revised: The linear combination of neighborhood features is lifted to the tangent space, which is an intrinsic operation in differential manifolds [28, 13].

Original: Specifically, Q -GCN aggregates neighbours’ embeddings in the tangent space of the reference point o before passing through a tangential activation function, and then projects the updated representation back to the manifold.
Revised: Specifically, Q -GCN aggregates neighbours’ embeddings in the tangent space of the reference point o before passing through a tangential activation function, and then projects the updated representation back to the manifold.

Original: Formally, at each layer ℓ , the updated features of each node i are defined as Eq.
Revised: Formally, at each layer ℓ , the updated features of each node i are defined as Eq.

Original: [Equation]
Revised: [Equation]

Original: where σ ( · ) is the activation function, β ℓ and β ℓ +1 are two layer-wise curvatures, N ( i ) denotes the one-hop neighborhoods of node i , and the ⊗ , ⊕ denote two basic operations, i.e. tangential transformation and bias translation, respectively.
Revised: where σ ( · ) is the activation function, β ℓ and β ℓ +1 are two layer-wise curvatures, N ( i ) denotes the one-hop neighborhoods of node i , and the ⊗ , ⊕ denote two basic operations, i.e. tangential transformation and bias translation, respectively.

Original: Tangential transformation.
Revised: Tangential transformation.

Original: We perform Euclidean transformations on the tangent space by leveraging the expmap and logmap in Eq.
Revised: We perform Euclidean transformations on the tangent space by leveraging the expmap and logmap in Eq.

Original: Specifically, we first project the hidden feature into the tangent space of south pole o =
Revised: Specifically, we first project the hidden feature into the tangent space of south pole o =

Original: [ | β | , 0 , , ..., 0] using logmap and then perform Euclidean matrix multiplication.
Revised: [ | β | , 0 , , ..., 0] using logmap and then perform Euclidean matrix multiplication.

Original: Afterwards, the transformed features are mapped back to the manifold using expmap .
Revised: Afterwards, the transformed features are mapped back to the manifold using expmap .

Original: Formally, at each layer ℓ , the tangential transformation is given by W ℓ ⊗ β
Revised: Formally, at each layer ℓ , the tangential transformation is given by W ℓ ⊗ β

Original: [Equation]
Revised: [Equation]

Original: where ⊗ β denotes the pseudo-hyperboloid tangential multiplication, and W ℓ ∈ R d ′ ×
Revised: where ⊗ β denotes the pseudo-hyperboloid tangential multiplication, and W ℓ ∈ R d ′ ×

Original: d denotes the layer-wise learnable matrix in Euclidean space.
Revised: d denotes the layer-wise learnable matrix in Euclidean space.

Original: Bias translation.
Revised: Bias translation.

Original: It is noteworthy that simply stacking multiple layers of the tangential transformation would collapse the composition [28, 12], i.e. exp β o ...
Revised: It is noteworthy that simply stacking multiple layers of the tangential transformation would collapse the composition [28, 12], i.e. exp β o ...

Original: ( W 1 log β
Revised: ( W 1 log β

Original: o (exp β o ( W 0 log β
Revised: o (exp β o ( W 0 log β

Original: o ( x ))))
Revised: o ( x ))))

Original: =
Revised: =

Original: exp β
Revised: exp β

Original: o ( W 0 × W 1 × ...
Revised: o ( W 0 × W 1 × ...

Original: × log β
Revised: × log β

Original: o ( x )) , which means that these multiplications can simply be performed in Euclidean space except the first logmap and last expmap .
Revised: o ( x )) , which means that these multiplications can simply be performed in Euclidean space except the first logmap and last expmap .

Original: To avoid model collapsing , we perform bias translation after the tangential transformation.
Revised: To avoid model collapsing , we perform bias translation after the tangential transformation.

Original: By the means of pseudo-hyperboloid parallel transport , the bias translation can be performed by parallel transporting a tangent vector b ℓ ∈ T o Q s,tβ to the tangent space of the point of interest.
Revised: By the means of pseudo-hyperboloid parallel transport , the bias translation can be performed by parallel transporting a tangent vector b ℓ ∈ T o Q s,tβ to the tangent space of the point of interest.

Original: Finally, the transported tangent vector is mapped back to the manifold with expmap .
Revised: Finally, the transported tangent vector is mapped back to the manifold with expmap .

Original: Considering that (cid:100) exp( · ) is only defined at point x ∈ Q s,tβ with the space dimension x s = 0 , we perform the original exp Q s,tβ ( · ) at the point of interest.
Revised: Considering that (cid:100) exp( · ) is only defined at point x ∈ Q s,tβ with the space dimension x s = 0 , we perform the original exp Q s,tβ ( · ) at the point of interest.

Original: The bias translation is formally given by:
Revised: The bias translation is formally given by:

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) h ℓ = W ℓ ⊗ β h ℓ , ⊕ β denotes the pseudo-hyperboloid addition.
Revised: where (cid:101) h ℓ = W ℓ ⊗ β h ℓ , ⊕ β denotes the pseudo-hyperboloid addition.

Original: For the broken cases where ⟨ o , (cid:101) h ℓ ⟩ t ≥ | β | , the parallel transport P β o → (cid:101) h ℓ is not defined.
Revised: For the broken cases where ⟨ o , (cid:101) h ℓ ⟩ t ≥ | β | , the parallel transport P β o → (cid:101) h ℓ is not defined.

Original: In this case, we parallel transport b ℓ to the tangent space of the antipodal point − (cid:101) h ℓ , and then perform exp β − (cid:101) h ℓ to map it back to the manifold.
Revised: In this case, we parallel transport b ℓ to the tangent space of the antipodal point − (cid:101) h ℓ , and then perform exp β − (cid:101) h ℓ to map it back to the manifold.

Original: Note that the case ⟨ o , (cid:101) h ℓ ⟩ t =
Revised: Note that the case ⟨ o , (cid:101) h ℓ ⟩ t =

Original: | β | occurs if and only if h = − o , in which case P β o →− (cid:101) h ℓ (cid:0) b ℓ (cid:1) =
Revised: | β | occurs if and only if h = − o , in which case P β o →− (cid:101) h ℓ (cid:0) b ℓ (cid:1) =

Original: P β o →− o (cid:0) b ℓ (cid:1) = − b .
Revised: P β o →− o (cid:0) b ℓ (cid:1) = − b .

Original: 3.3 Model training
Revised: 3.3 Model training

Original: Having introduced all the building blocks, Q -GCN stacks multiple pseudo-Riemannian GCN layers and the final embeddings at the last layer can then be used to perform downstream tasks.
Revised: Having introduced all the building blocks, Q -GCN stacks multiple pseudo-Riemannian GCN layers and the final embeddings at the last layer can then be used to perform downstream tasks.

Original: For graph reconstruction, the objective is to map all nodes into a low-dimensional space such that the connected nodes are closer than unconnected nodes.
Revised: For graph reconstruction, the objective is to map all nodes into a low-dimensional space such that the connected nodes are closer than unconnected nodes.

Original: Following [17, 21], we minimize the loss function
Revised: Following [17, 21], we minimize the loss function

Original: L
Revised: L

Original: [Equation]
Revised: [Equation]

Original: d
Revised: d

Original: [Equation]
Revised: [Equation]

Original: ) under the set of connected relations D in the graph, where
Revised: ) under the set of connected relations D in the graph, where

Original: E ( u ) = { v | ( u, v ) / ∈ D, v ̸ = u } is the set of negative examples for node u , d ( · ) is the distance function defined in Eq.
Revised: E ( u ) = { v | ( u, v ) / ∈ D, v ̸ = u } is the set of negative examples for node u , d ( · ) is the distance function defined in Eq.

Original: For node classification, we map the output of the last layer of Q -GCN to the tangent space, and then perform Euclidean multinomial logistic regression.
Revised: For node classification, we map the output of the last layer of Q -GCN to the tangent space, and then perform Euclidean multinomial logistic regression.

Original: For link prediction, we utilize the Fermi-Dirac decoder [8] to compute probability scores for edges, formally given by P ( e uv ∈ E| Θ) =
Revised: For link prediction, we utilize the Fermi-Dirac decoder [8] to compute probability scores for edges, formally given by P ( e uv ∈ E| Θ) =

Original: [Equation]
Revised: [Equation]

Original: 1 , where r and t are hyperparameters, d ( u , v ) is the distance function in the embedding space.
Revised: 1 , where r and t are hyperparameters, d ( u , v ) is the distance function in the embedding space.

Original: We then train Q -GCN by minimizing the cross-entropy loss using negative sampling.
Revised: We then train Q -GCN by minimizing the cross-entropy loss using negative sampling.

Original: Optimization.
Revised: Optimization.

Original: Although the model is built in Q s,tβ , the trainable parameters are all defined in Euclidean space through the diffeomorphic mappings.
Revised: Although the model is built in Q s,tβ , the trainable parameters are all defined in Euclidean space through the diffeomorphic mappings.

Original: Following the standard tangential optimization strategy [28], the parameters can be optimized via Euclidean optimization by applying layer-wise diffeomorphic expmap and logmap .
Revised: Following the standard tangential optimization strategy [28], the parameters can be optimized via Euclidean optimization by applying layer-wise diffeomorphic expmap and logmap .

Original: One optional strategy is to use dedicated optimization like [21], which we left for our future work.
Revised: One optional strategy is to use dedicated optimization like [21], which we left for our future work.

Original: Complexity analysis.
Revised: Complexity analysis.

Original: The time complexity is the same as a vanilla GCN given by O ( | V | dd ′ + | E | d ′ ) , where | V | and | E | are the number of nodes and edges, d and d ′ are the dimension of input and hidden features.
Revised: The time complexity is the same as a vanilla GCN given by O ( | V | dd ′ + | E | d ′ ) , where | V | and | E | are the number of nodes and edges, d and d ′ are the dimension of input and hidden features.

Original: The computation can be parallelized across all nodes.
Revised: The computation can be parallelized across all nodes.

Original: Similar to other non-Euclidean GCNs [28, 29, 14], the mapping from manifolds to the tangent space consume additional computation resources, compared with Euclidean GCNs, which is within the acceptable limits.
Revised: Similar to other non-Euclidean GCNs [28, 29, 14], the mapping from manifolds to the tangent space consume additional computation resources, compared with Euclidean GCNs, which is within the acceptable limits.

Original:  Experiments
Revised:  Experiments

Original: We evaluate the effectiveness of Q -GCN on graph reconstruction, node classification and link prediction.
Revised: We evaluate the effectiveness of Q -GCN on graph reconstruction, node classification and link prediction.

Original: Firstly, we study the geometric properties of the used datasets including the graph sectional curvature [17] and the δ -hyperbolicity [7].
Revised: Firstly, we study the geometric properties of the used datasets including the graph sectional curvature [17] and the δ -hyperbolicity [7].

Original: Fig.
Revised: Fig.

Original: 4 in the Appendix shows the histograms of sectional curvature and the mean sectional curvature for all datasets.
Revised: 4 in the Appendix shows the histograms of sectional curvature and the mean sectional curvature for all datasets.

Original: It can be seen that all datasets have both positive and negative sectional curvatures, showcasing that all graphs contain mixed graph topologies.
Revised: It can be seen that all datasets have both positive and negative sectional curvatures, showcasing that all graphs contain mixed graph topologies.

Original: To further analyze the degree of the hierarchy, we apply δ -hyperbolicity to identify the tree-likeness, as shown in Table 9 in the Appendix.
Revised: To further analyze the degree of the hierarchy, we apply δ -hyperbolicity to identify the tree-likeness, as shown in Table 9 in the Appendix.

Original: We conjecture that the datasets with positive graph sectional curvature or larger δ -hyperbolicity should be suitable for pseudo-hyperboloid with a smaller time dimension, while datasets with negative graph sectional curvature or smaller δ -hyperbolicity should be aligned well with pseudo-hyperboloid with a larger time dimension.
Revised: We conjecture that the datasets with positive graph sectional curvature or larger δ -hyperbolicity should be suitable for pseudo-hyperboloid with a smaller time dimension, while datasets with negative graph sectional curvature or smaller δ -hyperbolicity should be aligned well with pseudo-hyperboloid with a larger time dimension.

Original: 4.1 Graph reconstruction
Revised: 4.1 Graph reconstruction

Original: Datasets and baselines.
Revised: Datasets and baselines.

Original: We benchmark graph reconstruction on four real-world graphs including 1) Web-Edu [30]: a web network consisting of the .edu domain; 2) Power [31]: a power grid distribution network with backbone structure; 3) Bio-Worm [32]: a worms gene network; 4) Facebook [33]: a dense social network from Facebook.
Revised: We benchmark graph reconstruction on four real-world graphs including 1) Web-Edu [30]: a web network consisting of the .edu domain; 2) Power [31]: a power grid distribution network with backbone structure; 3) Bio-Worm [32]: a worms gene network; 4) Facebook [33]: a dense social network from Facebook.

Original: We compare our method with Euclidean GCN [2], hyperbolic GCN (HGCN) [28], spherical GCN, and product manifold GCNs ( κ -GCN) [29] with three signatures (i.e. H 5 × H 5 , H 5 × S 5 and S 5 × S 5 ).
Revised: We compare our method with Euclidean GCN [2], hyperbolic GCN (HGCN) [28], spherical GCN, and product manifold GCNs ( κ -GCN) [29] with three signatures (i.e. H 5 × H 5 , H 5 × S 5 and S 5 × S 5 ).

Original: Besides, five variants of our model are implemented with different time dimension in [1 , 3 , 5 , 7 , 10] for comparison.
Revised: Besides, five variants of our model are implemented with different time dimension in [1 , 3 , 5 , 7 , 10] for comparison.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Experimental settings.
Revised: Experimental settings.

Original: Following [17, 21, 29]
Revised: Following [17, 21, 29]

Original: [Table]
Revised: [Table]

Original: we use one-hot embeddings as initial node features.
Revised: we use one-hot embeddings as initial node features.

Original: To avoid the time dimensions being 0 , we uniformly perturb each dimension with a small random value in the interval [ − ϵ, ϵ ] , where ϵ = 0 .
Revised: To avoid the time dimensions being 0 , we uniformly perturb each dimension with a small random value in the interval [ − ϵ, ϵ ] , where ϵ = 0 .

Original: 02 in practice.
Revised: 02 in practice.

Original: In addition [Table] the same 10 -dimensional embedding and 2 hidden layers are used for all baselines to ensure a fair comparison.
Revised: In addition, the same 10 -dimensional embedding and 2 hidden layers are used for all baselines to ensure a fair comparison.

Original: The learning rate is set to 0 .
Revised: The learning rate is set to 0 .

Original: 01 , the learning rate of curvature is set to 0 . 0001 .
Revised: 01 , the learning rate of curvature is set to 0 . 0001 .

Original: Q -GCN is implemented with the Adam optimizer.
Revised: Q -GCN is implemented with the Adam optimizer.

Original: We repeat the experiments 10 times via different random seeds influencing weight initialization and data batching.
Revised: We repeat the experiments 10 times via different random seeds influencing weight initialization and data batching.

Original: Results.
Revised: Results.

Original: Table 1 shows the mean average precision (mAP) [17] results of graph reconstruction on four datasets.
Revised: Table 1 shows the mean average precision (mAP) [17] results of graph reconstruction on four datasets.

Original: It shows that Q -GCN achieves the best performance across all benchmarks compared with both Riemannian space and product manifolds.
Revised: It shows that Q -GCN achieves the best performance across all benchmarks compared with both Riemannian space and product manifolds.

Original: We observe that by setting proper signatures, the product spaces perform better than a single geometry.
Revised: We observe that by setting proper signatures, the product spaces perform better than a single geometry.

Original: It is consistent with our statement that the expression power of a single view geometry is limited.
Revised: It is consistent with our statement that the expression power of a single view geometry is limited.

Original: Specifically, all the top three results are achieved by Q -GCN, with one exception on Power where H 5 × S 5 achieved the third-best performance.
Revised: Specifically, all the top three results are achieved by Q -GCN, with one exception on Power where H 5 × S 5 achieved the third-best performance.

Original: More precisely, for datasets that have smaller graph sectional curvature like Web-Edu, Power and Bio-Worm, Q 7 , 3 perform the best, while Q 3 , 7 perform the best on Facebook with positive sectional curvature.
Revised: More precisely, for datasets that have smaller graph sectional curvature like Web-Edu, Power and Bio-Worm, Q 7 , 3 perform the best, while Q 3 , 7 perform the best on Facebook with positive sectional curvature.

Original: We conjecture that the number of time dimensions controls the geometry of the pseudo-hyperboloid.
Revised: We conjecture that the number of time dimensions controls the geometry of the pseudo-hyperboloid.

Original: We find that the graphs with more hierarchical structures are inclined to be embedded with fewer time dimensions.
Revised: We find that the graphs with more hierarchical structures are inclined to be embedded with fewer time dimensions.

Original: By analyzing the sectional curvature in Fig.
Revised: By analyzing the sectional curvature in Fig.

Original: 4, we find that this makes sense as the mean sectional curvature of Power, Bio-Wormnet and Web-
Revised: 4, we find that this makes sense as the mean sectional curvature of Power, Bio-Wormnet and Web-

Original: Edu are negative while it is negative for Facebook.
Revised: Edu are negative while it is negative for Facebook.

Original: Such results give us an intuition to determine the best time dimension based on the geometric properties of graphs.
Revised: Such results give us an intuition to determine the best time dimension based on the geometric properties of graphs.

Original: 4.2 Node classification and link prediction
Revised: 4.2 Node classification and link prediction

Original: Datasets and baselines.
Revised: Datasets and baselines.

Original: We consider four benchmark datasets: Airport, Pubmed, Citeseer and
Revised: We consider four benchmark datasets: Airport, Pubmed, Citeseer and

Original: Cora, where Airport is airline networks, Pubmed, Citeseer and Cora are three citation networks.
Revised: Cora, where Airport is airline networks, Pubmed, Citeseer and Cora are three citation networks.

Original: We observe that the graph sectional curvatures of the four datasets are consistently negative without significant differences in Fig.
Revised: We observe that the graph sectional curvatures of the four datasets are consistently negative without significant differences in Fig.

Original: 4, hence we report the additional δ -hyperbolicity for comparison in
Revised: 4, hence we report the additional δ -hyperbolicity for comparison in

Original: Table 2.
Revised: Table 2.

Original: GCN [2], GAT [3], SAGE [34] and SGC [4] are used as Euclidean GCN counterparts.
Revised: GCN [2], GAT [3], SAGE [34] and SGC [4] are used as Euclidean GCN counterparts.

Original: For non-Euclidean GCN baselines, we compare HGCN [28] and κ -GCN [29] with its three variants asexplained before.
Revised: For non-Euclidean GCN baselines, we compare HGCN [28] and κ -GCN [29] with its three variants as

Original: For Q -GCN, we empirically set the time dimension as [1 , 2 , 3 , 14 , 15 , 16] as six
Revised: [Table]

Original: [Table]
Revised: [Table]

Revised: explained before.

Revised: For Q -GCN

Original: [Table]
Revised: [Table]

Original:  variants since these settings best reflect the geometric properties of hyperbolic and spherical space, respectively.
Revised: we empirically set the time dimension as [1 , 2 , 3 , 14 , 15 , 16] as six variants since these settings best reflect the geometric properties of hyperbolic and spherical space, respectively.

Original: Experimental settings.
Revised: Experimental settings.

Original: For node classification, we use the same dataset split as [35] for citation datasets, where 20 nodes per class are used for training, and 500 nodes are used for validation and 1000 nodes are used for testing.
Revised: For node classification, we use the same dataset split as [35] for citation datasets, where 20 nodes per class are used for training, and 500 nodes are used for validation and 1000 nodes are used for testing.

Original: For Airport, we split the dataset into 70 / 15 / 15 .
Revised: For Airport, we split the dataset into 70 / 15 / 15 .

Original: For link prediction, the edges are split into 85 / 5 / 10 percent for training, validation and testing for all datasets.
Revised: For link prediction, the edges are split into 85 / 5 / 10 percent for training, validation and testing for all datasets.

Original: To ensure a fair comparison, we set the same 16 -dimension hidden embedding, 0 .
Revised: To ensure a fair comparison, we set the same 16 -dimension hidden embedding, 0 .

Original: 01 initial learning rate and 0 .
Revised: 01 initial learning rate and 0 .

Original: 0001 learning rate for curvature.
Revised: 0001 learning rate for curvature.

Original: The optimal regularization with weight decay, dropout rate, the number of layers and activation functions are obtained by grid search for each method.
Revised: The optimal regularization with weight decay, dropout rate, the number of layers and activation functions are obtained by grid search for each method.

Original: We report the mean accuracy over 10 random seeds influencing weight initialization and batching sequence.
Revised: We report the mean accuracy over 10 random seeds influencing weight initialization and batching sequence.

Original: Results.
Revised: Results.

Original: Table 2 shows the averaged ROC AUC for link prediction, and F1 score for node classification.
Revised: Table 2 shows the averaged ROC AUC for link prediction, and F1 score for node classification.

Original: As we can see from the δ -hyperbolicity, Airport and Pubmed are more hierarchical than CiteSeer and Cora.
Revised: As we can see from the δ -hyperbolicity, Airport and Pubmed are more hierarchical than CiteSeer and Cora.

Original: For Airport and Pubmed with dominating hierarchical properties (lower δ ), Q -GCNs with fewer time dimensions achieve the results on par with hyperbolic space based methods such as HGCN [28], κ -GCN ( H 16 ).
Revised: For Airport and Pubmed with dominating hierarchical properties (lower δ ), Q -GCNs with fewer time dimensions achieve the results on par with hyperbolic space based methods such as HGCN [28], κ -GCN ( H 16 ).

Original: While for CiteSeer and Cora with less tree-like properties (higher δ ), Q -GCNs achieve the state-of-the-art results, showcasing the flexibility of our model to embed complex graphs with different curvatures.
Revised: While for CiteSeer and Cora with less tree-like properties (higher δ ), Q -GCNs achieve the state-of-the-art results, showcasing the flexibility of our model to embed complex graphs with different curvatures.

Original: More specifically, Q -GCN with more time dimensions consistently performs best on Cora.
Revised: More specifically, Q -GCN with more time dimensions consistently performs best on Cora.

Original: While for CiteSeer, albeit Q -GCN achieves the best results, the corresponding best variants are not consistent on both tasks.
Revised: While for CiteSeer, albeit Q -GCN achieves the best results, the corresponding best variants are not consistent on both tasks.

Original: 4.3 Parameter sensitivity and analysis
Revised: 4.3 Parameter sensitivity and analysis

Original: [Figure]
Revised: [Figure]

Original: Time dimension.
Revised: Time dimension.

Original: We study the influence of time dimension for graph reconstruction by setting varying number of time dimensions under the condition of s + t = 10 .
Revised: We study the influence of time dimension for graph reconstruction by setting varying number of time dimensions under the condition of s + t = 10 .

Original: Fig.
Revised: Fig.

Original: 2 shows that the time dimension t acts as a knob for controlling the geometric properties of Q s,tβ .
Revised: 2 shows that the time dimension t acts as a knob for controlling the geometric properties of Q s,tβ .

Original: The best performance are achieved by neither hyperboloid ( t = 1 ) nor sphere cases ( t = 10 ), showcasing the advantages of Q s,tβ on representing graphs of mixed topologies.
Revised: The best performance are achieved by neither hyperboloid ( t = 1 ) nor sphere cases ( t = 10 ), showcasing the advantages of Q s,tβ on representing graphs of mixed topologies.

Original: It shows that on Web-Edu, Power and Bio-Worm with smaller mean sectional curvature, after the optimal value is reached at a lower t , the performance decrease as t increases.
Revised: It shows that on Web-Edu, Power and Bio-Worm with smaller mean sectional curvature, after the optimal value is reached at a lower t , the performance decrease as t increases.

Original: While on Facebook with larger (positive) mean sectional curvature, as t rises, the effect gradually increases until it reaches a peak at a higher t .
Revised: While on Facebook with larger (positive) mean sectional curvature, as t rises, the effect gradually increases until it reaches a peak at a higher t .

Original: It is consistent with our hypothesis that graphs with more hierarchical structure are inclined to be embedded in Q s,tβ with smaller t , while cyclical data is aligned well with larger t .
Revised: It is consistent with our hypothesis that graphs with more hierarchical structure are inclined to be embedded in Q s,tβ with smaller t , while cyclical data is aligned well with larger t .

Original: The results give us an intuition to determine the best time dimension based on the geometric properties of graphs.
Revised: The results give us an intuition to determine the best time dimension based on the geometric properties of graphs.

Original: [Figure]
Revised: [Figure]

Original: Q -NN VS Q -GCN.
Revised: Q -NN VS Q -GCN.

Original: We also introduce Q -NN, a generalization of MLP into pseudo-Riemannian manifold, defined as multiple layers of f ( x ) = σ ⊗ ( W ⊗ β x ⊕ β b ) , where σ ⊗ is the tangential activation.
Revised: We also introduce Q -NN, a generalization of MLP into pseudo-Riemannian manifold, defined as multiple layers of f ( x ) = σ ⊗ ( W ⊗ β x ⊕ β b ) , where σ ⊗ is the tangential activation.

Original: Table 3 shows that Q -NN with appropriate time dimension outperforms MLP and HNN on node classification, showcasing the expression power of pseudo-hyperboloid.
Revised: Table 3 shows that Q -NN with appropriate time dimension outperforms MLP and HNN on node classification, showcasing the expression power of pseudo-hyperboloid.

Original: Furthermore, compared with the results of Q -GCN in Table 2, Q -GCN performs better than Q -NN, suggesting that the benefits of the neighborhood aggregation equipped with the proposed GCN operations.
Revised: Furthermore, compared with the results of Q -GCN in Table 2, Q -GCN performs better than Q -NN, suggesting that the benefits of the neighborhood aggregation equipped with the proposed GCN operations.

Original: Computation efficiency.
Revised: Computation efficiency.

Original: We compare the running time of Q -GCN, GCN and Prod-GCN per epoch.
Revised: We compare the running time of Q -GCN, GCN and Prod-GCN per epoch.

Original: Table 4 shows that Q -GCN achieves higher efficiency than Prod-GCN ( H 5 × S 5 ).
Revised: Table 4 shows that Q -GCN achieves higher efficiency than Prod-GCN ( H 5 × S 5 ).

Original: This is mainly owing to that the component R in our diffeomorphic manifold ( S × R ) runs faster than non-Euclidean components in H 5 × S 5 .
Revised: This is mainly owing to that the component R in our diffeomorphic manifold ( S × R ) runs faster than non-Euclidean components in H 5 × S 5 .

Original: The additional running time mainly comes from the mapping operations and the projection from the time dimensions to S .
Revised: The additional running time mainly comes from the mapping operations and the projection from the time dimensions to S .

Original: The running time grows when increasing the number of time dimensions.
Revised: The running time grows when increasing the number of time dimensions.

Original: Overall, albeit slower than Euclidean GCN, the running time of all variants of
Revised: Overall, albeit slower than Euclidean GCN, the running time of all variants of

Original: Q -GCN is smaller than the twice of time in Euclidean GCN, which is within the acceptable limits.
Revised: Q -GCN is smaller than the twice of time in Euclidean GCN, which is within the acceptable limits.

Original: Visualization.
Revised: Visualization.

Original: To visualize the embeddings learned by Q -GCN, we use UMAP tool 3 to project the learned embeddings for Link Prediction on Pubmed and Cora into low-dimensional spherical and hyperbolic spaces.
Revised: To visualize the embeddings learned by Q -GCN, we use UMAP tool 3 to project the learned embeddings for Link Prediction on Pubmed and Cora into low-dimensional spherical and hyperbolic spaces.

Original: The projections include spherical projection into 3D sphere, hyperbolic projection into 3D plane, and 2D Poincar ´ e disk.
Revised: The projections include spherical projection into 3D sphere, hyperbolic projection into 3D plane, and 2D Poincar ´ e disk.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3 (a,b), for Pubmed with more hyperbolic structures, the class separability is more significant in hyperbolic projection than that is in spherical projection.
Revised: 3 (a,b), for Pubmed with more hyperbolic structures, the class separability is more significant in hyperbolic projection than that is in spherical projection.

Original: While the corresponding result is opposite for less tree-like Cora.
Revised: While the corresponding result is opposite for less tree-like Cora.

Original: Furthermore, Fig.
Revised: Furthermore, Fig.

Original: 3 (c) provides a more clear insight of the hierarchy.
Revised: 3 (c) provides a more clear insight of the hierarchy.

Original: It shows that there are more hub nodes near the origin of Poincar ´ e disk in Pubmed than in Cora, showcasing the dominating tree-likeness of Pubmed.
Revised: It shows that there are more hub nodes near the origin of Poincar ´ e disk in Pubmed than in Cora, showcasing the dominating tree-likeness of Pubmed.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we generalize GCNs to pseudo-Riemannian manifolds of constant nonzero curvature with elegant theories of diffeomorphic geometry tools.
Revised: In this paper, we generalize GCNs to pseudo-Riemannian manifolds of constant nonzero curvature with elegant theories of diffeomorphic geometry tools.

Original: The proposed Q -GCN have the flexibility to fit complex graphs with mixed curvatures and have shown promising results on graph reconstruction, node classification and link prediction.
Revised: The proposed Q -GCN have the flexibility to fit complex graphs with mixed curvatures and have shown promising results on graph reconstruction, node classification and link prediction.

Original: One limitation might be the choice of time dimension, we provide some insights to decide the best time dimension but this could still be improved, which we left for our future work.
Revised: One limitation might be the choice of time dimension, we provide some insights to decide the best time dimension but this could still be improved, which we left for our future work.

Original: The developed geodesic tools are application-agnostic and could be extended to more deep learning methods.
Revised: The developed geodesic tools are application-agnostic and could be extended to more deep learning methods.

Original: We foresee our work would shed light on the direction of non-Euclidean geometric deep learning.
Revised: We foresee our work would shed light on the direction of non-Euclidean geometric deep learning.

Original: Acknowledgments
Revised: Acknowledgments

Original: The authors thank the International Max Planck Research School for Intelligent Systems (IMPRSIS) for supporting Bo Xiong.
Revised: The authors thank the International Max Planck Research School for Intelligent Systems (IMPRSIS) for supporting Bo Xiong.

Original: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No: 860801.
Revised: Bo Xiong was funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No: 860801.

Original: Nico Potyka was partially funded by DFG projects Evowipe/COFFEE.
Revised: Nico Potyka was partially funded by DFG projects Evowipe/COFFEE.

Revised: Shirui Pan was supported in part by an ARC Future Fellowship (FT210100097).

Original: Supplemental Material of "Pseudo-Riemannian Graph Convolution Networks" Broader Impact
Revised: Supplemental Material of "Pseudo-Riemannian Graph Convolution Networks" Broader Impact

Original: Going beyond Riemannian manifolds, we extend GCNs to the pseudo-Riemannian manifolds, which is a smooth manifold furnished with indefinite metric.
Revised: Going beyond Riemannian manifolds, we extend GCNs to the pseudo-Riemannian manifolds, which is a smooth manifold furnished with indefinite metric.

Original: Our contribution is both theoretical and practical.
Revised: Our contribution is both theoretical and practical.

Original: We introduce necessary geodesic tools in geodesically disconnected pseudo-Riemannian manifolds to help machine learning researchers to define vector operations like matrix multiplication and vector addition.
Revised: We introduce necessary geodesic tools in geodesically disconnected pseudo-Riemannian manifolds to help machine learning researchers to define vector operations like matrix multiplication and vector addition.

Original: These operations are application agnostic and could be applied in the context of machine learning algorithms.
Revised: These operations are application agnostic and could be applied in the context of machine learning algorithms.

Original: Q -GCN is a flexible framework and is especially powerful on modeling real-world graphs such as social networks and molecular networks which usually exhibit heterogeneous topological structures.
Revised: Q -GCN is a flexible framework and is especially powerful on modeling real-world graphs such as social networks and molecular networks which usually exhibit heterogeneous topological structures.

Original: We foresee our method can be applied to more practical settings such as recommender systems and drug discovery, and also stimulate more applications of pseudo-Riemannian geometry in machine learning and related communities.
Revised: We foresee our method can be applied to more practical settings such as recommender systems and drug discovery, and also stimulate more applications of pseudo-Riemannian geometry in machine learning and related communities.

Original: However, we emphasize that current models might suffer from the same problems as other deep models such as limited interpretability, and thus, are not capable of replacing human expertise.
Revised: However, we emphasize that current models might suffer from the same problems as other deep models such as limited interpretability, and thus, are not capable of replacing human expertise.

Original: We advocate researchers to focus more on the interpretability of non-Euclidean graph embeddings.
Revised: We advocate researchers to focus more on the interpretability of non-Euclidean graph embeddings.

Original: We structurize our supplementary materials as follows:
Revised: We structurize our supplementary materials as follows:

Original: • Sec.
Revised: • Sec.

Original: A summarizes the frequently used notations and the conventions.
Revised: A summarizes the frequently used notations and the conventions.

Original: • Sec.
Revised: • Sec.

Original: B reviews some related works on Riemannian embeddings and the application of pseudo-Riemannian geometry in machine learning.
Revised: B reviews some related works on Riemannian embeddings and the application of pseudo-Riemannian geometry in machine learning.

Original: • Sec.
Revised: • Sec.

Original: C reviews pseudo-Riemannian geometry and provides some standard geodesic tools in pseudo-hyperboloid.
Revised: C reviews pseudo-Riemannian geometry and provides some standard geodesic tools in pseudo-hyperboloid.

Original: • Sec.
Revised: • Sec.

Original: D provides necessary proofs of the theorems mentioned in the paper.
Revised: D provides necessary proofs of the theorems mentioned in the paper.

Original: • Sec.
Revised: • Sec.

Original: E provides more details about the experimental setup and analysis.
Revised: E provides more details about the experimental setup and analysis.

Original: A Notation
Revised: A Notation

Original: In this paper, we denote the points on manifolds by boldface Roman characters.
Revised: In this paper, we denote the points on manifolds by boldface Roman characters.

Original: The tangent vectors on tangent spaces are denoted by boldface Greek fonts.
Revised: The tangent vectors on tangent spaces are denoted by boldface Greek fonts.

Original: The notations are summarized in Table 5.
Revised: The notations are summarized in Table 5.

Original: B Related Works
Revised: B Related Works

Original: Riemannian embedding.
Revised: Riemannian embedding.

Original: Non-Euclidean Riemannian spaces have recently gained extensive attention in learning representation for non-euclidean data.
Revised: Non-Euclidean Riemannian spaces have recently gained extensive attention in learning representation for non-euclidean data.

Original: [36] was the first work to learn hierarchical embeddings in hyperbolic space for link prediction.
Revised: [36] was the first work to learn hierarchical embeddings in hyperbolic space for link prediction.

Original: Following this work, [37] applied hyperbolic embedding in word embedding.
Revised: Following this work, [37] applied hyperbolic embedding in word embedding.

Original: [38] proposed a translation model in hyperbolic space for multirelational graph embedding, and [39] proposed a hyperbolic embedding for answering complex logical queries.
Revised: [38] proposed a translation model in hyperbolic space for multirelational graph embedding, and [39] proposed a hyperbolic embedding for answering complex logical queries.

Original: On the other hand, spherical space offers benefits for embedding spherical or cyclical data [11, 40, 41].
Revised: On the other hand, spherical space offers benefits for embedding spherical or cyclical data [11, 40, 41].

Original: For mixed curvature space, product manifolds [17, 18] on Euclidean, hyperbolic and spherical space has been proposed to embed graph and knowledge graphs [42] with mixed structures.
Revised: For mixed curvature space, product manifolds [17, 18] on Euclidean, hyperbolic and spherical space has been proposed to embed graph and knowledge graphs [42] with mixed structures.

Original: HNN [12, 43] extended neural networks into the hyperbolic space and defined some fundamental neural operations.
Revised: HNN [12, 43] extended neural networks into the hyperbolic space and defined some fundamental neural operations.

Original: Riemannian GCN.
Revised: Riemannian GCN.

Original: Going beyond GCNs in Euclidean space [1, 2], HGNN [13] and HGCN [28] first extended GCNs into hyperbolic space, achieving state-of-the-art performance on learning graph embedding for scale-free networks.
Revised: Going beyond GCNs in Euclidean space [1, 2], HGNN [13] and HGCN [28] first extended GCNs into hyperbolic space, achieving state-of-the-art performance on learning graph embedding for scale-free networks.

Original: The main solution is to move the graph convolutional operations (e.g. feature transformation , aggregation ) into the tangent space of manifolds and perform
Revised: The main solution is to move the graph convolutional operations (e.g. feature transformation , aggregation ) into the tangent space of manifolds and perform

Original: Euclidean operations on the space.
Revised: Euclidean operations on the space.

Original: To model graphs of mixed topologies, κ -GCN [29] unifies curvatures in a κ -stereographic model and extends GCNs into the products of Riemannian projection
Revised: To model graphs of mixed topologies, κ -GCN [29] unifies curvatures in a κ -stereographic model and extends GCNs into the products of Riemannian projection

Original: [Table]
Revised: [Table]

Original: manifolds.
Revised: manifolds.

Original: GIL [14] proposed graph geometric interaction learning that models interaction between hyperbolic and Euclidean space.
Revised: GIL [14] proposed graph geometric interaction learning that models interaction between hyperbolic and Euclidean space.

Original: Different from these works, Q -GCN learns graph embeddings in a single pseudo-Riemannian space with indefinite metrics while capturing mixed graph structures.
Revised: Different from these works, Q -GCN learns graph embeddings in a single pseudo-Riemannian space with indefinite metrics while capturing mixed graph structures.

Original: Pseudo-Riemannian machine learning.
Revised: Pseudo-Riemannian machine learning.

Original: Few works [5, 27, 21, 22, 20] have explored the application of pseudo-Riemannian geometry in machine learning.
Revised: Few works [5, 27, 21, 22, 20] have explored the application of pseudo-Riemannian geometry in machine learning.

Original: [5] first applied pseudo-Riemannian geometry (or pseudo-Euclidean space ) to embed non-metric data that preserves local information.
Revised: [5] first applied pseudo-Riemannian geometry (or pseudo-Euclidean space ) to embed non-metric data that preserves local information.

Original: [20] exploited Lorentzian spacetime on embedding directed acyclic graphs, [27] developed a manifold learning framework on the pseudo-Riemannian manifold and its submanifolds.
Revised: [20] exploited Lorentzian spacetime on embedding directed acyclic graphs, [27] developed a manifold learning framework on the pseudo-Riemannian manifold and its submanifolds.

Original: More recently, [21] proposed learning graph embeddings on pseudo-hyperboloid and provided some geodesic tools.
Revised: More recently, [21] proposed learning graph embeddings on pseudo-hyperboloid and provided some geodesic tools.

Original: [22] further extended it into directed graph embedding with novel tools.
Revised: [22] further extended it into directed graph embedding with novel tools.

Original: [44] first studied the pseudoRiemannian geometry in the setting of deep learning.
Revised: [44] first studied the pseudoRiemannian geometry in the setting of deep learning.

Original: However, their embedding is learned in a quotient manifold.
Revised: However, their embedding is learned in a quotient manifold.

Original: Different from [44]
Revised: Different from [44]

Original: [Table]
Revised: [Table]

Original: our embedding is directly learned from the pseudo-hyperboloid.
Revised: our embedding is directly learned from the pseudo-hyperboloid.

Original: C Review of Pseudo-Riemannian Geometry C.1 Pseudo-Riemannian manifold
Revised: C Review of Pseudo-Riemannian Geometry C.1 Pseudo-Riemannian manifold

Original: The pseudo-Riemannian manifold M is equipped with a pseudo-Riemannian metric g x : T x M ×
Revised: The pseudo-Riemannian manifold M is equipped with a pseudo-Riemannian metric g x : T x M ×

Original: T x M at point x .
Revised: T x M at point x .

Original: The metric g x is a nondegenerate symmetric bilinear form, which means that if for a given ξ ∈ T x M , for any ζ ∈ T x M we have g x ( ξ , ζ ) = 0 , then ξ = 0 .
Revised: The metric g x is a nondegenerate symmetric bilinear form, which means that if for a given ξ ∈ T x M , for any ζ ∈ T x M we have g x ( ξ , ζ ) = 0 , then ξ = 0 .

Original: The M manifold has the constant sectional curvature of 1 /β and constant mean curvature κ = | β | − 1 / 2 .
Revised: The M manifold has the constant sectional curvature of 1 /β and constant mean curvature κ = | β | − 1 / 2 .

Original: We recommend the reader to refer [23, 45] for details.
Revised: We recommend the reader to refer [23, 45] for details.

Original: C.2 Geodesic tools of pseudo-hyperboloid
Revised: C.2 Geodesic tools of pseudo-hyperboloid

Original: Although pseudo-hyperboloid is a geodescially disconnected manifold, previous works [21, 27] have defined some standard geodesic tools that do not consider the broken cases which we recall below.
Revised: Although pseudo-hyperboloid is a geodescially disconnected manifold, previous works [21, 27] have defined some standard geodesic tools that do not consider the broken cases which we recall below.

Original: Geodesic.
Revised: Geodesic.

Original: The geodesics of pseudo-hyperboloid are a combination of the hyperbolic, flat and spherical cases, depending on the sign of ⟨ ξ , ξ ⟩ t .
Revised: The geodesics of pseudo-hyperboloid are a combination of the hyperbolic, flat and spherical cases, depending on the sign of ⟨ ξ , ξ ⟩ t .

Original: Formally, the geodesic γ x → ξ ( τ ) of Q s,tβ with β < 0 is defined as Eq.
Revised: Formally, the geodesic γ x → ξ ( τ ) of Q s,tβ with β < 0 is defined as Eq.

Original: [Equation]
Revised: [Equation]

Original: Exponential and logarithmic maps.
Revised: Exponential and logarithmic maps.

Original: The exponential and logarithmic maps in the pseudohyperboloid manifold are defined as exp x : T x Q s,tβ → Q s,tβ and log x : Q s,tβ → T x Q s,tβ , respectively.
Revised: The exponential and logarithmic maps in the pseudohyperboloid manifold are defined as exp x : T x Q s,tβ → Q s,tβ and log x : Q s,tβ → T x Q s,tβ , respectively.

Original: We have the following closed form expressions of the exponential and the logarithmic maps [21], which allow us to perform operations on points on the pseudo-hyperboloid or pseudo-sphere manifolds by mapping them to tangent spaces and vice-versa, formally given by
Revised: We have the following closed form expressions of the exponential and the logarithmic maps [21], which allow us to perform operations on points on the pseudo-hyperboloid or pseudo-sphere manifolds by mapping them to tangent spaces and vice-versa, formally given by

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: [Equation]
Revised: [Equation]

Original: It is worth mentioning that not all points on the pseudo-hyperboloid manifold are connected by a geodesic.
Revised: It is worth mentioning that not all points on the pseudo-hyperboloid manifold are connected by a geodesic.

Original: For these broken points, there does not exist a tangent vector ξ such that y = exp x ( ξ ) .
Revised: For these broken points, there does not exist a tangent vector ξ such that y = exp x ( ξ ) .

Original: Namely, the logarithmic map is not defined if ⟨ x , y ⟩ ≥ | β | .
Revised: Namely, the logarithmic map is not defined if ⟨ x , y ⟩ ≥ | β | .

Original: Distance.
Revised: Distance.

Original: By the means of the geodesic, the distance between x and y in pseudo-Riemannian manifolds Q s,tβ is defined as the arc length of geodesic γ ( τ ) , given by Eq.
Revised: By the means of the geodesic, the distance between x and y in pseudo-Riemannian manifolds Q s,tβ is defined as the arc length of geodesic γ ( τ ) , given by Eq.

Original: [Equation]
Revised: [Equation]

Original: Parallel transport.
Revised: Parallel transport.

Original: Given the geodesic γ ( τ ) on Q s,tβ passing through x ∈ Q s,tβ with the tangent direction ξ ∈ T x Q s,tβ , the parallel transport of ζ ∈ T x Q s,tβ is defined as Eq.
Revised: Given the geodesic γ ( τ ) on Q s,tβ passing through x ∈ Q s,tβ with the tangent direction ξ ∈ T x Q s,tβ , the parallel transport of ζ ∈ T x Q s,tβ is defined as Eq.

Original: [Equation]
Revised: [Equation]

Original: if
Revised: if

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: if ⟨ ξ , ξ ⟩ <⟨ ζ
Revised: if ⟨ ξ , ξ ⟩ <⟨ ζ

Original: [Equation]
Revised: [Equation]

Original: ξ ⟩ (cid:0) τ x
Revised: ξ ⟩ (cid:0) τ x

Original: [Equation]
Revised: [Equation]

Original: ξ (cid:1) + ζ , otherwise (12) where ξ = log x ( y ) .
Revised: ξ (cid:1) + ζ , otherwise (12) where ξ = log x ( y ) .

Original: D Proof of Theorems D.1 Proof of Theorem
Revised: D Proof of Theorems D.1 Proof of Theorem

Original: Theorem 1 (Theorem 4.1 in [21]) .
Revised: Theorem 1 (Theorem 4.1 in [21]) .

Original: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ :
Revised: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ :

Original: Q s,tβ → S t 1 × R s that maps x into the product manifolds of an unit sphere and the Euclidean space, the mapping and its inverse are given by,
Revised: Q s,tβ → S t 1 × R s that maps x into the product manifolds of an unit sphere and the Euclidean space, the mapping and its inverse are given by,

Original: [Equation]
Revised: [Equation]

Original: (13) where x = (cid:18) ts (cid:19) ∈
Revised: (13) where x = (cid:18) ts (cid:19) ∈

Original: Q s,tβ with t ∈ R t +1 ∗ and s ∈ R s .
Revised: Q s,tβ with t ∈ R t +1 ∗ and s ∈ R s .

Original: z = (cid:18) uv
Revised: z = (cid:18) uv

Original: (cid:19) ∈ S t 1 × R s with u ∈ S t 1 and v ∈ R s .
Revised: (cid:19) ∈ S t 1 × R s with u ∈ S t 1 and v ∈ R s .

Original: Please refer Appendix C.5 in [21] for proof of this theorem.
Revised: Please refer Appendix C.5 in [21] for proof of this theorem.

Original: D.2 Proof of Theorem
Revised: D.2 Proof of Theorem

Original: Theorem 2.
Revised: Theorem 2.

Original: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ : Q s,tβ → S t − β × R s that maps x into the product manifolds of a sphere and the Euclidean space, the mapping and its inverse are given by,
Revised: For any point x ∈ Q s,tβ , there exists a diffeomorphism ψ : Q s,tβ → S t − β × R s that maps x into the product manifolds of a sphere and the Euclidean space, the mapping and its inverse are given by,

Original: [Equation]
Revised: [Equation]

Original: where x = (cid:18) ts (cid:19) ∈ Q s,tβ with t ∈ R t ∗ and s ∈ R s .
Revised: where x = (cid:18) ts (cid:19) ∈ Q s,tβ with t ∈ R t ∗ and s ∈ R s .

Original: z = (cid:18) uv (cid:19) ∈ S t − β ×
Revised: z = (cid:18) uv (cid:19) ∈ S t − β ×

Original: R s with u ∈ S t − β and v ∈ R s .
Revised: R s with u ∈ S t − β and v ∈ R s .

Original: Proof.
Revised: Proof.

Original: It is easy to show that the ψ and ψ − 1 are smooth functions as they only involve with a linear mapping with a constant scaling vector.
Revised: It is easy to show that the ψ and ψ − 1 are smooth functions as they only involve with a linear mapping with a constant scaling vector.

Original: Hence, we only need to show that ψ (cid:0) ψ − 1 ( z ) (cid:1) =
Revised: Hence, we only need to show that ψ (cid:0) ψ − 1 ( z ) (cid:1) =

Original: z and ψ − 1 ( ψ ( x ))
Revised: z and ψ − 1 ( ψ ( x ))

Original: =
Revised: =

Original: x .
Revised: x .

Original: Here, we consider space dimensions and time dimensions separately.
Revised: Here, we consider space dimensions and time dimensions separately.

Original: For space dimensions, the mapping of the space dimensions of x to the space dimensions of z is an identity function (i.e. v = s , s = v ).
Revised: For space dimensions, the mapping of the space dimensions of x to the space dimensions of z is an identity function (i.e. v = s , s = v ).

Original: Thus, we only need to show the invertibility of the mappings taking time dimensions as inputs.
Revised: Thus, we only need to show the invertibility of the mappings taking time dimensions as inputs.

Original: For time dimensions, we first show that:
Revised: For time dimensions, we first show that:

Original: [Equation]
Revised: [Equation]

Original: Note that the last equality can be inferred by the fact that x ∈ Q s,tβ and β < 0 .
Revised: Note that the last equality can be inferred by the fact that x ∈ Q s,tβ and β < 0 .

Original: We then show that:
Revised: We then show that:

Original: [Equation]
Revised: [Equation]

Original: Note that the last equality can be inferred by the fact that u ∈ S t − β and β < 0 .
Revised: Note that the last equality can be inferred by the fact that u ∈ S t − β and β < 0 .

Original: D.3 Proof of Theorem
Revised: D.3 Proof of Theorem

Original: Theorem 3.
Revised: Theorem 3.

Original: For any reference point x = (cid:18) ts (cid:19) ∈ Q s,tβ with space dimension s = 0 , the induced tangent space of Q s,tβ is equal to the tangent space of its diffeomorphic manifold S t − β ×
Revised: For any reference point x = (cid:18) ts (cid:19) ∈ Q s,tβ with space dimension s = 0 , the induced tangent space of Q s,tβ is equal to the tangent space of its diffeomorphic manifold S t − β ×

Original: R s , namely,
Revised: R s , namely,

Original: T ψ ( x ) ( S t − β × R s ) =
Revised: T ψ ( x ) ( S t − β × R s ) =

Original: T x Q s,tβ .
Revised: T x Q s,tβ .

Original: Proof.
Revised: Proof.

Original: For any point x = (cid:18) ts (cid:19) ∈ Q s,tβ with s = 0 , the corresponding point in the diffeomorphic manifold is ψ ( x ) = (cid:18) (cid:112) | β | t ∥ t ∥ (cid:19) .
Revised: For any point x = (cid:18) ts (cid:19) ∈ Q s,tβ with s = 0 , the corresponding point in the diffeomorphic manifold is ψ ( x ) = (cid:18) (cid:112) | β | t ∥ t ∥ (cid:19) .

Original: Based on the definition of tangent space, for any tangent vector ξ ∈ T x Q s,tβ , ⟨ x , ξ ⟩ t = 0 implies − t ξ t + 0 ξ s = 0 , which means t ξ t = 0 .
Revised: Based on the definition of tangent space, for any tangent vector ξ ∈ T x Q s,tβ , ⟨ x , ξ ⟩ t = 0 implies − t ξ t + 0 ξ s = 0 , which means t ξ t = 0 .

Original: Thus, ⟨ ψ ( x ) , ξ ⟩ t = 0 .
Revised: Thus, ⟨ ψ ( x ) , ξ ⟩ t = 0 .

Original: Based on the definition of tangent space, ξ ∈ T ψ ( x ) ( S t − β × R s ) .
Revised: Based on the definition of tangent space, ξ ∈ T ψ ( x ) ( S t − β × R s ) .

Original: D.4 Proof of Theorem
Revised: D.4 Proof of Theorem

Original: Theorem 4.
Revised: Theorem 4.

Original: For any point x ∈ Q s,t β , the union of the normal neighborhood of x and the normal neighborhood of its antipodal point − x cover the entire manifold.
Revised: For any point x ∈ Q s,t β , the union of the normal neighborhood of x and the normal neighborhood of its antipodal point − x cover the entire manifold.

Original: Namely, U x ∪ U − x = Q s,tβ .
Revised: Namely, U x ∪ U − x = Q s,tβ .

Original: Proof.
Revised: Proof.

Original: For any point x ∈ Q s,tβ and y / ∈ U x .
Revised: For any point x ∈ Q s,tβ and y / ∈ U x .

Original: Based on the definition of normal neighborhood, ⟨ x , y ⟩ t ≥ | β | → ⟨− x , y ⟩ t ≤ | β | .
Revised: Based on the definition of normal neighborhood, ⟨ x , y ⟩ t ≥ | β | → ⟨− x , y ⟩ t ≤ | β | .

Original: Thus, y ∈ U − x ,
Revised: Thus, y ∈ U − x ,

Original: E Supplementary experimental details
Revised: E Supplementary experimental details

Original: In this section, we present the details of our experiments and implementations, as well as some additional analyses.
Revised: In this section, we present the details of our experiments and implementations, as well as some additional analyses.

Original: E.1 Graph Reconstruction
Revised: E.1 Graph Reconstruction

Original: For graph reconstruction, one straightforward method is to minimize the graph distortion [29, 17] given by
Revised: For graph reconstruction, one straightforward method is to minimize the graph distortion [29, 17] given by

Original: [Equation]
Revised: [Equation]

Original: where d ( u , v ) is the distance function in the embedding space and d G ( u, v ) is the graph distance (the length of shortest path) between node u and v , |
Revised: where d ( u , v ) is the distance function in the embedding space and d G ( u, v ) is the graph distance (the length of shortest path) between node u and v , |

Original: V | is the number of nodes in graph.
Revised: V | is the number of nodes in graph.

Original: The objective function is to preserve all pairwise graph distances.
Revised: The objective function is to preserve all pairwise graph distances.

Original: Motivated by the fact that most of graphs are partially observable, we minimize an alternative loss function [21, 36] that preserves local graph distance, given by,
Revised: Motivated by the fact that most of graphs are partially observable, we minimize an alternative loss function [21, 36] that preserves local graph distance, given by,

Original: [Equation]
Revised: [Equation]

Original: where D is the connected relations in the graph, E ( u ) = { v | ( u, v ) / ∈ D ∪ u } is the set of negative examples for node u , d ( u , v ) is the distance function in the embedding space.
Revised: where D is the connected relations in the graph, E ( u ) = { v | ( u, v ) / ∈ D ∪ u } is the set of negative examples for node u , d ( u , v ) is the distance function in the embedding space.

Original: For evaluation, we apply the mean average precision (mAP) to evaluate the graph reconstruction task.
Revised: For evaluation, we apply the mean average precision (mAP) to evaluate the graph reconstruction task.

Original: mAP is a local metric that measures the average proportion of the nearest points of a node which are actually its neighbors in the original graph.
Revised: mAP is a local metric that measures the average proportion of the nearest points of a node which are actually its neighbors in the original graph.

Original: The mAP is defined as Eq.
Revised: The mAP is defined as Eq.

Original: [Equation]
Revised: [Equation]

Original: where f is the embedding function, | V | is the number of nodes in graph, deg( u ) is the degree of node u , N u is the one-hop neighborhoods in the graph, R u,v denotes whether two nodes u and v are connected.
Revised: where f is the embedding function, | V | is the number of nodes in graph, deg( u ) is the degree of node u , N u is the one-hop neighborhoods in the graph, R u,v denotes whether two nodes u and v are connected.

Original: Implementation details.
Revised: Implementation details.

Original: We implement our models by Pytorch and Geoopt tool 4 .
Revised: We implement our models by Pytorch and Geoopt tool 4 .

Original: Our models and baselines are built upon the codes 5 .
Revised: Our models and baselines are built upon the codes 5 .

Original: For baselines of product space and κ -GCN, we use the code 6 .
Revised: For baselines of product space and κ -GCN, we use the code 6 .

Original: Our models are trained on NVIDIA A100 with 4 GPU cards.
Revised: Our models are trained on NVIDIA A100 with 4 GPU cards.

Original: Similar to [28], we use the pre-tranined embeddings on link prediction as the initialization of node classification, to preserve graph structures information.
Revised: Similar to [28], we use the pre-tranined embeddings on link prediction as the initialization of node classification, to preserve graph structures information.

Original: All non-Euclidean methods do not use attention mechanism for a fair comparison.
Revised: All non-Euclidean methods do not use attention mechanism for a fair comparison.

Original: For node classification and link prediction, the optimal hyper-parameters for all methods are obtained by the same grid search strategy.
Revised: For node classification and link prediction, the optimal hyper-parameters for all methods are obtained by the same grid search strategy.

Original: The ranges of grid search are summarized in Table 6.
Revised: The ranges of grid search are summarized in Table 6.

Original: [Table]
Revised: [Table]

Original: Numerical instability.
Revised: Numerical instability.

Original: In practice, we find that the numerical computation in pseudo-hyperboloid is not stable due to the limited machine precision and rounding.
Revised: In practice, we find that the numerical computation in pseudo-hyperboloid is not stable due to the limited machine precision and rounding.

Original: To ensure the learned embeddings and tangent vectors to remain on the manifold and tangent spaces, we conduct projection operations via Eq.
Revised: To ensure the learned embeddings and tangent vectors to remain on the manifold and tangent spaces, we conduct projection operations via Eq.

Original: (20) and Eq.
Revised: (20) and Eq.

Original: (21) after each vector operations.
Revised: (21) after each vector operations.

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where ψ can be instantiated by Theorem 2.
Revised: where ψ can be instantiated by Theorem 2.

Original: Note that ψ ( · ) is an identity function when x is already localted in Q s,tβ .
Revised: Note that ψ ( · ) is an identity function when x is already localted in Q s,tβ .

Original: For input x / ∈ Q s,tβ , applying ψ ( · ) can project the x back to Q s,tβ , from which ψ ( · ) is no longer an identity.
Revised: For input x / ∈ Q s,tβ , applying ψ ( · ) can project the x back to Q s,tβ , from which ψ ( · ) is no longer an identity.

Original: [Equation]
Revised: [Equation]

Original: 21) where z is the tangent vector to be projected.
Revised: 21) where z is the tangent vector to be projected.

Original: Trainable curvature.
Revised: Trainable curvature.

Original: Similar to the trainable curvature in the hyperbolic space [28], we give an analogue of Theorem 4.1 in [28] as following corollary.
Revised: Similar to the trainable curvature in the hyperbolic space [28], we give an analogue of Theorem 4.1 in [28] as following corollary.

Original: Corollary 1.
Revised: Corollary 1.

Original: For any two points x , y ∈ Q s,tβ , there exists a mapping ϕ : Q s,tβ → Q s,tβ ′ that maps x , y into x ′ ,
Revised: For any two points x , y ∈ Q s,tβ , there exists a mapping ϕ : Q s,tβ → Q s,tβ ′ that maps x , y into x ′ ,

Original: y ′ ∈
Revised: y ′ ∈

Original: Q s,tβ ′ , such that the pseudo-Riemannian inner product is scaled by a constant factor β ′ β .
Revised: Q s,tβ ′ , such that the pseudo-Riemannian inner product is scaled by a constant factor β ′ β .

Original: Namely ⟨ x , y ⟩ t = β ′ β ⟨ x ′ , y ′ ⟩ t .
Revised: Namely ⟨ x , y ⟩ t = β ′ β ⟨ x ′ , y ′ ⟩ t .

Original: The mapping is given by ϕ ( x )
Revised: The mapping is given by ϕ ( x )

Original: = (cid:113) β
Revised: = (cid:113) β

Original: ′ β x .
Revised: ′ β x .

Original: Corollary 1 shows that assuming infinite machine precision, pseudo-hyperboloids of varying curvatures should have the same expressive power.
Revised: Corollary 1 shows that assuming infinite machine precision, pseudo-hyperboloids of varying curvatures should have the same expressive power.

Original: However, due to the limited machine precision [28], we set the curvature 1 /β as a trainable parameter to capture the best scale of the embeddings.
Revised: However, due to the limited machine precision [28], we set the curvature 1 /β as a trainable parameter to capture the best scale of the embeddings.

Original: Skip connection.
Revised: Skip connection.

Original: In preliminary experiments of graph reconstruction, we found that the performance of GCN, HGCN, κ -GCN and Q -GCN could not compete with the method in [17] that directly optimizes the distance function.
Revised: In preliminary experiments of graph reconstruction, we found that the performance of GCN, HGCN, κ -GCN and Q -GCN could not compete with the method in [17] that directly optimizes the distance function.

Original: We hypothesized that this was probably caused by over-smoothing of graph convolution kernel.
Revised: We hypothesized that this was probably caused by over-smoothing of graph convolution kernel.

Original: Therefore, we decided to apply skip-connection strategy [46, 47, 48] to avoid oversmoothing.
Revised: Therefore, we decided to apply skip-connection strategy [46, 47, 48] to avoid oversmoothing.

Original: Specifically, we take the mean of the hidden output of each layer to the decoder.
Revised: Specifically, we take the mean of the hidden output of each layer to the decoder.

Original: In the paper, we report the graph reconstruction results of GCN, HGCN, κ -GCN and Q -GCN applied with skip-connection.
Revised: In the paper, we report the graph reconstruction results of GCN, HGCN, κ -GCN and Q -GCN applied with skip-connection.

Original: Dataset analysis.
Revised: Dataset analysis.

Original: Table 7 and Table 8 summarize the statistics of the datasets we used for our experiments.
Revised: Table 7 and Table 8 summarize the statistics of the datasets we used for our experiments.

Original: For each dataset, we report the number of nodes, the number of edges, the number of classes and the feature dimensions.
Revised: For each dataset, we report the number of nodes, the number of edges, the number of classes and the feature dimensions.

Original: Besides, we measure the graph sectional curvature to identify the dominating geometry of each dataset.
Revised: Besides, we measure the graph sectional curvature to identify the dominating geometry of each dataset.

Original: Fig.
Revised: Fig.

Original: 4 shows the histograms of sectional curvature and the mean sectional curvature for all datasets.
Revised: 4 shows the histograms of sectional curvature and the mean sectional curvature for all datasets.

Original: It can be seen that all datasets have both positive and negative sectional curvatures, showcasing that all graphs contain mixed geometric structures.
Revised: It can be seen that all datasets have both positive and negative sectional curvatures, showcasing that all graphs contain mixed geometric structures.

Original: However, the datasets possess a preference for the negative half sectional curvature except the case of Facebook that has dominating positive sectional curvature, showcasing that most of real-world graphs exhibit more hierarchical structures than cyclic structures.
Revised: However, the datasets possess a preference for the negative half sectional curvature except the case of Facebook that has dominating positive sectional curvature, showcasing that most of real-world graphs exhibit more hierarchical structures than cyclic structures.

Original: To further analyze the degree of hierarchy of each graph, we apply δ -hyperbolicity to identify the tree-likeness.
Revised: To further analyze the degree of hierarchy of each graph, we apply δ -hyperbolicity to identify the tree-likeness.

Original: Table 9 shows the δ -hyperbolicity of different datasets.
Revised: Table 9 shows the δ -hyperbolicity of different datasets.

Original: It shows that although most of the datasets have negative mean sectional curvature, their degrees of hierarchy vary significantly.
Revised: It shows that although most of the datasets have negative mean sectional curvature, their degrees of hierarchy vary significantly.

Original: For example, Airport and Pubmed are more hierarchical than CiteSeer and Cora.
Revised: For example, Airport and Pubmed are more hierarchical than CiteSeer and Cora.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Algorithm 1 describes the training process of the proposed method.
Revised: Algorithm 1 describes the training process of the proposed method.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: # N
Revised: # N

Original: [Figure]
Revised: [Figure]

Original: s citeseer mean 1.
Revised: s citeseer mean 1.

Original: [Figure]
Revised: [Figure]

Original: # N o d e s cora mean
Revised: # N o d e s cora mean

Original: Input:
Revised: Input:

Original: The input graph G =
Revised: The input graph G =

Original: ( V , E ) , the input node features x , the time dimension t , the space dimension s , the initial curvature β , the number of epoches E , the number of layers L , the initial learning rate, the activation function σ , the layer-wise trainable weight W and bias term b .
Revised: ( V , E ) , the input node features x , the time dimension t , the space dimension s , the initial curvature β , the number of epoches E , the number of layers L , the initial learning rate, the activation function σ , the layer-wise trainable weight W and bias term b .

Original: Output:
Revised: Output:

Original: The trained model, i.e., the parameters W , b and β .
Revised: The trained model, i.e., the parameters W , b and β .

Original: The resulting embeddings h of nodes.
Revised: The resulting embeddings h of nodes.

Original: 1: Feature initialization: map the input Euclidean features x into Pseudo-Riemannian space by the double projection h 

Revised: 1: Feature initialization: map the input Euclidean features x into Pseudo-Riemannian space by the double projection h

Original: =
Revised: =

Original: ( ψ − 1 ◦ ψ )( x ) where ψ and ψ − 1 are given by ▷ Eq.
Revised: ( ψ − 1 ◦ ψ )( x ) where ψ and ψ − 1 are given by ▷ Eq.

Original: 2: for e ∈ [0 , E ) do 3: for l ∈ [0 , L ) do 4: for i ∈ V do 5: h ℓi = W ℓ ⊗ β h ℓi :=
Revised: 2: for e ∈ [0 , E ) do 3: for l ∈ [0 , L ) do 4: for i ∈ V do 5: h ℓi = W ℓ ⊗ β h ℓi :=

Original: (cid:100)
Revised: (cid:100)

Original: exp β
Revised: exp β

Original: o (cid:18) W ℓ (cid:99) log β o (cid:0) h ℓi (cid:1)(cid:19)
Revised: o (cid:18) W ℓ (cid:99) log β o (cid:0) h ℓi (cid:1)(cid:19)

Original: ( Feature transformation ) 6: h ℓi = (cid:101) h
Revised: ( Feature transformation ) 6: h ℓi = (cid:101) h

Original: i ℓ ⊕ β b ℓ :=    exp β (cid:102) h iℓ (cid:16) P β o → (cid:102)
Revised: i ℓ ⊕ β b ℓ :=    exp β (cid:102) h iℓ (cid:16) P β o → (cid:102)

Original: h iℓ (cid:0) b ℓ (cid:1)(cid:17) , if (cid:68) o , (cid:101) h i ℓ (cid:69) t <
Revised: h iℓ (cid:0) b ℓ (cid:1)(cid:17) , if (cid:68) o , (cid:101) h i ℓ (cid:69) t <

Original: | β | − exp β − (cid:102) h iℓ (cid:16) P β o →− (cid:102)
Revised: | β | − exp β − (cid:102) h iℓ (cid:16) P β o →− (cid:102)

Original: h iℓ (cid:0) b ℓ (cid:1)(cid:17)
Revised: h iℓ (cid:0) b ℓ (cid:1)(cid:17)

Original: [Equation]
Revised: [Equation]

Original: ( Bias translation ) 7:
Revised: ( Bias translation ) 7:

Original: [Equation]
Revised: [Equation]

Original: aggregation ) 8: end for 9:
Revised: aggregation ) 8: end for 9:

Original: //Computing the task-dependent loss L .
Revised: //Computing the task-dependent loss L .

Original: 10: //Updating the model parameters W , b , and β using Adam optimizer.
Revised: 10: //Updating the model parameters W , b , and β using Adam optimizer.

Original: 11: if convergence then 12: break loop.
Revised: 11: if convergence then 12: break loop.

Original: 13: end if 14: end for 15: end for 16: //Task-dependent inference.
Revised: 13: end if 14: end for 15: end for 16: //Task-dependent inference.

Original: 17: return W , b , β and h .
Revised: 17: return W , b , β and h .