Original: Mathmatically, it satisfies the following recursive relation:
Revised: Mathmatically, it satisfies the following recursive relation:

Original: [Equation]
Revised: [Equation]

Original: ifu = v,
Revised: if

Original: [Equation]
Revised: [Equation]

Original: if u and v are in different connected components ,
Revised: and v are in different connected components ,

Original: [Equation]
Revised: [Equation]

Original: The above equation can be used to calculate CTD and thus RD, as we will show later.
Revised: The above equation can be used to calculate CTD and thus RD, as we will show later.

Original: Resistance Distance is a graph metric .
Revised: Resistance Distance is a graph metric .

Original: We say a function d G : V × V → R is a graph metric if it is non-negative, positive semidefinite, symmetric, and satisfies triangular inequality.
Revised: We say a function d G : V × V → R is a graph metric if it is non-negative, positive semidefinite, symmetric, and satisfies triangular inequality.

Original: Let G be a connected graph.
Revised: Let G be a connected graph.

Original: Then Resistance Distance dis R G is a valid graph metric because:
Revised: Then Resistance Distance dis R G is a valid graph metric because:

Original: • (Positive semidefiniteness)
Revised: • (Positive semidefiniteness)

Original: dis R G ( u, v ) ≥ 0 holds for any u, v ∈ V .
Revised: dis R G ( u, v ) ≥ 0 holds for any u, v ∈ V .

Original: Moreover, dis R G ( u, v ) = 

Revised: Moreover, dis R G ( u, v ) =

Original: iff u = v .
Revised: iff u = v .

Original: • (Symmetry) dis
Revised: • (Symmetry) dis

Original: R G ( u, v ) =
Revised: R G ( u, v ) =

Original: dis R G ( v, u ) holds for any u, v ∈ V .
Revised: dis R G ( v, u ) holds for any u, v ∈ V .

Original: • (Triangular Inequality)
Revised: • (Triangular Inequality)

Original: For any u, v, w ∈ V , dis R G ( u, v ) + dis R G ( v, w ) ≥ dis R G ( u, w ) .
Revised: For any u, v, w ∈ V , dis R G ( u, v ) + dis R G ( v, w ) ≥ dis R G ( u, w ) .

Original: This can be seen from the definition of CTD, since dis C G ( u, v )+dis C G ( v, w ) is equal to the average hitting time from u to w under the condition of passing node v , which is obviously larger than dis R G ( u, w ) .
Revised: This can be seen from the definition of CTD, since dis C G ( u, v )+dis C G ( v, w ) is equal to the average hitting time from u to w under the condition of passing node v , which is obviously larger than dis R G ( u, w ) .

Original: Comparing RD with SPD .
Revised: Comparing RD with SPD .

Original: It is easy to see that RD is always no larger than SPD,
Revised: It is easy to see that RD is always no larger than SPD,

Original: i.e. dis R G ( u, v ) ≤ dis G ( u, v ) .
Revised: i.e. dis R G ( u, v ) ≤ dis G ( u, v ) .

Original: This is because for any subgraph G ′ of G , we have dis R G ( u, v ) ≤ dis R G ′ ( u, v ) , and when
Revised: This is because for any subgraph G ′ of G , we have dis R G ( u, v ) ≤ dis R G ′ ( u, v ) , and when

Original: G ′ is chosen to contain only the edges that belong to the shortest path between u and v , we have dis R G
Revised: G ′ is chosen to contain only the edges that belong to the shortest path between u and v , we have dis R G

Original: ( u, v ) =
Revised: ( u, v ) =

Original: dis G ( u, v ) .
Revised: dis G ( u, v ) .

Original: Therefore, the range of RD is the same as SPD, i.e. 0 ≤ dis R G ( u, v ) ≤ n − 1 .
Revised: Therefore, the range of RD is the same as SPD, i.e. 0 ≤ dis R G ( u, v ) ≤ n − 1 .

Original: However, unlike SPD which is an integer, RD can be a general rational number.
Revised: However, unlike SPD which is an integer, RD can be a general rational number.

Original: RD can thus be seen as a more fine-grained distance metric than SPD.
Revised: RD can thus be seen as a more fine-grained distance metric than SPD.

Original: Nevertheless, RD is still discrete and there are only finitely many possible values of dis R G ( u, v ) when n is fixed.
Revised: Nevertheless, RD is still discrete and there are only finitely many possible values of dis R G ( u, v ) when n is fixed.

Original: Relationship to graph Laplacian .
Revised: Relationship to graph Laplacian .

Original: We have the following theorem:
Revised: We have the following theorem:

Original: Theorem E.1.
Revised: Theorem E.1.

Original: Let G =
Revised: Let G =

Original: ( V , E ) be a connected graph, V = [ n ] , and let L ∈ S n be the graph
Revised: ( V , E ) be a connected graph, V = [ n ] , and let L ∈ S n be the graph

Original: Laplacian.
Revised: Laplacian.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where M ∈ S n is a symmetric matrix defined as
Revised: where M ∈ S n is a symmetric matrix defined as

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Denote d = (deg G (1) , · · · , deg G ( n )) ⊤ .
Revised: Denote d = (deg G (1) , · · · , deg G ( n )) ⊤ .

Original: Define the probability matrix P such that P ij =if { i, j } / ∈ E and P ij = 1 / deg G ( i ) if { i, j } ∈ E .
Revised: Define the probability matrix P such that P ij =if { i, j } / ∈ E and P ij = 1 / deg G ( i ) if { i, j } ∈ E .

Original: Then for any i ̸
Revised: Then for any i ̸

Original: = j , (25) can be equivalently written as
Revised: = j , (25) can be equivalently written as

Original: [Equation]
Revised: [Equation]

Original: Now define a matrix
Revised: Now define a matrix

Original: ˜ H such that ˜ H ij = 1 + (cid:80) nk
Revised: ˜ H such that ˜ H ij = 1 + (cid:80) nk

Original: =1 P ik ˜ H kj − P ij ˜ H jj , then ˜ H ij = h ( i, j ) for all i ̸ =
Revised: =1 P ik ˜ H kj − P ij ˜ H jj , then ˜ H ij = h ( i, j ) for all i ̸ =

Original: j (although ˜ H ii ̸
Revised: j (although ˜ H ii ̸

Original: = 

Revised: =

Original: = h ( i, i ) ).
Revised: = h ( i, i ) ).

Original: ˜ H can be equivalently written as
Revised: ˜ H can be equivalently written as

Original: [Equation]
Revised: [Equation]

Original:  where diag( ˜ H ) is the diagnal matrix with elements
Revised: 27) where diag( ˜ H ) is the diagnal matrix with elements

Original: ˜ H ii for i ∈ [ n ] .
Revised: ˜ H ii for i ∈ [ n ] .

Original: We first calculate diag( ˜ H ) .
Revised: We first calculate diag( ˜ H ) .

Original: Noting that d ⊤ P = d , we have
Revised: Noting that d ⊤ P = d , we have

Original: [Equation]
Revised: [Equation]

Original: , and thus d ⊤ diag( ˜ H ) =
Revised: and thus d ⊤ diag( ˜ H ) =

Original: d ⊤ 11 ⊤ , namely
Revised: d ⊤ 11 ⊤ , namely

Original: [Equation]
Revised: [Equation]

Original: Now define H =
Revised: Now define H =

Original: ˜
Revised: ˜

Original: H − diag( ˜ H ) , then H ij = h
Revised: H − diag( ˜ H ) , then H ij = h

Original: ( i, j ) for all i, j ∈ [ n ] .
Revised: ( i, j ) for all i, j ∈ [ n ] .

Original: We will calculate H in the following proof.
Revised: We will calculate H in the following proof.

Original: We first write (27) equivalently as H + diag( ˜ H ) =
Revised: We first write (27) equivalently as H + diag( ˜ H ) =

Original: 11 ⊤ + PH .
Revised: 11 ⊤ + PH .

Original: Then by multiplying D , we have
Revised: Then by multiplying D , we have

Original: [Equation]
Revised: [Equation]

Original: Using the fact that D ( I − P )
Revised: Using the fact that D ( I − P )

Original: =
Revised: =

Original: L and (28), we obtain
Revised: L and (28), we obtain

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Next, noting that L1 = 0 , we have
Revised: Next, noting that L1 = 0 , we have

Original: [Equation]
Revised: [Equation]

Original: One important property is that the matrix (cid:0) L + 1 n 11 ⊤ (cid:1) is invertible (see Gutman & Xiao (2004,
Revised: One important property is that the matrix (cid:0) L + 1 n 11 ⊤ (cid:1) is invertible (see Gutman & Xiao (2004,

Original: Theorem 4) for a proof).
Revised: Theorem 4) for a proof).

Original: Combining (30) and (31) we have 
Revised: Combining (30) and (31) we have (cid:18)

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: By taking diagonal elements and noting that diag( H )
Revised: By taking diagonal elements and noting that diag( H )

Original: =
Revised: =

Original: O , we otain
Revised: O , we otain

Original: [Equation]
Revised: [Equation]

Original: Namely,
Revised: Namely,

Original: [Equation]
Revised: [Equation]

Original: Substituting (34) into (32) yields
Revised: Substituting (34) into (32) yields

Original: [Equation]
Revised: [Equation]

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: This finally yields dis R G ( i, j )
Revised: This finally yields dis R G ( i, j )

Original: = 12 m dis C G ( i, [Equation]
Revised: = 12 m dis C G ( i, j ) =

Original: = 12 m ( H + [Equation]
Revised: 12 m ( H + H ⊤ ) =

Original: ⊤

Original: [Equation] = M i,i + M j,j − 2 M i,j and concludes the proof.
Revised: M i,i + M j,j − 2 M i,j and concludes the proof.

Original: Computational Complexity .
Revised: Computational Complexity .

Original: The graph Laplacian can be calculated in O ( n 2 ) time, and M can be calculated by matrix inversion which requires O ( n 3 ) time.
Revised: The graph Laplacian can be calculated in O ( n 2 ) time, and M can be calculated by matrix inversion which requires O ( n 3 ) time.

Original: Therefore, the overall computational complexity is O ( n 3 ) (or O ( n 2 . 376 ) using advanced matrix multiplication algorithms).
Revised: Therefore, the overall computational complexity is O ( n 3 ) (or O ( n 2 . 376 ) using advanced matrix multiplication algorithms).

Original: For sparse graphs typically encountered in practice (i.e. m =
Revised: For sparse graphs typically encountered in practice (i.e. m =

Original: o ( n 2 ) ), one may similarly ask whether a complexity that depends on m can be achieved.
Revised: o ( n 2 ) ), one may similarly ask whether a complexity that depends on m can be achieved.

Original: We conjecture that it should be possible.
Revised: We conjecture that it should be possible.

Original: Below, we give another algorithm to calculate (cid:0) L + 1 n 11 ⊤ (cid:1) − 1 .
Revised: Below, we give another algorithm to calculate (cid:0) L + 1 n 11 ⊤ (cid:1) − 1 .

Original: Note that the graph Laplacian L can be equivalently written as L = EE ⊤ , where E ∈ R n × m is defined asE
Revised: Note that the graph Laplacian L can be equivalently written as L = EE ⊤ , where E ∈ R n × m is defined as

Original: [Equation]
Revised: [Equation]

Original: if ϵ j = { i, k } and k > i − 1 if ϵ j = { i, k } andk < i if i / ∈
Revised: = { i, [Equation] and

Original: ϵ j (37) where we denote E = { ϵ 1 , · · · , ϵ m } .
Revised: [Equation] 37) where we denote E = { ϵ 1 , · · · , ϵ m } .

Original: Let E =
Revised: Let E =

Original: [ e 1 , · · · , e m ] where e i ∈ R n , then M = (cid:0) 1 n 11 ⊤ + (cid:80) mi
Revised: [ e 1 , · · · , e m ] where e i ∈ R n , then M = (cid:0) 1 n 11 ⊤ + (cid:80) mi

Original: =1 e i e ⊤ i (cid:1) − 1 .
Revised: =1 e i e ⊤ i (cid:1) − 1 .

Original: Noting that each e i is highly sparse with only two non-zero elements.
Revised: Noting that each e i is highly sparse with only two non-zero elements.

Original: We suspect that one can obtain an O ( nm ) complexity using techniques similar to the ShermanMorrison-Woodbury update.
Revised: We suspect that one can obtain an O ( nm ) complexity using techniques similar to the ShermanMorrison-Woodbury update.

Original: We leave it as an open problem.
Revised: We leave it as an open problem.

Original: E. T RANSFORMER - BASED IMPLEMENTATION
Revised: E. T RANSFORMER - BASED IMPLEMENTATION

Original: Graphormer-GD .
Revised: Graphormer-GD .

Original: The model is built on the Graphormer (Ying et al., 2021a) model, which use the
Revised: The model is built on the Graphormer (Ying et al., 2021a) model, which use the

Original: Transformer (Vaswani et al., 2017) as the backbone network.
Revised: Transformer (Vaswani et al., 2017) as the backbone network.

Original: A Transformer block consists of two layers: a self-attention layer followed by a feed-forward layer, with both layers having normalization (e.g., LayerNorm (Ba et al., 2016)) and skip connections (He et al., 2016).
Revised: A Transformer block consists of two layers: a self-attention layer followed by a feed-forward layer, with both layers having normalization (e.g., LayerNorm (Ba et al., 2016)) and skip connections (He et al., 2016).

Original: Denote X ( l ) ∈ R n × d as the input to the ( l + 1) -th block and define X (0) =
Revised: Denote X ( l ) ∈ R n × d as the input to the ( l + 1) -th block and define X (0) =

Original: X , where n is the number of nodes and d is the feature dimension.
Revised: X , where n is the number of nodes and d is the feature dimension.

Original: For an input X ( l ) , the ( l + 1) -th block works as follows:
Revised: For an input X ( l ) , the ( l + 1) -th block works as follows:

Original: [Equation]
Revised: [Equation]

Original: where W l,hO ∈ R d H × d , W l,hQ , W l,hK , W l,hV ∈ R d × d H , W l 1 ∈ R d × r , W l 2 ∈ R r × d , H is the number of attention heads, d H is the dimension of each head, and r is the dimension of the hidden layer.
Revised: where W l,hO ∈ R d H × d , W l,hQ , W l,hK , W l,hV ∈ R d × d H , W l 1 ∈ R d × r , W l 2 ∈ R r × d , H is the number of attention heads, d H is the dimension of each head, and r is the dimension of the hidden layer.

Original: A h ( X ) is usually referred to as the attention matrix.
Revised: A h ( X ) is usually referred to as the attention matrix.

Original: Note that the self-attention layer and the feed-forward layer introduced in (39) and (40) do not encode any structural information of the input graph.
Revised: Note that the self-attention layer and the feed-forward layer introduced in (39) and (40) do not encode any structural information of the input graph.

Original: As stated in Section 4, we incorporate the distance information into the attention layers of our Graphormer-GD model.
Revised: As stated in Section 4, we incorporate the distance information into the attention layers of our Graphormer-GD model.

Original: The calculation of the attention matrix in (38) is modified as:
Revised: The calculation of the attention matrix in (38) is modified as:

Original: [Equation]
Revised: [Equation]

Original: where D ∈ R n × n is the distance matrix such that D uv = d G ( u, v ) , ϕ h 1 and ϕ h 2 are element-wise functions applied to D , and ⊙ denotes the element-wise multiplication.
Revised: where D ∈ R n × n is the distance matrix such that D uv = d G ( u, v ) , ϕ h 1 and ϕ h 2 are element-wise functions applied to D , and ⊙ denotes the element-wise multiplication.

Original: In this way, the graph structural information can be captured by our Graphormer-GD model.
Revised: In this way, the graph structural information can be captured by our Graphormer-GD model.

Original: As stated in Section 4, we mainly consider two distance metrics: Shortest Path Distance dis G and
Revised: As stated in Section 4, we mainly consider two distance metrics: Shortest Path Distance dis G and

Original: Resistance Distance dis R G .
Revised: Resistance Distance dis R G .

Original: For SPD, we follow Ying et al.
Revised: For SPD, we follow Ying et al.

Original: (2021a) to use their shortest path distance encoding.
Revised: (2021a) to use their shortest path distance encoding.

Original: Formally, let D SPD be the SPD matrix
Revised: Formally, let D SPD be the SPD matrix

Original: such that D SPD uv =
Revised: such that D SPD uv =

Original: dis G ( u, v ) .
Revised: dis G ( u, v ) .

Original: The function ϕand ϕ 2 can simply be parameterized by two learnable vectors v 1 and v 2 , so that ϕ 1 ( D SPD uv ) is a learnable scalar corresponding to v 1 D SPD uv (and similarly for ϕ 2 ).
Revised: The function ϕand ϕ 2 can simply be parameterized by two learnable vectors v 1 and v 2 , so that ϕ 1 ( D SPD uv ) is a learnable scalar corresponding to v 1 D SPD uv (and similarly for ϕ 2 ).

Original: If two nodes u and v are not in the same connected component, i.e., D SPD uv = ∞ , a special learnable scalar is assigned.
Revised: If two nodes u and v are not in the same connected component, i.e., D SPD uv = ∞ , a special learnable scalar is assigned.

Original: For RD, we use the Gaussian Basis kernels (Scholkopf et al., 1997) to encode the value since it may not be an integer.
Revised: For RD, we use the Gaussian Basis kernels (Scholkopf et al., 1997) to encode the value since it may not be an integer.

Original: The encoded values from different Gaussian Basis kernels are concatenated and further transformed by a two-layer MLP.
Revised: The encoded values from different Gaussian Basis kernels are concatenated and further transformed by a two-layer MLP.

Original: We integrate both the SPD encoding and the RD encoding to obtain ϕ l,h 1 ( D ) and ϕ l,h 2 ( D ) .
Revised: We integrate both the SPD encoding and the RD encoding to obtain ϕ l,h 1 ( D ) and ϕ l,h 2 ( D ) .

Original: Note that these two matrices are parameterized by different sets of parameters.
Revised: Note that these two matrices are parameterized by different sets of parameters.

Original: Following Ying et al.
Revised: Following Ying et al.

Original: (2021a), we also incorporate the degree of each node in the input layer using a degree embedding.
Revised: (2021a), we also incorporate the degree of each node in the input layer using a degree embedding.

Original: Relationship between Graphormer-GD and GD-WL .
Revised: Relationship between Graphormer-GD and GD-WL .

Original: As stated in Section 4, the expressive power of Graphormer-GD is at most as powerful as GD-WL.
Revised: As stated in Section 4, the expressive power of Graphormer-GD is at most as powerful as GD-WL.

Original: We will prove that it is actually as powerful as
Revised: We will prove that it is actually as powerful as

Original: GD-WL under mild assumptions.
Revised: GD-WL under mild assumptions.

Original: We first restate the Lemma 5 from Xu et al.
Revised: We first restate the Lemma 5 from Xu et al.

Original: (2019), which shows that sum aggregators can represent injective functions over multisets.
Revised: (2019), which shows that sum aggregators can represent injective functions over multisets.

Original: Lemma E.2.
Revised: Lemma E.2.

Original: (Xu et al., 2019, Lemma 5) Assume the set X is countable.
Revised: (Xu et al., 2019, Lemma 5) Assume the set X is countable.

Original: Then there exists a function f : X → R n so that the function h ( ˆ X ) :
Revised: Then there exists a function f : X → R n so that the function h ( ˆ X ) :

Original: = (cid:80) x ∈ ˆ X f ( x ) is unique for each multiset ˆ X ⊂ X of bounded size.
Revised: = (cid:80) x ∈ ˆ X f ( x ) is unique for each multiset ˆ X ⊂ X of bounded size.

Original: Moreover, any multiset function g can be decomposed as g ( ˆ X ) =
Revised: Moreover, any multiset function g can be decomposed as g ( ˆ X ) =

Original: ϕ
Revised: ϕ

Original: ( (cid:80) x ∈
Revised: ( (cid:80) x ∈

Original: ˆ X f ( x ))
Revised: ˆ X f ( x ))

Original: for some function ϕ .
Revised: for some function ϕ .

Original: We are now ready to present the detailed proof of the Theorem 4.4, which is restated as follows:
Revised: We are now ready to present the detailed proof of the Theorem 4.4, which is restated as follows:

Original: Theorem E.3.
Revised: Theorem E.3.

Original: Graphormer-GD is at most as powerful as GD-WL.
Revised: Graphormer-GD is at most as powerful as GD-WL.

Original: Moreover, when choosing proper functions ϕ
Revised: Moreover, when choosing proper functions ϕ

Original: h 1 and ϕ h 2 and using a sufficiently large number of heads and layers, Graphormer-GD is as powerful as GD-WL.
Revised: h 1 and ϕ h 2 and using a sufficiently large number of heads and layers, Graphormer-GD is as powerful as GD-WL.

Original: Proof.
Revised: Proof.

Original: Consider all graphs with no more than n nodes.
Revised: Consider all graphs with no more than n nodes.

Original: The total number of possible values of both
Revised: The total number of possible values of both

Original: SPD and RD are thus finite and depends on n (see Appendix E.2).
Revised: SPD and RD are thus finite and depends on n (see Appendix E.2).

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: denote the set of all possible pairs (dis G ( u, v ) , dis R G ( u, v )) .
Revised: denote the set of all possible pairs (dis G ( u, v ) , dis R G ( u, v )) .

Original: Since D n is finite, we can list its elements as D n = { d G, 1 , · · · , d G, |D n | } .
Revised: Since D n is finite, we can list its elements as D n = { d G, 1 , · · · , d G, |D n | } .

Original: Without abuse of notation, denote d G ( u, v ) =
Revised: Without abuse of notation, denote d G ( u, v ) =

Original: (dis G ( u, v ) , dis R G ( u, v )) .
Revised: (dis G ( u, v ) , dis R G ( u, v )) .

Original: Then the GD-WL aggregation in (3) can be reformulated as follows:
Revised: Then the GD-WL aggregation in (3) can be reformulated as follows:

Original: [Equation]
Revised: [Equation]

Original: where χ t,kG ( v) :
Revised: where χ t,kG ( v

Original: = {{ χ t − 1 G ( u ) : u∈ V , d G ( u, v ) =
Revised: [Equation] {{ χ t − 1 G [Equation] u

Original: d G,k }} .
Revised: [Equation] : u ∈

Original: [Equation]
Revised: [Equation]

Original: Intuitively, this reformulation indicates that in each iteration, GD-WL updates the color of node v by hashing a tuple of color multisets, where each multiset is obtained by injectively aggregating the colors of all nodes u ∈ V with certain distance configuration to node v .
Revised: Intuitively, this reformulation indicates that in each iteration, GD-WL updates the color of node v by hashing a tuple of color multisets, where each multiset is obtained by injectively aggregating the colors of all nodes u ∈ V with certain distance configuration to node v .

Original: Therefore, to express
Revised: Therefore, to express

Original: GD-WL, the model suffices to update the representation of each node following the above procedure.
Revised: GD-WL, the model suffices to update the representation of each node following the above procedure.

Original: We show Graphormer-GD can achieve this goal.
Revised: We show Graphormer-GD can achieve this goal.

Original: Recall that for the h -th head, the attention matrix is defined as ϕ h 1 ( D ) ⊙ softmax (cid:0) XW h Q ( XW h K ) ⊤ + ϕ h 2 ( D ) (cid:1) .
Revised: Recall that for the h -th head, the attention matrix is defined as ϕ h 1 ( D ) ⊙ softmax (cid:0) XW hQ ( XW hK ) ⊤ + ϕ h 2 ( D ) (cid:1) .

Original: For the function ϕ h 1 , we define it to be the indicator function ϕ
Revised: For the function ϕ h 1 , we define it to be the indicator function ϕ

Original: h 1 ( d ) :
Revised: h 1 ( d ) :

Original: = I ( d = d G,h ) .
Revised: = I ( d = d G,h ) .

Original: For the function ϕ h 2 , we set it to be constant irrespective to the matrix D .
Revised: For the function ϕ h 2 , we set it to be constant irrespective to the matrix D .

Original: Let W hQ , W hK be zero matrices.
Revised: Let W hQ , W hK be zero matrices.

Original: It can be seen that the term softmax (cid:0) XW hQ ( XW hK ) ⊤ + ϕ h 2 ( D ) (cid:1) reduces to 1 |V| 11 ⊤ , and thus for each node v , the output in the h -th attention head is the sum aggregation of representations of node u satisfying d G ( u, v ) =
Revised: It can be seen that the term softmax (cid:0) XW hQ ( XW hK ) ⊤ + ϕ h 2 ( D ) (cid:1) reduces to 1 |V| 11 ⊤ , and thus for each node v , the output in the h -th attention head is the sum aggregation of representations of node u satisfying d G ( u, v ) =

Original: d G,h .
Revised: d G,h .

Original: Formally,
Revised: Formally,

Original: [Equation]
Revised: [Equation]

Original: Note that the constant 1 |V| can be extracted with an additional head and be concatenated to the node representations.
Revised: Note that the constant 1 |V| can be extracted with an additional head and be concatenated to the node representations.

Original: Moreover, the node representation X is processed via the feed-forward network in the previous layer (see (40).
Revised: Moreover, the node representation X is processed via the feed-forward network in the previous layer (see (40).

Original: Thus, we can invoke Lemma E.2 and prove that the h -th attention head in Graphormer-GD can implement an injective aggregating function for {{ χ t − 1 G ( u ) : u ∈
Revised: Thus, we can invoke Lemma E.2 and prove that the h -th attention head in Graphormer-GD can implement an injective aggregating function for {{ χ t − 1 G ( u ) : u ∈

Original: V , d G ( u, v ) =
Revised: V , d G ( u, v ) =

Original: d G,h }} .
Revised: d G,h }} .

Original: Therefore, by using a sufficiently large number of attention heads, the multiset representations χ t,kG , k ∈ [ |D n | ] can be injectively obtained.
Revised: Therefore, by using a sufficiently large number of attention heads, the multiset representations χ t,kG , k ∈ [ |D n | ] can be injectively obtained.

Original: Finally, the multi-head attention defined in (39) is equivalent to first concatenating the output of each attention head and then using a linear mapping to transform the results.
Revised: Finally, the multi-head attention defined in (39) is equivalent to first concatenating the output of each attention head and then using a linear mapping to transform the results.

Original: Thus, the concatenation is clearly an injective mapping of the tuple of multisets (cid:16) χ t, 1 G , χ t, 2 G , ..., χ t, |D n | G (cid:17) .
Revised: Thus, the concatenation is clearly an injective mapping of the tuple of multisets (cid:16) χ t, 1 G , χ t, 2 G , ..., χ t, |D n | G (cid:17) .

Original: When the linear mapping has irrelational weights, the projection will also be injective.
Revised: When the linear mapping has irrelational weights, the projection will also be injective.

Original: Therefore, one attention layer followed by the feed-forward network can implement the aggregation formula (42).
Revised: Therefore, one attention layer followed by the feed-forward network can implement the aggregation formula (42).

Original: Thus, our
Revised: Thus, our

Original: Graphormer-GD is able to simulate the GD-WL when using a sufficiant number of layers, which concludes the proof.
Revised: Graphormer-GD is able to simulate the GD-WL when using a sufficiant number of layers, which concludes the proof.

Original: F E XPERIMENTAL D ETAILS F. S YNTHETIC T ASKS
Revised: F E XPERIMENTAL D ETAILS F. S YNTHETIC T ASKS

Original: Data Generation and Evaluation Metrics.
Revised: Data Generation and Evaluation Metrics.

Original: We carefully design several graph generators to examine the expressive power of compared models on graph biconnectivity tasks.
Revised: We carefully design several graph generators to examine the expressive power of compared models on graph biconnectivity tasks.

Original: First, we include the two families of graphs presented in Examples C.9 and C.10 (Appendix C.2).
Revised: First, we include the two families of graphs presented in Examples C.9 and C.10 (Appendix C.2).

Original: We further introduce a rich family of regular graphs with both cut vertices and cut edges.
Revised: We further introduce a rich family of regular graphs with both cut vertices and cut edges.

Original: Each graph in this family is constructed by first randomly generating several connected components and then linking them via cut edges while simultaneously ensuring that each node has the same degree.
Revised: Each graph in this family is constructed by first randomly generating several connected components and then linking them via cut edges while simultaneously ensuring that each node has the same degree.

Original: Combining the above three families of hard graphs, we online generate data instances to train the compared models.
Revised: Combining the above three families of hard graphs, we online generate data instances to train the compared models.

Original: For each data instance, the total number of nodes is upper bounded by 120.
Revised: For each data instance, the total number of nodes is upper bounded by 120.

Original: We use graph-level accuracy as the metric.
Revised: We use graph-level accuracy as the metric.

Original: That is, for each graph, the prediction of the model is considered correct only when all and only the cut vertices/edges are correctly identified.
Revised: That is, for each graph, the prediction of the model is considered correct only when all and only the cut vertices/edges are correctly identified.

Original: We use different seeds to repeat the experiments 5 times and report the average accuracy.
Revised: We use different seeds to repeat the experiments 5 times and report the average accuracy.

Original: Baselines.
Revised: Baselines.

Original: We choose several baselines with their expressive power being at different levels.
Revised: We choose several baselines with their expressive power being at different levels.

Original: First, we consider classic MPNNs including GCN (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018), and GIN (Bouritsas et al., 2022).
Revised: First, we consider classic MPNNs including GCN (Kipf & Welling, 2017), GAT (Veliˇckovi´c et al., 2018), and GIN (Bouritsas et al., 2022).

Original: The expressive power of these GNNs is proven to be at most as powerful as the 1-WL test (Xu et al., 2019).
Revised: The expressive power of these GNNs is proven to be at most as powerful as the 1-WL test (Xu et al., 2019).

Original: We also compare the Graph Substructure Network (Bouritsas et al., 2022), which extracts graph substructures to improve the expressive power of
Revised: We also compare the Graph Substructure Network (Bouritsas et al., 2022), which extracts graph substructures to improve the expressive power of

Original: MPNNs.
Revised: MPNNs.

Original: The substructure counts are incorporated into node features or the aggregation procedure.
Revised: The substructure counts are incorporated into node features or the aggregation procedure.

Original: Lastly, we also compare the Graphormer model (Ying et al., 2021a), which achieved impressive performance in several world competitions (Ying et al., 2021b; Shi et al., 2022).
Revised: Lastly, we also compare the Graphormer model (Ying et al., 2021a), which achieved impressive performance in several world competitions (Ying et al., 2021b; Shi et al., 2022; Luo et al., 2022a).

Original: Settings.
Revised: Settings.

Original: We employ a 6-layer Graphormer-GD model.
Revised: We employ a 6-layer Graphormer-GD model.

Original: The dimension of hidden layers and feedforward layers is set to 768.
Revised: The dimension of hidden layers and feedforward layers is set to 768.

Original: The number of Gaussian Basis kernels is set to 128.
Revised: The number of Gaussian Basis kernels is set to 128.

Original: The number of attention heads is set to 64.
Revised: The number of attention heads is set to 64.

Original: The batch size is set to 32.
Revised: The batch size is set to 32.

Original: We use AdamW (Kingma & Ba, 2014) as the optimizer and set its hyperparameter ϵ to 1e-8 and ( β 1 , β 2 ) to (0.9, 0.999).
Revised: We use AdamW (Kingma & Ba, 2014) as the optimizer and set its hyperparameter ϵ to 1e-8 and ( β 1 , β 2 ) to (0.9, 0.999).

Original: The peak learning rate is set to 9e-5.
Revised: The peak learning rate is set to 9e-5.

Original: The model is trained for 100k steps with a 6K-step warm-up stage.
Revised: The model is trained for 100k steps with a 6K-step warm-up stage.

Original: After the warm-up stage, the learning rate decays linearly to 0.
Revised: After the warm-up stage, the learning rate decays linearly to 0.

Original: All models are trained on 1 NVIDIA Tesla V100 GPU.
Revised: All models are trained on 1 NVIDIA Tesla V100 GPU.

Original: F. R EAL - WORLD T ASKS
Revised: F. R EAL - WORLD T ASKS

Original: We conduct experiments on the popular benchmark dataset: ZINC from Benchmarking-GNNs (Dwivedi et al., 2020).
Revised: We conduct experiments on the popular benchmark dataset: ZINC from Benchmarking-GNNs (Dwivedi et al., 2020).

Original: It is a real-world dataset that consists of 250K molecular graphs.
Revised: It is a real-world dataset that consists of 250K molecular graphs.

Original: The task is to predict the constrained solubility of a molecule, which is an important chemical property for drug discovery.
Revised: The task is to predict the constrained solubility of a molecule, which is an important chemical property for drug discovery.

Original: We train our models on both the ZINC-Full and ZINC-Subset (12K selected graphs following Dwivedi et al.
Revised: We train our models on both the ZINC-Full and ZINC-Subset (12K selected graphs following Dwivedi et al.

Original: Baselines.
Revised: Baselines.

Original: For a fair comparison, we set the parameter budget of the model to be around 500K following Dwivedi et al.
Revised: For a fair comparison, we set the parameter budget of the model to be around 500K following Dwivedi et al.

Original: We compare our Graphormer-GD with several competitive baselines, which mainly fall into five categories: Message Passing Neural Networks (MPNNs), High-order
Revised: We compare our Graphormer-GD with several competitive baselines, which mainly fall into five categories: Message Passing Neural Networks (MPNNs), High-order

Original: GNNs, Substructure-based GNNs, Subgraph GNNs, and Graph Transformers.
Revised: GNNs, Substructure-based GNNs, Subgraph GNNs, and Graph Transformers.

Original: First, we compare several classic MPNNs including Graph Convolution Network (GCN) (Kipf &
Revised: First, we compare several classic MPNNs including Graph Convolution Network (GCN) (Kipf &

Original: Welling, 2017), Graph Isomorphism Network (GIN) (Xu et al., 2019), Graph Attention Network (GAT) (Veliˇckovi´c et al., 2018), GraphSAGE (Hamilton et al., 2017) and MPNN(sum) (Gilmer et al., 2017).
Revised: Welling, 2017), Graph Isomorphism Network (GIN) (Xu et al., 2019), Graph Attention Network (GAT) (Veli ˇ ckovi ´ c et al., 2018), GraphSAGE (Hamilton et al., 2017) and MPNN(sum) (Gilmer et al., 2017).

Original: Besides, we also include several popularly used models.
Revised: Besides, we also include several popularly used models.

Original: Mixture Model Network (MoNet) (Monti et al., 2017) introduces a general architecture to learn on graphs and manifolds using the Bayesian Gaussian Mixture Model.
Revised: Mixture Model Network (MoNet) (Monti et al., 2017) introduces a general architecture to learn on graphs and manifolds using the Bayesian Gaussian Mixture Model.

Original: Gated Graph ConvNet (GatedGCN) considers residual connections, batch normalization, and edge gates to design an anisotropic variant of GCN.
Revised: Gated Graph ConvNet (GatedGCN) considers residual connections, batch normalization, and edge gates to design an anisotropic variant of GCN.

Original: We compare the GatedGCN with positional encodings.
Revised: We compare the GatedGCN with positional encodings.

Original: Principal Neighborhood Aggregation (PNA) (Corso et al., 2020) combines multiple aggregators with degree-scalers.
Revised: Principal Neighborhood Aggregation (PNA) (Corso et al., 2020) combines multiple aggregators with degree-scalers.

Original: Second, we compare two higher-order Graph Neural Networks: RingGNN (Chen et al., 2019) and 3WLGNN (Maron et al., 2019a) following Dwivedi et al.
Revised: Second, we compare two higher-order Graph Neural Networks: RingGNN (Chen et al., 2019) and 3WLGNN (Maron et al., 2019a) following Dwivedi et al.

Original: RingGNN extends the family of order-2 Graph G -invariant Networks without going into higher order tensors and is able to distinguish between non-isomorphic regular graphs where order-2 G -invariant networks provably fail.
Revised: RingGNN extends the family of order-2 Graph G -invariant Networks without going into higher order tensors and is able to distinguish between non-isomorphic regular graphs where order-2 G -invariant networks provably fail.

Original: 3WLGNN uses rank-2 tensors to build the neural network and is proved to be equivalent to the 3-WL test on graph isomorphism problems.
Revised: 3WLGNN uses rank-2 tensors to build the neural network and is proved to be equivalent to the 3-WL test on graph isomorphism problems.

Original: Third, we compare two representative types of substructure-based GNNs.
Revised: Third, we compare two representative types of substructure-based GNNs.

Original: The Graph Substructure
Revised: The Graph Substructure

Original: Network (Bouritsas et al., 2022) extracts graph substructures to improve the expressive power of
Revised: Network (Bouritsas et al., 2022) extracts graph substructures to improve the expressive power of

Original: MPNNs.
Revised: MPNNs.

Original: The substructure counts are incorporated into node features or the aggregation procedure.
Revised: The substructure counts are incorporated into node features or the aggregation procedure.

Original: We also compare the Cellular Isomorphism Network (Bodnar et al., 2021a), which extends theoretical results on Simplicial Complexes to regular Cell Complexes.
Revised: We also compare the Cellular Isomorphism Network (Bodnar et al., 2021a), which extends theo- retical results on Simplicial Complexes to regular Cell Complexes.

Original: Such generalization provides a powerful set of graph “lifting” transformations with a hierarchical message passing procedure.
Revised: Such generalization provides a powerful set of graph “lifting” transformations with a hierarchical message passing procedure.

Original: Moreover, we compare several Subgraph GNNs.
Revised: Moreover, we compare several Subgraph GNNs.

Original: Nested Graph Neural Network (NGNN) (Zhang & Li, 2021) represents a graph with rooted subgraphs instead of rooted subtrees.
Revised: Nested Graph Neural Network (NGNN) (Zhang & Li, 2021) represents a graph with rooted subgraphs instead of rooted subtrees.

Original: It extracts a local subgraph around each node and applies a base GNN to each subgraph to learn a subgraph representation.
Revised: It extracts a local subgraph around each node and applies a base GNN to each subgraph to learn a subgraph representation.

Original: The whole-graph representation is then obtained by pooling these subgraph representations.
Revised: The whole-graph representation is then obtained by pooling these subgraph representations.

Original: GNN-AK (Zhao et al., 2022) follows a similar manner to develop Subgraph GNNs with different generation policies.
Revised: GNN-AK (Zhao et al., 2022) follows a similar manner to develop Subgraph GNNs with different generation policies.

Original: Equivariant Subgraph Aggregation Networks (ESAN) (Bevilacqua et al., 2022) develops a unified framework that includes per-layer aggregation across subgraphs, which are generated using pre-defined policies like edge deletion and ego-networks.
Revised: Equivariant Subgraph Aggregation Networks (ESAN) (Bevilacqua et al., 2022) develops a unified framework that includes per-layer aggregation across subgraphs, which are generated using pre-defined policies like edge deletion and ego-networks.

Revised: Subgraph Union Network (SUN) (Frasca et al., 2022) is developed based on the symmetry analysis of a series of existing

Revised: Subgraph GNNs and an upper bound on their expressive power, which theoretically unifies previous architectures and performs well across several graph representation learning benchmarks.

Original: Last, we compare several Graph Transformer models.
Revised: Last, we compare several Graph Transformer models.

Original: GraphTransformer (GT) (Dwivedi & Bresson, 2021) uses the Transformer model on graph tasks, which only aggregates the information from neighbor nodes to ensure graph sparsity, and proposes to use Laplacian eigenvector as positional encoding.
Revised: GraphTransformer (GT) (Dwivedi & Bresson, 2021) uses the Transformer model on graph tasks, which only aggregates the information from neighbor nodes to ensure graph sparsity, and proposes to use Laplacian eigenvector as positional encoding.

Original: Spectral Attention Network (SAN) (Kreuzer et al., 2021) uses a learned positional encoding (LPE) that can take advantage of the full Laplacian spectrum to learn the position of each node in a given graph.
Revised: Spectral Attention Network (SAN) (Kreuzer et al., 2021) uses a learned positional encoding (LPE) that can take advantage of the full Laplacian spectrum to learn the position of each node in a given graph.

Original: Graphormer (Ying et al., 2021a) develops the centrality encoding, spatial encoding, and edge encoding to incorporate the graph structure information into the Transformer model.
Revised: Graphormer (Ying et al., 2021a) develops the centrality encoding, spatial encoding, and edge encoding to incorporate the graph structure information into the Transformer model.

Revised: Universal RPE (URPE) (Luo et al., 2022b) first shows that there exist continuous sequenceto-sequence functions which RPE-based Transformers cannot approximate, and develops a novel and universal attention module called Universal RPE-based Attention.

Revised: The effectiveness of URPE has been verified across language and graph benchmarks (e.g., the ZINC dataset).

Original: Settings.
Revised: Settings.

Original: Our Graphormer-GD consists of 12 layers.
Revised: Our Graphormer-GD consists of 12 layers.

Original: The dimension of hidden layers and feedforward layers are set to 80.
Revised: The dimension of hidden layers and feedforward layers are set to 80.

Original: The number of Gaussian Basis kernels is set to 128.
Revised: The number of Gaussian Basis kernels is set to 128.

Original: The number of attention heads is set to 8.
Revised: The number of attention heads is set to 8.

Original: The batch size is selected from [128, 256, 512].
Revised: The batch size is selected from [128, 256, 512].

Original: We use AdamW (Kingma & Ba, 2014) as the optimizer, and set its hyperparameter ϵ to 1e-8 and ( β 1 , β 2 ) to (0.9, 0.999).
Revised: We use AdamW (Kingma & Ba, 2014) as the optimizer, and set its hyperparameter ϵ to 1e-8 and ( β 1 , β 2 ) to (0.9, 0.999).

Original: The peak learning rate is selected from [4e-4, 5e-4].
Revised: The peak learning rate is selected from [4e-4, 5e-4].

Original: The model is trained for 600k and 800k steps with a 60K-step warm-up stage for ZINC-Subset and ZINC-Full respectively.
Revised: The model is trained for 600k and 800k steps with a 60K-step warm-up stage for ZINC-Subset and ZINC-Full respectively.

Original: After the warm-up stage, the learning rate decays linearly to zero.
Revised: After the warm-up stage, the learning rate decays linearly to zero.

Original: The dropout ratio is selected from [0.0, 0.1].
Revised: The dropout ratio is selected from [0.0, 0.1].

Original: The weight decay is selected from [0.0, 0.01].
Revised: The weight decay is selected from [0.0, 0.01].

Original: All models are trained on 4 NVIDIA Tesla V100 GPUs.
Revised: All models are trained on 4 NVIDIA Tesla V100 GPUs.

Original: F. M ORE T ASKS
Revised: F. M ORE T ASKS

Original: Node-level Tasks.
Revised: Node-level Tasks.

Original: We further conduct experiments on real-world node-level tasks.
Revised: We further conduct experiments on real-world node-level tasks.

Original: Following Li et al.
Revised: Following Li et al.

Original: (2020), we benchmark our model on two real-world graphs: Brazil-Airports and EuropeAirports, both of which are air traffic networks and are collected by Ackland et al.
Revised: (2020), we benchmark our model on two real-world graphs: Brazil-Airports and EuropeAirports, both of which are air traffic networks and are collected by Ackland et al.

Original: from the government websites.
Revised: from the government websites.

Original: The nodes in each graph represent airports and each edge represents that there are commercial flights between the connected nodes.
Revised: The nodes in each graph represent airports and each edge represents that there are commercial flights between the connected nodes.

Original: The Brazil-Airports graph hasnodes, 1038 edges in total and its diameter is 5.
Revised: The Brazil-Airports graph hasnodes, 1038 edges in total and its diameter is 5.

Original: The Europe-Airports graph has 399 nodes,edges in total and its diameter is 5.
Revised: The Europe-Airports graph has 399 nodes,edges in total and its diameter is 5.

Original: The airport nodes are divided into 4 different levels according to the annual passenger flow distribution by 3 quantiles: 25%, 50%, and 75%.
Revised: The airport nodes are divided into 4 different levels according to the annual passenger flow distribution by 3 quantiles: 25%, 50%, and 75%.

Original: The task is to predict the level of each airport node.
Revised: The task is to predict the level of each airport node.

Original: We follow Li et al.
Revised: We follow Li et al.

Original: (2020) to split the nodes of each graph into train/validation/test subsets with the ratio being 0.8/0.1/0.1, respectively.
Revised: (2020) to split the nodes of each graph into train/validation/test subsets with the ratio being 0.8/0.1/0.1, respectively.

Original: The test accuracy of the best checkpoint on the validation set is reported.
Revised: The test accuracy of the best checkpoint on the validation set is reported.

Original: We use different seeds to repeat the experiments 20 times and report the average accuracy.
Revised: We use different seeds to repeat the experiments 20 times and report the average accuracy.

Original: Following Li et al.
Revised: Following Li et al.

Original: (2020), we choose several competitive baselines including classical MPNNs (GCN, GraphSAGE, GIN), Struc2vec and Distance-encoding based GNNs (DE-GNN-SPD, DEGNN-LP, DEA-GNN-SPD).
Revised: (2020), we choose several competitive baselines including classical MPNNs (GCN, GraphSAGE, GIN), Struc2vec and Distance-encoding based GNNs (DE-GNN-SPD, DEGNN-LP, DEA-GNN-SPD).

Original: We refer interested readers to Li et al.
Revised: We refer interested readers to Li et al.

Original: (2020) for detailed descriptions of baselines.
Revised: (2020) for detailed descriptions of baselines.

Original: For our Graphormer-GD, the dimension of hidden layers and feed-forward layers are set to 80.
Revised: For our Graphormer-GD, the dimension of hidden layers and feed-forward layers are set to 80.

Original: The number of layers is selected from [3, 6].
Revised: The number of layers is selected from [3, 6].

Original: The number of Gaussian Basis kernels is set to 128.
Revised: The number of Gaussian Basis kernels is set to 128.

Original: The number of attention heads is set to 8.
Revised: The number of attention heads is set to 8.

Original: The batch size is selected from [4, 8, 16, 32].
Revised: The batch size is selected from [4, 8, 16, 32].

Original: We use AdamW (Kingma & Ba, 2014) as the optimizer, and set its hyperparameter ϵ to 1e-8 and ( β 1 , β 2 ) to (0.9, 0.999).
Revised: We use AdamW (Kingma & Ba, 2014) as the optimizer, and set its hyperparameter ϵ to 1e-8 and ( β 1 , β 2 ) to (0.9, 0.999).

Original: The peak learning rate is selected from [2e-4, 7e-5, 4e-5].
Revised: The peak learning rate is selected from [2e-4, 7e-5, 4e-5].

Original: The total number of training steps is selected from [500, 1000, 2000].
Revised: The total number of training steps is selected from [500, 1000, 2000].

Original: The ratio of the warm-up stage is set to 10%.
Revised: The ratio of the warm-up stage is set to 10%.

Original: After the warm-up stage, the learning rate decays linearly to zero.
Revised: After the warm-up stage, the learning rate decays linearly to zero.

Original: The dropout ratio is selected from [0.0, 0.1, 0.5].
Revised: The dropout ratio is selected from [0.0, 0.1, 0.5].

Original: All models are trained on 1 NVIDIA Tesla V100 GPUs.
Revised: All models are trained on 1 NVIDIA Tesla V100 GPUs.

Original: The results are presented in Table 5.
Revised: The results are presented in Table 5.

Original: We can see that our model outperforms these baselines on both datasets with a slightly larger variance value due to the small scale of the datasets.
Revised: We can see that our model outperforms these baselines on both datasets with a slightly larger variance value due to the small scale of the datasets.

Original: [Table]
Revised: [Table]

Original: F. E FFICIENCY E VALUATION
Revised: F. E FFICIENCY E VALUATION

Original: We further conduct experiments to measure the efficiency of our approach by profiling the time cost per training epoch.
Revised: We further conduct experiments to measure the efficiency of our approach by profiling the time cost per training epoch.

Original: We compare the efficiency of Graphormer-GD with other baselines along with the number of model parameters on the ZINC-subset from Dwivedi et al.
Revised: We compare the efficiency of Graphormer-GD with other baselines along with the number of model parameters on the ZINC-subset from Dwivedi et al.

Original: The number of layers and the hidden dimension of our Graphormer-GD are set to 12 and 80 respectively.
Revised: The number of layers and the hidden dimension of our Graphormer-GD are set to 12 and 80 respectively.

Original: The number of attention heads is set to 8.
Revised: The number of attention heads is set to 8.

Original: The batch size is set to 128, which is the same as the settings of all baselines.
Revised: The batch size is set to 128, which is the same as the settings of all baselines.

Original: We run profiling of all models on a 16GB NVIDIA Tesla V100 GPU.
Revised: We run profiling of all models on a 16GB NVIDIA Tesla V100 GPU.

Original: For all baselines, we evaluate the time costs based on the publicly available codes of Dwivedi et al.
Revised: For all baselines, we evaluate the time costs based on the publicly available codes of Dwivedi et al.

Original: and Ying et al.
Revised: and Ying et al.

Original: (2021a).
Revised: (2021a).

Original: The results are presented in Table 6.
Revised: The results are presented in Table 6.

Original: From Table 6, we can draw the following conclusions.
Revised: From Table [Table] we can draw the following conclusions.

Revised: Firstly

Original: Firstly, the efficiency of Graphormer-GD is in the same order of magnitude as classic MPNNs despite the fact that the computation complexity of Graphormer-GD is higher than MPNNs (i.e., Θ( n 2 ) v.s. Θ( n + m ) for a graph with n nodes and m edges).
Revised: [Table] the efficiency of Graphormer-GD is in the same order of magnitude as classic MPNNs despite the fact that the computation complexity of Graphormer-GD is higher than MPNNs (i [Table] e., Θ( n 2 ) v

Revised: [Table]

Revised: s. Θ( n + m ) for a graph with n nodes and m edges).

Original: This may be due to the high parallelizability of the Transformer layers.
Revised: This may be due to the high parallelizability of the Transformer layers.

Original: Secondly,
Revised: Secondly,

Original: Graphormer-GD is much more efficient than higher-order GNNs as reflected by the computation complexity in Table 1.
Revised: Graphormer-GD is much more efficient than higher-order GNNs as reflected by the computation complexity in Table 1.

Original: Finally, Graphormer-GD is almost as efficient as the original Graphormer, since the newly introduced module to encode the Resistance Distance takes negligible additional time compared to that of the whole architecture.
Revised: Finally, Graphormer-GD is almost as efficient as the original Graphormer, since the newly introduced module to encode the Resistance Distance takes negligible additional time compared to that of the whole architecture

Original: [Table]
Revised: [Table]

Revised: [Table]