Original: U NION S UBGRAPH N EURAL N ETWORKS
Revised: U NION S UBGRAPH N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Graph Neural Networks (GNNs) are widely used for graph representation learning in many application domains.
Revised: Graph Neural Networks (GNNs) are widely used for graph representation learning in many application domains.

Original: The expressiveness of GNNs is upper-bounded by 1-dimensional Weisfeiler-Lehman (1-WL) test as they operate on rooted subtrees in message passing.
Revised: The expressiveness of GNNs is upper-bounded by 1-dimensional Weisfeiler-Lehman (1-WL) test as they operate on rooted subtrees in message passing.

Original: In this paper, we empower GNNs by injecting neighborconnectivity information extracted from a new type of substructures.
Revised: In this paper, we empower GNNs by injecting neighborconnectivity information extracted from a new type of substructures.

Original: We first investigate different kinds of connectivities existing in a local neighborhood and identify a substructure called union subgraph, which is able to capture the complete picture of the neighborhood.
Revised: We first investigate different kinds of connectivities existing in a local neighborhood and identify a substructure called union subgraph, which is able to capture the complete picture of the 1-hop neighborhood of an edge.

Original: We then design a shortest-path-based substructure descriptor that possesses three nice properties and can effectively encode the high-order connectivities in union subgraphs.
Revised: We then design a shortestpath-based substructure descriptor that possesses three nice properties and can effectively encode the high-order connectivities in union subgraphs.

Original: By infusing the encoded neighbor connectivities, we propose a novel model, namely Union Subgraph Neural Network (UnionSNN), which is proven to be strictly more powerful than 1-WL in distinguishing non-isomorphic graphs.
Revised: By infusing the encoded neighbor connectivities, we propose a novel model, namely Union Subgraph Neural Network (UnionSNN), which is proven to be strictly more powerful than 1-WL in distinguishing non-isomorphic graphs.

Original: Our extensive experiments on both graphlevel and node-levelclassification tasks demonstrate that UnionSNN outperforms state-of-the-art baseline models, with competitive computational efficiency.
Revised: Our extensive experiments on both graph-level and node-level tasks demonstrate that UnionSNN outperforms state-of-the-art baseline models, with competitive computational efficiency.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: With the ubiquity of graph-structured data emerging from various modern applications, Graph Neural Networks (GNNs) have gained increasing attention from both researchers and practitioners.
Revised: With the ubiquity of graph-structured data emerging from various modern applications, Graph Neural Networks (GNNs) have gained increasing attention from both researchers and practitioners.

Original: GNNs have been applied to many application domains, including quantum chemistry (Duvenaud et al., 2015; Dai et al., 2016), social sciences (Ying et al., 2018; Fan et al., 2019), and brain networks (Ahmedt-Aristizabal et al., 2021).
Revised: GNNs have been applied to many application domains, including quantum chemistry (Duvenaud et al., 2015; Dai et al., 2016), social sciences (Ying et al., 2018; Fan et al., 2019), and brain networks (Ahmedt-Aristizabal et al., 2021).

Original: GNNs also attained promising results on the tasks of graph classification (Xu et al., 2018;
Revised: GNNs also attained promising results on the tasks of graph classification (Xu et al., 2018;

Original: Ying et al., 2018), node classification (Kipf & Welling, 2016; Hamilton et al., 2017), link prediction (Zhang & Chen, 2018), etc.
Revised: Ying et al., 2018), node classification (Kipf & Welling, 2016; Hamilton et al., 2017), link prediction (Zhang & Chen, 2018), etc.

Original: Recently, a limitation in the expressiveness of GNNs has been identified.
Revised: Recently, a limitation in the expressiveness of GNNs has been identified.

Original: Xu et al.
Revised: Xu et al.

Original: (2018) shows that GNNs are at most as powerful as 1-dimentional Weisfeiler-Lehman (1-WL) test (Weisfeiler &
Revised: (2018) shows that GNNs are at most as powerful as 1-dimentional Weisfeiler-Lehman (1-WL) test (Weisfeiler &

Original: Leman, 1968) in distinguishing non-isomorphic graph structures.
Revised: Leman, 1968) in distinguishing non-isomorphic graph structures.

Original: This is because a vanilla GNN essentially operates on a subtree rooted at each node in its message passing, i.e., it treats every neighbor of the node equally in its message aggregation.
Revised: This is because a vanilla GNN essentially operates on a subtree rooted at each node in its message passing, i.e., it treats every neighbor of the node equally in its message aggregation.

Original: In this regard, it overlooks any discrepancy that may exist in the connectivities between neighbors.
Revised: In this regard, it overlooks any discrepancy that may exist in the connectivities between neighbors.

Original: To address this limitation, efforts have been devoted to incorporating local substructure information to GNNs.
Revised: To address this limitation, efforts have been devoted to incorporating local substructure information to GNNs.

Original: Several studies add such information to nodes by augmenting node features via subgraph isomorphism counting Zhao et al. (2021), positional encoding (Dwivedi et al., 2021), node identities (Bouritsas et al., 2022), etc.
Revised: Several studies attempt to encode local substructure information, such as induced subgraph (Zhao et al., 2021), overlap subgraph (Wijesinghe & Wang, 2021) and spatial encoding (Bouritsas et al., 2022), to enhance GNNs’ expressiveness.

Revised: But the local structure they choose are not able to capture the complete picture of the 1-hop neighborhood of an edge.

Original: Some others incorporate structural information to edges in message passing via the attention mechanism (Veliˇckovi´c et al.,2017), overlap subgraphs (Wijesinghe & Wang, 2021), etc.
Revised: Some others incorporate shortest path information to edges in message passing via distance encoding (Li et al., 2020), adaptive breath/depth functions (Liu et al., 2019), affinity matrix (Wan et al., 2021), etc., to control the message from different distance of neighbors.

Original: Though these studies are helpful in improving the expressiveness of GNNs, each of them suffers from one or more drawbacks: (1) The local substructures used are either designed in a handcrafted manner or unable to capture thecomplete neighborhood; (2) The substructure descriptor used to encode the substructure may overlook some connectivities between neighbors; and (3) The computational cost of the model becomes high  due to the incorporation of certain substructures.
Revised: However, the substructure descriptor used to encode the substructure may overlook some connectivities between neighbors. Furthermore, some of above models would also suffer from high computational cost due to the incorporation of certain substructures.

Original: In this paper, we aim to develop a model that overcomes the above drawbacks and yet is able to empower GNNs’ expressiveness.
Revised: In this paper, we aim to develop a model that overcomes the above drawbacks and yet is able to empower GNNs’ expressiveness.

Original: (1) We define a new type of substructures named union subgraphs, each capturing the entire closed neighborhood w.r.t.
Revised: (1) We define a new type of substructures named union subgraphs, each capturing the entire closed neighborhood w.r.t.

Original: an edge.
Revised: an edge.

Original: (2) We design an effective substructure descriptor that encodes high-order connectivities.
Revised: (2) We design an effective substructure descriptor that encodes high-order connectivities.

Original: (3) We propose a new model, namely Union
Revised: (3) We propose a new model, namely Union

Original: Subgraph Neural Network (UnionSNN), which is strictly more expressive than GNNs (1-WL) in theory and also computationally efficient in practice.
Revised: Subgraph Neural Network (UnionSNN), which is strictly more expressive than GNNs (1-WL) in theory and also computationally efficient in practice.

Original: Our contributions are summarized as follows:
Revised: Our contributions are summarized as follows:

Original: • We investigate different types of connectivities existing in the local neighborhood and identify the substructure, named “union subgraph”, that is able to capture the complete neighborhood.
Revised: • We investigate different types of connectivities existing in the local neighborhood and identify the substructure, named “union subgraph”, that is able to capture the complete neighborhood.

Original: • We abstract three desired properties for a good substructure descriptor and design a shortest-path-based descriptor that possesses all the properties with high-order connectivities encoded.
Revised: • We abstract three desired properties for a good substructure descriptor and design a shortest-path-based descriptor that possesses all the properties with high-order connectivities encoded.

Original: • We propose a new model, UnionSNN, which injects the information extracted from union subgraphs in message passing.
Revised: • We propose a new model, UnionSNN, which injects the information extracted from union subgraphs in message passing.

Original: We theoretically prove that UnionSNN is more expressive than 1-WL.
Revised: We theoretically prove that UnionSNN is more expressive than 1-WL.

Revised: We also show that UnionSNN at least stronger than 3-WL on some cases.

Original: • We perform extensive experiments on both graph-level and node-levelclassification tasks.
Revised: • We perform extensive experiments on both graph-level and node-level tasks.

Original: The results justify the superiority of UnionSNN over state-of-the-art baselines in terms of both effectiveness and efficiency.
Revised: The results justify the superiority of UnionSNN over state-of-the-art baselines in terms of both effectiveness and efficiency.

Original:  R ELATED W ORK 2. S UBSTRUCTURE - ENHANCED GNN S
Revised:  R ELATED W ORK 2. S UBSTRUCTURE - ENHANCED GNN S

Original: In recent years, several novel GNN architectures have been designed to enhance their expressiveness by encoding local substructures.
Revised: In recent years, several novel GNN architectures have been designed to enhance their expressiveness by encoding local substructures.

Original: GraphSNN (Wijesinghe & Wang, 2021) brings the information of overlap subgraphs into the massage passing scheme as a structural coefficient.
Revised: GraphSNN (Wijesinghe & Wang, 2021) brings the information of overlap subgraphs into the massage passing scheme as a structural coefficient.

Original: However, the overlap subgraph and the substructure descriptor used by GraphSNN are is not powerful enough to distinguish all non-isomorphic substructures in the 1-hop neighborhood.
Revised: However, the overlap subgraph and the substructure descriptor used by GraphSNN are is not powerful enough to distinguish all non-isomorphic substructures in the 1-hop neighborhood.

Original: Zhao et al.
Revised: Zhao et al.

Original: (2021) encode the induced subgraph for each node and inject it into node representations.
Revised: (2021) encode the induced subgraph for each node and inject it into node representations.

Original: Graph Substructure Network (Bouritsas et al., 2022) introduces structural biases in the aggregation function to break the symmetry in message passing.
Revised: Graph Substructure Network (Bouritsas et al., 2022) introduces structural biases in the aggregation function to break the symmetry in message passing.

Original: For these two methods, the neighborhood under consideration should be pre-defined, and the subgraph matching is extremely expensive ( O ( n k ) for k -tuple substructure) when the substructure gets large.
Revised: For these two methods, the neighborhood under consideration should be pre-defined, and the subgraph matching is extremely expensive ( O ( n k ) for k -tuple substructure) when the substructure gets large.

Original: Other transformer-based methods incorporate local structural information via positional encoding.
Revised: Other transformer-based methods incorporate local structural information via positional encoding.

Original: Graphormer (Ying et al., 2021) combines the node degree and shortest path information for spatial encoding.
Revised: Graphormer (Ying et al., 2021) combines the node degree and shortest path information for spatial encoding.

Original: Dwivedi et al.
Revised: Dwivedi et al.

Original: (2021) introduce the random-walk matrix and Graph Laplacian matrix to add the canonical positional information into nodes.
Revised: (2021) introduce the random-walk matrix and Graph Laplacian matrix to add the canonical positional information into nodes.

Original: These positional encodings only consider relative distances from the center node and ignore high-order connectivities between the neighbors.
Revised: These positional encodings only consider relative distances from the center node and ignore high-order connectivities between the neighbors.

Original: 2. P ATH -R ELATED GNN S
Revised: 2. P ATH -R ELATED GNN S

Original: A significant amount of works have focused on the application of shortest paths and other shortestpath-based techniques to GNNs.
Revised: A significant amount of works have focused on the application of shortest paths and other shortestpath-based techniques to GNNs.

Original:  GeniePath (Liu et al., 2019) proposes an adaptive breath function to learn the importance of different-sized neighborhoods and an adaptive depth function to extract and filter signals from neighbors within different distances.
Revised: Li et al. (2020) presents a distance encoding to augment node features and control the receptive field of message passing. GeniePath (Liu et al., 2019) proposes an adaptive breath function to learn the importance of different-sized neighborhoods and an adaptive depth function to extract and filter signals from neighbors within different distances.

Original: PathGNN (Tang et al., 2020) imitates how the Bellman-Ford algorithm solves the shortest path problem in generating weights when updating node features.
Revised: PathGNN (Tang et al., 2020) imitates how the Bellman-Ford algorithm solves the shortest path problem in generating weights when updating node features.

Original: CustomGNN (Chen et al., 2021) samples subgraphs from random walks and uses a sequence model to extract the high-level semantic information in the path re-weighting step.

Original: SPN (Abboud et al., 2022) designs a scheme, in which the representation of a node is propagated to each node in its shortest path neighborhood.
Revised: SPN (Abboud et al., 2022) designs a scheme, in which the representation of a node is propagated to each node in its shortest path neighborhood.

Original: Some recent works adapt the concept of curvature from differential geometry to reflect the connectivity between nodes and the possible bottleneck effects.
Revised: Some recent works adapt the concept of curvature from differential geometry to reflect the connectivity between nodes and the possible bottleneck effects.

Original: CurvGN (Ye et al., 2019) reflects how easily information flows between two nodes by graph curvature information, and exploits curvature to reweigh different channels of messages.
Revised: CurvGN (Ye et al., 2019) reflects how easily information flows between two nodes by graph curvature information, and exploits curvature to reweigh different channels of messages.

Original: Topping et al.
Revised: Topping et al.

Original: (2021) propose Balanced Forman curvature that better reflects the edges having bottleneck effects, and alleviates the over-squashing problem of GNNs by rewiring graphs.
Revised: (2021) propose Balanced Forman curvature that better reflects the edges having bottleneck effects, and alleviates the over-squashing problem of GNNs by rewiring graphs.

Revised: SNALS (Wan et al., 2021) utilizes an affinity matrix based on shortest paths to encode the structural information of hyperedges.

Original: Our method is different from these existing methods by introducing a shortest-path-based substructure descriptor for distinguishing non-isomorphic substructures in the message passing of MPNNs.
Revised: Our method is different from these existing methods by introducing a shortest-path-based substructure descriptor for distinguishing non-isomorphic substructures in the message passing of MPNNs.

Original:  L OCAL S UBSTRUCTURES TO E MPOWER MPNN S
Revised:  L OCAL S UBSTRUCTURES TO E MPOWER MPNN S

Original: In this section, we first introduce MPNNs.
Revised: In this section, we first introduce MPNNs.

Original: We then investigate what kind of local substructures are beneficial to improve the expressiveness of MPNNs.
Revised: We then investigate what kind of local substructures are beneficial to improve the expressiveness of MPNNs.

Original: 3. M ASSAGE P ASSING N EURAL N ETWORKS
Revised: 3. M ASSAGE P ASSING N EURAL N ETWORKS

Original: We represent a graph as G =
Revised: We represent a graph as G =

Original: ( V, E, X ) , where V = { v 1 , ..., v n } is the set of nodes, E ∈ V × V is the set of edges, and X = { x v | v ∈ V } is the set of node features.
Revised: ( V, E, X ) , where V = { v 1 , ..., v n } is the set of nodes, E ∈ V × V is the set of edges, and X = { x v | v ∈ V } is the set of node features.

Original: The set of neighbors of node v is denoted by N ( v ) = { u ∈ V | ( v, u ) ∈ E } .
Revised: The set of neighbors of node v is denoted by N ( v ) = { u ∈ V | ( v, u ) ∈ E } .

Original: As defined in Xu et al.
Revised: As defined in Xu et al.

Original: (2018), the l -th layer of an
Revised: (2018), the l -th layer of an

Original: MPNN can be written as:
Revised: MPNN can be written as:

Original: [Equation]
Revised: [Equation]

Original: where h ( l ) v is the representation of node v at the l -th layer, h (0) v = x v , AGG( · ) and MSG( · ) denote the aggregation and message functions, respectively.
Revised: where h ( l ) v is the representation of node v at the l -th layer, h (0) v = x v , AGG( · ) and MSG( · ) denote the aggregation and message functions, respectively.

Original: 3. L OCAL S UBSTRUCTURES TO I MPROVE MPNN S
Revised: 3. L OCAL S UBSTRUCTURES TO I MPROVE MPNN S

Original: According to Eq.
Revised: According to Eq.

Original: (1), MPNN updates the representation of a node isotropously at each layer and ignores the structural connections between the neighbors of the node.
Revised: (1), MPNN updates the representation of a node isotropously at each layer and ignores the structural connections between the neighbors of the node.

Original: Essentially, the local substructure utilized in the message passing of MPNN is a subtree rooted at the node.
Revised: Essentially, the local substructure utilized in the message passing of MPNN is a subtree rooted at the node.

Original: Consequently, if two non-isomorphic graphs have the same set of rooted subtrees, they could not be distinguished by MPNN (and also 1-WL).
Revised: Consequently, if two non-isomorphic graphs have the same set of rooted subtrees, they could not be distinguished by MPNN (and also 1-WL).

Original: Such an example is shown in Figure 1(a).
Revised: Such an example is shown in Figure 1(a).

Original: A simple fix to this problem is to encode the local structural information about each neighbor, based on which neighbors are treated unequally in the message passing.
Revised: A simple fix to this problem is to encode the local structural information about each neighbor, based on which neighbors are treated unequally in the message passing.

Original: One natural question arises: which substructure best characterizes local information?
Revised: One natural question arises: which substructure shall we choose to characterize the 1-hop local information?

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: To answer the above question, we consider two adjacent nodes v and u , and discuss different types of edges that may exist in their neighbor sets, N ( v ) and N ( u ) .
Revised: To answer the above question, we consider two adjacent nodes v and u , and discuss different types of edges that may exist in their neighbor sets, N ( v ) and N ( u ) .

Original: We define the closed neighbor set of node v as ˜ N ( v ) =
Revised: We define the closed neighbor set of node v as ˜ N ( v ) =

Original: N ( v ) ∪ { v } .
Revised: N ( v ) ∪ { v } .

Original: The induced subgraph of ˜ N ( v ) is denoted by S v , which defines the closed neighborhood of v .
Revised: The induced subgraph of ˜ N ( v ) is denoted by S v , which defines the closed neighborhood of v .

Original: The common closed neighbor set of v and u is N vu =
Revised: The common closed neighbor set of v and u is N vu =

Original: ˜ N ( v ) ∩ ˜ N ( u ) and the exclusive neighbor set of v w.r.t u is defined as N − u v = ˜ N ( v ) − N vu .
Revised: ˜ N ( v ) ∩ ˜ N ( u ) and the exclusive neighbor set of v w.r.t u is defined as N − u v = ˜ N ( v ) − N vu .

Original: As shown in Figure 1(b), there are four types of edges in the closed neighborhood of { v, u } :
Revised: As shown in Figure 1(b), there are four types of edges in the closed neighborhood of { v, u } :

Original: • E vu 1 ∈ N vu ×N vu : edges between the common closed neighbors of v and u , such as ( a, b ) ; • E vu 2 ∈ ( N vu
Revised: • E vu 1 ∈ N vu ×N vu : edges between the common closed neighbors of v and u , such as ( a, b ) ; • E vu 2 ∈ ( N vu

Original: ×N − u v ) ∪ ( N vu ×N − v u ) : edges between a common closed neighbor of v and u and an exclusive neighbor of v / u , such as ( a, d ) ; • E vu∈ N − u v × N − v u : edges between two exclusive neighbors of v and u from different sides, such as ( c, d ) ; • E vu 4 ∈ ( N − u v × N − u v ) ∪ ( N − v u × N − v u ) :
Revised: ×N − u v ) ∪ ( N vu ×N − v u ) : edges between a common closed neighbor of v and u and an exclusive neighbor of v / u , such as ( a, d ) ; • E vu∈ N − u v × N − v u : edges between two exclusive neighbors of v and u from different sides, such as ( c, d ) ; • E vu 4 ∈ ( N − u v × N − u v ) ∪ ( N − v u × N − v u ) :

Original: edges between two exclusive neighbors of v or u from the same side, such as ( d, f ) .
Revised: edges between two exclusive neighbors of v or u from the same side, such as ( d, f ) .

Original: We now discuss three different local substructures, each capturing a different set of edges.
Revised: We now discuss three different local substructures, each capturing a different set of edges.

Original: Overlap subgraph .
Revised: Overlap subgraph.

Revised: (Wijesinghe & Wang, 2021)

Original: The overlap subgraph of two adjacent nodes v and u is defined as S v ∩ u=
Revised: The overlap subgraph of two adjacent nodes v and u is defined as S v ∩ u

Original:  S v ∩ S u .
Revised: = S v ∩ S u .

Original: The overlap subgraph contains only edges in E vu 1 .
Revised: The overlap subgraph contains only edges in E vu 1 .

Original: Union minus subgraph .
Revised: Union minus subgraph.

Original: The union minus subgraph of two adjacent nodes v and uis defined as
Revised: The union minus subgraph of two adjacent nodes v and u

Original: S − v ∪ u = S v ∪ S u .

Original: The union minus subgraph consists of edges in E vu 1 , E vu 2 and E vu 4 .

Original: Union subgraph .
Revised: Union subgraph .

Original: The union subgraph of two adjacent nodes v and u , denoted as S v ∪ u , is defined as the induced subgraph of ˜ N ( v ) ∪ ˜ N ( u ) .
Revised: The union subgraph of two adjacent nodes v and u , denoted as S v ∪ u , is defined as the induced subgraph of ˜ N ( v ) ∪ ˜ N ( u ) .

Original: The union subgraph contains all four types of edges mentioned above.
Revised: The union subgraph contains all four types of edges mentioned above.

Original: It is obvious that the union subgraph captures the whole picture of the neighborhood of two adjacent nodes.
Revised: It is obvious that the union subgraph captures the whole picture of the neighborhood of two adjacent nodes.

Original: It covers all kinds of connectivities in the neighborhood, which serves as a perfect local substructure for improving the expressiveness of MPNNs.
Revised: It covers all kinds of connectivities in the neighborhood, which serves as a perfect local substructure for improving the expressiveness of MPNNs.

Revised: Note that we restrict the discussion to the 1-hop neighborhood because we aim to develop a model based on the MPNN architecture, in which a single layer of convolution is performed on the 1-hop neighbors.

Original: 3. U NION I SOMORPHISM
Revised: 3. U NION I SOMORPHISM

Original: In this subsection, we define the isomorphic relationship between the neighborhoods of two nodes based on the concept of union subgraphs.
Revised: In this subsection, we define the isomorphic relationship between the neighborhoods of two nodes based on the concept of union subgraphs.

Original: The definition follows that of overlap isomorphism in
Revised: The definition follows that of overlap isomorphism in

Original: Wijesinghe & Wang (2021).
Revised: Wijesinghe & Wang (2021).

Original: Overlap Isomorphism .
Revised: Overlap Isomorphism .

Original: S i and S j are overlap-isomorphic, denoted as S i ≃ overlap S j , if there exists a bijective mapping g : ˜ N ( i ) →
Revised: S i and S j are overlap-isomorphic, denoted as S i ≃ overlap S j , if there exists a bijective mapping g : ˜ N ( i ) →

Original: ˜ N ( j ) such that g ( i ) =
Revised: ˜ N ( j ) such that g ( i ) =

Original: j , and for any v ∈ N ( i ) and g ( v ) = u ,
Revised: j , and for any v ∈ N ( i ) and g ( v ) = u ,

Original: S i ∩ v and S j ∩ u are isomorphic.
Revised: S i ∩ v and S j ∩ u are isomorphic (ordinary graph isomorphic).

Original: Union Isomorphism .
Revised: Union Isomorphism .

Original: S i and S j are union-isomorphic, denoted as S i ≃ union S j , if there exists a bijective mapping g : ˜ N ( i ) →
Revised: S i and S j are union-isomorphic, denoted as S i ≃ union S j , if there exists a bijective mapping g : ˜ N ( i ) →

Original: ˜ N ( j ) such that g ( i ) =
Revised: ˜ N ( j ) such that g ( i ) =

Original: j , and for any v ∈ N ( i ) and g ( v ) = u , S
Revised: j , and for any v ∈ N ( i ) and g ( v ) = u , S

Original: i ∪ v and S j ∪ u are isomorphic.
Revised: i ∪ v and S j ∪ u are isomorphic (ordinary graph isomorphic).

Original: u v u v a b c d u v a b c d v      u      a      b      c
Revised: u v u v a b c d u v a b c d v      u      a      b      c

Original: [Figure]
Revised: [Figure]

Original: Theorem 1.
Revised: Theorem 1.

Original: If S i ≃ union S j , then S i ≃ overlap S j ; but not vice versa.
Revised: If S i ≃ union S j , then S i ≃ overlap S j ; but not vice versa.

Original: Theorem 1 states that union-isomorphism is stronger than overlap-isomorphism.
Revised: Theorem 1 states that union-isomorphism is stronger than overlap-isomorphism.

Original: The proofs of all theorems in this paper are provided in Appendix A. Figure 2 shows a pair of non-isomorphic graphs that are distinguishable under union-isomorphism but not overlap-isomorphism or 1-WL (subtree).
Revised: The proofs of all theorems in this paper are provided in Appendix B. Figure 2 shows a pair of non-isomorphic graphs that are distinguishable under union-isomorphism but not overlap-isomorphism or 1-WL (subtree).

Original: The union subgraph of v and one of its arbitrary neighbors is G 1 itself (same for node u ).
Revised: The union subgraph of v and one of its arbitrary neighbors is G 1 itself (same for node u ).

Original: Since Gand G 2 are not isomorphic, they are not union-isomorphic either.
Revised: Since Gand G 2 are not isomorphic, they are not union-isomorphic either.

Original:  U NION SNN
Revised:  U NION SNN

Original: In this section, we first discuss how to design our substructure descriptor so that it well captures the structural information in union subgraphs with several desired properties.
Revised: In this section, we first discuss how to design our substructure descriptor so that it well captures the structural information in union subgraphs with several desired properties.

Original: We then present our model
Revised: We then present our model

Original: UnionSNN, which effectively incorporates the information encoded by the substructure descriptor to
Revised: UnionSNN, which effectively incorporates the information encoded by the substructure descriptor to

Original: MPNNs.
Revised: MPNNs.

Original: Finally, we show that UnionSNN has a stronger expressiveness than 1-WL and is superior to GraphSNN in its design.
Revised: Finally, we show that UnionSNN has a stronger expressiveness than 1-WL and is superior to GraphSNN in its design.

Original: 4. D ESIGN OF SUBSTRUCTURE DESCRIPTOR FUNCTION
Revised: 4. D ESIGN OF SUBSTRUCTURE DESCRIPTOR FUNCTION

Original: Let U =
Revised: Let U =

Original: { S v ∪ u | ( v, u ) ∈ E } be the set of union subgraphs in G .
Revised: { S v ∪ u | ( v, u ) ∈ E } be the set of union subgraphs in G .

Original: In order to fuse the information of union subgraphs in message passing, we need to define a function f ( · ) to describe the structural information of each S v ∪ u ∈ U .
Revised: In order to fuse the information of union subgraphs in message passing, we need to define a function f ( · ) to describe the structural information of each S v ∪ u ∈ U .

Original: Ideally, given two union subgraphs centered at node v , S v ∪ u =
Revised: Ideally, given two union subgraphs centered at node v , S v ∪ u =

Original:  V v ∪ u , E v ∪ u ) 
Revised: ( V v ∪ u , E v ∪ u ) and S v ∪ u ′

Original: and
Revised: =

Original: S v ∪ u ′ = ( V v ∪ u ′ , E v ∪ u ′ ) , we want f ( S v ∪ u ) =
Revised: ( V v ∪ u ′ , E v ∪ u ′ ) , we want f ( S v ∪ u ) =

Original: f ( S v ∪ u ′ )
Revised: f ( S v ∪ u ′ )

Original: iff S v ∪ u and S v ∪ u ′ are isomorphic.
Revised: iff S v ∪ u and S v ∪ u ′ are isomorphic.

Original: We abstract the following properties of a good substructure descriptor function f ( · ) :
Revised: We abstract the following properties of a good substructure descriptor function f ( · ) :

Original: • Size Awareness .
Revised: • Size Awareness .

Original: f ( S v ∪ u ) ̸
Revised: f ( S v ∪ u ) ̸

Original: = f ( S v ∪ u ′ )
Revised: = f ( S v ∪ u ′ )

Original: if | V v ∪ u | ̸ = | V v ∪ u ′ | or | E v ∪ u | ̸ = | E v ∪ u ′ | ; • Connectivity Awareness .
Revised: if | V v ∪ u | ̸ = | V v ∪ u ′ | or | E v ∪ u | ̸ = | E v ∪ u ′ | ; • Connectivity Awareness .

Original: f ( S v ∪ u ) ̸
Revised: f ( S v ∪ u ) ̸

Original: = f ( S v ∪ u ′ )
Revised: = f ( S v ∪ u ′ )

Original: if | V v ∪ u | = | V v ∪ u ′ | and | E v ∪ u | = | E v ∪ u ′ |
Revised: if | V v ∪ u | = | V v ∪ u ′ | and | E v ∪ u | = | E v ∪ u ′ |

Original: but S v ∪ u and S v ∪ u ′ are not isomorphic; • Isomorphic Invariance .
Revised: but S v ∪ u and S v ∪ u ′ are not isomorphic; • Isomorphic Invariance .

Original: f ( S v ∪ u ) =
Revised: f ( S v ∪ u ) =

Original: f ( S v ∪ u ′ )
Revised: f ( S v ∪ u ′ )

Original: if S v ∪ u and S v ∪ u ′ are isomorphic.
Revised: if S v ∪ u and S v ∪ u ′ are isomorphic.

Original: Figure 3 illustrates the properties.
Revised: Figure 3 illustrates the properties.

Original: [Figure]
Revised: [Figure]

Original: Herein, we design f ( · ) as a function that transformsS v ∪ u to a path matrix P vu ∈ R | V v |×| V | such that each 
Revised: Herein, we design f ( · ) as a function that transforms such that each entry:

Original: [Equation]
Revised: [Equation]

Original: where PathLen( · ) denotes the length of the shortest path between i and j in S v ∪ u .
Revised: where PathLen( · ) denotes the length of the shortest path between i and j in S v ∪ u .

Original: We choose the path matrix over the adjacency matrix or the Laplacian matrix as it explicitly encodes high-order connectivities between the neighbors.
Revised: We choose the path matrix over the adjacency matrix or the Laplacian matrix as it explicitly encodes high-order connectivities between the neighbors.

Original: In addition, with a fixed order of nodes, we can get a unique
Revised: In addition, with a fixed order of nodes, we can get a unique

Original: P vu for a given S v ∪ u , and vice versa.
Revised: P vu for a given S v ∪ u , and vice versa.

Original: We formulate it in Theorem 2.
Revised: We formulate it in Theorem 2.

Original: Theorem 2.
Revised: Theorem 2.

Original: With a fixed order of nodes in the path matrix, we can obtain a unique path matrix P vu for a given union subgraph S v ∪ u , and vice versa.
Revised: With a fixed order of nodes in the path matrix, we can obtain a unique path matrix P vu for a given union subgraph S v ∪ u , and vice versa.

Original: It is obvious that our proposed f ( · ) satisfies the above-mentioned three properties, with a node permutation applied in the isomorphic case.
Revised: It is obvious that our proposed f ( · ) satisfies the above-mentioned three properties, with a node permutation applied in the isomorphic case.

Original: Discussion on other substructure descriptor functions .
Revised: Discussion on other substructure descriptor functions .

Original: In literature, some other functions have also been proposed to describe graph substructures.
Revised: In literature, some other functions have also been proposed to describe graph substructures.

Original: (1) Edge Betweenness (Brandes, 2001) is defined by the number of shortest paths between any pair of nodes in a (sub)graph G that pass through an edge.
Revised: (1) Edge Betweenness (Brandes, 2001) is defined by the number of shortest paths between any pair of nodes in a (sub)graph G that pass through an edge.

Original: When applying the edge betweenness to ( v, u ) in S v ∪ u , the metric would remain the same on two different union subgraphs, one with an edge in E vu 4 and one without.
Revised: When applying the edge betweenness to ( v, u ) in S v ∪ u , the metric would remain the same on two different union subgraphs, one with an edge in E vu 4 and one without.

Original: This shows that edge betweenness does not satisfy Size Awareness; (2) Wijesinghe & Wang (2021) put forward a substructure descriptor as a function of the number of nodes and edges.
Revised: This shows that edge betweenness does not satisfy Size Awareness; (2) Wijesinghe & Wang (2021) put forward a substructure descriptor as a function of the number of nodes and edges.

Original: This descriptor fails to distinguish non-isomorphic subgraphs with the same size, and thus does not satisfy Connectivity Awareness; (3) Discrete Graph Curvature, e.g., Olliveier Ricci curvature (Ollivier, 2009; Lin et al., 2011), has been introduced to MPNNs in recent years (Ye et al., 2019).
Revised: This descriptor fails to distinguish non-isomorphic subgraphs with the same size, and thus does not satisfy Connectivity Awareness; (3) Discrete Graph Curvature, e.g., Olliveier Ricci curvature (Ollivier, 2009; Lin et al., 2011), has been introduced to MPNNs in recent years (Ye et al., 2019).

Original: Ricci curvature first computes for each node a probability vector of length | V | that characterizes a uniform propagation distribution in the neighborhood.
Revised: Ricci curvature first computes for each node a probability vector of length | V | that characterizes a uniform propagation distribution in the neighborhood.

Original: It then defines the curvature of two adjacent nodes as the Wasserstein distance of their corresponding probability vectors.
Revised: It then defines the curvature of two adjacent nodes as the Wasserstein distance of their corresponding probability vectors.

Original: Similar to edge betweenness, curvature doesn’t take into account the edges in E vu 4 in its computation and thus does not satisfy Size Awareness either.
Revised: Similar to edge betweenness, curvature doesn’t take into account the edges in E vu 4 in its computation and thus does not satisfy Size Awareness either.

Original: We detail the definitions of these substructure descriptor functions in Appendix B.
Revised: We detail the definitions of these substructure descriptor functions in Appendix C.

Original:  N ETWORK DESIGN
Revised: 4. N ETWORK DESIGN

Original: For the path matrix of an edge ( v, u ) to be used in message passing, we need to further encode it as a scalar.
Revised: For the path matrix of an edge ( v, u ) to be used in message passing, we need to further encode it as a scalar.

Original: We choose to perform Singular Value Decomposition (SVD) (Horn & Johnson, 2012) on the path matrix and extract the singular values:
Revised: We choose to perform Singular Value Decomposition (SVD) (Horn & Johnson, 2012) on the path matrix and extract the singular values:

Original: [Equation]
Revised: [Equation]

Original: The sum of the singular values of P vu , denoted as a vu = sum( Σ vu ) , is used as the local structural coefficient of the edge ( v, u ) ∈ E .
Revised: The sum of the singular values of P vu , denoted as a vu = sum( Σ vu ) , is used as the local structural coefficient of the edge ( v, u ) ∈ E .

Original: Note that since the local structure never changes in message passing, we can compute the structural coefficients in preprocessing before the training starts.
Revised: Note that since the local structure never changes in message passing, we can compute the structural coefficients in preprocessing before the training starts.

Original: A nice property of this structural coefficient is that, it is permutation invariant thanks to the use of
Revised: A nice property of this structural coefficient is that, it is permutation invariant thanks to the use of

Original: SVD and the sum operator.
Revised: SVD and the sum operator.

Original: With an arbitrary order of nodes, the computed a vu remains the same, which removes the condition required by Theorem 2.
Revised: With an arbitrary order of nodes, the computed a vu remains the same, which removes the condition required by Theorem 2.

Original: We now present our model, namely Union Subgraph Neural Network (UnionSNN), which utilizes union-subgraph-based structural coefficients to incorporate local substructures in message passing.
Revised: We now present our model, namely Union Subgraph Neural Network (UnionSNN), which utilizes union-subgraph-based structural coefficients to incorporate local substructures in message passing.

Original: For each vertex v ∈ V , the node representation at the l -th layer is generated by:
Revised: For each vertex v ∈ V , the node representation at the l -th layer is generated by:

Original: [Equation]
Revised: [Equation]

Original: where ϵ ( l − 1) is a learnable scalar parameter and ˜ a vu =
Revised: where ϵ ( l − 1) is a learnable scalar parameter and ˜ a vu =

Original: [Equation]
Revised: [Equation]

Original: MLP [Equation] ) denotes a multilayer perceptron (MLP) with a non-linear function ReLU.
Revised: MLP 1 ( · ) denotes a multilayer perceptron (MLP) with a non-linear function ReLU.

Original: To transform the weight ˜ a vu to align with the multi-channel representation h ( l − 1) u , we follow Ye et al.
Revised: To transform the weight ˜ a vu to align with the multi-channel representation h ( l − 1) u , we follow Ye et al.

Original: and apply a transformation function
Revised: and apply a transformation function

Original: Trans( · ) for better expressiveness and easier training:
Revised: Trans( · ) for better expressiveness and easier training:

Original: [Equation]
Revised: [Equation]

Original: where MLP 2 denotes an MLP with ReLU and a channel-wise softmax function softmax( · ) normalizes the outputs of MLP separately on each channel.
Revised: where MLP 2 denotes an MLP with ReLU and a channel-wise softmax function softmax( · ) normalizes the outputs of MLP separately on each channel.

Original: 4. E XPRESSIVE POWER OF U NION SNN
Revised: 4. E XPRESSIVE POWER OF U NION SNN

Original: We formalize the following theorem to show that UnionSNN is more powerful than 1-WL test in terms of expressive power.
Revised: We formalize the following theorem to show that UnionSNN is more powerful than 1-WL test in terms of expressive power.

Original: Theorem 3.
Revised: Theorem 3.

Original: UnionSNN is more expressive than 1-WL in testing non-isomorphic graphs.
Revised: UnionSNN is more expressive than 1-WL in testing non-isomorphic graphs.

Original: The stronger expressiveness of UnionSNN over 1-WL is credited to its use of union subgraphs, with an effective encoding of local neighborhood connectivities via the shortest-path-based design of structural coefficients.
Revised: The stronger expressiveness of UnionSNN over 1-WL is credited to its use of union subgraphs, with an effective encoding of local neighborhood connectivities via the shortest-path-based design of structural coefficients.

Revised: We further provide a special case to show some graphs can be distinguished by UnionSNN but not by 3-WL or GraphSNN in Appendix D.

Original: [Figure]
Revised: [Figure]

Original: Design Comparisons with GraphSNN .
Revised: Design Comparisons with GraphSNN .

Original: Our UnionSNN is similar to GraphSNN in the sense that both improve the expressiveness of
Revised: Our UnionSNN is similar to GraphSNN in the sense that both improve the expressiveness of

Original: MPNNs (and 1-WL) by injecting the information of local substructures.
Revised: MPNNs (and 1-WL) by injecting the information of local substructures.

Original: However, UnionSNN is superior to GraphSNN in the following aspects.
Revised: However, UnionSNN is superior to GraphSNN in the following aspects.

Original: (1) Union subgraphs in UnionSNN are stronger than overlap subgraphs in GraphSNN, as ensured by Theorem 1.
Revised: (1) Union subgraphs in UnionSNN are stronger than overlap subgraphs in GraphSNN, as ensured by Theorem 1.

Original: (2) The shortest-path-based substructure descriptor designed in UnionSNN is more powerful than that in GraphSNN: the latter fails to possess the property of Connectivity Awareness (as elaborated in Section 4.1).
Revised: (2) The shortest-path-based substructure descriptor designed in UnionSNN is more powerful than that in GraphSNN: the latter fails to possess the property of Connectivity Awareness (as elaborated in Section 4.1).

Original: An example of two non-isomorphic subgraphs S v ∩ u and S v ′ ∩ u ′ is shown in Figure 4.
Revised: An example of two non-isomorphic subgraphs S v ∩ u and S v ′ ∩ u ′ is shown in Figure 4.

Original: They have the same structural coefficients in
Revised: They have the same structural coefficients in

Original: GraphSNN.
Revised: GraphSNN.

Original: The aggregation function in UnionSNN works on adjacent nodes in the input graph, while that in GraphSNN utilizes the structural coefficients on all pairs of nodes (regardless of their adjacency).
Revised: The aggregation function in UnionSNN works on adjacent nodes in the input graph, while that in GraphSNN utilizes the structural coefficients on all pairs of nodes (regardless of their adjacency).

Original: Consequently, GraphSNN requires to pad the adjacency matrix and feature matrix of each graph to the maximum graph size, which significantly increases the computational complexity.
Revised: Consequently, GraphSNN requires to pad the adjacency matrix and feature matrix of each graph to the maximum graph size, which significantly increases the computational complexity.

Original: The advantages of UnionSNN over GraphSNN are also evidenced by experimental results in Section 5.4.
Revised: The advantages of UnionSNN over GraphSNN are also evidenced by experimental results in Section 5.4.

Original:  E XPERIMENTAL S TUDY
Revised:  E XPERIMENTAL S TUDY

Original: In this section, we evaluate the effectiveness of our proposed model under various settings and aim to answer the following research questions: RQ1.
Revised: In this section, we evaluate the effectiveness of our proposed model under various settings and aim to answer the following research questions: RQ1.

Original: Can UnionSNN outperform existing MPNNs?
Revised: Can UnionSNN outperform existing MPNNs?

Original: RQ2.
Revised: RQ2.

Original: Can other GNNs benefit from our structural coefficient?
Revised: Can other GNNs benefit from our structural coefficient?

Original: RQ3.
Revised: RQ3.

Original: How do different components affect the performance of UnionSNN?
Revised: How do different components affect the performance of UnionSNN?

Original: RQ4.
Revised: RQ4.

Original: Is our runtime competitive with other substructure descriptors?
Revised: Is our runtime competitive with other substructure descriptors?

Original: We conduct experiments on two tasks: graph classification and node classification.
Revised: We conduct experiments on three tasks: graph classification, graph regression and node classification.

Original: Datasets.
Revised: Datasets.

Original: For graph classification, we use ten benchmark datasets.
Revised: For graph classification, we use twelve benchmark datasets.

Original: Eight of them were selected from the TUDataset (Kersting et al., 2016), including MUTAG, PROTEINS, ENZYMES, DD,
Revised: Eight of them were selected from the TUDataset (Kersting et al., 2016), including MUTAG, PROTEINS, ENZYMES, DD,

Original: FRANKENSTEIN (denoted as FRANK in our tables), Tox21, NCI1 and NCI109.
Revised: FRANKENSTEIN (denoted as FRANK in our tables), Tox21, NCI1 and NCI109.

Original: The other two datasets OGBG-MOLHIV and OGBG-MOLBBBP were selected from Open Graph Benchmark (Hu et al., 2020).
Revised: The other two datasets OGBG-MOLHIV and OGBG-MOLBBBP were selected from Open Graph Benchmark (Hu et al., 2020).

Revised: For Graph Regression, we conduct experiments on ZINC10k and ZINC-full datasets (Dwivedi et al., 2020).

Original: For node classification, we test on five datasets, including citation networks (Cora, Citeseer, and PubMed (Sen et al., 2008)) and Amazon co-purchase networks (Computer and Photo (McAuley et al., 2015)).
Revised: For node classification, we test on five datasets, including citation networks (Cora, Citeseer, and PubMed (Sen et al., 2008)) and Amazon co-purchase networks (Computer and Photo (McAuley et al., 2015)).

Original: These datasets cover various graph sizes and densities.
Revised: These datasets cover various graph sizes and densities.

Original: The statistics of datasets are summarized in Appendix C.
Revised: The statistics of datasets are summarized in Appendix E.

Original: Baseline Models.
Revised: Baseline Models.

Original: We select various GNN models as baselines, including (1) classical MPNNs such as GCN (Kipf & Welling, 2016), GIN (Xu et al., 2018), GraphSAGE (Hamilton et al., 2017),
Revised: We select various GNN models as baselines, including (1) classical MPNNs such as GCN (Kipf & Welling, 2016), GIN (Xu et al., 2018), GraphSAGE (Hamilton et al., 2017),

Original: GAT (Veliˇckovi´c et al., 2017); (2) WL-based GNNs such as 3WL-GNN (Maron et al., 2019); (3) transformer-based methods such as UGformer (Nguyen et al., 2019); (4) state-of-the-art graph pooling methods such as MEWISPool (Nouranizadeh et al., 2021); (5) methods introducing structural information by shortest paths or curvature, such as GeniePath (Liu et al., 2019) and CurvGN (Ye et al., 2019); (6) GNNs with positional encoding, such as GatedGCN-LSPE (Dwivedi et al., 2021); (7) GraphSNN (Wijesinghe & Wang, 2021).
Revised: GAT (Veliˇckovi´c et al., 2017); (2) WL-based GNNs such as 3WL-GNN (Maron et al., 2019); (3) transformer-based methods such as UGformer (Nguyen et al., 2019); (4) state-of-the-art graph pooling methods such as MEWISPool (Nouranizadeh et al., 2021); (5) methods introducing structural information by shortest paths or curvature, such as GeniePath (Liu et al., 2019), CurvGN (Ye et al., 2019) and NestedGIN (Zhang & Li, 2021); (6) GNNs with positional encoding, such as GatedGCNLSPE (Dwivedi et al., 2021); (7) GraphSNN (Wijesinghe & Wang, 2021).

Original: Model implementation details are provided in Appendix D.
Revised: Model implementation details are provided in Appendix F.

Original: 5.C LASSIFICATION P ERFORMANCE 
Revised: 5. P ERFORMANCE ON D IFFERENT G RAPH T ASKS MUTAG PROTEINS ENZYMES

Original: [Table] [Table] GeniePath

Original: [Table]
Revised: [Table]

Original: Graph Classification .
Revised: Graph Classification .

Original: We report the results on 8 TUDatasets in Table 1 and the results on 2 OGB datasets in Table 10 (Appendix E).
Revised: We report the results on 8 TUDatasets in Table 1 and the results on 2 OGB datasets in Table 9 (Appendix G).

Original: Our UnionSNN outperforms all baselines in 8 out of 10 datasets.
Revised: Our UnionSNN outperforms all baselines in 8 out of 10 datasets.

Original: We further apply our structural coefficient as a plugin component to two MPNNs: UnionGCN for
Revised: We further apply our structural coefficient as a plugin component to two MPNNs: UnionGCN for

Original: GCN and UnionGIN for GIN.
Revised: GCN and UnionGIN for GIN.

Original: The results show that our structural coefficient is able to boost up the performance of the base model in almost all cases, with an improvement of up to 6.15%.
Revised: The results show that our structural coefficient is able to boost up the performance of the base model in almost all cases, with an improvement of up to 6.15%.

Revised: The graph regression results are shown in Table 10 (Appendix G.2).

Original: Node Classification .
Revised: Node Classification .

Original: We report the results of node classification in Table 2.
Revised: We report the results of node classification in Table 2.

Original: Our model achieves the best performance on all 5 datasets.
Revised: Our model achieves the best performance on all 5 datasets.

Original: When injecting our structural coefficient to existing models such as GCN, GIN, and GraphSNN, their performance improves in almost all cases. 
Revised: When injecting our structural coefficient to existing models such as GCN, GIN, and GraphSNN, their performance improves in almost all cases. [Table]

Revised: Average Accuracy ± Standard Deviation) over 10 runs.

Revised: The best result is highlighted in bold .

Revised: The winner between a base model with and without our structural coefficient injected is underlined.

Original: 5. A BLATION S TUDY
Revised: 5. A BLATION S TUDY

Original: In this subsection, we validate empirically the design choices made in different components of our model: (1) the local substructure; (2) the substructure descriptor; (3) the encoding method from a path matrix to a scalar.
Revised: In this subsection, we validate empirically the design choices made in different components of our model: (1) the local substructure; (2) the substructure descriptor; (3) the encoding method from a path matrix to a scalar.

Original: All experiments are conducted on 6 graph classification datasets.
Revised: All experiments are conducted on 6 graph classification datasets.

Original: Local substructure .
Revised: Local substructure .

Original: We test three types of local substructures defined in Section 3.2: overlap subgraphs, union minus subgraphs and union subgraphs.
Revised: We test three types of local substructures defined in Section 3.2: overlap subgraphs, union minus subgraphs and union subgraphs.

Original: They are denoted as “overlap”, “minus”, and “union” respectively in Table 3.
Revised: They are denoted as “overlap”, “minus”, and “union” respectively in Table 3.

Original: We can see that the best results are achieved by using unionrelated substructures: 4 on union subgraphs and 2 on union minus subgraphs.
Revised: We can see that the best results are achieved by using unionrelated substructures: 4 on union subgraphs and 2 on union minus subgraphs.

Original: [Table]
Revised: [Table]

Original: Substructure descriptor .
Revised: Substructure descriptor .

Original: We compare our substructure descriptor with four existing ones discussed in Section 4.1.
Revised: We compare our substructure descriptor with four existing ones discussed in Section 4.1.

Original: We replace the substructure descriptor in UnionSNN with edge betweenness, node/edge counting, Ricci curvature [Table] and Laplacian matrix (other components unchanged), and obtain four variants, namely BetSNN, CountSNN, CurvSNN [Table]
Revised: We replace the substructure descriptor in UnionSNN with edge betweenness, node/edge counting, Ricci curvature, and Laplacian matrix (other components unchanged), and obtain four variants, namely BetSNN, CountSNN, CurvSNN, and LapSNN.

Original: and LapSNN.

Original: As shown in Table 4, our
Revised: As shown in Table 4, our

Original: UnionSNN is a clear winner: it achieves the best result on 5 out of 6 datasets.
Revised: UnionSNN is a clear winner: it achieves the best result on 5 out of 6 datasets.

Revised: More experiments based on GCN and GIN are provided in Appendix G.3.

Original: This demonstrates that the path matrix better captures substructure information
Revised: These experiments demonstrate that the path matrix better captures substructure information.

Original: [Table]
Revised: [Table]

Original: Path matrix encoding method .
Revised: Path matrix encoding method .

Original: We test three methods that transform a path matrix to a scalar: (1) sum of all elements in the path matrix (matrix sum); (2) maximum eigenvalue of the path matrix (eigen max); (3) sum of all singular values of the matrix (svd sum) used by UnionSNN in Section 4.2.
Revised: We test three methods that transform a path matrix to a scalar: (1) sum of all elements in the path matrix (matrix sum); (2) maximum eigenvalue of the path matrix (eigen max); (3) sum of all singular values of the matrix (svd sum) used by UnionSNN in Section 4.2.

Original: Table 5 shows that the encoding method “svd sum” performs the best on 5 out of 6 datasets.[Table]
Revised: Table 5 shows that the encoding method “svd sum” performs the best on 5 out of 6 datasets.

Original: 5. C ASE S TUDY
Revised: 5. C ASE S TUDY

Original: In this subsection, we investigate how the proposed structural coefficient a vu reflects local connectivities. We work on an example union subgraph S v ∪ u in Figure 5 and modify its nodes/edges to study how the coefficient a vu varies with the local structural change. u v a b d c f e
Revised: [Table]

Original: Modify edge ! !" -ab 23. +bc 22. -ab+bc 23.
Revised: [Table]

Original: -ad 23. -de 23. +df 21. -vc 24.
Revised: e

Original: -uc 24. -vb 24. Modify node ! !"
Revised: [Table]

Original: -a 20. -b 20. -c 19. -d 19.
Revised: In this subsection, we investigate how the proposed structural coefficient a vu reflects local connectivities.

Original: -e 18. -f 17. Original a !" 23.
Revised: We work on an example union subgraph S v ∪ u in Figure 5 and modify its nodes/edges to study how the coefficient a vu varies with the local structural change.

Original: In this case study, we have the following observations: (1) with the set of nodes unchanged, deleting an edge increases a vu ; (2) deleting a node (and its incident edges) decreases a vu ; (3) the four types of edges in the closed neighborhood (Section 3.2) have different effects to a vu : E vu 1 <E vu<
Revised: We have the following observations: (1) with the set of nodes unchanged, deleting an edge increases a vu ; (2) deleting a node (and its incident edges) decreases a vu ; (3) the four types of edges in the closed neighborhood (Section 3.2) have different effects to a vu : E vu 1 <E vu<

Original: E vu<E vu(by comparing -ab, -ad, -de, and +df).
Revised: E vu<E vu(by comparing -ab, -ad, -de, and +df).

Original: These observations indicate that a smaller coefficient will be assigned to an edge with a denser local substructure.
Revised: These observations indicate that a smaller coefficient will be assigned to an edge with a denser local substructure.

Original: This matches with our expectation that the coefficient should be small for an edge in a highly connected neighborhood.
Revised: This matches with our expectation that the coefficient should be small for an edge in a highly connected neighborhood.

Original: The rationale is, such edges are less important in message passing as the information between their two incident nodes can flow through more paths.
Revised: The rationale is, such edges are less important in message passing as the information between their two incident nodes can flow through more paths.

Original: By using the coefficients that well capture local connectivities, the massages from different neighbors could be properly adjusted when passing to the center node.
Revised: By using the coefficients that well capture local connectivities, the massages from different neighbors could be properly adjusted when passing to the center node.

Original: This also explains the effectiveness of UnionSNN in the performance experiments.
Revised: This also explains the effectiveness of UnionSNN in the performance experiments.

Original: 5. E FFICIENCY A NALYSIS
Revised: 5. E FFICIENCY A NALYSIS

Original: In this subsection, we conduct experiments on PROTEINS, DD and FRANKENSTEIN datasets, which cover various number of graphs and graph sizes.PROTEINS DD FRANK Betweenness [Table]
Revised: In this subsection, we conduct experiments on PROTEINS, DD and FRANKENSTEIN datasets, which cover various number of graphs and graph sizes.

Original: [Table]

Original: : Time cost (seconds) for computing structural coefficients in preprocessing.

Original: Preprocessing computational cost .

Original: UnionSNN computes structural coefficients in preprocessing.

Original: We compare its preprocessing time with the time needed in baseline models for pre-computing their substructure descriptors, including edge betweenness (Betweenness) in BetSNN, node/edge counting (Count ne) in GraphSNN, and Ricci curvature (Curvature) in

Original: CurvGN.

Original: As shown in Table 6, the preprocessing time of UnionSNN is comparable to that of other models.

Original: This demonstrates that our proposed structural coefficient derived by SVD on the path matrix is able to improve classification performance without significantly sacrificing the preprocessing cost.

Original: [Table]
Revised: [Table]

Original: Runtime computational cost .
Revised: Runtime computational cost .

Original: We conduct an experiment to compare the total runtime cost of UnionSNN with those in other MPNNs.
Revised: We conduct an experiment to compare the total runtime cost of UnionSNN with those in other MPNNs.

Original: The results are reported in Table 7.
Revised: The results are reported in Table 6.

Original: Although UnionSNN runs slightly slower than GCN and GIN, it runs over 4.56 times faster than WL-based MPNN (3WL-GNN) and is comparable to MPNN with positional encoding (GatedGCN-LSPE).
Revised: Although UnionSNN runs slightly slower than GCN and GIN, it runs over 4.56 times faster than WL-based MPNN (3WL-GNN) and is comparable to MPNN with positional encoding (GatedGCN-LSPE).

Original: Compared with 
Revised: Compared with GraphSNN,

Original: GraphSNN, UnionSNN runs significantly faster: the efficiency improvement approaches an order of magnitude on datasets with large graphs, e.g., DD.
Revised: UnionSNN runs significantly faster: the efficiency improvement approaches an order of magnitude on datasets with large graphs, e.g., DD.

Original: This is because UnionSNN does not need to pad the adjacency matrix and the feature matrix of each graph to the maximum graph size in the dataset, as what GraphSNN does.
Revised: This is because UnionSNN does not need to pad the adjacency matrix and the feature matrix of each graph to the maximum graph size in the dataset, as what GraphSNN does.

Revised: The analysis of preprocessing computational cost are provided in Appendix G.4.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper, we propose UnionSNN, a model that is strictly more powerful than 1-WL in distinguishing non-isomorphic graphs.
Revised: In this paper, we propose UnionSNN, a model that is strictly more powerful than 1-WL in distinguishing non-isomorphic graphs.

Original: UnionSNN is empowered by an effective shortest-path-based substructure descriptor applied to union subgraphs.
Revised: UnionSNN is empowered by an effective shortest-path-based substructure descriptor applied to union subgraphs.

Original: Our experimental results show that UnionSNN outperforms state-of-the-art baselines on both graph-level and node-level classification tasks, without sacrificing its computational efficiency.
Revised: Our experimental results show that UnionSNN outperforms state-of-the-art baselines on both graph-level and node-level classification tasks, without sacrificing its computational efficiency.

Original: The strength of union subgraphs in capturing neighbor connectivities and benefiting message passing is demonstrated.
Revised: The strength of union subgraphs in capturing neighbor connectivities and benefiting message passing is demonstrated.

Original: When applying union subgraphs to existing models, their performance is improved by up to 6.15%.
Revised: When applying union subgraphs to existing models, their performance is improved by up to 6.15%.

Original: A P ROOFS OF T HEOREMS A. P ROOF OF T HEOREM
Revised: A A N E XAMPLE FOR O UR I SOMORPHISM C ONCEPTS

Revised: [Figure]

Revised: We provide an example for our subgraph and isomorphism concepts and how they are related.

Revised: As shown in Figure 6, we have two subgraphs S v 1 and S v 2 .

Revised: These two subgraphs are overlapisomorphic.

Revised: The bijective mapping g is: g ( k 1 ) =

Revised: g ( k 2 ) , ∀ k ∈ { v, a, b, c, d, e, f, g, h } .

Revised: Take a pair of corresponding overlap subgraphs as an example: S v 1 ∩ g 1 and S v 2 ∩ g 2 .

Revised: They are based on two red edges ( v 1 , g 1 ) , and ( v 2 , g 2 ) and the rest of edges in the overlap subgraphs are colored in blue.

Revised: It is easy to see that S v 1 ∩ g 1 and S v 2 ∩ g 2 are isomorphic (ordinary one).

Revised: In this ordinary graph isomorphism, its bijective mapping between the nodes is not required to be the same as g , just as in the case of S v 1 ∩ g 1 and S v 2 ∩ g 2 .

Revised: It could be the same or different, as long as the ordinary graph isomorphism holds between this pair of overlap subgraphs.

Revised: In this example, any pair of corresponding overlap subgraphs defined under g are isomorphic (ordinary one).

Revised: In fact, all overlap subgraphs have the same structure in this example.

Revised: In this sense, the concept of “overlap isomorphism” does not look at the neighborhood based on a single edge, but captures the neighborhoods of all edges and thus the “overlap” connectivities of the two subgraphs S v 1 and S v 2 .

Revised: B P ROOFS OF T HEOREMS B. P ROOF OF T HEOREM

Original: Theorem 1.
Revised: Theorem 1.

Original: If S i ≃ union S j , then S i ≃ overlap S j ; but not vice versa.
Revised: If S i ≃ union S j , then S i ≃ overlap S j ; but not vice versa.

Original: Proof.
Revised: Proof.

Original: By the definition of union isomorphism, if S i ≃ union S j , then there exists a bijective mapping g :
Revised: By the definition of union isomorphism, if S i ≃ union S j , then there exists a bijective mapping g :

Original: ˜ N ( i ) →
Revised: ˜ N ( i ) →

Original: ˜ N ( j ) such that g ( i ) =
Revised: ˜ N ( j ) such that g ( i ) =

Original: j and for any v ∈ N ( i ) and g ( v ) = u , S
Revised: j and for any v ∈ N ( i ) and g ( v ) = u , S

Original: i ∪ v and S j ∪ u are isomorphic.
Revised: i ∪ v and S j ∪ u are isomorphic.

Original: Since g ( i ) =
Revised: Since g ( i ) =

Original: j and g ( v ) = u , according to the definition of graph isomorphism, v 1 ∈ ˜ N
Revised: j and g ( v ) = u , according to the definition of graph isomorphism, v 1 ∈ ˜ N

Original: ( i ) ∩ ˜ N ( v )
Revised: ( i ) ∩ ˜ N ( v )

Original: if and only if g ( v 1 ) ∈ ˜ N ( j ) ∩ ˜ N ( u ) .
Revised: if and only if g ( v 1 ) ∈ ˜ N ( j ) ∩ ˜ N ( u ) .

Original: Therefore, g ( · ) is a bijective mapping from ˜ N ( i ) ∩ ˜ N ( v ) to ˜ N ( j ) ∩ ˜ N ( u ) , and v 1 , v 2 ∈ ˜ N
Revised: Therefore, g ( · ) is a bijective mapping from ˜ N ( i ) ∩ ˜ N ( v ) to ˜ N ( j ) ∩ ˜ N ( u ) , and v 1 , v 2 ∈ ˜ N

Original: ( i ) ∩ ˜ N ( v ) are adjacent if and only if g ( v 1 ) , g ( v 2 ) ∈
Revised: ( i ) ∩ ˜ N ( v ) are adjacent if and only if g ( v 1 ) , g ( v 2 ) ∈

Original: ˜ N ( j ) ∩ ˜ N ( u ) are adjacent.
Revised: ˜ N ( j ) ∩ ˜ N ( u ) are adjacent.

Original: This proves that S i ∩ v and S j ∩ u are isomorphic, and thus
Revised: This proves that S i ∩ v and S j ∩ u are isomorphic, and thus

Original: S i ≃ overlap S j .
Revised: S i ≃ overlap S j .

Original: On the contrary, it is possible that S i ≃ overlap S j but S
Revised: On the contrary, it is possible that S i ≃ overlap S j but S

Original: i ̸≃ union S j , as shown by graphs G 1 and G 2 in Figure 2.
Revised: i ̸≃ union S j , as shown by graphs G 1 and G 2 in Figure 2.

Original: □
Revised: □

Original: A. P ROOF OF T HEOREM
Revised: B. P ROOF OF T HEOREM

Original: Theorem 2.
Revised: Theorem 2.

Original: With a fixed order of nodes in the path matrix, we can obtain a unique path matrix P vu for a given union subgraph S v ∪ u , and vice versa.
Revised: With a fixed order of nodes in the path matrix, we can obtain a unique path matrix P vu for a given union subgraph S v ∪ u , and vice versa.

Original: Proof.
Revised: Proof.

Original: We prove the theorem in two steps.
Revised: We prove the theorem in two steps.

Original: Step 1: We prove that we get a unique P vu from a given S v ∪ u .
Revised: Step 1: We prove that we get a unique P vu from a given S v ∪ u .

Original: With the node order fixed, the rows and the columns of P vu are fixed.
Revised: With the node order fixed, the rows and the columns of P vu are fixed.

Original: Since P vu stores the lengths of all-pair shortest paths,the matrix is unique given an input union subgraph.
Revised: Since P vu stores the lengths of all-pair shortest paths,

Original: Step 2: We prove that we can recover a unique S v ∪ u from a given P vu .
Revised: Step 2: We prove that we can recover a unique S v ∪ u from a given P vu .

Original: The node set of S v ∪ u can be recovered from the row (or column) indices of P vu .
Revised: The node set of S v ∪ u can be recovered from the row (or column) indices of P vu .

Original: The edge set of S v ∪ u can be recovered from the entries in P vu with the value of ”1”.
Revised: The edge set of S v ∪ u can be recovered from the entries in P vu with the value of ”1”.

Original: Both the node set and the edge set can be uniquely constructed from P vu , and thus S v ∪ u is unique.
Revised: Both the node set and the edge set can be uniquely constructed from P vu , and thus S v ∪ u is unique.

Original: □
Revised: □

Original: A. P ROOF OF T HEOREM
Revised: B. P ROOF OF T HEOREM

Original: The proof of Theorem 3 follows a similar flow to that of Theorem 4 in (Wijesinghe & Wang, 2021).
Revised: The proof of Theorem 3 follows a similar flow to that of Theorem 4 in (Wijesinghe & Wang, 2021).

Original: We first define the concept of subtree isomorphism.
Revised: We first define the concept of subtree isomorphism.

Original: Let h v denote the node feature of a node v ∈ V .
Revised: Let h v denote the node feature of a node v ∈ V .

Original: Subtree Isomorphism.
Revised: Subtree Isomorphism.

Original: S i and S j are subtree-isomorphic, denoted as S i ≃ subtree S j , if there exists a bijective mapping g : ˜ N ( i ) →
Revised: S i and S j are subtree-isomorphic, denoted as S i ≃ subtree S j , if there exists a bijective mapping g : ˜ N ( i ) →

Original: ˜ N ( j ) such that g ( i ) =
Revised: ˜ N ( j ) such that g ( i ) =

Original: j , and for any v ∈ N ( i ) and g ( v ) = u , h v = h u .
Revised: j , and for any v ∈ N ( i ) and g ( v ) = u , h v = h u .

Original: We assume that H , A and W are three countable sets that H is the node feature space, A is the structural coefficient space, and W = { a ij h i | a ij ∈ A , h i ∈ H} .
Revised: We assume that H , A and W are three countable sets that H is the node feature space, A is the structural coefficient space, and W = { a ij h i | a ij ∈ A , h i ∈ H} .

Original: Suppose H and W are two multisets that contain elements from H and W , respectively, and | H | = | W | .
Revised: Suppose H and W are two multisets that contain elements from H and W , respectively, and | H | = | W | .

Original: In order to prove
Revised: In order to prove

Original: Theorem 3, we need to use Lemmas 1 and 2 in Wijesinghe & Wang (2021).
Revised: Theorem 3, we need to use Lemmas 1 and 2 in Wijesinghe & Wang (2021).

Original: To be self-contained, we repeat the lemmas here and refer the readers to Appendix C of Wijesinghe & Wang (2021) for the proofs.
Revised: To be self-contained, we repeat the lemmas here and refer the readers to Appendix C of Wijesinghe & Wang (2021) for the proofs.

Original: Lemma 1.
Revised: Lemma 1.

Original: There exists a function f s.t.
Revised: There exists a function f s.t.

Original: π ( H, W ) =
Revised: π ( H, W ) =

Original: (cid:80) h ∈ H,w ∈ W f ( h, w ) is unique for any distinct pair of multisets ( H, W ) .
Revised: (cid:80) h ∈ H,w ∈ W f ( h, w ) is unique for any distinct pair of multisets ( H, W ) .

Original: Lemma 2.
Revised: Lemma 2.

Original: There exists a function f s.t.
Revised: There exists a function f s.t.

Original: π ′ ( h v , H, W ) =
Revised: π ′ ( h v , H, W ) =

Original: γf ( h v , | H | h v ) + (cid:80) h ∈ H,w ∈ W f ( h, w ) is unique for any distinct ( h v , H, W ) , where h v ∈ H, | H | h v ∈ W , and γ can be an irrational number.
Revised: γf ( h v , | H | h v ) + (cid:80) h ∈ H,w ∈ W f ( h, w ) is unique for any distinct ( h v , H, W ) , where h v ∈ H, | H | h v ∈ W , and γ can be an irrational number.

Original: From the lemmas above, we can now prove Theorem 3:
Revised: From the lemmas above, we can now prove Theorem 3:

Original: Theorem 3.
Revised: Theorem 3.

Original: UnionSNN is more expressive than 1-WL in testing non-isomorphic graphs.
Revised: UnionSNN is more expressive than 1-WL in testing non-isomorphic graphs.

Original: Proof.
Revised: Proof.

Original: By Theorem 3 in (Wijesinghe & Wang, 2021), if a GNN M satisfies the two following conditions, then M is strictly more powerful than 1-WL in distinguishing non-isomorphic graphs.
Revised: By Theorem 3 in (Wijesinghe & Wang, 2021), if a GNN M satisfies the two following conditions, then M is strictly more powerful than 1-WL in distinguishing non-isomorphic graphs.

Original: M can distinguish at least one pair of neighborhood subgraphs S i and S j such that S i and S j are subtree-isomorphic, but they are not isomorphic, and {{ ˜ a iv | v ∈ N ( i ) }} ̸
Revised: M can distinguish at least one pair of neighborhood subgraphs S i and S j such that S i and S j are subtree-isomorphic, but they are not isomorphic, and {{ ˜ a iv | v ∈ N ( i ) }} ̸

Original: = {{ ˜ a ju | u ∈ N ( j ) }} , where ˜ a vu is the normalised value of a vu ; 2.
Revised: = {{ ˜ a ju | u ∈ N ( j ) }} , where ˜ a vu is the normalised value of a vu ; 2.

Original: The aggregation scheme Φ( h ( t ) v , {{ h ( t ) u | u ∈ N ( v ) }} , {{ (˜ a uv , h ( t ) u ) | u ∈ N ( v ) }} ) is injective.
Revised: The aggregation scheme Φ( h ( t ) v , {{ h ( t ) u | u ∈ N ( v ) }} , {{ (˜ a uv , h ( t ) u ) | u ∈ N ( v ) }} ) is injective.

Original: For condition 1, the pair of graphs in Figure 2 satisfies the condition, and can be distinguished by
Revised: For condition 1, the pair of graphs in Figure 2 satisfies the condition, and can be distinguished by

Original: UnionSNN as they are not union-isomorphic.
Revised: UnionSNN as they are not union-isomorphic.

Original: For condition 2, by Lemmas 1 and 2 and the fact that the MLP Trans( · ) is a universal approximator and can model and learn the required functions, we conclude that UnionSNN satisfies this condition.
Revised: For condition 2, by Lemmas 1 and 2 and the fact that the MLP Trans( · ) is a universal approximator and can model and learn the required functions, we conclude that UnionSNN satisfies this condition.

Original: Therefore, UnionSNN is more expressive than 1-WL in testing non-isomorphic graphs.
Revised: Therefore, UnionSNN is more expressive than 1-WL in testing non-isomorphic graphs.

Original: □
Revised: □

Original: B D EFINITIONS OF THREE OTHER SUBSTRUCTURE DESCRIPTOR FUNCTIONS
Revised: C D EFINITIONS OF THREE OTHER SUBSTRUCTURE DESCRIPTOR FUNCTIONS

Original: Edge Betweenness.
Revised: Edge Betweenness.

Original: The betweenness centrality of an edge e ∈ E in a graph G =
Revised: The betweenness centrality of an edge e ∈ E in a graph G =

Original: ( V, E, X ) is defined as the sum of the fraction of all-pair shortest paths that pass through e :
Revised: ( V, E, X ) is defined as the sum of the fraction of all-pair shortest paths that pass through e :

Original: [Equation]
Revised: [Equation]

Original: where σ ( v, u, G ) is the number of shortest paths between v and u , and σ ( v, u, G | e ) is the number of those paths passing through edge e .
Revised: where σ ( v, u, G ) is the number of shortest paths between v and u , and σ ( v, u, G | e ) is the number of those paths passing through edge e .

Original: Node/Edge Counting.
Revised: Node/Edge Counting.

Original: GraphSNN (Wijesinghe & Wang, 2021) defines a structural coefficient based on the number of nodes and edges in the (sub)graph.
Revised: GraphSNN (Wijesinghe & Wang, 2021) defines a structural coefficient based on the number of nodes and edges in the (sub)graph.

Original: When applied to the union subgraph, we have
Revised: When applied to the union subgraph, we have

Original: [Equation]
Revised: [Equation]

Original: where λ = 1 for node classification and λ = 2 for graph classification.
Revised: where λ = 1 for node classification and λ = 2 for graph classification.

Original: Ollivier Ricci Curvature Given a node v ∈ V in a graph G =
Revised: Ollivier Ricci Curvature. Given a node v ∈ V in a graph G =

Original: ( V, E, X ) , a probability vector of v is defined as:
Revised: ( V, E, X ) , a probability vector of v is defined as:

Original: [Equation]
Revised: [Equation]

Original: where α ∈ [0 , 1) is a hyperparameter.
Revised: where α ∈ [0 , 1) is a hyperparameter.

Original: Following Ye et al.
Revised: Following Ye et al.

Original: (2019), we use α = 0 .
Revised: (2019), we use α = 0 .

Original: 5 in all our experiments.
Revised: 5 in all our experiments.

Original: The α -Ricci curvature κ αvu on an edge ( v
Revised: The α -Ricci curvature κ αvu on an edge ( v

Original: , u ) ∈ E is defined by:
Revised: , u ) ∈ E is defined by:

Original: [Equation]
Revised: [Equation]

Original: where Wass( · ) denotes the Wasserstein distance and d ( · ) denotes the shortest path length between v and u .
Revised: where Wass( · ) denotes the Wasserstein distance and d ( · ) denotes the shortest path length between v and u .

Original: The Wasserstein distance can be estimated by the optimal transportation distance, which can be solved by the following linear programming:
Revised: The Wasserstein distance can be estimated by the optimal transportation distance, which can be solved by the following linear programming:

Original: [Equation]
Revised: [Equation]

Original: C D ATASETS S TATISTICS
Revised: D T HE C ONNECTION WITH H IGHER - ORDER WL T ESTS

Revised: As discussed in Papp & Wattenhofer (2022), high-order WL tests are concepts of global comparisons over two graphs. Therefore, it is arguable if the WL hierarchy is a suitable tool to measure the expressiveness of GNN extensions as the latter focus on locality. Nonetheless, there exist some graph structures such that the 3-WL test and GraphSNN are not stronger than UnionSNN, i.e., some graphs can be distinguished by UnionSNN but not by 3-WL and GraphSNN.

Revised: Similar analyses have been performed in existing works, such as Balcilar et al.

Revised: and Bodnar et al.

Revised: As an example, UnionSNN can distinguish the 4x4 Rook’s graph and the Shrikhande graph (as shown in Figure 7, which is cited from Huang et al.

Revised: (2022)), while 3-WL and GraphSNN cannot, which suggests that UnionSNN is stronger than 3-WL and GraphSNN on such graphs.

Revised: This could be explained by the fact that the number of 4-cycles in these two graphs are different, and UnionSNN is able to reflect the number of 4-cycles with the consideration of E vu 3 edges in union subgraphs.

Revised: E D ATASETS S TATISTICS

Original: Statistics of the datasets used are summarized in Tables 8 and 9.
Revised: Statistics of the datasets used are summarized in Tables 7 and 8.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: D I MPLEMENTATION D ETAILS
Revised: F I MPLEMENTATION D ETAILS

Original: The settings of our experiments mainly follow those in Dwivedi et al.
Revised: The settings of our experiments mainly follow those in Dwivedi et al.

Original: For graph classification datasets, we split each one into 8:1:1 for training, validation and test, respectively.
Revised: For graph classification on TUDataset, we split each one into 8:1:1 for training, validation and test, respectively.

Original: We evaluate each model with the same random seed for 10-fold cross-validation and report the average accuracy for
Revised: We evaluate each model with the same random seed for 10-fold cross-validation and report the average accuracy.

Original: TUDataset.

Original: As for  OGB datasets, we use the scaffold splits run 10 times with different random seeds and report the average ROC-AUC due to the class imbalance issue.
Revised: As for graph classification on OGB datasets, we use the scaffold splits run 10 times with different random seeds and report the average ROC-AUC due to the class imbalance issue.

Revised: For graph regression, we use the data split of ZINC10k and ZINC-full datasets from Dwivedi et al.

Revised: We keep the total parameters of all GNNs less than 100K and 500K on ZINC10k and ZINC-full datasets, respectively, by adjust the hidden dimensions.

Original: For node classification, we use the standard split in the original paper and report the average accuracy for 10 runs with different random seeds.
Revised: For node classification, we use the standard split in the original paper and report the average accuracy for 10 runs with different random seeds.

Original: We choose the best values of the learning rate from { 0.02, 0.01, 0.005, 0.001 } , weight decay from { 0.005, 0.001, 0.0005 } , hidden dimension from { 16, 64, 128, 256 } and dropout from { 0, 0.2, 0.4, 0.6, 0.8 } .
Revised: We choose the best values of the learning rate from { 0.02, 0.01, 0.005, 0.001 } , weight decay from { 0.005, 0.001, 0.0005 } , hidden dimension from { 16, 64, 128, 256 } and dropout from { 0, 0.2, 0.4, 0.6, 0.8 } .

Original: The whole network is trained in an end-to-end manner using the Adam optimizer (Kingma & Ba, 2014).
Revised: The whole network is trained in an end-to-end manner using the Adam optimizer (Kingma & Ba, 2014).

Original: We use the early stopping criterion, i.e., we stop the training once there is no further improvement on the validation loss during 25 epochs.
Revised: We use the early stopping criterion, i.e., we stop the training once there is no further improvement on the validation loss during 25 epochs.

Original: All experiments were conducted in a Linux server with Intel(R) Core(TM) i9-10940X CPU (3.30GHz), GeForce GTX 3090 GPU, and 125GB RAM.
Revised: All experiments were conducted in a Linux server with Intel(R) Core(TM) i9-10940X CPU (3.30GHz), GeForce GTX 3090 GPU, and 125GB RAM.

Original: E M ORE EXPERIMENTAL RESULTS 
Revised: G M ORE EXPERIMENTAL RESULTS G. G RAPH C LASSIFICATION ON OGB DATASETS .

Revised: We conduct experiments on OGBG-MOLHIV and OGBG-MOLBBBP datasets. As shown in Table 9, UnionSNN outperforms all 6 baseline methods. [Table]

Revised: G. G RAPH R EGRESSION

Revised: We conduct experiments on ZINC10k and ZINC-full datasets

Revised: [Table]

Revised: as shown in Table 10, UnionSNN outperforms all 7 baseline methods

Revised: [Table]

Revised: and the performance of MPNNs with our structural coefficient (UnionGCN, UnionGIN and UnionGraphSAGE) are also beyond their counterparts.

Revised: We are not reporting the result of CurvGN on ZINC-full since the preprocessing of computes Ricci Curvature would take more than 100 hours.

Revised: [Table]

Revised: The best result is highlighted in bold .

Revised: The winner between a base model with and without our structural coefficient injected is underlined .

Revised: G. S UBSTRUCTURE D ESCRIPTOR A BLATION S TUDY ON O THER MPNN VARIANTS

Revised: In this subsection

Revised: [Table]

Revised: we tried to plugin different substructure descriptors into GCN and GIN, and the results are reported in Table 11 and Table 12.

Revised: The best results are highlighted in bold and the second-best results are highlighted in italic.

Revised: The experiments show that the path matrix (UnionGCN and UnionGIN) can always get the top 2 results comparing to other substructure descriptors.

Revised: This demonstrates the effectiveness of the substructure descriptor in UnionSNN.

Original: [Table]
Revised: [Table]

Revised: G. E FFICIENCY A NALYSIS FOR P REPROCESSING

Revised: [Table]

Revised: [Table]

Revised: Preprocessing computational cost .

Revised: UnionSNN computes structural coefficients in preprocessing.

Revised: We compare its preprocessing time with the time needed in baseline models for pre-computing their substructure descriptors, including edge betweenness (Betweenness) in BetSNN, node/edge counting (Count ne) in GraphSNN, and Ricci curvature (Curvature) in

Revised: CurvGN.

Revised: As shown in Table 13, the preprocessing time of UnionSNN is comparable to that of other models.

Revised: This demonstrates that our proposed structural coefficient derived by SVD on the path matrix is able to improve classification performance without significantly sacrificing the preprocessing cost.