From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Looking forward to further feedback
comment: Dear reviewers,

Thanks for the time and effort of all reviewers on our work. We have carefully addressed all the comments. Do let us know if there is any other concern. If all concerns are addressed satisfactorily, we would appreciate it if reviewers could kindly consider raising the rating. 

Thank you very much!

Best,

The authors

From: ICLR.cc/2023/Conference/Paper1386/Reviewer_S6Hp
title: Thanks 
comment: Thank you so much for the response. I think all the added experiments and discussions are useful to improve the paper. I greatly appreciate them. I still lean to accept this work though I can hardly give a strong recommendation due to the limitation in the technical contribution of this work.

From: ICLR.cc/2023/Conference/Paper1386/Area_Chair_jkuK
title: Please read and respond to the rebuttal 
comment: Dear reviewers,

The authors have responded to your reviews. Since they cannot update the submission after Nov 18, please make an effort to engage with them as soon as possible. Thank you very much!

AC

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Response to Reviewer S6Hp
comment: •	[Significance of the Work.] Many thanks for this insightful comment. As discussed in [1], the comparison of the expressive power between different GNN extensions remains an open question. In particular, high-order WL tests are concepts of global comparisons over two graphs. Therefore, it is arguable if the WL hierarchy is a suitable tool to measure the expressiveness of GNN extensions as the latter focus on locality. Nonetheless, there exist some graph structures such that the 3-WL test and GraphSNN are not stronger than UnionSNN, i.e., some graphs can be distinguished by UnionSNN but not by 3-WL and GraphSNN. Similar analyses have been performed in existing works [2, 3]. As an example, UnionSNN can distinguish the 4x4 Rook’s graph and the Shrikhande graph (as shown in Figure 7, Page 15 in our revision), while 3-WL and GraphSNN cannot, which suggests that UnionSNN is stronger than 3-WL and GraphSNN on such graphs. This could be explained by the fact that the number of 4-cycles in these two graphs are different, and UnionSNN is able to reflect the number of 4-cycles with the consideration of $E^{vu}_3$ edges in union subgraphs. We have formulated this analysis in Appendix D in the revision.

•	[Missing References.] Thank you for pointing out these missing references. Our UnionSNN leverages the shortest path matrix to encode substructure information for message reweighting, while DE-GNN uses distance encoding to augment node features and to control the receptive field of message passing. SNALS utilizes an affinity matrix based on shortest paths to encode the structural information of hyperedges. The affinity matrix is designed to measure the connectivity between two hypergraphs (target nodes and their nearby nodes) instead of representing the local substructure itself. NGNN represents a graph by pooling the rooted subgraph instead of the rooted subtree. However, which substructure to choose and how to capture the complete information of local substructures have not been well investigated. Unlike the quantitative analysis in NGNN, we perform a qualitative theoretical analysis in Appendix B.3 to prove that our UnionSNN is strictly more expressive than 1-WL for distinguishing non-isomorphic graphs. We have added the discussions on the above methods to our related work in the revision (Page 2). We have also included NGNN as a new baseline in our graph classification experiments and reported the results in Table 1. The results show that UnionSNN outperforms NGNN in all cases. 

|           | MUTAG        | PROTEINS     | ENZYMES      | DD           | NCI1         | NCI109       | FRANKENSTEIN | Tox21         |
|-----------|--------------|--------------|--------------|--------------|--------------|--------------|--------------|---------------|
| NestedGIN | 86.23 ± 8.82 | 68.55 ± 3.22 | 54.67 ± 9.99 | 70.04 ± 4.32 | 82.04 ± 2.23 | 79.94 ± 1.59 | 67.07 ± 1.46 | 91.42 ± 1.18  |
| UnionSNN  | **87.31 ± 5.29** | **75.02 ± 2.50** | **68.17 ± 5.70** | **76.74 ± 2.04** | **82.34 ± 1.93** | **81.61 ± 1.78** | **67.83 ± 1.99** | **91.76 ± 0.85**  |

•	[More Experiments.] Many thanks for this suggestion. We provide additional experiments for Graph Regression on two new datasets: ZINC10k with 12k graphs and ZINC-full with 250k graphs. The data split of ZINC10k is (train: val: test=10k: 1k: 1k). The data split of ZINC-full follows that in its original paper. We report the MAE on the test set. The results show that, (1) UnionSNN outperforms all baselines, and (2) our proposed substructure descriptor can improve the performance of various MPNNs. We didn’t report the result of CGN on ZINC-full since the preprocessing of Ricci Curvature computation took more than 100 hours. We have added this set of results in the revision (Appendix G.2).

|                | ZINC10k | ZINC-full  |
|----------------|---------|------------|
| GCN            | 0.4652  | 0.1159     |
| UnionGCN       | *0.4211*  | *0.0879*     |
| GIN            | 0.3745  | 0.1496     |
| UnionGIN       | *0.3607*  | *0.1343*     |
| GraphSAGE      | 0.4468  | 0.1219     |
| UnionGraphSAGE | *0.4404*  | *0.1134*     |
| GAT            | 0.4869  | 0.1112     |
| GatedGCN       | 0.4425  | 0.0919     |
| CGN            | 0.4512  | -          |
| 3WLGNN         | 0.3616  | 0.0964     |
| UnionSNN       | **0.3566**  | **0.0843**     |

[1] Papp P A and Wattenhofer R. A Theoretical Comparison of Graph Neural Network Extensions[C]// International Conference on Machine Learning. PMLR.
[2] Balcilar M, Héroux P, Gauzere B, et al. Breaking the limits of message passing graph neural networks[C]//International Conference on Machine Learning. PMLR, 2021: 599-608.
[3] Bodnar C, Frasca F, Wang Y, et al. Weisfeiler and lehman go topological: Message passing simplicial networks[C]//International Conference on Machine Learning. PMLR, 2021: 1026-1037.

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Response to Reviewer TUPm (2/2)
comment: •	[Why Union Minus Subgraphs sometimes perform better than Union Subgraphs?] Thank you for raising this question. In the process of finding the answer to this question, we find that union minus subgraphs were not extracted correctly in our previous experiments. We correct this bug, re-run the experiments, and update the results. Union outperforms Minus in all cases, except for the MUTAG dataset. The difference between a union subgraph and a union minus subgraph is: the former includes edges in $E^{vu}_3$, which form 4-cycles. Therefore, union subgraphs are aware of 4-cycles, while union minus subgraphs are not. None of the graphs in MUTAG contains 4-cycles and thus the performance of Union and Minus is the same on MUTAG.

|         | MUTAG        | PROTEINS     | ENZYMES      | DD            | NCI1         | NCI109        |
|---------|--------------|--------------|--------------|---------------|--------------|---------------|
| overlap | 85.70 ± 7.40 | 71.33 ± 5.35 | 65.00 ± 5.63 | 73.43 ± 4.07  | 73.58 ± 1.73 | 72.96 ± 2.01  |
| minus   | **87.31 ± 5.29** | 68.70 ± 3.61 | 65.33 ± 4.58 |  74.79 ± 4.63 | 80.66 ± 1.90 | 78.70 ± 2.48  |
| union   | **87.31 ± 5.29** | **75.02 ± 2.50** | **68.17 ± 5.70** | **76.74 ± 2.04**  | **82.34 ± 1.93** | **81.61 ± 1.78**  |

•	[Application to Other Graph Tasks.] Thanks for the suggestion. We provide additional experiments for Graph Regression on two new datasets: ZINC10k with 12k graphs and ZINC-full with 250k graphs. The data split of ZINC10k is (train: val: test=10k: 1k: 1k). The data split of ZINC-full follows [1]. We report the MAE on the test set. The results show that, (1) UnionSNN outperforms all baselines, and (2) our proposed substructure descriptor can improve the performance of various MPNNs. We didn’t report the result of CGN on ZINC-full since the preprocessing of Ricci Curvature computation took more than 100 hours. We have added this set of results in the revision (Appendix G.2).

|                | ZINC10k | ZINC-full  |
|----------------|---------|------------|
| GCN            | 0.4652  | 0.1159     |
| UnionGCN       | *0.4211*  | *0.0879*     |
| GIN            | 0.3745  | 0.1496     |
| UnionGIN       | *0.3607*  | *0.1343*     |
| GraphSAGE      | 0.4468  | 0.1219     |
| UnionGraphSAGE | *0.4404*  | *0.1134*     |
| GAT            | 0.4869  | 0.1112     |
| GatedGCN       | 0.4425  | 0.0919     |
| CGN            | 0.4512  | -          |
| 3WLGNN         | 0.3616  | 0.0964     |
| UnionSNN       | **0.3566**  | **0.0843**     |

[1] Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982, 2020.

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Response to Reviewer TUPm (1/2)
comment: •	[Performance of UnionSNN (Simple Architecture).] Thanks for pointing this out. The design of UnionSNN’s architecture is just a simple implementation to verify the effect of (1) the local substructure we choose to characterize neighborhood information, (2) our design of the substructure descriptor. Based on the results, it is true that the architecture of UnionSNN is not always the best when compared to other architectures. Nonetheless, UnionSNN still achieves the top 2 results on all 8 datasets. This shows that our proposed union subgraphs and the shortest-path-based descriptor are effective even with a simple network architecture.

•	[Performance of Different Substructure Descriptors on Other Architectures.] Thanks for the suggestion. To clarify, our ablation study for substructure descriptors in the submitted version was based on the UnionSNN architecture instead of the GraphSNN architecture. Following your suggestion, we test the performance of different substructure descriptors on GCN and GIN, and the results are reported below. The best results are highlighted in bold and the second-best results are italic. The experiments show that our substructure descriptor (UnionGCN and UnionGIN) achieves the top 2 performance in 7 out of 8 settings. We have added this set of results in the revision (Appendix G.3).

|              | MUTAG        | PROTEINS     | ENZYMES      | NCI109        |
|--------------|--------------|--------------|--------------|---------------|
| GCN          | 77.13 ± 5.24 | 73.89 ± 2.85 | 64.33 ± 5.83 | 75.91 ± 1.53  |
| BetGCN       | 76.61 ± 8.86 | 73.23 ± 6.45 | 63.04 ± 8.70 | 76.12 ± 1.35  |
| CountGCN     | 78.56 ± 4.74 | 75.02 ± 4.49 | **64.67 ± 7.14** | 77.84 ± 1.36  |
| CurvatureGCN | **84.59 ± 7.30** | 73.94 ± 6.01 | 61.32 ± 2.64 | 78.35 ± 1.74  |
| LapGCN       | 79.13 ± 6.83 | *74.24 ± 5.52* | *64.57 ± 2.35* | **80.33 ± 1.59**  |
| UnionGCN     | *81.87 ± 3.81* | **75.02 ± 2.50** | **64.67 ± 7.14** | *79.50 ± 1.82*  |

|              | MUTAG        | PROTEINS     | ENZYMES      | NCI109        |
|--------------|--------------|--------------|--------------|---------------|
| GIN          | 86.23 ± 8.17 | 72.86 ± 4.14 | 65.83 ± 5.93 | 80.95 ± 1.87  |
| BetGIN       | 86.70 ± 5.42 | *73.31 ± 3.20* | 66.83 ± 7.94 | 81.42 ± 1.80  |
| CountGIN     | 87.25 ± 4.78 | **73.49 ± 3.91** | 66.67 ± 4.53 | *81.83 ± 1.62*  |
| CurvatureGIN | 86.62 ± 5.40 | 62.86 ± 8.65 | 64.67 ± 5.72 | 80.96 ± 2.29  |
| LapGIN       | *87.81 ± 5.24* | 71.87 ± 4.13 | **68.50 ± 7.97** | 81.54 ± 2.38  |
| UnionGIN     | **88.86 ± 4.33** | 73.22 ± 3.90 | *67.83 ± 6.10* | **82.24 ± 1.24**  |

•	[Performance of Our Proposed Substructure Descriptor on Other Methods.] Thank you for the suggestion. We plugin our proposed substructure descriptor to GatedGCN and GraphSAGE and the results are reported below. We have also tried to apply to GAT. However, we are not able to get the results due to limited time and computation resources. The results show that the incorporation of our substructure descriptor into GatedGCN and GraphSAGE improves the performance in 14 out of 16 settings. We have added the results in Table 1 of the revision, which once again demonstrates the effectiveness of our proposed substructure descriptor.

|                | MUTAG         | PROTEINS     | ENZYMES      | DD           | NCI1         | NCI109       | FRANKENSTEIN | Tox21         |
|----------------|---------------|--------------|--------------|--------------|--------------|--------------|--------------|---------------|
| GatedGCN       | 77.11 ± 10.05 | 76.18 ± 3.12 | 66.83 ± 5.08 | **72.58 ± 3.04** | 80.32 ± 2.07 | 78.19 ± 2.39 | 61.40 ± 1.92 | 90.83 ± 0.96  |
| UnionGatedGCN  | **77.14 ± 8.14**  | **76.91 ± 3.06** | **67.83 ± 6.87** | 72.50 ± 2.22 | **80.95 ± 2.11** | **78.21 ± 2.58** | **61.47 ± 2.54** | **91.31 ± 0.89**  |
| GraphSAGE      | 80.38 ± 10.98 | **74.87 ± 3.38** | 52.50 ± 5.69 | 73.10 ± 4.34 | 63.94 ± 2.52 | 65.46 ± 1.12 | 52.95 ± 4.01 | 88.37 ± 0.00  |
| UnionGraphSAGE | **83.04 ± 8.70**  | 74.57 ± 2.35 | **58.32 ± 2.64** | **73.85 ± 4.46** | **69.36 ± 1.64** | **69.87 ± 1.04** | **56.75 ± 3.85** | **88.59 ± 0.12**  |

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Response to Reviewer 6BpF (2/2)
comment: 
- [Definition of “Union Isomorphism”] The concept of “union isomorphism” is the same as that of “overlap isomorphism”, except that “union subgraphs” are used in place of “overlap subgraphs” as the pair of subgraphs in the ordinary graph isomorphism part.
In the above example subgraphs $S_{v_1}$ and $S_{v_2}$, there doesn’t exist a bijective mapping $g$, based on which the corresponding pairs of union subgraphs are isomorphic (ordinary one). The union subgraph $S_{v_1 \cup k}$ for any $k \in $ {$a, b, c, d, e, f, g, h$} is $S_{v_1}$ itself, and the same for $S_{v_2}$. Since $S_{v_1}$ and $S_{v_2}$ are not isomorphic (ordinary one), these two graphs are not union-isomorphic.

- [Complete Picture of the Neighborhood.] Thanks for pointing out this confusion. We didn’t specify the context clearly in this statement. We mean that the substructures we define by the concept of "union subgraphs" are the largest possible if we consider the 1-hop neighborhood of an edge. We have rephrased the sentence in the Abstract to be more specific. May we ask if there are more doubts after "that it is" in this point?

- [Novelty and Merits.] Many thanks for the comment. We emphasize that, comparing to other subgraph-based approaches and other substructure descriptors (as discussed respectively in Sections 3.2 and 4.1), our method (1) chooses a better local substructure that characterizes the complete 1-hop neighborhood information, and (2) designs a substructure descriptor that meets three desirable properties to guarantee its ability to distinguish non-isomorphic (sub)graphs. We have modified the Introduction to better position our work.

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Response to Reviewer 6BpF (1/2)
comment: - [“Complete” and “Best”.] Many thanks for the comment. In our work, we discuss the local structure that represents the relationships between 1-hop neighbors. We restrict the discussion to the 1-hop neighborhood because we aim to develop a model based on the MPNN architecture, in which a single layer of convolution is performed on the 1-hop neighbors. In this regard, the union subgraph we choose is able to characterize the complete picture of the 1-hop neighborhood. For the “best”, we use it to characterize the properties of the substructure descriptor. We list down three desirable properties for distinguishing non-isomorphic substructures and show that the proposed shortest-path-based descriptor satisfies the three properties and meanwhile retains the information of a union subgraph (Theorem 2). We have modified the corresponding places in the revision (Page 4) to avoid overclaims.

- [Definition of “Overlap Isomorphism”] Sorry for the confusion on this concept. The last “isomorphic” in the definition refers to the ordinary “graph isomorphism”. The concept of “overlap-isomorphic” is defined over two subgraphs $S_i$ and $S_j$, where $S_i$ is an induced subgraph of $\tilde{N}(i) = N(i) \cup$ {$i$} (i.e., the 1-hop neighborhood of a node $i$). The bijective mapping $g$ is between the node sets of $S_i$ and $S_j$, i.e., $\tilde{N}(i) $ and $\tilde{N}(j)$. The definition of "overlap-isomorphic" requires that for any $v \in N(i)$, the pair of corresponding overlap subgraphs $S_{i \cap v}$ and $S_{g(i) \cap g(v)}$ are isomorphic (ordinary graph isomorphic), where $g(i) = j$ and $g(v) = u$. Therefore, “overlap isomorphism” is defined based on the ordinary graph isomorphism of any pair of overlap subgraphs, where each corresponding pair is defined based on $g$. Note that when we talk about an “overlap subgraph”, denoted as $S_{i \cap v}$, it refers to a subgraph based on an edge $(i, v)$, and is defined as $S_i \cap S_v$. It contains only edges in $E^{iv}_1$. 

	We give an example for this concept. As shown in the Figure 6, Page 13 in our revision, we have two subgraphs $S_{v_1}$ and $S_{v_2}$. These two subgraphs are overlap-isomorphic. The bijective mapping $g$ is: $g(k_1) = g(k_2)$, $\forall k \in$  {$v, a, b, c, d, e, f, g, h$}. Take a pair of corresponding overlap subgraphs as an example: $S_{v_1 \cap g_1}$ and $S_{v_2 \cap g_2}$. They are based on two red edges $(v_1, g_1)$, and $(v_2, g_2)$ and the rest of the edges in the overlap subgraphs are colored in blue. It is easy to see that $S_{v_1 \cap g_1}$ and $S_{v_2 \cap g_2}$ are isomorphic (ordinary one). In this ordinary graph isomorphism, its bijective mapping between the nodes is not required to be the same as $g$, just as in the case of $S_{v_1 \cap g_1}$ and $S_{v_2 \cap g_2}$. It could be the same or different, as long as the ordinary graph isomorphism holds between this pair of overlap subgraphs. In this example, any pair of corresponding overlap subgraphs defined under $g$ are isomorphic (ordinary one). In fact, all overlap subgraphs have the same structure in this example. In this sense, the concept of “overlap isomorphism” doesn’t look at the neighborhood based on a single edge, but captures the neighborhoods of all edges and thus the “overlap” connectivities of the two subgraphs $S_{v_1}$ and $S_{v_2}$. 

	We have added the example in the revision (Appendix A) for better understanding.

From: ICLR.cc/2023/Conference/Paper1386/Authors
title: Response to Reviewer 5gGi
comment: 
•	[Analysis of Expressive Power.] Thank you for the comment. As discussed in [1], the comparison of the expressive power between different GNN extensions remains an open question. In particular, high-order WL tests are concepts of global comparisons over two graphs. Therefore, it is arguable if the WL hierarchy is a suitable tool to measure the expressiveness of GNN extensions as the latter focus on locality. Nonetheless, there exist some graph structures such that the 3-WL test and GraphSNN are not stronger than UnionSNN, i.e., some graphs can be distinguished by UnionSNN but not by 3-WL and GraphSNN. Similar analyses have been performed in existing works [2, 3]. As an example, UnionSNN can distinguish the 4x4 Rook’s graph and the Shrikhande graph (as shown in Figure 7, Page 15 in our revision), while 3-WL test and GraphSNN cannot, which suggests that UnionSNN is stronger than 3-WL and GraphSNN on such graphs. This could be explained by the fact that the number of 4-cycles in these two graphs are different, and UnionSNN is able to reflect the number of 4-cycles with the consideration of $E^{vu}_3$ edges in union subgraphs.  We have formulated this analysis in Appendix D, in the revision.
 
•	[Use of Shortest Paths.] We are not the first to use shortest paths in GNNs. However, our way of using shortest paths is different from recent works discussed in Section 2.2. We use shortest paths to transform an entire local subgraph to a matrix for further edge encoding, while Graphormer encodes the shortest path information to the bias of the softmax attention as spatial encoding and SPN adopts shortest paths to guide the multi-scale graph convolution. In both Graphormer and SPN, only the shortest distances from the central node are taken into account and the distances between neighbor nodes are completely ignored.

•	[Novelty and Presentation.] Many thanks for the comment. We emphasize that, comparing to other subgraph-based approaches and other substructure descriptors (as discussed respectively in Sections 3.2 and 4.1), our method (1) chooses a better local substructure that characterizes the complete 1-hop neighborhood information, and (2) designs a substructure descriptor that meets three desirable properties to guarantee its ability to distinguish non-isomorphic (sub)graphs. We have modified the Introduction to better position our work.

[1] Papp P A and Wattenhofer R. A Theoretical Comparison of Graph Neural Network Extensions[C]// International Conference on Machine Learning. PMLR.

[2] Balcilar M, Héroux P, Gauzere B, et al. Breaking the limits of message passing graph neural networks[C]//International Conference on Machine Learning. PMLR, 2021: 599-608.

[3] Bodnar C, Frasca F, Wang Y, et al. Weisfeiler and lehman go topological: Message passing simplicial networks[C]//International Conference on Machine Learning. PMLR, 2021: 1026-1037.

[4] Huang Y, Peng X, Ma J, et al. Boosting the Cycle Counting Power of Graph Neural Networks with I $^ 2$-GNNs[J]. arXiv preprint arXiv:2210.13978, 2022.

From: ICLR.cc/2023/Conference/Paper1386/Reviewer_5gGi
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes a new graph neural network model called Union Subgraph Neural Network (UnionSNN), where the core idea is to consider a certain class of subgraphs induced by two adjacent nodes. This model therefore can be seen another example of subgraph-based graph neural networks, which uses shortest path distances as descriptors. Authors show that the model is strictly more expressive than 1-WL, hence surpassing the expressive power of classical graph neural networks. They also conduct an empirical analysis on model variations, including different descriptors, and report their findings.

strength_and_weaknesses: Strengths:

- The study of subgraph based aggregation is an important direction, and there is a value in discussing the advantages/disadvantages of different descriptors.

Weaknesses:

- The paper is really hard to follow: The concepts are relatively simple but the writing hinders readability.
- The formal analysis is very limited. Theorem 3 is a straight-forward observation (even, if we were to ignore existing results). It also doesn't tell us anything about the upper bound in the expressive power of UnionSNNs.
- The idea of using shortest paths dates back to shortest path graph kernels, and is also used in multiple recent works authors cite, e.g., SPNs (Abboud et al., 2022), Graphormer (Ying et al., 2021).
- The work is not located well in the existing literature: it is unclear why this model is superior to other subgraph-based approaches, or other shortest-path related approaches. Some of these works are credited in the paper, but there is a lack of discussion in terms of model properties as well as in terms of empirical analysis. The proposed model is a simple extension of GIN which uses the descriptors.


clarity,_quality,_novelty_and_reproducibility: There is significant room for improvement in the presentation. What is the exact motivation of this paper?  How does it compare to existing approaches and what are potential advantages/disadvantages? It is hard for me to answer these questions in the current state of the paper. In my opinion, this paper could be better placed as a review of existing subgraph models using different descriptors and comparing them under different assumptions and on different empirical setups. The proposed models do not appear novel to me.
summary_of_the_review: The paper is largely incremental on ideas from the literature, but a better framing of the paper may be possible since there are some interesting empirical insights. However, the current state of the paper clearly falls under the acceptance threshold.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper1386/Reviewer_6BpF
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper propose a new method using "union subgraph" to capture neighborhood information of two nodes to enhance information of nodes in GNNs. 
strength_and_weaknesses: I find the paper overclaim the concept they defined to be "complete" and "the best" without any proper ground.
 
- The concept of union subgraph is simply a subgraph induced by the union of two neighborhood node set. It definitely cover all its "subgraphs". Is it a perfect local structure? It depends on what is local structure. Is it the best to characterize local structure? It strongly depend on the definition of "the best". I think the paper overclaim the property of this concept. From I could see, its is a superset of other sets and by this logic, it is not "the best" as it is a subset of its superset... 
- The concept of "overlap isomorphism" is defined based on the concept of "isomorphism" of any pair of union subgraphs. However, the definition of isomorphism of union subgraph is not defined. I have to assume that two set are isomorphic if they have the same cardinality. Is this true? Does the bijective mapping of two overlap subgraphs in  "overlap isomorphism"  has anything to do with the bijective mapping "g" in the definition of S_i,S_j ovelap isomorphism
-  The same problem can be seen with union isomorphism. 
- This comes back to the motivation of the paper to capture a "complete picture of the neighborhood", which is not defined. I'm guessing that it is 
clarity,_quality,_novelty_and_reproducibility: - The paper is basically very well presented in most part. Background and paper organization, experiments are all clear, except for the key concepts of the paper. 
- The paper didn't have argument to convince its claim of quality. There might be some difference to previous work that can be novel but not presented in a proper way. 
summary_of_the_review: The paper need to claim the concept they defined properly with information it provides/not provide, comparing with previous work to show its contribution. For the moment, it does not convince readers of its merit yet.
correctness: 1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1386/Reviewer_TUPm
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper proposes an edge feature augmentation based on distance matrix on edge-enclosing subgraph which is useful to boost MPNN-like architectures on graph and node classification tasks.
strength_and_weaknesses: ## Strength

- This papers proposes a feature engineer technique which can be plugged into any MPNN-like achiectures to improve their expressivity for graph and node classification tasks.


## Weakness

The experiment part should be improved.

- From current experiment results, it looks like that your archiecture UnionSNN is not good (nearly half of the cases are not improving), thus I think the outcome here is using your proposed edge feature augmentation can improve performance. However, to show this conclusion, you should thoroughly compare performance of the same MPNN-like architecture with all other substructure descriptors as augmentation methods. I only see such a comparison on GraphSNN which do not always yield best performance, thus it is not enough.
- Besides, your method should work for any MPNN-like architecture, but it is only used for GCN, GIN and GraphSNN, why it is not applied on other methods in Table 1 and 2? I do not see any barricade at least for GAT, GatedGCN and GraphSAGE.
- Is there any theoretical explanation of minus performance over union in Table 3? Since minus is always a subset of union, such behavior is not straight and need explanation.

Another minor concern is why you limit only on node and graph classification. You are utilizing enclosing subgraphs for all edges, and as far as I know, there is work [1] utilziing similar things, but for link prediction.

[1] Chamberlain, Benjamin Paul, et al. "Graph Neural Networks for Link Prediction with Subgraph Sketching." arXiv preprint arXiv:2209.15486 (2022).
clarity,_quality,_novelty_and_reproducibility: Quality, clarity and novelty are all good.
summary_of_the_review: This paper proposes edge feature engineering which will be helpful for all MPNN-like architectures in graph and node classification. However, to precisely addressing that the proposal is state-of-the-art, more experiments are required.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1386/Reviewer_S6Hp
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This work proposes to use structural features constructed based on the union subgraph of two nodes as the edge feature between two nodes, to further improve the expressive power of graph neural networks. The paper gives the reasons why it chooses union subgraphs, and why it uses the spectrum of a path-based matrix as the structural feature. Experiments demonstrates the benefits of the adopted structural features.  
strength_and_weaknesses: Strengthes:
1. The idea to use structural features of a union subgraph of a pair of nodes is new for graph level tasks, although it is not new in general to improve GNN expressive powers. The first works consider injecting structural features into GNNs are [1]. Also, [2] exactly uses the spectral norm of the matrix of shortest-path-length distances as structural features. 

2. The motivations behind different designs are explained very clear and paired with examples. I appreciate this effort. 

3. The types of experiments are extensive, though the used datasets are not representative enough. The experimental results generally show the benefits of the proposed approach. At the beginning, I have some concerns on the computation complexity, while the authors defend themselves well by providing efficiency analysis. 

Weaknesses:
1. My big concern is about the significance of this work. This is another method to outperform 1WL GNN. There have been so many works on this topic. Different works may propose different designs, while how optimal this work is in theory, and why this work is better than others? These are not super clear in this paper. The authors have used a few examples to justify the points, while to me this reads not strong enough. In particular, the method is only provably more expressive than 1WL. How far it can reach is unclear. 

Also, constructing structural features to improve GNN expressive power is not a new idea. The authors miss discussing many works on the similar topics. Here, I just mention three closely related missing references [1,2,3]. [1] is the first work to inject structural features (distance features) to improve GNN expressive power with provable results. [1] originally proposes to use shortest path distances to a pair (or a set) of nodes to define structural features and gives theoretical analysis. [2] uses the spectral norm of the shortest path-distance-defined matrix like the one in this work to define structural features. These works should be discussed. Also,  can the authors provide any quantitive theoretical analysis results like [1] and [3] ([3] also leveraged structural features within a subgraph around each node)? And, can the authors compare the adopted method with more powerful GNNs in theory such as 3WL GNN but not just in practice?

2. Another concern is about the experiments. Different aspects have been evaluated in experiments and this is great. However, since the theoretical contribution is not that sufficient, more challenging datasets and more extensive experiments are expected. For example, most adopted datasets are small. Also, only a subset of OGB graph-level datasets has been used. Also, graph-level regression tasks over datasets such as QM9 have not been used to evaluate expressive power. Typically, I do not ask for these many experiments. However, as the theoretical contribution is not that sufficient, the experiments are expected to be extensive. 

Currently, I giving a marginal recommendation is by assuming that the authors may use more challenging datasets and tasks to evaluate the method. If the authors can indeed provide such evaluations and/or provide a more solid theory for the work, I would like to give a strong recommendation. 


[1] Distance encoding: design provably more powerful neural networks for graph representation learning, Li et al. NeurIPS 2020
[2] Principled hyperedge prediction with structural spectral features and neural networks, Wang et al. arxiv' 2021
[3] Nested Graph Neural Networks, Zhang & Li, NeurIPS 2022
clarity,_quality,_novelty_and_reproducibility: Clarity: Excellent. Very clear, easy to follow, with nice examples

Quality: Okay while not outstanding. Reasonable technical contributions with some reasonable experiments. 

Novelty: Some novelty while not groundbreaking (an old concept used in a new way)

Reproducibility: I did not check the codes while I think the experiments read reasonable to me.  
summary_of_the_review: Overall, I think this work is a solid work while with some but not significant technical contributions. Therefore, I give a marginal recommendation given the condition that the authors may use more challenging datasets and tasks to evaluate their methods. I would like to give strong recommendation if the new experiments are interesting or new theory is provided. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold