Edge Representation Learning with Hypergraphs

Abstract: Graph neural networks have recently achieved remarkable success in representing graph-structured data, with rapid progress in both the node embedding and graph pooling methods. Yet, they mostly focus on capturing information from the nodes considering their connectivity, and not much work has been done in representing the edges, which are essential components of a graph. However, for tasks such as graph reconstruction and generation, as well as graph classification tasks for which the edges are important for discrimination, accurately representing edges of a given graph is crucial to the success of the graph representation learning. To this end, we propose a novel edge representation learning framework based on Dual Hypergraph Transformation (DHT), which transforms the edges of a graph into the nodes of a hypergraph. This dual hypergraph construction allows us to apply message-passing techniques for node representations to edges. After obtaining edge representations from the hypergraphs, we then cluster or drop edges to obtain holistic graph-level edge representations. We validate our edge representation learning method with hypergraphs on diverse graph datasets for graph representation and generation performance, on which our method largely outperforms existing graph representation learning methods. Moreover, our edge representation learning and pooling method also largely outperforms state-of-the-art graph pooling methods on graph classification, not only because of its accurate edge representation learning, but also due to its lossless compression of the nodes and removal of irrelevant edges for effective message-passing.


Reviews and author responses:

 We sincerely thank you for your constructive and helpful comments. We appreciate your positive comments that our work has a good motivation focusing on an interesting problem, which is original and novel, and further the proposed idea is good. We address all your concerns below:

---
**Question 1-1:** In terms of the originality and novelty, this proposed idea is good. However, it is a little disappointing that there are not many technical contributions in this paper. 

**Answer:** Thank you for your thoughtful acknowledgement of our work as it is original and novel. However, we strongly believe that the proposal of a novel edge representation learning framework with the hypergraph duality is strong enough to be considered as a standalone contribution, since our method is universal and versatile as it eliminates the need to design special message passing for edges.

---
**Question 1-2:** The proposed idea is simple and many researchers can come up with such an idea.

**Answer:** Exploiting hypergraph duality for edge representation learning is indeed conceptually very simple to understand, but this does not automatically mean that coming up with the idea is simple which anyone can come up with. If exploiting hypergraph duality is such a simple idea for edge representations, then why hasn't anyone proposed it before, while it yields impressive performance on a variety of graph-related tasks, such as graph classification, reconstruction, and generation? We believe that the main reason why such a “simple” idea has not been explored thus far, is because researchers have overlooked the needs of learning explicit representations for edges. We have shed fresh light on the problem, demonstrating its importance on graph classification, reconstruction, and generation, which is another very important contribution of this work.

---
**Question 1-3:** Basically, by modeling the edges as another type of nodes, the original graph can be seen as a bipartite graph, and it is common to transform bipartite graphs to hypergraphs in a wide range of graph-based tasks such as recommendation.

**Answer:** We did not claim that we have invented the new transformation scheme for nodes to edges, whereas our novelty is rather on the proposal of Dual Hypergraph Transformation for edge representation learning, which exploits hypergraph duality. Moreover, converting a given graph to a hypergraph by using a bipartite graph as an intermediate structure incurs additional complexity, which is inefficient compared to our proposed transformation that transforms the original graph to the dual hypergraph with one transpose operation of the incidence matrix.


---
**Question 2:** When performing the message-passing among edges (nodes in the hypergraph), the proposed method needs to look up the adjacency matrix to find neighboring nodes. The process is similar to that of hypergraph neural networks which have been proposed and studied several years ago.

**Answer:** This is a misunderstanding about the focus of our work. Our work focuses on learning accurate edge representations using the dual hypergraphs, while existing works on hypergraph representation learning (e.g., hypergraph neural networks) aim to learn the node representations. We do not claim to have proposed a message passing scheme on a hypergraph for the first time, and have acknowledged existing works in the related work section. We discuss the specific differences between existing works on hypergraph representation learning and ours below: 

**Existing works on hypergraph representation learning**
1) Their aim is to learn accurate node representations on the hypergraph, with few works that use the edge features for solely augmenting the node representations.
2) They do not explicitly learn the (hyper)edge representations, nor learn graph-level edge representation.

**Our edge representation learning with Dual Hypergraph Transformation**
1) Our aim is to **learn accurate edge representation of the conventional graph using the dual hypergraph**, which realizes the message passing of nodes to edges with a simple transformation.
2) We also propose novel **edge pooling methods**, HyperDrop and HyperCluster, for obtaining **graph-level** edge representations.

Thus, while the message passing scheme over hypergraphs looks similar, the works for targeting representation learning of hypergraphs are orthogonal to us.


---
**Question 3:** The authors claimed that “Although acquiring the neighboring node set from the incidence matrix costs O(m), compared to using the adjacency matrix which costs O(n), this cost can be ignored by using the sparse implementation of DHT explained above” in page 5. Actually, both the hypergraph incident matrix and the graph adjacency matrix can have sparse implementations. This cannot be the advantage of the proposed framework.

**Answer:** This is a misunderstanding of our method, since the sparse implementation of DHT is still more efficient compared to the sparse implementation of the adjacency matrix. The transformation cost from the edge list to the hyperedge list (i.e., the edge list of the dual hypergraph) is negligible as we perform the operation of reshaping the edge list (see Section A.3 of the supplementary file for details), while the cost from the edge adjacency to the edge list is $O(m \times d_{\text{max}})$ (Monti et al., 2018), where $d_{\text{max}}$ is the maximum degree of the original graph. Thus, when the number of edges $m$ or $d_{\text{max}}$ increases, our method using the dual hypergraph is more efficient than existing methods, which we experimentally verified by measuring the transformation cost of a random graph with varying edge sizes as in the Table below.

Specifically, in the following table, we compare the transformation cost of our DHT (in seconds) against the edge adjacency transformation, on graphs with 1,000 nodes with varying the number of edges, generated from the Erdos-Renyi graph. The results are the average over 100 different runs.

| $\text{num of edges} \over \text{num of nodes}$ of the original graph | 2 | 4 | 8 | 16 | 32 | 64 |
| --- | --- | --- | --- | --- | --- | --- |
| Edge Adjacency | 32.78 | 65.93 | 131.36 | 260.92 | 527.44 | 1071.31 |
| Dual Hypergraph (Ours) | **0.13** | **0.18** | **0.26** | **0.45** | **0.81** | **1.37** |

---
**Question 4-1:** As for the pooling methods, it is not clear what the advantages of HyperCluster and HyperDrop when compared with methods such as DiffPool. Besides, these two pooling methods are intuitively designed. No new ideas are provided in this part.

**Answer:** This is a critical misunderstanding. While the HyperCluster and HyperDrop are direct adoptions of existing node pooling methods, the effects of pooling on edges are clearly different from the effects of pooling on nodes, and moreover, our edge pooling methods have more advantages against the node pooling methods. Specifically, as we have clearly stated the limitations of existing node pooling methods in lines 220-223, they suffer from the inevitable loss of both node and edge information. However, our proposed edge pooling methods overcome these through obtaining the global edge representation by clustering similar edges, or preserving the node while removing the task-irrelevant edges.



---
**Question 4-2:** It looks like that these two pooling methods, HyperCluster and HyperDrop, cannot be trained in an end-to-end way.

**Answer:** This is a factual misunderstanding, as our edge pooling methods are **trained in an end-to-end way**. In other words, we train the model, from representing the edges to pooling them, in an end-to-end fashion (see Figure 1 for the edge representation learning framework with edge pooling schemes).

---
**Question 5:** The experimental results are insignificant. Only small improvements are observed, which would question the necessity of applying the proposed framework to graph-based tasks.

**Answer:** The reported results in reconstruction of edges and graphs, and also the results in graph generation show that our model significantly outperforms existing baselines including the models for edge representation learning. Also, our model achieved statistically significant results compared to baselines in graph classification as well. Thus, the experimental results are significant.


---
**Question 6:** The number of edges is usually hundreds of times the number of nodes in real-world networks. 1) Learning edge representations requires far more memory compared with learning node representations. 2) Whether the hyperCluster and Hyperdrop can be performed before representation learning to tackle the memory issue?

**Answer:** This is a critical misunderstanding of our work, since 1) the proposed Dual Hypergraph Transformation (DHT) does not require additional memory, and 2) the edge pooling methods can be performed before representation learning to reduce the size of the graph, but this scheme is meaningless as we cannot meaningfully represent the edges into the latent space for task-adaptive pooling.

To be more specific for the former question, DHT interchanges the structural role of nodes and edges by simply transposing the incidence matrix along with the swapping of node and edge features, which is explained in lines 185-186. Thus, these rearranging operations do not require additional memory. Also, we can further implement the DHT more memory-efficiently, by using the sparse matrix (i.e., edge list) instead of the dense matrix (i.e., adjacency matrix), which is described in Section A.3 of the supplementary file.


---
**References**
* Monti et al. Dual-Primal Graph Convolutional Networks. arXiv preprint, 2018.


 We sincerely appreciate your time and effort to engage with us as well as your detailed comments. We would be happy to address the remaining concerns below.

---
**Question 1-1:** I still don't understand how the aggregation described in the example corresponds to equation (4) in the original paper. 

**Answer:** The aggregation that we explained in the previous response is based on the **sparse implementation of equation (4)**, as we have explained in lines 186-189 and lines 200-201 in the main paper, and in Section A.3 of the supplementary file, to take the advantage of efficiency on message passing.

To be specific, equation (4) can be sparsely implemented, as in the example, by using the **hyperedge list**instead of the incidence matrix when finding the neighboring node set. This sparse approach achieves the aggregation by **passing the messages for each hyperedge**in the dual hypergraph. This can be done since we use the hyperedge list which contains the information of incidence between nodes and hyperedges.


**Question 1-2:** The aggregation mechanism of DHT is the same mechanism used in the line graph GNN.

**Answer:** There is a misunderstanding between the **aggregation mechanism**and the **aggregation result**. As you have correctly explained, the result of the aggregation (or precisely the adjacency between the edges) is the same, but the **aggregation scheme is completely different**. We have clarified from the previous example, that the message passing in the dual hypergraph costs $\mathbf{O(n)}$ whereas the message passing on the line graph costs $\mathbf{O(n^2)}$. 

**Question 2:** I agree the edge information (bond, double bond) is necessary since the hydrogen atoms are dropped. I am satisfied with this point as long as the authors clarify this in the main text.

**Answer:** Thank you for your detailed and constructive feedback. We will make sure to clarify the details in the revision.

---
We hope the above responses satisfactorily address your points, and please let us know if you have anything else we should address.

Thanks, Authors.


 Thanks for the example, however, I still don't understand how the aggregation described in the example corresponds to equation (4) in the original paper. In particular, after DHT, the nodes in the transformed graph G* corresponds to edges in the original graph. Now, let's look at equation 4:

For each node e, it summarize messages from neighboring nodes (e.g. f), where the neighboring is defined by the transposed incidence matrix. Since the e and f are both nodes in the hypergraph (which are edge in the original graph), they are adjacent if they are connected to the same hype-edge in G*. In other words, edge e in the original graph pull information from a different edge f, if they are incident to the same node. This is the same aggregation mechanism used in the line graph GNN. Please correct me if I am not understanding it correctly.


I agree the edge information (bond, double bond) is necessary since the hydrogen atoms are dropped. I am satisfied with this point as long as the authors clarify this in the main text.

 We appreciate your response, and further address your remaining concerns below.

---
**Question 1:** I ask the author to clarify how the aggregation mechanism is different from the aggregation on the line graph, not how the graph transformation is different since it does not really matter.

**Answer:** There seems to be a misunderstanding, as our DHT has more benefits on both the **transformation** and the **aggregation**, compared to the line graph transformation.  In specific, our DHT is efficient in terms of  **the message passing**induced by the difference in the aggregation scheme, as well as the **the transformation**induced by the distinction between the dual hypergraph and the line graph.

First of all, the aggregation on the dual hypergraph is completely different from the aggregation on the line graph. This is because the dual hypergraph can model **higher-order dependencies** among multiple nodes, whereas the line graph cannot. The pairwise connectivity among three nodes and the single connectivity defined by a hyperedge is clearly different, which is the significance of the hypergraph and the related studies.

To understand this difference more intuitively, we would like to explain the aggregation (i.e., message passing) mechanism with a **specific example**. Suppose that the input graph $G$ is a **star graph** with one hub and $n$ nodes connected to the hub (thus, total $n+1$ nodes and $n$ edges).

- We first consider the **aggregation in the line graph** of $G$, which is a clique with $n$ nodes, namely $1,2,...,n$. Then, the undirected edges of this clique are defined as following $n(n-1)/2$ edges $[[1, 2], [1, 3], ..., [n-1, n]]$, which is indeed the two-dimensional edge list (we omit the reverse orders, such as 2 to 1, for simplicity). The message passing denotes the information propagation from the start node to the end node, which first propagates the information from the start node to the incidence edge (for example, 1 to [1, 2] for the propagation of 1 to 2), and then further propagates the information of the intermediate edge to the end node (for example, [1, 2] to 2 for the propagation of 1 to 2). Therefore, the complexity of message passing can be approximated as $\mathbf{O(n^2)}$.

- However, the **aggregation in the dual hypergraph is different from the aggregation in the line graph**. The dual hypergraph of $G$ is a hypergraph with $n$ nodes, namely $1,2,...,n$, connected to a single hyperedge $e_0$, and each node $i$ has a single self-loop $e_i$. Then, the hyperedge list is denoted as follows: $[[1, e_0], ..., [n, e_0],[1,e_1],...,[n,e_n]]$ which indicates the incidence between nodes and edges. The message passing on the hypergraph first propagates the information from start nodes to intermediate hyperedges (for example, {$1,...,n$} to $e_0$ or $1$ to $e_1$), and then propagate the aggregated information on the intermediate hyperedges to the end nodes (for example, $e_0$ to {$1,...,n$} or $e_1$ to $1$). **Thus, we can approximate the complexity of message passing on the dual hypergraph as $\mathbf{O(n)}$, which is much more efficient compared to the line graph message passing $\mathbf{O(n^2)}$.**

To sum up, the **message passing complexity of the dual hypergraph has exactly the same asymptotic complexity as the message passing on the original graph**, which is $O(m)$ in the general case, where $m$ is the number of edges of the original graph, whereas the **message passing complexity of the line graph can be increased to the quadratic of the number of nodes**.

Furthermore, even putting aside the difference in the aggregation scheme, our **DHT has clear advantages compared to using the line graph**, as we clearly explained in the answer to the previous Question 1-1. We restate the advantages below:

1) Our DHT is **bijective**, but the line graph transformation is **not injective**. This means that two different (non-isomorphic) graphs may be transformed into the same line graph, in which case a graph could be reconstructed into another graph, or two different graphs are considered as isomorphic.

2) Our DHT **does not incur any loss of node information**. However, with line graphs, since the nodes of the original graph do not correspond one-to-one to the edges, **the node information may be lost**during the transformation. This is a fatal flaw for tasks such as graph reconstruction and classification.

3) While our DHT can be done with **almost no cost**by applying simple transpose operations to the incident matrices, the line graph transformation is **not scalable**. 


**Question 2-1:** The hydrogen can be dropped in molecular graphs, but only when they are connected to carbon atom. 

**Answer:** This is a misunderstanding between a **molecular graph** and a **molecular skeletal formula**. For drawing the molecular skeletal formula, you are indeed right that hydrogens are dropped only when they are connected to carbon. However, by using the term molecular graph, we indicate the graph of a molecule perceived by GNNs, not a molecular skeletal formula commonly used in chemistry. The mentioned Wikipedia reference **does not refer to a molecular graph**, but plots **molecular skeletal formula**. We would like to emphasize again that it is a very common practice in GNN papers to **consider only heavy atoms as nodes to obtain the compact form**(David et al., 2020). We have cited numerous GNN papers that treat molecular graphs as we explained -- drop hydrogens and only consider heavy atoms -- in our last response. We will clarify these details related to molecular graphs in the subsequent revision, but please note that the detailed definition of the molecular graphs is not the main focus of our work.


**Question 2-2:** I did not know that the heavy atoms are dropped in the modeling, nor is it explicitly stated anywhere in the paper. The authors shall clarify that in the paper. 

**Answer:** It seems you have misunderstood our response, since we have clearly explained that molecular graphs **only consider the heavy atoms**, instead of dropping the heavy atoms in the modeling. Also, we already clearly stated that we follow the experimental setting of existing papers in Section C of the supplementary file, nonetheless we will further clarify the usage of heavy atoms in the subsequent revision.

**Question 2-3:** When I say geometry, I mean the 3D structures of the molecules. If all the atoms are modeled, then the geometry alone determines the molecular propriety. If the heavy atoms are dropped in the modeling, then bond information is indeed necessary.

**Answer:** Molecular datasets may not contain the information about the 3D structures, and the base GNNs (R-GCN in MolGAN and MPNN in MARS) from the existing papers do not utilize the 3D information and only rely on nodes (and/or edges) of molecular graphs to predict molecular properties. Therefore, accurately representing the edge information is critical to the success of the tasks, as we have experimentally shown in our paper.

---
**References**
* David et al., Molecular representations in ai-driven drug discovery: a review and practical guide. Journal of Cheminformatics 2020.


 1. Regarding the Lack of Novelty (Dual Hypergraph Transformation vs. Line Graph)

I do know that DHT and line graph construction are different. However, if you look at your equation (4), this aggregation is the **exactly the same** as if you first did the line graph transformation, then perform standard GNN on the line graph (and that has already been done in [2] as I pointed out). The fact the authors use the concept DHT does not change the fact that the underlying aggregation mechanism is the same.

If I am not understanding correctly, I ask the author to clarify how the aggregation mechanism is different, **not** how the graph transformation is different since it does not really matter.

2. Regarding my comment about Tylenol and NAPQI & my comment that the molecular propriety are determined by geometry.
 - Yes, the hydrogen can be dropped in molecular graphs, but only when they are connected to **carbon** atom. If you go to wikipedia page, https://en.wikipedia.org/wiki/Paracetamol, you can see how they are typically plotted.
 - I did not know that the heavy atoms are dropped in the modeling, nor is it explicitly stated anywhere in the paper. The authors shall clarify that in the paper. When I say geometry, I mean the 3D structures of the molecules. If all the atoms are modeled, then the geometry alone determines the molecular propriety. If the heavy atoms are dropped in the modeling, then bond information are indeed necessary.

 We sincerely thank you for your time and efforts in reviewing our work. We initially address all your concerns below:

---
**Question 1-1:** I am not very impressed with the novelty of this paper. To me, the net effect of EHGNN is simply running standard GNN on a transformed line graph (Godsil & Royle, 2001), which has been done in the past (for example see Cai et al. (2020)). 

**Answer:** This is a critical misunderstanding between the **dual hypergraph** used in our work and the **line graph** (i.e., dual graph) you refer to. The former is a hypergraph that can model higher-order dependencies among multiple nodes, while the latter is a conventional graph. For example, in Figure 1 of our paper, the resulting dual hypergraph has a hyperedge connecting three nodes, while the line graph cannot model such a high-order interaction. We list the **main differences between our Dual Hypergraph Transformation (DHT) and the line graph construction** in below:
1) Our DHT is **bijective** as shown in lines 180-184, and in Figure 2 in our main paper. However, the line graph transformation is **not injective**. This means that two different (non-isomorphic) graphs may be transformed into the same line graph, in which case a graph could be reconstructed into another graph, or two different graphs are considered as isomorphic.
2) Our DHT **does not incur any loss of node information**. However, with line graphs, since the nodes of the original graph do not correspond one-to-one to the edges, **the node information may be lost**during the transformation. This is a fatal flaw for tasks such as graph reconstruction and classification.
3) While our DHT can be done with **almost no cost** by applying simple transpose operations to the incident matrices, the line graph transformation is **not scalable**. First, the line graph construction itself requires at least $O(m)$ complexity where $m$ is the number of edges of the original graph. We further show the inefficiency of line graph construction, with the experimental results in the table below. Also, the constructed line graphs may have excessively large edges if the original graphs have very large degrees (e.g., scale-free graphs and community graphs). Specifically, the number of edges of the line graph has the size of $O(d^2_{\text{max}})$ where $d_{\text{max}}$ is the max degree of the original graph.

In the following table, we compare the transformation cost of our DHT (in seconds) with the line graph transformation, on graphs with 1,000 nodes with varying the number of edges, generated from the Erdos-Renyi graph. The results are the average over 100 different runs. 

| $\text{num of edges} \over \text{num of nodes}$ of the original graph | 2 | 4 | 8 | 16 | 32 | 64 |
| --- | --- | --- | --- | --- | --- | --- |
| Line graph | 32.78 | 65.93 | 131.36 | 260.92 | 527.44 | 1071.31 |
| Dual Hypergraph (Ours) | **0.13** | **0.18** | **0.26** | **0.45** | **0.81** | **1.37** |

We believe that this comparison against existing line-graph transformation for edge representation learning will be a valuable addition to better understand the novelty and the advantage of our work, and will include it in the paper in the revision.

---
**Question 1-2:** If the method itself is not novel, judging the significance of the empirical results is not meaningful.

**Answer:** Since there seems to be a misunderstanding of the main novelty of our work, we briefly recap it again. Our main contributions are the proposal of **1) explicit edge representation learning** by exploiting the **2) hypergraph duality**, which has almost no cost and allows applying conventional message passing algorithms and pooling methods for nodes, to edges. Further, we show that learning **explicit edge representations** is critical in improving the performance of **general graph-related tasks**, such as graph classification, reconstruction, and generation tasks. 

---
**Question 2-1:** The example in the introduction (figure 1) is simply not scientifically correct, since Tylenol would have two extra hydrogen atoms and is therefore different from NAPQI even from the perspective of regular GNN.

**Answer:** This is a critical misunderstanding between molecules and molecular graphs. We clearly stated as **molecular graphs** in the caption of Figure 1. To construct graph representations of molecules, it is a common practice to treat any hydrogens in a molecule implicitly, only considering heavy atoms (atoms besides hydrogen) as nodes to obtain the compact form (David et al., 2020). In addition to this, basically most of the models for graph generation that handle graph-structured molecular inputs (e.g., MolGAN (Cao & Kipf, 2018), MARS (Xie et al., 2021), GCPN (You et al., 2018), GraphAF (Shi et al., 2019), MoFlow (Zang & Wang, 2020), GraphDF (Luo et al., 2021)) only consider heavy atoms indeed. Therefore, the two example graphs are valid in terms of nodes in the perspective of the model’s input.

---
**Question 2-2:** In general, molecular propriety does not depend on the edges (whether it is single bond or double bond) but rather depends on geometries, despite the fact that traditionally chemists use different edges (bond) for intuitive understanding purposes.

**Answer:** This is also a critical misunderstanding of the edges of molecules.
* If the **geometries** refer to the configuration of atoms, edges themselves change the atom configuration as the bond type determines the remaining valency of incident atoms, thereby changing the number of hydrogens connected. For example, benzene and cyclohexane have the same configuration of heavy atoms, but their chemical properties are clearly different as the former is aromatic while the latter is not.
* If the **geometries** refer to the 3D structures of molecules, the bond type is still critical to geometries, since the number of hydrogens changes as explained above, and the bond length and rotational property change as well. Single bonds can rotate freely, while double bonds with extra $\pi$ bonds cannot.

---
**References**
* Chris Godsil & Gordon Royle. Algebraic Graph Theory. Springer 2001.
* Cai et al. Line Graph Neural Networks for Link Prediction, IEEE Transaction on Pattern Analysis and Machine Intelligence 2021.
* David et al., Molecular representations in ai-driven drug discovery: a review and practical guide. Journal of Cheminformatics 2020.
* Cao & Kipf. MolGAN: An implicit generative model for small molecular graphs. ICML 2018 Workshop on Theoretical Foundations and Applications of Deep Generative Models.
* Xie et al. Mars: Markov molecular sampling for multi-objective drug discovery. ICLR 2021.
* You et al. Graph convolutional policy network for goal-directed molecular graph generation. NIPS 2018.
* Shi et al.  GraphAF:  a flow-based autoregressive model for molecular graph generation. ICLR 2019.
* Zang & Wang.  MoFlow:  an invertible flow model for generating molecular graphs. KDD 2020.
* Luo et al. GraphDF:  A discrete flow model for molecular graph generation. ICML 2021.

 We sincerely thank you for your constructive and helpful comments. We appreciate your positive comments that our approach is simple and easy to implement. We initially address all your concerns below:

---
**Question 1:** Novelty: Converting an input graph to a hypergraph is a known approach. The authors combine this with graph convolution network which is also common in graph representation learning. This limits the novelty of the approach.

**Answer:** This is a critical misunderstanding. We did not claim that we have invented or discovered the hypergraph duality (Berge, 1973; Scheinerman & Ullman, 2011). Also, we did not claim that we have proposed the novel graph neural network over the hypergraphs. Our novelty rather comes from the **edge representation learning using Dual Hypergraph Transformation**which converts a graph into its dual form with simple transpose operations applied to incident matrices, which has not been done before. Also, from the problem-wise view, we shed new light on the **importance of edge representation learning for general graph-related tasks** (e.g., graph classification, reconstruction, and generation), which has been relatively overlooked due to the complexity of the edge representation learning. 

---
**Question 2:** Further, HyperCluster and HyperDrop are also simple implementations of clustering and dropping nodes.

**Answer:** While HyperCluster and HyperDrop are straightforward adoptions of conventional graph pooling methods, their **effects are completely different from conventional graph pooling methods**, since they can remove edges or cluster edges which is not possible with the conventional pooling methods. Also, we analyzed their effectiveness coming from the lossless compression of node information with extensive experiments (Figure 3, 5, 8 and Table 2), and we believe that this is an important contribution.

---
**Question 3-1:** Experiments - Dataset Size and Time: The authors have shown results on several tasks including edge and graph reconstruction, graph generation and graph and node classification. As they are changing to edge space, the authors should show the time taken in comparison to node-based approaches to showcase the utility of the approach.

**Answer:** The complexity of edge-level message passing with EHGNN is the same as the complexity of node-level message passing with GNN. This is because the **complexity for message-passing is $O(m)$ for both the original graph and the dual hypergraph**, where $m$ is the number of edges.

To be specific, since each node in the dual hypergraph has a degree of two (see Section A.3 of the supplementary file; lines 62-64), the hyperedge list of the dual hypergraph has a dimensionality of $2m$, thus the message passing complexity on the dual hypergraph using the hyperedge list is $O(m)$ regardless of the type of a given graph. This complexity is the same for the original graph using the edge list.

Although the analytic complexity of message passing is identical, we further empirically validate the computational complexity, by measuring the time cost for message passing (in seconds), on both random (Erdos-Renyi) and scale-free (Barabasi-Albert) networks with $3,000$ nodes and $11,984$ edges following the densification law (i.e., $m \propto n^{1.18}$, where $n$ and $m$ is the number of nodes and edges, respectively) (Leskovec et al., 2005) of the internet graph. As shown in the table below, the message passing time for nodes and edges are **almost identical**on two different types of graphs.

| Graph | Time for node-level message passing (sec) | Time for edge-level message passing (sec) |
| --- | --- | --- |
| Erdos-Renyi | 0.0031 | 0.0034 |
| Barabasi-Albert | 0.0029 | 0.0032 |

---
**Question 3-2:** Most of these data sets are small with no more than 6000 edges.

**Answer:** We want to clarify that the datasets we used are widely used on benchmark tasks, which are sufficiently large, for graph-level tasks, such as graph reconstruction, generation, and classification. 

---
**Question 4:** For node and graph classification, authors have not compared their results with the state-of-the-art methods. For e.g., graph classification using HGP-SL (Zhang et al., 2019) gets accuracy of 84.91% and 80.96% on PROTEINS and D&D respectively. The authors achieve an accuracy of 75.39% and 78.39% which is lower.

**Answer:** Our experimental settings are different to HGP-SL, therefore the results are not directly comparable to us. As in lines 320-332 of Appendix C.3, we strictly follow the fair evaluation setting of Errica et al. (2020) while HGP-SL did not.

While we already compared 12 baselines for graph classification in Table 3, we further measure the performance of HGP-SL for the PROTEIN and D&D datasets, under the fair experimental setup. As shown in the table below, our model obviously outperforms HGP-SL.

| | D&D| PROTEINS |
| --- | --- | --- |
| HGP-SL | 74.25 $\pm$ 1.48 | 71.91 $\pm$ 1.58 |
| HyperDrop (Ours) | **78.74** $\pm$ 0.68 | **75.30** $\pm$ 0.45 |
| HyperDrop+GMT (Ours) | **78.39** $\pm$ 0.33 | **75.39** $\pm$ 0.26 |

---
**Question 5:** Similarly, node classification using GCN-LPA (Wang et al., 2020) achieves an accuracy of 88.5% and 78.7% on CORA and Citeseer respectively. HyperDrop achieves a performance of around 75-80% and 65-70% respectively. Please comment on the difference in numbers between state-of-the-art and paper's performance.

**Answer:** This is a misunderstanding about the purpose of the node classification tasks. In Figure 10, we do not target to achieve state-of-the-art performance on the node classification tasks, but we aim to alleviate the oversmoothing issues and further shows our method’s applicability toward node-level tasks, which was not possible with conventional node-level pooling methods since they inevitably drop or collapse nodes. Furthermore, the suggested GCN-LPA can be orthogonally used with our work, and thus not directly comparable.

---
**Question 6:** Further, for edge and graph reconstruction, the authors should show performance comparison with approaches such as Structural Deep Network Embedding (SDNE) (Wang et al., 2016) which has shown to perform well on this task.

**Answer:** This is a critical misunderstanding about our reconstruction tasks. The suggested SDNE (Wang et al., 2016) compresses the size of the features of the nodes, while we compress the number of nodes in the given graph. Also, the mentioned work proposes to embed nodes on the one given graph with the reconstruction of it, while we aim to reconstruct the many graphs which are even unseen during training. Therefore, SDNE is not applicable to our reconstruction tasks.

---
**References**
* Berge. Graphs and hypergraphs. 1973.
* Scheinerman & Ullman. Fractional graph theory: a rational approach to the theory of graphs. Courier Corporation, 2011.
* Leskovec et al. Graphs over time: densification laws, shrinking diameters and possible explanations. KDD 2005.
* Errica et al. A fair comparison of graph neural networks for graph classification. ICLR 2020.
* Zhang et al. Hierarchical Graph Pooling with Structure Learning. arXiv 2019.
* Wang et al. Unifying Graph Convolutional Neural Networks and Label Propagation. arXiv 2020.
* Wang et al. Structural Deep Network Embedding. KDD 2016.

 We thank you for your time and effort in reviewing our paper. We sincerely appreciate your positive comments that our approach is clearly presented, easy to follow, and achieves good results, suggesting significant potential in better incorporating edge information within current GNN models. We initially address all your concerns below:

---
**Question 1-1:** The novelty of the approach seems limited, and appears to be somewhat overstated. In particular, the related works mentioned in the paper seem highly similar in essence to the separate components (pooling, DHT) being proposed, and the differences raised (e.g., lines 106-107) do not come across as conceptually substantial.

**Answer:** This is a critical misunderstanding of the contribution of our work. Regarding Dual Hypergraph Transformation (DHT), we do not claim that we have invented or discovered hypergraph duality, which is a well-known concept in graph theory (Berge, 1973; Scheinerman & Ullman, 2011). Our novelty is rather on the proposal of Dual Hypergraph Transformation for **edge representation learning**, which allows to apply conventional message passing and graph pooling algorithms for node representation learning, to edges. 

Applying well-known message passing and graph pooling methods for edge representation learning, does not show the lack of originality of our work, but is rather the main strength of our method since it means that we do not need to reinvent the wheels for edge representation learning. Previous edge representation learning methods do not allow to treat the edges in the same manner as the nodes, and thus cannot utilize existing message passing and pooling methods for nodes.

Also, while our pooling, HyperCluster and HyperDrop, can be seen as similar to existing graph pooling methods, their results are completely different, since ours can cluster edges or identify task-relevant edges, which is not possible with the conventional pooling methods. Our work analyzed the effectiveness of edge pooling coming from the lossless compression of node information, and we believe that this is an important contribution.

---
**Question 1-2:** In my view, the main strength of this paper lies in highlighting the value of edge information, and performing a thorough empirical study toward showing this point, and arguing how this approach manages to bridge this gap. Therefore, I suggest that the authors emphasize their arguments more, and develop their work more along this direction.

**Answer:** Thank you for thoughtfully acknowledging the strengths of our work as highlighting the value of edge information and performing thorough empirical studies. However, we strongly believe that the main strength of our work is the proposal of Dual Hypergraph Transformation (DHT) for edge representation learning, which transforms edges to nodes to realize existing message passing and pooling for nodes to edges. Our proposed methods are not only powerful as verified by the extensive experiments on a variety of graph-related tasks as you said, but also are surprisingly simple and universal, as they eliminate the need to design special message passing or pooling methods specifically tailored for edges. However, as suggested, we will emphasize the value of learning edge representations more in the subsequent revision.

---
**Question 2-1:** The EHGNN framework does not jointly capture both node and edge information. This model then suffers in terms of capturing node information, which then motivates the use of a parallel second model for standard node representation learning.

**Answer:** This is a critical misunderstanding of our learning scheme, as the nodes and edges are jointly learned with GNN and EHGNN respectively in our method, which does not suffer in capturing node information. To be more specific, the representations of both nodes and edges are simultaneously used to make predictions corresponding to their specific down-stream tasks, thus, when performing back-propagation, the weights of GNN and EHGNN are jointly updated considering both nodes and edges at once.

---
**Question 2-2:** EHGNN successfully captures edge information in its representations, and in doing so vastly outperforms standard GNNs which only implicitly handle edges, but this is not surprising, as the model is explicitly modeled for this setting.

**Answer:** As you said, the limitation of standard GNNs is to only implicitly handle edges, and we found such learning schemes are highly suboptimal in substantial graph-level tasks. Thus, while the outperforming performances of our EHGNN against standard GNNs could be expected, it is a meaningful and novel contribution to point out and confirm the importance of learning explicit edge representations for holistic graph-level representation, which was largely overlooked.

---
**Question 2-3:** This framework is not conceptually more powerful than existing models, as it simply applies standard GNNs on a dual graph, and this limits its significance.

**Answer:** This is a critical misunderstanding about the contribution of our work. Exploiting hypergraph duality for edge representation learning is **not a straightforward idea**since none of the existing works have done so. It indeed is conceptually very simple to understand, but this does not automatically mean that coming up with the idea is simple. Also, the simplicity of our method does not limit its significance, but rather further expand the versatility of our work, since it enables the realization of any off-the-shelf message passing and pooling for nodes to edges, and eliminates the need to design special message passing or pooling methods specifically tailored for edges.

---
**Question 2-4:** A more balanced presentation of the results, framing EHGNN more in terms of a means of highlighting the current gaps in edge representation learning, and less so as a standalone model, should be considered.

**Answer:** Thank you for your suggestion. Although we have highlighted the necessity and the significance of learning accurate edge representations with the example of molecules (Figure 1) and from the results of various tasks, as suggested, we will emphasize more on the importance of learning edges for graph-level representations in the subsequent revision. 
However, we strongly believe that the learning scheme of EHGNN, which realizes message passing and pooling of edges using the dual hypergraphs, is also valuable as we can apply existing methods designed for nodes, to edges, with a simple transformation.

---
**Question 3-1:** I find that the tasks in the experimental section should be defined within the main body of the paper, as I had to search for these definitions and infer them from the supplementary material to fully understand the results and the discussions. 

**Answer:** We apologize for the inconvenience. Due to the space limits, we moved the specific details to the supplementary file, however we will add more details in the main paper for the subsequent revision.

---
**Question 3-2:** Concepts such as the compression ratio in that section should also be made clearer and be presented more carefully.

**Answer:** Compression ratio denotes the **ratio of the number of nodes (edges) in the compressed graphs to the number of nodes (edges) in the original graph**, which is equal to the pooling ratio. Intuitively, a compression ratio close to 0 means the number of edges after pooling is small. We will clarify this definition in the subsequent revision.

---
**References**
* Berge. Graphs and hypergraphs. 1973.
* Scheinerman & Ullman. Fractional graph theory: a rational approach to the theory of graphs. Courier Corporation, 2011.


This paper propose Edge HyperGraph Neural Network to learn edge representation. Given an input graph, it first perform a dual hyper-graph transformation, which convert a node in the original graph to an edge in the transformed graph; then it run standard GNN on the transformed graph. The proposed method empirically work well on many graph learning tasks, ranging from edge reconstruction, graph reconstruction to node classification.  I am not very impressed with the novelty of this paper. To me, the net effect of EHGNN is simply running standard GNN on a transformed line graph [1], which has been done in the past (for example see [2]). In particular:

 - Quote 197-198 in the main text: "Here N (e; M^T) is the neighboring node set of the node e in G∗, which we obtain using the incidence matrix M^T of G∗." and also line 42-43 in the appendix "this is because two edges are said to be adjacent if they share the same node". It seems to me that two nodes in the hyper-graph are connected if the corresponding edges in the original graph are incident to the same node. If this is the case, then the EHGNN is simply the "line graph neural network"[2, Section II.C] under a fancy name. 

If the method itself is not novel, judging the significance of the empirical results are not meaningful. Moreover,

 - The example in the introduction (figure 1) is simply not scientifically correct, since Tylenol would have two extra hydrogen atoms and is therefore different from NAPQI even from the perspective of regular GNN. In general, molecular propriety does not depend on the edges (whether it is single bond or double bond) but rather depend on geometries, despite the fact that traditionally chemists use different edges (bond) for intuitive understanding purposes. I understand the author here is just trying to make a point about the importance of edge representation, but please use a valid example to motivate the method.

[1] Chris Godsil and Gordon Royle. 2001. Algebraic Graph Theory. Springer

[2] Cai et al. Line Graph Neural Networks for Link Prediction, 2020 Societal Impact: There is no significant negative societal impact associated with this research.

This paper focuses on learning accurate edge representations for tasks such as graph reconstruction and generation where the importance of discriminative edge representations is often being neglected in most previous research. The authors introduce a novel edge representation learning framework based on the proposed hypergraph transformation which enables message passing over edges. Besides, the authors propose two pooling methods to cluster and drop edges to obtain holistic graph-level representations. They have conducted extensive experiments on multiple benchmarks to validate the effectiveness of the proposed framework and pooling methods on several important graph-based tasks.   This paper has a good motivation and focuses on an interesting problem. Intuitively, accurate edge representations may be as effective as node representations for graph-based tasks. Existing works mostly pay attention to node representation learning and how to learn useful edge representation is under-explored. The authors propose to transform normal graphs to hypergraph, so as to fulfill message passing among edges. In terms of the originality and novelty, this proposed idea is good. 
However, it is a little disappointing that there are not many technical contributions in this paper. The proposed idea is simple and many researchers can come up with such an idea. Basically, by modeling the edges as another type of nodes, the original graph can be seen as a bipartite graph, and it is common to transform bipartite graphs to hypergraphs in a wide range of graph-based tasks such as recommendation. 
When performing the message-passing among edges (nodes in the hypergraph), the proposed method needs to look up the adjacency matrix to find neighboring nodes. The process is similar to that of hypergraph neural networks which have been proposed and studied several years ago. The authors claimed that “Although acquiring the neighboring node set from the incidence matrix costs O(m), compared to using the adjacency matrix which costs O(n), this cost can be ignored by using the sparse implementation of DHT explained above” in page 5. Actually, both the hypergraph incident matrix and the graph adjacency matrix can have sparse implementations. This cannot be the advantage of the proposed framework.
As for the pooling methods, it is not clear what the advantages of HyperCluster and HyperDrop when compared with methods such as DiffPool. It looks like that these two pooling methods cannot be trained in an end-to-end way, which is the advantage of DiffPool. Besides, these two pooling methods are intuitively designed. No new ideas are provided in this part.
The quality of the presentation is good and this paper is easy to follow. However, the experimental results are insignificant. Only small improvements are observed, which would question the necessity of applying the proposed framework to graph-based tasks.
 The number of edges is usually hundreds of times the number of nodes in real-world network. Learning edge representations requires far more memory compared with learning node representations. The authors should address this problem. Whether the hyperCluster and Hyperdrop can be performed before representation learning to tackle the memory issue?

This paper introduces an edge representation learning framework using Dual Hypergraph Transformation which transforms input graph to hypergraph and uses graph convolution network on the hypergraph to obtain edge representations.
They further introduce Hypercluster and Hyperdrop functions as pooling to get graph representations for global tasks such as graph classification.
Message passing in hypergraphs ensures that information of edges in the original graphs is not lost.  Overall the approach is simple and easy to implement.

However, I have the following concerns:
1. Novelty: Converting an input graph to a hypergraph to get edge representations is a known approach. The authors combine this with graph convolution network which is also common in graph representation learning. This limits the novelty of the approach. Further, HyperCluster and HyperDrop are also simple implementations of clustering and dropping nodes.
2. Experiments - Dataset Size and Time: The authors have shown results on several tasks including edge and graph reconstruction, graph generation and graph and node classification. However, most of these data sets are small with no more than 6000 edges. As they are changing to edge space, the authors should show how the model performs on big data sets as well as the time taken in comparison to node based approaches to showcase the utility of the approach. 
3. Experiments - Baselines and Performance: For node and graph classification, authors have not compared their results with the state-of-the-art methods. For e.g., graph classification using HGP-SL (https://arxiv.org/pdf/1911.05954v3.pdf) gets accuracy of 84.91% and 80.96% on PROTEINS and D&D respectively. The authors achieve an accuracy of 75.39% and 78.39% which is lower. Similarly, node classification using GCN-LPA (https://arxiv.org/pdf/2002.06755v1.pdf) achieves accuracy of 88.5% and 78.7% on CORA and Citeseer respectively. HyperDrop achieves a performance of around 75-80% and 65-70% respectively. Please comment on the difference in numbers between state-of-the-art and paper's performance. Further, for edge and graph reconstruction, the authors should show performance comparison with approaches such as Structural Deep Network Embedding (SDNE) which has shown to perform well on this task. Yes

The paper introduces a framework, EHGNN, through which edge representations can explicitly be learned via a combination of standard message passing and a dual hypergraph transform (DHT). More concretely, DHT converts graph edges into nodes and graph nodes into hyper-edges, yielding a dual graph over which standard message passing techniques are subsequently applied to yield edge representations. Furthermore, the paper proposes two new edge-based pooling mechanisms, namely HyperCluster (a clustering-based pooling approach), and HyperDrop (which selects a set of hyper-edges to be eliminated, and thus only preserves the most informative hyper-edges), to combine edge information and obtain graph-level information.

In addition to EGGNN, the paper presents related work on edge-aware node aggregation GNNs, as well as similar pooling mechanisms, and highlights the importance of explicitly factoring in edge information and representing this with dedicated embeddings. Moreover, the work presents a comprehensive empirical study of EHGNN, both in isolation and in combination with node representation GNNs, on several graph representation learning tasks (e.g. graph reconstruction, graph classification, graph generation) and a large set of standard benchmarks, and shows that the model achieves substantial performance improvements, which can largely be attributed to its explicit representation of edge structures.  The paper proposes a simple method to explicitly represent edge information which builds on the literature in message passing GNNs and leverages an intuitive graph duality structure. This approach is clearly presented, easy to follow, and achieves good results, suggesting significant potential in better incorporating edge information within current GNN models. Moreover, the paper conducts a thorough empirical analysis, and convincingly argues for the value of edge information in the field of graph representation learning. However, the paper has some key limitations that need to be addressed.

First, the novelty of the approach seems limited, and appears to be somewhat overstated. In particular, the related works mentioned in the paper seem highly similar in essence to the separate components (pooling, DHT) being proposed, and the differences raised (e.g., lines 106-107) do not come across as conceptually substantial.  In my view, the main strength of this paper lies in highlighting the value of edge information, and performing a thorough empirical study toward showing this point, and arguing how this approach manages to bridge this gap. Therefore, I suggest that the authors emphasize their arguments more, and develop their work more along this direction.

Second, the EHGNN framework, as the authors acknowledge, does not jointly capture both node and edge information. Indeed, EHGNN successfully captures edge information in its representations, and in doing so vastly outperforms standard GNNs which only implicitly handle edges, but (1) this is not surprising, as the model is explicitly modelled for this setting, and, more importantly, (2) this model then suffers in terms of capturing node information, which then motivates the use of a parallel second model for standard node representation learning. Therefore, this framework is not conceptually more powerful than existing models, as it simply applies these models on a dual graph, and this limits its significance. Hence, a more balanced presentation of the results, framing EHGNN more in terms of a means of highlighting the current gaps in edge representation learning, and less so as a standalone model, should be considered. I agree with the authors' comment in the supplementary material that a joint model is an interesting avenue for future work, but also think that this view should be better reflected in the main body of the paper, and that such a model is what is ideally needed to make this paper more conceptually significant.

Finally, I have some minor comments about the writing of the paper. For example, I find that the tasks in the experimental section should be defined within the main body of the paper, as I had to search for these definitions and infer them from the supplementary material to fully understand the results and the discussions. Moreover, concepts such as the compression ratio in that section should also be made clearer and be presented more carefully. 

All in all, the paper makes a thorough empirical study to confirm intuitions about the importance of edge representations within graph representation learning. However, its proposed approach, EHGNN, is not sufficiently novel, and fails to properly capture node information, which current GNNs do well. This limits its applicability without combining with a standard GNN, and makes that the model itself does not provide any conceptual improvement over existing models. Hence, this paper does not currently make sufficient contributions in my opinion to motivate acceptance.  The authors have acknowledged the limitation of EHGNN in terms of capturing node information, and mention solving this as an avenue for future work in the supplementary material. However, their overall presentation of EHGNN can be improved to better reflect this limitation, in line with my comments in my main review.

Meta-review: This paper proposes edge representation learning framework in graphs based on a hyper graph transformation. The reviewers agreed that the problem motivation has some merits. However, all the reviewers had hard time discerning what’s the novelty of the proposed work, and the significance of the contribution. In future submission, it would help to have a better description on the novelty of the significance of the contribution, and contrast it with existing work more clearly. 



Decision: accept