Paper: From Graphs to Hypergraphs: Hypergraph Projection and its Reconstruction (ICLR-2024)

Reviewer comment (Experiments):
The paper lacks a comparison of their method with several other works in the domain. I have listed them in the Questions section.

Author response:
**Weakness 1**

Please see our answers to Question 5 and 6.

Reviewer comment (Theory):
Conditions stated in Theorem 1 are explained as necessary conditions to reconstruct a hypergraph from a given projection using the max-clique algorithm. But are these conditions sufficient? Is there any theoretical justification for that?

Author response:
**Question 1** 

Conditions stated in Theorem 1 are stated as both necessary and sufficient conditions, as indicated by the “if and only if” in Theorem 1. If the reviewer is mainly concerned about the text below Theorem 1 that explains its significance, we have added the phrase ``necessary and sufficient” to avoid confusion. We have also updated our proof in Appendix B.1 to highlight that both directions are true.

Reviewer comment (Experiments):
The clique-sampler tries to sample for as many as possible hyperedges. Will it not affect the size of hyperedges being predicted?

Author response:
**Question 2**

The clique-sampler indeed samples as many hyperedges as possible since it needs to maximize its utility objective under the budget constraint.  We design the sampling mechanism, however, to positively affect the size of hyperedges, i.e., making their sizes diverse and in line with the distribution in the training hypergraph.  This is accomplished by our Algo.1: Appendix F.2.1 elaborates on this, with additional numerical results obtained on DBLP; Table 7 is also very helpful as it examines more properties of reconstructions. Below is a short summary:

In each iteration, Algo. 1 tends to sample cliques from a different column of Figure 4(a)’s heatmap.  Cliques of the same size have diminishing sampling gain, due to the overlapping of $\mathcal{E}_{n,k}$’s that have the same $k$. Meanwhile, cliques of different sizes remain unaffected in each sampling iteration. This inherent exclusivity encourages more diverse hyperedge sizes in sampling, which reduces the chance of a skewed size distribution in the reconstruction result. Of course, ultimately the sampling is still guided by the distribution of hyperedges in the training hypergraph.

Reviewer comment (Evaluation):
The performance of the proposed method is not very different from the baselines on hypergraphs where the average hyperedge size is large. Is there any justification for that? Also, from the predicted hyperedges, is it possible to provide a stratified evaluation to understand how the proposed method works on large vs small hyperedges?

Author response:
**Question 3 & Weakness 3**

We have added a new section, Appendix F.3, to answer this question systematically through a linear regression on the performance gap.  

As a short summary, we found that (1) our advantage over baselines is most prominent on dense hypergraphs, because dense hypergraphs are usually hardest to reconstruct; (2) our advantage is relatively less prominent on datasets with larger hyperedges, because baselines like max cliques, clique covering, and Bayesian-MDL naturally favor large hyperedge in reconstruction. In these cases, the baseline’s assumption is aligned with the dataset property.
Please read F.3 for more details.

Reviewer comment (Theory):
In the Introduction section, whether the hyperedges are observable or not actually depends on the objective of experiments conducted on the system. For example, protein-protein, gene-gene interactions have semantics defined for pairwise interactions where protein complexes are inherently super-dyadic relations of proteins, and experiments observe them in the protein complex form [1].

Author response:
**Question 4 & Question 6a**  

As introduced in Section 5.4 and elaborated in Appendix F.4, a realistic use case we have demonstrated is multiprotein complex reconstruction. This is a widely-studied application that naturally fits the motivation for hypergraph reconstruction, since it is a case where some of the most readily available measurement technologies can only record pairwise interactions in a system that is fundamentally composed of hyperedges.  In particular, we understand the reviewer’s point that it can be an experimental choice as to whether one should detect protein interactions as pairs or groups, but as we elaborate in Appendix F.4, pair-based detection methods such as Y2H are much more accessible and faster than group-based detection methods such as TAP-MS. As a result, a large amount of the data so far generated from protein interaction studies have come from Y2H screenings. This often leads to a realistic demand for researchers to recover groups (hyperedges) from pair detection results. Please refer to Appendix F.4 for more details.

Reviewer comment (Novelty):
The problem of hyperedge prediction is actually relevant. The hyperedge prediction methods consume a hypergraph as input, and the right way to see it is every graph is also a hypergraph (more precisely, 2-uniform hypergraph), and the methods [3] where a set of candidate hyperedges is not used can identify underlying hyperedges. Especially when you are using the sample hypergraphs from a given domain, methods like HPRA can generate new hyperedges using the known hyperedge degree distribution.

Author response:
**Question 6c**

We agree with the reviewer that the suggested HPRA method is relevant, and we have added it as a baseline for comparison (see the updated Table 3). 

HPRA is a novel method for hyperedge prediction, and it addresses the huge search space of hyperedges very well. It also has the advantage of being lightweight, interpretable, and requiring no tuning at all. As Table 3 shows, the HPRA is a strong baseline that achieves competitive performance among other baselines. However, we still observe a gap in its performance with our proposed method for hypergraph reconstruction, for two reasons:

-  HPRA does not reconstruct all hyperedges from scratch. Instead, it requires at least 80% - 90 % of the hyperedges to already exist in the test hypergraph (See “Evaluation of HPRA” in Section 4 of its paper, where they use 5-fold or 10-fold split). When we treat edges in the projected graph as 2-uniform hyperedges, this violates the underlying assumption that the hypergraph to predict (reconstruct) is ``almost’’ complete, leading to two issues:

    -  Forcing HPRA to reconstruct from scratch causes its errors in the early stage of reconstruction to quickly accumulate. Notice that HPRA’s prediction of each new hyperedge depends on its previous predictions, i.e., it is self-regressing.

    - Treating the hypergraph as a 2-uniform graph significantly alters the assumed hypergraph structure. In other words, it essentially degrades the hypergraph predictions into samplings from the landing probabilities of random walks on the projected graph.

- HPRA uses the training hypergraph (“observed hyperedges”) in a limited way. It mainly learns about the size distribution of hyperedges from the training hypergraph. In comparison, our method not only learns about size distributions (via $\rho(n,k)$’s), but more importantly learns from the fine-grained statistics about hyperedges’ locations, as well as the location’s intricate correlation with their surrounding structures.

Reviewer comment (Novelty):
The approaches outlined in the paper are somewhat basic (which itself is not a weakness, as the problem is novel and the contributions fundamental). What are potential future directions for the design of possibly more sophisticated methods?

Author response:
We agree with the reviewer to discuss more about future directions. We have followed the advice to add an entire section, “More Discussions on Limitations and Future Work”, in Appendix G.

One direction is to consider propagating the error signal (e.g., loss gradient) from the hyperedge classifier back to the clique sampler. We have appended an additional section, Appendix G, to formulate the challenge in this interesting future direction, as well as a conjectured solution.

Another direction we may consider is the usage of attributes in the learning-based framework. Our current method targets the most difficult version of the reconstruction problem: we don’t assume nodes or edges to have attributes, and the reconstruction is purely based on leveraging structural information in the projected graph. It would be interesting to think about how we can effectively integrate node attributes into the picture. This is nontrivial though because the projected graph has special clique structures (and with max cliques that we have preprocessed). Therefore, how to conduct graph learning most effectively on these type of clique structures could be something worth exploring.

Reviewer comment (Theory):
What other types of information could be taken into account to improve guarantees for hypergraph reconstruction?

Author response:
Another direction we may consider is the usage of attributes in the learning-based framework. Our current method targets the most difficult version of the reconstruction problem: we don’t assume nodes or edges to have attributes, and the reconstruction is purely based on leveraging structural information in the projected graph. It would be interesting to think about how we can effectively integrate node attributes into the picture. This is nontrivial though because the projected graph has special clique structures (and with max cliques that we have preprocessed). Therefore, how to conduct graph learning most effectively on these type of clique structures could be something worth exploring.

Reviewer comment (Experiments):
Are there any natural strategies you'd expect to perform better than looking at maximal cliques, for the case of no side information?

Author response:
We have counted on the “(Edge) Clique Covering” baseline to outperform the “Max Cliques” baseline. As introduced in Appendix C.2, “Clique Covering” is a classical algorithmic problem that aims to use the least number of cliques to cover all the edges of the projected graph.  This is a very natural baseline for reconstructing hyperedges. This is also the only baseline that strictly enforces the principle of parsimony --  a popular rule of thumb in science. Therefore, we thought it should perform better than max cliques.
In practice, our experimental results in Table 3 show that “Clique Covering” does outperform “Max Cliques” on 5 out of the 8 datasets.  Meanwhile, however, it also performs very badly on Enron, P.School, and H. School. We conclude that the principle of parsimony is useful in some cases of reconstruction, but not (even close to) good in many others. In fact, we think there may not exist an (unsupervised) strategy that is universally applicable to reconstructing any real-world hypergraphs, which necessitates the introduction of a learning-based reconstruction method.

Reviewer comment (Novelty):
I'm uncertain about the necessity of addressing this problem.

Author response:
First, a foundational motivation to study the problem of hypergraph reconstruction is that we have very limited understanding about the consequences of hypergraph projection, a process that implicitly happens very frequently in network analysis. This is a source of concern both conceptually and in applications because we don’t know what we miss when we drastically simplify complex systems involving hypergraphs in this way. We address this question in two fundamental ways: (i) We begin by presenting several theorems to mathematically characterize the consequences of hypergraph projection in general terms (we consider these to be some of our most important findings); and (ii) we study the problem that is the inverse of hypergraph projection, which is hypergraph reconstruction. 
 
Regarding the actual hypergraph reconstruction problem, it has the following benefits:

- The reconstructed hypergraphs themselves are very useful data representations, because they crucially tell us which group of nodes in a projected graph are really interacting with each other at a time. Therefore, the reconstructed hypergraphs can facilitate scientific analysis that are infeasible or less accurate in projected graphs. Examples include analysis of protein complexes and social interactions; in both of these cases, as we discuss in the paper, natural ways of measuring the system produce graph representations for what are in fact hypergraph structures.
-  The reconstructed hypergraphs also facilitate downstream tasks, though they do not exist for the sole purpose of improving any one specific downstream task. We demonstrate this through the link prediction task. Please read more in our answer to “Weakness 3 & Question 1”.
-  Even the instances of hypergraph reconstruction that yield poor performance results have value, since they help us identify the classes of hypergraphs for which projection risks causing the greatest amount of information loss.  For example, one interesting finding in our experiment is that hypergraphs with large hyperedges are relatively safer to project, while it is more important to avoid projecting hypergraphs with high node degrees.

Reviewer comment (Experiments):
It would be advantageous to incorporate supplementary experiments to illustrate the benefits of utilizing reconstructed hypergraphs. Although the experiments show the proposed method's strong performance in reconstructing hypergraphs, its impact on downstream tasks remains unclear.

Author response:
We have followed the reviewer’s suggestion to add another use case in Appendix F.6, showing reconstructed hypergraph’s benefit in the classical task of node clustering. In particular, we show that the hypergraph reconstruction crucially recovers much information about the higher-order cluster structures in a social interaction network.
We believe that the benefits of reconstructed hypergraph in downstream tasks have now been well substantiated by the three concrete use cases: node ranking (F.4), link prediction (F.5), and node clustering (F.6).

Reviewer comment (Experiments):
In experiments for link prediction (In F.4), the authors append several structural features of hyperedges to the final link embeddings. However, in cases of projected graphs, additional structural features cannot be used. Is it a fair comparision?

Author response:
The improvement on link prediction in Appendix F.4 does originate from the use of additional structural information. However, this is exactly what we have wanted to demonstrate. Without a hypergraph reconstruction method, the classifier for link prediction is unable to use higher-order structural features in the (testing) projected graph. This limitation exists despite the availability of a training hypergraph together as input. This is because there is no mapping between nodes in the testing projected graph and nodes in the training hypergraph. Therefore, we can’t use training hypergraph to construct structural features for nodes in the tested projected graph.

Hypergraph reconstruction, on the other hand, creates a mechanism to use information in the training hypergraph when the task is defined on the testing projected graph (by first reconstructing the testing hypergraph).  This is an important sense in which hypergraph reconstruction — by providing access to an inferred hypergraph structure — unlocks the potential for using available side information that is not directly compatible with graph structures. 

We also hope to emphasize that we do not intend to compete against the current state-of-the-art method for link prediction. The main purpose of this experiment is to show that the reconstructed hypergraph contains structural information that is useful and handy for many analysis tasks, including link prediction.

Reviewer comment (Evaluation):
Could the performance improvement in experiments on link prediction have originated from the use of additional structural information?

Author response:
The improvement on link prediction in Appendix F.4 does originate from the use of additional structural information. However, this is exactly what we have wanted to demonstrate. Without a hypergraph reconstruction method, the classifier for link prediction is unable to use higher-order structural features in the (testing) projected graph. This limitation exists despite the availability of a training hypergraph together as input. This is because there is no mapping between nodes in the testing projected graph and nodes in the training hypergraph. Therefore, we can’t use training hypergraph to construct structural features for nodes in the tested projected graph.

Hypergraph reconstruction, on the other hand, creates a mechanism to use information in the training hypergraph when the task is defined on the testing projected graph (by first reconstructing the testing hypergraph).  This is an important sense in which hypergraph reconstruction — by providing access to an inferred hypergraph structure — unlocks the potential for using available side information that is not directly compatible with graph structures. 

We also hope to emphasize that we do not intend to compete against the current state-of-the-art method for link prediction. The main purpose of this experiment is to show that the reconstructed hypergraph contains structural information that is useful and handy for many analysis tasks, including link prediction.

Reviewer comment (Experiments):
Could the authors demonstrate the benefits of reconstructed hypergraphs through additional experiments?

Author response:
We have followed the reviewer’s suggestion to add another use case in Appendix F.6, showing reconstructed hypergraph’s benefit in the classical task of node clustering. In particular, we show that the hypergraph reconstruction crucially recovers much information about the higher-order cluster structures in a social interaction network.
We believe that the benefits of reconstructed hypergraph in downstream tasks have now been well substantiated by the three concrete use cases: node ranking (F.4), link prediction (F.5), and node clustering (F.6).

Reviewer comment (Novelty):
The graph representations (e.g., clique expansions) and hypergraph representations have been compared in many contexts, which are largely ignored in the paper (see [R1]-[R3]).  
- [R1] The why, how, and when of representations for complex systems  
- [R2] How Much and When Do We Need Higher-order Information in Hypergraphs? A Case Study on Hyperedge Prediction  
- [R3] HNHN: Hypergraph Networks with Hyperedge Neurons

Author response:
We have followed the suggestion to add an additional subsection, Appendix C.1, which includes more extensive discussion on references R1-R3. We appreciate these references as they also help highlight the importance of our studied topic.

Reviewer comment (Evaluation):
The comparison between the structures of the original hypergraphs and the reconstructed ones is limited, relying on basic statistical metrics. It's important to note that hypergraph structures can exhibit variations even when their basic statistics align. To conduct a more comprehensive comparison, additional measures and analytical tools should be considered [R4].  
- [R4] Mining of Real-world Hypergraphs: Patterns, Tools, and Generators

Author response:
We have followed the suggestion to add an additional subsection, Appendix F.2.2, to our manuscript, which reports numerical results on some of the more advanced statistics, including simplicial closure, degree distribution, singular-value distribution, density, and diameter. We will keep working on expanding these numerical comparisons.