Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose the HyperQuery framework for predicting hyperedges between nodes in hypergraphs. It can also be used to predicted hyperedge types, e.g., in a knowledge hypergraph. Link prediction in hypergraphs is much less studied and more difficult than link prediction in simple graphs. The proposed approach defines an embedding for each hyperedge. These are then aggregated to compute an embedding for each node. They use hypergraph clustering to initialize the embeddings. The authors demonstrate improved empirical results for both knowledge hypergraph completion and hyperedge prediction on several real data sets.
strength_and_weaknesses: Strengths:
- Considers the task of hyperedge prediction, which is more difficult and less well studied than link prediction in simple graphs.
- Improvements in accuracy on both prediction tasks is impressive compared to prior methods, including recently proposed neural hyperedge prediction methods HyperSAGNN and NHP.

Weaknesses:
- Presentation is unclear in many respects. Although the paper is mostly free from grammatical errors, the text and figures are confusing in many aspects. See my questions below.
- Different elements of the proposed HyperQuery framework seem to be chosen and combined in ad-hoc manner that doesn't provide much insight into why they work.
- Novelty of different elements in the HyperQuery framework is unclear. 

Questions:
1. What is Figure 2 supposed to show? On the left pane, I can't tell what the hyperedges are. What are the colored dashed lines supposed to show? Are these the hops in the random walk? What is the right pane supposed to show?
2. What are the quantities \#train, \#valid, and \#test referring to? They don't appear to map to either the number of entities or relations. Are these counts of hyperedges?
3. I don't understand the evaluation in Section 6.1. What counts as a hit for hit\@1 and hit\@3? Is there a way to give partial credit if HyperQuery is given a query for a subset of entities that have a hyperedge? For example, if a hyperedge exists between $(e_1, e_2, e_3, e_4)$ but the query is for $(e_1, e_2, e_3)$, what is the correct answer, and how is it evaluated?

Typos and minor issues:
- Figure 1: Prediect -> Predict
- Section 5.2, first line: varience -> variance
- Placement of Figure 2 on page 3 seems odd. It is not referenced anywhere around that page, and not until Section 4.1 on page 5. Perhaps Figure 2 should be moved after Figure 3, which is referenced much earlier. Similarly, Figure 5 is in a strange location, showing up before Tables 2-4 despite not being referred to until afterwards.
- Inconsistent use of the notation $\mathcal{N}(\cdot)$. $\mathcal{N}(v_i)$ is stated to denote the neighborhood of node $v_i$ (incorrectly denoted as $N(v_i)$ in the Figure 3 caption). Then what is $\mathcal{N}(e)$ for a hyperedge $e$? Is it a union of the neighborhoods for all nodes $v_i \in e$?
clarity,_quality,_novelty_and_reproducibility: - Clarity: Low. Unlike most papers that are poorly presented, this paper is mostly free of grammatical errors and typos, but the descriptions of key components in the text and figures are either missing details or unclear.
- Quality: Medium. Proposed framework and experimental evaluation appear to be technically sound, but I can't say that with high confidence due to the poor presentation.
- Novelty: Medium. There appear to be some novel components in the proposed HyperQuery framework, but the main novelty is in addressing the hyperedge prediction problem, where there is not much existing literature.
- Reproducibility: Medium. Code is provided, although I did not examine it in detail. There are not enough details in the experiment descriptions and supplements for me to reproduce the experiments. I'm not even certain of how the evaluation is being conducted, as indicated by my questions above.

summary_of_the_review: This paper addresses an important task, hyperedge prediction in hypergraphs, that has not been very well studied previously. The proposed HyperQuery framework appears sound and may have some novel components, but I am left confused about many elements of this paper. It is quite strange to me that the text in the paper is grammatically sound but lacks clarity in descriptions. Similarly, the figures look good, but I am not sure what several of them are supposed to show.

I think there is a lot of potential in this paper but would not recommend it for publication at this time. If the presentation can be significantly improved, it could become a high quality paper in the future.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The model first generates natural labels by a clustering algorithm and then performs hyperedge prediction by hypergraph convolution.
strength_and_weaknesses: Strength：
1. The method in this paper shows consistent superiority on experiments with multiple data sets.
2. The authors identified the shortcomings of their previous work and innovatively improved it through simple methods.

Weaknesses：
1. The abstract and the introduction section does not state the background of the task, the motivation of the method and where the innovation lies. Instead, the authors define the hypergraph, knowledge hypergraph, and hypergraph prediction tasks in this section, which I think should be explained in the model section. 
2. The captions for Figures 2 and 3 are too long. The description of the model should be placed in the body of the text.
3. The composition of the pipeline should be further rethought to allow the reader to better understand the model details. 
4. What are equations 1 and 2 for? They are not used in the subsequent objective function Eq. 6.
5. The logic of the article is not clear, making it difficult for the reader to catch the point. The authors should rethink the organization of their paper to better explain their work.
clarity,_quality,_novelty_and_reproducibility: The quality and clarity of the writing needs to be improved. Th originality is good.
summary_of_the_review: This work identified the shortcomings of the previous work and improved it using simple methods. But the writing is not really satisfactory. The author should revise it significantly to make the work more acceptable to the readers.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors study the problem of hyperedge classification in hypergraphs. The authors generalize the idea of GNN to carry out message-passing type methods to alternate between edge and node representations. Experiments are carried out on several datasets with comparison to existing hyperedge classification methods.
strength_and_weaknesses: Strength
1.The authors study an interesting and practical problem to generalize edge classification to hyperedge with many real-world applications.
2.The proposed method is a natural generalization to GNN from graph to hyper-graph with alternating node and edge representation learning.
3.The authors carry out experiments on several datasets with comparison to existing state-of-art methods.

Weakness
1.The authors should provide more details on the empirical evaluation. First, how important hyper-parameters are set like the number of iteration of message-passing k. Also it would be better to have an experiment on the sensitivity of the parameters. Second, for hyper-edge classification, what is the evaluation protocol e.g. how are the negative samples generated and what is positive to negative ratio etc. In Table 4, does HQ the same as HyperQuery. Also, what is the exact definition of the minimax operation and the bilinear operation used?
2.Besides hyper-graph based approaches, the authors should compare the proposed method to graph-based methods as well. A neutral choice would be casting the problem as a graph classification problem by flattening all hyper-edges. Comparison to some simple heuristics like graph structure features like number of edges would also be very helpful. 
3.The authors mention the classification of hyperedge types. Seems no experiment is carried out in section 6? Also in section 6.2, why use 70% for test and only use 20% for training?

clarity,_quality,_novelty_and_reproducibility: The clarity in both methodology and evaluation needs some improvement. The technical contribution and novelty is marginally significant.
summary_of_the_review: The authors studied an interesting problem with a natural generalization of GNN to hypergraph as solution. However, both clarity and empirical needs some improvements.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new framework for higher-order link prediction. The paper creates hyperedges embedding (local information) and combines this information with clustering embedding (global information). Both pieces of information were supposed to be combined using an MLP for the final classification process.

strength_and_weaknesses: The introduction is well-written and stated the main problem. The related work covers several points of topics and papers. The methodology of the experiment section is also strong. It has several baselines, different problems are considered, and an ablation study is applied to check the importance of each stage of the framework. However, there are several weaknesses that must be improved. 

The most important issue is the clarity of the paper. The paper tries to be general, so different approaches can be considered. This can be observed with the use of functions such as AGG, \Omega, and others. However, this generality loses the main focus of the paper, and the framework becomes complex (even though it is simple). So, in general, the three sections explaining the main framework must be improved. 

The use of space can also be improved. For example, most of subsection 4.1 can be resumed in a single paragraph. Figure 2 can be explained in a single paragraph, and figure 1 from the appendix could be incorporated into the paper. You can also add the time complexity of the framework (a small part is mentioned in the appendix). This will make the paper stronger. 

Another problem is reproducibility. Unfortunately, several details are omitted. From the use of the clustering algorithm, to all the hyperparameters used in the framework. 

Minor comments:
Figure 1: it says project instead of paper. 
Figure 1 ("right" part): it says Prediect
Figure 5: use a white background, increase the size of the font for the legend, and improve the caption mentioning that it is for a specific experiment.

clarity,_quality,_novelty_and_reproducibility: The paper is novel. It is a new framework that combines local and global characteristics for higher-order link prediction. With respect to quality, as with every framework, it is a combination of multiple methods. So, it seems fine, and no problem can be observed. However, there are important problems with respect to clarity and reproducibility.

The clarity must be improved before publication. Even though it seems a simple framework based on the figures, it is very complicated to understand the process. Unfortunately, this also affects the reproducibility of the paper (even though the codes are available). Just as an example, the clustering process is based on "state-of-the-art hierarchical partitioning based clustering tools", but the final algorithm is not mentioned in the paper. Similarly, the selection of the number of clusters is supposed to be based on equation 7, but again, it is difficult to figure out the selection given that almost no description is given. Similar problems can be observed in sections 3 and 5. 

The previous problems also affect the reproducibility of the experiments. For example, no details are given about the MLP and the number of clusters for each problem. What is the clustering algorithm (hierarchical is just mentioned)? What are the characteristics of the MLP? How do you select the number of clusters?

summary_of_the_review: The paper is well-written and proposes a new interesting framework. The results are very interesting, and the work could be publishable. However, the current state of the paper must be improved in terms of clarity and reproducibility.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a hyperedge prediction framework. Messages are passed between nodes and hyperedges alternatively and final hyperedge representations are pooled from node embeddings. The main weakness of the paper is that the idea of treating hypergraph as bipartite graph and performing message passing between hyperedges and nodes is not new. Many previous works have by default used this scheme. The presentation in this paper is also bad. There are many typos and notation issues (e.g., the AGG() never appears in Eq.9). The figures are generally too small. The citation format is wrong, with many in-text citations that should be replaced by parenthesis citations. All the above make the paper hard to read. Also, many used datasets miss citing or discussing their original papers.
justification_for_why_not_higher_score: The paper looks like a practice submission by student without careful proofreading. It is far from the average presentation level of ICLR papers.
Decision: Reject