Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes novel methods for few-shot KG completion. They identify two issues with existing methods for this task -- (a) They learn entity-level information from local nbr aggregators. (the paper jointly takes into account the nbr of head and tail entity for triple context) (b) They learn relation-level information using a sequential model (while the sequentiality assumption is invalid -- the paper use transformers instead). The authors proposed HiRe, which takes into account triplet-level interactions, context-level interactions, and entity-level information for query (h,r,?). MAML based training strategy used to train the model. The model shows improved performance on 2 benchmark datasets - Nell-One and Wiki-One. The ablation study demonstrates the importance of the key components of the model, where transformer-based MRL outshined.
strength_and_weaknesses: The paper address an important problem in KBC. It identifies valid issues in existing methods. However:
1. Paper writing can be improved. Certain sections are hard to understand (especially sections 4.2/4.3/4.4). Since these sections form the core of the paper it is imperative to write them very clearly. 
Adding details of MAML training startegy, MSA and SAB in appendix should be helpful.
2. \bigoplus used before definition
3. Some comments on the scalability of the model will be insightful.
4. Why did the authors not look at (?, r, t_j) queries along with (h_j, r, ?) and report the mean (the standard way of evaluation in KBC).
5. Definition 2: Consider adding an \exist r.
6. Some understanding of how the models perform on conventional KG completion datasets (where relations are associated with many more triples) is also important.
7. Authors should use a more competitive version of ComplEx (ComplEx-N3) for comparison. (See Lacroix, Timothée, Nicolas Usunier, and Guillaume Obozinski. "Canonical tensor decomposition for knowledge base completion." International Conference on Machine Learning. PMLR, 2018.)
Also why did the authors choose a translation model (sec 4.3) in place of more competitive KBC models like ComplEx?
clarity,_quality,_novelty_and_reproducibility: 
1. It is imperative to share the code of the model, to be able to reproduce the results.
2. The paper identifies valid gaps/issues in previous techniques and provides sensical first steps to address them.
3. Further see answer to Q2
summary_of_the_review: I feel the paper has interesting contribution but I believe the quality of paper writing needs to be improved.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: - Real-world KGs are incomplete and suffer from the long-tail distribution over the relations.
- Thus the performance (on completion tasks) on the low-frequency relations is poor.
- Prediction of the tail entity given the (head, relation, ?), is considered a few-shot completion problem, in which one could try to learn meta representation for these relations from the limited amount of reference (head, relation, tail) observed.
- In order to learn the meta-relation representation, the authors try to jointly capture three levels of relational information:
  a. Context level b. Triplet level c. Entity level.
- Finally, they propose a meta-learning-based optimization approach to obtain an effective embedding for the tail relation that achieves a few-shot generalization
strength_and_weaknesses: Strength:
1. The authors very succinctly point out the gap in the existing few-shot paradigm, viz, inconsistent sequential assumption, and lack of context awareness. 
2. Their extensive ablation study indicates the necessity of each component of the model, e.g., the set-based transformer used in triplet-level relation capture was justified by introducing simple sum and LSTMs.

Weakness:
 A few places where complicated model choices were not motivated properly. For example, why authors picked the TransE or distance-based framework for entity-level generalization? The pairwise interaction of the triples could have been done using any sort of permutation invariant operation, why SAB was picked?
clarity,_quality,_novelty_and_reproducibility: N/A, see weakness.
summary_of_the_review: The proper correctly identifies the problem with the existing approaches and addresses them with noble techniques. For that reason I feel the paper is a suitable candidate for acceptance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a hierarchical relational learning framework for few-shot KG completion. The authors proposes jointly capturing three levels of relational information for enriching entity and relation embeddings, i.e., entity-level, triplet-level and context-level. Experiments on two benchmark datasets demonstrate the effectiveness of the proposed model.
strength_and_weaknesses: Strength:

1. The idea of using multiple different levels of information for relational learning on KGs is interesting;

2. The paper is well-written and the technical details are basically presented clearly;

3. The experimental results demonstrate significant improvements over baseline methods.

Weaknesses:

1. The novelty of this work is limited;

2. The term "hierarchical" in the title is confusing;

3. Some of the technical details are not clearly presented;

See below for details of weaknesses.
clarity,_quality,_novelty_and_reproducibility: 1. The novelty of this work is limited. The authors propose using three levels of information: entity-level, triplet-level, and context level, which has been extensively explored in the literature (For entity-level and triplet-level, all KG models use those information, of course; for context-level, there has been a lot of work studying the context information of entities and triplets in KGs. All GNN-based methods can be seen as context-aware). The components of multi-head self-attention, contrastive learning, and translation-based KG representation learning (MTransD) are also extensively explored in the literature.

2. The term "hierarchical" in the title is confusing. When I first read the title I thought that the authors somehow construct a hierarchical structure among relations. However, it actually refers to the three-levels of information. It may not be appropriate to call such information "hierarchical", since there is no hierarchy here.

3. When using contrastive learning to construct a false context for a given triplet, are all (r, t) pairs in context corrupted, or only one (r, t) pair is corrupted? If all (r, t) pairs are corrupted, this false sample may be too weak and can be easily identified, but if only one (r, t) pair is corrupted, this context may be too similar to the true one and is hard for the model to distinguish.

4. For the two parts of the loss, i.e., Q_r and L_c, are entity and relation embeddings shared across the two modules, or they have two sets of separate embeddings?

5. For the MTransD part, did the authors try other translational-based methods, such as TransE, TransR, TransH?

6. For the context, the authors only consider one-hop neighbors as the context information, but prior work has shown that more neighbors are helpful to model entities and triplets. It is recommended that the authors try to increase the hop of neighbors to see if the model performance can be further improved.

summary_of_the_review: This paper studied an interesting problem in KG representation learning. The paper is basically well-written and the experimental results are promising. However, the technical novelty is limited, and some of the technical details are not clearly presented. Overall, I think this paper is marginally below the acceptance threshold of ICLR.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper studies the few-shot KG completion problem and proposes a hierarchical relation learning method for this task. The authors propose three levels of relational information to learn and refine the meta-representation of few-shot relations. Specifically, a contrastive learning-based context-level relation learning is proposed to update the entity embeddings, and a transformer-based triplet-level relation learning is proposed to learn the meta-relation representation, and an MTransD is proposed as the score function. The authors conduct experiments on widely used datasets.
strength_and_weaknesses: Strengths:
1. The paper is well-organized and the proposed method is easy to follow.
2. The studied problem is an important research problem for KG completion.

Weaknesses:
1. The proposed method is based on previous works and uses different components to replace the existing ones. The incremental improvement shows the lack of novelty in this work.
2. The motivation for the proposed method is not clear and hard to understand why the proposed design is necessary.
3. The lack of an ablation study to demonstrate the superiority of the MTransD.

clarity,_quality,_novelty_and_reproducibility: Clarity:
1. The claim of “the potential of leveraging context-level relation information” is inaccurate because the existing work aggregates information from the local neighbors.
2. The authors claim the existing works lose critical information. First, I wonder how many triples will have shared information. Second, why the previous methods cannot capture the shared entity? I think the previous methods implicitly encode the shared entity in their embeddings. The claim needs more explanations.
3. The motivation for using the transformer-based meta-relation learner is not clear. Why is the utilization of LSTM based on an unrealistic assumption and what is the impact caused by the insensitive of the size of the reference set and the permutation-invariant? And Why these two properties are essential?
4. The advantages of the proposed MTransD are not clarified. Why not use the existing translation-based methods? And what is the disadvantage of these methods?
5. The ablation study is necessary to demonstrate the advantage of MTransD compared with TransE, RotateE… the related algorithms.

Quality and Novelty:
This work replaces the key components in existing works and proposes an incremental solution to improve the performance of few-shot KG completion. The proposed method is reasonable but lacks novelty.

Reproducibility:
The authors provide the data and code for reproducing their results.
summary_of_the_review: The paper proposes an incremental solution for the few-shot KG completion and lacks novelty to some extent. And some claims and motivations are not clear and need further explanations. Thus I think the quality of this paper is below the standard bar of ICLR and would not like to accept it.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a new method for few shot KGC, i.e., KGC for relations in the long tail. The main idea is to use shared context between head and tail entity neighborhoods in  embedding the triple via contrastive learning. In particular, three capture three levels of relational information: a. Context level b. Triplet level c. Entity level. They call this "hierarchical relational learning", although thats a bit of misnomer.

Main strengths are (1) careful analysis of research gaps in existing works, which result in this solution, (2) extensive ablation study, and (3) impressive experimental results on one of the two datasets; good results on the 2nd.

Main weaknesses are (1) incremental novelty: it appears that most ideas are present in other works, this particular combination is not. (2) name hierarchical learning or hierarchical relation learning is a misnomer -- since it is neither doing hierarchical learning not learning hierarchy of relations. 

On balance, the paper can be accepted. Incremental novelty is considered an issue with most papers these days, since coming up with a new building block has become harder and harder. This paper has several other advantages as discussed above. Three of the reviewers (8,6,5) did not respond to author comments. One reviewer (3) increased their score to 5. Authors have diligently answered most questions. 
justification_for_why_not_higher_score: Incremental novelty
justification_for_why_not_lower_score: Strong results, good analysis.
Decision: Accept: poster