Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This work proposed to use prototype entities to enhance the knowledge base completion methods. Specifically, the KBC methods firstly look for the alternative entities that share the same value with the query entity and relation as the prototype entities. The authors show that the prototypes have closed-form expressions when combining with translation-based KBC models like TransE. The experimental results show it outperforms existing methods. Besides, this work also claims the connections between instance-based learning and rule-based learning. Further it encourages the rethinking of interpretability of rule-based reasoning from the angle of instance-based equivalence.

strengths_and_weaknesses: Strengths:
1. The idea is well-motivated and prototype-based methods have been shown useful for many downstream tasks such as relation extraction etc. It is good to apply to knowledge base completion tasks. 

2. The closed-form solution of looking for prototype entities is interesting under the assumption of translation-based models. 

3. The proposed instance-based learning method can achieve competitive performance and then combining with translation-based methods outperforms the state-of-the-art methods. 


Weakness:
1. The introduction of logical rules as prototype-based inference is vague. One concrete example in Section 5 is not convincing and clearly states the connections or significant differences between prototype-based inference and rule-based reasoning. 
2. The IBL rules should be formally defined in Section 5. 
3. The claim that replacing the rule-based reasoning with instance-based learning is a bit weak from the limited experimental results. 

questions: 1. Can you demonstrate more examples of IBL rules obtained from the proposed methods? It could be better to have detailed comparisons with the rules from previous work. 

2. The assumption is a bit in line with collaborative filtering in the recommendations. Have you thought of any solutions from the perspective of matrix factorization? 

3. Can the prototype-based methods generalize to unseen relations or OOD scenarios? 

limitations: N.A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces a new paradigm for KBC task by exploring instance based learning. They show that most of the 
rule learning models mostly learn instance-based rules and using only those rules they can still perform close to state of the art. 
Paper also proposes ways to finding prototypes and propose models on how IBL can be combined with other approaches. 
Experimental results show that its competent on several benchmarks though it doesn't outperform existing systems. 
Using existing embedding methods to select prototype seems novel and builds on top of the existing literature. 


Any comments on the practical scenarios with large KBs and long skewed entities and relationships? Is it 
is to find a prototype entity to answer the queries for relatively less represented entity types and relationships?


strengths_and_weaknesses: Strengths: 
Bringing IBL to KBC space and see how it performs with respect to current systems

weaknesses: 
Experimental results are not on par with other benchmarks. 
Not sure about how it can extend to practical use cases with long tail entities.
questions: Please see summary
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes Instance-Based Learning for translational embeddings, and evaluates its performance in knowledge-base construction tasks. The paper first defines the general concept of prototyping in translational embedding training using relational tuples, then discuss methods to train with both translational loss and IBL loss, which is through a linear combination of their respective loss functions controlled using a hyper-parameter weight alpha. The paper also presents experimental results on 4 KBC datasets, which largely surpass performances of all but 1 graph network based approach, and does so with much less computational complexity. The paper also interestingly present the finding that it seems like rule-based reasoning doesn't necessarily always find semantically relevant rules.
strengths_and_weaknesses: Strength:
1. The paper is clearly written and presented. The mathematically formulation is well done and easy to understand.
2. The paper proposes a method with relatively low-computational cost (compared to graph neural networks) to complement existing translational embedding learning method.
3. The paper shows that the proposed method is highly competitive in all experimented datasets.

Weakness:
I could not find major weaknesses in this paper. I think it would be better to include numbers of the authors' implementation of TransE, TransR, etc. (without IBE loss) in the evaluation table as an ablation study perhaps. But all in all, IBLE itself is already an interesting approach.

questions: 1. One could probably argue, that prototyping is potentially a GNN approach with limited depth (1). Recent work on GNN for KBC have considerable difficulty with computational costs, which led to a series of path-based learning algorithms. Based on the simplicity and strong performance of CIBLE, I wonder if it could be extended to a GNN with greater depth?

2. Prototyping like in CIBLE does have advantages over plain TransR and RotatE as demonstrated in this paper. However, I couldn't figure out why there seems to be quite a gap between the performance of TransE/IBLE against CIBLE, but not so much between TransE and IBLE. I wish the authors could maybe explain a bit on why this is the case. It would be especially helpful if the authors could include their own implementation of TransR's evaluation on the table if not TransE/R-RotatE as well, since as I understand CIBLE is the combination of IBLE loss and TransR, this should not be very difficult.
limitations: I do not see potential negative societal impact.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper studies the knowledge base completion problem and proposes an instance-based learning method that utilizes the prototypes to help the inference of a given query. Then the relation information can be used in the prototype modeling and could obtain better performance. The proposed IBL can be combined with the traditional translation models to achieve the best performance. The authors also compare their model with other existing methods of rule discovery. IBL role provides interpretability by establishing instance-based equivalence relations.
strengths_and_weaknesses: Strengths:
- Most important baselines are included in the experiments.
- The case study and visualization are interesting.
- The conducted experiments are comprehensive.

Weaknesses:
- The proposed instance-based method needs to work with the traditional translation models to obtain the best performance. It looks like a neighborhood aggregation to enhance the translation models as they do not consider the graph structure information. Based on that, the paper provides an incremental strategy to improve the existing methods.
- The presentation about the logical rule is not clear and not easy to follow.
- The motivation for combining IBL and Translational models is not clarified.
questions: - The proposed instance-based learning looks like the neighboring entities of a relation can be aggregated to help infer the missing facts. Can you please discuss the differences between the proposed and GNN-based methods, e.g., R-GCN?
- The motivation for combining IBL and Translational models is not clear. The authors mention the objectives are compatible. However, they did not clarify why the IBL cannot work well alone and have to work with the translation models.
- It is not easy to understand why the IBL rule provides better interpretability than existing methods. I wonder why the IBL using instance-based equivalence relations can work better than the methods adopting semantic relevance.
limitations: Please discuss the limitations and negative societal impact of the work.
review_title: null

Meta-review: metareview: Develops a new paradigm for knowledge base completion, based around instance based learning.   The paper has motivation, some supporting theory and good empirical work.  

Reviewers ZosV and DVJo mention a relationship with GNN/GCNs which should be further discussed in the paper.
The paper has some simple grammar and spelling errors that should be fixed up, though no reviewers mentioned this.
The authors have some enlightening discussion with reviewers, for instance on definitions, that should be included in the paper.
confidence: Certain
Decision: Accept