Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: [Update after Rebuttal: Increased confidence, Accept]

[REMARK: This is an emergency review, that is, the reviewer did neither select autonomously into this paper out of interest/expertise nor was the reviewer granted with sufficient time (with respect to standard time frames) to appropriately provide a review. Please take the review with caution, the confidence level will be adjusted accordingly.]

The presented paper proposes a bridge between the otherwise disjoint model classes of factorization-based models (FM) and the currently popular graph neural networks (GNN). The motivation for this lies in the fact that the success stories of said models remains disjoint as well i.e., GNNs are successful widely but in tasks such as multi-relational link prediction they fall short to FMs. The authors bridge aforementioned gap by providing a reformulation (Theorem 3.1) that shows how one can identify Message Passing a la GNN within FM. To corroborate on their theoretical result, the authors provide a first model living within the identified (or expected) spectrum of FM-GNN models and give an empirical account of said model (referred to as ReFactor GNN) showing that the addendum to plain GNN indeed improves the shortcomings.
strengths_and_weaknesses: **Strengths**

Studying the shortcomings of GNN in advent of their rise as dominant model class for structured-input predictions is an important step towards improving the model class to cope with more classical tasks such as knowledge graph completion. Bridging the gap to other existing model classes such as FMs, helps to broaden the understanding of what qualitative differences between our modelling choices of interest exist. It also enables (a foundation to) compilation back-and-forth and therefore allows for flexible use of existing tools. The presentation of the ideas is detailed and complimented by several modalities including the rigorous mathematization and a key figure. The empirical results suggest a sensible view on FM-GNNs and the idea of a spectrum of such models (which might have been identified by this work) leaves room for further investigations.

**Weaknesses**

IMHO there is only so much a contribution can propose and certainly the key results presented should not be deluted, therefore, the following weaknesses are rather extensions to be considered upon the existing work. Considering GNN literature, several theoretical findings have been made regarding their expressivity, tractability, etc. and in a similar fashion analyzing the presented initial FM-GNN, ReFactor GNN, would benefit our understanding of where to place the model within the zoo of existing methods. Furthermore, the focus has been on DistMult but also on optimizers such as SGD or AdaGrad (as also explicitly mentioned by the authors), yet, there exists the impression of more general schemes (the FM-GNN spectrum) and thereby it would be sensible to already provide a disclaimer on how the design choices already (strongly or weakly) influence the subsequent results.
questions: The questions are derived mostly from aspects mentioned in the "Weaknesses" section. I'd invite the authors to answer these questions to overcome the observed weaknesses for improving the paper to ultimately increase the chances of both contribution and visibility to the community. The list of questions is unordered and they range in quality and speed of expected answer (some are minor, quickly answerable whereas others are more fundamental, crucial, maybe difficult to answer):
* How could we classify/analyze the spectrum of FM-GNN models?
* In which way do the design choices (DistMult, optimizer etc.) affect the key result?
* What type of guarantees could we possibly give to ReFactorGNN?
limitations: No contradictions or any sort of relevant mistake have been detected in the paper. The clarity of the paper is a clear advantage, even if (likely unavoidable) dense at times. Existing bodies of work are being referenced accordingly throughout the paper.

Sufficient details for reproduction are being provided. Also with actual code (however, the code was not reviewed). All key assumptions for the method are being pointed out explicitly. Societal impact is not being discussed explicitly, however, this was not necessary to begin with IMHO.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work proposes ReFactor GNNs that makes use of the message passing property of GNNs in order to cast he hugely successful factorization based embedding techniques, here DistMult, as a GNN. The idea is to formulate gradient descent in factorization based models as message passing in GNNs. The experimental section shows that the proposed model works well on different knowledge base completion tasks.
strengths_and_weaknesses: Strengths:

1. The idea is simple and quite intuitive and yet very clever at the same time. It's great to see this connection between DistMult's and GNNs in a more formal fashion.

2. The paper is written nicely and is easily understandable.

Weaknesses:

1. The fact that the results presented are for $L=\infty$ after explicitly saying that they "periodically clear the node state cache every L full batches" seems a bit strange. Is there any reason for not presenting resulting for L other than $\infty$?

2. There should be some comparisons on the link prediction task especially with RGCN. 

I have done an emergency review for this paper and thus have lowered my confidence. Overall, I like the idea behind the paper and would like to hear more from the authors in the rebuttal. My major concern remains the experimental part of the paper.

questions: In addition to the above 2 points raised in weaknesses, I have a few more questions:

1. What is the running time of the proposed ReFactor GNNs?

2. It would be great to have some comment on the expressiveness of the ReFACTOR GNNs.
limitations: I do not foresee any potential negative societal impact of this work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: **DISCLAIMER: This is not my area of expertise, and this paper is very specific and quite technical. Nevertheless, the program chairs asked to review the submission based on my experience and broad knowledge anyway.  I do try.**

This paper proposes a new approach to the problem of knowledge graph completion. This problem resisted recent advances in geometric deep learning and the best approaches so far are still based on factorization-based models (FB).
Here the authors note that FB models can be seen as GNNs "by reformulating gradient descent on node embeddings using message-passing primitives" and use this idea to build a new hybrid model called Refactor GNN, that achieves the performance of FB models using 1/10 parameters, sometimes even fixing a new state of the art.

strengths_and_weaknesses: ### Originality
I am not able to assess whether these ideas were already existing under a different name. In particular I am not able to judge the novelty with respect to [28].

### Quality
As far as I can tell, this work is of good quality. I appreciated the uncertainty estimated using different random seeds reported in the appendix. I peeked at the code and it seems well organized, even if mostly bare of comments.

### Clarity
This paper is very specific and quite technical. The background section was easy to follow, which is a good thing, then for a non-expert it becomes quite harsh.

### Significance

Suppose I have to bet on how many citations this paper will receive in the next five years.

Following the stats of accepted papers at Neurips 2014 tracked in 2019 (as can be found at https://github.com/rubai5/MLcites) a bookmaker could give the following betting odds:

- A: Between 0 and 5 cits, proposed at 10/1.
- B: Between 5 and 25 cits, proposed at 3/1.
- C: Between 25 and 75 cits, proposed at 3/1.
- D: More than 75 cits, proposed at 6/1.

I would bet my money on A, maybe adjusting to B given the higher rate of citations in the NLP field. Actually this bet is based on my broad knowledge, and should be understood as a bet on the doom of knowledge graphs (KGs). The problem discussed here is very specific to KGs, and to me it seems that many use-cases of KGs were recently robbed by large language models, a short time ago enriched also with explicit memory and retrieval methods [RETRO].


[RETRO] Borgeaud, Sebastian, et al. "Improving language models by retrieving from trillions of tokens." arXiv preprint arXiv:2112.04426 (2021).  

questions: I do not have relevant questions to ask. My score is borderline because I am not able to judge too many aspects of this paper. 
limitations: There seems to be adequate discussion of the limitations, but I can't make an assessment.
review_title: null

Meta-review: metareview: the paper proposes ReFactorGNNs, a unified view on factorization-based models (FMs) and graph neural networks (GNNs). It shows how to how FMs can be cast as GNNs by reformulating the gradient descent
procedure as message-passing operations. The reviewers agree that the idea is "intuitive and yet very clever at the same time", as one of the reviewers puts it. Connecting FMs just as DistMul with GNNs is great, and the paper is well written and presented. The one more negative reviewer expresses a low confidence and provides an informed outsider view on the paper, which is actually rather positive. 
I fully agree though I would also like to provide some more links to related work:
https://arxiv.org/abs/1509.08535
https://forum.stanford.edu/events/posterslides/MessagePassingforMatrixFactorization.pdf
http://pfister.ee.duke.edu/papers/Kim-istc10.pdf
Moreover, another indication is that transformers are quite expressive, and transformers are GNNs. I I would like to encourage the authors to provide a more high-level view on the problem, and then argue that the presented approach makes this intuition very concrete. Moreover, it would be great to discuss more future work. For instance, it seems possible now to learn new FM approaches automatically from data. Kind of "FM discovery" or "Learning to Factorize". This is all very exciting. 
confidence: Certain
Decision: Accept