Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes to provide a novel gradient-based meta-learning framework (Meta-Graph) for a few shot link prediction task. More specifically, they generate an effective parameter initialization for a local link prediction model for any unseen graph by leveraging higher-order gradients and introducing graph signature function into graph neural network framework. The authors validate the proposed model through several experiments.

This paper reads well and the results appear sound. I personally find the idea of incorporating the structural signature for input graphs into the GCN to modulate the parameters of the inference model very interesting. Moreover, the provided experiments support the authors’ intuition and arguments.

As for the drawbacks, I find the relationship to the prior works partly unclear. Moreover, it would be nice if the authors could also provide some ideas for future research directions, such as the prospects of using their approach for improving link prediction models and incorporating Meta-Graph in other domains like molecules structure. My concerns are as follows:

•    I am wondering if you can adopt R-GCN [1] instead of the GCN model for extending the Meta-Graph to multi-relational graphs? 
•    I suggest considering ranking metrics such as MRR and HITS@ to further evaluate the performance of Meta-Graph. 
•    Comparing the performance of Meta-Graph and MAML in Table 2, I am wondering about the reason behind the fact that the distance between these models’ performance decreases by increasing the number of edges in PPI and FirstMMDB, but increases in Ego-AMINER? Further, I am wondering why Meta-Graph performance drops in Ego-AMINER after increasing the number of edges from 10% to 20% (Table 3)?
•    I suggest providing a comparison of computational complexity between Meta-Graph and the baselines.

[1] Schlichtkrull, Michael, et al. "Modeling relational data with graph convolutional networks".

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: Overview: In this paper, a meta-learning approach is proposed to perform link prediction across multi-graphs with scarce data. To do so, each graph is treated as a link prediction "task". Different from the tasks in conventional meta-learning, the graphs here are generally non i.i.d. Based on the variational graph auto-coders, the method learns two important components: the global parameters used for GNNs and a graph signature function used to modulate the parameters of inference model based on the history of observed training graphs. 
The idea of formulating link prediction as a few-shot learning problem and solving it via multi-graph meta-learning is novel. The approach basically takes advantage of meta-learning and is expected to generalize well across graphs. The numerical study is extensive in discussing the properties of meta-graph and showing more attractive empirical performance than the existing approaches. 


Comments:

1. The explanation of Figure 1 needs to be improved. In its current form, the figure does not illustrate the idea of this paper very clearly. It would be better to provide more explanations on the graphical model for meta-graph and the meta-graph architecture in the context. 

2. The description of Algorithm 1 is confusing in some places. For example, the operation in Line 6 is commented as *compute graph signature*. However, what's actually going on is to set the signature function to stop gradient computation which seems not matching the comment. 

3. In section 4.2, the 5-gradient update AUC is reported to show the fast adaptation of meta-graph. However, in my opinion, it could be much more informative to show the accuracies of different gradient steps, like what has done in the experimental study of MAML. I don’t think the current experimental results can well explain the property of fast adaptation. 

=== update after author response ===

Thank you for your response.   I find my main concerns properly addressed in the feedback. 
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper presents a new link prediction framework in case of having seen only a small fraction of the graph. The premise is that the predictor is trained on other similar graphs. The predictor (a variational graph autoencoder) performs transfer learning through gradient-based metalearning with a combination of global and local parameters. 

The paper is conceptually solid, however, it is hard to claim novelty in individual ideas in the paper. The major concern I have is with the benchmarks. It seems like the (meta) training data is very similar to the validation data. So, even though it is supposed to be a few shot prediction problem, it seems like the meta-training phase has already provided more than a "few" training points. Please correct me if this observation is incorrect. My rating is neutral so far (not an option in the system) and ready to change upon comments from the authors.
review_title: Official Blind Review #1

Meta-review: This paper presents a new link prediction framework in the case of small amount labels using meta learning methods. The reviewers think the problem is important, and the proposed approach is a modification of meta learning to this case. However, the method is not compared to other knowledge graph completion methods such as TransE, RotaE, Neural Tensor Factorization in benchmark dataset such as Fb15k and freebase.  Adding these comparisons can make the paper more convincing. 
Decision: Reject