Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The authors introduce strategies for pre-training graph neural networks. Pre-training is done at the node level as well as at the graph level. They evaluate their approaches on two domains, biology and chemistry on a number of downstream tasks. They find that not all pre-training strategies work well and can in fact lead to negative transfer. However, they find that pre-training in general helps over non pre-training.

Overall, this paper was well written with useful illustrations and clear motivations. The authors evaluate their models over a number of datasets. Experimental construction and analysis also seems sound.

I would have liked to see a bit more analysis as to why some pre-training strategies work over others. However, the authors mention that this is in their planned future work.

Also, in figure 4, the authors mention that their pre-trained models tend to converge faster. However, this does not take into account the time already spent on pre-training. Perhaps the authors can include some results as to the total time taken as well as amortized total time over a number of different downstream tasks.

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes new pre-training strategies for GNN with both a node-level and a graph-level pretraining. For the node-level pretraining, the goal is to map nodes with similar surrounding structures to nearby context (similarly to word2vec). The main problem is that directly predicting the context is intractable because of combinatorial explosion. The main idea is then to use an additional GNN to encode the context and to learn simultaneously the main GNN and the context GNN via negative sampling. Another method used is attribute masking where some masked node and edge attributes need to be predicted by the GNN. For graph-level pretraining, some general graph properties need to be predicted by the graph.
Experiments are conducted on datasets in the chemistry domain and the biology domain showing the benefit of the pre-training.

The paper addresses an important and timely problem. It is a pity that the code is not provided. In particular, the node-level pretraining described in section 3.1.1. seems rather complicated to implement as a context graph needs to be computed for each node in the graph. In particular I do not think the satement 'all the pre-training methods are at most linear with respect to the number of edges' made in appendix F is correct.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The paper proposes pre-training strategies (PT) for graph neural networks (GNN) from both node and graph levels. Two new large-scale pre-training datasets are created and extensive experiments are conducted to demonstrate the benefits of PT upon different GNN architectures. I am relative positive for this work. Detail review of different aspects and questions are as follows. 

Novelty: As far as I know, this work is among the earliest works to think about GNN pre-training. The most similar paper at the same period is [Z Hu, arXiv:1905.13728]. I read both papers and found they have similar idea about PT although they have different designs. This paper leverages graph structure (e.g., context neighbors) and supervised labels/attributes (e.g., node attributes, graph labels) for PT. These strategies are not surprising for me and the novelty is incremental. 

Experiment: The experiments are overall good. The authors created two new large scale pre-training graph datasets. Experimental results of different GNN architectures w/o different PT for different tasks are provided. Comparing to non-pretraining GNN, the improvements are significant for most cases. 

Writing: The writing is good and easy to follow. 

Questions: I would like to see more discussion about difference between this work and [Z Hu, arXiv:1905.13728]. Comparing to the other work, what are strengths of this work? In addition, have the authors compared the performances of their work and [Z Hu, arXiv:1905.13728] using the same data? 
review_title: Official Blind Review #3

Meta-review: All three reviewers are consistently positive on this paper. Thus an accept is recommended.
Decision: Accept (Spotlight)