Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: ##########################################################################

Summary:
This paper incorporates AdaBoost into the deep graph neural network architecture, which has the ability to efficiently extract knowledge from high-order neighbors and then integrates knowledge from different hops of neighbors into the network in an Adaboost way. It solves the problem of oversmoothing. Extensive experiments show the effectiveness of the proposed method. 

##########################################################################

Pro:
+ The paper is clear and well organized.
+ The introduction of AdaBoost into the deep GNN is novel and interesting.
+ The comparison with several existing methods is well analyzed in terms of both model architectures and computational advantages. 
+ Extensive experiments are conducted to demonstrate the consistent state-of-the-art performance of the proposed method.

##########################################################################

Cons:
- Although the same classifier architecture is adopted for $f^{(l)}_\theta$, their parameters are different, which is different from RNN. It is better to avoid this confusion.

- It would be better to include some discussion of the global attention methods (e.g., [Puny et al., 2020] ) and sampling-based methods (e.g., [Zeng et al., 2020]).

References: 
Puny et al. From Graph Low-Rank Global Attention to 2-FWL Approximation. ICML Workshop Graph Representation Learning and Beyond, 2020.

Zeng et al. Graph sampling-based inductive learning method. ICLR ’20, 2020.  
review_title: Official Blind Review #4

final_score: rating: 7: Good paper, accept
review_content: Summary
In this paper, the authors study graph convolutional networks, where they propose to use AdaBoost for Deep GCNs. This method makes it possible to use information from multi-hop neighbours. Computationally, the proposed method is efficient, which is illustrated through various experiments. The paper is well written with good clarity, while the proposed method is novel and significant to the research community. 

Reasons for recommending acceptance
- To that of reviewer's knowledge, the proposed scheme is novel. The method address the issue of using information for higher-order neighbours, without increasing the computational complexity. 
- comprehensive experiments across multiple datasets, evaluating AdaGCN in terms of computational efficiency, accuracy, dependency on the number of layers. 

Questions
- While comparing MixHop against AdaGCN, the authors mention that AdaGCN does have generalization guarantees from Boosting theory. This statement is loose, and a formal justification may be needed. 
- In Figure 4 (Right), where the epoch time is measured against the number of layers, AdaGCN is shown to have nearly constant time w.r.t. layers. Some more explanation would be useful in understanding this. 




review_title: Adaboosting GCNs

final_score: rating: 6: Marginally above acceptance threshold
review_content: Overall, the proposed AdaGCN model could incorporate the different hops of neighbors into the network in an Adaboost way without improving the computational cost. That has been confirmed by the theoretical comparison with other baselines and the experimental results.

Pros:
[1] It proposed a novel deep graph neural network by incorporating AdaBoost into the computation of network.
[2] It compared to the existing related work to illustrate the benefits of the proposed AdaGCN.
[3] The experiments demonstrate its effectiveness of efficiency of AdaGCN on encoding the high-order graph structure information.

Cons:
[1] The simplified graph convolution might be vulnerable to the noisy nodes. That is, when there exists one noisy node with abnormal attributes, this simplified graph convolution might be significantly degraded. Thus, the higher-order convolution in the proposed method might become worse.
[2] In table 2, it is confusing why the implemented methods (e.g., PPNP (ours), APPNP (ours)) have lower performance than the results reported in the literature (Klicpera et al., 2018).
[3] In Section 4.3, it shows that Fast-GCN has fastest speed in Pubmed, but slower than GCN on Cora-ML and MS-Academic datasets. That might need more explanation since intuitively the goal of Fast-GCN is to improve the efficiency of GCN.
review_title: This paper proposed a novel RNN-like deep graph neural network architecture by incorporating AdaBoost into the computation of network for extracting the knowledge from high-order local neighborhood.

final_score: rating: 5: Marginally below acceptance threshold
review_content: By integrating Adaboosting and a fully connected layer, this paper provides a new graph neural network structure. The objective of this paper is to design a deeper graph models in an efficient way for better performance. The computational efficiency and performance of the proposed algorithm are evaluated using the task of node property prediction on several public datasets. This is a new variant of GNN, but the quality this paper is lower than the expectation regarding to the clarity and organisation. 

Pros: 
1.	The algorithm integrated Adaboosting for graph data. Thus, AdaGCN could utilise different levels of node features for final prediction. 
2.	The method is optimised in a layer-wise way rather than the traditional GCN optimisation, which is similar to the optimisation of recurrent neural networks. 
3.	Authors compared the structure of AdaGCN with that of other GNN variants. 
4.	For the experiments, the proposed algorithm is more computationally efficient, and achieves better performance on the task of node property prediction.  The performance of AdaGCN is slightly more robust than previous methods. The performance drop is not observed within 10 layers for AdaGCN as shown in Fig 3. 

Cons: 
1.	Speaking of the state of art performance, GraphSAGE with LSTM also achieves a 95.4% F1 score on the Reddit dataset for node classification tasks. Thus, the authors may need to compare the training time and performance with more recent algorithms, like ClusterGCN and GraphSAGE. 
2.	The paper is not well written. Many typos are discovered. For example, extra space is added in the first sentence after equation 3. Meanwhile, the punctuation around equations is not consistent. For the full sentence following an equation, one would place a full stop after the equation. However, there is no full stop after equations 5, 6, 7, and 8. Abbreviations, such as "JK", "APPNP", and "PPNP", are used before introduction. 
3.	Some notations are confusing and misleading. $K$ refers to the number of node categories, and $k$ refers to a category of a node. Meanwhile, $w_i$ and $W^{l}$ have completely irrelevant definitions. 
4.	To evaluate the efficiency of different GCN approaches, the authors listed the per-epoch training time of methods. The implementation of GCN with different frameworks would result in the large variance of training time. It is better that the authors could include the time and memory complexity of each algorithm. 
5.	Fig 4, after 10 layers, it is not clear whether the linear trend would continue. This result is a bit misleading.

review_title: AdaGCN: Adaboosting Graph Convolutional Networks Into Deep Models

Meta-review: Three of the reviewers are very positive about this work, and R3 is slightly concerned about the datasets, writing, and notations etc. The authors responded to these concerns in detail and have agreed to take care of these comments. Thus an accept is recommended based on the understanding that the authors will fulfil their commitments.
Decision: Accept (Poster)