Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper proposes a semi supervised learning algorithm for graph node classification. The Algorithm is inspired from Graph Neural Networks and more precisely graph convolutional NNs recently proposed by ref (Kipf et al  2016)) in the paper.  These NNs alternate 2 types of layers:  non linear projection and diffusion, the latter incorporates the graph relational information by constraining neighbor nodes to have close representations according to some “graph metrics”. The authors propose a model with simplified projection layers and more sophisticated diffusion ones, incorporating a simple attention mechanism. Experiments are performed on citation textual datasets. Comparisons with published results on the same datasets are presented.

The paper is clear and develops interesting ideas relevant to semi-supervised graph node classification. One finding is that simple models perform as well as more complex ones in this setting where labeled data is scarce. Another one is the importance of integrating relational information for classifying nodes when it is available. The attention mechanism itself is extremely simple, and learns one parameter per diffusion layers. One parameter weights correlations between node embeddings in a diffusion layer. I understand that you tried more complex attention mechanisms, but the one finally selected is barely an attention mechanism and rather a simple “importance” weight. This is not a criticism, but this makes the title somewhat misleading. The experiments show that the proposed model is state of the art for graph node classification. The performance is on par with some other recent models according to table 2. The other tests are also interesting, but the comparison could have been extended to other models e.g. GCN.
You advocate the role of the diffusion layers, and in the experiments you stack 3 to 4 such layers. It would be interesting to have indications on the compromise performance/ number of diffusion layers and on the evolution of these performances when adding such layers.
The bibliography on semi-supervised learning in graphs for classification is light and should be enhanced.
Overall this is an interesting paper with nice findings. The originality is however relatively limited in a field where many recent papers have been proposed, and the experiments need to be completed.

review_title: Interesting paper with nice findings. The originality is however relatively limited in a field where many recent papers have been proposed, and the experiments need to be completed.

final_score: rating: 6: Marginally above acceptance threshold
review_content: SUMMARY.

The paper presents an extension of graph convolutional networks.
Graph convolutional networks are able to model nodes in a graph taking into consideration the structure of the graph.
The authors propose two extensions of GCNs, they first remove intermediate non-linearities from the GCN computation, and then they add an attention mechanism in the aggregation layer, in order to weight the contribution of neighboring nodes in the creation of the new node representation.
Interestingly, the proposed linear model obtains results that are on-par with the state-of-the-art model, and the linear model with attention outperforms the state-of-the-art models on several standard benchmarks.


----------

OVERALL JUDGMENT
The paper is, for the most part, clear, although some improvement on the presentation would be good (see below).
An important issue the authors should address is the notation consistency, the indexes i and j are used for defining nodes and labels, please use another index for labels.
It is very interesting that stripping standard GCN out of nonlinearities gives pretty much the same results, I would appreciate if the authors could give some insights of why this is the case.
It seems to me that an important experiment is missing here, have the authors tried to apply the attention model with the standard GCN?
I like the idea of using a very minimal attention mechanism. The similarity function used for the attention (cosine) is symmetric, this means that if two nodes are connected in both directions, they will be equally important for each other. But intuitively this is not true in general. It would be interesting if the authors could elaborate a bit more on the choice of the similarity function.


----------

DETAILED COMMENTS
Page 2. I do not understand the point of so many details on Graph Laplacian Regularization.
Page 2. The use of the term 'skip-grams' is somewhat odd, it is not clear what the authors mean with that.
Page 3. 'the natural random walk' ???
Bottom of page 4. When the authors introduce the attention based network also introduce the input/embedding layer, I believe there is a better place to do so instead of that together with the most important contribution of the paper.

review_title: few questions

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes graph-based neural network in which weights from neighboring nodes are adaptively determined. The paper shows importance of propagation layer while showing the non-linear layer does not have significant effect. Further the proposed method also provides class relation based on the edge-wise relevance.

The paper is easy to follow and the idea would be reasonable. 

Importance of the propagation layer than the non-linear layer is interesting, and I think it is worth showing.

Variance of results of AGNN is comparable or even smaller than GLN. This is a bit surprising because AGNN would be more complicated computation than GLN. Is there any good explanation of this low variance of AGNN?

Interpretation of Figure 2 is not clear. All colored nodes except for the thick circle are labeled node? I couldn't judge those predictions are appropriate or not.
review_title: idea would be reasonable and constains interesting insight

Meta-review: A version of GCNs of Kipf and Welling is introduced with (1) no non-linearity; (2) a basic form of (softmax) attention over neighbors where the attention scores are computed as the cosine of endpoints' representations (scaled with a single learned scalar). There is a moderate improvement on Citeseer, Cora, Pubmed.

Since the use of gates with GCNs / Graph neural networks is becoming increasingly common (starting perhaps with GGSNNs of Li et al, ICLR 2016)) and using attention in graph neural networks is also not new  (see reviews and comments for references), the novelty is very limited.  In order to make the submission more convincing the authors could: (1) present results on harder datasets; (2)  carefully evaluate against other forms of attention (i.e. previous work).

As it stands, though it is interesting to see that such simple model performs well on the three datasets, I do not see it as an ICLR paper.

Pros:
-- a simple model, achieves results close / on par with state of the art

Cons:
-- limited originality
-- either results on harder datasets or / and evaluation agains other forms of attention (i.e. previous work) are needed



Decision: Reject