Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: You say that any off the shelf classifier can be used to make the prediction based on embedding, what are the actual classifiers you used in your experiments?
review_title: How do you go from embedding to final classification?

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper studies the graph embedding problem by using the encoder-decoder method. The experimental study on real network data sets show the features extracted by the proposed model is good for classification.

Strong points of this paper:
  1. The idea of using the methods from natural language processing to graph mining is quite interesting.
  2. The organization of the paper is clear

Weak points of this paper:
  1. Comparisons with state-of-art methods (Graph Kernels) is missing. 
  2. The problem is not well motivated, are there any application of this. What is the different from the graph kernel methods? The comparison with graph kernel is missing. 
  3. Need more experiment to demonstrate the power of their feature extraction methods. (Clustering, Search, Prediction etc.)
  4. Presentation of the paper is weak. There are lots of typos and unclear statements. 
  5. The author mentioned about the graph kernel things, but in the experiment they didn't compare them. Also, only compare the classification accuracy by using the proposed method is not enough.
review_title: Comparison with Graph kernels Missing

final_score: rating: null
review_content: What is the classifier that you used for ECFP? In the text you only say that you used a library from a prior work. I think that for a correct comparison you should  use the same classifier for ECFP and skip-graph embeddings, because your dataset is very small and skewed, and the choice of regularization might be very important.
review_title: Classifier for ECFP

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes an unsupervised graph embedding learning method based on random walk and skip-thought model. They show promising results compared to several competitors on four chemical compound datasets.

Strength:
1, The idea of learning the graph embedding by applying skip-thought model to random walk sequences is interesting. 
2, The paper is well organized.

Weakness:
1, As the current datasets are small (e.g., the average number of nodes per graph is around 30), it would be great to explore larger graph datasets to further investigate the method. 
2, Comparisons with recent work like LINE and node2vec are missing. You can compare them easily by applying the same aggregation strategy to their node embeddings.

Detailed Questions:
1, The description about how to split the random walk sequence into 3 sub-sequences is missing. Also, the line “l_min >= (n_k - 1), … >= l_max” in section 2.2.2 is a mistake.
2, Can you provide the standard deviations of the 5-fold cross validation in Table 2? I’m curious about how stable the algorithm is.
review_title: Experiments Are Incomplete

final_score: rating: 7: Good paper, accept
review_content: The paper presents a method to learn graph embeddings in a unsupervised way using random walks. It is well written and the execution appears quite accurate. The area of learning whole graph representations does not seem to be very well explored in general, and the proposed approach enjoys having very few competitors.

In a nutshell, the idea is to linearize the graph using random walks and to compute the embedding of the central segment of each walk using the skip-thought criterion. Being not an expert in biology, I can not comment whether or not this makes sense, but the gains reported in Table 2 are quite significant. 

An anonymous public comment compared this work to a number of others in which the problem of learning representations of nodes is considered. While this is arguably a different goal, one natural baseline would be to pool these representations using mean- or max- pooling. It would very interesting to do such a comparison, especially given that the considered approach heavily relies on pooling (see Figure 3(c))

To sum up, I think it is a nice paper, and with more baselines I would be ready to further increase the numerical score.  

review_title: Good paper

final_score: rating: 6: Marginally above acceptance threshold
review_content: Authors take the skip-graph architecture (Kiros 2015) and apply it to classifying labeled graphs (molecular graphs). They do it by creating many sentences by walking the graph randomly, and asking the model to predict previous part and next part from the middle part. Activations of the decoder part of this model on a walk generated from a new graph are used as features for a binary classifier use to predict whether the molecule has anti-cancer properties.

Paper is well written, except that evaluation section is missing details of how the embedding is used for actual classification (ie, what classifier is used)

Unfortunately I'm not familiar with the dataset and how hard it is to achieve the results they demonstrate, that would be the important factor to weight on the papers acceptance.
review_title: An extension of skip-graph architecture to classifying similar molecular graphs 

Meta-review: The idea of applying skip-graphs to this graph domain to learn embeddings is good. The results demonstrate that this approach is competitive, but do not show a clear advantage. This is difficult, as the variety of approaches in this area is rapidly increasing. But comparisons to other methods could be improved, notably deep graph kernels.
Decision: Reject