Paper:
Community detection in graphs is of central importance in graph mining, machine learning and network science. Detecting overlapping communities is especially challenging, and remains an open problem. Motivated by the success of graphbased deep learning in other graph-related tasks, we study the applicability of this framework for overlapping community detection. We propose a probabilistic model for overlapping community detection based on the graph neural network architecture. Despite its simplicity, our model outperforms the existing approaches in the community recovery task by a large margin. Moreover, due to the inductive formulation, the proposed model is able to perform out-of-sample community detection for nodes that were not present at training time.

Review:
This paper proposes Deep Overlapping Community detection model (DOC), a graph convolutional network (GCN) based community detection algorithm for network data. The model is a simple combination of GCN and existing framework for community detection. The proposed algorithm is compared to baselines on various datasets, and demonstrated to be accurate in many cases.
I think the paper does not deal with one of the most important aspects of network modeling - the degree heterogeneity of nodes. Many works reported that lack of degree corrections would result in bad estimates of community structures [1,2,3]. Probably including the degrees as feature of nodes would be helpful. 
Regarding the stochastic gradient descent by edge subsampling, I think the authors should mention [4], where the idea of edge subsampling in stochastic gradient descent setting was introduced before this work. Also, it is worth noting that we may lose some important distributional properties in graphs if we naively subsample from it [5]. For instance, sampling from positive and negative pairs to balance the class contribution may distort the sparsity and degree distributions of subsampled graphs. 
If we choose to use Bernoulli-Poisson link function, we can reduce the time complexity of likelihood and gradient computation to O(N + E), where N is the number of nodes and E is the number of edges, with the auxiliary variable trick introduced in [6]. In that case we don't really have to worry about subsampling. Why didn't you consider applying this to your model?
Regarding the experiments, I think some important baselines are missing [3, 6]. Also, I wonder whether the proposed algorithm would scale to the graphs with more than 100,000 nodes. 
References
[1] B. Karrer and M. E. J. Newman. Stochastic blockmodels and community structure in networks. Physical Review E, 83(1):016107, 2011.
[2] P. K. Gopalan, C. Wang, and D. Blei. Modeling overlapping communities with node popularities. NIPS 2013.
[3] A. Todeschini, X. Miscouridou and F. Caron. Exchangeable Random Measures for Sparse and Modular Graphs with Overlapping Communities. CoRR 2016.
[4] J. Lee, C. Heakulani, Z. Ghahramani, L. F. James, and S. Choi. Bayesian inference on random simple graphs with power law degree distributions. ICML 2017.
[5] P. Orbanz. Subsampling large graphs and invariance in networks. CoRR 2017.
[6] M. Zhou. Infinite edge partition models for overlapping community detection and link prediction. AISTATS 2015