Paper:
Graph representation learning serves as the core of many important prediction tasks, ranging from product recommendation in online marketing to fraud detection in financial domain. Real-life graphs are usually large with complex local neighborhood, where each node is described by a rich set of features and easily connects to dozens or even hundreds of neighbors. Most existing graph learning techniques rely on neighborhood aggregation, however, the complexity on reallife graphs is usually high, posing non-trivial overfitting risk during model training. In this paper, we present Neural Sparsification (NeuralSparse), a supervised graph sparsification technique that mitigates the overfitting risk by reducing the complexity of input graphs. Our method takes both structural and non-structural information as input, utilizes deep neural networks to parameterize the sparsification process, and optimizes the parameters by feedback signals from downstream tasks. Under the NeuralSparse framework, supervised graph sparsification could seamlessly connect with existing graph neural networks for more robust performance on testing data. Experimental results on both benchmark and private datasets show that, NeuralSparse can effectively improve testing accuracy and bring up to 7.4% improvement when working with existing graph neural networks on node classification tasks.

Review:
The authors propose a supervised graph sparsification technique that "mitigates the overfitting risk by reducing the
complexity of input graphs."
The idea is as follows: there is a sparsification network which samples subgraphs (adjacency matrices) by computing a probability distribution for each edge and drawing the existence of each edge from this distribution. The sparsified graph is then fed to a GNN that computes a classification loss. Since the authors use the Gumbel-softmax trick the method is end-to-end differentiable and the output consists of a node classifier and a graph generative model that can be used to sample sparsified graphs. 
The paper covers an interesting topic and results in some good numbers on standard benchmark datasets. I also like the idea to use the Gumbel-softmax trick to make the entire model differentiable. 
Unfortunately, the authors miss to cite and discuss highly related work [1] (ICML 2019). In this work, the authors also maintain a graph generative model (also by parameterizing edges but with iid Bernoulli RVs), also sampling graphs from this generative model, and also using these sampled graphs to train a GNN for semi-supervised node classification. The resulting model is also end-to-end differentiable. Instead of using a Gumbel-softmax to keep the method differentiable (given that we have discrete random variables) the authors propose a novel way to compute gradients for the parameters of the Bernoulli RVs by posing the problem as a bilevel optimization problem. In [1] the graph cannot only be sparsified but also enriched with edges that might be beneficial for the classification accuracy. Indeed, it was shown that adding edges is more beneficial than removing edges. The results on Cora and Citeseer are better than the results reported by the authors in this submission. At the very least, the authors should familiarize themselves, discuss, and compare empirically to [1]. 
The existence of this previous work also reduces the novelty of the proposed approach. However, if a  comparison to [1] would be added, the submission could be seen as an alternative instance of the framework presented in [1]. I would be willing to increase my score to accept, if the authors provide such a comparison in an updated version. 
[1] https://arxiv.org/abs/1903.11960
-----
I've read the rebuttal and while I don't agree necessarily with all statements made by the authors, I am happy to increase my score based on the discussion of previous work.