Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper derives a matrix-factorization approach for node classification. The approach is closely related to GCN. The authors show that the proposed approach outperforms GCN and Planetoid empirically.

Though empirically appealing, this paper has a few pitfalls that need be addressed.

1. The wording "unifying" is a misnomer. The title "unifying graph convolutional networks" hallucinates a framework that unifies several neural network architectures, which is not precise. In reality, the authors propose a learning objective that consists of two loss terms, the classification loss and the structure loss. The classification loss is nothing but the usual GCN. The structure loss is the contribution of the paper. The derivation of this term starts from GCN and a Laplacian smoothing argument, and arrives at a matrix factorization form through a series of modeling modifications. By and large, the title is misleading.

2. The wording "correctness of our theoretical analysis" is dubious. The paper does not present a theoretical analysis. The derivation of the matrix factorization is only a modeling process. In no mathematical sense the factorization is equivalent to GCN.

3. The alternating training is questionable. The authors propose alternately optimizing the structure loss and the classification loss. Since taking the gradient of the whole loss function is straightforward in all graph neural network approaches, it is unclear why the authors prefer the alternating optimization approach. Supplementing a convergence plot and comparing the two approaches may help, if the alternating approach is indeed better.

4. The "distributed computing" component needs more substantiation. It is unclear whether this phrase actually means the concept familiar by the parallel computing community. Therein, computation is done by using several machines communicated through networked protocols. Machine setting, parallel implementation details, and speedup are the primary interests in distributed computing. All information should be reported.

Questions:

1. First sentence of section 5. What does "all-round" mean?

2. Stability Analysis. What is b? The reader does not find a definition elsewhere. A probably related concept is alpha (see eqn (8)).

3. Figure 3(b) shows that larger b leads to poorer performance. The authors state that a larger b means a stronger emphasis on the structure loss. Consequently, it appears that putting more emphasis on the structure term leads to poorer performance. Then, does it mean that the structure term is a useless contribution?

review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: 
The work poses an interesting question: Are GCNs (and GNNs) just special types of matrix factorization methods? Unfortunately, the short answer is **no**, which goes against what the authors say. 

Until recently I thought like the authors, but the concurrent work [1] (On the Equivalence between Node Embeddings and Structural Graph Representations) https://openreview.net/forum?id=SJxzFySKwH changed my mind. 
The work of (Li et al., 2018) shows that nearby nodes tend to get similar representations. There is mounting experimental evidence of that being the case in real-world graphs (e.g., https://arxiv.org/abs/1908.08572). But [1] shows that GCNs and GNNs are fundamentally different from matrix factorization methods, regardless of the loss function used to learn the embeddings. Consider Figure 1 in [1], and it is easy to see that matrix factorization will give different embeddings to the Lynx and Orca nodes, while GCNs and GNNs must give the same embedding. Even if we connect the graphs through the Spruce and the Zooplankton nodes, their conclusion would not change. Matrix factorization (as broadly understood) will give embeddings that can even be used to cluster nodes. The eigenvectors of the symmetric Laplacian encode the diffusion of a type of random walk and nodes that are far away in the graph must have different embeddings (because through the diffusion operator, they are far away).

In GCNs, the convergence of the embeddings is better explained by the mixing of a random walk (Theorem 1 of (Xu et al., 2018)), which, in the special case of a GCN, converges to 1/sqrt(degree of node), as shown by (Li et al., 2018) in their Theorem 1 for the symmetric Laplacian. This is unrelated to what we get in matrix factorization as explained earlier. 

What is wrong with the math: Equation (11) is equated with matrix factorization, but note that it does not account for nonedges, while matrix factorization accounts for nonedges. This issue is more clear in Equation (14). The problem happens when the paper jumps from Equation (14), which is correct but not MF, to Equation (15) which is MF but unrelated to Equation (14). The argument is that “negative edges sampling is used, for better convergence”… sorry, not for better convergence, it completely changes the optimization objective. Hence, GCNs are not matrix factorization methods.

I think the paper is a valiant effort, but unfortunately the core premise is incorrect. The jump from Equation (14) to equation (15) cannot be justified, and I believe showcases a fundamental flaw the argument. I do not see a way to fix the paper. I vote to reject it. 

[1] On the Equivalence between Node Embeddings and Structural Graph Representations, https://openreview.net/forum?id=SJxzFySKwH 
Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K.I. and Jegelka, S., 2018. Representation learning on graphs with jumping knowledge networks. ICML 2018.
Li, Qimai, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. AAAI, 2018.


--------------

Read rebuttal. Will keep my original assessment.

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: In this paper, the authors propose a new method for semi-supervised node classification by drawing connection between GCN and MF. The authors borrow the idea of convergence of GCN as Laplacian Smoothing. With this observation, the authors propose a joint loss with two components: classification loss and structure loss for the similarity between embedding of neighboring nodes. The authors train the parameters via optimizing the two losses alternatively. Experiments are carried out on seven networks with comparison to baselines.

Strength:
1. It is an interesting and innovative idea to draw connection between GCN and MF.
2. The propose method is more suitable for distributed setting. With negative sampling for structure loss, both structure batch and classification batch can be constructed locally with only one-hop information.
3. The authors carry out experiments on seven real-world networks with ablation study for components in the model. Moreover, the authors carry out comparison to baselines in distributed setting.

Weakness:
1. The connection of GCN to MF is very indirect. It holds only when the GCN converges to the Laplacian smoothing. It is not clear whether this holds empirically. Moreover, there are too much intermediate steps and approximation between the Laplacian smoothing to the matrix factorization. As far as I am concerned, the connection is closer to node embedding versus matrix factorization.
2. Given that GCN serves as Laplacian smoothing, it would be great if the authors can simply add additional regularization on dis(h_i, h_i) for (v_i, v_j)\in E. Moreover, there is no reference and description to the Planetoid* algorithm.
3. The authors use alternative batches between structure and classification loss. It would be interesting to see if joint training the two loss in mini-batch among a node and its neighbors can leads to any difference.
3. The authors report only accuracy as evaluation metrics. It would be better If the authors could report recall@K and F1 score as well.

review_title: Official Blind Review #3

Meta-review: The paper makes an interesting attempt at connecting graph convolutional neural networks (GCN) with matrix factorization (MF) and then develops a MF solution that achieves similar prediction performance as GCN. 

While the work is a good attempt, the work suffers from two major issues: (1)  the connection between GCN and other related models have been examined recently. The paper did not provide additional insights; (2) some parts of the derivations could be problematic. 

The paper could be a good publication in the future if the motivation of the work can be repositioned. 
Decision: Reject