Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper presents an interesting new graph neural network technique customized for the specific setting of signed graphs. This is a paper I have reviewed before.

There has been an increasing interest in the last several years on the problem of clustering/link prediction/representation learning of signed graphs, where edge weights are allowed to take either positive or negative values. The main contribution is the end to end pipeline targeted at link sign prediction and the feature diffusion step. The signed random walk diffusion is neat, and appears to be new I believe, in this context. The numerical results are somewhat convincing, with improvements in the single digit percentages; on some data sets fairly small, but consistent improvement across the board. There is some theoretical component, which is rather light, but this is fine.

Might have been good to compare against non GCN based methods, for example 
Chiang, Kai-Yang, et al. "Prediction and clustering in signed networks: a local to global perspective." The Journal of Machine Learning Research 15.1 (2014): 1177-1213.
Perhaps also worth placing this work in the context of some of the growing literature of signed clustering/link prediction that typically rely on spectral methods/matrix completion. However, since the authors already compare against non-GCN methods and it is fine to leave this for future work. 

It would have been good to see a comparison on synthetic data as well. There exist signed stochastic block models which have been looked at, which could be augmented with features, or use no feature information at all. Another compelling story could be made around better clustering performance, by leveraging the embedding obtained via the authors’ proposed approach. There is a growing literature on clustering signed graphs, and it would be interesting to see to what extend the proposed method embedding is applicable in that setting and how cluster recovery performance looks like. 

Regarding the features - they play a central role in the paper, but towards the end the reader finds out that the input signed graphs do not have initial node features.. It would be have been good to identify a data set where the nodes come with some covariate information. Or, as stated above, a synthetic model could be put forth, with a set (or subset) of features which correlate with cluster membership/edge sign. 

In SBM model, it would be interesting to see how does the link prediction accuracy vary with the number of clusters, and also the graph sparsity. Most spectral methods out there for this task face difficulties when handling very sparse graphs (ie, typically below the threshold connectivity limit of (log n)/n, both theoretically and empirically, and require various regularisation techniques). It would be good for the authors to comment on this aspect (very sparse graphs), as it could be a strong edge over spectral methods which tend to underperform in the very space regime.

Might also be interesting to comment on how seed information could be used, if available? See for example some of the literature on polarization in signed graphs. 

Overall the paper appears to be technically sound, clearly written and very well structured, and I think overall a good addition to the signed network embedding literature.
review_title: Signed network embedding

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes to leverage signed random walk as hidden representation propagations to construct a signed graph diffusion network model. 

Pros: 
1. The idea of the proposed model is simple but very effective according to the experiment evaluations.
2. It's very interesting that the proposed model achieves better performance as the number of layers increases, even up to K=10.
3. The writing of the paper is in a good shape and everything is clear to follow.
4. The authors also prove the convergence of the proposed diffusion layer.

Cons:
1. My major concern is the novelty of the paper. The key part of the model (i.e., signed diffusion layer) totally borrows from the existing work. Even the figure used in this paper is same as the reference paper (e.g., Figure 2 (a) in this paper vs. Figure 2 (b) in reference paper).
      Jung, Jinhong, et al. "Personalized ranking in signed networks using signed random walk with restart." 2016 IEEE 16th 
      International Conference on Data Mining (ICDM). IEEE, 2016.
2. In addition, authors need to compare with the above paper as well as it's very effective in link prediction even though it's not an embedding-based method.
review_title: Graph Diffusion Network on signed networks

final_score: rating: 6: Marginally above acceptance threshold
review_content: Main Idea:

In this paper, the author studied the problem of node embedding in signed networks. The authors proposed SGDNet which combines the idea of diffusion/random work in signed networks and Residual connection in GCN. The network is trained directly with classification loss on edge sign prediction. The authors carried out extensive experiments on several real-world networks with comparison to several state-of-the-art methods. The proposed method showed superior performance in the sign prediction task.

Strength:
The paper is technically sound and well written. The authors provide analysis on the convergence of the diffusion kernel to show that the injection of input signals prevents the degradation with depth increment.
The authors carried out thorough experiments on four real-world networks with comparison to several strong baselines.
The authors carried out parameter sensitivity analysis on two of the important parameters: diffusion depth and signal injection strength.

Weakness
The authors trained the model and evaluated performance only on existing edges with signs. However, in practice, it is usually useful to predict the existence of edges or not in the network. It is interesting to see how the method generalizes to this situation which more resembles the graph reconstruction task.
The comparison of depth in SGCN with K is kind of unfair. A more direct comparison would be the number of layers L which controls the depth of the network. Since there are no trainable parameters involved in the diffusion step.
The authors mentioned that the embedding dimension is fixed to 32 for all methods. First, as an important hyperparameter, it is interesting to see how performance of different methods varies with it. Second, the embedding dimension of the proposed method is effective 64 as positive and negative are separated and combined.

Questions
In equation (2), what is the motivation to inject h only to positive but not the negative as well.
How is m^{(0)}_t and p^{(0)}_t initialized. Are they initialized to all zero?

review_title: Official Blind Review #1

final_score: rating: 4: Ok but not good enough - rejection
review_content: The strength:

- Empirical results show improvements of the proposed method over the baseline methods, including methods for unsigned graphs, signed embedding emthods, and signed GCN methods not addressing over-smoothing.

The weakness:

- The proposed method is a straightforward integration of existing methods on signed graphs and on handling over-smoothing in GCNs. There is very little new idea in the proposed method.

- The convergence result (Theorem 1) is straightforward based on linear algebra, and it is only for the diffusion part in each GCN layer. There is no overall theoretical results on the SGDNet architecture.

Overall, the method proposed is a simple combination of past methods on handling signed graphs and handling over-smoothing in graphs. It is just that the two has not been combined together, and so the only contribution I see is the combination of these two methods and verify it in the signed edge prediction task. The theoretical result on convergence is only on the diffusion convergence at each layer, and it is a straightforward application of the linear algebra. It is unclear to me why we need a diffusion convergence at each layer and then also need GCN with multiple layers. What is the connection between the diffusion steps K and the GCN layers L? 

In summary, the paper shows improvement when combining previous methods on signed networks and on handling over-smoothing. It may fit into a second-tier conference to record the result, but I feel that it does not meet the high bar of ICLR.
review_title: The paper propose SGDNet, Signed Graph Diffusion Network, to perform end-to-end node presentation and signed link prediction. It also considers the over-smoothing issue and use local feature injection to prevent over-smoothing.

Meta-review: The paper addresses an interesting problem of clustering/link prediction/representation learning of signed graphs, where edge weights are allowed to take either positive or negative values. The paper proposed an end to end pipeline targeted at link sign prediction and the feature diffusion step. The reviewers think the proposed method is a straightforward integration of existing methods, and the convergence result is straightforward. The paper can be improved by including more novel ideas or analysis. 
Decision: Reject