Title: Predicting Global Label Relationship Matrix for Graph Neural Networks under Heterophily
Venue: NeurIPS 2023
Accepted: yes
Keywords: graph neural networks

heterophily problem

global label relationship matrix
Abstract: Graph Neural Networks (GNNs) have been shown to achieve remarkable performance on node classification tasks by exploiting both graph structures and node features. The majority of existing GNNs rely on the implicit homophily assumption. Recent studies have demonstrated that GNNs may struggle to model heterophilous graphs where nodes with different labels are more likely connected. To address this issue, we propose a generic GNN applicable to both homophilous and heterophilous graphs, namely Low-Rank Graph Neural Network (LRGNN). Our analysis demonstrates that a signed graph's global label relationship matrix has a low rank. This insight inspires us to predict the label relationship matrix by solving a robust low-rank matrix approximation problem, as prior research has proven that low-rank approximation could achieve perfect recovery under certain conditions. The experimental results reveal that the solution bears a strong resemblance to the label relationship matrix, presenting two advantages for graph modeling: a block diagonal structure and varying distributions of within-class and between-class entries.

Predicting Global Label Relationship Matrix for Graph Neural Networks under Heterophily

Abstract

Graph Neural Networks (GNNs) have been shown to achieve remarkable performance on node classification tasks by exploiting both graph structures and node features. The majority of existing GNNs rely on the implicit homophily assumption. Recent studies have demonstrated that GNNs may struggle to model heterophilous graphs where nodes with different labels are more likely connected. To address this issue, we propose a generic GNN applicable to both homophilous and heterophilous graphs, namely Low-Rank Graph Neural Network (LRGNN). Our analysis demonstrates that a signed graph's global label relationship matrix has a low rank. This insight inspires us to predict the label relationship matrix by solving a robust low-rank matrix approximation problem, as prior research has proven that low-rank approximation could achieve perfect recovery under certain conditions. The experimental results reveal that the solution bears a strong resemblance to the label relationship matrix, presenting two advantages for graph modeling: a block diagonal structure and varying distributions of within-class and between-class entries.

1 Introduction

Graphs (or networks) are ubiquitous in various fields, such as social networks [Tang et al., 2013, Xu et al., 2015, 2019b], biology [Guzzi and Zitnik, 2022], and chemistry [Gilmer et al., 2017]. Many real-world networks follow the Homophily assumption, i . e ., linked nodes tend to share the same label or have similar features; while for graphs with heterophily, nodes with different labels are more likely to form a link. For example, many people tend to connect with people of the opposite sex in dating networks. Graph Neural Networks (GNNs) [Kipf and Welling, 2017, Velickovic et al., 2018] have shown significant success in tackling a diverse set of graph mining tasks, such as NLP [Song and King, 2022, Chen et al., 2023] and clustering [Huang et al., 2023, Kang et al., 2020]. There are also studies dedicated to enhancing the representational power of GNNs [Liang et al., 2023, Xu et al., 2019a]. Recently, researchers have observed that GNNs may face difficulties in learning on graphs

𝑇

!

: Class 1

: Class 2

: Class 3

: Positive Link

: Negative Link

1

1

-1

-1

1

1

-1

-1

-1

-1

1

-1

-1

-1

-1

1

LRGNN:

+

+

+

𝑇

"

-

+

-

𝑇

#

+

-

+

𝑇

$

-

-

-

with heterophily due to the smoothing operation, which tends to generate similar node representations for adjacent nodes, even though their labels differ [Abu-El-Haija et al., 2019, Zhu et al., 2020].

Various designs [Zhu et al., 2020, Chien et al., 2021, Lim et al., 2021] have been proposed to enhance the performance of GNNs under heterophilous scenarios (see Zheng et al. [2022] for a survey). Among them, high-pass filters are the most frequently used components as they can push away a node from its neighbors in the embedding space, aligning with the characteristic of heterophily. Spectral-based methods [Chien et al., 2021, Luan et al., 2021] combine high-pass filters with low-pass ones by fusing the outputs from intermediate layers. However, these methods cannot capture the node-level homophily ratio, as they use only one convolutional filter type in each layer. Alternatively, some methods [Bo et al., 2021, Yang et al., 2021] enable neural networks to learn the aggregation coefficients. Specifically, these methods update node representation by computing a learned signed weighted combination of neighbors' representations by adopting graph attention function [Velickovic et al., 2018]. However, the graph attention function has been proven to compute a form of static attention: the ranking of attention scores remains unconditioned on the query [Brody et al., 2022].

According to Zhu et al. [2020], under specific conditions, a node's 2-hop neighborhood is expected to be predominantly homophilic. To capitalize on this result, they suggest amplifying a node's influence using its 2-hop neighbors. Building on this, GloGNN [Li et al., 2022] introduces a subspace clustering approach to derive a global coefficient matrix. Nonetheless, Liu et al. [2010] proves that this matrix's between-class entries are almost zero. This indicates that while GloGNN taps into global homophily, it rarely employs remote heterophilous nodes for feature propagation. Moreover, the subspace clustering approach ensures dense within-class matrix entries but does not account for the sign of these entries.

In this paper, we aim at recovering a global label relationship matrix ˆ Z such that ˆ Z i,j = 1 if node v i and v j have the same label (homophilous) and -1 otherwise (heterophilous) . In this case, we exploit global homophily and heterophily by assigning proper signs. Determining ˆ Z i,j , which requires knowledge of node labels, seems impractical due to the unavailability of many labels during training. Here a question arises: can we infer the unknown ˆ Z i,j from a small set of observations?

Unlike previous neural network-based methods, we employ sign inference to solve the problem. Signed social networks (SSNs) utilize positive links to denote friendship between two users and negative links to represent enmity. Sign inference is the task of predicting the signs of unknown links using the knowledge of available ones. The structural balance theory [Cartwright and Harary, 1956], a fundamental principle of SSN, postulates that individuals in signed networks are disposed to following specific patterns of 'an enemy of my friend is my enemy' (e-f-e), 'a friend of my friend is my friend' (f-f-f), 'a friend of my enemy is my enemy' (f-e-e), and 'an enemy of my enemy is my friend' (e-e-f). However, the last assumption could contradict actual scenarios. Therefore, a more general notion, called weak balance, was introduced to address this issue [Davis, 1967].

We study signed graphs, wherein a negative edge signifies that the two nodes possess distinct labels. Figure 1 illustrates that triads in signed graphs follow the weak balance theory, which suggests that the signed graph has a global low-rank structure [Hsieh et al., 2012]. An intriguing finding [Candès and Tao, 2010] demonstrates that if a matrix has a low rank, it can be accurately recovered from a

limited number of observations. Motivated by this, we approximate ˆ Z with the solution to a low-rank matrix approximation problem, which is then used for feature propagation.

Since the solution is a low-rank matrix, our new model is termed Low-Rank Graph Neural Network (LRGNN). We summarize the contributions of this paper as:

We show that triads in signed graphs obey weak balance, based on which the unknown label relationship between two nodes can be inferred by looking up the known ones.

We prove that a variant of signed GNNs shows a tendency to follow the structural balance theory, which can be enhanced by eliminating the faulty assumption from the model design.

We propose an effective model LRGNN. Using both real-world and synthetic datasets, we provide comprehensive experimental results, thereby highlighting the superior performance of LRGNN over other state-of-the-art methods.

2 Preliminaries

We first introduce the notations used in this paper.

Notations . Denote by G = ( V , E ) an undirected graph, where V and E denote the node set and edge set, respectively. The nodes are described by a node feature matrix X ∈ R n × f , where n and f are the number of nodes and number of features per node, respectively. Y ∈ R n × c is the node label matrix. The neighbor set of node v i is denoted by N i . Let A ∈ R n × n denote the adjacency matrix where A i,j = 1 if ( i, j ) ∈ E and 0 otherwise. We define a label relationship matrix ˆ Z ∈ R n × n for a signed graph, where ˆ Z i,j = 1 if ( i, j ) share the same label and -1 otherwise. Let ˜ A be a signed adjacency matrix indicated by an observation set Ω , with ˜ A i,j = ˆ Z i,j if ( i, j ) ∈ Ω and 0 otherwise. The Frobenius norm of a matrix is given by ∥ · ∥ 2 F . A i, : represents the i -th row of matrix A .

2.1 Link Sign Prediction via Low-Rank Matrix Completion

Next, we discuss the problem of link sign prediction and the theoretical support for using low-rank matrix completion.

Link Sign Prediction. The link sign prediction task involves determining the signs of unobserved links in a network, where some links are already labeled as positive or negative. This work specifically considers scenarios where only one link exists between each node pair ( i, j ) ∈ V × V . To be precise, given ˜ A partly observed from ˆ Z , the objective is to predict the signs of the unknown entries of ˆ Z . Note that the observed entries can be constructed using available labels from the training set. This task is difficult in general because of the low supervised ratio. For instance, if the supervised ratio is 30% , we need to guess 91% of entries of ˆ Z using the knowledge of the rest 9% of entries.

Here is a fact that significantly changes the task's premise, rendering it practical. The rank of ˆ Z is equivalent to the number of classes if c ≥ 2 . This low-rank structure is described by the weak balance theory: for homophilous node pair ( i, j ) and any v k ∈ V , if ˆ Z i,k = 1 then ˆ Z j,k = 1 (f-f-f), if ˆ Z i,k = -1 then ˆ Z j,k = -1 (f-e-e); for heterophilous ( i, j ) , if ˆ Z i,k = 1 then ˆ Z j,k = -1 (e-f-e). Therefore, ˆ Z has c linearly independent distinct rows if c ≥ 2 (rank ( ˆ Z ) = c ). Inspired by this finding, we employ Matrix Completion to address this challenge.

Matrix Completion. The task of matrix completion involves predicting the missing values in an unknown matrix based on a limited number of observed entries in the observation set Ω . Exact recovery of a matrix from a small Ω is impossible without making any assumption about the matrix. But the search for solutions becomes meaningful as soon as the unknown matrix has a low rank; numerous algorithms can accomplish a near-perfect recovery with high probability under mild assumptions. This is known as the Low-Rank Matrix Completion (LRMC) problem.

Low-Rank Matrix Completion. Given a signed adjacency matrix ˜ A and the observation set Ω , the task of LRMC is to find the lowest-rank solution among all the feasible solutions as follows,

where Z is the decision variable and P Ω ( · ) is the projection onto the observation set. Unfortunately, (1) is an NP-hard problem without known algorithms capable of solving problems in practical time [Candès and Tao, 2010]. A matrix of rank r has exactly r nonzero singular values. Hence, the rank function is simply the number of nonvanishing singular values. As the l 1 -minimization is the tightest convex relaxation of the combinatorial l 0 -minimization problem [Candès and Plan, 2010]. An alternative is the nuclear minimization problem,

where ∥ Z ∥ * denotes the nuclear norm of Z , which is the sum of the singular values. Equivalently we can reformulate Eq.(2) in Lagrange form

where λ ≥ 0 is a regularization parameter. A surprising result involving LRMC is that under certain assumptions, the missing entries of ˜ A can be accurately predicted.

Theorem 1 . [Candès and Tao, 2010] Let ˆ Z ∈ R n × n be a fixed matrix of rank r = O (1) obeying the strong incoherence property with parameter ε . Suppose we observe m entries of ˆ Z with locations sampled uniformly at random. Then there is a positive numerical constant C such that if

then ˆ Z is the unique solution to Eq.(2) with probability at least 1 -n -3 . In other words: with high probability, nuclear-norm minimization recovers all the entries of ˆ Z with no error.

Solving Eq.(2) requires the computation of a SVD of a potentially large matrix, which can be computationally expensive. To reduce this cost, we can use an SVD-free low-rank matrix factorization (LRMF) problem formulated as follows:

where c ≤ q ≪ n is the operating rank. There is a remarkable fact that ties the LRMF problem and the nuclear-norm regularized problem (3) [Mazumder et al., 2010]: by selecting a q ≥ c , the solution to Eq.(5) also provides a solution to Eq.(3). The equivalence between the solutions allows us to approximate ˆ Z by the solution to Eq.(5).

We present some fundamental facts that support the recovery of ˆ Z . The first fact is that the rank of ˆ Z is deterministic. Ideally, q = rank ( ˆ Z ) should be the optimal choice [Hastie et al., 2015]. The second fact is that ˆ Z i, ; = ˆ Z j, : holds for any homophilous ( i, j ) . This implies that class imbalance is the upper bound of the coherence parameter ε . In most cases, ε = O (1) . Furthermore, if ˜ A i,j = 1 is observed, v i and v j can combine their observations. For example, if ˜ A i,k is observed, then ˜ A j,k is also ascertainable, regardless of whether it is directly observed or not ( ˜ A i,k = ˜ A j,k ). As a result, there is a significant increase in the observation rate.

In real-life data, we typically witness a deviation from the assumption that observations exhibit uniform distribution. Nonetheless, this deviation only moderately influences the performance. Suppose a node lacks any observed samples. In that case, an exact recovery is theoretically impossible, but it would not hinder the recovery of other rows. Low-rank approximation methods have been applied to practical applications with a fair amount of success, such as the Netflix problem [Koren et al., 2009].

3 Structural Balance Theory and Signed GNNs

We consider a variant of signed GNNs [Bo et al., 2021] that updates node representations as

where f θ ( · , · ) refers to a neural network that measures the similarity between the inputs and returns a scalar ranging in [ -1 , 1] . Let y i,j = 1 if two neighboring nodes v i and v j belong to the same class and y i,j = -1 otherwise. Let < i, j, k > be a triad such that v j is a neighbor of v i and v k is a neighbor of v j , while v i and v k are not directly linked. We have the following result.

Theorem 2 . Consider that we apply a L -layer signed GNN ( L ≥ 2 ) on a triad. The output of the signed GNN is equivalent to H ( L ) = ZH (0) , where Z is a matrix. Assume that each coefficient α i,j is independent of other coefficients and the probability that the model can precisely predict the sign of each coefficient is p , namely Pr ( sign ( α ( l ) i,j ) = y i,j ) = p . Also, assume that all the self-coefficients are positive. The probability that at least one of Z i,j and Z j,k are correct in sign and the product Z i,j Z j,k Z i,k > 0 is given by p b . Then p b is monotonically increasing concerning p . Especially, if p = 1 , Z i,j Z j,k Z i,k > 0 always holds.

Remark 1 . A triad is said to be balanced iff its product is positive as the four types of triads given by balance theory have two or zero negative signs.

The above result suggests that multi-layer signed GNNs may implicitly implement the structural balance theory as they frequently create balanced triads. When p equals 1, all triads are balanced, indicating that the model is following the idea that 'a heterophilous neighbor of my heterophilous neighbor is a homophilous node.' Nonetheless, this notion does not necessarily hold under multi-class classification tasks. Hence, even if the model can predict the label relationship between direct neighbors accurately, it will still likely predict the relationship between 2-hop neighbors inaccurately. This suggests that multiple-layer signed GNNs are sub-optimal. Signed Graph Convolutional Networks (SignedGNNs) [Derr et al., 2018] utilizes the balance theory to model signed graphs. By eliminating the assumption 'e-e-f' (see Supplement for implementa-

4

8

12

16

20

24

#Layer

55

60

65

70

75

80

85

Accuracy (%)

WB-SignedGCN

SignedGCN

tion details), we obtain a weakly balanced version of SignedGNNs termed WB-SignedGNNs. To evaluate their vulnerability to over-smoothing, we compared the performance of WB-SignedGCN and SignedGCN at different depths, ranging from 4 to 24. As Figure 2 illustrates, SignedGCN suffer from over-smoothing, and this problem can be largely resolved by dropping the incorrect assumption in the model's design. Such a minute adjustment in the design can result in substantial performance enhancement for 24-layer SignedGCN (80% vs. 53% on Wisconsin dataset).

4 Approach

In this section, we present the overall framework of the Low-Rank Graph Neural Networks.

4.1 Predicting the label relationship matrix via robust low-rank approximation

Note that in ˆ Z , the information on the original graph topology is lost: we cannot tell whether two given nodes are linked or not since all entries are nonzero. To preserve the graph topology information, following Lim et al. [2021], we first apply MLPs to fuse feature matrix and adjacency matrix into a lower-dimensional matrix H (0) ∈ R n × c ,

where 0 < µ < 1 is the balance term. Next, we utilize LRMF to recover ˆ Z with the edge set being the observation set ( Ω = E ), matching the time complexity of vanilla GCNs [Kipf and Welling, 2017]. Furthermore, an attention term [Bahdanau et al., 2015] is incorporated into Eq.(5) as a supplement to the matrix completion. We approximate ˆ Z with Z = U * V T * by solving the following robust low-rank matrix factorization problem,

where λ > 0 and γ > 0 are hyper-parameters, W a parameter matrix, h (0) T i W h (0) j an attention coefficient, τ i,j a parameter of the capped norm. Low-rank matrix approximation models are prone

to outliers due to the square loss function [Keshavan et al., 2010]. To mitigate this issue, this work applies capped norm to limit the contribution of the approximation error (( UV T ) i,j -˜ A i,j ) 2 . This strategy has been extensively utilized by researchers [Nie et al., 2014, 2017, Jiang et al., 2015]. Note that here each sample ( i, j ) has an independent parameter τ i,j which will be explained in detail later. The min operator render the function non-differentiable, and the projection function prevents us from obtaining a closed-form solution. It seems that solving Eq.(8) is infeasible. Using the surrogate function and Majorization-Minimization algorithm [Hastie et al., 2015], we can ensure that the objective function does not increase after each iteration. The optimization algorithm, initialization method, and proof of convergence can be found in the Supplement.

Once we have derived U * V T * by minimizing Eq.(8), the output of the model is formulated as

where the identity matrix I n is used to augment the ego-information and 0 < β < 1 is a hyperparameter. A 1-layer network is sufficient to aggregate all nodes' representations for a node, it also averts the implicit usage of the balance theory.

Remark 2 . The time complexity of LRGNN is proportional to the number of edges. A detailed discussion can be found in the Supplement.

4.2 Parameterized outlier detection

This section outlines a practical algorithm for detecting outliers based on the reliability of the estimated ˜ A i,j . To generate the signed adjacency matrix ˜ A , we can use any off-the-shelf neural network classifier to generate pseudo labels. Additionally, the known node labels in training set T V and label matrix Y can also be utilized. Similar to Zhu et al. [2021], the pseudo labels are generated as follows,

where f NN ( · ) denotes a trained neural network named estimator, ⊙ the Hadamard product, and O i, : = 1 if i ∈ T V and 0 otherwise. The signed adjacency matrix is defined as

Here 0 < δ < 1 is a parameter that controls the ratio of negative edge weights. ⟨· , ·⟩ denotes the inner product of two vectors. Although this strategy appears simple, it is highly effective. The critical point here is that we do not need to know the exact classes of the two nodes and we just care about whether they belong to the same class. Let p be the accuracy of the estimator and assume that there is a uniform distribution chance of being any incorrect class, namely 1 -p c -1 .

The estimator has a probability of (1 -p )( pc + c -2) ( c -1) 2 to identify any two heterophilous nodes as homophilous nodes. This probability is less than 1 -p , provided that c ≥ 3 and p < 1 . For instance, when the accuracy of the estimator is 50% and c = 5 , for heterophilous ( i, j ) , the probability of assigning them different labels is about 82 . 8% , which is significantly higher than 50% . In our experiments, we use simple estimators including GCN and MLP for fairness.

The succeeding section discusses the design of τ i,j , which can be expressed as τ i,j = c i,j · τ , where

0 < c i,j < 1 is a parameter. We adopt a widely accepted definition of outliers [Yeh, 2007],

where q 1 and q 3 are the first and third quartiles, respectively. Mathematically, c i,j is defined as

where [ a 1 , a 2 , a 3 ] = Softmax ( W a ) , W a ∈ R 1 × 3 , || denotes vector concatenation, and e is the natural logarithm.

The Euclidean norm term measures the quality of the pseudo label since a uniformly distributed ∥ ˆ Y i, : ∥ 2 2 suggests that the estimator has no confidence in the class of v i . The absolute value term takes into account that extreme values are typically more reliable. For example, a zero ˜ A i,j indicates that the relationship between v i and v j is ambiguous. The third term employs graph attention [Velickovic et al., 2018, Brody et al., 2022] to adapt c i,j accordingly.

5 Experiment

This section evaluates the performance of LRGNN. We put the experimental results w.r.t. ablation study and robustness in the Supplement due to space limitation.

Datasets. We use three homophilous datasets including Cora, Citeseer and Pubmed [Yang et al., 2016], along with 6 heterophilous datasets released in Pei et al. [2020] and Rozemberczki et al. [2021], and three large-scale heterophilous graphs [Lim et al., 2021]. The training/validation/testing splits used in this paper are the same as [Li et al., 2022].

Baselines. We compare LRGNN with 11 baselines, including (1) classic GNN models: vanilla GCN [Kipf and Welling, 2017],and GAT [Velickovic et al., 2018]. (2) models specifically designed to handle heterophily: H2GCN [Zhu et al., 2020], GPR-GNN [Chien et al., 2021], WRGAT [Suresh et al., 2021], LINKX [Lim et al., 2021], GGCN [Yan et al., 2021], ACM-GCN [Luan et al., 2021], GloGNN++ [Li et al., 2022], and Ordered GNN (OGNN) [Song et al., 2023]. (3) 2-layer MLP. We specifically choose GloGNN++ and ACM-GCN as they generally outperforme the other variants proposed in their respective papers.

Node classification results. Table 2 and Table 3 provide a summary of the test accuracy of the tested methods on 12 datasets with diverse homophily ratios and scales. Some baselines experience out-ofmemory errors on large datasets, thus the corresponding results are excluded. From the tables, we can

3

5

7

9

11 13 15 17 19 21

Operating rank

q

0.10

0.15

0.20

0.25

0.30

0.35

Wisconsin

3

5

7

9

11 13 15 17 19 21

Operating rank

q

0.28

0.30

0.32

0.34

0.36

0.38

0.40

Chameleon

3

5

7

9

11 13 15 17 19 21

Operating rank

q

0.30

0.32

0.34

0.36

0.38

3

5

7

9

11 13 15 17 19 21

Operating rank

q

0.05

0.10

0.15

0.20

0.25

0.30

Cornell

Texas

Wisconsin

Cornell

104

within-class

within-class

within-class

within-class

103

between-class

103

between-class

103

between-class

106

105

between-class

Frequency

104

102

102

102

103

101

101

101

102

101

100

100

100

100

5

4

3

2

-1

0

-0.4

0.2

0.0

0.2

0.4

-1.00-0.75 -0.50-0.250.000.250.500.75

1.0

0.5

0.0

0.5

1.0

AggregationCoefficients

Aggregation Coefficients

AggregationCoefficients

Aggregation Coefficients

draw several observations. Firstly, MLP is a strong baseline for heterophilous datasets, outperforming GCN and GAT on Texas, Wisconsin, and Cornell datasets. Secondly, methods designed specifically for heterophilous graphs generally perform better than MLP and traditional GNNs. Thirdly, H2GCN, WRGAT, GGCN, ACM-GCN, GloGNN++, and LRGNN are the top performers on small datasets. However, H2GCN, WRGAT, and GGCN lack scalability, thereby limiting their widespread adoption for large datasets. Fourthly, LRGNN performs the best in terms of the average rank, emerging as the winner or runner-up on all datasets except for Actor.

The results suggest that LRGNN consistently offers superior performance on both homophilous and heterophilous graphs. Notably, LRGNN achieved the highest score on Squirrel, with approximately 19% improvement over the runner-up score achieved by OGNN. The excellent performance on Squirrel and Chameleon could be attributed to the relatively large average node degrees of these two datasets, providing more observations to recover the label relationship matrix. Moreover, the performance of LRGNN on these three large datasets demonstrates the scalability of our model.

Sign inference accuracy w.r.t. operating rank q . We investigate the effect of the operating rank q on the recovery loss: err = 1 2 n 2 ∑ i,j | sign (( U * V T * ) i,j ) -sign ( ˆ Z i,j ) | . 3 shows that the error decreases initially when q increases, then it increases gradually after reaching the minimum point. The best result is consistently achieved when q is approximately 5 , the number of classes. Empirically, this supports the use of low-rank approximation to predict the label relationship matrix: the appropriate dimensionality for the factors can be determined based on the number of classes. In a standard LRMF problem, the rank of the matrix of interest is inaccessible. Therefore, we have to select a sufficiently large q to ensure that q surpasses the rank, which guarantees that the solution to LRMF problem provides a solution to the LRMC problem. Nevertheless, selecting excessively large q can substantially increase the number of iterations required to converge.

Visualizing the predicted label relationship matrix Z . There are two main desirable traits of ˆ Z we want Z to possess. The first is that within-class entries are positive while between-class entries are negative. This ensures that nodes are embedded close to their homophilous nodes and distant from the heterophilous nodes in the embedding space. The second is that ˆ Z i , : and ˆ Z j , : are identical if they share the same label, so that ˆ Z i , : X = ˆ Z j , : X . Consequently, we visualize Z in two different styles.

First, as shown in Figure 4, dividing the entries into two groups reveals that the distribution is similar to a Gaussian distribution. Although most entries converge around zero, within-class entries and between-class entries are positioned on opposite sides of the zero.

To illustrate the second trait, we visualize Z with reordered rows and columns based on ground truth labels. As shown in Figure 5, the matrices exhibit the structure of a block diagonal matrix. But here the entries of sub-matrices along the diagonal are positive, whereas off-diagonal sub-matrices contain

Texas

Wisconsin

Cornell

Chameleon

0

50

100

150

0

0

50

100

150

200

250

+0

0

50

100

150

0

0

500

1000

1500

2000

1.00

0.75

50

500

0.50

50

50

0.25

100

1000

0.00

100

100

-0.25

150

1500 -

-0.50

-0.75

150 -

200

150 -

-1.00

2000

250

negative entries. On closer inspection of the figure, we notice that nodes in the same class share a high degree of similarity, with the rows' pixels showing comparable patterns. In conclusion, Z exhibits both desirable traits of ˆ Z . This helps explain the effectiveness of LRGNN.

SyntheticGraphs

Real-WorldGraph

Avg.Degree=0.5

Avg.Degree=5.0

Avg.Degree=20.0

Texas

0.6

1.0

1.0

0.9

0.5

0.8

0.8

0.8

?

GloGNN++

GCN

MLP

0.4

0.6

0.6

ACMGCN

0.7

LINKX

0.3

0.4

0.4

0.6

LRGNN

H2GCN

0.2

0.2

0.2

0.1

0.3

0.5

0.7

0.9

0.1

0.3

0.5

0.7

0.9

0.1

0.3

0.5

0.7

0.9

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Node-Level HomophilyRatio

Node-Level HomophilyRatio

Node-Level HomophilyRatio

a

Results on synthetic graphs. To comprehensively evaluate the performance of LRGNN, we leverage random partition graphs [Kim and Oh, 2021] generated by the stochastic block model. The node features are sampled from Gaussian distributions where the centers of clusters are vertices of a hypercube. Notably, the distance between Gaussian distribution means is small, thus, it is challenging to differentiate between node features of distinct classes. This is evident by MLP's unsatisfactory performance on the synthetic graphs. We use 15 synthetic random graphs with varying node-level homophily ratios and average degrees. More information about the synthetic graphs can be found in the Supplement.

It can be observed from Figure 6 that, when the homophily ratio or node degree is low, LRGNN is the only model that outperforms MLP. Also, LRGNN performs significantly better than other models when the homophily ratio is below 0 . 5 . These results suggest that LRGNN performs particularly well on random graphs, which we attribute to two reasons. Firstly, the observed entries of these graphs are uniformly distributed, meeting one of the conditions for exact recovery. Secondly, to derive aggregation coefficients, low-rank approximation methods utilize only the observed entries of ˆ Z , while neural network based methods rely on the raw features. However, in this case, node features are not that informative since the means of Gaussian distributions are close. To validate the second reason, we use the Texas dataset and degrad the quality of its features by adding Gaussian variables. We obtain a degraded feature matrix by X ′ i,j = X i,j + ϵ j , with ϵ j i.i.d. sampled from a Gaussian distribution N (0 , σ 2 ) . Note that the original features will be overwhelmed by the Gaussian random variables if a large σ is applied. We compare the performance of representative models that included GloGNN++, H2GCN, and MLP with that of LRGNN. It can be observed that, as the features get less informative, the performance of GloGNN++, H2GCN, and MLP decreases significantly, while LRGNN's accuracy remains above 80% . In addition to that, a large σ also makes their training process erratic, which is accompanied by significant error bars. These empirical results confirm our previous analysis.

6 Conclusion

This paper presents a method for extending GNNs to heterophilous graphs through the use of a label relationship matrix. In order to utilize the low-rank properties of weakly-balanced graphs, we propose a robust low-rank matrix approximation technique for the prediction of the label relationship matrix. The proposed LRGNN has been thoroughly evaluated through extensive experiments. We conduct extensive experiments to evaluate our proposed LRGNN. Our results indicate that LRGNN outperforms other baseline methods on both synthetic and real-world graphs. Notably, when the node degree distribution of the graph conforms to a uniform distribution, LRGNN exhibits significantly superior performance over other baseline methods.

7 Limitations

The main theoretical support for our approach assumes a uniform distribution of node degrees. However, in practice, node degree distributions frequently follow a power-law distribution that is non-uniform. Although the main results of this paper are obtained using real-world graphs with nonuniform node degree distributions, it is unclear how this affects the theoretical results. Additionally, it is acknowledged that LRGNN's performance is affected by the accuracy of the generated pseudo labels. The supplementary material provides the corresponding experiment that indicates LRGNN demonstrates a non-sensitive response to this impact. However, further exploration and discussion are helpful to investigate the theoretical implications fully.

References

Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 21-29. PMLR, 2019.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Yoshua Bengio and Yann LeCun, editors, ICLR 2015, 2015.

Deyu Bo, Xiao Wang, Chuan Shi, and Huawei Shen. Beyond low-frequency information in graph convolutional networks. In AAAI, pages 3950-3957. AAAI Press, 2021.

Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? In ICLR. OpenReview.net, 2022.

M. Buchanan and Andrew W. Fitzgibbon. Damped newton algorithms for matrix factorization with missing data. In CVPR (2), pages 316-322. IEEE Computer Society, 2005.

Emmanuel J. Candès and Yaniv Plan. Matrix completion with noise. Proc. IEEE, 98(6):925-936, 2010. doi: 10.1109/JPROC.2009.2035722.

Emmanuel J. Candès and Terence Tao. The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory, 56(5):2053-2080, 2010. doi: 10.1109/TIT.2010.2044061.

Dorwin Cartwright and Frank Harary. Structural balance: a generalization of heider's theory. Psychological review, 63(5):277, 1956.

Pei Chen. Optimization algorithms on subspaces: Revisiting missing data problem in low-rank matrix. Int. J. Comput. Vis., 80(1):125-142, 2008.

Yankai Chen, Yifei Zhang, Menglin Yang, Zixing Song, Chen Ma, and Irwin King. WSFE: wasserstein sub-graph feature encoder for effective user segmentation in collaborative filtering. CoRR, abs/2305.04410, 2023.

Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In ICLR. OpenReview.net, 2021.

Mark A. Davenport and Justin K. Romberg. An overview of low-rank matrix recovery from incomplete observations. IEEE J. Sel. Top. Signal Process., 10(4):608-622, 2016.

James A Davis. Clustering and structural balance in graphs. Human relations, 20(2):181-187, 1967.

Tyler Derr, Yao Ma, and Jiliang Tang. Signed graph convolutional networks. In IEEE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018, pages 929-934. IEEE Computer Society, 2018. doi: 10.1109/ICDM.2018.00113.

Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019.

Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Doina Precup and Yee Whye Teh, editors, ICML 2017, volume 70 of Proceedings of Machine Learning Research, pages 1263-1272. PMLR, 2017.

Pietro Hiram Guzzi and Marinka Zitnik. Editorial deep learning and graph embeddings for network biology. IEEE ACM Trans. Comput. Biol. Bioinform., 19(2):653-654, 2022. doi: 10.1109/TCBB. 2021.3110279.

Trevor Hastie, Rahul Mazumder, Jason D. Lee, and Reza Zadeh. Matrix completion and low-rank SVD via fast alternating least squares. J. Mach. Learn. Res., 16:3367-3402, 2015.

Cho-Jui Hsieh, Kai-Yang Chiang, and Inderjit S. Dhillon. Low rank modeling of signed networks. In KDD, pages 507-515. ACM, 2012.

Zongmo Huang, Yazhou Ren, Xiaorong Pu, Shudong Huang, Zenglin Xu, and Lifang He. Selfsupervised graph attention networks for deep weighted multi-view clustering. In Brian Williams, Yiling Chen, and Jennifer Neville, editors, AAAI, pages 7936-7943. AAAI Press, 2023. doi: 10.1609/aaai.v37i7.25960.

Du Q. Huynh, R. Hartley, and Anders Heyden. Outlier correction in image sequences for the affine camera. In ICCV, pages 585-590. IEEE Computer Society, 2003.

Wenhao Jiang, Feiping Nie, and Heng Huang. Robust dictionary learning with capped l1-norm. In Qiang Yang and Michael J. Wooldridge, editors, IJCAI 2015, pages 3590-3596. AAAI Press, 2015.

Zhao Kang, Wangtao Zhou, Zhitong Zhao, Junming Shao, Meng Han, and Zenglin Xu. Large-scale multi-view subspace clustering in linear time. In AAAI, pages 4412-4419. AAAI Press, 2020. doi: 10.1609/aaai.v34i04.5867.

Raghunandan H. Keshavan, Andrea Montanari, and Sewoong Oh. Matrix completion from a few entries. IEEE Trans. Inf. Theory, 56(6):2980-2998, 2010. doi: 10.1109/TIT.2010.2046205.

Dongkwan Kim and Alice Oh. How to find your friendly neighborhood: Graph attention design with self-supervision. In ICLR. OpenReview.net, 2021.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR (Poster). OpenReview.net, 2017.

Yehuda Koren, Robert M. Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, 2009. doi: 10.1109/MC.2009.263.

Xiang Li, Renyu Zhu, Yao Cheng, Caihua Shan, Siqiang Luo, Dongsheng Li, and Weining Qian. Finding global homophily in graph neural networks when meeting heterophily. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 13242-13256. PMLR, 2022.

Langzhang Liang, Zenglin Xu, Zixing Song, Irwin King, Yuan Qi, and Jieping Ye. Tackling long-tailed distribution issue in graph neural networks via normalization. IEEE Transactions on Knowledge and Data Engineering, 2023.

Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao, and Ser-Nam Lim. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In NeurIPS, pages 20887-20902, 2021.

Guangcan Liu, Zhouchen Lin, and Yong Yu. Robust subspace segmentation by low-rank representation. In ICML, pages 663-670. Omnipress, 2010.

Meng Liu, Hongyang Gao, and Shuiwang Ji. Towards deeper graph neural networks. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD '20, pages 338-348. ACM, 2020. doi: 10.1145/3394486.3403076.

Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wen Chang, and Doina Precup. Is heterophily A real nightmare for graph neural networks to do node classification? CoRR, abs/2109.05641, 2021.

Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. J. Mach. Learn. Res., 11:2287-2322, 2010. doi: 10.5555/ 1756006.1859931.

Feiping Nie, Jianjun Yuan, and Heng Huang. Optimal mean robust principal component analysis. In ICML 2014, volume 32 of JMLR Workshop and Conference Proceedings, pages 1062-1070. JMLR.org, 2014.

Feiping Nie, Zhouyuan Huo, and Heng Huang. Joint capped norms minimization for robust matrix recovery. In Carles Sierra, editor, IJCAI 2017, pages 2557-2563. ijcai.org, 2017. doi: 10.24963/ ijcai.2017/356.

Takayuki Okatani and Koichiro Deguchi. On the wiberg algorithm for matrix factorization in the presence of missing components. Int. J. Comput. Vis., 72(3):329-337, 2007.

Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gcn: Geometric graph convolutional networks. In ICLR. OpenReview.net, 2020.

Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of gnns under heterophily: Are we really making progress? In ICLR 2023. OpenReview.net, 2023.

Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. J. Complex Networks, 9(2), 2021.

Heung-Yeung Shum, Katsushi Ikeuchi, and Raj Reddy. Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Trans. Pattern Anal. Mach. Intell., 17(9):854-867, 1995.

Yunchong Song, Chenghu Zhou, Xinbing Wang, and Zhouhan Lin. Ordered GNN: ordering message passing to deal with heterophily and over-smoothing. CoRR, abs/2302.01524, 2023. doi: 10. 48550/arXiv.2302.01524.

Zixing Song and Irwin King. Hierarchical heterogeneous graph attention network for syntax-aware summarization. In AAAI, pages 11340-11348. AAAI Press, 2022.

Nathan Srebro, Jason D. M. Rennie, and Tommi S. Jaakkola. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 17 (NIPS 2004), pages 1329-1336, 2004.

Susheel Suresh, Vinith Budde, Jennifer Neville, Pan Li, and Jianzhu Ma. Breaking the limit of graph neural networks by improving the assortativity of graphs with local mixing patterns. In KDD, pages 1541-1551. ACM, 2021.

Jiliang Tang, Xia Hu, Huiji Gao, and Huan Liu. Exploiting local and global social context for recommendation. In IJCAI, pages 2712-2718. IJCAI/AAAI, 2013.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR (Poster). OpenReview.net, 2018.

T Wiberg. Computation of principal components when data are missing. In Proc. of Second Symp. Computational Statistics, pages 229-236, 1976.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net, 2019a.

Zenglin Xu, Feng Yan, and Yuan (Alan) Qi. Bayesian nonparametric models for multiway data analysis. IEEE Trans. Pattern Anal. Mach. Intell., 37(2):475-487, 2015.

Zenglin Xu, Bin Liu, Shandian Zhe, Haoli Bai, Zihan Wang, and Jennifer Neville. Variational random function model for network modeling. IEEE Trans. Neural Networks Learn. Syst., 30(1):318-324, 2019b.

Yujun Yan, Milad Hashemi, Kevin Swersky, Yaoqing Yang, and Danai Koutra. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. CoRR, abs/2102.06462, 2021.

Liang Yang, Mengzhe Li, Liyang Liu, Bingxin Niu, Chuan Wang, Xiaochun Cao, and Yuanfang Guo. Diverse message passing for attribute with heterophily. In NeurIPS, pages 4751-4763, 2021.

Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In ICML, volume 48 of JMLR Workshop and Conference Proceedings, pages 40-48. JMLR.org, 2016.

Arthur B. Yeh. A modern introduction to probability and statistics. Technometrics, 49(3):359, 2007. doi: 10.1198/tech.2007.s502.

Xin Zheng, Yixin Liu, Shirui Pan, Miao Zhang, Di Jin, and Philip S. Yu. Graph neural networks for graphs with heterophily: A survey. CoRR, abs/2202.07082, 2022.

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In NeurIPS, 2020.

Jiong Zhu, Ryan A. Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K. Ahmed, and Danai Koutra. Graph neural networks with heterophily. In AAAI, pages 11168-11176. AAAI Press, 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a Low-Rank Graph Neural Network (LRGNN) that can model both homophilous and heterophilous graphs via signed propogation (two nodes with the same class label share a positive edge and with different class share a negative edge). The authors predict the label relationship matrix by solving a robust low-rank matrix approximation problem, which results in a block diagonal structure and varying distributions of within-class and between-class entries.\n\nSTRENGTHS:\nClarity: The paper is generally easy to follow.\n\nNovelty: The authors propose a low-rank matrix approximation technique for predicting the label relationship matrix and enhancing the representation power of nodes. This technique is designed to utilize the low-rank properties of weakly-balanced graphs. \n\nExperiments: The authors have conducted extensive experiments on both synthetic and real-world graphs to evaluate the proposed method. The results indicate that LRGNN outperforms other baseline methods on both synthetic and real-world graphs.\n\nPresentation: The paper is well-structured and many visualization analysis are provided to better assess the qualitative and quantitive analysis of the proposed model.\n\nWEAKNESSES:\nAssumption of Uniform Distribution: The main theoretical support for the approach assumes a uniform distribution of node degrees. However, in practice, node degree distributions frequently follow a power-law distribution that is non-uniform. Although the main results of this paper are obtained using real-world graphs with non-uniform node degree distributions, it is unclear how this affects the theoretical results. \n\nDependence on Pseudo Labels: It is mentioned that LRGNN’s performance is affected by the accuracy of the generated pseudo labels. Although the supplementary material provides an experiment that indicates LRGNN demonstrates a non-sensitive response to this impact, further exploration and discussion are needed to fully investigate the theoretical implications. Additionally, the motivation of using signed labels for edges is not strong enough becuase 1) only applies for homogenous graphs. 2) When the # of class labels is high, such strategy may lose a lot class-related information so it may not generalize.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces sign links to model both the homophilous and heterophilous relationships in the global graph, thus extending the applicability of GNNs beyond heterophily. Additionally, the paper leverages the weak balance theory to support the existence of a global low-rank structure in the signed graph and formulates sign link prediction as a robust low-rank matrix approximation problem. Experimental results demonstrate the effectiveness and performance improvement of the proposed approach.\n\nSTRENGTHS:\nClarity:\n1. The submission demonstrates clear writing, readability, and a well-organized structure.\n2. The supplementary material provides additional details that enhance the reproducibility of the results.\n\nQuality:\n1. The claims put forth in the submission are strongly backed by theoretical and empirical analyses.\n2. The inclusion of the structural balance theory and conducting accuracy comparison, ablation, and visualization experiments significantly bolsters the overall quality of the submission.\n\nWEAKNESSES:\nOriginality\n1. The originality of the contributions can be strengthened by explaining the differences and comparing them with several relevant existing works, such as FAGCN [1], HOG-GCN [2], etc.\n2. In particular, the description in the paper about how the proposed signed GNN can better work for heterophily problems is insufficient.\n\nSignificance\n1. Most heterophily datasets considered in the paper have issues (e.g., small scale, train-test data leakage, etc.) as pointed out by recent publications on high-quality heterophily datasets [3].\n2. Strengthening the significance of the contributions can be achieved by incorporating important references and conducting experimental comparisons.\n\n[1]Bo, D., Wang, X., Shi, C., & Shen, H. (2021). Beyond low-frequency information in graph convolutional networks. In AAAI, 35, 3950-3957.\n\n[2]Wang, T., Jin, D., Wang, R., He, D., & Huang, Y. (2022). Powerful graph convolutional networks with adaptive propagation mechanism for homophily and heterophily. In AAAI, 36, 4210-4218.\n\n[3]Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., & Prokhorenkova, L. (2023). A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. In ICLR.\n\nQUESTIONS:\n1. The idea behind LRGNN bears similarities to FAGCN; however, the author fails to provide a detailed explanation of the differences between them.\n2. There is a lack of experimental comparisons with state-of-the-art (SOTA) models such as FAGCN [1], DMP [4], and Ordered GNN [5].\n3. The paper lacks experimental comparisons with other popular large-scale datasets, as mentioned in [6], such as pokec, snap-patents, and twitch-gamers.\n4. Figure 3 solely displays the recovery loss on heterophily graphs. It would be valuable to include the results on homophily graphs as well.\n\n[4]Yang, L., Li, M., Liu, L., Wang, C., Cao, X., & Guo, Y. (2021). Diverse message passing for attribute with heterophily. In NeurIPS, 34, 4751-4763.\n\n[5]Song, Y., Zhou, C., Wang, X., & Lin, Z. (2023). Ordered GNN: Ordering message passing to deal with heterophily and over-smoothing. In ICLR.\n\n[6]Lim, D., Hohne, F., Li, X., Huang, S. L., Gupta, V., Bhalerao, O., & Lim, S. N. (2021). Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In NeurIPS, 34, 20887-20902.\n\n---------\nUpdate: Most of the above concerns have been addressed.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGraph Neural Networks (GNNs) do not operate effectively when the graph is heterophilic with respect to the task. Addressing this issue is an important problem. The paper starts by analyzing SignedGNN model and demonstrates through a simple analysis that SignedGNNs might be implicitly implementing balance theory (at least for binary scenario). This fails in multi-class setting because e-e-f is not right. To address this issue, the aggregation is modified to remove the e-e-f aggregation, creating the weak-balanced model. However, this requires the knowledge of friends and enemies. Another model is used to predict this relationship labels. The actual model incorporates this signedGCN model along with a matrix factorization component that compares low rank behavior with the learned representations. So, the learnt representations capture low rank matrix factorization behavior while aggregating over reasonably right neighbors.\n\nSTRENGTHS:\n1. **Easy Approach**: The proposed approach is fairly simple.\n2. **Clarity**: The paper was easy to read and follow.\n3. **Good Results**: The presented results are good.\n\nWEAKNESSES:\n1. **Weak Novelty**: The proposed approach is similar to GloGNN [A]. Also, the low rank pattern has been observed in some of the prior works [B]. [C] also builds a compatibility matrix to use in their model. However, what this paper has achieved is to put these ideas together in an effective way.\n2. **Unintuitive modeling choices**: The definition of $c_{ij}$ is quite unintuitive. While the individual terms have been explained, the way they are put together is inexplicable.\n3. **Large number of hyperparameters**: The number of hyperparameters go up significantly in this approach. It would be useful to have a hyperparameter sensitivity study.\n\n[A] Finding Global Homophily in Graph Neural Networks When Meeting Heterophily, ICML 2022.\n[B] Simple Truncated SVD based Model for Node Classification on Heterophilic Graphs, KDD 2021 Workshop on Deep Learning on Graphs.\n[C] Graph Neural Networks with Heterophily, AAAI 2021.\n\nQUESTIONS:\n1. While individual terms of $c_{ij}$ have been explained in the writing and some ablation study in the supplementary have been performed to show how well it works. It will still be good to understand why the three terms can be put in a very simple additive model combined with some values through Softmax.\n2. It will be good to perform some hyper-parameter sensitivity study.\n3. The \"Impact of the Accuracy of the Estimated Signed Adjacency Matrix\" was a very interesting section. It seems almost unintuitive that the model is robust to arbitrary corruption of the estimated signed adjacency matrix. However, would it not indicate that the proposed approach could also work with SignedGNN instead of WB-SignedGCN?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper consider predicting the global label relationship matrix as a low rank matrix completion problem. This formulation is based on the observation that the rank of the global label relationship matrix shall equal the number of classes. The authors proposed an efficient solution to the low rank matrix completion problem and the predicted global relationship matrix is used to mix features into the node embedding for predicting node labels. The authors compared the proposed method with a series of baselines and find the proposed method consistently outperform or on-par with the state-of-the-art methods.\n\nBesides the LRGNN, the authors also prove that a variant of signed GNNs shows a tendency to follow the structural balance\n72 theory, which can be enhanced by eliminating the faulty assumption from the model design.\n\nSTRENGTHS:\n1. The idea of computing the global label relationship matrix and then use it as a one-hop GNN is quite novel.\n2. The formulation of predicting the global label relationship matrix as a signed low rank matrix completion problem is elegant.\n3. The paper is extremely well-written and the logic well organized.\n\n*I am not able to verify the math behind the matrix completion solution, but the rest of the method looks sound to me*\n\nWEAKNESSES:\nThe weakness of the paper is well-discussed in the last section.\n\nQUESTIONS:\nIt is not clear to me how the \\tilde{A} entries are defined. In particular, assuming a graph has 30% of labels being known, then only roughly 9% of the edge signs are known. I am not sure how the rest of the values are determined, are they simply set to 0?\n\nFurthermore, I am not sure how intuitively matrix completion would help determine the label relation of a node with other nodes in the graph, if the graph has no edges that has a know sign. In particular, the authors states:\n\n*Finally, at least one observation is present per row and per column (\\tilde{A}_i,i = 1)*\n\nYet I am not sure why that information alone would be helpful.\n\nwould appreciate if you can provide more intuition\n\nLIMITATIONS:\n...", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the LRGNN method, which aims to enhance the performance on heterophilous graphs by re-constructing the signed relationships. The approach involves optimizing the low-rank signed matrix using the SVD-free LRMF strategy.\n\nSTRENGTHS:\nThe utilization of the low-rank signed relationship matrix in heterophilous graph representation is intriguing and promising.\n\nWEAKNESSES:\nWeaknesses:\n\n1. The limitations on the first contribution. \nOne of the limitations of this method is its reliance on the data split. The success of low-rank matrix completion is highly dependent on the availability and quality of the signed adjacency matrix and the observation set. In cases where the observation set is small, there may be nodes that have no neighbors selected, as mentioned in Line 140. This poses challenges in effectively guessing the values of U and V, potentially impacting the accuracy of the low-rank matrix completion process.\n\n2. Limited novelty:\nAlthough the method incorporates low-rank optimization, the overall novelty of the approach is somewhat limited. The designed method in Section 4 shares similarities with other existing methods. For instance, when compared to the general compatibility matrix used in [1,2], the primary difference lies in both matrices' objective to measure whether two nodes belong to the same class.\n\n3. Concerns with the experiments:  \na. Effectiveness of low-rank recovery of Z (Eq. 7 and 10):\nTo further justify the effectiveness of the low-rank recovery of Z, it is essential to provide evaluations for Eq. (7) and (10). Demonstrating the accuracy and reliability of the low-rank recovery process would strengthen the paper's claims.  \nb. Parameter numbers:  Including the parameter numbers in the paper would provide valuable insights, especially regarding the growth of the matrices U and V in correlation with the number of nodes. This information would contribute to a more comprehensive understanding of the method's scalability and resource requirements.\n\n\nDetailed comments:\n1. Fig.1: In panel t3, the color of the circle appears to be incorrect. Please ensure that the color \ncorresponds accurately to the represented class.\n\n2. Fig.4 Wisconsin: There seems to be a discrepancy in the edge numbers of in-class and between-class connections. Given that this dataset is heterophilous, the number of edges within the same class and between different classes should differ significantly. Please review and verify the edge counts for accuracy.\n\n[1] Graph Neural Networks with Heterophily.   \n[2] Powerful Graph Convolutioal Networks with Adaptive Propagation Mechanism for Homophily and Heterophily.\n\nQUESTIONS:\nPlease check the weakness\n\nLIMITATIONS:\nPlease check the weakness", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-20"}