Signed Graph Diffusion Network

1 INTRODUCTION . Given a signed social graph , how can we learn appropriate node representations to infer the signs of missing edges ? Signed social graphs model trust relationships between people with positive ( trust ) and negative ( distrust ) edges . Many online social services such as Epinions ( Guha et al. , 2004 ) and Slashdot ( Kunegis et al. , 2009 ) that allow users to express their opinions are naturally represented as signed social graphs . Such graphs have attracted considerable attention for diverse applications including link sign prediction ( Leskovec et al. , 2010a ; Kumar et al. , 2016 ) , node ranking ( Jung et al. , 2016 ; Li et al. , 2019b ) , community analysis ( Yang et al. , 2007 ; Chu et al. , 2016 ) , graph generation ( Derr et al. , 2018a ; Jung et al. , 2020 ) , and anomaly detection ( Kumar et al. , 2014 ) . Node representation learning is a fundamental building block for analyzing graph data , and many researchers have put tremendous efforts into developing effective models for unsigned graphs . Graph convolutional networks ( GCN ) and their variants ( Kipf & Welling , 2017 ; Velickovic et al. , 2018 ) have spurred great attention in machine learning community , and recent works ( Klicpera et al. , 2019 ; Li et al. , 2019a ) have demonstrated stunning progress by handling the performance degradation caused by over-smoothing ( Li et al. , 2018 ; Oono & Suzuki , 2020 ) ( i.e. , node representations become indistinguishable as the number of propagation increases ) or the vanishing gradient problem ( Li et al. , 2019a ) in the first generation of GCN models . However , all of these models have a limited performance on node representation learning in signed graphs since they only consider unsigned edges under the homophily assumption ( Kipf & Welling , 2017 ) . Many studies have been recently conducted to consider such signed edges , and they are categorized into network embedding and GCN-based models . Network embedding ( Kim et al. , 2018 ; Xu et al. , 2019b ) learns the representations of nodes by optimizing an unsupervised loss that primarily aims to locate two nodes ’ embeddings closely ( or far ) if they are positively ( or negatively ) connected . However , they are not trained jointly with a specific task in an end-to-end manner , i.e. , latent features and the task are trained separately . Thus , their performance is limited unless each of them is tuned delicately . GCN-based models ( Derr et al. , 2018b ; Li et al. , 2020 ) have extended the graph convolutions to signed graphs using balance theory ( Holland & Leinhardt , 1971 ) in order to properly propagate node features on signed edges . However , these models are directly extended from existing GCNs without consideration of the over-smoothing problem that degrades their performance . This problem hinders them from exploiting more information from multi-hop neighbors for learning node features in signed graphs . We propose SGDNET ( SIGNED GRAPH DIFFUSION NETWORK ) , a novel graph neural network for node representation learning in signed graphs . Our main contributions are summarized as follows : • End-to-end learning . We design SGDNET that performs end-to-end node representation learning . Given a signed graph , SGDNET produces node embeddings through multiple signed graph diffusion ( SGD ) layers ( Figure 1 ( a ) ) , which are fed into a loss function of a specific task such as link sign prediction . • Novel feature diffusion . We propose a signed random walk diffusion method that prop- agates node embeddings on signed edges based on random walks considering signs , and injects local features ( Figure 1 ( c ) ) . This enables SGDNET to learn distinguishable node representations considering multi-hop neighbors while preserving local information . • Experiments . Extensive experiments show that SGDNET effectively learns node represen- tations of signed social graphs for link sign prediction , giving at least 3.9 % higher accuracy than the state-of-the-art models in real datasets ( Table 2 ) . 2 RELATED WORK . 2.1 GRAPH CONVOLUTIONAL NETWORKS ON UNSIGNED GRAPHS . Graph convolutional network ( GCN ) ( Kipf & Welling , 2017 ) models the latent representation of a node by employing a convolutional operation on the features of its neighbors . Various GCN-based approaches ( Kipf & Welling , 2017 ; Velickovic et al. , 2018 ; Hamilton et al. , 2017 ) have aroused considerable attention since they enable diverse graph supervised tasks ( Kipf & Welling , 2017 ; Yao et al. , 2019 ; Xu et al. , 2019a ) to be performed concisely under an end-to-end framework . However , the first generation of GCN models exhibit performance degradation due to the over-smoothing and vanishing gradient problems . Several works ( Li et al. , 2018 ; Oono & Suzuki , 2020 ) have theoretically revealed the over-smoothing problem . Also , Li et al . ( Li et al. , 2019a ) have empirically shown that stacking more GCN layers leads to the vanishing gradient problem as in convolutional neural networks ( He et al. , 2016 ) . Consequently , most GCN-based models ( Kipf & Welling , 2017 ; Velickovic et al. , 2018 ; Hamilton et al. , 2017 ) are shallow ; i.e. , they do not use the feature information in faraway nodes when modeling node embeddings . A recent research direction aims at resolving the limitation . Klicpera et al . ( Klicpera et al. , 2019 ) proposed APPNP exploiting Personalized PageRank ( Jeh & Widom , 2003 ) to not only propagate hidden node embeddings far but also preserve local features , thereby preventing aggregated features from being over-smoothed . Li et al . ( Li et al. , 2019a ) suggested ResGCN adding skip connections between GCN layers , as in ResNet ( He et al. , 2016 ) . However , all of these models do not provide how to use signed edges since they are based on the homophily assumption ( Kipf & Welling , 2017 ) , i.e. , users having connections are likely to be similar , which is not valid for negative edges . As opposed to the homophily , negative edges have the semantics of heterophily ( Rogers , 2010 ) , i.e. , users having connections are dissimilar . Although these methods can still be applied to signed graphs by ignoring the edge signs , their trained features have limited capacity . 2.2 NETWORK EMBEDDING AND GRAPH CONVOLUTIONAL NETWORKS ON SIGNED GRAPHS . Traditional methods on network embedding extract latent node features specialized for signed graphs in an unsupervised manner . Kim et al . ( Kim et al. , 2018 ) proposed SIDE which optimizes a likelihood over direct and indirect signed connections on truncated random walks sampled from a signed graph . Xu et al . ( Xu et al. , 2019b ) developed SLF considering positive , negative , and non-linked relationships between nodes to learn non-negative node embeddings . However , such approaches are not end-to-end , i.e. , they are not directly optimized for solving a supervised task such as link prediction . There are recent progresses on end-to-end learning on signed networks under the GCN framework . Derr et al . ( Derr et al. , 2018b ) proposed SGCN which extends the GCN mechanism to signed graphs considering balanced and unbalanced relationships supported by structural balance theory ( Holland & Leinhardt , 1971 ) . Yu et al . ( Li et al. , 2020 ) developed SNEA using attention techniques to reveal the importance of these relationships . However , such state-of-the-art models do not consider the over-smoothing problem since they are directly extended from GCN . 3 PROPOSED METHOD . We propose SGDNET ( SIGNED GRAPH DIFFUSION NETWORK ) , a novel end-to-end model for node representation learning in signed graphs . Our SGDNET aims to properly aggregate node features on signed edges , and to effectively use the features of multi-hop neighbors so that generated features are not over-smoothed . Our main ideas are to diffuse node features along random walks considering the signs of edges , and to inject local node features at each aggregation . Figure 1 depicts the overall architecture of SGDNET . Given a signed graph G and initial node features X ∈ Rn×d0 as shown in Figure 1 ( a ) , SGDNET extracts the final node embeddings H ( L ) ∈ Rn×dL through multiple SGD layers where n is the number of nodes , L is the number of SGD layers , and dl is the embedding dimension of the l-th layer . Then , H ( L ) is fed into a loss function of a specific task so that they are jointly trained in an end-to-end framework . Given H ( l−1 ) , the l-th SGD layer aims to learn H ( l ) based on feature transformations and signed random walk diffusion Fd ( · ) as shown in Figure 1 ( b ) . The layer also uses the skip connection to prevent the vanishing gradient problem when the depth of SGDNET increases . Figure 1 ( c ) illustrates the intuition behind the signed random walk diffusion . Each node has two features corresponding to positive and negative surfers , respectively . The surfer flips its sign when moving along negative edges , while the sign is kept along positive edges . For example , the positive ( or negative ) surfer becomes positive at node v if it moves from a positively connected node u ( or a negatively connected node t ) . As a result , the aggregated features at node v become similar to those connected by positive edges ( e.g. , node u ) , and different from those connected by negative edges ( e.g. , node t ) . In other words , it satisfies homophily and heterophily at the same time while unsigned GCNs can not handle the heterophily of negative edges . Furthermore , we inject the local feature ( i.e. , the input feature of the module ) of node v at each aggregation so that the resulting features remain distinguishable during the diffusion . 3.1 SIGNED GRAPH DIFFUSION LAYER . Given a signed graph G and the node embeddings H ( l−1 ) from the previous layer , the l-th SGD layer learns new embeddings H ( l ) as shown in Figure 1 ( b ) . It first transforms H ( l−1 ) into hidden features H̃ ( l ) as H̃ ( l ) = H ( l−1 ) W ( l ) t with a learnable parameter W ( l ) t ∈ Rdl−1×dl . Then , it applies the signed random walk diffusion which is represented as the function Fd ( G , H̃ ( l ) ) that returns P ( l ) ∈ Rn×dl and M ( l ) ∈ Rn×dl as the positive and the negative embeddings , respectively ( details in Section 3.2 ) . The embeddings are concatenated and transformed as follows : H ( l ) = φ ( [ P ( l ) ||M ( l ) ] W ( l ) n + H ( l−1 ) ) ( 1 ) where φ ( · ) is a non-linear activator such as tanh , || denotes horizontal concatenation of two matrices , and W ( l ) n ∈ R2dl×dl is a trainable weight matrix that learns a relationship between P ( l ) and M ( l ) . We use the skip connection ( He et al. , 2016 ; Li et al. , 2019a ) with H ( l−1 ) in Equation ( 1 ) to avoid the vanishing gradient issue which frequently occurs when multiple layers are stacked .

Summary: In this paper, the author studied the problem of node embedding in signed networks. The authors proposed SGDNet which combines the idea of diffusion/random work in signed networks and Residual connection in GCN. The network is trained directly with classification loss on edge sign prediction. The authors carried out extensive experiments on several real-world networks with comparison to several state-of-the-art methods. The proposed method showed superior performance in the sign prediction task.