Weighted Line Graph Convolutional Networks

1 INTRODUCTION . Graph neural networks ( Gori et al. , 2005 ; Scarselli et al. , 2009 ; Hamilton et al. , 2017 ) have shown to be competent in solving challenging tasks in the field of network embedding . Many tasks have been significantly advanced by graph deep learning methods such as node classification tasks ( Kipf & Welling , 2017 ; Veličković et al. , 2017 ; Gao et al. , 2018 ) , graph classification tasks ( Ying et al. , 2018 ; Zhang et al. , 2018 ) , link prediction tasks ( Zhang & Chen , 2018 ; Zhou et al. , 2019 ) , and community detection tasks ( Chen et al. , 2019 ) . Currently , most graph neural networks capture the relationships among nodes through message passing operations . Recently , some works ( Chen et al. , 2019 ) use extra graph structures such as line graphs to enhance message passing operations in graph neural networks from different graph perspectives . A line graph is a graph that is derived from an original graph to represent connectivity between edges in the original graph . Since line graphs can encode the topology information , message passing operations on line graphs can enhance network embeddings in graph neural networks . However , graph neural networks that leverage line graph structures need to deal with two challenging issues ; those are bias and inefficiency . Topology information in original graphs is encoded in line graphs but in a biased way . In particular , node features are either overstated or understated depending on their degrees . Besides , line graphs can be much bigger graphs than original graphs depending on the graph density . Message passing operations of graph neural networks on line graphs lead to significant use of computational resources . In this work , we propose to construct a weighted line graph that can correct biases in encoded topology information of line graphs . To this end , we assign each edge in a line graph a normalized weight such that each node in the line graph has a weighted degree of 2 . In this weighted line graph , the dynamics of node features are the same as those in its original graph . Based on our weighted line graph , we propose a weighted line graph convolution layer ( WLGCL ) that performs a message passing operation on both original graph structures and weighted line graph structures . To address inefficiency issues existing in graph neural networks that use line graph structures , we further propose to implement our WLGCL via an incidence matrix , which can dramatically reduce the usage of computational resources . Based on our WLGCL , we build a family of weighted line graph convolutional networks ( WLGCNs ) . We evaluate our methods on graph classification tasks and show that WLGCNs consistently outperform previous state-of-the-art models . Experiments on simulated data demonstrate the efficiency advantage of our implementation . 2 BACKGROUND AND RELATED WORK . In graph theory , a line graph is a graph derived from an undirected graph . It represents the connectivity among edges in the original graph . Given a graph G , the corresponding line graph L ( G ) is constructed by using edges in G as vertices in L ( G ) . Two nodes in L ( G ) are adjacent if they share a common end node in the graph G ( Golumbic , 2004 ) . Note that the edges ( a , b ) and ( b , a ) in an undirected graph G correspond to the same vertex in the line graph L ( G ) . The Whitney graph isomorphism theorem ( Thatte , 2005 ) stated that a line graph has a one-to-one correspondence to its original graph . This theorem guarantees that the line graph can encode the topology information in the original graph . Recently , some works ( Monti et al. , 2018 ; Chen et al. , 2019 ; Bandyopadhyay et al. , 2019 ; Jiang et al. , 2019 ) proposes to use the line graph structure to enhance the message passing operations in graph neural networks . Since the line graph can encode the topology information , the message passing on the line graph can enhance the network embeddings in graph neural networks . In graph neural networks that use line graph structures , features are passed and transformed in both the original graph structures and the line graph structures , thereby leading to better feature learnings and performances . 3 WEIGHTED LINE GRAPH CONVOLUTIONAL NETWORKS . In this work , we propose the weighted line graph to address the bias in the line graph when encoding graph topology information . Based on our weighted line graph , we propose the weighted line graph convolution layer ( WLGCL ) for better feature learning by leveraging line graph structures . Besides , graph neural networks using line graphs consume excessive computational resources . To solve the inefficiency issue , we propose to use the incidence matrix to implement the WLGCL , which can dramatically reduce the usage of computational resources . 3.1 BENEFIT AND BIAS OF LINE GRAPH REPRESENTATIONS . In this section , we describe the benefit and bias of using line graph representations . Benefit In message-passing operations , edges are usually given equal importance and edge features are not well explored . This can constrain the capacity of GNNs , especially on graphs with edge features . In the chemistry domain , a compound can be converted into a graph , where atoms are nodes and chemical bonds are edges . On such kinds of graphs , edges have different properties and thus different importance . However , message-passing operations underestimate the importance of edges . To address this issue , the line graph structure can be used to leverage edge features and different edge importance ( Jiang et al. , 2019 ; Chen et al. , 2019 ; Zhu et al. , 2019 ) . The line graph , by its nature , enables graph neural networks to encode and propagate edge features in the graph . The line graph neural networks that take advantage of line graph structures have shown to be promising on graph-related tasks ( Chen et al. , 2019 ; Xiong et al. , 2019 ; Yao et al. , 2019 ) . By encoding node and edge features simultaneously , line graph neural networks enhance the feature learning on graphs . Bias According to the Whitney graph isomorphism theorem , the line graph L ( G ) encodes the topology information of the original graph G , but the dynamics and topology of G are not correctly represented in L ( G ) ( Evans & Lambiotte , 2009 ) . As described in the previous section , each edge in the graph G corresponds to a vertex in the line graph L ( G ) . The features of each edge contain features of its two end nodes . A vertex with a degree d in the original graph G will generate d× ( d− 1 ) /2 edges in the line graph L ( G ) . The message passing frequency of this node ’ s features will change from O ( d ) in the original graph to O ( d2 ) in the line graph . From this point , the line graph encodes the topology information in the original graph but in a biased way . In the original graph , a node ’ s features will be passed to d neighbors . But in the corresponding line graph , the information will be passed to d× ( d− 1 ) /2 nodes . The topology structure in the line graph L ( G ) will overstate the importance of features for nodes with high degrees in the graph . On the contrary , the nodes with smaller degrees will be relatively understated , thereby leading to biased topology information encoded in the line graph . Note that popular adjacency matrix normalization methods ( Kipf & Welling , 2017 ; Veličković et al. , 2017 ; Gao & Ji , 2019 ; Gong & Cheng , 2019 ) can not address this issue . 3.2 WEIGHTED LINE GRAPH . In the previous section , we show that the line graph L ( G ) constructed from the original graph G encodes biased graph topology information . To address this issue , we propose to construct a weighted line graph that assigns normalized weights to edges . In a regular line graph L ( G ) , each edge is assigned an equal weight of 1 , thereby leading to a biased encoding of the graph topology information . To correct the bias , we need to normalize edge weights in the line graph . Considering each edge in G has two ends , it is intuitive to normalize the weighted degree of the corresponding node in L ( G ) to 2 . To this end , the weight for an edge in the adjacency matrix F of L ( G ) is computed as : F ( a , b ) , ( b , c ) = { 1 Db if a 6= c 1 Db + 1Da , if a = c ( 1 ) where a , b , and c are nodes in the graph G , ( a , b ) and ( b , c ) are edges in the graph G that are connected by the node b. Db is the degree of the node b in the graph G. To facilitate the message passing operation , we add self-loops on the weighted line graph WL ( G ) . The weights for self-loop edges computed by the second case consider the fact that they are self-connected by both ends . Figure 2 illustrates an example of a graph and its corresponding weighted line graph . Theorem 1 . Given the edge weights in the weighted line graph WL ( G ) defined by Eq . ( 1 ) , the weighted degree for a node ( a , b ) in WL ( G ) is 2 . The proof of Theorem 1 is provided in the supplementary material . By constructing the weighted line graph with normalized edge weights defined in Eq . ( 1 ) , each node ( a , b ) has a weighted degree of 2 . Given a node a with a degree of d , it has d related edges in G and d related nodes in L ( G ) . The message passing frequency of node a ’ s features in the weighted line graph WL ( G ) is ∑d i=1 2 = O ( d ) , which is consistent with that in the original graph G. Thus , the weighted line graph encodes the topology information of the original graph in an unbiased way . 3.3 WEIGHTED LINE GRAPH CONVOLUTION LAYER . In this section , we propose the weighted line graph convolution layer ( WLGCL ) that leverages our proposed weighted line graph for feature representations learnings . In this layer , node features are passed and aggregated in both the original graph G and the corresponding weighted line graph WL ( G ) . Suppose an undirected attributed graph G has N nodes and E edges . Each node and each edge in the graph contains Cn and Ce features , respectively . In the layer ` , an adjacency matrix A ( ` ) ∈ RN×N , a node feature matrix X ( ` ) ∈ RN×Cn , and a edge feature matrix Y ( ` ) ∈ RE×Ce are used to represent the graph connectivity , node features , and edge features , respectively . Here , we construct the adjacency matrix F ( ` ) ∈ RE×E of the corresponding weighted line graph . The layer-wise propagation rule of the weighted line graph convolution layer ` is defined as : Ŷ ( ` ) = F ( ` ) Y ( ` ) , ∈RE×Ce ( 2 ) K ( ` ) L = B ( ` ) Ŷ ( ` ) , ∈RN×Ce ( 3 ) K ( ` ) = A ( ` ) X ( ` ) , ∈RN×Cn ( 4 ) X ( ` +1 ) = K ( ` ) W ( ` ) +K ( ` ) L W ( ` ) L , ∈R N×C′ ( 5 ) where W ( ` ) ∈ RCn×C′ and W ( ` ) L ∈ RCe×C ′ are matrices of trainable parameters . B ( ` ) ∈ RN×E is the incidence matrix of the graph G that shows the connectivity between nodes and edges . To enable message passing on the line graph L ( G ) , each edge in the graph G needs to have features . However , edge features are not available on some graphs . To address this issue , we can compute features for an edge ( a , b ) by summing up features of its two end nodes : Y ( ` ) ( a , b ) = X ( ` ) a +X ( ` ) b . Here , we use the summation operation to ensure the permutation invariant property in this layer . Then , we perform message passing and aggregation on the line graph in Eq . ( 2 ) . With updated edges features , Eq . ( 3 ) generates new nodes features with edge features Y ( ` ) . Eq . ( 4 ) performs feature passing and aggregation on the graph G , which leads to aggregated nodes features K ( ` ) . In Eq . ( 5 ) , aggregated features from the graph G and the line graph L ( G ) are transformed and combined , which produces the output feature matrix X ( ` +1 ) . Note that we can apply popular adjacency matrix normalization methods ( Kipf & Welling , 2017 ) on the adjacency matrix A ( ` ) , the line graph adjacency matrix F ( ` ) , and the incidence matrix B ( ` ) . In the WLGCL , we use the line graph structure as a complement to the original graph structure , thereby leading to enhanced feature learnings . Here , we use a simple feature aggregation method as used in GCN ( Kipf & Welling , 2017 ) . Other complicated and advanced feature aggregation methods such as GAT ( Veličković et al. , 2017 ) can be easily applied by changing Eq . ( 2 ) and Eq . ( 4 ) accordingly . Figure 3 provides an illustration of our WLGCL .

Summary: The paper proposed a GNN model based on a weighted line graph, which adds weights to the line graph for the original input graph in a node/graph property prediction task. The line graph is a graph built on the original graph but with edges as nodes. A new convolution called weighted line graph convolution layer (WLGCL) is proposed to overcome the issue of "biased topological information" of the line graph. The weights for the line graph in WLGCL are computed based on the node degree of the original graph, which implies the node degree in the line graph is always 2. The WLGCL can be implemented for different kinds of graph convolution, which rule incorporates graph connectivity, node features and edge features.