Weighted Line Graph Convolutional Networks

1 INTRODUCTION . Graph neural networks ( Gori et al. , 2005 ; Scarselli et al. , 2009 ; Hamilton et al. , 2017 ) have shown to be competent in solving challenging tasks in the field of network embedding . Many tasks have been significantly advanced by graph deep learning methods such as node classification tasks ( Kipf & Welling , 2017 ; Veličković et al. , 2017 ; Gao et al. , 2018 ) , graph classification tasks ( Ying et al. , 2018 ; Zhang et al. , 2018 ) , link prediction tasks ( Zhang & Chen , 2018 ; Zhou et al. , 2019 ) , and community detection tasks ( Chen et al. , 2019 ) . Currently , most graph neural networks capture the relationships among nodes through message passing operations . Recently , some works ( Chen et al. , 2019 ) use extra graph structures such as line graphs to enhance message passing operations in graph neural networks from different graph perspectives . A line graph is a graph that is derived from an original graph to represent connectivity between edges in the original graph . Since line graphs can encode the topology information , message passing operations on line graphs can enhance network embeddings in graph neural networks . However , graph neural networks that leverage line graph structures need to deal with two challenging issues ; those are bias and inefficiency . Topology information in original graphs is encoded in line graphs but in a biased way . In particular , node features are either overstated or understated depending on their degrees . Besides , line graphs can be much bigger graphs than original graphs depending on the graph density . Message passing operations of graph neural networks on line graphs lead to significant use of computational resources . In this work , we propose to construct a weighted line graph that can correct biases in encoded topology information of line graphs . To this end , we assign each edge in a line graph a normalized weight such that each node in the line graph has a weighted degree of 2 . In this weighted line graph , the dynamics of node features are the same as those in its original graph . Based on our weighted line graph , we propose a weighted line graph convolution layer ( WLGCL ) that performs a message passing operation on both original graph structures and weighted line graph structures . To address inefficiency issues existing in graph neural networks that use line graph structures , we further propose to implement our WLGCL via an incidence matrix , which can dramatically reduce the usage of computational resources . Based on our WLGCL , we build a family of weighted line graph convolutional networks ( WLGCNs ) . We evaluate our methods on graph classification tasks and show that WLGCNs consistently outperform previous state-of-the-art models . Experiments on simulated data demonstrate the efficiency advantage of our implementation . 2 BACKGROUND AND RELATED WORK . In graph theory , a line graph is a graph derived from an undirected graph . It represents the connectivity among edges in the original graph . Given a graph G , the corresponding line graph L ( G ) is constructed by using edges in G as vertices in L ( G ) . Two nodes in L ( G ) are adjacent if they share a common end node in the graph G ( Golumbic , 2004 ) . Note that the edges ( a , b ) and ( b , a ) in an undirected graph G correspond to the same vertex in the line graph L ( G ) . The Whitney graph isomorphism theorem ( Thatte , 2005 ) stated that a line graph has a one-to-one correspondence to its original graph . This theorem guarantees that the line graph can encode the topology information in the original graph . Recently , some works ( Monti et al. , 2018 ; Chen et al. , 2019 ; Bandyopadhyay et al. , 2019 ; Jiang et al. , 2019 ) proposes to use the line graph structure to enhance the message passing operations in graph neural networks . Since the line graph can encode the topology information , the message passing on the line graph can enhance the network embeddings in graph neural networks . In graph neural networks that use line graph structures , features are passed and transformed in both the original graph structures and the line graph structures , thereby leading to better feature learnings and performances . 3 WEIGHTED LINE GRAPH CONVOLUTIONAL NETWORKS . In this work , we propose the weighted line graph to address the bias in the line graph when encoding graph topology information . Based on our weighted line graph , we propose the weighted line graph convolution layer ( WLGCL ) for better feature learning by leveraging line graph structures . Besides , graph neural networks using line graphs consume excessive computational resources . To solve the inefficiency issue , we propose to use the incidence matrix to implement the WLGCL , which can dramatically reduce the usage of computational resources . 3.1 BENEFIT AND BIAS OF LINE GRAPH REPRESENTATIONS . In this section , we describe the benefit and bias of using line graph representations . Benefit In message-passing operations , edges are usually given equal importance and edge features are not well explored . This can constrain the capacity of GNNs , especially on graphs with edge features . In the chemistry domain , a compound can be converted into a graph , where atoms are nodes and chemical bonds are edges . On such kinds of graphs , edges have different properties and thus different importance . However , message-passing operations underestimate the importance of edges . To address this issue , the line graph structure can be used to leverage edge features and different edge importance ( Jiang et al. , 2019 ; Chen et al. , 2019 ; Zhu et al. , 2019 ) . The line graph , by its nature , enables graph neural networks to encode and propagate edge features in the graph . The line graph neural networks that take advantage of line graph structures have shown to be promising on graph-related tasks ( Chen et al. , 2019 ; Xiong et al. , 2019 ; Yao et al. , 2019 ) . By encoding node and edge features simultaneously , line graph neural networks enhance the feature learning on graphs . Bias According to the Whitney graph isomorphism theorem , the line graph L ( G ) encodes the topology information of the original graph G , but the dynamics and topology of G are not correctly represented in L ( G ) ( Evans & Lambiotte , 2009 ) . As described in the previous section , each edge in the graph G corresponds to a vertex in the line graph L ( G ) . The features of each edge contain features of its two end nodes . A vertex with a degree d in the original graph G will generate d× ( d− 1 ) /2 edges in the line graph L ( G ) . The message passing frequency of this node ’ s features will change from O ( d ) in the original graph to O ( d2 ) in the line graph . From this point , the line graph encodes the topology information in the original graph but in a biased way . In the original graph , a node ’ s features will be passed to d neighbors . But in the corresponding line graph , the information will be passed to d× ( d− 1 ) /2 nodes . The topology structure in the line graph L ( G ) will overstate the importance of features for nodes with high degrees in the graph . On the contrary , the nodes with smaller degrees will be relatively understated , thereby leading to biased topology information encoded in the line graph . Note that popular adjacency matrix normalization methods ( Kipf & Welling , 2017 ; Veličković et al. , 2017 ; Gao & Ji , 2019 ; Gong & Cheng , 2019 ) can not address this issue . 3.2 WEIGHTED LINE GRAPH . In the previous section , we show that the line graph L ( G ) constructed from the original graph G encodes biased graph topology information . To address this issue , we propose to construct a weighted line graph that assigns normalized weights to edges . In a regular line graph L ( G ) , each edge is assigned an equal weight of 1 , thereby leading to a biased encoding of the graph topology information . To correct the bias , we need to normalize edge weights in the line graph . Considering each edge in G has two ends , it is intuitive to normalize the weighted degree of the corresponding node in L ( G ) to 2 . To this end , the weight for an edge in the adjacency matrix F of L ( G ) is computed as : F ( a , b ) , ( b , c ) = { 1 Db if a 6= c 1 Db + 1Da , if a = c ( 1 ) where a , b , and c are nodes in the graph G , ( a , b ) and ( b , c ) are edges in the graph G that are connected by the node b. Db is the degree of the node b in the graph G. To facilitate the message passing operation , we add self-loops on the weighted line graph WL ( G ) . The weights for self-loop edges computed by the second case consider the fact that they are self-connected by both ends . Figure 2 illustrates an example of a graph and its corresponding weighted line graph . Theorem 1 . Given the edge weights in the weighted line graph WL ( G ) defined by Eq . ( 1 ) , the weighted degree for a node ( a , b ) in WL ( G ) is 2 . The proof of Theorem 1 is provided in the supplementary material . By constructing the weighted line graph with normalized edge weights defined in Eq . ( 1 ) , each node ( a , b ) has a weighted degree of 2 . Given a node a with a degree of d , it has d related edges in G and d related nodes in L ( G ) . The message passing frequency of node a ’ s features in the weighted line graph WL ( G ) is ∑d i=1 2 = O ( d ) , which is consistent with that in the original graph G. Thus , the weighted line graph encodes the topology information of the original graph in an unbiased way . 3.3 WEIGHTED LINE GRAPH CONVOLUTION LAYER . In this section , we propose the weighted line graph convolution layer ( WLGCL ) that leverages our proposed weighted line graph for feature representations learnings . In this layer , node features are passed and aggregated in both the original graph G and the corresponding weighted line graph WL ( G ) . Suppose an undirected attributed graph G has N nodes and E edges . Each node and each edge in the graph contains Cn and Ce features , respectively . In the layer ` , an adjacency matrix A ( ` ) ∈ RN×N , a node feature matrix X ( ` ) ∈ RN×Cn , and a edge feature matrix Y ( ` ) ∈ RE×Ce are used to represent the graph connectivity , node features , and edge features , respectively . Here , we construct the adjacency matrix F ( ` ) ∈ RE×E of the corresponding weighted line graph . The layer-wise propagation rule of the weighted line graph convolution layer ` is defined as : Ŷ ( ` ) = F ( ` ) Y ( ` ) , ∈RE×Ce ( 2 ) K ( ` ) L = B ( ` ) Ŷ ( ` ) , ∈RN×Ce ( 3 ) K ( ` ) = A ( ` ) X ( ` ) , ∈RN×Cn ( 4 ) X ( ` +1 ) = K ( ` ) W ( ` ) +K ( ` ) L W ( ` ) L , ∈R N×C′ ( 5 ) where W ( ` ) ∈ RCn×C′ and W ( ` ) L ∈ RCe×C ′ are matrices of trainable parameters . B ( ` ) ∈ RN×E is the incidence matrix of the graph G that shows the connectivity between nodes and edges . To enable message passing on the line graph L ( G ) , each edge in the graph G needs to have features . However , edge features are not available on some graphs . To address this issue , we can compute features for an edge ( a , b ) by summing up features of its two end nodes : Y ( ` ) ( a , b ) = X ( ` ) a +X ( ` ) b . Here , we use the summation operation to ensure the permutation invariant property in this layer . Then , we perform message passing and aggregation on the line graph in Eq . ( 2 ) . With updated edges features , Eq . ( 3 ) generates new nodes features with edge features Y ( ` ) . Eq . ( 4 ) performs feature passing and aggregation on the graph G , which leads to aggregated nodes features K ( ` ) . In Eq . ( 5 ) , aggregated features from the graph G and the line graph L ( G ) are transformed and combined , which produces the output feature matrix X ( ` +1 ) . Note that we can apply popular adjacency matrix normalization methods ( Kipf & Welling , 2017 ) on the adjacency matrix A ( ` ) , the line graph adjacency matrix F ( ` ) , and the incidence matrix B ( ` ) . In the WLGCL , we use the line graph structure as a complement to the original graph structure , thereby leading to enhanced feature learnings . Here , we use a simple feature aggregation method as used in GCN ( Kipf & Welling , 2017 ) . Other complicated and advanced feature aggregation methods such as GAT ( Veličković et al. , 2017 ) can be easily applied by changing Eq . ( 2 ) and Eq . ( 4 ) accordingly . Figure 3 provides an illustration of our WLGCL .

Summary: This paper introduces a weighted line graph formulation (WLGCL) which corrects the over-counting ("bias") of high-degree node features in a line-graph based convolutional network. Further, the paper uses Incidence Matrix to implement WLGCL updates which reduces the space complexity ($O(N^4) \to O(N^3)$) and time complexity ($O(N^4 C) \to O(N^4)$) compared to the naive implementation. The paper shows empirical evaluation on downstream task of graph classification and shows gain in accuracy.