KS-GNN: Keywords Search over Incomplete Graphs via Graphs Neural Network

1 Introduction . Keyword search is an important research topic which allows users to provide query keywords and returns the most relevant results . The keyword search over graph data [ 1 ] usually retrieves top-k subtrees or subgraphs which contain all the query keywords ranked according to some criteria . For example , He et al . [ 2 ] propose a general scoring function considering both graph structure and content , and they aim to find top-k nodes where each node can reach all query keywords , and the sum of its shortest path distances to these keywords is as small as possible . This ranking method is commonly used in later graph keyword search works [ 3 , 4 ] . ∗Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . All existing studies assume that the graph data is complete and has no missing information . However , real-world graphs usually have some missing edges [ 5 ] and missing attributes on some nodes [ 6 ] . This renders previous graph keyword search methods unable to find exact answers when dealing with such incomplete graphs . Figure 1 shows an example keyword search query over graphs . Given q= { c , e , f } , on the left graph G with no missing information , the best node is v4 , since it contains keywords c and f , it can reach v5 containing e , and its sum of the shortest path distances to all query keywords is the smallest which is 1 ( the shortest distances of v4 to c , e , and f are 0 , 1 , and 0 ) . However , on the right graph G′ which has a missing edge and a node with missing attributes , the result becomes v1 , v2 or v6 , and the subtree consists of { v1 , v2 , v6 } , with a total distance of 2 . To handle the missing information , one simple idea is to first utilize some state-of-the-art graph completion models ( such as SAT [ 6 ] ) to predict the missing information and then apply the existing algorithms ( such as BLINKS [ 2 ] ) to find the answers from the graph with predicted keywords and edges . However , such a completed graph contains many noises and errors comparing with the original graph , and thus this method has poor performance as shown in our experimental study . To capture the latent information of the incomplete graphs , we propose to utilize the graph neural network ( GNN ) for graph keyword search . GNN has been widely applied in tasks such as link prediction [ 7 , 8 , 9 ] , node classification [ 10 , 11 ] , and node clustering [ 6 , 12 ] , but existing models can not be directly applied to keyword search since they usually embed all the features ( keywords ) of a node into a single vector and they can not obtain the representation for the individual query keywords . We firstly design two naïve approaches based on GNN and dimension reduction . To achieve better performance , we propose a novel auto-encoder and GNN-based model using the message passing mechanism , called KS-GNN . The model mainly consists of three components : an encoder that transforms the original keyword information to low-dimensional embedding vectors ; a decoder that aims to reconstruct the high-dimensional representation of keywords from the embedding ; a message passing-based aggregation mechanism that preserves the shortest path information between keywords and the target node . Different from the existing graph keyword search works , we propose to leverage GNN to obtain representative node embedding that contains the keyword information , taking the latent graph structure , keyword distribution , and keyword frequency information into account . Meantime , the proposed KS-GNN is able to encode the input query as a low-dimensional vector by its learned powerful encoder , and the results are obtained by computing the similarity between the query embedding and node embeddings . This also speeds up query processing time to linear complexity . The main contributions of our approach are as follows : • To our best knowledge , this is the first work on keyword search in graphs with missing information . • We propose an auto-encoder and GNN-based model KS-GNN to solve the problem effectively without having to know the complete information of the input graph . • The experimental results on four real-world datasets show that our proposed model consistently outperforms several baseline methods . 2 Related Work . Keyword Search in Graphs . Keyword search over graph data aims to find the top-k subtrees or subgraphs according to some ranking criteria . The conventional methods design algorithms assuming that the graphs have complete information . For example , DBXplorer [ 13 ] proposes to utilize the number of the answer ’ s edges as the scoring function . BANKS [ 14 ] model tuples as nodes in a graph and then performs keyword search using proximity-based ranking . He et al . [ 2 ] propose a general ranking function considering both graph structure and content . BLINKS also builds an efficient bi-level index structure to improve efficiency . Kargar and An , motivated by the Steiner tree problem , use the total edge weight in ranking answers [ 15 ] . There also exists studies on keyword search in temporal graphs [ 16 ] , uncertain graphs [ 17 ] , knowledge graphs [ 18 ] , RDF graphs [ 19 ] , etc . However , real-world graphs may usually be incomplete . As all the state-of-art keyword search methods retrieve the exact answer , the missing information ( keywords or edges ) imposes a significant effect on the query results . To address this issue , we propose a graph representation learning-based solution to solve the top-k keyword search problem on incomplete graphs . Graph Neural Networks . As a powerful branch of graph representation learning methods , the graph neural network has been widely used in recent years due to its excellent performance . The models in the early years are usually based on the so-called graph convolutional network ( GCN ) [ 20 , 21 , 22 ] , which is based on the Fourier transform theory of graphs developed by Shuman et al . [ 23 ] . However , research in recent years has shown that the GCN-based methods can be represented by the message passing mechanism , which is more consistent with the experimental results [ 24 ] . The Graph Attention Network ( GAT ) [ 11 ] is one of the representatives of graph neural networks based on the message passing mechanism . GAT introduces the attention mechanism to calculate the attention coefficient between nodes and then uses it to assign different weights to neighbors ’ information . Based on the auto-encoder and GCN , Graph auto-encoder ( GAE ) [ 25 ] is proposed to reconstruct the adjacency matrix . Moreover , there are some GNN-based works that aim to predict and impute missing data to a data matrix [ 6 , 26 , 27 ] . However , all the methods mentioned above can not directly handle the graph keyword search problem . To our best knowledge , this is the first work that leverages GNN to process keyword search on incomplete graphs . 3 Problem Statement . A graph keyword search query q = ( wq1 , wq2 , ... , wqm ) contains a set of query keywords , and it searches relevant results from a graph2 G = ( V , E , W ) , where each node v ∈ V , each edge e ∈ E , and each keyword w ∈ W . For each node v , it is associated with a set of keywords { wv1 , wv2 , ... , wvn } . In this work , we study the keyword search problem over an incomplete graph . To alleviate the effect of the missing information to keyword search over incomplete graphs , we assume that in the original graph the query results are obtained by applying the BLINKS scheme [ 2 ] ( a commonly used graph keyword search method ) . Given a query q , let s ( v , q ) denote the score of the node v. According to [ 2 , 3 , 28 , 29 , 30 ] , s ( v , q ) = ∑m i=1 distmin ( v , wqi ) , where distmin ( v , wqi ) computes the shortest path distance from node v to a node containing wqi . BLINKS aims to find top-k nodes where each node can reach all query keywords in the graph , and the scores of the k nodes measured by s ( v , q ) are the smallest . E.g. , in Figure 1 ( a ) , s ( v4 , q ) = distmin ( v4 , c ) + distmin ( v4 , e ) + distmin ( v4 , f ) = 1 . Problem Definition . Given an incomplete graph G′ = ( V , E ′ , W ′ , rw , re ) , where E ′ ⊆ E , W ′ ⊆ W , and the proportions of nodes with missing keywords and of missing edges in G are denoted by rw and re , respectively . Given a query q , the incomplete graph top-k keyword search problem aims to find a set S = ( v1 , v2 , ... , vk ) of k nodes from G′ such that for any vertex v′ /∈ S , s ( v′ , q ) ≥ max ( { s ( vi , q ) |vi ∈ S } ) . 4 Proposed Methods . We propose to solve the keyword search problem with an unsupervised graph representation learning method , since the representative low-dimensional node embeddings can capture the latent information of the input incomplete graph and thus can help recovering the missing information . In addition , low dimensional node embeddings can speed up the query processing by comparing the node embedding with the generated query embedding at the cost of linear complexity . In this section , we first propose two naïve methods that are based on GNN and dimensionality reduction methods , and then we introduce our proposed KS-GNN in details . 4.1 Naïve Methods . Conv-OH . Graph convolutional layer has been widely used in GNNs , which enables GNN models to gather information from neighbor nodes . Our first naïve method Conv-OH utilizes the graph 2For ease of presentation , we focus on the undirected graphs , and it is easy to extend the proposed method in directed graphs by passing messages along the edges . convolutional layer and takes the one-hot encoding of keywords as the input feature for nodes due to the scoring function mentioned in Section 3 . Conv-OH is able to return the same answer as does BLINKS [ 2 ] , if the graph has no missing information and l is large enough . Frg . 2 shows the message passing and aggregation of Conv-OH . Specifically , with |V| = N and |W| = M , the one-hot encoding of node v is denoted by x = { 0 , 1 } M with hv , j = 1 if wj is a keyword associated with v and 0 otherwise . Therefore , the input feature matrix is denoted by X ∈ { 0 , 1 } N×M . Let Hl denote the output node embedding of the l-th layer and hlv denote the v-th row of Hl , we have : hl+1v = Aggregate ( { f ( hlu ) , ∀u ∈ N ( v ) } ∪ { hlv } ) , ( 1 ) where H0 = X , f ( · ) denotes a transform function and N ( v ) denotes the neighbors of node v. Using the combined distance as the scoring function ( described in Section 3 ) , Eq . ( 1 ) can be written as : hl+1v = Ω ( { 1 ( hlu ) ◦ ( hlu + 1 ) , ∀u ∈ N ( v ) } ∪ { hlv } ) , ( 2 ) where Ω ( · ) denotes the element-wise minimum function which ignores zeros , ◦ denotes the elementwise product , and 1 ( · ) denotes the indicator function that the element of 1 ( h ) is 1 if the input element is positive and 0 otherwise . For instance , Ω ( { [ 0 , 0 , 1 , 1 ] , [ 2 , 0 , 0 , 2 ] } ) = [ 2 , 0 , 1 , 1 ] , and 1 ( [ 0 , 2 , 0 , 3 ] ) = [ 0 , 1 , 0 , 1 ] . With Eq . ( 2 ) , the output of Conv-OH is an N ×M matrix , denoted by Z . Note that there is no dimensionality reduction in Conv-OH and thus this method consumes huge space . The node embedding hlv of v also represents the shortest path distances between the keywords and v. Specifically , if hlv , i > 0 , it means that the shortest path distance between v and wi is h l v , i − 1 , and v can not reach wi within l hops if hlv , i = 0 . For the query processing , given q , we can obtain the one-hot encoding of q , denoted by xq . Therefore , given the output node embedding Z , the sum of graph shortest-path distances between nodes and the query keywords can be computed with xqZ > , and the space complexity is O ( NM ) . It is obvious that Conv-OH can not deal with the missing information , but it provides some hints to propose more advanced methods . Conv-PCA . Principal component analysis ( PCA ) is a classic dimensionality reduction technique in multivariate statistical analysis [ 31 ] . In order to facilitate data storage and query processing , we propose another naïve PCA-based method to solve the keyword search problem . Given the one-hot encoding matrix X as the input feature matrix , PCA is able to keep d principal components of X with Xp = XU > , where the rows of U ∈ Rd×M form an orthogonal basis for the d features that are decorrelated [ 32 ] . It is worth noting that we can obtain the reconstructed feature matrix X′ with X′ = XpU . The learning object is to minimize Lpca = ||X′ −X||22 , where || · ||2 denotes the L2 norm . Inspired by the graph convolutional layers of Conv-OH , we propose Conv-PCA with the convolutional layers as below : hl+1v = max ( { αhlu , ∀u ∈ N ( v ) } ∪ { hlv } ) , ( 3 ) where α ∈ ( 0 , 1 ) is a decay parameter used to estimate the shortest path distances in Eq . ( 3 ) , since the dimension is reduced from M to d and thus it is difficult to discriminate M keywords within the d dimensions ( d M ) . Specifically , a larger cumulative decay corresponds to a larger shortest path distance . Conv-PCA takes Xp as the initial node embedding ( H0 = Xp ) . This mechanism performs better than directly using PCA in experiments as shown in Section 5.3 . For the query processing , given q , we can obtain the query embedding hq = xqU > . Therefore , given the output node embedding Z , the similarity scores between nodes and the query keywords can be computed by hqZ > with linear space complexity O ( dN ) , where the dimension d is a small constant . Compared with Conv-PCA , Conv-OH utilizes each element of hv to record the shortest path distance between keywords and v and can not reduce the dimension of node embedding . Conv-PCA can more efficiently process the keyword search query and requires less space than Conv-OH , but neither of them can well handle the missing information in incomplete graphs .

Summary: This paper introduces KS-GNN, Keywords Search GNN over Incomplete Graphs. More specifically, KS-GNN employs joint training on autoencoder (dimension deduction), keyword-based node similarity in subgraphs, and keyword frequency regularization to facilitate a complex keyword search task on incomplete graphs (missing keywords in node attributes and edges). Experimental results have proven the effectiveness of the proposed KS-GNN over multiple baseline approaches compared to various standard benchmarks.