LGESQL : Line Graph Enhanced Text-to- SQL Model with Mixed Local and Non-Local Relations (2021)

Abstract: This work aims to tackle the challenging heterogeneous graph encoding problem in the text-to-SQL task. Previous methods are typically node-centric and merely utilize different weight matrices to parameterize edge types, which 1) ignore the rich semantics embedded in the topological structure of edges, and 2) fail to distinguish local and non-local relations for each node. To this end, we propose a Line Graph Enhanced Text-to-SQL (LGESQL) model to mine the underlying relational features without constructing meta-paths. By virtue of the line graph, messages propagate more efficiently through not only connections between nodes, but also the topology of directed edges. Furthermore, both local and non-local relations are integrated distinctively during the graph iteration. We also design an auxiliary task called graph pruning to improve the discriminative capability of the encoder. Our framework achieves state-of-the-art results (62.8% with Glove, 72.0% with Electra) on the cross-domain text-to-SQL benchmark Spider at the time of writing.

## introduction the text-to-sql task @xcite aims to convert a natural language question into a sql query, given the corresponding database schema. it has been widely studied in both academic and industrial communities to build natural language interfaces to databases @xcite . one daunting problem is how to jointly encode the question words and database schema items (including tables and columns), as well as various relations among these heterogeneous inputs. typically, previous literature utilizes a node-centric graph neural network (gnn, scarselli et al., 2008) * the corresponding authors are lu chen and kai yu. to aggregate information from neighboring nodes. gnnsql @xcite adopts a relational graph convolution network @xcite to take into account different edge types between schema items, such as t-has-c relationship foot_0 , primary key and foreign key constraints. however, these edge features are directly retrieved from a fixed-size parameter matrix and may suffer from the drawback: unaware of contextualized information, especially the structural topology of edges. meta-path is defined as a composite relation linking two objects, which can be used to capture multi-hop semantics. for example, in figure 1 (a), relation q-exactmatch-c and c-belongsto-t can form a 2-hop meta-path indicating that some table t has one column exactly mentioned in the question. although ratsql @xcite introduces some useful meta-paths such as c-sametable-c, it treats all relations, either 1-hop or multi-hop, in the same manner (relative position embedding, @xcite in a complete graph. without distinguishing local and non-local neighbors, see figure 1(b) , each node will attend to all the other nodes equally, which may lead to the notorious over-smoothing problem @xcite . besides, meta-paths are currently constructed by domain experts or explored by breadthfirst search @xcite . unfortunately, the number of possible meta-paths increases exponentially with the path length, and selecting the most important subset among them is an np-complete problem @xcite . to address the above limitations, we propose a line graph enhanced text-to-sql model (lgesql), which explicitly considers the topological structure of edges. according to the definition of a line graph @xcite , we firstly construct an edge-centric graph from the original node-centric graph. these two graphs capture the structural topology of nodes and edges, respectively. iteratively, each node in either graph gathers information from its neighborhood and incorporates edge features from the dual graph to update its representation. as for the node-centric graph, we combine both local and non-local edge features into the computation. local edge features denote 1-hop relations and are dynamically provided by node embeddings in the line graph, while non-local edge features are directly extracted from a parameter matrix. this distinction encourages the model to pay more attention to local edge features while maintaining information from multihop neighbors. additionally, we propose an auxiliary task called graph pruning. it introduces an inductive bias that the heterogeneous graph encoder of text-to-sql should be intelligent to extract the golden schema items related to the question from the entire database schema graph. experimental results on benchmark spider @xcite demonstrate that our lgesql model promotes the exact set match accuracy to 62.8% (with glove, @xcite ) and 72.0% (with pretrained language model elec-tra, @xcite . our main contributions are summarized as follows: ## preliminaries problem definition given a natural language question @xmath0, q 2 , • • • , q |q| ) with length |q| and the corresponding database schema @xmath1, the target is to generate a sql query y. the database schema s contains multiple tables @xmath2, t 2 , • • • } and columns throughout our discussion, we use the term local to denote relations with path length 1, while nonlocal relations refer to meta-paths longer than 1. the relational adjacency matrix r n contains both local and non-local relations, see appendix a for enumeration. function f maps the source and target node index tuple (s, t) into the "edge" index @xmath3, t) in g e . the reverse mapping is f -1 . in the line graph g e , a directed edge r e @xmath4, iff the target node of edge r n f -1 (i) and the source node of edge r n f -1 (j) in g n are exactly the same node. actually, r e ij captures the information flow in meta-path r n f -1 (i) • r n f -1 (j) . we prevent backtracking cases where two reverse edges will not be connected in g e , illustrated in figure 2 . we only utilize local relations in r n as the node set v e to avoid creating too many nodes in the line graph g e . symmetrically, each edge in r e can be uniquely identified by the node in v n . for example, in the upper right part of figure 2 , the edge between nodes "e1" and "e2" in the line graph can be represented by the middle node with double solid borderlines in the original graph. 3 method after constructing the line graph, we utilize the classic encoder-decoder architecture @xcite as the backbone of our model. lgesql consists of three parts: a graph input module, a line graph enhanced hidden module, and a graph output module (see figure 3 for an overview). the first two modules aim to map the input heterogeneous graph g n into node embeddings @xmath5, where d is the graph hidden size. the graph output module retrieves and transforms x into the target sql query y. ## experiments in this section, we evaluate our lgesql model in different settings. codes are public available foot_4 . ## related work encoding problem for text-to-sql to tackle the joint encoding problem of the question and database schema, @xcite proposes "column attention" strategy to gather information from columns for each question word. typesql @xcite incorporates prior knowledge of column types and schema linking as additional input features. @xcite and @xcite deal with the graph structure of database schema via gnn. editsql @xcite considers "co-attention" between question words and database schema nodes similar to the common practice in text matching @xcite . @xcite further leverages the database content to augment the column representation. the most advanced method ratsql @xcite , utilizes a complete relational graph attention neural network to handle various pre-defined relations. in this work, we further consider both local and non-local, dynamic and static edge features among different types of nodes with a line graph. heterogeneous graph neural network apart from the structural topology, a heterogeneous graph @xcite also contains multiple types of nodes and edges. to address the heterogeneity of node attributes, @xcite designs a type-based content encoder and @xcite utilizes a type-specific linear transformation. for edges, relational graph convolution network @xcite and relational graph attention network (rgat, @xcite have been proposed to parameterize different relations. han @xcite converts the original heterogeneous graph into multiple homogeneous graphs and applies a hierarchical attention mechanism to the meta-path-based sub-graphs. similar ideas have been adopted in dialogue state tracking @xcite , dialogue policy learning @xcite ) and text matching @xcite to handle heterogeneous inputs. in another branch, @xcite , @xcite and @xcite construct the line graph of the original graph and explicitly model the computation over edge features. in this work, we borrow the idea of a line graph and update both node and edge features via iteration over dual graphs. ## conclusion in this work, we utilize the line graph to update the edge features in the heterogeneous graph for the text-to-sql task. through the iteration over the structural connections in the line graph, local edges can incorporate multi-hop relational features and capture significant meta-paths. by further integrating non-local relations, the encoder can learn from multiple views and attend to remote nodes with shortcuts. in the future, we will investigate more useful meta-paths and explore more effective methods to deal with different meta-path-based neighbors.