G^3R : A Graph-Guided Generate-and-Rerank Framework for Complex and Cross-domain Text-to- SQL Generation (2023)

Abstract: We present a framework called G3R for complex and cross-domain Text-to-SQL generation. G3R aims to address two limitations of current approaches: (1) The structure of the abstract syntax tree (AST) is not fully explored during the decoding process which is crucial for complex SQL generation; (2) Domain knowledge is not incorporated to enhance their ability to generalise to unseen domains. G3R consists of a graph-guided SQL generator and a knowledge-enhanced re-ranking mechanism. Firstly, during the decoding process, An AST-Grammar bipartite graph is constructed for both the AST and corresponding grammar rules of the generated partial SQL query. The graph-guided SQL generator captures its structural information and fuses heterogeneous information to predict the action sequence which can construct the AST for the corresponding SQL query uniquely. Then, in the inference stage, a knowledge-enhanced re-ranking mechanism is proposed to introduce domain knowledge to re-rank candidate SQL queries from the beam output and choose the final answer. The SQL ranker is based on pre-trained language models (PLM) and contrastive learning with hybrid prompt tuning is incorporated to stimulate the knowledge of PLMs and make it more discriminative. The proposed approach achieves state-of-the-art results on the Spider and Spider-DK benchmarks, which are challenging complex and cross-domain benchmarks for Text-to-SQL semantic analysis.

## introduction complex and cross-domain text-to-sql generation aims to translate natural language utterances into structurally complex sql queries, where no database overlaps in training and testing. the challenges mainly lie in two aspects: (1) zero-shot cross-domain setting, databases are not overlapped recently, various works have been proposed for complex and cross-domain text-to-sql generation based on the encoder-decoder paradigm. in order to improve generalizability to unseen database schemas, many approaches focus on the encoder part. graph-based approaches @xcite have been proposed for jointly encoding the natural language utterance and relational structure of the database schema, thereby capturing the semantic relationships between them. for generating complex sql queries, most decoding approaches @xcite @xcite pre-define abstract syntax description language (asdl) rules and autoregressively generate the sql query as an ast. specifically, an ast is produced progressively in a pre-order depth-first traversal order based on a sequence of actions. in this way, the skeleton of complex sql queries can be generated flexibly. despite much progress on complex and crossdomain text-to-sql generation, there are still two limitations. firstly, the structure of ast is not fully explored during the decoding process which is crucial for generating complex sql queries correctly. they predict the next action only based on the previous action sequence, without considering the structural information of the ast and how it changes dynamically. in addition, domain knowledge required by different domains is not incorporated, which results in the model not being well generalised to new domains. as shown in to address the aforementioned limitations, this paper proposes a graph-guided generate-and-rerank framework (g 3 r) for complex and crossdomain text-to-sql generation. it consists of a graph-guided sql generator and a knowledgeenhanced re-ranking mechanism. to make better use of the structural information of ast, we construct an ast-grammar bipartite graph for both the ast and grammar rules of the generated partial sql query. a graph-guided sql generator is proposed to capture structural changes of the bipartite graph dynamically and fuse the heterogeneous information from the encoder to predict the action sequence. furthermore, we try to improve generalizability to unseen domains from a different perspective, by introducing a knowledge-enhanced re-ranking mechanism to choose the best sql query from the beam output in the inference stage. plm is adopted as an sql re-ranker and hybrid prompt tuning is adopted to make the re-ranking task similar to language modelling, stimulating the knowledge in plms to bridge the gap between different domains. in addition, sometimes the differences between candidate sql queries are subtle and contrastive learning is adopted to push away the distance of candidate queries' representations and thus make them more distinguishable. in summary, the main contributions of this paper are listed as follows: our work is related to two lines of research, complex and cross-domain text-to-sql generation and prompt tuning. ## preliminaries autoregressive top-down sql generation in this paper, we tackle the sql generation problem by generating the ast through syntax-based autoregressive top-down decoding @xcite @xcite , which guarantees to decode of syntactically valid sql queries. the generation process is considered to be the sequential application of actions. we define three actions including applyrule, selecttable and selectcolumn. the applyrule action applies a grammar rule to r1: @xmath0: @xmath1, select) sql r1 sql_unit r2 from select ast node grammar rule node figure 1: an example of the ast-grammar bipartite graph. the nodes in the dashed box represent the new nodes and edges generated after the "sql_unit" node is expanded using grammar rule 2. expand a non-terminal node in the ast, while the selecttable and selectcolumn actions populate a terminal node by appending a table or a column name from the database schema. in ast, non-terminal nodes sketch the general structure of the target sql query, while terminal nodes correspond to operations, constants and variables. the decoding process does not end until all leaf nodes are terminal nodes and the sql generation is regarded as a sequence generation problem. problem setting given a natural language question @xmath2, w 2 , ..., w n } and a corresponding database schema @xmath3, t , the goal is to generate an action sequence â, which can construct an ast and deterministically be converted to a sql query y. here the database schema s consists of tables @xmath4, t 2 , ..., t |t | }, columns @xmath5, c 2 , ..., c |c| } and a set of foreign key, primary key column pairs describing relations between columns. ast-grammar bipartite graph a bipartite graph is a simple graph where the graph vertices are decomposed into two disjoint sets such that no two graph vertices within the same set are adjacent. in order to represent the structure of the partial ast and grammar rules that have been generated, we design an ast-grammar bipartite graph input self-attention with res. & norm. encoder attention with res. & norm. predictor schema representation feed forward with res. & norm. n× positional embeddings ## conclusion in this paper, a novel graph-guided generated-and-rerank framework is proposed for complex and cross-domain text-to-sql generation. in specific, we design the ast-grammar bipartite graph and propose a graph-guided sql generator to capture the structural information of the generated complex sql query. a knowledge-enhanced re-ranking mechanism is proposed to introduce domain knowledge to bridge the gap between different domains and re-rank candidate sql queries generated from top-ranked action sequences. experimental results on the spider and spider-dk benchmark datasets show that the proposed method outperforms other competitive test-to-sql baselines.