HIE - SQL : History Information Enhanced Network for Context-Dependent Text-to- SQL Semantic Parsing (2022)

Abstract: Recently, context-dependent text-to-SQL semantic parsing which translates natural language into SQL in an interaction process has attracted a lot of attentions. Previous works leverage context dependence information either from interaction history utterances or previous predicted queries but fail in taking advantage of both of them since of the mismatch between the natural language and logic-form SQL. In this work, we propose a History Information Enhanced text-to-SQL model (HIE-SQL) to exploit context dependence information from both history utterances and the last predicted SQL query. In view of the mismatch, we treat natural language and SQL as two modalities and propose a bimodal pre-trained model to bridge the gap between them. Besides, we design a schema-linking graph to enhance connections from utterances and the SQL query to database schema. We show our history information enhanced methods improve the performance of HIE-SQL by a significant margin, which achieves new state-of-the-art results on two context-dependent text-to-SQL benchmarks, the SparC and CoSQL datasets, at the writing time.

## introduction conversation user interfaces to databases have launched a new research hotspot in text-to-sql semantic parsing @xcite @xcite @xcite @xcite and benefited us in industry @xcite . most previous works focus on the context-independent text-to-sql task and propose many competitive models. some models @xcite even surprisingly work well on the context-dependent figure 1 : an example of context-dependent text-to-sql interaction in cosql where u i is the utterance of turn i and s i is the corresponding sql query for u i . the tokens with red color are the history information that should be considered in later predictions. it is context-independent if we just consider the prediction of s 1 . compared with context-independent text-to-sql semantic parsing, context-dependent text-to-sql semantic parsing are more challenging since of the various types of dependence in utterances which make models vulnerable to parsing errors. as r 2 sql @xcite considers, different context dependencies between two adjacent utterances require the model to establish dynamic connections between utterances and database schema carefully. however, context information is not only from the last utterance. long-range dependence is also the case in cosql as the prediction of s 3 depends on "the name of the teachers and the courses" in u 1 in figure 1 . a workable proposition for longrange dependence is to inherit context information from previous predicted sql queries. but it is not a piece of cake to take advantage of previously predicted queries since of the mismatch between natural language and logic-form sql. as @xcite conclude, roughly encoding the last predicted sql query and utterances takes the wooden spoon while easily concatenation of interaction history utterances and current utterance appears to be strikingly competitive in their evaluation of 13 existing context modeling methods. in this paper, we propose a history information enhanced network to make full use of both history interactive utterances and previous predicted sql queries. we first treat the logic-form sql query as another modality with natural language. we present sqlbert, a bimodal pre-trained model for sql and natural language which is able to capture the semantic connection and bridge the gap between sql and natural language. it produces general-purpose representations and supports our context-dependent text-to-sql semantic parsing. besides, we propose a history information enhanced schema-linking graph to represent the relations among current utterance, interaction history utterances, the last predicted query, and corresponding database schema. considering it is weird to shift a topic back and forth in an interaction, we assume that the long-range dependence is successive. for example, that s 3 depends on u 1 implies that s 2 does too in figure 1 . in that case, we can leverage the long-range dependence from the last predicted query. therefore, unlike the previous schema-linking graph just with utterances and database schema @xcite , the last predicted query takes part in our graph. besides, we distinguish current utterance and interaction history utterances in the schema-linking graph. we encode the schema-linking relations with relative self-attention mechanism @xcite . in our experiments, the proposed methods of sqlbert and the history information enhanced schema-linking substantially improve the performance of our model. at the time of writing, our model ranks first on both two large-scale crossdomain context-dependent text-to-sql leaderboards, sparc @xcite and cosql @xcite . specifically, our model achieves a 64.6% question match and 42.9% interaction match accuracy on sparc, and a 53.9% question match and 24.6% interaction match accuracy on cosql. ## related work text-to-sql semantic parsing follows a long line of research on semantic parsing from natural language to logical language @xcite @xcite . recently, context-independent text-to-sql semantic parsing has been well studied. spider @xcite ) is a famous dataset for the complex and cross-domain context-independent text-to-sql task. some works @xcite apply graph neural networks to encode database schema. @xcite succeed in appling deep transformers to the context-independent text-to-sql task. @xcite employ a treebased decoder to match sql grammar. @xcite improve the tree-based decoder by a bottom-up method. @xcite refine the sequence-based decoder via carefully designed restriction rules. @xcite and @xcite propose sql intermediate representations to bridge the gap between natural language and sql. @xcite study the role of schemalinking in text-to-sql semantic parsing. @xcite propose a unified framework to capture the schema-linking. lin et al. (2020) represent the schema-linking as a tagged sequence. @xcite further integrate non-local and local features via taking advantage of both schema-linking graph and its corresponding line graph. besides, many previous works @xcite @xcite focus on pre-train models for context-independent text-to-sql semantic parsing. with more attentions on context-dependent textto-sql semantic parsing, existing works have been devoted to the context-dependent text-to-sql task. sparc @xcite and cosql @xcite datasets are specially proposed for the task. editsql @xcite and ist-sql @xcite focus on taking advantages of the last predicted query for the prediction of current query. editsql tries to copy the overlap tokens from the last predicted query, while ist-sql proposes an interaction state tracking method to encode the information from the last predicted query. igsql @xcite ) and r 2 sql @xcite leverages the contextual information among the current utterance, interaction history utterances and database schema via context-aware dynamic graphs. notably, r 2 sql simulates the information by connecting the schema graphs with the tokens in interactive utterances. @xcite creatively propose a context-aware pre-trained language model. however, the problem of making full use of both interaction history utterances and predicted queries for the context-dependent text-to-sql task remains open. ## hie-sql first, we formally define the conversational textto-sql semantic parsing problem. in the rest of the section, we detail the architecture of history information enhanced text-to-sql model (hie-sql). ## conclusion we present hie-sql, a history information enhanced context-dependent text-to-sql model, which targets at explicitly capturing the contextdependence from both interaction history utterances and the last predicted sql query. with the help of the proposed bimodal pre-trained model, sqlbert, hie-sql bridge the gap between the utterances and predicted sql despite the mismatch of natural language and logic-form sql. moreover, we also introduce a method of schema-linking to enhance the connections among utterances, sql query, and database schema. taken together, hie-sql achieves consistent improvements on the context-dependent text-to-sql task, especially in the interaction match metric. hie-sql achieves new state-of-the-art results on two famous context-dependent text-to-sql datasets, sparc and cosql.