A Review of Cross-Domain Text-to- SQL Models (2020)

Abstract: WikiSQL and Spider, the large-scale cross-domain text-to-SQL datasets, have attracted much attention from the research community. The leaderboards of WikiSQL and Spider show that many researchers propose their models trying to solve the text-to-SQL problem. This paper first divides the top models in these two leaderboards into two paradigms. We then present details not mentioned in their original paper by evaluating the key components, including schema linking, pretrained word embeddings, and reasoning assistance modules. Based on the analysis of these models, we want to promote understanding of the text-to-SQL field and find out some interesting future works, for example, it is worth studying the text-to-SQL problem in an environment where it is more challenging to build schema linking and also worth studying combing the advantage of each model toward text-to-SQL.

## introduction text-to-sql is a task to translate the natural language query (input) written by users into the sql query (output) automatically. for example, in in this paper, we discuss the top models for the wikisql and spider benchmarks. since relatively high generation accuracy has already been achieved for the wikisql benchmark, and the sql structures in spider cover all sql structures in wik-isql, we focus more on models designed for spider. this paper starts from the comparison of the overall paradigms of the models and then discusses the key modules used by most models. overall, our contributions are as follows: question: what airports don't have departing or arriving flights? sql: select airportname from airports where airportcode not in ( select sourceairport from flights union select destairport from flights ) similarities: although wikisql and spider are cross-domain settings, most sql queries do not need domain knowledge during generation. the domain knowledge is a consensus that only exists in a specific field and will not be clearly stated in the question. for example, in a scenario where domain knowledge is needed, ask for 'good restaurant' can correspond to a where condition '@xmath0.5' since this domain rates the restaurants ranging from 0 to 5 stars. some different domain examples only replace the words related to schema item names, keeping the same sentence structure. besides, most sentences directly use words related to schema item names instead of synonyms, which allows the model to locate the schema items by word matching. for example, in ## the paradigms of text-to-sql we only discuss two paradigms achieving relatively high performance in the text-to-sql task, shown in figure 1 , ignoring the paradigms such as directly using a seq2seq to generate sql. most models follow the paradigm one but are based on different neural network architecture, while ie-sql (anonymous, 2020) brings a new paradigm achieving the sota result in wikisql benchmark. ## reasoning assistance module some sql clauses in spider need reasoning to generate, but wikisql has almost no such clauses. for example, we cannot make out the join on clause directly from the question in figure 2 . existing models mainly implement reasoning through graph neural networks and intermediate representation. ## conclusion we discuss the existing cross-domain sota textto-sql models from the whole to the detailed modules to give a clear picture of the current textto-sql research progress. we illustrate that pretrained embeddings improve the models by constructing a better schema linking and a more accurate sql structure through experiments. this paper also provide many details that are not mentioned in the original papers, such as . however, due to space limitations, this paper cannot cover all the details of these sota models. we hope this paper will help you understand the key connections and differences between the previous models and have a comprehensive understanding of the text-to-sql field.