Natural SQL : Making SQL Easier to Infer from Natural Language Specifications (2021)

Abstract: Addressing the mismatch between natural language descriptions and the corresponding SQL queries is a key challenge for text-to-SQL translation. To bridge this gap, we propose an SQL intermediate representation (IR) called Natural SQL (NatSQL). Specifically, NatSQL preserves the core functionalities of SQL, while it simplifies the queries as follows: (1) dispensing with operators and keywords such as GROUP BY, HAVING, FROM, JOIN ON, which are usually hard to find counterparts in the text descriptions; (2) removing the need of nested subqueries and set operators; and (3) making the schema linking easier by reducing the required number of schema items. On Spider, a challenging text-to-SQL benchmark that contains complex and nested SQL queries, we demonstrate that NatSQL outperforms other IRs, and significantly improves the performance of several previous SOTA models. Furthermore, for existing models that do not support executable SQL generation, NatSQL easily enables them to generate executable SQL queries, and achieves the new state-of-the-art execution accuracy.

## introduction automatic generation of sql queries from natural language (nl) has been studied in the literature for a number of years @xcite @xcite @xcite @xcite . more recently, wikisql @xcite , the first large-scale cross-domain text-to-sql dataset, has attracted much attention from the research community @xcite @xcite . although the current state-of-the-art approach has achieved over 90% execution accuracy on wikisql @xcite , since the sql 1 our code and dataset are available at https://github.com/ygan/natsql . to synthesize sql queries with more complex structures, intermediate representation (ir) is widely employed by the previous sota models on the spider dataset @xcite @xcite . however, previous irs are either too complicated or have limited coverage of sql structures. besides, although the existing irs eliminate part of the mismatch between intent expressed in nl and the implementation details in sql, there is still some mismatch that can be further eliminated by improving the ir. in this work, we present natural sql (natsql), a new intermediate representation that offers simplified queries over other irs, while preserving a high coverage of sql structures. more importantly, natsql further eliminates the mismatch between nl and sql, and can easily support executable sql generation. figure 1 presents a sample comparison between natsql and other irs. we observe that there is a mismatch between the nl word 'and' and the intersect sql keyword, since in another similar question shown in figure 5 , the 'and' no longer corresponds to the intersect keyword. to translate the nl question into a corresponding query, previous irs need the models to distinguish whether the word 'and' corresponds to intersect, this is not required for natsql. among all irs, natsql provides the simplest and shortest translation, while the natsql structure also aligns best with the nl question. natsql preserves the core functionalities of sql, while simplifying the queries as follows: question : : m : which film has more than 5 actors and less than 3 in the inventory? sql : : select t1.title from film as t1 join film_actor as t2 on t1.@xmath0.film_id group by t1.film_id having count(*) > 5 intersect select t1.title from film as t1 join inventory as t2 on t1.@xmath1.film_id group by t1.film_id having count(*) < 3 the ir of rat-sql : (remove the join on clause) : select title from film, film_actor group by film_id having count(*) > 5 intersect select title from film, inventory group by film_id having count(*) < 3 the ir of syntaxsql : (remove the join on and from clause) : select film.title group by film.film_id having count(*) > 5 intersect select film.title group by film.film_id having count(*) < 3 semql : (remove the join on, from and group by clause. merge the having and where clause) select film.title where count(film_actor.*) > 5 intersect select film.title where count(inventory.*) < 3 natsql : (further remove the set operators based on semql) : select film.title where count(film_actor.*) > 5 and count(inventory.*) < 3 figure 1: a sample question in spider dataset with corresponding sql and irs. we compare natsql with sql and other irs by incorporating them into existing open-source neural network models that achieve competitive performance on spider. our experiments show that natsql boosts the performance of these existing models, and outperforms both sql and other irs. in particular, equipping rat-sql+gap with natsql achieves a new state-of-the-art execution accuracy on the spider benchmark. these results suggest that to improve the ability of text-to-sql models to understand and reason about the nl descriptions, designing irs to better reveal the correspondence between natural language and query languages is a promising direction. ## review: text-to-sql paradigm most existing text-to-sql models generate the sql keywords (blue character in figure 1 ) and sql schema items (black character in figure 1 ) separately. based on this paradigm, we investigate how we can design an ir to improve both sql keyword generation and schema item generation. ## conclusion in this paper, we propose natsql, a new sql intermediate representation that reduces the difficulty of schema linking and simplifies the sql structure. by incorporating natsql into existing neural models for text-to-sql generation, we show that natsql is easier to infer from natural language specification than the full-fledged sql and other intermediate representation languages. furthermore, natsql enables existing models to easily generate executable sql queries without modifying their architecture. experimental results on the challenging spider benchmark demonstrate that natsql consistently improves the prediction performance of several neural network architectures and achieves the state-of-the-art, showing the effectiveness of our approach.