Semantic Decomposition of Question and SQL for Text-to- SQL Parsing (2023)

Abstract: Text-to-SQL semantic parsing faces challenges in generalizing to cross-domain and complex queries. Recent research has employed a question decomposition strategy to enhance the parsing of complex SQL queries.However, this strategy encounters two major obstacles: (1) existing datasets lack question decomposition; (2) due to the syntactic complexity of SQL, most complex queries cannot be disentangled into sub-queries that can be readily recomposed. To address these challenges, we propose a new modular Query Plan Language (QPL) that systematically decomposes SQL queries into simple and regular sub-queries. We develop a translator from SQL to QPL by leveraging analysis of SQL server query optimization plans, and we augment the Spider dataset with QPL programs. Experimental results demonstrate that the modular nature of QPL benefits existing semantic-parsing architectures, and training text-to-QPL parsers is more effective than text-to-SQL parsing for semantically equivalent queries. The QPL approach offers two additional advantages: (1) QPL programs can be paraphrased as simple questions, which allows us to create a dataset of (complex question, decomposed questions). Training on this dataset, we obtain a Question Decomposer for data retrieval that is sensitive to database schemas. (2) QPL is more accessible to non-experts for complex queries, leading to more interpretable output from the semantic parser.

## introduction querying and exploring complex relational data stores necessitate programming skills and domainspecific knowledge of the data. text-to-sql semantic parsing allows non-expert programmers to formulate questions in natural language, convert the questions into sql, and inspect the execution results. while recent progress has been remarkable on this task, general cross-domain text-to-sql models still face challenges on complex schemas and queries. state of the art text-to-sql models show performance above 90% for easy queries, but fall to about 50% on complex ones (see in this paper, we address the challenge of dealing with complex data retrieval questions through a compositional approach. based on the success of the question decomposition approach for multi-hop question answering, recent work in semantic parsing has also investigated ways to deal with complex sql queries with a question decomposition (qd) strategy. in another direction, previous attempts have focused on splitting complex sql queries into spans (e.g., aggregation operators, join criteria, column selection) and generating each span separately. in our approach, we start from a semantic analysis of the sql query. we introduce a new intermediary language, which we call query plan language (qpl) that is modular and decomposable. qpl can be directly executed on sql databases through direct translation to modular sql @xcite . we design qpl to be both easier to learn with modern neural architectures than sql and easier to interpret by nonexperts. the overall approach is illustrated in fig. 1 . we develop an automatic translation method from sql to qpl. on the basis of the modular qpl program, we also learn how to generate a natural language decomposition of the original question. ## question decomposition we use the qpl plans automatically computed from sql queries in the dataset to derive a set of question decompositions (qd) that are grounded in the qpl steps, as shown in fig. 1 . we investigate three usages of this qd method: (1) qp @xmath0: we learn how to generate a qd given a qpl plan; this is useful at inference time, to present the predicted qpl to non-expert users, as a more readable form of the plan; (2) @xmath1; (3) q + @xmath2, firstly generates a corresponding qd, and then predicts a qpl plan based on (q+qd). ## error analysis the most challenging error type is related to queries that involve aggregate operations (group by and aggregated values such as count, sum or max). the spider development set includes 20 different schemas. ## conclusion we presented a method to improve compositional learning of complex text-to-sql models based on qpl, a new executable and modular intermediary language that is derived from sql through semantic transformation. we provide software tools to automatically translate sql queries into qpl and to execute qpl plans by translating them to cte sql statements. we also compiled spider-qpl, a version of spider which includes qpl and question decomposition for all examples. our experiments indicate that qpl is easier to learn using fine-tuned llms (our text-to-qpl model achieves sota results on fine-tuned models on spider dev set without db values, especially on hard and extra-hard queries); and easier to interpret by users than sql for complex queries. on the basis of the computed qpl plans, we derived a new form of question decomposition and trained a question decomposer that is sensitive to the target database schema, in contrast to existing generic question decomposers. given a predicted qpl plan, we can derive a readable qd that increases the interpretability of the predicted plan. in future work, we plan to further exploit the modularity of qpl plans for data augmentation and to explore multi-step inference techniques. we have started experimenting with an auto-regressive model which predicts qpl plans line by line. our error analysis indicates that enhancing the model to further take advantage of the database values and foreign-keys has the potential to increase the robustness of this multi-step approach. we are also exploring whether users can provide interactive feedback on predicted qd as a way to guide qpl prediction. for example: simple schema encoding: pets_1 table student (stuid, lname, fname, age, sex, major, advisor, city_code) table pets (petid, pettype, pet_age, weight) table has_pet (stuid, petid) rich schema encoding: pets_1 create table student ( stuid number, lname text, fname text, age number, sex text, major number, advisor number, city_code text, primary key ( stuid )) create table pets ( petid number, pettype text ( dog ), pet_age number, weight number, primary key ( petid )) create table has_pet ( stuid number, petid number, foreign key ( stuid ) references student ( stuid ), foreign key ( petid ) references pets ( petid )) values are added after each column when an n-gram from the question is found as one of the values in one of the rows of the table. for example, for the question "how much does the youngest dog weigh?", the n-gram "dog" is found in the values of the column pettype. in this case, the value annotation pettype text ( dog ) is encoded.