An Exploratory Study on Model Compression for Text-to- SQL (2023)

Abstract: Text-to-SQL translates user queries into SQL statements that can retrieve relevant answers from relational databases. Recent approaches to Text-to-SQL rely on pre-trained language models that are computationally expensive and technically challenging to deploy in real-world applications that require real-time or on-device processing capabilities. In this paper, we perform a focused study on the feasibility of applying recent model compression techniques to sketch-based and sequence-to-sequence Text-to-SQL models. Our results reveal that sketch-based Text-to-SQL models generally have higher inference efficiency and respond better to model compression than sequence-to-sequence models, making them ideal for real-world deployments, especially in use cases with simple SQL statements.

## introduction text-to-sql is an important task that has been gaining the attention of researchers over the years. formally, given a query q and a relational database d, the goal of text-to-sql is to build a model f such that @xmath0, d | θ) where θ is a vector of model parameters and s is a predicted sql statement which we can use to retrieve the answer to q from d. text-to-sql has many potential applications that can improve our standard of living. for example, medical chatbots can convert user queries into sql statements and then use them to retrieve relevant information from medical knowledge bases. industry can leverage text-to-sql tools to help employees shorten the time needed to write complex sql queries, thereby improving overall work productivity. the recent emergence of complex text-to-sql datasets containing complicated sql and crosstable setup has driven researchers to develop huge models that encode various complex relationships between table schema and query with large pretrained language models such as bert @xcite and t5 @xcite . these models are usually sequence-to-sequence models that generate sql statements sequentially or sketch-based models that use classifiers to fill in the slots of sql templates. however, despite achieving state-of-the-art performances on benchmark datasets, such models are usually both memory and computationally expensive, making it technically challenging to deploy them in memory-constrained real-world applications that require low inference latency. therefore, to deploy state-of-the-art text-to-sql models in real-world production environments, we must drastically improve the inference time and reduce the number of parameters in these models. we turn to the field of model compression @xcite for solutions that can speed up inference without significantly hurting model performance. formally, the goal of model compression is to reduce f to a smaller model f ′ such that s ′ = f ′ (q, d | θ ′ ). ideally, we want s ′ to be the same as s and dim(θ ′ ) to be much smaller than dim(θ). in this paper, we thoroughly examine the feasibility of using model compression techniques to build faster and more accurate text-to-sql models that we can successfully deploy in the real world. for this, we carefully apply a few model compression methods to representative sequence-to-sequence or sketch-based text-to-sql models on three datasets: wikisql, spider, and tableqa. the main findings of this paper are: (i) sketch-based models generally respond well to model compression techniques, while sequence-to-sequence models show mixed results, (ii) we observe better speed improvements in sketch-based models as their slot-filling components are much faster than the decoding components of sequence-to-sequence models. (iii) model compression techniques work poorly on state-of-the-art text-to-sql models built on pre-trained encoder-decoder language models such as t5. we hope our findings can empower practitioners to make more informed decisions when selecting text-to-sql models and compressing them appropriately for real-world deployments. we conduct model compression experiments on several datasets as shown in tableqa @xcite is a chinese text-to-sql dataset containing 64,891 question-sql pairs over 6000 tables extracted from online documents such as financial reports or spreadsheets. difficulty of datasets wikisql and tableqa are considered simple datasets because they only contain sql queries covering the select and where clauses, and each database has only one single table. contrarily, spider contains large samples of complex sql instances that connect multiple tables with primary and foreign keys with more advanced clauses such as nested queries, join on, and order/group by. ## experiment setup in most cases, we follow the recommended configurations in corresponding papers. we may adjust the batch sizes and learning rates slightly to fit the experiments on our hardware. we train our models on servers with either nvidia gv100 gpu (32gb) or rtx a6000 (45gb) but calculate inference speeds by running models on only cpus with batch size set to one, which better mimics the situations in the real world. for all datasets, we use their dev sets as the test sets and create new train-dev sets in the ratio of 4 to 1 from the original train set. we early stop our models based on the esm scores on dev sets and report average test set esm scores over 5 different runs. other than pi-card, we use bert-large for all english datasets and roberta-zh @xcite for tableqa. ## conclusion this paper investigates whether we can use model compression to improve the inference efficiency of recent text-to-sql models that rely heavily on large pre-trained language models. our results show that on simple text-to-sql datasets, we can deploy simple strategies such as layer pruning to obtain a 5-6x speedup without significantly hurting model performances. we also observe that sketchbased models generally respond better to model compression than sequence-to-sequence models. however, we are not able to effectively compress picard on the spider dataset and we would tackle this problem as a future work.