SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL
Authors: name: Ruoxi Sun; name: Sercan O Arik; name: Hootan Nakhost; name: Hanjun Dai; name: Rajarishi Sinha; name: Pengcheng Yin; name: Tomas Pfister
Published: 2023-05-26 21:39:05

Abstract: One impressive emergent capability of large language models (LLMs) is
generation of code, including Structured Query Language (SQL) for databases.
For the task of converting natural language text to SQL queries, Text-to-SQL,
adaptation of LLMs is of paramount importance, both in in-context learning and
fine-tuning settings, depending on the amount of adaptation data used. In this
paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on
PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is
based on an execution-based self-consistency prompting approach designed for
Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our
best knowledge is the first to outperform previous state-of-the-art with
fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the
fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying
SQL-PaLM to real-world scenarios we further evaluate its robustness on other
challenging variants of Spider and demonstrate the superior generalization
capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate
the impressive intelligent capabilities and various success enablers of
LLM-based Text-to-SQL.