Learning SQL Like a Human: Structure-Aware Curriculum Learning for Text-to- SQL Generation (2025)

Abstract: The Text-to-SQL capabilities of large language allow users to interact with databases using natural language. While current models struggle with handling complex queries, especially involving multi-table joins and reasoning. To address this gap, we propose to construct a model, namely SAC-SQL, with synthetic training samples followed by a structure-aware curriculum learning framework for enhancing SQL generation. Our approach begins with a supervised fine-tuning (SFT) stage, where we train open-source models on a synthetically constructed, cross-domain SQL dataset with diverse structural patterns. Moreover, we introduce a unified structure difficulty scoring function to partition the training samples into non-overlapping curriculum phases, guiding the model progressively learning from simpler to more complex SQL structures. Extensive experiments are conducted and the results show that SAC-SQL achieves better results than the baselines, and significantly narrows the performance gap between open-source and close-source models on Spider and Bird benchmarks.

## introduction text-to-sql generation task aims to automatically translate natural language questions into executable sql queries, allowing users to interact with databases using plain language @xcite . this task reduces the technical barrier for data access and plays a key role in natural language interfaces to structured data @xcite . several methods have been proposed to achieve the goal. some methods adopt prompt engineering to solve this task. furthermore, recent methods have focused on collecting related data at scale and applying instruction tuning to enhance model performance. representative models such as gpt-* corresponding author 4 @xcite and codellama @xcite have demonstrated remarkable progress on the text-to-sql task. however, existing methods still struggle with handling complex queries that involve multi-table joins and reasoning. according to our statistics, codellama achieves an execution accuracy of 86% for easy queries compared to only 40% for complex queries foot_0 . furthermore, we performed an error analysis on the 30 incorrectly predicted complex samples (totaling 60 individual errors, with some examples containing multiple issues). as shown in figure 1 , the distribution of error types revealed that incorrect joins accounted for the largest share (30%), followed by nested query errors (23.3%), aggregation misuse (16.7%), incorrect where clause (13.3%), syntax errors (10%), and ambiguous column references (6.7%). these findings clearly indicate that these llms struggle particularly with structurally complex queries such as join operations and nested queries. to overcome this challenge, in this paper, we proposed a novel model, sac-sql, which adopts synthetic training samples followed by a structureaware curriculum learning framework for enhancing sql generation. our model builds upon two key insights: first, text-to-sql difficulty is governed by query structure, and current models fail to generalize well to complex structures when trained in a uniform manner @xcite . second, high-quality real-world text-to-sql datasets are scarce and difficult to obtain, limiting the diversity and complexity of training examples. thus, we propose to construct a high-quality, synthetically generated sql dataset that covers a wide variety of query structures and schema domains. besides, we propose a two-stage training strategy to train the models. at the first stage, the synthesized dataset is used to supervise fine-tune open-source language models to provide them with essential sql syntax and schema grounding. by explicitly controlling the composition of sql structures during generation, we ensure that more examples contain low-frequency but high-difficulty patterns. this helps alleviate the long-tail distribution problem and enhances the model's ability to generalize to rare but semantically important query types @xcite . at the second stage, we introduce a structureaware curriculum learning framework to further enhance the model's ability to handle structurally complex queries. we design a unified function based on structural components, interaction patterns, syntactic tree depth, and generation uncertainty (via normalized nll). training samples are ranked and divided into non-overlapping phases of increasing difficulty, allowing the model to learn sql structure in a staged, cognitively aligned way. combining these two-stage processes, we develop sac-sql, a curriculum-enhanced model trained atop an sft-initialized open-source llm. experiments on spider and bird benchmarks show that sac-sql achieves better results than existing methods, and surpasses existing open-source baselines. besides, it outperforms gpt-4 and its derivatives in executing complex sql queries, highlighting the effectiveness of combining data design and curriculum scheduling in improving sql compositionality and reasoning. considering the characteristics of the synthetic sample, we design a structure-aware curriculum learning with difficulty measure at the second stage. ## methodology our training process consists of two major stages: a supervised fine-tuning (sft) phase and a subsequent structure-aware curriculum learning phase. the overall goal is to improve the ability of large language models to handle structurally complex sql queries by gradually exposing them to increasingly difficult training examples. figure 2 illustrates the full pipeline. ## conclusion in this paper, we propose sac-sql, a model trained synthetic training samples followed by a structure-aware curriculum learning framework for enhancing sql generation. it has integrated highquality synthetic data, a unified difficulty scoring function, and phase-wise curriculum scheduling to guide the model through progressively more complex sql structures. we have introduced sac-sql that achieves state-of-the-art execution accuracy on the spider and bird benchmarks, even surpassing several closed-source models such as gpt-4 in structurally demanding scenarios.