Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL
Authors: Mohammadreza Pourreza
Shayan Talaei
Ruoxi Sun
Xingchen Wan
Hailong Li
Azalia Mirhoseini
Amin Saberi
Sercan O Arik
Venue: COLM 2025 (2025)
Keywords: Text-to-SQL
Reinforcement Learning
Database
TL;DR: Enhancing the reasoning ability of LLMs in text-to-SQL domain by employing GRPO and partial rewards.

Abstract: Text-to-SQL is a challenging task involving multiple reasoning-intensive subtasks, including natural language understanding, database schema comprehension, and precise SQL query formulation. Existing approaches often rely on handcrafted reasoning paths with inductive biases that can limit their overall effectiveness. Motivated by the recent success of reasoning-enhanced models such as DeepSeek R1 and OpenAI O1, which effectively leverage reward-driven self-exploration to enhance reasoning capabilities and generalization, we propose a novel set of partial rewards tailored specifically for the Text-to-SQL task. Our reward set includes schema-linking, partial reward from AI feedback, n-gram similarity, and syntax check rewards, explicitly designed to address the reward sparsity issue prevalent in reinforcement learning (RL). Leveraging group relative policy optimization (GRPO), our approach explicitly encourages large language models (LLMs) to develop intrinsic reasoning skills necessary for accurate SQL query generation. With models of different sizes, we demonstrate that RL-only training with our proposed rewards consistently achieves higher accuracy and superior generalization compared to supervised fine-tuning (SFT). Remarkably, our RL-trained14B-parameter model significantly outperforms larger proprietary models, e.g. O3-Mini by 4% and Gemini-1.5-Pro-002 by 3% on the BIRD benchmark. These highlight the efficacy of our proposed RL-training framework with partial rewards for enhancing both accuracy and reasoning capabilities in Text-to-SQL tasks.

Published as a conference paper at COLM 2025
Reasoning-SQL:
Reinforcement Learning with Partial Re- wards for Reasoning-Enhancee Text-to-SQL
Mohammadreza Pourreza'
Shayan Talaei*
Ruoxi Sun Google DeepMind ruoxis@google com
Google Cloud pourrezaggoogle.co com
Stanford University
stalaei@stanf corttaannorddede.
Xingchen Wan
Hailong Li
Azalia Mirhoseini
Google Cloud xingchenw@google.c com
Google Cloud mailongii@googee.oom com
Stanford University azal Stalia@stanfi fonford.cee ord.edu
Amin Saberi
Sercan Ö. Ank
Stanford University daberi@stanfor edu
Google Cloud
soari coarikegoogle.com com
Abstract
Text-to-SQL a challenging task involving multiple reasoning-intensiee subtasks, including natural language understanding, database schema comprehension, and precise SQL query formulation. Existing approaches often rely on handcrafted reasoning paths with inductive biases that can limit their overall effectiveness. Motivated by the recent success of reasoning-enhanced models such as DeepSeek Ri (Guo et al., 2025) and OpenAl ol (Jaech et al., 2024), which effectively leverage reward-driven self-exploration to enhance reasoning capabilities and generalization, propose a novel set of partial rewards tailored specifically for the Text-to- SQL. task. Our reward set includes schema-linking, AT feedback, N-gram similarity, and syntax check, explicitly designed to address the reward sparsity issue prevalent in reinforcement learning (RL). Leveraging group relative policy optimization (GRPO), our approach explicitly encourages large language models (LLMs) to develop intrinsic reasoning skills neces- sary for accurate SQL query generation. With models of different sizes, we demonstrate that RL-only training with our proposed rewards consistently achieves higher accuracy and superior generalization compared to super- vised fine-tuning (SFT). Remarkably, our RL-trained 14B-parameter model significantly outperforms larger proprietary models, e.g. 03-mini by 4% by 3% on the BIRD benchmark. These highlight the efficacy of our proposed RL-training framework with partial rewards for enhancing both Text-to-SQL accuracy and reasoning capabilities.
and Geminí-1.5-Pro-
1 Introduction
Bridging the gap between human language and database queries, Text-to-SQL focuses on translating natural language questions into executable SQL. The advent of large language models (LLMs) has yielded significant progress, with most state-of-the-art Text-to-SQL. systems now relying on LLMs for synthesizing SQL queries. However, the performance of these methods is inherently limited by the reasoning capabilities of the underlying models, making it essential to explicitly enhance reasoning in the Text-to-SQL domain. Accurate SQL generation generraton requires both deep understanding of natural language and sophisticated reasoning over complex database schemas (Cao et al., 2021; Talaei et al., 2024; Pourreza & Rafiei, 2024a). Conventional approaches for LLM adaptation, predominantly based
'Fqual contribution.

---

Published as a conference paper at COLM 2025
supervised fine-tuning (SFT) (Pourreza & Rafiei, 2024b; Li et al., 2024b) or few-shot prompting (Gao et al., 2023; Pourreza & Rafiei, 2024a), often fall short when the task demands extended reasoning, especially in cases of ambiguous multi-step queries.
Recent advances in large reasoning models, such as OpenAl's ol (Jaech et al., 2024) and DeepSeek-RR R1 (Guo et al., 2025), have shown that additional compute during inference to produce judicious reasoning trajectories can significantly improve response quality. Motivated by these, we propose Reasoning-SQU
learning framework with reinforcement learning (RL) to enhance the reasoning process of LLMs for Text-to-SQL by encouraging the generation of detailed intermediate reasoning steps that ultimately lead to more accurate SQL queries.
A core component of RL-based training is the reward function, which is crucial for guiding the model. For Text-to-SQI, the most intuitive reward would be the execution accuracy. However, its binary and sparse nature provides limited feedback when the model partially captures correct logic or schema relationships (Nguyen et al., 2025). Reward sparsity is notable challenge in many RL-based training approaches, often impeding the optimization of policy models (Blier & Ollivier, 2021; Rengarajan et al., 2022; Jaderberg et al., 2016). To address this, we design a composite reward that integrates multiple partial rewards to address the reward sparsity problem for Text-to-SQL, namely LLM-as-a-Judge, Syntax Check, Schema Linking, and N-gram Similarity rewards. We employ Group Relative Policy Optimization (GRPO) (Shao et al., 2024) to effectively integrate these reward signals. By generating multiple candidate queries per input and evaluating them relative to each other, our approach provides robust and informative feedback mechanism that directly optimizes both the intermediate reasoning process and the final execution accuracy.
Our extensive ablation studies on various reward designs underscore the effectiveness of our proposed partial rewards in boosting model performance beyond what execution accuracy alone can achieve. We validate Reasoning-St including BIRD, Spider, Spider-DK, and Spider-Syn (Li et al., 2024c), demonstrating that smaller models trained with our approach not only outperform conventional SFT methods but also exceed much larger proprietary base models, with gains of 4% over 03-mini and 3% over Gemini-1.5-Pro-OU0
on several challenging benchmarks,
Moreover, our models integrated into standard text-to-SQL. pipelines deliver a state-of-the-art 72.78% execution accuracy, all at 93% lower monetary inference cost. We further establish the superiority of the model's 's emergent reasoning capabilities by showing that its naturally evolved, structured reasoning outperforms well- designed, hand-crafted step-by-step approaches.
Our main contributions can be summarized as: (1) Automatic RL-based Reasoning Opti- mization: We introduce, Reasoning-SQQ!. the first RL-based framework that automatically optimizes the reasoning process of LLMs for the Text-to-SQL task. (2) Novel Partial Rewards Suite: We propose a novel suite of partial rewards specifically designed for Text-to-SQL, with comprehensive ablation studies demonstrating their effectiveness. (3) State-of-the fi--the-Art and Cost-Effective Performance: Our method achieves state-of-the-art results on the BIRD test set among the best open-source models, competing with significantly larger proprietary models while being much more cost effective.
2 Related Works
Text-to-SQL. Early Text-to-SQL methods used rule-based and template-driven approaches with handcrafted grammars and keyword matching (Androutsopoulos al., 1995; Hristidis et al., 2003; Li & Jagadish, 2014; Popescu et al., 2003; 2004). With deep learning, the focus models like Seq2SQI. (Zhong
shifted to neural semantic parsing using sequence-to-sequene et al., 2017) and SQLNet (Xu et al., 2017), which employed reinforcement learning to boost execution accuracy (Zhong et al., 2017; Xu et al., 2017; Popescu et al., 2004; Li & Jagadish, 2014). Later work further advanced these techniques by integrating schema linking, structured decoding with graph-based encoders, and constrained generation (e.g., IRNet (Guo et al., 2019) and RAT-SQL (Wang et al., 2019; Cai et al., 2021)), along with incremental parsing strategies such as PICARD to ensure syntactic validity (Scholak et al., 2021; Xu et al., 2017; Popescu et al., 2004; Li & Jagadish, 2014). LLMs have revolutionized Text-to-
2

---

Published as a conference paper at COLM 2025
SEWARDS
Σ 1%--%
COMPLETIONS
XLA
STRTAA
SCEENA
ereasoninge un w/reesoninge
WHESEE csame Contra Gosta AND
Num NumTstTokr DI9C UNIT
creasozing </reasonings
WHiRE csame Contra Coscal AND
0.75
0.8
0.8
4.1
school
MODEL
NumTalTukr D1SO LIII
srcosoning-
Query
OEPO
SELECT snams, Mor.HunTTTTae
FROM sutucores AND NOT
0.5
0.5
0.52
2.02
oname
NULL
sname
NumTstTakr DESC
creasp-ings-... ... </reasonings SELECT school FROM entecooe
county "Contra Costa' NOT Num TomTatTukr DESG LIITT
0.25 0.33
0.52
2.35
Figure/ 1: Overview of the GRPO-based Text-to-SQL training pipeline. For each natural language prompt q and its associated database schema, the policy model πi. generates group of candidate SQL queries. Each candidate evaluated using suite of reward functions produce composite reward. These rewards are then used to compute advantages and update the policy via GRPO.
SQL by enabling zero-shot and few-shot learning through advanced prompt engineering and in-context learning (Brown et al., 2020; Wei et al., 2022; Dong et al., 2023; Pourreza Rafiei, 2024a; Gao et al., 2023). Recent methods use multi-step schema linking, self-correction, and execution feedback-to iteratively refine SQL generation (Pourreza et al., 2024; Wang et al., 2023; Sun et al., 2023a; Xie et al., 2025; Gao et al., 2024; Talaei et al., 2024), while other approaches directly fine-tune open-source LLMs large Text-to-SQIe corpora (Li et al., 2024b;c;; Nan et al., 2023; Pourreza & Rafiei, 2024b). Although reinforcement learning was initially employed to improve SQL generation by leveraging execution feedback (Zhong et al., 2017; Xu et al., 2017; Popescu et al., 2004; Li & Jagadish, 2014), more recent RL-based methods that incorporate intermediate reward signals and chain-of-thought prompting (Nguyen et al., 2025; Zelikman et al., 2022; Jaderberg et al., 2016) have only yielded modest improvements compared to techniques that directly capitalize on the reasoning capabilities of LLMs (Kojima et al., 2022; Sun et al., 2023b).
appelines-interratiin
RL for reasoning. OpenAI's O-series models achieve strong performance in complex rea- soning tasks, such as mathematical problem-solving code generation, and logical deduction, primarily due to their use of chain-of-thoughe prompting combined with RL training (Jaech et al., 2024; Wei et al., 2022). Building these successes, the research community has worked to reproduce and extend these capabilities (Qin et al., 2024; Zhao et al., 2024). No- tably, DeepSeek R1 (Guo et al., 2025) and Kimi-K1.5 (Team et al., 2025) have emerged as strong contenders, leveraging RL-based training to foster autonomous internal reasoning and reflection (Shao et al., 2024). This progress has spurred further work in enhancing performance on beasoning-intensti ware evolution (Wei et al., 2025), mathematical reasoning (Luo al., 2025), and visual maze solving (Dao & Vu, 2025). Despite Text-to-SQL requiring similarly sophisticated reasoning, involving user intent interpretation, schema comprehension, and precise SQL query con- struction, there remains notable gap in explicitly applying RL-based training methods to improve LLM reasoning in this domain.
tasks such as code generation (Zeng et al., 2025), soft-
3 Methodology
To train a reasoning language model for Text-to-SQL, we employ an RL framework built upon GRPO and complemented by set of carefully crafted reward functions. This combi- nation is specifically designed to provide task-specific guidance, addressing key hurdles in Text-to-SQL generation, including accurate schema linking, the generation of valid SQL syntax, and the limitations posed by sparse execution feedback. An overview of our training flow is presented in Figure 1.
3

---

Published as a conference paper at COLM 2025
3.1 Reinforcement Learning Protocol
For RL fine-tuning of LLMs, methods based on policy optimization, such as Proximal Policy Optimization (PPO) (Schulman et al., 2017) and GRPÓ (Shao et al., 2024), have been explored. Given the demonstrated effectiveness of GRPO in training models like R1 (Guo et al., 2025) and its advantages over PPO, including eliminating the need for separate value model and reducing memory requirements, we focus on GRPO to effectively optimize the policy model The for SQL generation. For each input, consisting of natural language question q and its associated database schema, the model generates a group of G candidate SQL queries, (01, ................................ ......................))........ Each candidate is evaluated using a composite reward function that is designed to capture both the end-goal of correct execution and several intermediate quality measures essential to Text-to-SQL.
GRPO leverages the relative performance of candidates within the group to compute advantage A for each output, guiding policy updates according to the following objective:
1 G
no(0i19)
7ep (0:ilg)
A
JCRPO (0)= =
L
min
A; clip Tal (0:19)
1- e, e,1 (e) 41)
G
ol (o: (0
- (KL Aret),
(1)
where ae is the policy before the update, nee is the reference policy (typically the initial model), and and B are hyperparameters controlling the update step and divergence regularization. By generating multiple candidates per input, GRPO naturally accommodates the inherent ambiguities and challenges of mapping natural language to SQL queries, ensuring that feedback is both robust and informative.
3.2 Reward Design
Having a well-shaped reward is key to the efficacy of RL training (Akalin & Loutfi, 2021; Bouktif et al., 2023; Trella et al., 2023). For SeX, the most intuitive reward would be the execution accuracy (Nguyen et al., 2025; Li et al., 2024c), which can check if the generated SQL queries produce the expected output after getting executed, commonly known as Reinforcement Learning from Execution Feedback (RLEF). However, the binary and sparse nature of it would create challenges for RL optimization (Blier & Ollivier, 2021; Rengarajan et al., 2022; Jaderberg et al., 2016), as many candidates receive the same feedback despite varying degrees of correctness, and the infrequent reward signals offer limited guidance for differentiating between nearly correct and completely incorrect outputs. To mitigate this, we introduce a set of partial rewards that provide more granular guidance, such as LLM-as-a-judge reward, commonly referred to as Reinforcement Learning from Al Feedback (RLAIF). The final reward used to optimize the policy model is the weighted sum of all partial and the execution accuracy rewards. Figure 2 provides example of how each of the rewards is calculated given the gold (ground truth) and a generated query.
Execution accuracy reward (RLEF). This reward directly assesses whether the generated SQL. query, when executed against the database, produces the correct results. For each candidate query, we perform an execution alongside the ground-truth query and compare their resulting outputs, with the perfect match earning the full reward. While execution accuracy serves as the ultimate benchmark aligned with the downstream goals, its binary nature results in sparse reward signal. This sparsity is limitation, as many queries might contain elements of the correct logic but still fail to yield the exact correct output, thus receiving no reward.
LLM-as-a-judge reward (RLAIF). To supplement binary execution feedback, we employ an LLM as a judge. Using specially designed prompt (provided in Appendix Section A.3) that incorporates specific rubrics for comparing SQL queries, the LLM evaluates candidate queries against the ground truth. This evaluation is performed exclusively for queries with zero execution accuracy, assessing them based on criteria such as logical consistency, structural similarity, and semantic correctness. This provides nuanced partial reward, differentiating between various incorrect answers. The x in Figure 1 indicates a correct query, thus requiring no additional AI feedback.
4

---

Published as a conference paper at COLM 2025
Which school in Contra Costa has the highest number of test takers?
Gold Query
4S Generated Query
SELECT aname FROM catecore WHERE Conte Costa' AND sname NOT NULL ORDER BY NumTetTakr DEBO LIMIT
SELECT School School FROM satscore WHERE Cname-Contra Costa AND sname IB NOT NULL ORDER BY NumTotTakr DESC LIMIT
ADDE LIM
-entacore -cname
-Smame -Eatocore -NumTasTakr
(SELECT rrama, FROM sutscore, m]
[SELECT school, FROM satscore, ...
correct, The column in clause with sname".
JACCARD SIMILARITY
JACCARD
Figure 2: Example partial reward calculation for generated SQL query, illustrating how each reward component, execution accuracy, llm-as-a-judge (AI Feedback), syntax check, schema linking, and N-gram similarity, is derived by comparing the candidate query with the gold (ground truth) query.
Syntax check reward. Syntactic validity is fundamental prerequisite for any query to be considered meaningful or executable. We design corresponding reward so that positive score is assigned when the generated SQL query is syntactically valid and executes without errors, regardless of whether it returns the exact correct result. This reward strategy aids in distinguishing between incorrect queries by rewarding syntactic correctness, thereby instructing the model to prioritize well-formed query generation.
Schema linking reward. Accurate schema linking critical in Text-to-SQL (Wang et al., 2019; Pourreza & Rafiei, 2024a; Talaei et al., 2024; Caferoğlu & Ulusoy, 2024), as the model must correctly identify and reference the relevant tables and columns. Our schema linking reward quantifies the Jaccard similarity between the set of used schema items in the candi- date query relative to the gold query, directly addressing the challenge of mapping natural language entities to the correct parts of the database schema. Figure 2 illustrates an example of schema linking reward computation for a pair of predicted query and gold query.
N-gram similarity reward. The structured nature of SQL allows leveraging token-level overlap as a measure of similarity. The N-gram similarity reward computes the Jaccard similarity between N-grams in the candidate and gold SQL queries. Given SQL's inherent hierarchical syntax and structured format, this reward promotes alignment with the correct query's 's lexical and syntactic structure, even when minor differences exist. Figure 2 illustrates the computation of N-gram similarity using bigrams (N=2), as employed in this paper.
Format reward. Finally, we include a format reward that encourages adherence to a prede- fined output structure (e.g., using <reasoning> and <answer> tags). Outputs that conform to this pattern receive reward boost, thereby enhancing clarity and consistency. This structure also triggers zero-shot chain-of-thought reasoning in the policy model, which progressively improves as training advances to optimize for the reward.
Together, the weighted sum of these evaluation functions constitutes the final composite reward. Formally, given evaluation functions fi each providing a reward ff and cor- responding weights Wf. the final composite reward is defined as: r = E wf r.. In our experiments, the weights are carefully chosen to ensure that no incorrect SQL query can achieve higher overall reward than correct query. This reward aggregation yields dense, informative feedback, significantly enhancing the efficacy of the GRPO training framework to guide the model toward generating SQL queries that are both syntactically valid and semantically accurate (Section 5.1). Figure illustrates the overall training pipeline where candidate queries are generated, evaluated through the specialized reward functions, and reinforced the model's s reasoning policy Tg towards generating more accurate SQL queries. For the pseudo-code of our algorithm please refer to Appendix A.7.
5

---

Published as a conference paper at COLM 2025
3.3 Supervised Fine-Tuning Baselines
To establish robust baselines, we implement two SFT approaches: direct SQL prediction and the bootstrapped STaR-SFT method (Zelikman et al., 2022). In the direct SFT paradigm, the model is trained via maximum likelihood estimation on gold SQL queries, serving as reliable benchmark. To assess the benefits of training on the Self-Taught Reasoner (STaR)
reasoning traces with SFT,
we introduce a second framework (Zelikman et al., 2022). The STaR-SFT method explicitly integrates chain-of- thought (Co'T) reasoning by leveraging the Soride-and-onqueee prompt from CHASE-SQL (Pourreza al., 2024) and the Gemini-1.5-Pro 2022). When these CoT sequences yield correct SQL outputs, the corresponding samples are added the training set; otherwise, the ground truth SQL provided hint to guide the model toward producing accurate CoT chains.
model to generate CoT traces (Wei al.,
4 Experimental Setup
In our experiments, we are going to address the following research questions:
How do different reward functions and the inclusion of reasoning affect performance relative to SFT? RQ2) Baseline Comparison: How does Reasoning-SQU compare to existing Text-to-SQL methods in execution accuracy and schema linking? RQ3) Emergent Reasoning: How does structured reasoning emerge during RL training, and how does it compare with human-designed strategies? RQ4) Generalization Analysis: How well does Reasoning-Se Masonnnn-SSLLe generalize to out-of-distribution SQL queries and related tasks?
RQ1) Training Objective Ablation:
4.1 Baselines and Benchmarks
Benchmarks and metrics. We trained our models on the BIRD training set (Li et al., 2024c), comprising 9,428 Question-SQL. pairs from 70 databases across domains like airlines, movies, and sales. To address known issues with noisy and ambiguous queries (Pourreza et al., 2024; Talaei et al., 2024; Li et al., 2024b), we filtered out samples flagged incorrect by both Gemini-2.0--llsh and GPT-44, resulting in 8,026 training examples. For evaluation, we primarily use the BIRD benchmark and include Spider, Spider-DK (Gan et al., 2021b), and Spider-Syn (Gan et al., 2021a) to assess generalization. While BIRD captures real-world SQL challenges with diverse schemas and noisy queries, Spider provides an out- ditto---istribution test with queries from broad range of databases. Spider-Syn tests robustness through paraphrased questions using schema-related synonyms. Spider-DK modifies Spider queries with added domain knowledge to reflect realistic paraphrasing. We report execution accuracy (EX), requiring exact row match.
Training settings. We employ open-source models from the Qwen2.5-Coder (Hui et al., 2024) family, specifically the 3B, 7B, and 14B variants, for our experiments. We select this model family primarily due to its strong performance and the availability of multiple model sizes, allowing us to adapt to different computational constraints and use cases. In our work, a model trained with all of our proposed partial rewards is denoted with the subscript (allrewards) (e.g., --coeer--- (all rewards)). Since most small LLMs, such as those used in this paper, have limited context window sizes-are and to decouple schema linking from SQL generation for improved performance (Li al., 2024b; Pourreza & Rafiei, 2024b)-we first for schema linking, following the method
filter the database schema using Gemini-1.5- proposed in (Talaei et al., 2024). To ensure that the filtered schema contains all necessary information, combine it with the ground truth schema used in the corresponding SQL query. However, during inference, as the ground truth answer is unavailable, we only filter the database schema using Gemini-1.5-pp manageable schema. he weights assigned to the partial rewards in this work are as follows: Wexee = 3, Wjudge 2, and all other partial reward weights set to 1. More details of the training hyperparameters and settings are provided in Appendix section A.1.
to provide the model with smaller, more

---

Published as a conference paper at COLM 2025
Text-to-SQL pipelines. State-of-the-art Text-to-SQL methods typically use multi-stage pipelines rather than a single LLM call, incorporating steps like value retrieval, schema linking, query generation, self-correction, and selection (Pourreza et al., 2024; Gao et al., 2024; Lee et al., 2024; Wang et al., 2023). For fair comparison, we integrate our trained models into the CHASE Following CHASE-SQL's setup, we train binary pairwise verifier utilizing the open-source Qwen2.5-Coder 14B for query selection.
pipeline (Pourreza et al., 2024), replacing Gemini-1.5-Pro.
Results
5.1 Training Objective Ablation
In this section, we provide detailed analysis of the performance of the Qwen2.5-Coder 7B model trained with different reward functions and compare it against models trained using SFT and the base Qwenn25-Coder-Instuuc model (prompts for training are provided in Appendix A). For SFT, we consider direct SQL prediction as well the STaR-based (Zelikman et al., 2022) approach, which includes a CoT reasoning step before generating the SQL query. In our GRPO experiments, we explore various combinations of the reward functions described in Section 3.2. Additionally, to evaluate the importance of step-by-step report results from GRPO training on model that was
reasoning for SQL generation, not required to output these reasoning steps. To ensure consistency in evaluation, we use Gemini-1.5-PPe 5-PPro-002 for schema linking, following the CHESS framework (Talaei et al., 2024), and filter the database schema before feeding it into the models. We further report the fltered-schene performance by providing the correct database schema for each question, representing the best achievable performance given perfect schema linking module.
Table 1: Execution Accuracy (EX) of the Qwen2.5-Coder-77 5-Coder-7B model trained with different RL rewards and SFT datasets. In addition to overall EX and filtered-sdd -schemm (FS) EX, we report the Syntax Check, Jaccard Similarity of Schema Items, and Jaccard Similarity of N-gram metrics. The upward arrows and delta values indicate the direct improvement in the corresponding metric when associated reward is added.
Post-training Method
Model
Thinking
Syntax
Schema
N-gram
EX (%)
(%)
No Post-training
Qwen2.5-Coder-78
93,02
91.57
56.09
58.73
64.01
Owee2.5-codee776 SFT
61.53
69.23
Qwen2.5 SB.
94.71
90.71
58.02
62.84
68.12
rowards)
94.71
92.03
58.02
62.25
68.12
95.04
92.17
57.62
62.32
68.90
58.34
62.84
68.97
CRPO training
Dwen225-coder-7BB.... (wochem)))
95.34
93.04
58.12
63.10
69.03
Qwen2 Qwen2.5-coder-2. 5-ooddee-B-BBeaaa.gaaa
95.78
92.26
62.08
63.08
68.9
0wen2.5 coder-7 Bondor-7Binngnncceee
95.76
92.95
58.34
63.10
69.23
Qwen2 (reen2,5-codr--BB.. (ecodde:eBB.acaaaaaala..w.. messeceessnns
96.60
92.85
62.26
63.75
69.94 29.94
Qwen2.5 coone.-- coder-7B RLAIF)
69.16
Quven..-codee--7B, all rewards)
96.21
92.91
61.18
64.01
71.66
As shown in Table 1, using our proposed partial reward functions yields higher performance compared to training with the sparse execution accuracy reward alone. Notably, the addition of each reward directly improves its corresponding metric. For example, adding the syntax reward boosts the Syntax Check score by 1.37, incorporating the schema reward raises the Jaccard Similarity of Schema Items by 0.78, and including the N-gram reward increases the Jaccard Similarity of N-gram by 3.92. These improvements, highlighted with upward arrows and delta values, emphasize that each reward has a targeted effect while simultaneously contributing to the overall execution accuracy. Overall, our proposed method improves the base model's performance by 6.77% whereas conventional SFT training alone results 4.11% performance gain. Moreover, comparing our CRPO-trained model with reasoning against the model that is GRPO-trained but only predicts the SQL query, we observe 2% performance gain, demonstrating the importance of step-by-step reasoning for SQL generation. We further investigate the behavior of models under varying test-time compute budgets, with additional details in Appendix A.4.
7

---

Published as a conference paper at COLM 2025
5.2 Baselines Comparison
Models comparison. We report the performance of Qwen2.5-Coder models with 3B, 7B, and 14B parameters, trained using GRPÓ with all partial rewards on the BIRD development set. We compare these models against their respective base versions trained with SFT, as well as the base model itself. Additionally, we benchmark their performance against SOTA models, including 03-mini, and execution accuracy with schema linking, where 1--ini-1-5-Pro-002 serves as the schema linker together with filtered-scheme accuracy, which is measured using the correct schema. As shown in Table 2, models trained with our proposed reward functions consistently outperform their SFT-trained counterparts, with gap scaling with the model size (indicated by up arrows). More importantly, the 14B model surpasses the latest reasoning reasoning-focused focused model, o3-mini, by a significant margin of 4%. The only model that performs on par with our largest model is Gemini-2.0-Flash. This demonstrates the effectiveness in introducing reasoning capabilities, achieving SOTA performance with 14B model that can be deployed on single GPU.
Following the approach in Section 5.1, we evaluate
Table 2: Execution Accuracy (EX) of various models trained with all RL rewards, the SFT model, and selected SOTA models. EX reported with schema linking, while the filtered-schema EX is measured using the correct schema.
Model
Thinking
EX (%)
Filtered-Schema EX (%)
GPT-4o-mini
X
60.82
66.29
03-mini
/
61.34
69.88
Gemini-220-flash
X
66.49
71.9
Gemini-1.5-pro-002
X
62.25
67.47
Owen2.5-coder-33
X
45.17
48.17
Qwen2.5-coder-3BB5 SFT
x
55.9
63.62
Qwen2.5-coder-3B::n rewards)
V
58.67
65.31
+1.69
Qwen2.5-coder-7B
X
58.73
64.01
Dwen2.5-coder-7B555
X
61.53
68.12
0wen2.5-coder-7B:an rewards)
64.01 V
70.66
+2.54
Qwen2.5-coder-14B 14 SFT
X
63.1
68.77
X
63.75
68.83
Qwen2.5-coder-144B rewards)
V
65.31
72.03 1 +3.20
Reasoning-SQL previous text-to-SQL methods. Recent Text-to-SQL systems, particu- larly on benchmarks like BIRD, adopt pipeline frameworks where SQL generatio is just one part of larger process (Pourreza et al., 2024; Gao et al., 2024; Xie et al., 2025; Lee al., 2024; Wang et al., 2023). These pipelines typically incorporate schema linking (to simplify schemas and reduce LLM confusion (Talaei et al., 2024)), initial SQL generation using filtered schemas, iterative self-correction, and a query selection nechanism-eitee (Pourreza et al., 2024; Gao et al., 2024) or self-consistency driven (Sun et al., 2023a; Gao et al., 2023). To assess our GRPO-trained models in this context, we integrate them into the CHASE-SQL, framework for schema linking, SQL generation, and self-correction, and replace Gemini-1.5-pri with fine-tuned Qwwn2.5-Coder-144 model for query selection. As shown in Table 3, our approach achieves 72.29% and 72.78% execution accuracy on the BIRD development and test sets, respectively. Our approach bridges the gap between open-source and proprietary solutions, positioning it as the ttat-off-the-rt among small open-source models while outperforming many GPT-based systems. Additionally, it is important to note that our approach attains performance on par with CHASE-SQL 93% lower cost, as detailed in Appendix Section A.5.1.
model-base
8

---

Published as a conference paper at COLM 2025
Table 3: Execution Accuracy (EX) of our model with the CHASE-SQL pipeline and comparison with all methods on the BIRD development set. "All" denotes the use of all introduced rewards combined.
Method
Model
Model Size
Dev EX (%)
Test EX (%)
CHASE-SQL.
Gemini-1.5-pro
Unknown
74.46
74.79
(Pourreza et al., 2024)
XiYan-SQI,
Ensemble of Models
Unknown
73.34
75.63
(Gao et al,,
OpenSearch-SQL.V. v2
GPT-4o
Unknown
69.3
72.28
al,
CHESSIR+CG++UT
Gemini-1.5-pro
Unknown
68.31
71.10
(Talaei et al., 2024)
Distillery
GPT-4o
Unknown
67.21
71.83
(Maamari et al,, 2024)
XiYan-SQL
QwenCoder-32B
32B
67.01
69.03
(Gao et al., 2024)
E-SQL
GPT-4o
Unknown
65.68
66.29
(Caferoglu & Ulusoy,2022
CodeS-15B
CodeS
15B
56.47
60.37
(Li et al., 2024b)
Think2SQL
Qwen2.5-7B
7B
58.1
(Papicchio et al., 2025)
DTS-SQL
DeepSeek-7B
7B
55.8
60.31
(Pourreza & Rafiei, 2024b)
Reasoning-SOO SQL (Ours)
Qwen2.5- rewards)
14B
72.29
72.78
Reasoning-SQI. (Ours)
Qwen2.5-7BB all rewards)
7B
68.05
Reasoning SQL (Ours)
Swen2.5-coder-7B SFa Qwen2.5-coder-7
7B
66.92
Reasoning SQL (Ours)
7B
66.12
Quastion: Who the owner the post "Elloiting priors from experte*?
USER
this question. need owner stored the users tablc, and the title of the post stored the poots tablc. We need to join these two tobles based on the 'Id' column, which co common both tab-es, The 18 column the pests teble comeiponds to the Parentic column in the users table.
Reasoning:
the
titlec "Ellotting prfors from experts". The Information about the
which the forsian kay reforemoing
columi Answer: SELECT T2 YaDDisplaMemme FROM posts AS INNER JOIN usera AS ON T1. P1.Prrntidd-2.id
users toole. On have joined the tables, we select DisplayNa
WHERE TL.iile 'Elieiting priora from experts'
MODEL AT STEP OF TRAINING
Reasoning: To enswer this question, we nead to fine the owner of the post with the title "Eliciting priora from experta" Based on the
pravided cable schema, have two relevant tab les: posts users: posts table concaine title posts and the users
Here's stap-by-step by-step breakdown
derive the
1. iddettif the tables involved: We need to access the possti table to find the post with the title "Elio ting priors from experter and the
users tab get display name the owner. join
pposte'
foreign key referencing the
'usere'
two tables
column.
3. specify the join type: We use INNER JOIN becsuse we only want to oo solder poots that have corresponding user the "usore" table.
Specify the conditions: We need to fllter resulto to only ino udd post with thte "Ellolting piiore from experto": We can do this by adcling 'WHERE" o-suse.
required columns:
'user''
this
Wrise the SQL query: Putting all topether, the SQL query
Answer: SELECT 72. DisplayNeme FROM posts As (NNER JOIN users AS 72 ON m...merUsarii--2..e
'éllciiing priora experts'
MODEL AT STEP 200 OF TRAINING
Figure An example illustrating the model's improvement in reasoning. step 200 of training, the model adopts structured approach to SQL synthesis and correctly identifies the join condition.
5.3 Emergent Reasoning
In Section 5.1, we demonstrate that our reasoning model Qwen2.5-coder-7B:
rewards)
outperforms other baselines. Here, we further examine the reasoning strategy the model has developed. As illustrated in Figure 3, during early training stages the base model produces non-structured, sub-optimal reasoning trace for synthesizing the SQL query, resulting in an incorrect "JOIN" operation. However, as training progresses, the model adopts a more structured reasoning style, yielding the correct SQL output. Notably, we do not enforce any particular reasoning format rather, the effective reasoning style emerges naturally as the model optimizes for designed rewards (for more examples see Appendix A.2). For further insights into the evolution of execution accuracy and reasoning efficiency during training, please refer to Appendix A.6.
We then ask whether the emergent reasoning pattern outperforms human-designed thinking formats. We generate thinking traces using three human-designed strategies, Divide-and-

---

Published as a conference paper at COLM 2025
Conquer (DC), Query Plan (QP) (Pourreza et al., 2024), and ACT-SQL (Zhang et al., 2023), for three randomly selected instances from the BIRD training set. Then, for each set of traces, the three human-designed strategies and our model's emergent reasoning, design corresponding prompts using these thinking traces as few-shot examples. Finally, we evaluate the in-context learning performance of two base models, Swee-14B-codeeinstee and CodeGemma-7b-it, on the BIRD dev set. As shown in Table 4, prompts based on the emergent Reasoning-SQL style consistently outperform those using human-designed strategies for the two base models, highlighting both the efficacy of the learned reasoning style and its transferability across different models.
Table 4:
Performance comparison gener- ated from emergent reasoning vs. human- designed strategies on the BIRD dev set.
Table 5: Comparison of performance on Spider, Spider- DK, and Spider-Syn (EX).
Thinking
Spider
SpiderDK
Model
Prompt
Dev EX(%)
GPT-ko-mini
76.3
71.02
67:00
ACT-SQL
62.84
<3-mini
78.82
71.77
73.01
Qwen-14B
Divide-and-Conuee
60.75
Gemini-2. 2.-flahh
Cemini-1.5-pro-l
80.46
81.23
74:15
67.66
71.56
73.99
code-inst
Query Plan
61.92
Reasoning-SQL
64.21
Qwen? 5-vdder-78
Owent 5-vxlat-78 SFT
77.85
68.06
66.34
62.8
66.4
51.98
ACT-SQI.
48.37
Qwen2 5-aolar-7B moandss
78.72
73.27
69.34
CodeGemma 7B-it
Divide-and-Conquer
48.82
Qwent 5-vdirr-488
29.40
71.4
72.14
Query Plan
Reasoning-SQU
50.19
48.56
Qwen2 soodar-14B, cowade
Qwen2 5-axder-14885 14B SFT
75.04
61.43
73.03
69.15
61.00
72.63
5.4
Generalization Analysis
We evaluate the generalization capabilities of our approach three benchmarks, Spider, Spider-DK, and Spider-Syn, which feature distinct SQL query and natural language question distributions. To ensure a fair comparison, Gemini-1.5-Pro model to isolate the relevant schema components, as detailed in Section 5.1. As shown in Table 5, our GRPO-trained model outperforms state-of-the-art alternatives like o3-mini and Gemini-1.5-PP reveal that while our RL-trained models consistently improve upon the base models, the SFT-trained models underperform even relative to the base, confirming our observation that RL fosters robust generalization whereas SFT tends to favor memorization (Chu et al., 2025).
first performed schema linking using the
with on all three benchmarks. Furthermore, our experiments
Conclusion
In this paper, we introduce a novel RL-based approach for the Text-to-SQL. task, focusing on addressing critical reasoning subtasks, such schema comprehension, query generation, and self-correction. Leveraging a set of carefully designed partial rewards and employing GRPO, we train different LLMs, significantly improving their reasoning and generalization capabilities. Our extensive experiments on benchmarks such as BIRD, Spider, Spider- DK, and Spider-SYN demonstrate that our RL-trained models consistently outperform models trained through SFT. Notably, our 14B-parameter model achieves state-of-the-a performance on the BIRD benchmark, surpassing significantly larger proprietary models. Our work underscores the potential of RL-based training methods and partial rewards to significantly advance the performance and generalization of smaller, open-source LLMs, reducing the performance gap between proprietary and open-source solutions in Text-to- SQL applications and enabling reasoning abilities.
References
Neziha Akalin and Amy Loutfi. Reinforcement learning approaches in social robotics. Sensors, (14) 2021.
Ion Androutsopoulos, Graeme D Ritchie, and Peter Thanisch. Natural language interfaces to databases-an introduction. Natural language engineering, 1(1):29-81 81, 1995.
10

---

Published as a conference paper at COLM 2025
Léonard Blier and Yann Ollivier. Unbiased methods for multi-goal reinforcement learning. arv preprint arXiv:2106.08863, 2021.
Salah Bouktif, Abderraouf Cheniki, Ali Ouni, and Hesham El-Sayed. Deep reinforcement learning for traffic signal control with consistent state and reward design approach. Knowleedge-Base Based Systems, 267:110440, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Adoances in neural information processing systems, 33:1877-1901, 2020.
Hasan Alp Caferoğlu and Özgür Ulusoy. E-sql: Direct schema linking via question enrich- ment in text-to-sql. Xiv preprint ar arXiv:2409.16751 2024.
Ruichu Cai, Jinjie Yuan, Boyan Xu, and Zhifeng Hao. Sadga: Structure-aware dual graph aggregation network for text-to-sql. Advances in Neural Information Processing Systems, 34: 7664-7676, 2021.
Ruisheng Cao, Lu Chen, Zhi Chen, Yanbin Zhao, Su Zhu, and Kai Yu. Lgesql: line graph enhanced text-to-sql model with mixed local and non-local relations. ar arviv preprint arXiv:2106.01099, 2021.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sit memorizes, rl generalizes: comparative study of foundation model post-training. arXiv preprint ar X1c1:2201..1111, 2025.
Alan Dao and Dinh Bach Vu. Alphamaze: Enhancing large language models' spatial intelligence via grpo. arXiv preprint arXiv:2502.14669 2025.
Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, Jinshu Lin, Dongfang Lou, et al. C3: Zero-shot text-to-sql with chatgpt. arXiv preprint Xiv: 07300, 2023.
Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, and Pengsheng Huang. Towards robustness of text-to-SQL. models against syn- onym substitution. pp. 2505-2515, Online, August 2021a. Association for Computational Linguistics. doi: 10.18653 v112021.aclllongg 1955. URL https://aclantthol..g or 2021. acl-long. 195.
Yujian Gan, Xinyun Chen, and Matthew Purver. Exploring underexplored limitations of cross-domain text-to-sq generalization. ar preprint a=Xtn:21099.0115,, 2021b.
Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. Text-to-sql empowered by large language models: benchmark evaluation. ar Xiv preprint Xiv:2308. 15363, 2023.
Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, al. Xiyan-sql: multi-generator ensemble framework for text-to-sql. Xiv preprint ar arXio:2411.08599 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, ar arXiv preprint ar Xiv:25011112943 2025.
Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. Towards complex text-to-sql in cross-domain database with intermediate representation. ar Xiv preprint orXto:1905.08055 2019.
Vagelis Hristidis, Yannis Papakonstantinou, and Luis Gravano. Efficient ir-style keyword search over relational databases. In Proceedings 2003 VLDB Conference, pp. 850-861. Elsevier, 2003.
11

---

Published as a conference paper at COLM 2025
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report. arXiv preprint (rXiv:2409.12186, 2024.
Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, and Koray Kavukcuoglu. Reinforcement learning with unsupervised auxil- iary tasks. ar arviv preprint arx: 1.05977, 2016.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai system card. ar preprint ar axXit:2412..67220 2024.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.
Large language models are zero-shot reasoners. Advances in neural information processing systemss 35:22199 22213,,, 2022.
Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation. arviv preprint arXiv:2405.07467, 2024.
Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. The dawn of natural language to sql: Are we fully ready? arXiv preprint ar arXiv:2406.01222 01266, 2024a.
Fei Li and Hosagrahar Jagadish. Constructing an interactive natural language interface for relational databases. Proceedings of the VLDB Endowment, 8(1):73-84, 2014.
Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. Codes: Towards building open-source language models for text-to-sql. Proceedings of the ACM on Management of Data, 2(3):1-28, 2024b.
Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. Can Ilm already serve as database interface? big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems, 36, 2024c.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol-preview with 1.5b model by scaling rl, 2025. Notion Blog.
Karime Maamari, Fadhil Abubaker, Daniel Jaroslawicz, and Amine Mhedhbi. The death of schema linking? text-to-sql in the age of well-reasoned language models. arXiv preprint ar arXit:2408.07702 2024.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. Xiv preprint ar XXx::2501.19399 2025.
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, and Dragomir Radev. Enhancing few-shot text-to-sql capabilities of large language models: A study on prompt design strategies. arv preprint ar arXiv:2305.12588 2023.
Xuan-Bang Nguyen, Xuan-Hieu Phan, and Massimo Piccardi. Fine-tuning text-to-sql models with reinforcement-eerrnnn training objectives. Natural Language Processing Journal, pp. 100135, 2025.
Simone Papicchio, Simone Rossi, Luca Cagliero, and Paolo Papotti. Think2sql: Reinforce Ilm reasoning capabilities for text2sql. Xiv preprint Xxx::2504.11077, 2025.
12

---

Published as a conference paper at COLM 2025
Ana-Maria Popescu, Oren Etzioni, and Henry Kautz. Towards theory of natural language interfaces to databases. In Proceedings of the 8th international conference on Intelligent user interfaces, pp. 149-157, 2003.
Ana-Maria Popescu, Alex Armanasu, Oren Etzioni, David Ko, and Alexander Yates. Modern natural language interfaces to databases: Composing statistical parsing with semantic tractability. In COLING 2004 Proceedings of the 20th International Conference on Computa- fional Linguistics, pp. 141-147, 2004.
Mohammadreza Pourreza and Davood Rafiei. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. Adoances in Neural Information Processing Systems, 36, 2024a.
Mohammadreza Pourreza and Davood Rafiei. Dts-sql: Decomposed text-to-sql with small large language models. arXiv preprint arXiv:2402.01117, 2024b.
Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gau- rav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O Arik. Chase-sql: Multi-path reasoning and preference optimized candidate selection in text-to-sql. Xiv preprint ar Xiv:2410.019933, 2024.
Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. O1 replication journey: A strategic progress report- part Xiv preprint ar arXiv:2410.1898) 2024.
Desik Rengarajan, Gargi Vaidya, Akshay Sarvesh, Dileep Kalathil, and Srinivas Shakkottai. Reinforcement learning with sparse rewards using guidance from offline demonstration. Xrv preprint ar 2rX1:22002.04628, 2022.
Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. Picard: Parsing incremen- tally for constrained ruto-regressive decoding from language models. arXiv preprint arXiv:2109.05093, 2021.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. Xiv preprint ar *xXe:1707.06347 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint ar arXio:2402.03300, 2024.
Charlie Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling I1m test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03311 2024.
Ruoxi Sun, Sercan Ö Arik, Alex Muzio, Lesly Miculicich, Satya Gundabathula, Pengcheng Yin, Hanjun Dai, Hootan Nakhost, Rajarishi Sinha, Zifeng Wang, et al. Sql-palm: Improved large language model adaptation for text-to-sq! (extended). ar Xiv preprint or drXiv:2306.00739, 2023a.
Ruoxi Sun, Sercan Ô Arik, Rajarishi Sinha, Hootan Nakhost, Hanjun Dai, Pengcheng Yin, and Tomas Pfister. Sqlprompt: In-context text-to-sql with minimal labeled data. arxiv preprint arXio:2311.0288. 2023b.
Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint ar 4rX1:22405.1655, 2024.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. ar Xiv preprint ar arx:::2001. 12599, 2025.
13

---

Published as a conference paper at COLM 2025
Anna L Trella, Kelly W Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, and Susan A Murphy. Reward design for an online reinforcement learning algorithm supporting oral self-care. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 15724-15730, 2023.
Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers. arXiv preprint #rXio:1911.04942, 2019.
Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Qian-Wen Zhang, Zhao Yan, and Zhoujun Li. Mac-sql: Multi-agent collaboration for text-to-sql. arxiv preprint #XXv:2312.11122, 2023.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V
Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24833 2022.
Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida Wang. Swe-rl: Advancing Ilm reasoning via reinforcement learning on open software evolution. arXiv preprint arXtr:2502.18449, 2025.
Xiangjin Xie, Guangwei Xu, Lingyan Zhao, and Ruijie Guo. Opensearch-sql: Enhancing text- to-sql with dynamic few-shot and consistency alignment. ar preprint arXiv:2502.1491 149113, 2025.
Xiaojun Xu, Chang Liu, and Dawn Song. Sqinet: Generating structured queries from natural language without reinforcement learning, arXiv preprint arXit:1711.044330 2017.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances Neural Information Processing Systems, 35:15476-15488, 2022.
Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. Ace- coder: Acing coder rl via automated test-case synthesis. arXiv preprint ar af1io:2502.01718, 2025.
Hanchong Zhang, Ruisheng Cao, Lu Chen, Hongshen Xu, and Kai Yu. Act-sql: In-context learning for text-to-sql with automatically-genenndii chain-of-thought, 2023. URL https: //arxiv.orgg/// abs//310, 17342.
Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-ol: Towards open reasoning models for open- ended solutions. arxiv preprint arXio:2411..4440, 2024.
Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. ar Xiv preprint ar 1rXic:1709.00103, 2017.
14

---

Published as a conference paper at COLM 2025
A
Appendix
A.1 Training details
In this section, we provide detailed information about the training hyperparameters and settings utilized in this study. For training the Qwen-2.5-Coder models with GRPO, we employed a small learning rate of le-6 using a constant learning rate scheduler with 0.1% warm-up ratio, effective batch size of 32, and trained the models for 3 epochs the filtered training dataset. For each input prompt, we generated 6 completions as the group size requirement for GRPO training. All GRPO training experiments were conducted on NVIDIA H100 GPUs. We observed during GRPO training that permatting-relatesr rewards were optimized first, followed by the task-specific rewards, highlighting an interesting learning pattern within the model.
For supervised fine-tuning (SFT), we used a comparatively larger learning rate of 1e-5 with a cosine learning rate scheduler, maintaining the same effective batch size of 32 and training for 3 epochs.
A.2 Example Output
In this section, provide example output from the Qwen2.5-Coder-144. model, demonstrating the step-by-step reasoning performed by the model prior to generat- ing the final SQL query. This reasoning varies for each question and adapts according to the conditions required by the model.
rewards)
What the webpages for the Los Angeles County school that 2,000 3,000 takers?
USER
<reasoning>
the question, 2,000
find wabpages of schools in Los County that have step-by-step
SQL
MODEL
!''Ieentffy
Columns:**
We need
'satscores' table
on number (CDSCode).
'schools
get webpages schools.
'satscores' table has columns NNmmattakr (number takers)
'schools' table has columns 'Website' 'CDSCode.
Tables:'
We need to join 'satscores' table the 'schools' table on 'ods' column, which is the primary key in satscores' and a foreign key in schools'.
3. *"Fiteer Data:**
We need to filter the data include only schools Los Angeles County, This can be done by checking the 'County' column in the schools' table. We also need to filter the date to include only schools with between 2,000 and 3,000 test takers. This can be done by checking the 'NumTatTakr" column in the satscores" table.
4. **Seleot the Required Column:** We need to select the Website' column from the 'schools' table.
5. *Ennuree Distinct Values:**
Since we selecting webpages, we should ensure that get distinct values to avoid duplicates.
6. **Construct the SQL Query:**
Use INNER JOIN join the 'satscores'' table with the 'schools' table the 'ods' column. county number 'SLECC DISTINCT filter get unique webpages.
'WHERE'
</reasonings>
"'sql
SELECT
satscores
INNER schools AS ON T1.eds T2.CDSCode
WHERE T2.County 'Los Angeles' TENNmTstTakr BETWEEN 2000 3000;
</answer>
Figure 4: An example output of our GRPO-trained model.
Below also provide another example output of the model during the early steps of the training (step 10) and compare it with the reasoning output of the model at one epoch of training.
15

---

Published as a conference paper at COLM 2025
Question:What was the age of the youngest patient when they initially arrived at the hospital?
patient's birth from their first they initially arrived the hospital. The age can Answer: SELECT DISTINGT stritime("*N y 'First Date') strftimet ("YY, Birthday) AS FROM Patient WHERE Birthday NOT NULL ORDER BY age ASC LIMIT'S
Reasoning: answer question, need to age youngest
subtracting
MODEL STEP TRAINING
this question, need find the calculated by aubtracting the year of the patient's birthday from the year of their initial arrival date. Here step-by-step breakdown of how denive the SQL cuery. Tebles"": information
Reasoning:
when initially arrived hospital.
""ddentiyy
inc uding
initial
Pattent" table.
2. "Beetormine the Columns Needed* We need select the 'iistt Date' and 'Birthday' columns to calculate the age.
*Calculate the Aga*: age calculated using the 'SUBTRACT" function in Squite. We need extract the year from both the
Birthoay using
function. Patient* To find youngest patlent,
**iind
order the patients their ascending order select
first row
5. *Construct the SQL Query"";
only considering patients volid dates. the SUSTRACT function calculate function
STRFTTMEE ensure
and 'Birthday..
-Order the results by age in ascending order to find the youngest patient.
clause
only
Birthday) age
NOT NULL ORDER
LIMIT
MODEL AT STEP 200 OF TRAINING
Figure 5: An example output of the model during the training process at step 10 and step 200.
A.2.1 Model Checkpoints
In this section, we present the outputs generated by the Qwen2.5-Coder-3B: rewards) model at checkpoints 250, 500, 750, and the final checkpoint at 849, where the output at the last checkpoint provides the correct answer. The outputs correspond to the following question:
Question: In race No. 45, what is the abbreviated code of the driver whose Q3 time 0:01:33?
16

---

Published as a conference paper at COLM 2025
Checkpoint-25-ouupput
<reasoning>
To answer this question, we need to find the abbreviated code for the driver who had specific Q3 time for race No. 45. The key columns involved are 'raceld' and 'driverld', which foreign keys in the 'qualifying' table that reference the 'driverld' in the 'drivers' table. We will use 'OIN'' clause link these tables. The 'raceld' is specified as the query, and the "q3' column is filtered to only include times with the format 'M:S5%', which indicates the Q3 time.
Here's step-by-step breakdown of how to derive the SQL query:
**Identify the relevant tables**: We need to access data from the 'qualifying' ta- ble to get the Q3 time and the 'drivers' table to get the abbreviated code.
2. *Determine the join condition**, The 'qualifyld' in the 'qualifying' table is the same as will join these tables this common column. **Specify the join type**; Since we want find specific driver, we will use an 'NNER JOIN' to ensure that we only consider rows where there a matching driver.
the 'driverId' in the 'drivers' table,
4. **Filter by race number**; We will filter the results to only include race No. 45 by adding 'WHERE' clause with 'raceld = 45'.
**Filter by Q3 time** We will filter the results to only include drivers who have Q3 time of 'M:SS%' which indicates the Q3 time.
**Select the required columns**. We will select the 'code' column from the 'drivers' table, which contains the abbreviated code for the driver.
Use "DISTINCT** Since keyword to ensure that each driver's abbreviated code is listed only once.
interested in unique drivers, will use the 'DISTINCT'
</reasoning>
<answer>
"ssql
SELECT DISTINCT T2code FROM qualifying T1 INNER JOIN drivers AS ON TI.driverld d2driverld WHERE Tl.raceld = AND T1.q3 LIKE "M:SS"
</answer>
Checkpoint-50-ouupput
<reasoning>
To answer this question, we need find the abbreviated code for driver who participated in race No. 45 and had Q3 time of 0:01:33. Here's a step-by-step breakdown of how we derive the SQL query:
1. **1den **Identify the relevant tables**; We need to access the 'drivers' table get the abbreviated codes and the 'qualifying' table to get the Q3 times for each race.
**Dttemmine the join condition**, The common column between the two tables is 'driverld', which links each driver to their qualifying times.
3. **Specify the join type"*: Since need to link each driver to their qualifying times, use 'NNEE JOIN' between the 'drivers' and 'qualifying' tables the 'driverld' column. **Filter the race**; We need filter the results to only include the race with raceld of 45. This is done using a 'WHERE' clause on the 'raceld' column.
5. **Fitter the driver**: We need to filter the results to only include the driver who had a Q3 time of 0:01:33. This is done using 'WHERE' clause the 'q3' column, which matches the pattern 'M.SS%'
**Select the abbreviated code**; The abbreviated code is stored in the 'code' column of the 'drivers' table. We select this column from the 'drivers' table.
7. *Construct the ** Combining all the steps, we get the SQL query as follows </reasoning>
<answer>
"ssll
SELECT Clccoe FROM drivers AS T1 INNER JOIN qualifying AS T2 ON T1.driverld 12.driverld WHERE r2.aaceld 45 AND T2.q3 '1:01:33'
<lanswer>
17

---

Published as a conference paper at COLM 2025
Checkpoint-750-ouppuu
<reasoning>
To answer this question, we need to find the abbreviated code of the driver who had specific Q3 time in race No. 45. Here's step-by-step breakdown of how derive the SQL query: *Identify the relevant tables**: We need information from both the 'drivers' and "qualifying' tables. The 'drivers' table contains the abbreviated code of the drivers, and the "qualifying' table contains the Q3 times for each driver and race.
2. **Dtermiine the join condition ** The 'driverld' in the 'drivers' table is the foreign key in the 'qualifying' table, which links the two tables. We will use this as the join condition to link the driver's code with their Q3 times.
**Specify the join type**, We will use an INNER JOIN' to combine the rows from both tables where the join condition is met.
4. "*Fitter the results**: We need to filter the results to only include the race No. 45. This can be done by adding 'WHERE' clause the query with the condition 'raceld = 45'. **Filter the results further**; We need filter the results to only include the driver who had a specific Q3 time. This can be done by adding a 'WHERE' clause to the query with the condition '43 LIKE '0:01:33%"' 6. **Select the required columns** We need to select the abbreviated code of the driver. This can be done by adding 'SELECT' statement to the query with the column 'code' from the 'drivers' table.
** *Construct the SQL query**: Based on the above steps, the SQL query can be constructed as follows
</reasoning>
<answer>
"'sql
SELECT T1.code FROM drivers AS T1 INNER JOIN qualifying AS T2 ON T1.driverld 12.driverld WHERE 12.raceld 45 AND 12.q3 LIKE '0.01:33%
</answer>
Checkpoint-849-outppl
<reasoning>
To answer this question, need to find the abbreviated code of specific driver in given race where the Q3 time 0:01:33. Let's break down the steps required derive the SQL query:
"Identiffy the relevant tables**: We need to access the 'drivers' table for the abbreviated code and the 'qualifying' table to get the Q3 time.
2. **Determine the join condition**, The 'driverld' in the 'qualifying' table is the primary the 'driverld' the 'drivers' table. Therefore, will join the 'qualifying'
key that links table with the 'drivers' table on this common column.
3. **Specify the join type** Since we need to find the driver with a specific Q3 time, we will use an 'NNNR JOIN' to ensure that only consider drivers who have matching record in the 'qualifying' table.
**Filter by raceld**; We need to filter the results only include the race with done using the 'raceld' column in the 'qualifying'' table.
This
5. *Filteer by Q3 ** We need to filter the results to only include the driver with a Q3 time of 0:01:33. This is done using the 'q3' column in the 'qualifying' table, with the condition 'q3 LIKE 11:3%%"
**Select the abbreviated code**; We need to select the 'code' column from the 'drivers' table, which contains the abbreviated code of the drivers. Therefore, the final SQL query is </reasoning>
<answer>
"'sql
SELECT c2.c FROM qualifying AS T1 INNER JOIN drivers AS T2 ON T1.driverld T2.driverld WHERE ra.ael 45 AND T1.q3 LIKE '1:33%'
<lanswer>
18

---

Published as a conference paper at COLM 2025
A.3 Prompts
LLM-as-a-Judge Prompt
You are experienced database expert. Your task is to evaluate generated SQL query by comparing the ground truth (gold) query and then ussign score between 0.0 and 2.0. higher score indicates the predicted query is more correct, while score of 0.0 means is completely incorrect.
Follow these evaluation rules strictly:
SELECT Clause:
Only select columns that are mentioned in the user's 's question.
Do not include unnecessary columns or values.
Aggregation (MAX/MIN):
Always perform JOINs before applying MAX( or MIN).
3. ORDER BY with Distinct Values:
Use GROUP BY <column> before an ORDER BY <column> DECI to ensure distinct values.
Handling NULLs:
If column may contain NULL values (indicated by "None" in value examples oT explicitly mentioned), include JOIN or WHERE <column> IS NOT NULL clause.
5. FROM/JOIN Clauses:
Only include the tables essential answering the question.
6. Strictly Follow Hints:
Adhere to hints provided with the question.
7. Thorough Question Analysis:
Ensure all conditions and requirements mentioned in the question are ad- dressed.
8. DISTINCT Keyword:
Use SELECT DISTINCT when the question requires unique values (e-g., IDs, URLs) or when column statistics (Value Statics) indicate its necessity.
Column Selection:
Carefully analyze column descriptions and hints to choose the correct column when similar columns exist across tables.
10. String Concatenation:
Do not any string concatenation methods (e.g., I1) in the SELECT clause.
11. JOIN Preference:
Prefer using INNER over nested SELECT statements.
12. Date Processing:
Use STRFTIME() for any date manipulations (e.g.. STRFTIME("'N) (?XY', SOMETIME) to extract the year).
You are provided with the following inputs:
Question: [QUESTION]
Hint: (HINT)
Gold Query: (GOLD-QUERY]
Predicted Query: (PREDICTED QUERY)
Based on the above, return single numeric score between 0.0 and 2.0 that reflects how correct the predicted query is compared to the gold query. Respond with only the score and no additional explanation.
19

---

Published as a conference paper at COLM 2025
Database Admin Instructions
SELECT Clause:
Only select columns mentioned in the user's question.
Avoid unnecessary columns or values.
Aggregation (MAX/MIN):
Always perform JOINs before using MAX( or MINO.
ORDER with Distinct Values:
Use GROUP BY <column> before ORDER BY <column> DESI to ensure distinct values.
4. Handling NULLs:
If a column may contain NULL values (indicated by "None" explicitly tioned), include
WHERE <column> IS NOT NULL.
5. FROM/JOIN Clauses:
Include only essential tables for answering the question.
6. Strictly Follow Hints:
Adhere to all provided hints.
7. Thorough Question Analysis:
Address all conditions mentioned in the question.
8. DISTINCT Keyword:
Use SELECT DISTINCT when unique values (e.g., IDs, URLs) are needed.
9. Column Selection:
Analyze column descriptions and hints carefully to choose correctly when similar columns exist.
10. String Concatenation:
Never use || || or other concatenation in SELECT.
11. JOIN Preference:
Prioritize INNER JOIN over nested SELECT statements.
12. Date Processing:
Use STRFTIME() for date manipulations (e.g., STRFTIME (M ('XY', SOMETIME)).
20

---

Published as a conference paper at COLM 2025
SQL Generation Divide-and-Conquer Prompt (USED FOR STaR-SFT)
You are an experienced database expert. Now you need to generate a SQLite query given the database information, question and some additional information. The database structure is defined by table schemas (some columns provide additional column descriptions in the options).
Given the table schema information description and the Question, you will be given table creation statements and you need understand the database and columns.
You will be using method called "recursive divide-and-conge approach to SQL query generation from natural language."
Here is high-level description of the steps:
Divide (Decompose Sub-question with Pseudo SQL); The complex natural lan- guage question is recursively broken down into simpler sub-questions. Each sub- question targets specific piece of information or logic required for the final SQL query.
2. Conquer (Real SQL for sub-questions): For each sub-question (and the main ques- tion initially), a "pseudo-SQL" fragment is formulated. This pseudo-SQU represents the intended SQL logic but might have placeholders for answers to the decomposed sub-questions.
3. Combine (Reassemble): Once all sub-questions are resolved and their correspond- ing SQL fragments are generated, the process reverses. The SQL fragments recursively combined by replacing the placeholders in the pseudo-SQL with the actual generated SQL from the lower levels.
4. Final Output: This bottom-up assembly culminates in the complete and correct SQL query that answers the original complex question.
Database admin instructions: [DATABASE ADMIN INSTRUCTIONS PLACEHOLDER/
Now is the real question, following the instruction and examples, generate the SQLite Query with Recursive Divide-and-Conquer approach. Follow all steps from the strategy. When you get the final query, output the query string ONLY the format "* "'sql "' Make sure you only output one single query.
Table creation statements
SERABASE_SCHEMAS
Question
[QUESTION] (HINT)
Answer
Repeating the question and generating the SQL with Recursive Divide-and- Conquer.
21

---

Published as a conference paper at COLM 2025
SQL Reasoning Prompt (USED FOR GRPO)
You are an experienced database expert. Now you need to generate a SQL query given the database information, question and some additional information. The database structure is defined by the following table schemas (comments efter '-' provide additional column descriptions).
Instructions:
Note that the "Example Values" are actual values from the column. Some columns might contain the values that are directly related the question. Use this information to justify which columns to use.
Given the table schema information description and the Question, you will be given table creation statements and you need to understand the database and columns to generate single SQLite query that can answer the user's 's question.
Database admin instructions: (DATABASE ADMIN INSTRUCTIONS PLACEHOLDERI
[Table creation statements]
CATTABASESCHEEMES
Now is the real question, following the instruction and examples, generate the SQL query.
Question:
(QUESTION] Hint: (HINT)
Respond the following format:
<reasoning>
Your detailed and step-by-step thinking path toward finding the correct SQL query
</reasoning>
<answer>
" sqql
Your predicted SQL query
</answer>
Now is your turn to respond in the above format.
SQL Generation Supervised Fine-tuning Prompt (USED FOR SFT)
You are experienced database expert. Now you need to generate SQL query given the database information, question and some additional information. The database siructure defined by the following table schemas (comments after provide additional column descriptions).
Instructions:
Note that the "Example Values" are actual values from the column. Some columns might contain values directly related to the question. Use this information to justify which columns to use.
Given the table schema information description and the Question, will be provided table creation statements. You need to understand the database and columns generate single SQLite query that answers the user's 's question.
Database admin instructions: (DATABASE ADMIN INSTRUCTIONS PLACEHOLDER
[Question]
QUESTION] (HINT))
Respond strictly in the following format:
"c "sql
Your predicted SQL query
22

---

Published as a conference paper at COLM 2025
In-context learning example for Reasoning-SQU format
Table creation statements:
CREATE TABLE Product
ProductID INTEGER primary key autoincrement, StandardCost REAL not null, -- Example Values: '00.0)'
Name TEXT not null unique, -- Example Values: '('AWC Logo Cap'
CREATE TABLE ProductCostHistory
foreign key (ProductID) references Product(ProductID)) StandardCost REAL not null, Example Values: '(12.0278,) ProductID INTEGER not null, Example Values: *(707))* StartDate DATE not null, -- primary key (ProductID, StartDate),
Question:
List the products whereby the standard cost is $80 more than previous standard cost in history. Hint: ABSTRACCTprotuttSannddCCCC. CostHis- (ory.StandardCost)>88
Reasoning:
To answer the question, we need to find products where the current stan- dard cost is $80 more than their previous standard cost in history. Here's a step-by-step breakdown of how to derive the SQL query:
1. Identify Relevant Tables and Columns: We need data from the Product table to get the product names and their current standard costs. We need data from the ProductCostHisce
table to get the historical standard costs. Determine the Join Condition: Join the Product table with ProductCostHistory on the ProductID colurn.
2.
3. Filter Based on the Condition: Include only those products where the current standard cost is $80 more than the previous standard cost.
4. Select the Required Columns: Select Name column from Product.
5. Ensure Distinct Values: Ensure distinct values through the join condi-
tion.
Answer:
"'sql
SELECT DISTINCT P.Name FROM Product P INNER JOIN Product- CostHistory CH ON P.ProductID CH.ProductID WHERE PSStadardCost CH.StandardCost 80;
23

---

Published as a conference paper at COLM 2025
In-context learning example for Query-prompt format
Table creation statements:
CREATE TABLE Product
ProductID INTEGER primary key autoincrement, StandardCost REAL not null, -- Example Values: '(0.0))
Name TEXT not null unique, -- Example Values: '('AWC Logo Cap'
CREATE TABLE ProductCostHistory
foreign key (ProductID) references Product(ProductID) StandardCost REAL not null, Example Values: '(12.0278,) ProductID INTEGER not null, Example Values: *(707))* StartDate DATE not null, -- primary key (ProductID, StartDate),
Question:
List the products whereby the standard cost is $80 than previ- ous standard cost history. Hint: SpBTRACT StatTarrrrruccttSttaaarrddss, Sectiistor.SStadaaddoooo
StandardCost,
Query Plan Guided SQL Generation:
Step-by-Step Reasoning:
1. Initialization:
Open tables: Product, ProductCostHistory.
2. Columns Needed:
Product: StandardCost (current), Name.
ProductCostHissoryy StandardCost (historical), ProductID.
3. Table JOIN Operation:
INNER JOIN Product.ProductID ProducttCsstiisttrry.Pooodc
4. Filter Condition:
Select products where current StandardCost exceeds historical StandardCost by more than $80:
Product. Standardostt ProductCosttiittrry.Sttadddddddd >
5. Select Final Column:
DISTINCT product names (Product. Name).
Final Optimized SQL Query:
a "'sql
SELECT DISTINCT P.Name FROM Product INNER JOIN Product-
Costoistory CH ON P.ProductID CH.ProductID WHERE PStandardCost
Staddannadd 80;
24

---

Published as a conference paper at COLM 2025
In-context learning example for Divide-and-conquer format
Table creation statements:
CREATE TABLE Product
ProductID INTEGER primary key autoincrement, StandardCost REAL not null, -- Example Values: '(0.0.)
Name TEXT not null unique, -- Example Values: '('AWC Logo Cap',
CREATE TABLE ProductCostHistory
foreign key (ProductID) references Product(ProductID) StandardCost REAL not null, Example Values: '(12.0278,) ProductID INTEGER not null, Example Values: *(707))* StartDate DATE not null, -- primary key (ProductID, StartDate),
Question:
List the products whereby the standard cost is $80 than previ- ous standard cost in history. Hint: SUBTRACT (rtnrrrrdcttttttadrrdssssss,
CostHistorr.SttadardCoo StandardCost))80
Divide and Conquer:
Main Question Analysis:
Identify products whose current standard cost is at least $80 greater than histor- ical costs.
Join tables Product and ProductCostHistoonn ProductID.
Pseudo SQL: SELECT DISTINCT Product. Name FROM Product INNER JOIN
ProductCostHistory ON Product.ProductID ProductoosstiissoryyPooooo Prodott ProductID WHERE <condition>;
2. Sub Sub-question (StandardCost difference):
Condition: Product. StandardCost CroduccCosttisstooystttaaddddddd 80
Assemble SQL:
SQL condition: Product. StandardCost pooducttosttiistory.ttaaddde 80
Complete SQL query:
Simplification and Optimization:
Query is straightforward and optimized.
Final SQL Query:
# "ssll
SELECT DISTINCT P.Name FROM Product P INNER JOIN Product- CostHistory CH ON P.ProductID- CH.ProductID WHERE PStandardCost
CH.StandardCost > 80;
25

---

Published as a conference paper at COLM 2025
In-context learning example for ACT-SQL format
Table creation statements:
CREATE TABLE Product
ProductID INTEGER primary key autoincrement, StandardCost REAL not null, -- Example Values: '(0.0.)
Name TEXT not null unique, -- Example Values: '('AWC Logo Cap'
CREATE TABLE ProductCostHistory
foreign key (ProductID) references Product(ProductID) StandardCost REAL not null, Example Values: '(12.0278,) ProductID INTEGER not null, Example Values: '(707,)', StartDate DATE not null, -- primary key (ProductID, StartDate),
Question:
List the products whereby the standard cost is $80 than previ- ous standard cost history. Hint: Sp StandardCost, (productt.SanddarCCsssss CostHistorr.Staaddrddosss StandardCost))00
Query Plan Guided SQL Generation:
Reasoning:
According to "list the products", columns [Product Name] may be used.
According "standard cost is $80 more than previous standard cost", columns
[ StandardCost, ProductCostHistory.. www.ctoottttttoyySSttnaddttsss may be used.
Tables [Product, ProductCosttiistory) will be joined using [ProductID].
Condition [(Product. StandardCott ProductCosthisttry. StandardCost) > 80] must be applied.
Final Answer:
"'sql
SELECT DISTINCT P.Name FROM Product AS P INNER JOIN Pro- ductCostHiistory AS CH ON P.ProductID CH.ProdtcIID WHERE (PStandardCost- CH.StandardCost) > 80;
26

---

Published as a conference paper at COLM 2025
A.4 Test-time Compute Analysis
Scaling test-time compute (Snell et al., 2024) involves allocating additional computational resources during inference through methods such as Chain-of-Thought prompting (Wei et al., 2022), the Best-of-N approach (generating N candidate answers and selecting the best one either programmatically via test cases or using a verifier model) (Lightman al., 2023; Pourreza al., 2024; Gao et al., 2024; Lee et al., 2024), or by simply increasing the number of reasoning tokens (e.g,, adding "wait" tokens at the reasoning step) (Muennighoff et al., 2025). Among these methods, the Best-of-N approach is widely adopted in the Text-to-SQL domain (Pourreza et al., 2024; Gao et al., 2024; Lee et al., 2024). Two critical metrics influence the effectiveness of the Best-of-N method: the Pass@K performance, indicating the upper bound achievable by selecting from multiple candidates, and the average quality of the candidate pool (Average@K). To compare the performance of our GRPO-trained models against the standard SFT-trained models, we evaluate the Pass@K and Average@K metrics for the Swwe2.5-Coder-144 SFT model and our GRPO-trained Qwen2.5-Coder
model in Figure 6. To enhance candidate diversity, we increased the sampling temperature to 0.5, resulting in a slight performance drop for the GRPO-trained model (from 65.5% to 64.66%). However, the drop was more significant for the SFT-trained model (from 63.75% to 60.82%). Notably, although the SFT model demonstrates greater randomness, thus achieving slightly higher Pass@K, our GRPO-trained model maintains significantly higher average performance across various candidate sizes (K). These results indicate that the GRPO-trained model generates candidates of consistently higher quality, albeit with slightly reduced diversity.
0.750
Execution Accuracy
0.725
0.700
0.675
0.650
0.625
SFT PesSSK
GRPO Average@
0.500
SFT Averagg:@
10
Number candidates (K)
Figure 6: Pass@K and perrraceeeeooo for the GRPO and SFT trained models on bird development set.
A.5 Difficulty and Cost Analysis
In this section, we report the performance of our GRPO-trained models with different sizes, 3B and 7B, across the three difficulty levels of questions in the BIRD development set and compare with STaR-SFT trained reasoning models. Additionally, we analyze the length of the reasoning portion in the LLM's output for each difficulty level. As shown in Table 6, performance of our GRPO-trained models are consistently better than STaR-SFT models across all difficulty levels with much less reasoning characters. Furthermore, for all models, the number of characters in the reasoning section of the output grows as the question difficulty increases, indicating greater need for detailed reasoning in more complex queries. Furthermore, recognizing that the number of schema links required for query serves as an indicator of query complexity (Pourreza & Rafiei, 2024a), we conducted an additional analysis comparing the length of reasoning characters generated by models with the number of schema links. Our analysis revealed a positive correlation between these metrics, as illustrated in Figure
27

---

Published as a conference paper at COLM 2025
Table 6: Execution Accuracy (EX) of our model with CHASE-SQL pipeline and comparison with all methods on BIRD development set. "All" denotes the use all introduced rewards combined.
set.
EX (%)
Ressoning Chars Maderale
Model
Simple
Moderate
Challenging
Simple
Challenging
Qwen2. Seodee--B--STRR
20.27
45.56
2546
Swwen2.5rodee-7BB.p (i))
Qwen2. 5-oddrr-B-STaale
62.18
43.1
38,62
1579
2000
2313
Qwen2.5 Swee2.5-exder:33
67.78
46.76
38.62
1306
1606
1666
Numbe of Characters
2200
2000
2800
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0
Number of Schema Links (Columns)
Figure 7: Length of reasoning segments generated by our GRPO-trained model across queries requiring varying numbers of schema links.
A.5.1 Detailed Cost Analysis
In this section, we compare the cost of using our Qwen2.5-Coder-1455 model within the CHASE-SQL pipeline (Pourreza et al., 2024) against the original CHASE-SQL implementa- tion, which utilizes Gemini-1.5-pro-002 for all LLM calls in its pipeline. Figure 8 illustrates the average cost comparison per question across each database. The results indicate that our model achieves comparable performance (1% lower than Gemini-1.5-pro) lower cost, highlighting the significant contribution of our work in reducing the expenses associated with proprietary models for SQL generation. the Gemini-1.5-pr model, we estimated the token costs at approximately $1.25 per 1M input tokens and $5.00 per 1M output tokens. In contrast, our model's 'ss estimated costs are around $0.08 per 1M input tokens and $0.18 per 1M output tokens, contributing to this cost reduction.
93%
0.6
Methad Outpul
Our Method Total Price
Ouiput
CHASE "itel Price
wice
0.3
022
0.1
saperhoro
games
thronoois prstiction
oldenie sctasis
euapean fostball
audden Clob
codebaus costerurity
Barmula.x
abdet card specializing
fonrccal
Insicoingy
Databases
Figure 8: Cost analysis of using our Qwen2.5-coder-14B: (alt) vs using Gemini-1.5-pro- for the SQL generation with CHASE-SQL pipeline across different databases of the BIRD development set.
28

---

Published as a conference paper at COLM 2025
A.6 Training Dynamics: Execution Accuracy and Reasoning Efficiency
In this section, we provide detailed analysis of the model's execution accuracy and the corresponding number of reasoning characters generated during training. We conducted this analysis by evaluating the model's accuracy and reasoning length at intervals of 10 training steps, starting from 0 and continuing up to 200 steps. To perform an effective and efficient evaluation, we used a 10% sub-sample of the BIRD development set across these 20 evaluation points. Figure illustrates that as training progresses, the accuracy of the model generally increases while the average number of reasoning characters tends to decrease. This reduction is partly due to our prompt's requirement for the model to always provide its reasoning steps before providing the answer. Initially, at early stages of training, the model leverages its zero-shot chain-of-thought capabilities to produce extensive reasoning steps. However, training advances, the model increasingly standardizes and condenses its reasoning process, converging toward a more succinct and consistent format.
Acc May (itaal
Chars douel
My
Char
Twiskng Stope
Figure 9: Accuracy and number of reasoning characters the Qwen2.5-33 (att) model during the first 200 steps of the training.
A.7 Pseudo-Code
In this section, we present the high-level pseudo-code detailing our GRPO-based rein- forcement learning training algorithm 1 for Text-to-SQL. The pseudo-code outlines the generation of candidate SQL queries, the computation of a composite reward, including execution accuracy, LLM-based judge feedback with the judge prompt, schema linking, n-gram similarity, and format adherence, and the subsequent policy update via the GRPO objective. This detailed overview is provided to enhance reproducibilit, and to offer further clarity on the methodological aspects discussed in the main paper.
29

---

Published as a conference paper at COLM 2025
Algorithm 1 GRPO-bbased RL Training for Text-to-SQL
Input:
Policy model ne (initialized from pretrained LLM)
Reference model Tre (copy of the pretrained LLM)
Old policy model Teaaa (frozen snapshot of Te before update)
Training dataset D consisting of questiin-schemma pairs (q. and gold SQL queries
Number of candidates per example G
Hyperparameters: clipping threshold e, divergence regularizer B
Reward weights: Wexec, Woudge Wsyntax, Wschema/ Wngram, Wfrmmt
Output: Updated policy model
foreach fraining slep do
foreach mini-hatch of examples Sgold) C D do foreach example (g, S, gold) in the mini-batch do // Generate candidate SQL queries
Sample G candidates {o To g, S)
// Evaluate rewards for each candidate foreach candidate do Texec f- ((eee((0; = exec(gold)
if You = 0 then
fjudge + LLM-judge(0, gold)) E [0, 1]
end
else
judge f
end
Ysyntax + [syntactically va)id
rschema + Jaccard Jaccard (schema (o;), schema schemaggold) goll))
Ingram (accard(ngramss (o;), ngrams (gold)
Nfrrat [[; matches required format)
// Aggregate weighted rewards
ri 7 Wexecrexec + juduge + Diyyntax'synaa
+ Wschema + ('ngram/ngramt + WiormatFiormal
end
// Compute advantages foreach candidate 0; do , E
end
// GRPO policy update Compute loss:
min x)(oilg,5)
Te (Dilg, e, e,1+,) A1)
na(0;19, 5)
[GRPO(0) + E
AG clip
Te (0ilq,
kI. (Te [err
Update + (0-=vVo(-CRRO0000 // Periodically update old policy snapshot
end end
end
30


Reviews:
[{"review_id": "ZAr94hH8jk", "reviewer": "Reviewer_ChoJ", "summary": "Reasoning-SQL introduces an RL framework for text2sql, leveraging tailored partial rewards (execution accuracy, syntax, schema linking, n-gram similarity, and llm-as-a-judge) to overcome reward sparsity. Using GRPO, it trains LLMs to generate more accurate SQL in comparison with larger proprietary models on benchmarks like BIRD and Spider. Authors also argue Reasoning-SQL is more cost-effective.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This work presents a RL training paradigm with new reward design choices tailored specifically for SQL generation tasks. Unlike conventional RL approaches that primarily rely on execution accuracy, Reasoning-SQL introduces a composite reward and demonstrate promising results in this direction.", "reasons_to_reject": "1. complex reward functions may lead models to exploit unintended behavior. Even though the authors study how each reward by itself perform, it doesn't seem the authors have provided evidence on the justification of the particular reward combination choice (i.e. ablation on different combinations of rewards is lacking). \n2.  The paper introduces a composite reward system, the authors argue it's necessary to capture the multifaceted nature of SQL generation tasks. However, components like the llm-as-a-judge reward introduce uncertainty, which confilct with recent studies that advocate for simple, verifiable reward designs to enhance performance and mitigate reward hacking risks [1, 2]. Therefore, justification for using a composite reward should be rigorously evaluated against simpler alternatives. But this part of experiment is lacking.\n3. The evaluation primarily benchmarks Reasoning-SQL against proprietary models. However, it omits comparisons with recent open-source models such as XiYanSQL [3], OmniSQL [4] and Think2SQL [5]. \n4. no comparison of PPO and GRPO results on the reward design choice.\n5. Appendix A.4 indicates that Reasoning-SQL tends to produce less diverse SQL queries compared to the SFT baseline, with only a ~7% improvement from greedy sampling. This reduced diversity could pose challenges in applications requiring varied query formulations, potentially limiting the model's generalizability.\n\n[1] Mroueh, Youssef. \"Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification.\" arXiv preprint arXiv:2503.06639 (2025).\n[2] Meng, F., et al. \"Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.\" arXiv preprint arXiv:2503.07365 (2025).\n[3] Gao, Yingqi, et al. \"Xiyan-sql: A multi-generator ensemble framework for text-to-sql.\" arXiv preprint arXiv:2411.08599 (2024).\n[4] Li, Haoyang, et al. \"Omnisql: Synthesizing high-quality text-to-sql data at scale.\" arXiv preprint arXiv:2503.02240 (2025).\n[5] Papicchio, Simone, et al. \"Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQL.\" arXiv preprint arXiv:2504.15077 (2025).", "questions_to_authors": "See weekness.", "extra_scores": {}}, {"review_id": "t4hXE89W3b", "reviewer": "Reviewer_yRRV", "summary": "Inspired by DeepSeek R1's reinforcement learning (RL) training using GRPO, this work investigates the application of RL to the Text-to-SQL task. The authors developed a reasoning-enhanced Text2SQL system on top of Qwen2.5-Coder and demonstrated its effectiveness through both quantitative evaluations—such as performance on the in-domain BIRD benchmark and out-of-domain Spider-related datasets—and qualitative analysis, highlighting emergent reasoning capabilities. Additionally, by integrating their RL-trained language model with the state-of-the-art CHASE framework, they achieved performance competitive with leading closed-source models on the Text2SQL task.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. To further improve the performance, the authors designed multiple rewards, which are key innovations to make sure GRPO based RL training works for Text2SQL task. These rewards design can inspire others who also want to apply RL training to other tasks.\n2. Tthe final RL trained model shows competitve performance as other systems that rely on closed-source LLMs, the gap between open-source and closed-source is even smaller.", "reasons_to_reject": "1. Given this claim -- \"our RL-trained 14B-parameter model significantly outperforms larger proprietary models, e.g. o3-mini by 4% and Gemini-1.5-Pro-002 by 3% on the BIRD benchmark\", it is pretty misleading, based on Table 2, the base model (Qwen2.5-coder-14B) already outperforms o3-mini and Gemini-1.5-Pro-002. Their RL-trained model just shows further performance gains. However, the baseline setup for Qwen2.5-coder-14B is not clear, is it zero-shot or few-shot? if few-shot, static or dynamic exemplars? we should prepare solid baseline results to make sure readers really understand the original Text2SQL performance with Qwen2.5-coder-14B.\n2. Table 1 shows FS EX result, it doesn't make sense to show this result gold schema input, since one of the reward design is for schema linking, and this is critical for Text2SQL, we should avoid showing this cheat setting to highlight the good results numbers.\n3.  For Table 3, the paper only shows CHASE + their RL-trained Qwen2.5-coder, how about CHASE with original Qwen2.5-coder? if the performance is also comparable, does that mean their RL training cannot improve Qwen2.5-coder's upperbound performance for Text2SQL capability?", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SBwbu6pmg0", "reviewer": "Reviewer_sLu2", "summary": "The authors proposed a reinforcement learning (RL) based SQL generation system with large language model. The method utilizes a set of partial rewards, including, schema linking, AI feedback, n-gram and syntax check et al, to alleviate reward sparsity issues in RL.  Extensive experiments are conducted to evaluate the proposed method and it shows better results than the baselines and interesting emerging reasoning ability.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well written and easy to follow in general.  \n- Extensive experiments have been conducted to demonstrate the ability of the proposed method, such as ablation study, different model sizes, and different evaluation settings etc. \n- Good results are achieved", "reasons_to_reject": "- Most of partial rewards, such as AI feedback, scheme linking, are n-gram similarity require gold SQL answers.  It limits the usage of the proposed method.", "questions_to_authors": "Most partial rewards are based on gold SQL answers, which is also used in SFT baseline. Will the SFT baseline be helpful for the RL learning? For example, can SFT baseline is used as initialization or used as another reward?\n\nTypo:\nline 250 ``he weights --> The weights''", "extra_scores": {}}, {"review_id": "tFAS34c62z", "reviewer": "Reviewer_KUeS", "summary": "In this paper, the authors have proposed a training pipeline for reasoning-based text-to-SQL tasks. The authors have proposed several different rule-based rewards to improve performance of reinforcement learning. The experiments are extensive and show solid improvements compared with the baseline systems.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The writing is easy to follow.\n2. This study has covered some implementation details and experience that could be helpful to the community for further research, especially to the text-to-SQL research.\n3. It validates the success and generalization of rule-based reinforcement learning.\n4. [After Rebuttal] The updated experiments show improvements compared with llm-as-judge baseline built on the SFT policy.", "reasons_to_reject": "1. Each single item of the rule-based rewards (i.e., syntax, schema linking, n-gram similarity, format) demonstrate marginal improvements. Considering the less stableness of reinforcement learning, I have some concern of whether these rewards can make some true contributions to the final performance.\n2. Besides, by comparing the baselines using rule-based rewards with the one using only llm-as-judge, can we say that llm-as-judge can already bypass the hand-crafted reward design? Again, the improvements by applying rule-based rewards to llm-as-judge based model is somewhat marginal.", "questions_to_authors": "When prompting LLMs, will the schema information of the underlying database also provided to the model? From the prompt examples, I didn't see such information.", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors present a RL framework for text2SQL generation that employs a composite reward system (using execution accuracy, syntax checking, etc.) to address reward sparsity. The authors show competitive performance against larger models and also show emergent reasoning capabilities while also being cost sensitive.

Reviewers had initially raised concerns about (i) the justification for the composite reward design (ii) misleading performance claims and experimental setup clarity (iii) requirement of gold answers (iv) marginal reward improvement and training stability — but the authors were able to satisfactorily address a large number of concerns.

Overall, the paper makes good technical contributions with the composite design and empirical results demonstrating incremental progress with practical applications.
Decision: Accept