Title: SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: NL2SQL

Reinforcement Learning

Synthetic Data

Model Reasoning
Abstract: Natural Language to SQL (NL2SQL) enables intuitive interactions with databases by transforming natural language queries into structured SQL statements.  Despite recent advancements in enhancing human-computer interaction within database applications, significant challenges persist, particularly regarding the inference performance in complex scenarios involving multi-table joins and nested queries.   Current methodologies primarily utilize supervised fine-tuning (SFT) to train the NL2SQL model, which may limit adaptability and interpretability in new environments (e.g., finance and healthcare).
In order to enhance the reasoning performance of the NL2SQL model in the above complex situations, we introduce SQL-R1, a novel NL2SQL reasoning model trained by the reinforcement learning (RL) algorithms.
We design a specialized RL-based reward function tailored for NL2SQL tasks and discussed the impact of cold start and synthetic data on the effectiveness of intensive training. In addition, we achieve competitive accuracy using only a tiny amount of synthetic NL2SQL data for augmented training and further explore data engineering for RL. In existing experiments, SQL-R1 achieves execution accuracy of 88.6\% and 67.1\% on the benchmark Spider and BIRD, respectively. The code is available at https://github.com/IDEA-FinAI/SQL-R1.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SQL-R1, a novel approach for Natural Language to SQL (NL2SQL) that leverages reinforcement learning (RL) fine-tuning of large language models (LLMs). Motivated by the success of RL-based training in improving reasoning capabilities of LLMs, the authors employ Group Relative Policy Optimization (GRPO) to enhance SQL generation performance. The work also addresses several key research questions such as the optimal cold start strategy for RL fine-tuning in the NL2SQL domain, and the characteristics and requirements of high-quality training data for learning effective NL2SQL models. The authors demonstrate that SQL-R1 achieves state-of-the-art performance on two standard benchmarks, Spider and BIRD.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n**Timely Application of RL Fine-tuning to NL2SQL.** The application of reinforcement learning fine-tuning to NL2SQL is well-motivated given the recent success of RL methods in improving LLM reasoning capabilities. Since NL2SQL requires complex reasoning to generate accurate SQL statements, this approach represents a natural and promising direction for the field.\n\n**Comprehensive Investigation of Key Research Questions.** Beyond proposing the core method, the authors systematically investigate important practical questions, including the necessity of supervised fine-tuning as a prerequisite (cold start problem) and the requirement of high-quality training data. These investigations provide valuable insights that advance understanding of RL fine-tuning for structured generation tasks.\n\n**Thorough Experimental Evaluation.** The paper includes comprehensive comparisons against diverse NL2SQL baselines across multiple datasets and model scales. The ablation studies on reward design components are particularly valuable for understanding which aspects of the approach contribute most to performance gains.\n\n\n## Weaknesses\n\n**Limited Performance Gains.** While the method achieves competitive results, the improvements over existing state-of-the-art methods are modest and inconsistent. For instance, on Spider test set with Qwen2.5-Coder-7B, SQL-R1 achieves 88.7% versus OmniSQL's 88.9%. The performance gaps vary across different experimental settings, raising questions about the practical significance of the approach.\n\n**Insufficient Baseline Analysis.** The experimental section compares against numerous prior methods, but the paper lacks adequate explanation of these baselines in the related work or methodology sections. Readers would benefit from understanding the key differences between SQL-R1 and these comparative methods, particularly how they differ in their approach to the NL2SQL problem.\n\nQUESTIONS:\n**Missing Competitive Baselines in Figure 2.** Figure 2 appears to omit several recent competitive methods that could significantly impact the performance comparison. Notable missing baselines include SQL-o1 and Alpha-SQL, among others. This omission may present an incomplete picture of the current state-of-the-art and could mislead readers about the relative performance of SQL-R1. Could the authors include these methods in their comparison or provide justification for their exclusion?\n\n**Selective Dataset Analysis.** In lines 250-251, the authors discuss the effectiveness of selective datasets for RL fine-tuning by referencing base model results which are not in the main paper. Can you add the guidance where the reader check for the base model results? Additionally, to show the effectiveness of the selective dataset, it would be better to compare the SQL-R1 trained from the selective dataset and default dataset. Can you test this also?\n\n**Performance Analysis by SQL Complexity.** Given that RL fine-tuning with specialized rewards should theoretically be more beneficial for complex reasoning tasks, it would be valuable to see a breakdown of performance across different SQL complexity levels (Easy/Medium/Hard). This analysis would help readers understand when and why the proposed method provides the most benefit. Can you provide these level-wise performance comparisons?\n\n**Relationship to Recent RL Fine-tuning Effectiveness Debates.** Recent work [1] has raised important questions about the effectiveness of RL fine-tuning, demonstrating that reasoning-optimized models can underperform on simple tasks while showing improvements on medium-complexity problems when given additional computational budget. Given these findings, how does SQL-R1's performance align with this complexity-dependent effectiveness pattern? Do the authors observe similar patterns in SQL-R1's results? More broadly, can the authors provide analysis and discussion of their method through this lens, including whether SQL-R1 exhibits the complexity-dependent benefits and limitations observed in recent RL fine-tuning research?\n\n[1] https://machinelearning.apple.com/research/illusion-of-thinking\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInspired by the success of the RL training approach on language models, SQL-R1 uses GRPO to train open-source language models (Qwen-2.5 family) on the Text-to-SQL task. They introduce four rewards for the training, including format, executability, execution accuracy, and length reward, that, in combination, guide the RL training process. This work shows that an open-source model trained with RL can outperform the SFT-trained counterpart, reaching a comparable performance to some larger proprietary models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nS1) The paper is well-presented, easy to follow, and provides clear explanations of methodologies and findings.\n\nS2) The experimental details are clearly explained and well-documented, helping with the reproducibility of the results.\n\nS3) The authors have shown extensive experimental results across multiple model sizes, using two well-established and rigorous benchmarks for Text-to-SQL (Spider and BIRD)\n\nS4) Ablation studies are presented for the training data, model size, and inference-budget, providing insights into the individual impact of each component on the overall performance.\n\nWeaknesses:\n\nW1) Novelty: The primary issue with this work is the lack of novelty. As cited by the paper itself, Reasoning-SQL has a very similar experimental setting and has already validated most of the claims made in this work. Specifically, Reasoning-SQL previously adopted GRPO for Text-to-SQL and employed rewards very similar to the format, executability, and execution accuracy rewards used here. The length reward introduced in this work is also a known concept from existing RL literature.\n\n\nW2) Lack of cost analysis: Although this work leverages inference-time scaling (extended reasoning and parallel sampling) to enhance performance, the accuracy/cost trade-offs introduced by scaling are not discussed. Moreover, the paper lacks cost comparison with existing methods, an important aspect for practical deployment in Text-to-SQL systems.\n\n\nW3) BIRD's test set performance report: Reporting the performance on the private BIRD's test set is a conventional standard for fair comparison in the Text-to-SQL domain. Without the report on the BIRD test set or the confidence intervals of the reported results, it would be hard to assess the results against existing methods.\n\nQUESTIONS:\nQ1) Figure 9 shows the performance of the 7B model reaching around 63% using self-consistency on the BIRD dev set for 8 SQL candidates, whereas Table 1 reports this as 66.6%. Could the authors clarify this discrepancy?\n\nQ2) The results reported in Reasoning-SQL represent the zero-shot performance of the model without parallel sampling and self-consistency. However, Table 1 indicates the use of self-consistency, differing from the original report. Did the authors reproduce the results from Reasoning-SQL and other comparable works? It would be helpful to add a column reporting the sampling budget of each method for a clearer comparison.\n\nQ3) How were the weights for the partial rewards determined? Specifically, considering the execution score computation depends on the format reward, how was the training conducted for the \"w/o Sf\" scenario in Table 3? Surprisingly, the Sr reward, which directly measures performance on the downstream task, seems to have the least impact. Could you elaborate on why this might be the case?\n\nQ4) Moreover, I am interested in understanding the impact of the length reward on model performance. How does this reward specifically guide the model toward generating better SQL queries? Could you provide qualitative examples comparing the reasoning processes of models trained with and without this reward?\n\nLIMITATIONS:\nThe main limitation of the work is its lack of novelty, and I would appreciate it if the authors could elaborate on that.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SQL-R1, a novel NL2SQL model trained with reinforcement learning to improve reasoning and generalization in complex SQL generation tasks. By leveraging the GRPO algorithm and a structured reward function incorporating format, execution, result correctness, and reasoning trace quality, SQL-R1 produces both accurate and interpretable SQL outputs. The model is trained on synthetic data (SynSQL-2.5M) with a cold-start SFT phase, followed by RL fine-tuning. Experimental results on Spider and BIRD benchmarks show that SQL-R1 achieves competitive performance with closed-source models like GPT-4 while offering improved transparency and efficiency, especially on smaller base models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Clear Motivation for RL in NL2SQL: The paper is well-motivated by the limitations of SFT, especially in generalization and interpretability, - which are crucial in high-stakes domains like healthcare and finance.\n- Custom Reward Design: The reward function, combining execution success, result correctness, formatting, and length penalties/rewards, is thoughtful and tailored to the NL2SQL task.\n- Strong Benchmarking: SQL-R1 is evaluated across multiple model sizes and datasets (Spider, BIRD), showing competitive results against state-of-the-art systems, including GPT-4 and Gemini, especially considering it uses open-source LLMs.\n\nWeaknesses\n- Marginal Algorithmic Novelty: While the application of GRPO to NL2SQL is novel, the reinforcement learning framework itself is reused from prior work (e.g., DeepSeek-R1) with limited modification. The paper would benefit from a clearer discussion of what is novel versus reused.\n- Limited Analysis of Errors, Generalization and Reward shaping: The paper does not sufficiently analyze where the model fails (e.g., complex joins, ambiguous queries), nor does it test generalization on other tasks. Ablations are done with reward shaping techniques, but  not enough analysis is given, i.e. how sensitive is the reward design. No error breakdown on failure cases, i.e. harder SQL generation improvement is bigger etc?\n\nQUESTIONS:\n1.Spider dataset is an old dataset which could have contamination in pretrained models, why don't you use the Spider-2 dataset?\n2. In the abstract, it's claimed that generalization is better with RL training, where can we tell there is generalization improvement?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

comments: SUMMARY:
The paper proposes SQL-R1, an open-source NL2SQL system that fine-tunes a family of Qwen2.5-Coder models with GRPO. The main idea is to move beyond supervised fine-tuning by designing an RL reward that combines format, executability, result correctness and answer length. With a 7 B backbone, SQL-R1 reaches 88.7 % execution accuracy on Spider-Test and 66.6% on BIRD, outperforming other open LLMs and approaching closed models such as GPT-4o.

STRENGTHS AND WEAKNESSES:
### Strengths
1. This paper is among the first attempts to adopt GRPO for Text2SQL and has a clear layered reward definition that are well-motivated.
2. Large margins over prior open models on Spider and solid BIRD gains, achieved with modest RL data.
3. The results demonstrate that a 7B model trained with RL can match or beat much larger supervised models.
4. Good ablation study. Also the examples are clear in the paper, with interpretability in the reasoning process.

### Weaknesses
1. Experiments only cover SQLite; claim of cross-dialect robustness is untested — can authors provide PostgreSQL / MySQL results? Datasets like [Spider 2.0](https://spider2-sql.github.io/) might be a good add.
2. Eight-candidate self-consistency improves accuracy but may raise inference latency. Concrete numbers would help.
3. Spider and BIRD are human-curated, but training is wholly synthetic; real-world noise may hurt the performance. I'd like to see more discussion on robustness and transfer to very different domains, or multi-table queries.
4. The paper's novelty is not very significant, as it applies GRPO to a specific task.

QUESTIONS:
How much slower is the self-consistency approach used to get the performance number? Is it the same approach across baselines?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03