Optimizing Reasoning for Text-to- SQL with Execution Feedback (2025)

Abstract: Text-to-SQL demands precise reasoning to convert natural language questions into structured queries. While large language models (LLMs) excel in many reasoning tasks, their ability to leverage Chain-of-Thought (CoT) reasoning for text-to-SQL remains underexplored. We identify critical limitations: zero-shot CoT offers minimal gains, and Direct Preference Optimization (DPO) applied without CoT yields marginal improvements. We propose ExCoT-DPO, a novel framework that iteratively optimizes open-source LLMs by combining CoT reasoning with off-policy and on-policy DPO, relying solely on execution accuracy as feedback. This approach eliminates the need for reward models or human-annotated preferences. Our experimental results demonstrate significant performance gains: ExCoT-DPO improves execution accuracy on BIRD from 57.37% to 68.51% and on Spider from 78.81% to 86.59% for LLaMA-3 70B, with Qwen-2.5-Coder demonstrating similar improvements. Our best model achieves state-of-the-art performance in the single-model setting on both BIRD and Spider datasets.

## introduction text-to-sql, the task of translating natural language questions into accurate sql queries @xcite , requires both domain knowledge and precise reasoning about query structures. this task is particularly challenging due to its reliance on schema linking (matching natural language entities to database columns), handling compositional syntax (e.g., nested queries, joins), and resolving ambiguities in user intent. in recent years, large language models (llms) have demonstrated remarkable capabilities in generating coherent, contextually rich responses with reasoning across a 1 our code is released at arctic training and trained models are released at huggingface: qwen-2.5-coder-arctic-excot-32b and llama-3.1-arctic-excot-70b range of domains @xcite @xcite . these models are often trained with preference learning @xcite to promote the quality of their reasoning and accuracy on downstream tasks. however, it remains an open question how these llms can effectively leverage step-by-step reasoning, also referred to as chain-of-thought (cot, @xcite , to improve performance in text-to-sql, where errors in intermediate logic (e.g., misaligned joins or aggregation) often propagate irreversibly to final sql outputs. prior work in text-to-sql has explored specialized decoding strategies like execution-guided parsing @xcite or constrained inference @xcite , but these methods typically focus on syntax correction rather than improving the model's intrinsic reasoning. meanwhile, preference optimization techniques like dpo have shown promise in aligning model outputs with human preferences in dialogue and summarization @xcite , yet their applicability to structured tasks like text-to-sql remains understudied. our investigations reveal three key observations that clarify the role of cot in this setting: (1) zeroshot cot -without any dedicated training -does not provides improvement over baseline methods in text-to-sql tasks, likely because unstructured reasoning steps fail to address schema-specific dependencies or syntactic constraints; (2) directly applying direct preference optimization (dpo) to text-to-sql problem without incorporating a cot does not substantially boost performance, as the absence of intermediate reasoning steps limits the feedback signal's ability to pinpoint logical errors; and (3) when cot is integrated, even a simple execution-based feedback signal can significantly enhance the model's reasoning through dpo. we posit that the presence of intermediate reasoning steps in cot exposes errors in logic or structure, thereby enabling the dpo process to correct those errors more effectively. building on these observations, we propose a novel framework, namely execution-guided chain-of-thought direct preference optimization (excot-dpo), as illustrated in figure 1 . excot-dpo aligns open-source llms to text-to-sql tasks by combining cot with off-policy and onpolicy iterative dpo, using only execution accuracy as a feedback mechanism. crucially, this approach does not rely on a specialized reward model or human-annotated preference data; instead, it leverages the model's own reasoning traces and downstream execution outcomes for iterative improvement. notably, a similar conclusion in the domain of math and coding has also been reached by a concurrent research @xcite ) that the answer correctness alone can be an effective signal. our experiments demonstrate that the introduction of cot into a dpo training loop allows the model to refine its reasoning steps, ultimately leading to more accurate sql queries. in this paper, we make the following contributions: (1) we demonstrate that zero-shot cot does not offer gains in text-to-sql, highlighting the necessity of explicit training and feedback mechanisms. meanwhile, we show that applying dpo without cot yields only marginal benefits, underscoring the importance of intermediate reasoning paths for effective alignment. (2) we introduce a simple yet powerful alignment strategy based solely on execution feedback (i.e., whether a generated query is correct), demonstrating that detailed cot reasoning provides sufficient internal signals for the model to self-correct. (3) we release models that achieve state-of-the-art performance in the single-model text-to-sql setting. our contributions include both the llama-3.1 70b-based and qwen-2.5-coder 32b-based models, with our best model reaching 68.51% and 68.53% execution accuracy on the challenging bird benchmark dev set and test set, respectively. our work has broader implications for aligning llms to structured output tasks. by showing that execution feedback and self-generated reasoning traces suffice for iterative improvement, we reduce reliance on costly human annotations or reward models. this approach could generalize to other domains requiring precise structured generation, such as code synthesis or logical reasoning, where intermediate steps naturally admit validation (e.g., unit tests or formal verification). ## related work the text-to-sql task focuses on converting natural language queries into sql commands to enable easy interaction with structured databases. early work in this domain, such as spider @xcite , established the complexity of the task by introducing benchmarks for cross-domain generalization and complex queries. recent approaches to text-to-sql can be broadly categorized into finetuning-based and prompting-based methods. fine-tuning-based methods fine-tuning pretrained language models has been a dominant strategy in text-to-sql research. rat-sql @xcite introduced a relation-aware schema encoding mechanism to enhance schema linking and query comprehension. picard @xcite improved sql query generation with constrained decoding, ensuring syntactic validity during generation. resdsql @xcite decoupled schema linking and skeleton parsing for better modularity and adaptability. furthermore, codes @xcite contributed open-source models that emphasized transparency and reproducibility in fine-tuned text-to-sql systems. prompting-based methods prompting large language models (llms) has gained attention due to its flexibility and minimal reliance on fine-tuning. act-sql @xcite applied chain-ofthought reasoning in prompts to handle complex queries. @xcite evaluated llms on text-to-sql tasks in zero-shot and few-shot settings, highlighting their effectiveness in reducing task-specific training. din-sql @xcite used a decomposed in-context learning approach, leveraging sub-query decomposition and self-correction. similarly, chess @xcite proposed contextual harnessing to streamline sql synthesis. chase-sql @xcite employed multi-agent modeling with divide-and-conquer, chain-of-thought reasoning, and instance-aware example generation. chase-sql utilizes multiple prompts to generate sql queries and refines them through a selection mechanism, optimizing for higher accuracy by selecting the best candidate from a set of generated sql queries. other works have tackled key challenges like schema linking, ambiguity resolution, and confidence calibration. semantic matching in crush4sql @xcite addressed schema hallucination for improved linking accuracy. reflexion @xcite incorporated verbal reinforcement learning to iteratively refine sql outputs. calibration techniques, such as ramachandran and sarawagi (2024), utilized model probabilities for better confidence scoring, while @xcite explored strategies for eliciting calibrated confidence directly from llms. xiyan-sql @xcite introduces a multi-generator ensemble framework that builds on similar concepts from chase-sql. it combines multiple sql generators and refines the output through an ensemble strategy, improving both the diversity and accuracy of generated sql queries. xiyan-sql uses a refiner to correct errors in the generated queries and a selection model to choose the best candidate from multiple generated sql queries. the framework also integrates in-context learning and supervised fine-tuning to further enhance the quality of the sql generation. different from these works, excot-dpo unifies cot and direct preference optimization using only execution accuracy, eliminating the need for reward models or human annotation. this enables iterative, self-guided improvement of open-source llms, bridging cot reasoning and preference alignment in text-to-sql. while our work demonstrates strong single-model performance, excot-dpo is orthogonal to prompting-based frameworks and can be combined with those. ## conclusion and future work in this paper, we show that combining chain-ofthought (cot) reasoning with direct preference optimization (dpo) can substantially boost text-to-sql performance. our excot-dpo framework relies solely on execution accuracy to iteratively refine open-source llms, eliminating the need for reward models or human annotations. the resulting gains on both bird and spider demonstrate that aligning cot with dpo offers a powerful path toward more accurate and self-guided text-to-sql generation. combined with sft, excot-dpo improves execution accuracy on bird from 57.37% to 68.51% and on spider from 78.81% to 86.59% for llama-3.1 70b while also proven to be effective on another llm, qwen-2.5-coder. concurrent works (deepseek-ai, 2025; kimi-ai, 2025) show that a free-style cot process with the simple execution feedback can significantly improve model's performance tasks. we constraint our cot template in this work, and we leave this free-style format as our feature work. also, this work only applies offline-dpo to enhance model's text-to-sql capability. more advanced approach, like online-dpo @xcite , ppo @xcite , and grpo @xcite , are left for feature exploration.