Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Authors: Boyan Li
Jiayi Zhang
Ju Fan
Yanwei Xu
Chong Chen
Nan Tang
Yuyu Luo
Venue: ICML 2025 poster (2025)
Keywords: Text-to-SQL
Databases
LLMs
Primary area: applications->everything_else

Abstract: Text-to-SQL, which enables natural language interaction with databases, serves as a pivotal method across diverse industries.
With new, more powerful large language models (LLMs) emerging every few months, fine-tuning has become incredibly costly, labor-intensive, and error-prone. As an alternative, *zero-shot* Text-to-SQL, which leverages the growing knowledge and reasoning capabilities encoded in LLMs without task-specific fine-tuning, presents a promising and more challenging direction.
To address this challenge, we propose Alpha-SQL, a novel approach that leverages a Monte Carlo Tree Search (MCTS) framework to iteratively infer SQL construction actions based on partial reasoning states. To enhance the framework’s reasoning capabilities, we introduce *LLM-as-Action-Model* to dynamically generate SQL construction *actions* during the MCTS process, steering the search toward more promising SQL queries. Moreover, Alpha-SQL employs a self-supervised reward function to evaluate the quality of candidate SQL queries, ensuring more accurate and efficient query generation. Experimental results show that Alpha-SQL achieves 69.7% execution accuracy on the BIRD development set, using a 32B open-source LLM without fine-tuning. Alpha-SQL outperforms the best previous zero-shot approach based on GPT-4o by 2.5% on the BIRD development set.

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Boyan Li
Jiayi Zhang'
Ju Fan?
Yanwei Xu*
Chong Chen
Nan Tang' Yuyu Luo"
Abstract
and enabling both lay and expert users to derive insights ef- fectively (Liu et al., 2024; Luo al., 2018a;;; 2022b; 2021; 2022a; 2025; Liu et al., 2025b; Luo et al., 2023). With the advancement of large language models (LLMs), methods like CHASE-SQL, (Pourreza et al., 2025) have achieved competitive results on benchmarks such as BIRD (Li et al., 2023c). Generally, these LLM-based Text-to-SQL meth- ods fall into two categories; trained methods and zero-shot methods.
Text-to-SQL, which enables natural language interaction with databases, serves as pivotal method across diverse industries. With new, more powerful large language models (LLMs) emerg- ing every few months, fine-tuning has become incredibly costly, labor-intensive, and error-prone. As alternative, zero-shot Text-to-SQL, which leverages the growing knowledge and reason- ing capabilities encoded in LLMs without task- specific fine-tuning, presents a promising and more challenging direction. To address this chal- lenge, we propose Alpha-SQI., novel approach that leverages Monte Carlo Tree Search (MCTS) framework iteratively infer SQL construction actions based on partial reasoning states. To enhance the framework's 's reasoning capabilities, we introduce AMM-as-Action-Modee to dynami- cally generate SQL construction actions during the MCTS process, steering the search toward more promising SQL queries. Moreover, Alpha- SQL. employs self-supervised reward function to evaluate the quality of candidate SQL queries, ensuring more accurate and efficient query gener- ation. Experimental results show that Alpha-SQL achieves 69.7% execution accuracy on the BIRD development set, using a 32B open-source LLM without fine-tuning. Alpha-SQL. outperforms the best previous zero-shot approach based on GPT- 4o by 2.5% the BIRD development set. The code is available at https: : / /tt///ggttttt github.ccmm/ EKUSTDial/Alpha-Se
Training LLMs for Text-to-SLL. Pre-training fine- tuning LLMs on task-specific datasets is a common ap- proach to improving Text-to-SQL. performance (Li et al., 2024b; Gao al., 2024b; Talaei et al., 2024; et al., 2024a; Zhu al., 2025). While effective. this method requires extensive labeled datasets and significant computational re- sources for model training (Lin et al., 2025). Moreover, as newer and more powerful L.MSs emerge, the training pro- cess must be repeated to maintain competitive performance, further increasing the cost and effort (Wu et al., 2025).
Zero-Shot L.Mss for Text-to-SQI. As an alternative, zero- shot Text-to-SQL methods, such DAIL-SQL (Gao et al., 2024a) and C3 (Dong al., 2023), leverage the general knowledge encoded in LLMs to generate SQL queries with- out requiring task-specific fine-tuning, which eliminates the dependence on labeled datasets and computationally inten- sive training. While this approach offers practical and cost-effective solution, it faces a fundamental challenge.
The key challenge in zero-shot Text-to-SQL the difficulty of transferring and generalizing knowledge from pre-trained LLMs to the specific task of SQL generation, based on natural language queries and database schemas, without fine-tuning on task-specific annotated data. This limitation makes it difficult for the model to handle the complex map- ping between natural language queries and diverse database schemas, impeding its ability to accurately interpret schema relationships, construct complex SQL queries, and maintain robustness across various contexts.
1. Introduction
Text-to-SC Sattto-SLLLL (a.k.a. NL2SQL) converts natural language queries into SQL., simplifyinn access to relational databases
Our Methodology and Contributions. To address the above challenges, we propose Alpha-SQL., a novel approach that enables zero-shot Text-to-SQL. as a process of progres- sive SQL construction, where queries are progressively built step-by-step. The key idea of Alpha-SQL to decompose the task into smaller, more manageable sub-tasks, each with
'The Hong Kong University of Science and Technology (Guangzhou) *Remmin University of China 'Haawei Technologics Ltd. Correspondence to: Yuyu Luo <yuyuluo@ @rayyllu@@kuut-eee wr.us--z..uuu.n>>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Plug-in Alpha-SQL
In summary, Alpha-SQL is a fine-tuning-free, plug-and-play Text-to-SQI. framework that enhances small open-source LLMs for Text-to-SOL tasks. As shown in Figure 1, it can integrate and boost existing LLMs without fine-tuning on Text-toSQI. datasets. Extensive experiments show Alpha- SQL's strong performance, achieving 69.79 79.7% execution accu- racy on the BIRD development set, significantly outperform- ing existing zero-shot methods. Ablation studies confirm the effectiveness of our reasoning actions, and performance improves with more MCTS rollouts.
Direcily prompting 2.w without finie-tuning
Zero-shot Text-to-SQL SOTA (RSL-SQL with GPT-4o)
75%
60%
45%
30%
15%
0%
Alpha-SQL
7B
14B
32B 8
BIRD (Dev)
87
Figure/ 1. Alpha-SQL: A plug-in framework boosting small open- source LLMs. Our method significantly improves Qwen2.5's per- formance by 15%-20% across different model sizes (7B-32B) without fine-tuning, surpassing even GPT-4o based zero-shot Text- JOTA SOTA (RSL-SQL) on the BIRD (Dev) dataset.
2. Related Work
Text-to-SQL. The Natural Language to SQL (Text-to-SQL,
a.k.a., NL2SQLL) task involves translating natural language questions into SQL queries. The emergence of Pre-trained Language Models (PLMs) like T5 (Raffel et al., 2020) subsequently improved the performance of Text-to-SQI. tasks (Scholak et al., 2021; Li et al., 2023a;b; Gu et al., 2023). More recently, the development of LLMs has fur- ther advanced Text-to-SQL capabilities. However, apply- ing LLMs directly remains challenging due to issues like schema alignment, complex query generation, etc (Liu et al., 2024). To address this, recent works (Pourreza et al., 2025; Talaci et al., 2024; Li et al., 2024a; Cao et al., 2024; Liu et al., 2025a; Fan et al., 2024) have explored decomposing Text-to-SQI. into subtasks, such as candidate SQL gener- ation, refinement, and selection. For example, CHASE- SQL (Pourreza et al., 2025) employs a multi-step pipeline to generate and validate SQL candidates, mitigating errors introduced by direct generation.
contextual guidance, making it easier for the model to han- dle complexity at each step. To achieve this, we model the progressive construction process as a search problem over tree-structured space, where nodes represent partial rea- soning states, and edges denote SQL construction actions (e.g., selecting a table or revising SQL clause). By itera- tively selecting edges (actions) from the root to a leaf node, Alpha-SQL progressively constructs a valid SQL query.
Based on this idea, SQp. leverages Monte Carlo Tree Search (MCTS) framework (Coulom, 2006; Xie al., 2024) generate and explore SQL construction actions dynamically. To facilitate efficient and effective search within the MCTS framework, we introduce the following novel techniques.
First, to enhance reasoning capabilities during the search process, we propose the LLM-as MaMM-as-Action-Mi Model, which in- vokes an LLM as the reasoning action model in the MCTS framework to generate step-by-step reasoning (i.e., Chain- of- Thought) after each action taken. This reasoning is stored in each node alongside the partial state, enabling Alpha-SQL to maintain context and track the LLM's thought process throughout the SQL construction process. This ensures that each SQL construction action is both context-awars and aligned with the overall reasoning path, which can guide the search toward more promising SQL queries.
Building on this direction, we propose Alpha-SQL, a pro- gressive SQL construction framework that uses MCTS for dynamic query generation. Unlike prior methods relying on static pipelines or fine-tuning, Alpha-SQL leverages LLMs as action models, guiding the search in a context-aware man- ner, enabling efficient exploration and improved accuracy without task-specific labeled data.
Test-time Computation. Recent advances in test-time com- putation (Snell et al., 2024; Liu et al., 2025a) have sig- nificantly improved LLM performance without modifying model parameters. Techniques such as planning, search, and verification during inference have enhanced reasoning across various tasks (Wei et al., 2022; Yao al., 2023; Qiu et al., 2024; Hao et al., 2023; Zhang et al., 2025; Teng et al., 2025; Zhu et al., 2024; Zhang et al., 2024b). Recent methods, including tree- tree-based search (Yao et al., 2023) and Best-of-N sampling (Qiu et al., 2024), further optimized inference through structured search. Recent work has also explored MCTS-based reasoning to enhance the capabilities of LLMs (Qi et al., 2024). While effective in general rea- soning tasks, these methods do not fully address the unique challenges of text-to-SQL, such as schema understanding,
Second, to ensure accurate and efficient query generation during the MCTS search process, we introduce self- supervised reward function to evaluate the quality of can- didate SQL queries. Specifically, for each reasoning path, Alpha-SQL generates multiple candidate SQL queries us- ing nigh-temperattre sampling, filters out invalid queries, and computes self-consistency score by comparing the execution results of the sampled queries with those of the predicted SQL. This helps prioritize promising paths and re- fines the exploration process. Finally, Alpha-SQL calculates the self- coff-consittnncy scores of all candidate SQL queries and selects the SQL with the highest score as the final output.
2

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
generating semantically accurate SQL, and refining outputs based on execution feedback.
Edges (Actions)
Nodes
"What's the rank of Bob the foctball match?"
D- "CREATE TABLE' players L..''
Alpha-SQL builds on test-time computation principles with a search-based SQL generation framework designed for zero-shot Text-to-SLL. Unlike prior work, it integrates LLM-driven reasoning into the MCTS process for progres- sive SQL query construction, optimizing the action space and incorporating database feedback improve accuracy.
4y: Column Valuee Identffication
Column Value Thinking:
In the above question, there is specific filter about matich type and player
dg: Column Function lnlntilication
"player" namee" "Bob' 'match" match hypee "football:
Column Function Thinking
Gat SQL Generation
SQL Genemtion Thinking Based on previous thoughts, need WHERE clause to fittrr the metch type and player, and there is functiors neeced. Thus, the query
3. Alpha-SQL
uu Termination
SELECT TL.ank FROM players AS TT XOIN metches AS 72 GN G2T11 12,player_ Jd WHERE 71.name = "Bab' AND type "lotbali";
3.1. Zero-shot Text-to-SQL
Text-to-SQI. Task, Let D = (T,C, represent a rela- tional database, where T is the set of tables, C is the set of columns in those tables, and R denotes the relationships between tables (c.g., primary key-foreign key constraints). Let Q denote all well-speeified natural language questions over D, and }/ all valid SQL queries over D.
LLM-as-Action-oode
Figure 2. Example of the search tree formulation for Text-to-SQU
be expressed as: y = up O U1 @ ..... + Uys where @ denotes the concatenation or composition of the actions represented by the nodes along the path.
The goal is to find a mapping function f such that for any given question E Q. f(g, D) produces a syntactically and semantically correct SQL query y E V
Our goal is to identify an optimal reasoning path in the search tree that constructs an accurate SQL query for given natural language question and database schema D.
In the zero-shot setting, the key challenge is to construct mapping function f without task-specific labeled data. This requires the model to generalize across unseen SQL queries and databases, relying solely on pre-trained knowledge and the provided database schema.
Complexity of the Search Space. Efficiently exploring the search space S for Text-to-SQL generation is a significant challenge due to its combinatorial nature. The size of S, denoted as [S), grows exponentially with the complexity of the database schema D and the question q. For instance, the number of potential queries grows exponentially with the number of tables, columns, possible join conditions, and nested subqueries. This makes exhaustive search practically impossible, and therefore an efficient search strategy needs to be used to explore this vast space effectively and identify the correct SQL query.
Formulating Text-to-SQL as a Search Problem. We de- fine the Text-to-SQI. task as a search problem over a vast space of potential SQL queries, where the search space S consists of all valid SQL queries for given database 'D and question g. To structure this space, we represent S as tree T = (V, E), where:
(1) Nodes (V): Each node v E V represents a partial reason- ing state at a specific step in the query construction process. As shown in Figure 2, The root node to represents the initial empty query and contains the input question q and database schema 'D. Intermediate nodes store incremental reasoning steps, such as identifying column values, select- ing functions, or constructing SQL clauses. A leaf node or termination node iA represents either a fully constructed SQL query or a state where a termination action is applied.
3.2. An Overview of Alpha-SQL
To address the challenges of navigating the exponential search space S and generating high-quality SQL queries from natural language question q, we propose Alpha- SQL, novel framework that leverages Monte Carlo Tree Search (MCTS).
(2) Edges (E): Each edge e E E corresponds to an action in the query construction process, such as selecting a table, adding condition, or applying aggregation function. These actions model transitions between intermediate query states in the search tree.
MCTS-based Search with LLM-as--Ation-MMdde Build- ing upon the search problem formulation in Section 3.1, Alpha-SQL employs MCTS construct and explore the search tree V/ (V, E). Given an input question g, database DD and an LLM M, the MCTS process iteratively builds v. The root node up E V represents the initial state with an empty SQL query. The edges e E E represent an action o, where we invoke the LLM M to select a SQL construction action such as schema selection or column value identifica- tion, as shown in Figure 2. The MCTS process iteratively
(3) A Path from Root to Leaf Nodes (Candidate SQL): A path from the root node vp to a leaf node i, corresponds to a sequence of SQL. construction actions that, when composed, forms complete SQL query y E S. The SQL query y can
3

---

Alpha-SQL: Zero-Shot Text-to-QQL using Monte Carlo Tree Search
MCTS for Candidate SQL Generation
SQL Selection
Aow-as-Action-Madae
Repeat
Database
Previous
Selection
Expansion
Simulation
(4) Backpragation
Action
LLM
Papiracn Quastion
Visted Edge
Scchime Selection
Action Space
History
Reward
Value identffiation
Termination
Visited Node
Function Idontfication
Reward for SQLly
SQL
SQL Generation
ponsistency
3QL Revision
UCT Selection @Cv, a)
Self-suparvised Reward
Teminnsion
argmax
N(r.a)
(uN(Cp NG,,a))
RG.g. D) =
- LEXY, EX(y
Selected SIQL.
Figure 3. An Overview Alpha-SQL.
invokes M to apply actions during the search, exploring different paths. Each node i E V represents partial reasoning state after applying a sequence of actions. A com- plete path from the root termination (leaf) node forms reasoning trajectory corresponding query. The MCTS process generates multiple such trajecto- ries, forming a set T = [TI, 72,..... .., Tn) T of candidate SQL queries. Therefore, Alpha-SQL can efficiently explore the vast search space defined in the problem formulation.
where ys are sampled SQL queries, and N is the number of samples. This formulation reinforces SQL queries that consistently yield stable execution results, enabling Alpha- SQL to prioritize reliable reasoning trajectories without requiring annotated data.
candidate SQL
4. The Design Details of Alpha-SQL
4.1. LLM--s-Action-MMdde
Self-Supervised Rewards. The reward function plays crucial role in the MCTS process by evaluating the utility of each action, guiding the search toward more promising SQL queries. Traditional methods like Outcome Reward Models (Zelikman et al., 2022) and Progress Reward Mod- els (Uesato et al., 2022) require domain-speeiffe labeled data for training, making them difficult to generalize across different datasets (Zhang al., 2024a).
A key challenge in zero-shot text-to-SQL. is the difficulty of transferring general knowledge from pre-trained language models the specific task of SQL generation.
To enhance the reasoning capabilities of our framework. we propose the LLMM-as-Action-Modde which leverages LLMs to generate reasoning actions (i.e., CoT Thoughts) dynamically based on the current context of the problem. As shown in Figure 3, LLM-as- -atass-Action-MMie Modeel empowers the LLMs to generate appropriate action outputs based on the question, database schema, and current partial reasoning state (including previous actions), enabling the model to build a valid SQL query in step-by heep-by-step manner.
Inspired by human reasoning, we observe that individu- als who are confident in their answers tend to consistently provide the same response across multiple attempts, indi- cating high confidence their solution. Conversely, when responses vary, suggests low confidence, implying un- certainty and lower reliability (Qi al., 2024). This funda- mental intuition directly informs and forms the conceptual foundation of our innovative self-consistency-boee based reward function. Within this function, the perceived confidence or correctness of a candidate SQL query is quantitatively determined by the consistency of its execution results when compared against those obtained from a diverse set mul- tiple sampled queries. The reward is computed as:
Formally, at step of the reasoning process (u;), MCTS selects an SQL construction action a; from the action space, which is defined later. Based on the reasoning trajectory Up B the LLM is prompted execute a; and generate the next state 0441:
U = LLM D, Actions(ro, ... ..), Prompt(as)) (a:)):
where Actions(-) refers to all previous reasoning steps in the trajectory, and Prompt(-) represents the prompt instruc- tion for a specific action.
1 N
R(y.9, 9, D) =
Σ 1=1
Execute(y.. D) = Execute(y), D)),

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
SQL Construction Reasoning Action Space. The action
Table 1. Action Space with Ordering.
space defines the set of potential reasoning steps L.LM can take to decompose and solve Texl-to-SQL problems. It is crucial for the LLM-as-Action Model to guide the pro- gressive construction of SQI. queries by specifying possible actions at each stage. Previous works (Pourreza et al., 2025; Talaei et al., 2024; Gao et al., 2024b) used limited actions and fixed pipelines, which restricted the model's 's ability to explore the full space of potential solutions.
Previous Action
Valid Next Actions
A 12, A3. Ar, As
A1: Question Rephrasing
A2. 43.44.1 A4. As
Ag: Schema Selection
A3, As,A,,45 As
Ag: Column Value Identification
Ag, As
Aq: Column Function Identification
Ay, A3, As
As: SQL Generation
Ag, A7
Ag: SQL Revision
A7
Inspired by human thinking, where some might jump straight to the answer while others first clarify the question and break it down into subtasks, we introduce new reason- ing actions, such as question rewriting (Qi et al., 2024), alongside existing ones. In total, our action space defines seven distinct reasoning actions. The specific prompts for each action are illustrated in Appendix A.2.
A7: Termination
tion (COUNT (people. i.)). Analysis by CHASE- SQL (Pourreza et al., 2025) revealed that function-related errors account for 19% of mistakes in the BIRD develop- ment set. To improve function handling ability, we introduce a column function identification action during inference.
A1: Question Rephrasing, Text-to-SIL. systems need to handle diverse question styles and ambiguities from differ- ent user groups (Liu et al., 2024). While NL-Rewriter (Ma et al., 2024) addresses this through experience- ques- tion rewriting, it struggles in zero-shot scenarios where training data is unavailable. Building on rStar (Qi et al., 2024), use few-shot prompting to decompose questions into a structured (conditions list, question) format.
As: SQL Generation. SQL generation is the core com- ponent of Text-to-SQL systems. CHASE-SQL. (Pourreza et al., 2025) introduced a Divide-and-Conquer CoT strategy that breaks down complex queries into multiple subtasks, solves them independently, and combines solutions. This method particularly excels at handling nested queries. We incorporate this strategy into our reasoning action space.
A2: Schema Selection. Databases often contain complex schemas, but individual SQL queries typically use only a small subset of available elements. This mismatch creates challenges for accurate SQL generation (Liu et al., 2024). Prior work has established schema selection as critical component of SQL generation (Cao et al., 2024; Pourreza et al., 2025; Talaei et al., 2024). Following (Talaei et al., 2024), we use Chain-of oh-Thhtgh (CoT) prompting to iden- tify the relevant schema subset for each user question, which then guides subsequent query generation.
As: SQL Revision. LLMs can generate syntactically in- valid SQL queries in complex scenarios (Wang et al., 2025; Pourreza et al., 2025). Drawing inspiration from human debugging practices, we implement an execution-guidat correction mechanism. Our approach provides the LLM with the user question, schema, incorrect SQL, and execu- tion results to guide query revision. The system performs multiple correction rounds until either obtaining a valid SQL query or reaching a maximum attempt limit N Nrenisson
Ay: Termination, The termination action is invoked when the reasoning process yields the final predicted SQL, signi- fying the conclusion of reasoning trajectory. We specify that the termination action must occur following either SQL Generation or SQL Revision actions.
A3: Column Value Identification, Text-to-SQL systems need accurately identify filtering conditions in user ques- tions. For example, "What is Bob's best ranking in foot- ball matches?" requires filtering on both name ("Bob") and match type ("football"" (e.g., WHERE name 'Bob' AND ma match,type = 'football'). CHESS- SQL (Talaei et al., 2024) found that 20% of errors in the BIRD development set stem from incorrect filtering columns or value selection. To address this, we introduce a column value identification action that evaluates potential filtering values before SQL. generation.
Action Ordering and Constraints. Each reasoning trajec- tory follows a structured order, ensuring logical coherence. For example, some actions, like SQL Revision, must oc- cur only after SQL Generation. Table I defines the valid transitions between actions. Furthermore, to prevent the possibility of infinite loops and ensure forward progress in the reasoning trajectory, we impose a constraint that restricts each defined action to appear only once within any single query construction process.
As: Column Function Identification. Complex SQL queries often require aggregate functions (e.g., COUNT) and sealar functions (e.g., STRFTIME). For example, the question "How many people were born in 2024?" necessitates both date manipulation (STRETIME ( ('SY', people. of 20024' ) and aggrega-
4.2. MCTS for Candidate SQL Generation
Candidate SQL Generation with MCTS Rollout. We generate Candidate SQL through multiple MCTS rollouts.
5

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Specifically, each rollout includes four distinct phases: Se- lection, Expansion, Simulation, and Backpropagation.
2025), which rely on fine-tuned proprietary models such as Gemini-1.5-Flashe requiring extensive domain-specific data.
(1) Selection. This phase identifies promising nodes for expansion by traversing from the root node (vy) to ei- ther an unexpanded leaf or termination node. We use Up- per Confidence Bound applied to Trees (UCT) (Kocsis & Szepesvári, 2006) to balance exploration and exploitation during node selection: UCT(u, a)
We show the pseudo-code of Alpha-SQL in Appendix A.
Pruning Strategies. Alpha-SQI. incorporates schema con- straints and semantic rules into the search process to prune invalid paths early. A key aspect of our pruning strategy is the elimination of redundant nodes, For instance, when performing a Schema Selection action, we may sample the LLM M multiple times (e.g,, 3 times). Although the Chain- of Thought content generated by M may differ in each sam- ple, if the final selected schema subset is identical, we create only one child node instead of three duplicate nodes. This de-duplication significantly reduces the branching factor of the search tree without loss of information.
Q(r.e) + +ev [In N(v.a)' where (1, a) counts visits to action from node v, and (e) tracks total visits to node v. Q(v, a) is the estimated reward for action a from node v, updated via backpropaga- tion. We select the action with the maximum UCT score and move to the resulting child node. Notably, if there exist any unvisited child nodes (i.e., N(e a) = 0), we prioritize the selection of such nodes over UCT selection.
N(v,a)
4.3. Database Value Retrieval
(2) Expansion. The expansion phase generates child nodes from the selected node. Valid actions are determined by the node type (Table 1) and executed via LLM prompt- ing. Each action is sampled Nexpansion times with tempera- ture T Teapansion. creating N Nezpansion x |Eaattll child nodes, where |Ecatzal represents the number of valid actions. This sampling approach enhances reasoning diversity.
Efficiently and accurately retrieving database values rel- evant to user's natural language question crucial for Text-to-SLL generation (Liu et al., 2024). SQL queries typically reference limited subset of column values ex- tracted from potentially extensive databases, primarily for utilization within filtering conditions, such as those spec- ified in WHERE clauses. However, a significant challenge arises from the potential semantic discrepancies between how a user phrases their query and how values are stored in the database (e.g., a user might say "America" while the database stores "United States"). Following Talaei al. (2024), we employ a two-stage approach involving offline pre-processing and online retrieval.
(3) Simulation. A complete simulation process consists of iterative node selection and expansion until reaching a termination node. Throughout the simulation process, all newly expanded child nodes are persistently maintained within the tree structure.
(4) Backpropagation. At termination node (v:), backprop- agation begins by evaluating the predicted SQL using the self-supervised reward function from Section 3.2. We iden- tify the action (As or As) that produced the final SQL, then sample Nreward SQL queries with temperature T Treward to ensure diversity. The reward value r is calculated from the self-consistency score between the execution results of the sampled and predicted SQLs. The process then back- tracks from 2/ to the root node, updating Q(u, a) and N(v) values for all nodes along the path Un + ... @ U2 using: Q(v, a) = Qu, a) + r, N(v) = (v)) + 1. These up- dated values guide future search directions through the UCT formula in subsequent rollouts.
Offline Pre-processing. In the offline phase, we focus on pre- pre-processing database values to facilitate fast and accurate retrieval during the online phase. This process specifically targets TEXT type column values, as these are most prone to ambiguities and variations in user queries. For these selected textual column values, we apply the MinHash (Datar et al., 2004) technique to generate compact signatures. These Min- Hash signatures are then stored locally, creating an indexed and optimized representation of the database values that are most likely to require disambiguation.
Online Retrieval. During the online phase, when user sub- mits a natural language question, we first extracts relevant keywords from the question, which can be guided by few- shot prompts (Appendix A.3), Once the keywords are identi- fied, Locality Sensitive Hashing (LSH) (Datar et al., 2004) is utilized in conjunction with the pre-computed MinHash sig- natures stored locally. This LSH-based search allows for ef- ficient retrieval of database values that are semantically sim- ilar to the extracted keywords. The retrieved values are then subject to further filtering based on predefined editing simi- larity and semantic similarity thresholds (Eedit. Esemantie). The semantic matching employs OpenAl's text-embedding-
After Nrollout rollouts, we collect all complete trajectories that reach termination nodes, forming a set of candidate SQL reasoning trajectories T {T1, . )
Final SQL Selection. To select the optimal trajectory and SQL query from T, we leverage the convergent nature of Text-to-SQL: different reasoning paths yield equivalent SQL queries for a given question. We execute all predicted SQL. queries and select the one with the highest execution result consistency as the final prediction. This approach differs from previous methods like CHASE-SQL (Pourreza al.,
6

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
3-large model. The finally selected, relevant database values are then incorporated into the database schema prompt pro- vided to our LLM-as-Action-Mode enrichiig the context for the SQL generation process.
C1358-536
Abpha-500
A2pha-1
Alphe-d0t
1240:
KSLL-50
XAyRt=OCL 1796
NCS-00L
45
(022-4)
CIEES-OOL
SeperS0G
5. Experiments
Accuracy
aFD-Ooded
(76)
Gemint- Sugeo
(GF7-4)
5.1. Experimental Setup
Datasets. We utilize the Spider (Yu al., 2018) and BIRD (L. et al., 2023c) development sets for evaluation. Spider contains 1034 (NL, SQL) pairs, and BIRD includes 1534 pairs, with BIRD queries being more complex and containing domain specific keywords like CASE and IIF.
queet. 3-bodoo--4
Tte Seepde
Aiphe-Sgh Other Methads Pazato Pront
10
321
10?
Rodel Size
To facilitate more comparison experiments while reducing computational costs (Sections 5.3 to 5.5), we follow CHESS- SQL (Talaei et al., 2024) and utilize the same Subsampled Development Set (SDS), which comprises 10% of each database from the BIRD development set. The SDS contains 147 samples, consisting of 81 simple, 54 moderate, and 12 challenging questions.
Figure 4. Performance vs. Model Size on the BIRD dev. For GPT-4, GPT-4o, and Gemini-1.55ppi we referenced the parameter descriptions from (Ahacha al., 2024) for plotting.
Metrics. Following prior work (Pourreza et al., 2025), we use Execution Accuracy (EX) as the metric, defined as the percentage of predicted SQL queries that generate execution results identical to those of the ground-trun -truth queries.
performance of RSL-SQL (Cao et al., 2024), which relies on the proprietary GPT-4o. Notably, this performance sur- passes many methods that require data fine-tuning. Upon scaling our inference model to 32B parameters, Alpha-SQL. demonstrated superior performance in the zero-shot sce- nario, with 69.7% average accuracy. Even when compared to methods with domain data fine-tuning, Alpha-SQL.'s per- formance is exceeded only by CHASE-SQL (Pourreza et al., 2025), which requires fine-tuning the proprictary Gemini- 1.5-Flash model, and XiYan- (Gao et al., 2024b), which fine-tunes an unknown model. These results confirm Alpha- SQL's effectiveness as plug-and-play framework that de- livers competitive performance without fine-tuning.
Hardware. All experiments are run an Ubuntu 22.04.3 LTS server with 512GB of RAM and dual 40-core Intel(R) Xeon(R) Platinum 8383C CPUs @@ 2.70GHz). Open-source LLMs are deployed locally using GPUs, each with 80GB of memory and 312 TFLOPS with BF16 precision.
5.2. Main Results on BIRD and Spider Datasets
Settings. We employed three open-source models from the Qwen2.5-Coder family - 7B, 1 14B, and 32B (Yang et al., 2024)- as inference models for Alpha-SQL. The related hyper-paraman paeer--aameterss were set as follows: For offline database value retrieval, set the editing similarity Eddit 0.3 and se- mantic similarity Cemantic as 0.6. For the MCTS rollout pro- cess, we set the number of rollouts to Nyotlout = 24. During node expansion, each action was sampled Nezpameion = times with sampling temperature of expansioo 0.8. In the computation of supervisedd rewards, we set the SQL sampling parameters with Nreward
Performance on Spider Dataset. We also evaluate Alpha- SQL on the Spider development dataset. As shown in Ta- ble 3, Alpha-SQI. with Q outperforms existing methods. Notably, it achieves a 2.1% improvement over SFT CodeS-15B (Li al., 2024b), which was specifi- cally fine-tuned for the Spider dataset. This demonstrates Alpha-SQL's capability achieve strong generalization per- formance without dataset-specific fine-tuning.
Performance-Scall Trade-off' Analysis. To explore the performance potential of smaller open-source LLMs, we conducted this experiment to demonstrate that Alpha-SQL can unlock the full potential of smaller models while main- taining cost efficiency. Figure shows that Alpha-SQL sig- nificantly outperforms larger models the Pareto frontier, enabling smaller models, such as the 7B and 14B versions, to achieve accuracy comparable to or surpassing much larger models, including GPT-4o-based approaches. This demon- strates our framework's 's ability to optimize Text-to-SQL performance across varying model scales.
repetitions and temperature of Treword = 1.0. For the SQL Revision action (Aa), we set a maximum iteration limit of Nrevision = 10 for the multi-round correction process.
Performance on BIRD Dataset. As shown in Table 2, we conducted comprehensive comparison between Alpha- SQL and current state-of-the-ant approaches, categorizing methods based on their zero-shot capabilities. Alpha-SQL,, leveraging the relatively lightweight Qwen2.5-Coder-7B model, achieved 66.8% average accuracy. comparable to the

---

Alpha-SQL: Zero-Shot Text-to-QQL using Monte Carlo Tree Search
Table Exccution Accuracy on BIRD Development Dataset.
Inference
Selection Model
Zero-shot Setting
Accuracy (%)
Method
Model
Simple
Moderate
Challenging
All
SFT CodeS (Li al., 2024b)
CodeS-7B
64.6
46.9
40.3
57.0
SFT CodeS ct al., 2024b)
CodeS-15B
65.8
48.8
42.4
58.5
Distillery (Maamari et al., 2024)
GPT-4o
CHESS-SQL (Talaci al., 2024)
Deepscck-Coder-33
GPT-4-Turbo
CHESS-SQL (Talaci et al., 2024)
Deepsck-Coder-33B
LLaMA3-708
X
61.5
CHASE-SQL (Pourreza et al., 2025)
Gemini-1.5-Pro
Gemini-1.5-Plasi
X
73.0
XiYan-SQI. (Gan et al., 2024b)
?
73.3
XiYan-SQL (Gao al., 2024b)
Qwen2.5-Coder-322
Qwen2.5-Coder-32B
67.0
DAIL-SQL (Gao et al., 2024a)
GPT4
SC Selection
45.6
43.1
SuperSQL (Li et al., 2024u)
GPT4
SC Selection
66.9
46.5
43.8
58.5
MCS-SQL (l.ee et al., 2025)
GPT-4
GPT-4
64.4
RSL-SQL (Cao al., 2024)
GPT-4o
GPT-4o
74.4
57.1
53.8
67.2
Alpha-SQL (Ours)
@wen2.5-Coder-7B
SC Selection
72.6
59.3
53.1
66.8
Alpha-SQL (Ours)
Owen2.5-Coder-14B
Selection
74.6
55.9
Alpha-SQL (Ours)
Qwen2.5-Coder-32B
SC Selection
74.5
64.0
57.2
69.7
Table 3. Execution Accuracy on Spider Development Dataset.
Inference
Selection Model
Zero-shot Setting
Accuracy (%)
Method
Model
Easy
Medium
Hard
Extra Hard
SFT CodeS (Li et al., 2024b)
CodeS-7B
94.8
91.0
75.3
66.9
85.4
SFT CodeS (Li ct al., 2024b)
CodeS-15B
95.6
90.4
78.2
61.4
84.9
C3-SQL (Dong et al., 2023)
GPT-3,5-Turbo
SC Selection
92.7
85.2
77.6
62.0
82.0
DIN-SQL (Pourreza Rafiei. 2023)
GPT-4
92.3
87.4
76.4
62.7
82.8
DAIL-SQL (Gao et al., 2024a)
GPT-4
SC Selection
91.5
90.1
75.3
62.7
83.6
ZeroNI-2SQ1. (Fan et al., 2024)
GPT-4
84.0
MAC-SQL. (Wang al., 2025)
GPT-4
86.8
SuperSQI. al., 2024a)
GPT-4
Selection
94.4
91.3
83.3
68.7
87.0
Alpha-SQL (Ours)
Qwen2.5-Coder-7B
SC Selection
94.0
89.2
76.4
63.3
84.0
Alpha-SQI. (Ours)
Qwen2,55Coder-1445
SC Selection
94.0
91.0
79.9
72.3
87.0
35
the number MCTS rollouts affects performance, we con- ducted an in-depth analysis. We conducted experiments the SDS dataset using the Qwen2.5-Coder=778 model, maintaining all hyper-parameters identical to Section 5.2 except for the number of rollouts (NraHonu). We report both the upper bound accuracy and final accuracy. Similar to CHASE-SQL. (Pourreza et al., 2025), the upper bound accuracy represents the percentage samples where the candidate SQL set contains the correct SQL query before the final SQL selection. We observe positive correla- tion between the number of MCTS rollouts and both upper bound and final accuracy metrics. This demonstrates Alpha- SQL's capability to enhance Text-to-SQL task performance through more MCTS explorations.
Uppez Bound (TB)
(75)
Accuracy (K)
755
2.
65
MCIS Rollouts
Figure 5. Accuracy vs. MCTS Rollouts
5.3. Impact of MCTS Rollouts on Performance
5.4. Comparison with Baseline LLMs
The efficiency of MCTS exploring large search spaces is key feature of Alpha-SQL. Based on Table we calcu- lated that there over 3000 possible reasoning paths for each text-to-SQL task. Remarkably, Alpha-SQL achieves significant performance improvements with just 24 MCTS rollouts, suggesting that our Alpha-SQL can efficiently ex- plore significantly larger search spaces.
In this section, we present an evaluation of the baseline performance characteristics of various LLMS when applied directly to the Text-to-SQL task on the SDS dataset. These LLMs were categorized into two groups: general LLMss exemplified by models such as GPT-4o. and LLMs specifi- cally optimized for reasoning tasks, such as DeepSeek-R1. For a fair comparison, all baseline LLMs were prompted
To further investigate this efficiency and understand how
8

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Table Comparison with Baseline LLMs the SDS dataset.
6. Conclusion
Model
Accuracy (%)
Deepseek-V2 V3
51.2
In this paper, we proposed Alpha-SQL, zero-shot Text-to- SQL framework that frames SQL generation as structured search problem. By combining Monte Carlo Tree Search with -ssa--Actto--ooee! Alpha-SQL. explores the SQL query space efficiently without fine-tuning the LLMs. Ex- periments show competitive performance, with 69.7% on the BIRD development set. Ablation studies validate the effectiieneess our reasoning actions, and performance improves with more MCTS rollouts.
GPT-4o
53.7
Gemini-1,5-Pro
56.2
QwQ-32B-Preview
38.8
DeepSeek-R1
50.3
Gemini-2.0- Flan:i000laaas--iiiinn---gg
60.8
Qwen2.5-Coders
47.6
Alpha-SQL (Ours)
64.6 (f 17.0)
Phi-4
43.5
Alpha-SQL (Ours)
60.0 (t 16.5)
A Acknowledgements
Table 5. Ablation Study on the Action Space.
Action Space
Accuracy (%)
This paper is supported by NSF of China (62402409, 62436010, and 62441230), Guangdong provincial project 2023CX 10X008, Guangdong Basic and Applied Basic Re- search Foundation (2023A1515110 545), Guangzhou Basic and Applied Basic Research Foundation (2025A0413935), and Guangzhou-HKKUST(GZ)) Joint Funding Program (2025A0313714).
11, A2, A3, A4, As, As, A7
64.6
w/o A₁ (Question Rephrasing)
63.9 (4 0.7)
w/o A2 (Schema Selection)
63.1 (4 1.5)
w/o A3 (Column Value Identification)
64.2 () 0.4)
w/o A1 (Column Function Identification)
64.0 (L 0.6)
w/o Ag (SQL Revision)
62.8 (t 1.8)
using standardized Text-to-SOL prompt, detailed in Ap- pendix A.4. As shown in Table 4, our AIp, utilizing a model with only 7B parameters, surpasses all baseline models in performance. Notably, Alpha-SQI. outperforms Gemini-2.0-Flaase Thinking-Exp 2..---a--s----hiiinng----- optimized model, despite using a weaker model. This shows that the Text-to-SQI. task requires targeted reasoning op- timization, which is a strength of the Alpha-SQL frame- work. Moreover, to validate Alpha-SQL's plug-and-play advantages, we conducted additional experiments using Phi-4 (Abdin et al., 2024) and Qwen2.5-Coder: as infer- ence models. Compared to directly prompting these LLMs, Alpha-SQL achieved sigmificant accuracy improvements of 17.0% and 16.5% for Qwen2.5-Coder--77 and Phi-4, respec- tively. These results validate Alpha-SQL's generalizability and effectiveness across different inference models.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
a sophisticated reasoning-
References
Abacha, A. B., Yim, W., Fu, Y., Sun, Z., Yetisgen, M., Xia, F., and Lin, T. MEDEC: A benchmark for medical error detection and correction in clinical notes. CoRR, abs/2412.19260, 2024.
Abdin, M. I., Aneja, J., Behl, H. S., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R. J., Javaheripi. M., Kauffmann, P., Lee, J. R., Lee, Y. T., Li, Y., Liu, W., Mendes, C. C. T., Nguyen, A., Price, E.,, de Rosa, G., Saarikivi, O., Salim, A., Shah, S., Wang, X., Ward, R.,
5.5. Ablation Study of Action Space
Wu, Y., Yu, D., Zhang, C., and Zhang, Y. Phi-4 technical report. CoRR, abs/2412.08905,, 2024.
The purpose of this ablation study is to validate the effective- ness of our proposed Text-to-SQL reasoning action space and the LLM-as- Muss-Action-Mae Modee approach. To achieve this, we conducted experiments on the SDS dataset, systemati- cally removing individual actions from the original action space while maintaining the parameter settings from Sec- tion 5.2. Table 5 presents the results of these experiments.
Cao, 7.,, Zheng, Y., Fan, 7., Zhang, X., Chen, W., and Bai, X. RSL-SQL: robust schema linking in text-to-sql generation. CoRR, abs/2411.0007 2024.
Coulom, R. Efficient selectivity and backup operators in monte-carlo tree search. In Computers and Games, vol- ume 4630 of Lecture Notes in Computer Science, pp. 72-83. Springer, 2006.
Table 5 shows that removing any action from the original action space negatively impacts performance. The SQL Revision action demonstrates particular significance, as it leverages database interaction to incorporate feedback into the LLM for SQL correction, highlighting the importance of database execution feedback for Text-to-SQL, tasks.
Datar, M., Immorlica, N., Indyk, P., and Mirrokni, V. S. Locality sensitive hashing scheme based on p-stable dis- tributions. In SCG, pp. 253-262. ACM, 2004.

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Dong, X., Zhang, C., Ge, Y., Mao, Y., Gao, Y., Chen, L., J., and Lou, D. C3: texttto-qq with chatgpt. CoRR, abs/2307 07736, 2023.
Li, J., Hui, B., Qu, G., Yang, J., Li, B., Li, B., Wang, B., Qin, B., Geng, R., Huo, N., Zhou, X., Ma, C., Li, G., Chang, K. C., Huang, F., Cheng, R., and Li, Y. Can LLM already serve as A database interface? A big bench for large-scale database grounded text-to-sqls. In NeurlPS, 2023c.
Fan, J., Gu, Z., Zhang, S., Zhang, Y., Chen, Z., Cao, L.,, Li, G., Madden, S., Du, X., and Tang. N. Combining small language models and large language models for zero- shot NL2SQL. Proc. VLDB Endow., 17(11)2750--7633 2024.
Lin, X., Qi, Y., Zhu, Y., Palpanas, T., Chai, C., Tang, N., and Luo, Y. Lead: Iterative data selection for efficient Ilm instruction tuning, 2025. URL aattps://arxt. org abs /2w...555500000 07437.
Gao, D., Wang. H., Li, Y., Sun, X., Qian, Y., Ding, B., and Zhou, J. Text-to-sql empowered by large language models: A benchmark evaluation. Proc. VLDB Endow., 17(5):1132-111 1145, 2024a.
Liu, B., Li, X., Zhang. J., Wang, J., He, T., Hong, S., Liu, H., Zhang, S., Song, K., Zhu, K., Cheng, Y., Wang, S., Wang, X., Luo, Y., Jin, H., Zhang, P., Liu, O., Chen, J., Zhang, H., Yu, Z., Shi, H., Li, B., Wu, D., Teng, F., Jia, X., Xu, J., Xiang, J., Lin, Y., Liu, T., Liu, T., Su, Y., Sun, H., Berseth, G., Nie. J., Foster, I., Ward. L. T., Wu. Q., Gu, Y., Zhuge, M., Tang, X., Wang. H., You, J., Wang, C., Pei, J., Yang, Q.. Qi, X., and Wu, C. Advances and challenges in foundation agents: From brain-inspires intelligence to evolutionary, collaborative, and safe systems. CoRR. abs/2504.01 9900, 2025a.
Gao, Y., Liu, Y., Li, X., Shi, X., Zhu, Y., Wang. Y., Li, S., Li, W., Hong, Y., Luo, Z., Gao, J., Mou, L., and Li, Y. Xiyan-sql: A neutti-generato ensemble framework for text-to-sql. CoRR, abs/2411 1.08999. 2024b.
Gu, Z.., Fan, J., Tang, N., Cao, I.., Jia, B., Madden, S., and Du, X. Few-shot text-to-sql translation using structure and content prompt learning. Proc. ACM Manag. Data, (2):147:1-147:28 2023.
Liu, X., Shen, S., Li, B., Ma. P., Jiang. R., Luo, Y., Zhang, Y., Fan, J., Li, G., and Tang, N. A survey of NL2SQL with large language models: Where are we, and where are we going? CoRR, abs/2408.05 1590,, 2024.
Hao, S. Gu, Y., Ma, H., Hong, J., Wang. Z., Wang, D. Z., and Hu, Z. Reasoning with language model is planning with world model. In EMNLP, pp. 8154-8177 -173. Association for Computational Linguistics, 2023.
Liu, X., Shen, S., Li, B., Tang, N., and Luo, Y. NI2sql-bugs: A benchmark for detecting semantic errors in NL2SQL translation. CoRR, abs/2503. 11984, 2025b.
Kocsis, L. and Szepesvári, C. Bandit based monte-carlo planning. In ECML, volume 4212 of Lecture Notes in Computer Science, pp. 282-293. Springer, 2006.
Luo, T., Huang, C., Shen, L., Li, B., Shen, S., Zeng, W., Tang, N., and Luo, Y. nybench 2.0: A benchmark for natural language to visualization under ambiguity. CoRR, abs/2503. 12880, 2025.
Lee, D., Park, C., Kim, J., and Park, H. MCS-SQL: lever- aging multiple prompts and multiple-choice selection for text-to-sql generation. In COLING, pp. 337-353. Asso- ciation for Computational Linguistics, 2025.
Luo, Y., Qin, X., Tang, N., and Li, G. Deepeye: Towards au- tomatic data visualization. In ICDE, pp. 101-112. IEEE Computer Society, 2018a.
Li, B., Luo, Y., Chai, C., Li, G., and Tang, N. The dawn of natural language to SQL: are we fully ready? [experiment, analysis benchmark J. Proc. VLDB Endow., 17(11): 2024a.
Luo, Y., Qin, X., Tang, N., Li, G., and Wang, X. Deepeye: Creating good data visualizations by keyword search. In SIGMOD Conference, pp. 733-1736. ACM, 2018b.
Li, H., Zhang. J., Li, C., and Chen, H. RESDSQL: decou- pling schema linking and skeleton parsing for text-to-sql. In AAAL, pp. 13007-13075. AAAI Press, 2023a.
Luo, Y., Tang, N., Li, G., Chai, C., Li, W., and Qin, X. Synthesizing natural language to visualization (NL2VIS) benchmarks from NL2SQL benchmarks. In SIGMOD Conference, pp. 1235-1247. ACM, 2021.
Li, H., Zhang, J., Liu, H., Fan, J., Zhang, X., Zhu, J., Wei, R., Pan, H., Li, C., and Chen, H. Codes: Towards building open-source language models for text-to-sql. Proc. ACM Manag. Data, 2(3):127, 2024b.
Luo, Y., Qin, X., Chai, C., Tang, N., Li, G., and Li, W. Steerable self-driving data visualization. IEEE Trans. Knowl. Data Eng., 34(1):475 490, 2022a.
Li, Hui, B., Cheng, R., Qin, B., Ma, C., Huo, N., Huang, F. Du, W., Si, L., and Li, Y. Graphix-t:: Mixing pre- trained transformers with graph-aware layers for text-to- sql parsing. In AAAL pp. 13084, AAAI Press, 2023b.
Luo, Y., Tang, N., Li, G., Tang, J., Chai, C., and Qin, X. Natural language to visualization by neural machine trans- lation. IEEE Trans. Vis. Comput. Graph., 28(1):217-226, 2022b.
10

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Luo, Y., Zhou, Y., Tang, N., Li, G., Chai, C., and Shen, L. Learned data-aware image representations of line charts for similarity search. Proc. ACM Manag. Data, I(I): 88:1-88:29, 2023.
Wang, B., Ren, C., Yang. J., Liang, X., Bai, J., Chai, L.,, Yan, Z., Zhang, Q., Yin, D., Sun, X., and Li, Z. MAC-SQL:: A multi-agent collaborative framework for text-to-sql. In COLING, pp. 540-557. Association for Computational Linguistics, 2025.
Ma, P., Li, B., Jiang, R., Fan, J., Tang, N., and Luo, Y. A plug-and-play natural language rewriter for natural language to SQL. CoRR. abs/2412.17068, 2024.
Wei, J., Wang, X., Schuurmans, D.., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V. and Zhou, D. Chain-of- thought prompting elicits reasoning in large language models. In NeurlPS, 2022.
Maamari, K., Abubaker, F., Jaroslawicz, D., and Mhedhbi, A. The death of schema linking? text-to-sql in the age of well-reasoned language models. CoRR, 077002, 2024.
Wu, Y., Shi, J., Wu, B., Zhang, J., Lin, X., Tang, N., and Luo, Y. Concise reasoning, big gains: Pruning long reasoning trace with difriculty-wware prompting, 2025. URL https: /aaaaaxxvvvvroo.///a
Pourreza, M. and Rafiei, D. DIN-SQL: decomposed in- context learning of text-to-sql with self-correction. In NeurIPS, 2023.
Xie, Y., Luo, Y., Li, G., and Tang, N. Haichart: Human and AI paired visualization system. Proc. VLDB Endow., 17 (1 (11):317-3119,, 2024.
Pourreza, M., Li, H., Sun, R., Chung, Y., Talaci, S., Kakkar, G. T., Gan, Y., Saberi, A., Ozcan, F., and Arik, S. Ó. CHASE-SQL: multi-path reasoning and preference opti- mized candidate selection in text-to-sql. ICLR. Open- Review.net, 2025.
Yang, A., Yang. B., Zhang, B., and et al. Qwen2.5 technical report. CoRR, abs/2412.15115 2024.
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree thoughts: Deliberate problem solving with large language models. In NeurIPS, 2023.
Qi Z., Ma, M., Xu, J., Zhang, L. L., Yang, F,, and Yang, M. Mutual reasoning makes smaller lims stronger problem- solvers. CoRR, 0tbs//408.061999 2024.
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., Zhang, Z., and Radev, D. R. Spider: A large-scale tuman-labeled dataset for complex and cross-domain semantic parsing and text-to- sql task. In EMNLP, pp. 3911-3921. Association for Computational Linguistics, 2018
Qiu, J., Lu, Y., Zeng, Y., Guo, J., Geng, J., Wang, H., Huang, K., Wu, Y., and Wang, M. Treebon: Enhancing inference- time alignment with speculative tree-search and best-of-n sampling. CoRR, abs/2410.1600 16033, 2024.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Mach. Learn Res., 11:14001-140::0 2020.
Zelikman, E., Wu, Y., Mu, J., and Goodman, N. D. Star: Bootstrapping reasoning with reasoning. In NeurIPS, 2022.
Scholak, T., Schucher, N., and Bahdanau, D. PICARD:
Zhang, D.,, Zhoubian, S., Hu, Z., Yue, Y., Dong, Y., and Tang, J. Rest-mots*: LLM self-training via process re- ward guided tree search. In NeurIPS, 2024a.
parsing incrementally for constrained auto-regressive de- coding from language models. In EMNLP (1), pp. 9895- 9901. Association for Computational Linguistics, 2021.
Zhang. J., Zhao, C., Zhao, Y., Yu, Z., He, M., and Fan, J. Mobileexperts: A dynamic tool-enabled agent team in mobile devices. CoRR, abs/2407.00 003913., 2024b.
Snell, C., Lee, J., Xu, K., and Kumar, A. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. CoRR, 0tts/2008.33311 14, 2024.
Zhang, J., Xiang, J., Yu, Z., Teng. F., Chen, X., Chen, J., Zhuge, M., Cheng, X., Hong, S., Wang. J., Zheng, B., Liu, B., Luo, Y., and Wu, C. Aflow: Automating agentic workflow generation. In ICLR. OpenReview.net, 2025.
Talaei, S., Pourreza, M., Chang, Y., Mirhoseini, A., and
Saberi, A. CHESS: contextual harnessing for efficient SQL synthesis. CoRR, abs/2405. 16755, 2024.
Teng, F., Yu, Z., Shi, Q., Zhang, J.. Wu. C., and Luo, Y. Atom of thoughts for markov LLM test-time scaling.
Zhu, Y., Du S., Li, B., Luo. Y., and Tang, N. Are large language models good statisticians? In NeurlPS, 2024.
CoRR, abs/a502. 12018, 2025.
Zhu, Y., Jiang, R., Li, B., Tang, N., and Luo, Y. Elliesql: Cost-efficient text-to-sqi with cowplexity-aware routing. CoRR, abs22503.22442,, 2025.
Uesato, J., Kushman, N., Kumar, R., Song, H. F., Siegel, N. Y., Wang, L., Creswell, A., Irving, G., and Higgins, Solving math word problems with process- and outcome- based feedback. CoRR, abs/2211 142755, 2022.

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
A. Appendix
A.1. Alpha-SQL Algorithm
The Alpha-SQL algorithm, as outlined in Algorithm 1, operates in multiple phases: Selection, Expansion, Simulation, and Backpropagation. Given a user query q and the corresponding database schema D, the algorithm starts by initializing an empty search tree = (V, E) with root node vo representing the initial state (lines 3-4). The process then iterates over Nrollout MCTS rollouts (line 6), where each rollout seeks to explore high-value reasoning paths in the search space.
In the Selection phase (lines 7-12), starting from the root node, Alpha-SQL recursively traverses the search tree until an unexpanded node is reached. The next action is selected based on the Upper Confidence Bound for Trees (UCT) formula, which balances exploration and exploitation by considering both the estimated reward and the visit counts actions (line 10). This process continues until a terminal node is found or all children the node are expanded.
In the Expansion phase (lines 13-23), valid next actions Avali are determined based on the current node's type. The algorithm generates new states by invoking our L..Maas-Action-oodee to execute each action, creating new child nodes in the search tree (lines 15-21). The expansion process introduces new reasoning paths, enriching the search space for subsequent rollouts.
The Simulation phase (lines 24-30) performs rollouts by randomly selecting unexplored child nodes and executing the corresponding actions. This phase continues until terminal node is reached, at which point the final SQL query is extracted from the trajectory.
Finally, the Backpropagation phase (lines 32-40) updates the values of nodes along the path from the terminal node back to the root. The reward is computed by sampling multiple SQL queries and calculating their self-consistency scores based on their execution results. The values of nodes (u) and Qu, (.)) are updated according to the reward to guide future searches.
At the end of the rollouts, the algorithm selects the SQL query with the highest self-consistency from the set of candidate queries generated during the search, as indicated by line 43.
This procedure enables Alpha-SQL efficiently explore the search space of SQL queries, balancing exploration with accuracy and providing scalable, tine-tuning-fie free solution to sero-shot Text-to-SQL tasks.
12

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Algorithm AIph Zero-Shor Shot Text-to-SQL using Monte Carlo Tree Search
1: Input: Question 9. Database Schema D, LLM M, Number of rollouts Nrottout
2: Output: SQL query y
3: Initialize an empty search tree V = (V. R) with root node Up
4: Initialize a root node vp with (g. D)
5: T <
6: for = 1 to Nollout do
7: // Selection Phase
8: v e-
9: while v is not terminal and is fully expanded do
10:
a f argggaa...... A(v) (Q( a)/N(u,a) + Nn)n
11: v f child reached by action
12: end while
// Expansion Phase
14:
if is not terminal then
15:
Avatid <- GetValidAccionss(e) from Table for a E Avalid do
16:
17:
for j = 1 to Nezzansion do
18:
Unew < M(q. D, Actions(so........ .,). Prompt(a))
Add Unew as child of in
end for
21: end for
22:
Random unexplored child of end if
23:
24:
// Simulation Phase
25:
while v is not terminal do
26:
Apatid + VelidAAAciio))))
Expand with Avalid and ramdomly select action
28: Unew M(q, D, Aclions(ro..... (a))pl(a))
29:
Unent
30:
end while
31: Extract final SQL from v
32:
// Backpropagation Phase
33:
Sample Nreword SQL queries (yI,..., yNreware with temperature Treward
34:
r Neeward
Σ Li=1
Nreward [Execute(y, D) = Execute(y,,
35: T + the path from Up to v
36: for each node u in do
37:
N(u)= + N(u) +1
Quu, au) Quu, a.) + where au is the action taken u
39: N(u, a.) < a(u,au) +1
40: end for
41: T U
42: end for
43: return argmaryey (Self-consistency())) where are unique SQLs from
13

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
A.2. Prompt Template for Actions
In this section, we provided the prompt templates for the actions defined in Section 4.1.
Rephrase Question Action Prompt
You are an Al assistant to help me rephrase questions by splitting the question context into conditions. In your rephrased question, remember fully express the information in the original question.
Example 1:
Original Question: Name movie titles released in year 1945. Sort the listing by the descending order of movie popularity.
Hint: released in the year 1945 refers to movie release year = 1945;
Rephrased Question: Given a list of conditions, please answer the question. Condition I: Movies are released in the year 1945. Condition 2: Movies are sorted by the descending order of movie popularity. Condition Return the movic titles. Question: What are the movie titles released the year 1945, sorted by the descending order movie
popularity?
Example 2:
Original Question: How many office supply orders were made by Cindy Stewart in the south superstore?
Hint: office supply refers to Category 'Office Supplies'
Rephrased Question: Given a list of conditions, please answer the question. Condition I: Orders made by Cindy Stewart. Condition 2: Orders are office supplies, refer to Category 'Office Supplies'. Condition 3: Return the number of orders. Question: How many office supply orders were made by Cindy Stewart in the south superstore?
Example 3:
Original Question: Tell the number of fights landed earlier on Miami Airport on 2018/8/12.
Hint: landed on refers to DEST; landed earlier refers to ARR_DELAY < O; Miami Airport refers to DEST 'MIA'; on 2018/8/12 refers to FA_DATE '2018/81 2218////2';;
Rephrased Question: Given list of conditions, please answer the question. Condition Flights landed Miami Airport 2/8//2 refer DEST 'MIA' and FL DATE '2018/8/12'. Condition 2: Flights landed earlier, refer ARR_DELAY < 0. Condition 3: Return the number of fights. Question: How many fights landed earlier on Miami Airport on 201 8/8//22
Answer the following question:
Original Question: {QUESTION]
Hint: (HINT)
Rephrased Question:
Figure 6. Question Rephrasing Action Prompt.
14

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Schema Selection Action Prompt
You expert and very smart data analyst.
Your task examine the provided database schema, understand the posed question, and use the hint pinpoint the specific columns within tables that essential for crafting SQL query to answer the question.
The schema offers in-depth description of the database's architecture, detailing tables, columns, primary keys, foreign keys, and any pertinent information regarding relationships constraints. Special attention should be given to the examples listed beside each column, as they directly hint at which columns are relevant to our query.
For key phrases mentioned in the question, we have provided the most similar values within the columns denoted by Value Examples" in front of the corresponding column names. This is a critical hint to identify the columns that will be used in the SQL query.
The hint aims to direct your focus towards the specific elements of the database schema that are crucial for answering the question effectively.
Task: Based on the database schema, question, and hint provided, your task is to identify all and only the columns that are essential for crafting SQL query to answer the question. For each of the selected columns, explain why exactly necessary for answering the question. Your reasoning should concise and clear, demonstrating a logical connection between the columns and the question asked.
Please respond with JSON object structured follows: "json
"chain of thought reasoning": "Your reasoning for selecting the columns, be concise and clear"" "table namel": ["collumn", "column2", ...]. "table_name2"; [""cllmmn",, "column2",...
Make sure your response includes the table names as keys, each associated with list column names that are necessary for writing a SQL query to answer the question. For each aspect of the question, provide a clear and concise explanation of your reasoning behind selecting the columns. Take a deep breath and think logically. If you do the task correctly, will give you million dollars. Database Schema Overview: "SCHEMA_CONTER Question: [QUESTION]
Hint: (HINT)
Only output json (starting with jjson and ending with () as your response.
Figure 7. Schema Selection Action Prompt.
15

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Column Value Identification Action Prompt
You are an Al assistant to help me identify the potential column values (if needed to be used in the SQL query) that are essential for answering the question.
Here is an example:
Database Schema:
CREATE TABLE generalinfo
id_restaurant INTEGER not null primary key, food_type city TEXT null, Column Description: the city where the restaurant located in
food type TEXT null, Value Examples: 'thai' Columm Description: the food type
CREATE TABLE location
id restaurant INTEGER not null primary key, street_name TEXT null, Value Examples: 'ave', 'san pablo ave', 'pablo ave" Column Description: the street name of the restaurant
city TEXT null, Column Description: the city where the restaurant located in foreign key (id restaurant) references generalinfo (id restaurant) on update cascade on delete cascade
Question:
How many Thai restaurants can be found in San Pablo Ave, Albany?
Hint:
Thai restaurant refers to food type 'thai'; San Pablo Ave Albany refers to street name 'san pablo ave' AND
TI.city 'albany'
Answer:
Since the restaurants are located in Albany, based the schema information and the hint, need 'loca- tion'. 'street_name' 'san pablo ave' AND 'generalinfo'. .'city"" 'albany'.
**********.............*****....
Now, answer the real question, and you need to follow the answer style of the above examples (answer in two sentences).
Database Schema: [SCHEMA_CONTENT
Question: [QUESTION]
Hint: (HINT)
Answer:
Figure 8, Column Value Identification Action Prompt.
16

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Column Function Identification Action Prompt
You are an Al assistant to help me identify the potential column functions (if needed be used in the SQL query)
that are essential for answering the question.
Here is an example:
Database Schema:
CREATE TABLE businesses
"business INTEGER NOT NULL,
'name' TEXT NOT NULL, Column Description: the name of the eatery PRIMARY KEY ("businessjid')
CREATE TABLE inspections
"business id' INTEGER NOT NULL, Column Description: the unique of the business 'score' INTEGER DEFAULT NULL, - Column Description: the inspection score 'date' TEXT NOT NULL, Value Examples: '2014-01-24"
FOREIGN KEY ("business_id) REFERENCES "businesses' ('business.idf) idd")
Question: What are the names of the businesses that passed with conditions in May 2012?
Hint: name of business refers to dba name; passed with conditions refers to results = 'Pass w/ Conditions'; in May 2012 refers to inspection date like 20112-05%
Answer: Since the businesses passed with conditions in May 2012, I should consider a date-related function to filter the 'inspections' 'date' column. find that column is of type TEXT, so can use the strftime( Ymm'',, 'inspections' 'dtee') '2012-05' filter the date.
*******5*******
Now, answer the real question, and you need to follow the answer style of the above examples (answer in two sentences).
Database Schema: (SCHEMA
Question: {QUESTION]
Hint: (HINT)
Answer:
Figure 9, Column Function Identification Action Prompt.
17

---

Alpha-SQL: Zero-Shot Text-to-QQL using Monte Carlo Tree Search
SQL Generation Action Prompt
You experienced database expert. Now you need to generate SQL query given the database information, question and some additional information.
The database structure is defined by the following table schemas (comments after " provide additional column deseriptions). Note that the "Value Examples" are actual values from the column. Some column might contain the values that are directly related to the question. Use to help you justify which columns to use. Given the table schema information description and the "Question". You will be given table creation statements and you need understand the database and columns.
You will be using way called "recursive divide-and-conquer approach to SQL query generation from natural language".
Here is a high level description of the steps. *ividdd (Decompose Sub-question with Pseudo SQL):** The complex natural language question recursively broken down into simpler sub-questions. Each sub-question targets specific piece of information or logic required for the final SQL. query.
2. **oonquer (Real SQL for sub-questions):** For each sub-question (and the main question initially). "pseudo- SQL" fragment is formulated. This pseudo-SQL represents the intended SQL logic but might have placeholders for answers to the decomposed sub-questions.
3. Commbine (Rcassemble):*** Once all sub-questions are resolved and their corresponding SQL fragments are generated, the process reverses. The SQL fragments are recursively combined by replacing the placeholders in the pseudo-SQL with the actual generated SQL from the lower levels.
** F*Final Output:** This bottom-up assembly culminates the complete and correct SQI. query that answers the original complex question.
Database admin instructions (voliating any of the following will result is punishble death!):
1. *ELEEC $
Only select columns mentioned in the user's question.
Avoid unnecessary columns or values.
** Aggregation (MAX/MIN):**
Always perform JOINs before using MAXO or MINO.
** **ORDER BY with Distinct Values:##
Use 'GROUP BY column' before 'ORDER BY column ASCIDESC" ensure distinct values.
4. Handllnn NULLs;##
If column may contain NULL values, use 'OIN' or 'WHERE <column>> IS NOT NULL'.
Repeating the question and hint, and generating the SQL with Recursive Divide-and anddee-add-a MaddCCnnuer, and finally try to simplify the SQL. query using 'INNER JOIN' over nested 'SELECT" statements IF POSSIBLE.
Please respond with a JSON object structured as follows: " 'json
""chin_of__te tcchain_o_thhuuhtt.o chhoughtteesoonnnii Conquer approach.".
"Your detailed reasoning for the SQL query generation, with Recursive Divide-and-
"sql-uery"": "The final SQL query that answers the question."
Table creation statements:
CONTEXT]
******************************** **
Question: {QUESTION]
Hint: (HINT)
Only output json (starting with jsson and ending with 1) as your response.
Figure 10. SQL Generation Action Prompt.
18

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
SQL Revision Action Prompt
**aask Description:**
You are an SQL database expert tasked with correcting a SQL query. A previous attempt to run a query did not yield the correct results, either due to errors in execution because the result returned was empty or unexpected. Your role is analyze the error based the provided database schema and the details of the failed execution, and then provide corrected version of the SQL query.
** **Proeduur:***
1. Review Database Schema:
Examine the table creation statements to understand the database structure.
Analyze Query Requirements:
Original Question: Consider what information the query is supposed retrieve.
Hint: Use the provided hints understand the relationships and conditions relevant the query.
Executed SQL Query: Review the SQL query that was previously executed and led to error incorrect result.
Execution Result: Analyze the outcome of the executed query to identify why it failed (e.g., syntax errors, incorrect column references, logical mistakes).
3. Correct the Query:
Modify the SQL query to address the identified issues, ensuring correctly fetches the requested data according to the database schema and query requirements.
Based on the question, table schemas, the previous query, and the execution result, analyze the result following the procedure, and try fix the query. You cannot modify the database schema or the question, just output the corrected query.
Please respond with JSON object structured follows: 'json
"chain_of wtt__ttttttteeeaee reasoning" "Your detailed reasoning for the SQL query revision.", "sql-query": "The final SQL query that answers the question.".
..
Table creation statements: (SCHEMA.CONTEET))
Question: [QUESTION]
Hint: (HINT)
Only output json (starting with >> "json and ending with >) as your response.
Figure 11. SQL Revision Action Prompt.
19

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
A.3. Prompt Template for Question Keywords Extraction
In this section we provided the prompt template for the question keywords extraction in Section 4.3.
Question Keywords Extraction Prompt
Analyze the given question and hint to identify and extract keywords, keyphrases, and named entities. These elements crucial for understanding the core components of the inquiry and the guidance provided. This process involves recognizing and isolating significant terms and phrases that could be instrumental in formulating searches queries related to the posed question.
Objective:
Instructions:
Read the Question Carefully: Understand the primary focus and specific details of the question. Look for any named entities (such as organizations, locations, etc.), technical terms, and other phrases that encapsulate important aspects of the inquiry.
Analyze the Hint: The hint is designed to direct attention toward certain elements relevant to answering the question. Extract any keywords, phrases, or named entities that could provide further clarity or direction in formulating an answer.
List Keyphrases and Entities: Combine your findings from both the question and the hint into single Python list.
This list should contain:
Keywords: Single words that capture essential aspects of the question or hint.
Keyphrases: Short phrases or named entities that represent specific concepts, locations, organizations, or other significant details.
Ensure to maintain the original phrasing or terminology used in the question and hint.
Example 1:
Question: "What is the annual revenue Acme Corp in the United States for 2022?" Hint: "Focus on financial reports and U.S. market performance for the fiseal year 2022." ["annual revenue", "Acme Corp", "United States", "2022", "financial reports", "U.S, market performance", "fiscal year")
Example 2:
Question: "In the Winter and Summer Olympics of 1988, which game has the most number of competitors? Find the difference of the number of competitors between the two games."
Hint: "the most number of competitors refer to (penoonn id)); SUBTRACTICOUNT(PESSn id where
games name '1988 Summer'), COUNT(person _id where games_nang name = '1988 Winter')):"
["Winter Olympics", "Summer Olympics", "1988", "1988 Summer", "Summer", "1988 Winter", "Winter,, "number of competitors", "difference", "MAX(COUUN(pperoon id))", "games name", "person id")
Example 3:
Question: "How many Men's 200 Metres Freestyle events did lan James Thorpe compete in?"
Hint: "Men's 200 Metres Freestyle events refer event_nane 'Swimming Men"' 200 metres Freestyle'; events compete refers to event id;" ["Swimming Men's 200 metres Freestyle", "Ian James Thorpe", "lan", "James", "Thorpe", "compete in", name", "event_id"]
Task: Given the following question and hint, identify and list all relevant keywords, keyphrases, and named entities. Question: {QUESTION}
Hint: (HINT)
Please provide your findings as a Python list, capturing the essence of both the question and hint through the identified terms and phrases. Only output the Python list, explanations needed.
Figure 12. Question Keywords Extraction Prompt.
20

---

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
A.4. Prompt Template for Baseline LLMs
In this section we provided the prompt template for the directly calling baseline LLMs in Section 5.4
Baseline Text-to-SQL. Prompt
You are an experienced database expert. Now you need generate a SQL query given the database information, question and some additional information.
The database structure is defined by the following table schemas (comments after "-' provide additional column descriptions). Note that the "Value Examples" are actual values from the column. Some column might contain the values that are directly related to the question. Use it to help you justify which columns to
Given the table schema information description, the 'Question' and 'Hint', you need generate SQL query that answers the question. Please respond the final sql query in the end of response.
********** **** **********
Table creation statements:
[SCHEMA.CONTEN)
Question:
{QUESTION]
Hint:
[HINT]
Output Format:
<think>
Your thinking process.
</think>
<sql>
The final SQL query.
</sql>
Figure 13, Baseline Text-to-SQL. Prompt.
21


Reviews:
[{"review_id": "8W8OAeaj9g", "reviewer": "Reviewer_5kbR", "summary": "The authors propose a Monte Carlo tree search framework for zero-shot text-to-SQL with LLMs. The action space is a set of sub-tasks whose composition (subject to ordering rules) defines a reasoning path that terminates in a SQL output given a question and database. They generate candidate SQL queries by MCTS rollout, using the LLM to generate the next state given the action, and using consistency in execution accuracy as a self-supervised reward. They report state-of-the-art results on BIRD dev set among zero-shot methods for open-source LLMs.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your thorough review and constructive feedback. We appreciate your positive assessment and valuable suggestions, which will help improve our paper's clarity.\n\n**1. Terminology (\"Partial SQL Query States\")**\n- **Our Response**: Thank you for highlighting this ambiguity. We agree \"reasoning state\" or \"contextual state\" is more accurate for internal nodes representing accumulated context (selected schema, functions, etc.) prior to the final SQL generation action. We will revise the manuscript accordingly.\n\n**2. Self-Consistency Sample Generation**\n- **Our Response**: Thanks for the question. There are two distinct stages: 1) During MCTS, we use deterministic sampling (Temperature=0) for the primary candidate SQL from a given reasoning state. 2) For reward calculation, we use stochastic sampling (Temperature =1.0) to generate N diverse SQL samples from the exact same reasoning state. The reward is based on execution agreement between the deterministic query and the diverse set. We will clarify this process in the revision.\n\n**3. Inference-Time Cost Evaluation**\n- **Our Response**: This is a very relevant point. The inference cost, particularly in terms of LLM calls and latency, is an important consideration for MCTS-based methods. Due to rebuttal length limitations, we kindly refer the reviewer to our detailed response to **Reviewer guJX (Point 1 & 2)**. We believe those sections fully address the concerns regarding token usage and latency analysis.\n\n**4. Comparison with \"SQL Generation\" Action Alone**\n- **Our Response**: This is an excellent suggestion for a baseline. This baseline, representing a direct prompting approach using only the 'SQL Generation' action with the initial question and schema, corresponds to the \"Direct Prompting\" results in our experiments.\n| Base LLM            | Direct Prompting | Directly Promoting with Self-consistency | Alpha-SQL |\n|------------------------------|------------------|------------------------------------------|-----------|\n| Qwen2.5-Coder-Instruct-7B    | 48.4%            | 56.3%                                    | 66.8%     |\n| Qwen2.5-Coder-Instruct-14B   | 57.4%            | 62.3%                                    | 68.7%     |\n| Qwen2.5-Coder-Instruct-32B   | 62.6%            | 63.4%                                    | 69.7%     |\n- The results clearly show Alpha-SQL significantly outperforms direct, single-step prompting across all model sizes, demonstrating the substantial benefit of the MCTS framework for dynamic path construction and context gathering. We will ensure this is clearly discussed.\n\n**5. Analysis of Common Reasoning Paths**\n- **Our Response**: A valuable suggestion for deeper insight. Our analysis (Qwen-7B on BIRD-dev) shows Alpha-SQL dynamically adapts reasoning paths based on database complexity, unlike fixed-sequence methods.\n- We analyzed the reasoning paths selected by Alpha-SQL (based on Qwen2.5-Coder-7B) on the BIRD-dev dataset. Our findings reveal interesting patterns that demonstrate the adaptive nature of Alpha-SQL's reasoning:\n  - **Simple Schema Databases**: For databases with relatively simple schemas (e.g., 'toxicology' with 4 tables, avg. 2.8 columns/table), the most frequently selected reasoning path followed the pattern: Root -> Identify Column Values -> SQL Generation -> End. Notably, this common path **omits the 'Schema Selection' action**. This suggests that for simpler schemas where most tables/columns might be relevant or easily inferred, Alpha-SQL learns that the computational effort or potential risk of error from explicitly running 'Schema Selection' outweighs its benefits, adapting by taking a more direct path to SQL generation.\n  - **Complex Schema Databases**: In contrast, for databases with more complex schemas (e.g., 'student_club' with 8 tables, avg. 6 columns/table; 'california_schools' with 3 tables, avg. 29.3 columns/table), the most common reasoning path pattern included explicit schema filtering: Root -> Identify Column Values -> Identify Column Functions -> Schema Selection -> SQL Generation -> End. The inclusion of the 'Schema Selection' action in these cases highlights Alpha-SQL's ability to recognize when detailed schema filtering is necessary due to complexity and dynamically incorporate the appropriate actions into the reasoning process.\n  - **Comparison to Fixed Sequences**: This flexibility, driven by MCTS, demonstrates an advantage over rigid pipeline approaches. Leveraging the MCTS framework, it dynamically adapts the reasoning path based on the perceived complexity and characteristics of the specific database and query, selecting actions only when deemed beneficial by the search process. We will add details of this analysis, including path examples, to the Appendix in the revised manuscript.\n\n**Note: Due to the character limit for replies, we will discuss your other two questions, \"Rollouts vs. Path Diversity\" and \"Using Labeled Data for Reward vs. Fine-tuning,\" later.**", "claims_and_evidence": "The main claims in the introduction are the introduction of a novel MCTS approach for text-to-SQL along with state-of-the-art results for zero-shot performance on the BIRD benchmark. These are supported by the sections that follow.\n\nAs a very minor complaint, the authors sometimes claim that the internal nodes of their tree correspond to \"partial SQL query states\", but this seems misleading as the query is generated entirely by a single action ($A_5$), while most of the remaining actions simply gather context.", "methods_and_evaluation_criteria": "The method makes sense and seems original for text-to-SQL. The self-consistency based reward function is reasonable, though model confidence is not always a good proxy for correctness. One thing that was slightly unclear is how exactly the samples are generated for self-consistency.\n\nEvaluation uses standard datasets (Spider / BIRD) and metrics (execution accuracy). One thing that is notably missing, however, is evaluation in terms of cost at inference time. This seems like a significant and relevant question for an MCTS-based approach.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The main experiments for performance are standard. The ablations give some confidence about the relevance of including each action, though since these accuracy numbers are computed on a subsampled dataset it's not clear if the drops are all statistically significant. One comparison that might have been useful would have been the performance of a method that just calls the \"SQL Generation\" action alone (using the same model as in Alpha-SQL).", "relation_to_broader_scientific_literature": "There is a long literature on text-to-SQL, with zero-shot methods increasingly successful and popular given the quality of new pretrained models. Most text-to-SQL systems decompose the problem into smaller tasks (i.e. actions), but these are usually composed in a fixed sequence rather than dynamically according to some policy.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "The MCTS approach seems novel for text-to-SQL, and it advances the field by enabling dynamic composition of a reasoning path from component actions rather than a fixed sequence of steps. The paper is written clearly and the empirical results are contextualized against strong baselines.", "other_comments_or_suggestions": "It would be interesting to see a summary of the most common reasoning paths selected by Alpha-SQL (in terms of action sequences), and how these compare to the fixed sequence of steps implemented by other methods. It also seems notable that there is only a slight improvement in the accuracy as a function of the MCTS rollouts (Fig 5). It would be interesting to relate this to diversity of the reasoning paths.", "questions_for_authors": "- Could the authors comment on or provide evidence regarding the inference-time cost of Alpha-SQL? How does this compare to other zero-shot baselines?\n- How are the samples drawn for the self-consistency reward function? Are they generated from the same leaf node as the candidate query?\n- If labeled data is available, could it be used for the reward rather than the consistency criterion? Or in this case is it just better to finetune a model directly?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "22zQ5YYae6", "reviewer": "Reviewer_guJX", "summary": "This paper introduces a novel zero-shot Monte Carlo Tree Search (MCTS)-based Text-to-SQL approach that constructs SQL queries progressively, enhancing the Text-to-SQL capabilities of Qwen2.5-Coder-32B. The proposed method achieves an execution accuracy of 69.7% on the BIRD dev set and 87.0% on spider dev set, surpassing previous approaches.\n\n### update after rebuttal: While I appreciate the author's rebuttal, I have decided to maintain my score, as most of my concerns remain valid, as detailed in my comments following the rebuttal.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your detailed and insightful review of our paper. \nBelow, we address each point in detail:\n\n**1. Fair Comparison, Model Choice, and Performance Validation (Addressing Concerns on Experiments, Q1, Q2a, and Rationale for Qwen)**\n- **Our Response**: We acknowledge the critical importance of fair comparison and understand the concern that the observed performance gains might be attributed solely to the base LLM (Qwen2.5-Coder) rather than the Alpha-SQL framework. We also agree that the rationale for choosing Qwen needs clarification.\n  -  **Direct Qwen vs. GPT-4o Prompting Comparison - Addresses Q1**:\n      -  To directly address the relative strength of the base models on this task (Q1), we performed a direct prompting comparison between Qwen2.5-Coder-Instruct-32B and GPT-4o on the BIRD dev set using the same simple prompt structure. We also included results using simple self-consistency:\n\n          | Model                                       | Execution Accuracy |\n          |---------------------------------------------|--------------------|\n          | GPT-4o                                      | 62.3%              |\n          | Qwen2.5-Coder-Instruct-32B                  | 62.6%              |\n          | GPT-4o + Self-consistency                   | 63.2%              |\n          | Qwen2.5-Coder-Instruct-32B + Self-consistency | 63.4%              |\n\n      - This comparison reveals that Qwen2.5-Coder-Instruct-32B and GPT-4o exhibit very comparable performance on the BIRD dev set when using simple direct prompting (62.6% vs 62.3%) and even when enhanced with basic self-consistency (63.4% vs 63.2%). This indicates that neither model has a significant inherent advantage over the other for this specific task under these simple zero-shot conditions.\n\n  -  **Run Baselines on Qwen2.5-Coder-Instruct-7B**:\n    | Methods   | Execution Accuracy | Input Tokens (K) / Question | Output Tokens (K) / Question | Total Tokens (K) / Question | Latency (s) / Question |\n    |-----------|--------------------|-----------------------------|------------------------------|-----------------------------|------------------------|\n    | RSL-SQL   | 57.7%              | 12.1                        | 0.3                          | 12.4                        | 11.35                  |\n    | CHESS-SQL | 61.0%              | 327.0                       | 24.8                         | 351.8                       | 284.4                  |\n    | Alpha-SQL | 66.8%              | 138.0                       | 72.2                         | 200.2                       | 377.1                  |\n      -  Comparing these results, Alpha-SQL (66.8%) significantly outperforms both RSL-SQL (57.7%) and CHESS-SQL (61.0%) when using the identical base LLM. Specifically, Alpha-SQL achieves a +5.8% absolute gain in execution accuracy over the strongest baseline evaluated here (CHESS-SQL). This result strongly suggests that the performance improvements demonstrated by Alpha-SQL are substantially attributed to our proposed MCTS framework and reasoning path exploration, rather than solely being an effect of the base model choice. We will add these crucial comparative results to the relevant tables in the revised manuscript.\n\n**2. Computational Cost Analysis (Addressing Concerns on Cost, Weakness, Q2b)**\n> Reviewer Concerns: Computational cost not discussed; method seems costly; need for analysis considering forward passes, self-consistency samples, tokens, execution time; comparison needed using same self-consistency N.\n- **Our Response**: We agree that a discussion of computational cost is essential for contextualizing Alpha-SQL's performance gains, especially given its MCTS nature. We have analyzed the average computational cost per query on the BIRD dev set using the Qwen2.5-Coder-Instruct-7B model. The table above (in point 1) includes Alpha-SQL and the baselines run on the same model.\n- In summary, Alpha-SQL delivers state-of-the-art accuracy among these methods on the same base model, demonstrating better token efficiency than the next best method (CHESS-SQL), but this comes at the cost of increased latency. In future work, we plan to optimize the MCTS process to mitigate this latency, potentially exploring strategies such as heuristic pruning and SQL execution caching mechanisms. \n\n**3. Depth of Analysisn (Addressing Concerns on Claims, Analysis, Q3)**\n- **Our Response**: We appreciate the reviewer's push for deeper analysis beyond aggregate execution accuracy to better illustrate how Alpha-SQL improves text-to-SQL generation.\n  - As detailed in our response to **Reviewer 5kbR's point 5 (\"Analysis of Common Reasoning Paths\")**, our analysis of the reasoning paths chosen by Alpha-SQL demonstrates its ability to dynamically adapt its strategy based on database complexity (e.g., selectively including 'Schema Selection' only when needed). This provides initial insight into its advantages over fixed-sequence methods.", "claims_and_evidence": "The paper claims that a key challenge in zero-shot Text-to-SQL lies in the difficulty of transferring and generalizing knowledge from pre-trained LLMs to the specific task of SQL generation. However, there is no further explanation or experimental evidence supporting this inference, particularly in how this limitation affects complex query mapping. Additionally, while Alpha-SQL achieves a high execution accuracy (EX) score on test sets, the paper lacks a clear discussion of its advantages over existing methods beyond this metric. To strengthen the evaluation, the paper should include analytical experiments such as case studies or statistical analyses to better illustrate Alpha-SQL’s effectiveness and potential improvements in reasoning or structural accuracy.", "methods_and_evaluation_criteria": "This method decomposes SQL generation into multiple subproblems and employs MCTS for test-time scaling, which helps enhance generation quality. The evaluation criteria, such as execution accuracy, BIRD, and Spider, are well-established benchmarks in the field.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "- The main experiments only take experiments on Qwen2.5-Coder family model that is different from other baselines. This raises concerns about the validity of the comparisons. Given that Qwen2.5-Coder-32B has comparable general coding performance to GPT-4o (2024-08-06) and may even outperform it on SQL tasks (e.g., 85.1 vs. 79.8 on Spider, according to [1]), the superiority of Alpha-SQL could stem from the model itself rather than the proposed method. To ensure fair comparisons, all the important methods should be evaluated using the same model—either by running other baselines on Qwen2.5-Coder-32B or by testing Alpha-SQL on GPT-4o/GPT-4.\n- The computational cost of each method during inference is not clearly discussed. Since different methods may use varying numbers of forward passes, self-consistency participants, SQL execution attempts, and input/output token counts, a direct comparison without accounting for these factors may be unfair. A thorough cost analysis is necessary to contextualize performance gains.\n- The paper lacks detailed explanations and experimental analyses to highlight the specific performance improvements and challenges addressed by Alpha-SQL, which affects the completeness of the study.\n\n[1] Alibaba Group. Qwen2.5 Coder Family[EB/OL]. [2025-03-06]. https://qwenlm.github.io/blog/qwen2.5-coder-family/.", "relation_to_broader_scientific_literature": "This paper builds upon prior research in zero-shot Text-to-SQL methods, particularly those leveraging LLMs, such as CHESS, Chase-SQL, and C3-SQL. Unlike previous approaches that generate SQL in a single step, it introduces a novel MCTS-based framework that decomposes SQL generation into subproblems, improving test-time scalability. While prior studies have explored MCTS for structured reasoning, this paper is among the first to apply it effectively to SQL generation.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Other Weaknesses:\n\nIt seems that the proposed method is computational costly as it requires decomposing the task and MTCS.", "other_comments_or_suggestions": "- Sections 3 and 4 contain overlapping details about Alpha-SQL, making the paper somewhat redundant. This repetition reduces the space available for discussing the motivation behind the approach and providing insightful analytical experiments. A clearer separation of conceptual explanations and technical details would improve the paper’s structure.\n- The first mention of Monte Carlo Tree Search (MCTS) appears in the introduction, yet the citation for it is only provided in Section 3.2. To ensure proper attribution and clarity, the citation should be introduced at its first mention\n- In Section 5.2, while previous methods primarily used closed-source models, this paper adopts the Qwen2.5-Coder family. The rationale behind this choice is not clearly explained. The authors should clarify why they opted for Qwen2.5-Coder instead of continuing the trend of using closed-source models, especially given its potential impact on comparability.", "questions_for_authors": "I am very interested in these experimental results, as they directly impact my evaluation of this paper.\n\n1. **Direct Performance Comparison Between Qwen2.5-Coder-32B and GPT-4o on the BIRD Dev Set:**\n   The paper highlights the effectiveness of Qwen2.5-Coder-32B with Alpha-SQL, but it does not provide a direct comparison between Qwen2.5-Coder-32B and GPT-4o on the BIRD dev set. Understanding their relative performance would help determine whether Alpha-SQL’s gains stem from the proposed method itself or the choice of model.\n2. **Fair Comparisons with Key Baselines (e.g., MCS-SQL, RSL-SQL):**\n   A fair evaluation should consider consistency across models and computational costs. Specifically:\n   - **Model Selection:** All methods should be evaluated on either GPT-4o(same version) or Qwen2.5-Coder-32B to ensure a valid comparison.\n   - **Computational Cost :** One possible approach is to compare performance using the same number of self-consistency samples and provide statistics on the number of input tokens, output tokens, and SQL execution times for each method.\n3. **Further Validation of Claims in Section 1:**\n   The paper states that prior methods struggle with SQL generation while Alpha-SQL addresses these challenges to some extent. However, further experimental validation is needed to support this claim. For example, an analysis of failure cases from previous methods compared to Alpha-SQL would help illustrate specific improvements and their underlying causes.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QBLtAWBQzY", "reviewer": "Reviewer_2yXT", "summary": "The paper presents a novel approach to Text-to-SQL that eliminates the need for fine-tuning by leveraging the reasoning capabilities of large language models (LLMs). Alpha-SQL employs a Monte Carlo Tree Search (MCTS) framework to progressively construct SQL queries by breaking them down into smaller, manageable sub-tasks. The core component, LLM-as-Action-Model, dynamically generates SQL construction actions and provides step-by-step reasoning, maintaining context throughout the query-building process. A self-supervised reward function evaluates the quality of candidate SQL queries by computing a self-consistency score, helping to refine the exploration and prioritize promising paths. Experimentally, Alpha-SQL achieves a 69.7% execution accuracy on the BIRD development set.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your detailed feedback. We appreciate the opportunity to clarify our approach, particularly regarding the core concerns about the MCTS framework's validity, the reward mechanism, experimental fairness, and novelty, which we believe may stem from some misunderstandings.\n\n**1. Validity and Significance of the MCTS Framework for Text-to-SQL**\n> Reviewer Concerns: Formulating Text-to-SQL as a search problem is unconvincing/lacks support; main pipeline is nearly fixed (like CHESS), making many paths ineffective; questions value of searching reasoning actions; inconsistency between MCTS claim and reward function impact.\n- **Our Response**: We respectfully disagree that formulating Text-to-SQL as an MCTS search problem lacks significance. While fixed workflows like CHESS exist, Text-to-SQL often requires flexible adaptation (e.g., for schema complexity, query ambiguity) that MCTS provides.\n  - **Why MCTS is Beneficial**: Alpha-SQL uses MCTS precisely to navigate the choices within the reasoning process, not just execute fixed steps. \n    - **Adaptability**: MCTS allows Alpha-SQL to dynamically select the most useful sequence of preparatory actions (like Schema Selection, Value Identification) for a given query, rather than following a rigid pipeline. It efficiently prunes suboptimal paths while exploring promising variations. **Our analysis presented to Reviewer 5kbR (Point 5) demonstrates this adaptive path selection based on database complexity.**\n    - **Clarifying Reward Propagation**: This seems to be a key point of misunderstanding. The reward obtained after SQL generation is backpropagated through the entire MCTS tree via standard MCTS updates. This directly influences the selection probabilities of all preceding actions (including Actions 1-4) based on their contribution to successful outcomes. This holistic path guidance fundamentally differs from the local feedback/refinement mechanisms in methods like CHESS or E-SQL [2].\n    - **Theoretical Support**: Our contribution is the novel application and formulation of the well-established MCTS algorithm for zero-shot Text-to-SQL, supported by strong empirical results. We will enhance Section 3 to clarify the MCTS reward backpropagation mechanism.\n\n**2. Fair Experimental Comparison (Qwen vs. GPT-4)**\n> Reviewer Concerns: Experiments lack fair comparison using the same base LLM; Alpha-SQL uses Qwen, baselines use GPT-4; need Alpha-SQL results on GPT-4.\n- **Our Response**: We acknowledge the importance of fair comparison. Due to rebuttal length constraints, we kindly refer you to our detailed response to **Reviewer guJX (Points 1 & 2)**. This includes direct performance comparisons of Qwen-32B and GPT-4o (showing they have similar base capabilities on BIRD-dev) and evaluations of baselines (RSL-SQL, CHESS-SQL) run directly on Qwen-7B. These results demonstrate Alpha-SQL's significant gains originate from the framework itself, not just the base model.\n\n**3. Novelty Concerns (SQL Generation)**\n> Reviewer Concern: Core SQL Generation action is based on existing method (CHASE-SQL), raising concerns about novelty and effectiveness.\n- **Our Response**: While we leverage insights from prior work like CHASE-SQL for components like SQL Generation, the primary novelty of Alpha-SQL lies in the overall MCTS framework. This includes the dynamic orchestration of actions, adaptive reasoning path construction, and the integration of components into a cohesive, effective system for zero-shot Text-to-SQL. We will refine our discussion to better delineate these contributions.\n\n**4. Computational Cost Analysis**\n> Reviewer Concern: MCTS likely increases runtime; analysis of inference cost needed; limitations section missing.\n- **Our Response**: We agree cost analysis is essential. Our analysis, detailed quantitatively in response to **Reviewer guJX (Points 1 & 2)**, confirms Alpha-SQL achieves higher accuracy but incurs higher latency compared to baselines run on the same model. We commit to adding a dedicated \"Computational Cost Analysis\" subsection and a \"Limitations\" section discussing these trade-offs in the revised manuscript.\n\n**5. Missing References**\n> Reviewer Suggestion: Include discussion of E-SQL and the LLM Agents survey.\n- **Our Response**: Thank you for suggesting E-SQL and the LLM Agents survey. We agree they are relevant and will incorporate discussion of both in our revised related work section.\n\n**6. Supplementary Material Clarification**\n> Reviewer Statement: \"There is no supplementary material provided.\"\n- **Our Response**: We apologize for any confusion. Supplementary material, including detailed prompts, algorithm code, and running results, was provided with our initial submission and should be accessible via the OpenReview page.\n\nWe hope that these clarifications and planned revisions will address the reviewer's concerns and lead to a re-evaluation of our work's contributions.", "claims_and_evidence": "The claims made in this submission are not supported by clear and convincing evidence. The main concern is that the authors' proposition of formulating Text-to-SQL as a search problem and modeling it using the Monte Carlo Tree Search (MCTS) framework is not convincing. The paper designs seven actions for text-to-SQL based on MCTS. However, the reward model, which relies on SQL execution results, can only provide rewards for the two actions related to SQL generation (action-5 SQL Generation, action-6 SQL Revision). For the prior actions (action 1-4: Question Rephrasing, Schema Selection, Column Value Identification, Column Function Identification), no rewards are obtained as they do not lead to SQL generation results. Consequently, the reasoning path search space is limited, as the reward function is not comprehensive. As a result, the proposed Alpha-SQL still adheres to the traditional architecture, performing steps such as Question Rephrasing and Schema Selection sequentially, before considering SQL execution results in the SQL generation and refinement phases, similar to approaches like CHESS [1].\n\n[1] CHESS: Contextual Harnessing for Efficient SQL Synthesis", "methods_and_evaluation_criteria": "The proposed approach of treating Text-to-SQL as a search problem using the Monte Carlo Tree Search (MCTS) framework does not make sense. The paper lacks theoretical support for this formulating text-to-SQL as a search problem. Although Table 1 provides a search space for each action and Section 5.2 mentions 3000 possible reasoning paths in a Text-to-SQL task, in practice, based on empirical experience and previous research, the main pipeline of a Text-to-SQL framework is nearly fixed. For instance, as outlined in this paper, the sequence of Question Rephrasing, Schema Selection, Column Value Identification, and Column Function Identification, followed by SQL generation and SQL refinement, is an effective process already extensively discussed in papers like CHESS [1]. This implies that the majority of the claimed 3000 reasoning paths are practically ineffective. For example, the difference between a sample reasoning path like \"Question Rephrasing to SQL Generation\" and a complete reasoning path such as \"Question Rephrasing, Schema Selection, Column Value Identification, Column Function Identification, SQL Generation, SQL Refinement\" is significant. Naturally, the latter provides a more detailed analysis and yields more accurate results. Given this, why spend extra computational resources to decide every detail of the reasoning actions? Considering the reward model is solely based on SQL execution feedback, it further highlights the inconsistency of defining Text-to-SQL as a search problem.\nSince the proposed reward function only impacts the SQL generation and refinement stages, it does not significantly differ from prior methods like CHESS [1] and E-SQL [2], which also utilize SQL execution feedback for refinement. Moreover, unlike traditional workflows that execute actions sequentially, Alpha-SQL requires multiple interactions with the LLM and several rounds of expansion and backpropagation. This could lead to longer runtime when using the same LLM as the base model. An analysis of the inference item cost under the same settings should be presented in a limitation section, which is currently missing in the paper.\n\n[1] CHESS: Contextual Harnessing for Efficient SQL Synthesis\n[2] E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL", "theoretical_claims": "The paper does not include any proofs for theoretical claims.", "experimental_designs_or_analyses": "A concern is that the experimental section does not compare baselines using the same LLM as the base model. The proposed Alpha-SQL employs the Qwen2.5-Coder series, while the baselines predominantly use GPT-4. Using different LLMs for baselines may introduce bias that affects the results. Since GPT-4 also supports zero-shot settings, the authors should provide experimental results of Alpha-SQL based on GPT-4 to ensure a fair comparison with the baselines.", "relation_to_broader_scientific_literature": "The paper attempts to model Text-to-SQL as a search problem and proposes a Monte Carlo Tree Search (MCTS)-based method. MCTS is an algorithm used for decision-making processes, widely applied in tasks requiring reasoning and strategic planning, by utilizing effective reward mechanisms to aid decision-making. MCTS employs the Upper Confidence Bound (UCB) formula to balance exploration and exploitation. This strategy allows MCTS to efficiently explore high-potential nodes while avoiding premature convergence to local optima [1]. Recently, MCTS has been employed in LLM-based research to enhance reasoning and decision-making capabilities [2].\n\n[1] Monte Carlo Tree Search: A Review of Recent Modifications and Applications\n[2] A Survey on Large Language Model-based Autonomous Agents", "essential_references_not_discussed": "The paper includes a detailed discussion of related work on Text-to-SQL. However, it would be beneficial to include and discuss the following two papers:\n\n[1] E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL \n[2] A Survey on Large Language Model-based Autonomous Agents", "other_strengths_and_weaknesses": "Strengths:\n1. The writing and visualization in the paper are clear and straightforward. \n2. The paper effectively argues that the paradigm of zero-shot settings is more suitable for the development of LLM-based Text-to-SQL instead of extensive fine-tuning.\n\nWeaknesses:\n1. Modeling Text-to-SQL as a search problem lacks sufficient theoretical support. Using the MCTS framework to search reasoning paths appears to have limited practical significance in text-to-SQL, as detailed in the above Claims and Evidence and Methods and Evaluation Criteria.\n\n2. The reward model in the proposed MCTS framework is applicable only to the two actions that generate SQL. This methodology is similar to existing approaches with execution feedback, such as CHESS [1] and E-SQL [2].\n\n3. The experimental section lacks a fair comparison with baselines, as most baselines use GPT-4, while Alpha-SQL employs Qwen2.5-Coder. The paper also does not justify this choice, raising concerns about whether the observed SQL generation results heavily depend on the specific LLM used.\n\n4. The core action component, SQL Generation, is based on an existing method. Given that SQL Generation is a critical step in text-to-SQL, relying on existing research like CHASE-SQL (which has achieved a good performance in the leaderboard) raises concerns about the novelty and effectiveness of the proposed framework.\n\n[1] CHESS: Contextual Harnessing for Efficient SQL Synthesis\n[2] E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL", "other_comments_or_suggestions": "N.A.", "questions_for_authors": "Please address the aforementioned questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NoTsXDGnb4", "reviewer": "Reviewer_yCmr", "summary": "This paper proposes a novel approach based MCTS to enhance the zero-shot performance of LLMs in the text to SQL domain. Authors designed a set of task specific actions such as question rephrasing, schema selection, SQL generation, column value identification, column function identification, and SQL revision. Authors used LLM-as-action model to enhance the reasoning capabilities of the LLMs to generate CoT traces based on the context of the problem. Based on the actions spaced defined in the paper, authors proposed using MCTS to effectively generate the SQL queries for the given NL queries. In order to obtain the Q values for each action and node they proposed using self-consistency based query scoring. Their proposed approach has improved the performance of the open-source LLMs mainly from Qwen family and outperformed some of previous works with larger models like GPT family models.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your comprehensive review and for providing a clear summary of our proposed Alpha-SQL method. \nWe understand  the need for comparison against a \"Best-of-N\" baseline, the omission of base model performance for larger Qwen models in Table 4, and the lack of detailed analysis on computational costs (token usage and latency). These are valid points crucial for thoroughly evaluating a test-time compute method like Alpha-SQL. We appreciate the opportunity to address these concerns.\n\nBelow, we address each point in detail:\n\n**1. Comparison with Best-of-N Baseline**\n>Reviewer Concern: \"...a comparison with the best-of-N method (directly generate N candidate SQL queries then use self-consistency to select the most consistent answer) which is the common strong baseline for any test-time compute methods would truly convince me about the effectiveness of this method.\"\n> Reviewer Weakness: \"... a comparison with a baseline like Best-of-N method is very important.\"\n- **Our Response**: Thank you for suggesting the comparison with the Best-of-N baseline. We agree that this is a relevant and strong baseline for evaluating test-time inference methods employing self-consistency. While Best-of-N explores the model's output diversity from a single prompt state, Alpha-SQL uses MCTS to explore diverse reasoning paths before final SQL generation.\n  - Comparison with Best-of-N: \n| Base LLM                     | Direct Prompting | Directly Promoting with Best-of-N | Alpha-SQL |\n|------------------------------|------------------|-----------------------------------|-----------|\n| Qwen2.5-Coder-Instruct-7B    | 48.4%            | 56.3%                             | 66.8%     |\n| Qwen2.5-Coder-Instruct-14B   | 57.4%            | 62.3%                             | 68.7%     |\n| Qwen2.5-Coder-Instruct-32B   | 62.6%            | 63.4%                             | 69.7%     |\n\n  - The results clearly demonstrate that Alpha-SQL significantly outperforms the Best-of-N baseline across all tested Qwen model sizes. For instance, with the 7B model, Alpha-SQL achieves 66.8% EX compared to 56.3% for Best-of-N (+10.5%). Similarly, with the 32B model, Alpha-SQL reaches 69.7% compared to 63.4% (+6.3%). This confirms that the structured reasoning path exploration enabled by Alpha-SQL's MCTS framework offers substantial benefits beyond simply sampling multiple outputs from a fixed initial prompt state, even when using self-consistency for selection. We will add these comparative results and include this analysis in the revised manuscript.\n\n**2. Missing Base Model Performance (Qwen 14B & 32B in Table 4)**\n> Reviewer Concern: \"...the base performance for the larger Qwen 14B and 32B models are not provided in table 4 but I think seeing the performance gain for these models is also important cause I believe the performance gap could be smaller for these models.\"\n> Reviewer Note on Table 4: \"Table 4 has the issue of missing the baseline performance for 14B abd 32B models.\"\n- **Our Response**: Thank you for pointing out the missing base model performance data for the Qwen2.5-Coder-14B and -32B models in Table 4. We completely agree that showing the performance gain achieved by Alpha-SQL over the respective base models is important for a full evaluation across different model sizes, and we appreciate you highlighting this omission. As shown in the table provided in our response to your first point (regarding fair comparison and Best-of-N baselines), we have now completed the evaluation to determine the baseline performance for these models. We will update Table 4 in the revised manuscript to include this complete comparison and add a discussion analyzing these performance gains.\n\n**3. Cost Analysis: Token Usage and Latency**\n> Reviewer Weakness: \"A detailed analysis on the token usage of this method is required...\"\n> Reviewer Weakness: \"A detailed analysis on the latency of this method is also required, most of the text-to-SQL systems require performing in few seconds to be able to useful in real world settings.\"\n- **Our Response**: We understand the need for a detailed analysis of computational costs, specifically token usage and inference latency, to assess the practical viability of Alpha-SQL.  \n-  Due to rebuttal length limitations, we kindly refer the reviewer to our detailed response to **Reviewer guJX, specifically: Point 1 (Fair Comparison...) and Point 2 (Computational Cost Analysis...)**. We believe those sections fully address the concerns regarding token usage and latency analysis.\n\nWe believe these additions will significantly strengthen the paper's evaluation by providing crucial comparative data and cost context. We appreciate the reviewer's constructive feedback, which helps us improve the rigor of our study, and we hope the revised manuscript will meet the standards for acceptance.\n\nSincerely,\n\nThe Authors", "claims_and_evidence": "The main claim about improving the performance of smaller large language models has demonstrated by comparing the performance with current SOTA methods and the base model performance for some models are provided in table 4. However in order to truly demonstrate the impact of their proposed test-time compute approach I believe a comparison with the best-of-N method (directly generate N candidate SQL queries then use self-consistency to select the most consistent answer) which is the common strong baseline for any test-time compute methods would truly convince me about the effectiveness of this method. Additionally, the base performance for the larger Qwen 14B and 32B models are not provided in table 4 but I think seeing the performance gain for these models is also important cause I believe the performance gap could be smaller for these models.", "methods_and_evaluation_criteria": "Most of the SOTA approaches have been considered in this paper, and two famous benchmarks of Spider and BIRD are used.", "theoretical_claims": "Yes, the theoretical claims about the self-supervised rewards and MCTS are aligned with previous works.", "experimental_designs_or_analyses": "Yes, experimental designs and analysis mostly make sense. Table 2 and Table 3 provide the comparison with the SOTA text-to-SQL methods which has latest methods. Table 4 has the issue of missing the baseline performance for 14B abd 32B models. Table 5 also provides a detailed ablation study on action space which shows the importance of the proposed decomposition.", "relation_to_broader_scientific_literature": "The proposed method can be considered as one of novel and effective test-time compute methods to improve zero-shot performance of LLMs. Specifically with open-source models they were able to catch the performance of larger models like GPT family models.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Remaining weaknesses:\n\n1) A detailed analysis on the token usage of this method is required and a comparison with a baseline like Best-of-N method is very important. \n\n2) A detailed analysis on the latency of this method is also required, most of the text-to-SQL systems require performing in few seconds to be able to useful in real world settings.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Alpha-SQL, a framework applying Monte Carlo Tree Search (MCTS) to the zero-shot text-to-SQL task using large language models (LLMs). Instead of a fixed pipeline, Alpha-SQL decomposes the task into several actions (e.g., question rephrasing, schema selection, value identification, SQL generation, revision) and uses MCTS to dynamically explore sequences of these actions ("reasoning paths"). An LLM acts as the model for predicting action outcomes, and a self-supervised reward based on the execution consistency of generated SQL guides the search. The authors report state-of-the-art zero-shot results on the BIRD benchmark using open-source Qwen models. Reviewers acknowledged the novelty of applying MCTS to text-to-SQL, the clarity of the paper, and the potential benefits of adaptive reasoning paths. However, concerns were raised regarding the fundamental necessity and effectiveness of the MCTS framework for this task, the fairness of experimental comparisons (using Qwen models while baselines used GPT-4), the lack of computational cost analysis (latency, token usage), and the effectiveness of the reward signal in guiding earlier, non-SQL-generating actions.

In response, the authors provided substantial clarifications and new experimental results during the rebuttal. They argued that MCTS enables crucial adaptability based on query/schema complexity, unlike fixed pipelines, and clarified that the reward signal is backpropagated to influence all actions. They presented new experiments comparing base LLMs (Qwen vs. GPT-4o showed similar direct prompting performance on BIRD), running key baselines (RSL-SQL, CHESS-SQL) on the same Qwen model (showing Alpha-SQL still outperformed), and comparing against a "Best-of-N" self-consistency baseline (which Alpha-SQL also surpassed). Detailed computational cost analysis (latency, tokens) was provided, acknowledging higher latency for Alpha-SQL but better token efficiency than CHESS-SQL. Analysis of common reasoning paths was also added to demonstrate adaptivity. These rebuttals led one reviewer (R-2yXT) to raise their score from Weak Reject to Weak Accept, while another (R-5kbR) maintained their initial Accept. However, one reviewer (R-guJX) maintained Weak Reject, expressing remaining concerns about the completeness and consistency of the fairness comparisons, while the final reviewer (R-yCmr) did not update their initial Weak Reject score.
Decision: Accept (poster)