TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring

## Abstract

Abstract Text-to-SQL enables users to interact with databases using natural language, simplifying the retrieval and synthesis of information.
Despite the remarkable success of large language models (LLMs) in translating natural language questions into SQL queries, widespread deployment remains limited due to two primary challenges.
First, the effective use of text-to-SQL models depends on users’ understanding of the model’s capabilities—the scope of questions the model can correctly answer.
Second, the absence of abstention mechanisms can lead to incorrect SQL generation going unnoticed, thereby undermining trust in the model’s output.
To enable wider deployment, it is crucial to address these challenges in model design and enhance model evaluation to build trust in the model’s output.
To this end, we introduce TrustSQL , a novel comprehensive benchmark designed to evaluate text-to-SQL reliability —defined as a model’s ability to correctly handle any type of input question by generating correct SQL queries for feasible questions and abstaining from generating infeasible ones ( e.g. , due to schema incompatibility or functionalities beyond SQL).
We evaluate existing methods using a novel penalty-based scoring metric with two modeling approaches: (1) pipeline-based methods combining SQL generators with infeasible question detectors and SQL error detectors for abstention; and (2) unified methods using a single model for the entire task.
Our experimental results reveal that achieving high scores under severe penalties requires significant effort and provide a new perspective on developing text-to-SQL models for safer deployment. TrustSQL is available at https://github.com/glee4810/TrustSQL .

## 1 Introduction

Text-to-SQL is a task that involves generating SQL queries from natural language (NL) questions, a challenge that has persisted for several decades [].
This task is crucial both within and beyond the realm of machine learning and natural language processing (NLP), due to the ubiquity of databases in various fields such as medicine, finance, retail, and telecommunications.
Accessing structured data stored in these databases through natural language can unlock the true potential of big data, enabling domain experts to uncover new quantitative insights [].

To effectively utilize these models, users need to be familiar with their capabilities. These include the types of questions each individual model can accurately answer, the information it has access to, and what operations can or cannot be performed via SQL. However, many potential users who could greatly benefit from the model are unfamiliar with these capabilities and may find it challenging to use, as their questions may frequently fall outside the model’s scope (beyond the correct SQL in Figure 1(b)). The challenge intensifies in real-world deployment scenarios where user questions include not only those feasible for the model but also infeasible ones, such as requests requiring knowledge beyond the database content or SQL functionalities (as shown in Figure 1(a)). Moreover, the absence of abstention mechanisms in current text-to-SQL modeling can lead to incorrect model outputs for both feasible and infeasible questions going unnoticed, potentially outweighing the benefits of model use. If a model, by design, could selectively answer correct SQL queries while abstaining from answering the rest, it would significantly boost user confidence, thereby enhancing trust in the model and encouraging its use [, ].

Figure: (a)
Refer to caption: x1.png

To address the aforementioned challenges, we present TrustSQL, a new benchmark designed to assess text-to-SQL reliability.
We define text-to-SQL reliability as a model’s ability to handle any input questions correctly through accurate SQL generation and abstention, thereby reducing the red and yellow regions by covering the entire region in green, as shown in Figure 1(c)).
Unlike standard text-to-SQL tasks defined within the feasible region (Figure 1(b)), real user questions span both feasible and infeasible task regions, including requests beyond the database content and SQL functionalities (Figure 1(a)).
We constructed this new comprehensive setting using three domain-specific datasets—ATIS [], Advising [], and EHRSQL []—along with one cross-domain dataset—Spider [].
In constructing TrustSQL, we reflected common error cases undermining text-to-SQL reliability through careful data splitting and incorporation of various infeasible questions, similar to red teaming.
We then evaluated existing methods using a novel penalty-based scoring metric designed specifically to measure text-to-SQL reliability (details in Section 3.2).
Our experimental results reveal that while most advanced methods seem to perform well in penalty-less settings, achieving high scores under severe penalty settings remains highly challenging.
From the high penalty perspective, knowing when to abstain precisely carries more weight than rashly attempting to generate SQL queries, underscoring the need for substantial efforts towards gaining trust in model outputs.

## 2 Dataset Construction

In this section, we begin by identifying common error cases that undermine text-to-SQL reliability during deployment, causing answers to reside in the red regions in Figure 1(c). Then, we introduce our strategies to incorporate each case into the dataset construction process. Below are the common error cases in text-to-SQL deployment:

- •
Complexity of SQL generation: User questions, especially those posed by domain experts, are often too complex for models to answer accurately. This complexity arises from the inherent difficulty of the queries [, , , ], as in Table 7 in Supplementary B, or from the model encountering unfamiliar phrases and entities specific to the database [, ]. Models that generate SQL without the ability to abstain from such cases risk compromising trust in their output. We include these dimensions of query difficulty [P1] and familiarity [P2] in feasible data construction.
- •
Ambiguity in user queries: User questions are often ambiguous, particularly when they refer to columns or values (i.e., database records) without precise specification. Resolving the user’s true intent involves multi-turn interactions for clarification, which is beyond the scope of this work.
- •
Schema incompatibility: Users who are unfamiliar with the exact database schema may inadvertently ask questions that cannot be processed because the information they seek does not exist in the existing columns. Previous works construct such cases by deleting and perturbing existing columns [, ]. Instead, we include three distinct types within this case [P3] to provide more granular views on model behavior.
- •
Scope of SQL functionalities: Users may not always be familiar with the scope of SQL functionalities, leading to requests that require complex numerical operations, data analysis, or scheduling appointments, which are beyond standard SQL execution. We include such questions [P4] in infeasible data construction.
- •
Need for external knowledge: Users may ask questions that involve factual world knowledge that goes far beyond the database content, either fully (e.g., ‘What airport is in Wilder, KY?’) or partially (e.g., ‘Find me patients born after WWII’). We consider the former cases [P5] when constructing infeasible questions, while excluding the latter.

Table: Table 1: Sample infeasible questions for each type for the orchestra database in Spider, containing 4 tables with 23 columns. Samples for domain-specific databases are not shown here for brevity. ‘Note’ indicates hypothetical columns in existing tables (i.e., columns not present in the database we use for text-to-SQL) or a task name used for question annotation.

### 2.1 Feasible Data Construction

##### Dataset selection

To establish the benchmark data covering comprehensive settings, we selected three complex domain-specific text-to-SQL datasets, namely, ATIS (flight information) [], Advising (education) [], and EHRSQL (healthcare) [].
Additionally, we included one cross-domain dataset, Spider.
The distinction between domain-specific and cross-domain text-to-SQL settings, along with the complexity of each dataset, is detailed in Supplementary B.

##### Data preprocessing

Our preliminary data analysis revealed that ATIS and Advising require significant preprocessing due to issues such as numerous query duplications, inconsistencies, and poor-quality natural language (NL) questions (each issue discussed in Supplementary G).
We implemented several strategies to correct these issues, including creating new question paraphrases for all input NL questions using GPT-4, followed by a manual review by the authors.
For further details, please refer to Supplementary G).
The databases of the four datasets and their corresponding question-SQL pairs, after preprocessing, are included as questions in the feasible task region (light blue region in Figure 1(a)).
To reflect the query difficulty [P1], we adopted the SQL hardness criteria following [] across all datasets.
Detailed statistics for each hardness level are provided in Table 7 in Supplementary B.
Query familiarity [P2] is later introduced in the data splitting process (§2.3).

**Table 2: Dataset statistics of TrustSQL. TrustSQL includes an equal number of unanswerable and answerable questions in the test sets. For $Q$, the three numbers, separated by ’/’, denote the size of training, validation, and test sets.**
|  | ATIS | Advising | EHRQSL | Spider |
| --- | --- | --- | --- | --- |
| All Questions ($Q\cup Q^{c}$) | 2555 | 2769 | 7473 | 8561 |
| Feasible Question ($Q$) | 1114 / 489 / 476 | 1170 / 533 / 533 | 4674 / 931 / 934 | 7000 / 507 / 527 |
| Infeasible Question ($Q^{c}$) | 476 | 533 | 934 | 527 |
| ^⌞ non-sql | 99 | 110 | 191 | 100 |
| ^⌞ surface | 93 | 102 | 185 | 100 |
| ^⌞ unrelated | 94 | 110 | 187 | 100 |
| ^⌞ related | 98 | 107 | 189 | 100 |
| ^⌞ ext-know | 92 | 104 | 182 | 127 |

### 2.2 Infeasible Data Construction

Text-to-SQL models deployed in real-world scenarios often encounter user requests that fall into the infeasible task region (gray region in Figure 1(a)) due to going beyond the database content or SQL functionalities, and thus do not have corresponding SQL queries.
To evaluate models in a more realistic environment, we manually annotated NL questions by incorporating three common infeasible error cases as follows:

##### Schema incompatibility

[P3]: Instead of randomly dropping existing columns in the database [, ], we introduce the concept of hypothetical columns as target columns for infeasible questions. Those questions are asked as if their corresponding SQL queries exist, but in fact do not, since no such columns exist in the database. We categorized these into three types: column-surface, column-related, and column-unrelated, representing attacks based on surface-form and semantic column similarities and dissimilarities, respectively.

- •
column-surface: Questions referring to hypothetical columns that resemble existing columns in different tables, or have similar surface forms (e.g., a hypothetical ‘age’ column in the ‘Professionals’ table that resembles the ‘age’ column in the ‘Dogs’ table in the ‘dog_kennels’ database).
- •
column-related: Questions referring to hypothetical columns that could naturally integrate into the existing database schema (e.g., a hypothetical column named ‘duration’ in the ‘song’ table). This question type can be seen as a manual version of the ADD perturbations in ADVETA [].
- •
column-unrelated: Questions referring to hypothetical columns that are randomly sampled from other databases, which are mostly unnatural (e.g., a hypothetical ‘product_category’ column in the ‘battles’ database).

##### Beyond SQL functionalities

[P4]: To capture diverse user requests that exceed SQL execution (non-sql), we compiled a list of NLP and data analytics tasks and used GPT-4 to identify what could be done as an AI assistant if any tools can be used given the database content. Based on a list of 27 tasks (e.g., anomaly detection, visualization, and web search), we manually annotated NL questions relevant to or referring to existing columns in each database (thus, context-driven annotation), simulating users expecting the model to perform tasks not solely addressed through SQL.

##### Need for external knowledge

[P5]: To construct NL questions involving external knowledge (ext-know), we prompted GPT-4 to generate questions that require domain expertise, qualitative insights, or causal understanding. We continued this process until we achieved a balanced set of samples comparable to other types of infeasible questions.

After manual annotation of infeasible NL questions, the authors used GPT-4 to iteratively review them to ensure that they were truly infeasible.
The sample annotated questions are provided in Table 1.

### 2.3 Data Splitting

The core principle for data splitting is to create varying query familiarity [P2] for feasible data and introduce the unexpectedness of infeasible data for model evaluation. Table 2 summarizes the overall data statistics in TrustSQL.

##### Feasible data

We define ‘seen SQL’ and ‘unseen SQL’ to introduce query familiarity [P2] (only applicable within feasible data).
Samples are considered ‘seen SQL’ when their corresponding SQL structures(^3^33A SQL structure refers to a specific combination of SQL components with specific tables and columns, excluding actual data values.) are present in the training set but have different paraphrased NL questions in the validation and test sets.
‘Unseen SQL’ samples have SQL structures not present in the training set.
We use these different query familiarities [P2] included in the validation and test sets to analyze model performances.
The data splitting process for domain-specific datasets is as follows: 1) Split feasible data into training, validation, and test sets in a 6:2:2 ratio based on NL questions (same SQL structures across splits, but different NL questions); 2) Remove 20% of non-overlapping SQL structures from the validation and test sets; 3) Remove the total 40% of these SQL structures from the training set, resulting in 60% ‘seen SQL’ in all splits, while validation and test sets contain 80% SQL structures (60% ‘seen SQL’ and 20% ‘unseen SQL’). For Spider, we divide the development split in half for new validation and test sets (10 databases each), all ‘unseen SQL,’ with no overlaps with the training set.

##### Infeasible data

We add the annotated infeasible NL questions of five types (column-surface, column-related, column-unrelated, non-sql, and ext-know) all to the test set in balanced proportions to reflect their unexpected encounter in deployment.

## 3 Benchmarking Text-to-SQL Reliability

### 3.1 Task

Reliable text-to-SQL aims to develop models that only generate correct SQL queries while abstaining in any other cases, including both feasible ($\mathcal{Q}$) and infeasible questions ($\mathcal{Q}^{c}$), to enhance text-to-SQL reliability.
Optimized under this principle, users can effectively use these models without having extensive knowledge about the models’ capabilities.
They can also be less concerned about the model producing incorrect outputs, as it would abstain from answering questions beyond its capabilities.
This increased reliability will lead to wider adoption of text-to-SQL models, empowering domain experts to utilize natural language interfaces to access and analyze the vast amounts of data stored in databases.

### 3.2 Evaluation Metric

In evaluating reliable text-to-SQL performance, we categorize the model’s outputs into three distinct categories: desirable (green), undesirable (red), and neutral (yellow), as illustrated in Figure 1(c).
These categories reflect different types of consequences brought about by the model’s outputs.
Desirable outputs, providing utility, include correct SQL generation (I) and abstentions (V).
However, the severity of the consequences of undesirable outputs can vary based on the context of the deployment.
In less critical settings (e.g., experimental scenarios), we might be lenient towards these undesirable outputs and focus solely on the model’s utility.
In contrast, in high-stakes environments (e.g., hospitals), undesirable outputs can have severe repercussions, potentially outweighing any utility provided by the model.
Therefore, we define the relative magnitude of the negative impact compared to the positive impact as risk.
This risk includes incorrect SQL generations (III) and abstentions (IV), weighted by a penalty factor, $c$, namely the ratio of the negative impact to the positive impact.
Neutral (yellow) cases (II) are neither strictly desirable nor strictly undesirable; therefore, we consider them neither utility nor risk.
Examples of instances for each case are provided in Appendix E.

In summary, text-to-SQL reliability can be quantified as the difference between utility and risk. To formalize this notion mathematically, we define this penalty-based scoring for text-to-SQL—reliability score (RS)—as follows:

$$ $\phi_{c}(x)=\begin{cases}1&\text{if $x\in\mathcal{Q}$; $g(x)=1$; $Acc(x)=1$,} \\ 0&\text{if $x\in\mathcal{Q}$; $g(x)=0$,}\\ -c&\text{if $x\in\mathcal{Q}$; $g(x)=1$; $Acc(x)=0$,}\\ -c&\text{if $x\in\mathcal{Q}^{c}$; $g(x)=1$,}\\ 1&\text{if $x\in\mathcal{Q}^{c}$; $g(x)=0$,}\\ \end{cases}$ (1) $$

where $Acc(x)$ is the execution accuracy in text-to-SQL and $g(x)=1$ and $g(x)=0$ indicate the model’s decision to answer or abstain, respectively.

For each sample, a score of 1 is assigned when the SQL is generated correctly for $x\in\mathcal{Q}$ (I) or when the model abstains for $x\in\mathcal{Q}^{c}$ (V), indicating utility.
A score of 0 0 is given when the model decides not to generate an SQL for $x\in\mathcal{Q}$ (II), indicating no risk and no utility.
A penalty of $-c$ is assigned when the generated SQL is incorrect (III) or when the model attempts to generate an SQL for $x\in\mathcal{Q}^{c}$ (IV), indicating risk.
Actual examples of each case are provided in Supplementary E.
After calculating each sample-level score, the aggregate RS can be computed by taking the mean, $\frac{1}{N}\sum_{x}\phi_{c}(x)$, resulting in $\phi_{c}$.

In our experiments, we consider three options for the penalty value $c$: 0, 10, and $N$, where $N$ represents the total number of samples to be evaluated.

- •
$c=0$: The most lenient scenario. This does not penalize any mistakes made by the model. If we only handle feasible questions, this score is equivalent to the standard text-to-SQL accuracy.
- •
$c=10$: A moderately strict scenario. Every 10 correct answers weigh the same as one incorrect answer.
- •
$c=N$: The most strict scenario. A single mistake, even if all other answers were correct, leads to a negative score.

An RS($c$) of 100% indicates perfect text-to-SQL reliability, meaning the model can handle all inputs correctly (i.e., covering all the regions with green in Figure 1(c)). RS($c$) > 0 means the utility is greater than the risk, making it deployable in the context of text-to-SQL reliability.

## 4 Experiments

### 4.1 Modeling

In this section, we describe various methods for developing models to enhance text-to-SQL reliability. We categorize the methods into two broad modeling approaches: pipeline-based and unified. For each approach, we develop both open-source and commercial API-based models, as these choices are often utilized one or the other depending on various factors in deployment, such as budget constraints, technical expertise, and privacy concerns.

Table: Table 3: Model comparison in TrustSQL involving both feasible and infeasible questions evaluated in $\phi_{{}_{0}}$, $\phi_{{}_{10}}$, and $\phi_{{}_{N}}$. AT, AD, EH, and SP refer to ATIS, Advising, EHRSQL, and Spider, respectively. Abstain-All blindly abstains from predicting all input questions, reflecting the ratio of infeasible questions. All in % for readability.

#### 4.1.1 Pipeline-based Approach

In conventional NLP task formulations, enhancing text-to-SQL reliability can be addressed by combining three different tasks: infeasible question detection [, ], SQL generation [, ], and SQL error detection [].

##### Open-source

For SQL generation, we use T5 (T5-3B), feeding questions followed by a serialized schema listing all tables and columns []. To incorporate abstention mechanisms, we use SQLCoder (sqlcoder-7b-2) for infeasible question detection (before SQL generation) and error detection (after SQL generation). For infeasible question detection, we fine-tune SQLCoder on TriageSQL [], denoted as CLSM. For error detection, we train SQLCoder based on the generated SQLs and their sample-level accuracy from trained T5 on the TrustSQL validation set plus samples of infeasible questions in TriageSQL, denoted as ErrorM. Details about training these models are provided in Supplementary C.
When denoting the combined methods, we use arrows to indicate data flow between models, such as CLSM → T5 indicating that question detection comes first, followed by SQL generation.

##### API-based

For SQL generation, we employ three methods using GPT-3.5 (gpt-3.5-turbo-0125): SQLPrompt, SQL generation with few-shot prompting [], DIN-SQL [], a state-of-the-art task decomposition framework in Spider, and MAC-SQL [], a state-of-the-art multi-agent framework in BIRD [].
We adjust text prompts for DIN-SQL and MAC-SQL for domain-specific datasets.
For abstention, we use GPT-4o for both infeasible question detection and error detection. We provide the modeling details in Supplementary D.

#### 4.1.2 Unified Approach

To achieve text-to-SQL reliability using a single model, we explore various text-to-SQL prompts that incorporate abstention mechanisms. Additionally, we utilize multiple uncertainty estimation methods, leveraging SQL generators’ hidden representations and probabilities, to enable abstention.

##### Open-source

For SQL generation, we use T5 (T5-3B) due to its wide use in selective generation [] and calibration [] in NLP. For abstention, we leverage the hidden representations of the input and the output probabilities by adopting uncertainty estimation to detect samples that are out-of-distribution (OOD) for abstention. Below are the methods we use:

- •
Entropy-based: Each sample is assigned the maximum entropy value among the generated SQL tokens, denoted as MaxEnt [].
- •
Probability-based: Each sample is assigned the minimum of the highest token probabilities from the token sequence, denoted as MaxProb [].
- •
Distance-based: Mahalanobis distance calculation based on the distribution of the training data representations, denoted as FeatMD [, ], and relative Mahalanobis distance calculation with a background distribution using the TriageSQL dataset, denoted as FeatRMD [, ].

These methods require calibrating thresholds to draw a decision boundary between in-distribution and out-of-distribution. Our proposed heuristic method is provided in Supplementary C.2.

Table: Table 4: Model comparison in TrustSQL using only feasible data. In this table, $\phi_{{}_{0}}$ is equivalent to execution accuracy in standard text-to-SQL. The scores in shaded cells represent SQL performance without abstention.

##### API-based

We use two prompting methods with GPT-3.5, as explained below:

- •
SQLPrompt[DEMO]: Few-shot demonstrations of similar feasible questions with their corresponding SQL queries and similar infeasible questions, each labeled with ‘not_answerable’ []. The order of demonstrations is randomly chosen.
- •
SQLPrompt[Voting]: Using the same prompts as SQLPrompt, we sample the answers five times [] and consider the SQL generation only if all samples unanimously agree; otherwise, we abstain.

## 5 Results

### 5.1 Model Comparison

##### Full Data

Table 3 shows the performance of each method in the TrustSQL benchmark. For open-source models, the probability-based (i.e., T5[MaxEnt]) and entropy-based (i.e., T5[MaxProb]) methods in the unified approach consistently outperform other methods across different penalty settings. For API-based models, CLSP → SQLPrompt performs best in RS(0), while CLSP → SQLPrompt → ErrorP excels in stricter settings (i.e., RS(10) and RS(N)). This indicates that both question detection and error detection are crucial for enhancing text-to-SQL reliability.
SQLPrompt[Voting] shows notable performance across all penalties, particularly given its simplicity.

##### Feasible Data Only

Evaluating models on feasible data measures unnecessary abstention and the impact of error detectors, indicated by the performance drop between the SQL performance without abstention (shaded cells) and the models with abstention in RS(0). Similar to the full data setting, open-source models, T5[MaxEnt] and T5[MaxProb], tend to perform best with increasing penalties.
API-based models using the unified approach, particularly SQLPrompt[Voting], outperform SQLPrompt[Demo] as the penalty increases. High performance in RS(0) indicates frequent abstention, resulting in lower performance with higher penalties. A more detailed analysis of each component in pipeline-based methods is provided in Supplementary F.

### 5.2 Discussion

##### Q1: Why is All-abstain considered deployable in text-to-SQL reliability?

In the context of standard text-to-SQL, All-abstain in Table 3 cannot be considered deployable since it contradicts the very purpose of the task. However, in the context of text-to-SQL reliability, All-abstain achieves an RS of 50 on all $c$ by correctly handling 50% of all input questions (i.e., the proportion of infeasible questions in TrustSQL) with certainty. This approach may be more reliable (absolutely no risk in all settings) than a model optimized for standard text-to-SQL, which always generates SQL queries that may be incorrect (risk potentially higher than utility).

##### Q2: What is an easy way to enhance text-to-SQL reliability?

An easy way to enhance the RS to at least a deployable level (i.e., >0) in strict settings is to generate only those SQL queries for which the model has high confidence, while abstaining from generating any when uncertain. The experimental results demonstrate that this approach is particularly effective in settings where incorrect generation carries a high penalty. For instance, as shown in Table 3, the T5[FeatMD] model on Spider generates only one correct SQL query and abstains from the rest due to our strict heuristic thresholding (see Supplementary C.2). Efforts to reduce risk can gain more RS scores than efforts to increase utility. However, achieving higher RS (i.e., >50%) with such cautious SQL generation requires another line of effort, which has not been explored extensively.

##### Q3: Why is TrustSQL necessary as a text-to-SQL benchmark?

Existing benchmarks lack penalty-based scoring and considerations for infeasible questions. TrustSQL addresses these gaps by modeling SQL generation with abstention and handling error cases from infeasible question detection, which are then propagated to SQL generation and error detection. This comprehensive approach encourages new modeling aspects to enhance reliability.
For instance, the entries column-surface and non-sql in Table 1 are examples that fail to be handled using GPT-4o-based infeasible question detection and error detection methods (i.e., CLSP → SQLPrompt → ErrorP).
Conventional task settings cannot explore such cases, showing how these models respond to infeasible questions. For instance, the query ‘SELECT avg(attendance) FROM show’ is generated for the question ‘What is the average attendance capacity for all the shows?’ (where only the attendance column exists, not the capacity), returning a result of 1326.4 from the database.
TrustSQL’s comprehensive approach provides a new perspective on error cases in text-to-SQL generation, encouraging the development of more reliable models.

## 6 Related Works

##### Model Reliability in Text-to-SQL

Text-to-SQL has attracted significant attention for its practical applications. Key single-domain datasets include ATIS [], GeoQuery [], Advising [], SEDE [], and EHRSQL []. For cross-domain tasks, Spider [] and BIRD [] serve as essential benchmarks for SQL generation.
However, these benchmarks assume that all input questions should generate SQL, lacking incentives for developing models to abstain. This absence of safety measures undermines reliable deployment. Some efforts focus on detecting infeasible questions given database schemas [, ], but they do not integrate SQL generation or consider the error-handling capabilities of text-to-SQL models. Our work aims to integrate comprehensive reliability measures, aligning the benchmark more closely with model trust.

##### Uncertainty Estimation in NLP

Uncertainty estimation is vital for AI reliability []. The methods include probability-based methods (calculating entropies or model-assigned probabilities)[, , , ] and distance-based methods (using metrics such as the Mahalanobis distance to compare new inputs with training data)[, , , ]. Probability-based methods gauge model confidence, enhancing output reliability. Distance-based methods identify outliers likely to produce low-quality outputs, promoting safer AI deployment. Our work employs both approaches to develop abstention mechanisms for text-to-SQL models.

## 7 Conclusion and Future Direction

In this work, we present TrustSQL, a new benchmark designed to assess text-to-SQL reliability, defined as models’ ability to correctly handle any type of input question by generating correct SQL queries for feasible questions and abstaining from infeasible ones. To demonstrate this, we constructed the benchmark based on common error cases that users unfamiliar with the model’s capabilities may encounter.
We developed and assessed two approaches: pipeline-based and unified methods, using existing modeling strategies. Our experimental results, using a penalty-based scoring metric, show that addressing this task requires various modeling approaches and opens new avenues for model development.

Though we have carefully designed the benchmark, there are several limitations. First, despite our efforts to cover a wide range of infeasible questions, our benchmark does not encompass all those found in real-world scenarios. Secondly, our work focuses on single-turn text-to-SQL tasks and does not address the complexities of disambiguating uncertainty in entity and schema linking or user intents. Finally, results of commercial API-based models in our experiment may not be reproducible if any updates are made.

Future research directions include developing agent-based methods that can plan ahead to achieve high performance in RS(N), bypassing the significant need for strategies such as prompt engineering and domain adaptation, accelerating wider adoption of text-to-SQL across diverse fields.

## Supplementary Contents

## Appendix A Datasheet for Datasets

[supplementary]

### A.1 Motivation

- •
For what purpose was the dataset created?
TrustSQL is designed to evaluate text-to-SQL reliability—defined as a model’s ability to correctly handle any type of input question by generating correct SQL queries for feasible questions and abstaining from generating infeasible ones (e.g., due to schema incompatibility or functionalities beyond SQL). By incorporating both aspects of feasible and infeasible questions in the benchmark, we aim to assess existing models’ ability to gain trust in their outputs through appropriate abstention, encouraging wider adoption of text-to-SQL models across various fields beyond experimental settings.
- •
Who created the dataset (e.g., which team, research group) and on
behalf of which entity (e.g., company, institution, organization)?
The authors of this paper.
- •
Who funded the creation of the dataset? If there is an associated grant, please provide the name of the grantor and the grant name and number.
This work was supported by Institute for Information & communications Technology Promotion(IITP) grant (No.RS-2019-II190075) and the National Research Foundation of Korea (NRF) grant (NRF-2020H1D3A2A03100945) funded by the Korea government (MSIT).

### A.2 Composition

- •
What do the instances that comprise the dataset represent (e.g., documents, photos, people, countries)?
TrustSQL contains natural language questions and their corresponding SQL queries (if feasible); infeasible questions do not have the corresponding SQL.
- •
How many instances are there in total (of each type, if appropriate)?
There are about 21.4K data instances (18.9K feasible; 2.5K infeasible).
- •
Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set?
This is the complete data for TrustSQL.
- •
What data does each instance consist of?
TrustSQL contains two sets of data: question-SQL pairs for feasible questions and questions only for infeasible questions.
- •
Is there a label or target associated with each instance?
Labels are SQL queries.
- •
Is any information missing from individual instances? If so, please provide a description, explaining why this information is missing (e.g., because it was unavailable). This does not include intentionally removed information, but might include, e.g., redacted text.
N/A.
- •
Are relationships between individual instances made explicit (e.g., users’ movie ratings, social network links)?
N/A.
- •
Are there recommended data splits (e.g., training, development/validation, testing)?
We provide pre-split datasets.
- •
Are there any errors, sources of noise, or redundancies in the dataset?
We did our best to ensure each data instance best serves its purpose without errors or noise. Any unexpected errors or noise may occur in the future, and we will be willing to correct them if detected.
- •
Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)?
The dataset is self-contained if users comply with the licenses in the downloadable link.
- •
Does the dataset contain data that might be considered confidential (e.g., data that is protected by legal privilege or by doctor– patient confidentiality, data that includes the content of individuals’ non-public communications)?
N/A.
- •
Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety?
N/A.
- •
Does the dataset relate to people?
The EHRSQL portion is related to people.
- •
Does the dataset identify any subpopulations (e.g., by age, gender)?
No.
- •
Is it possible to identify individuals (i.e., one or more natural persons), either directly or indirectly (i.e., in combination with other data) from the dataset?
No.
- •
Does the dataset contain data that might be considered sensitive in any way (e.g., data that reveals race or ethnic origins, sexual orientations, religious beliefs, political opinions or union memberships, or locations; financial or health data; biometric or genetic data; forms of government identification, such as social security numbers; criminal history)?
The EHRSQL portion of question-SQL pairs is linked to the MIMIC-IV Demo, which is already de-identified following HIPAA standards and completely openly available to anyone who complies with its license.

### A.3 Collection Process

- •
How was the data associated with each instance acquired?
We curated and modified question-SQL pairs in TrustSQL based on the existing text-to-SQL datasets and their databases (feasible questions). Infeasible questions were manually annotated by the authors based on the existing databases.
- •
What mechanisms or procedures were used to collect the data (e.g., hardware apparatuses or sensors, manual human curation, software programs, software APIs)?
We mainly used Excel, Google Sheets, and Python to collect, process, and annotate the data. In addition, we used OpenAI’s GPT-4 to generate paraphrases for each question template.
- •
If the dataset is a sample from a larger set, what was the sampling strategy (e.g., deterministic, probabilistic with specific sampling probabilities)?
N/A.
- •
Who was involved in the data collection process (e.g., students, crowdworkers, contractors) and how were they compensated (e.g., how much were crowdworkers paid)?
There were three parts that required human involvement in the data collection process: preprocessing question-SQL data, question paraphrasing, and infeasible question annotation for each database. In all cases, the authors conducted all the processes with the help of GPT-4 if necessary. If GPT-4 was used, the authors manually reviewed all the data to maintain high quality.
- •
Over what timeframe was the data collected?
N/A.
- •
Were any ethical review processes conducted (e.g., by an institutional review board)?
N/A.
- •
Does the dataset relate to people?
The EHRSQL portion is related to people.
- •
Did you collect the data from the individuals in question directly, or obtain it via third parties or other sources (e.g., websites)?
N/A.
- •
Were the individuals in question notified about the data collection?
N/A.
- •
Did the individuals in question consent to the collection and use of their data?
N/A.
- •
If consent was obtained, were the consenting individuals provided with a mechanism to revoke their consent in the future or for certain uses?
N/A.
- •
Has an analysis of the potential impact of the dataset and its use on data subjects (e.g., a data protection impact analysis) been conducted?
The EHRSQL database we used is already de-identified.

### A.4 Preprocessing/cleaning/labeling

- •
Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)?
Extensive preprocessing was done to construct TrustSQL. Details are provided in Section G.
- •
Was the “raw” data saved in addition to the preprocessed/cleaned/labeled data (e.g., to support unanticipated future uses)?
N/A.
- •
Is the software that was used to preprocess/clean/label the data available?
Preprocessing, cleaning, and annotation are done using Excel, Google Sheets, and Python.

### A.5 Uses

- •
Has the dataset been used for any tasks already?
Raw datasets are used for text-to-SQL modeling.
- •
Is there a repository that links to any or all papers or systems that use the dataset?
The raw version of ATIS and Advising is available at [https://github.com/jkkummerfeld/text2sql-data](https://github.com/jkkummerfeld/text2sql-data). EHRSQL is available at [https://github.com/glee4810/ehrsql-2024](https://github.com/glee4810/ehrsql-2024). Spider is available at [https://github.com/taoyds/spider](https://github.com/taoyds/spider).
- •
What (other) tasks could the dataset be used for?
N/A.
- •
Is there anything about the composition of the dataset or the way it was collected and preprocessed/cleaned/labeled that might impact future uses?
N/A.
- •
Are there tasks for which the dataset should not be used?
N/A.

### A.6 Distribution

- •
Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf of which the dataset was created?
No.
- •
How will the dataset will be distributed (e.g., tarball on website, API, GitHub)?
The dataset is released at [https://github.com/glee4810/TrustSQL](https://github.com/glee4810/TrustSQL).
- •
When will the dataset be distributed?
Now.
- •
Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of use (ToU)?
The dataset is released under the CC-BY-4.0 license.
- •
Have any third parties imposed IP-based or other restrictions on the data associated with the instances?
No.
- •
Do any export controls or other regulatory restrictions apply to the dataset or to individual instances?
No.

### A.7 Maintenance

- •
Who will be supporting/hosting/maintaining the dataset?
The authors of the paper.
- •
How can the owner/curator/manager of the dataset be contacted (e.g., email address)?
Contact the first author ([gyubok.lee@kaist.ac.kr](gyubok.lee@kaist.ac.kr)) or other authors.
- •
Is there an erratum?
No.
- •
Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete instances)?
If any correction is needed, we plan to upload a new version.
- •
If the dataset relates to people, are there applicable limits on the retention of the data associated with the instances (e.g., were the individuals in question told that their data would be retained for a fixed period of time and then deleted)?
N/A
- •
Will older versions of the dataset continue to be supported/hosted/maintained?
We plan to maintain only the newest version.
- •
If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for them to do so?
Contact the authors of the paper.

[supplementary]

## Appendix B Text-to-SQL Dataset Complexity

### B.1 Dataset Selection for Seed Feasible Questions

Table: Table 5: Open-source text-to-SQL datasets in the literature.

##### Single-domain

We select datasets that feature complex SQL queries (Avg. SQL Length) and database schemas (# Table/DB and # Column/DB). SEDE [] is excluded from our dataset selection because its SQL queries are too inconsistent (raw queries written by real users at Stack Exchange) and lack accompanying executable databases. As a result, we selected ATIS, Advising, and EHRSQL, representing diverse domains with complex queries.

##### Cross-domain

We select Spider over other datasets for the following reasons: WikiSQL is limited by single-table questions (# Table/DB) and simplified SQL queries. KaggleDBQA offers realistic questions based on actual Web databases, but its schema is considered simple (# Table/DB) for our needs. BIRD features questions and complex SQL queries with their mapping evidence for SQL generation. However, this reliance on evidence diverges from our objective to include both feasible and infeasible questions, leading us to exclude it from our dataset selection. Therefore, Spider is the most suitable choice for our research.

### B.2 Query Difficulty in TrustSQL

For assigning query difficulty [P1], we adopted the SQL hardness criteria following DIN-SQL [] across all datasets, classifying samples into three levels: easy, medium, and hard. The easy category includes single-table queries that do not require joins or nesting. The medium category includes queries that involve joins (including implicit joins) but excludes nesting. The hard category covers queries that can contain joins, sub-queries, and set operations.

Table 6 shows the data statistics for each hardness level for feasible questions. Infeasible questions do not have hardness levels but can be categorized into five question types (statistics in Table 2) used for their annotations. Samples of feasible questions and their corresponding SQL queries by hardness level are in Table 7.

Table: Table 6: Data statistics for feasible questions by hardness level. EHRSQL contains no medium-level samples because its annotation process avoided using joins and used nesting instead.

Table: Table 7: Sample feasible questions. SQL queries in domain-specific datasets are very long, posing a significant challenge in handling lengthy queries for the model. In contrast, those in cross-domain datasets are relatively short but require database generalization. EHRSQL does not have any medium-difficulty samples.

## Appendix C Modeling Details of Open-Source Models

### C.1 Pipeline-based Approach

##### SQL Generation ( T5 )

We use T5-3B as the backbone model. The input consists of questions followed by a serialized schema listing all tables and columns for each database []. The model is trained using BF16 with the Adam optimizer at a learning rate of 0.0001, continuing until the validation loss stops decreasing. Each model is independently trained on four different datasets using NVIDIA RTX A6000 GPUs.

##### Infeasible Question Detection ( CLS M )

We fine-tune sqlcoder-7b-2 for classifying question types (small talk, ambiguous, lack of data, unanswerable by SQL, and answerable) using the TriageSQL [] training set. This task is trained for one epoch on NVIDIA RTX A6000 GPUs and evaluated on TrustSQL’s test set, categorizing answerable questions as feasible and others as infeasible.

##### SQL Error Detection ( Error M )

We also utilize sqlcoder-7b-2 for detecting SQL errors. The training data is derived from the sample-level accuracy of SQL generation for TrustSQL’s validation set, combined with sampled infeasible data from TriageSQL (the same number of data points as the TrustSQL validation set). Correct labels correspond to accurate SQL generation for feasible questions in the TrustSQL validation set, while incorrect labels include both inaccurate SQL generation for feasible questions in the TrustSQL validation set and infeasible questions from TriageSQL. This task is also trained for one epoch on NVIDIA RTX A6000 GPUs.

### C.2 Unified Approach

We leverage the same T5 model used in the pipeline-based approach for the unified approach. The methods for abstention in this approach involve probability-based (i.e., MaxEnt and MaxProb) and distance-based (i.e., FeatMD and FeatRMD) methods, which require fixed thresholds to draw the boundary between SQL generation and abstention from the model outputs.

##### Threshold Selection Method

Figure 2 shows the steps for our heuristic threshold method, and Figure 3 presents the visualization of the thresholds for MaxEnt, MaxProb, FeatMD, and FeatRMD.

Figure: Figure 2: Samples sorted in descending confidence. 14 samples are illustrated in the figure.
Refer to caption: x4.png

Figure: (a) T5[MaxEnt]
Refer to caption: extracted/5705204/figure/maxent.png

## Appendix D Modeling Details of Commercial API-based Models

### D.1 Pipeline-based Approach

##### SQL Generation

For all methods described below, we used GPT-3.5 (gpt-3.5-turbo-0125) instead of GPT-4 due to budget constraints. We considered using GPT-4o, but the proposed prompts for DIN-SQL and MAC-SQL were not compatible with the new model in a stable manner, particularly due to model output formatting issues. Therefore GPT-4o is used only for CLSP and ErrorP.

- •
SQLPrompt [] We use the same schema representation (Columns=[]+FK) as DIN-SQL. For question representation following the schema, we use a retrieval-based prompt (8 samples x 1 database) with a dense retriever (all-mpnet-base-v2) for the single-domain settings, while a fixed prompt (8 samples x 5 databases, randomly selected) is used for the cross-database setting.
- •
DIN-SQL [] We adjust prompts for single-domain datasets, except for the self-correction module at the end. This step is necessary because the original prompts are limited to the SQL structures and syntax found in Spider, making them insufficient for handling the complexity of domain-specific text-to-SQL data.
- •
MAC-SQL [] We adjust the selector and decomposer prompts, except for the refiner, for both single-domain and cross-domain datasets according to their database schemas and samples as task demonstrations.

##### Infeasible Question Detection ( CLS P )

We use CreateTable + SelectCol3 to represent database schemas. This method has shown the best performance among various schema representation choices []. For the single-domain settings, we utilize 16 feasible questions (retrieved from both the TrustSQL training and validation sets) and 16 infeasible questions (retrieved from TriageSQL) with a dense retriever (all-mpnet-base-v2), following the schema representation. For the cross-domain setting, we utilize two feasible and two infeasible questions (retrieved from TriageSQL) from 8 different random databases.

##### SQL Error Detection ( Error p )

This model is designed to hold back the model output if any errors exist in the generated SQL. The prompt is a modified version of the self-correction module in DIN-SQL [], starting from Columns=[]+FK as schema representation, followed by the input question and generated SQL. The instruction is: Based on the question and predicted SQL, are you sure the SQL below is correct? If you consider the SQL to be correct, answer with ‘correct’. If not, answer with ‘incorrect’. Only output your response without explanation.

### D.2 Unified Approach

Based on the prompt used in SQLPrompt, SQLPrompt[Demo] uses the same number of infeasible questions as a few-shot demonstration (these samples are randomly shuffled with feasible ones), while SQLPrompt[Voting] uses exactly the same prompt as SQLPrompt but is sampled five times to check for unanimity in SQL generation; otherwise, it abstains.

## Appendix E Qualitative Analysis for Reliability Scores

Figure: (a) Example of (I). A correct SQL query is generated for a feasible question.
Refer to caption: x5.png

Figure: (d) Example of (IV). A SQL query is generated for an infeasible question.
Refer to caption: x8.png

## Appendix F Detailed Model Performance

### F.1 Performance of Correct SQL among all Model Outputs

While we evaluate model performance in terms of the green region over the whole region in Figure 1(c) (I+V)/(I+II+III+IV+V) and through the scores provided in Table 3 under different penalties, we take a different view: the ratio of correct SQL to all SQL generation (i.e., model output without abstention) (I)/(I+III+IV), as shown in Table 8. In high-penalty scenarios, SQL generation should be done conservatively only if it is certain, and this ratio inversely indicates the extent of incorrect SQL generation going unnoticed by each model.

Table: Table 8: The first line of each cell indicates the ratio of correct SQL to all SQL generations. The second line indicates the number of correct SQL divided by the sum of generated SQL without abstention for both feasible and infeasible questions.

The results in Table 8 that (1) unified methods tend to have higher ratios of correct SQL per total number of generations compared to pipeline-based methods, and (2) models that are conservative (with low values in the denominators) do not necessarily have the highest ratios of correct SQL.

### F.2 Performance of Infeasible Question Detection for Pipeline-based Methods

Tables 9 and 10 show the performance of infeasible question detection by pipeline-based methods.

Table: Table 9: Performance of infeasible question detection for CLSM and CLSP by different hardness levels in feasible questions. The scores indicate the ratios of feasible questions not being filtered.

The results in Table 9 indicate that (1) the performances do not necessarily correlate with the hardness level, as the levels are assigned based on the query difficulty [P1], not on the question in natural language. (2) In most cases, seen questions tend to work better than unseen questions because similar questions were either trained or seen as demonstrations by the model.

Table: Table 10: Performance of infeasible question detection for CLSM and CLSP by five question types used for infeasible question annotation. The scores indicate the ratios of infeasible questions being filtered.

The results in Table 10 indicate that (1) schema incompatibility attacks, in order of difficulty, are surface > unrelated > related; (2) CLSP performs well, especially in non-sql; and (3) ext-know is generally well filtered out by both methods.

### F.3 Performance of SQL Generation for Pipeline-based Methods

Table: Table 11: Performance of SQL generators on feasible questions only, without any abstention mechanisms.

The results in Table 11 indicate that (1) the performances generally correlate with the hardness level, suggesting the soundness of the criteria of hardness we set. However, MAC-SQL and T5 are unable to generate any correct SQL queries for unseen questions on ATIS, especially the easy samples. (2) In most cases, seen questions tend to work better than unseen questions because similar questions and SQL pairs are either trained or seen as demonstrations by the model.

### F.4 Performance of Error Detection for Pipeline-based Methods

Table 12 shows the performance of each method according to these scoring rules: 1) +1 for correct SQL generation without abstention, 2) +1 for incorrect SQL generation with abstention, and 3) +0 for all other cases. This scoring focuses on measuring desirable abstention scenarios for feasible questions. Table 13 shows the performance of each method according to these scoring rules: 1) +1 for abstention and 2) +0 otherwise. This scoring is designed to evaluate the methods’ ability to correctly abstain from answering infeasible questions.

Table: Table 12: Performance of error detection on feasible questions.

We found that (1) the performances do not necessarily correlate with the hardness level for T5 → ErrorM, but the performance is generally higher as the hardness level increases for SQLPrompt → ErrorP, and (2) the performances of seen questions were higher than those of the unseen counterparts for each dataset.

Table: Table 13: Performance of error detection on infeasible questions.

The results in Table 13 indicate that (1) schema incompatibility attacks, in order of difficulty, are surface > unrelated > related; (2) T5 → ErrorM detects non-sql and ext-know much better than SQLPrompt → ErrorP.

## Appendix G Data Preprocessing

In analyzing the ATIS and Advising datasets(^4^44We utilized the improved version of the original datasets, where the questions are grouped together in terms of the same SQL structures [].), we observed several annotation issues: query duplication, limited paraphrase diversity per SQL query, null results, and data inconsistencies in both NL and SQL data. In this section, we outline the preprocessing steps taken to address these issues.

##### Query duplication

Significant query duplication was found in both ATIS and Advising, where questions conveying the same meaning are matched with differently labeled SQL structures []. While these are not critical in terms of data quality, this may result in misleading outcomes when evaluating model performance on ‘seen SQL’ and ‘unseen SQL’ settings after data splitting. To address this, we identified NL paraphrases sharing identical placeholders (e.g., ‘I need a flight from city_name1 to city_name0’ and ‘Which airline provides service in city_name0 and city_name1’ both use city_name0 and city_name1 as placeholders.) and then assessed their semantic equivalence. For any SQL structures with identical semantics, we kept the one with more NL paraphrases and merged the others into it.

##### Limited paraphrase diversity

Our findings indicate that over 90% of SQL queries in ATIS have fewer than 10 paraphrases per SQL query (e.g., ‘I need a flight from city_name1 to city_name0’ and ‘Give me a list of flights going from city_name1 to city_name0’ are two different paraphrases for the same SQL structure). Additionally, both ATIS and Advising exhibit limited diversity and quality in NL questions. To improve question quality, we used ChatGPT to generate 10 paraphrases for each SQL query, followed by manual review to ensure they reflect the intended SQL meaning.

During our review, we specifically focused on avoiding inconsistencies. We noticed that the original questions in ATIS and Advising often contain mismatches between the columns targeted by the SQL queries and those referenced in the NL questions. For example, there were multiple instances where an NL question referred to people’s names, but the corresponding SQL query selected their IDs instead of their names e.g., INSTRUCTOR.INSTRUCTOR_ID rather than INSTRUCTOR.INSTRUCTOR_NAME). This could result in potential false positives in execution accuracy.

##### Null results

We refer to null results as cases where ground-truth SQLs return ‘Null’ or ‘[]’ as a result of execution, which may cause false positives in text-to-SQL model evaluation. We observed numerous data samples in Advising (and some in ATIS) fitting this scenario. Consequently, we manually corrected these cases in both ATIS and Advising by sampling a ‘valid’ combination of placeholder values for the questions and SQLs. The word ‘combination’ is used here because questions often contain multiple placeholders, which need their realized values to coexist for successful execution without null results.

##### Inconsistent text-to-SQL assumptions

ATIS and Advising contains questions with relative time expressions lacking a clear current time reference (e.g., I need a flight from city_name1 to city_name0 ‘today’). Moreover, the current time, inferred from NL questions and evident in SQL queries, varies across samples. Additionally, Advising contains SQL queries that assume a specific student without explicitly mentioning that in NL questions (e.g., Can ‘I’ take department0 number0 in semester0 year0?). To resolve these inconsistencies, we converted relative time expressions to their absolute-time equivalents (e.g., ‘today’ => ‘on month_number0 / day_number0 / year0’), and specified the student’s identity when needed.

## Appendix H URL to GitHub and Croissant Metadata Record

- •
Github: [https://github.com/glee4810/TrustSQL](https://github.com/glee4810/TrustSQL)
- •
Croissant Metadata Record: [https://huggingface.co/api/datasets/glee4810/TrustSQL](https://huggingface.co/api/datasets/glee4810/TrustSQL)

## Appendix I Author Statement

The authors bear all responsibility in case of violation of rights, etc. associated with the TrustSQL dataset.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. Lines 200–201 mention that all questions were reviewed to ensure clarity, with adjustments made for SQL queries that were too similar. How was clarity ensured during annotation, and what metric was used to determine if a question was clear or if two SQL queries were overly similar?

2. Who are the annotators, and how are disagreements in annotation resolved?
soundness: 2
strengths: 1. The paper introduces the TrustSQL dataset, aimed at evaluating text-to-SQL systems on both feasible and infeasible questions. The re-annotated datasets could serve as a valuable resource for the text-to-SQL community. 

2. The paper evaluates multiple text-to-SQL systems, including the SOTA fine-tuned text-to-SQL model (SQLCoder) and a general-purpose LLM (GPT-4), using both classifier-based and uncertainty-estimation approaches for detecting infeasible questions.
summary: The paper introduces TrustSQL, a benchmark designed to assess the reliability of text-to-SQL systems in handling both feasible and infeasible questions. To build the TrustSQL dataset, the authors re-annotated three existing datasets: ATIS, Advising, and EHRSQL. TrustSQL’s evaluation metric considers both question types, incorporating a novel metric, the Reliability Score. With this metric, correctly answering feasible questions and identifying infeasible questions yield a positive score of 1, while other responses result in a score of 0 or a negative penalty of -C.

The paper evaluates both classifier-based methods, which use a sub-model classifier to distinguish feasible and infeasible questions, and uncertainty-estimation-based methods, which rely on SQL generation uncertainty.
weaknesses: 1. Re-annotation is a central component of the paper and should be included in the main content rather than the appendix, especially given that one page of space remains available. 

2. The authors find that many text-to-SQL models can produce responses that are potentially more harmful than helpful, but this observation depends on the choice of weight C in the penalty. The choices for C (set at 1, N/2, and N) seem arbitrary. The paper lacks a discussion on the rationale behind these values and whether any human studies informed this choice.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 1
questions: Can you provide practical advice or criteria for selecting the penalty parameter 'c' in the Reliability Score? 
How sensitive is the RS to different values of 'c'? 
Have you considered that frequent abstentions might affect user experience?
soundness: 3
strengths: The paper makes a significant contribution by introducing TrustSQL, a benchmark that fills a critical gap in the evaluation of text-to-SQL models. By incorporating infeasible questions and proposing the Reliability Score (RS), the authors provide a fresh perspective on assessing model reliability, which is crucial for real-world deployment.
The authors conduct thorough experiments using both fine-tuning and in-context learning settings, integrating various abstention mechanisms. The meticulous re-annotation of existing datasets and the careful construction of infeasible questions enhance the quality and relevance of the benchmark. The paper's focus on reliability and safety has the potential to influence future research and practices in the field.
summary: The paper introduces TrustSQL, a new benchmark designed to evaluate the reliability of text-to-SQL models. It addresses two significant gaps in existing benchmarks: the lack of infeasible questions that cannot be mapped to SQL queries and the absence of abstention mechanisms in current models. TrustSQL is constructed by re-annotating three datasets—ATIS, Advising, and EHRSQL—and incorporating infeasible questions to provide a comprehensive evaluation. The authors propose a novel metric called the Reliability Score (RS), which quantifies a model's helpfulness relative to its harmfulness, adjusted by a user-defined penalty. They evaluate text-to-SQL models integrated with various abstention mechanisms, such as classifiers and uncertainty estimation methods. The experiments reveal that most models fail to meet safety requirements under high penalty settings, underscoring the need for developing models that ensure a certain degree of reliability alongside SQL generation accuracy.
weaknesses: The paper evaluates several abstention mechanisms but could provide deeper insights into why certain methods perform better under specific conditions. A more thorough analysis of the trade-offs between coverage and risk for each mechanism would be beneficial.
The Reliability Score depends heavily on the user-defined penalty parameter. The paper does not offer sufficient guidance on how practitioners should choose this parameter in practice. A sensitivity analysis or guidelines would make the RS metric more practical.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. The proposed classification of infeasible questions into missing-schema, ambiguous, and non-SQL categories may not be exhaustive. Are there other types of infeasible or unanswerable questions that the classification does not cover, such as questions based on inaccurate premises, malformed queries, or those that require external knowledge beyond the database schema? How would these additional categories impact model performance and evaluation? It would be helpful if the authors could discuss the potential for other classes of infeasible questions and how their benchmark could accommodate them.
2. Regarding the penalty c, is the selection and sensitivity of the c value thoroughly explored? Why choose 1, 0.5N, and N? Could the authors elaborate on how it was determined? Are there recommended strategies or guidelines for practitioners to select appropriate values in different application contexts?
soundness: 3
strengths: 1. The introduction of TrustSQL fills a critical gap in the evaluation of text-to-SQL models by incorporating infeasible questions.
2. The Reliability Score metric provides a meaningful way to quantify the trade-off between a model's helpfulness and harmfulness.
3. The paper provides an evaluation of different abstention mechanisms and their effects on model reliability.
summary: This paper introduces TrustSQL, a benchmark aiming to evaluate the reliability of text-to-SQL models in handling both feasible and infeasible natural language questions. The authors identify limitations in existing benchmarks, specifically the lack of infeasible questions and abstention mechanisms in models, which can lead to incorrect or harmful responses. They propose the Reliability Score (RS) metric to quantify a model's helpfulness relative to its potential harmfulness, assigning positive scores for correct decisions and penalizing incorrect ones based on a user-defined penalty. The authors re-annotate three domain-specific datasets—ATIS, Advising, and EHRSQL—to include infeasible questions categorized into missing schema, ambiguous, and non-SQL types. Various text-to-SQL models integrated with abstention mechanisms are evaluated using this benchmark.
weaknesses: 1. Insufficient Comparison with Existing Work: The paper lacks a thorough and clear comparison with existing works that address similar challenges in handling infeasible questions in text-to-SQL tasks. Prior studies, such as TriageSQL and more recent research on hallucination problems in LLMs, have explored methods for detecting and managing unanswerable or infeasible queries. The paper does not adequately position its contributions within the context of these existing approaches. Moreover, it does not provide empirical comparisons with previous benchmarks and methods dealing with infeasible questions.
2. Lack of Quality Analysis for the Proposed Dataset: The paper does not provide quality metrics or detailed analyses to demonstrate the high quality of the re-annotated datasets and the newly added infeasible questions. Quality metrics such as inter-annotator agreement, dataset statistics, or validation processes are essential to establish the reliability and usefulness of the proposed benchmark. The author could also consider adding a user study or real-world deployment to validate whether the proposed RS metric aligns with actual user satisfaction or trust in the models.
3. Limited Evaluation Scope and Generalizability Concerns: The evaluation is limited to three simple domain-specific datasets, which may restrict the generalizability to other domains or more complex schemas found in cross-domain datasets like Spider and BIRD.

---

confidence: 5
contribution: 1
initial_rating: 3
presentation: 1
questions: 1) Why did you choose to re-annotate existing datasets with just limited domains like ATIS and EHRSQL rather than leveraging larger benchmarks such as SPIDER and BIRD which contains ambiguous questions covering multiple domains already? What are trade-offs they considered in choosing these specific datasets over more diverse ones like SPIDER or BIRD? Could you explain the specific guidelines used for generating infeasible questions, and how your approach meaningfully improves upon template-based methods like EHR-SQL? Please provide concrete examples that demonstrate how your method yields more diverse or higher-quality data compared to existing techniques. 

2) Could you describe the **full annotation process** and **quality control** procedures used in your work? Specifically, how many annotators were involved, what were their expertise levels, how were they recruited, and what compensation was provided? Additionally, did you employ expert panels for review, and what specific measures were implemented to ensure the quality and consistency of annotations across the dataset?

3) How does your system address the complexity of SQL query equivalence in its classification scheme? For example, how are questions handled when they have multiple valid SQL representations (e.g., using `MAX` versus `ORDER BY LIMIT 1`), or when queries can be written both with and without nesting? Furthermore, why weren't factors such as natural language complexity or database schema constraints included in your difficulty assessment? See details in Weakness.

4) Could you provide a quantitative comparison of the diversity and distribution of questions in your dataset relative to existing datasets? This should include specifics on the types of templates, distribution across question categories, and empirical evidence demonstrating how your approach achieves greater diversity. How do these distributions align with the natural distribution of questions in real-world scenarios as authors claimed?

5) In terms of RS metric: What is the recommended range for the penalty factor `c`, and how should it be adjusted for different user groups, such as SQL experts or novices? What empirical evidence supports the choice of these ranges? Additionally, could you clarify the role and definition of large `\Phi{}` parameter and the rationale behind using `N/2` in your formula? How sensitive is model performance to changes in these parameter values? See details in Weakness.

6) What was the reason behind selecting SQLCoder-2 as the primary benchmark model? Can you provide details about its architecture (i.e., whether it is encoder- or decoder-based) and explain why other mainstream large language models (LLMs) were not included in the comparison? How does performance of SQLCoder-2 compare to other SOTA models such as CodeS on standard benchmarks?

7) One crucial point of concern is the separate implementation of CL methods for SQLCoder-2 and uncertainty estimation (UE)-based methods for T5. It would be more informative to apply both methods to the same model for a more direct comparison, as was done with GPT-4o. If there are specific reasons preventing SQLCoder-2 from being used with UE-based methods or preventing T5 from using CL-based methods, these limitations should be explicitly stated. This would help clarify the generalizability of these techniques. Additionally, the description of the CL-based method implementation is vague, making it difficult to understand how it differs from existing methods like DTE . If CL-based methods are a core contribution of this work, the paper should include a comparison with prior methods and highlight the improvements made in this work.

Please see other questions in Weakness for details.
soundness: 1
strengths: 1) This paper proposes an important research problem of infeasible questions in the text-to-SQL domain.
2) Authors introduce a novel evaluation metric to assess model reliability regarding the potential harm of infeasible questions.
3) The work incorporates these features by re-annotating three existing datasets.
4) Experiments demonstrate the impact of abstention mechanisms and the Reliability Score in evaluating text-to-SQL model reliability.
summary: This paper introduces TrustSQL, a new benchmark for evaluating text-to-SQL model reliability. Current benchmarks overlook real-world scenarios with unanswerable questions, and existing models miss abstention mechanisms. TrustSQL addresses these issues by including both answerable and unanswerable questions, introducing a new scoring system rewarding correct answers and abstentions while penalizing errors. To faciliate evaluation, authors re-annotated three datasets and added infeasible questions to formalize a comprehensive benchmark. Experiments and related analysis reveal potential issues of current text-to-SQL systems when facing such safety requirements.
weaknesses: ### Unclear Dataset Construction:
1) The paper re-annotates three existing datasets (ATIS, Advising, and EHRSQL), which are limited. The generalizability of the findings across other domains or more diverse database schemas is uncertain. In contrast, benchmarks like SPIDER and BIRD include test sets with broader domain coverage (e.g., > 10 domains), making the generalizability of the dataset in this paper more limited in comparison.

2) The creation of "infeasible questions" appears to rely on manual annotation, potentially introducing bias or inconsistency. However, the authors do not provide **detailed guidelines** on the generation process for these infeasible questions. Appendix D only offers a high-level overview of data re-annotation without elaborating on **specific types** of infeasible questions or **how these types were defined and re-annotated**. Discussing measures taken to ensure consistency across annotators and mitigate potential biases in the manual annotation process would be more helpful. Additionally, while the authors criticize related work (e.g., EHR-SQL [2]) for using a **template-based** method to generate infeasible questions, their approach similarly employs **templates** (as suggested in Lines 1362–1363). This raises the question of how their method fundamentally differs from EHR-SQL. Furthermore, **no comparison** of data distributions between the authors’ dataset and related work, such as [1] and [2], is provided. Both of these works offer clear, detailed taxonomies of ambiguous with distributions or unanswerable (infeasible) question types, which is missed in this paper. The single table provided (Table 1) and Section 4.2 are quite insufficient; more detailed charts or a comprehensive taxonomy are necessary, especially since this is a central problem the paper aims to address. Also authors just state which papers that they follow to define for each category without discussing motivations of each, and it also seems the contribution of this work is incremental and limited. 

3) When constructing benchmarks, it is essential to clearly describe the process of crowdsourcing. The paper should detail how many annotators were involved, what their expertise levels were, how the recruitment process worked, and what methods were used for data evaluation. Were expert panels involved? What were the criteria for evaluating the quality of annotations, and how was the workflow structured? Additionally, without information on compensations or incentives for crowdsourcing, it is difficult to evaluate the overall quality and reliability of the benchmark. These crucial details are missing, making it harder for readers to trust the quality and fairness of the dataset.

4) The paper does not adequately differentiate its contributions from similar works. Both [1] and [2] have already addressed infeasible questions (often referred to as ambiguous or unanswerable questions). The distinction in this work between their approach and previous works is only briefly mentioned, and a single sentence in the Related Work section does not sufficiently clarify the novelty or improvements conveyed by this work. For example, the statement "do not account for SQL generation errors" (Line 110) is vague without examples or tables. It is unclear why SQL generation errors are a critical focus. Do the authors envision text-to-SQL systems returning SQL queries with an explicit “unreliable” label to users? If so, why not fix the errors if they are detectable? Also as examples showing in Figure 1, "reliable" Text-to-SQL systems return an empty response by abstention. In this case, [1], an encoder-based classifier, can also achieve this by translating it from negative labels.

Similarly, the authors argue that template-based methods reduce diversity (as stated in Appendix D), but they also implement templates to structure data, followed by paraphrasing (as in [3]). The lack of a **diversity comparison** between their work and [2] weakens their claim that their methods result in more diverse infeasible question types.  The diversity the authors refer to in Appendix D seems to pertain to linguistic variation, not the diversity of categories of infeasible questions, which considered not quite important. Did this modification lead to some strong contributions of reliability of text-to-SQL systems?  Without experimental proof, it is hard to assert that template-based data generation inherently leads to less diversity. Furthermore, I also think the contribtion is quite limited if the diversity just refers to linguistic difference since [3] proposes much more formats of diversity including lingusitc modification and [1][2] already mention this problem. Also, following [3], please present and append your main types of templates

Given the availability of SPIDER and BIRD, which contain more domains and template-free data annotations, it remains unclear why the authors chose not to leverage these existing resources. These datasets naturally include ambiguous questions, as demonstrated in [4] and [5]. Why not explore them on such existing datasets, but spend more resources in re-annotation?

5) Question Familarity definition and detection is not clear and stable. As we know the same question may have multiple forms of gt SQLs. Clustering by GT SQLs from dataset may not be objective. For example, `highest score` can refer to SQLs with `max()` or `order by limit 1`, would your algorithm consider SQLs with such two keywords as infamiliar questions? This definition is questionable and not intuitive. The same with Question Difficulty in which authors mistakenly assume each question has the only one GT SQL. For example, they consider questions with no JOINs but no nesting in GT SQLs as `Medium`. How about its GT SQL is sub-queried such as `SELECT ... In (SELECT ...)`. It doesn't contain `JOIN` but nesting, what category should belong to? However, it could also be generated as a single `Join`, which should be considered as `Easy`. As pointed in Appendix A in BIRD, the only SQL difficutly cannot represent all difficulties of text-to-SQL tasks. The NL and environment with different complexity of DB schema and constrains should also be considered since it represent "text-to". In a short, I think the whole definitions in the second paragraph of Section 4.3 are not rigorous leading to unfair evaluation setting and distributions especially given authors didn't present a clear distribution and statitic analysis of their benchmark. 

### RS Metric:
RS relies heavily on a user-defined penalty factor c. While this allows for flexibility, it also introduces subjectivity. Different penalty values could lead to significantly different conclusions about model reliability, and there's no clear guidance on how to choose an appropriate penalty for a given application. And no appropriate range of c has been discussed. And given this work was positioned as reflections of real-world text-to-SQLs. It would be better disccuss which ranges of c selected can represent which groups of people. For example, whether data analysis with strong SQL knowledge requires just small penalty since they can fix issues if just small alerts are allowed. If this user have high-privacy considerations or totally unaware of SQL knowledge, this penalty could be set to large? Did different range of c will lead to performance shift of LLM or strategy based on GPT-4o?  As a core contribution, the absence of detailed analysis about c's implications and practical application is a significant weakness. 

Also, what does \Phi{} mean in Line 351-352, and N/2, this is hard to understand and i didn't find where the big \Phi was defined before. And why implement these three metrics here?

### Experiments:

1) The authors claim to include SQLCoder-2 as the state-of-the-art (SOTA) model. However, several crucial points about this are unclear. First, there are no links, references, or citations to provide more information about SQLCoder-2, and I couldn't locate it on established leaderboards such as SPIDER or BIRD. Additionally, the paper misses details on whether SQLCoder-2 is used as an encoder or decoder-based language model. It would be more appropriate to benchmark against well-recognized models like CodeS [6], which is a widely accepted SOTA model on the BIRD leaderboard. Testing a single model (SQLCoder-2) seems insufficient, and the paper would benefit from comparing performance with other popular large language models (LLMs) like CodeLlama, LLaMA 3/3.1, DeepCoder, and StarCoder. Furthermore, the decision to fine-tune models on the additional dataset TriageSQL, rather than using the training set of TrustSQL, requires more justification. Since this is a benchmark study, it raises concerns about whether other users would also need to fine-tune their models on TriageSQL before proceeding with TrustSQL. Lastly, the mention of "sub-model" is ambiguous—if this is referring to a mixture-of-experts (MoE) system, it should be explicitly stated. Otherwise, it is unclear why a single model's performance is not evaluated. This leaves questions regarding whether the authors are testing text-to-SQL systems as a whole or focusing purely on MoE systems.

2) Another point of concern is the separate implementation of CL methods for SQLCoder-2 and uncertainty estimation (UE)-based methods for T5. It would be more informative to apply both methods to the same model for a more direct comparison, as was done with GPT-4o. If there are specific reasons preventing SQLCoder-2 from being used with UE-based methods or preventing T5 from using CL-based methods, these limitations should be explicitly stated. This would help clarify the generalizability of these techniques. Additionally, the description of the CL-based method implementation is vague, making it difficult to understand how it differs from existing methods like DTE [1]. If CL-based methods are a core contribution of this work, the paper should include a comparison with prior methods and highlight the improvements made in this work.

3) The explanation of how uncertainty estimation (UE) is applied to GPT-4o is unclear. In Line 118, the authors state that the UE-based method can enhance LLM safety by qualifying model confidence, but the connection between this claim and the four dimensions defined in the Related Work section is not sufficiently explained. Moreover, the paper does not clarify how model confidence is represented for GPT-4o, given that it is a closed-source model. This is a crucial point that needs further elaboration to ensure the reproducibility and transparency of the method.




[1] Know what I don’t know: Handling ambiguous and unknown questions for text-to-SQL (wang et al., ACL 2024) \
[2] Ehrsql: A practical text-to-sql benchmark for electronic health records (Lee et al., NeurIPS 2024) \
[3] Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness (Chang et al., ICLR 2023) \
[4] Evaluating cross-domain text-to-sql models and benchmarks (Pourreza et al., EMNLP 2023) \
[5] Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmark (Wretblad et al., 2024) \
[6] CodeS: Towards Building Open-source Language Models for Text-to-SQL, (Li et al., SIGMOD 2024)