## 3 Model Design and Implementation
### Pre-Processing
The BIRD dataset is broader, not a specialized dataset for finance. It covers games, sports, coding, finance, biology, etc. Compared with the Spider 2.0 dataset, the BIRD dataset has apparent advantages in NL2 SQL experiments in the financial field. First, the BIRD dataset focuses on the complex table structure and business logic unique to the financial industry, covering many financial statements, transaction records, and customer data, which can more realistically reflect the characteristics of financial data. Secondly, the queries in the BIRD dataset often contain more specific operations related to the financial field, such as financial calculations, risk assessments, and dynamic queries, which enables the model to better adapt to the needs of practical applications. In contrast, although the Spider 2.0 dataset covers multiple fields, it lacks dedicated data and scenarios for the financial industry, so its performance in financial tasks may not be as good as the BIRD dataset. The following are the databases used for the NL2 SQL question test and the number of questions in each database. In Table 2, the input data is the BIRD development dataset dev ([[https://bird-bench.oss-cn-beijing.aliyuncs.com/dev.zip](https://bird-bench.oss-cn-beijing.aliyuncs.com/dev.zip)]([https://bird-bench.oss-cn-beijing.aliyuncs.com/dev.zip](https://bird-bench.oss-cn-beijing.aliyuncs.com/dev.zip)), (accessed on 8 October 2024)). This dataset has 1533 natural language query questions. We select EX (Execution Accuracy), F1 (Partial Match Score), and VES (Value Execution Score) metrics to facilitate better comparative testing and conduct experiments on the dataset's Simple, Moderate, and Challenging types, comparing the results with 11 typical NL2 SQL methods. EX (Execution Accuracy)

EX refers to Execution Accuracy, which indicates whether the results returned by the new SQL queries are the same as those of the target queries when executed on the database. If the results match those of target SQL queries, they are counted as correct. EX mainly focuses on the accuracy of the query execution results rather than whether the specific structure of the SQL queries matches exactly. Total EX (Total Execution Accuracy) is usually used to measure the accuracy of SQL queries generated by the NL2 SQL model when executed. It indicates the proportion of SQL \begin{table}
\begin{tabular}{l r} \hline \hline
**Database** & **Number of Questions** \\ \hline card\_games & 191 \\ \hline codebase\_community & 186 \\ \hline formula\_1 & 174 \\ \hline thrombosis\_prediction & 163 \\ \hline student\_club & 158 \\ \hline toxicology & 145 \\ \hline superhero & 129 \\ \hline european\_football\_2 & 129 \\ \hline financial & 106 \\ \hline california\_schools & 89 \\ \hline debit\_card\_specializing & 64 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Databases and number of questions per database. queries generated by the model in all input samples that are correctly executed and return expected results. The calculation formula is as follows:

\[\text{Total}\ EX=\frac{Number\ of\ Correct\ Executions}{Total\ Number\ of\ Samples} \tag{2}\]

In Formula (2), the Number of Correct Executions refers to the number of samples in which the generated SQL query can correctly return the expected results during actual execution. The total Number of Samples refers to the total number of samples of all-natural language queries in the test set. The total EX value is between 0 and 1, usually expressed as a percentage. It is used to evaluate the model's overall performance in generating SQL queries in the test set and correctly returning results. 1. F1 (Partial Match Score)

The F1 score is used to measure the similarity between the SQL generated by the model and the target SQL. By comparing the overlap in syntax and structure between the generated SQL and the target SQL, the F1 metric provides a finer-grained assessment of the SQL generation effectiveness.