## 3 Research Methodology
### Dataset and Query Preparation  
The TREQS dataset was developed to advance text-to-SQL generation tasks, particularly within the complex and sensitive healthcare domain. This dataset is derived from the extensively documented and deidentified MIMIC-III database, encompassing a wealth of electronic medical records from over 46,520 patients. By providing carefully curated question-query pairs alongside a relational database structure, TREQS enables the creation of systems capable of translating natural language queries into SQL commands, thus facilitating direct and efficient interaction with healthcare databases.  
The TREQS dataset is built around a relational database that organizes patient data into five primary tables. These tables cover demographic information, diagnostic records, procedural interventions, prescription details, and laboratory test results. For example, the demographic table contains fields such as patient ID, age, sex, admission details, and mortality status, providing a foundational overview of each patient's medical history. The diagnoses table maps the ICD-9 codes to their respective descriptions, while the procedures table records the procedural codes and descriptions of medical treatments. Similarly, prescriptions and laboratory test tables provide information on medication administration and diagnostic results, respectively.  
A key feature of the TREQS dataset is that it includes natural language question-query pairs specifically designed to train and evaluate text-to-SQL models. These pairs consist of two components. The first component comprises template questions systematically generated using predefined patterns to ensure comprehensive coverage of various database fields. The second component includes natural language questions, which are manually rephrased versions of the template questions. This rephrasing process introduces linguistic variability and improves the data set's applicability to model real-world query interactions.  
For example, a template question such as \"Retrieve all female patients over 60 years of age\" could be rephrased as \"Can you show me the list of women older than 60?\" Both questions are linked to the same SQL query, enabling robust model training.  
We used a total of 1000 natural language queries from the TREQS dataset, originating from the MIMIC III dataset, a widely used deidentified critical care dataset. Each query was carefully designed to reflect realistic use cases, such as retrieving patient demographics, diagnoses, or lab results. To ensure a robust evaluation, queries were executed five times per, resulting in 5000 queries per total. This repetition enabled the assessment of model consistency across multiple runs, as large language models (LLMs) are well known for their inherent variability in responses, even when presented with identical inputs. By running each query multiple times, our objective was to measure the stability of the model's outputs, identify any significant deviations, and evaluate the reliability of their performance under repeated conditions.  
Table 1 illustrates some example question-query pairs from the TREQS dataset. These examples highlight the diversity in the phrasing of the questions and the corresponding SQL queries used to extract data from the relational database.