EHR - S eq SQL : A Sequential Text-to- SQL Dataset For Interactively Exploring Electronic Health Records (2024)

Abstract: In this paper, we introduce EHR-SeqSQL, a novel sequential text-to-SQL dataset for Electronic Health Record (EHR) databases. EHR-SeqSQL is designed to address critical yet underexplored aspects in text-to-SQL parsing: interactivity, compositionality, and efficiency. To the best of our knowledge, EHR-SeqSQL is not only the largest but also the first medical text-to-SQL dataset benchmark to include sequential and contextual questions. We provide a data split and the new test set designed to assess compositional generalization ability. Our experiments demonstrate the superiority of a multi-turn approach over a single-turn approach in learning compositionality. Additionally, our dataset integrates specially crafted tokens into SQL queries to improve execution efficiency. With EHR-SeqSQL, we aim to bridge the gap between practical needs and academic research in the text-to-SQL domain.

## introduction text-to-sql provides a practical opportunity for non-experts to explore databases, even without prior knowledge of the database operations. electronic health records (ehrs) are large-scale relational databases (rdbs) storing vast and comprehensive patient data @xcite . medical experts often ask questions that require highly complex reasoning across multiple tables and access to a vast number of records within a single query @xcite . handling such complexity in large-scale databases remains a significant challenge in the current text-to-sql research @xcite . several efforts have been made to construct textto-sql datasets for ehrs. mimic-sql @xcite is the first text-to-sql dataset that targets a subset of mimic-iii @xcite ), * these authors contributed equally to this work. one of the widely-used open-source ehr databases, consisting of 26 tables. drugehrqa @xcite provides the first question answering dataset that incorporates both structured tables and unstructured notes from ehrs. ehrsql @xcite is a dataset curated based on a survey from various medical experts, reflecting the diverse information needs of the actual medical field. still, there are important yet underexplored objectives for the practical application of text-to-sql models. first, the text-to-sql system should incorporate interactivity. most of the existing text-to-sql research assumes a single-turn scenario, whereas the process of exploring information in real-world situations is often continuous @xcite . for better user usability, it is desirable for the system to be interactive as well. second, the text-to-sql system should learn compositionality. realistically, datasets cannot capture all the diverse needs of users. therefore, it is crucial that the model can handle a wider range of queries especially when these unseen queries comprise components (i.e. sub-queries) that the model has explicitly seen during training. this is the compositional generalization ability, which is challenging even for the highly proficient language models @xcite . lastly, the text-to-sql system should consider efficiency. real-world databases are often significantly larger than academic ones @xcite . efficient query execution is crucial, given the magnitude of the databases in actual hospitals (e.g. mimic-iii dataset includes the medical history of 46k patients). this becomes even more significant when we consider a real-time interaction scenario between a text-to-sql model and a user @xcite . in this work, we introduce ehr-seqsql which addresses all three objectives. our contribution is as follows: ## data construction we aim to convert text-sql pairs of ehrsql to multi-turn text-sql pairs that embrace interactivity and query execution efficiency. note that ehrsql contains text-sql samples for two different ehr sources, namely mimic-iii @xcite and eicu @xcite . we use the mimic-iii version of ehrsql, given its complex schema and wider adoption by the nlp community. questions for eicu and the questions that cannot be transformed into a proper sql query are not the scope of this dataset but can be addressed in the future work. our data construction process is divided into two stages: sql decomposition and natural language question (nlq) generation. initially, we break down each sql query of ehrsql into a sequence of subqueries to create more granular meanings of queries. next, we create a corresponding nlq for each subquery. compared to nlq decomposition, sql decomposition enables systematic breakdown of a question without any loss of information, due to the strict, rule-based nature of sql syntax. the overall process is shown in figure 2 and we provide detailed explanations in the following sections. ## experiments we now empirically demonstrate the benefits of ehr-seqsql in terms of two types of compositional generalization ( § 5.1, § 5.2), as well as the effectiveness of the special tokens ( § 5.3). ## conclusion we present ehr-seqsql, the first and the largest sequential text-to-sql dataset designed for ehr databases, aimed at improving interactivity, compositionality, and efficiency in text-to-sql parsing. our experiments show that the question decomposition significantly enhances the model's generalization capabilities for unseen interaction goals and extends its applicability to longer, untrained interactions. moreover, the introduction of novel sql-like tokens, which reuse query and execution results from prior queries, enhances execution efficiency and improves the model performance. we believe our dataset will serve as a valuable testbed for assessing contextual and interactive text-to-sql tasks and bridge the gap between industry needs and academic research.