Wonder at Chemotimelines 2024: M ed T imeline: An End-to-End NLP System for Timeline Extraction from Clinical Narratives (2024)

Abstract: Extracting timeline information from clinical narratives is critical for cancer research and practice using electronic health records (EHRs). In this study, we apply MedTimeline, our end-to-end hybrid NLP system combining large language model, deep learning with knowledge engineering, to the ChemoTimeLine challenge subtasks. Our experiment results in 0.83, 0.90, 0.84, and 0.53, 0.63, 0.39, respectively, for subtask1 and subtask2 in breast, melanoma and ovarian cancer.

## introduction patients' medical history plays a crucial role in guiding the decisions made by clinicians. yet, the vast majority of temporal information, along with the medical events, is embedded in clinical narratives. for instance, details such as the timing of chemotherapy administration for cancer patients, particularly those referred to the current hospital from other healthcare facilities, are often documented within clinical notes during patient consultations with physicians. there is a pressing need to automatically extract timeline information from clinical narratives to facilitate the understanding of disease progression and treatment efficacy and enhance the quality of cancer research and patient care based on electronic health records (ehrs). large language models (llms), trained on a large amount of unstructured text and then applied to a task through instructive prompts @xcite , have recently shown great value in information extraction and garnered significant attention. we developed medtimeline, an end-to-end hybrid natural language processing (nlp) system, which combines llms and deep learning to support knowledge engineering for timeline information extraction. in this chemotimeline challenge, we applied medtimeline to the two subtasks and had it evaluated based on the tasks-specific data @xcite . ## related work in the 2012 i2b2 clinical temporal relations challenge, sohn et al. constructed an automated system, i.e., medtime, that leveraged the framework of heideltime, for timex3 extraction from clinical text @xcite . the system extracts temporal information, including date, time, duration, and frequency, along with their normalized values, demonstrating superior performance. in addition, using the thyme corpus @xcite , liu et al. developed an attention-based neural network model to extract containment relations within sentences of clinical narratives @xcite , which outperformed the existing state-of-the-art neural network models at the time. nlp systems derived from challenges are usually limited to functioning within the confines of the tasks they're specifically designed for. consequently, wang et al. further expanded their nlp work to patient-level event temporal relation extraction based on real ehr data @xcite . their results revealed that complete data related to patients' journeys was important for accurate identification of diagnosis dates. in addition, domain knowledge, e.g., chemotherapy drug and transplant names of multiple myeloma, and histology cell type of lung cancer were critical for event temporal relation extraction. in addition, this study demonstrated the usability of medtime and medtagger, resource-driven open-source uima-based frameworks with the capacity to incorporate knowledge engineering @xcite @xcite , for ehr-based cancer research. ## methodology in this section, we present our solution, medtimeline, an end-to-end nlp system comprising an event entity (chemotherapy entity for subtask 2) extractor, a temporal entity extractor (subtask 2), and a patient-level timeline aggregator (subtasks 1 patient timeline aggregator chemo entities temporal entities gold standard chatgpt 4 task 1 output patient timeline synthesized sentences bio-lm* task 2 output medtagger linked sentences medtime clinical notes (training set of subtask1) clinical notes (training set of subtask1) entity relation few shot event extractor time extractor medtime/medtagger figure 1: architecture of medtimeline and 2). the architecture of medtimeline includes two well-established knowledge engineering nlp pipelines (medtagger as event extractor and med-time as temporal expression extractor) from the open health nlp (ohnlp) consortium, a contextaware deep learning open-source architecture, and an llm-empowered data augmentation pipeline (figure 1 ). specifically, the data augmentation pipeline incorporates chatgpt to generate synthetic data to facilitate the fine-tuning of a pretrained language model for temporal relation classification within the timeline aggregator. ## conclusion we present medtimeline, an end-to-end hybrid nlp system generalizable to any medical events for patients' timeline extraction, and evaluate it based on the chemotimeline challenge data. our system ranks the second place in subtask 1 and the third place in subtask 2. in the future, we will continue to develop the medtimeline, and tailor it to the scenarios of various medical events.