KCL ab at Chemotimelines 2024: End-to-end system for chemotherapy timeline extraction – Subtask2 (2024)

Abstract: This paper presents our participation in the Chemotimelines 2024 subtask2, focusing on the development of an end-to-end system for chemotherapy timeline extraction. We initially adopt a basic framework from subtask2, utilizing Apache cTAKES for entity recognition and a BERT-based model for classifying the temporal relationship between chemotherapy events and associated times. Subsequently, we enhance this pipeline through two key directions: first, by expanding the exploration of the system, achieved by extending the search dictionary of cTAKES with the UMLS database; second, by reducing false positives through preprocessing of clinical notes and implementing filters to reduce the potential errors from the BERT-based model. To validate the effectiveness of our framework, we conduct extensive experiments using clinical notes from breast, ovarian, and melanoma cancer cases. Our results demonstrate improvements over the previous approach.

## introduction in recent years, the rapid development and widespread implementation of electronic health records (ehrs) have created a significant demand for the clinical notes processing and information extraction within the realm of medical research. @xcite particularly, the extraction of temporal information, encompassing temporal expressions, temporal events, and temporal relations, has created new opportunities for dynamic treatment studies @xcite . among the various treatment modalities, chemotherapy stands out as one of the most critical and widely used approaches in cancer therapy. ehrs with temporal information offer a unique advantage by providing a chronological roadmap of patient-specific treatments. these timelines play a key role in understanding the effectiveness of chemotherapy, evaluating treatment responses, and identifying patterns in patient outcomes. they serve as invaluable resources for researchers investigating the interplay among treatment protocols, tumor biology, and patient characteristics. by analyzing these timelines, researchers can uncover trends, predictors of response and potential markers for treatment success or failure. as such, the construction of accurate and comprehensive chemotherapy treatment timelines is not only an academic pursuit but a practical clinical necessity in advancing cancer care and improving outcomes. however, this task presents notable challenges due to the domain-specific nature of ehrs, namely, variations in writing style and quality, lack of text structure, and the pervasive presence of redundant information. moreover, the creation of annotated corpora manually is a resourceintensive process, demanding substantial human effort and time. consequently, numerous research efforts have turned to employ rulebased, machine-learning, or hybrid methods to extract general temporal information from clinical narratives @xcite @xcite . notably, despite these efforts, there are currently no available tools designed specifically for extracting timelines to contextualize cancer treatment. hence, this competition subtask aims to fill this gap by developing an end-to-end system for chemotherapy timeline extraction (*jiarui @xcite . this system not only addresses the urgent need for accurate and ## results in chemotimelines 2024 subtask2, our team achieved the 3 rd highest rank in the average scores, with f1 scores of 0.68 for breast cancer (rank #1), 0.49 for melanoma (rank #3), and 0.45 for ovarian cancer (rank #7) ( due to the unavailability of the test set, we present results from the development set to analyze the strengths and weaknesses of our pipeline. as depicted in our implemented preprocessing procedure and filtering step effectively reduced false positives not only from the integrated dictionary but also from cases prone to misclassification by the tlink model. however, this also led to the exclusion of some true pairs from the gold timelines. for example, some patients only had "rad" files, which do not pertain to chemotherapy details, resulting in missed records that impact our evaluation significantly. additionally, some chemotherapy pairs were solely mentioned in the plan section, such as "we will give chemo cycle 2 today." while we expected subsequent confirmed notes, they were not present, resulting in the omission of such pairs from our analysis. reviewing the test results ( ## conclusion and future work this paper details our efforts in the chemotimelines 2024 subtask2, focusing on the development of an end-to-end system for chemotherapy timeline extraction. our experiments utilizing clinical notes from breast, ovarian cancer, and melanoma cases have demonstrated the enhancements made to our pipeline. these enhancements include expanding the system's capabilities by leveraging the umls database and implementing preprocessing and directional filtering procedures to effectively reduce false positives. future works could potentially include firstly creating a more detailed and precise dictionary using the umls, with specific terms tailored to different cancer types, and establishing a synonymous dictionary to prevent duplication of terms. secondly, it is crucial to exercise caution when removing files such as "rad" and "sp," especially in cases where patients only possess these notes. finally, exploring the use of chatgpt and appropriate prompts as an alternative to the tlink classifier, which is currently fine-tuned from pubmedbert, would be a valuable exercise. chatgpt's superior understanding of sentence context could prove beneficial for those classifications that do not require specific domain knowledge.