M ed T em2.0: Prompt-based Temporal Classification of Treatment Events from Discharge Summaries (2023)

Abstract: Discharge summaries are comprehensive medical records that encompass vital information about a patient’s hospital stay. A crucial aspect of discharge summaries is the temporal information of treatments administered throughout the patient’s illness. With an extensive volume of clinical documents, manually extracting and compiling a patient’s medication list can be laborious, time-consuming, and susceptible to errors. The objective of this paper is to build upon the recent development on clinical NLP by temporally classifying treatments in clinical texts, specifically determining whether a treatment was administered between the time of admission and discharge from the hospital. State-of-the-art NLP methods including prompt-based learning on Generative Pre-trained Transformers (GPTs) models and fine-tuning on pre-trained language models (PLMs) such as BERT were employed to classify temporal relations between treatments and hospitalisation periods in discharge summaries. Fine-tuning with the BERT model achieved an F1 score of 92.45% and a balanced accuracy of 77.56%, while prompt learning using the T5 model and mixed templates resulted in an F1 score of 90.89% and a balanced accuracy of 72.07%.Our codes and data are available at https://github.com/HECTA-UoM/MedTem .

## introduction clinical texts contain important temporal information, such as medication start and end dates, appointment dates, and diagnosis dates. extracting this information can provide insights into a patient's medical history and allow doctors to make more informed decisions about their treatment. however, this process requires a significant amount of time and effort. to help healthcare professionals make informed decisions more efficiently, leading to better patient outcomes, we designed the project medtem, medication and treatment event extraction and their relation modelling with temporal information. by using natural language processing (nlp) methods to extract temporal information from clinical texts, doctors can spend less time deciphering medical records and more time focusing on providing the best care possible to their patients. this study reports findings from medtem2.0, a follow-up work from our previous investigation medtem @xcite . clinical texts can be challenging to process due to their unstructured nature and the use of medical jargon. thus, developing effective nlp techniques for extracting temporal information from clinical texts is crucial for improving healthcare outcomes. the primary goal of this work is to classify temporal information related to medication, surgeries, and other treatments within electronic health records (ehrs) to determine if these treatments occurred during the hospitalisation period. this work aims to develop a system capable of classifying temporal information using prompt-based learning (pbl) from texts, which could aid healthcare professionals in understanding patients' medical histories and facilitate research in clinical text mining. as an example, in to the best of our knowledge, this is the first attempt at using prompt-based learning for the temporal classification of treatments in the clinical domain, with the following outcomes: 1) we established a high baseline score with 90.89% f1 measurement and 72.07% balanced accuracy by using prompt-based learning, demonstrating the clinical free text admission date discharge date doctor's note 02/22/92 03/08/92 she was, therefore, cleared for the operating room, and on 3/3/92, she underwent a left carotid endarterectomy, with continuous electroencephalogram monitoring and vein patch angioplasty, which was uneventful . ## conclusion and future work in this work, two state-of-the-art approaches were developed to classify the relative timing of treatments in hospital discharge summaries, focusing on determining whether a treatment was administered during hospitalisation or not. these approaches used cutting-edge pre-trained language models, bert, gpt-2, and t5, in conjunction with prompt-based learning and fine-tuning paradigms. both approaches achieved f1 scores of 91.79% and 92.45%, and balanced accuracy of 75.08% and 77.56%, respectively, on the n2c2 2012 temporal relations dataset. the primary challenge was accurately classifying the "off" class due to data imbalance and complex semantic meanings that made it difficult for the models to make correct decisions. future work could investigate the impact of fixed tokens on mixed template performance or the role of longer sequence lengths in soft templates for improved understanding. additionally, a more comprehensive comparison of prompt learning and traditional fine-tuning can be conducted across various clinical domain tasks, using frozen plms in conjunction with few-shot learning methods.