D eakin NLP at P rob S um 2023: Clinical Progress Note Summarization with Rules and Language M odels C linical Progress Note Summarization with Rules and Languague Models (2023)

Abstract: This paper summarizes two approaches developed for BioNLP2023 workshop task 1A: clinical problem list summarization. We develop two types of methods with either rules or pre-trained language models. In the rule-based summarization model, we leverage UMLS (Unified Medical Language System) and a negation detector to extract text spans to represent the summary. We also fine tune three pre-trained language models (BART, T5 and GPT2) to generate the summaries. Experiment results show the rule based system returns extractive summaries but lower ROUGE-L score (0.043), while the fine tuned T5 returns a higher ROUGE-L score (0.208).

## introduction clinical progress note is a typical type of text format for doctors and nurses to keep a record of a patient's up-to-date status. automatic summarization of daily progress notes can not only saves doctors' time, but also benefit the structured representation for patient record. in bionlp workshop task 1a foot_1 , the task of problem list summarization @xcite aims to generate a list of diagnoses and problems in a patient's daily care plan using input from the providers' progress notes during hospitalization. different from generic text summarization, the amount of the note and summary pairs is limited, e.g., there are only 765 annotated records which can be used for training, compared to the hundreds of thousands in xsum @xcite . recent methods @xcite used pre-trained language models, such as t5 @xcite and bart @xcite , to adapt to the task of clinical text summarization. in this paper, we will show the two basic methods that we used for this summarization task: the first one is a rule based system which leverages umls and a negation detector for high frequency medical concept filtering, and these concepts are used to extract text spans. the second approach fine tunes pre-trained language models, including bart @xcite , t5 @xcite and gpt2 @xcite . experiments on a blind test set shows that the fine tuned t5 achieves the best performance, the result is well aligned with that in @xcite . ## related work recent work on clinical text summarization focus on two applications: pure clinical note summarization @xcite and clinical dialogue summarization @xcite . @xcite proposed three sentence classification models that extracts sentences from progress notes. @xcite constructed an english dataset of 109,000 hospitalizations (2m source notes) and their corresponding summary proxy. more recently, @xcite used multi-head attention-based mechanism to perform extractive summarization of meaningful phrases on clinical notes. @xcite introduced a soft prompt-based calibration on mitigating performance variability in clinical notes summarization. ## exploration of clinical progress notes the clinical progress notes were extracted from mimic-iii, a publicly available dataset of deidentified ehr data from approximately 60,000 hospital icu admissions at beth israel deaconess medical center in boston, massachusetts @xcite . a subset of the above progress notes were annotated with the soap format with four components: subjective, objective, assessment, assessment: 37 yo m s/p total thryoidectomy for multi-nodular goiter that was impinging airway that appears to have a protected airway with no signs of compromise. subjective: no overnight events able to void on own pain well controlled with morphine amoxicillin unspecified lamictal (oral) (lamotrigine) rash; ... objective: icterus. constricted pupils and perrla/eomi. mmm. op clear. anterior neck c/d/i cardiac: regular rhythm, normal rate. normal s1, s2. no murmurs, rubs. ... summary: s/p total thyroidectomy plan. the subjective component are mainly from the patient's own description of the symptoms, the objective component concludes structured clinical data such as blood test result, the assessment component is the doctor's passive and active diagnoses, the plan component lists the medical problems and the corresponding treatment plans. in this task, the training data includes the first three components , and a more detailed annotation for the plan component is used as the problem summary, in which there are direct and indirect problems. ## clinical progress note summarization modelling we develop two types of clinical note summarization systems based on either rules or pre-trained language models. for rule based systems, it is largely relied on domain knowledge (e.g. umls) and expert information (e.g. doctors' pre-diagnosis). for pre-trained langague models, fine tuning them needs a large amount of high quality data. even though nowdays pre-traied language models show superior performance on many nlp tasks, for this specific task we develop our own summarization rules without too much expert involvement. ## conclusion in this paper, we summarized the models we developed for the bionlp2023 workshop task 1a. in the age of large language models, we 8 the result of the best ranking system is taken from https: //codalab.lisn.upsaclay.fr/competitions/12388 on 28 april 2023.