A New Public Corpus for Clinical Section Identification: M ed S ec I d (2022)

Abstract: The process by which sections in a document are demarcated and labeled is known as section identification. Such sections are helpful to the reader when searching for information and contextualizing specific topics. The goal of this work is to segment the sections of clinical medical domain documentation. The primary contribution of this work is MedSecId, a publicly available set of 2,002 fully annotated medical notes from the MIMIC-III. We include several baselines, source code, a pretrained model and analysis of the data showing a relationship between medical concepts across sections using principal component analysis.

## introduction most unstructured medical text found in electronic health record systems (ehrs) written by medical staff have conceptually well defined sections. for example, discharge summaries are technical medical documents, written by physicians when the patient is discharged, which describe the patient's hospital stay and surrounding circumstances of their illness. as shown by the example in figure 1 , discharge summaries consist of named sections, typically in a specific sequence, such as the history of present illness; this type of section appears both in discharge summaries and in physician notes that describe a chronology of an illness that begins with the admission of the patient. whereas sections often have headers, section identification (si) is more challenging than simply parsing the first several leading header tokens of the respective section (underlined in figure 1 ). while the first several tokens can be helpful in identifying a section, their naming often varies. for example, the 6 th section in figure 1 starts with header tokens preoperative laboratory data, but the section type is labs-imaging. there are also cases where the header tokens are missing, as shown in admission date: [**2126-2-7**] discharge date: [**2126-2-20**] date of birth: [**2069-4-1**] sex: m history-of-present-illness history of present illness: mr. [**known lastname **] is a 56-year-old male who experienced chest. . . past-medical-history past medical history: hypertension, former smoker with a 4-pack per day history for which he. . . social-history social history: he lives alone, and he works at [**hospital3 2576**] as a cargo transporter. medication-history medications on admission: aspirin 325 mg p.o. once a day, toprol-xl 50 mg p.o. once a day. allergies allergies: he had no known drug allergies. labs-imaging preoperative laboratory data: white count 6.0, hematocrit 33.3, platelet count 329,000.. . . hospital-course on exam he had a left facial droop, status post his childhood polio. temperature of 97.5, heart rate 65 in sinus. . . discharge-diagnosis discharge diagnoses: 1. status post coronary artery bypass grafting x 3.. . . discharge-instructions discharge instructions: he was instructed to make an appointment. . . discharge-medications medications on discharge: 1. aspirin enteric coated 81 mg p.o. once a day. 2. colace 100 mg p.o. twice a day. . . he was discharged to home with vna services in good condition on [**2126-2-20**]... the 7 th section (hospital-course) in the same figure, or where sections have several header text spans placed throughout the section. adding to this challenge is the non-uniformity of the text, lack of section boundary syntax, and copy-pasted text from other notes or from structured data such as patient vital signs. while discharge summary sectioning helps a physician locate specific information, the primary impetus for the structure and content stems from the ongoing dispute between providers and healthcare insurance companies in the united states. providers are limited by how much they can bill for relatively simple medical procedures, but increasingly complex procedures garner more revenue with proper documentation. specifically, medical billing staff and insurance companies use relative value units (rvus), which is a monetary unit updated annually and currently set at $34.30. the number of rvus billed is based on the composition and number of sections included in the medical notes per guidelines set by the centers for medicare and medicaid services 1 . for this reason, providers are encouraged to write medical notes to maximize rvus out of necessity @xcite even though physician training lacks such emphasis. in contrast, medical residents are evaluated with the objective structured clinical examination (osce), which is a student examination that evaluates students based on direct observation @xcite . however, the exam's evaluation with respect to medical note authoring and structure uses a very different criteria and omits rvus @xcite . the necessity of a particular structure in medical notes, for the purpose of patient care and arguably more important insurance billing requirements, highlights the need for understanding sectioning. however, the motivation for understanding si is not limited to the medical field, it has a bearing on other medical nlp tasks. since each section contains specific information, si is often the first step in a medical nlp pipeline and can lead to downstream propagation errors causing poor task specific results if not properly executed. examples of downstream tasks that benefit from si include medical summarization, entity linking and natural language understanding and extraction. while academic text segmentation has garnered interest @xcite , no publicly available medical si annotated corpora exists @xcite . for this reason, we believe medsecid is the first medical section identification dataset. it was created from 2,002 medical notes annotated by two attending physicians and one senior resident physician at the university of illinois chicago (ui health). the annotation dataset is 1 https://www.cms.gov/regulations-and-guidance comprehensive with 2,558k annotated tokens or 97.3% of the entire corpus (see ## related work sectioning medline abstracts was explored by @xcite using a support vector machine (svm). this classifier was used to label sentences as introduction, method, result, or conclusion and showed promising results using a bag-of-words approach. sequence based approaches @xcite were also used to section scientific abstracts into objective, methods, results, and conclusion labels using a conditional random field (crf) model producing a sentence level accuracy of 95.5%. while academic abstract segmentation was a well explored area @xcite , @xcite were the first to apply statistical methods to the medical domain to automatically classify sections of clinical free text into sections. their method used in, out, begin (iob) annotation @xcite with labels to mark named sections. for example, b-hpi indicates a beginning token for the history of present illness section. their dataset consisted of annotating the 2010 i2b2 corpus with a section header and medical ontology label, and obtained an f-measure of 0.92 for the concept extraction task @xcite . a maximum entropy (maxent) model @xcite and beam search were used for classification to produce the iob sequence for token tagging. along with maxent, other non-neural network methods, such as svm and crf models continue to be popular with few exceptions as detailed in the comprehensive survey of @xcite . one such exception @xcite used a long-short term memory (lstm) model with word-to-vector (word2vec) embeddings @xcite for a binary classification of section boundaries. even though the corpus consists of dictated and transcribed notes, they show that neural methods work for the section segmentation task. other notable neural network (nn) text segmentation works use convolutional neural networks (cnns) over sentence embeddings with a softmax over the output of a bi-directional long-short term memory (bilstm) layer to demarcate sections as a binary classification across both medical and non-medical datasets @xcite . @xcite also used a lstm in a network that aggregates features across fast-text word embeddings using a concatenated segment pooling lstm (s-lstm) for non-medical wikipeda articles @xcite . the work of @xcite most closely resembles our si work. however, their model classifies only the four soap (subjective, objective, assessment and plan) sections available in the corpus leaving the others as future work. their methods also only have been tested against the flair framework, which uses concatenated static word embeddings that are fine tuned locally for the task on the 2010 i2b2 corpus. our method includes finetuning the bert embeddings themselves as an endto-end joint learning process. additionally, they have provided no annotation pipeline or process to create a semi-supervised or bootstrapped corpus. our work includes medical domain specific experiments with various word embedding combinations and novel data analysis using the unified medical language system (umls) @xcite and cui2vec @xcite (see section 3.3). it also includes other methods and network experimental configurations the authors have not yet tried as they used the flair framework "out of the box". another significant difference is their annotations are not available foot_0 while we classify 50 sections and provide our code with annotations publicly. ## dataset medsecid is a subset of the mimic-iii version 1.4 corpus @xcite that we annotated; mimic-iii is publicly available foot_1 and consists of critical care unit ehr records from the beth israel deaconess medical center in boston, massachusetts. the dataset contains 58,976 hospital admissions across 46,520 patients who were admitted to the intensive care unit (icu) surgical, medical, and neonatal departments. it includes 2,083,180 unstructured medical text notes handwritten by medical professionals across several disciplines and contains 15 categories, such as discharge summaries and radiology notes. we created a curated annotation set consisting of text spans taken from a random sample across five categories of mimic-iii medical notes foot_2 , including discharge summaries, radiology, consult , echo, and physician progress notes (see ## methods because the section text spans do not break on tokens, we cast our task as a named entity recognition (ner) using in, out (io) encoding 9 on a 50 way classification @xmath0. using this encoding, we created several baselines across two bilstm models 10 for the purpose of future work benchmarking. these baselines include majority label metrics, a token bilstm-crf, and a sentinel bert embedding @xcite lstm model (see figure 3 ). aside from adjusting the lstm hidden size, gradient clipping, and number of epochs, all parameters were held constant across all experiments (see appendix b for all hyperparameters used). ## results the baseline models described in section 4 were each trained until the validation loss converged, then early stopped. the results are summarized in from the majority label, it's clear the models perform comparatively well as shown in the summary results in the bert sent does not lag far behind, but its performance using sentinel embeddings does not capture sections as well as the token level models despite long document length. performance significantly improved and models converged faster with the use of gradient clipping to alleviate issues of lstm exploding gradients @xcite . ## conclusions and future work we presented medsecid, a comprehensive dataset of 2,002 medical annotations from the mimic-iii corpus across five note types and 50 sections. the dataset contains section types, headers and patient age annotations. our dataset shows promising baseline results from simple models such as bilstms with diverse inputs, but still leaves room for improvement by more sophisticated models. we expect performance using our models to improve pipelines that use rule based methods for si as mentioned in section 3.2. these pipelines include discharge note summarization, and other downstream tasks that would benefit from having header and non-section text removed such as training word embeddings such as clinicalbert. hyperparameter tuning with the baseline models is a next logical step for further work. another obvious opportunity to improve performance is to concatenate cui2vec embeddings in the input layer as described in section 3.3. other future work includes comparing the results using the synthetic tokens in place of pseudo tokens, which would shed light on how models learn with more realistic data.