A Hybrid Deep Learning Approach for Spatial Trigger Extraction from Radiology Reports (2020)

Abstract: Radiology reports contain important clinical information about patients which are often tied through spatial expressions. Spatial expressions (or triggers) are mainly used to describe the positioning of radiographic findings or medical devices with respect to some anatomical structures. As the expressions result from the mental visualization of the radiologist’s interpretations, they are varied and complex. The focus of this work is to automatically identify the spatial expression terms from three different radiology sub-domains. We propose a hybrid deep learning-based NLP method that includes – 1) generating a set of candidate spatial triggers by exact match with the known trigger terms from the training data, 2) applying domain-specific constraints to filter the candidate triggers, and 3) utilizing a BERT-based classifier to predict whether a candidate trigger is a true spatial trigger or not. The results are promising, with an improvement of 24 points in the average F1 measure compared to a standard BERT-based sequence labeler.

## introduction radiology reports contain a radiologist's interpretations of an imaging study of a patient. the mental interpretations often get expressed through descriptions of important radiological entities with reference to a particular anatomical structure @xcite . the radiological entities whose positions are described mainly include radiographic findings (e.g., clinical findings like interstitial emphysema and imaging observations like groundglass opacity) and medical devices (e.g., endotracheal tube and central venous catheter). there exists a wide variation in the spatial language used by radiologists in expressing the exact positioning of the radiological entities. limited research has focused on effectively identifying the spatial expressions from multiple imaging modalities. therefore, the focus of this work is to investigate different automatic approaches to extract the spatial expressions from the report sentences along with highlighting the various challenges involved in this task. these extracted spatial expressions, if predicted accurately, can also facilitate clinical applications such as automatic labeling of radiographic images for training image classifiers @xcite . identification of spatial expressions in a sentence forms the foundation for other downstream spatial information extraction tasks. much of the clinically relevant information appears in the context of a spatial expression. consider the following sentence: a lytic lesion at the left vertex extending into the epidural region, scalp and soft tissues is grossly unchanged in appearance. here, we note that it is very crucial to accurately identify the spatial expressions such as at and extending into. firstly, these trigger terms denote the specific positioning of the lesion by associating the lesion term with anatomical entities such as left vertex and epidural region. secondly, this indirectly helps in identifying all the modifier information about the lesion including its density (i.e., lytic) and status (i.e., unchanged in appearance). spatial expressions are also used to describe the positioning of the medical devices that are inserted into specific body locations. radiologists often document the current position status of the devices (e.g., malpositioned, satisfactory position) and often times indicate their changes in positioning. the following is an example of the radiologist's interpretation about a device position: picc line enters from left arm, descends to the lower inferior vena cava, then turns and extends peripherally to left subclavian vein. this captures the mental visualization of the radiologist as they interpret the specific position of the picc line from the corresponding image. we note that there are diverse expressions belonging to multiple part-of-speech categories (e.g., verbs, prepositions, verbs followed by prepositions) that the radiologists use in documenting the spatial position of both findings and devices. in this work, our aim is to identify all the spatial expressions given a radiology report sentence. we experiment with a pre-trained language model, bert @xcite , used as a sequence labeler to extract the spatial expressions (or triggers). we further propose a hybrid deep learning method where we use bert as a classifier in combination with domain-dependent heuristics. specifically, in this hybrid approach, we first extract the candidate trigger terms from the sentences with high recall leveraging the terms from the training corpus. we then filter the candidates by applying a set of radiology-specific constraints. finally, we utilize bert as a classification model to identify if each of the filtered candidate terms is a trigger expression or not. ## related work some previous studies have focused on extracting spatial relations from radiology reports @xcite . however, both these studies are specific to appendicitis-related reports. our previous work has also aimed at identifying spatial expressions (mainly prepositional) from chest x-ray reports @xcite . moreover, all these studies have focused on identifying spatial relations associated only with radiographic findings. we aim to identify more complex and varied spatial expressions associated with descriptions of both findings and medical devices. importantly, descriptions of devices often utilize far richer spatial language, as shown in the picc line example above. both in the general and medical domains, hybrid deep learning approaches have been used lately for various natural language processing (nlp) tasks such as document classification @xcite and named entity recognition @xcite . a recent work has also demonstrated the promising results of applying a hybrid approach for extracting clinical information from ct scan reports @xcite . moreover, many nlp tasks have leveraged the contextualized representations of pretrained language models such as bert. however, not much effort has been directed toward building hybrid methods based on bert. extracting spatial expressions from text often requires domain knowledge of language characteristics. thus, we investigate the impact of combining radiology-specific constraints with a bert-based model to extract spatial expressions from radiology reports. ## dataset we use a dataset of 400 radiology reports containing annotated spatial expressions @xcite . these reports are taken from the mimic iii clinical corpus @xcite . our dataset consists of an equal distribution of three different imaging modalities, namely, chest x-rays, brain mris, and babygrams. some basic statistics related to the spatial expressions in this dataset are shown in ## evaluation and experimental settings we perform 10-fold cross validation (cv) to evaluate the performance of both the bert-based methods. for each of the 10 iterations, reports in 8 folds are used for training and 1 fold each are used for validation and testing. average precision, recall, and f1 measures are reported for these methods. we also report these performance metric values for the rule-based methods (both for exact matching and exact matching + constraints) by evaluating on the same test folds. note that we train the bert classifier using the candidates directly obtained after the exact matching step. while evaluating, we apply the additional domain constraints over the candidate triggers generated from exact matching. we make this decision based on the results of our preliminary experiments. we use the bert base variant for both sequence labeling and classification models. the models are pre-trained on mimic-iii clinical notes @xcite for 320k steps. the maximum sequence length for both the tasks is set at 128 and learning rate at 2e-5. based on the validation set performance, we select the number of training epochs as 4. we use the cased version of the models. ## results the results of spatial trigger extraction are shown in we notice that the sequence labeling method obtained high precision and low recall. the exact matching achieved a much improved recall (96.77%) compared to the sequence labeling system. however, it resulted in too many false positive spatial triggers, mainly because of common prepositional and verb terms such as of, with, and are (as indicated by a very low precision). we achieved slightly better precision by applying constraints over the exact matched triggers (shown in the third row of ## discussion we focus on extracting varied spatial expressions from radiology reports using a sequence labeling method as well as a hybrid approach that first applies domain-specific rules to extract the candidate triggers and later employs a deep learning-based classifier to judge every candidate. our proposed method (exact matching + constraints + bertbased classification) achieves much improved average f1 measure in cv. ## conclusion this work proposes a bert-based hybrid method to extract spatial expressions from radiology reports. this method achieves satisfactory performance with an average f1 measure of 81.10 over 10-fold cv. we also extract spatial expressions by formulating the problem as a sequence labeling task (used as baseline). we find that the bert-based sequence labeling model suffers from low recall. our proposed hybrid approach combining radiologyspecific constraints with a bert-based classifier helps to improve the recall by around 36%. we also address some of the challenges involved in the task of spatial trigger extraction in the radiology domain. we plan to further improve the performance of the system by adding more granular domain constraints as well as evaluate the generalizability of the method across multi-institutional datasets.