Automatic recognition of abdominal lymph nodes from clinical text (2020)

Abstract: Lymph node status plays a pivotal role in the treatment of cancer. The extraction of lymph nodes from radiology text reports enables large-scale training of lymph node detection on MRI. In this work, we first propose an ontology of 41 types of abdominal lymph nodes with a hierarchical relationship. We then introduce an end-to-end approach based on the combination of rules and transformer-based methods to detect these abdominal lymph node mentions and classify their types from the MRI radiology reports. We demonstrate the superior performance of a model fine-tuned on MRI reports using BlueBERT, called MriBERT. We find that MriBERT outperforms the rule-based labeler (0.957 vs 0.644 in micro weighted F1-score) as well as other BERT-based variations (0.913 - 0.928). We make the code and MriBERT publicly available at https://github.com/ncbi-nlp/bluebert , with the hope that this method can facilitate the development of medical report annotators to produce labels from scratch at scale.

## introduction lymph nodes are organs of the lymphatic system that are present throughout the body. their status plays a pivotal role in the staging and treatment of cancer @xcite . the development of deep learning (dl) for computer vision has led to increasing interest in applying dl-based ai to identify and segment lymph nodes and detect lymph nodes and detecting lymph node metastasis in imaging studies, such as magnetic resonance imaging (mri). applications of machine learning to mri not only contribute to improving diagnostic accuracy but also reduce the workload of radiologists and enable them to spend additional time on highlevel decision-making tasks. however, dl algorithms need to be sufficiently trained and evaluated using large-scale data before clinical adoption. unlike general computer vision tasks, medical image analysis currently does not have enough annotated data (comparable to imagenet and ms coco), which is mainly because the conventional methods for harvesting labels cannot be applied in the clinical domain, as it requires extensive clinical expertise and because of security and privacy issues. therefore, there is an unmet need to construct a large-scale annotated dataset of lymph nodes to increase the generalizability and robustness of the dl algorithms. radiologists report any abnormal lymph node detected in computed tomography (ct) and mri exams by describing the regional name (type) of the lymph node. example mri scans and annotations are shown in figure 1 , where the radiologist describes the lymph node with the sentence "abdominal/pelvic lymph nodes: there is intraperitoneal and retroperitoneal lymphadenopathy, for example, enlarged mesenteric/peripancreatic lymph node measuring bookmark1[[(2.8 cm x 1.3 cm) (series 6, image 24)]], periportal lymph node measuring bookmark2[[(2.8 cm x 1.7 cm) (series 6, image 19)]], retroperitoneal left paraaortic lymph node conglomerate measuring bookmark3[[(3.8 cm x 3.1 cm) (series 6, image 22)]], and retroperitoneal aortocaval lymph node measuring book-mark4[[(2.0 cm x 1.3 cm) (series 6, image 25)]]". the radiologist places a hyperlink (hereafter "bookmark") in the context to refer to the specified lymph node annotation in the image. therefore, clinical reports provide a detailed and personalized account of assessments, offering a better context for clinical decision making and follow up. natural language processing (nlp) has been explored recently to unlock evidence buried in clinthere is intraperitoneal and retroperitoneal lymphadenopathy, for example enlarged mesenteric/peripancreatic lymph node measuring bookmark1[[(2.8 cm x 1.3 cm) (series 6, image 24)]], periportal lymph node measuring bookmark2[[(2.8 cm x 1.7 cm) (series 6, image 19)]], retroperitoneal left paraaortic lymph node conglomerate measuring book-mark3[[(3.8 cm x 3.1 cm) (series 6, image 22)]], and retroperitoneal aortocaval lymph node measuring book-mark4[[(2.0 cm x 1.3 cm) (series 6, image 25)]]. " to tackle these obstacles and challenges, this paper outlines a framework based on deep learning to harvest lymph node annotations and construct an annotated dataset of lymph nodes by automatically extracting lymph nodes from clinical reports. the contributions of this study are threefold: (1) we construct an ontology of 41 types of abdominal lymph nodes with a hierarchical relationship. the rest of the paper is organized as follows. we first present related work in section 2. then, we describe the method to construct the ontology and dataset in section 3, followed by our experimental setup, results, and discussion in section 4. we conclude with future work in the last section. ## related work in recent years, there has been considerable interest in harvesting information and knowledge from freetext on electronic health records (ehrs) @xcite . however, manually annotating a large dataset to fulfill the needs of deep learning models downstream is time-consuming and expensive. therefore, researchers have applied nlp systems to identify structured labels from radiology reports @xcite @xcite . previous efforts in this area have focused mostly on two directions. one is the rule-based methods. negex, in combination with the unified medical language system (umls), is a widely used algorithm that utilizes regular expressions to determine the negative concepts in the clinical narratives @xcite @xcite . negbio extended negex by utilizing universal dependencies and sub-graph matching to detect both negative and uncertain lung diseases in chest x-rays and was used to generate labels for the nih chest x-ray and mimic-iii-cxr datasets @xcite @xcite . the chexpert labeler further extended negbio by increasing the rule sets and improving the nlp pipeline to construct report-level disease annotations @xcite . chexpert++ trained a hybrid ruleand bert-based labeler on the radiograph domain but offers additional commentary on the utility of active-learning strategies to inform the interplay between the hybrid and rule-based labeler @xcite . the other direction is to apply machine learning methods to construct labels @xcite @xcite . huang et al. described a hybrid approach to automatically detect negations in clinical radiology reports @xcite . clark et al. combine machine learning (conditional random field and maximum entropy) and rules to determine the assertion status of medical problems mentioned in clinical reports @xcite . recently, deep learning approaches have also been studied intensively. @xcite cnns to classify pulmonary embolism in chest ct reports @xcite . drozdov et al. compared thirteen supervised classifiers and demonstrate that bidirectional long short-term memory (bilstm) networks with attention mechanisms effectively identify labels in cxr reports @xcite . wood et al. present a transformer-based network for brain magnetic resonance imaging (mri) radiology report classification, which automates this task by assigning image labels based on free-text expert radiology reports @xcite . smit et al. introduced a bert-based approach to medical image report labeling that exploits both the scale of available rule-based systems and the quality of expert annotations @xcite . ## methods in this section, we first describe the process of constructing the abdominal lymph node ontology and gold-standard labels from the mri reports associated with lymph nodes on mri images. then we demonstrate the development of the transformerbased method to detect lymph nodes from the reports. ## conclusion in this study, we introduced an ontology of 41 types of abdominal lymph nodes with a hierarchical relationship. we then proposed an end-to-end framework for combining rules and deep learning for accurate bookmark classification for lymph node types from mri reports. in this framework, the rule-based method is first used to extract sentences with potential lymph node bookmarks. then a bert-based model pretrained on mri reports was used to classify each bookmark into one of 41 types of abdominal lymph node, non-abdominal lymph nodes, or not a lymph node. we evaluated our framework on 2,099 bookmarks manually annotated by a radiological expert. we also compared our framework with a rule-based system and other bert-based models. we find that our framework achieved 0.912 in f1-score, which outperforms the rule-based system and other bert variations. our study has several limitations. first, our model is limited to the 41 abdominal lymph nodes. while we believe the list is comprehensive, we may miss some lymph node types due to training corpus bias. second, our evaluation is performed on a single corpus. cross-institutional experiments need to be performed in the future to evaluate the generalizability of the model. while our work only scratches the surface of using text mining techniques and deep learning to extract the lymph node from radiology reports, we hope it will shed light on the development of generalizable nlp models that can extract highly accurate labels.