Biomedical NER using Novel Schema and Distant Supervision (2022)

Abstract: Biomedical Named Entity Recognition (BMNER) is one of the most important tasks in the field of biomedical text mining. Most work so far on this task has not focused on identification of discontinuous and overlapping entities, even though they are present in significant fractions in real-life biomedical datasets. In this paper, we introduce a novel annotation schema to capture complex entities, and explore the effects of distant supervision on our deep-learning sequence labelling model. For BMNER task, our annotation schema outperforms other BIO-based annotation schemes on the same model. We also achieve higher F1-scores than state-of-the-art models on multiple corpora without fine-tuning embeddings, highlighting the efficacy of neural feature extraction using our model.

## introduction named entity recognition (ner) consists of identification and classification of named entities in text. biomedical ner (bmner) is a crucial problem in healthcare as it is the initial step in solving various tasks, such as relation extraction, semantic role labeling, and clinical decision making @xcite . as compared to ner in other domains, bm-ner is a difficult task as labelled data in biomedical domain is less in amount and expensive to obtain, and it requires identification of complex entities that are not common in other domains @xcite . recently, deep learning approaches using large unstructured data, such as bi-lstm with crf @xcite and bert @xcite models have been used to obtain state-of-the-art results on bmner. the most common annotation scheme for ner is bio tagging, where b is for beginning of entity, i for inside of entity, and o for outside of entity. a major assumption of bio tagging is that an entity is composed of continuous and non-overlapping tokens. as complex entities that defy these assumptions frequently occur in biomedical records, a new scheme is needed to capture them. for this purpose, biohd @xcite was introduced to represent discontinuous entities that may overlap with four new tags : (bh,ih) as shared head tags and (bd,id) as non-shared non-head tags. however, this scheme fails to capture discontinuous entities that have more than two spans. in this paper, we propose a novel annotation schema biodt that overcomes this limitation of biohd. our schema includes shared non-head tags and non-shared head tags, and hence captures entities with more than two spans, which biohd fails to do. distant supervision is a method to generate labelled data from unlabelled data using existing knowledge @xcite that is particularly useful to create data for supervised learning algorithms which require large amounts of data. we use this method for bmner to compensate for the lack of labelled data in the biomedical domain. as we are using an rnn (recurrent neural network) which requires a large amount of training data, distant supervision helps in increasing the amount of annotated records without human effort. in summary, the main contributions of this paper are as follows: existing solutions for bmner include traditional ner methods such as dictionary or rulebased approaches, as well as supervised machine learning methods like markov models @xcite , conditional random fields (crfs) @xcite @xcite and support vector machine (svm) @xcite . lately, deep learning approaches using large unstructured data, such as bi-lstm with crf @xcite and bert @xcite models have been used to obtain state-of-the-art results on bmner. to deal with scarcity of token-level annotated data required in deep-learning models, some weaksupervision and distant-supervision solutions have been proposed. for the task of bmner, mathew et al. @xcite introduced a weaklysupervised data augmentation approach for identification of proteins in biocreative challenge vi track 1 dataset @xcite , using a reference set of entity names from knowledge bases like uniprot @xcite to identify entity mentions on unlabelled data. in 2016, lee et al. proposed a bagging-based approach using active learning with distant supervision, that uses a semiautomatically constructed dictionary of named entities from wikipedia @xcite . to the best of our knowledge, no prior work has been done to study the effects of distant supervision on complex entities for ner. to deal with annotation of complex entities, many methods have been proposed. annotation schemes like biohd @xcite and biohd1234 @xcite were proposed with four and ten additional tags, respectively, to the commonly used bio schema. these schemes gave near state-of-the-art results with simple machine learning models. methods such as representing sentences as hypergraphs @xcite (muis and lu, 2016), transition-based models that uses specialized actions and attention mechanisms @xcite , and representing ner task as a structured multi-label classification problem @xcite have also been explored. additionally, a two-stage approach that first detects all continuous parts, then combines them to form discontiguous entities using a classifier @xcite has also been proposed. ## annotation schema we introduce a new annotation schema called biodt, which consists of 11 tags: the traditional bio tags, and 8 additional tags as described below. extracting entities from biodt tagged sentence: discontinuous entities can be obtained from a biodt tagged sentence using the following simple rules : ## approach and architecture we use a bilstm-crf network to assign labels for ner, as presented in figure 2 . bilstm-crf is an rnn (recurrent neural network), and is formed by the combination of a bilstm (bidirectional long-short memory) and a crf (conditional network field). for each sentence, the bilstm forms a vector representation for each word, preserving backward and forward context. this vector representation is then used as the input to the crf, which predicts labels for the words of the sentence. the labels at the crf output layer are decoded using the viterbi algorithm. ## datasets we experiment on two datasets from the biomedical domain: share 2013 @xcite and share 2014 @xcite . the datasets contain clinical free-text notes, which include discharge summaries, echo-reports, and radiology reports. an annotated named entity can contain any number of continuous spans, and it maps to a concept in the disorder semantic group of snomed-ct @xcite . since a significant fraction(almost 10%) of the mentions in these datasets are discontinuous(from ## results and analysis for evaluation, we have used scripts provided in share tasks to calculate f-score (f) to evaluate the efficiency of the models in our experimentation. our baseline model is a bilstm-crf that uses the features and embeddings mentioned in 4.1, as proposed by @xcite . we faced a replication crisis while attempting to reproduce the results presented in @xcite using the proposed biohd1234 schema. hence, we were unable to compare the performance of our schema with that of biohd1234. ## conclusion in this paper, we introduced a novel annotation schema to identify named entities in biomedical data. we have also shown that for the same model, our annotation scheme gives better performance than other bio-based complex annotation schemes for discontinuous entities. we also explore the distant supervision paradigm to increase our training set for bioner. using this, we have achieved stateof-the-art results.