F uzzy BIO : A Proposal for Fuzzy Representation of Discontinuous Entities (2021)

Abstract: Discontinuous entities pose a challenge to named entity recognition (NER). These phenomena occur commonly in the biomedical domain. As a solution, expansions of the BIO representation scheme that can handle these entity types are commonly used (i.e. BIOHD). However, the extra tag types make the NER task more difficult to learn. In this paper we propose an alternative; a fuzzy continuous BIO scheme (FuzzyBIO). We focus on the task of Adverse Drug Response extraction and normalization to compare FuzzyBIO to BIOHD. We find that FuzzyBIO improves recall of NER for two of three data sets and results in a higher percentage of correctly identified disjoint and composite entities for all data sets. Using FuzzyBIO also improves end-to-end performance for continuous and composite entities in two of three data sets. Since FuzzyBIO improves performance for some data sets and the conversion from BIOHD to FuzzyBIO is straightforward, we recommend investigating which is more effective for any data set containing discontinuous entities.

## introduction adverse drug reactions (adrs), harmful reactions that result from the intake of medication, pose a major health concern @xcite . due to the limitations of clinical trials on the one hand @xcite and reporting systems after release on the market on the other hand @xcite , many adrs remain undiscovered. therefore, both social media and clinical reports are being explored by the research community as alternative information sources for the semi-automatic discovery of adrs @xcite . one particular challenge for the extraction of adrs from text is the presence of discontinuous entities. these can be either composite entities (i.e. some words belong to multiple entities), such as 'lack of sleep and appetite', or disjoint entities (i.e. split entities), such as 'eyes are feeling dry'. these phenomena occur more commonly in the clinical than general domain. in fact, @xcite reported that discontinuous mentions in clinical text account for about 10% of all adr mentions. none of the traditional versions of the bio representation scheme (b: beginning of entity, i: inside entity and o: outside entity) or common extensions such as iobes (e: end of entity, s: singleton entity) foot_0 were designed to handle such mentions @xcite . therefore, @xcite proposed extending the bio scheme with two additional tags: the 'h' for words shared by multiple mentions and 'd' for parts of discontinuous mentions not shared by other mentions. this resulted in four new tag types (hb-, hi-, db-and di-). their biohd representation was broadly adopted by the community @xcite @xcite . although the biohd scheme allows for precise representation of entities, the extra tag types make the task more difficult for models to learn. straightforward bio rules such as 'an entity always starts with a b' are no longer valid under the biohd scheme. in this paper we argue that a more simple bio representation in which discontinuous entities are transformed into continuous sequences by including all non-entity tokens in between would improve adr extraction by being easier to learn and reintroducing these straightforward rules. we coin this representation fuzzybio. some examples of entities represented with biohd and fuzzybio can be seen in sentence 1 muscles are constantly quivering ! biohd db o o di o fuzzybio b i i i o sentence 2 i have pain in my hands and upper arms biohd o o hb hi hi db o db di fuzzybio o o b i i i i i i in this paper we present fuzzybio, a fuzzy continuous bio representation of discontinuous entities. moreover, we show it is beneficial for end-toend adr discovery. our representation is applicable to other domains as well. we release our code for the purpose of follow-up research. foot_1 ## related work the first shared task to deal with discontinuous medical entities was semeval 2014 task 7 @xcite . prior to this task, discontinuous entities were often excluded (e.g. uzuner et al. (2011) ) or each part was represented as a separate continuous entity and later reassembled @xcite . various representations were proposed but the only one able to distinguish between those that share a head word (i.e composite entities) and those that do not (e.g. disjoint entities) was the biohd scheme @xcite . this scheme was later analysed in more detail and compared to two baseline approaches: (1) ignoring all discontinuous entities and (2) representing separate parts of discontinuous entities as individual entities @xcite . in comparison to the baseline approaches, the biohd scheme could improve recognition of both discontinuous but also continuous entities, likely due to its ability to distinguish between the two. tang et al. (2015) also proposed a further extension (biohd1234) in which numbers were added to refer to which entity a non-head ('d') entity should be combined with foot_2 , effectively expanding the scheme from 7 to 13 tags. this representation was able to outperform biohd due to its ability to correctly represent multiple discontinuous entities and discontinuous entities with more than one non-head part. however, as neither biohd nor biohd1234 could handle multiple head entities in one sentence, @xcite proposed a multilabel bio representation in which tokens can be labeled with more than one tag, and each tag corresponds to one entity. for ner of adverse drug responses, this novel representation managed to outperform biohd. similarly, @xcite allowed for multiple labels per token for extracting disorders from scientific articles. despite its limitations @xcite , the biohd scheme is commonly adopted @xcite @xcite . we propose an alternative, simpler representation scheme that could improve extraction by being easier to learn. ## data we use two data sets of social media posts annotated for adr: cadec @xcite and psytar @xcite . the former was also used by @xcite . both contain posts from medical fora on askapatient.com. additionally, we used a data set of clinical records annotated for disorder mentions, namely the semeval 2014 task 7 data (pradhan et al., 2014) that builds on the clef ehealth 2013 corpus used by @xcite . see ## discussion in answer to rq1, we find that the fuzzybio scheme benefits overall recall during ner for two of the three data sets. for these data sets, it also leads to a higher percentage of correctly identified entities, both continuous and discontinuous. for the third data set (psytar), more discontinuous entities are extracted correctly when using fuzzybio compared to the biohd scheme. however, more continuous entities are missed. in answer to rq2, we find that for the same two data sets (cadec and clef) the end-to-end adr extraction is improved for continuous and composite entities. however, for the remaining data set (psytar) the end-to-end performance is lower for all entity types. we believe that the difference between psytar and the other data sets may be related to either the low number of discontinuous entities or the low number of composite entities in the psytar data, which may have hindered the training of an ner model for these entity types. an alternative explanation is that fuzzybio is less beneficial for easier ner tasks: the initial ner performance with biohd is far higher for psytar (f 1 of 0.771) than for the other data sets (f 1 of 0.586 and 0.312). the difference between psytar and the other data sets is unlikely to be related to the relative percentage of discontinuous entities, as this is similar to that of the clef data (5.8 vs 6.2%), or the nature of the data, as cadec contains forum posts from the same website. another result that stands out is the lower endto-end performance for disjoint entities when using the fuzzybio scheme despite initial gains in the extraction of disjoint entities for all data sets. we suspect that normalization of these entities is made more challenging by the words in between the disjoint parts of the entity that are now included in the extracted entity. therefore, in future work, we plan to investigate post-processing steps such as the removal of stop words which may improve the normalization of the more noisy disjoint entities represented with fuzzybio. as our representation is applicable for representing any type of discontinuous entity, future work may also include testing fuzzybio in other domains. although the improvement in ner is compa-rable for disjoint entities in medical records and user-generated content, the negative impact on normalization of disjoint entities is far stronger for the medical records. one might expect the normalization to decrease more strongly if more non-entity words (i.e. more noise) were included, but the median amount of non-entity words included in the disjoint entities is equal for all data sets (on average 1 word is added). manual analysis also does not reveal a difference between the type of non-entity word included; they appear to mostly be stopwords. thus, the most likely explanation for this difference is that the training examples for normalization from the user-generated data are already more noisy than their counterparts from the medical records. consequently, the normalization algorithm for user-generated data might be better at dealing with noise. future work could investigate whether training with the noisy examples instead of the original entities would be beneficial. fuzzybio appears to be more beneficial for endto-end extraction of composite entities in the medical records (+5.7) than in the user-generated data (+1.3 and -3.1). however, the number of non-entity words that is included is not lower for the medical records (median of 2 words added) compared to the user-generated data (median of 1 for cadec and 3 for psytar). thus, this difference does not appear to be due to an increase in the fuzziness of the entities. we also find some support for our hypothesis that the biohd representation makes the ner task more difficult for bert models to learn than the fuzzybio representation. overall the bert models have difficulty learning the additional tag types; the precision for h-and d-tags is consistently lower than the precision for b-tags. in fact, on the psytar data which contains only few overlapping entities (1.7%), the h-tag was never predicted. it seems that fuzzybio makes the task easier in two ways, namely by standardizing entities into continuous sequences that always start with a btag and by excluding rare tags such as the h-tag. standardizing entities makes it easier for the model to learn the underlying rules and excluding rare tags removes a goal for which there are only few examples available. ## conclusion as recommendation for future ner tasks, we expect fuzzybio to especially benefit ner for diffi-cult tasks with a fair amount of discontinuous entities. however, since the conversion from biohd to fuzzybio is straightforward and deterministic, we recommend to experimentally compare which of the two representations is more effective for any data set that includes discontinuous entities.