Comparing and combining some popular NER approaches on Biomedical tasks (2023)

Abstract: We compare three simple and popular approaches for NER: 1) SEQ (sequence labeling with a linear token classifier) 2) SeqCRF (sequence labeling with Conditional Random Fields), and 3) SpanPred (span prediction with boundary token embeddings). We compare the approaches on 4 biomedical NER tasks: GENIA, NCBI-Disease, LivingNER (Spanish), and SocialDisNER (Spanish). The SpanPred model demonstrates state-of-the-art performance on LivingNER and SocialDisNER, improving F1 by 1.3 and 0.6 F1 respectively. The SeqCRF model also demonstrates state-of-the-art performance on LivingNER and SocialDisNER, improving F1 by 0.2 F1 and 0.7 respectively. The SEQ model is competitive with the state-of-the-art on LivingNER dataset. We explore some simple ways of combining the three approaches. We find that majority voting consistently gives high precision and high F1 across all 4 datasets. Lastly, we implement a system that learns to combine SEQ’s and SpanPred’s predictions, generating systems that give high recall and high F1 across all 4 datasets. On the GENIA dataset, we find that our learned combiner system significantly boosts F1(+1.2) and recall(+2.1) over the systems being combined.

## introduction ner has frequently been formulated as a sequencelabeling problem @xcite @xcite in which a model learns to label each token using a labeling scheme such as @xcite . however, in recent years people have also formulated the ner task as a span-prediction problem @xcite @xcite where spans of text are represented and labeled with entity types. let seq be the simplest sequence-labeling model which represents each token using a language model and then classifies each token-representation with a linear layer. let seqcrf be another popular sequence-labeling model which is identical to seq model except that the token representations from the language model are fed into a linear-chain conditional random field layer @xcite . let spanpred @xcite be a model that represents every possible span of text using two token-embeddings located at the its boundary, and then classifies every span-representation using a linear layer. we describe all three models in detail in section 4. we evaluate seq, seqcrf, and spanpred models on four biomedical ner tasks: genia @xcite , ncbi-disease @xcite , livingner(spanish)(miranda-escalada et al., 2022), and socialdisner(spanish) @xcite . despite being simple, the spanpred and crf models improve the state-of-the-art on the livingner and socialdisner tasks. @xcite show that the sequencelabeling approaches(eg. seq and seqcrf) and span-prediction approaches(eg. spanpred) have different strengths and weaknesses while having similar(f1) performance. this motivated us to try and combine seq, seqcrf, and spanpred models using two simple methods and study the results. we refer to the two simple methods as union and majvote. union is inspired by the set(mathematical) union operation and it simply involves "unioning" the sets of predictions made by the models. majvote is the classic majority voting method. we find that majvote can yield systems that have both high precision and high f1. inspired by the boost in recall(and the corresponding drop in precision) resulting from the union method, we implemented a combiner system (which we refer to as meta) that aims to combat the drop in precision as a result of the union method. we find that meta shows very promising signs of increasing precision while preserving high recall and high f1. meta borrows ideas from work on generating span representations using "solid markers" @xcite @xcite , work on using prompts @xcite , and work by @xcite to combine the span-prediction and sequence-labeling approaches using the span-prediction approach. ## preliminaries let every prediction p of an ner system be a tuple of the form @xmath0, entitytype, beginoffset, endoffset) which consists of the identifier of the sample/text in which the entity is found, the type of the entity, and the beginning and ending offsets for the entity. ## preprocessing for genia and ncbi-disease, each sample is an english sentence. for socialdisner, each sample is an entire spanish tweet. for livingner, we use the flert @xcite approach for document-level ner, in which each spanish sentence is surrounded by a context of 100 characters to the left and 100 characters to the right.