Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization (2023)

Abstract: Biomedical named entity recognition is one of the core tasks in biomedical natural language processing (BioNLP). To tackle this task, numerous supervised/distantly supervised approaches have been proposed. Despite their remarkable success, these approaches inescapably demand laborious human effort. To alleviate the need of human effort, dictionary-based approaches have been proposed to extract named entities simply based on a given dictionary. However, one downside of existing dictionary-based approaches is that they are challenged to identify concept synonyms that are not listed in the given dictionary, which we refer as the synonym generalization problem. In this study, we propose a novel Synonym Generalization (SynGen) framework that recognizes the biomedical concepts contained in the input text using span-based predictions. In particular, SynGen introduces two regularization terms, namely, (1) a synonym distance regularizer; and (2) a noise perturbation regularizer, to minimize the synonym generalization error. To demonstrate the effectiveness of our approach, we provide a theoretical analysis of the bound of synonym generalization error. We extensively evaluate our approach on a wide range of benchmarks and the results verify that SynGen outperforms previous dictionary-based models by notable margins. Lastly, we provide a detailed analysis to further reveal the merits and inner-workings of our approach.

## introduction biomedical named entity recognition (bioner) @xcite @xcite is one of the core tasks in biomedical natural language processing (bionlp). it aims to identify phrases that refer to biomedical entities, thereby serving as the fundamental component for numerous downstream bionlp tasks @xcite . existing bioner approaches can be generally classified into three categories, i.e. (1) supervised methods; (2) distantly supervised methods; and although dictionary-based approaches do not re-quire human intervention or in-domain corpus, they suffer from the synonym generalization problem, i.e. the dictionary only contains a limited number of synonyms of the biomedical concepts that appear in the text. therefore, if an entity synonym in the text is not explicitly mentioned by the dictionary, it cannot be recognized. this problem severely undermines the recall of dictionary-based methods as, potentially, a huge amount of synonyms are not contained in the dictionary. to address the synonym generalization problem, we propose syngen (synonym generalization)a novel framework that generalizes the synonyms contained in the given dictionary to a broader domain. figure 1 presents an overview of our approach. (1) in the training stage, syngen first samples the synonyms from a given dictionary as the positive samples. meanwhile, the negative samples are obtained by sampling spans from a general biomedical corpus. then, it fine-tunes a pre-trained model to classify the positive and negative samples. in particular, syngen introduces two novel regularizers, namely a synonym distance regularizer which reduces the spatial distance; and a noise perturbation regularizer which reduces the gap of synonyms' predictions, to minimize the synonym generalization error. these regularizers make the dictionary concepts generalizable to the entire domain. (2) during the inference stage, the input text is split into several spans and sent into the finetuned model to predict which spans are biomedical named entities. to demonstrate the effectiveness of the proposed approach, we provide a theoretical analysis to show that both of our proposed regularizers lead to the reduction of the synonym generalization error. we extensively test our approach on five wellestablished benchmarks and illustrate that syngen brings notable performance improvements over previous dictionary-based models on most evaluation metrics. our results highlight the benefit of both of our proposed regularization methods through detailed ablation studies. furthermore, we validate the effectiveness of syngen under the fewshot setup, notably, with about 20% of the data, it achieves performances that are comparable to the results obtained with a full dictionary. in summary, our contributions are: umls c5203670 covid-19 c5203670 covid-19 virus c0016627 h5n1 mitophagy controls mitochondria quality to maintain cellular … nsf ge the patient infected by covid-19 virus ✗ ✗ … ✓ ✓ ✗ ✗ … ✗ ✓ syngen sdr npr ϵ f ( ŝ i + ν) f ( ŝ i ) mitophagy controls covid-19 disease cellular homeostasis … mitophagy controls cellular homeostasis … the patient patient infected … covid-19 covid-19 virus training stage inference stage m 4 s m 7 n c x p i z n h a k 2 i q v x 2 j n 6 z 8 4 m 6 t q x a m x d u f c y z 3 3 h j f n 2 r j u l 5 n b w f x a x s m v f t b w n z a 3 i t s 7 d s 0 t r w i n s c 5 v m 8 s a c i z o z t d d a t n w f e c h p 4 1 w c j n 5 j t u q n j p i 1 g x j g k s 4 j 1 • we provide a theoretical analysis showing that the optimization of syngen is equivalent to minimizing the synonym generalization error. ## methodology in this section, we first give the definition of the dictionary-based biomedical ner task. then, we introduce our synonym generalization (i.e. syn-gen) framework, following by the details of the synonym distance regularizer and the noise perturbation regularizer. lastly, we provide a theoretical analysis on the problem of synonym generalization to show the effectiveness of our proposed approach. ## related works existing ner methods can be divided into three categories, namely, supervised methods, distantly supervised methods, and dictionary-based methods. in supervised methods, @xcite propose to pre-train a biomedical corpus and then fine-tune the ner model. wang et al. (2019b); cho and lee (2019); weber et al. (2019, 2021) develop several toolkits by jointly training ner model on multiple datasets. however, supervised annotation is quite expensive and human labor-intensive. in the distantly supervised methods, lison et al. (2020, 2021); ghiasvand and kate (2018); meng et al. (2021); liang et al. (2020) propose to first conduct a weak annotation and then train the bioner model on it. fries et al. (2017); basaldella et al. (2020) propose to use utilize a well-designed regex and special case rules to generate weakly supervised data while wang et al. (2019a); shang et al. ( 2020 ) train an in-domain phase model and make a carefully tailored dictionary. however, these methods still need extra effort to prepare a high-quality training set. in the dictionary-based methods, zhang and elhadad (2013) propose a rule-based system to extract entities. aronson ( 2001); divita et al. (2014); soldaini and goharian (2016); giannakopoulos et al. (2017); he (2017); basaldella et al. (2020) propose to apply exact string matching methods to extract the named entities. @xcite ; @xcite propose to extract the entity names by calculating the string similarity scores while quickumls @xcite propose to use more string similarity scores @xcite to do a fast approximate dictionary matching. to the best of our knowledge, there is still no existing work that gives a theoretical analysis of the synonyms generalization problem and proposes a corresponding method to solve it. ## conclusion this paper proposes a novel synonym generalization framework, i.e. syngen, to solve the bioner task with a dictionary. we propose two novel regularizers to further make the terms generalizable to the full domain. we conduct a comprehensive theoretical analysis of the synonym generalization problem in the dictionary-based biomedical ner task to further show the effectiveness of the proposed components. we extensively evaluate our approach on a wide range of benchmarks and the results verify that syngen outperforms previous dictionary-based models by notable margins.