Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization
Authors: Zihao Fu
Yixuan Su
Zaiqiao Meng
Nigel Collier
Venue: EMNLP 2023 Main (2023)
Keywords: Biomedical named entity recognition
NER
BioNLP
Synonym Generalization
TL;DR: The study proposes a novel Synonym Generalization framework for biomedical named entity recognition, which introduces two regularization terms to minimize the synonym generalization error.

Abstract: Biomedical named entity recognition is one of the core tasks in biomedical natural language processing (BioNLP). To tackle this task, numerous supervised/distantly supervised approaches have been proposed. Despite their remarkable success, these approaches inescapably demand laborious human effort. To alleviate the need of human effort, dictionary-based approaches have been proposed to extract named entities simply based on a given dictionary. However, one downside of existing dictionary-based approaches is that they are challenged to identify concept synonyms that are not listed in the given dictionary, which we refer as the synonym generalization problem. 
In this study, we propose a novel Synonym Generalization (SynGen) framework that recognizes the biomedical concepts contained in the input text using span-based predictions. In particular, SynGen introduces two regularization terms, namely, (1) a synonym distance regularizer; and (2) a noise perturbation regularizer, to minimize the synonym generalization error. To demonstrate the effectiveness of our approach, we provide a theoretical analysis of the bound of synonym generalization error. We extensively evaluate our approach on a wide range of benchmarks and the results verify that SynGen outperforms previous dictionary-based models by notable margins. Lastly, we provide a detailed analysis to further reveal the merits and inner-workings of our approach.

Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization
Zihao Fu
Yixuan Su Zaiqiao Meng Nigel Collier
Language Technology Lab, University Cambridge
Cohere
School of Computing Science, University of Glasgow
[zf268, nhc30)0cam ac. uk
yi vyixuan@cohere.cm com
Q zaiqiao. meng@glasgow.cm ac. uk
Abstract
numerous downstream BioNLP tasks (Leaman and Gonzalez, 2008; Kocaman and Talby, 2021).
Biomedical named entity recognition is one of the core tasks biomedical natural lan- guage processing (BioNI.P). To tackle this task, numerous supervised/ddistanty supervised approaches have been proposed. Despite their remarkable success, these approaches in- escapably demand laborious human effort. To alleviate the need of human effort, dictionary- based approaches have been proposed to ex- tract named entities simply based on a given dictionary. However, downside of existing dictionary-bew -based approaches is that they are challenged identify concept synonyms that are not listed in the given dictionary, which we refer as the synonym generalization problem.
Existing BioNER approaches can be generally classified into three categories, i.e. (1) supervised methods; (2) distantly supervised methods; and (3) dictionary-based methods. Supervised meth- ods (Wang et al., 2019b; Lee et al., 2020; We- ber et al., 2021) train the BioNER model based large-scale human-annotated data. However, annotating large-scale BioNER data expensive as it requires intensive domain-specific human la- bor. To alleviate this problem, distantly supervised methods (Frics et al., 2017; Zhang et al., 2021; Zhou et al., 2022) create a weakly annotated train- ing data based on an in-domain training corpus. Nonetheless, the creation of the weakly annotated data still demands significant amount of human effort (Fries et al., 2017; Wang al., 2019a; Shang et al., 2020). For instance, the preparation of the in-domain training corpus could be challenging as the corpus is expected to contain the corresponding target entities. To this end, most existing methods (Wang et al., 2019a; Shang et al., 2020) simply use the original training set without the annota- tion as the in-domain corpus, which greatly limits their applicability more general domains. In con- trast the pnperviied/dditaatty--spperes ods, dictionary- based methods able to train the model without luman-annotated data. Most of the cxisting dictionary- based frameworks (Aronson, 2001; Song et al., 2015; Soldaini and Goharian, 2016; Nayel et al., 2019; Basaldella et al., 2020) identify phrases by matching the spans of the given sentence with entities of a dictionary, thereby avoid- ing the need of extra human involvement or in- domain corpus. As human/expert involvement in the biomedical domain is usually much more ex- pensive than in the general domain, in this paper, we focus our study on the dictionary- based method for the task of BioNER. Although dictionary- based approaches do not re-
In this study, we propose novel Synonym Generalization (SynGen) framework that rec- ognizes the biomedical concepts contained in the input text using span-based predictions. In particular, SynGen introduces two regulariza- tion terms, namely, (1) synonym distance reg- ularizer: and (2) a noise perturbation regular- izer, minimize the synonym generalization error. To demonstrate the effectiveness of our approach, we provide a theoretical analysis of the bound of synonym generalization emorr. We extensively evaluate our approach on wide range of benchmarks and the results verify that SynGen outperforms previous dictionary-bu based models by notable margins. Lastly, we provide a detailed analysis to further reveal the merits and inner-workings of our approach."
meth-
1 Introduction
Biomedical Named Entity Recognition (BioNER) (Settles, 2004; Habibi et al., 2017: 7: Song et al., 2021; Sun et al., 2021) is one of the core tasks in biomed- ical natural language processing (BioNI.P). It aims to identify phrases that refer to biomedical entities, thereby serving as the fundamental component for
'The data and source code of this paper can be ohtained from ht tttps: / //ubthub.bbtttt www/fuzimiiaeetooo www.waww..../aeeeoaaooo htttthh/////eyyiieeeee SynGen

---

quire human intervention in-domain corpus, they suffer from the synonym generalization problem, i.e. the dictionary only contains a limited number of synonyms of the biomedical concepts that appear in the text. Therefore, an entity synonym in the text is not explicitly mentioned by the dictionary, it cannot be recognized. This problem severely un- dermines the recall of dictionary-baan -bsed methods as, potentially, huge amount of synonyms not contained in the dictionary.
PublMed Milophagy controls
Mitophagy controls
Covid-1 19 disease
mitochondria quality maintain cellular
cellular homeostasis
UMLS
Training Stage
C5203670 Covid-19 C5203670 Covid-19 virus C0016627 H5N1
Mitophagy controls
Cellular homeostasis
The patient
SynGen
palient infected
11,+.0 100L
Covid-19 Covid-19 19 virus
To address the synonym generalization problem. we propose SynGen (Synonym Generalization) a novel framework that generalizes the synonyms contained in the given dictionary to broader do- main. Figure 1 presents overview of our ap- proach. (1) In the training stage, SynGen first sam- ples the synonyms from a given dictionary as the positive samples. Meanwhile, the negative sam- ples are obtained by sampling spans from general biomedical corpus. Then, it fine-tunes a pre-trained model to classify the positive and negative samples. In particular, SynGen introduces two novel regu- larizers, namely a synonym distance regularizer which reduces the spatial distance; and noise perturbation regularizer which reduces the gap of synonyms' predictions, minimize the synonym generalization error. These regularizers make the dictionary concepts generalizable to the entire do- main. (2) During the inference stage, the input text is split into several spans and sent into the fine- tuned model to predict which spans are biomedical named entities. To demonstrate the effectiveness of the proposed approach, ical analysis show that both of proposed regularizers lead to the reduction of the synonym generalization error.
The patient infected by Covid-19 virus
Inference Stage
Figure I: SynGen framework. represents the training steps while --+ represents the inference steps.
We provide theoretical analysis showing that the optimization of SynGen is equivalent to minimizing the synonym generalization error.
We conduct extensive experiments and anal- yses to demonstrate the effectiveness of our proposed approach.
2 Methodology
In this section, we first give the definition of the dietionary-based biomedical NER task. Then, we introduce our Synonym Generalization (i.e. Syn- Gen) framework, following by the details of the synonym distance regularizer and the noise pertur- bation regularizer. Lastly, we provide theoretical analysis on the problem of synonym generalization to show the effectiveness of our proposed approach.
provide theoret-
2.1 Task Definition
We extensively test our approach on five well- established benchmarks and illustrate that SynGen brings notable performance improvements over pre- vious dictionary-base based models on most evaluation metrics. Our results highlight the benefit of both of our proposed regularization methods through detailed ablation studies. Furthermore, we vali- date the effectiveness of SynGen under the few- shot setup, notably, with about 20% of the data, achieves performances that are comparable to the results obtained with full dictionary. In summary, our contributions are:
Given biomedical domain D (e.g. disease do- main chemical domain), we denote the set of all possible biomedical entities in D as S {8) {81, 8| where 8; denotes the i-th entity and [S] denotes the size of S. Then, given an input text x, the task of biomedical NER to identify sub- spans inside x that belong to S, namely finding
( (bye)):""" .... Flbbees VV E [1, k),, Fbbees] E S). where k is the number of spans, and b, E [1, are the beginning and the ending token indices in æ for the i-th span, respectively.
However, in real-life scenarios, it is impracti- cal to enumerate all possible entities in S. Nor- mally, we only have access to a dictionary S C S, S = {ō1.... 8 { which contains a subset
We propose SynGen - novel dictionary- based method to solve the BioNER task via synonym generalization.

---

entities that belong to 8.2 Thereby, the goal of dictionary-baas biomedical NER is then to max- imally recognize the biomedical entities from the input text conditioned on the available dictionary.
the spans that are biomedical entities), we encode all sampled spans into the embedding space and
remove the samples that are close to the entities contained in the given dictionary. Specifically, we denote the set of sampled negative spans as N. Then, Vn, E EN, it satisfies
2.2 Synonym Generalization Framework
Figure/ depicts an overview of the proposed Syn- Gen framework. (1) In the training stage ($2.2.1), it samples synonyms from a given dictionary (e.g. UMLS) positive samples. Meanwhile, the nega- tive samples are obtained by sampling spans from a general biomedical corpus (e.g. PubMed). Then, SynGen learns to classify positive and negative samples through the cross-entropy objective. More- over, we propose two regularization methods (i.c. synonym distance regularization and noise pertur- bation regularization), which have been proved to be able to mitigate the synonym generalization er- ror (detailed in $2.3). (2) In the inference stage ($2.2.1), SynGen splits the input text into differ- ent spans and scores them separately following a greedy extraction strategy.
min v&cS
|IF(())- F(R,))) > tas (3)
where la is a hyper-parameter that specifies the threshold of minimal distance, and F(-) an off- the-shelf encoder model. In our experiments, we use SAPBert (Liu et al., 202la,b) as the model Fi(.) since it is originally designed to aggregate the synonyms of the same biomedical concept in the adjacent space.
Synonym Distance Regularizer (SDR). Intu- itively, if the distinct synonyms of single con- cept are concentrated in a small region, is easier for the model to correctly identify them. Thereby, to equip our model with the ability to extract dis- tinct synonyms of the same biomedical concept, we propose a novel regularization term- Synonymm Distance Regularizer (SDR). During training, SDR first samples an anchor concept 8, and its asso- ciated concept &p from the dictionary S.4 Then, random negative sample no E N is sampled. Finally, SDR is computed by imposing a triplet margin loss (Chechik et al., 2010) between the en- coded sampled synonyms and the sampled negative term as
2.2.1 Training
During training, we first sample a biomedical entity & (i.e. the positive sample) from the dictionary S and encode its representation as A; F(S,), where E(-) is an encoder model such as BERT (Kenton and Toutanova, 2019). The probability of & being biomedical entity is then modelled
p( $.) = a (MLP(F:)).
(1)
R2 max{ Apll - - Full + 7s. 0}, (4) where Vo is a pre-defined margin, and A. - E(sa).
where MLP(.) contains multiple linear layers and (-) is the sigmoid function. Meanwhile, we sam- ple negative text span n; from a general biomed- ical corpus, i.c. PubMed? and compute its repre- sentation as well as its probability as r; = E(n)) and p(ñ,) = o (ML.(ñ,)). respectively. Lastly. the training objective of our model is defined as
Pp F((S)) and F.. = E(ūx), respectively. In $2.3, we provide rigorous theoretical anal- ysis to show why reducing the distance between synonyms leads minimizing the synonym gener- alization error.
S
Noise Perturbation Regularizer (NPR). An- other way to mitigate the scoring gap between biomedical synonyms is to reduce the sharpness of the scoring function's 's landscape (Foret et al., 2020; Andriushchenko and Flammarion, 2022). This is because the synonyms of one biomedical entity are expected distribute in close-by region. Based on this motivation, we propose new Noise Pertur- bation Regularizer (NPR) that defined as
1
L. =
(n.( + In(1 - p(n.)). (2)
S
Negative Sampling Filtering (NSF). To obtain the negative sample n; we first sample spans with a random length from the PubMed corpus. Then, to avoid erroneously sampling the false negatives (i.e.
"There are many biomedical dietionaries that are publicly available, such as UMLS (Bodenreider. 2004), Snomed CT settps://wwwe snomed. org). MeSHI (https://mesh) nmmm nih. gov). etc.
Rr = ||p( + v) p((:) l|,
(5)
pettts::/pubmed.neee ncbi. alm. gov/download/
"The å and & share the same concept ID.

---

where &; is biomedical entity sampled from the dictionary S and is a Gaussian noise vector. In- tuitively, NPR tries to flatten the landscape of the loss function by minimizing the loss gap between vectors within a close-by region. More discussion of increasing the function flatness can be found in Foret et al. (2020); Bahri et al. (2022). In $2.3, we theoretically show why NPR also leads to the reduction of synonym generalization error.
recognized entity will never be recognized again because they will not be contained in the corre- sponding validation sequence.
2.3 Theoretical Analysis
Most of the existing dictionary- based frameworks suffer from a common problem that terms outside of the given dictionary cannot be easily recog- nized, which we refer to as the synonyms gener- alization problem. To understand why the Syn- Gen framework can resolve this problem, we give theoretical analysis focusing the cor- rectness of entities in S. Specifically, given bounded negative log- likkkiikhhood loss function f(r) In (MLP(r)) E b0,
Overall Loss Function. The overall learning ob-
jective of our SynGen is defined as
C= + RRg + 3R..
(6)
E(s), it tends to O an entity a S is correctly classified as positive. Otherwise, f(r) tends to b if the en- tity 8 is wrongly recognized as a non-biomedical phrase. Following the traditional generalization error (Shalev-Shwartz and Ben- Ben-David, 2014), we further define the average empirical error for enti- ties in the dictionary S as R 1 better analyze the generalization error for all syn- onyms, we consider the most pessimistic enror gap between f and the error of arbitrary S, namely f (()))). Then, the synonym generalization error can be defined as follows:
where and are tunable hyperparameters that regulate the importance of the two regularizers.
2.2.2 Inference
During inference, given the input text æ, Syn- Gen first splits into spans with different lengths, namely, X { < i S S |x].] i I m,). where m, is the maximum length of the span. Then, we compute the score of every span T[yy] E X as p())))) with Equation (1). We select the spans whose score is higher than a pre-defined threshold tp as candidates, which are then further filtered by the greedy extraction strategy as intro- duced below.
E (P..). To
Definition 1 (synonym generalization error). Given a loss function f(r) E [0, the synonym generalization ervor is defined as:
Greedy Extraction (GE). It has been observed that a lot of biomedical terms are nested (Finkel and Manning, 2009; Marinho et al., 2019). For example, the entity T-cell prolymphocytic leukemia contains sub-entities T-cell and leukemia. How- ever, in most of the existing BioNER approaches, if one sentence x contains the entity T-cell prolym- phocytic leukemia, these approaches usually only identify it as a single biomedical entity and ignore its sub-entities T-cell or leukemia. To address this issue, our SynGen applies a greedy extraction (GE) strategy to post-process, the extracted biomedical terms. In particular, our GE strategy first ranks the recognized terms by their length in descending orders as T = {t1, t2..... tn In j, |ta] [tyl) and set the initial validation sequence (1) æ. Then, it checks the ranked terms from t, t,. the term & is a sub-sequence of the valida- tion sequence (+() (i.c. Ep.q < [z( such that 4 x = will recognize the term 6; as a biomedical entity and set new validation sequence x (++1) by removing all t,'s occurrence in the se- quence (() As a result, the sub-entities inside a
E, (uspp(((((s)) - R). SES
It can be observed from Definition 1 that small E, implies error (EE()) for arbitrary s will no de- viate too far from R. Therefore, training ʃ with the dictionary terms S generalizable to the entities in the whole domain S. To give a theoretical analysis of Es, we further assume that S is an n-net of S, namely, Vs E S, =s E S, such that || - s S Intuitively, given an entity s E S, we can always find an entity á in the dictionary S within distance We further assume that ʃ is s-Lipschitz, namely, If - f(y) S x|| x - y||- Then, we have the following bound hold.
Theorem 1 (Synonym Generalization Error Bound), If $ an c-net SSS. f E [o, b) is n-Lisschitz continuous. We have with probability at least 5,
S. The loss function
In S + In
ln 2
E. < (REE + bbl
+ b
(7)
2
S

---

The proof can be found in Appendix A. It can be observed from Theorem 1 that reducing the syn- onym distance upper bound e or function f's Lips- chitz constant A can help reduce the generalization error gap g.g-
2021a,b), BioBert (Lee et al., 2020), SciBert (Belt- agy al., 2019), and Bert (Kenton and Toutanova. 2019) (Detailed comparison can be found in Ap- pendix C). Our experiments are conducted on a server with NVIDIA GeForce RTX 3090 GPUs. For all the experiments, we report the average per- formance of our used three metrics over 10 runs.
Theorem 1 explains why both SDR and NPR are able to help improve the NER performance. (1) For SDR, it allows the model to learn to reduce the distance between synonyms, which is equivalent reducing e of Equation (7). Therefore, it helps to reduce the synonym generalization error upper bound. (2) For NPR, helps reducing the Lipschitz constant & because given a vector v, minimizing R., is equivalent to minimizing
3.2 Comparison Models
We compare our model with baseline models de- pending on different kinds of annotations or extra efforts. The supervised models (BioBert and SBM) require golden annotations. The distantly super- vised models (SBMCross, SWELLSHARK and AutoNER) depend on several different kinds of an- notation or extra efforts which will be discussed in the corresponding models. The dictionary- based models mainly use the UMLS dictionary.
+ v) s(+) [
[ v) f(a:)) (#):
17
+ v) 2.1
as v fixed during the parameter optimization procedure. Therefore, optimizing R2 is a necessary condition for reducing f's Lipschitz constant k.
BioBert (Lee al., 2020) first pre-trains an en- coder model with biomedical corpus and then fine- tunes the model on annotated NER datasets.
3 Experiments
SBM is standard Span-Based Model (Lee et al., 2017: Luan et al., 2018, 2019; Wadden et al., 2019; Zhong and Chen, 2021) for NER task. We use the implementation by Zhong and Chen (2021). SBMCross utilizes the same model as SBM. We follow the setting of Langnickel and Fluck (2021) to train the model on one dataset and test it on the other dataset in the same domain which is referred as the in-domain annotation. For example, in the NCBI task, we train the model on the BC5CDR- D dataset with SBM and report the results on the NCBI test set.
3.1 Experimental Setup
We evaluate all the models on 6 popular BioNER datasets, including two in the disease domain (i.e. NCBI disease (Dogan et al., 2014) and BC5CDR- D (Li et al., 2016)), two in the chemical domain (i.e. BC4CHEMD (Krallinger et al., 2015) and BCSCDR-C (Li et al., 2016)) and two in the species domain (i.e. Species-800 (Pafilis al., 2013) and LINNAEUS (Gerner et al., 2010)). Note that BC5CDR-D and BC5CDR-C are splits of the BC5CDR dataset (Li et al., 2016) for evaluating the capability of recognizing entities in the disease and chemical domains respectively, following Lee et al. (2020). We evaluate the performance by re- porting the Precision (P), Recall (R), and F1 scores. The entity name dictionary used in our model is extracted by the concepts' synonyms in diseases, chemicals, and species partition from UMLS (Bo- denreider, 2004). The negative spans are randomly sampled from the PubMed corpus" 3.
SWELLSHARK (Fries et al., 2017) proposes to first annotate corpus with weak supervision. Then uses the weakly annotated dataset to train an NER model. It requires extra expert effort for designing effective regular expressions as well as hand-tuning for some particular special cases.
AutoNER (Wang et al., 2019a; Shang et al., 2020) propose first train an AutoPhase (Shang et al., 2018) tool and then tailor domain-specific dic- tionary based on the given in-domain corpus. The corpus is the original training set corpus without human annotation. Afterwards, it trains the NER model based on the distantly annotated data. We also report the ablation experiments by removing the dictionary tailor replacing the in-domain cor- pus with evenly sampled PubMed corpus.
We tune the hyper-parameters of SDR (i.e. a), NPR (i.e. 3), threshold constant tul tp and max- imal span length m, by using grid search on the development set and report the results on the test set. The hyper-parameter search space well as the best corresponding setting can be found in Ap- pendix D. We use PubMedBert (Gu al., 2021) as the backbone model by comparing the development set results from PubMedBert, SAPBert (Liu et al.,
EmbSim first uses a pre-trained model to encode the input spans and the entities in the UMLS dictio-

---

NCBI
BC5CDR-D
BCSCDR-C
BCACHEMD
Species-800
LINNAEUS
AVG
Model
P
R
Fi
P
R
F
P
R
F
R
F
P
R
F1
P
R
F
R
F1
Dietinney- -8oood Distanily Supervied
BioBert
91.2
89.7
86.5
87.8
87.2
93.7
93.3
93.5
92
12.
75.4
90.8
85.8
88.2
87.5
87.6
87.5
SBM*
88.4
88.9
88.6
83.4
86.4
84.9
93.2
93.6
93.4
92.0
86.6
89.2
99.5
91.6
95.4
99.8
80.1
88.9
92.8
87.9
90.1
SBMCross ?
75.9
58.3
66.0
70.1
61.3
65.4
94,1
86,4
90.1
72.
63.2
67.4
64.2
64.5
64.3
78.8
45.8
57.9
75.9
63.2
68.5
SWELLSHARK*
64.7
69.7
67.1
80.7
77.6
79.1
88.3
88.3
88.3
77.9
78.5
78.2
AutONEER
79.4
72.0
75.5
86.2
67.9
76.0
85.2
84.2
84.7
91.1
18.9
31.3
86.6
90.9
88.7
92
95.6
93.8
86.8
71.6
75.0
AutoNER w/o DT
66.8
32.4
43.6
72.0
17.3
27.9
89.7
67.3
76.9
90.7
19.7
32.4
57.6
50.7
53.9
88
39.0
54.1
77.5
37.7
48.1
AutoNER IDe
19.1 31.2
94.2
37.3
31.2
83.6
62.8
45.9
AutoNER wo DT+IDC
57.9 9.7 16,6
63.0
13.5
22.8
92.8
39.3
55.2
60.9
24.6
35.1
59.8
25.0
35.3
80.1
33.0
46.8
69.1
24.2
35.3
EmbSim
56.7
24.9
34.6
61.8
14.3
23.2
71.7
61.2 66.0
47.
24.7
32,4
49.0
34,2
40.3
80
42.9
55.9
61.2
33.7
42.1
MataMap
61.8
27.8
38.4
69.3
13.3
22.3
65.9
63.5
64.7
33.1
25.2
28.6
56,9
48.7
52.5
85.5
44.3
58.3
62.1
37.1
44.1
MetaMap (Uncused)
$8.4 27.5
37.4
63.5
184
28.6
94,8
64.1
76.5
86.2
24.0
37.5
49.1
52.3
50.6
79.1
49.6
61.0
71.
39.3
48.6
SPED
30.1
39.9
68.2
65.6
63.9
64.8
33.0
25.4
28.7
56.0
49.4
52.5
85
44.7
58.7
61.2
38.0
44.7
TF-IFF
26.1
29.7
27.7
32.0
22.6
26.4
74.1
65.4
69.5
19.1
39.3
25.7
42.5
21.4
28.4
77.3
40.5
53
45.2
36.5
38.5
QuickUMES
17.2
28.4
93.5
93.2
56.9
28.1
61.7
46.7
88.2
44.7
59.3
SynGien
68.8
64.1
66.2
63.8
63.4
63.5
85.0
83.9
84.4
56.4
S1.1
53.6
58.8
65.7
62.0
84.9
66.2
74.4
69.6
65.7
67.4
Table I: Main results. We repeat each experiment for 10 runs and report the averaged scores, For BioBert and SWELLSHARK, we report the score from the original paper. We mark the extra effort involved with superseripts, where A is gold annotations; is in-domain annotations; is regex design; A is special case tuning: is in-domain corpus; is dictionary tailor. The bold values indicate the best performance among the dictionary-based models. The standard deviation analysis is in Figure
nary into the corresponding vector representation. Then, it calculates the minimal distance between a given span and an arbitrary entity in the dictio- nary. It recognizes spans with minimal distances smaller than a threshold as named entities. We re- port the performance based on BioBert. For the results using other backbone models, please refer to Appendix C.
3.3 Experimental Results
Main Results. We first compare the overall per- formance of our proposed SynGen framework with the baseline models, and the results are shown in Table I. It can be observed that: (1) Our proposed SynGen model outperforms all other dictionary- based models in terms of F1 score. This is because it alleviates the synonym generalization problem and can extract entities outside of the dictionary. As a result, the recall scores are significantly im- proved. (2) By comparing SBM and SBMCross, can find that the performance is very sensitive to the selection of the in-domain corpus. Even us- ing the golden annotation of another training set in the same domain can lead to a sharp performance decrease. Therefore, preparing a good in-domain corpus is quite challenging, (3) SynGen is already comparable to the performance of SBMCross with average F1 scores of 67.4 and 68.5 respectively. It shows that our dictionary dietionnryy-baeed model is com- parable supervised model without in-domain data. (4) The precision score (P) of QuickUMLS is very high. This is because it is mainly based on matching exact terms in the dictionary and may not easily make mistakes. However, it cannot handle the synonyms out of the UMLS dictionary. As a result, it fails to retrieve adequate synonyms, lead- ing to low recall scores. (5) By comparing the ablation experiments for AutoNER, we can con- clude that the human labor-intensive dictionary tai- lor and the in-domain corpus selection are very important for distantly supervised methods. With- out the dictionary tailor or the in-domain corpus,
MetaMap (Aronson, 2001; Divita et al., 2014; Sol- daini and Goharian, 2016) proposes to perform exact concept mapping of spans with the entities in the UMLS dictionary. We report the results for both cased and uncased matching.
SPED (Rudniy al., 2012: Song al., 2015) cal- culates the Shortest Path Edit Distances between the query span and each entity in the dictionary. Then recognizes a span as a biomedical entity the minimal distance ratio is smaller than a specific threshold.
TF-IDF follows the model of Ujiie et al. (2021). We remove the component utilizing the annotated data and only keep the if- features to make it comparable to other models without extra effort.
UMLS (Soldaini and Goharian, 2016) is a fast upproximate UMLS dictionary matching sys- tem for medical entity extraction. It utilizes Sim- string (Okazaki and Tsujii, 2010) as the matching model. We report the scores based on Jaccard dis- tance. For the performance using other distances, please refer to Appendix C.

---

NCBI
BCSC
BCSC
BC4C HEMD
Speci
LINN
Backbone of NCBI
DCSC DR-D
BCSC
BC4C HEMD
Speci e5-800
LINN ABUS
AVG
DR-D
DR-C
e5-800 ABUS
Syntion
DR-C
SynGen
66.2
63.55
84.4
53.6
62.0
74.4
67.4
PuhMedBert
66.2
63.5
84.4
53.6
62.0
74.4
67.4
w/o SDR
66.2
63.1
80.8
51.1
60.6
73.0
65.8
SAPBent
66.3
62.1
83.8
42.4
58.1
71.6
64,0
we NPR
66.2
62.8
78.0
49.7
60.3
73.1
65.0
BioBert
62.4
57.4
81.0
31.7
52.4
59.3
57.4
w/ NPR-SDR
64.7
58.7
76.9
49.0
59.7
72.0
63.5
Bert
62.0
56.1
79.6
47.1
55.2
62.1
60.4
Wo NSP
57.4
81.0
53.1
63.7
w/o GE
49.8
54.4
69.4
33.4
52.1
71.7
55.1
Table Comparison of different backbone models".
Table 2: Ablation study?.
by the SDR component does help improve the overall performance, we plot how the synonym distance changes as the SDR weight increases Figure 6. Specifically, we train the model with dif- ferent
the performance drops sharply. It should also be noted that our model outperforms the AutoNER model without in-domain corpus or dictionary tai- loring significantly. It shows the effectiveness of our model without using extra effort (e.g. tailored dictionary).
a and measure the synonym distance by sampling 10,000 synonym pairs from UMLS. Then, we calculate the average distance between synonym name pairs for different As suggested in Figure 6, a increases, the synonym distance decreases which shows the effectiveness of the SDR component in controlling the synonym distance. On the other hand, we also plot how the evaluation scores change as the synonym distance increases in Figure 2. For the previous results, we further split the distance intervals into 8 segments and get the average overall performance in each interval. The results indicate that as the synonym distance is regularized (decreases), the overall per- formance increases. This observation shows the effectiveness of our proposed SDR component and justifies the analysis in Theorem 1.
Ablation Study. In order to show the effective- ness of our model's each component, we conduct several ablation experiments over different variants of our model, and the results are shown in Table 2 and Table 3. We have the following observations from the two tables. (1) By comparing the vanilla SynGen model with its variants, such as SynGen w/o NPR, SynGen w/o SDR, and SynGen w/o NPR + SDR, we can observe that removing any compo- nent results in performance drop, which means that our proposed NPR and SDR component can effec- tively improve the performance. This observation further verifies the correctness of our theory in The- orem 1. (2) By comparing SynGen with SynGen w/o NSF, we can find that the negative sample fil- tering component is also quite important in finding proper negative samples. (3) By comparing Syn- Gen with SynGen w/o GE model, we also observe a significant performance drop, which concludes that the greedy extraction strategy does help improve the overall performance. Specifically, it helps im- prove the precision it will not extract sub-entities inside other entities and thus avoid false positive ex- tractions. (4) We also try to use different backbone models including PubMedBert, SAPBert, BioBert, Bert, and SciBert and the results are shown in Ta- ble 3. Our experiments show that PubMedBert per- forms the best. These results may be caused by the different training corpus used by each pre-trained model.
Influence of Noise Perturbation. usefulness of our proposed NPR component, we plot how the scores change as the NPR weight (i.c. 3) changes. The results are shown in Figure 3. It indicates that as 3 increases, all metrics including precision, recall, and F1 scores increase. This ob- servation shows that the NPR component does help improve the performance which also justifies our theoretical analysis.
To show the
Few-Shot Analysis. To further show that our SynGen framework can applied the few-shot scenarios, we run the model with part of the original dictionary entries with the dictionary size ratio ranging in {0.1%, 0.2%, 0.5%. 1%, 2%, 5%, 10%, 20%, 30%, 40%, 50% 60%, 70%, 80%, 90%, 100%).
Influence of Synonym Distance Regularizer. To show that regularizing the synonym distance
The results shown in Figure 4. To better show the capability of few-shot learning, we also draw the same figure for the MetaMap model as shown in Figure 5. It can be concluded from the results that in SynGen, when the dictionary size is very
'The full results including precision and recall results can found Appendix "The full results including precision and recall can he found in Appendix C.

---

0.3
0.8
BESCDR-D
NCBI
BESCDR-C
0.6
0.58
0.62
0.4
4C 0.4
0.56
Precision Recall
0.60
Precision
BC5CDR-D
0.2
BCSCDR-C
0.0
0.58
0.0
0.2
0.4
0.6
0.0
Synonym distance
Few
(%)
Figure 2: Influence of syn- onym distance.
Figure NPR.
3: Influence of Figure 4: Few-shot analy-
Figure 3: Few-shot analy- sis for MetaMap.
si8.
7.50
SynGen w/f NPR-SDR
Synotym disrance
SynGien
Label
UMLS
7.25
7.00
manic depressive
6.75
illness
affective disorders
6.50
atrophic benign
epidemnlyyiis
4
bullosa
6.00
renal cell carcinoma
/
0.00
0.01
0.03
0.04
0.05
Friedreich
progressive optic atrophy
Figure 6: Influence of synonym distance regularizer's weight a.
primury prosiate
X
X
X
cancer
nystagmus
XX
chronic
copper
overload disease
small, the performance increases as the dictionary size increases. Using nearly only 20% dictionary entries can achieve comparable results using the full of dictionaries. However, in the MetaMap model, the performance increases linearly as the dictionary size increases. It shows that the word match based model cannot handle the few-shot cases. This observation shows the potency of using our approach to undertake few-shot learning. It should be noted that the performance stops increasing after a certain ratio. This observation can also be explained with Theorem Increasing the dictionary size can only mitigate the second term in Equation (7). After the second term decreases to a certain amount, the first term dominates the error and continues increasing the dictionary size cannot further reduce the upper bound. This observation can further verify the correctness of our theoretical analysis. To further should consider
Table 4: Case Study.
performs the model variants without the proposed components, which further validates the effective- ness consistency of each component of our SynGen and the effectiveness of the overall SynGen frame- work.
Case Study. We conduct a case study to demon- strate how our proposed NPR and SDR compo- nents enhance performance. As shown in Table 4, we select range of terms from the NCBI cor- pus. Using SynGen, both with and without the NPR+SDR component, predict whether each term is biomedical term. A term predicted biomedical term is marked with check mark (V); otherwise, it's marked with a cross (X). Our findings reveal that SynGen accurately identifies certain terms like "maternal UPD 15" as biomed- ical, even if they are not indexed with the UMLS. However, without the NPR+SDR components, the system cannot recognize such terms, underscoring the significance of these components. Morcover, SynGen is designed to prevent misclassification of common words (e.g.. "man". "breast") and peculiar spans like "t (3; 15)" biomedical entities. With- out our proposed components, the system might erroneously categorize them, further emphasizing
improve the performance, reducing the first term in Equation (7) and this is how our proposed SDR and NPR work.
Standard Deviation Analysis. To further show the effectiveness and consistency of our proposed components, conduct standard deviation anal- ysis shown in Figure We run each model for 10 runs with different random seeds and draw the box plot of the F1 scores. We can see from Fig- ure 7 that our proposed SynGen consistently out-

---

NCBI
BC5CDR-D
BC5CDR-C
BC4CHEMD
0.85
0.5
0.80
0.6
0.75
0.5
SynGen wio SDR wi NPR d
0.65
0.3
0.4
SynGen w/o SDR w/o NPR
w/o NPRWi Wiee w/o GE
w/o Wh Worffss w/o GE
SynGen wo SDR w/o NPR
wi NPR woo
w/o GE
SynGen w/o SDR w/o NPR w/o NPR WoRR w/o GE
Figure 7: The box plot of each model's F1 score over 10 runs.
the essential role of the NPR+SDR components in SynGen.
there is still no existing work that gives a theoretical analysis of the synonyms generalization problem and proposes a corresponding method to solve it.
4
Related Works
Existing NER methods can be divided into three categories, namely, supervised methods, distantly supervised methods, and dictionary-based methods. In supervised methods, Lee et al. (2020) propose to pre-train biomedical corpus and then fine-tune the NER model. Wang et al. (2019b); Cho and Lee (2019); Weber al. (2019, 2021) develop several toolkits by jointly training NER model on multiple datasets. However, supervised annotation is quite expensive and human labor-intensive.
Conclusion
This paper proposes novel synonym generaliza- tion framework, i.e. SynGen, to solve the BioNER task with dictionary. We propose two novel reg- ularizers further make the terms generalizable to the full domain. We conduct a comprehensive theoretical analysis of the synonym generalization problem in the dictionary-based biomedical NER task to further show the effectiveness of the pro- posed components. We extensively evaluate our approach wide range of benchmarks and the results verify that SynGen outperforms previous dictionary- models by notable margins.
In the distantly supervised methods, Lison et al. (2020, 2021); Ghiasvand and Kate (2018); Meng et al. (2021); Liang al. (2020) propose first con- duct weak annotation and then train the BioNER model on it. Fries al. (2017); Basaldella al. (2020) propose to use utilize a well-designed regex and special case rules to generate weakly super- vised data while Wang et al. (2019a); Shang et al. (2020) train an in-domain phase model and make a carefully tailored dictionary. However, these meth- ods still need extra effort to prepare high-quality training set.
Limitations
Although the dictionary-based methods achieve considerable improvements, there is still an over- all performance gap compared with the super- vised models. Therefore, for domains with well- annotated data, it is still recommended to apply the supervised model. Our proposed SynGen frame- work is suggested be applied domains where there is no well-annotated data.
In the dictionary-based methods, Zhang and El- hadad (2013) propose rule-based system ex-
tract entities. Aronson (2001); Divita et al. (2014);
Soldaini and Goharian (2016); Giannakopoulos et al. (2017); He (2017); Basaldella et al. (2020) propose to apply exact string matching meth- ods to extract the named entities. Rudniy et al. (2012); Song et al. (2015) propose to extract the entity names by calculating the string similarity scores while QuickUMLS (Soldaini and Goharian, 2016) propose to use more string similarity scores (Okazaki and Tsujüi 2010) to do a fast approximate dictionary matching. To the best our knowledge,
Broader Impact Statement
This paper focuses on biomedical named entity recognition. The named entity recognition task is standard NLP task and we do not make any new dataset while all used datasets are properly cited. This work does cause any kind of safety security concerns and it also does not raise any human rights concerns or environmental concerns.

---

References
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. 2020. Sharpness-awan aware mini- mization for efficiently improving generalization. In International Conference on Learning Representa- lions.
Maksym Andriushchenko and Nicolas Flammarion. 2022. Towards understanding sharpness-aware mini- mization. In International Conference on Machine Learning, pages 639-668. PMLR.
Jason Fries, Sen Wu, Alex Ratner, and Christopher Ré. 2017. Swellshark: A generative model for biomed- ical named entity recognition without labeled data. arXiv preprint arXiv: 1eXivv17744 1704006360.
Alan R Aronson. 2001. Effective mapping of biomed- ical text to the umls metathesaurus: the metamap program. Proceedings of the AMIA Symposium, page 17. American Medical Informatics Association.
Martin Gemerr, Goran Nenadic, and Casey M Bergman. 2010. Linnaeus: a species name identification sys- tem for biomedical literature. BMC bioinformatics. 11(1):1-17.
Dara Bahri, Hossein Mobahi, and Yi Tay. 2022. Sharpness-aware minimization improves language model generalization. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7360- 7371.
Omid Ghiasvand and Rohit J Kate. 2018. Learning for clinical named entity recognition without man- ual annotations. Informatics in Medicine Unlocked, 13:122-127.
Marco Basaldella, Fangyu Liu, Ehsan Shareghi, and Nigel Collier. 2020. Cometa: A corpus for medical entity linking in the social media. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3122- 3137.
Athanasios Giannakopoulos, Claudiu Musat, Andreea Hossmann, and Michael Baeriswyl. 2017. Unsuper- vised aspect term extraction with b-Istm & erf using automatically labelled datasets. In Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 180-188.
Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Scibert: pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615-3620
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama. Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Nomain-specific lan- guage model pretraining for biomedical natural lan- guage processing. ACM Transactions on Compuring for Healthcare (HEALTH), 3(1):1-23.
Olivier Bodenreider. 2004. The unified medical lan- guage system (umls): integrating biomedical termi- nology. Nucleic acids research, 32: D270.
Maryam Habibi. Leon Weber, Mariana Neves. David Luis Wiegandt, and Ulf Leser. 2017. Deep learning with word embeddings improves biomed- ical named entity recognition. Bioinformatiiss 33( 33(4433337
Gal Chechik, Varun Sharma, Uri Shalit, and Samy Ben- gio. 2010. Large scale online learning of image simi- larity through ranking. Journal of Machine Learning Research, 11(3).
Wenqi He. 2017. Autoentity: automated entity detec- tion from massive text corpora.
Hyejin Cho and Hyunju Lee. 2019. Biomedical named entity recognition using deep neural networks with contextual information. BMC bioinformatics, 20.1):1-11.
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171 4171-4186.
Guy Divita, Qing T Zeng, Adi V Gundlapalli, Scott Duvall, Jonathan Nebeker, and Matthew H Samore. 2014. Sophia: a expedient umls concept extraction annotator. In AMIA Annual Symposium Proceedings, volume 2014, page 467. American Medical Informat- ies Association.
Veysel Kocaman and David Talby. 2021. Biomedical named entity recognition at scale. In International Conference on Pattern Recognition, pages 635 646.. Springer.
Martin Krallinger, Obdulia Rabal, Florian Leitner, Miguel Vazquez, David Salgado, Zhiyong Lu, Robert Leaman, Yanan Lu, Donghong Ji, Daniel M Lowe, et al. 2015. The chemdner corpus of chemicals and drugs and its annotation principles. Journal of chem- informatics, 7(1):1117.
Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014. Nebi disease corpus: a resource for dis- ease name recognition and concept normalization. Journal of biomedical informatics, 47:1-10.
Jenny Rose Finkel and Christopher D Manning. 2009. Nested named entity recognition. In Proceedings of the 2009 conference on empirical methods in natural language processing, pages 141-150.
Lisa Langnickel and Juliane Fluck. 2021. We are not ready yet: limitations of transfer learning for disease named entity recognition. bioRxiv.

---

Robert Leaman and Graciela Gonzalez. 2008, Ban- ner: an executable survey of advances in biomedical named entity recognition. In Biocomputing 2008, pages 652-663. World Scientific.
graph construction. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing, pages 3219-3233
Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hamnaneh Hajishirzi. 2019. general framework for information extraction using dynamic span graphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Pupers), pages 3036-3046.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang- 2020. Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234.
Kenton Lee. Luheng He, Mike Lewis, and Luke Zettle- moyer. 2017. End-to-end neural coreference reso- lution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188-
7ita Marinho, Alfonso Mendes, Sehastiao Miranda, and David Nogueira. 2019. Hierarchical nested named entity recognition. In Proceedings of the 2nd Clinicall Natural Language Processing Workshop, pages 28- 34.
Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sci- aky, Chih-Hssuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. 2016. Biocreative v edr task corpus: resource for chemical disease relation extraction. Database, 2016.
Yu Meng. Yunyi Zhang, Jiaxin Huang, Xuan Wang. Yu Zhang, Heng Ji, and Jiawei Han. 2021. Distamtly- supervised named entity recognition with noise- robust learning and language model augmented self- training. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing. pages 10367-10378.
Chen Liang, Yue Yu, Haoming Jiang. Siawpeng Er, Rui- jia Wang. Tuo Zhao, and Chao Zhang. 2020. Bond: Bert-assisted open-domain named entity recognition with distant supervision. In Proceedings of the 26th ACM SIGKDD International Conference Knowl- edge Discovery & Data Mining, pages 1054-1064.
Hamada A Nayel et al. 2019. Integrating dictionary feature into a deep learning model for disease named entity recognition. arXiv preprint arliv:1911.0. 1.01600.
Naoaki Okazaki and Jun ichi Tsujii. 2010. Simple and efficient algorithm for approximate dictionary match- ing. In Proceedings of the 23rd International Con- ference on Computational Linguistics (Coling 2010). pages 851-859.
Pierre Lison, Jeremy Barnes, and Aliaksandr Hubin. 2021. skweak: Weak supervision made easy for nip. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 337-346.
Evangelos Pafilis, Sune P Frankild, Lucia Fanini, Sarah Faulwetter, Christina Pavloudi, Aikaterini Vasileiadou, Christos Arvanitidis, and Lars Juhl Jensen. 2013. The species and organisms resources for fast and accurate identification of taxonomic names in text. PloS one, 866)e65390
Pierre Lison, Jeremy Barnes, Aliaksandr Hubin, and Samia Touileb. 2020. Named entity recognition with- out labelled data; A weak supervision approach. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1518- 1533.
Alex Rudniy, James Geller, and Min Song. 2012. His- togram difference string distance for enhancing on- tology integration in bioinformatics. In 4th Interna- tional Conference on Bioinformatics and Computa- tional Biology 2012, BICoB 2012, pages 108-113.
Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. 2021a. Self-alignment pretraining for biomedical entity representations. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional 1. Linguistics: Human Language Technologies, pages 4228 2238.438.
Burr Settles. 2004. Biomedical named entity recogni- tion using conditional random fields and rich feature sets. In Proceedings of the international joint work- shop on natural language processing in biomedicine and its applications WNL.PAA/BoNN.P.. pages 107- 110.
Fangyu Liu, Ivan Vulić, Anna Korhonen, and Nigel Collier. 2021b. Learning Nomain-specialicn repre- sentations for cross-lingull biomedical entity linking. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ith International Joint Conference on Natural Lan- guage Processing (Volume 2: Short Papers), pages 565-574.
Shai Shalev-Shwartz and Shai Ben-David 2014, Un- derstanding machine learning: From theory to algo- rithms. Cambridge university press.
Jingbo Shang. Jialu Liu, Meng Jiang. Xiang Ren. Clare R Voss, and Jiawei Han. 2018. Automated phrase mining from massive text corpora. IEEE Transactions on Knowledge and Data Engineering. 30( 1825-18777.
Yi Luan, Luheng He, Mari Ostendorf, and Hannanch Hajishirzi. 2018 Multi-task identification of entities, relations, and coreference for scientific knowledge

---

Jingbo Shang, Liyuan Liu, Xiaotao Gu, Xiang Ren. Teng Ren, and Jiawei Han. 2020. Learning named entity lagger using domain-specific dictionary. In 2018 Conference on Empirical Methods in Natural Language Processing. EMNLP 2018, pages 2054 2064. Association for Computational Linguisties.
Shaodian Zhang and Noémie Elhadad. 2013. Unsuper- vised biomedical named entity recognition: Experi- ments with clinical and biological texts. Journal of biomedical informatics. 46(6): 1088-1098.
Xinghua Zhang, Bowen Yu, Tingwen Liu, Zhenyu Zhang, Jiawei Sheng, Xue Mengge, and Hongbo Xu. 2021. Improving distantl--superiiae entity recognition with celf-collaborativee denoising learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10746-10757.
Luca Soldaini and Nazli Goharian. 2016. Quickumls: a fast, unsupervised approach for medical concept extraction. In MedIR workshop, sigir, pages 1-4.
named
Bosheng Song, Fen Li, Yuansheng Liu, and Xiangxiang 7eng. 2021. Deep learning methods for biomed- ical named entity recognition: a survey and qual- itative comparison. Briesings in Bioinformatics, b2x6).bbab222.
Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 202] Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 50-61.
Min Song, Hwanjo Yu, and Wook-Shin Han. 2015. De- veloping hybrid dietionary-basee bio-entity recog- nition technique. BMC medical informatics and deci- sion making, 15(1))1-8.
Kang Zhou, Yuepei Li, and Qi Li. 2022. Distantly supervised named entity recognition via confidence- based multi-class positive and unlabeled learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pages 7 7198-7211.
Cong Sun, Zhihao Yang. Lei Wang. Yin Zhang, Hongfei Lin, and Jian Wang. 2021. Biomedical named en- tity recognition using bert in the machine reading comprehension framework. Journal of Biomedical Informaticx, 10:1999.
Shogo Ujiie, Hayate Iso, Shuntaro Yada, Shoko Wakamiya, and Eiji Aramaki. 2021. End-to-end biomedical entity linking with span-based dictionary matching. In Proceedings of the 20th Workshop on Biomedical Language Processing, pages 162-167.
David Wadden, Ulme Wennberg, Luan, and Han- naneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)) pages 5784-5789.
Xuan Wang. Yu Zhang, Qi Li, Xiang Ren, Jingbo Shang, and Jiawei Han. 2019a. Distantly supervised biomed- ical named entity recognition with dictionary expan- sion. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 496- 503. IEEE.
Xuan Wang, Zhang, Xiang Ren, Yuhao Zhang, Marinka Zitnik, Jingbo Shang, Curtis Langlotz, and Jiawei Han. 2019b. Cross-type biomedical named en- tity recognition with deep multi-task learning. Bioin- formatics. 17450):7755-155.
Leon Weber, Jannes Münchmeyer, Tim Rocktäschel, Maryam Habibi, and Ulf Leser. 2019. Huner: Im- proving biomedical ner with pretraining, Bioinfor- matics.
Leon Weber, Mario Sänger, Jannes Münchmeyer. Maryam Habibi, UIf Leser, and Alan Akbik. 2021. Hunflair: easy-to-use tool for state-of-the--rt biomedical named entity recognition. Bioinformatics, 37(17):2722-2794

---

Appendix. Supplementary Material
A Proof of Theorem
We denote the encoded representation of biomedical named entities E 8 as E(s) P where [S] P and denote the encoded named entities representation in UMLS
P where |P] |S].
Theorem 1 (Synonym Generalization Error Bound), If S is an e-net of S. The loss function f E [0, b] is x-Lipschitz continuous. We have with probability at least 8,
In S +
In
Es (ASE (ne+bb)
b//
(7)
2
Proof. Vr E P. P. E P such that ||- S We assume that VA E P the loss f(A) is bounded
f(f) [0, b). Therefore
f(r) f(r) f(F) + ((F) f(F) +
$ kk|r-- - ill + f(f)
S KE +
On the other hand, as P is evenly sampled from P, namely P = (7lr P)- vf. we have Ef(r)
- Bf(()) - E! Ef(A:)
- E( ER
Therefore,
f(r) R = f(r) - EE((r + Ef(r) R = - - EEf(r) + Eй
(8)
For the first term ʃ(r) - Er(r) in Equation (8). by Hoeffding, we have
212
p((r)) - ES(r) 2 20 exp (-The + b)2
Then, take all r E P. we have
p(rr 2f))
((f(rr) - Ef(r) 2()
TEP
((f(r) Bf(r) 2 1))
PEP
2f2
S Σ exp
(Re + b)2
rCP
2t2
|P|
exp
(Re + 6)²
Therefore,
242
- < = 1- |P|
exp
(KE + b)2
Then denote |P| exp
(-
212 (-xxrtbj²) we have with probability at least 5.
f(r) - Ef(rr s (KC + b) In P In
(9)
2
Similarly, for the second term ER R in Equation (8), by Hoeffding, we have

---

2/2
2|
p(ERR
-R R2 t) S exp
IP|62
exp
r2
Then denote cxp -2 we have with probability at least
8,
In
f(r) Ef(r) - V
(10)
2/P)
By combining Equation (9) and Equation (10), we have
In |P|
In
In
p
f(r) R < (NE + b)
+ b
2
2/P1
In IP|
In
In 1
f(r) Es(r) + +B((r) - Ri < (KE + +bll
+ +bV
2/
2p (f(r) - Bf(r) < (RE (Re++b) In IP| 1nd
In i
(Ef(r) R b/
2|
=(1 - 5)²
21 - 28
Therefore, vʃ, Vr E P, we have with probability at least 1 - 26,
In PP|
Im 6
In
f(r) R < (nC + b)
+ b
2
2/P)
Finally, by taking the supremum of the lefthand side terms, have with probability least 1 - 8,
sup(f(r) R) (ne + b) rCP
In PP
In
b
2 In
J
2
P
B Illustration of SDR and NPR
Figure shows the intuition of how SDR and NPR improve the performance. On the left-hand side, it shows the synonyms and the corresponding loss function value without SDR and NPR. On the right-hand side, it shows the synonym points and the regularized function value. The results indicate that when applied the SDR, the distance between the synonyms for the same concept concentrates more closely with each other. On the other hand, with the NPR, the Lipschitz constant for the function decrease, and the landscape for the function becomes quite flat. As result, the function value for the synonyms is more close to each other.
Full Results
Table 5 shows the full results of the experimental results.
D Hyper-Parameter Tuning
The paper-parameter with the corresponding search space are listed Table 6.
E Advantages and Disadvantages for NER methods
In Table 7, we compare the advantages and disadvantages of different NER paradigms.

---

E
3
Figure Intuition of SDR and NPR.
NCBI
BCSCDR-D
BCSCDR-C
RCACHEMD
Species-800
LINNAEUS
AVG
Model
R
Fi
F
P
R
F
F
P
R
Fj
P
R
F
F5
BioBert
88.2
91.2
89.7
86.5
87.8
87.2
93.7
93.3
93.5
92.8
919
92
72.8
75.4
74.1
90.8
85.8
88.2
87.5
87.6
87.5
Diisanty Supervised
SBM
88.4
88.9
88.6
834
864
849
93.2
93.6
93
9.
86.6
89
99.5
91.6
95.
99.8
80.1
88.9
92.8
87.9
90.1
SBMCross
75.9
58.3
66.0
70.1
61.3
65.4
94.1
86.4
90.1
72.2
63.2
67.4
64.2
64.5
64.3
78.8
45.8
57.9
75.9
63.2
68.5
SWELLSHARK
64.7
69.7
67.1
80.7
77.6
79.1
88.3
88.3
88.3
77.9
78.5
78.2
AutoNER
72.0
75.5
31.3
95.6
AutoNER w/o D
66.8
32.4
43.6
72.0
17.3
27.9
89.7
67.3
76.9
90.7
19.7
32.4
57.6
50.7
53.9
88.4
39.0
54.1
77.5
37.7
48.1
AutoNER w/o IDe!
85.1
19.1 31.2
87.1
40.4
55.2
94.2
37.3
53.4
91.2
18.8
31.2
83.6
18.5
30.3
90
62.8
74.1
88.6
32.8
45.9
AutoNTER w/o DT
$7.9 9.7 16.6
63.0
13.9
22.8
92.8
39.3
55.2
60.9
24.6
35.1
59.8
25.0
35.3
80.
330
46.8
69.
24.2
35.3
EmbSim (BioBert)
56.7
24.9 34.6
61.8
71.7
.1.
66.0
47.4
32.4
49.0
80
42.9
55.9
42.1
Dictomey-Booo
EmbSim (PubMedBert)
55.6
24.6 34,2
61.0
13.1
21.6
70,9
61.7
66.0
46.2
24.8
32.3
50.8
29.7
37.4
80.3
42.2
55.4
60.8
32.7
41.2
EmbSim (SAPBert)
31.9
EmbSim (Word2vec)
46.2 27.6 34,5
50.2
15.3
23.5
64.9
64,8
64.9
37.9
30.4
33.7
40.6
27.2
32.6
69
41.7
52.
51.5
34,5
40.2
MetaMap
27.8 38.4
69.3
22.3
65.9
63.5
64.7
33.1
25.2
28.6
56.9
48.7
44.3
58.3
62.1
37.1
44,1
MetaMap (Uncased)
58.4 27.5 37.4
63.5
18.4
28.6
94.8
64.1
76.5
86.2
24.0
37.5
49.1
52.3
50.6
79.1
49.6
61.0
71.9
39.3
48.6
SPED
59.3 30.1 39.9
68.2
14.3
23.7
65.6
63.9
64.8
33.0
25.4
28.7
56.0
49.4
52.5
85.3
44.7
58.7
61.2
38.0
44.7
TF-DF
26.1 29.7 27.7
32.0
22.6
26.4
74.1
65.4
69.5
19.1
39.3
25.7
42.5
21.4
28.4
77.3
40.5
53.1
45.2
36.5
38.5
QuickUMLS (Dice)
61.2 20.9 31.2
76,1
25.2
37.8
82.0
57.8
67.8
60.1
20.6
30.7
$1.9
46.7
49.1
76.6
50.2
60.6
68.0
36.9
46.2
QuicKUMLS (Cosine)
52.6
62.7
58.7
QuickUMM S (Jaccun))
80.4 17.2 28.4
93.5
14.5
25.1
93.2
56.9
70.7
82.7
16.9
28.1
61.7
46.7
53.2
88.2
44.7
59.3
K3.3
32.8
44.1
QuicKUMI (Overlup)
7.5
5.7
SynGen
68.8 64.1 66.2
63.8
63,4
63.5
85.0
83,9
84,4
56.4
51.1
53.6
58.8
65.7
62.0
84.9
66.2
74.4
69.6
65.7
67.4
SynGen (SAPBert)
70.0 63.2
66.3
65.4
59.3
62.
84.4
83.2
83.8
51.0
40.7
42.4
56.7
59.5
58.
83.0
63.0
71.6
68.4
61.5
64.0
SyuGen (BioBert)
62.4
79.1
46.3
Syutien (Bert)
60.1 64.2
62.0
56.5
55.6
56.1
79.4
80.0
79.6
46.4
48.2
47.1
55.1
55.4
55.2
81.3
50.2
62.1
63.1
58.9
60.4
SynGen (SciBert)
66.6
61.6 56.5
55.7
83.5
78.9
81
55.3
49.3
54.5
80.4
52.8
63.7
64.7
Table 5: Main results. We repeat each experiment for runs and report the averaged scores. For BioBert and SSA.LLSAARK, report the score from the original paper. We mark the extra effort involved with superscripts, where is gold annotations; 9 is in-domain annotations: is regex design; A is special case tuning: 9 is in-domain corpus; i is dictionary tailor. The standard deviation analysis is in Figure 7.
Search Space
NCBI BCSCOR-D BCSCDR-C BOACHEMD Spacies-000 LINNABUS
SDR weight
(0.05, O. 0.2. 0.5. 1.0, 2.0]
0.1
0.1
0.1
0.1
0.1
0.2
weight
0.1, 2.0)
NSF minimal distance 1g
7.0
7.0
6.0
Inference probubiiiy threshold
(0.30.4...5.66..7)
0.5
0.6
0.5
Inference maximal spin longth m.,
16.8.10.123
8
8
Table 6: Search space for hyper-parameters.
Paradigm
Advantage
Disadvautage
Demanding largo-scale annotated data.
Supervised
performance.
Distantly
Requiring extra effort, e.g. regex rule design.
human annotation.
Supervised
Only one dictionary is needed.
Dietionary-based
performance.
Table 7: Comparison between different NER paradigms.


Reviews:
[{"review_id": "MdJncHrI0e", "reviewer": "Reviewer_GYVt", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors achieve significant performance improvements against other dictionary-based frameworks across a large number of datasets. Additionally, analysis based on synonym distance, few-shot scenarios based on multiple dictionary sizes, and hyper-parameter optimization (on SDR and NDR) are provided and offer further insight on the effectiveness of SynGen.\n\nThe authors provide a thorough and detailed theoretical analysis of their framework.\n\nThe paper is clear and concise, making it easy to follow.\n", "reasons_to_reject": "Ideally, there would be a case study outlining the effectiveness of SynGen via an example.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors introduce a dictionary-based biomedical NER framework that’s backed by two important components: a synonym distance regularizer and a noise perturbation regularizer. The authors demonstrated the effectiveness of their methodology against many baselines across multiple datasets.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "A9VtD6vQW1", "reviewer": "Reviewer_tQnM", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Well written paper with through and carefully designed experiments.", "reasons_to_reject": "No major reasons to reject.", "questions_to_authors": "", "paper_topic_and_main_contributions": "Paper introduces a novel synonymy generalization framework (SynGen). It recognizes biomedical concepts included in input text using span-based predictions. Paper introduces two regularization methods SDR and NPR to minimize the synonymy generalization error. Extensive evaluations and ablation studies are done to verify the performance of SynGen.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xQz3Z0ldLz", "reviewer": "Reviewer_JQ58", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- the method is well and clearly described.\n- experiments are performed on several corpora and the results are compared to state-of-the-art approaches. The system performs better than similar type of methods.\n- a theoretical analysis of the synonymy generalization.", "reasons_to_reject": "- distant supervised approaches achieves better results. But such approaches requires annotated data.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper focuses on the concept synonym identification in the context of biomedical named entity recognition.\nThe contribution is an approach  which takes advantage of the information issued from dictionary and large  language model to identify concept synonyms.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Xe10hdnFu3", "reviewer": "Reviewer_gSYy", "summary": "", "questions": "A) I'm a bit confused about the Greedy Extraction section: the introduction of the section seems to indicate that the issue is to design a method that is able to recognize nested entities. However, the details given afterwards seem to indicate that all nested entities will actually be removed. Could the authors clarify what the method does?\n\nB) It's not clear what dictionary tailoring in the AutoNER approach actually entails. Is that a step that requires extra manual work? If it is fully automatic, then I don't think it is a weakness of the method. As far as I can tell, the most important difference between AutoNER and the proposed SynGen is that AutoNER requires a task specific corpus, while SynGen does not.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- novel and interesting approach that does not require manual annotation, manually designed rules or a task-specific corpus.\n- fairly good performance, significantly outperforming other dictionary-based approaches.\n- the code is made available", "reasons_to_reject": "- I'm not sure how accurate the few-shot experiment results (Figure 4) are: at 100%, performance seems lower than what is reported in Table 1, especially for the BC5CDR-D corpus (0.4 vs. 0.635) \n- A few details/clarifications are needed: the \"Greedy Extraction\" part is unclear, and so is the AutoNER approach (see questions below)", "questions_to_authors": "A) I'm a bit confused about the Greedy Extraction section: the introduction of the section seems to indicate that the issue is to design a method that is able to recognize nested entities. However, the details given afterwards seem to indicate that all nested entities will actually be removed. Could the authors clarify what the method does?\n\nB) It's not clear what dictionary tailoring in the AutoNER approach actually entails. Is that a step that requires extra manual work? If it is fully automatic, then I don't think it is a weakness of the method. As far as I can tell, the most important difference between AutoNER and the proposed SynGen is that AutoNER requires a task specific corpus, while SynGen does not.", "paper_topic_and_main_contributions": "This paper investigates biomedical entity recognition. The authors propose a novel dictionary-based approach that does not require manual effort. It does not require a manually annotated training corpus, manually designed rules, or even a task-specific corpus (such as the unlabeled version of a training corpus for a given task). They formulate the task as a synonym generalization problem, i.e. recognizing text spans as synonyms of dictionary concepts. They build an MLP classifier to classify text spans as positive or negative instances, which they train using synonyms from a medical dictionary (the UMLS) as positive examples and text spans extracted from a general biomedical corpus (PubMed) as negative examples. Text spans are encoded using a BERT-based model. The authors propose two regularization steps to improve the generalizability of synonym recognition: one that regularizes the distance between synonyms in the embedding space and one that minimizes the loss gap between synonyms. The authors evaluate their method on 6 BioNER tasks and compare it to several existing supervised, distantly supervised and dictionary-based approaches. They show that their method significantly outperforms all existing dictionary-based approaches.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores a method for biomedical NER which introduces a synonym generalization framework to improve dictionary-based approaches for this problem. The reviews have minor concerns about the clarity for some details, but the rebuttal has addressed them. The authors are encouraged to update the paper according to the rebuttal and discussion with the reviewers.
Decision: Accept-Main