BioFEG: Generate Latent Features for Biomedical Entity Linking
Authors: Xuhui Sui
Ying Zhang
Xiangrui Cai
Kehui Song
Baohang Zhou
Xiaojie Yuan
Wensheng Zhang
Venue: EMNLP 2023 Main (2023)
Keywords: Biomedical entity linking
Unseen entities

Abstract: Biomedical entity linking is an essential task in biomedical text processing, which aims to map entity mentions in biomedical text, such as clinical notes, to standard terms in a given knowledge base.  However, this task is challenging due to the rarity of many biomedical entities in real-world scenarios, which often leads to a lack of annotated data for them. Limited by understanding these unseen entities, traditional biomedical entity linking models suffer from multiple types of linking errors. In this paper, we propose a novel latent feature generation framework BioFEG to address these challenges. Specifically, our BioFEG leverages domain knowledge to train a generative adversarial network, which generates latent semantic features of corresponding mentions for unseen entities. Utilizing these features, we fine-tune our entity encoder to capture fine-grained coherence information of unseen entities and better understand them. This allows models to make linking decisions more accurately, particularly for ambiguous mentions involving rare entities. Extensive experiments on the two benchmark datasets demonstrate the superiority of our proposed framework.

BioFEG: Generate Latent Features for Biomedical Entity Linking
Xuhui Sui', Ying Zhang'; Xiangrui Cai', Kehui Song', Baohang Zhou', Xiaojie Yuan', Wensheng Zhang"
College of Computer Science, VCIP, TMCC, TBI Center, Nankai University, China Institute of Automation, Chinese Academy of Sciences, China x zhwubaohangg@dbiie..... hoob nankai edu cn {yingzhang, caixr, y juuanxjeennnlle edu. cn, zhangwensheng@hho tmain com
Abstract
allowing arsenic [detrxification] over broad concentration range.
Biomedical entity linking is essential task biomedical text processing, which aims to map entity mentions in biomedical text to stan- dard terms in given knowledge base. How- ever, this task is challenging due to the rar- ity of many biomedical entities in real-worke scenarios, which leads to a lack of annotated data for them. Limited by understanding these unseen entities, traditional biomedical entity linking models suffer from multiple types of linking errors. In this paper, we propose novel latent feature generation framework BioFEG to address these challenges. Specifically, our BioFT: leverages domain knowledge train a generative adversarial network, which gener- ates latent semantic features of corresponding mentions for unseen entities. Utilizing these features, we fine-tune our entity encoder to cap- ture ane-grrieedd coherence information of un- seen entities and better understand them. This allows models to make linking decisions more accurately, particularly for ambiguous tions involving rare entities. Extensive experi- ments on the two benchmark datasets demon- strate the superiority of our proposed method.
cellular detoxification
Liver cetionnication
Severe hip osteoarthritis [Saittall] spinopelvic malalignment
Sagitial plane
Sagilial sinus
Sagittal stratum
which rupidly progressed to the tright lawe).
x
Lateral part of right knee
Structure of right knee
(a) Examples of three types of linking errors for biomedical entity linking. The entities in blue are unseen entities and in yellow are seen entities.
Thmiiing
Mention
Mention 2
Retrieser
Ecoodddgge base
Meniion 3
Minaiion
Pine ture
Eaiity 3
Iuaity 5
Exiiity
Genented Frsturs
Test
2
Mention 5
Mestion 6
Ganevaooe
(b) Overview our proposed BioFRG.
Figure 1: The motivation illustration of our proposed BioFEG. Linking errors may occur if unseen entities are not sufficiently understood. Our BioFEG generates latent semantic features corresponding mentions for unseen entities to fine-tune entity encoders capture fine-gre ine-grained coherence signals of unseen entities.
1 Introduction
Biomedical entity linking assigns biomedical en- tity mentions in texts to corresponding canonical concepts in a knowledge base, is key task in the biomedical NLP area. This task plays vital role in bridging unstructured text and structured biomedi- cal knowledge bases, making an essential compo- nent several downstream medical-related applica- tions, including knowledge discovery and medical diagnosis (Joseph et al., 2016), high-throughhut phenotyping (Yu et al., 2015), literature searching (Zheng et al., 2015) and biomedical question an- swering (Lamurias and Couto, 2019).
2019; Sung et al., 2020; Liu al., 2021; Yuan al., 2022) have achieved high performance the past few years, they often overlook entities that never appear at training, commonly known as unseen enti- ties. In the biomedical field, there are many unseen entities lacking training data due to the presence of rare diseases that occur infrequently in literature. This restricts our knowledge and comprehension of unseen entities and hinders the overall performance of biomedical entity linking.
Although many previous biomedical entity link- ing methods (Leaman and Lu, 2016; Phan al.,
Due to the limited understanding of unseen en- tities, there are three types of linking errors for
Corresponding author.

---

biomedical entity linking, as illustrated in Figure la. The first type involves linking mentions of un- seen entities to wrong seen entities, as the top case where the mention "detoxification" may be linked to the wrong entity "Liver detoxification". The second type is linking mentions of seen entities to wrong unseen entities, like in the middle case where the mention "Sagittal" is confused with un- seen entities "Sagittal sinus" and "Sagittal stratum". The most challenging type is the third type, which oceurs when linking mentions of unseen entities to wrong unseen entities. As in the bottom case, the mention "right knee" can not be correctly linked without sufficiently understanding both "Lateral part of right knee" and "Structure of right knee".
pseudo data in the latent feature space for un- seen entities to simultancously deal with all three types of linking errors caused by insuffi- ciently understanding of unseen entities.
We propose a novel BioFEG framework, which leverages domain knowledge to gen- erate semantically meaningful latent features for unseen entities without any labeled data and fine-tunes the entity encoder with these features to achieve higher accuracy.
We compare our BioFEG with state-of- the-rrr biomedical entity linking approaches on two benchmark datasets: MedMentions (Mohan and Li, 2019) and BC5CDR (Li et al., 2016). Experimental results demonstrate the superi- ority of our proposed framework.
Some previous works (Angell et al., 2021; Agar- wal al., 2022) have realized the challenge of unseen entities. They cluster mentions by utilizing mention-mente coreference relationships to bet- ter disambiguate mentions. However, considering the rarity of unseen entities, most clusters of them only contain few mentions. This makes it difficult to utilize mention-mentinn relationships of unseen entities. Thus, these methods good at managing the second type of linking error linking mentions of seen entities to wrong unseen entities but still seem troubling to deal with the first and third type of linking errors.
Related Work
2.1 Biomedical Entity Linking
Biomedical entity linking is also named as biomedi- cal entity normalization or biomedical entity disam- biguation. Diverse approaches have been proposed to explore this task the last few years. While tra- ditional biomedical entity linking studies (Hanisch et al., 2005; Kang et al., 2013; Cho et al., 2017) incorporate heuristic rules to normalize entities, recent ofate-of-the-art approaches of biomedical en- tity linking (Phan et al., 2019; Sung et al., 2020; Liu et al., 2021; Lai et al., 2021; Yuan et al., 2022: Zhang al., 2022) encode mentions and entities into a common space and link mentions to the near- est entity. However, all these works ignore entities that never appear at training.
To address these problems, we propose a novel latent FEature Generation framework for Biomedical entity linking in this paper, BioFEG in short. Our BioFEG is iterative framework and each iteration contains three steps, which is shown in Figure lb. In the first step, we train biomedical entity linking retriever using the training set consist- ing of the training pairs of seen entities. To better address the second type of linking error, we utilize hard negative sampling there, which is simple but effective. In the second step, we leverage domain knowledge of seen entities to train generative ad- versarial network (GAN) (Goodfellow et al., 2014) learn generate corresponding mention latent semantic features. In the third step, we fine-tune the entity encoder our retriever by utilizing the generated latent features of unseen entities. This captures line-grinedd coherence signals of unseen entities to simultancously handle all three types of linking errors.
Some previous works (Angell et al., 2021; Varma et al., 2021; Agarwal et al., 2022) have focused on generalizing to unseen entities to improve the per- formance of biomedical entity linking. Angell et al. (2021) and Agarwal et al. (2022) utilize mention- mention coreference relationships cluster men- tions to provide another way to jointly make linking predictions. However, considering the rarity of un- seen entities in the biomedical domain, most of them only have one or two mentions to group to- gether. Thus, these methods fail to deal with the first and third types of linking errors by utilizing mention-mention relationships. Varma al. (2021) introduces additional structural knowledge from WikiData into unseen entities to understand them better. However, the accuracy of mapping entities in biomedical knowledge bases to WikiData is only
The main contributions of this paper are summa- rized as follows:
For the first time, we focus on generating

---

Retneve
Rerank
Real/ Fake
Discriminatar
Score
Score
Score
FFNN
Real mention features
Genernlor
Generaled latent
features
BERT
BERT
Noise
BERT
B
[CLS) en (SEP)
(CLS) me (SEP)
Domain knowledge
[CLS] SJ [SEP)
[CL.S] me [SEP] on [SEP]
Slep Retriever Treining
Step 2: GAN Trsining
Slep 3: Retriever Fine-luning
leeration
The overall architecture BioFEG framework, which contains two phases: retrieve and rerank. our retrieve phase, it performs three steps iteratively.
Figure 2:
80.2%. Wrong structural knowledge will have a greater negative impact.
knowledge base of entities, where each entity is identified by a Concept Unique ID (CUI). Each mention with context m, is associated with its own CUI u. We denote the union of all entities in the knowledge base as F = ojj=l where ej is single entity and there are E entities in the knowledge base. Our objective is to predict the gold CUT u", i.e., the target entity CUI corresponding to the men- tion my as follows:
Therefore, in this paper, we try to find another way, that is generating corresponding mention la- tent semantic features of unseen entities to fine- tune the entity encoder of our biomedical entity linking retriever to deal with all three types of link- ing errors simultancously without introducing any cross-domain error information.
[i]
2.2 Zero-Shot Entity Linking
u CUC CUI P(eylm;;)) (1)
Zero-shot entity linking (Logeswaran et al., 2019) is the task that mentions need to be linked to unseen entities of new domains. Considering that also focus on unseen entities in this paper, we briefly discuss this task there. State-of-the-art zero-sho shot entity linking approaches (Wu et al., 2020; Yao et al., 2020; Tang et al., 2021; Sui et al., 2022) employ a retrieve-rerank pipeline to improve the performance. Some biomedical entity linking meth- ods (Angell et al., 2021; Varma et al., 2021; Agar- wal et al., 2022) and our proposed BioFEG also follow this pipeline. However, zero-shot entity linking is a more challenging task, since all test samples are from unseen entities of new domains. our biomedical entity linking setting, consider- ing that the test set and training set are both from the biomedical domain and only a part of test sam- ples are from unseen entities, there is rich domain knowledge that can be leveraged.
where CUT(.) returns the CUI of the entity ej and is the trainable parameters of models.
3.2 Overall Framework
Figure 2 shows the overall architecture of BioFEG. Following previous works (Angell al., 2021; Varma al., 2021: Agarwal et al., 2022), our framework contains two phases: retrieve and rerank. Retrieve phase is to generate candidate sets for cach mention from all entities of knowledge base and rerank phase is to predict the gold CUI. During the retrieve phase, it performs three steps iteratively. In the first step, we train our biomedi- cal entity linking retriever by utilizing the training set. In the second step, we train a GAN model learn to generate latent semantic features for corre- sponding mentions. the third step, we utilize the generated latent features of unseen entities to fine- tune the entity encoder of our retriever to capture fine-grained coherence signals of unseen entities.
3 Methodology
3.1 Task Definition
3.3 Step 1: Retriever Training
Our biomedical entity linking task takes as input a biomedical text containing multiple mentions whose positions are known. Additionally, we have
Following Varma (2021), we utilize the bi- encoder architecture proposed by Wu et al. (2020) as our biomedical entity linking retriever. This

---

architecture contains two encoders: entity encoder and mention encoder learn the representations of candidate entities knowledge bases and mentions with context respectively.
3.4 Step 2: GAN Training
For unseen entities, the entity encoder of our re- triever is never trained with corresponding mention- entity pairs due to the lack of such data. We pro- pose to utilize GAN (Goodfellow et al., 2014) to generate pscudo data in the latent feature space for unseen entities to fine-tune the entity encoder enhance the understanding of unseen entities.
The inputs of each entity and mention are respec- tively constructed as:
[CLS] en [SEP], [CLS] me [SEP]
(2)
where en = title [ENT] des, and me ctrtl [Ms] mention [Mc] ctztr. The title, des, mention, clall, clztr are the word-piece tokens of the entity title, entity description, mention, context before and after the mention respectively. [ENT] is a special token to separate the entity title and its description, [Ms] and [Me] are special tokens to tag the mention.
Specifically, we utilize WGAN-GP (Gulrajani et al., 2017) to generate corresponding mention la- tent features based on the domain knowledge of each entity. The more domain knowledge we lever- age. the more accurate the generated features will be. However, for fair comparison, we only lever- age the entity description our domain knowl- edge. We utilize another BERT encoder (BBET))) to obtain the domain knowledge representation yd- which also the output of the last hidden layer corresponding to the position of [CLS] token.
Both our entity encoder and mention encoder are individual BERT encoders (θBERT, (BERT; ) We feed the entity and mention inputs into them to obtain vector representations Ye and ym, which are the last layers of the output of the [CLS] to- ken. The score s s(m;; eÿ) of each mention m, and entity ej is calculated the inner product between corresponding representations:
The generator combines the domain knowledge representation ys and a random Gaussian noise vec- tor 2 as the input while generating fake mention latent features Dm. We take mention representa- tions in the biomedical entity linking retriever ym as our real mention latent features. The discrimina- tor combines the domain knowledge representation ya and a latent feature vector 7 m (can be fake or real latent feature) as the input while return- ing real-valued score D m, mm.3)) to decide how realistic the ym is. The discriminator is trained to distinguish the real feature and the generated feature, while the generator is trained to fool the discriminator. Following Gulrajani et al. (2017), the loss of our GAN model is computed follows:
s(m,, e2) ym, Ve,
For the negative sampling, in the first iteration, our retriever is trained on in-batch negatives, which is commonly used random negative sampling method. Within a batch, all other entities except the corresponding entity are treated as negative sam- ples of the mention. In the following iterations, we utilize hard negative sampling to train our retriever, which chooses negative samples according to the score s (m,, ej). The entities with higher scores ex- cept the corresponding entity are negative samples of the mention. This is simple but effective way handle the linking errors of linking mentions of seen entities wrong unseen entities.
LG E.p.Ptt m.)) ((>)) Eg-ptmm [D(+]]
A A.gwpiim [[IV (())ll 1 (5)
where z ~ pldmmal is the joint distribution of latent features Vm and domain knowledge repre- sentations . ~ p(ūm.Na) is the joint distribution of fake latent features Dm and yd. a ~ p(im..a) is the joint distribution of latent features im and ya while ÿ.. = m + (1 c)um with (0, 1), and À is a gradient penalty coefficient. During the training stage, we only use the training set con- sisting of the training pairs of seen entities and minimax the loss minmaxLa.
During the training stage, the objective of our biomedical entity linking retriever is to maximize the score of the corresponding entity of the mention with respect to negative sampling entities. Thus, for each mention-entity pair (m;, c;), the loss function is calculated as follows:
CRer (m,, ex)+ log Σ exp(s(mm, e;)) (4)
G
where e; is the corresponding entity of the mention m4, N is the number of all sampling entities of the mention, which contains the gold entity and negative sampling entities.
3.5
Step 3: Retriever Fine-Tuning
After training the GAN model, we use its generator to generate corresponding mention latent seman-

---

tic features for unseen entities. Then we fine-tune the entity encoder BØBERT) ) of our biomedical en- tity linking retriever with these generated latent features. The process of obtaining entity repre- sentations Ye is the same as step 1. We utilize Eq. 3 to calculate the matching score s(m,, e2) be- tween each generated mention latent feature Mm, and obtained entity representation Vey. And also choose negative samples in the hard way. For each pair (Um:: ej) of generated mention latent fea- tures and their corresponding unseen entities, we fine-tune the entity encoder by optimizing the loss function of Eq. 4.
Num. of
Split
MedMentions
BC5CDR
Train
120K
18K
Mentions
Dev
40K
934
Test
40K
10K
Train
19K
2K
Unique Entities
Dev
281
Test
Dev 42.3%
19.9%
Unseen Entities
Test 42.5%
35.2%
Table 1: Overall statisties of the MedMentions and BCSCDR dataset.
Experiments
4.1 Datasets
3.6 Reranking Phase
Following previous works (Angell et al., 2021; Varma et al., 2021), we evaluate our proposed framework BioFEG under two public biomedical entity linking datasets: MedMentions (Mohan and Li, 2019) and BC5CDR (Li et al., 2016). Table shows the overall statistics of these two datasets.
Through the retrieve phase, we generate a candi- date entity set for each mention. The goal of our reranking phase is to predict the corresponding en- tity from the candidate set and obtain the predicted CUI for each mention. Following previous works (Angell et al., 2021; Varma et al., 2021; Agarwal al., 2022), utilize the cross-encoder architec- ture rerank entities. We utilize Eq. to obtain the inputs of each entity and mention with context. The input of our reranking phase is the concatena- tion of these two inputs after removing the [CLS] token of the entity input.
MedMentions is the largest biomedical entity linking dataset, which contains 4392 abstracts from PubMed. The dataset is labeled to link to the 2017AA full-version of UMLS. Following previ- ous works (Angell et al., 2021; Varma et al., 2021; Agarwal al., 2022), we use the ST21PV subset of MedMentions dataset. There are large number of entities that are unseen entities (never seen during the training stage) in the development and test set, over 42% of entities.
We feed the input into another BERT encoder (OBERT,) to obtain the mention-candidate repre- sentation em,e. which is also the vector in the last hidden layer corresponding the position of the [CLS] token. For mention m, and one of its can- didate entity eg. we put the representation Imm.e to a feed-forward neural network FFNN and obtain the matching score 8% between my and eş by using a SoftMax function:
BC5CDR contains 1500 PubMed abstracts di- vided into training (500), development (500) and test (500) subsets. The mentions are linked MESH knowledge base, which is much smaller than UMLS. In this dataset, 19.9% and 35.2% unique entities in development and test set never appear in the training set.
exp($k)
9p FFNN((Mmae)) 8k-
2j=1 exp($;)
4.2 Implementation Details
where K is the number of candidate entities of the mention m,. We utilize the cross-entropy our loss function, which is computed as follows:
In experiments, following (Angell et al., 2021; Varma al., 2021: Agarwal al., 2022), all the BERT encoders used are the BERT-base version. The maximum sequence length of words for men- tion with context, entity, and domain knowledge are all set to 128. Any string over the maximum length is truncated. We optimize all loss functions using AdamW (Loshchilov and Hutter, 2017). The evaluation metric is the accuracy. All scores are averaged 5 runs using different seeds.
CRerank -4 log s (1 log(1 - 9k) (7)
where 4 E (0, 1), 4 equals to while the candi- date entity 6 the corresponding gold entity of
the mention m, otherwise it equals to the value 0. Finally, we can obtain the predicted gold CUI by calculating as follows:
For the retrieval phase, the total iteration number is 3. The learning rate is Ic-5 for training, and it is set to le-7 and le-9 for BC5CDR and MedMet-
CUI CArg(Ar sR)
(8)
where CUI() returns the CUI of the entity ek-

---

MedMentions
BCSCDR
Models
Overall
Seen
Unseen
Overall
Seen
Unseen
N-GRAM TF- IDF
50.9
50.9
51.0
86.9
89.2
74.6
BIOSYN (Sung et al., 2020)
72.5
76.5
58.7
87.8
89.0
81.1
SAPBERT (Liu et al., 2021)
69.8
72.9
58.9
85.2
85.8
82.0
INDEPENDENT (Logeswaran et al., 2019)
72.8
75.9
61.9
90.5
94.0
73.6
CLUSTERING-BAAE BBSSED (Angell et al., 2021)
74.1
77.3
62.9
91.3
94.9
73.8
DATA-INTEGRAATIN (Varma et al., 2021)
74.8
79.7
57.8
91.9
94.5
77.5
ARBORESCENCE (Agarwal et al., 2022)
75.73
79.97
60.99
BioFEG (Ours)
76.68
79.91
65.42
93.39
95.57
81.19
Table 2: Experimental results the MedMentions dataset and BC5CDR dataset. Seen and Unseen represent the
sets of mentions whose corresponding entities are seen and unseen during training, respectively. We report the results of baselines according to (Angell al., 2021; Varma al., 2021; Agarwal et al., 2022). All scores of our BioFEG are averaged 5 runs using different random seeds. In the results, the highest values are in bold.
nions respectively during the fine-tume toning stage. The number of negative samples is 64. For the GAN, we use single-layer fully-connected network with hidden size of 768 for both the generator and dis- criminator. The gradient penalty coefficient À in Eq. 5 is set to 10. We train the diseriminator iterations in each generator training iteration. We train the GAN model 80 epochs with learning rate 5e-5. For the reranking phase, we choose the top 64 candidate entities as the candidate set for each mention. We train the reranker 5 epochs with learning rate of 2e-5. Our experimental code is available here
imum spanning arborescences over mentions and entities across documents to further improve the CLUSTERING-BASED approach.
4.4 Overall Performance
Table shows the experimental results our BioFEG and other baselines on the MedMentions and BCSCDR datasets. We can observe that our BioFEG outperforms all other methods on both the two datasets and achieves new state-of-the-an per- formance, which demonstrates the effectiveness of our BioFEG. The overall accuracy improvement on Medmentions and BC5CDR are 0.95% and 1.49%, respectively. We can also find that methods consid- ering the unseen entity problem (INDEPENDENT, CLUSTERING-BASEED, DATA-INTEGRATION, ARBORESCENCE, and our proposed BioFEG) perform better than other methods. This is consis- tent with our belief that the lack of understanding of unseen entities hinders the overall performance of biomedical entity linking.
4.3 Baselines
For the quantitative evaluation of our proposed BioFEG, we utilize the following state-of-the-art biomedical entity linking methods for compari- son. The n-gram if-idf model is traditional tf-idf based information retrieval approach. BIOSYN (Sung et al., 2020) utilizes the synonym marginal- ization technique to learn biomedical entity rep- resentations. SAPBERT (Liu et al., 2021) is a self-alignment pre-training approach to repre- sent biomedical entities. INDEPENDENT (Lo- geswaran et al.. 2019) uses the cross-encoder archi- tecture make linking decisions. CLUSTERING- BASED (Angell et al., 2021) is the first work con- sider the problem of unseen entities biomedical entity linking. utilizes mention mention-mention corefer- ence relationships to provide another way to jointly make linking predictions. DATA--INEGGAAION (Varma et al., 2021) introduces additional struc- tural knowledge from WikiData into biomedical entities enhance entity representations. AR- BORESCENCE (Agarwal et al., 2022) builds min-
In terms of the results of MedMentions, our BioFEG achieves similar performance with AR- BORESCENCE on the seen set, but notably outper- forms it on the unseen set. This demonstrates the superiority of our BioFEG, which generates latent semantic features of corresponding mentions for unseen entities to fine-tune the entity encoder to capture fine-grained coherence signals of unseen entities. Regarding the results of BC5CDR, our BioFEG achieves better results than all other base- lines except for SAPBERT on both the seen and unseen sets. SAPBERT introduces training pairs from the UMLS knowledge base and uses metric learning to train language model. This may result in some unseen entities having been trained in these training pairs. Though SAPBERT performs well
hnttps://githhr cwm/ www.iuuuuiuuuuBBee

---

MedMentions
BC5CDR
Models
Overall
Scen
Unscen
Overall
Seen
Unseen
BioFEG
76.68
79,91
65.42
93.39
95.57
81.19
w/o generated latent features
74.87
79.18
59.85
92.93
95.39
79.23
w/o hard negative sampling
74.89
78.83
61.16
93.02
95.29
80.33
w/o reranking phase
74.08
78.73
57.89
92.45
95.53
75.31
Table 3: Ablation study of our proposed BioFEG. We mainly evaluate the effect of three components: the fine-tuning with generated latent features, the hard negative sampling and the reranking phase.
Models
Type
Type
Type 3
Total
90-
Retrieve-Rerank
1691
1203
1084
3978
ARBORESCENCE
1717
1070
3703
80
BioFEG (Ours)
1332
1029
3450
Recall
70
TF-DDE
Table 4: The number of three types of linking errors on the MedMentions test set. Type 1: linking mentions of unseen entities to wrong seen entities, Type 2: linking mentions of seen entities wrong unseen entities, and Type 3: linking mentions of unseen entities to wrong unseen entities.
K-NN NEGATIVES
60-
IN-RATCH NEGATIVES
ARBORESCENCE
BioFEG
Number of Candidates
Figure 3: Retrieval results on Medmentions. The evalu- ation metric is recall, which measures whether the gold entity is in the top K candidates for each mention. The results of baselines come from (Agarwal et al., 2022), and the recall is micro average recall over all mentions.
cantly outperforms BioFEG w/o reranking phase, emphasizing the necessity of the reranking phase after retrieving entities. The deep tross-attention between mentions and candidate entities produces more consistent gains.
on the unseen set, appears challenging learn good representations of seen entities. Overall, these results confirm the effectiveness of our proposed BioFEG in performing biomedical entity linking in the presence of unseen entities.
5.2 Retrieval Performance
We report the retrieval results on McdMentions Figure 3. We can find that our proposed BioFEG consistently outperforms all baseline approaches for all K values, which demonstrates the effective- ness of our proposed BioFEG in the biomedical entity linking retrieval phase. For recall @ 1, di- rectly linking at the retrieval phase, our BioFEG improves over ARBORESCENCE 1.77%. As the number of candidates increases, there is a tendency for all models to saturate. Despite this, we still find that our proposed BioFEG outperforms the ARBORESCENCE for 1 1.45% at recall@64 64.
5 Analysis
5.1 Ablation Studies
To gain a better understanding of our proposed BioFEG, we conduct a series of ablation studies, as presented in Table 3. We can find that our BioFEG w/o generated latent features and BioFEG w/o hard negative sampling both result in decrease in over- all accuracy, confirming the effectiveness of these two key components. The performance of BioFEG w/o generated latent features drops more on the unseen set, while the performance of BioFEG w/o hard negative sampling drops more on the seen set. Fine-tuning with our generated latent features helps our entity encoder understand unseen entities better. Our hard negative sampling finds the most informative training samples for efficient training. It avoids easy training samples dominating the gra- dients to harm the learning process, which provides more fine- -rae-gr coherence information for seen entities. We also observe that our BioFEG signifi-
5.3 Handling Three Types of Linking Errors
As noted in our stated contributions, our BioFEG possesses the ability to tackle three types of link- ing errors caused by insufficiently understanding of unseen entities. Thus, we conducted an analysis to compare the number of the three types of link- ing errors among BioFEG and two baselines. The ARBORESCENCE represents the current state-of- the-art, while the Setrieve-Rerant serves as the base model for both ARBORESCENCE and our BioFEG, utilizing a bi-encoder in the retrieval

---

Error Type
Mention
Predicted
Annotated
Statistics
HO (High Overlap)
creatinine hicarbonate
creatinine
chromium
0.6%
AS (Ambiguous Substring)
sodium hicarbonate
bicarhonates
7.7%
RS (Redundant String)
inorganic arsenic
arsenic
arsenicals
10.5%
(Abbreviation)
amp anotia
adenosine monophosphate
ampicillin
7.5%
LO (Low Overlap)
anodontia
congenital microtia
59.5%
OT (Others)
brain damage
brain diseases
brain injuries
14.2%
Table 5: Examples and statistics of each error type on the BC5CDR test set.
phase and a cross-encoder in the reranking phase. The results are shown in Table 4.
on the context. AS is the error case whose mention string is a substring of the predicted entity, while RS is the case whose predicted entity is substring of the mention string. AB is string is a form of abbreviation, which may need additional tools to replace them with the expanded form. LO denotes an error case where the mention string is very different from the surface of its gold entity. OT represents an error that does not fall into any of the above types.
For the type 2 linking errors, ARBORESCENCE performs better than Retrieve-Rerane and our pro- posed BioFEG, which demonstrates the superior- ity of utilizing mnntionnmennionn coreference rela- tionships to cluster mentions to provide another way to jointly make linking decisions. However, it performs worse and slightly better than Retrieve- Rerank on type 1 and type linking errors, respec- tively. This is consistent with our claim that the methods based on mention--ention coreference re- lationships are good at handling the type linking error but still seem troubling to deal with type and type 3 linking errors.
that the mention
In Table 5, we can observe that most errors be- long LO. This is the most challenging error type, since it requires more external knowledge and more complex reasoning to overcome the huge differ- ence between the surface of mention string and gold entity. The available information may not be sufficient to make right linking decisions. There are also many errors are AS and RS, the mentions of them are often highly ambiguous and could refer to many different entities. We can also find 14.2% errors belong OT, most of which are caused by the gold entity being too semantically similar other entities, such as "brain injuries" and "bruin diseases". In general, considering the limitations of annotation and available knowledge, our BioFEG has almost reached the upper bound performance of the biomedical entity linking task.
We observe that our BioFEG significantly out- performs Retrieve-Rerank on handling all the three types of linking errors, especially type linking error. Our BioFEG utilizes the generated latent semantic features of unseen entities to fine-tune the entity encoder to capture fine-grained coherence signal of unseen entities. This is significant for biomedical entity linking models to understand un- seen entities better and simultancously handle the three types of linking errors.
5.4 Error Analysis
To better understand the behavior of our proposed BioFEG, we perform error analysis the test set, with focus on the BC5CDR dataset due to the high number of error samples in MedMentions. We manually examined all error cases and categorize them into six error types: HO (High Overlap), AS (Ambiguous Substring), RS (Redundant String), AB (Abbreviation), LO (Low Overlap), and OT (Others). The examples and statistics of each error type are shown in Table 5.
Conclusion
In this paper, we focus on rare entities in biomed- ical entity linking. To address the challenge of lacking training data and insufficiently understand- ing of unseen entities, we propose BioFEG, a novel framework to generate pseudo data in the latent fea- ture space for unseen entities to fine-tune the entity encoder of our retriever. This captures more fine- grained coherence signals of unseen entities, allow- ing for simultaneously handling all the three types of linking errors caused by insufficiently under- standing of unseen entities. Experimental results on two public biomedical entity linking datasets demonstrate the effectiveness of utilizing generated
HO is error where the mention string is iden- tical to the predicted entity but the predicted entity is not the annotated entity. This error type may be caused by annotation errors or occur when the same mention is interpreted in different ways depending

---

latent semantic features and our proposed BioFFE achieves state-of-the-art performance.
Courville, and Yoshua Bengio. 2014. Generative ad- versarial nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neu- ral Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 2672- 2680.
Limitations
Although our BioFEG has demonstrated its effec- tiveness in the biomedical entity linking task, there are still some limitations to be addressed in the fu- ture. The primary limitation is to generate seman- tically more meaningful corresponding mention latent features for unseen entities. We argue that the more domain knowledge we leverage, the more accurate the generated features will be. However, for fair comparison with baselines, we only lever- age the entity description as our domain knowledge, We will investigate to utilize more domain knowl- edge to generate more accurate latent features in the future work.
Ishaan Guirajani, Faruk Ahmed, Martín Arjovsky, Vin- cent Dumoulin, and Aaron C. Courville. 2017. Im- proved training of wasserstein gans. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Sys- tems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5767-5777.
Daniel Hanisch, Katrin Fundel, Geinz-Theodor Mevis- sen, Ralf Zimmer, and Juliane Fluck. 2005. Prominer: rule-based protein and gene entity recognition. BMC Bioinform., 6(S-1).
Shaini Joseph, Ram Shankar Barai, Rasika Bhujbal- rao, and Susan Idicula-Thomen 2016. PCOSKB: A knowledgebase on genes, diseases, ontology terms and biochemical pathways associated with polycystic ovary syndrome. Nucleic Acids Res., 44(Database- Issue):1032 1035.
Acknowledgments
This research is supported by the Chinese Sci- entific and Technical Innovation Project 2030 (2018AAA0102100), the National Natural Science Foundation of China (No. 62272250, 62002178,
Ning Kang, Bharat Singh, Zubair Afzal, Erik M. van Mulligen, and Jan A. Kors. 2013. Using rule-based natural language processing to improve disease nor- malization in biomedical text. J. Am. Medical Infor- matics Assoc., 20(5):876-88)
U1936206), the Natural Science Foundation of Tianjin, China (No. 22JCJQJC00150, 22JC- QNJC015800, the Fundamental Research Funds for the Central Universities (No. 63231149).
Tuan Manh Lai, Heng Ji, and ChengXiang Zhai 2021. BERT might be overkill: A tiny but effective biomed- ical entity linker based on residual convolutional neu- ral networks. In Findings of the Association for Com- putational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 1631-1639, Association for Computa- tional Linguistics.
References
Dhruv Agarwal, Rico Angell, Nicholas Monath, and Andrew McCallum. 2022. Entity linking via explicit mention-mention coreference modeling. In Proceed- ings of the 2022 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 4644 4658. Association for Computational Linguistics.
Andre Lamurias and Francisco M. Couto. 2019. Lasige- biotm at MEDIQA 2019: Biomedical question an- swering using bidirectional transformers and named entity recognition. In Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACE 2019, Florence, Italy, August 1, 2019, pages 523-527, Association for Computational Linguistics.
Rico Angell, Nicholas Monath. Sunil Moban, Nishant Yadav, and Andrew McCallum. 2021. Clustering- based inference for biomedical entity linking. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, NAACL-HLI 2021, Online, June 6- 11, 2021, pages 2598 2608. Association for Computational Linguis- tics.
Robert Leaman and Zhiyong Lu. 2016. Taggerone: joint named entity recognition and normalization with meri-markov models. Bioinform., 32(18):2839- 2846.
Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sci- aky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016. Biocreative V CDR task corpus: a resource for chemical disease relation extraction. Database J. Biol. Databases Curation, 2016.
Hyejin Cho, Wonjun Choi, and Hyunju Lec. 2017. A method for named entity normalization in biomedical articles: application to diseases and plants. BMC Bioinform., 18(1 (:1:::5::1-451:11
Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng. Marco Basaldella, and Nigel Collier. 2021. Self-alignment pretraining for biomedical entity representations. In
Ian Goodfellow. Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C.

---

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 4228 4228--2388 Association for Computational Linguis- tics.
tion in biomedical text. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16- 20 November, 2021, pages 4566-4575 Association for Computational Linguistics.
Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero- shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6397- 6407. Association for Computational Linguistics.
Lajanugen Logeswaran, Ming- Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, and Honglak Lee. 2019. Zero-shot entity linking by reading entity de- scriptions. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Vol- ume 1: Long Papers. pages 3449 3460. Association for Computational I Linguistics.
Zonghai Yao, I Liangliang Cao, and Huapu Pan. 2020. Zero-shot entity linking with efficient long range se- quence modeling. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL, pages 2517-2522 Association for Computational Linguistics.
Ilya Loshchilov and Frank Hutter. 2017. Fixing weight decay regularization in adam. CoRR,
abs/1771.05101.
Sunil Mohan and Donghui Li. 2019. Medmentions: A large biomedical corpus annotated with UMLS con- cepts. In Isi Conference on Automated Knowledge Base Construction, AKBC 2019, Amherst, MA, USA, May 20-22, 2019.
Sheng Yu, Katherine P. Liao, Stanley Y. Shaw, Vi- vian S. Gainer, Susanne E. Churchill, Peter Szolovits, Shawn N. Murphy, Isaac S. Kohane, and Tianxi Cai. 2015. Toward high-throughy ased automated feature extraction and selection from knowledge sources. J. Am. Medical Informatics As- soc., 2265):993-1000.
phenotyping: unbi-
Minh C. Phan, Aixin Sun, and Yi Tay. 2019. Robust representation learning of biomedical names. Pro- ceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Pa- pers, pages 3275-3285. Association for Computa- tional Linguisties.
Hongyi Yuan, Zheng Yuan, and Sheng Yu. 2022. Gen- erative biomedical entity linking via knowledge base- guided pre-training and synonyms-aware fine-tuning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 4038 4048. Association for Computa- tional Linguistics.
Xuhui Sui, Ying Zhang, Kehui Song, Baohang Zhou, Guoqing Zhao, Xin Wei, and Xiaojic Yuan. 2022. Im- proving zero-shot entity linking candidate generation with ultra-fine entity type information. In Proceed- ings of the 29th International Conference on Compu- tational Linguistics, COLING 2022, Gyeongju. Re- public of Korea, October 12-17, 2022, pages 2429- 2437. International Committee on Computational Linguistics.
Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Nau- mann, Jianfeng Gao, and Hoifung Poon. 2022. Knowledge-rich self supervuueviiioon for biomedical en- tity linking. In Findings of the Association for Com- putational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 868-880, Association for Computational Linguistics.
Mujeen Sung, Hwisang Jeon, Jinhyuk Lee. and Jac- woo Kang. 2020. Biomedical entity representations with synonym marginalization. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, ACL 2020, Online, July 5-10. 2020, pages 3641-3650. Association for Computa- tional Linguistics.
Jinguang Zheng, Daniel Howsmon, Boliang Zhang, Juergen Hahn, Deborah L. McGuinness, James A. Hendler, and Heng Ji. 2015. Entity linking for biomedical literature. BMC Medical Informatics De- cis. Mak., 15-S(5-
Hongyin Tang, Xingwu Sun, Beihong Jin, and Fuzheng Zhang. 2021. A bidirectional multi parairap read- ing model for zero-shot entity linking. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thin Third Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Ar- tificial Intelligence, EAAT 2021, Virtual Event, Febru- ary 2-9, 2021, pages 13889-13899 AAAI Press.
Maya Varma, Laurel J. Orr, Sen Wu, Megan Leszezyn- ski, Xiao Ling, and Christopher Ré. 2021. Cross- domain data integration for named entity disambigua-


Reviews:
[{"review_id": "o19XoI0LfN", "reviewer": "Reviewer_gxqK", "summary": "", "questions": "I wonder if the proposed method also works for other domains. It seems the framework is not specific to the bio domain at all. If the only required domain knowledge is entity descriptions, there are a lot of knowledge bases that can provide those for various domains. Is there a specific reason that you constrain your study to the bio domain?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and sincerely appreciate your positive feedback. We would like to address the specific questions below.\n\n1.Work for other domains\n\nWe are confident that our proposed method can be applied effectively to other specialized domains, only if these domains meet the following three conditions: 1) a large amount of seen entity training data, 2) the existence of unseen entities that are prevalent in these domains and 3) sufficient domain knowledge to generate latent features. The bio domain meets all these conditions simultaneously, which motivated us to devise this method to generate pseudo data in the latent feature space for unseen entities to enhance overall performance. Additionally, the bio domain exhibits more advanced baselines and possesses more comprehensive datasets. Hence, this paper primarily focuses on the bio domain.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The problem setting is realistic: Indeed, in real-world applications, an entity linking system should better perform well on both seen and unseen entities.\n- The proposed method is interesting and makes sense to me.\n- The writing is clear and very easy to follow.\n- The experiment section is solid and it answers almost all questions in my mind.", "reasons_to_reject": "None. I believe this paper should be accepted.", "questions_to_authors": "I wonder if the proposed method also works for other domains. It seems the framework is not specific to the bio domain at all. If the only required domain knowledge is entity descriptions, there are a lot of knowledge bases that can provide those for various domains. Is there a specific reason that you constrain your study to the bio domain?", "paper_topic_and_main_contributions": "This paper proposes an interesting solution to biomedical entity linking where the test set contains both seen and unseen entities during training. It uses a GAN to learn latent representations for unseen entities so that the correct unseen entity has a higher chance of being captured at top k by the retriever compared to existing methods.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nO7hzKeBJo", "reviewer": "Reviewer_v9F6", "summary": "", "questions": "1. What’s the key motivation and novelty of this paper? \n2. Have you ever thought about the potential of your method to apply the Large Language Models (LLMs, such as GPT-4)?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive suggestions and sincerely appreciate your positive feedback. We will improve the presentation to improve this paper in the revised version. We would like to address the specific questions below.\n\n1.Key motivation and novelty\n\nIn the biomedical field, there are many unseen entities lacking training data due to the presence of rare diseases that occur infrequently in literature. Limited by understanding these unseen entities, there are three types of linking errors. However, previous works are good at managing the second type of linking errors but still seem troubling to deal with the other two types of linking errors. This motivates us to find a way to simultaneously deal with all three types of linking errors to improve the performance. Thus, in this paper, we propose a novel framework to generate pseudo data in the latent feature space for unseen entities to fine-tune the entity encoder of our retriever. This captures more fine-grained coherence signals of unseen entities, allowing for simultaneously handling all the three types of linking errors.\n\n2.Significance and robustness analysis\n\nFor the significance analysis, our results over all baselines are statistically significant with p < 0.05 according to the t-test. Robustness analysis usually modifies the input to verify the robustness of the model. To the best of our knowledge, all previous biomedical entity linking works have not included the robustness analysis. In the multimodel entity linking setting, [1] designs a robustness analysis to demonstrate that their model can better utilize visual information to compensate for the textual noise. However, in our biomedical entity linking setting, visual information is not available, and the textual information in the biomedical domain is typically rigorous and complete. Therefore, we argue that designing a robustness analysis by adding textual noise in this paper is unnecessary.\n\n3.Large language models\n\nLarge language models (such as GPT-4) are pre-trained using general domain resources. However, the focus of this paper is specifically on the biomedical domain. Some previous works [2, 3] have demonstrated that large language models perform significantly worse than domain-specific language models (such as BioBERT). This is because domain-specific language models are pre-trained on vast biomedical text corpora and possess extensive domain knowledge. Hence, using LLMs is not appropriate either in this task or for our proposed method.\n\n[1] Wang et al, 2022, Multimodal Entity Linking with Gated Hierarchical Fusion and Contrastive Training, SIGIR 2022.\n\n[2] Moradi et al, 2021, GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain, CORR.\n\n[3] Gutiérrez et al, 2022, Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again, In Findings of the Association for Computational Linguistics: EMNLP 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea of proposing a novel latent feature generation framework BioFEG to address multiple types of linking errors in biomedical entity linking is meaningful and interesting.\n2. This paper presents the design principle of the BioFEG methodology in detail, which has the potential to motivate more related works for addressing the linking errors in biomedical entity linking tasks.\n3. The experimental part is rich, and a lot of studies have been done on the proposed BioFEG framework.\nIn general, this paper is well-organized and is a good work that can promote the development of biomedical entity linking.", "reasons_to_reject": "1. The presentation of the paper should be improved. The expressions in the current version are not natural and elegant enough, and it would be better to improve them with the help of an English native speaker.\n2. The key motivation and novelty of this paper should be discussed more.\n3. The experiments are rich and diverse. But to a certain extent, this paper is lack significance and robustness analysis and it would be better to include them. \n4. With the rapid development of Large Language Models (LLMs, such as GPT-4), it is necessary to discuss the potential of LLMs for this task.", "questions_to_authors": "1. What’s the key motivation and novelty of this paper? \n2. Have you ever thought about the potential of your method to apply the Large Language Models (LLMs, such as GPT-4)?", "paper_topic_and_main_contributions": "This paper proposes a novel latent feature generation framework BioFEG to address multiple types of linking errors in biomedical entity linking. Specifically, BioFEG leverages domain knowledge to train a generative adversarial network, which generates latent semantic features of corresponding mentions for unseen entities. Utilizing these features, they fine-tune an entity encoder to capture fine-grained coherence information of unseen entities and better understand them, which allows models to make linking decisions more accurately, particularly for ambiguous mentions involving rare entities. Extensive experiments on the two benchmark datasets demonstrate the superiority of the proposed BioFEG framework.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The presentation of the paper should be improved. The expressions in the current version are not natural and elegant enough, and it would be better to improve them with the help of an English native speaker."}}, {"review_id": "oLT6MyUNXx", "reviewer": "Reviewer_msFt", "summary": "", "questions": "The paper is very clear and Section 5 answered all the questions I came up with while reading the first part of the paper. I don’t really have any technical questions.\n\n- Out of curiosity, did you analyze all 3450 errors (Table 4) to get the statistics in section 5.4?\n- Do you plan to release a version of your codebase?\n", "limitations": "", "rating": 4, "confidence": 5, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the insightful suggestions and we sincerely appreciate your positive feedback. We would like to revise all typos and address the specific questions below.\n\n1.Analysis of 3450 errors in the MedMentions dataset\n\nWe didn’t analyze the 3450 errors in the MedMentions dataset (Table 4) to get the statistics in section 5.4. We performed the statistics by entirely manual examining, and 3450 errors is a very large number. Thus, we perform the error analysis on the smaller dataset BC5CDR, which has only a few hundred errors.\n\n2.Source code\n\nWe will release the complete code for all experiments to GitHub after this paper is accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- the proposed method is novel, interesting, and makes sense\n- the method is presented in a clear and understandable way\n- the ablation studies and the extra analyses answer most of the questions that a reader would have while reading the paper\n- the proposed method improves the performance on unseen entity normalization compared with previous models\n", "reasons_to_reject": "None that comes to mind.\n", "questions_to_authors": "The paper is very clear and Section 5 answered all the questions I came up with while reading the first part of the paper. I don’t really have any technical questions.\n\n- Out of curiosity, did you analyze all 3450 errors (Table 4) to get the statistics in section 5.4?\n- Do you plan to release a version of your codebase?\n", "paper_topic_and_main_contributions": "The paper tackles the problem of medical entity normalization, and in particular how to improve the performance of the models on unseen entities (normalizing mentions whose entities are never seen at training time).\n\nThe proposed approach trains a mention and an entity embedder to generate meaningful embeddings on the training (seen) data. \\\nDomain knowledge is used to generate fake mention embeddings for unseen entities with a GAN model. \\\nThese fake mention embeddings are then used to further finetune the entity embedder. \\\nAn additional re-ranking step ensures maximum performance after the two previous models have retrieved the best candidate entities to normalize a mention.\n\nThe authors compare their proposed model with several strong baselines, showing improved performance on unseen entities. The ablation studies conducted also highlight the importance of the single components of the proposed architecture.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 55: “Figure. 1a” →”Figure 1a”\n- Figure 2: minor suggestion, you could add the number of the BERT models in the image BERT_1, …, BERT_4. Although the difference is already clear thanks to the different colors, it would further tie together the text and figure, and make the figure more readable if printed in grayscale.\n- Line 182: “there are rich” → “there is rich” (?)\n- Line 229-235: the description of the input is clear enough, but it would be even better with an example of the input for a real entity / mention. The meaning of “title” and “description” might be unclear, I guess you are using information from UMLS, but it would be good to state it somewhere.\n"}}]

Author rebuttal:

Meta-review: This paper's problem setting is realistic, the proposed method is interesting and clear, and the experiments are solid and address all relevant questions.
Decision: Accept-Main