Controllable Contrastive Generation for Multilingual Biomedical Entity Linking
Authors: Tiantian Zhu
Yang Qin
Qingcai Chen
Xin Mu
Changlong Yu
Yang Xiang
Venue: EMNLP 2023 Main (2023)
Keywords: Multilingual Biomedical Entity Linking
Controllable Generation
Contrastive Learning

Abstract: Multilingual biomedical entity linking (MBEL) aims to map language-specific mentions in the biomedical text to standardized concepts in a multilingual knowledge base (KB) such as Unified Medical Language System (UMLS). In this paper, we propose Con2GEN, a prompt-based controllable contrastive generation framework for MBEL, which summarizes multidimensional information of the UMLS concept mentioned in biomedical text into a natural sentence following a predefined template. Instead of tackling the MBEL problem with a discriminative classifier, we formulate it as a sequence-to-sequence generation task, which better exploits the shared dependencies between source mentions and target entities. Moreover, Con2GEN matches against UMLS concepts in as many languages and types as possible, hence facilitating cross-information disambiguation. Extensive experiments show that our model achieves promising performance improvements compared with several state-of-the-art techniques on the XL-BEL and the Mantra GSC datasets spanning 12 typologically diverse languages.

Controllable Contrastive Generation for Multilingual Biomedical Entity Linking
Tiantian Zhu', Yang Qin', Qingcai Chen 1:2;; Xin Mu?, Changlong Yu', Yang Xiang?
Harbin Institute of Technology (Shenzhen), Shenzhen, China PPnng Cheng Laboratory, Shenzhen, China
The Hong Kong University of Science and Technology, Hong Kong, China zhu. tiantianl 10@gmail. com, {csyqin, qingcai. chen]@hit edw. cn, {mux, xiangy) ac. cn, cyuaqêcse .st. hk
Abstract
Mention in G German
Context: ist Erstbeschriiber Sülisoof
[Stevia] (CUP C1018893), durunter Rebandiusid A.
Multilingual biomedical entity linking (MBEL) aims to map lunguage-specfiic mentions in the biomedical text standardized concepts in multilingual knowledge base (KB) such Uni- fied Medical Language System (UMLS). In this paper, we propose Con2GEN, prompt- based controllable contrastive generation framework for MBEL, which summarizes multidimen- sional information of the UMLS concept tioned in biomedical text into a natural sentence following a predefined template. Instead of tackling the MBEL problem with discrimina- tive classifier, we formulate it as a sequence-to- to- sequence generation task, which better exploits the shared dependencies between source men- tions and target entities. Moreover, Con2GEN matches against UMLS concepts in as many languages and types as possible, hence facilitat- ing cross-information disambiguation. Exten- sive experiments show that our model achieves promising performance improvements com- pared with several state-of-the-arr techniques the XL-BEL and the Mantra GSC datasets spanning 12 typologically diverse languages.
Linking
Top Candidates
Multidimensional Information
I Language: English
Name: Stevia CUI: C0075246
Type: Organic Chemical
Type: Pharmicologic Substance
Language: German
CUI: C1018893
Type: Plant
Name: Stevia
Language: Fnglish
CUI: C2370269
Type: Organic Chemical
Language: Spanish
(organiseno)
CUI: C1018893
Type: Plant
Figure I:
An example multilingual biomedical en- tity linking, where the mention is in red. The col- ored boxes represent the top candidates retrieved by SAPBERT Among them, the orange box indi- cates the incorrectly linked entity and the green box indicates the matching entity.
(typically English) (Bitton et al., 2020; Liu et al., 2021a; Zhu et al., 2022), these approaches cannot be effectively applied to other languages due the huge discrepancies between multilingual and monolingual versions of the biomedical entity link- ing (BEL) task. Firstly, resources for non-Eng oon-nnglish BEL are scarce, hindering the development of mul- tilingual research. For example, although the Uni- fied Medical Language System (UMLS) (Bodenrei- der, 2004) is often regarded as a large multilingual KB in the biomedical domain, more than 70% of concept synonyms are in English, while the mi- nority of concept names are in other languages. Secondly, the ambiguity problem in MBEL is more severe compared to monolingual BEI,, since named entities with the same surface form in different languages can express distinct semantics, which makes the task of MBEL more challenging. A dominant paradigm of most current MBEL
1 Introduction
Multilingual biomedical entity linking (MBEL) is an essential yet challenging task for domain- specific natural language understanding, which refers to the process of mapping mentions in source language to standard entities in multi- lingual knowledge base (KB). Suecesses in build- ing MBEL systems can assist various medical re- search, such as biomedical knowledge probing (Meng et al., 2022) and clinical decision support (Chen et al., 2022), which need to determine the distinct meanings of biomedical terms, regardless of language.
Although promising breakthroughs have been achieved in monolingual biomedical entity linking
"Corresponding authors.

---

solutions is to provide a single representation for each biomedical term by fine-tuning multilingual pre-trained models, such as mBERT (Devlin et al., 2019) or XLM-R (Conneau et al., 2020), on phrase- level synonyms extracted from the UMLS (Liu et al., 2021b; Yuan et al., 2022b). However, these methods still suffer from two major drawbacks, namely indirect and uninformative. Concretely, the indirect problem refers to the candidate entities ranked with a dot-product between dense vectors between the mention and the entity, which fails to capture the interactions between them. The unin- formative problem refers to the fact that current work solves the MBEL task via shallow matching based on surface forms, which ignores the deep un- derlying information of entities, such as language and context.
matches against UMLS concepts in as many lan- guages and types as possible, which allows the transfer of the latent knowledge of multilingual pre-trained models about the task. In this way, the decoder of Con2GEN can he guided to generate in limited space by filling the fixed slots in the pre- defined template. Moreover, we utilize contrastive learning to alleviate the exposure bias problem in generation (An et al., 2022) an autoregressive model trained only using the ground truths exhibits inferior generalization performance. Finally, we evaluate the performance of our model using the XL-BEL (Liu et al., 2021b) and the Mantra GSC (Kors et al., 2015) datasets spanning 12 typologi- cally diverse languages, and against several state- of-the-art (SOTA) baselines. From the extensive experimental results, the proposed Con2GEN strat- egy is demonstrated to be effective in multilingual disambiguation based on multidimensional infor- mation of bio-entities. The code is available at https://itthub.com saggituubbcoom/iaaiiaaznuut TiantianZuul 10/Con2GEN.
Consider the example in Figure 1, where Ste- via is German mention of the concept C1018893 in UMLS. In this case, although the top candi- date entity retrieved by SAPBERT multi (Liu et al., 2021b) has the same surface form as the mention, is not the target entity. Specifically, the entity Stevia is an English synonym for the concept C0075246. of type Organic Chemical or Pharmacologic Sub- stance, which is an ambiguous instance across lan- guages and types. However, if the MBEL model understands the context of mentions and utilizes the language and type information of entities, the target concept may have a greater chance of being retrieved, which will make the entity disambigua- tion across languages more precise.
To sum up, the contributions of this paper are as follows: (1) For the first time, we propose the Con2GEN, novel controllable contrastive genera- tive method, to address the ambiguity challenge of the MBEL task; (2) We design constrained decod- ing algorithm consisting of predefined template and contrastive learning mechanism for entity multidimensional information injection during in- ference; (3) Extensive experiments show that our model achieves promising performance improve- ments on three public MBEL datasets spanning 12 diverse languages.
Motivated by the above observations, in this paper, we propose a prompt-based controllable contrastive generation (Con2GEN) algorithm to disambiguate biomedical entities across diverse languages. To address the indirect challenge, we exploit a sequence-to-senee to-suueece architecture to ef- fectively cross-encode the mention and the entity capture more latent interactions. Generative models inherently have a better ability to align in- put and output spaces than discriminative models, since discriminative models need to iteratively fit atomic labels to learn aligned features (Cao et al., 2022). In detail, Con2GEN summarizes multidi- mensional information of the UMLS concept men- tioned in biomedical text into natural sentence following a predefined template, mainly aiming to solve the uninformative challenge and reduce ambiguity. Specifically, the predefined template is as close as possible to natural language and
2 Related Work
2.1 Multilingual Biomedical Entity Linking
Multilingual biomedical entity linking (MBEL) task is an extended version of monolingual biomed- ical entity linking (Zhu et al.. 2022: Liu al., 2021a: la; Wu et al., 2020) and cross- lions-lingual biomed- ical entity linking (XBEL) (Rijhwani et al., 2019; Bitton et al., 2020), which focuses on mapping an input mention from biomedical text in a spe- cific language to its associated entity in curated multilingual KB. In monolingual biomedical entity linking, mentions always match the KB language, and entities in other languages are discarded. How- ever, the monolingual formulation of biomedical entity linking has the problem that the KB might miss several entries for languages with limited cov- erage of entity descriptors. The XBEL formulation

---

alleviates this problem to some extent and con- siders mentions expressed in different languages and linked to a monolingual KB (usually English) (Roller et al., 2018; Bitton et al., 2020). Although both the MBEL and XBEL formulations exploit inter-language links to identify entities in other lan- guages, XBEL requires the target KB to be mono- lingual, which might still miss several entries in the KB.
ing algorithm whose output sequences contain non- semantic indicators such as ">>". However, such non-semantic indicators express little semantic in- formation and have a gap with natural language, which may mislead the decoding process. Unlike the decoding algorithm of mGENRE, Con2GEN generates the associated entities by filling the fixed slots in the predefined natural language template, which allows Con2GEN to handle various entity types and alleviate ambiguity biomedical tasks. Furthermore, GenBioËL (Yuan et al., 2022a) is the first to explore generative EL in the biomedi- cal domain. However, the differences between our work and GenBioEL are as follows: (1) Different decoding objectives: GenBioEL only generates the entity name at the decoding phase, ignoring the multidimensional information of bio-entities and increasing the difficulty of disambiguation, Instead, we proposed constrained decoding algorithm con- sisting of a predefined template for multidimen- sional information injection during inference. (2) Different training manners: GenBioEL pre-trained the model to improve the generalization ability. In- stead, we utilized multi-task learning jointly op- timize the contrastive loss and the task loss and meanwhile strengthen the proposed Con2GEN model.
In this work, we focus on MBEL, which is more general since it can link to entities that might not be necessarily represented in the target monolingual KB but in any of the available languages. In par- ticular, we use UMLS (Bodenreider, 2004) as our multilingual biomedical KB, as is a rich concep- tual ontology differentiated by Concept Unique IDs (CUIs) and relations between them across 25 lan- guages. Moreover, we maintain as much language and type information as possible, hence learning the connections between the source language and entity names in different languages.
2.2 Sequence to Sequence Text Generation
The sequence-to-sequenee models (Sutskever et al., 2014) learn to generate an output sequence of ar- bitrary length from an input sequence, and have demonstrated its flexibility and effectiveness in a wide range of tasks compared to classification- based methods. Early generative models used recur- rent neural networks (RNNs) and attention mecha- nism (Bahdanau et al., 2015) to capture long-term dependencies between the source and the target sequences. Later, the Transformer (Vaswani et al., 2017) was proposed, which solely based on at- tention mechanisms, dispensing with recurrence and convolutions, and showed significant perfor- mance improvement. Recently, Transformer based pre-trained language models (De Cao et al., 2021; Lewis et al., 2020; Raffel et al., 2020; Liu et al.. 2020) provide a new trend to utilize massive text corpora and have become the baseline for most text generation tasks due their impressive perfor- mance.
In this paper, we aim to facilitate MBEL with a sequence-to-sequenee generation method to cap- ture the dependencies between mentions and enti- ties. Specifically, we propose novel prompt-based controllable contrastive decoding algorithm, which can guide the generation process using predefined natural language template and contrastive learning. In this way. the multidimensional knowledge biomedical entities can be injected and exploited during inference.
3 Method
3.1 Problem Statement
Given a biomedical mention m in free text x of language I, and a multilingual KB (e.g., UMLS) of n entities E = {c1, (21.02-- c2. cx). the MBEL system will output the target entity from E. In this work, we assume that each mention has its own golden CUI and each CUI has at least one synonym, regard- less of language. Note that the type and language information of the bio-entities is required, which differs from the general domain setting. In the gen- eral domain, we assume that each entity deseriptor contains a name that concisely describes the entity,
Close to this work, mGENRE (Cao et al., 2022) also employed neural generation models for mul- tilingual entity linking, mapping mentions to the Wikipedia titles in multiple languages. However, mGENRE focuses on multilingual entity linking in the general domain and cannot be simply trans- ferred to the biomedical domain. Furthermore, mGENRE applies a sequence-to-stree to structure decod-

---

e.g., the Wikipedia title (with language identifier) can be used to uniquely identify an entity descrip- tor, while in the biomedical domain entity names with languages are still ambiguous combinations. Note that mapping the output to CUT is non-trivial task. However, by adding multidimensional infor- mation, ambiguity has been greatly reduced. In the experiment, if the combination of entity name, type, and language cannot uniquely identify CUI, we will randomly select one CUI from them. More details are provided in Appendix A.I.
follows:
Po(y))) II pa (uelree,
where MKi = yn-.. 1.-1, y is the identifier of entity, and θ denotes the parameters of the model M.. Moreover, since the input and the output tokens of our model are both natural language words, we adopt the multilingual pre-tree trained language model mBART (Liu al., 2020) as our base model, which will take advantage of the general multilingual text generation knowledge.
3.2 Model Architecture
this work, we frame the MBEL task as a sequence-to-sequence generation problem and the overall architecture is illustrated in Figure 2. Given the token sequence x 21,.... ........ -||r| of the mention along with contextual information, Con2GEN directly generates the most relevant entity e according to specific template T(-) via an encoder-decoder architecture. Concretely, Con2GEN first encodes the input x, and then uses a prompt-based controllable contrastive decoding algorithm to generate the corresponding entity in natural language: a sequence of tokens y. In the fol- lowing sections, we introduce in detail the neural conditional text generation network and the prompt- based controllable contrastive decoding algorithm proposed in this work.
3.4 Prompt-based Controllable Decoding
3.4.1 Template
By designing appropriate template, encour- age Con2GEN better utilize the multidimen- sional information of entities, therefore, solve the uninformative problem. The desired prompt not only provides information but also defines the output format. Specifically, we create a natural language template T(-), in which we list all mean- ingful elements of an entity in the format of place- holders: T =[Ent] of type [Ent Type] in [Ent Lang]. Among them, [Ent] denotes the entity name. [Ent Type] is the semantic type of the en- tity and we use an empty string for entities without type information UMLS. [Ent Lang] denotes the target language.
3.3 Neural Conditional Text Generation Network
3.4.2 Controllable Decoding
A neural conditional text generation model (Sutskever et al., 2014) M = (f.g) g) generates the target sequence with respect to the source se- quence where f and denotes the encoder and decoder, respectively. To this end, Con2GEN first encodes the input biomedical sequence into the hidden vector representations H hf...., "......,",, via a lulti-layer transformer encoder f:
Naturally, the most straightforward solution for decoding is the greedy algorithm (Germann et al., 2001), which selects the token with the highest probability p at each step í. However, the greedy decoding algorithm may cause invalid entities that are not in the vocabulary of the UMLS. In addition.
the greedy decoding algorithm is uncontrollable since it cannot output sequences in the template format we designed, further ignoring the useful knowledge of entities.
H" ((11...,,
(I)
Taking the above needs into consideration, we employ a controllable decoding algorithm based on the trie tree (Cormen et al., 2009; Lu et al., 2021) to only output the valid token from constrained vocabulary set at each decoding step i. During controllable decoding, the multidimenssioall infor- (2)
During decoding, g utilizes the sequential hidden vectors Hº to predict the target entity token-by- token. Specifically, at step i, the token y; and the decoder state hę are computed as follows:
Mv. h = : :A...................... m
mation of entities is injected into the predefined template as a prompt for the decoder. Specifically, at each step i, the based decoding algorithm will dynamically traverse the children nodes from
Therefore, the entire conditional probability p(y]a) of generating the output sequence y given the in- put sequence x can be progressively calculated as

---

Seg2Sog Loss CaLL
Template:
[Bct]]
of type (Ent Type! [Gnt Leng)
Positive Output Space
Negative Output Space
Contrastive
Psrudomonas acruginosa infection type diseose syndrome E English.
Parudomonas aeruginosa infection of type injury poisoning in English.
Infection psrudomenas aeruginosa of type diseaus sy syudrome English.
AR 89 N S of type organisa function in Jpannsee.
Decoder
lanrrety:
20.00 Bi N O S type diszase syndrume Japanese,
infective polyarthritis type diseuse syndrome in English.
Chrunic
t Koream
2
type
syndrome
сус(Мリр)ン aaaaneee.
type pathologic
Encoder
Infecrtón por Psnadomonas aeruginosa of type disease syndrome Spanish.
t P.F. of type pathologic function Koream,
......
Imput (START) Chronic Infection [EDD]
with reduced lung
tusction, daster rate of lung deciine, increased rates of etacebbtions and shorter survival.
Figure 2: Con2GEN: the input a biomedical text where the mention is signaled with special tokens [START] and [END], and the output is the entity identifier following the predefined template. We use a prompt-based controllable decoder to generate entity names as well as types and languages. This example is real instance from our system.
the current state. For ease of understanding, we give an example of a complete decoding process by executing trie tree search in Appendix A.2.
hard negative samples during inference. Recently, contrastive learning provides new idea for alle- viating the exposure bias problem of generation models by additionally constructing positive and negative pairs during training (An et al., 2022). Contrastive learning optimizes model parameters by learning efficient representation space for the data, in which positive pairs pulled together and negative pairs are pushed apart.
Since the vocabulary set of UMLS E is very large (over 10M deduplicated entities) and the cost of ranking each entity in F. is prohibitive, we fur- ther restrict the generation space at inference time. First, employ the beam search approximate strategy (Sutskever et al., 2014) search for the top-b entities in E instead of scoring entities, which greatly improves inference efficiency. More- over, modern entity linking systems (Ledell Wu, 2020; Zhu et al., 2022) tend to apply candidate re- trieval first and then apply a finer-graine reranking instead of scoring all entities at once in order to save computational cost. Inspired by this, we also leverage the SAPBERT, multi (Liu al., 2021b) to generate top-k candidates with similar surface form for each input mention and further constrain the decoding generation space.
In this paper, we expect that by contrasting hard
Algorithm Contrastive learning training strategy
Input: Initialize the generation model M @' θ one training instance d (s, y). where input sequence, y output sequence
1: for epoch e = to do for batch b = 1 to B do Batch data D
2:
3:
(d), de....... p
4:
for d, E D do
5:
Randomly select two positive outputs vy. yft from the synonym set of the same CUI Randomly select three negative outputs v, y y," from the union hard negatives and samples with type information replaced Compute representations h² O(a, My)) hat O(zz, 5f h O(z,. ".'), A. O(x,, v ). ns O(2,, "s)
3.5 Training
6:
3.5.1
Contrastive Learning
Typically, generation systems are trained with teacher forcing with the ground truth labels in the training phase and exposed to incorrectly gen- erated tokens during the inference phase, thus en- countering the exposure bias problem (An al., 2022). Although the traditional neural generation method no longer needs negative sampling, the trained model will lose the ability to discriminate
8:
Compute contrastive loss Ces with 12, 19 1,1,, k,',,,,, 1,,.. end for
Compute average batch and update param- eter
end for

---

negatives, the proposed Con2GEN model can be sensitive to the entity semantics and further allevi- ate the exposure bias problem. Towards this, we first create the positive and negative pairs for each training sample. Specifically, for a specific training sample (z, y). we keep the input I constant and vary the output y. Then, we filter out the synonyms in different languages with the same CUI as y to build the positive set. Moreover, the negative in- stances come from two sources: 1) we replace the
we explored different ways to combine the NLL and contrastive loss. We utilized a hyper-parameter "label weight" to balance the NLL and contrastive loss, and did a grid search from 0 to with a step size of 0.1. And we found when the label weight equals 0.5, the performance is best on the valida- tion set.
Experiments
4.1 Datasets
type information in the template of y in order to cre- ate the semantically confusing negative instances; 2) we regard the incorrect candidates retrieved by SAPBERT multi as hard negative instances.
We used three datasets to evaluate the proposed Con2GEN model. Descriptions of the datasets and their statistics provided in Table 1. XL-BEL (Liu et al., 2021b) contains sentences from 10 lan- guage dumps of Wikipedia, with mentions linked to the 2020A full version of UMLS concepts. The size of the concept set in UMLS is 4, 281, 184. and the size of all surface form/synonym set is 12, 671, 936, And the concepts in UMLS belong to 127 semantic types. For each language, 1, 000 in- stances are randomly selected for the final test sets, where each evaluation instance consists of triplet (sentence, mention, CUT). EMEA and Patent are two subsets of the Multilingual Annotation of (4) Named Entities and Terminological Resource Ac- quisition gold-standard corpora (the Mantra GSC) (Kors et al.. 2015), which belong to different text types: European Medicines Agency (EMEA) and Patent. The Mantra GSC contains French, German, Spanish, and Dutch corpora that include medical terms mapped standard terms in SNOMED-CT, McSH, and MedDRA of the 2020AA full version of UMLS, including 801, 038 CUIs.
In the training phase, for each training sam- ple (zz, y). we randomly select two positive sam- ples (x, y²), (zt, ) and three negative samples (I, y²1), ), (2, y² from the additionally constructed positive and negative sets, respectively. Then we adopt the average outputs of M as the in- stance representation for modeling the contrastive objective. Inspired by InfoNCE (van den Oord et al., 2018), we define an objective LCr in the contrastive manner:
(PP)
LeL logg sim (kp + Ei petmeAP.A75)
where h2- (II, ("),")) h O y²t), hm @(x,y) ")", h O(r, )²²2), nm O y²*). o denotes the parameters of the model M. The details of the contrastive learning training process are described in Algorithm I.
3.5.2 Training Objective
Moreover, due to the lack of training data for the MBEL task, we utilize the Medmentions En- glish biomedical entity linking dataset (Mohan and Li, 2019) to construct the multilingual contextual training data. In order to solve the problem that the input of training data is all in English, we used the weakly supervised data augmentation method to increase the multilingual characteristies of the training data. Specifically, we swap the English mention of input with the multilingual entity of out- put. Since the Medmentions dataset only contains English instances, we use this simple method to generate multilingual input. We preserve many multilingual entities as possible, which plays role in smoothing multilingual data. We didn't use any translator in this process, which will not introduce any extra noise. The final constructed training and validation sets contain 3, 136, 568 and 412, 905 in-
This section describes the training objective of the proposed Con2GEN model. Specifically, given a training instance (x, y) pair, where x is the source sequence, and y is the target sequence, we first com- pute the following negative likeliidl (NLL)
loss:
ly Σ
ENLL
log polss v)).
(5)
where 0 is the parameters of model M. Simulta- neously, for each instance, we compute the con- trastive loss LC1, as described in the previous sec- tion. Hence, the final loss function L is the sum of the two, where L = LNLI LCL. By combining the NLL and contrastive loss in the training phase, the Con2GEN model can be jointly optimized in a multi-task learning manner, which will improve the generalization ability of the model. Moreover,

---

+
XL-BEL
EMEA
Patent
Train
Dey
Num.
EN
ES
DE
F1
RU
TR
KO
ZH
JA
TH
ES
FR
NL
DE
FR
DE
Sentences
34,
10, 058
994 986
996
995
995
966
971
988
984
963
100
100
100 100
50 50
Mentions 3, 136, 568
412, 905
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
432
431
434
425
342
348
Entities
25, 529
12, 535
807
889
908
816
816
694
829
875
822
704
295
304
304
299
223
223
Table 1: Statisties of different multilingual biomedical entity linking datasets used in this work.
stances, respectively.
was not compared in most biomedical studies. We compared with both the base and finetuned results of mGENRE in Table 2.
4.2 Evaluation Metrics and Baselines
For evaluation, we report Accuracy for evaluating the performance of the proposed model against the baseline models following the previous work (Cao et al., 2022). Appendix A.3 gives detailed descriptions of baselines.
In addition, analyze the contribution of each part of our model, we cumulatively removed com- ponents and listed the performance comparison of the proposed model and its variants in Table The w/o type, w/o lang, and w/o type & lang meth- ods represent the use of partial dimensional in- formation of entities rather than multidimensional information, resulting in a drop on performance. Even so, Con2GEN's variants with partial dimen- sional information are still strong baselines com- pared with other baselines. The contrastive mecha- nism is removed from the w/o CL method, which drops the Accuracy point of test sets compared with Con2GEN. The contrastive learning strategy can identify the hard examples and is demonstrated to be effective on 12 language subsets from Table 2. We believe the improvements are significant since we only trained one single model which outper- forms basclines almost all languages, Although the contrastive learning strategy improved the aver- age accuracy by only 0.2 points, it worked across diverse languages, which not trivial. Moreover, we added an ablation experiment of Con2GEN without using the controllable decoding algorithm, and the performance dropped significantly, which demonstrates the effectiveness of the proposed con- trollable decoding. In conclusion, the template with both language and type information is better than other templates in generation and the model that integrates contrastive learning and controllable decoding has the best performance.
4.3 Implementation Details
We used Adam optimizer with learning rate of le 6. B1 0.9, B2 0.999, eps= le 8, and with warm up for 500 steps for model training. The dropout rate and attention dropout are set to 0.1 and 0.1. We clip the gradient norm within 0.1. The la- bel smoothing of NLL loss is 0.1 and weight decay is 0.01. We used max 1, 024 tokens and variable batch size(>> 15). The training epoch of the model Constrained Beam
are set 3. At test time, Search with beams. We implemented our model using the fairseq library (Ott et al., 2019). Note that all parameters are selected based on the best performance on the development set.
4.4
Experimental Results
4.4.1
Main Results
To evaluate our approach, we compared the pro- posed Con2GEN model with several SOTA models on the XL-BEL and the Mantra GSC EMEA/Patent datasets and listed the performance of each model in Table 2. As shown in Table 2, our model out- performed all other models, which improved 1.3 Accuracy points on average over the baselines on XL-BEL,, EMEA and Patent test sets. We note that the discriminative PLMs (mBERT, XLM-R) under- performed the generative PLM (mBART), which demonstrates the advantage of generative models in solving the indirect problem of MBEL. Since Bio- PRO (Zhu al., 2022) context-infused method in the monolingual BEL task, we finetuned it on the multilingual training data and reported the results in Table 2. Note that mGENRE (Cao et al., 2022) is a SOTA method in the general domain, which
4,4.2 Resolution of the Ambiguity Problem
In order to study the resolution of different mod- els on the ambiguity problem, we compared the top 1 candidates prediction's 's performance of our proposed model with SAPBERT multi. which is a surface form matching based MBEL model. As mentioned before, the ambiguous phenomenon is severe the MBEL task, which exists not only in different contexts but also across languages, in-

---

Dutaset->
XL-BEL
EMEA
Patent
Avg
Methodl
TR
KO
ZH
DE
HR
DE
mBERT al. 2019)
8.0
0.6
7.0
4.6
XLM-R (Cormeau sl., 2020)
0.5
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
mHAKT (Liu 2020)
15.0
9.0
30.0
27.2
22-1
BioPROr-inetune (Zhu et al., 2022)
80.4
46.0
27.2
19.0
34.2
34.3
12.7
12.7
19.8
12.9
53.9
50.3
47.5
46.5
55.0
50.0
37.7
SAPBERT (Liu et 202
86.2
15.4
28.5
19.9
38
1
35.5
11.
14.5
23.1
14.0
58.3
53.
55.5
53.6
63.5
61.2
41.8
CODER (Yaan er 2022b)
51.4
31.0
20.3
37.3
35.4
1.4
14.4
22.6
56.9
63.6
66.9
63.4
64.9
65.8
40.9
mGENRR (Cao al., 2022)
85.2
48.9
30.9
23.8
39.9
36.9
15.7
16.7
23.9
16.0
56.5
54.8
49.3
52.7
61.1
55.6
41.9
müB:NE--lnncuue (Coo al., 2022)
41.5
39.5
61.2
Con2GEN (wlo type)
24.6
38.0
39.8
16.9
59.5
49.5
52.5
66.4
65.5
43.4
Con2GEN (win lang)
86.0
62.1
31.2
24.7
39.5
40.4
17.1
17.3
26.3
18.2
63.2
67.8
54.1
06.1
64.9
64.7
44.5
Con2GEN (wlo type lang)
39.5
16.8
24.5
53.4
66.4
65.5
Con2GENN (wlo CL.)
86.3
52:2
30.6
24.9
39.7
40.3
17.2
17.3
25.4
18.
63.0
56.9
51.8
51.8
64.9
61.9
44.3
Con2GEN (wlo controllable decoding)
52.7
4.5
5.0
2.0
3.3
7.1
5.3
6.4
12.5
13.2
20.0
17.1
15.1
23.4
20.7
13.3
Con2GEN
86.6
52.7
30.8
24.4
39.5
40.5
17.4
17.1
25.6
19.2
63.9
57.1
51.4
55.5
64.0
66.1
44.5
Table 2: Performance on the XL-BEL and the Mantra GSC EMEA Patent datasets in comparison with the SOTA methods. Avg refers the average performance across all target languages. Top: SOTA methods. Bottom: Con2GEN with and without different components. "w/o type" denotes removing type information from the template. "wlo lang" denotes removing language information from the template. "w/o type & lang" denotes removing type and language information from the template. "w/o CL." denotes training without the contrastive learning strategy. "w/o controllable decoding" denotes without using the controllable decoding. Bold denotes the best result in the column.
Dutasct-
XL-BBL
Avg
Method_
DE
RU
TR
ZH
TH
mGENRR
85.2
48.9
30.9
23.8
39.9
36.9
15.7
16.7
23.9
16.0
33.8
+finetune
85.4
49.8
31.3
24.8
41.5
39.5
16.0
16.9
23.9
17.2
34.6
+finetune+type
87.71
50.77
30.4
24.5
41.2
40.1
16.7t
16.4
25.3"
18.0"
35.1"
+Linetune+LL
85.3
50.01
31.71
24.8
42.5
39.91
16.4
16.6
24.3
17.5
34.9"
+finctunettpe+CT..
86.81
50.97
30.7
24.8
41.9
40.2
16.9t
16.7
24.9
18.3
35.2"
Table 3: Accuracy of mGENRE's variants on the XL-BEL dataset. "+finetume" denotes finetuning on the multilingual biomedical training data we constructed. "*finetune+type" denotes finetuning with type information. "+finctune+CL" denotes finetuning with the contrastive learning strategy. "ffnetunc+typ=+CL" denotes finctuning with type information and contrastive learning strategy.
creasing the challenge of the task. Table 4 lists the number of wrongly predicted top 1 candidates, the number of ambiguous examples in wrongly predicted top candidates, and the proportion of ambiguous instances in the overall top 1 errors on the XL-BEL test set. It shown that our model can greatly reduce the proportion of ambiguous examples among the total misclassified examples, which demonstrates the effectiveness of the pro- posed method in addressing the ambiguity chal- lenge and understanding the deep semantics of mentions and entities.
forms the base mGENRE model on each XL-BEL test set, which demonstrates the effectiveness of the proposed weakly supervised data augmenta- tion method. Moreover, assume that the in- jection of multidimensional information may be useful to the mGENRE generation as well. Thus, wc reported the "+finctune+type" results in Ta- ble 3, which improves the accuracy by 0.5 on av- erage compared to the base model. Note that the "+finetun++type" variant of mGENRE still under- performed our model, which demonstrates the ef- fectiveness of the proposed natural language tem- plate. since both mGENRE and our Con2GEN uti- lize controllable decoding inference, yet their output templates differ.
4.4.3 Efffet of Multidimensional Information Injection
From Table 2, observe that mGENRE is strong baseline even without finetuning on the multilin- gual biomedical dataset. To further study the po- tential of mGENRE on the MBEL task, we fine- tuned it on the training data we constructed. As can be seen from Table 3, "+finetune" method outper-
4.4.4 Effect of Contrastive Learning
As shown in Table 2, the proposed model achieved better performance with the contrastive learning strategy compared with the "w/o CL" method on 12 subsets. To further study the effect of contrastive

---

Source
Meation with contest
SAPBERT
Con2GEN
In den Sathorpn Taglnalaa" "Iandomnaae"" "Vhbro" "Satnonnla"" und [[TAIT]]
JiN Drensse
Shipelb Hartedtu
Shigella Indet man sichuge Kunbhtatrgeerr Metsch Tee und Plance.
85
La Förmiz de Parkand es una tenuis uniizzda para estiazr la cantiadt de fiuid de repisickis regaenia pars prinaras 24 luras to pacseti [STilT]] quenado (END)
Quemante) ES| cunltattveoomcee
Quematura (rg.toms)) BS polsoning
Intury
de ganutiiar qua
www.eeennnnnninnnnnnnnneee vstali..
F1
kincckisi Wagiiamppsssussi on viyy Poljess Amcrilan liniioiia ja Untoca- seatamtin puralauteeepperr Iitarniann wis (STARI]) parmavallebjja [END))
Parvanocte EN Bokeeysez
Mazosue parmia (oceniono (E3) Mamnnal
fomavallabjj letakase Muteen Scclartile.
94 242/5/p olst Po
B S 8 2190, [ENO] 2074 2013 2I 84
AB 50%
2: 2 US 2 ( [START] +
+ Ko) location
(producto
R1PE2 HM 0 2 Mo M K.
mediciaa Phermsologs sobotance
Josssan wascordpaaeee
[START]
Yossun pantunarraaa I BU Barndical
Kanur Ell Thamuecaiic
recooppaccaaurrrrcc [END].
denaal material
poscslats
11
Kromat igpllstnnde sloipe (STAKE)) meligronntaminn (END) en gärdien yaklt, skusmio bucndi Iansitnmddr.
Maligurn EEN Qquellettse concept
8:5:5 JA peopee
F #t. [STakT] HMM (100)) #HP9
IEIA (EIIAACCHlcco Cell
absu J8 Plazt
ARR 4N4
Figure 3: Examples of top 1 candidates retrieved by SAPBERT multi and Con2GEN, Bold denotes mentions. Underline refers to the golden entities.
Method
SAPBERT muits
Con2GEN
entities. However, it clearly indicates that the pro- posed method can make up for this disadvantage and make comprehensive judgments by leveraging the contextual and language information of entities
Languagel
#Err
#Amb
Percent
#Err
#Amh Percent
EN
138
87
63.0%
134
56
41.8%
ES
516
108
20.9%
473
37
7.8%
DE
715
0.6%
692
2
0.3%+
and mentions.
FI
801
12
1.5%
756
4
0.5%+
RU
616
21
3.4%
602
5
0.8%
TR
644
16
2.5%
595
8
1.3%
5 Conclusion
KO
852
42
4.9%
826
30
3.6%+
ZH
855
21
2.8%
829
17
2.1%
In this paper, focus both the indirect and the uninformative challenges in multilingual biomed- ical entity linking. Towards this, we propose Con2GEN, a prompt-based controllable contrastive generation method. Our approach outperforms baselines on 3 benchmark datasets, including XL- BEL, Mantra GSC EMEA, and Mantra GSC Patent. Further experiments and analysis demonstrate that the controllable contrastive decoding strategy of our model could considerably reduce ambiguity and meanwhile gain high performance.
769
152
19.8%
714
102
13.7%
TH
859
2
0.2%
808
1
0.1%
Avg
676.5
46.8
6.9%
645.9
26.2
4.1%*
Table Statistics the ambiguous instances in mispre- dictions XL-BEL test set. "#Err" refers the number of top candidates predicted incorrectly. "#Amb" refers to the number of ambiguous instances in top candi- dates predicted incorrectly. "Percent" refers to the ratio ambiguous instances to error instances.
learning, we had initially hoped that the contrastive learning's capability would make other neural net- works work better, and we add it to the mGENRE during finetuning. In Table 3, the improvements of the mGENRE with contrastive learning can be achieved most languages of the XL-BEL dataset, which demonstrates that our contrastive learning framework supports other coouuncce-o-scquenne models as well.
Limitations
One limitation of this work is that the generative model used this paper requires a large amount of training data and is computationally expensive. Fur- thermore, due to the lack of multilingual domain- specific training data, we utilize English text and a multilingual knowledge base to construct the mul- tilingual contextual training data. However, the constructed data may be biased for two reasons. Firstly, the number of non oon-English entities in the multilingual knowledge base is small, which lim- its the linguistic diversity of generated data. Sec- ondly, we only replace the English mentions with non-Enge Mon-English synonyms, and the remaining context remains in English.
4.4.5 Case Study
To illustrate the effectiveness of the proposed model, we listed the top 1 candidates retrieved by AAPBERTmait and our Con2GEN in Figure respectively. We can infer from Figure 3 that SAPBERT mults prefers entities with similar surface form as the mentions, but may miss real matching

---

Acknowledgments
cross lingual representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Associa- rion for Computational Linguistics, ACI. 2020, On- line, July 5-10, 2020, pages 8440-8451, Association for Computational Linguistics.
This work was supported in part by the Na- tional Key Research and Development Program of China (No. 2022ZD01 16002), National Natu- ral Science Foundation of China (No. 62006061,
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, 2009. Introduction Algorithms, 3rd Edition. MIT Press.
61872113, 62106115, 62276075, 62106114), Sci- ence and Technology Planning Project of Shen- and Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologics, China (No. 2022B1212010009
zhen (No. (XX200198001122200066
Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. Autoregressive entity retrieval. In 9th International Conference on Learning Repre- sentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Jacob Devlin, Ming-Was Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pages 4171 1171-4186. Association for Computational Linguistics.
References
Chenxin An. Jiangtao Feng, Kai Lv, Lingpeng Kong, Xipeng Qiu, and Xuanjing Huang. 2022. Cont: Contrastive neural text generation. arXiv preprint arXiv:2205.14690
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings.
Ulrich Germann, Michael Jahr, Kevin Knight, Daniel Marcu, and Kenji Yamada. 2001. Fast decoding and optimal decoding for machine translation. In Asso- ciation for Computational Linguistic, 39th Annual Meeting and 10th Conference of the European Chap- ter, Proceedings of the Conference, July 9-11, 2001, Toulouse, France, pages 228-235. Morgan Kaufmann Publishers.
Yonatan Bitton, Raphael Cohen, Tamar Schifter, Eitan Bachmat, Michael Elhadad, and Noémie Elhadad. 2020. Cross-lingual unified medical language system entity linking in online health communities. J. Am. Medical Informatics Assoc., 27(10) 1592.
Jan A. Kors, Simon Clematide, Saber A. Akhondi, Erik M. van Mulligen, and Dietrich Rebholz- Schuhmann. 2015. A multilingual gold-standard cor- pus for biomedical concept recognition: the mantra GSC. Am. Medical Informatics Assoc., 22(5):948- 956.
Olivier Bodenreider. 2004. The unified medical lan- guage system (UMLS): integrating biomedical termi- nology. Nucleic Acids Res., DaDaaaaaae-Issuee22000 270.
Nicola De Cao, Ledell Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke Zettlemoyer, Nicola Cancedda, Sebastian Riedel, and Fabio Petroni. 2022. Multilingual autoregressive entity linking. Trans. Assoc. Comput. Linguistics, 10:274-290.
Martin Josifoski Sebastian Riedel Luke Zettlemoyer Ledell Wu, Fabio Petroni. 2020. Zero- shot entity linking with dense entity retrieval. In EMNLP.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: demoising sequence-to-seqqencce pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, ACI. 2020, Online, 5-10, 2020,, pages 7871-7880. Association for Computational Linguistics.
Junying Chen, Dongfang Li, Qingcai Chen, Wenxiu Zhou, and Xin Liu. 2022. Diaformer: Automatic diagnosis via symptoms sequence generation. In Thirty-Sixih AAAI Conference on Artificial Intelli- gence, AAAI 2022, ThirtyFourth Conference on In- novative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22- March 1, 2022, pages 4432- 4440. AAAI Press.
Fangyu Liu, Ehsan Shareghi. Zaiqiao Meng, Marco Basaldella, and Nigel Collier. 2021a. Self-alignment pretraining for biomedical entity representations. Proc. of NAACL, pages 4228-4238.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised
Fangyu Liu, Ivan Vulie, Anna Korhonen, and Nigel
Collier. 2021b. Learning domain- -eeain--peeiallied repre- sentations for cross-lingual biomedical entity linking.

---

In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Lan- guage Processing, ACL/JCCNPP 2021, (Volume 2: Short Pupers), Virtual Event, August 1-6, 2021, pages 565-574. Association for Computational Linguistics.
Roland Roller, Madeleine Kittner, Dirk Weissenbom. and Ulf Leser. 2018. Cross-lingual candidate search for biomedical concept normalization. CoRR, abs/ 01111800501666.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Sys- tems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Mon- treal, Quebec, Canada, pages 3104-3112.
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li. Sergey Edunov, Marjan Ghazvininejad, Mike Lewis. and Luke Zettlemoyer. 2020. Multilingual denoising pre- training for neural machine translation. Trans. Assoc. Comput. Linguistics, 8:726-742.
Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. CoRR, hatp//007.037555
Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, and Shaoyi Chen. 2021. Text2event: Controllable sequence-to- structure generation for end-to-end event extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing, ACL/IJCNLP 2021. (Volume 1: Long Papers). Virtual Event, August 1-6, 2021, pages 2796-2806. Association for Computational Linguis- tics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017, Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 2017, Long Beach, CA, USA, pages 5998-6008.
Ledell Wu, Fabio Petroni, Martin Josifoski. Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero- shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6397- 6407. Association for Computational Linguisties.
Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi, Yixuan Su, Charlotte Collins, and Nigel Collier. 2022. Rewire- then- then-probe: A contrastive recipe for probing biomed- ical knowledge of pre-trained language models. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics ( (Volume 1: Long Pupers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pages 4798-48918 Association for Computa- tional Linguistics.
Hongyi Yuan, Zheng Yuan, and Sheng Yu. 2022a. Gen- erative biomedical entity linking via knowledge base- guided pre-training and synonyms-ave aware fine-tuning, In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, NAACI. 2022, Seattle, WA, United States, July 10-15, 2022, pages 4038-4048, Association for Computa- tional Linguistics.
Sunil Mohan and Donghui Li. 2019. Medmentions: A large biomedical corpus annotated with UMLS concepts. In Proc. of AKBC.
Myle Ott, Sergey Edunov, Alexci Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli, 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HUP 2019, Minneapo- lis, MN, USA, June 2-7, 2019, Demonstrations, pages 48 53. Association for Computational Linguistics.
Zheng Yuan, Zhengyun Zhao, Haixia Sun, Jiao Li, Fei Wang, and Sheng Yu. 2022b. CODER: knowledge-infusee cross-lingual medical term em- bedding for term normalization. J. Biomed. Informat- ics, 22:110398
Tiantian Zhu, Yang Qin. Qingcai Chen, Baotian Hu, and Yang Xiang. 2022. Enhancing entity representations with prompt learning for biomedical entity linking. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 4036-4042. ijcai.org.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 21:140:1-140:61
Shruti Rijhwani, Jiateng Xie, Graham Neubig, and Jaime G. Carbonell. 2019. Zero-shot neural transfer
A Appendix
for cross-lingual entity linking. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty- First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1. 2019, pages 6924 6931. AAAI Press.
A.1 Mapping Output to CUIs
SAPBERT, mults only considers surface form match- ing and therefore cannot disambiguate the same en- tity name with different CUIs. However, we utilize multidimensional information, and ambiguity has

---

been greatly reduced. Moreover, if the gold entity is one of the matching entities, SAPBERT mults will consider it correct, but we randomly select one CUI from them for evaluation. Thus, if we adopt the
of XLM-R in the experiments is the same mBERT.
mBART (Liu al., 2020) is multilingual sequence-tor -euueuecet---equeecee sequence pre-training method that produces significant performance gains across a wide variety of tasks. We didn't finetune it on the training set.
evaluation method of SAPBERT multi- the accuracy scores will improve than what we reported.
A.2 Example of Trie Tree Search
For ease understanding, we give an example of a complete decoding process by executing a trie tree search in Figure 4. For example, there is generation path "ROOT-RE--O type-inorganic chemical-in-Jappnes, in Figure 4. Correspond- ingly, there must be record like "Name: R Language: Japanese, Type: inorganic chemical" UMLS. Among them, the information of name and language can be found in the UMLS file "MR- CONSO.RRI", and the information of type can be found in the UMLS file "MRSTY.RRF". We only wrote data processing code to integrate multi- dimensional information without modification.
BioPRO (Zhu et al., 2022) proposes two-stage linking algorithm to enhance the monolingual entity representations based on prompt learning, Since BioPRO is context- iiffuute-ifueed method in the monolingual BEL task, we finetuned it on the multilingual training data.
SAPBERT multi (Liu et al., 2021b) proposes a multilingual extension of the self-alignment pre-training technique to improve domain- specialized representations in resource-lean lan- guages. SAPBERT modti restricted the target modt
UMLS ontology to only include CUIS that ap- pear in WikiMed (62, 531 CUIs, 399, 931 en- tity names), which greatly reduced the knowl- edge base size and task difficulty. However, we utilized the 2020AA full version of UMLS for evaluation (4, 281, 184 CUIs, 12, 671, 936 entity names), which much harder. The results in Table 2 produced using the officially released code of SAPBERT malti-
ROOT
cettia curbane
stope
sftpp
otopp
d'ops
(formentogg
dhesical
pracedire
shankcl
rabattinc
pracekzs
Aepeeee
Jipaeee
Fagish
Eaglss
Sparii
Riglts
Kanwn
CODER (Yuan al., 2022b) proposes a pre- training method to use both synonyms and rela- tions from the UMLS to direct the generation multilingual biomedical term embeddings.
The trie tree of the controllable decoding algorithm for entity generation.
Figure 4:
A.3 Descriptions of Baselines
mGENRE (Cao et al., 2022) proposes an autore- gressive formulation to the multilingual entity linking problem the general domain. We com- pared with both the base and finetuned results of mGENRE in Table 2.
For the evaluation of the proposed model, we use the following SOTA baseline methods for compari- son.
mBERT (Devlin et al., 2019) is multilingual contextualized language representation encoder that is pre-trained using bidirectional transform- ers. Following the previous work (Liu et al., 2021b), we use the pre-trained mBERT to com- pute vector representations. Specifically, [CLS] of the last layer's s output is used as the final rep- resentation. At inference, given query represen- tation, nearest neighbor search is used to rank all candidates' representations.
XLM-R (Conneau et al., 2020) is a transformer- based multilingual masked language model pre- trained on text in 100 languages. The use


Reviews:
[{"review_id": "tNuFZ0o38A", "reviewer": "Reviewer_4M3G", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**(R denotes reasons to reject)**\n\n**R.** It is true that the main innovative ideas are oriented from preexisting works. However, multilingual BEL is a challenging task due to the lack of training data and the need for disambiguation across languages. Thus, in this paper, we not only simply apply preexisting approaches to the MBEL task. Firstly, we propose a novel prompt-based controllable contrastive decoding algorithm, which can guide the generation process using a predefined natural language template and contrastive learning. In this way, the multidimensional knowledge of biomedical entities can be injected and exploited during inference. Secondly, we alleviate the problem of lacking MBEL training data by proposing a weakly supervised data augmentation method. Finally, extensive experiments show that our model achieves promising performance improvements spanning 12 typologically diverse languages. \n\nAs suggested, we have conducted more comprehensive experiments to prove the algorithm's performance and make the experimental results more convincing.\n\nFirstly, we added the ablation study of Con2GEN without using controllable decoding. The experimental results are as follows: Accuracy on XL-BEL dataset: EN: 52.7, ES: 4.5, DE: 5.0, FI: 2.0, RU: 3.3, TR: 7.1, KO: 5.3, ZH: 6.4, JA: 12.5, TH: 4.6. Accuracy on EMEA dataset: ES: 13.2, FR: 20.0, NL: 17.1, DE: 15.1. Accuracy on Patent dataset: FR: 23.4, DE: 20.7. The average accuracy point across all target languages is 13.3, which dropped significantly compared to Con2GEN, demonstrating the effectiveness of the proposed controllable decoding.\n\nSecondly, we added experiments of finetuning mGENRE on EMEA and Patent datasets. The fine-tuning performances of mGENRE on all datasets are as follows: Accuracy on XL-BEL dataset: EN: 85.4, ES: 49.8, DE: 31.3, FI: 24.8, RU: 41.5, TR: 39.5, KO: 16.0, ZH: 16.9, JA: 23.9, TH: 17.2. Accuracy on EMEA dataset: ES: 62.0, FR: 56.4, NL: 55.1, DE: 54.1. Accuracy on Patent dataset: FR: 61.2, DE: 55.7. The average accuracy of finetuned mGNERE is 85.4, which is worse than our Con2GEN.\n\nFinally, we added the experiment of “mGENRE+finetune+type+CL”, and the accuracy on each language of the XL-BEL dataset is as follows: EN: 86.8, ES: 50.9, DE: 30.7, FI: 24.8, RU: 41.9, TR: 40.2, KO: 16.9, ZH: 16.7, JA: 24.9, TH: 18.3, Avg: 35.2. Our proposed Con2GEN still outperforms the method mGENRE+finetune+type+CL.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses a significant problem by proposing a prompt-based controllable contrastive generation framework, and several improvement techniques have been designed. The experimental evaluation shows that the proposed algorithm improves the preexisting algorithms. ", "reasons_to_reject": "The main innovative ideas are oriented from preexisting works in NLP, wherein commonly adopted. LLM, Multi-Prompt, and Curriculum Learning might further improve the optimization technique via language awareness. Moreover, there are significant differences between the general and medical domains, and the domain migration strategy might significantly impact the model performance. Therefore, they should conduct more comprehensive experiments to prove the algorithm's performance and make the experimental results more convincing.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper studies the problem of multilingual biomedical entity linking, which is essential for various downstream applications. Specifically, the paper proposes a prompt-based controllable contrastive generation framework by summarizing the UMLS concept's multidimensional information mentioned in biomedical text into a natural sentence following a predefined template.\nFurthermore, the proposed method matches against UMLS concepts in as many languages and types as possible, facilitating cross-information disambiguation. \nExperimental results show that the proposed algorithm is competitive with SOTA baselines.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9axbuibdXR", "reviewer": "Reviewer_F782", "summary": "", "questions": "1. I am not clear what the authors mean by 'fails to capture the interactions between them' (line 68) - what interactions are you referring to? \n\n2. I do not believe this is completely true about current work only solving the MBEL task via shallow matching on surface forms ignoring the context. Please see: Wu et al 2019 (https://aclanthology.org/2020.emnlp-main.519/)\n\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**(R denotes reasons to reject and Q denotes questions for the authors)**\n\n**R1. & Q2.** Wu et al., 2019 propose the BLINK model to solve the monolingual entity linking problem in the general domain, which also considers contextual information. However, monolingual entity linking approaches cannot be effectively applied to other languages due to the huge discrepancies between multilingual and monolingual versions of the entity linking task. Note that in Table 1, we also reported the results of BioPRO (Zhu et al., 2022), which is a SOTA method in the monolingual BEL task and incorporates context. Thank you for pointing out these citations, we will carefully proofread our manuscript in the next revision.\n\n**R2.** Following the previous work (SAPBERTmulti (Liu et al., 2021b)), we use these pre-trained baselines to compute vector representations. Specifically, [CLS] of the last layer’s output is used as the final representation. At inference, given a query representation, a nearest neighbor search is used to rank all candidates’ representations.\n\n**R3.** SAPBERTmulti (Liu et al., 2021b) restricted the target UMLS ontology to only include CUIs that appear in WikiMed (62,531 CUIs, 399,931 entity names), which greatly reduced the knowledge base size and task difficulty. However, we utilized the 2020AA full version of UMLS for evaluation (4,281,184 CUIs, 12,671,936 entity names), which is much harder. The results in Table 1 are produced using the officially released code of SAPBERTmulti (Liu et al., 2021b).\n\n**Q1.** Our intention is that discriminative classifiers need to iteratively fit atomic labels to learn aligned features, which is an indirect manner. However, generative models inherently have a better ability to align input and output spaces than discriminative models, which will better exploit the shared dependencies (interactions) between source mentions and target entities.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Explores biomedical entity linking across 12 different languages\n2. Conduct an evaluation across three different datasets \n3. Interesting approach that leverages summarising text into a predefined template to generate the entities. \n", "reasons_to_reject": "I really liked the approach, however:\n\n1 the authors disregard previous work that has incorporated context which needs to be addressed\n\n2. it is not clear to me how all the baseline comparisons were conducted (e.g. what is the mBERT method in Table 1 - the appendix only describes the language model itself - not how the classification was conducted)\n\n3. it is also not clear where the accuracy of all of the results is coming from (e.g. SAPBERTmulti Liu et al 2021b - where did the 86.2 in EN XL-BEL come from - I don't see that result in the cited paper)", "questions_to_authors": "1. I am not clear what the authors mean by 'fails to capture the interactions between them' (line 68) - what interactions are you referring to? \n\n2. I do not believe this is completely true about current work only solving the MBEL task via shallow matching on surface forms ignoring the context. Please see: Wu et al 2019 (https://aclanthology.org/2020.emnlp-main.519/)\n\n", "paper_topic_and_main_contributions": "The authors propose a framework for multilingual biomedical entity linking that focuses on generating natural language sentences summarizing information from UMLS concepts in biomedical texts. Their approach utilizes a sequence-to-sequence generation task and is designed to match UMLS concepts across various languages and types, facilitating the disambiguation of cross-information. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "w6V4gAmUQh", "reviewer": "Reviewer_LBDQ", "summary": "", "questions": "1) What are the statistics for the training set? Why only use the smaller subset for evaluation?\n2) How do you swap english mentions and non-english entities of the data? Any translator in this process?\n3) I would expect more details on each baseline, like how do you use mBERT or XLM-R for the task, multiclass classification or just using them to compute vector representations? What are their output spaces and what are the sizes of their training data if it is trained?\n4) It seems mGENRE is not trained at table 1, are other models also not trained? What are the fine-tuning performances of mGENRE on all datasets? Are they better than the fine-tuned Con2Gen?\n5) What are the performances of models without using constraint decoding algorithms? \n6) Have you explored different ways to combine the NLL and contrastive loss?\n7) How do you map the output of the sequence-to-sequence model to CUIs? Is it 1-to-1 mapping?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**(R denotes reasons to reject, Q denotes questions for the authors, and T denotes typos grammar style and presentation improvements)**\n\n**R1. & Q1.** For the sake of fairness, we follow the same test sets as SAPBERTmulti (Liu et al., 2021b) and CODER (Yuan et al., 2022). Since SAPBERTmulti (Liu et al., 2021b) and CODER (Yuan et al., 2022) used smaller subsets for evaluation, we are consistent with these existing publications. We briefly describe the statistics of the training and validation sets in lines 446-448. Specifically, more statistics about the training and validation sets are as follows: For the training set, the number of sentences is 30,349, the number of mentions is 3,136,568, and the number of entities is 25,529. For the validation set, the number of sentences is 10,058, the number of mentions is 412,905, and the number of entities is 12,535.\n\n**R2.** We believe the improvements are significant since we only trained one single model which outperforms baselines on almost all languages. Although the contrastive learning strategy improved the average accuracy by only 0.2 points, it worked across 9 diverse languages, which is not trivial. Moreover, we can infer from Table 3 that the proposed contrastive learning strategy also makes mGENRE work better, demonstrating its generalization. Although the paper “Contrastive Neural Text Generation” also utilizes contrastive learning, our training objectives are quite different from theirs. Concretely, for each instance, we constructed “semantically confusing negatives” and “hard negatives” specifically for the MBEL task. Moreover, when computing the contrastive loss, we only utilized the representations of the decoder since the representations of the decoder are the same, however, the paper “Contrastive Neural Text Generation” utilized both the representations of the source and the representations of the target.\n\n**R3.** Since mGENRE is the extended version of GENRE in multilingual scenarios, we only cited mGENRE in our manuscript. Thank you for pointing this out, we will cite GENRE in the next revision. For the sake of fairness, we have added experiments of finetuning mGENRE on EMEA and Patent datasets. The fine-tuning performances of mGENRE on all datasets are as follows: Accuracy on XL-BEL dataset: EN: 85.4, ES: 49.8, DE: 31.3, FI: 24.8, RU: 41.5, TR: 39.5, KO: 16.0, ZH: 16.9, JA: 23.9, TH: 17.2. Accuracy on EMEA dataset: ES: 62.0, FR: 56.4, NL: 55.1, DE: 54.1. Accuracy on Patent dataset: FR: 61.2, DE: 55.7. The average accuracy of finetuned mGNERE is 85.4, which is worse than our Con2GEN.\n\n**R4. & Q5.** As suggested, we have added the ablation study of Con2GEN without using controllable decoding. The experimental results are as follows: Accuracy on XL-BEL dataset: EN: 52.7, ES: 4.5, DE: 5.0, FI: 2.0, RU: 3.3, TR: 7.1, KO: 5.3, ZH: 6.4, JA: 12.5, TH: 4.6. Accuracy on EMEA dataset: ES: 13.2, FR: 20.0, NL: 17.1, DE: 15.1. Accuracy on Patent dataset: FR: 23.4, DE: 20.7. The average accuracy point across all target languages is 13.3, which dropped significantly compared to Con2GEN, demonstrating the effectiveness of the proposed controllable decoding. \n\n**R5. & Q7.** It is true that mapping the output to CUI is a non-trivial task. We mentioned in lines 234-242 that in the general domain the Wikipedia title can be used to uniquely identify an entity, however, in the biomedical domain, bio-entity names with type and language information are still ambiguous combinations. However, by adding multi-dimensional information, ambiguity has been greatly reduced. In the experiment, if the combination of entity name, type, and language cannot uniquely identify a CUI, we will randomly select one CUI from them. Moreover, if we combine these CUIs, the performance will be further improved.\n\n**Q2.** We propose a straightforward idea of swapping the English mention of input with the multilingual entity of output. Since the Medmentions dataset only contains English instances, we use this simple method to generate multilingual input. We didn’t use any translator in this process, which will not introduce any extra noise. Moreover, we discuss the possible limitations of our constructed data in lines 592-603.\n\n**Q3.** Following the previous work (SAPBERTmulti (Liu et al., 2021b)), we use these pre-trained baselines to compute vector representations. Specifically, [CLS] of the last layer’s output is used as the final representation. At inference, given a query representation, a nearest neighbor search is used to rank all candidates’ representations.\n\n**Q4.** Following the previous work (SAPBERTmulti (Liu et al., 2021b)), we also didn’t train other models. The fine-tuning performances of mGENRE on all datasets are as follows: Accuracy on XL-BEL dataset: EN: 85.4, ES: 49.8, DE: 31.3, FI: 24.8, RU: 41.5, TR: 39.5, KO: 16.0, ZH: 16.9, JA: 23.9, TH: 17.2. Accuracy on EMEA dataset: ES: 62.0, FR: 56.4, NL: 55.1, DE: 54.1. Accuracy on Patent dataset: FR: 61.2, DE: 55.7. The average accuracy of finetuned mGNERE is 85.4, which is worse than our Con2GEN.\n\n**Q6.** We have explored different ways to combine the NLL and contrastive loss. Specifically, we utilized a hyper-parameter “label weight” to balance the NLL and contrastive loss, and did a grid search from 0 to 1 with a step size of 0.1. We found when the label weight equals 0.5, the performance is best on the validation set. Due to the length limit, we didn’t mention it in the manuscript. \n\n**T.** Thank you for pointing out the flaws in our writing, we will carefully proofread our manuscript in the next revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper combines a few techniques (prompt based controllable decoding, and contrastive learning for text generation) together for the MBEL task, which is a pretty important task in the biomedical domain. Their approach achieves better performances than a few baseline approaches. And they also conduct a few ablation studies to understand the contribution of each component.", "reasons_to_reject": "I have a few main concerns about their paper: 1) for each language from the XL-BEL dataset, only 1000 instances are selected from each dataset, and also a few hundreds examples from EMEA and patent dataset, why only use such smaller subset for evaluation, not the entire dataset from XL-BEL? Also more statistics about the training and validation data. 2) it’s unclear whether the performances are significant or not? Especially for the model without using contrastive learning, it seems only 0.2 percent lower than the complete model, which does not support their claim of the contrastive learning, and it seems their ideas are from the paper “Contrastive Neural Text Generation”, which does not bring new things to the reader. 3) their controllable decoding algorithm is very similar to the work of GENRE, they ignore to mention this part in the related work and method section. Also, it seems fine-tuning mGENRE on the same data may achieve better performances, but the authors do not evaluate all the data and compare it fairly. 4) There are also a few ablation studies I expect to see, like the performances of models without using controllable decoding, and different hard examples for contrastive learning. 5) The output of the sequence-to-sequence model is the entity text plus its type and language information,however the evaluation target is the CUI. Mapping the output to CUI is not a trivial task, but authors did not mention anything for this. In general, I like the task of this paper and the idea of combining different techniques, but I would expect more details for points 1) - 5).", "questions_to_authors": "1) What are the statistics for the training set? Why only use the smaller subset for evaluation?\n2) How do you swap english mentions and non-english entities of the data? Any translator in this process?\n3) I would expect more details on each baseline, like how do you use mBERT or XLM-R for the task, multiclass classification or just using them to compute vector representations? What are their output spaces and what are the sizes of their training data if it is trained?\n4) It seems mGENRE is not trained at table 1, are other models also not trained? What are the fine-tuning performances of mGENRE on all datasets? Are they better than the fine-tuned Con2Gen?\n5) What are the performances of models without using constraint decoding algorithms? \n6) Have you explored different ways to combine the NLL and contrastive loss?\n7) How do you map the output of the sequence-to-sequence model to CUIs? Is it 1-to-1 mapping?", "paper_topic_and_main_contributions": "This paper presents a new approach for multilingual biomedical entity linking (MBEL); instead of using the classifier-based approach or vector space model, they formulate the task as a sequence-to-sequence generation task, and propose a prompt-based controllable contrastive generation framework. They design a prompt template to encourage the model to learn both semantic type and language information about the entity name; their controllable decoding algorithms reduces the output space thus improving the computational cost during inference; their contrastive training objective could  learn more discriminative information about hard negative examples. They evaluate their approach on multiple datasets including 1000 test examples for each one of the 10 languages from XL-BEL dataset, and a few 100 examples from EMEA and patent dataset, their approach achieves better performances over a few baseline models.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1) section 3.3 could be removed or shortened.\n2) Merge table 3 to table 1, and highlight whether these baseline are trained or not, and how they are trained."}, "extra_text": {}}, {"review_id": "A5zf0aNVU3", "reviewer": "Reviewer_iTH6", "summary": "", "questions": "A. How is your work different from the work from Yuan, H., Yuan, Z. and Yu, S., 2022, July. Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning?\n\nB. Are indirect and uninformative introduces by authors?\n\nC. What is the need to combine both the constative loss and NLL?\n\nD. Accuracy of mGENRE is compared for several cases staring from base to +finetune to +finetune+CL, why not +finetune+type+CL?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**(R denotes reasons to reject, Q denotes questions for the authors, and T denotes typos grammar style and presentation improvements)**\n\n**R1.** We gave an example of a complete decoding process by executing a trie tree search in Appendix A.1. For example, there is a generation path “ROOT-炭酸-of type-inorganic chemical-in-Japanese” in Table 4. Correspondingly, there must be a record like “Name: 炭酸, Language: Japanese, Type: inorganic chemical” in UMLS. Among them, the information of name and language can be found in the UMLS file “MRCONSO.RRF”, and the information of type can be found in the UMLS file “MRSTY.RRF”. We only wrote a data processing code to integrate multi-dimensional information without modification. Moreover, we added an ablation experiment of Con2GEN without using a trie-based controllable decoding algorithm, and the performance dropped significantly, which demonstrates the effectiveness of the proposed trie-based decoding. The results are listed as follows: Accuracy on XL-BEL dataset: EN: 52.7, ES: 4.5, DE: 5.0, FI: 2.0, RU: 3.3, TR: 7.1, KO: 5.3, ZH: 6.4, JA: 12.5, TH: 4.6. Accuracy on EMEA dataset: ES: 13.2, FR: 20.0, NL: 17.1, DE: 15.1. Accuracy on Patent dataset: FR: 23.4, DE: 20.7. The average accuracy point across all subsets is 13.3.\n\n**R2. & QC.** By combining the contrastive loss and NLL in the training phase, the Con2GEN model can be jointly optimized in a multi-task learning manner, which will improve the generalization ability of the model. Moreover, we explored different ways to combine the NLL and contrastive loss. We utilized a hyper-parameter “label weight” to balance the NLL and contrastive loss, and did a grid search from 0 to 1 with a step size of 0.1. And we found when the label weight equals 0.5, the performance is best on the validation set.\n\n**R3. & QB.** “Indirect” and “Uninformative” are different drawbacks, which are introduced in lines 66-75. Specifically, “Indirect” denotes discriminative classifiers need to iteratively fit atomic labels to learn aligned features, while generative models can directly align input and output spaces. “Uninformative” denotes ignoring the multi-dimensional information of entities, such as language, type, and context. \n\n**R4.** It is true that generative models are not a novel concept in BEL, however, multilingual BEL is much more challenging due to the lack of training data and the need for disambiguation across languages. Thus, in this paper, we not only simply apply the generative models to the MBEL task. Firstly, we propose a novel prompt-based controllable contrastive decoding algorithm, which can guide the generation process using a predefined natural language template and contrastive learning. In this way, the multidimensional knowledge of biomedical entities can be injected and exploited during inference. Secondly, we alleviate the problem of lacking MBEL training data by proposing a weakly supervised data augmentation method. Finally, extensive experiments show that our model achieves promising performance improvements spanning 12 typologically diverse languages. \n\n**QA.** The differences between our work and the paper “Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning” are as follows: (1) Different decoding objectives: Yuan. et al., 2022 only generate the entity name at the decoding phase, ignoring the multidimensional information of bio-entities and increasing the difficulty of disambiguation. In our paper, we proposed a constrained decoding algorithm consisting of a predefined template for multidimensional information injection during inference. (2) Different training manners: Yuan. et al., 2022 pre-trained the model to improve the generalization ability. Instead, we utilized multi-task learning to jointly optimize the contrastive loss and the task loss and meanwhile to strengthen the proposed Con2GEN model.\n\n**QD.** In order to analyze the contribution of “type” and “CL” separately, we reported the performances of “+finetune+type” and “+finetune+CL” methods in Table 3, respectively. As suggested, we have added the experiment of “+finetune+type+CL”, and the accuracy on each language of the XL-BEL dataset is as follows: \n| mGENRE | EN | ES | DE | FI | RU | TR | KO | ZH | JA | TH | Avg |\n|-------------------|-------|------|------|------|------|------|------|------|------|------|------|\n| +finetune+type+CL         | 86.8  | 50.9 | 30.7 | 24.8 | 41.9 | 40.2 |16.9 | 16.7 | 24.9 | 18.3 | 35.2 | \n\n**T.** Thank you for pointing out the flaws in our writing, we will carefully proofread our manuscript in the next revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The experiments are thorough.", "reasons_to_reject": "-Authors refer to why greedy algorithms limitations but the alternative they use (Trie tree based decoding) is not explained clearly. E.g., -UMLS is not originally structured as you depict. Did you have to do any modifications?\n-What is the need to combine both the constative loss and NLL?\n-Are indirect and uninformative introduces by authors? Not convinced that they are two different drawbacks.\n-Generative entity linking is not a novel concept in Biomedical entity linking, so the motivation is for multilingual aspect of BEL, but this is not adequately addressed in the paper.", "questions_to_authors": "A. How is your work different from the work from Yuan, H., Yuan, Z. and Yu, S., 2022, July. Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning?\n\nB. Are indirect and uninformative introduces by authors?\n\nC. What is the need to combine both the constative loss and NLL?\n\nD. Accuracy of mGENRE is compared for several cases staring from base to +finetune to +finetune+CL, why not +finetune+type+CL?", "paper_topic_and_main_contributions": "The paper frames the MBEL task as a generative task opposed to a discriminative task. Authors compare the performance of their Con2GEn approach with SOTA with ablation studies and show improved performances. Paper also discusses about improvements achieved through contrastive learning and multidimensional information injection.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Yuan, H., Yuan, Z. and Yu, S., 2022, July. Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 4038-4048).", "Typos_Grammar_Style_and_Presentation_Improvements": "There were several instances where strong claims were not supported by citations (e.g., claim starting in line 098, claim starting at line 041, )\nCite InfoNCE.\nThe main figure (2) is not illustrative enough to understand how the UMLS is utilized in the process, how the Trie tree decoding is utilized etc.\nA through revision is needed to make the paper more readable."}}]

Author rebuttal:

Meta-review: There is a consensus among the reviewers and most questions have been answered in the author response.
Decision: Accept-Main