Controllable Contrastive Generation for Multilingual Biomedical Entity Linking (2023)

Abstract: Multilingual biomedical entity linking (MBEL) aims to map language-specific mentions in the biomedical text to standardized concepts in a multilingual knowledge base (KB) such as Unified Medical Language System (UMLS). In this paper, we propose Con2GEN, a prompt-based controllable contrastive generation framework for MBEL, which summarizes multidimensional information of the UMLS concept mentioned in biomedical text into a natural sentence following a predefined template. Instead of tackling the MBEL problem with a discriminative classifier, we formulate it as a sequence-to-sequence generation task, which better exploits the shared dependencies between source mentions and target entities. Moreover, Con2GEN matches against UMLS concepts in as many languages and types as possible, hence facilitating cross-information disambiguation. Extensive experiments show that our model achieves promising performance improvements compared with several state-of-the-art techniques on the XL-BEL and the Mantra GSC datasets spanning 12 typologically diverse languages.

## introduction multilingual biomedical entity linking (mbel) is an essential yet challenging task for domainspecific natural language understanding, which refers to the process of mapping mentions in a source language to standard entities in a multilingual knowledge base (kb). successes in building mbel systems can assist various medical research, such as biomedical knowledge probing @xcite and clinical decision support @xcite , which need to determine the distinct meanings of biomedical terms, regardless of language. although promising breakthroughs have been achieved in monolingual biomedical entity linking (typically english) @xcite @xcite , these approaches cannot be effectively applied to other languages due to the huge discrepancies between multilingual and monolingual versions of the biomedical entity linking (bel) task. firstly, resources for non-english bel are scarce, hindering the development of multilingual research. for example, although the unified medical language system (umls) @xcite ) is often regarded as a large multilingual kb in the biomedical domain, more than 70% of concept synonyms are in english, while the minority of concept names are in other languages. secondly, the ambiguity problem in mbel is more severe compared to monolingual bel, since named entities with the same surface form in different languages can express distinct semantics, which makes the task of mbel more challenging. a dominant paradigm of most current mbel solutions is to provide a single representation for each biomedical term by fine-tuning multilingual pre-trained models, such as mbert @xcite or xlm-r @xcite , on phraselevel synonyms extracted from the umls @xcite . however, these methods still suffer from two major drawbacks, namely indirect and uninformative. concretely, the indirect problem refers to the candidate entities are ranked with a dot-product between dense vectors between the mention and the entity, which fails to capture the interactions between them. the uninformative problem refers to the fact that current work solves the mbel task via shallow matching based on surface forms, which ignores the deep underlying information of entities, such as language and context. consider the example in figure 1 , where stevia is a german mention of the concept c1018893 in umls. in this case, although the top 1 candidate entity retrieved by sapbert multi @xcite has the same surface form as the mention, it is not the target entity. specifically, the entity stevia is an english synonym for the concept c0075246, of type organic chemical or pharmacologic substance, which is an ambiguous instance across languages and types. however, if the mbel model understands the context of mentions and utilizes the language and type information of entities, the target concept may have a greater chance of being retrieved, which will make the entity disambiguation across languages more precise. motivated by the above observations, in this paper, we propose a prompt-based controllable contrastive generation (con2gen) algorithm to disambiguate biomedical entities across diverse languages. to address the indirect challenge, we exploit a sequence-to-sequence architecture to effectively cross-encode the mention and the entity to capture more latent interactions. generative models inherently have a better ability to align input and output spaces than discriminative models, since discriminative models need to iteratively fit atomic labels to learn aligned features @xcite . in detail, con2gen summarizes multidimensional information of the umls concept mentioned in biomedical text into a natural sentence following a predefined template, mainly aiming to solve the uninformative challenge and reduce ambiguity. specifically, the predefined template is as close as possible to natural language and matches against umls concepts in as many languages and types as possible, which allows the transfer of the latent knowledge of multilingual pre-trained models about the task. in this way, the decoder of con2gen can be guided to generate in a limited space by filling the fixed slots in the predefined template. moreover, we utilize contrastive learning to alleviate the exposure bias problem in generation @xcite ) -an autoregressive model trained only using the ground truths exhibits inferior generalization performance. finally, we evaluate the performance of our model using the xl-bel @xcite and the mantra gsc @xcite datasets spanning 12 typologically diverse languages, and against several stateof-the-art (sota) baselines. from the extensive experimental results, the proposed con2gen strategy is demonstrated to be effective in multilingual disambiguation based on multidimensional information of bio-entities. the code is available at https://github.com/tiantianzhu110/con2gen . to sum up, the contributions of this paper are as follows: (1) for the first time, we propose the con2gen, a novel controllable contrastive generative method, to address the ambiguity challenge of the mbel task; (2) we design a constrained decoding algorithm consisting of a predefined template and a contrastive learning mechanism for entity multidimensional information injection during inference; (3) extensive experiments show that our model achieves promising performance improvements on three public mbel datasets spanning 12 diverse languages. ## conclusion in this paper, we focus on both the indirect and the uninformative challenges in multilingual biomedical entity linking. towards this, we propose con2gen, a prompt-based controllable contrastive generation method. our approach outperforms baselines on 3 benchmark datasets, including xl-bel, mantra gsc emea, and mantra gsc patent. further experiments and analysis demonstrate that the controllable contrastive decoding strategy of our model could considerably reduce ambiguity and meanwhile gain high performance.