Biomedical Concept Normalization by Leveraging Hypernyms (2021)

Abstract: Biomedical Concept Normalization (BCN) is widely used in biomedical text processing as a fundamental module. Owing to numerous surface variants of biomedical concepts, BCN still remains challenging and unsolved. In this paper, we exploit biomedical concept hypernyms to facilitate BCN. We propose Biomedical Concept Normalizer with Hypernyms (BCNH), a novel framework that adopts list-wise training to make use of both hypernyms and synonyms, and also employs norm constraint on the representation of hypernym-hyponym entity pairs. The experimental results show that BCNH outperforms the previous state-of-the-art model on the NCBI dataset.

## introduction biomedical concept normalization (bcn) plays an important and prerequisite role in biomedical text processing. the goal of bcn is to link the entity mention in the context to its normalized cui (unique concept identifier) in the biomedical dictionaries such as umls @xcite ), snomed-ct @xcite and @xcite . figure 1 is an example of bcn from ncbi dataset @xcite , the mention b-cell non-hodgkins lymphomas should be linked to d016393 lymphoma, b-cell in the medic @xcite dictionary. recent works on bcn usually adopt encoders like cnn @xcite , lstm @xcite , elmo @xcite or biobert @xcite @xcite to embed both the mention and the concept's name entities, and then feed the representations to the following classifier or ranking network to determine the corresponding concept in the biomedical dictionary. however, biomedical dictionaries are generally sparse in nature: a concept is usually provided with only cui, referred name (recommended concept name string), synonyms (acceptable name variants, synonyms), and related concepts (mainly hypernym concepts). therefore, effectively using the limited information in the biomedical dictionary where the candidate entities came from is paramount for the bcn task. for concept's synonym entities, recent bne @xcite and biosyn @xcite tries to make full use of them by synonym marginalization to enhance biomedical entity representation and achieved consistent performance improvement. unfortunately, previous works generally ignore concept hypernym hierarchy structure, which is exactly the initial motivation of biomedical dictionary: organization of thousands of concepts under a unified and multi-level hierarchical classification schema. we believe that leveraging hypernym information in the biomedical dictionary can improve the bcn performance based on two intuitions. first, hard negative sampling @xcite is vital for the bcn model's discriminating ability and a hypernym is a hard negative example for its hyponym naturally. second, injecting the hypernym hierarchy information during the training process is beneficial for encoders, since currently used encoders like biobert only encodes the context semantics in biomedical corpora instead of the biomedical concept structural information. to this end, we propose biomedical concept normalizer with hypernyms (bcnh), a novel framework combining the list-wise cross entropy loss with norm constraint on hypernym-hyponym entity pairs. concretely, we reformulate the candidate target list as a three-level relevance list to consider both synonyms and hypernyms, and apply the list-wise cross entropy loss. on the one hand, synonyms help to encode surface name variants, on the other hand, hypernyms help encode hierarchical structural information. we also apply the norm constraint on the embedding of hypernym-hyponym entity pairs to further preserve the principal hypernym relation. specifically, for a hypernym-hyponym entity pair (e hyper , e hypo ), we constraint that the norm of hypernym entity e hyper is larger than that of e hyper in a multi-task manner. we conduct experiments on the ncbi dataset and outperforms the previous state-of-the-art model. to sum up, the contributions of this paper are as follows. first, for the first time, we reformulate the candidate target list as a three-level relevance list and apply the list-wise loss to attend all candidate entities. second, we innovatively use norm constraint to model the hypernym-hyponym relation, preserving the hierarchy structure information inside the entity representation. the proposed bcnh outperforms the previous state-of-the-art model on the ncbi dataset, leading to an improvement of 0.73 % on top1 accuracy. ## methodology the architecture of our framework is illustrated in figure 2 . our model is composed of three parts: candidate generator to generate the candidate entities from the dictionary, list-wise ranker to train the encoder, hypernym normalizer to apply the hypernym-hyponym norm constraint. ## conclusion in this paper, we propose bcnh to leverage hypernyms in the biomedical concept normalization task. we adopts both list-wise training and norm constraint with the help of hypernym information. the experimental results on the ncbi dataset show that bcnh outperforms previous state-of-the-art models.