Biomedical Entity Representations with Synonym Marginalization (2020)

Abstract: Biomedical named entities often play important roles in many biomedical text mining tools. However, due to the incompleteness of provided synonyms and numerous variations in their surface forms, normalization of biomedical entities is very challenging. In this paper, we focus on learning representations of biomedical entities solely based on the synonyms of entities. To learn from the incomplete synonyms, we use a model-based candidate selection and maximize the marginal likelihood of the synonyms present in top candidates. Our model-based candidates are iteratively updated to contain more difficult negative samples as our model evolves. In this way, we avoid the explicit pre-selection of negative samples from more than 400K candidates. On four biomedical entity normalization datasets having three different entity types (disease, chemical, adverse reaction), our model BioSyn consistently outperforms previous state-of-the-art models almost reaching the upper bound on each dataset.

## introduction biomedical named entities are frequently used as key features in biomedical text mining. from biomedical relation extraction @xcite to literature search engines @xcite , many studies are utilizing biomedical named entities as a basic building block of their methodologies. while the extraction of the biomedical named entities is studied extensively @xcite , the normalization of extracted named entities is also crucial for improving the precision of downstream tasks @xcite . ## experimental results we use five different versions of our model to see the effect of each module in our framework. first, biosyn denotes our proposed model with default hyperparameters described in section 4.1. biosyn (s-score) and biosyn (d-score) use only sparse scores or dense scores for the predictions at inference time, respectively. to see the effect of different dense ratios, biosyn (α = 0.0) uses only sparse candidates and biosyn (α = 1.0) uses only dense candidates during training. ## analysis 6.1 iterative candidate samples ## conclusion in this study, we introduce biosyn that utilizes the synonym marginalization technique and the iterative candidate retrieval for learning biomedical entity representations. on four biomedical entity normalization datasets, our experiment shows that our model achieves state-of-the-art performance on all datasets, improving previous scores up to 2.6%. although the datasets used in our experiments are in english, we expect that our methodology would work in any language as long as there is a synonym dictionary for the language. for future work, an extrinsic evaluation of our methods is needed to prove the effectiveness of learned biomedical entity representations and to prove the quality of the entity normalization in downstream tasks.