W iki UMLS : Aligning UMLS to W ikipedia via Cross-lingual Neural Ranking (2020)

Abstract: We present our work on aligning the Unified Medical Language System (UMLS) to Wikipedia, to facilitate manual alignment of the two resources. We propose a cross-lingual neural reranking model to match a UMLS concept with a Wikipedia page, which achieves a recall@1of 72%, a substantial improvement of 20% over word- and char-level BM25, enabling manual alignment with minimal effort. We release our resources, including ranked Wikipedia pages for 700k UMLSconcepts, and WikiUMLS, a dataset for training and evaluation of alignment models between UMLS and Wikipedia collected from Wikidata. This will provide easier access to Wikipedia for health professionals, patients, and NLP systems, including in multilingual settings.

## introduction the unified medical language system (umls) foot_1 is a controlled vocabulary resource, enabling standardisation of biomedical terminology and interoperability of electronic health systems across the world @xcite . umls has good coverage in only a handful languages such as english, impeding its uptake in health systems in different language settings @xcite . in addition, concept definitions in umls are either missing or very short, limiting its use as a medical encyclopedia. wikipedia, on the other hand, is a crowd-sourced encyclopedia that is a primary source of online medical knowledge for practitioners, students, and the general public @xcite @xcite . wikipedia hosts an increasing number of health-related articles @xcite . wikipedia articles exist in 100s of languages, far exceeding the multilingual support in umls. together, these characteristics point to the value of wikipedia for patient-facing health information. our goal in this work is to align umls concepts to their corresponding wikidata and through that to their available wikipedia concepts, to expand the language support for umls terminology with little effort. this will have a direct impact on patients worldwide by facilitating adoption of umls (including clinical terminology snomed-ct @xcite or medical subject headings, mesh @xcite ) in international healthcare systems, and also supporting medical information seeking needs of patients with varying linguistic backgrounds. for example, a patient whose native language is not english might receive a discharge summary in english with mentions of symptoms, diagnoses, or medications in formal clinical language. this language may not match wikipedia titles exactly, making search hard (and noisy). aligning umls and wikipedia facilitates such information seeking, makes medical information accessible in a patient's native language, and supports the need for patient access to information in consumer-friendlier terms @xcite . most common medical terms are covered in wikipedia; for instance, it is estimated that 80% of snomed-ct terms have a corresponding wikipedia page @xcite . our contributions are as follows: (1) we use the multilingual resources in umls and wikidata in a neural ranking model to retrieve wikipedia articles given a umls concept, and achieve a recall@1 of 72%, representing a 20% increase over a bm25 model; (2) we release our new wikiumls dataset (collected from wikidata) for training and evaluation of umls-to-wikipedia alignment models; and (3) we release the output of our cross-lingual ranking model as a large-scale silver-standard alignment of umls and wikipedia, facilitating new applications that can make use of the alignment. ## background umls combines multiple vocabularies into a unified coding system by mapping terms referring to the same concept to a single concept id (cui). a single cui can have several aliases in different languages from various vocabularies. a large proportion of umls concepts (roughly 100k) have descriptions that are very short (mostly one sentence), and so are not adequate for dissemination of knowledge or information access. medical entity pages in wikipedia have longer descriptions, are connected to other entities through hyperlinks, and are enriched by the wikidata @xcite knowledge graph. wikipedia is rich in content and multilinguality, as anyone can contribute to or revise an article. for example, the diabetes wikipedia page exists in more than 134 languages, compared to only a dozen languages in umls. knowledge-base alignment: there are two main approaches for aligning knowledge bases such as wikipedia and umls: (1) embedding-based alignment, and (2) string and semantic matching. in embedding-based methods, entity embeddings are learnt from text co-occurrence statistics or knowledgegraph (kg) relations separately, and are then aligned using a seed alignment dictionary @xcite , or adversarial learning @xcite . however, the text-based methods suffer from non-comparable corpora @xcite , and the kg methods have not been tried on wikidata because of its large scale and entity variety. string and semantic matching methods are based on similarity between the entity names or descriptions in the two knowledge-bases (kbs). each entity in kb 1 is used as a query against all the entities in kb 2 using a scoring method such as cosine similarity or bm25. entities are represented by bag of words or bag of character n-grams. the result of these cheap methods can be further improved by supervised (neural) ranking models @xcite @xcite . particularly when query and candidate documents are represented by an encoder such as bert @xcite , pre-trained on massive amounts of text data, neural rerankers perform substantially better than ir-based methods @xcite . our methods fit within this group. ## method given a umls concept c i represented by query q @xmath0, . . . , t n i }, where t n i is an alias term for c i in umls, we use the english wikipedia augmented with multilingual wikidata as a document collection @xmath1, . . . , d |d| }, to retrieve page d j matching concept c i . each page is represented by its title, text (only for candidate generation), and multilingual aliases from wikidata. we follow a two-stage retrieval procedure: (1) candidate generation, where an ir method (e.g. bm25) is used to retrieve related documents; and (2) reranking of the top k candidates via a learn-to-rank method @xcite . ## results the performance of the candidate generation methods for word-(bm25) and character-level (tfidf char ) retrieval, and also the reranking methods (bert, biobert, and bert multi ) are shown in figure 1 . in all top k positions, bm25 outperforms tfidf char , which is surprising given that tfidf char was reported to achieve strong performance for entity linking in @xcite . 5 as shown in ## conclusions we proposed passage pair ranking models based on pretrained contextual encodings for aligning umls and wikipedia, to help bridge between health information systems, and empower consumers with understanding of their health condition. we developed a dataset, wikiumls, for training and testing alignment models between the two knowledge-bases, and proposed neural reranking models that substantially outperform bm25. we showed that the use of multilingual aliases in bert multi substantially improves recall@1 compared to biobert (72 vs. 68). the use of subword information such as bpe @xcite as used in xlm (conneau and lample, 2019) might improve performance, which we leave for future work. utilising the relationships between concepts in umls and wikipedia (through wikidata) to align the two knowledge graphs is also an interesting future direction. we also intend to release a large wikipedia-based entity linking (el) dataset by using the top-ranked wikipedia pages for umls queries, to be used in state-of-the-art el models such as zeshel @xcite .