Knowledge-Rich Self-Supervision for Biomedical Entity Linking (2022)

Abstract: Entity linking faces significant challenges such as prolific variations and prevalent ambiguities, especially in high-value domains with myriad entities. Standard classification approaches suffer from the annotation bottleneck and cannot effectively handle unseen entities. Zero-shot entity linking has emerged as a promising direction for generalizing to new entities, but it still requires example gold entity mentions during training and canonical descriptions for all entities, both of which are rarely available outside of Wikipedia. In this paper, we explore Knowledge-RIch Self-Supervision (KRISS) for biomedical entity linking, by leveraging readily available domain knowledge. In training, it generates self-supervised mention examples on unlabeled text using a domain ontology and trains a contextual encoder using contrastive learning. For inference, it samples self-supervised mentions as prototypes for each entity and conducts linking by mapping the test mention to the most similar prototype. Our approach can easily incorporate entity descriptions and gold mention labels if available. We conducted extensive experiments on seven standard datasets spanning biomedical literature and clinical notes. Without using any labeled information, our method produces KRISSBERT, a universal entity linker for four million UMLS entities that attains new state of the art, outperforming prior self-supervised methods by as much as 20 absolute points in accuracy. We released KRISSBERT at https://aka.ms/krissbert.

## introduction entity linking maps mentions to unique entities in a target knowledge base @xcite . it can be viewed as the extreme case of named entity recognition and entity typing, where the category number swells to tens of thousands or even millions. entity linking is particularly challenging in high-value domains such as biomedicine, where variations and ambiguities abound. for instance, depending on the context, "pdf" may refer to a gene (peptide deformylase, mitochondrial), or file type (portable document format). similarly, "er" could refer to emergency room, the organelle endoplasmicreticulum, or the estrogen receptor gene. moreover, the number of entities in domains such as biomedicine can be very large. the unified medical language system (umls), a representative ontology for biomedicine, contains over three million entities @xcite . standard classification approaches such as medlinker @xcite require example gold mentions for each entity and cannot effectively handle new entities for which there are no labeled examples in training. recently, zeroshot entity linking has emerged as a promising direction for generalizing to unseen entities @xcite , by learning to encode contextual mentions for similarity comparison against reference entity descriptions. existing methods, however, require example gold entity mentions during training, as well as canonical descriptions for all entities. while applicable to wikipedia entities, these methods are hard to generalize to other domains, where such labeled information is rarely available at scale. in this paper, we explore knowledge-rich self-supervision (kriss) for entity linking by leveraging readily available domain knowledge to compensate for the lack of labeled information (figure 1 ). for entity linking, the most relevant knowledge source is the domain ontology. the core of an ontology is the entity list, which specifies the unique identifier and canonical name for each entity and is the prerequisite for entity linking. our method only requires the entity list and unlabeled text, which are readily available in any domain. in training, kriss uses the entity list to generate self-supervised mention examples from unlabeled ... the effect of alcohol consumption on violence related-injuries assessed in the er and to show how behavioral sciences ... ## knowledge-rich self-supervision for entity linking entity linking grounds textual mentions to unique entities in a given database/dictionary. formally, the goal of entity linking is to learn a function link : (m, t ) → e that maps mention m in the context t to the unique entity e. self-supervised entity linking assumes no access to any gold mention examples. the knowledge-rich self-supervision setting (kriss) assumes that only a domain ontology o and an unlabeled text corpus t are available. in particular, we require the availability of an entity list, which specifies for each entity the unique identifier and a canonical name. entity list is the prerequisite for entity linking, as it provides the targets for linking. our framework can also incorporate other knowledge in the ontology ( §3.5). ## conclusion we propose knowledge-rich self-supervised entity linking by conducting contrastive learning on mention examples generated from unlabeled text using available domain knowledge. experiments on seven standard biomedical entity linking datasets show that our proposed krissbert outperforms prior state of the art by as much as 20 points in accuracy. future directions include: further improving self-supervision quality; incorporating additional knowledge; applications to other domains.