BLAR : Biomedical Local Acronym Resolver (2021)

Abstract: NLP has emerged as an essential tool to extract knowledge from the exponentially increasing volumes of biomedical texts. Many NLP tasks, such as named entity recognition and named entity normalization, are especially challenging in the biomedical domain partly because of the prolific use of acronyms. Long names for diseases, bacteria, and chemicals are often replaced by acronyms. We propose Biomedical Local Acronym Resolver (BLAR), a high-performing acronym resolver that leverages state-of-the-art (SOTA) pre-trained language models to accurately resolve local acronyms in biomedical texts. We test BLAR on the Ab3P corpus and achieve state-of-the-art results compared to the current best-performing local acronym resolution algorithms and models.

## introduction in the past decade, natural language processing (nlp) has greatly advanced in the biomedical domain. given the troves of biomedical texts, nlp has emerged as a critical tool for knowledge extraction. nlp has been used to automatically analyze clinical notes, electronic medical records, biological literature, and other biomedical texts in the hopes of unearthing new knowledge and deeper insights. acronyms are especially common in science and even more so in biomedical publications, as authors regularly seek to shorten the long names for diseases, bacteria, and chemicals. @xcite documented acronym use in more than 24 million scientific article titles and 18 million scientific articles published between 1950 and 2019. they report that 19% of titles and 73% of abstracts contain acronyms. of the more than one million unique acronyms in their data, 0.2% appeared regularly and most acronyms, 79%, appeared less than 10 times. acronym resolution (ar) can be performed by either leveraging acronym definitions found in the text (referred to as local ar) or by consulting external resources, such as ontologies (known as disambiguation or global ar). while a lot of progress has been recently done on the latter, local ar has seen surprisingly little recent work. in particular, the sota approaches in local ar are rulebased or simple machine learning approaches from more than a decade ago. as a result, this task has not benefited from recent advances in transformers @xcite . to address this issue, in this work we focus on local ar where we try to answer the question: can transformers be leveraged to further improve traditional local ar approaches? to answer this question, we present biomedical local acronym resolver (blar); a transformerbased model designed to resolve local acronyms in biomedical texts. in particular, this work makes the following contributions: ## background and related work there are a few challenges inherent in acronym resolution that make a simple dictionary-lookup and our approach is applicable to both "local ar" and "acronym dictionary compilation." our approach specifically targets local acronym resolution and acronym dictionary compilation. local acronyms appear as a pair of entities featuring a short-form (sf) entity and a corresponding longform (lf) entity. historically, local acronym resolution has been handled by @xcite @xcite and @xcite demonstrated the best performance of local acronym resolution. they used a combination of hand-crafted filters to identify sf-lf pairs. @xcite introduced the first local acronym resolution model that leveraged machine learning. it produced sota results with the help of four sets of hand-crafted features, including rule-based text filters. @xcite further improved upon local acronym resolution by introducing a hybrid machine learning and rulebase model that does not rely on labeled data. they extract potential sf-lf pairs from pubmed articles using rules similar to the rules developed by sohn et al. and train a classifier to identify sf-lf pairs. our approach to local acronym resolution is simple in its architecture yet novel in its application. our two-stage model leverages transfer learning from modern, sota pretrained transformers and is able to learn the features of short-form and longform acronym pairs without the help of a predefined dictionary, hand-crafted features, filters, or rules. our model processes batches of documents, such as abstracts from pubmed, and creates an acronym dictionary specific to each inputted document. ## method the intuition behind local acronym resolution is that authors of scientific publications commonly define the acronyms that they employ later on in the document. this is typically done by defining acronyms within the text in the form of pairs of short-form (sf) and corresponding long-form (lf) entities. we can then use the identified sf-lf acronym pairs to either resolve the acronyms appearing in the input document or populate an sf-lf dictionary that can be used to accurately resolve future uses of the sf versions of the acronyms in the remainder of the text. showing the various tagged entities of a short and long-form acronym pair. we use a bilou (beginning, inside, last, outside, unit) tagging scheme @xcite to identify long-form (lf) entities, short-form (sf) entities, and parenthesis (pr) enclosing a paired sf or lf entity. identifying the definitions of sf-lf pairs poses two major challenges: first, one has to identify the location in the text where the definition of an sf-lf pair is provided. second, one has to identify the exact span (i.e., text) of both the short and long-form within the definition. two-step ar: following the above structure, blar splits the problem into two separate subtasks: ## results and discussion to measure blar's performance, we first compare it against sota local ar approaches. as explained in the background and previous work section, to the best of our knowledge, local acronym resolution has not seen significant advances since 2009. more recent acronym resolution works have focused instead on disambiguation acronym resolution, still relying on simpler rule-based algorithms for local acronym resolution @xcite . as a result, we compare blar to @xcite , @xcite , and @xcite , which represent the sota in local acronym resolution. ## conclusion and future work local acronym resolution has seen limited progress in recent years and has not benefited from the recent advancements in machine learning approaches. to address this problem, we develop blar; a deeplearning model that leverages a two-step architecture on top of pre-trained language models to identify sf-lf pairs in input documents. our experimental results show that blar outperforms other local acronym resolution approaches and achieves state-of-the-art performance. we release blar and its source code for public use. as part of our future work, we will be exploring two threads: first, further improving the model based on our error analysis, and second, exploring how blar (which in this case has been fine-tuned for the scientific domain) can be extended to cover acronyms found in other domains. we believe future work could also focus on a hybrid model that leverages both deep-learning and rule-based algorithms.