ABB - BERT : A BERT model for disambiguating abbreviations and contractions (2021)

Abstract: Abbreviations and contractions are commonly found in text across different domains. For example, doctors’ notes contain many contractions that can be personalized based on their choices. Existing spelling correction models are not suitable to handle expansions because of many reductions of characters in words. In this work, we propose ABB-BERT, a BERT-based model, which deals with an ambiguous language containing abbreviations and contractions. ABB-BERT can rank them from thousands of options and is designed for scale. It is trained on Wikipedia text, and the algorithm allows it to be fine-tuned with little compute to get better performance for a domain or person. We are publicly releasing the training dataset for abbreviations and contractions derived from Wikipedia.

## introduction we use abbreviations and contractions (called "short forms" in this paper) while quickly typing on digital apps. they are used to save time or effort in typing and may be unique to us; therefore, sometimes, only we can understand them. there is no reliable dictionary of short forms to be referred because it can be specific to a context or a person. the short forms often have multiple meanings depending on the domain or the person. in ## abb-bert the goal of the algorithm is to rank the options for short forms. as shown in 1, the input sentence x may contain one or more short forms. we assume that short forms' location in the sentence and the character composition is known for training