Learning to Recognize Dialect Features (2021)

Abstract: Building NLP systems that serve everyone requires accounting for dialect differences. But dialects are not monolithic entities: rather, distinctions between and within dialects are captured by the presence, absence, and frequency of dozens of dialect features in speech and text, such as the deletion of the copula in “He ∅ running”. In this paper, we introduce the task of dialect feature detection, and present two multitask learning approaches, both based on pretrained transformers. For most dialects, large-scale annotated corpora for these features are unavailable, making it difficult to train recognizers. We train our models on a small number of minimal pairs, building on how linguists typically define dialect features. Evaluation on a test set of 22 dialect features of Indian English demonstrates that these models learn to recognize many features with high accuracy, and that a few minimal pairs can be as effective for training as thousands of labeled examples. We also demonstrate the downstream applicability of dialect feature detection both as a measure of dialect density and as a dialect classifier.

## introduction dialect variation is a pervasive property of language, which must be accounted for if we are to build robust natural language processing (nlp) systems that serve everyone. linguists do not characterize dialects as simple categories, but rather as collections of correlated features @xcite , such as the one shown in figure 1 ; speakers of any given dialect vary regarding which features they employ, how frequently, and in which contexts. in comparison to approaches that classify speakers or documents across dialects (typically using metadata such as geolocation), the feature-based perspective has several advantages: (1) allowing for fine-grained comparisons of speakers or documents ## data and features of indian english we develop methods for detecting 22 dialect features associated with indian english. although india has over 125 million english speakers -making it the world's second largest english-speaking population -there is relatively little nlp research focused on indian english. our methods are not designed exclusively for specific properties of indian english; many of the features that are associated with indian english are also present in other dialects of english. we use two sources of data in our study: an annotated corpus ( § 2.1) and a dataset of minimal pairs ( § 2.2). for evaluation, we use corpus annotations exclusively. the features are described in ## models and training we train models to recognize dialect features by fine-tuning the bert-base uncased transformer architecture @xcite . we consider two strategies for constructing training data, and two architectures for learning across multiple features. ## results on dialect feature detection in this section, we present results on the detection of individual dialect features. using the features shown in ## measuring dialect density a dialect density measure (ddm) is an aggregate over multiple dialect features that tracks the vernacularity of a passage of speech or text. such measures are frequently used in dialectology (van hofwegen and wolfram, 2010), and are also useful in research on education (e.g., @xcite . recently, a ddm was used to evaluate the performance of speech recognition systems by the density of aave features @xcite . the use of ddms reflects the reality that speakers construct individual styles drawing on linguistic repertoires such as dialects to varying degrees @xcite . this necessitates a more nuanced description for speakers and texts than a discrete dialect category. following prior work (e.g., van hofwegen and wolfram, 2010) we construct dialect density measures from feature detectors by counting the predicted number of features in each utterance, and dividing by the number of tokens. for the learningbased feature detectors (minimal pairs and corpus examples), we include partial counts from the detection probability; for the regular expression detectors, we simply count the number of matches and dividing by the number of tokens. in addition, we construct a ddm based on a document classifier: we train a classifier to distinguish indian english from u.s. english, and then use its predictive probability as the ddm. these ddms are then compared on two tasks: distinguishing indian and u.s. english, and correlation with the density of expert-annotated features. the classifier is trained by fine-tuning bert, using a prediction head on the [cls] token. ## conclusion we introduce the task of dialect feature detection and demonstrate that it is possible to construct dialect feature recognizers using only a small number of minimal pairs -in most cases, just five positive and negative examples per feature. this makes it possible to apply computational analysis to the many dialects for which labeled data does not exist. future work will extend this approach to multiple dialects, focusing on cases in which features are shared across two or more dialects. this lays the groundwork for the creation of dialectbased "checklists" @xcite to assess the performance of nlp systems across the diverse range of linguistic phenomena that may occur in any given language.