Extracting Lexical Features from Dialects via Interpretable Dialect Classifiers (2024)

Abstract: Identifying linguistic differences between dialects of a language often requires expert knowledge and meticulous human analysis. This is largely due to the complexity and nuance involved in studying various dialects. We present a novel approach to extract distinguishing lexical features of dialects by utilizing interpretable dialect classifiers, even in the absence of human experts. We explore both post-hoc and intrinsic approaches to interpretability, conduct experiments on Mandarin, Italian, and Low Saxon, and experimentally demonstrate that our method successfully identifies key language-specific lexical features that contribute to dialectal variations.

## introduction dialects and closely related languages exhibit subtle but significant variations, reflecting regional, social, and cultural differences @xcite . identifying and distinguishing differences between these dialects is of great importance in linguistics, language preservation, and natural language processing (nlp) research @xcite . traditionally, identifying the specific linguistic features that distinguish dialects of a language has relied on manual analysis and expert knowledge @xcite , as the differences between these dialects could be subtle and hard to detect without linguistic expertise @xcite . this process is also time-consuming and is usually language-specific due to the peculiarities that different languages exhibit. extracting distinctive words of particular dialects is essential for studying dialectal variation, especially in dialectology @xcite . in this work, we focus on extracting wordlevel distinguishing and salient features in dialects, 1 data and code are available: https://github.com/ ruoyuxie/interpretable_dialect_classifier also called 'shibboleths' @xcite . we propose an automated workflow that could potentially assist researchers such as dialectologists and corpus linguists in their analysis of dialectal variations. to achieve this, we leverage strong neural classifiers of dialects @xcite @xcite and pair them with model interpretability techniques to extract these features. we hypothesize that there are certain distinguishing features in dialects that the models learn during training, which enables them to make accurate predictions at test time. we utilize post-hoc @xcite and intrinsic @xcite ; arik and pfister, 2020) feature attribution explanation methods to extract these features from model interpretations, in the form of local explanations from dialect classifiers. our experiments focus on three language groups: mandarin, italian, low saxon, and their respective dialects. we demonstrate the effectiveness of our approach through automatic evaluation, human evaluation, and extensive analysis. we use known lexical features of some of the dialects we consider and show the viability of using explanation methods to extract unique dialectal features. ## background and related work the importance of interpreting model decisions has increasingly been recognized in recently @xcite @xcite , primarily due to its role in deciphering the inner workings of black-box models. two main approaches include: (i) post-hoc methods @xcite @xcite which provide insights into the predictions of pre-existing models based on model internals; (ii) intrinsic methods @xcite @xcite where interpretability is an integral feature that is optimized concurrently with the model's main primary task during training. this paper also closely relates to dialect classification @xcite @xcite . some highperforming neural dialect classifiers are able to @xmath0, 2023). most languages worldwide are under-resourced @xcite , and basic tasks like identifying verbs and nouns within a dialect can be challenging, given that syntactic parsing is primarily efficient for well-resourced languages @xcite . in this work, we focus on extracting lexical differences between dialects, as they show potential in distinguishing dialectal variations. the idea of automatically extracting linguistic features is not new @xcite . however, we identify these features through the lens of model explanations by using feature-attribution methods. to the best of our knowledge, this is the first study to undertake such an endeavor. ## method we present a simple yet effective method to extract lexical differences that distinguish dialects using explanations obtained from interpretable dialect classifiers as shown in figure 1 . ## experiments in this section, we present multiple experiments and demonstrate that our method results in distinguishing and meaningful feature extraction. for the main results, we focus on mandarin (cn-tw) and sicilian-italian (scn-it). a total of four annotators are involved in the evaluation process. foot_1 ## conclusion in this work, we introduce a novel approach for capturing language-unique lexical features from dialects through interpretable dialect classifiers. we utilize both post-hoc and intrinsic explanation methods and experiment on three language groups -mandarin, italian, and low saxon, and their respective dialects, conducting extensive evaluation and analysis to showcase the effectiveness of our method. in the future, we plan to broaden this approach to address additional linguistic aspect beyond the lexical level (appendix d.3). more broadly, our paper takes a first step to assess how interpretability techniques can be used to unearth lexical and, potentially, other linguistic features. by doing so, we hope to provide a framework that future studies can build upon.