Lexical Normalization for Code-switched Data and its Effect on POS Tagging (2021)

Abstract: Lexical normalization, the translation of non-canonical data to standard language, has shown to improve the performance of many natural language processing tasks on social media. Yet, using multiple languages in one utterance, also called code-switching (CS), is frequently overlooked by these normalization systems, despite its common use in social media. In this paper, we propose three normalization models specifically designed to handle code-switched data which we evaluate for two language pairs: Indonesian-English and Turkish-German. For the latter, we introduce novel normalization layers and their corresponding language ID and POS tags for the dataset, and evaluate the downstream effect of normalization on POS tagging. Results show that our CS-tailored normalization models significantly outperform monolingual ones, and lead to 5.4% relative performance increase for POS tagging as compared to unnormalized input.

## introduction social media provide an invaluable source of information for natural language processing (nlp) systems. its informative and spontaneous nature leads to many interesting phenomena, like nonstandard words, spelling errors and abbreviations. one particularly challenging and interesting phenomenon is the use of multiple languages within the same utterance, which is also called codeswitching (cs) @xcite @xcite . because most nlp models are designed to process canonical and monolingual data, their performance drops enormously when having to process social media data @xcite . one solution to this problem is lexical normalization: the translation of non-standard (e.g. social media) text to its canonical form @xcite . previous work has shown that by standardizing the data, we can improve the robustness of nlp systems @xcite @xcite . nevertheless these systems overlook code-switching. (1) shows a code-switched tweet (upper) and its normalization annotation (lower), taken from an indonesian-english cs corpus @xcite (indonesian in bold). this example demonstrates that cs complicates normalization, because it can be unclear in which language to normalize (e.g., ak is normalized to aku 'i' in indonesian. english-only normalization systems would probably normalize it to ok). recently, there has been an increasing interest in the automatic processing of cs data, however, there has not been much work on its lexical normalization. to the best of our knowledge, only @xcite focus entirely on lexical normalization for cs data in their work. for other works, normalization is a preprocessing step for downstream tasks: chunking @xcite , parsing @xcite , or machine translation @xcite . these cs normalizers are either rule-based and language-specific @xcite or combine (hindi) back-transliteration and normalization @xcite @xcite ) thus, they are not directly applicable to other lexical normalization datasets. in this work, the current state-of-the-art model for most languages is monoise (van der goot, 2019), which is based on this two-step approach. a variety of modules are used for the generation of candidates. for the ranking, monoise complements features from the generation step with additional features, which are all combined in a random forest classifier that predicts the probability that a candidate is a 'correct' candidate. monoise is described in more detail in section 4.2. more recently, sequence-tosequence models @xcite and contextual embeddings @xcite have been used for the lexical normalization task. these approaches have been shown to reach performances close to monoise on english benchmarks. like most nlp tasks, most research on normalization has been done on english datasets @xcite . however, there has been some efforts on other languages, where usually only one language is considered, we refer to @xcite and @xcite for an overview of available resources. ## data in this section we first describe the design decisions of the novel turkish-german dataset, then we compare some basic statistics together with the existing indonesian-english dataset @xcite . ## models in this section we describe the models used for word-level language identification (4.1), lexical normalization (4.2) and pos tagging (4.3). ## evaluation in this section we evaluate each of the three subtasks (lid, normalization, pos), where for the latter two we also examine the effect of exploiting the prediction of the previous tasks. unless mentioned otherwise, we report the results of 10-fold crossvalidation on the training split of the data. for all experiments, we use a paired bootstrap test on the sentence level with 1,000 samples to test significance. for all results, we order the models by the complexity of the implementation as compared to monoise (first fragments, as the original model can be used as a black box, then multilingual because it does not need a language classifier, and finally the language-aware model). an * next to results denotes a significant difference for @xmath0.05, of a model always as compared to the previous model (corresponding to the previous column in ## conclusion code-switching provides many challenges for nlp systems. in this work we attempt to overcome some of these challenges by normalizing the data, and evaluating the downstream effect of this for pos tagging. for evaluation we use an indonesian-english dataset @xcite as well as a german-turkish dataset (c ¸etinoglu, 2016), for which we provided novel normalization layers and adapted existing lid and pos annotation. we proposed three different models to normalize cs data. the two best-performing models are language-aware and multilingual. the first model exploits language labels, to identify for which language to generate features, whereas the second model combines features for both languages. the differences in performance between these two systems was not significant for any of the 10-fold experiments nor on the test data, so in most cases the multilingual model would be preferable, as it has no dependence on language labels. we showed that normalizing the input before pos tagging results in significantly higher pos accuracies for cs data. gold normalization experiments showed that there is still room for improvement for normalization models to help pos tagging. an interesting property of the proposed model is that it does not have to be trained on intrasentential cs data. in fact, it can be trained on a mix of two monolingual datasets, thereby handling many more language pairs. we hope to evaluate this setup if resources (i.e., normalization test data for a cs language pair, and monolingual normalization training data for both languages) become available.