CL - M o N oise: Cross-lingual Lexical Normalization (2021)

Abstract: Social media is notoriously difficult to process for existing natural language processing tools, because of spelling errors, non-standard words, shortenings, non-standard capitalization and punctuation. One method to circumvent these issues is to normalize input data before processing. Most previous work has focused on only one language, which is mostly English. In this paper, we are the first to propose a model for cross-lingual normalization, with which we participate in the WNUT 2021 shared task. To this end, we use MoNoise as a starting point, and make a simple adaptation for cross-lingual application. Our proposed model outperforms the leave-as-is baseline provided by the organizers which copies the input. Furthermore, we explore a completely different model which converts the task to a sequence labeling task. Performance of this second system is low, as it does not take capitalization into account in our implementation.

## introduction lexical normalization is the task of converting non-canonical text to its canonical equivalent on the word level. as common for natural language processing (nlp) tasks, most of the previous work is done on english data @xcite . however, for lexical normalization there have also been many attempts for other language-(pair)s @xcite @xcite @xcite @xcite @xcite @xcite , which have been combined into one benchmark for the wnut 2021 shared task @xcite . even though data has been available for multiple languages, most work focused on one language, and to the best of our knowledge no one has attempted to solve this task cross-lingually. if successful, a cross-lingual lexical normalization model would open up possibilities for lexical normalization for languages in which no training data is available. in this work, we use the monoise model (van der goot, 2019a) as a starting point, the only normalization model that is open source and has models available for the languages we target. furthermore, it is heavily dependent on raw data to generate candidates and features, which makes it relatively easy to adapt it for a cross-lingual setup (section 2). we refer to our new model cl-monoise. in addition to our cross-lingual model, we also evaluate an out-of-the-box sequence labeler, we use the string2string task-type of machamp (van der goot et al., 2021b), which was originally created for the purpose of lemmatization.