V i L ex N orm: A Lexical Normalization Corpus for V ietnamese Social Media Text (2024)

Abstract: Lexical normalization, a fundamental task in Natural Language Processing (NLP), involves the transformation of words into their canonical forms. This process has been proven to benefit various downstream NLP tasks greatly. In this work, we introduce Vietnamese Lexical Normalization (ViLexNorm), the first-ever corpus developed for the Vietnamese lexical normalization task. The corpus comprises over 10,000 pairs of sentences meticulously annotated by human annotators, sourced from public comments on Vietnam’s most popular social media platforms. Various methods were used to evaluate our corpus, and the best-performing system achieved a result of 57.74% using the Error Reduction Rate (ERR) metric (van der Goot, 2019a) with the Leave-As-Is (LAI) baseline. For extrinsic evaluation, employing the model trained on ViLexNorm demonstrates the positive impact of the Vietnamese lexical normalization task on other NLP tasks. Our corpus is publicly available exclusively for research purposes.

## introduction in 2022, there were more than 72 million users of social networks in vietnam, accounting for approximately 73.7% of the total population 2 . the rapid growth of social media has resulted in a significant increase in the volume of data exchanged over the internet. however, because the data is spontaneous, it naturally contains a wide range of linguistic variances, both intended (e.g., slang, leet speak, puns) and unintended (e.g., mistakes). this presents significant challenges for natural language processing software (e.g., @xcite , which is primarily aimed at analyzing canonical text. one possible approach to enhance the performance of these systems is to normalize the text, thereby increasing its resemblance to the data that nlp systems were originally developed and trained. this task is also called lexical normalization; see figure 1 for the normalization of "c h ế t trong tôi, một ty chưa nóiii" (english: dying within me, a love yet unspoken). in this paper, we define our task of lexical normalization by @xcite , expressed by the following formulation: ## related work the landscape of lexical normalization research has witnessed significant growth and diversification across various languages over the past decade. this section provides an overview of the foundational work in english and extends to include developments in languages other than english, highlighting the emergence of corpora, advancements in normalization systems, and the downstream impact of lexical normalization on diverse nlp tasks. since the foundational work of @xcite with lexnorm1.1 a decade ago, lexical normalization has sparked interest in english and several other languages. in the realm of english, the task was followed by subsequent corpora such as lexnorm1.2 @xcite and lexnorm2015 @xcite . moving to languages other than english, several corpora were established. croatian saw the creation of reldi-normtagner-hr 2.0 @xcite , while serbian had reldi-normtagner-sr 2.0 @xcite . slovenian, too, had its representation with janes-tag 2.0 @xcite . danish was addressed by the development of dan+ @xcite . italian also had a dataset introduced by @xcite . shifting the focus to asian languages, higashiyama et al. (2021) introduced a notable corpus for japanese. additionally, barik et al. (2019) presented a corpus for code-mixed indonesian-english, and makhija et al. (2020) developed hinglishnorm for code-mixed hindi-english. remarkably, a shared task on multilingual lexical normalization (multilexnorm by van der goot et al., 2021) has provided a benchmark including 12 language variants. alongside the establishment of corpora, advancements in normalization systems, as exemplified by monoise by @xcite , have showcased promising outcomes. furthermore, lexical normalization has been demonstrated to boost various downstream nlp tasks, such as named entity recognition @xcite , pos tagging (zupan et al., 2019), dependency and constituency parsing (van der goot et al., 2020), sentiment analysis @xcite , and machine translation @xcite . however, the studies have yet to be applied to vietnamese. research efforts have primarily focused on the detection and correction of vietnamese spelling errors (e.g., @xcite @xcite , which are mostly unintended. to the best of our knowledge, vilexnorm stands as the first work to examine both advertent and inadvertent variations in spelling, encompassing all classifications outlined by @xcite except phrasal abbreviations. ## corpus creation in this section, we illustrate our corpus creation. the overview process is depicted in figure 2 . ## intrinsic evaluation this section focuses on the intrinsic evaluation of vilexnorm, examining its empirical performance through diverse experiments and methodologies. we explore methods ranging from pretransformer structures to transformer-based structures in the lexical normalization task. subsequently, we outline the experimental setup, including data configurations, training procedures, and metrics. finally, we present evaluation results, analyzing each method's performance and offering insights into the efficiency and effectiveness of vilexnorm. ## extrinsic evaluation this section extends the assessment of vilexnorm beyond intrinsic measures, exploring its impact on downstream nlp tasks. through experiments, we investigate how the normalization system enhances performance in emotion recognition, hate speech detection, and spam detection. we also assess its efficacy in scenarios without vietnamese diacritics, providing insights into its adaptability and real-world effectiveness. ## conclusion and future work our paper introduced vilexnorm, a novel corpus expressly designed for the lexical normalization task of vietnamese social media data. the corpus analysis demonstrated captivating characteristics of the vietnamese language used on social media. we conducted empirical evaluations employing various methods on this corpus, and the bartpho syllable model emerged as the top performer, achieving an impressive err score of 57.74% and a precision score of 93.32%. additionally, we harnessed the potential of vilexnorm to assess the impact of lexical normalization on downstream nlp tasks, and the results were encouraging. as the pioneering effort in the lexical normalization task for vietnamese, we hope that our corpus contributes to the diversity of the multilingual lexical normalization task. furthermore, we expect this work to motivate and inspire further exploration and research in handling noisy data on the internet, advancing the field of lexical normalization in vietnamese nlp research. promising avenues for advancement in this task are considered for our future research. our roadmap includes not only expanding the corpus in both scale and diversity but also incorporating a variety of vietnamese language variants found across the internet (e.g., text lacking diacritic marks). additionally, we intend to conduct a thorough analysis of agreement, exploring metrics like cohen's kappa score @xcite , to gain deeper insights into the quality and consistency of the corpus. moreover, we are inclined towards a comprehensive exploration and adaptation of state-of-the-art models and methods, including monoise (van der goot, 2019a), with the goal of identifying optimal solutions for the lexical normalization task and ad-vancing the development of highly effective multilingual lexical normalization models that can effectively bridge language-specific gaps. another important aspect of our future work involves expanding the scope of extrinsic evaluations to encompass a broader range of nlp tasks, including dependency parsing and pos tagging @xcite . these tasks require label adjustments during normalization due to the monosyllabic nature of vietnamese, necessitating the investigation of adaptive methods for monosyllabic languages and contributing to a more diverse language landscape in practical language processing scenarios.