A Parallel Corpus for V ietnamese Central-Northern Dialect Text Transfer (2023)

Abstract: The Vietnamese language embodies dialectal variants closely attached to the nation’s three macro-regions: the Northern, Central and Southern regions. As the northern dialect forms the basis of the standard language, it’s considered the prestige dialect. While the northern dialect differs from the remaining two in certain aspects, it almost shares an identical lexicon with the southern dialect, making the textual attributes nearly interchangeable. In contrast, the central dialect possesses a number of unique vocabularies and is less mutually intelligible to the standard dialect. Through preliminary experiments, we observe that current NLP models do not possess understandings of the Vietnamese central dialect text, which most likely originates from the lack of resources. To facilitate research on this domain, we introduce a new parallel corpus for Vietnamese central-northern dialect text transfer. Via exhaustive benchmarking, we discover monolingual language models’ superiority over their multilingual counterparts on the dialect transfer task. We further demonstrate that fine-tuned transfer models can seamlessly improve the performance of existing NLP systems on the central dialect domain with dedicated results in translation and text-image retrieval tasks.

## introduction owing to the rapid development of natural language processing in the past few years, research on vietnamese nlp have also benefited in terms of resources and modelling techniques. in particular, many task-specific vietnamese datasets and dedicated language models have been released to the community @xcite @xcite @xcite @xcite . these benchmarks and models, as they facilitate the research on vietnamese com-* corresponding author putational linguistics, have limitations in that they solely focus on the standard vietnamese text. figure 1 : industry-level translation systems respond differently regarding the central dialect geographically, vietnamese provinces are categorized into three macro-regions: the northern, central and southern regions -bringing forth different dialects @xcite . among these, the northern dialect is often treated as the standard i.e. the defacto text style of the language @xcite . compared to the northern, the southern dialect differs in terms of pronunciation but is mostly similar in terms of lexicon, making the two mutually intelligible @xcite . in contrast, the central dialect, besides the phonology deviation, possesses a significant number of vocabulary differences and is thus less mutually intelligible to the remaining two @xcite . in fact, the central dialect is often perceived as "peculiar" or even "difficult to understand" by speakers of other regions due to the existence of various unique words @xcite . with a population comprising of nearly one-third of the country, the central region, along with its dialect, remains an important and idiosyncratic part of the nation's culture @xcite @xcite . however, existing research works, despite massive advancement in recent years, have mainly focused on the standard dialect so far @xcite @xcite @xcite , neglecting other variants with potentially rich linguistic values. furthermore, state-of-the-art industry-level nlp systems, regardless of being production-ready, do not possess understandings of the non-standard dialect. an illustrative example is presented in figure 1 where a text utterance in the central dialect is input into several renowned translation systems including google translate foot_0 & yandex translate foot_1 . here the central-style utterance differs from the northern variety at two words răng and ngá, which corresponds to the words sao and ngứa in the northern counterpart. the word răng means why/how in the central dialect, but it also means teeth in the northern (standard) dialect. in contrast, the word ngá means itchy and is lexically unique to the central dialect. taken together, the input text can be translated as for some reasons i start to feel really itchy or simply oh i feel so itchy. we can observe that while the translation outputs of the northernstyle utterance are highly relevant, the predicted outputs with respect to the central-style utterance are inapposite. in particular, all systems seem to mistake the meaning of the word răng as teeth and do not properly grasp the meaning of the word ngá. despite a slight lexical difference between the two dialects, no system manages to correctly translate the central-style utterance, or even close, producing completely unconnected contents. this phenomenon can be considered a well-known bias in current nlp research, where developed models only function towards a particular, majority group of users, ignoring the needs of other minor communities e.g. translation for high-resource languages versus low-resource ones @xcite @xcite . in the case of vietnamese language, it manifested as the dialectal bias where the non-prestige (i.e. central) is not comprehended, even by well-tailored systems -a conceivable issue stemming from the lack of appropriate training data. inclined to provide a remedy to this dilemma, we introduce a new parallel corpus encompassing the central and northern dialect. constructed through manual effort under strict quality-control, the corpus is specifically designed for the dialect transfer task, with meaningful applications towards facilitating the development of more well-rounded nlp models that are not only potent towards the standard text but can also handle the central-style wordings. we extensively evaluate several monolingual and multilingual language models on their ability to shift the dialect of an input utterance, which requires deep linguistics understandings of the vietnamese language. in addition, we provide experiments and discussions on the corpus's applications beyond mere dialect transfer, that is, adapting prevailing nlp models to the central dialect domain without the needs to re-train. our contributions can be summarized as follows: in addition, we observe that all experimented generative models suffer from a fine-tuningpre-training mismatch. ## dataset construction in this section, we describe the procedure to construct a parallel corpus for vietnamese centralnorthern dialect text transfer. ## experiments & discussions experiment settings. we partitioned the dataset with 80%/10%/10% ratios to form the training/validation/test splits. for benchmarking the dialect transfer task, we fine-tuned a set of pre-trained generative language models: mbart (multilingual bart) @xcite , bartpho-syllable (vietnamese bart, operating on syllable-level data), bartpho-word (similar to the prior but uses wordlevel data), bartpho-syllable-base and bartphoword-base (the base variants) @xcite . during training, we used a batch size of 32 and a learning rate of 1e -6 along with the adamw @xcite optimizer with lin-ear decay schedulers. all models were fine-tuned for a maximum of 300 epochs with early stopping. all settings were implemented with the pytorch @xcite framework and the transformers @xcite library. for each model, the top-5 checkpoints with lowest validation losses were selected and evaluated on the test set. we used greedy decoding in all experiments unless explicitly mentioned otherwise. we also considered sbert-based @xcite retrieval baselines as lower bounds. in particular, we adopted two multilingual sbert models @xcite denoted as retrieval-m1 and retrieval-m2 along with a publicly available vietnamese sbert model hereafter abbreviated as retrieval-vi. in each direction, we first encoded the input utterance and retrieved the sample with the closest semantic distance 9 in the target dialect from the training set. for details on pre-trained checkpoints, please see appendix a. to evaluate the quality of predicted sequences, we adopted a set of automatic evaluation metrics: rouge @xcite , bleu @xcite and meteor @xcite . for the predictions of generative models, we also conducted human evaluations in which each participant was presented with the gold sequence and predicted outputs from 5 systems 10 , conditioned on 100 random test samples. predicted outputs were shuffled and the participants were not aware of the different models. each participant then picked out the sequence that he/she thought was the most suitable conversion. for each direction, we employed 3 participants with approriate backgrounds (e.g. centralorigin participants for northern-to-central transition and vice versa). upon voting, we further held a meeting to resolve conflicts among raters where a linguistics specialist also partook in. for the two directions, we obtained corresponding fleiss' kappa scores @xcite of 0.6447 and 0.6284 which implied substantial agreement. dialect transfer. we present the results for two transfer directions in correctness fluency style chatgpt (zero-shot) 5% 37% 54% chatgpt (few-shot) 9% 39% 58% bartpho (fine-tuned) 82% 86% 95% to study how anomalous the central dialect is on both the word-and sequence-level, we fine-tuned a set of auto-encoder language models on two tasks: central-style word extraction and dialect detection. the prior can be treated as a sequence labelling task whereas the latter resembles a sequence classification task. note that the word extraction task is non-trivial, as central-style words can have the same lexical appearance with certain northern-style words in which cases the models have to disambiguate them through context information. for example, the word răng means how/why in the central dialect but also denotes teeth in the northern dialect. in our experiments, we employed the multilingual xlm-roberta models @xcite and the monolingual phobert models @xcite (large and base variants) 19 . for every model, we picked the top-5 checkpoints with the best validation performance (accuracy or f1) and accordingly reported the mean and standard deviation. training hyperparameters (learning rate, batch size, etc) remain the same as in previous experiments. 19 phobert-base-v2 uses more pre-training data, but its architecture is the same as phobert-base at the sequence-level ( ## related works research on language varieties, specifically dialects, has been actively developed among many languages including those with latin @xcite @xcite @xcite @xcite @xcite and non-latin writing systems @xcite @xcite . the research areas mostly revolve around dialect classification @xcite and normalization @xcite , but also span related downstream tasks such as hate speech detection @xcite , sentiment analysis @xcite , part-ofspeech tagging @xcite and eyetracking @xcite . for the vietnamese language, dialect-related research remains bounded in traditional linguistics research @xcite @xcite , whereas computational linguistics mainly puts the focus on the standard dialect @xcite @xcite . amid the small number of works considering dialectal differences, only signal processing and speech-related tasks are explored @xcite @xcite , centering on the tonal (phonetic) deviations between dialects while the textual attributes and their effects on downstream tasks remain unexplored. in recent years, a plethora of vietnamese nlp datasets have been released to facilitate the development of downstream tasks including intent detection and slot filling @xcite , speech translation @xcite , machine translation @xcite , named entity recognition @xcite ) and text-to-sql @xcite . these corpora have the same limitations in that they do not take into account the (non-standard) central dialect and only target the standard text. while they accelerate the progress on a number of tasks, systems trained on these datasets potentially carry the same drawbacks in that they are not apt to confront the central-style wordings which might in turn cause the existing unfairness to become more severe. as growing needs for text style transfer emerge, the field has been actively receiving attention from the research communities @xcite . the settings vastly differ per situational basis, ranging from business use cases such as formality @xcite , politeness @xcite , authorship @xcite , simplicity @xcite to scenarios that aim at mitigating social issues such as toxicity @xcite , sarcasm @xcite , sentiment @xcite , biases @xcite . among them, many are developed with extended applications in mind i.e. facilitating the progress of other tasks such as paraphrasing @xcite , summarizing @xcite or producing style-specific translation @xcite . in our work, we focus on a more language-oriented setting, that is, the transfer between different dialects (i.e. the central and northern dialects), with meaningful pertinence towards more inclusive nlp models. ## conclusion in this paper, we tackle the dialect transfer problem for the vietnamese language. in particular, we introduce a new benchmark for vietnamese central-northern dialect text transfer. through immense experiments, we discover the deficiencies of multilingual models for the task compared to the monolingual counterparts. we also highlight the performance bias of existing nlp systems regarding the vietnamese central dialect. as a prospective remedy, we further demonstrate that the finetuned transfer modules can empower existing models to address the central-style wordings without the needs for re-training.