Multi-Dialect V ietnamese: Task, Dataset, Baseline Models and Challenges (2024)

Abstract: Vietnamese, a low-resource language, is typically categorized into three primary dialect groups that belong to Northern, Central, and Southern Vietnam. However, each province within these regions exhibits its own distinct pronunciation variations. Despite the existence of various speech recognition datasets, none of them has provided a fine-grained classification of the 63 dialects specific to individual provinces of Vietnam. To address this gap, we introduce Vietnamese Multi-Dialect (ViMD) dataset, a novel comprehensive dataset capturing the rich diversity of 63 provincial dialects spoken across Vietnam. Our dataset comprises 102.56 hours of audio, consisting of approximately 19,000 utterances, and the associated transcripts contain over 1.2 million words. To provide benchmarks and simultaneously demonstrate the challenges of our dataset, we fine-tune state-of-the-art pre-trained models for two downstream tasks: (1) Dialect identification and (2) Speech recognition. The empirical results suggest two implications including the influence of geographical factors on dialects, and the constraints of current approaches in speech recognition tasks involving multi-dialect speech data. Our dataset is available for research purposes.

## introduction the vietnamese language can be divided into three major dialects (regional dialects): northern, central, and southern dialect @xcite @xcite , each associated with distinct region and characterized by unique phonetic characteristic @xcite @xcite . however, even within these regions, the dialects unique to each province (provincial dialect) maintain noticeable differences @xcite . therefore, to obtain a more detailed insight into the vietnamese dialects, we need to consider a more granular level, particularly provincial dialects, rather than limiting our examination to regional dialects alone. it is worth noting that vietnamese is a monosyllabic language in which each word is a single syllable and words are separated by spaces. moreover, in the vietnamese context, the terms 'accent' and 'dialect' are used interchangeably @xcite and consequently, for the sake of consistency, this research adopts the term 'dialect'. in recent years, vietnamese speech-related research has made remarkable strides; however, the issue of multi-dialectal variations within the language has posed a significant challenge @xcite @xcite . in an effort to tackle this obstacle, several multi-dialect vietnamese corpora have been published @xcite @xcite . however, these datasets exhibit two limitations: (1) all of the mentioned datasets encompass only three to five groups of dialects, (2) several datasets are not publicly available. motivated by these two limitations, we introduce the vimd speech dataset, a novel resource encompassing 63 provincial dialects, representing all 63 provinces of vietnam. the audio data is curated from publicly available sources. the transcripts undergo a semi-automatic labeling process, followed by a rigorous manual verification to guarantee the quality of dataset. furthermore, each record includes additional attributes such as speaker identification codes and gender, allowing the dataset to support various speech-related tasks. the contributions of our study are as follows: ## basic phonetic structure of vietnamese vietnamese, as a monosyllabic tonal language, is structured with three key components: initial, final, and tone. in particular, the final segment is composed of three elements: onset, nucleus, and coda. these components are detailed in the two mandatory elements to construct a syllable, highlighted in bold, are tone and nucleus. for instance, the word 'bạn' (friend) exemplifies all three key components, where 'b' is the initial, 'an' is the final, and the high-broken tone is represented by the diacritic below the nucleus 'a'. on the other hand, the word 'u' (lump) has only the two mandatory elements, with 'u' as the nucleus and the mid tone. there are six tones in vietnamese: mid tone, high-rising, low-falling, low-rising, high-broken, and low-broken. each tone, when combined with a syllable, conveys a distinct meaning. for instance, considering a word with the initial consonant 'b' and the nucleus 'a': mid tone (bathree), high-rising (bá -uncle), low-falling (bà -grandmother), low-rising (bả -poison), high-broken (bã -waste), and low-broken (bạ -randomly). the differences in pitch contour between the tones are represented in ## related work numerous speech corpora spanning diverse linguistic and dialectical backgrounds have been curated to facilitate dialect classification and speech recognition tasks. ## experiments and results we conduct experiments on our dataset through two tasks encompassing dialect identification and speech recognition. the process is presented through the following sections: experimental design, baselines evaluation metrics, data preprocessing, and results. ## dicussion our analysis of the results yields two notable findings encompassing (1) geographical influences on dialects and (2) multi-dialect data challenges for speech recognition approach. the following are our conjectures based on experimental results. the cause could also stem from the training data for pre-trained models or other factors. ## conclusion we introduce vimd, a novel dataset covering all 63 provincial dialects in vietnam. we carry out experiments on the two tasks of dialect identification and speech recognition, employing various stateof-the-art models to establish baseline benchmarks. the results facilitate a more in-depth investigation of dialects, including the impact of geographical factors on dialectal variations, and pose challenges for speech recognition models in tackling the multidialect aspect of the vietnamese language. in addition, we hope that our process will help expand both the scale and quality of the other datasets. we also expect that this dataset will facilitate future research aimed at enhancing the performance of di and sr tasks, as well as other related speech tasks, especially for the vietnamese language.