Disfluency Detection for V ietnamese (2022)

Abstract: In this paper, we present the first empirical study for Vietnamese disfluency detection. To conduct this study, we first create a disfluency detection dataset for Vietnamese, with manual annotations over two disfluency types. We then empirically perform experiments using strong baseline models, and find that: automatic Vietnamese word segmentation improves the disfluency detection performances of the baselines, and the highest performance results are obtained by fine-tuning pre-trained language models in which the monolingual model PhoBERT for Vietnamese does better than the multilingual model XLM-R.

## introduction humans do not always exactly predetermine what they intend to say, hence leading to interruptions in natural conversations. this phenomena is informally referred to as disfluency @xcite . disfluencies are highly ubiquitous in human conversations. with the increasing popularity of task-oriented dialogue systems, it is essential to improve the capacity of the systems in dealing with many kinds of distractor sources. note that a vast majority of spoken language understanding (slu) models used in the dialogue systems are trained on well-formed input text without disfluencies. however, there is a significant mismatch between the fluent training corpora and the real-world inputs of disfluent utterances/speech transcripts for those models, resulting in serious performance degradation in practical applications. hence, disfluency detection that identifies (and then removes) disfluencies to produce fluent versions of the disfluent inputs is a crucial component of real-world slu/dialogue systems. almost all benchmark datasets for the disfluency detection task, such as switchboard @xcite , callhome @xcite and child @xcite , are ex-clusively for english. therefore, the development of disfluency detection systems has been largely limited to the english language. from a societal, linguistic, machine learning, cultural and normative, and cognitive perspective @xcite , it is worth investigating the disfluency detection task for languages other than english, e.g. vietnamese. in particular, it is interesting to study whether the difference in linguistic characteristics might add difficulties to developing disfluency detection systems to non-english languages, e.g. investigating the influence of vietnamese word segmentation @xcite on the vietnamese disfluency detection task. despite being the 17th most spoken language in the world @xcite with about 100m speakers, to our best knowledge, there is no previous study as well as no public dataset available for disfluency detection in vietnamese. we fill the gap in the literature by conducting the first empirical study for vietnamese disfluency detection. to conduct this study, we first create a dataset for vietnamese disfluency detection through two manual phases, including: (i) adding contextual disfluencies into an existing fluent dataset of 5871 utterances @xcite , and (ii) annotating the added disfluencies with two different disfluency types. on our dataset, we then formulate the vietnamese disfluency detection task as a sequence labeling problem and empirically investigate strong baselines, including bilstm-cnn-crf @xcite and pre-trained language models xlm-r @xcite and @xcite . we find that: (i) automatic vietnamese word segmentation helps improve disfluency detection performances, and (ii) the highest performance results are obtained by fine-tuning the pre-trained language models, in which the monolingual model phobert outperforms the multilingual model xlm-r. we publicly release our dataset at: https://github. com/vinairesearch/phodisfluency . ## related work among disfluency detection datasets with manual annotations for english @xcite @xcite @xcite , the switchboard dataset @xcite is the most commonly used benchmark for developing and evaluating disfluency detection models. the disfluency detection models generally fall into three main categories of approaches based on noisy channel, parsing and sequence tagging. noisy channel-based disfluency detection models use tree adjoining grammar-based channel models to assign high probabilities to exact copy reparandum words @xcite , and also use language model scores as features to a maxent reranker @xcite . parsingbased models detect disfluencies and the syntactic structure of the sentence utterance simultaneously @xcite @xcite ; however, these models require large annotated training datasets that contain both disfluencies and syntactic structures. sequence tagging approaches formulate the disfluency detection task as a sequence labeling problem to label individual words by disfluency types or simply fluent/disfluent tags @xcite @xcite @xcite . among the disfluency detection approaches, the sequence tagging ones that fine-tune pre-trained language models @xcite produce the state-ofthe-art performances @xcite . ## our dataset our approach to creating a disfluency detection dataset for vietnamese is first to manually add contextual disfluencies as distractors into an existing fluent dataset. this first phase is inspired by @xcite who present a disfluent derivative of the question answering dataset squad @xcite . we choose phoatis consisting of 5871 utterance transcripts @xcite as our base fluent vietnamese dataset. after adding disfluencies to phoatis, we manually annotate disfluent words using disfluency types. ## conclusion in this paper, we have presented the first study for vietnamese disfluency detection. we create a vietnamese disfluency detection and empirically conduct experiments on this dataset to compare strong baseline models as well as perform detailed error analysis. experimental results show that the input representations and the pre-trained language models have positive influences on this vietnamese disfluency detection task.