Planning and Generating Natural and Diverse Disfluent Texts as Augmentation for Disfluency Detection (2020)

Abstract: Existing approaches to disfluency detection heavily depend on human-annotated data. Numbers of data augmentation methods have been proposed to alleviate the dependence on labeled data. However, current augmentation approaches such as random insertion or repetition fail to resemble training corpus well and usually resulted in unnatural and limited types of disfluencies. In this work, we propose a simple Planner-Generator based disfluency generation model to generate natural and diverse disfluent texts as augmented data, where the Planner decides on where to insert disfluent segments and the Generator follows the prediction to generate corresponding disfluent segments. We further utilize this augmented data for pretraining and leverage it for the task of disfluency detection. Experiments demonstrated that our two-stage disfluency generation model outperforms existing baselines; those disfluent sentences generated significantly aided the task of disfluency detection and led to state-of-the-art performance on Switchboard corpus.

## introduction disfluency is a para-linguistic concept defining the interruption to the flow of speech @xcite . as shown in figure 1 , a standard annotation of the disfluency structure indicates the reparandum (the region to repair), an optional interregnum (filled pauses, discourse cue words, etc.) and the associated repair (corrected linguistic materials) @xcite . disfluency @xcite mainly deals with identifying and removing reparandum foot_0 , i want a flight [ to boston + {um} to denver ] reparandum interregnum repair type example repetition they they learn to share. deletion this is just happened yesterday. substitution it's nothing but wood up here down here. reparandum in disfluency can be categorized as repetition, deletion and substitution @xcite , as shown in neural models have achieved reasonable performance in disfluency detection on english switchboard (swbd) corpus @xcite . such models involve applying pretrained models to conduct disfluency detection as sequence tagging or seq2seq tasks @xcite . recently, data augmentation techniques are also used to generate augmented disfluent sentences for model pretraining. those models are limited in that the augmented data is generated based on simple heuristics such as random repetition or insertion of ngrams @xcite . sentences generated methods example random repetition (1) that 's that 's really good. random insertion (2) of a that 's really good. generation (ours) (3) it 's that 's really good. generation (ours) (4) that would be more be worth doing. to address this gap, we propose a generation based data augmentation method to generate natural and diverse disfluent sentences to further improve the performance of disfluency detection. this method is similar to back-translation @xcite , which has been shown effective as a way of data augmentation in machine translation. different from classical neural generation models, our generation model is in a two-stage generation manner, motivated by coarse-to-fine decoding @xcite . specifically, given a fluent sentence as the input, in the first stage, a planner selects the positions of where to insert reparandum; in the second stage, a generator generates disfluent segments accordingly for the predicted areas. compared to generic end-to-end generation, our two-stage model separates the generation task into two steps: where to generate and what to generate. such breakdown enables the model to only generate disfluent segments rather than the whole disfluent sentences, and to carry naturally labeled data as augmentation for disfluency detection. as shown in to sum up, our contributions are as follows: disfluency generation betz et al. (2015) and adell et al. (2006) used heuristic rules to generate filled pauses, repetitions in disfluent speech generation. their works demonstrate that disfluency generation enhances the naturalness and intelligibility of speech generated by text-to-speech (tts) systems. wang et al. (2018) and wang et al. (2019) randomly inserted or repeated ngrams to generate augmented disfluent sentences. disfluent sentences generated in this method have low affinity to disfluent sentences from the benchmark dataset, and they contain few substitutions, causing limited diversity. to achieve better affinity and diversity, we design generation based data augmentation which generates natural disfluent sentences that can then be directly used to train the disfluency detection model. we adapt multi-stage coarse-to-fine neural decoders (dong and lapata, 2018) for generation tasks to design our disfluency generation model. disfluency detection disfluency detection models mainly fall into four categories. the first one utilizes noisy channel models (zwarts and johnson, 2011; lou and johnson, 2018), which require tree adjoining grammar (tag) based transducer in the channel model. the second category leverages phrase structure, which is often related to transitionbased parsing yet requires annotated syntactic structure @xcite @xcite . the third category frames the task as a sequence tagging task @xcite @xcite @xcite , and the last one employs end-to-end encoder-decoder models @xcite to detect disfluent segments automatically. traditional disfluency detection models often required additional features (e.g. pos tags) @xcite , syntactic annotations or external tools (e.g. tag based transducer) (lou and johnson, 2018) for learning. recent disfluency detection approaches leveraged neural representations and obtained comparable results. for instance, lou et al. (2018) adopted cnn and introduced the auto-correlation operator which models more accurate word relations and similarities in order to capture "rough copies". however, most of them still heavily depend on human-annotated data. as a result, different kinds of data augmentation approaches and pretraining have been designed to alleviate such dependence. for example, bach and huang (2019) incorporated elmo (peters et al., 2018) to sequence tagging model and dong et al. (2019) used a pretrained denoising autoencoder to initialize the encoder-decoder model. wang et al. (2019) achieved state-of-the-art performance by using data augmentation and bert (devlin et al., 2018) in a sequence tagging task, and wang et al. (2018) obtained similar performance by using the same data augmentation methods in an encoder-decoder fashion. these aforementioned data-augmentation methods created augmented disfluent sentences only by randomly inserting or repeating ngrams. to this end, we introduce generation based data augmentation to first generate disfluencies and then use them for sequence tagging of disfluency detection. note that there was a similar trend in grammatical error detection. felice and yuan (2014); kasewa et al. (2018) generated sentences with grammatical errors to augment the training data of grammatical error detection. ## conclusion this work presents a simple two-stage disfluency generation model to generate natural and diverse disfluent texts. we further used them as augmented data for pretraining and aiding the task of disfluency detection. experiments demonstrate that our proposed disfluency generation model outperformed existing baselines; those disfluent sentences generated significantly aided the task of disfluency detection and led to state-of-the-art performances.