Generating Fluent Translations from Disfluent Text Without Access to Fluent References: IIT B ombay@ IWSLT 2020 (2020)

Abstract: Machine translation systems perform reasonably well when the input is well-formed speech or text. Conversational speech is spontaneous and inherently consists of many disfluencies. Producing fluent translations of disfluent source text would typically require parallel disfluent to fluent training data. However, fluent translations of spontaneous speech are an additional resource that is tedious to obtain. This work describes the submission of IIT Bombay to the Conversational Speech Translation challenge at IWSLT 2020. We specifically tackle the problem of disfluency removal in disfluent-to-fluent text-to-text translation assuming no access to fluent references during training. Common patterns of disfluency are extracted from disfluent references and a noise induction model is used to simulate them starting from a clean monolingual corpus. This synthetically constructed dataset is then considered as a proxy for labeled data during training. We also make use of additional fluent text in the target language to help generate fluent translations. This work uses no fluent references during training and beats a baseline model by a margin of 4.21 and 3.11 BLEU points where the baseline uses disfluent and fluent references, respectively. Index Terms- disfluency removal, machine translation, noise induction, leveraging monolingual data, denoising for disfluency removal.

## introduction and related work spoken language translation often suffers due to the presence of disfluencies. in conversational speech, speakers often use disfluencies such as filler words, repetitions of fillers, repetitions of fluent phrases, false starts, and corrections which do not occur in the text. standard machine translation and spoken translation systems perform competitively when the input is well-formed text or rehearsed speech as in ted talks or broadcast news @xcite @xcite . with the increasing popularity of end-to-end speech translation systems @xcite , one may not want disfluency removal to be treated as an intermediate step between asr and mt. it might be more desirable for disfluency removal to be handled within the model itself, or as a separate post-processing step. to produce fluent translations from disfluent text, one would typically require access to disfluent speech (or text) and its corresponding fluent translations during training. while some corpora with labeled disfluencies exist @xcite , only subsets have been translated and/or released. @xcite introduced a set of fluent references for the fisher spanish-english conversational speech corpus @xcite . this has enabled a new task of endto-end training and evaluation on fluent references. @xcite reports results using a speechto-text model trained on this corpus using both fluent and disfluent translations. however, fluent translations of disfluent speech or text are a scarce resource. it would be highly desirable to build a system for disfluency removal that does not rely on fluent references. in this work, we propose a framework for disfluency removal that utilizes a simple noise induction technique for data augmentation using fluent monolingual text in the target language. during denoising, such disfluent text is trained jointly with parallel disfluent-to-disfluent textual translation data, thus simultaneously optimizing the objectives of disfluency removal and translation. this work describes the submission of iit bombay to the conversational speech translation challenge at iwslt 2020 @xcite . we release code for our proposed approach at the following url. 1 section 2 describes the details of the data used and the proposed noise induction technique to leverage monolingual data. section 3 describes our proposed architecture. we then present experimental details in section 4, followed by our main results in section 4.2. finally, we analyze our model outputs in section 5 and present our conclusions in section 6. ## model this section describes the proposed architecture for disfluency removal and translation. since our objective is two-fold, which is disfluency removal and translation, section 3.2.1 first presents our denoising module which is aimed at achieving the task of disfluency removal and section 3.2.2 describes how translation is achieved by our model. ## analysis in this section, we discuss different types of examples that were generated by our model and how they differ from the disfluent reference and the fluent reference. output is the generated translation from our implementation, disfluent ref and fluent ref are the disfluent and fluent references, respectively. it should be noted that fluent references were not used during training and are only being shown here for the sake of comparison foot_5 . segment comparison: deletion, insertion, shift. figure 2 shows that the filler word oh has been omitted in the generated output. in figure 3 , we observe that the repetition of the fluent phrase peruvian peruvian is handled correctly, but not the repetition of yes. in figures 4, the output carefully rejects um, along with legitimately paraphrasing the sentence as a result of the language model that it has learned from the corpus. in figures 5, the disfluency right yes yes has been completely removed. instead of choosing yes, it replaced it with well sure, but the disfluency has been removed. ## conclusion in this work, we propose a model for generating fluent translations from disfluent text without any access to fluent references during training. we rely on having access to monolingual fluent text in the target language, which is largely available for most languages. we extract disfluency patterns by examining the disfluent text and inject disfluencies to create a parallel disfluent-to-fluent text corpus in the target language. we compare our results at different levels of disfluency induction and show significant improvements over a competitive baseline. for future work, we aim at building more sophisticated and rich disfluency induction models. in this work, we focused on the text-to-text setting. we will look at extending this approach to a speechto-text spoken translation task, with disfluency removal being an auxiliary task and investigate how to meaningfully tie parameters across an audio encoder and a text encoder. furthermore, we only report standard quantitative metrics like bleu and meteor here. more detailed human evaluations may better highlight the benefits and limitations of our approach. we also believe that our proposed approach can be easily applied to other language pairs and hope to verify this as part of future work.