ELITR Non-Native Speech Translation at IWSLT 2020 (2020)

Abstract: This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020. We describe systems for offline ASR, real-time ASR, and our cascaded approach to offline SLT and real-time SLT. We select our primary candidates from a pool of pre-existing systems, develop a new end-to-end general ASR system, and a hybrid ASR trained on non-native speech. The provided small validation set prevents us from carrying out a complex validation, but we submit all the unselected candidates for contrastive evaluation on the test set.

## introduction this paper describes the submission of the eu project elitr (european live translator) 1 to the non-native speech translation task at iwslt 2020 @xcite . it is a result of a collaboration of project partners charles university (cuni), karlsruhe institute of technology (kit), and university of edinburgh (uedin), relying on the infrastructure provided to the project by pervoice company. the non-native speech translation shared task at iwslt 2020 complements other iwslt tasks by new challenges. source speech is non-native english. it is spontaneous, sometimes disfluent, and some of the recordings come from a particularly noisy environment. the speakers often have a significant non-native accent. in-domain training data are not available. they consist only of native out-domain speech and non-spoken parallel corpora. the validation data are limited to 6 manually transcribed documents, from which only 4 have reference translations. the target languages are czech and german. the task objectives are quality and simultaneity, unlike the previous tasks, which focused only on the quality. despite the complexity, the resulting systems can be potentially appreciated by many users attending an event in a language they do not speak or having difficulties understanding due to unfamiliar non-native accents or unusual vocabulary. we build on our experience from the past iwslt and wmt tasks, see e.g. @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite . each of the participating institutions has offered independent asr and mt systems trained for various purposes and previous shared tasks. we also create some new systems for this task and deployment for the purposes of the elitr project. our short-term motivation for this work is to connect the existing systems into a working cascade for slt and evaluate it empirically, end-to-end. in the long-term, we want to advance state of the art in non-native speech translation. ## overview of our submissions this paper is a joint report for two primary submissions, for online and offline sub-track of the non-native simultaneous speech translation task. first, we collected all asr systems that were available for us (section 3.1) and evaluated them on the validation set (section 3.2). we selected the best candidate for offline asr to serve as the source for offline slt. then, from the asr systems, which are usable in online mode, we selected the best candidate for online asr and as a source for online slt. in the next step (section 4), we punctuated and truecased the online asr outputs of the validation set, segmented them to individual sentences, and translated them by all the mt systems we had available (section 5.1). we integrated the online asrs and mts into our platform for online slt (sections 5.2 and 5.3). we compared them using automatic mt quality measures and by simple human decision, to compensate for the very limited and thus unreliable validation set (section 5.4). we selected the best candidate systems for each target language, for czech and german. both best candidate mt systems are very fast (see section 5.5). therefore, we use them both for the online slt, where the low translation time is critical, and for offline slt. in addition to the primary submissions, we included all the other candidate systems and some public services as contrastive submissions. ## automatic speech recognition this section describes our automatic speech recognition systems and their selection. ## punctuation and segmentation all our asr systems output unpunctuated, often all lowercased text. the mt systems are designed mostly for individual sentences with proper casing and punctuation. to overcome this, we first insert punctuation and casing to the asr output. then, we split it into individual sentences by the punctuation marks by a rule-based language-dependent moses sentence splitter @xcite . depending on the asr system, we use one of two possible punctuators. both of them are usable in online mode. ## machine translation this section describes the translation part of slt. ## conclusion we presented elitr submission for non-native slt at iwslt 2020. we observe a significant qualitative difference between the end-to-end offline asr methods and hybrid online methods. the component that constrains the offline slt from real-time processing is the asr, not the mt. we selected the best candidates from a pool of pre-existing and newly developed components, and submitted our primary submissions, although the size of the development set limits us from a reli-able validation. therefore, we submitted all our unselected candidates for contrastive evaluation on the test set. for the results, we refer to @xcite . eunah cho, jan niehues, kevin kilgour, and alex waibel. 2015. punctuation insertion for real-time spoken language translation. in iwslt. eunah cho, jan niehues, and alex waibel. 2012. segmentation and punctuation prediction in speech language translation using a monolingual translation system. in iwslt. eunah cho, jan niehues, and alex waibel. 2017. nmtbased segmentation and punctuation insertion for real-time spoken language translation. in inter-speech. florian dessloch, thanh-le ha, markus müller, jan niehues, thai-son nguyen, ngoc-quan pham, elizabeth salesky, matthias sperber, sebastian stüker, thomas zenkel, and alexander waibel. 2018. kit lecture translator: multilingual speech translation with one-shot learning. in coling: system demonstrations. dario franceschini et al. 2020. removing european language barriers with innovative machine translation technology. in lrec iwltp. in print. melvin johnson, mike schuster, quoc v. le, maxim krikun, yonghui wu, zhifeng chen, nikhil thorat, fernanda viégas, martin wattenberg, greg corrado, macduff hughes, and jeffrey dean. 2017. google's multilingual neural machine translation system: enabling zero-shot translation. transactions of the association for computational linguistics, 5:339-351. marcin junczys-dowmunt et al. 2018. marian: fast neural machine translation in c++. in acl system demonstrations. ondrej klejch, joachim fainberg, peter bell, and steve renals. 2019. lattice-based unsupervised testtime adaptation of neural networkacoustic models. corr, abs/1906.11521. philipp koehn et al. 2007. moses: open source toolkit for statistical machine translation. in acl interactive poster and demonstration sessions. taku kudo and john richardson. 2018. sentencepiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. in emnlp: system demonstrations. lucian vlad lita, abe ittycheriah, salim roukos, and nanda kambhatla. 2003. truecasing. in acl.