Dynamic Sentence Boundary Detection for Simultaneous Translation (2020)

Abstract: Simultaneous Translation is a great challenge in which translation starts before the source sentence finished. Most studies take transcription as input and focus on balancing translation quality and latency for each sentence. However, most ASR systems can not provide accurate sentence boundaries in realtime. Thus it is a key problem to segment sentences for the word streaming before translation. In this paper, we propose a novel method for sentence boundary detection that takes it as a multi-class classification task under the end-to-end pre-training framework. Experiments show significant improvements both in terms of translation quality and latency.

## introduction simultaneous translation aims to translate the speech of a source language into a target language as quickly as possible without interrupting the speaker. typically, a simultaneous translation system is comprised of an auto-speech-recognition (asr) model and a machine translation (mt) model. the asr model transforms the audio signal into the text of source language and the mt model translates the source text into the target language. recent studies on simultaneous translation @xcite @xcite focus on the trade-off between translation quality and latency. they explore a policy that determines when to begin translating with the input of a stream of transcription. however, there is a gap between transcription and asr that some asr model doesn't provide punctuations or cannot provide accurate punctuation in realtime, while the transcription is always well-formed. see figure 1 for illustration. without sentence boundaries, the state-of-the-art wait-k model takes insufficient text as input and produces an incorrect translation. therefore, sentence boundary detection (or sentence segmentation) foot_0 plays an important role to narrow the gap between the asr and transcription. a good segmentation will not only improve translation quality but also reduce latency. studies of sentence segmentation falls into one of the following two bins: other researches utilized language model, either based on n-gram @xcite or recurrent neural network (rnn) @xcite . in this paper, we use classification to solve the problem of sentence segmentation from the perspective of text. instead of predicting a sentence boundary for a certain position, we propose a multiposition boundary prediction approach. specifically, for a source text @xmath0, ..., x t }, we calculate the probability of predicting sentence boundary figure 1 : an english-to-german example that translates from a streaming source with and without sentence boundaries. we take the wait-k model @xcite for illustration, @xmath1. the wait3 model first performs three read (wait) action at the beginning of each sentence (as shown in blue), and then alternating one read with one write action in the following steps. given the input source without sentence boundaries (in the 4 th line), the wait3 model (in the 5 th line) doesn't take the three read action at the beginning of following sentences. therefore, the english phrase "it's going to", which should have been translated as "wird", produced a meaningless translation "es ist geht dass" with limited context during wait3 model inference. ## background recent studies show that the pre-training and finetuning framework achieves significant improvements in various nlp tasks. generally, a model is first pre-trained on large unlabeled data. after that, on the fine-tuning step, the model is initialized by the parameters obtained by the pre-training step and fine-tuned using labeled data for specific tasks. @xcite proposed a generalized framework bert, to learn language representations based on a deep transformer @xcite encoder. rather than traditionally train a language model from-left-to-right or from-rightto-left, they proposed a masked language model (mlm) that randomly replace some tokens in a sequence by a placeholder (mask) and trained the model to predict the original tokens. they also pre-train the model for the next sentence prediction (nsp) task that is to predict whether a sentence is the subsequent sentence of the first sentence. @xcite proposed a pre-training framework ernie, by integrating more knowledge. rather than masking single tokens, they proposed to mask a group of words on different levels, such as entities, phrases, etc. the model achieves state-of-theart performances on many nlp tasks. in this paper, we train our model under the ernie framework. ## our method given a streaming input @xmath2, ..., x t , ..., x t }, the task of sentence segmentation is to determine whether x @xmath3. thus the task can be considered as a classification problem, that is p(y t |x, θ), where y @xmath4, 1}. however, in simultaneous translation scenario, the latency is unacceptable if we take the full source text as contextual information. thus we should limit the context size and make a decision dynamically. as the input is a word streaming, the sentence boundary detection problem can be transformed as, whether there exists a sentence boundary until the current word x t . thus we can use the word streaming as a context to make a prediction. we propose a multi-class classification model to predict the probability of a few words before x t as sentence boundaries (section 3.1). we use the ernie framework to first pre-train a language representation and then fine-tune it to sentence boundary detection (section 3.2). we also propose a dynamic voted inference strategy (section 3.3). ## experiment experiments are conducted on english-german (en-de) simultaneous translation. we evaluate 1) the f-score foot_1 of sentence boundary detection and 2) case-sensitive tokenized 4-gram bleu @xcite as the final translation effect of the segmented sentences. to reduce the impact of the asr system, we use the transcription without punctuation in both training and evaluation. the datasets used in our experiments are listed in we evaluate our model and two existing methods listed below: our classifier in dynamic-base and dynamicforce is trained under ernie base framework. we use the released 6 parameters obtained at pretraining step as initialization. in the fine-tuning stage, we use a learning rate of 2e -5 . 5 we only keep the two classes of period and φ in this work 6 https://github.com/paddlepaddle/ernie ## related work sentence boundary detection has been explored for years, but the majority of these work focuses on offline punctuation restoration, instead of applied in simultaneous translation. existing work can be divided into two classes according to the model input. ## conclusion in this paper, we propose an online sentence boundary detection approach. with the input of streaming words, our model predicts the probability of multiple positions rather than a certain position. by adding this adjacent position constraint and using dynamic prediction, our method achieves higher accuracy with lower latency. we also incorporate the pre-trained technique, ernie to implement our classification model. the empirical results on iwslt2014 demonstrate that our approach achieves significant improvements of 0.19 f-score on sentence segmentation and 1.55 bleu points compared with the language-model based methods.