Does Simultaneous Speech Translation need Simultaneous Models? (2022)

Abstract: In simultaneous speech translation (SimulST), finding the best trade-off between high output quality and low latency is a challenging task. To meet the latency constraints posed by different application scenarios, multiple dedicated SimulST models are usually trained and maintained, generating high computational costs. In this paper, also motivated by the increased sensitivity towards sustainable AI, we investigate whether a single model trained offline can serve both offline and simultaneous applications under different latency regimes without additional training or adaptation. Experiments on en->de, es show that, aside from facilitating the adoption of well-established offline architectures and training strategies without affecting latency, offline training achieves similar or better quality compared to the standard SimulST training protocol, also being competitive with the state-of-the-art system.

## introduction many application contexts, such as conferences and lectures, require automatic speech translation (st) to be performed in real-time. to meet this requirement, simultaneous st (simulst) systems strive not only for high output quality but also for low latency (i.e. the elapsed time between the speaker's utterance of a word and the generation of its translation in the target language). balancing quality and latency is extremely complex as the two objectives are conflicting: in general, the more a system waits -which implies higher latency -the better it translates thanks to a larger context to rely on. simulst models manage the quality-latency trade-off by means of a decision policy: the rule that determines whether a system has to wait for more input or to emit one or more target words. the most popular decision policy is the wait-k, a straightforward heuristic that prescribes to wait for * work done when working at fbk. a predefined number of words before starting to generate the translation. initially proposed by @xcite for simultaneous machine translation (simulmt), the wait-k is now widely adopted in simulst @xcite @xcite @xcite thanks to its simplicity. apart from wait-k, other attempts have been made to develop decision policies learned by the simulst system itself @xcite @xcite , all resulting in computationally expensive models with limited diffusion. regardless of the decision policy, simulst systems are usually trained simulating the conditions faced at inference time, that is with only a partial input available @xcite @xcite @xcite @xcite . since the size of the partial input -and consequently of the context that the simulst system can exploit to translate -varies according to the latency requirements imposed by real-world applications, foot_0 several models must be trained and maintained to accommodate different quality-latency trade-offs. this results in high computational costs that contrast with rising awareness on the need to reduce energy consumption @xcite towards more sustainable ai @xcite . so far, the benefits of training systems on partial inputs have been taken for granted and, although works employing models trained in offline mode are documented in literature @xcite , the indispensability of simultaneous training in simulst has never been demonstrated. with an eye at the burden and environmental impact of training multiple dedicated models for different tasks -offline, simultaneous -and latency regimes, in this work we address the following question: does simultaneous speech translation actually need models trained in simultaneous mode? to this end, we experiment with a single, easy-tomaintain offline model, which can effectively serve both the simultaneous and offline tasks. specifically, we explore the application of the widely adopted wait-k policy to the offline-trained st system only at inference time, bypassing any additional training neither to adapt the model to the simultaneous scenario nor to accommodate different latency requirements. 2 through experiments on two language directions (@xmath0, es}), having respectively different and similar word ordering with respect to the source, we show that: ## leveraging offline solutions offline training brings considerable advantages in terms of reducing the computational costs of simulst technology. first, only one model can be trained and maintained to serve both offline and simultaneous tasks without performance degradation. second, contrary to the simultaneous-training mode, the choice of the word detection strategy at run-time does not depend on the strategy used during training. rather, it can be made according to the specific use case, making the offline-trained model more flexible. this also means that other decision policies can be applied to the offline-trained system without the need to re-train it from scratch. using a single offline-trained model not only speeds up its development but also opens up the possibility to directly adopt powerful offline architectures and techniques without performing any additional training nor adaptation to the simultaneous scenario. in the following, we test this hypothesis to find out whether recent architectural improvements (section 6.1) and data augmentation techniques (section 6.2) designed for offline st also have a positive impact in simulst. in recent years, many architectures have been proposed to address the offline st task @xcite @xcite . among them, the conformer @xcite has recently shown impressive results both in speech recognition, for which it was initially proposed, and in speech translation @xcite . the main aspects characterizing this encoder-decoder architecture are related to the encoder part. inspired by the macaron-net @xcite , the conformer encoder is built with a sandwich structure and integrates the relative sinusoidal positional encoding scheme @xcite . given the promising results it achieved in the offline scenario, we choose to test if this architecture also brings quality and latency gains in simulst. since we found in section 3 that fixed and adaptive word detection strategies have their own use cases (their best results are observed at different latency regimes, respectively low for fixed and mediumhigh for adaptive), we compare conformer-and transformer-based architectures using both strategies. for the offline training of conformer, we follow the same procedure used for transformer. details about the model hyper-parameters are presented in appendix a.2. ## comparison with the state of the art so far, we discovered that scaling to better performing architectures and more data further improves the simultaneous results of offline-trained models. but how good is their performance compared to the state of the art in simulst? to answer this question, we compare our best system, the offline-trained conformer with adaptive word detection, with the cross attention augmented transducer @xcite ) -caat -used by the winning submissions at iwslt 2021 @xcite and 2022 @xcite . inspired by the recurrent neural network transducer by @xcite , caat is made of three transformer stacks: the encoder, the predictor, and the joiner. these three elements are jointly trained in simultaneous to optimize the quality of the translations while keeping latency under control. for training and testing the caat architecture, we use the code published by the authors and adopt the same hyper-parameters of their paper. as the performance of the caat model is sensitive to sequence-level kd @xcite show a 2 bleu degradation without it), we compare it with the offline-trained conformer model using the same data settings -see section 6.2. we report the caat results obtained by adopting both the greedy search used in our simulst settings and the beam search used by @xcite . as suggested by @xcite , we also compute the computational aware (ca) version of the laal metric (laal ca ), which is defined as the time elapsed from the beginning of the generation process to the prediction of the partial target. 7 since laal ca represents the real wall-clock elapsed time experienced by the user, it gives a more reliable evaluation of the simulst performance in a real-time scenario. for the sake of completeness, we also report the results of average lagging @xcite in appendix c. we present the comparison in figure 5 . from the laal-bleu curves, we see that, at low latency regime, the caat model (in solid red) outperforms our offline-trained conformer model (in solid blue) by 2 bleu on en-de and 4 bleu on en-es. however, moving to medium-high latency regime, the conformer significantly outperforms caat, reaching gains of 4 bleu on en-de and 2 bleu on en-es. we can also notice a degradation of the caat en-de translation quality that is caused by an under-generation problem at higher latency, for which we give details in appendix b. when it comes to laal ca -bleu, the scenario changes, bringing caat curves much closer to those of conformer. the state of the art still outperforms the conformer at lower latency but in this case, waiting about 100/200ms more, the conformer performance starts to improve consistently. comparing the laal-and laal ca -bleu curves, we see that our offline-trained system is more coherent between computational and noncomputational aware metrics: while conformer has a computational overhead of 400/500ms, caat requires 1400/1500ms more than its ideal laal. the caat greedy curves (dotted red) show only a little improvement in latency compared to the beam search (solid red), suggesting that its higher computational cost does not depend on the generation strategy but on other factors like its complex and more computationally expensive architecture. all in all, we can say that, compared to the state of the art in simulst, the lower performance of our offline-trained conformer at low latency regime is balanced by consistently higher bleu scores at medium and high latency. ## conclusions to reduce the potentially large amount of experiments usually performed to build simulst models, we explored the use of a single offline-trained model to serve both the offline and simultaneous tasks. through comparison with native simulst systems, we showed that our offline-trained model can be successfully used in real-time, achieving comparable or even better results. to further enhance its performance, we investigated the adoption of consolidated techniques and emerging architectures from offline research, showing consistent improvements also in the simultaneous scenario. the benefits of offline training indicate the potential of applying this method without the need for any additional training or adaptation. besides facilitating system deployment, another important advantage of building and reusing one single model to rule both tasks is the drastic reduction of the carbon footprint of st training (by a factor of 9 in our evaluation setting). this represents an important step in response to rising concerns about the ai energy consumption and environmental impact toward more sustainable development. as regards simulst evaluation, the differences between results computed with noncomputationally and computationally aware latency metrics suggest that including computational time in the measurements heavily influences the outcomes of system comparisons. in our particular case, the differences in latency between the offline-trained models and the state of the art ob-served in terms of the non-computationally aware laal metric become smaller when considering its computational aware version. although lower latency is theoretically reached by the state of the art caat model, this comes at the cost of a more complex and computationally expensive architecture that shows its limitations at inference time. we therefore invite the simulst community to use computationally aware metrics for more sound evaluations, referring to ideal metrics only in the absence of similar testing assets, as machines with comparable computational power.