Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters (2024)

Abstract: Simultaneous machine translation aims at solving the task of real-time translation by starting to translate before consuming the full input, which poses challenges in terms of balancing quality and latency of the translation. The wait-k policy offers a solution by starting to translate after consuming words, where the choice of the number k directly affects the latency and quality. In applications where we seek to keep the choice over latency and quality at inference, the wait-k policy obliges us to train more than one model. In this paper, we address the challenge of building one model that can fulfil multiple latency levels and we achieve this by introducing lightweight adapter modules into the decoder. The adapters are trained to be specialized for different wait-k values and compared to other techniques they offer more flexibility to allow for reaping the benefits of parameter sharing and minimizing interference. Additionally, we show that by combining with an adaptive strategy, we can further improve the results. Experiments on two language directions show that our method outperforms or competes with other strong baselines on most latency values.

## introduction simultaneous machine translation (simt) aims at reducing the latency of translation systems. in scenarios with low latency demands, such as conferences or lectures, translating with minimum delay is crucial. in order to reduce the latency, simt models start translating before consuming the full input sentence, which improves the latency but affects the quality of the translation, because of limited access to enough source context to make a correct prediction. simt techniques design a strategy to decide when to make a read (i.e. wait for more source tokens) or write (i.e. output a new token) action. the strategy has to balance the trade-off between quality and latency by making more read or write actions. making more read actions will lead to improved quality but will hinder the latency, while the opposite is true for making more write actions. fixed policies design a strategy that is detached from whether there is sufficient context to make a write action @xcite @xcite . for instance, the wait-k policy @xcite trains the model to make k number of read actions before every write action. the value of k has a direct impact on the quality and latency of the translation and since it is decided during training, wait-k models have to be trained with latency in mind, which means that in order to support multiple latency levels, we need to train multiple models. the multi-path training @xcite was introduced to solve this issue by sampling the value of k randomly during training, which results in a model that supports multiple latency levels. this technique was shown to benefit the inference at lower wait-k values by improving the results, but it neglects that parameter sharing between all the wait-k values might introduce interference. @xcite addressed the interference issue by using mixture-of-experts (moe), where each head of the multi-head attention is treated as an expert and is trained on different wait-k values. this has proven to be a successful technique, but the number of wait-k experts we can introduce depends on the number of heads in the transformer model, which limits the flexibility in terms of balancing parameter sharing and interference between the wait-k paths. our method relies on inserting lightweight adapters @xcite for this purpose. the number of the adapters and their capacity can be easily adjusted depending on the wait-k values we intend to support and the complexity of the language direction. dynamic strategies have gained increased atten-tion in recent years @xcite @xcite @xcite due to their effectiveness. dynamic strategies strive to strike a balance between latency and quality by making as much read actions as necessary and as much write actions as possible. the decision to read or write is made dynamically based on the context (which can be the received input and the previous target tokens) at each decoding step. although dynamic strategies achieve state-of-the-art results, they often require specialized training techniques @xcite @xcite that can balance between latency and quality when generating read/write actions, or even require the training of multiple models @xcite to support multiple latency levels. in order to take advantage of the dynamic wait-k strategies, we adopt a strategy that composes multiple wait-k models during inference (we refer to this as adaptive wait-k @xcite ) to work with wait-k adapters instead. this brings efficiency and cost benefits as only one model is required to satisfy multiple latency levels and also improves performance compared to other strong baselines including adaptive wait-k. in summary, our main contributions are the following: ## method our method is composed of two steps: first we train a single model that can support multiple fixed waitk values by using wait-k adapters, then we rely on the probability that the model assigns to the most likely token in order to build an adaptive strategy, where we decide a read or write action based on a predefined probability threshold. ## analysis in this section, we look into how the performance changes in response to varying the value of k max , then we provide a wall-clock time comparison between adapters wait-k and multi-path wait-k. moreover, we experiment with how balancing between parameter sharing and interference by adjusting the adapter lagging impacts the performance, and also experiment with varying the bottleneck size in order to discern the impact of the complexity of the adapters. at last, we analyze the l2-norm of the adapter representations to discover which adapter layers are involved in the prediction. ## conclusion in this paper, we employ adapters to build a simt model that can support multiple latency levels at inference. we use the multi-path training and show that by adding wait-k adapters we can flexibly balance parameter sharing and interference between the wait-k paths. furthermore, we adopt a simple adaptive strategy and show that it further improves the results. by comparing against strong adaptive and fixed strategies, we find that our method achieves better or competitive results on most latency levels.