Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy (2021)

Abstract: Simultaneous machine translation (SiMT) generates translation before reading the entire source sentence and hence it has to trade off between translation quality and latency. To fulfill the requirements of different translation quality and latency in practical applications, the previous methods usually need to train multiple SiMT models for different latency levels, resulting in large computational costs. In this paper, we propose a universal SiMT model with Mixture-of-Experts Wait-k Policy to achieve the best translation quality under arbitrary latency with only one trained model. Specifically, our method employs multi-head attention to accomplish the mixture of experts where each head is treated as a wait-k expert with its own waiting words number, and given a test latency and source inputs, the weights of the experts are accordingly adjusted to produce the best translation. Experiments on three datasets show that our method outperforms all the strong baselines under different latency, including the state-of-the-art adaptive policy.

## introduction simultaneous machine translation (simt) @xcite @xcite begins outputting translation before reading the entire source sentence and hence has a lower latency compared to fullsentence machine translation. in practical applications, simt usually has to fulfill the requirements with different levels of latency. for example, a live broadcast requires a lower latency to provide smooth translation while a formal conference focuses on translation quality and allows for a slightly higher latency. therefore, an excellent simt model should be able to maintain high translation quality under different latency levels. figure 1: performance of wait-k models with different k train v.s. k test on iwslt15 @xmath0. k train and k test mean the number of source tokens to wait before performing translation during training and testing, respectively. the best translation performance under different latency with only one model @xcite . with fixed policy, e.g., wait-k policy @xcite , the simt model has to wait for a fixed number of source words to be fed and then read one source word and output one target word alternately. in wait-k policy, the number of words to wait for can be different during training and testing, denoted as k train and k test respectively, and the latency is determined by k test . figure 1 gives the performance of the model trained with k train under different k test , and the results show that under different k test the simt model with the best performance corresponds to different k train . as a result, multiple models should be maintained for the best performance under different latency. with adaptive policy, the simt model dynamically adjusts the waiting of source tokens for better translation by directly involving the latency in the loss function @xcite . although the adaptive policy achieves the state-of-the-art performance on the open datasets, multiple models need to be trained for different latency as the change of model latency is realized by the alteration of the loss function during training. therefore, to perform simt under different latency, both kinds of methods require training multiple models for different latency, leading to large costs. under these grounds, we propose a universal simultaneous machine translation model which can self-adapt to different latency, so that only one model is trained for different latency. to this end, we propose a mixture-of-experts wait-k policy (moe wait-k policy) for simt where each expert employs the wait-k policy with its own number of waiting source words. for the mixture of experts, we can consider that different experts correspond to different parameter subspaces @xcite , and fortunately the multi-head attention is designed to explore different subspaces with different heads @xcite . therefore, we employ multihead attention as the implementation manner of moe by assigning different heads with different waiting words number (wait-1,wait-3,wait-5,• • • ). then, the outputs of different heads (aka experts) are combined with different weights, which are dynamically adjusted to achieve the best translation under different latency. experiments on iwslt15 @xmath1, wmt16 @xmath2, our method can outperform strong baselines under all latency, including the state-of-the-art adaptive policy. further analyses show the promising improvements of our method on efficiency and robustness. ## background our method is based on mixture-of-experts approach, multi-head attention and wait-k policy, so we first briefly introduce them respectively. ## the proposed method in this section, we first view multi-head attention from the perspective of the mixture of experts, and then introduce our method based on it. ## analysis we conducted extensive analyses to understand the specific improvements of our method. unless otherwise specified, all the results are reported on de-en with transformer-base(8 heads). ## conclusion and future work in this paper, we propose mixture-of-experts waitk policy to develop a universal simt, which can perform high quality simt under arbitrary latency to fulfill different scenarios. experiments and analyses show that our method achieves promising results on performance, efficiency and robustness. in the future, since moe wait-k develops a universal simt model with high quality, it can be applied as a simt kernel to cooperate with refined external policy, to further improve performance.