USST ’s System for A uto S im T rans 2022 (2022)

Abstract: This paper describes our submitted text-to-text Simultaneous translation (ST) system, which won the second place in the Chinese→English streaming translation task of AutoSimTrans 2022. Our baseline system is a BPE-based Transformer model trained with the PaddlePaddle framework. In our experiments, we employ data synthesis and ensemble approaches to enhance the base model. In order to bridge the gap between general domain and spoken domain, we select in-domain data from general corpus and mixed then with spoken corpus for mixed fine tuning. Finally, we adopt fixed wait-k policy to transfer our full-sentence translation model to simultaneous translation model. Experiments on the development data show that our system outperforms than the baseline system.

## introduction simultaneous translation @xcite consists in generating a translation before the source speaker finishes speaking. it is widely used in many real-time scenarios such as international conferences, business negotiations and legal proceedings. the challenge of simultaneous machine translation is to find a read-write policy that balances translation quality and latency. the translation quality will decline if the machine translation system reads insufficient source information. when reading wider source text, latency will increase. recent read-write policies can be divided into two categories: fixed policies such as wait-k @xcite , wait-if* @xcite , and adaptive policies such as mocha @xcite , milk @xcite and mu @xcite . fixed policies are simple to implement, but they neglect contextual information, which might result in quality reduction. dynamic policies are more flexible, they can learn from data to achieve better quality/latency trade-offs, but accordingly difficult to train. in our system, we train a transformer @xcite with a deep encoder @xcite as baseline for abtaining rich source representations, besides we initialize the model with the method mentioned in deepnet @xcite in order to stabilize the training of the deeper model. at the pre-training stage, we firstly pretrain our model on a large general corpus, then we utilize data synthesis methods such as self-training and back-translation to improve model quality. during the fine-tuning phase, we first apply finetuning on a small spoken corpus. for better domain adaptation, we adopt mixed fine-tuning @xcite , which trains on a mixed dataset that includes a subsampled general corpus and an upsampled spoken corpus. thirdly, we propose a method called "in-domain mixed fine-tuning", which further improve the bleu score than mixed finetuning. specifically, inspired by in-domain data filtering @xcite , we mixed upsampled spoken data with selected in-domain data from general corpus rather than random subsampled. in the final stage, we employ the wait-k policy to convert the full-sentence translation model into a prefix-to-prefix architecture that predicts target words with only the source sentence's prefixes. after waiting for k-1 source subwords, the system reads a source subword and then predicts a target subword alternately @xmath0. an example of wait 1 is shown in figure 1 . the contributions of this paper are as follows: ## data we participate in the chinese-english streaming transcription track , where each sentence is broken into lines whose length is incremented by one word until the sentence is completed. an example is shown in similar to @xcite , we preprocess the data as follows: see ## experiments our system is implemented with the paddlepaddle foot_6 framework, and our experiments are carried out on ai studio foot_7 with 4 nvidia v100 gpu each of which has 32 gb memory. (we also benchmarked our code against fairseq foot_8 , see appendix a) ## conclusion in this paper we describe our chinese-to-english simultaneous translation system, which uses a deep transformer to improve translation quality and adopts wait-k policy @xcite to reduce latency. besides, for better domain adaption, we combined mixed fine-tuning @xcite with in-domain data filtering @xcite and proposed a new domain adaption method called "in-domain mixed fine-tuning", which is empirically more effective than fine-tuning and mixed fine-tuning. in our future work, we plan to validate the effective of our proposed in-domain mixed fine-tuning on more datasets, while investigating some novel domain adaption methods. we also plan to research on some dynamic read-write policies in order to better balance quality and latency for simultaneous translation tasks.