M i SS : An Assistant for Multi-Style Simultaneous Translation (2021)

Abstract: In this paper, we present MiSS , an assistant for multi-style simultaneous translation. Our proposed translation system has five key features: highly accurate translation, simultaneous translation, translation for multiple text styles, back-translation for translation quality evaluation, and grammatical error correction. With this system, we aim to provide a complete translation experience for machine translation users. Our design goals are high translation accuracy, real-time translation, flexibility, and measurable translation quality. Compared with the free commercial translation systems commonly used, our translation assistance system regards the machine translation application as a more complete and fully-featured tool for users. By incorporating additional features and giving the user better control over their experience, we improve translation efficiency and performance. Additionally, our assistant system combines machine translation, grammatical error correction, and interactive edits, and uses a crowdsourcing mode to collect more data for further training to improve both the machine translation and grammatical error correction models. A short video demonstrating our system is available at https://www.youtube.com/watch?v=ZGCo7KtRKd8 .

## introduction with the increasing technological development of the world and the acceleration of globalization, people from different languages and cultural backgrounds communicate more and more, and the needs of translation are becoming more and more important and diverse. although traditional manual translation works well, with the increasing frequency of international communication, traditional manual translation far from meets demand, and machine translation has correspondingly risen in popularity @xcite . recently, neural machine translation (nmt), especially transformer-based nmt, has emerged as a promising approach with the potential to address many of the shortcomings of traditional rulebased or statistics-based machine translation systems @xcite @xcite . this has significantly improved the performance of machine translation and other related tasks @xcite . although neural machine translation has made tremendous improvements and is relatively highperforming, because human language is so complex, machine translation is often still only used as an assistance tool rather than the sole entity responsible for translation. there are several popular and large existing commercial machine translation systems that provide users with effective translation (e.g., google translator, bing translator, amazon translate, and baidu translate). as nmt is still very imprecise, however, these web services fall short, as they do not provide sufficient information to users in how good each translation is, which is particularly pertinent to those who have not mastered the target language. voicetra foot_0 included back-translation in the machine translation system to alleviate this deficiency; however, this practice requires users to perform additional manual evaluations, which brings new usage costs. in mainstream machine translation systems, sentences or paragraphs are used as the units of translation, which means that it takes a relatively long time to provide users with translated content. simultaneous machine translation, translating sentences in real-time while the user speaks or types, can significantly reduce this translation time, but its performance lags behind that of standard nmt. although some commercial machine translation systems such as google and baidu have introduced simultaneous translation feature, due to the integration of simultaneous translation and whole-sentence translation, users cannot easily control whether the system uses simultaneous translation or whole-sentence translation, and the automated control of commercial systems sometimes does not follow the user's wishes. since user input errors are unavoidable for any human-computer interaction system, the quality of nmt system also has been shown to significantly degrade when confronted with source-side noise @xcite @xcite . the previous grammatical error detection and correction work focused on computer-aided writing systems. some existing computer-aided writing systems (grammarly foot_1 and pigai foot_2 , write&improve foot_3 , and lingglewrite foot_4 ) detect and correct grammatical errors; however, systems such as these have had little attention when considered in the context of input error detection or correction for commercial machine translation systems, as their main focus is generally posttranslation editing. high quality domain specific machine translation systems are in high demand whereas general purpose mt has limited applications because different machine translation users want to generate translations that can be used in the scenario. on the one hand, general purpose translation systems usually perform poorly @xcite . on the other hand, appropriate translation is also a very important goal to pursue. there are two typical methods to achieve this goal. one is to use the domain adaptation method to obtain a domainspecific model from the existing general machine translation model through transfer learning. the other is to adopt an conditional translation decoder to integrate various domains into the same model and generate translations according to different input conditions @xcite . at present, the commercial machine translation system mainly adopts the former one, but it also brings the additional deployment cost. considering the deficiencies of existing systems, the new needs of users, and the current development of natural language processing, we developed a web-based machine translation demonstration system miss. in this system, we tried to integrate several new features to provide better services for users. with miss, users can get real-time translations while writing, flexible control in switching between real-time translation and whole-sentence translation, informative back-translation feedback and scoring, and input error detection and revision suggestions. in addition, the system also supports user interactions that modify the translations or inputs, which provides crowdsourced data for further improving the performance of our machine translation and grammatical error correction. notably, there were also several interactive translation systems in the past, such as casmacat @xcite , @xcite , @xcite , and inmt @xcite . the distinctions lie in the abilities of the systems and the features to adapt to the latest user needs. ## the miss system there are 5 features in our miss translation system: simultaneous translation, back-translation for quality evaluation, grammatical error correction, multi-style translation, and crowdsourcing data collection. the system is available at http: //miss.x2brain.com/ until november 12, 2021. we show a screenshot of the system in figure 1 . in the following subsections, we will describe each component of the system. ## implementation and training the full system consists of 4 neural models: (1) a multi-style nmt model, (2) a simultaneous nmt model, (3) a grammatical error correction model, and (4) a bert model. in our current miss release, we translate between three languages (english (en), chinse (zh), and japanese (ja)) for demonstration. for the multi-style nmt model, we implement ctrl-nmt using the public fairseq @xcite toolkit. in our system, we adopt the transformer (big) setting as in @xcite . we did not choose a deeper or wider transformer @xcite ) model because we wanted to balance performance and efficiency. as in @xcite , we used a data-dependent gaussian prior objective (d2gpo) during the nmt model training process for better generalization. due to resource constraints, our currently deployed model does not perform back-translation of larger sentences. provider style num. en-zh wmt20 formal 28.3m ai challenger18 oral 12.9m en-ja wmt20 formal 17.7m ted+bsd oral 0.25m for translation quality evaluation, we measure the semantic equivalence using bertscore, an automated evaluation metric that computes token similarity using contextual embeddings. we use roberta-large, bert-base-chinese, and bertbase-japanese-char as the respective initial embedding sources for our english, chinese, and japanese evaluation models. as @xcite observed that fine-tuning the pre-trained con- provider num. en pie-synthetic 9m lang-8 947k nucle 56k fce 34k w&i+locness 34k zh nlpcc2018-gec 1.3m hsk+lang8 cged ja lang8 3.1m ## evaluation we conducted empirical experiments on our models to evaluate the performance of important components in our system. for the nmt component, we chose the wmt2020 test set newstest2020 as the evaluation set for formal en-zh and en-ja translation and the development set of the ai challenger 2018 competition as the evaluation set for oral zh-en translation. in @xmath0, we used multi-bleu as our evaluation metric, and we adopted the moses tokenizer for word tokenization, while in @xmath1, we used character-level multi-bleu to remove the influence of different segmenters on bleu score. for the standard and simultaneous machine translation components, we used the same evaluation sets and metrics. for the gec component, we followed common practice in the gec task @xcite and used precision (p), recall (r), and f 0 .5 to evaluate our models on all three languages. we evaluated english at the word level and chinese and japanese at the character level. we chose the test set of the conll-2014 shared task as our evaluation set for our english gec model. for chinese and japanese, we extracted 5000 sentences from the original training set for the development set and 5000 sentences for the test set and used the rest as the training set. er-rant foot_5 was used to convert parallel files to the m2 format for subsequent scoring with the m 2 scorer @xcite . the results of our models for standard nmt and simultaneous nmt are shown in in ctrl-nmt, through the incorporation control codes, we found that the translation performance for specific styles using the single model was equivalent to or, in some cases, better than that of training separate models. this shows that the transformer-based model sufficiently accommodates the generation of multiple styles of language, and leveraging the language commonalities between different styles can bring additional im-and gunicorn are used to wrap the model into a microservice interface for external calls. nginx is used to distribute static resources and balance load. we use a basic web ui to make our service accessible to users. in addition, mongodb is adopted to store the users' logs, which the system collects. ## conclusion and future work in this paper, we presented a translation system, miss. this system supports multi-style machine translation, simultaneous machine translation, grammatical error detection and correction, and back-translation-based quality evaluation. our goal in developing this system is providing users with a more fluid machine translation experience. using the research of the nlp community, we were able to introduce a variety of translation and translation-related tools to help users. in addition, we leverage the user's operations and feedback in the system as a source of crowdsourced information to potentially use in further improving the performance of the system. compared with existing commercial translation systems, our system can provide a more comprehensive experience. with this work, we also lay out steps to take to further improve the machine translation user experience: improve the consistency of translation by integrating document-level context, enhance the performance of models by incorporating backtranslation using monolingual data, include more language styles such as academic translation, and explore the data collected through crowdsourcing for further improving overall performance.