OPUS - MT – Building open translation services for the World (2020)

Abstract: This paper presents OPUS-MT a project that focuses on the development of free resources and tools for machine translation. The current status is a repository of over 1,000 pre-trained neural machine translation models that are ready to be launched in on-line translation services. For this we also provide open source implementations of web applications that can run efficiently on average desktop hardware with a straightforward setup and installation.

## introduction equality among people requires, among other things, the ability to access information in the same way as others independent of the linguistic background of the individual user. achieving this goal becomes an even more important challenge in a globalized world with digital channels and information flows being the most decisive factor in our integration in modern societies. language barriers can lead to severe disadvantages and discrimination not to mention conflicts caused by simple misunderstandings based on broken communication. linguistic discrimination leads to frustration, isolation and racism and the lack of technological language support may also cause what is known as the digital language death @xcite . machine translation (mt) has developed into a useful tool that diminishes and partially removes such language barriers. modern mt engines enable people to communicate, to access information in foreign languages and to build efficient resources for new communities. the mission of opus-mt 1 is to provide open translation services and tools that are free from commercial interests and restrictions. the idea is to make automatic translation accessible for anyone in a transparent and secure way without exploitation plans and hidden agendas compromising privacy and placing marketing strategies. we also want to focus on the support of minority and low-resource languages with the aim to introduce a community effort for the benefit of all. opus-mt has successfully launched its first pilot system and currently collaborates with the wikimedia foundation in the setup of translation ser-vices for the production of wikipedia content in new languages based on more elaborated resources available in, e.g. english. currently, the project provides over 1,000 pre-trained translation models that are free to download and use. opus-mt also contains open-source software for launching translation services as web applications. the on-going effort focuses on the improvement of translation quality, language coverage and emphasizes specific test cases to study the applicability of the approach. more details about the implementation and current status of the project are given below. ## opus-mt models the models that we train are based on state-of-theart transformer-based neural machine translation (nmt). we apply marian-nmt foot_0 in our framework, a stable production-ready nmt toolbox with efficient training and decoding capabilities @xcite . our models are trained on freely available parallel corpora collected in the large bitext repository opus foot_1 @xcite . the architecture is based on a standard transformer setup with 6 self-attentive layers in both, the encoder and decoder network with 8 attention heads in each layer. the hyper-parameters follow the general recommendations given in the documentation of the software. all the details can be seen in the training procedures that we also release as open source in our github repository. 4 opus-mt supports both, bilingual as well as multilingual models. for the latter, we apply the language label approach proposed by @xcite procedures that make it easy to train a large number of translation models from the existing data in the opus collection. the procedures take care of proper pre-processing and training setups to enable batch-processes without the immediate need for further adjustments. we try to reduce the burden of time-consuming optimization and focus on rather generic models for the time being in order to quickly achieve a good language coverage without significantly compromising translation quality that can be achieved. we use common benchmarks and test sets that are extracted on the fly from held-out data to monitor the quality of the nmt models. test sets and results are released together with the models, preand post-processing scripts and basic information about their usage. the table of currently supported language pairs can be accessed on-line. 5 we also develop generic fine-tuning and data augmentation procedures that can be used to further improve the translation models. we implemented a pipeline for backtranslation of wikimedia content (coming from wikipedia, wikibooks, wikisource, etc.) to augment existing training data. backtranslation is known to significantly boost performance and to enable simple domain adaptation based on in-domain target language data. furthermore, we also provide procedures for fine-tuning that can adjust model parameters according to some small in-domain data set, another successful strategy for domain adaptation. the impact of fine-tuning and backtranslation can be seen on the example of the english-finnish opus-mt model listed in ## opus-mt servers finally, we also provide simple web applications that can be used to launch translation services based on the pre-trained models. the most straightforward setup is implemented as a dockerized tornadobesed web application that can be set up with a few simple commands. the configuration can be adjusted and extended to serve any bilingual trans- ## conclusions this paper presents opus-mt, a project that focuses on the development of free resources and tools for machine translation. the current status is a repository of over 1,000 pre-trained neural mt models that are ready to be launched in on-line translation services. for this we also provide opensource implementations of web applications that can run efficiently on average desktop hardware with a straightforward setup and installation. the work is supported by the european union's horizon 2020 projects fotran and memad as well as by the finnish it center for science, csc.