Neural Translation for the E uropean U nion ( NTEU ) Project (2020)

Abstract: The Neural Translation for the European Union (NTEU) project aims to build a neural engine farm with all European official language combinations for eTranslation, without the necessity to use a high-resourced language as a pivot. NTEU started in September 2019 and will run until August 2021.

## introduction normally, data for translation are available in english from or to another language. with a few exceptions, all etranslation mt engines include english as either source or target. thus, to translate between two non-english languages, english must be used as a pivot. the nteu partners, pangeanic, foot_1 tilde, foot_2 kan-tanmt foot_3 and sead foot_4 , have been awarded eu funds to build direct machine translation (mt) engines between any of the 24 eu official languages (e.g. spanish to german, croatian to italian, greek to polish, etc.) without pivoting through english (around 550 translation engines in total). ## approach nteu will provide a capacity service to etranslation by building a near-human-professionalquality neural engine farm which includes all eu language combinations. state-of-the-art technologies such as the transformer @xcite architecture will be implemented. moreover, lower-resourced languages (for example, irish or maltese) will be a challenge, and more effort will be required to obtain well-performing engines for them. in order to obtain the best results, we will experiment with techniques to supplement the original data, such as generating synthetic data by doing back-translation @xcite , checking of sentence alignments, transfer learning @xcite and unsupervised learning on a monolingual corpus @xcite . in addition to providing the trained engines, the nteu consortium will gather and clean data from all language combinations so that the engines can be retrained with other technologies in the future. as part of the national digital data gathering efforts, nteu will also act as a bridge between previous efforts, putting to work the results of the elrc foot_5 repository and other european data gathering efforts such as the nec tm foot_6 and paracrawl foot_7 projects. therefore, the project will promote the free flow of data between public administrations themselves and eu bodies. ## evaluation methodology the results of the mt will be manually evaluated in an open-source platform created by the consortium. following industry and wmt 9 practices for human evaluation, the evaluation dataset has been carefully chosen so as to represent realworld, whole, human-translated documents, purposely excluded from the training data. human evaluators will score the mt for fluency, adequacy and fit-for-purpose, in a range between 0 and 100. the human evaluator -who will be a native speaker of the target language, but not necessarily knowledgeable of the source languagewill be presented with the reference translation in context, plus two automatic translations (one from our engines and one from a third party state-ofthe-art system), both unidentified. each translation will be evaluated by two different human evaluators. the aim of the evaluation platform is to make human evaluation of mt faster, more efficient and more consistent. automatic evaluation using a larger dataset, also excluded from training, is foreseen as well. the objective is to extensively reduce the time required to build and approve production-ready mt engines with continuous evaluator feedback that reinforces data selection and training procedures. ## conclusion in conclusion, the nteu project will yield a stateof-the-art neural mt engine farm, which will include all eu language combinations without pivoting through english for etranslation. special emphasis will be put on the low-resourced language combinations. the collected data will be provided so it can be used in future technologies. moreover, an open-source platform will be developed to facilitate human evaluation of the mt engines. finally, the project will be promoted between public administrations and will put to work previous european data-gathering initiatives.