G o URMET – Machine Translation for Low-Resourced Languages (2022)

Abstract: The GoURMET project, funded by the European Commission’s H2020 program (under grant agreement 825299), develops models for machine translation, in particular for low-resourced languages. Data, models and software releases as well as the GoURMET Translate Tool are made available as open source.

## the project gourmet (global under-resourced media translation) started in january 2019 and runs until 30 june 2022. the consortium consists of five partners: the university of edinburgh (coordinator), university of alicante, university of amsterdam, and user partners bbc and deutsche welle (dw). @xcite the aim is to significantly improve the robustness and applicability of neural machine translation (nmt) for low-resourced language pairs and domains. this is in particular important because machine translation (mt) is increasingly used as a technology for supporting communication in a globalized world. the two international broadcasters participating in gourmet are faced with the need to use mt to support their editorial work, especially for languages for which such tools are currently hard to find or lack quality. the main objectives of the project are: the work is built around three use cases. the first is global content creation, where we use mt in multilingual content production, with editorial control. the second use case is media monitoring for low-resourced and especially strategically important languages. the third use case focuses on a specific topic, and the health sector, in particular covid, was selected for this purpose, fitting the news requirements over the past two years. the objective in the last use case is to apply transfer learning between topical domains. ## languages covered mt models were selected for sixteen lowresourced languages, jointly by user and technology partners and developed in different phases of the project. these languages are: amharic, bulgarian, burmese, gujarati, hausa, igbo, kyrgyz, macedonian, pashto, serbian, swahili, tamil, tigrinya. turkish, urdu, and yorubaall of them from and into english. pashto was selected as a "surprise language" and developed as a special case upon request in a period of two months. different factors were taken into account for the selection process, including strategic importance for the news partners, proximity of languages, research interest and complexity for the development of the models. ## research and development of the models each selected language was assigned to one technology partner, who developed the model. different methods were used among the consortium, allowing a comparison and evaluation of pros and cons of each development method, encouraging enhancement of processes and exchange among research partners. research was done as to the availability of data. data was gathered for each language, from external sources and user partner content. bilingual datasets were established and manually annotated by editors from bbc and/or dw in terms of their level of equivalence. novel approaches were used to enhance the results. one such approach is the multi-task learning data augmentation (mtl da), in which we generate additional parallel sentences which, despite being completely unlikely under the data distribution, systematically improve the quality of the resulting nmt system. the output proves to be more robust against domain shift and produce less hallucinations. we also produced a survey covering the state of the art in low-resource mt research. 2 ## evaluation and benchmarking the user partners evaluated the mt models using a customized evaluation process, including direct assessment (by native speakers of the low-resourced languages), gap filling (looking at english-language mt output) and post editing. specific assessment user interfaces (ui) and test sets were developed for this purpose. technical benchmarking provided a comparative analysis of gourmet models with google mt models using bleu-scores and chrf-scores. in addition, user partners benchmarked the mt output from an editorial point of view, including 2 https://arxiv.org/abs/2109.00486 considering the usefulness and adequacy of the respective models in the field and for different purposes (e.g. understanding or multilingual text production). ## applications for the models the models are trialed and implemented in several applications by the user partners in the project. first of all, an open-source gourmet translate tool has been developed as a customized ui for text translation in all gourmet languages. dw has incorporated it in the plain x (semi-)automated translation and subtitling platform and as an application of the selma 4 project. 3 https://bbcnewslabs.co.uk/projects/frank/ 4 https://selma-project.eu/