M a TIAS : Machine Translation to Inform Asylum Seekers (2024)

Abstract: This project aims to develop a multilingual notification system for asylum reception centres in Belgium using machine translation. The system will allow staff to communicate practical messages to residents in their own language. Ethnographically inspired fieldwork is being conducted in reception centres to understand current communication practices and ensure that the technology meets user needs. The quality and suitability of machine translation will be evaluated for three MT systems supporting all target languages. Automatic and manual evaluation methods will be used to assess translation quality, and terms of use, privacy and data protection conditions will be analysed.

## project phases matias is an interdisciplinary project that combines methods from linguistic ethnography and translation technology research. the project is divided into four phases: (1) practice-oriented research comprising ethnographic fieldwork and the evaluation and selection of suitable mt systems; ## ethnographic fieldwork the ethnographic fieldwork at the reception centres aims to (1) understand the current communication practices and to make an inventory of the most common messages (announcements, instructions) in the reception facilities. this inventory, which consists of frequent phrases and their translations, forms one input source of the context-specific translation memory; (2) gain an insight into the necessary preconditions to optimally use the developed technology (e.g. required expertise among users, attitudes and trust among users, receptivity towards this technology, it infrastructure, user security, etc.); and (3) gain an understanding of residents' communication preferences to determine which messaging system is best suited to link to the web platform. so far, the fieldwork has yielded important insights on users' expectations and potential usage obstacles. firstly, the participants have high expectations on translation accuracy. the staff believe the notification system will facilitate their communication practices significantly. residents, on the other hand, expressed their support for the system but would prefer technology that allows two-way messaging. secondly, low literacy in some residents has been identified as a major obstacle for the reception of messages in written form. therefore, we will explore possibilities of using textto-speech settings to enable the read-aloud function on different types of smartphones. finally, the fieldwork data highlights the benefits of incorporating the source language text into the message, rather than simply presenting the translated version. this practice not only serves as an invaluable resource for residents seeking more detailed information from staff, but also provides an excellent opportunity for residents to familiarise themselves with the local language. ## mt evaluation we will test the quality and suitability of three different mt systems that support all target languages: two commercial engines (modernmt and google translate), which both allow for some degree of customisation and one open-source model (meta ai's no language left behind-model). for each of these systems, we will also evaluate the impact of the source language on translation quality, in particular english versus dutch or french for translation into the minority languages. we will use both automatic and manual evaluation methods to assess mt quality. as a first step, mt quality will be assessed using reference-based automatic evaluation metrics such as bleu, ter, chrf, bleurt, bertscore and comet. translation memories linked to www.fedasilinfo.be were obtained from fedasil for all languages except dutch, french, german and spanish, from which we extracted reference translations. in addition, to create a test set, we identified 43 web pages from www.fedasilinfo.be that were most relevant to our project foot_1 and extracted and sentence aligned all textual information for the missing languages. from this set, we collected all sentences for which translations are available in all 14 languages. the resulting test set of 577 sentences will be used for all automatic evaluations. based on the results of the automatic evaluations, a decision will be taken as to which mt systems will be manually evaluated by language experts. in addition to assessing the mt quality, a thorough analysis of the terms of use for the three different mt systems will be conducted to gain a comprehensive understanding of the privacy and data protection conditions. furthermore, factors such as deployment features and the ease of integration into the web platform will be taken into consideration. based on the results obtained from these evaluations, a final decision on which engines to integrate will be made in consultation with fedasil.