Machine translation, translation errors, and adequacy: S panish- E nglish vs. S panish- R omanian (2023)

Abstract: This paper has two objectives: 1. To analyse the adequacy of using neural machine translation (NMT) for the translation of health information (from Spanish into English and Romanian) used in Spanish public health campaigns; and 2. To compare results considering these two linguistic combinations. Results show that post-editing is essential to improve the quality of the translations for both language combinations since they cannot be used as a primary resource for informing foreign users without post-editing. Moreover, Romanian translations require more post-editing. However, using NMT for informative texts combined with human post-editing can be used as a strategy to benefit from the potential of MT while at the same time ensuring the quality of the public service translations depending on the language combination and on the amount of time allotted for the task.

## introduction within the context of globalisation and crisis situations characterised by the increase of the migrants' percentage in europe, spain's foreign population kept growing in the last decade. in fact, it increased by 182,141 people in the first half of 2022 @xcite . this situation has affected both the general needs for social integration and access to public services (i.e., education, administration, healthcare, or social welfare) and specifically the communication needs to access these rights in several language pairs. on the other hand, as @xcite underline, healthcare providers who needed to be able to inform patients who did not understand spanish were also affected. one of the foreign populations that maintained its percentage in the last decade in spain is the romanian population (see, for example, that it had more than 600.000 people every year since 2008 according to @xcite . in the specific context of the healthcare settings, as one of the rights that the foreign population has, health campaigns are used as a tool to transmit essential information regarding healthcare to the general population, usually focusing on disease prevention or situations that can imply some level of risk. they also include or even specifically address the foreign population, who, due to communication difficulties or cultural differences, may not be aware of the risks they are exposed to and of the prevention measures that they should take. in fact, both linguistic and cultural differences can hinder the dissemination of healthcare-related materials @xcite . on the other hand, technology in general can assist in the provision of (public) services to both the general and the foreign population and facilitates the provision of services that were previously unavailable @xcite . technology is also fundamental in the translation and interpreting (t&i) sector, which is the channel that facilitates communication when the foreign population is involved. this channel is specifically relevant within the public service interpreting and translation (psit) context, that is, fields such as education, administration, social welfare, healthcare, and legal settings @xcite , to which the foreign population has free access. additionally, within the private t&i sector, technologies (especially cat tools and including machine translation (mt) and post-editing) are an essential part of translators' daily work to reduce costs, increase efficiency, and improve productivity @xcite . moreover, if an effective post-editing process is followed, the use of mt or terminology management tools can improve productivity, assist with textual consistency, and ensure better quality @xcite . furthermore, some other advantages should be considered, such as the fact that cat tools allow the translator to store their work so that they can use it again when they need to work with similar texts @xcite . however, the use of technology (especially mt), also generates debates among professionals regarding the quality of the product obtained due to "terminological inconsistencies, false meanings, and a clear lack of syntactic and stylistic systematicity" @xcite . finally, specialists have also been reluctant to incorporate translation tools in migrant support contexts @xcite . considering this context, this paper has two objectives: 1. to analyse the adequacy of using neural machine translation (nmt) for the translation of health information (from spanish into english and romanian) used in spanish public health campaigns. ## classifying translation errors: human and machine translation a translation error can be defined as an inappropriate equivalence @xcite . translation errors are directly related to translation problems @xcite especially because translation problems are seldom found in the translation process @xcite and can be identified in advance. in addition, they are different from translation difficulties since they do not depend on the translator's ability to solve problems as an individual @xcite . translation errors are fundamental when analysing and evaluating translation quality and can be classified based on different criteria. the most frequent categories one can find when defining and analysing translation errors are related to errors concerning either the source text or the target language and the two main phases of translation: comprehension and re-expression @xcite . moreover, the type of translator (human or mt) is another criterion to be considered. in fact, we believe that to identify the types of errors and analyse them correctly we need to be aware of the similarities and differences between the outputs of both types of translators. that is why we show, in false meanings. nonsenses. terminological improprieties. syntactic and lexical calques of the source language. addition. ## methodology the methodology used to obtain information is descriptive and it involves corpus compilation, error tagging using mt and the raw output evaluator tool, as well as counting and analysing translation errors. specifically, we took the following steps: ## results we chose three campaigns to test our hypotheses and show our results and several examples of translation errors have been included by type of error in furthermore, finally, we determined the percentage of translation errors considering the total number of words in the texts we analysed (539 words) and we observed that the percentage was much higher in the spanish-romanian combination than in the case of the spanish-english combination, as shown in ## discussions results show both similarities and differences considering the types of translation errors and the number of errors in the two language combinations involved considering the translations produced specifically in informative texts from health campaigns. first, we observed that mt involved a variety of translation errors in these types of texts (see tables 3 , 4 , and 5). the most common types of translation errors in both language combinations were false meanings, syntactic and lexical calques of the source language, terminological improprieties/lack of precision, grammar mistakes/errors, and the translation of lexicalised metaphors. we only found differences in the case of omissions and repetitions on the one hand (for the spanish-english combination), and spelling mistakes and loans on the other hand (for the spanish-romanian combination). second, if we consider the results obtained for each engine and each language combination, we can underline several differences. the romanian translations had more errors than the english translations in general. moreover, in the case of the english translations, deepl had more errors than gt, especially for terms with no context. on the opposite side, gt had more errors than deepl in the romanian translations, in some cases (false meanings and calques) with approx. 50% more errors in the gt output than in the deepl output. in fact, all the romanian texts included several examples of errors that hindered the correct transmission of the intended meaning: false meanings, syntactic and lexical calques of the source language, and terminological improprieties/imprecise expressions. therefore, although similar types of post-editing are required in both languages for adequacy purposes, there are important differences regarding the number of post-edits needed considering the two language combinations we compared. in this case, our results for informative texts from health campaigns suggest that the romanian translations require much more post-editing than the english translations to achieve adequate results. lastly, lack of coherence was also significant in both languages. third, despite the seriousness of the translation errors found in both language combinations and for both engines, the actual percentage of errors found was low considering the total number of words in the texts. this suggests that, although post-editing is required for adequate results that can be used to inform foreign users in a public setting, at least fragments of the texts translated are mostly functional. this means that the human effort required to apply post-editing strategies is generally low in both cases, especially for the english translations. ## conclusions this study focused on the level of adequacy of the mt produced specifically in health campaigns' information and on the differences considering two language combinations. the results we obtained helped us not only determine the level of adequacy of the translations for this type of texts but also reflect on their implications. thus, if we refer to the transmission of information, the number of errors, and the seriousness of the translation errors, we could state that mt is only relatively adequate in the case of the english translations and not adequate in the case of romanian translations. this verifies hypothesis 1, which suggests that mt alone is not adequate for the translation of informative health texts. moreover, they showed that the number of translation errors and post-edits required is higher (and more varied) for the spanish-romanian combination, thus verifying hypothesis 2. on the other hand, the results also made us reflect on the adequacy of using mt for informative texts combined with human postediting as a strategy to benefit from the potential of mt while at the same time ensuring the quality of the public service translations. this strategy seems to depend on the language combination and on the amount of time allotted for the task. therefore, english translations have a higher level of adequacy and potential since we found fewer translation errors and less error variety, which involves less post-editing time. in turn, romanian translations have a lower level of adequacy since we found more translation errors and more error variety, which involves more post-editing time. in general, considering the low percentage of translation errors in both language combinations, one could argue that these outputs can be defined as functional, considering that the main messages intended in the original texts can be understood, which is the main purpose when using mt. it seems that the translation engines analysed could be used as a starting point in the translation process of informative healthcare texts with adequate postediting strategies and the post-editing would not require a great amount of time from the translator. this process is easier when healthcare informative texts are involved since they tend to be repetitive and nmt can rely on a great amount of information available online especially in language combinations that involve english. however, we still have to keep in mind that, for the time being, mt engines cannot fully render the natural-sounding language that the human translation produces. in fact, as our analysis shows, despite their continuous development, both engines still had difficulties finding adequate translations for idiomatic expressions, metaphors, and even in the translation of everyday language and consistency in both language combinations. therefore, human post-editing is still essential to achieve a completely functional and understandable text that is adequate from the linguistic, social, and medical points of view. ultimately, mt translation will continue to be researched, specifically within the psit context and will continue to improve its quality. this means that the results with psit texts such as the ones analysed in this research will keep improving.