On Translating Technical Terminology: A Translation Workflow for Machine-Translated Acronyms (2024)

Abstract: The typical workflow for a professional translator to translate a document from its source language (SL) to a target language (TL) is not always focused on what many language models in natural language processing (NLP) do - predict the next word in a series of words. While high-resource languages like English and French are reported to achieve near human parity using common metrics for measurement such as BLEU and COMET, we find that an important step is being missed: the translation of technical terms, specifically acronyms. Some state-of-the art machine translation systems like Google Translate which are publicly available can be erroneous when dealing with acronyms - as much as 50% in our findings. This article addresses acronym disambiguation for MT systems by proposing an additional step to the SL-TL (FR-EN) translation workflow where we first offer a new acronym corpus for public consumption and then experiment with a search-based thresholding algorithm that achieves nearly 10% increase when compared to Google Translate and OpusMT.

## introduction with the myriad of artificial intelligence tools available for professional translators, it can be hard for translators to select solutions that address their core needs. ideally, translation approaches based on machine learning techniques should improve translator proficiency and achieve higher overall quality. one such approach focuses on technical terminology (tt) where domain-specific terms in the form of acronyms in a source language (sl) must be translated into their target language (tl) counterpart. tt is considered important to translators as it is one of the main sources of error a professional translator might encounter on a daily basis. the importance of tt is further displayed by the lat-est machine translation (mt) workshops @xcite @xcite that stress the importance of correctly addressing terminology issues-including correctness of technical terms. while modern mt systems do not seem to focus on acronym and term disambiguation foot_0 , workshops like the "machine translation using terminologies" workshop 2 @xcite clearly state that they focus on both translation accuracy and consistency. since the dominant metric used (bleu) for most mt approaches does not center so much on terminology expansion with acronyms and other mechanisms, we present in this article a novel method that hones in specifically on the day-to-day work in terminology that a professional translator may encounter, which has not been addressed by most of the recent literature. in this article, we present two main novelties that are based on the translation of acronyms: (1) the introduction of a new corpus made publicly available for others to use and (2) a fact-checking step that is used to verify the combination of a technical term and its acronym (long form (lf) and short form (sf)). we do this for several published articles in the tl, which is english. we aim to show that acronym disambiguation can improve term error rate by reducing the risk of default mt models that generally do not have an acronym approach. our claim is that translators can use this method as a novel verification step in the normal translation pipeline. we also believe that other automated work such as generative mt may be able to include this step as a mechanism for evaluation. to that end, we present the following sequence. first, we introduce relevant work in section 2. second, we describe our motivation and high-level proposal methodology in section 3. we then cover the details of our corpus creation in section 4.4. afterwards, we show the results of our sf/lf method in section 5 and finally we conclude our work in section 6. ## related work in the wmt 2023 shared task on machine translation with terminologies, @xcite emphasize the effectiveness of incorporating terminology dictionaries and respecting domain-specific terminology constraints. the authors also distinguish terminology incorporation from general mt methods. @xcite explore the use of masking to properly isolate and translate specific named entities such as terminology spans. their findings show that masking solves some problems, but relies entirely on the masks being properly aligned. @xcite propose a method for translating rare words such as technical terminology. the method, called dipmt, is a prompting technique that provides an llm with multiple translation choices from a dictionary as well as hints about their meaning for a subset of input words. it outperforms baselines for low resource and out-ofdomain mt. the authors also extract bilingual dictionaries from the training data to assist in this pro-cess. doing so allows for fine-grained control over the use of domain-specific terminology. @xcite stress the importance of taking terminology into account in neural mt and propose metrics to measure mt output consistency with regard to domain constraints. dagan and @xcite propose a system to identify technical terms in a source text as well as their translations. the system uses part-of-speech tagging and word alignment techniques to assist translators during the translation process. @xcite address the issue of translating collocations in a variety of domains. grefenstette (1999) offers an example-based method for dealing with terminology problems in translation as well as other nlp tasks. the method proposed uses search to find the most statistically likely translation of an entire noun phrase. @xcite provide a knowledge-based approach to translation that includes using word-sense disambiguation to semantically derive the meaning of a word before seeking a target translation corresponding to that meaning. skadin ¸š et al. ( 2013 ) demonstrate the use of a cloud-based terminology search system that fully integrates with statistical methods to address the need for domain-specific terms and their integration into neural mt systems. meanwhile, @xcite stress the importance of term verification and consistency in the translation process and propose using external terminological databases to assist in fact checking and correcting domain-specific terminology. ## background and motivation in order to better understand how ineffective acronym disambiguation may be for translators, we investigate the performance of lfs and their sf acronyms within the realm of commercial mt systems. we perform this necessary step in order to confirm our hypothesis that: acronym disambiguation in the current state-of-the-art french mt systems is not being addressed properly. in step 4 consists of the verification process, also known as "fact checking". typically, the translation process for technical terms involves a significant component of researching the meaning of a source language term, identifying multiple target language candidate terms, and finally, proceeding through the n-best list in order and seeking out the use of a chosen term in context in similar target language texts, written by experts in the field in question. 7 according to @xcite , professional translation term verification is done on the basis of observed frequency in a corpus; if enough experts use the selected term in context, it is considered to be valid. domain expertise from professional translation trade unions such as the ata foot_7 point to two or three sources being sufficient to substantiate use of a given term. we replicate that process using the search method below. we implement a boolean retrieval system that contains acronyms extracted from ab3p output on a crawl of arxiv and pubmed along with the long forms they map to and source paper id. if a sufficient number of sources have been found to employ the desired term-acronym pair (in the form cardiopulmonary resuscitation (cpr)), term validation is deemed to be successful and the term pair is returned to the user alongside the list of sources for verification. this re-appropriates the term verification method employed by professional translation agencies in the field (and facilitates verification by a reviewer, who may need to fact check term sources at a later stage). the translation of acronyms is further complicated by non-english languages opting to adopt a better known english acronym alongside a translation of the term. the french translation for "large language model" (grand modèle de langue) is condensed using the english acronym "llm," even though the acronym does not correspond to the first letters of each word. despite this limitation, our search step allows for the verification of such cases, as the pairing of term and acronym is likely to occur in the literature if they have found consensus in the field. thus, verification would succeed and the disambiguation step would not be performed. furthermore, fine tuning on corpora such as pubmed was foregone due to the non-compositionality of many technical terms; boolean search ensures that the term is verified as a fixed unit. while an exact match (e.g. 'rcp' to 'cpr') is the objective of our system, it is important to note that for evaluating the system we distinguish between agreement (an exact match) and verification (verified by a search) as noted: agreement -the candidate sf is an exact match with the gold sf. verification -the candidate sf was found near the lf in at least two published papers in the target language (english). ## results we compare the baselines first in when compared, our proposed technique, which includes search and verification, achieves 9.9% improvement (43.9%) for agreement and 17.8% improvement (32.7%) for verification compared to the baseline when using the opusmt system. google translate scores are also markedly higher, with 8.3% improvement (62.6%) and 13.6% (42.8%), respectively. it is clear that through the use of our proposed system, the acronym resolution is much higher for both agreement and verification. additionally, we illustrate the comparisons in more detail from a precision and recall perspective in ## conclusion professional translators must be well versed in the source and target languages that they are translating. translating technical terminology can be so important that it has been compared to the job of a terminologist @xcite . quality translations will take into account several units of measurement such as fluency, adequacy, and more. however, it has been the case in the past that, more often than not, terminology, specifically the translation of acronyms, is not included as a major improvement to a translator's pipeline. domain-specific standards (ghent ¸ulescu, 2015), nonetheless, have been set such that verification of terminology like acronyms is considered an important step in translation. translators and ai practitioners could benefit highly from the use of a system like the one presented in this article. we believe that our corpus and findings provide sufficient evidence and materials to reproduce a benefit to warrant future work on the topic.