Tailoring Machine Translation for Scientific Literature through Topic Filtering and Fuzzy Match Augmentation (2025)

Abstract: To enhance the accessibility of scientific literature in multiple languages and facilitate the exchange of information among scholars and a wider audience, there is a need for high-performing specialized machine translation (MT) engines. However, this requires efficient filtering and the use of domain-specific data. In this study, we investigate whether approaches for increasing training data using topic filtering and more efficient use of such data through exploiting fuzzy matches (i.e. similar translations to a given input; FMs) improve translation quality. We apply these techniques both to sequence-to-sequence MT models and off-the-shelf multilingual large language models (LLMs) in three scientific disciplines. Our results suggest that the combination of topic filtering and FM augmentation is an effective strategy for training neural machine translation (NMT) models from scratch, not only surpassing baseline NMT models but also delivering improved translation performance compared to smaller LLMs in terms of the number of parameters. Furthermore, we find that although FM augmentation through in-context learning generally improves LLM translation performance, limited domain-specific datasets can yield results comparable to those achieved with additional multi-domain datasets.

## introduction the use of a lingua franca like english for scholarly communication is, on the one hand, beneficial as it facilitates knowledge dissemination to a certain extent in the international research landscape. on the other hand, it leads to inequalities among researchers (in terms of understanding and writing) and scientific information written in different languages reaches a limited audience @xcite . machine translation (mt) is an important support for mitigating this problem and improving knowledge dissemination. for instance, with the support of mt systems, providing translations of abstracts, keywords, and full articles could become standard practice for research programs spanning multiple languages @xcite . more broadly, adopting translation as a standard practice could improve access to scientific research for scientists, students, educators, policymakers, journalists, and society as a whole @xcite . translating scientific texts is challenging due to specialized terminology, complex syntax, domainspecific discourse, and the fluid boundaries of scientific disciplines @xcite . moreover, these unique characteristics of scientific literature and the limited language resources for training mt systems further complicate the task for such systems. in the translations and open science project @xcite , which we refer to henceforth as taos, custom mt engines were trained for various scientific disciplines. this effort showed that it is challenging to collect parallel training data for scientific disciplines, as many texts are only available in one language (translation is not an activity that is habitually applied in scholarly communication because of disciplinary standards and because there is a shortage of human resources). in this paper, we approach the scarcity of scientifically oriented parallel training data for the @xmath0, and (2) exploiting fuzzy match (fm) augmentation techniques (i.e. leveraging the translation of sentences similar to a given input) to make more efficient use of the available data for nmt models as well as off-the-shelf llms. in this study, we mainly focus on training nmt models from scratch and less to llms for various reasons: llms (the training data of which are typically unknown) may present data leakage and thus suffer from unrepresentative evaluations; they require more substantial computational infrastructure for inference (nmt models can run on cpu, whereas this is far less obvious for llm models); and finally, the answers of instruct variants of llms need some post-processing. therefore, we do not provide a comprehensive comparison between (pretrained) nmt models and llms: while fine-tuning pre-trained nmt models and llms are common and effective, we merely focus on out-of-the-box translation capabilities of llms (i.e. zero-shot) and through in-context learning. ## conclusions and future work developing highly accurate mt systems for specialized scientific disciplines continues to be a significant challenge due to unique textual characteristics and the scarcity of language resources neces-figure 1 : results of the automatic evaluations for the best-performing configuration per model type selected in terms of comet scores (nmt vs. llms), per discipline. the highest metric scores per metric and discipline are highlighted in bold and statistically significant improvements are denoted by * , † , and ‡ , representing @xmath1.05, @xmath2.01, and @xmath3.001, respectively, based on the lowest p values obtained when compared to all other models. sary for building effective mt systems. in this study, we combined two existing methodologies, aiming to tailor mt systems for the scientific domain, namely topic filtering of large, multidomain datasets to extract relevant nmt training data and fm augmentation to utilize the available data more efficiently. to this end, we trained nmt models from scratch and employed four llms to evaluate their zero-shot and in-context learning capabilities. our experiments, which covered three scientific disciplines, namely neuroscience, climatology, and mobility, in the @xmath4, revealed that combining topic filtering with fm augmentation effectively enhances nmt models trained from scratch. however, although fm augmentation via in-context learning proved beneficial for most of the llms tested, the value of additional datasets in this context, regardless of whether they included topic filtering, remained inconclusive. our findings suggest that smaller, discipline-specific datasets could yield comparable results to larger datasets when employed for fm augmentation in this specific setting, while incurring significantly lower computational costs. furthermore, our findings enable a comparison between nmt models trained from scratch and llms (without further fine-tuning) for this task. we demonstrated that specialized nmt models can achieve superior translation performance compared to out-of-the-box llms in this discipline-specific scenario, with these improvements being more pronounced when compared to smaller llms with fewer parameters. therefore, it can be argued that these improvements in translation quality and other benefits, such as reduced inference costs, make nmt systems a viable option for translating scientific literature, particularly when computational resources are limited. however, given the positive correlation we observed between translation performance and the increasing number of llm parameters, our findings suggest that larger llms, even in the absence of further fine-tuning, could deliver better translation performance than such specialized nmt models. in future studies, we will test additional configurations with given datasets, for example, retrieving fms for the test/validation sets only from the discipline-specific datasets while using extra, larger datasets as additional nmt training data and for fm augmentation on the training set. moreover, we will investigate the effectiveness of additionally fine-tuning pre-trained nmt models and llms using the in-domain datasets, with or without fm augmentation (i.e. zero-vs. few-shot settings), as both approaches have been shown to further improve mt performance in previous studies.