Mind the Gap: Diverse NMT Models for Resource-Constrained Environments (2025)

Abstract: We present fast Neural Machine Translation models for 17 diverse languages, developed using Sequence-level Knowledge Distillation. Our selected languages span multiple language families and scripts, including low-resource languages. The distilled models achieve comparable performance while being 10x times faster than transformer-base and 35x times faster than transformer-big architectures. Our experiments reveal that teacher model quality and capacity strongly influence the distillation success, as well as the language script. We also explore the effectiveness of multilingual students. We release publicly our code and models in our Github repository: anonymised.

## introduction neural machine translation (nmt) has seen significant advancements with the advent of large language models (llms; @xcite . although llms often perform exceptionally well on high-resource languages, their performance on low-resource languages lags behind @xcite @xcite . nevertheless, recent advancements suggest that this gap may be narrowing @xcite . despite their high quality performance, llms come with substantial computational costs, requiring significant amount of traning data, high-end hardware and extensive energy consumption @xcite . these limitations make llms unsuitable for many real-world scenarios where resources are constrained, such as on-device translation, low-latency requirements, or environments with privacy concerns. the traditional sequence-to-sequence (seq2seq) transformer architecture @xcite , though not as versatile as llms, offers considerable advantages in terms of computational efficiency. these models can be optimized to run faster, consume less memory, and require fewer resources, making them a practical solution for many nmt applications @xcite . in this work, we leverage knowledge distillation (kd) @xcite to train compact seq2seq nmt models. kd allows the transfer of knowledge from a large, high-performing teacher model to a smaller, more efficient student model. we present fast nmt models for 17 diverse languages with english as the target language. the selected languages vary widely in terms of script, language family, and resource availability, including low-resource languages like north azerbaijani and high-resource languages like hindi. in our experiments, we address the following research questions (rq): rq1: how does the capacity gap affect the distillation quality?, rq2: to what extent does script influence the transfer of knowledge? and rq3: can we train multilingual students effectively?. ## related work we use sequence-level kd (seq-kd, @xcite , which has which has proven to be effective to do kd for nmt @xcite . in seq-kd, the teacher model is used to forward-translate all the sentences in the training data to create a distilled dataset. in the interpolated seq-kd variant, the teacher generates k-candidate translations, selecting the one with the highest smoothed sentence bleu @xcite with the reference. then, the student model is trained on the synthetically generated data. figure 1 illustrates this procedure. in this way, the lightweight student retains much of the teacher's performance while being optimized for speed and efficiency. several studies explore how to build compact nmt models. with the motivation of testing the time-efficiency of nmt systems, a shared task on nmt efficiency was organized for several years within the workshop on neural generation and translation @xcite @xcite . research has focused on various aspects, including compressing multilingual systems @xcite , investigating different architectures for student models @xcite , and understanding the effectiveness of kd @xcite . one widely adopted approach is the thin and deep architecture @xcite , characterized by a deep encoder and a shallow decoder @xcite , which has become a standard for compressing nmt models. we follow that approach in this work. ## methodology next, we describe the selected languages, datasets, tools, and teacher and student architectures used for our experiments. languages the 17 selected languages are listed in ## results tables 3 and 4 summarize the results of our distillation experiments in comet scores for bilingual and multilingual settings, respectively. we report spbleu scores in tables 5 and 6 in the appendix. on average, the students exhibit a drop of 2.9 comet points compared to their teachers. in general, we observe that our students maintain competitive performance, with high scores for several languages, including catalan, galician, hebrew, slovak, and malay. these results indicate that, despite the reduction in model size and complexity, these students still capture a significant portion of the teacher's knowledge. however, for languages like arabic, korean and japanese, the scores drop significantly. for japanese, we expect that our students do not outperform their teachers, due to the capacity limitations of the students when compared to their larger teachers, known as the capacity gap problem @xcite . however, our catalan student achieves a comet score 1.1 point higher than its teacher, correlating with a 90% human agreement that it outputs better translations @xcite . we also find an improved score for vietnamese, slovak and malay, though these improvements were less significant. ## discussion in this section, we address the research questions (rqs) posed in the introduction based on the results of our distillation experiments. rq1: how does the capacity gap between the teacher and student models affect the distillation quality? the capacity gap between the teacher and student models is a critical factor in distillation quality. we find that larger teachers (tf-big) lead to a more significant performance drop, with an average comet reduction of 2.2 compared to tf-base teachers, which exhibit an average of 1.1 comet. this directly correlates with the capacity gap problem: the smaller the gap in model size, the better the distillation. the compression ratios for tf-big teachers are 3.2 times larger, underscoring the complexity of transferring knowledge from a high-capacity teacher to a smaller student. rq2: to what extent does script influence the transfer of knowledge? we compare latin vs. non-latin scripts because english (the target language in all models) is in the latin script. students trained for latin script languages have an average of 1.2 comet, while non-latin script languages have a similar average of 3.5 comet. this difference indicates that script plays a role in the transfer of knowledge during distillation. with a fixed vocabulary size, a shared script between source and target lets sentencepiece build longer, more semantically rich subwords. in contrast, non-latin script languages yield shorter subwords, making knowledge transfer more difficult and reducing translation quality. rq3: can we train multilingual students effectively? the student models for the language families in ## conclusions and future work in this paper, we introduced fast mt models for 17 diverse languages, leveraging interpolated seq-kd to compress large teacher models into more efficient students. our experiments reveal that lowperforming teachers struggle to transfer knowledge effectively. we also demonstrate that the capacity gap between teacher and student models, as well as language script, significantly affect distillation performance. additionally, our results highlight the effectiveness of multilingual distillation for related languages. for future work, we plan to develop student models for additional languages. we also aim to expand our approach by distilling from a broader range of teacher models available on the hugging-face hub foot_4 and to further investigate cross-script knowledge transfer.