OpusDistillery : A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models (2025)

Abstract: In this work, we introduce OpusDistillery, a novel framework to streamline the Knowledge Distillation (KD) process of multilingual NMT models. OpusDistillery’s main features are the integration of openly available teacher models from OPUS-MT and Hugging Face, comprehensive multilingual support and robust GPU utilization tracking. We describe the tool in detail and discuss the individual contributions of its pipeline components, demonstrating its flexibility for different use cases. OpusDistillery is open-source and released under a permissive license, aiming to facilitate further research and development in the field of multilingual KD for any sequence-to-sequence task. Our code is available at https://github.com/Helsinki-NLP/OpusDistillery.

## introduction neural machine translation (nmt) has continuously improved, offering higher-quality translations and supporting an ever-increasing number of languages. however, these advancements come with significant computational costs. the resources required for both training and, more critically, using these models can be quite expensive. as a response to this trend, there has been a growing effort in the field to optimize these large systems by producing smaller models that are easier to deploy in practical settings. knowledge distillation (kd) @xcite is a compression technique that allows to build such systems. in kd, a powerful large model, referred to as the teacher, is distilled into a more compact model, faster and smaller in size, known as the student, that tries to match the performance of the teacher by mimicking its output. in this work, we introduce opusdistillery, a novel open-source toolkit for performing distillation of open nmt models in multilingual scenarios. we leverage publicly available tools and release our code in our github repository under the mozilla public license 2.0. we intend our pipeline to serve researchers as well as industry players in nmt or any sequence-to-sequence task. ## background and motivation our tool implements both standard sequence-level knowledge distillation (seq-kd) and its enhanced version, interpolated seq-kd. seq-kd, first introduced by @xcite , trains a student model on the sentence-level outputs produced by a teacher model. this process involves two main steps: (1) generating a synthetic dataset by forward translating the source text using the teacher model, and (2) training the student model on this generated data. despite its simplicity, seq-kd has been shown to outperform more sophisticated methods for multilingual nmt @xcite . building on seq-kd, @xcite further introduced sequence-level interpolation. it enhances seq-kd by using beam search to generate multiple translations (k-translations) and selecting the most similar sentence to the ground truth for distillation, based on smoothed sentence-bleu @xcite . this interpolated approach has been demonstrated to surpass the performance of standard seq-kd; however, the ground truth may not always be available, as distillation can also be performed using monolingual data only. the challenge of applying kd in multilingual settings is still underexplored. several studies have attempted to address this task @xcite @xcite @xcite , yet there is no standard framework available. to the best of our knowledge, there exists only one other open toolkit to perform multilingual seq-kd. stopes @xcite is a framework of modular pipelines developed within the nllb project that allows to recreate their distilled models for reproducibility purposes, but provides little flexibility. our motivation for developing opusdistillery is driven by the need to address this limitation. first, our pipeline provides a versatile toolkit that is easy to configure to perform systematic distillation for nmt in any kind of multilingual setting. second, we emphasize the use of external, openly available pre-trained teacher models, similar to the approach in @xcite . we advocate for the reuse of public models as a practical and economical solution. this approach not only leverages the continuous publication of new models in open-source repositories such as hugging face (hf) foot_0 , but also significantly reduces the costs associated with training from scratch. ## the opusdistillery pipeline opusdistillery is an extension of the firefox translation training pipeline (ftt) foot_1 . the ftt tool trains bilingual nmt teacher models and distills them to produce student models. it was originally developed within the bergamot project foot_2 for training efficient nmt models that can run locally in a web browser on cpu. the final student is a quantized model, fast at decoding and ready to be fed to the bergamot-translator application. 4 the pipeline works by feeding a yaml configuration file to snakemake @xcite , a workflow management system that enables the definition of computational pipelines through rules specifying their input and output files. when the expected output files of a particular rule are absent, snakemake systematically backtracks to identify and execute the necessary preceding rules in sequence to produce the required outputs. the tool uses the marian toolkit (junczys-dowmunt et al., 2018) for training and sentencepiece @xcite for segmentation. ## experiments to showcase the versatility and capabilities of the presented pipeline, we conduct a series of experiments. we train multilingual student models up until the student training step, without exporting, to showcase the impact of different components. specifically, we trained student models using the complete pipeline and perform ablation studies by excluding ce filtering, alignment, and both. languages following do and lee (2023), we perform our experiments focusing on selected language groups from and into english. for each group, we distill a student model from multiple teachers. we test three language families paired with english, each of them containing three languages and with diverse linguistic characteristics: data we use the parallel tatoeba translation challenge dataset, sampling up to 10 million sentence pairs per language pair when available. occitan, being a low-resource language, had a smaller dataset of approximately 200k sentences. we applied default cleaning and used the flores-200 development and test sets for evaluation. teacher models for each language pair, we selected the best opus-mt teacher available using the implemented feature of best teacher selection. each teacher model was also used as a backward model for the opposite translation direction for ce scoring. their performance is reported in ## conclusions and future work in this work, we have presented opusdistillery, an end-to-end pipeline to perform systematic multilingual distillation of open nmt models. through our experiments, we demonstrated its effectiveness and versatility by training english-centric models for three distinct language groups using the tatoeba challenge dataset. we explored the individual contributions of the ce filtering and guided alignment steps, revealing that simplifying the pipeline can slightly enhance student model performance. opusdistillery is open source and distributed under a permissive license. we hope that our research benefits the community by enabling them to perform distillation of publicly available models and to contribute to the development of more efficient and accessible language technologies. in future work, we plan to extend the pipeline to better accommodate multilingual scenarios by integrating additional tools, such as employing @xcite and incorporating monolingual data, which is now not implemented. furthermore, we aim to explore the use of large language models (llms) to enhance performance. additionally, we intend to implement alternative distillation strategies, such as word-level distillation @xcite .