## 1 Introduction  
Knowledge distillation (KD) is a popular technique for transferring knowledge from a large NN (teacher) to a typically smaller NN (student) [1, 2]. In this approach, the student model is trained on unlabeled data to mimic the teacher model's output, typically contain ing soft labels generated by the teacher on the same data. In case of an MT task, translations generated by the teacher model are used as soft labels for training the student model.  
Although KD has proven effective in many MT tasks [3, 4], training time and cost tend to increase with the volume of data used to train the student model. This presents a significant challenge for TSPs operating under constrained training budgets. Moreover, the CO\({}_{2}\) emissions produced during model training are typically proportional to the data size. While large state-of-the-art pretrained models such as NLLB [5] can be used to generate soft target labels to create the distilled training data, these labels are often not of good quality when low-resource languages are involved. The poor quality and noisy translations produced by the large pretrained models (e.g., encoder-decoder models such as NLLB) can become a critical issue for SKD, since the noisy labels can propagate errors and the student models can learn incorrect patterns from such distilled data.  
Because training cost usually scales proportionally with the number of instances used for learning, keeping noisy instances in the training data may unnecessarily increase training cost without any performance gain [6]. Moreover, removing instances with low-quality and erroneous labels from the distilled data can prevent the student model from learning incorrect patterns. In this regard, obtaining powerful hardware and relevant data to train neural models and deploying them in production within tight timelines and limited training budgets remains a critical challenge for TSPs seeking to quickly build efficient and high-quality MT systems for their clients.  
In our study, we aim to provide TSPs with an effective low-cost KD training solution for MT, balancing training data size and label quality with the goal of achieving little to no loss in translation quality. We exploit model confidence estimation measures for selecting instances with good-quality soft labels (translations) generated by the teacher model. Translations for which the teacher model exhibits strong confidence should ideally contribute more to the distillation process, while those with lower confidence should have a reduced impact. This research focuses on recommending suitable confidence thresholds that TSPs should use for KD in specific use cases under a limited training budget. In addition, we assess whether this strategy can substantially reduce training costs, particularly in terms of CO\({}_{2}\) emissions. For our investigation, we carried out experiments on the Urdu-to-English language pair, focusing on the following two specific use cases in an industrial translation project operating under constrained training budget conditions:  
* A small amount of bilingual data similar to the project data to be translated are available for training.
* Only limited source-language monolingual data similar to the project data to be translated are available for training.  
We found that the student models trained on the distilled data prepared following our strategy performed comparably or even outperformed the baseline in certain cases, while also significantly reducing carbon emission during the training process. The baseline models were trained following the standard KD training setup.  
The rest of this paper is organized as follows: in Section 2, we describe the related work; in Section 3, we describe the background; our datasets are explained in Section 4; Section 5 details the experimental setups; in Section 6, we discuss our results; finally, Section 7 concludes our work and discusses potential directions for future research.