## 7 Conclusions  
In this paper, we have employed model-based confidence estimation for choosing pseudo-target labels generated by a teacher model for training student models in an NMT knowledge distillation task. We investigated two translation industry use cases operating under a constrained training budget: (i) scenarios with limited bilingual data, and (ii) scenarios where only source-language monolingual data were available for student training. We conducted experiments for a low-resource translation task (Urdu-to-English translation) both with and without taking the teacher model's confidence measure into account. We considered NLLB-3.3B as our teacher model. Experimental results showed (i) that student models trained on the distilled data prepared using the confidence estimation-based filtering strategy performed on par or in some instances better than the student models trained on the standard distilled data, and (ii) that the filtering strategy can significantly reduce the cost of training the student model in terms of CO2 emissions. The proposed methods can help SMEs or MT users to build compact MT systems and deploy them on resource-limited devices either via the cloud or offline to provide greener translation service that is both low-cost and high-quality. We note that the primary goal of this research was to improve the efficiency of student training rather than to improve the quality or inference speeds of the student models themselves. Our strategy can be employed in any standard KD setup; for instance, if a high-quality compact NMT model is available as the student, then our methods can be employed to reduce training costs and CO2 emissions. We experimented with different threshold values in order to determine the impact of confidence-based data filtering in KD. We found that low threshold values (e.g., first quartile) can allow lower-quality examples to pass through the filter, which can reduce the  
Figure 7: CO2 emissions vs. confidence threshold. The models corresponding to a confidence threshold of 0.0 represent SKD, while the models corresponding to the confidence thresholds of the first (0.1295), second (0.2175), and third (0.3802) quartiles represent CEKD. student model's performance. The key findings from our threshold-based experiments are as follows: (i) when only monolingual data similar to the project data were available, a high confidence threshold (e.g., third quartile) for filtering led to the best KD-based MT systems; (ii) when limited bilingual data were available, an average confidence threshold (e.g., mean) was found to be most effective for KD in MT. In future research, we aim to further fine-tune the threshold parameter. This might involve randomly sampling a set of sentences translated by the teacher model and manually annotating them for quality, either using binary labels (e.g., good vs. bad) or continuous scores via direct assessment. This will make it possible to analyse the relationship between confidence scores and human quality labels, then choose the threshold that best separates high-quality examples from low-quality ones. In addition, we plan to focus on exploring the use of multiple teachers. We also intend to further investigate the extent to which multiple teachers can be beneficial in scenarios involving limited bilingual data and training budgets. Importantly, one of the limitations of this work is that our research focused on a single low-resource translation task. In future work, we plan to explore a larger number of languages, domains, and datasets. This research aimed at developing sustainable and budget-efficient neural networks for low-resource MT in order to provide TSPs with practical solutions for deploying compact MT models that maintain high translation quality while reducing CO2 emissions; however, this study did not address other critical challenges in MT, such as mitigating model bias and reducing hallucinations in translations. In future work, we plan to further investigate these aspects using student-teacher learning frameworks. Investigation, M.Z.