## 2 Related Work
In their paper \"Learning With Less: Computational Resources and Less Data for Knowledge Distillation from LLMs\", the authors of [33] introduced (LLKD), a novel method designed to improve the training of smaller language models by leveraging the extensive knowledge of larger and more powerful LLMs. The core challenge they addressed is the scarcity of labeled data for training smaller models, which are preferred for practical applications due to their lower computational demands. More specifically, LLKD enables LLMs to generate pseudolabels for readily available unlabeled data, effectively acting as a teacher to the smaller student models. Their proposed adaptive sample selection method prioritizesdata where the teacher LLM demonstrates high confidence in its labeling and where the student model indicates a high need for information, leading to more efficient and effective knowledge transfer and superior performance even with less training data. The work of [PERSON] et al. [34] is closely related to ours; there, the authors proposed a training procedure that leverages readily available monolingual data along with small and inexpensive dictionaries to pretrain NMT models. They introduced dictionary-preserving byte-pair encoding to better integrate rare dictionary words as well as a new \"cross-entropy difference\" active learning (AL) strategy to intelligently select the most informative sentences for human annotation. Their research demonstrates that this combined approach can significantly improve translation quality even with very small annotation budgets, outperforming conventional AL methods. In this context, application of deep learning algorithms can also be seen in different but related research domains [35] such as optimizing residential electricity costs and reducing CO\({}_{2}\) emissions via renewable energy integration. For instance, [36] developed a deep learning-based system that monitors and manages household electrical appliance usage with the aim of reducing electricity consumption. Their proposed approach demonstrated significant performance improvements, successfully lowering both energy costs and carbon emissions. Similarly, although our research focuses on minimizing computational costs in MT model training through techniques such as KD, our underlying goal aligns with that of [36], namely, reducing energy consumption, albeit in the context of smart grids rather than computational systems. Tiny NMT models usually have benefits [16] such as faster training times, reduced computational requirements, lower energy consumption, and the ability to be deployed on resource-constrained devices without significant performance degradation, leading to significantly lower CO\({}_{2}\) emissions. However, tiny models can see significantly compromised quality compared to larger models. These issues become worse when dealing with low-resource languages, which face problem in building NMT systems due to limited availability of high-quality data. As pointed out above, while large pretrained models such as NLLB can be used to generate soft target labels in order to create distilled training data, these labels are often not of good quality when low-resource languages are involved. The poor quality and noisy translations of large pretrained models can become a critical issue for sequence-level KD, since noisy or inaccurate labels can lead to the propagation of errors and the student model learning incorrect patterns from distilled data. The studies discussed above primarily focus on scenarios involving large fine-tuning datasets, and do not address situations where the teacher has a fixed budget and provides flawed annotations for the unlabeled data (e.g., in case of low-resource languages). In contrast, our work focuses on helping TSPs to operate under limited budgets by leveraging confidence-based filtering to mitigate flawed teacher annotations and reduce training costs without sacrificing translation quality.