Iterative Layer Pruning for Efficient Translation Inference (2025)

Abstract: Large language models (LLMs) have transformed many areas of natural language processing, including machine translation. However, efficient deployment of LLMs remains challenging due to their intensive computational requirements. In this paper, we address this challenge and present our submissions to the Model Compression track at the Conference on Machine Translation (WMT 2025). In our experiments, we investigate iterative layer pruning guided by layer importance analysis. We evaluate this method using the Aya-Expanse-8B model for translation from Czech to German, and from English to Egyptian Arabic. Our approach achieves substantial reductions in model size and inference time, while maintaining the translation quality of the baseline models.

## introduction large language models (llms) have demonstrated powerful capabilities in diverse natural language processing tasks, including translation. however, llms are often computationally intensive, making them impractical to deploy in real-world settings with limited resources. to enhance the efficiency of these models, researchers have explored various model compression techniques, aiming to reduce their computational requirements while preserving quality @xcite @xcite @xcite . aya expanse is an open-weight large language model with multilingual capabilities. the wmt 2025 model compression track @xcite required all submissions to be derived from the aya-expanse-8b model. this work focuses on translation from czech to german and from english to egyptian arabic. our experiments build on established work on iterative layer pruning guided by layer importance evaluation @xcite . we apply iterative layer pruning to the baseline model aya-expanse-8b foot_0 which originally consists of 32 layers and 8.03b parameters. this approach incrementally identifies and removes layers with minimal contribution to translation quality, one layer at a time. to this end, we conduct layer importance evaluation by measuring translation performance without each layer. after identifying and removing the least critical layer, we repeat the layer importance evaluation on the remaining layers until reaching our pruning target. the pruned model resulting from this process is then fine-tuned on the news commentary dataset. we have made three submissions; the primary submission is a 24-layer model with 6.28b parameters, and the two contrastive submissions are 20-layer and 16-layer models, with 5.41b and 4.54b parameters, respectively. ## data after layer pruning of the aya-expanse-8b model (cf. section 3), we need to fine-tune the pruned model on medium-sized training data to restore the translation quality of the baseline model. to this end, we use the news commentary dataset foot_1 which consists of news articles and their corresponding translations in several languages, including arabic, english, german, and czech. we start by rule-based filtering of the czech-to-german (ces-deu) news commentary dataset by removing duplicates, segments longer than 200 words, and those whose source/target length ratio is larger than 1.5 times. we also apply language detection with fasttext foot_2 @xcite with a 0.9 threshold. finally, we conduct semantic filtering using the muse model @xcite and sentence-transformers (reimers and gurevych, 2019) with a 0.7 threshold of semantic similarity between the source and target. the ces-deu news commentary dataset includes 250.4k segments before filtering, and 201.3k segments after filtering. 4 eventually, we split the dataset into train and test splits, where the test set includes 500 segments used for both testing and layer importance evaluation. then, we sample 100k of the training data, using 0 as the random seed in both cases. as the english-to-arabic news commentary dataset uses standard arabic, foot_4 we first apply the same rule-based and semantic filtering steps as those we employ while processing the czech-to-german dataset, which result in 84.3k segments. afterwards, we convert standard arabic text segments into egyptian arabic (arz) with gpt-4.1-mini, using the prompt in appendix a, providing a fixed verified example that includes the english source as well as both the standard arabic and egyptian arabic translations. for parameters, we use temperature 0.3 and top-p 1. after completing the generation of the synthetic egyptian arabic translations, we apply rule-based filtering, comparing the generated egyptian arabic text segments to the original english source. finally, we calculate the semantic similarity between the english source and the egyptian arabic target and select the 500 segments with the highest scores (0.91-0.98) for the "test" split, while the "train" split comprises the remaining 83.2k segments. 6 using this synthetic dataset to fine-tune our models yields clear quality gains compared to the baseline models, when evaluated on both the in-domain holdout test dataset (cf. ## iterative layer pruning as previous research demonstrates, iterative layer pruning achieves better quality than middle layer pruning @xcite . in this experimental setup, we apply iterative layer pruning to the aya-expanse-8b baseline model. this approach incrementally identifies and removes layers with minimal contribution to translation quality, one layer at a time. the pruned models resulting from this process are then fine-tuned on the training dataset. furthermore, knowledge distillation data from the teacher model can be added. fine-tuning the pruned model restores most of the baseline model's translation quality. the following points elaborate on the process. ## inference and evaluation for inference, we use greedy generation by disabling the sampling options, and setting the temperature argument to 0. we apply a simple translation prompt: "translate the following text from {source_language} to {target_language}:" to evaluate our systems, we calculated bleu @xcite , chrf++ @xcite , as implemented in the sacrebleu library foot_7 @xcite . for semantic evaluation, we use comet (rei et al., 2020). 9 ## results the process of iterative layer pruning has achieved model compression from 8.03b parameters to 6.28b, 5.41b, and 4.54b parameters, after removing 8, 12, and 16 layers, respectively. moreover, the quality degradation caused by pruning has been mitigated through fine-tuning on medium-sized data (80k-100k) and knowledge distillation. as demonstrated by in terms of inference performance, we observe that using vllm 11 @xcite as an inference engine instead of transformers increases the inference speed by more than four times when 11 https://github.com/vllm-project/vllm conducting the evaluation on one a40 48gb gpu (cf. figure 1 ). moreover, while 4-bit quantization using bitsandbytes @xcite reduces the memory footprint, pruning results in higher inference speed and throughput (cf. ## conclusions and future work in this work, we demonstrated that iterative layer pruning is an effective approach for compressing llms while retaining translation quality. the method relies on layer importance evaluation, followed by fine-tuning on a medium-sized dataset. this iterative layer pruning process reduces the model size and accelerates inference. to ensure reproducibility, we have made our code publicly available. 12 future research directions include investigating adaptive compression approaches that dynamically select appropriate model configurations based on real-time deployment constraints such as memory limits and latency requirements. moreover, we plan to assess our compression methods on a broader range of datasets, including both sentencelevel and document-level data. since aya-expanse is designed to follow textual instructions, exploring retrieval-augmented generation combined with few-shot prompting presents a promising opportunity for enhancing translation performance in compressed models.