The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities (2024)

Abstract: Fine-tuning large language models (LLMs) for machine translation has shown improvements in overall translation quality. However, it is unclear what is the impact of fine-tuning on desirable LLM behaviors that are not present in neural machine translation models, such as steerability, inherent document-level translation abilities, and the ability to produce less literal translations. We perform an extensive translation evaluation on the LLaMA and Falcon family of models with model size ranging from 7 billion up to 65 billion parameters.Our results show that while fine-tuning improves the general translation quality of LLMs, several abilities degrade. In particular, we observe a decline in the ability to perform formality steering, to produce technical translations through few-shot examples, and to perform document-level translation. On the other hand, we observe that the model produces less literal translations after fine-tuning on parallel data. We show that by including monolingual data as part of the fine-tuning data we can maintain the abilities while simultaneously enhancing overall translation quality. Our findings emphasize the need for fine-tuning strategies that preserve the benefits of LLMs for machine translation.

## introduction recent work has highlighted a range of qualitative advantages that large language models (llms) hold over neural machine translation (nmt) models. one significant advantage is the controllability of style and language variety which can be achieved through prompting and in-context learning @xcite @xcite . llms also exhibit inherent document-level translation abilities @xcite . another advantage is their ability to produce less literal translations @xcite . finally, llms have been shown to have * work done during an internship at amazon. recent studies have demonstrated that finetuning llms on parallel data further improves their translations as measured by metrics that reflect overall quality (such as comet) @xcite @xcite . however, relying on general translation quality metrics and generic test sets does not fully capture the nuanced abilities of llms in machine translation. this oversight raises questions about the retention of llm-specific advantages -such as controllability, document-level translation proficiency, and the production of less literal translations -after fine-tuning on parallel data. while it is clear that general machine translation quality improves through fine-tuning, there is a risk that llms lose their unique strengths due to catastrophic forgetting @xcite @xcite . determining the extent of this risk and comparing the effect of various fine-tuning strategies in preserving the qualitative benefits of llms remains an important yet unresolved question. we investigate how qualitative advantages of llms change when fine-tuning on parallel data. we consider llama and falcon models, with parameter counts ranging from 7 billion up to 65 billion. the llm properties we investigate are general translation quality, formality steerability, non-literalness in idiom translations, performance on specialized domains, and performance on document-level input which requires contextualisation of ambiguous tokens. we compare two fine-tuning strategies for varying data sizes (89k up to 1.4m) in six translation directions. our main findings and contributions are: our findings highlight the importance of creating fine-tuning approaches that enhance general translation quality while also preserving the distinctive capabilities of llms for machine translation. finetuning llms for mt there are multiple strategies for fine-tuning llms for machine translation. one approach makes use of either a small set of high-quality human-written translations or a set of translation instructions for fine-tuning @xcite @xcite . another line of work makes use of more traditional machine translation data: parallel data from the web, which is orders of magnitude larger compared to what is used in fine-tuning @xcite @xcite . these strategies are focused on improving general machine translation quality, but it remains unclear what happens to other abilities that are relevant for translation. we investigate the effect of fine-tuning on relevant abilities for translation using publicly available models. ## fine-tuning on a mix of monolingual and parallel data having established that fine-tuning on parallel data leads to a decline in various advantages of llms for machine translation, this section delves into strategies to mitigate this degradation. a potential approach to counteract degradation involves incorporating examples of desired behaviors during fine-tuning. for instance, the degradation of few-shot capabilities for domain adaptation can be partially mitigated by including few-shot examples during fine-tuning @xcite . however, our aim is to establish a more general solution that prevents degradation across a broader spectrum of behaviors, without the need to specifically include data for each behavior during fine-tuning. to achieve this, our experiments involve a blend of monolingual and parallel data in the fine-tuning phase. this strategy stems from the understanding that the pretraining on monolingual data contributed to these beneficial phenomena, and our goal is to retain these qualities during fine-tuning. ## conclusion we investigated how fine-tuning on parallel data affects the qualitative advantages of llms for machine translation. while previous research predominantly focused on summary quality metrics like comet, our findings reveal a more complex interplay between fine-tuning and llm capabilities. consistent with prior work, we find that fine-tuning enhances the general translation quality of llms. however, we show that fine-tuning adversely impacts several important qualitative advantages of llms. we observe declines in the abilities of llms to 1) perform formality steering, 2) perform technical translation through few-shot examples, as well as 3) a decrease in their document-level translation capabilities. the ability to produce non-literal translations shows improvement post fine-tuning, likely because the publicly available llms we investigate do not perform strongly on this task to begin with. furthermore, our results indicate that these degradations are more pronounced for larger fine-tuning datasets, even when generic translation quality continues to improve. these trends are consistent across different model scales (7b up to 65b), underscoring the generalizability of our findings. to prevent these degradations, we develop a fine-tuning method tailored for machine translation, that combines monolingual and parallel data. we show that this approach mitigates the degradation of llms' qualitative advantages, thereby preserving their capabilities while improving general translation quality.