Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? (2024)

Abstract: Traditionally, success in multilingual machine translation can be attributed to three key factors in training data: large volume, diverse translation directions, and high quality. In the current practice of fine-tuning large language models (LLMs) for translation, we revisit the importance of these factors. We find that LLMs display strong translation capability after being fine-tuned on as few as 32 parallel sentences and that fine-tuning on a single translation direction enables translation in multiple directions. However, the choice of direction is critical: fine-tuning LLMs with only English on the target side can lead to task misinterpretation, which hinders translation into non-English languages. Problems also arise when noisy synthetic data is placed on the target side, especially when the target language is well-represented in LLM pre-training. Yet interestingly, synthesized data in an under-represented language has a less pronounced effect. Our findings suggest that when adapting LLMs to translation, the requirement on data quantity can be eased but careful considerations are still crucial to prevent an LLM from exploiting unintended data biases.

## introduction large language models (llms) have reached new heights in various nlp tasks @xcite @xcite . supervised fine-tuning (sft, @xcite , alternatively, instruction tuning or simply fine-tuning in some literature) further prepares these models for better generalization and reliability in downstream tasks by training on task inputoutput data combined with instructions in natural languages @xcite @xcite . in this research direction, various works have studied the "scaling up" of sft data size, number of languages, etc @xcite . on the other hand, recent papers also embraced the philosophy of "less is more" by achieving strong results with a small set of high-quality training instances, claiming a "superficial alignment hypothesis" @xcite with similar findings by others. this work investigates the role of sft data in aligning llms to machine translation (mt), a cross-lingual generation task with high demands in practical domains. prior research has found finetuning to improve translation performance @xcite and more recent works also integrated continued pre-training with more data to provide further improvement @xcite . for encoder-decoder models, @xcite used little data to enable an english-centric model to translate between any two languages. nonetheless, the feasibility of "less is more" in llm translation fine-tuning is rather under-explored. in translation prompting, researchers have suggested that a model's translation capability can be attributed to the bilingual signals exposed during pre-training @xcite and task recognition in llm layers @xcite , hinting that the translation capability has been picked up during pre-training. a natural question follows: can we put reduced effort into data? from a data efficiency perspective, we squeeze the translation sft data to a mere size of 32 or the translation direction to 1 for multilingual translation, for which we believe llms already possess a strong pre-trained foundation in multilingual understanding and generation. beyond quantity and language diversity, we perform sft on synthesized data via machine translation, which is a common data augmentation practice for under-served languages. to summarize, our analysis is grounded in the task of mt, with "scaling down" in mind. in multiple dimensions-data size ( §3.2), translation direction ( §3.3 and §3.4), and data synthesis ( §3.5)-our findings verify, complement, and refine the existing superficial alignment hypothesis for fine-tuning llms for translation tasks: ## conclusion and future work in this work, we conduct an in-depth analysis of fine-tuning llms for translation. we demonstrate that llms is capable of translating in multiple directions after being fine-tuned with minimal lowquality training data in a single direction. while this suggests pre-trained llms inherently possess multilingual translation capabilities which only need to be unlocked by aligning with the correct task format, we discover pitfalls and lessons in aligning llms; while llms make efforts to adjust to the translation task, they are good at imitating other patterns such as the noise in the parallel data. future work could explore robust training methods that align llms with translation while minimizing the risk of overfitting to low-quality data.