T as T e: Teaching Large Language Models to Translate through Self-Reflection (2024)

Abstract: Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. Techniques like instruction tuning have effectively enhanced the proficiency of LLMs in the downstream task of machine translation. However, the existing approaches fail to yield satisfactory translation outputs that match the quality of supervised neural machine translation (NMT) systems. One plausible explanation for this discrepancy is that the straightforward prompts employed in these methodologies are unable to fully exploit the acquired instruction-following capabilities. To this end, we propose the \textbf{TasTe} framework, which stands for translating through self-reflection. The self-reflection process includes two stages of inference. In the first stage, LLMs are instructed to generate preliminary translations and conduct self-assessments on these translations simultaneously. In the second stage, LLMs are tasked to refine these preliminary translations according to the evaluation results. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods. Our work presents a promising approach to unleash the potential of LLMs and enhance their capabilities in MT. The codes and datasets are open-sourced at https://github.com/YutongWang1216/ReflectionLLMMT.

## introduction large language models (llms) like @xcite have recently demonstrated dramatic performance across a wide range of natural language processing tasks @xcite . their outstanding grasp of syntactic and semantic knowledge positions them as powerful instruments for the enhancement of machine src 但/but 口水鸡/poached chicken in chili sauce 整盒/whole box 是/is 生肉/row meat，没办 法/cannot 吃/eat， 是 否/whether 可 以/can 退款/refund? ref but the poached chicken in chili sauce is just a box of inedible raw chicken, and can i get a refund for it? normal but the whole box of chicken is raw meat, so you can't eat it, can you refund? stage 1 but the whole box is meat, can't eat, can you refund \n [bad] stage 2 but the whole box of spicy chicken is raw meat, so i can't eat it. can i get a refund? some studies recommend the tuning of relatively smaller llms for translation @xcite . instruction tuning of llms with a limited number of high-quality supervised instructions in machine translation tasks yields remarkable results in some instances @xcite @xcite . despite these achievements, these attempts still fail to fully leverage the capacity of llms due to their overly straightforward inference process. unlike supervised nmt models, llms generate translations through language modeling, which contains a more complicated inference process and relies more on inherent linguistic knowledge. studies such as chain-of-thought (cot) reveal that the introduction of intermediate reasoning steps in the inference process significantly increases the reasoning capabilities of language models @xcite . in this paper, we introduce taste, a method that aims at improving the translation performance of llms by instilling the ability to self-reflect on their own outputs. specifically, we segment the llm translation process into two stages of inference. in the first stage, llms are prompted to generate preliminary translations while simultaneously making quality predictions for these translations. in the second stage, we instruct llms to refine these preliminary translations based on the predicted quality levels to produce final candidates. an example of the proposed process can be found in our contributions are summarized as follows: ## related work efforts to enhance the translation performance of llms can be categorized into two research lines: prompt engineering and instruction tuning. prompt engineering aims to design proper prompt templates and introduce prior knowledge or supplementary information to support the inference process. dictionary-based approaches incorporate control hints in the prompt from bilingual or multilingual dictionaries to deal with rare words in source sentences @xcite . @xcite extracts translation-related knowledge, such as topics, by self-prompting to guide the translation process. studies in icl @xcite aim to provide llms with more relevant and high-quality translation exemplars. this approach assists llms in retrieving bilingual knowledge, facilitating the generation of translations of the highest possible quality @xcite . instruction tuning represents an efficient method to enhance the ability of llms to follow natural language instructions and yield outputs that align more closely with human preference in downstream zero-shot tasks @xcite @xcite . jiao et al. (2023) explore several translation instructions to improve the translation performance of llms. zeng et al. (2023) employ examples in comparison to instruct basic translation quality prediction draft refinement translate from chinese to english. 另据塔斯社1月16日报道，曼图罗夫告诉 记者，"车队"系列汽车将从2019年起对 外零售，各种配置的车型都能买到。 ### note: a translation with no errors could be ### response: translate from german to english, and label the translation quality as "good", "medium" or "bad". eine gewisse lautstärke gehört genrebedingt bei einem solchen [...] bei etwa 98 dezibel. ### response: translate from english to german. owning a dog brings with [...] respond to our consultation. ### hint: draft with quality label: [bad] der besitz eines [...] beratung zu reagieren. ### response: elsewhere, according to reports by tass [...] externally from 2019. all configured models will be available. at the "mint mosquito"-rocknacht, according to information [...] is about 98 decibels. [bad] der besitz eines hundes bringt bestimmte verantwortlichkeiten [...] auf unsere konsultation zu reagieren. ## results our main results are shown in there is no significant difference in translation performance observed between two different quality prediction approaches, (•)-qe and (•)-tc. this suggests that both of these approaches effectively aid llms in grasping the quality differences between varying translations. the models trained with fixed embedding layers consistently outperform their counterparts trained with full parameters across all language pairs and both evaluation metrics. we argue that this is because fixing embedding layers during fine-tuning effectively preserves the expressive capability of llms against word distribution biases within the training data. this facilitates the generalization of llms across the word domain, mitigating overfitting and thereby enhancing their capacity to produce robust and diverse translations. model ppl pred.↑ p↑ r↑ f1↑ bloomz -37.10 76.84 70.1 68.2 67.6 llama-2 0.00 80.33 70.5 70.1 69.8 we also train a merged model that handles qe and tc approaches simultaneously, and conduct a comparison of the translation performance across models of different scales. please refer to appendix a.3 and a.4 for more details. ## analysis unless mentioned otherwise, the subsequent experiments are conducted in the fixemb-tc setting. ## conclusion we introduce taste, a novel approach that enables llms to translate through the self-reflection process. our approach allows llms to initially generate a preliminary translation and autonomously assess its quality. subsequently, the translation is refined based on the evaluation results, resulting in the final candidate. our experiments and analyses provide evidence of the effectiveness of taste, as it successfully enhances the translation quality through the refinement process, consistently producing high-quality candidates across various translation directions. furthermore, our findings underscore that llms possess significant potential for the translation quality prediction task. the translation process can leverage this capacity to discern different qualities among translations, leading to the generation of high-quality outcomes.