Exploring Human-Like Translation Strategy with Large Language Models (2024)

Abstract: Large language models (LLMs) have demonstrated impressive capabilities in general scenarios, exhibiting a level of aptitude that approaches, in some aspects even surpasses, human-level intelligence. Among their numerous skills, the translation abilities of LLMs have received considerable attention. Compared to typical machine translation that focuses solely on source-to-target mapping, LLM-based translation can potentially mimic the human translation process, which might take preparatory steps to ensure high-quality translation. This work explores this possibility by proposing the MAPS framework, which stands for Multi-Aspect Prompting and Selection. Specifically, we enable LLMs first to analyze the given source sentence and induce three aspects of translation-related knowledge (keywords, topics, and relevant demonstrations) to guide the final translation process. Moreover, we employ a selection mechanism based on quality estimation to filter out noisy and unhelpful knowledge. Both automatic (3 LLMs × 11 directions × 2 automatic metrics) and human evaluation (preference study and MQM) demonstrate the effectiveness of MAPS. Further analysis shows that by mimicking the human translation process, MAPS reduces various translation errors such as hallucination, ambiguity, mistranslation, awkward style, untranslated text, and omission. Source code is available at https://github.com/zwhe99/MAPS-mt.

## introduction large language models (llms) have recently demonstrated remarkable general capabilities across a wide range of tasks, making substantial strides in the field of artificial general intelligence. these capabilities have led to llms exhibiting a certain degree of human-level intelligence, particularly in the areas of language understanding and generation @xcite @xcite . among numerous tasks, translation has emerged as a prominent area where llms have shown impressive capacity and competence @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite . this progress above harkens back to the long-term aspirations and dreams of earlier machine translation research in the 1960s @xcite : can llms employ a translation process similar to human translators? figure 1 illustrates the difference between the processes of machine and human translation. while conventional machine translation is typically a direct source-to-target mapping process, professional human translators tend to take preparatory steps when working with the given source text, including gathering and meticulously analyzing information such as keywords, topics, and relevant example sentences @xcite @xcite . these steps are critical for ensuring highquality translations that accurately capture the nuances of the source material. although recent figure 1 : the difference between machine and human translation in an @xmath0. typical neural machine translation is a source-to-target mapping process, while human translators can take complex steps to ensure the quality and accuracy of the translation. the primary focus of this paper is to explore whether llms can imitate the translation strategies employed by human translators. specifically, we aim to investigate whether llms can effectively preprocess the source text and leverage the relevant knowledge to improve their translations. to this end, we propose a method called maps, which stands for multi-aspect prompting and selection. maps prompts the llms to analyze the source sentence and elicit translation-related knowledge in three aspects: keywords, topics, and relevant demonstrations. this knowledge then guides the llm toward generating more accurate translations. to further enhance translation quality, we also employ a post-selection process to filter out unhelpful knowledge and select the best translation based on reference-free quality estimation (qe). we validate our approach across 11 translation directions (covering high-, mediumand low-resource language pairs) from wmt22 @xcite and 3 llms (text-davinci-003, alpaca, and vicuna). automatic evaluation shows that maps achieves significant improvement over other baselines in terms of comet and bleurt. further analysis emphasizes the importance of the extracted knowledge in resolving hallucination and ambiguity in translation. we also conduct human preference studies and multidimensional quality metrics (mqm) evaluation @xcite which show that maps produces more favorable translations by reducing mistranslation, awkward style, untranslated text, and omission errors. in contrast to other llm-based translation approaches, such as dictionary-based prompting @xcite and in-context learning (icl) @xcite , maps focuses on translating general scenarios without any preconceived assumptions about the domain of translation. as a result, maps does not require the preparation of any external ''datastore'', which might include a meticulously constructed glossary @xcite , dictionary @xcite , or sample pool @xcite , for specific language pairs and domains in advance. in summary, the contributions of this work are detailed as follows: ## maps: multi-aspect prompting and selection in this section, we introduce the maps framework. as depicted in figure 2 , maps consists of three steps-knowledge mining, integration, and selection. when mining the knowledge, the llm operates in a manner akin to a human translator, analyzing the source text and generating background knowledge that is beneficial to translation purposes. the acquired knowledge is integrated as contextual guidance, enabling the llm to produce translation candidates. however, the generated knowledge may contain noise (see §4.3 for further analysis). as a result, a filtering mechanism becomes necessary to select useful knowledge while filtering out unhelpful or noisy ones. ## analysis in this section, we conduct analyses to understand the maps framework. if not otherwise specified, maps comet-qe , text-davinci-003, and wmt22 en-zh are default tested method, model and language pair, respectively. ## conclusion this work introduces maps, a method that enables llms to mimic human translation strategy for achieving high-quality translation. maps allows llms to take preparatory steps before translation. specifically, llms analyze the given source text and generate three aspects of translation-related knowledge: keywords, topics, and relevant demonstrations. using a filtering mechanism based on quality estimation, the selected knowledge guides the llms' translation process. in experiments with text-davinci-003, alpaca, and vicuna, maps yields significant and consistent improvements across eleven translation directions from wmt22 and exhibits a higher upper bound of candidate selection. human evaluations show that maps provides more favorable translations by reducing mistranslation, awkward style, untranslated text, and omission errors. further analyses show that maps effectively resolves ambiguities and hallucinations in translation. future work includes designing more aspects of translation-related knowledge and better filtering mechanisms to improve the translation capabilities of llms further. another interesting direction is to explore the human-like translation strategy in training llms (e.g., instruction tuning).