Human-in-the-loop Machine Translation with Large Language Model (2023)

Abstract: The large language model (LLM) has garnered significant attention due to its in-context learning mechanisms and emergent capabilities. The research community has conducted several pilot studies to apply LLMs to machine translation tasks and evaluate their performance from diverse perspectives. However, previous research has primarily focused on the LLM itself and has not explored human intervention in the inference process of LLM. The characteristics of LLM, such as in-context learning and prompt engineering, closely mirror human cognitive abilities in language tasks, offering an intuitive solution for human-in-the-loop generation. In this study, we propose a human-in-the-loop pipeline that guides LLMs to produce customized outputs with revision instructions. The pipeline initiates by prompting the LLM to produce a draft translation, followed by the utilization of automatic retrieval or human feedback as supervision signals to enhance the LLM’s translation through in-context learning. The human-machine interactions generated in this pipeline are also stored in an external database to expand the in-context retrieval database, enabling us to leverage human supervision in an offline setting. We evaluate the proposed pipeline using the GPT-3.5-turbo API on five domain-specific benchmarks for German-English translation. The results demonstrate the effectiveness of the pipeline in tailoring in-domain translations and improving translation performance compared to direct translation instructions. Additionally, we discuss the experimental results from the following perspectives: 1) the effectiveness of different in-context retrieval methods; 2) the construction of a retrieval database under low-resource scenarios; 3) the observed differences across selected domains; 4) the quantitative analysis of sentence-level and word-level statistics; and 5) the qualitative analysis of representative translation cases.

## introduction large language models (llms) have exhibited remarkable proficiency in comprehending natural language prompts @xcite , enabling them to execute vari-ous controllable generation tasks based on human instructions. furthermore, llms can acquire knowledge from limited demonstrations that are relevant to the input data and generate desired outputs through analogy. this paradigm, known as in-context learning (icl) @xcite , represents a significant advancement in prompt engineering and offers insights into adapting llms to downstream tasks without the need for fine-tuning the models. machine translation (mt) serves as a representative sequence-to-sequence task that also requires tailoring models to produce domain-specific translations. traditional approaches to building domain-specific mt models involve fine-tuning pre-trained models with domain data or utilizing domain adaptation techniques to transfer in-domain mt models to out-of-domain requirements. however, these methods are suited for accessible, medium-scale mt models, which may not be suitable for llms. notably, certain llms available through application programming interfaces (api) lack accessible weight matrices. furthermore, optimizing llm parameters with domain data can be expensive under resource-limited scenarios. consequently, current research on llm-based mt predominantly focuses on icl, including in-context selection methods @xcite , in-context prompt engineering @xcite , and the systematic evaluation of llm-based mt @xcite . while these lines of research present empirical studies investigating icl in the mt task, they still exhibit a dearth of exploration in customizing llms for domain-specific needs. given the representative capabilities of llms, which rely on generating outputs based on provided instructions, it is intuitive to leverage llms to refine general mt outputs for different domains. nevertheless, there remain challenges and issues when utilizing icl to adapt llms for domain-specific needs. firstly, icl demonstrations for mt typically comprise source input and target reference, lacking domain features. secondly, the icl retrieval database is usually constructed using separate labeled data, and the retrieved demonstrations fail to capture llms' translation preferences. last but not least, the adaptation of black-box llms does not benefit from parameter optimization, thereby limiting adaptation methods to modifying icl inputs alone. in response to these challenges, we propose integrating llm-specific translation feedback into icl inputs, enabling the model to learn from both relevant input-output pairs and domain preferences. specifically, the proposed pipeline consists of two essential parts: feedback collection, and in-context refinement. to collect llm-specific feedback associated with domains, we first request the llm to produce domain-specific translations and obtain feedback by comparing its translation with a reference translation. the feedback takes the form of a sequence of revision instructions, indicating the necessary edits to transform the llm's translation into the reference translation. ideally, these revision instructions originate from human feedback sources. however, due to resource limitations, we simulate this process and generate synthetic human feedback in this study. subsequently, these translation texts and feedback are stored together in the icl retrieval database. for in-context refinement, when faced with new in-domain translation requests, the pipeline initially prompts the llm to generate a draft translation. it then retrieves similar translation pairs and their revision histories as in-context demonstrations tied to the specific domain. finally, the model refines the draft translation based on the retrieved domainspecific demonstrations. any new human-machine interactions generated within this pipeline are incorporated to expand the in-context retrieval database. overall, the primary concept revolves around enabling the llm to revise its outputs by learning from relevant domain-specific revision feedback. we conduct experiments using the proposed pipeline in the german-english translation direction across five domains, utilizing the gpt-3.5 turbo api as a testbed for the black-box llm. the results demonstrate that the proposed pipeline enhances domain translation performance in selected domains, as indicated by four automated evaluation metrics. to further elu- draft translation: "wie geht es dir?" => how are you doing! final translation: "wie geht es dir?" => how are you doing? you should revise: remove "doing". some revision examples: <@xmath0: "sagen sie, wie geht's dir?" => tell me, how are you! revision: replace "!" with "?" history: "wer schickte es dir?" => who it sent you? revision: remove "it". match score: 0.8 match score: 0.2 … add revision to database large language model feedback sources history database translation feedback store retrive translation feedback store retrive example: cidate the effectiveness of our approach, we discuss the pipeline and results through an ablation study, quantitative analysis, and qualitative analysis. ## methodology we leave the discussion of related work in the appendix a due to page limitation. the overall pipeline is illustrated in figure 1 . the feedback retrieved from a data store aims to provide domain-specific revision demonstrations to llm. furthermore, it can also be jointly used with external human supervision in real-world applications. in addition to the feedback that exists in the database, any newly produced feedback for current text will be also recorded into the database to be used as a candidate for icl retrieval in the future. ## conclusions and future work in this paper, we present an empirical study focused on enhancing the translation capabilities of the llm by integrating concrete feedback within the translation process. our objective is to establish a human-in-the-loop machine translation pipeline, where human feedback plays a pivotal role. to simulate this concept, we utilize an automated feedback method, leveraging the gpt-3.5 api as our testbed, which yields effective results. in the future, our plan is to collect human feedback to create a novel dataset and conduct experiments on the proposed pipeline using this dataset. this approach will enable us to further validate and implement our human-in-the-loop machine translation system in the context of llm, enhancing its practical applicability and performance.