P arro T : Translating during Chat using Large Language Models tuned with Human Translation and Feedback (2023)

Abstract: Large language models (LLMs) like ChatGPT have exhibited remarkable abilities on a wide range of natural language processing (NLP) tasks, including various machine translation abilities accomplished during chat. However, these models are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. Therefore, we propose ParroT, a framework to enhance and regulate the translation abilities during chat based on open-source LLMs (e.g., LLaMA), human-written translation and feedback data. Specifically, ParroT reformulates translation data into the instruction-following style, and introduces a “Hint” field for incorporating extra requirements to regulate the translation process. Accordingly, we propose three instruction types for finetuning ParroT models, including translation instruction, contrastive instruction, and error-guided instruction. Experiments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs significantly while error-guided instruction can lead to further improvement, which demonstrates the importance of learning from low-quality translations annotated by humans. We also demonstrate the potential of automatic evaluation tools in providing quality information of translations, when constructing error-guided instructions for directions that lack human annotation data. Please refer to our Github project for more implementation details: https://github.com/wxjiao/ParroT.

## introduction large language models (llms), designed in the instruction-following format, such as chatgpt and @xcite , have garnered considerable interest due to their remarkable abilities in comprehending instructions and generating humanlike responses. these versatile models can efficiently perform a wide range of natural language parrot 有什么问题？ translate the following sentences to english. processing (nlp) tasks within a single architecture, including question answering @xcite ), text summarization @xcite , grammatical error correction @xcite , and machine translation @xcite . consequently, they represent a significant stride toward the realization of artificial general intelligence (agi). machine translation, a quintessential nlp task, faces both challenges and opportunities presented by the emergence of llms. traditional machine translation encompasses several sub-tasks @xcite , such as bilingual translation @xcite , multilingual translation @xcite , terminology translation @xcite , quality estimation @xcite , and automatic postediting @xcite , among others. these tasks are typically addressed by individual models with limited cross-task interaction. however, current llms have the potential to revolutionize this inefficient approach and redefine the machine translation paradigm. on one hand, llms can leverage the benefits of various sub-tasks and seamlessly transition between them using only natural language instructions. for instance, if a user is dissatisfied with a translation result, they can request the llm to refine the translation implicitly (i.e., through automatic post-editing) or explicitly, by imposing constraints on specific entities (i.e., terminology translation). on the other hand, llms are expected to enhance the explainability of machine translation, ultimately leading to further improvements in translation quality. for example, users may want llms to compare two translations of a sentence (i.e., quality estimation) and provide an explanation for the discrepancies (i.e., error analysis), which can then be addressed in a targeted manner by the llm itself. however, superior llms like chatgpt and gpt-4 are only accessible through restricted apis, which creates barriers to new research and advancements in the field. therefore, developing comprehensive machine translation abilities upon open-source llms has become a critical and challenging research problem. in this paper, we propose the parrot framework to enhance and regulate the translation abilities of llms during chat by leveraging existing human-written translation and feedback data. to be compatible with chat, our framework reformulates translation data into the instruction-following style @xcite , and introduces a "hint" field for incorporating extra requirements to guide the translation process. accordingly, we propose three distinct instruction types: (1) translation instruction, that asks llms to generate translations based on source sentences. (2) contrastive instruction, that asks llms to generate the translations of two different systems with the preferred one at first. (3) error-guided instruction, that asks llms to generate the translations with humanannotated errors as the hint. the first instruction guarantees the basic translation ability of llms while the latter two regulate the llms to align with human feedbacks @xcite . we adopt the open-source llama @xcite and bloom (scao et al., 2022) models, and conduct instruction tuning on previous wmt validation data and multidimensional quality metric (mqm) human evaluation data. the resulting parrot models are evaluated on flores subsets and wmt22 test sets. our main findings are summarized as below: ## instruction pool in this section, we introduce the three distinct instruction types: translation instruction, contrastive instruction, and error-guided instruction. the first instruction guarantees the basic translation ability of llms while the latter two regulate the llms to align with human-written translation and feedback. ## conclusion we propose parrot to enhance and regulate the translation abilities during chat based on opensource llms, human-written translation and feedback data. we reformulate translation data into the instruction-following style, and introduce a "hint" field for incorporating extra requirements to regulate the translation process. accordingly, we propose three instruction types for finetuning parrot models, i.e., translation instruction, contrastive instruction, and error-guided instruction. experiments on flores subsets and wmt22 test sets suggest that translation instruction improves the translation performance of vanilla llms significantly while error-guided instruction can lead to further improvement, demonstrating the importance of learning from low-quality translations annotated by humans. while we only use three instruction types in this paper, it is natural to extend parrot to other hints (e.g., entity alignments), which we leave for future exploration.