Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing (2024)

Abstract: Large language models (LLMs) have demonstrated considerable success in various natural language processing tasks, but open-source LLMs have yet to attain state-of-the-art performance in Neural Machine Translation (NMT). Nevertheless, their significant performance in tasks demanding a broad understanding and contextual processing shows their potential for translation. To exploit these abilities, we investigate using LLMs for MT and explore recent parameter-efficient fine-tuning techniques. Surprisingly, our initial experiments found that fine-tuning with Q-LoRA for translation purposes led to performance improvements in terms of BLEU but degradation in COMET compared to in-context learning. To overcome this, we propose an alternative approach: adapting LLMs as Automatic Post-Editors (APE) rather than direct translators. Building on the ability of the LLM to handle long sequences, we also propose extending our approach to document-level translation. We show that leveraging Low-Rank-Adapter fine-tuning for APE can yield significant improvements across both sentence and document-level metrics while generalizing to out-of-domain data. Most notably, we achieve a state-of-the-art accuracy rate of 88.7% on the ContraPro test set, which assesses the model’s ability to resolve pronoun ambiguities when translating from English to German. Lastly, during manual post-editing for document-level translation, the source sentences are iteratively annotated, which can be used to refine further translations in the document. Here, we demonstrate that leveraging human corrections can significantly reduce the number of edits required for subsequent translations.

## introduction large language models (llms) are currently being explored for many natural language processing tasks such as question answering and dialogue applications @xcite , 2023; @xcite . moreover, they are even shown to achieve or surpass state-of-the-art performance based on traditional methods. this achievement demonstrates their ability to possess general understanding and process long inputs. given these strengths, llms might also be suitable for machine translation (mt) as many of them are also inherently multilingual from being trained on the web. however, llms for mt still remain an underexplored area of research. while there are initial works on using llms for mt at both sentence and document-level @xcite @xcite , the performance of open-source models in the range of 13b parameters still lags behind the current stateof-the-art neural mt (nmt) methods @xcite . nonetheless, closed models such as gpt-4 are on par or better depending on the language pair @xcite . hence, it is necessary to investigate adapting small-scale open-source models for for translation. notably, these methods mainly employ in-context-learning (icl) @xcite as fine-tuning these models often requires a significant amount of computational resources. hence, there might be a barrier to optimally adapting the llms for mt. parameter-efficient techniques for fine-tuning such as low-rank adapters (lora) @xcite were recently proposed to overcome large computational requirements. this enables a new adaptation process for llms. however, whether these techniques are sufficient for successful adaptation and better generalization is still unclear. this work investigates exploiting llms for mt at both sentence and document levels. we initially experiment using icl and parameter-efficient finetuning techniques such as q-lora to use llm for mt and find that adding adapters alone is insufficient and may even lead to degradation in comet (section 4). while full fine-tuning may resolve such discrepancies, it is important to efficiently adapt open-source llms with minimal computational resources. to mitigate this and still exploit the strengths of llms, we propose to adapt them as automatic post-editors (ape) correcting nmt systems hypothesis rather than direct translators. this approach offers several advantages. it introduces modularity, allowing state-of-the-art or customized nmt techniques to be applied independently, followed by llm improvements. additionally, llms can refine the sentence-level nmt systems output and generate consistent and coherent text using their ability to generate fluent and long documents. the cascaded system of nmt and llm offers modularity and can also enable the integration of human feedback. by feeding the llm humancorrected translations from previous sentences in the document, we show that it can leverage this feedback to improve the current sentence's translation. this process can be applied iteratively in practice, sentence by sentence, as the annotator progressively corrects the translated document (see figure 1 for iterative manual post-editing). we summarize our main findings and contributions below: our sentence-level llm ape demonstrates a successful fusion of knowledge from nmt systems and llms, leading to substantial en-hancements in translation quality. importantly, we observe that the llm ape exhibits robustness and can adeptly correct nmt systems, even for test sets from different domains that it was not explicitly trained on. we extend our approach to document-level ape and observe significant improvements in both sentence and document-level translation metrics. notably, we achieve a state-of-the-art accuracy of 88.7% on the contrapro english to german test set, underscoring the effectiveness of ape. ## approach: adapting llm for ape open-source llms may not be as proficient translators as state-of-the-art nmt systems due to no explicit training with large amounts of parallel data. however, llms being trained on the web containing data from several domains possess general knowledge that is lacking in an nmt model. moreover, they are capable of processing significantly longer inputs compared to a standard sentence-level nmt. given their strengths of knowledge and ability to process long sequences, we propose to use them for ape at both sentence and document levels. hence, we first generate translations with nmt and then perform ape with llms. our approach combines the translation capacity of nmt with fluency and understanding of llms. we use a technique similar to @xcite , which combines phrase-based and nmt models. we extend this approach to incorporate llms for ape at both sentence and document levels. we first explain our system's complete pipeline at both text representation levels of sentence and document. then, we describe how we train and create the data for each step in our setup. finally, we explain how human feedback can be easily integrated into our cascaded approach during manual pe. ## experimental setup models: in our proposed approach, we have a sentence-level nmt system that generates an initial hypothesis and an llm which then improves it. nonetheless, we want a strong nmt model to assess the benefits of using llms. furthermore, the model should be efficient to add more latency to the two step approach. therefore, we fine-tune the pre-trained deltalm 2 @xcite (△ lm) for initializing our nmt sentence-level model (refer to appendix c.2 for more details). we chose this 2 we use the △lm base model with 360m parameters model given its size and the performance in the constrained setup in the iwslt evaluation for multilingual track @xcite . we also ablate with nllb @xcite to compare with state-of-the-art nmt models (section 5.3). while for llm, we use the recently open-sourced llama-2-13b-chat-hf @xcite as it is instruction-finetuned and has reasonable compute and memory requirements when adapting with 4bit q-lora @xcite . during training (refer to appendix c.1 for more details), we mask the loss on the prompt, which means that the llm is exclusively trained to predict the reference given the source and hypothesis. datasets & metrics: we primarily focus on translating talks from english to german at a document-level. this choice is based on the large availability of document-level parallel data, the current state of sentence-level nmt quality, and the necessity for contextual information in this translation direction. for training our sentence nmt and posteditor llms, we utilize the must-c v3 corpus @xcite . this corpus aligns well with our objectives, containing parallel data annotated with talk ids for document-level translation. for testing, we report results on three test sets. first, we select a subset of the training corpus with the most contextual phenomena in the speeches. this choice stems from the need for context not always prevalent, and hence, standard tests may not suffice for document-level evaluation. to identify such contextual phenomena, we employ the muda tagger @xcite , which automatically tags words requiring contextual information in the training corpus. we select talks with the highest number of tags for each phenomenon, resulting in 14 talks in our test sets, addressing contextual occurrences related to pronouns, formality, and lexical cohesion. then, we remove the selected talks from our training data and use them for testing. to evaluate the robustness of our approach with out-of-domain data, we use the wmt21 news test set @xcite and the acl dev set from iwslt23 @xcite . although the acl data consists of talks, its content contains terminology and domains that are unlikely to be found in the training data. furthermore, both test sets are annotated at the document level, aligning with our experimental setup. compared to the total sentences, even when accounting for false positives in wmt and acl test sets. therefore, creating a custom test set was essential to reliably evaluate context usage and sentence-level translation quality. we also report scores on the contrapro (@xmath0. regarding metrics, we employ various methods to assess the quality at both the sentence and document levels. our report includes bleu @xcite , chrf2 @xcite using sacre-bleu @xcite , and comet 3 @xcite scores for sentence-level evaluation. to gauge the quality of word prediction using contextual information at the document level, we report precision, recall, and f1 scores for words detected by the muda tagger. while ape with llms seems intuitively advantageous, our first step is to empirically evaluate it against several baselines, including alternatives like in-context-learning (icl) @xcite 3 unbabel/wmt22-comet-da and fine-tuning with lora. to justify the development of a cascaded system with added computational complexity, we assess the following configurations foot_1 : ## automatic post-editing is necessary sentence-level nmt: we fine-tune △lm on the training data at sentence-level in conventional fashion. in-context-learning with llama2 13b: we prompt the model according to @xcite , selecting examples randomly or similar to the current prompt. to find sentences more closely related to our source, we extract sentence embeddings from our training data using sent-bert @xcite and retrieve the nearest neighbors for efficiency using faiss @xcite (llama2 13b icl). llama2 13b + adapters: leveraging the recent advancements in efficient fine-tuning of llms, we fine-tune with adapters using lora (hu et al., 2021) (training and hyper-parameter details can be found in appendix c.1). like the sentence-level nmt, we fine-tune it on all of our training data at the sentence level (llama2 13b mt). zero-shot post-editing with llama2 13b: finally, we consider the case of simple zero-shot pe to evaluate the in-built ability of the model to use the knowledge from another system and compare it to icl where it acts as a direct translator. results for the above setups are reported in furthermore, we find that in the case of icl, the selection strategy is relatively unimportant, with both random and faiss performing similarly as indicated by the scores in appendix b. addition-ally, increasing the number of exemplars in the prompt had a detrimental effect on our setup. moreover, adapting with lora yields the highest bleu and chrf2 scores of 28.92 and 55.9 when compared to setups that rely solely on the llm. however, it's worth noting that comet scores decrease compared to icl. these findings suggest that finetuning with q-lora on extensive parallel data may lead to higher scores in lexical metrics but degradation in comet. whether such a trend is also observed with full fine-tuning demands further exploration, and we do not perform it due to a lack of computational resources. zero-shot ape beats icl across metrics (comet included), unlike lora, showing llms' innate post-editing ability. however, it falls short of sentence-level △ lm. therefore, we propose to train the adapter for ape rather than direct translators to exploit llms. in this section, we evaluate the performance of improving sentence translations with ape. first, we discuss the results of improving translations generated only by △ lm on in-domain test data. then, we analyze the influence of moving away from our training conditions to assess the robustness of the model. we achieve this by first evaluating its performance on out-of-domain test sets and combining it with hypotheses generated by models other than △lm. ## llama2 as document-level post editors another motivation for our approach is to exploit llms ability to process long sequences for doc ape. in this section, we evaluate and analyze the performance of our doc ape model in detail. to gain insights on whether the doc ape with llama2 is beneficial, we compare it against several models. apart from the previously mentioned sentence-level models such as △ lm and llama2 + lora, we also extend them to the document-level by concatenating sentences @xcite . furthermore, we evaluate different decoding strategies and report both sentence and document-level metrics in after tuning on the dev data, we set the llama2 maximum chunk token sizes as 1024 for training and 256 for inference (see figure 2 for more information). this ensured at least 5 preceding sentences for most data, which we found to be reasonable given the computation requirements with large inputs. for △ lm doc2doc, we use a smaller chunk size (128 tokens) due to its limited capacity. concatenating sentences for doc2doc proves insufficient: our analysis reveals that models finetuned with △ lm and llama2 separately at the document level exhibit subpar performance when compared to the sentence-level △ lm across all considered metrics. this limitation likely stems from the scarcity of document-level parallel data, a common occurrence, particularly in the context of low-resource languages. this highlights the inadequacy of concatenation as a standalone approach in practical use cases. navigating the trade-off between sentence and document ape: doc ape models outperform sentence-level on bleu/chrf2 (despite slight comet dip of 0.003 between sent and doc ape), showing promise for document translation. the doc ape models leverage context effectively for pronouns and formality by achieving the best f1 scores of 0.75 and 0.74. however, it is still unclear why the comet score of doc ape model is worse while we observe improvements in all other metrics. impact of decoding strategy: doc ape's different decoding strategies (chunking, windowing) show no clear winner in the sentence or documentlevel metrics. batched sliding window, though computationally expensive, offer no significant advantage. thus, a continuous sliding window or chunking may be preferred for efficiency. however, further research across domains and languages is crucial to comprehensively understand doc2doc decoding strategies. ## conclusion our work highlights llms' potential for ape, significantly boosting nmt at both sentence and document levels. we showed that it enables modularity, deeper text understanding, and document-level quality boosted by llms' massive pretraining. for future work, we consider several research avenues. these include training the adapters on substantially larger volumes of document-level parallel data, assessing various open-source llms, and conducting similar experiments with low-resource languages and domain.