Prompting Large Language Models with Human Error Markings for Self-Correcting Machine Translation (2024)

Abstract: While large language models (LLMs) pre-trained on massive amounts of unpaired language data have reached the state-of-the-art in machine translation (MT) of general domain texts, post-editing (PE) is still required to correct errors and to enhance term translation quality in specialized domains. In this paper we present a pilot study of enhancing translation memories (TM) produced by PE (source segments, machine translations, and reference translations, henceforth called PE-TM) for the needs of correct and consistent term translation in technical domains. We investigate a light-weight two-step scenario where at inference time, a human translator marks errors in the first translation step, and in a second step a few similar examples are extracted from the PE-TM to prompt an LLM. Our experiment shows that the additional effort of augmenting translations with human error markings guides the LLM to focus on a correction of the marked errors, yielding consistent improvements over automatic PE (APE) and MT from scratch.

## introduction technical translation at large enterprises involves a large number of translation domains, for which translation memories and terminologies need to be maintained to support multi-domain mt systems and human post-editors in producing contextually adequate and consistent translation of technical terms @xcite . in this paper, we ask if ongoing human post-editing efforts that produce large databases consisting of source segments, machine translations, and reference translations, can be enhanced by light-weight human error markings. this could then be used to teach a translation system a focused self-correction of marked erroneous tokens from similar examples with error markings and corrections found in the pe-tm. such a setup could complement translation memories and terminology databases by up-to-date and domain-specific information in the pe-tm, and be used in a scenario where a user marks errors in mt hypotheses. incontext examples with high source-side similarity are then extracted from the pe-tm to prompt an llm to focus on a correction of the marked error interactively. we present a pilot study where we construct a pe-tm for the it domain, which is augmented by human error markings on machine translations. while for training purposes, error markings for the pe-tm could be obtained by automatic matching against human post-edits, this cannot be done at test time. we envisage a scenario where the error markings in the pe-tm are obtained by direct human annotation, simulating a realistic setup where a user only performs the light-weight task of error marking at test time. such a scenario could be feedback collection in the publishing of raw-mt. raw-mt could be shown to end-users who, if they notice an error in the translation, proceed to annotate tokens in the translation they perceive to be incorrect. the translation would then be flagged for review by a human translator, who then post-edits the translation and publishes their correction. this process results read the english text and the german translation hypothesis and then correct the output. incorrect words are inside of tags '<@xmath0. please use this feedback in your correction. if the hypothesis is already correct, do not make any changes. english: this environment variable can also be used to make sure that other operations are working on uploaded files, as well. hypothesis: @xmath1, um sicherzustellen , dass @xmath2. german: mittels dieser umgebungsvariable kann auch sichergestellt werden, dass auch andere operationen an hochgeladenen dateien arbeiten können. english: some important environment variables used by kde hypothesis: einige @xmath3, die von kde verwendet werden german: einige wichtige umgebungsvariablen, die von kde verwendet werden figure 1 : example of a 1-shot prompt for english-to-german translation. error markings are inside bold faced @xmath4. the demonstration example consists of a source segment in english (in green), a translation hypothesis in german (in blue), and a correction (in red). the test example shows a correction of the translation of "environment variable" from "umweltvariable" into "umgebungsvariable" learned by the llm (in bold-faced red). in the creation of (source, hypothesis, post-edit) triples with annotations for the pe-tm. this pe-tm is used to provide in-context examples for llm correction of annotated translation hypotheses. for example, the end-user who annotates raw-mt could then immediately be shown a new translation that takes the error markings into account. the results of our study show that selecting incontext examples based on similarity of sourceside embeddings and providing error markings on hypotheses lets the llm infer focused corrections of marked errors. furthermore, overall translation quality is improved over few-shot prompt-based translation and over automatic post-editing. an example 1-shot prompt and error-marked output is given in figure 1 . ## prior work the last year has seen a progression of the translation capabilities of decoder-only llms, pre-trained on unpaired language data, from lagging behind supervised systems @xcite to matching their translation quality @xcite , with only 5 examples of high-quality translation data used for in-context learning. however, mt in specialized domains still requires translation postediting in order to correct errors and to enhance term translation quality. @xcite recently showed that very large llms (openai, 2023) can perform zero-shot automatic translation post-editing for general domain data, at the price of hallucinated edits. this makes this setup impractical if high precision in domain-specific translation is key. for these purposes, manually crafted glossaries @xcite , dictionaries extracted in a separate step of unsupervised word-alignment @xcite , or translation memories accessed with fuzzy matching @xcite , have been used to aid prompt-based mt. our approach combines pe-tms with light-weight human error markings, achieving improvements over both ape and mt from scratch. the standard paradigm to incorporate token-level human error markings as learning signal is an adaptation of supervised learning from post-edits (see, for example, @xcite ) by penalizing erroneous tokens and rewarding correct tokens in a weighted maximum-likelihood objective @xcite @xcite @xcite . most approaches are conceptualized as fine-tuning applications, with error markings obtained by automatic matching against human post-edits or by direct human annotation. the approach that is closest to our work is quickedit @xcite . they train a model with separate encoders for source and error-marked hypothesis in order to improve upon the initial hypothesis by avoiding the marked tokens. similar to our approach, quickedit requires error-markings at inference time. while quickedit relies on supervised learning, our approach succeeds in teaching an llm to avoid marked tokens from a few demonstration examples of similar error patterns. more recent work by @xcite successfully uses feedback in form of error type and location that is predicted by a learned error pinpoint model. their work focuses on general domain translation and quality-estimation type feedback, in difference to the focused error markings on technical terms that we are interested in. we plan an extension of our work in the direction of using learned error markings in future work. our work is furthermore related to the more general issue of self-correction capabilities of llms. similar to the findings of @xcite , our work shows that in order to qualify as a correction rather than a mere change, automatic self-correction in llms @xcite needs to be guided by an oracle. in our case, the oracle consists of feedback on the errors in translation outputs of an llm, combined with a few examples of similar errors and their reference translations. ## data and models we collected english and german parallel data from open source software documentation and localization available on opus @xcite , as this data comes closest to our domain of interest. we concatenated data from gnome, kde4, kdedoc, php, and ubuntu to create our data set and filtered them with the following methods: we removed those segments containing fewer than five words or more than 25, those identified by fast-text @xcite as the wrong language, those with more than 20% of characters being nonalphanumeric, and those containing personally identifiable information. of the remaining data, we selected a subset of 1, 500 examples. for the purposes of this experiment, we were interested in models that support prompt-based interaction. furthermore, we are interested in the scenario where users use their judgment to guide a model towards a better translation based upon its original translation. large language models lend themselves well to this interaction because the same model can be used with prompt-based interaction to produce the original translations as well as for pro-viding extra information to aid in correction. these considerations decide in favor of using an llm over a traditional encoder-decoder based model typically used in production scenarios. therefore, we examine if the model that produced the hypothesis, llama 13b @xcite , can leverage feedback to correct its own mistakes. in addition to choosing this model because it supports prompting, it runs on a single gpu foot_0 , and the model will remain available in the future for reproducibility. the llama model was converted to huggingface transformers @xcite format for inference. we found that llama 13b frequently copies hypotheses including the error tags to its output as it was instructed not to make changes if the hypothesis is acceptable as-is. we therefore post-processed outputs by removing tags by regex. additionally, we test gpt-3.5 foot_1 in order to test a model larger than we can locally run. we use the "chatcompletion" api, send the query as the 'user' message, and set the temperature to 0. all hypotheses were generated with the above models using greedy decoding. ## feedback collection in order to simulate the previously proposed scenario of an end-user who annotates raw-mt errors, we turn to paid annotators. we generate translations of english source sentences and provide them with only the source and the hypothesis, as would be the case when getting feedback on raw-mt. these are then paired with references to create our pe-tm. ## results we show results across metrics for llama 13b and gpt-3.5 in llama 13b nominally improves bleu scores over the original hypotheses score (28.92) in all scenarios with mrk in the lead with 30.09, mt in second with 29.83 foot_5 and ape with 29.79. nominal improvements over the original hypotheses are also found according to the ter metric, albeit only for ape and mrk scenarios. the gpt model is already quite capable of finding errors in the hypotheses without error markings and the ape outputs achieve marking edits of 76.70 while mrk has a slight improvement of 78.36. worth noting is the reduction in unmarking edits when prompting gpt with mrk. mrk reduces unmarking edits to 13.90 from 15.85 with ape. this means that indicating specific errors can constrain the number of edits that the gpt model makes. additionally, nominal improvements of bleu and ter scores are found in the ape and mrk scenarios over mt with gpt 3.5 as well. mrk improves bleu to 39.31 from mt's 38.61. in the manual evaluation of marking edits, we found that ape made correct edits 32% of the time on average, while either making incorrect edits or not editing the rest. mrk on the other hand was judged correct 67% of the time on average. agreement in terms of krippendorff's alpha for sentence level ratings of ape is α = 0.82, while for mrk α = 0.55. as ape makes fewer edits overall, it is easier to classify as incorrect or not editing. for mrk there was disagreement on how to handle partial edits or if not all markings were edited, requiring individual judgement by each evaluator. ## conclusion we presented a pilot study to investigate the potential of augmenting a so-called pe-tm resource consisting of sources, machine translations, and human references, with human error markings in order to guide an llm to self-correct marked erroneous term translations. we find that the llm that produced the translation hypotheses identifies its own translations as correct, and therefore does not act on the instructions to correct errors. however, when prompted with error markings, the llm learns to act on them, doubling the number of edits to marked tokens, with nearly 70% of the edits being correct according to a human evaluation. in sum, our pilot study shows that the additional effort of error marking a machine translation at test time allows an llm translation system to learn focused corrections on marked errors from similar examples extracted from a pe-tm, leading to improved translation quality over ape and mt. in future work, we will investigate learned models for error markings. these require larger tms for reliable training of markings estimators, but also bear the promise of improved retrieval augmentation.