# Self-Retrieval from Distant Contexts for Document-Level Machine Translation

Ziqian Peng, Rachel Bawden, François Yvon

► To cite this version:

Ziqian Peng, Rachel Bawden, François Yvon. Self-Retrieval from Distant Contexts for Document-Level Machine Translation. WMT 2025 - Conference on Machine Translation, Association for Computational Linguistics, Nov 2025, Suzhou, China. hal-05353046

HAL Id: hal-05353046

https://hal.science/hal-05353046v1

Submitted on 7 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Self-Retrieval from Distant Contexts
for Document-Level Machine Translation

Ziqian Peng¹,² and Rachel Bawden² and François Yvon¹

¹Sorbonne Université & CNRS, ISIR, Paris, France

²Inria, Paris, France

{ziqian.peng,francois.yvon}@isir.upmc.fr rachel.bawden@inria.fr

Abstract

Document-level machine translation is a chal-
lenging task, as it requires modeling both short-
range and long-range dependencies to main-
tain the coherence and cohesion of the gen-
erated translation. However, these dependen-
cies are sparse, and most context-augmented
translation systems resort to two equally un-
satisfactory options: either to include maxi-
mally long contexts, hoping that the useful de-
pendencies are not lost in the noise; or to use
limited local contexts, at the risk of missing
relevant information. In this work, we study
a self-retrieval-augmented machine translation
framework (SELF-RAMT), aimed at inform-
ing translation decisions with informative local
and global contexts dynamically extracted from
the source and target texts. We examine the
effectiveness of this method using three large
language models, considering three criteria for
context selection. We carry out experiments on
TED talks as well as parallel scientific articles,
considering three translation directions. Our
results show that integrating distant contexts
with SELF-RAMT improves translation qual-
ity as measured by reference-based scores and
consistency metrics.

# 1 Introduction

Document-level machine translation (DLMT) is a
challenging task, as it requires modeling both short-
range and long-range dependencies to maintain the
coherence and cohesion of the generated transla-
tion. Inter-sentential contexts are indispensable for
the handling of phenomena such as co-reference,
lexical consistency, textual coherence and cohe-
siveness, which continue to be challenging for long
document translation (Bawden et al., 2018; Maruf
et al., 2019; Voita et al., 2019b; Fernandes et al.,
2023). Numerous approaches, reviewed in (Maruf
et al., 2021; Castilho and Knowles, 2024), have
been proposed to integrate these contexts. They
include segment concatenation (Tiedemann and

Scherrer, 2017; Bawden et al., 2018; Sun et al., 2022), architecture adaptation (Miculicich et al., 2018; Yang et al., 2019; Ma et al., 2020), training strategy optimization (Lupo et al., 2022b; Li et al., 2023; Wu et al., 2024), and multi-pass refinement (Voita et al., 2019a; Yu et al., 2020; Koneru et al., 2024). Past work also shows that various sources of contextual information contribute differently to translation quality; the local source and target context is the main resource for handling anaphoric references and word-sense disambiguation information (Bawden et al., 2018; Gete et al., 2022), whereas the global context, especially on the target side, holds information likely to improve coherence and cohesiveness of the full translated document (Pal et al., 2024).

Recent generative models such as Llama3
(Grattafiori et al., 2024) and GPT4 (OpenAI et al.,
2023) can process inputs up to hundreds of thou-
sands of tokens, creating new possibilities for the
inclusion of the whole source text, as well as al-
ready translated target segments, in the transla-
tion context. It however remains an open ques-
tion whether such architectures, relying on the self-
attention-mechanism (Vaswani et al., 2017), are
effectively able to identify relevant long-range de-
pendencies and actually improve DLMT (Wang
et al., 2024). This is because inter-sentential de-
dependencies can be sparsely distributed within a
document, whereas self-attention generates dense
patterns spreading out over the entire past text (Tay
et al., 2023; Liu et al., 2024). Therefore, most ap-
proaches to DLMT still consider a limited context
window size, usually up to 1024 tokens or a fixed
number of sentences.

In order to capture long-distance dependen-
cies without requiring the attention mechanism
to handle overly long contexts, we propose self-
retrieval augmentation for machine translation
(SELF-RAMT), aiming to take into account both
local and global dependencies, regardless of the
document length. In our approach, inter-sentential dependencies are precomputed for the full source document to identify past relevant segments for each translation unit. As soon as they are translated, these segments and their translation become available to inform the subsequent translation choices. In our implementation, which uses large language models (LLMs), such dynamic contexts are taken into account through in-context learning (ICL). Two scenarios are considered: (a) one where in-context examples correspond to correct translations, as in online learning, where the input sentences, once incrementally post-edited by a human translator, become available to revise the model (Álvaro Peris and Casacuberta, 2019),<fnref n="1" /> and (b) a fully automatic setup, with imperfect in-context translations, requiring no human intervention. In this context, our main research questions are as follows: (i) how to best identify and retrieve useful context segments, (ii) what improvement to MT quality do these retrieved contexts bring, and (iii) to what extent distant (as opposed to local) contexts actually enhance translation scores. We compare three criteria for context selection (cosine similarity with respect to LaBSE embeddings (COS), Best Match 25 (BM25) and point-wise mutual information (PMI)) and carry out experiments on three LLMs in three language directions (English to German (EN-DE), French (EN-FR), and Chinese (EN-ZH)), analyzing the impact of contexts, especially distant ones. Experimental data includes both TED talks (Cettolo et al., 2012) and a new dedicated parallel test set, MERSENNE, consisting of scientific articles for the EN-FR direction. Scientific articles offer an interesting use case to study term consistency in long document translation. Our investigation reveals that distant contexts retrieved with PMI provide valuable information that increases translation metric scores as well as term consistency. We make our code and data available.²

## 2 Related Work

**Document-level MT** DLMT research broadly falls into two categories: *Doc2Sent*, which involves translating each sentence individually using intra-document source and/or target context to aid translation, and *Doc2Doc*, which involves translating multiple sentences at once (Popescu-Belis, 2019; Maruf et al., 2021; Castilho and Knowles, 2024).

*Doc2Doc* approaches represent a simple strategy for effective context integration, maintaining better consistency and coherence than *Doc2Sent* methods (Li et al., 2020; Sun et al., 2022). However, *Doc2Doc* methods struggle to process very long sequences of sentences, as relevant information is sparse in global contexts (Lupo et al., 2022a; Wang et al., 2023), which can lead to the degradation of translation quality or omitted sentences (Zhuocheng et al., 2023; Li et al., 2023; Peng et al., 2025). To address this problem, several approaches have been explored, including context-aware attention (Maruf et al., 2019; Zheng et al., 2021; Yang et al., 2023), which selects important contexts according to the attention distribution, and dynamic context selection (Kang et al., 2020), which applies a reward model to identify varying numbers of useful context sentences within the local context, constrained by the complexity of reinforced training.

**LLM-based DLMT** Various LLM-based methods have been proposed for DLMT. Several works explore zero-shot prompting and ICL for *Doc2Doc* MT (Hendy et al., 2023; Karpinska and Iyyer, 2023) and study the best way to train LLMs for DLMT (Xu et al., 2024; Li et al., 2024; Guo et al., 2024; Alves et al., 2024), illustrating the importance of high-quality in-context demonstrations and fine-tuning parallel corpora. LLMs have also been used as post-editors, either through fine-tuning (Koneru et al., 2024; Li et al., 2025; Dong et al., 2025) or prompting for iterative translation refinement (Briakou et al., 2024; Wang et al., 2025a).

When it comes to integrating context, LLMs offer greater flexibility than traditional neural machine translation models. Various multi-aspect prompting techniques have been proposed to enhance the input by incorporating or automatically summarizing relevant information from the context. For instance, DELTA (Wang et al., 2025b) builds a dynamic context for each source sentence, heavily relying on LLM components to extract and assemble relevant information from the available source and target texts (including proper nouns, bilingual summaries of local contexts and relevant past sentences within a predefined context window). However, DELTA is computationally costly (a lot more so than SELF-RAMT), due to the multiple steps required for dynamic context extraction. SENT2SENT++ (Guo et al., 2025) incorporates two types of contexts: a static part, consisting of an au-

<footnote n="1">¹We refer to this scenario as *online in-context learning*.
²https://anr-matos.github.io/resources.</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 1: SELF-RAMT framework. It consists of four steps: I. Define the search domain (e.g. the past contexts); II. Compute contextual relevance scores for segments in the search domain; III. Retrieve relevant contexts according to the ranked scores and IV. Integrate the selected contexts to the inputs then generate the translations.</figcaption></figure>

tomatically generated bilingual summary of the full
source document, and a dynamic part, composed
of the previous source and target sentences. How-
ever, the use of static, automatic summaries (a) has
the effect of potentially changing the words of the
context, which can be detrimental to lexical consis-
tency and (b) does not ensure that the most relevant
information is accessible for each source sentence,
especially as documents increase in length.

**Retrieval-augmented MT** Choosing which con-
text to be included in MT can be seen as a type of
retrieval-augmented MT. In past works, retrieval-
augmented MT systems have mostly been designed
to mimic the use of translation memories by trans-
lators, which has a long history in MT (Kay, 1997).
Recent implementations of this idea for neural mod-
els encode the target side of relevant example(s)
together with the source sentence in an extended
translation context (Gu et al., 2018; Bulte and Tez-
can, 2019; Xia et al., 2019; Xu et al., 2020; He
et al., 2021; Cheng et al., 2022). Variants, relying
on both the source and target sides of the retrieved
example(s) are proposed by Pham et al. (2020) and
Reheman et al. (2023).

LLM-based MT systems seamlessly accommo-
date examples through in-context learning, where
examples of the translation task (the source and tar-
get sides of parallel samples) (Radford et al., 2019)
are inserted into the prompt. The optimal selection
of in-context examples has also been the focus of
recent research (Moslem et al., 2023; Vilar et al.,
2023; Zhang et al., 2023; Bawden and Yvon, 2023;
Agrawal et al., 2023; Cui et al., 2024; Zebaze et al.,
2025), also analyzed by Zaranis et al. (2024) and
Bouthors et al. (2024).

A key difference with SELF-RAMT is that these
methods retrieve examples from external resources,
instead of the input sequence, with the aim to find

similar examples that can be easily edited. Several
retrieval-augmented architectures have also been
proposed, e.g., by Rubin and Berant (2024), to
retrieve relevant contextual information from very
long input documents. These approaches have been
evaluated in language modeling tasks, but, to the
best of our knowledge, have not yet been applied
to MT.

3 Augmenting MT with Self-retrieval

**3.1 A Self-retrieval Framework for MT**

As illustrated in Figure 1, SELF-RAMT involves
translating each segment of an input document X
with relevant contexts retrieved within X. It con-
sists of four steps:

**I. Defining the Search Domain** We consider a Doc2Sent scenario, translating sentences $\mathbf{x}_i$ in a document $X = \langle \mathbf{x}_1 \dots \mathbf{x}_T \rangle$ using previous context sentences $\{\mathbf{x}_j, j < i\}$ in X.

**II. Contextual Relevance Scores** We compute contextual relevance scores $S(\mathbf{x}_i, \mathbf{x}_j)$ of candidate segments $\mathbf{x}_j$ for each $\mathbf{x}_i$, with the aim of improving the consistency and coherence of the resulting translations. Details are in Section 3.2.

**III. Context Retrieval** For each $\mathbf{x}_i$, $\mathbf{x}_j$ is selected as a contextual segment if $S(\mathbf{x}_i, \mathbf{x}_j)$ is among the top $K$ relevance scores. Additionally, $\mathbf{x}_j$ is disregarded if $S(\mathbf{x}_i, \mathbf{x}_j) \le \tau$, where $\tau$ represents the minimum value such that $\mathbf{x}_j$ is relevant to $\mathbf{x}_i$ for score $S$ (see Section 3.2). The resulting list of selected sentences, which we refer to as $C_i$, constitutes a dynamic context containing up to $K$ sentences.

**IV. Context-aware Translation** Contextual sentences selected in step III are included as few-shot demonstrations in the LLM prompt in the order
in which they appear in the original text. We use specific prompts for each LLM. Details about the prompt selection and the decoding process are given in Appendix B.

## 3.2 Context Selection Criteria

Multiple criteria can be used to identify the relevant contextual sentences (reviewed in (Bouthors et al., 2024) for retrieval-based MT). In our approach, contextual relevance is assessed based on source side similarity between segments, which enables us to pre-compute the relevant contexts for all $x_i$ prior to translation. Our hypothesis is that if $x_j, j < i$ is sufficiently similar to $x_i$, then $(x_j, y_j)$ will contain useful information when generating $y_i$. This enables us to vary the retrieval score while keeping the translation infrastructure unchanged. It is therefore simpler than the proposal of Wang et al. (2025b), where contexts are dynamically updated during the generation process. In our experiments, we consider three contextual relevance scores:

**COS** We compute the cosine similarity between the sentence embeddings using LaBSE (Feng et al., 2022). Only positive cosine similarities are taken into account (i.e. $\tau = 0$).

**BM25** We adapt BM25L (Lv and Zhai, 2011), a variant with length normalisation<fnref n="3" /> of the Best Match 25 (BM25) relevance score (Robertson and Zaragoza, 2009), which is a go-to method for retrieving lexically relevant segments in large data stores. Implementation details are in Appendix B.2.

**PMI** To better reflect contextual relevance, we also consider an alternative inspired by the Point-wise Cross-Mutual Information (P-CXMI) (Fernandes et al., 2021, 2023) and the likelihood difference (Shi et al., 2024; Pombal et al., 2024). We identify relevant contexts based on the point-wise mutual information (PMI) between $x_i$ and $x_j$, defined as:

$$ \text{PMI}(\mathbf{x}_i, \mathbf{x}_j) = -\frac{1}{l_i} \sum_{t=1}^{l_i} \log \frac{P_C(\mathbf{x}_{i,t} | \mathbf{x}_{i,<t})}{P_C(\mathbf{x}_{i,t} | \mathbf{x}_{i,<t}, \mathbf{x}_j)}, $$

where $l_i$ is the length of $x_i$. In other words, PMI($x_i, x_j$) measures how much the knowledge of $x_j$ reduces the uncertainty about $x_i$ for some autoregressive language model $P_C$. We disregard $x_j$ if PMI($x_i, x_j$) $\le \tau$ with $\tau = 0$.

**Baseline** We compare these relevance scores to four baselines: (i) Zero-shot, vanilla sentence-level MT, reflecting the basic non-contextual MT ability of LLMs, (ii) Past-$K$, where the local context is composed of $K$ previous sentences, (iii) Random-$K$, where we randomly select $K$ past sentences, and (iv) Indep-$K$ ($K$ independent examples generated by an LLM, the same for all sentences). More details are given in Appendix B.

# 4 Experimental Settings

## 4.1 Datasets

Our experiments rely on two test sets described below.

**IWSLT** Following Wang et al. (2025b) and Guo et al. (2025), we take the test sets of IWSLT2017⁴ (Cettolo et al., 2012) as our test sets, for three translation directions: English to German (DE), French (FR) and Chinese (ZH), with respectively 10, 12 and 12 TED talks.

**MERSENNE** Due to the scarcity of long document-level data, we curated a set of 23 published scientific articles and their translations for the EN-FR language pair.<fnref n="5" /> These articles are segmented into sentences and aligned into parallel articles. We refer to this test set as MERSENNE. More details on data preparation are given in Appendix A.

**Statistics of test sets** Table 1 reports the number of full documents and the number of sentences in our test sets. It also includes the average, minimum, and maximum length of sentences in LLAMA tokens, for the source and target languages. The average number of sentences is 119 for TED talks from IWSLT, and 192 for articles from MERSENNE.

## 4.2 Models and Inference Settings

We evaluate our framework with three open-weight medium-size multilingual LLMs using ICL: Llama3.1-8B-Instruct<fnref n="6" /> (LLAMA) (Grattafiori et al., 2024), EuroLLM-9B-Instruct<fnref n="7" /> (EUROLLM) (Martins et al., 2024, 2025), and Qwen2.5-7B-Instruct<fnref n="8" /> (QWEN) (Qwen et al., 2025). These models do

<footnote n="3">³Our code uses the Python implementation of Lù (2024).</footnote>

<footnote n="4">⁴https://wit3.fbk.eu/2017-01-d</footnote>

<footnote n="5">⁵https://www.centre-mersenne.org/</footnote>

<footnote n="6">⁶https://huggingface.co/meta-llama/Llama-3.</footnote>

<footnote>1-8B-Instruct</footnote>

<footnote n="7">⁷https://huggingface.co/utter-project/</footnote>

<footnote>EuroLLM-9B-Instruct</footnote>

<footnote n="8">⁸https://huggingface.co/Qwen/Qwen2.</footnote>

<footnote>5-7B-Instruct</footnote>
<table><caption>Table 1: Statistics of IWSLT and MERSENNE, including the number of documents (#doc) and sentences (#sent). ‘mean’, ‘min’, and ‘max’ correspond respectively to the average, minimum, or maximum length of sentences, measured in LLAMA tokens.</caption><thead><tr><th rowspan="2"></th><th colspan="3">IWSLT</th><th>MERSENNE</th></tr><tr><th>en-de</th><th>en-fr</th><th>en-zh</th><th>en-fr</th></tr></thead><tbody><tr><th>#doc</th><td>10</td><td>12</td><td>12</td><td>23</td></tr><tr><th>#sent</th><td>1138</td><td>1455</td><td>1459</td><td>4417</td></tr><tr><th>mean</th><td>20/25</td><td>21/26</td><td>20/24</td><td>36/53</td></tr><tr><th>min</th><td>2/2</td><td>2/2</td><td>2/2</td><td>1/1</td></tr><tr><th>max</th><td>106/143</td><td>93/121</td><td>93/117</td><td>256/348</td></tr></tbody></table>

not contain IWSLT nor parallel articles from MERSENNE in their pre-training data. Our experimental pipelines relies on vLLM (Kwon et al., 2023), an efficient framework for text generation. Decoding is performed with a beam width of 5 and a maximum number of new tokens of 256. To determine the impact of *K* (the maximum number of selected contexts), we vary *K* from 0 to 6. Regarding context selection, we compute PMI using LLAMA. For the Indep-K baseline, we generate 6 examples in the style of TED talks again using LLAMA. More details regarding the experimental setup are in Appendix B.

## 4.3 Metrics for DLMT

To evaluate general translation quality, we primarily rely on COMET (Rei et al., 2022) and its document-level variant (d-COMET) (Vernikos et al., 2022), with the reference-based model wmt22-comet-da. We also report BLEU<fnref n="9" /> (Papineni et al., 2002) and SLIDE (Raunak et al., 2024) with wmt22-cometkiwi-da with a window size of 8 sentences and a stride of 6. For lexical consistency, we compute Lexical Translation Consistency Ratio (LTCR) (Lyu et al., 2021; Wang et al., 2025b) for proper nouns annotated using spaCy<fnref n="10" /> and aligned using awesome-align (Dou and Neubig, 2021). We also conduct case studies to examine the effectiveness of SELF-RAMT.

# 5 Examining Context Selection Strategies

In this section, we aim to answer the following questions: (a) how effective are the context selection scores to identify relevant contexts? and (b) how similar are the retrieved segments when

<table><caption>Table 2: Extraction error rate on MERGEDTED using COS, PMI, and BM25, for *K* from 1 to 6, computed on the source (EN, left) or target texts (FR, right).</caption><thead><tr><th rowspan="2">K</th><th rowspan="2">rand.</th><th colspan="3">EN</th><th colspan="3">FR</th></tr><tr><th>COS</th><th>PMI</th><th>BM25</th><th>COS</th><th>PMI</th><th>BM25</th></tr></thead><tbody><tr><td>1</td><td>0.52</td><td>0.12</td><td>0.01</td><td>0.22</td><td>0.09</td><td>0.10</td><td>0.32</td></tr><tr><td>2</td><td>0.52</td><td>0.17</td><td>0.09</td><td>0.26</td><td>0.15</td><td>0.10</td><td>0.30</td></tr><tr><td>3</td><td>0.52</td><td>0.19</td><td>0.11</td><td>0.28</td><td>0.21</td><td>0.11</td><td>0.31</td></tr><tr><td>4</td><td>0.52</td><td>0.23</td><td>0.10</td><td>0.28</td><td>0.23</td><td>0.11</td><td>0.34</td></tr><tr><td>5</td><td>0.51</td><td>0.24</td><td>0.10</td><td>0.29</td><td>0.25</td><td>0.12</td><td>0.34</td></tr><tr><td>6</td><td>0.51</td><td>0.25</td><td>0.10</td><td>0.29</td><td>0.25</td><td>0.13</td><td>0.36</td></tr></tbody></table>

<table><caption>Table 3: Cover rate on MERGEDTED using COS, PMI, and BM25, for *K* from 1 to 6.</caption><thead><tr><th>K</th><th>rand.</th><th>COS</th><th>PMI</th><th>BM25</th></tr></thead><tbody><tr><td>1</td><td>0.03</td><td>0.39</td><td>0.54</td><td>0.43</td></tr><tr><td>2</td><td>0.05</td><td>0.47</td><td>0.52</td><td>0.53</td></tr><tr><td>3</td><td>0.07</td><td>0.60</td><td>0.46</td><td>0.56</td></tr><tr><td>4</td><td>0.08</td><td>0.61</td><td>0.46</td><td>0.58</td></tr><tr><td>5</td><td>0.09</td><td>0.63</td><td>0.55</td><td>0.62</td></tr><tr><td>6</td><td>0.11</td><td>0.64</td><td>0.61</td><td>0.62</td></tr></tbody></table>

retrieval is performed in the source text or in the target text? As discussed above, generating coherent texts ideally requires taking the target context into account. As only the source text is initially available, it is important to verify that source-based retrieval is a reliable substitute for target-based retrieval, simulated using the oracle reference.

**Method** To assess the context selection criteria for their sensitivity to coherence, we challenge their ability to distinguish sentences extracted from the same documents from other noise segments. Starting with a set of document pairs ($X^1, X^2$) both containing *n* sentences in the same language, we randomly shuffle sentences from $X^1$ with those of $X^2$, resulting in a combined document $X^{1,2}$. We then retrieve, from the first $2n-1$ segments of $X^{1,2}$, the *K* most relevant segments for the last sentence ($x_{2n}$), using each relevance score, and compute the extraction error rate *r*, defined as the proportion of selected sentences that do not belong to the same document as $x_{2n}$. For a set of *N* documents {$x_{1,2}^l, l = 1...N$}, from which we retrieve the *K* context sentences {$c_{l,1},..., c_{l,K}$} for each $x_{2n}$, *r* is computed as follows:

$$r = \frac{\sum_{l=1}^{N} \sum_{j=1}^{K} \mathbb{I}(\text{doc}(c_j) \neq \text{doc}(x_{2n}))}{N \times K},$$

where doc(c) returns the document index of its input segment c.

<footnote n="9">⁹We use SacreBLEU (Post, 2018) with signature: nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.4.0. We use the default zh tokenizer for translations into Chinese.</footnote>

<footnote n="10">¹⁰https://spacy.io/usage</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: BLEU (left) and s-COMET (right) scores for IWSLT EN-FR translated using EUROLLM, with source and reference as contexts, for *K* from 0 to 6.</figcaption></figure>

To compare the relevance scores computed using source and target texts, we also compute Kendall's τ (Kendall, 1938, 1945) between the relevance ranking respectively induced on the source and target texts, and average over the *N* documents. Finally, we also report the cover rate, defined as the ratio of context sentences $c_j$ recognized by retrieval using both $X^{1,2}$ and $Y^{1,2}$, which is the reference translation of $X^{1,2}$, among all selected contexts retrieved from $Y^{1,2}$.<fnref n="11" />

**MergedTED** We artificially construct shuffled documents $X^{1,2}$ and their translations $Y^{1,2}$ from the 12 EN-FR talks from IWSLT. We consider the first 30 sentences of each talk as a pseudo-document, and include all 66 possible pairs as $(X^1, X^2)$.¹² We refer to the resulting corpus as MERGEDTED.

**Results** For question (a), Table 2 reports the extraction error rate for COS, PMI and BM25, derived from MERGEDTED. We observe that, with the exception of PMI, error rates are quite high: already for *K* = 1, about 12% (resp. 22%) of the sentences retrieved by COS (resp. BM25) do not belong to the same talk as the focus sentence. In comparison, PMI error rates increase more slowly with the value of *K*. Regarding (b), we note that the error rates computed in the target language (FR) are only slightly higher than in the source (EN). Table 3 reports the cover rates of contextual segments selected in $X^{1,2}$ and in $Y^{1,2}$. Using the source document, the context criteria identify around half of the relevant contexts determined with the reference target document. Regarding the ranking of potential contexts, Kendall's τ between the relevance

scores derived from the source and the reference are 0.55, 0.45 and 0.55 for COS, PMI and BM25 respectively. These results provide an empirical support of our main hypotheses, in particular they confirm that we can effectively perform context selection by only looking at the source side of the input documents, yet identify relevant dependencies on the target side.

# 6 Results and Analyses

## 6.1 MT quality with SELF-RAMT

**The Impact of *K*** To determine the optimal value for *K* and the best relevance criteria, we examined the BLEU and s-COMET scores for *K* from 0 (i.e. Zero-shot) to 6, with pairs of source and reference as contexts. Figure 2 displays representative results with the EN-FR translation of IWSLT using EUROLLM, where the context-augmented translations perform better than Random-*K* and Indep-*K*. Furthermore, compared to s-COMET, BLEU scores distinguish better translations using selected contexts from Past-*K*, indicating that these contexts lead to greater lexical similarity between the translations and the references. For a trade-off between quality and complexity, we take *K* = 3 for the following experiments and analysis.

**Comparing Relevance Scores** We perform context-aware evaluations on IWSLT, reported in Table 4. The results show that all reference-based metrics, including BLEU, s-COMET, and d-COMET, classify PMI as the best or the second-best relevance score for all models and translation directions. In contrast, SLIDE scores are less conclusive, ranking PMI as the top-2 best systems 7 out of 9 times. This suggests that PMI performs better than COS and BM25. LTCR only prefers PMI for EN-FR, while for EN-DE and EN-ZH, baseline methods give higher scores. This highlights a small issue with this metric, when we use the oracle reference as context. Assume that the MT engine translates the first instance of a term *x* as *y*₁, different from the reference version (*y*₂); then, for all the subsequent instances of the same term, we may retrieve the translation of the first instance (*y*₂), making the system more inclined to generate the same translation (*y*₂), which is what we want. This will introduce a discrepancy between the first instance (*y*₁) and the remaining ones, which will be penalized by LTCR. By comparison, the baseline system may appear more consistent.

<footnote n="11">¹¹For the cover rate, we only count segments appearing in the same document as $x_{2n}$.

¹²We consider 10 different shuffled versions for each pair $(X^1, X^2)$, then report the average *r* and Kendall's τ.</footnote>
<table><caption>Table 4: Results for IWSLT (source and reference as context). We mark MT systems that are significantly better than zero-shot in COMET-based scores, for sentences excluding the first 20 ones (△), all sentences (δ), or in both cases (*), with p-value < 0.05. The top two scores are marked in bold (best) and underlined (second-best).</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="5">EUROLLM</th><th colspan="5">LLAMA</th><th colspan="5">QWEN</th></tr><tr><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th></tr></thead><tbody><tr><td rowspan="6">DE</td><td>0-shot</td><td>27.8</td><td>85.4</td><td>75.8</td><td><strong>81.8</strong></td><td><strong>95.8</strong></td><td>24.7</td><td>82.7</td><td>72.3</td><td>79.5</td><td><strong>95.2</strong></td><td>22.8</td><td>81.6</td><td>70.6</td><td>77.2</td><td><strong>91.2</strong></td></tr><tr><td>indeprand.</td><td>28.1</td><td>85.4</td><td>75.8</td><td><strong>81.8</strong></td><td><strong>96.0</strong></td><td>24.4</td><td>82.7</td><td>71.9</td><td>79.5</td><td><strong>95.9</strong></td><td>22.8</td><td>81.2</td><td>69.7</td><td>76.5</td><td><strong>91.8</strong></td></tr><tr><td>past</td><td>28.9</td><td>*85.7</td><td>*76.3</td><td><strong>81.8</strong></td><td><strong>95.9</strong></td><td>25.1</td><td>*83.3</td><td>*73.0</td><td>*79.9</td><td><strong>96.7</strong></td><td>22.6</td><td>*82.1</td><td>71.1</td><td>77.7</td><td><strong>92.8</strong></td></tr><tr><td>COS</td><td>29.7</td><td>*86.0</td><td>*76.8</td><td><strong>△82.0</strong></td><td><strong>95.8</strong></td><td>26.0</td><td>*83.7</td><td>*73.4</td><td>79.9</td><td>93.6</td><td>24.0</td><td>*82.6</td><td>*71.7</td><td>77.7</td><td>90.3</td></tr><tr><td>PMI</td><td><strong>30.2</strong></td><td><strong>*86.1</strong></td><td>*76.9</td><td><strong>81.8</strong></td><td><strong>93.3</strong></td><td><strong>26.8</strong></td><td>*83.9</td><td>*73.7</td><td>*80.0</td><td>94.5</td><td>24.4</td><td>*83.1</td><td>*72.5</td><td><strong>78.0</strong></td><td>88.8</td></tr><tr><td>BM25</td><td><strong>30.1</strong></td><td><strong><u>86.2</u></strong></td><td><strong><u>77.1</u></strong></td><td><strong>81.8</strong></td><td><strong>94.9</strong></td><td><strong>26.7</strong></td><td><strong><u>84.2</u></strong></td><td><strong><u>74.1</u></strong></td><td><strong><u>80.3</u></strong></td><td><strong>92.6</strong></td><td><strong>25.1</strong></td><td><strong><u>83.2</u></strong></td><td><strong><u>72.8</u></strong></td><td>77.8</td><td><strong>87.1</strong></td></tr><tr><td rowspan="6">FR</td><td></td><td>29.5</td><td>*86.1</td><td>*76.8</td><td><strong>81.8</strong></td><td><strong>93.6</strong></td><td>26.6</td><td>*83.8</td><td>*73.6</td><td>79.8</td><td>94.9</td><td><strong>24.9</strong></td><td>*83.0</td><td>*72.4</td><td><strong>77.9</strong></td><td><strong>89.5</strong></td></tr><tr><td>0-shot</td><td>40.1</td><td>86.4</td><td>76.8</td><td>83.3</td><td>88.9</td><td>36.5</td><td>84.4</td><td>74.0</td><td>81.8</td><td>87.2</td><td>34.5</td><td>83.9</td><td>73.2</td><td>80.7</td><td>89.6</td></tr><tr><td>indeprand.</td><td>41.3</td><td>86.4</td><td>77.0</td><td><strong>83.4</strong></td><td><strong>89.9</strong></td><td>36.5</td><td>84.3</td><td>74.0</td><td>81.8</td><td>87.5</td><td>34.8</td><td>84.0</td><td>*73.6</td><td>80.9</td><td>88.7</td></tr><tr><td>past</td><td>41.5</td><td>86.5</td><td>*77.2</td><td>83.3</td><td>88.0</td><td>37.2</td><td>*84.8</td><td>*74.5</td><td><strong>82.0</strong></td><td><strong>88.7</strong></td><td>35.2</td><td>*84.4</td><td>*73.9</td><td>*81.2</td><td>89.6</td></tr><tr><td>COS</td><td>42.4</td><td>*86.7</td><td>*77.6</td><td><strong>83.5</strong></td><td><strong>90.5</strong></td><td>38.0</td><td>*84.9</td><td>*74.8</td><td><strong>82.0</strong></td><td><strong>85.5</strong></td><td>36.5</td><td>*84.5</td><td>*74.3</td><td>*81.4</td><td>88.4</td></tr><tr><td>PMI</td><td><strong>42.8</strong></td><td><strong><u>86.8</u></strong></td><td><strong><u>77.7</u></strong></td><td>83.3</td><td>89.7</td><td><strong>38.6</strong></td><td><strong>85.0</strong></td><td><strong>75.0</strong></td><td><strong><sup>δ</sup>82.1</strong></td><td><strong>86.8</strong></td><td>36.7</td><td>*84.5</td><td>*74.3</td><td><strong>△80.8</strong></td><td>90.0</td></tr><tr><td rowspan="6">ZH</td><td>0-shot</td><td>30.1</td><td>84.4</td><td>73.3</td><td><strong>81.3</strong></td><td><strong>75.5</strong></td><td>28.2</td><td>83.2</td><td>70.6</td><td>79.2</td><td>75.9</td><td>29.2</td><td>83.2</td><td>71.6</td><td>78.6</td><td>79.0</td></tr><tr><td>indeprand.</td><td>30.5</td><td>*84.7</td><td>*73.8</td><td><strong>81.3</strong></td><td><strong>78.9</strong></td><td>29.1</td><td>*83.6</td><td>*71.9</td><td><strong>*79.7</strong></td><td><strong>76.4</strong></td><td>29.8</td><td>*83.7</td><td>*72.5</td><td>*79.5</td><td><strong>81.3</strong></td></tr><tr><td>past</td><td>30.8</td><td>*84.7</td><td>*74.0</td><td><strong>81.2</strong></td><td><strong>78.2</strong></td><td>29.4</td><td>*83.5</td><td>*71.6</td><td>79.1</td><td><strong>76.3</strong></td><td>30.3</td><td>*84.1</td><td>*73.0</td><td>*79.4</td><td><strong>81.5</strong></td></tr><tr><td>cos</td><td>31.6</td><td><strong><u>85.0</u></strong></td><td><strong><u>74.4</u></strong></td><td>81.2</td><td><strong>79.7</strong></td><td>30.6</td><td>*83.9</td><td>*72.4</td><td>79.3</td><td>74.7</td><td>31.5</td><td>*84.6</td><td>*73.7</td><td>*79.4</td><td>78.9</td></tr><tr><td>pmi</td><td>32.0</td><td>*84.9</td><td>*74.3</td><td>80.3</td><td>73.2</td><td>31.4</td><td><strong><u>83.9</u></strong></td><td><strong><u>72.5</u></strong></td><td>79.2</td><td>73.3</td><td>32.0</td><td>*84.5</td><td><strong><u>73.7</u></strong></td><td><strong><u>79.5</u></strong></td><td>75.9</td></tr><tr><td>BM25</td><td><strong>32.4</strong></td><td><strong><u>85.0</u></strong></td><td><strong><u>74.7</u></strong></td><td>81.0</td><td>74.0</td><td><strong>31.6</strong></td><td><strong><u>84.0</u></strong></td><td><strong><u>72.9</u></strong></td><td>79.3</td><td>75.1</td><td><strong>32.2</strong></td><td><strong><u>84.7</u></strong></td><td><strong><u>74.1</u></strong></td><td><strong><u>79.7</u></strong></td><td>75.1</td></tr><tr><td rowspan="6">SRC+REF</td><td></td><td>32.3</td><td><strong><u>85.0</u></strong></td><td><strong><u>74.4</u></strong></td><td>80.6</td><td>73.8</td><td><strong>31.5</strong></td><td><strong><u>83.9</u></strong></td><td><strong>72.4</strong></td><td>78.9</td><td>72.0</td><td><strong>32.1</strong></td><td><strong><u>84.5</u></strong></td><td><strong>73.6</strong></td><td><strong><u>79.5</u></strong></td><td>73.7</td></tr><tr><td>rand.</td><td>55.7</td><td>89.5</td><td>86.6</td><td>74.6</td><td><strong>92.8</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>past</td><td>58.7</td><td>*89.6</td><td>*86.8</td><td><strong><u>74.8</u></strong></td><td><strong>91.7</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>COS</td><td>59.8</td><td>*89.8</td><td>*87.0</td><td><strong><u>74.9</u></strong></td><td><strong>91.8</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>pmi</td><td><strong>60.8</strong></td><td><strong><u>89.9</u></strong></td><td><strong><u>87.2</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>90.9</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>BM25</td><td><strong>61.0</strong></td><td><strong><u>90.0</u></strong></td><td><strong><u>87.3</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>91.5</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="6">SRC+MT</td><td></td><td>60.5</td><td><strong><u>89.9</u></strong></td><td><strong><u>87.2</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>91.2</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>rand.</td><td>55.8</td><td>89.5</td><td>86.7</td><td>*74.7</td><td>91.4</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>past</td><td>55.0</td><td><strong><u>89.6</u></strong></td><td><strong><u>86.8</u></strong></td><td><strong><u>74.8</u></strong></td><td><strong>92.3</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>COS</td><td><strong>56.2</strong></td><td><strong><u>89.6</u></strong></td><td><strong><u>86.9</u></strong></td><td><strong><u>74.8</u></strong></td><td><strong>92.6</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>pmi</td><td><strong>56.3</strong></td><td><strong><u>89.6</u></strong></td><td><strong><u>86.9</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>92.8</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>BM25</td><td><strong>56.2</strong></td><td><strong><u>89.6</u></strong></td><td><strong><u>86.9</u></strong></td><td><strong><u>74.8</u></strong></td><td><strong>92.7</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

<table><caption>Table 5: Results for MERSENNE using EUROLLM. * indicates significant gains as in Table 4.</caption><thead><tr><th>Context</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th></tr></thead><tbody><tr><td>0-shot</td><td>55.7</td><td>89.5</td><td>86.6</td><td>74.6</td><td><strong>92.8</strong></td></tr><tr><td rowspan="6">SRC+REF</td><td>rand.</td><td>58.7</td><td>*89.6</td><td>*86.8</td><td><strong><u>74.8</u></strong></td><td><strong>91.7</strong></td></tr><tr><td>past</td><td>59.8</td><td>*89.8</td><td>*87.0</td><td><strong><u>74.9</u></strong></td><td><strong>91.8</strong></td></tr><tr><td>COS</td><td><strong>60.8</strong></td><td><strong><u>89.9</u></strong></td><td><strong><u>87.2</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>90.9</strong></td></tr><tr><td>pmi</td><td><strong>61.0</strong></td><td><strong><u>90.0</u></strong></td><td><strong><u>87.3</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>91.5</strong></td></tr><tr><td>BM25</td><td>60.5</td><td><strong><u>89.9</u></strong></td><td><strong><u>87.2</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>91.2</strong></td></tr><tr><td rowspan="6">SRC+MT</td><td>rand.</td><td>55.8</td><td>89.5</td><td>86.7</td><td>*74.7</td><td>91.4</td></tr><tr><td>past</td><td>55.0</td><td><strong><u>89.6</u></strong></td><td><strong><u>86.8</u></strong></td><td><strong><u>74.8</u></strong></td><td><strong>92.3</strong></td></tr><tr><td>COS</td><td><strong>56.2</strong></td><td><strong><u>89.6</u></strong></td><td><strong><u>86.9</u></strong></td><td><strong><u>74.8</u></strong></td><td><strong>92.6</strong></td></tr><tr><td>pmi</td><td><strong>56.3</strong></td><td><strong><u>89.6</u></strong></td><td><strong><u>86.9</u></strong></td><td><strong><u>74.9</u></strong></td><td><strong>92.8</strong></td></tr><tr><td>BM25</td><td><strong>56.2</strong></td><td><strong><u>89.6</u></strong></td><td><strong><u>86.9</u></strong></td><td><strong><u>74.8</u></strong></td><td><strong>92.7</strong></td></tr></tbody></table>

Note that this problem does not arise when we retrieve automatic translations, where we see the benefits of SELF-RAMT more clearly.

The evaluation results for MERSENNE and IWSLT translated with reference and automatic translations as context (using EUROLLM) are given in Tables 5 and 6. These results lead to the same conclusion that PMI is a good criterion for retrieving relevant past segments. As the LTCR scores rank the context selection methods differently across test sets, we conduct a follow-up case study presented in Section 6.3, which better highlights the contribution of selected contexts to term consistency. The complete scores for all MT systems are in Tables 11 and 12 in Appendix C.

## 6.2 Analysis of Context Contribution

Distance Contexts are Retrieved Contextual information plays a crucial role in DLMT. To analyze the contribution of relevant contexts (especially distant ones) to the translation quality, we bin sentences according to the distance (in number of sentences) to their most distant selected context. For example, all selected contexts of sentences in the group 0–20 are retrieved from the past 20 sentences, while there are contexts more distant than 64 sentences for members of the group 64–256.

<table><caption>Table 6: Results for IWSLT (source and MT as contexts) using EUROLLM. * indicates significant gains as in Table 4.</caption><thead><tr><th>SRC+MT</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th></tr></thead><tbody><tr><td rowspan="6">DE</td><td>0-shot</td><td>27.8</td><td>85.4</td><td>75.8</td><td>81.8</td><td>95.8</td></tr><tr><td>rand.</td><td><strong>28.5</strong></td><td><strong>*85.7</strong></td><td>*76.2</td><td><strong>82.0</strong></td><td>96.0</td></tr><tr><td>past</td><td><strong>28.5</strong></td><td><strong>*85.7</strong></td><td>*76.4</td><td><strong>82.1</strong></td><td><strong>96.6</strong></td></tr><tr><td>COS</td><td>28.3</td><td><strong><u>85.7</u></strong></td><td><strong>76.2</strong></td><td><strong>82.0</strong></td><td>96.5</td></tr><tr><td>pmi</td><td><strong>28.9</strong></td><td><strong><u>85.9</u></strong></td><td><strong>76.6</strong></td><td><strong>82.1</strong></td><td>95.4</td></tr><tr><td>BM25</td><td>28.3</td><td><strong><u>85.7</u></strong></td><td><strong>76.3</strong></td><td>81.9</td><td>95.2</td></tr><tr><td rowspan="6">FR</td><td>0-shot</td><td>40.1</td><td>86.4</td><td>76.8</td><td>83.3</td><td>88.9</td></tr><tr><td>rand.</td><td>41.2</td><td><strong>86.5</strong></td><td>*77.2</td><td><strong>83.4</strong></td><td>89.2</td></tr><tr><td>past</td><td>41.2</td><td><strong>86.5</strong></td><td>*77.3</td><td><strong>83.4</strong></td><td>87.6</td></tr><tr><td>COS</td><td>41.2</td><td>86.4</td><td>*77.2</td><td><strong>83.4</strong></td><td>92.1</td></tr><tr><td>pmi</td><td><strong>41.8</strong></td><td><strong><u>86.6</u></strong></td><td><strong>77.5</strong></td><td><strong>83.4</strong></td><td>90.9</td></tr><tr><td>BM25</td><td>41.5</td><td>86.4</td><td>77.2</td><td><strong>83.4</strong></td><td><strong>92.4</strong></td></tr><tr><td rowspan="6">ZH</td><td>0-shot</td><td>30.1</td><td>84.4</td><td>73.3</td><td><strong>81.3</strong></td><td>75.5</td></tr><tr><td>rand.</td><td>30.6</td><td><strong><u>84.6</u></strong></td><td>*73.7</td><td><strong>81.2</strong></td><td>83.1</td></tr><tr><td>past</td><td>30.7</td><td><strong>84.6</strong></td><td>*73.6</td><td><strong>81.2</strong></td><td>85.1</td></tr><tr><td>COS</td><td>30.7</td><td>84.5</td><td>*73.6</td><td><strong>81.2</strong></td><td>85.5</td></tr><tr><td>pmi</td><td><strong>30.8</strong></td><td><strong><u>84.7</u></strong></td><td><strong>73.9</strong></td><td><strong>81.2</strong></td><td><strong>90.0</strong></td></tr><tr><td>BM25</td><td>30.2</td><td>84.2</td><td>73.2</td><td>80.1</td><td>88.8</td></tr></tbody></table>

We then measure the ratio between the effective number of contexts occurring within a given interval (e.g., 20 – 40) and the maximal possible number of contexts in that interval.<fnref n="13" /> This analysis

<footnote n="13">¹³Which depends on the sentence position in a document: sentences in the initial paragraphs only have access to a re-</footnote>
<table><caption>Table 7: Distance between the translated sentence and selected contexts, for sentences appearing after the 40th sentence in IWSLT with K = 3. 'ratio' denotes the effective value ('nb') normalized by the number of selected contexts for sentences that have access to the corresponding distance interval.</caption><thead><tr><th rowspan="2">Range</th><th colspan="2">rand.</th><th colspan="2">COS</th><th colspan="2">PMI</th><th colspan="2">BM25</th></tr><tr><th>nb</th><th>ratio</th><th>nb</th><th>ratio</th><th>nb</th><th>ratio</th><th>nb</th><th>ratio</th></tr></thead><tbody><tr><td>0-20</td><td>2066</td><td>0.26</td><td>3322</td><td>0.41</td><td>4575</td><td>0.57</td><td>3432</td><td>0.42</td></tr><tr><td>20-40</td><td>2069</td><td>0.26</td><td>1708</td><td>0.21</td><td>1407</td><td>0.17</td><td>1568</td><td>0.19</td></tr><tr><td>40-80</td><td>2685</td><td>0.34</td><td>2028</td><td>0.25</td><td>1417</td><td>0.18</td><td>2116</td><td>0.27</td></tr><tr><td>80-120</td><td>900</td><td>0.21</td><td>740</td><td>0.17</td><td>498</td><td>0.12</td><td>663</td><td>0.16</td></tr><tr><td>120-256</td><td>356</td><td>0.20</td><td>278</td><td>0.16</td><td>179</td><td>0.10</td><td>297</td><td>0.17</td></tr></tbody></table>

<table><caption>Table 8: Retrieval statistics with respect to the distance between the translated sentence and selected contexts, for sentences appearing after the 20th sentence. Selection is performed with PMI and K = 3, for IWSLT (left) and MERSENNE (right).</caption><thead><tr><th rowspan="2">Range</th><th colspan="3">IWSLT</th><th colspan="2">MERSENNE</th></tr><tr><th>DE</th><th>FR</th><th>ZH</th><th>Range</th><th>FR</th></tr></thead><tbody><tr><td>0-20</td><td>337</td><td>390</td><td>373</td><td>0-20</td><td>966</td></tr><tr><td>20-40</td><td>241</td><td>350</td><td>352</td><td>20-40</td><td>770</td></tr><tr><td>40-64</td><td>125</td><td>223</td><td>236</td><td>40-64</td><td>650</td></tr><tr><td>64-256</td><td>235</td><td>252</td><td>258</td><td>64-128</td><td>995</td></tr><tr><td></td><td></td><td></td><td></td><td>128-320</td><td>570</td></tr></tbody></table>

is performed for the TED talks test set (IWSLT), for K = 3. We exclude from the analysis the first 40 sentences in each document, as the context they can access is limited. The corresponding statistics are in Table 7.

We observe that about half of the retrieved contexts are in the past 20 sentences, while a sizable portion of contexts are chosen in more distant part of the document (in the 20 – 80 range); more remote sentences, with a distance larger than 80 are also frequently selected. There is a clear variance between relevance scores: COS retrieves closer segments on average, whereas PMI and BM25 are more likely to extract more remote sentences.

**Distant Contexts Matter** For each group of sentences, we now compare the translations generated with PMI and with the baseline methods. Table 8 reports the corresponding retrieval statistics for these experiments, where we again group sentences by their position index in the source text.

Translation scores are in Table 9, where we compute the d-COMET difference between baselines and PMI-based retrieval. The upper part of Table 9 shows that, for translations of IWSLT with

<table><caption>Figure 3: Translations of the 72nd sentence of a MERSENNE article, using EUROLLM with MT as target-side context. The correct translations of “hemihedria” are underlined.</caption><tr><td>SRC</td><td>The hemihedria is, moreover, non-superposable.</td></tr><tr><td>REF</td><td>L'hémiédrie est, en outre, non superposable.</td></tr><tr><td>0-shot</td><td>La hemihedrie est, en outre, non superposable.</td></tr><tr><td>Past</td><td>De plus, l'hémimorphie n'est pas superposable.</td></tr><tr><td>PMI</td><td>De plus, l'hémiédrie n'est pas superposable.</td></tr></table>

<table><caption>Figure 4: EN–ZH translations of the 45th sentence from an IWSLT talk, using EUROLLM with MT as target-side context. PMI retrieves relevant contexts for “a match” that corresponds to a specie (see Figure 7 in Appendix C for more details.)</caption><tr><td>SRC:</td><td>And he found a match.</td></tr><tr><td>REF:</td><td>结果找到了一个配对！(a match)</td></tr><tr><td>0-shot:</td><td>他找到了合适的人。(an appropriate person)</td></tr><tr><td>Past:</td><td>他找到了匹配的物品。(a matched object)</td></tr><tr><td>PMI:</td><td>他找到了匹配的物种。(a matched species)</td></tr><tr><td colspan="2"><strong>The contexts of PMI:</strong></td></tr><tr><td>English:</td><td>It turns out that different species have slightly different structures of collagen, so if you get a collagen profile of an unknown bone, you can compare it to those of known species, and, who knows, maybe you get a match.</td></tr><tr><td>Chinese:</td><td>事实证明,不同物种的胶原蛋白结构略有不同,因此如果你得到一块未知骨头的胶原蛋白谱,你可以将其与已知物种的胶原蛋白谱进行比较,谁知道,也许能找到匹配。</td></tr><tr><td>English:</td><td>So she shipped him one of the fragments, FedEx.</td></tr><tr><td>Chinese:</td><td>于是她通过联邦快递把其中一块碎片寄给了他。</td></tr><tr><td>English:</td><td>LN: And he processed it, and compared it to 37 known and modern-day mammal species.</td></tr><tr><td>Chinese:</td><td>LN:他对它进行了处理,并将其与37种已知和现代哺乳动物进行了比较。</td></tr></table>

source and reference as contexts, integrating remote contexts selected by PMI leads to better translation quality than Zero-shot, Random and Past. The bottom part reports the results for translations with source and automatic translations as contexts. In this setting, translation using EUROLLM with PMI is still better than Zero-shot, Random and Past, with lesser performance gains. In contrast, for QWEN, PMI appears to do less well than Past. The performance of LLAMA depends on the language pair, PMI being best only for EN–ZH translations.

We report a similar analysis in Table 10 for MERSENNE corpus. The results show that PMI outperforms Zero-shot and Random for all models using reference or automatic translations as target side context. In all cases except four, PMI is better than Past in d-COMET, especially for QWEN.

## 6.3 A Case study: Lexical Consistency

We illustrate the benefits of retrieving relevant contexts for the adequate translation in context-dependent cases. A first example is in Figure 4: to translate *a match*, which corresponds here to a

<footnote>stricted contexts, while sentences occurring in the last position have a much larger set of contextual segments to chose from.</footnote>
<table><caption>Table 9: Average differences between the d-COMET of translations using PMI and three baselines. From top to bottom these are: zero-shot (sent), random (rand), past (past). We only consider all sentences occurring past the 20th sentence in IWSLT articles, and take into account the maximum distance between the selected contexts and the current sentence. * marks a significant difference with p-value < 0.05.</caption><thead><tr><th rowspan="2">dist</th><th colspan="3">DE</th><th colspan="3">FR</th><th colspan="3">ZH</th></tr><tr><th>EURO</th><th>LLAMA</th><th>QWEN</th><th>EURO</th><th>LLAMA</th><th>QWEN</th><th>EURO</th><th>LLAMA</th><th>QWEN</th></tr></thead><tbody><tr><th colspan="10">SRC+REF</th></tr><tr><td>PMI - sent</td><td>0-20<br>20-40<br>40-64<br>64-256</td><td>*1.4<br>*1.4<br>*1.4<br>*1.5</td><td>*2.3<br>*1.4<br>*2.2<br>*2.2</td><td>*2.7<br>*1.5<br>*2.1<br>*2.4</td><td>*1.0<br>*1.3<br>*1.6<br>*1.3</td><td>*1.2<br>*1.5<br>*1.9<br>*1.9</td><td>*1.6<br>*1.7<br>*1.9<br>*1.6</td><td>*1.5<br>*1.5<br>*1.5<br>*1.1</td><td>*2.8<br>*2.2<br>*2.9<br>*2.0</td><td>*2.7<br>*2.2<br>*2.9<br>*3.1</td></tr><tr><td>PMI - rand</td><td>0-20<br>20-40<br>40-64<br>64-256</td><td>*1.1<br>*0.9<br>0.3<br>*1.2</td><td>*1.4<br>0.7<br>*1.2<br>*1.7</td><td>*2.0<br>*1.3<br>*2.9<br>*1.8</td><td>*0.7<br>*1.0<br>0.5<br>*1.0</td><td>*1.3<br>*0.7<br>0.6<br>*0.8</td><td>0.5<br>*1.0<br>0.3<br>*0.8</td><td>*0.9<br>*1.0<br>*1.1<br>0.5</td><td>*2.1<br>*1.5<br>*1.2<br>*1.3</td><td>*1.6<br>*0.6<br>*1.5<br>*1.5</td></tr><tr><td>PMI - past</td><td>0-20<br>20-40<br>40-64<br>64-256</td><td>0.3<br>0.2<br>0.3<br>0.6</td><td>*0.8<br>*0.7<br>0.5<br>*1.3</td><td>0.8<br>0.7<br>*1.8<br>*1.9</td><td>0.2<br>*0.5<br>0.5<br>0.4</td><td>*0.9<br>0.5<br>*0.7<br>0.4</td><td>0.3<br>*0.7<br>*1.5<br>-0.4</td><td>0.2<br>0.4<br>0.4<br>-0.2</td><td>0.4<br>*0.8<br>0.5<br>*0.9</td><td>0.1<br>0.3<br>*1.0<br>*0.8</td></tr><tr><th colspan="10">SRC+MT</th></tr><tr><td>PMI - sent</td><td>0-20<br>20-40<br>40-64<br>64-256</td><td>*0.7<br>*0.9<br>*1.6<br>*0.8</td><td>0.2<br>-0.2<br>-0.3<br>-0.7</td><td>0.5<br>-0.7<br>0.8<br>-0.3</td><td>*0.5<br>*0.8<br>0.4<br>*0.8</td><td>-0.2<br>-0.3<br>0.2<br>*0.7</td><td>0.4<br>0.6<br>0.7<br>*1.2</td><td>*0.7<br>*0.7<br>0.4<br>*0.7</td><td>*1.5<br>*1.2<br>*1.3<br>*1.1</td><td>*1.6<br>*0.9<br>*1.1<br>*1.7</td></tr><tr><td>PMI - rand</td><td>0-20<br>20-40<br>40-64<br>64-256</td><td>0.5<br>*0.7<br>0.3<br>0.6</td><td>0.3<br>-0.5<br>-0.6<br>-0.5</td><td>0.4<br>-0.7<br>*2.2<br>-0.2</td><td>0.3<br>*0.5<br>0.3<br>0.3</td><td>0.2<br>-0.3<br>-0.2<br>0.2</td><td>0.3<br>*0.7<br>0.3<br>0.2</td><td>0.1<br>*0.6<br>0.3<br>0.4</td><td>*1.2<br>*1.3<br>0.5<br>*1.0</td><td>*0.8<br>-0.1<br>0.1<br>0.6</td></tr><tr><td>PMI - past</td><td>0-20<br>20-40<br>40-64<br>64-256</td><td>0.1<br>-0.0<br>*1.0<br>0.1</td><td>-0.5<br>-0.4<br>*0.9<br>*-0.8</td><td>-0.8<br>*-1.0<br>0.6<br>-0.2</td><td>0.1<br>*0.3<br>0.4<br>-0.1</td><td>-0.1<br>-0.5<br>0.3<br>-0.0</td><td>-0.2<br>0.2<br>0.7<br>-0.1</td><td>0.2<br>0.4<br>0.4<br>-0.0</td><td>0.5<br>*0.7<br>0.2<br>*0.8</td><td>-0.1<br>-0.3<br>-0.4<br>0.5</td></tr></tbody></table>

<table><caption>Table 10: Average differences between the d-COMET of translations using PMI and three baselines. From top to bottom these are: zero-shot (sent), random (rand), past (past). We only consider all sentences occurring past the 20th sentence in MERSENNE articles, and take into account the maximum distance between the selected contexts and the current sentence. * marks significant difference with p-value < 0.05.</caption><thead><tr><th rowspan="2">dist</th><th colspan="3">SRC+REF</th><th colspan="3">SRC+MT</th></tr><tr><th>EURO</th><th>LLAMA</th><th>QWEN</th><th>EURO</th><th>LLAMA</th><th>QWEN</th></tr></thead><tbody><tr><td>sent</td><td>0-20<br>20-40<br>40-64<br>64-120<br>120-320</td><td>*0.6<br>*0.7<br>*0.9<br>*0.7<br>*0.6</td><td>*0.9<br>*0.8<br>*0.7<br>*0.9<br>*1.0</td><td>*1.4<br>*1.6<br>*1.9<br>*1.5<br>*1.6</td><td>*0.3<br>*0.4<br>*0.3<br>*0.2<br>*0.4</td><td>*0.5<br>*0.3<br>*0.3<br>*0.3<br>*0.5</td><td>*0.8<br>*0.9<br>*1.2<br>*0.8<br>*0.6</td></tr><tr><td>rand</td><td>0-20<br>20-40<br>40-64<br>64-120<br>120-320</td><td>*0.3<br>*0.3<br>*0.6<br>*0.5<br>*0.4</td><td>*0.5<br>*0.5<br>*0.5<br>*0.6<br>*0.9</td><td>*0.8<br>*0.5<br>*0.9<br>*0.7<br>*0.7</td><td>0.0<br>*0.2<br>*0.3<br>0.3<br>0.1</td><td>*0.2<br>*0.3<br>*0.2<br>0.2<br>0.2</td><td>*0.3<br>*0.3<br>*0.5<br>*0.3<br>*0.1</td></tr><tr><td>past</td><td>0-20<br>20-40<br>40-64<br>64-120<br>120-320</td><td>-0.0<br>*0.2<br>*0.4<br>*0.3<br>0.2</td><td>*0.1<br>*0.2<br>0.1<br>*0.4<br>*1.0</td><td>*0.5<br>*0.5<br>*0.6<br>*0.5<br>*1.0</td><td>-0.0<br>0.1<br>0.1<br>0.1<br>-0.1</td><td>0.0<br>0.0<br>-0.3<br>0.1<br>0.0</td><td>*0.3<br>*0.2<br>*0.4<br>*0.2<br>0.4</td></tr></tbody></table>

matched species, PMI retrieves the relevant contexts and generates the correct translation, while Zero-shot translates it as “an appropriate person” and Past considers it as a matched object. Complementary details and another Chinese example are presented in Appendix C.

Distant contexts are necessary to ensure lexi-

cal consistency, especially when translating full articles, and we do observe some evidence for improved translation consistency. For example, in one particular MERSENNE article, the term *hemihedria* appears 12 times,<fnref n="14" /> with a consistent French reference translation *hémiédrie*. Table 3 shows the translation of the 72nd sentence using EUROLLM taking source and MT as context.

PMI achieves consistent translation for this term, while Past fails due to the absence of remote contexts, resulting in 11 translation errors with 4 variants. Zero-shot translates them into 6 different forms with only one correct translation, despite its high LTCR score.

Figure 8 and Figure 9 in Appendix C provide the contexts selected using PMI for the 72nd and the 159th sentences respectively, including the sentences in lines 51 and 58 that contain the term *hemihedria*.

## 7 Conclusion

To achieve better document-level machine translation for extra long documents, we propose the SELF-RAMT framework. By retrieving relevant sentences in a local translation memory composed

<footnote n="14">¹⁴Lines 26, 39, 51, 53, 56, 58, 72, 78, 89, 122, 159, 160.</footnote>
of sentences that have already been translated, and
possibly post-edited, in the same document, we ex-
pect to generate translations that are globally more
consistent. We carry out experiments to translate
full TED talks and a novel parallel corpus consist-
ing of scientific articles, using three LLMs with
in context learning, to integrate the retrieved past
contexts.

We further analyze the influence of distant con-
texts on translation quality, according to their dis-
tance from the current sentence. Our findings show
that incorporating distant contexts, selected using
context criteria such as PMI, can be useful for bet-
ter lexical consistency. Distant context also seem
to improve the general translation quality, as mea-
sured by reference-based scores.

Limitations

In this work, we only considered three open-weight
models, excluding close-source models such as
GPT4 (OpenAI et al., 2023) the use of which makes
results difficult, if possible at all, to reproduce for
scientific purposes. The translation of low-resource
languages is not included in our experiments, al-
though our framework could be beneficial in that
scenario. In our experimental settings, we process
each document sentence per sentence, defining the
translation unit and the contextually relevant sen-
tences based on this predefined segmentation. Seg-
menting and processing input documents in larger
chunks, made of several consecutive sentences, is
left for future work. Our evaluation was based
mainly on derivatives of BLEU and COMET met-
rics, with a variant of LTCR and some case studies.
While we reckon that some better metrics should be
applied to better reflect document translation qual-
ity, measuring for example, the consistency and
coherence of the translated text, we also can only
regret that such standard evaluation metrics have
not yet developed nor adopted by the community.

Ethics Statement

This work has conducted experiments and analysis
using open source models, tools, and corpus. We
see no ethical problem with this study.

Acknowledgments

This work was supported by the French national
agency (ANR) as part of the MaTOS project under
reference ANR-22-CE23-0033.<fnref n="15" /> Rachel Bawden

was also partly funded by her chair position in the
PRAIRIE institute funded by ANR as part of the
“Investissements d’avenir” programme under ref-
erence ANR19-P3IA-0001. The authors wish to
thank Célia Vaudaine and Caroline Rossi for giv-
ing access to the MERSENNE corpus. The authors
are also grateful to the anonymous reviewers for
their insightful comments and suggestions, and to
Paul Lerner and Lichao Zhu for their review and
feedback on a preliminary draft of this work.

References

Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke
Zettlemoyer, and Marjan Ghazvininejad. 2023. In-
context examples selection for machine translation.
*In Findings of the Association for Computational
Linguistics: ACL 2023*, pages 8857–8873, Toronto,
Canada. Association for Computational Linguistics.

Duarte Miguel Alves, José Pombal, Nuno M Guerreiro,
Pedro Henrique Martins, João Alves, Amin Farajian,
Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta
Agrawal, Pierre Colombo, José G. C. de Souza, and
Andre Martins. 2024. Tower: An open multilingual
large language model for translation-related tasks. In
*First Conference on Language Modeling*.

Rachel Bawden, Rico Sennrich, Alexandra Birch, and
Barry Haddow. 2018. Evaluating discourse phenom-
ena in neural machine translation. In *Proceedings of
the 2018 Conference of the North American Chap-
ter of the Association for Computational Linguistics:
Human Language Technologies, Volume 1 (Long Pa-
pers)*, pages 1304–1313, New Orleans, Louisiana.

Rachel Bawden and François Yvon. 2023. Investigating
the translation performance of a large multilingual
language model: the case of BLOOM. In *Proceed-
ings of the 24th Annual Conference of the European
Association for Machine Translation*, pages 157–170,
Tampere, Finland. European Association for Machine
Translation.

Maxime Bouthors, Josep Crego, and François Yvon.
2024. Retrieving examples from memory for re-
trieval augmented neural machine translation: A sys-
tematic comparison. In *Findings of the Association
for Computational Linguistics: NAACL 2024*, pages
3022–3039, Mexico City, Mexico. Association for
Computational Linguistics.

Eleftheria Briakou, Jiaming Luo, Colin Cherry, and Markus Freitag. 2024. Translating step-by-step: Decomposing the translation process for improved translation quality of long-form texts. In *Proceedings of the Ninth Conference on Machine Translation*, pages 1301–1317, Miami, Florida, USA. Association for Computational Linguistics.

Bram Bulte and Arda Tezcan. 2019. Neural fuzzy re-
pair: Integrating fuzzy matches into neural machine

<footnote n="15">¹⁵http://anr-matos.github.io/</footnote>
translation. In *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pages 1800–1809, Florence, Italy.

Sheila Castilho and Rebecca Knowles. 2024. A survey of context in neural machine translation and its evaluation. *Natural Language Processing*, page 1–31.

Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. WIT3: Web inventory of transcribed and translated talks. In *Proceedings of the 16th Annual Conference of the European Association for Machine Translation*, pages 261–268, Trento, Italy. European Association for Machine Translation.

Xin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao, and Rui Yan. 2022. Neural machine translation with contrastive translation memories. In *Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*, pages 3591–3601, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Menglong Cui, Jiangcun Du, Shaolin Zhu, and Deyi Xiong. 2024. Efficiently exploring large language models for document-level machine translation with in-context learning. In *Findings of the Association for Computational Linguistics ACL 2024*, pages 10885–10897, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.

Yichen Dong, Xinglin Lyu, Junhui Li, Daimeng Wei, Min Zhang, Shimin Tao, and Hao Yang. 2025. Two intermediate translations are better than one: Fine-tuning LLMs for document-level translation refinement. In *Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 14917–14933, Vienna, Austria. Association for Computational Linguistics.

Zi-Yi Dou and Graham Neubig. 2021. Word alignment by fine-tuning embeddings on parallel corpora. In *Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume*, pages 2112–2128, Online. Association for Computational Linguistics.

Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT sentence embedding. In *Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 878–891, Dublin, Ireland. Association for Computational Linguistics.

Patrick Fernandes, Kayo Yin, Emmy Liu, André Martins, and Graham Neubig. 2023. When does translation require context? a data-driven, multilingual exploration. In *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 606–626, Toronto, Canada. Association for Computational Linguistics.

Patrick Fernandes, Kayo Yin, Graham Neubig, and André F. T. Martins. 2021. Measuring and increasing

context usage in context-aware machine translation. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*, pages 6467–6478, Online. Association for Computational Linguistics.

Harritxu Gete, Thierry Etchegoyhen, David Ponce, Gorka Labaka, Nora Aranberri, Ander Corral, Xabier Saralegi, Igor Ellakuria, and Maite Martin. 2022. TANDO: A corpus for document-level machine translation. In *Proceedings of the Thirteenth Language Resources and Evaluation Conference*, pages 3026–3037, Marseille, France. European Language Resources Association.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schellen, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mirta, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, and Bobbie Chern et al. 2024. The Llama 3 Herd of Models. Preprint arXiv:2407.21783.

Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher. 2018. Non-autoregressive neural machine translation. In *International Conference on Learning Representations*.

Jiaxin Guo, Yuanchang Luo, Daimeng Wei, Ling Zhang, Zongyao Li, Hengchao Shang, Zhiqiang Rao, Shaojun Li, Jinlong Yang, Zhanglin Wu, and Hao Yang. 2025. Doc-Guided Sent2Sent++: A Sent2Sent++ Agent with Doc-Guided memory for Document-level Machine Translation. Preprint arXiv:2501.08523.

Jiaxin Guo, Hao Yang, Zongyao Li, Daimeng Wei, Hengchao Shang, and Xiaoyu Chen. 2024. A novel paradigm boosting translation capabilities of large language models. In *Findings of the Association for Computational Linguistics: NAACL 2024*, pages 639–649, Mexico City, Mexico. Association for Computational Linguistics.

Qiuxiang He, Guoping Huang, Qu Cui, Li Li, and Lemao Liu. 2021. Fast and accurate neural machine translation with translation memory. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*, pages 3170–3180, Online. Association for Computational Linguistics.

Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hasan Awadalla. 2023. How good are gpt models at machine translation? a comprehensive evaluation. Preprint arXiv:2302.09210.
Xiaomian Kang, Yang Zhao, Jiajun Zhang, and Chengqing Zong. 2020. Dynamic context selection for document-level neural machine translation via reinforcement learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2242–2254, Online. Association for Computational Linguistics.

Marzena Karpinska and Mohit Iyyer. 2023. Large language models effectively leverage document-level context for literary translation, but critical errors persist. In Proceedings of the Eighth Conference on Machine Translation, pages 419–451, Singapore. Association for Computational Linguistics.

Martin Kay. 1997. The proper place of men and machines in language translation. Machine Translation, 12(3–23).

Maurice G. Kendall. 1938. A new measure of rank correlation. Biometrika, 30(1-2):81–93.

Maurice G. Kendall. 1945. The treatment of ties in ranking problems. Biometrika, 33(3):239–251.

Sai Koneru, Miriam Exel, Matthias Huck, and Jan Niehues. 2024. Contextual refinement of translations: Large language models for sentence and document-level post-editing. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2711–2725, Mexico City, Mexico. Association for Computational Linguistics.

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.

Bei Li, Hui Liu, Ziyang Wang, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu, and Changliang Li. 2020. Does multi-encoder help? a case study on context-aware neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3512–3518, Online. Association for Computational Linguistics.

Chen Li, Meishan Zhang, Xuebo Liu, Zhaocong Li, Derek Wong, and Min Zhang. 2024. Towards demonstration-aware large language models for machine translation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13868–13881, Bangkok, Thailand. Association for Computational Linguistics.

Yachao Li, Junhui Li, Jing Jiang, Shimin Tao, Hao Yang, and Min Zhang. 2023. P-transformer: Towards better document-to-document neural machine translation. IEEE/ACM Trans. Audio, Speech and Lang. Proc., 31:3859–3870.

Zongyao Li, Zhiqiang Rao, Hengchao Shang, Jiaxin Guo, Shaojun Li, Daimeng Wei, and Hao Yang. 2025. Enhancing large language models for document-level translation post-editing using monolingual data. In Proceedings of the 31st International Conference on Computational Linguistics, pages 8830–8840, Abu Dhabi, UAE. Association for Computational Linguistics.

Lei Liu and Min Zhu. 2022. Bertalign: Improved word embedding-based sentence alignment for Chinese-English parallel corpora of literary texts. Digital Scholarship in the Humanities, 38(2):621–634.

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12:157–173.

Lorenzo Lupo, Marco Dinarelli, and Laurent Besacier. 2022a. Divide and rule: Effective pre-training for context-aware multi-encoder translation models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4557–4572, Dublin, Ireland. Association for Computational Linguistics.

Lorenzo Lupo, Marco Dinarelli, and Laurent Besacier. 2022b. Focused concatenation for context-aware neural machine translation. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 830–842, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

Yuanhua Lv and ChengXiang Zhai. 2011. When documents are very long, bm25 fails! In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '11, page 1103–1104, New York, NY, USA. Association for Computing Machinery.

Xinglin Lyu, Junhui Li, Zhengxian Gong, and Min Zhang. 2021. Encouraging lexical translation consistency for document-level neural machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3265–3277, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Xing Han Lù. 2024. Bm25s: Orders of magnitude faster lexical search via eager sparse scoring. Preprint arXiv:2407.03618.

Shuming Ma, Dongdong Zhang, and Ming Zhou. 2020. A simple and effective unified encoder for document-level machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3505–3511, Online. Association for Computational Linguistics.

Pedro Henrique Martins, João Alves, Patrick Fernandes, Nuno M. Guerreiro, Ricardo Rei, Amin Farajian, Mateusz Klimaszewski, Duarte M. Alves, José Pombal, Manuel Faysse, Pierre Colombo, François Yvon,
Barry Haddow, José G. C. de Souza, Alexandra Birch, and André F. T. Martins. 2025. EuroLLM-9B: Technical Report. Preprint arXiv:2506.04079.

Pedro Henrique Martins, Patrick Fernandes, João Alves, Nuno M. Guerreiro, Ricardo Rei, Duarte M. Alves, José Pombal, Amin Farajian, Manuel Faysse, Małteusz Klimaszewski, Pierre Colombo, Barry Haddow, José G. C. de Souza, Alexandra Birch, and André F. T. Martins. 2024. Eurollm: Multilingual language models for europe. Preprint arXiv:2409.16235.

Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2019. Selective attention for context-aware neural machine translation. In *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)*, pages 3092–3102, Minneapolis, Minnesota. Association for Computational Linguistics.

Sameen Maruf, Fahimeh Saleh, and Gholamreza Haffari. 2021. A survey on document-level neural machine translation: Methods and evaluation. *ACM Comput. Surv.*, 54(2).

Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018. Document-level neural machine translation with hierarchical attention networks. In *Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, pages 2947–2954, Brussels, Belgium. Association for Computational Linguistics.

Yasmin Moslem, Rejwanul Haque, John D. Kelleher, and Andy Way. 2023. Adaptive machine translation with large language models. In *Proceedings of the 24th Annual Conference of the European Association for Machine Translation*, pages 227–237, Tampere, Finland. European Association for Machine Translation.

Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021. Trankit: A light-weight transformer-based toolkit for multilingual natural language processing. In *Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations*, pages 80–90, Online. Association for Computational Linguistics.

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, and Lenny Bogdonoff et al. 2023. GPT-4 technical report. CoRR, abs/2303.08774.

Proyag Pal, Alexandra Birch, and Kenneth Heafield. 2024. Document-level machine translation with large-scale public parallel corpora. In *Proceedings of the 62nd Annual Meeting of the Association for*

*Computational Linguistics (Volume 1: Long Papers)*, pages 13185–13197, Bangkok, Thailand. Association for Computational Linguistics.

Kishore Papineni, Salim Roukos, Todd Ward, and Weijing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In *Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics*, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.

Ziqian Peng, Rachel Bawden, and François Yvon. 2025. Investigating length issues in document-level machine translation. In *Proceedings of Machine Translation Summit XX: Volume 1*, pages 4–23, Geneva, Switzerland. European Association for Machine Translation.

Minh Quang Pham, Jitao Xu, Josep Crego, François Yvon, and Jean Senellart. 2020. Priming neural machine translation. In *Proceedings of the Fifth Conference on Machine Translation*, pages 516–527, Online. Association for Computational Linguistics.

José Pombal, Sweta Agrawal, Patrick Fernandes, Emmanouil Zaranis, and André F. T. Martins. 2024. A context-aware framework for translation-mediated conversations. Preprint arXiv:2412.04205.

Andrei Popescu-Belis. 2019. Context in neural machine translation: A review of models and evaluations. Preprint arXiv:1901.09115.

Matt Post. 2018. A call for clarity in reporting BLEU scores. In *Proceedings of the Third Conference on Machine Translation: Research Papers*, pages 186–191, Belgium, Brussels. Association for Computational Linguistics.

Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, and Rui Men et al. 2025. Qwen2.5 technical report. Preprint arXiv:2412.15115.

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog.

Tharindu Ranasinghe, Constantin Orasan, and Ruslan Mitkov. 2020. TransQuest: Translation quality estimation with cross-lingual transformers. In *Proceedings of the 28th International Conference on Computational Linguistics*, pages 5070–5081, Barcelona, Spain (Online). International Committee on Computational Linguistics.

Vikas Raunak, Tom Kocmi, and Matt Post. 2024. SLIDE: Reference-free evaluation for machine translation using a sliding document window. In *Proceedings of the 2024 Conference of the North American*
*Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)*, pages 205–211, Mexico City, Mexico. Association for Computational Linguistics.

Abudurexiti Reheman, Tao Zhou, Yingfeng Luo, Di Yang, Tong Xiao, and Jingbo Zhu. 2023. Prompting neural machine translation with translation memories. In *Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence*, AAAI’23/IAAI’23/EAAI’23. AAAI Press.

Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: Unbelabel-IST 2022 submission for the metrics shared task. In *Proceedings of the Seventh Conference on Machine Translation (WMT)*, pages 578–585, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. *Found. Trends Inf. Retr.*, 3(4):333–389.

Ohad Rubin and Jonathan Berant. 2024. Retrieval-pretrained transformer: Long-range language modeling with self-retrieval. *Transactions of the Association for Computational Linguistics*, 12:1197–1213.

Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. Trusting your evidence: Hallucinate less with context-aware decoding. In *Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)*, pages 783–791, Mexico City, Mexico. Association for Computational Linguistics.

Zewei Sun, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Shujian Huang, Jiajun Chen, and Lei Li. 2022. Re-thinking document-level neural machine translation. In *Findings of the Association for Computational Linguistics: ACL 2022*, pages 3537–3548, Dublin, Ireland. Association for Computational Linguistics.

Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2023. Efficient Transformers: A Survey. *ACM Computing Surveys*, 55(6):1–28.

Jörg Tiedemann and Yves Scherrer. 2017. Neural machine translation with extended context. In *Proceedings of the Third Workshop on Discourse in Machine Translation*, pages 82–92, Copenhagen, Denmark. Association for Computational Linguistics.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In *Advances in Neural Information Processing Systems 30*, pages 5998–6008. Curran Associates, Inc.

Giorgos Vernikos, Brian Thompson, Prashant Mathur, and Marcello Federico. 2022. Embarrassingly easy document-level MT metrics: How to convert any pretrained metric into a document-level metric. In *Proceedings of the Seventh Conference on Machine Translation (WMT)*, pages 118–128, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. 2023. Prompting PaLM for translation: Assessing strategies and performance. In *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 15406–15427, Toronto, Canada. Association for Computational Linguistics.

Elena Voita, Rico Sennrich, and Ivan Titov. 2019a. Context-aware monolingual repair for neural machine translation. In *Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)*, pages 877–886, Hong Kong, China. Association for Computational Linguistics.

Elena Voita, Rico Sennrich, and Ivan Titov. 2019b. When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pages 1198–1212, Florence, Italy. Association for Computational Linguistics.

Kuang-Da Wang, Teng-Ruei Chen, Yu Heng Hung, Shuoyang Ding, Yueh-Hua Wu, Yu-Chiang Frank Wang, Chao-Han Huck Yang, Wen-Chih Peng, and Ping-Chun Hsieh. 2025a. Plan2align: Predictive planning based test-time preference alignment in paragraph-level machine translation. Preprint arXiv:2502.20795.

Longyue Wang, Zefeng Du, Wenxiang Jiao, Chenyang Lyu, Jianhui Pang, Leyang Cui, Kaiqiang Song, Derek Wong, Shuming Shi, and Zhaopeng Tu. 2024. Benchmarking and improving long-text translation with large language models. In *Findings of the Association for Computational Linguistics: ACL 2024*, pages 7175–7187, Bangkok, Thailand. Association for Computational Linguistics.

Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. 2023. Aug-menting language models with long-term memory. In *Thirty-seventh Conference on Neural Information Processing Systems*.

Yutong Wang, Jiali Zeng, Xuebo Liu, Derek F. Wong, Fandong Meng, Jie Zhou, and Min Zhang. 2025b. DELTA: An online document-level translation agent based on multi-level memory. In *The Thirteenth International Conference on Learning Representations*.

Minghao Wu, Yufei Wang, George Foster, Lizhen Qu, and Gholamreza Haffari. 2024. Importance-aware
data augmentation for document-level neural ma-
chine translation. In Proceedings of the 18th Confer-
ence of the European Chapter of the Association for
Computational Linguistics (Volume 1: Long Papers),
pages 740–752, St. Julian’s, Malta. Association for
Computational Linguistics.

Mengzhou Xia, Guoping Huang, Lemao Liu, and Shum-
ing Shi. 2019. Graph based translation memory for
neural machine translation. In Proceedings of the
Thirty-Third AAAI Conference on Artificial Intelli-
gence and Thirty-First Innovative Applications of Ar-
tificial Intelligence Conference and Ninth AAAI Sym-
posium on Educational Advances in Artificial Intelli-
gence, AAAI’19/IAAI’19/EAAI’19. AAAI Press.

Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Has-
san Awadalla. 2024. A paradigm shift in machine
translation: Boosting translation performance of
large language models. In The Twelfth International
Conference on Learning Representations.

Jitao Xu, Josep Crego, and Jean Senellart. 2020. Boost-
ing neural machine translation with similar transla-
tions. In Proceedings of the 58th Annual Meeting of
the Association for Computational Linguistics, pages
1580–1590, Online. Association for Computational
Linguistics.

Jian Yang, Yuwei Yin, Shuming Ma, Liqun Yang, Hongcheng Guo, Haoyang Huang, Dongdong Zhang, Yutao Zeng, Zhoujun Li, and Furu Wei. 2023. Hanoit: Enhancing context-aware translation via selective context. In Database Systems for Advanced Applications: 28th International Conference, DASFAA 2023, Tianjin, China, April 17–20, 2023, Proceedings, Part III, page 471–486, Berlin, Heidelberg. Springer-Verlag.

Zhengxin Yang, Jinchao Zhang, Fandong Meng, Shuhao
Gu, Yang Feng, and Jie Zhou. 2019. Enhancing con-
text modeling with a query-guided capsule network
for document-level translation. In Proceedings of
the 2019 Conference on Empirical Methods in Natu-
ral Language Processing and the 9th International
Joint Conference on Natural Language Processing
(EMNLP-IJCNLP), pages 1527–1537, Hong Kong,
China. Association for Computational Linguistics.

Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang
Ling, Lingpeng Kong, Phil Blunsom, and Chris Dyer.
2020. Better document-level machine translation
with Bayes’ rule. Transactions of the Association for
Computational Linguistics, 8:346–360.

Emmanouil Zaranis, Nuno M. Guerreiro, and Andre
Martins. 2024. Analyzing context contributions in
LLM-based machine translation. In Findings of the
Association for Computational Linguistics: EMNLP
2024, pages 14899–14924, Miami, Florida, USA.
Association for Computational Linguistics.

Armel Randy Zebaze, Benoît Sagot, and Rachel Baw-
den. 2025. In-context example selection via simi-
larity search improves low-resource machine trans-

lation. In Findings of the Association for Computa-
tional Linguistics: NAACL 2025, pages 1222–1252,
Albuquerque, New Mexico. Association for Compu-
tational Linguistics.

Biao Zhang, Barry Haddow, and Alexandra Birch. 2023. Prompting large language model for machine translation: a case study. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org.

Zaixiang Zheng, Xiang Yue, Shujian Huang, Jiajun
Chen, and Alexandra Birch. 2021. Towards making
the most of context in neural machine translation. In
Proceedings of the Twenty-Ninth International Joint
Conference on Artificial Intelligence, IJCAI’20.

Zhang Zhuocheng, Shuhao Gu, Min Zhang, and Yang Feng. 2023. Addressing the length bias challenge in document-level neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11545–11556, Singapore. Association for Computational Linguistics.

Álvaro Peris and Francisco Casacuberta. 2019. Online
learning for effort reduction in interactive neural ma-
chine translation. Computer Speech & Language,
58:98–126.

A Mersenne

Due to the scarcity of complete parallel scholarly
documents, we constructed MERSENNE, which con-
sists of 23 articles in English and their translation
into French prepared by the Mersenne Center.<fnref n="16" />
Nineteen of them report recent research in the geo-
sciences domain and the remaining four belong to
the chemical sciences. The translations are human
post-edits of an initial machine-translated version.

For each article, we first convert the curated html
page to plain texts using pandoc.<fnref n="17" /> Extra empty
lines and the symbol xa0 are removed before nor-
malizing the texts to the NFC<fnref n="18" /> format through
unicodedata.<fnref n="19" /> We then extract the article from
the processed plain text, excluding equations and
tables, which are reserved for future exploitation.
We segment the texts into sentences and align the
sentences to parallel articles. For sentence segmen-
tation, we use Trankit (Nguyen et al., 2021) as it
recognizes lists of citations well. Our alignment
tool is derived from BertAlign (Liu and Zhu, 2022),
which supports many-to-many alignments. All the
alignments matched with an empty string were
checked and manually adjusted whenever needed.

<footnote n="16">¹⁶https://www.centre-mersenne.org/</footnote>

<footnote n="17">¹⁷https://pandoc.org/</footnote>

<footnote n="18">¹⁸Normalization Form Canonical Composition.</footnote>

<footnote n="19">¹⁹https://docs.python.org/3/library/unicodedata.html</footnote>
We subsequently evaluated the aligned sentence
pairs using TransQuest (Ranasinghe et al., 2020).
All alignment scores were above 0.75, suggesting
that sentence alignment is mostly correct and that
all aligned sentences can be kept. Statistics regard-
ing the MERSENNE parallel articles can be found
in Table 1.

B Experimental Details

This section presents details about the inference
settings, the computation of relevance scores, and
the prompt patterns for ICL.

B.1 Inference

In our experiments, we obtain automatic translations through ICL using vLLM (Kwon et al., 2023) in bfloat16. We set the beam width to 5 and the temperature to 0. The minimum and maximum number of new tokens are 1 and 256 respectively.

For the decoding process, we propose a multi-
turn decoding algorithm to access the incremen-
tally generated target-side contexts. This involves
translating the *i*th sentences of all documents from
the same batch in parallel, and updating the pre-
selected contexts for the (*i* + 1)th sentences with
the generated translations. When contexts com-
prise source texts and reference translations of past
sentences, we integrate them in the LLM prompts
and decode them all at once.

Regarding context selection, we compute PMI
scores for source texts using LLAMA, perform-
ing computations in bfloat32 for IWSLT and in
bfloat16 for MERSENNE and MERGEDTED.

B.2 Implementation of BM25L

In practice, given a document *X*, we preprocess
each sentence,<fnref n="20" /> then compute the term frequency
and the inverse document frequency (IDF) for each
term in the whole document. We also precomputed
an IDF for terms from the training split of IWSLT-
2016 (Cettolo et al., 2012)²¹ for the EN–FR lan-
guage pair. Therefore, the IDF of a term in *X* is
replaced by the precomputed values if available.

B.3 Prompt Patterns

We integrate the bilingual context for DLMT in a
few-shot template for ICL. As all our MT engines

are instruction-tuned, we use the chat template. Regard-
garding prompt design, we empirically tested the
performance of the Past- *K* baseline using differ-
ent prompt patterns on the IWSLT TST2010 and
TST2011 test sets, for *K* ∈ {2, 6}. After disregard-
ing some patterns that lead to over-generation, we
selected the prompt templates in Figure 6, where
we integrate the few-shot contexts into system
prompts for EUROLLM and LLAMA, and into user
prompts for QWEN.

For each sentence, we apply the prompt pattern
without context to perform zero-shot translation.
For the Indep-K baseline, we maintain *K*-shot
demonstrations for all sentences. In practice, we
generate 6 examples in the style of TED talks using
LLAMA,<fnref n="22" /> then integrate the first *K* examples as
*K*-shot demonstrations.

C Complementary Evaluation Results

**Translation Quality** Based on the complete
evaluation scores for IWSLT with source and au-
tomatic translations as contexts (see Table 11), and
MERSENNE (see Table 12), we can confirm that in
general PMI performs better than COS and BM25,
as discussed in Section 6.1. On the other hand, we
also observed that the performance of LLAMA is
worse than EUROLLM. This quality degradation
has a strong influence on DLMT using multi-turn
decoding.

**Case Study** In this section, we provide addi-
tional examples that illustrate the need to integrate
remote contexts for term consistency when trans-
lating long documents, and relevance criteria such
as PMI can identify such contexts. This analysis
is complementary to the one in Section 6.3. For
example, a consistent translation of the term *hemi-
hedria* in the 159th sentence of a MERSENNE arti-
cle requires contexts more distant than the past 30
sentences, and contexts with the *K* highest PMI
scores include these relevant but remote sentences
(see Figure 9). Figure 5 displays another example
that contextual information from the past 10th sen-
tence is required for the consistent translation of
“the High Arctic”.

Figure 7 gives complementary information for
the example in Figure 4, including the two succes-
sive sentences of the source sentence to be trans-

<footnote n="20">²⁰The preprocessing consists of lowercasing, stop-word removal and stemming using PyStemmer: https://pypi.org/project/PyStemmer/.
²¹https://wit3.fbk.eu/2016-01</footnote>

<footnote n="22">²²We use the following chat template integrated in the sys-
tem prompt “You are a good translation assistant!” and the
user prompt “Give six few-shot examples to assist the transla-
tion from English to {tgtLang} in the style of TED talks.”</footnote>
<table><caption>Table 11: Evaluation for the translation of IWSLT, translated using source and automatic translation as context, for K = 3. We mark significantly positive difference between context-augmented methods and zero-shot MT for COMET-based scores, for sentences excluding the first 20 ones (Δ), all sentences (δ), or in both cases (*), with p-value < 0.05.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="5">EUROLLM</th><th colspan="5">LLAMA</th><th colspan="5">QWEN</th></tr><tr><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th></tr></thead><tbody><tr><td rowspan="5">DE</td><td>0-shot</td><td>27.8</td><td>85.40</td><td>75.77</td><td>81.83</td><td>95.85</td><td><u>24.7</u></td><td>82.73</td><td>72.25</td><td>79.52</td><td>95.23</td><td>22.8</td><td>81.64</td><td>70.61</td><td>77.21</td><td>91.16</td></tr><tr><td>indep</td><td>28.1</td><td>85.42</td><td>75.81</td><td>81.83</td><td>96.02</td><td>24.4</td><td>82.69</td><td>71.91</td><td>79.46</td><td>95.88</td><td>22.8</td><td>81.23</td><td>69.71</td><td>76.54</td><td>91.83</td></tr><tr><td>rand.</td><td><u>28.5</u></td><td>*85.70</td><td>*76.18</td><td>81.98</td><td>96.01</td><td>24.4</td><td>82.88</td><td>72.41</td><td>79.73</td><td>95.08</td><td>22.7</td><td>81.75</td><td>70.65</td><td>77.43</td><td>93.48</td></tr><tr><td>past</td><td><u>28.5</u></td><td>*85.69</td><td><u>*76.43</u></td><td><u>*82.07</u></td><td><u>96.57</u></td><td>24.5</td><td><u>*83.09</u></td><td>*72.71</td><td><u>*80.08</u></td><td>96.64</td><td><u>23.0</u></td><td><u>*81.93</u></td><td><u>71.07</u></td><td>77.74</td><td>91.61</td></tr><tr><td>COS</td><td>28.3</td><td>*85.69</td><td>*76.24</td><td>82.00</td><td>96.50</td><td>24.5</td><td>82.85</td><td>72.27</td><td>79.71</td><td><u>97.95</u></td><td>22.9</td><td><u>*82.15</u></td><td><u>71.19</u></td><td><u>77.81</u></td><td><u>95.72</u></td></tr><tr><td>PMI</td><td><u>28.9</u></td><td><u>*85.92</u></td><td><u>*76.62</u></td><td><u>*82.06</u></td><td>95.38</td><td>24.2</td><td>82.68</td><td>72.16</td><td>*79.97</td><td>96.04</td><td>22.3</td><td>81.69</td><td>70.75</td><td>77.72</td><td>91.58</td></tr><tr><td>BM25</td><td>28.3</td><td>*85.72</td><td>*76.32</td><td>81.94</td><td>95.24</td><td><u>24.9</u></td><td><u>*83.16</u></td><td><u>*72.76</u></td><td>79.72</td><td><u>97.82</u></td><td>22.9</td><td>81.72</td><td>70.74</td><td><u>77.90</u></td><td><u>93.75</u></td></tr><tr><td rowspan="5">FR</td><td>0-shot</td><td>40.1</td><td>86.40</td><td>76.84</td><td>83.34</td><td>88.89</td><td>36.5</td><td><u>84.36</u></td><td>73.98</td><td>81.78</td><td>87.20</td><td>34.5</td><td>83.87</td><td>73.20</td><td>80.68</td><td>89.64</td></tr><tr><td>indep</td><td>41.3</td><td>86.39</td><td>77.03</td><td>83.37</td><td>89.89</td><td>36.5</td><td>84.28</td><td>73.97</td><td>81.77</td><td>87.53</td><td>34.8</td><td>84.03</td><td>*73.58</td><td>80.88</td><td>88.66</td></tr><tr><td>rand.</td><td>41.2</td><td>86.48</td><td>*77.18</td><td>83.41</td><td>89.21</td><td>36.5</td><td><u>84.40</u></td><td>74.03</td><td>81.79</td><td>88.81</td><td>34.7</td><td>84.02</td><td>73.45</td><td><u>*81.17</u></td><td>89.91</td></tr><tr><td>past</td><td>41.2</td><td><u>86.52</u></td><td><u>*77.32</u></td><td><u>83.43</u></td><td>87.65</td><td><u>36.7</u></td><td>84.32</td><td><u>74.08</u></td><td><u>81.93</u></td><td>88.23</td><td>35.0</td><td>83.97</td><td>*73.71</td><td><u>71.12</u></td><td>89.46</td></tr><tr><td>COS</td><td>41.2</td><td>86.43</td><td>*77.16</td><td>83.43</td><td>92.14</td><td><u>36.7</u></td><td>84.27</td><td>73.89</td><td><u>81.93</u></td><td><u>92.53</u></td><td><u>35.1</u></td><td><u>84.07</u></td><td>73.52</td><td><u>71.05</u></td><td>93.35</td></tr><tr><td>PMI</td><td><u>41.8</u></td><td><u>*86.64</u></td><td><u>*77.53</u></td><td><u>83.45</u></td><td>90.87</td><td><u>36.7</u></td><td>84.27</td><td>73.99</td><td>81.84</td><td><u>93.63</u></td><td><u>35.2</u></td><td><u>84.07</u></td><td>*73.85</td><td>81.03</td><td><u>94.71</u></td></tr><tr><td>BM25</td><td><u>41.5</u></td><td>86.43</td><td>*77.18</td><td>83.43</td><td><u>92.43</u></td><td><u>36.6</u></td><td>84.29</td><td><u>74.12</u></td><td>81.73</td><td>92.22</td><td><u>35.1</u></td><td><u>84.17</u></td><td><u>*73.91</u></td><td><u>*81.35</u></td><td><u>94.38</u></td></tr><tr><td rowspan="5">ZH</td><td>0-shot</td><td>30.1</td><td>84.42</td><td>73.31</td><td>81.28</td><td>75.52</td><td><u>28.3</u></td><td>83.21</td><td>70.62</td><td>79.17</td><td>75.88</td><td>29.2</td><td>83.24</td><td>71.56</td><td>78.63</td><td>79.02</td></tr><tr><td>indep</td><td>30.5</td><td>*84.65</td><td><u>*73.76</u></td><td><u>81.30</u></td><td>78.91</td><td><u>29.1</u></td><td><u>*83.59</u></td><td><u>*71.89</u></td><td><u>*79.70</u></td><td>76.40</td><td>29.8</td><td>*83.73</td><td>*72.53</td><td>*79.54</td><td>81.34</td></tr><tr><td>rand.</td><td>30.6</td><td>*84.62</td><td>*73.65</td><td>81.22</td><td>83.08</td><td>26.6</td><td>83.10</td><td>70.87</td><td>79.14</td><td>78.70</td><td>29.9</td><td>*83.90</td><td>*72.49</td><td>*79.39</td><td>79.44</td></tr><tr><td>past</td><td>30.7</td><td>84.60</td><td>*73.64</td><td>81.18</td><td>85.12</td><td>26.9</td><td>83.20</td><td>*71.33</td><td>79.30</td><td>82.89</td><td>30.1</td><td><u>*84.02</u></td><td><u>*72.98</u></td><td><u>*79.64</u></td><td>87.26</td></tr><tr><td>COS</td><td>30.7</td><td>84.49</td><td>*73.56</td><td>81.20</td><td>85.50</td><td>27.1</td><td>82.98</td><td>*71.04</td><td>78.96</td><td>81.92</td><td>30.0</td><td>*83.93</td><td>*72.69</td><td>*79.55</td><td><u>89.94</u></td></tr><tr><td>PMI</td><td><u>30.8</u></td><td><u>*84.69</u></td><td><u>*73.86</u></td><td>81.23</td><td><u>90.02</u></td><td>27.3</td><td><u>71.86</u></td><td><u>*71.86</u></td><td><u>79.35</u></td><td><u>86.14</u></td><td><u>30.3</u></td><td><u>*84.00</u></td><td><u>*72.90</u></td><td><u>*79.57</u></td><td>86.11</td></tr><tr><td>BM25</td><td>30.2</td><td>84.21</td><td>73.23</td><td>80.15</td><td>88.79</td><td>26.8</td><td>83.12</td><td>*71.23</td><td>79.14</td><td><u>85.02</u></td><td><u>30.3</u></td><td><u>*83.87</u></td><td><u>*72.59</u></td><td><u>*79.22</u></td><td><u>88.37</u></td></tr></tbody></table>

<table><caption>Table 12: Evaluation for the translation of MERSENNE, translated using bilingual contexts with source and reference (REF, top) or source and automatic translation (MT, bottom), for K = 3. We mark significantly positive difference between context-augmented methods and zero-shot MT for COMET-based scores, for sentences excluding the first 20 ones (Δ), all sentences (δ), or in both cases (*), with p-value < 0.05.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="5">EUROLLM</th><th colspan="5">LLAMA</th><th colspan="5">QWEN</th></tr><tr><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th><th>BLEU</th><th>s-comet</th><th>d-comet</th><th>slide</th><th>LTCR</th></tr></thead><tbody><tr><td rowspan="5">REF</td><td>0-shot</td><td>55.7</td><td>89.48</td><td>86.64</td><td>74.60</td><td><b>92.85</b></td><td>51.0</td><td>88.41</td><td>85.34</td><td>73.20</td><td><b>92.59</b></td><td>47.6</td><td>87.58</td><td>84.18</td><td>71.99</td><td>90.12</td></tr><tr><td>rand.</td><td>58.7</td><td>*89.60</td><td>*86.78</td><td>*74.82</td><td>91.73</td><td>52.6</td><td>*88.65</td><td>*85.62</td><td>*73.39</td><td>92.09</td><td>51.8</td><td>*88.25</td><td>*85.01</td><td>*72.93</td><td>89.79</td></tr><tr><td>past</td><td>59.8</td><td>*89.80</td><td>*87.02</td><td>*74.88</td><td>91.79</td><td>53.8</td><td>*88.89</td><td>*85.89</td><td><b>*73.52</b></td><td>91.28</td><td>53.1</td><td>*88.37</td><td>*85.10</td><td>*72.91</td><td><b>90.74</b></td></tr><tr><td>COS</td><td><b>60.8</b></td><td>*89.94</td><td>*87.22</td><td>*74.88</td><td>90.89</td><td><b>55.0</b></td><td>*89.03</td><td>*86.04</td><td><b>*73.52</b></td><td>91.85</td><td><b>55.4</b></td><td>*88.68</td><td>*85.54</td><td><b>*73.33</b></td><td>90.06</td></tr><tr><td>PMI</td><td><b>61.0</b></td><td><b>*89.97</b></td><td><b>*87.26</b></td><td><b>*74.91</b></td><td>91.54</td><td><b>55.2</b></td><td><b>*89.14</b></td><td><b>*86.16</b></td><td><b>*73.51</b></td><td>92.32</td><td><b>55.8</b></td><td>*88.80</td><td>*85.67</td><td>*73.26</td><td>90.68</td></tr><tr><td>BM25</td><td>60.5</td><td>*89.91</td><td>*87.17</td><td>*74.90</td><td>91.21</td><td>54.9</td><td>*89.11</td><td>*86.09</td><td>*73.44</td><td>91.32</td><td><b>55.4</b></td><td><b>*88.82</b></td><td><b>*85.70</b></td><td>*73.24</td><td>89.98</td></tr><tr><td rowspan="5">MT</td><td>0-shot</td><td>55.7</td><td>89.48</td><td>86.64</td><td>74.60</td><td><b>92.85</b></td><td>51.0</td><td>88.41</td><td>85.34</td><td>73.20</td><td>92.59</td><td>47.6</td><td>87.58</td><td>84.18</td><td>71.99</td><td>90.12</td></tr><tr><td>rand.</td><td>55.8</td><td>89.51</td><td>86.72</td><td>*74.75</td><td>91.45</td><td><b>51.5</b></td><td>*88.55</td><td>*85.52</td><td>73.31</td><td>92.87</td><td>48.4</td><td>*88.01</td><td>*84.74</td><td>*72.81</td><td>90.63</td></tr><tr><td>past</td><td>55.0</td><td>*89.56</td><td>*86.84</td><td><u>*74.85</u></td><td>92.34</td><td><b>51.5</b></td><td>*88.63</td><td><b>*85.69</b></td><td><b>*73.57</b></td><td>92.89</td><td>48.3</td><td>*87.91</td><td>*84.74</td><td>*72.97</td><td>92.09</td></tr><tr><td>COS</td><td><b>56.2</b></td><td>*89.59</td><td>*86.87</td><td>*74.82</td><td>92.64</td><td><b>51.5</b></td><td>*88.61</td><td>*85.61</td><td>*73.44</td><td>93.62</td><td>48.7</td><td>*88.04</td><td>*84.90</td><td>*73.06</td><td>92.62</td></tr><tr><td>PMI</td><td><b>56.3</b></td><td>*89.61</td><td><b>*86.90</b></td><td><b>*74.88</b></td><td>92.77</td><td><b>51.6</b></td><td>*88.64</td><td>*85.67</td><td>*73.56</td><td><b>93.96</b></td><td><b>49.0</b></td><td><b>*88.09</b></td><td><b>*85.01</b></td><td><b>*73.16</b></td><td>92.79</td></tr><tr><td>BM25</td><td>56.2</td><td><b>*89.63</b></td><td><b>*86.90</b></td><td>*74.85</td><td>92.69</td><td><b>51.6</b></td><td><b>*88.66</b></td><td>*85.66</td><td>*73.45</td><td>93.94</td><td>48.5</td><td>*88.02</td><td>*84.89</td><td>*73.06</td><td><b>92.92</b></td></tr></tbody></table>
<im_start>system

Translate the following English source text to Chinese, considering the
provided English context and its Chinese translations:

**English:** that Natalia had dug out of the High Arctic belonged to ...

**Chinese:** Natalia 从加拿大北极高地挖出的东西属于……

**English:** So this camel would have been about nine feet tall, weighed around a ton.

**Chinese:** 因此,这只骆驼身高约9英尺,重约1吨。

**English:** But chances are the postcard image you have in your brain is one of these, the dromedary, quintessential desert creature -- hangs out in sandy, hot places like the Middle East and the Sahara, has a big old hump on its back for storing water for those long desert treks, has big, broad feet to help it tromp over sand dunes.

**Chinese:** 但是,很可能你脑海中浮现的明信片图像是其中一种,即单峰驼,这种动物是典型的沙漠生物,生活在中东和撒哈拉等炎热干燥的地方,背上有一大块驼峰,用来储存水分以应对漫长的沙漠探险,还有大而宽的脚,帮助它在沙丘上行走。<im_end>

<im_start>user

**English:** So how on earth would one of these guys end up in the High Arctic?

**Chinese:**<im_end>

<im_start>assistant

Figure 5: Selected contexts using PMI for the 56th sentence from a IWSLT talk for EN–ZH translation. It included the 46th sentence that contains the first mention of “the High Arctic”.

lated (“And he found a match”), and also the contexts used in Past and PMI, with $K = 3$. The contexts selected using PMI, stating that the person can “compare it to those of known species” and “get a match”, produce an adequate translation of “a match”.
<figure><img src="image_4.png" /><figcaption>Figure 6: The prompt patterns for EUROLLM, LLAMA and QWEN applied in our experiments.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 7: EN-ZH translations of the 45th sentence from an IWSLT talk, using EUROLLM with MT as target-side context. PMI retrieves relevant contexts in the 41th sentence for "a match", which means a matched species.</figcaption></figure>
<im_start>system

Translate the following English source text to French, considering the provided English context and its French translations:

**English:** In the memoir of May 22, 1848[4], it is the "tartrates" and the "paratartrates" which are primarily considered, but the young scientist seeks fruitful generalisations: "It will be said, and rightly so: All organic substances that deviate from the plane of polarisation when they are dissolved will therefore enjoy **hemihedria**.

**French:** Dans le mémoire du 22 mai 1848 [4], ce sont les « tartrates » et les « paratartrates » qui sont principalement considérés, mais le jeune scientifique cherche des généralisations fructueuses : « On dira, et à juste titre : toutes les substances organiques qui dévient du plan de polarisation lorsqu'elles sont dissoutes profiteront donc de l'hémiédrie.

**English:** It was even by studying this latter property that I was assured of the **hemihedria**, which I then realised via careful observation of the crystalline form.

**French:** C'est même en étudiant cette dernière propriété que j'ai été convaincu de l'hémiédrie, que j'ai ensuite confirmée par une observation attentive de la forme cristalline.

**English:** At this stage, Pasteur had moved away from the morphological study of the crystals to the study of the possible rotations of the plane of polarisation which they induced, which had led him to better characterise the **hemihedria** of tartrates.

**French:** À ce stade, Pasteur s'était éloigné de l'étude morphologique des cristaux pour étudier les rotations possibles du plan de polarisation qu'ils induisaient, ce qui l'avait conduit à mieux caractériser l'hémiédrie des tartrates.


<im_start>user

**English:** The **hemihedria** is, moreover, non-superposable.

**French:**


<im_start>assistant

Figure 8: Selected contexts for the 72^nd^ sentence of an MERSENNE article using PMI, including the 51^st^, 56^th^ and 58^th^ sentences containing the term *hemihedria*.
system

Translate the following English source text to French, considering the provided English context and its French translations:

**English:** His appointment to the University of Lille, in an industrial environment that led him to study amyl alcohols, helped to reorient his scientific activity, but he remained mainly driven by his hypothesis that “molecular dissymmetry” was the prerogative of the living.

**French:** Sa nomination à l'université de Lille, dans un environnement industriel qui l'a conduit à étudier les alcools amyles, a contribué à réorienter son activité scientifique, mais il est resté principalement guidé par son hypothèse selon laquelle la « dissymétrie moléculaire » était l'apanage du vivant.

**English:** In the memoir of May 22, 1848[4], it is the "tartrates" and the "paratartrates" which are primarily considered, but the young scientist seeks fruitful generalisations: "It will be said, and rightly so: All organic substances that deviate from the plane of polarisation when they are dissolved will therefore enjoy hemihedria.

**French:** Dans le mémoire du 22 mai 1848 [4], ce sont les « tartrates » et les « paratartrates » qui sont principalement considérés, mais le jeune scientifique cherche des généralisations fructueuses : « On dira, et à juste titre : toutes les substances organiques qui dévient du plan de polarisation lorsqu'elles sont dissoutes profiteront donc de l'hémiédrie.

**English:** At this stage, Pasteur had moved away from the morphological study of the crystals to the study of the possible rotations of the plane of polarisation which they induced, which had led him to better characterise the hemihedria of tartrates.

**French:** À ce stade, Pasteur s'était éloigné de l'étude morphologique des cristaux pour étudier les rotations possibles du plan de polarisation qu'ils induisaient, ce qui l'avait conduit à mieux caractériser l'hémiédrie des tartrates.



user

**English:** This preceded the second thesis, devoted to amyl alcohols, where Pasteur examined the crystallographic question thus posed, and where he writes to have not succeeded in inducing crystalline hemihedria.

**French:** 



assistant

<caption>Figure 9: Selected contexts for the 159th sentence of an MERSENNE article using PMI, including the 51st and 58th sentences containing the term hemihedria.</caption>