Enhancing Neural Machine Translation Through Target Language Data: A $k$NN-LM Approach for Domain Adaptation
Authors: Abudurexiti Reheman, Hongyu Liu, Junhao Ruan, Abudukeyumu Abudula, yingfeng luo, Tong Xiao, JingBo Zhu
Accepted at: acl25

Enhancing Neural Machine Translation Through Target Language Data: A kNN-LM Approach for Domain Adaptation

Abudurexit Reheman 1, Hongyu Liu 1, Junhao Ruan 1, Abudukeyumu Abudula 1, Yingfeng Luo 1, Tong Xiao 1,2, Jingbo Zhu 1,2 *,
1 School of Computer Science and Engineering, Northeastern University, Shenyang, China
2 NiuTrans Research, Shenyang, China
rexiti_neu@outlook.com
{xiaotong,zhujingbo}@mail.neu.edu.cn

Abstract
Neural machine translation (NMT) has advanced significantly, yet challenges remain in adapting to new domains. In scenarios where bilingual data is limited, this issue is further exacerbated. To address this, we propose kNN-LM-NMT, a method that leverages semantically similar target language sentences in the kNN framework. Our approach generates a probability distribution over these sentences during decoding, and this distribution is then interpolated with the NMT model’s distribution. Additionally, we introduce an n-gram-based approach to focus on similar fragments, enabling the model to avoid the noise introduced by the non-similar parts. To enhance accuracy, we further incorporate cross-lingual retrieval similarity to refine the kNN probability distribution. Extensive experiments on multi-domain datasets demonstrate significant performance improvements in both high-resource and low-resource scenarios. Our approach effectively extracts translation knowledge from limited target domain data, and well benefits from large-scale monolingual data for robust context representation.

1 Introduction
With the introduction of deep learning techniques, especially the revolutionary models like Transformer, neural machine translation (NMT) has made significant progress in recent years (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017). Despite these advancements, NMT often suffers from translating in new domains, which is called domain adaptation (Koehn and Knowles, 2017; Isabelle et al., 2017; Lee et al., 2018; Farajian et al., 2017). The primary reason for this issue is the domain mismatch between the training and test datasets. To address this, researchers have developed various strategies to enhance NMT model’s domain adaptation capabilities through the incorporation of external domain knowledge or similar examples (Cao and Xiong, 2018; Bulté and Tezcan, 2019; Xu et al., 2020; Meng et al., 2022; Reheman et al., 2023). A particularly promising approach integrates non-parametric methods with NMT, where the non-parametric methods rely on an external memory of additional translation examples (Khandelwal et al., 2021). However, the heavy reliance on high-quality ground-truth bilingual data limits the applicability of these methods in low-resource scenarios.

On the other hand, with the advantages of large data scale, broad domain coverage, and ease of access, monolingual data is widely employed to enhance NMT capabilities (Pang et al., 2024). Back-translation (Sennrich et al., 2016; Caswell et al., 2019; Marie et al., 2020) enriches training data by translating target language data back into the source language using a reverse translation model. Shallow fusion or deep fusion leverages language models that are trained on the target language data to constrain the translation process of NMT (Gülcehre et al., 2015, 2017; Sriram et al., 2018). Although these methods efficiently leverage monolingual data to enhance parametric models, they inevitably increase the training cost.

Recent non-parametric methods, UDA-kNN (Zheng et al., 2021b) and Pseudo-kNN-MT (Reheman et al., 2024), leverage target language data in the kNN-MT framework. The former introduces adapter modules that require NMT model retraining, while the latter pairs similar target sentences with the source input. However, this approach is vulnerable to the noise incorporated by the non-similar parts of the target retrievals and is constrained by the representation capability of the NMT model.

As a non-parametric method, k-nearest neighbor language model (kNN-LM) (Khandelwal et al., 2020) has achieved notable success in language modeling tasks. In the machine translation task,
since the decoder of a transformer NMT model learns a causal language model implicitly, if the target language instances that are semantically similar to the source input sentence can be provided, a non-parametric LM can be utilized to incorporate the relevant translation information.

Building on this idea, we propose our method, kNN-LM-NMT. Specifically, given some semantically similar target sentences, kNN-LM-NMT generates a probability distribution over these sentences in the kNN-LM framework. This probability distribution is subsequently interpolated with the NMT model’s distribution. Additionally, to effectively leverage the similar fragments from the target language sentences, we introduce an n-gram-based approach, which focuses on similar fragments. This enables the model to filter out non-similar parts of the target sentence. Furthermore, to differentiate the contributions of sentences with varying similarity levels, we incorporate cross-lingual retrieval similarity to refine next token probability of the kNN.

We validate our approach on multi-domain datasets for both high-resource and low-resource machine translation tasks. The results demonstrate that our method significantly enhances the translation performance of NMT on both scenarios.

In real-world application, our method can effectively extract useful translation knowledge from a small amount of similar target domain data. At the same time, large-scale target data enables training robust language models for context representation.

2 Preliminaries

For better understanding, we provide an overview of the fundamental concepts underlying kNN-LM and LaBSE.

2.1 kNN-LM

kNN-LM (Khandelwal et al., 2020) is a semi-parametric language modeling approach, which combines the traditional language model with nearest neighbor search techniques, allowing it to leverage large corpora of text data more effectively. Applying kNN-LM involves two main steps: datastore creation and inference using the datastore.

Datastore Creation. Let \( \mathcal{D} \) denote a collection of key-value pairs, where key is a high-dimensional representation of a context from a pretrained language model (LM), and value is the corresponding ground-truth next token. Let \( \mathcal{Y} = \{Y_1, Y_2, ..., Y_n\} \) be a set of sentences and let \( f(\cdot) \) be the mapping function that transfers the context into the vector representation using an LM. For all sentences in \( \mathcal{Y} \), the key-value datastore is created as:

\[
\mathcal{D} = \{(f(y_{1:t-1}), y_t), \forall y_t \in \mathcal{Y} \mid Y \in \mathcal{Y}\}.
\] (1)

Here, the datastore size is equal to the total number of tokens in \( \mathcal{Y} \).

Inference. During the inference phase, the representation of the previously generated tokens, \( y_{1:t-1} \), at each time-step is taken as a query, denoted as \( q = f(y_{1:t-1}) \), to retrieve k-nearest neighbors \( \mathcal{N} \) from the datastore \( \mathcal{D} \). This retrieval process employs vector distance measuring metrics, such as \( L^2 \) distance or cosine similarity, to ascertain the proximity of the context representation to the stored instances. Subsequently, the kNN distribution, \( p_{kNN} \), over the vocabulary is then obtained by normalizing the negative distances with temperature, and aggregating the probability of same tokens afterwards. The kNN distribution is formulated as:

\[
p_{kNN}(y_t|\hat{y}_{1:t-1}) \propto \sum_{(k_j, v_j) \in \mathcal{N}} \mathbb{I}_{y_j = v_j} \exp(-d(q, k_j)/T),
\] (2)

where \( d(\cdot, \cdot) \) represents the distance function that calculates the distances between the query vector and the keys from the datastore and \( T \) is the temperature.

In the end, the final generation probability is calculated by linearly interpolating the kNN distribution with the LM distribution, as:

\[
p(y_t \mid \hat{y}_{1:t-1}) = \lambda p_{kNN}(y_t \mid \hat{y}_{1:t-1}) + (1 - \lambda)p_{LM}(y_t \mid \hat{y}_{1:t-1}),
\] (3)

where \( \lambda \) is the interpolation hyperparameter.

2.2 LaBSE

LaBSE (Language-agnostic BERT Sentence Embedding) (Feng et al., 2022) is a multilingual sentence embedding model designed to generate language-agnostic embeddings that capture semantic meaning across diverse languages.

In cross-lingual retrieval tasks, LaBSE encodes sentences from different languages into a unified representation space. Each sentence is first processed through the model, resulting in a dense vector representation. These representations can
Figure 1: The illustration of the proposed kNN-LM-NMT method. CLS Embedder refers to the cross-lingual sentence embedding model. The Encoder and Decoder at the right-bottom refer to the encoder and decoder of a standard transformer NMT model.

be compared using similarity functions. By computing the similarity of sentences in different languages, LaBSE effectively retrieves semantically similar sentences across languages. In this paper, we use it to retrieve similar target sentences for the source input sentence.

3 Methodology

Given a limited set of similar sentences in the target language, our objective is to extract valuable knowledge from them to enhance the NMT by integrating this knowledge into the translation process. As a robust non-parametric language modeling method, kNN-LM effectively incorporates the probability of the next token into the target generation process through approximate similarity search. Moreover, since these similar sentences exhibit a certain degree of similarity to the input sentence, it is possible to leverage LMs to integrate the relevant translation knowledge from these sentences without referencing the source sentences.

3.1 Retrieving similar target Sentences

Given the source input sentence \( X \) and the target language dataset \( \mathcal{Y} = \{ Y_1, Y_2, ..., Y_n \} \), we utilize the cross-lingual embedding model, LaBSE, denoted as \( e(.) \), to derive their dense vector representations as follows:

\[
h_X = e(X),
\]
(4)
\[
\mathcal{H}_\mathcal{Y} = \{ e(Y) \mid Y \in \mathcal{Y} \}.
\]
(5)

Subsequently, we calculate the Euclidean Distances (also referred to as \( L^2 \) distances) between the source sentence embedding \( h_X \) and each target sentence embedding in \( \mathcal{H}_\mathcal{Y} \). Finally, we rank the target sentences based on their distances and select the top-\( k \) nearest ones as the final retrieved targets.

3.2 Integrating kNN-LM into NMT

Similar to other kNN-based methods, we first construct a kNN-LM datastore \( \mathcal{D} \) using the entire target language dataset. This process is identical to the datastore construction described in §2.1. Both this datastore and the target sentence vector datastore (used for retrieving similar target language sentences in §3.1) are built offline. During the decoding process, the kNN probability distribution for the next token is generated from \( \mathcal{D} \) and then interpolated with the probability distribution from the NMT model. The overall process of this method is illustrated in Figure 1.

At the first step of decoding, we extract a sentence-specific datastore \( \mathcal{D}_{\text{sim}} \) for the source sentence. Specifically, for an input sentence \( X \), we use the method described in §3.1 to obtain a set of similar target sentences, along with their cross-lingual distances. Then, we extract the corresponding keys and values from the kNN datastore based on the similar target sentence IDs and construct triples in the form of \( <key, value, dist> \). Here, “dist” is the sentence-level cross-lingual retrieval distance of a target sentence to which the token belongs. These triples are then stored to construct \( \mathcal{D}_{\text{sim}} \).

At each time-step of the decoding process, we input the previously generated target tokens \( y_{1:t-1} \) into the LM, the same model which is used to
build the kNN datastore, to obtain query vector \( q = f(y_{1:t-1}) \). Next, we use \( q \) to retrieve the k-nearest neighbors from \( \mathcal{D}_{\text{sim}} \) and construct the kNN probability distribution. Unlike the standard kNN probability generation method, we incorporate cross-lingual retrieval distances to refine the kNN probability. This method is described in detail in §3.4. Finally, the kNN distribution is interpolated with the NMT distribution as:

\[
p(y_t \mid x, \hat{y}_{1:t-1}) = \lambda p_{kNN}(y_t \mid \hat{y}_{1:t-1}) + (1-\lambda)p_{NMT}(y_t \mid x, \hat{y}_{1:t-1}).
\] (6)

3.3 Integrating n-gram-based kNN-LM with NMT

Since we utilize similar target language sentences, the non-similar parts may introduce noise. Additionally, due to the auto-regressive nature of causal LMs, the representation at a time-step inherently contains all the information of its prefix. To mitigate the negative impact of noise in the representation, we propose a local representation method based on n-gram sequences. An example of this method is illustrated in Figure 2.

First, we construct n-gram datastores using the target language dataset. Specifically, each sentence in \( \mathcal{Y} \), we input every n-gram segment into the LM to obtain its hidden representation as the key, with the corresponding next token as the value. Hence, the n-gram kNN datastore is constructed as:

\[
\mathcal{D}^{ngram} = \{ (f(y_{t-n:t-1}), y_t), \forall y_t \in Y \\
| Y \in \mathcal{Y} \}.
\] (7)

where \( f(\cdot) \) is the text-to-vector mapping function using an LM, \( Y \) is the target sentence, and \( y_{t-n:t-1} \) is the n-gram segment at time-step \( t \). Notably, we take the hidden representation \( h_t \) as the key for \( y_{t-n:t-1} \), reflecting the nature of casual LMs.

At the first time-step of decoding, we retrieve the similar target sentences and extract a sentence-specific n-gram datastore \( \mathcal{D}^{ngram}_{\text{sim}} \), using the same method described in §3.2. At each time-step of decoding, we extract n-gram segment \( y_{t-n:t-1} \) from the previously generated target sequence \( y_{1:t-1} \) and input \( y_{t-n:t-1} \) into the LM to obtain its representation, which is then taken as query to construct the n-gram kNN distribution from \( \mathcal{D}^{ngram}_{\text{sim}} \). If the length of \( y_{1:t-1} \) is insufficient to extract n-gram segment, the corresponding n-gram kNN distribution is not constructed for that time-step.

Furthermore, we construct multiple n-gram kNN distributions, and take the average of these distributions to obtain the final kNN distribution, which is then interpolated with NMT distribution as:

\[
p(y_t \mid x, \hat{y}_{1:t-1}) = \lambda \frac{1}{n} \sum_1^n p_{kNN}^{ngram} + (1-\lambda)p_{NMT}(y_t \mid x, \hat{y}_{1:t-1}).
\] (8)

3.4 Integrating Cross-lingual Similarity

The generation of kNN probabilities relies on the retrieved similar target sentences, independent of the source sentence information. Nevertheless, the cross-lingual retrieval distance reflects the degree of translational correspondence between them, providing essential information for kNN probability. Therefore, we propose two distinct approaches to integrate cross-lingual retrieval distances into the kNN probability generation framework.

Updating kNN Distance (UD). For an input sentence \( X \), we have the similar target sentence datastore \( \mathcal{D}_{\text{sim}} \) (we denote this in both kNN and n-gram kNN occasions). At time-step \( t \), we feed the previously generated target prefix \( \hat{y}_{1:t-1} \) or the n-gram segment \( \hat{y}_{t-n:t-1} \) into the LM to obtain its representation \( h_q \). Using \( h_q \) as the query (\( q = h_q \)), we retrieve k-nearest neighbors from \( \mathcal{D}_{\text{sim}} \), obtaining kNN distances \( d^{\text{knn}} \) and cross-lingual retrieval distances \( d^{\text{clr}} \) of the top-k neighbors. The kNN probability is constructed based on the multiplica-
tion of \( d^{clr} \) and \( d^{knn} \) as follows:

\[
p_{kNN}(y_t \mid x, \hat{y}_{1:t-1}) \propto \\
\sum_{(k_j, v_j, d^{clr}_j) \in \mathcal{D}_{sim}} \mathbb{I}_{y_j = v_j} \exp(-d^{clr}_j d^{knn}(q, k_j)/T).
\] (9)

**Updating kNN Probability (UP).** Unlike UD, this approach transforms the cross-lingual distances into similarity scores and updates kNN probability accordingly. Specifically, after retrieving \( k \)-nearest neighbors from the datastore, we compute their similarity scores \( s^{clr} \) by applying an exponential function to the negative cross-lingual distance:

\[
s^{clr}_j = \exp(-d^{clr}_j), \quad d^{clr}_j \in \mathcal{D}_{sim}.
\] (10)

Following the construction of the \( k \)NN probability, we multiply each neighbor’s probability by its corresponding similarity score and then aggregate probabilities of identical tokens. The updated \( k \)NN probability is thus formulated as:

\[
p_{kNN}(y_t \mid x, \hat{y}_{1:t-1}) \propto \\
\sum_{(k_j, v_j) \in \mathcal{D}_{sim}} \mathbb{I}_{y_t = v_j} \exp(-d^{knn}(q, k_j)/T) s^{clr}_j.
\] (11)

4 Experiments

We evaluate the effectiveness of our approach on a publicly available multi-domain dataset in high-resource and low-resource translation settings. We also explore the usability of LLM in the \( k \)NN component of our method.

**Models, Datasets and Evaluation Metrics.** We employ Facebook’s champion model on WMT19 German-English (De-En) news translation task¹ (Ng et al., 2019). For the \( k \)NN component, we use their WMT19 English LM ², trained on the monolingual Newscrawl dataset of that year. We test our method on the test sets of IT, Koran, Law and Medical domains of the Multi-domain dataset which is originally introduced in Koehn and Knowles (2017) and resplit by Aharoni and Goldberg (2020). We take the target training data of each domain as the monolingual target language dataset. The data statistics are given in Table 1. The translation quality is evaluated using SacreBLEU (Post, 2018) and COMET-23⁴ (Rei et al., 2022).

<table>
  <tr>
    <th>Split</th>
    <th>WMT19</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
  </tr>
  <tr>
    <td>Train</td>
    <td>33M</td>
    <td>223k</td>
    <td>17k</td>
    <td>467k</td>
    <td>248k</td>
  </tr>
  <tr>
    <td>Valid</td>
    <td>6002</td>
    <td>2000</td>
    <td>2000</td>
    <td>2000</td>
    <td>2000</td>
  </tr>
  <tr>
    <td>Test</td>
    <td>2000</td>
    <td>2000</td>
    <td>2000</td>
    <td>2000</td>
    <td>2000</td>
  </tr>
</table>

Table 1: Statistics of datasets.

**Experimental Settings.** We utilize the cross-lingual embedding model LaBSE (Feng et al., 2022) to convert the sentences from both source and target language datasets into sentence embeddings. We employ dense vector search library FAISS (Johnson et al., 2021) for both the cross-lingual retrieval and the \( k \)NN search from the datastore, using its built-in \( L^2 \) distance function for vector distance measuring. In the cross-lingual retrieval, we retrieve the top-32 similar sentences from the target dataset for each source input sentence. In the \( k \)NN search, we retrieve \( k = 8 \) neighbors from the \( k \)NN datastore. For the key of the \( k \)NN datastore, following the recommendations from Khandelwal et al. (2020); Xu et al. (2023), we extract the input for the “FFN” in the last layer of the LM decoder. Regarding the \( k \)NN temperature, we search it from {0.5, 1, 5, 10, 20, 50}. For interpolation, we search \( \lambda \) from {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}. For \( n \)-gram \( k \)NN method, we set the \( n \)-gram from “1-gram” to “5-gram”. For decoding, we set the beam size to 5 and length penalty to 1.0.

**Baselines.** We compare our method with existing ones that leverage target language data in NMT, especially with non-parametric \( k \)NN methods.

• NMT: The general-domain NMT model.
• Shallow Fusion (Gülçehre et al., 2017): Interpolates the probability distribution of a trained LM with NMT distribution during the inference. For fair comparison, we also finetune the LM on the training data of multi-domain dataset.
• BT (Sennrich et al., 2016): Standard back-translation approach that augments the training data by translating the target data back to the source language.
• Pseudo-\( k \)NN-MT (Reheman et al., 2024): A \( k \)NN-based method that creates pseudo-bilingual data by pairing similar target sentences with source sentences for \( k \)NN-MT.
• UDA-\( k \)NN (Zheng et al., 2021b): Incorporates target language data into \( k \)NN-MT by training a specialized adapter in the transformer NMT model.

¹https://dl.fbaipublicfiles.com/fairseq/models/wmt19.de-en.ffn8192.tar.gz
²https://dl.fbaipublicfiles.com/fairseq/models/lm/wmt19.en.tar.gz
³https://github.com/Unbabel/COMET
<table>
  <tr>
    <th rowspan="2">Methods</th>
    <th colspan="2">IT</th>
    <th colspan="2">Koran</th>
    <th colspan="2">Law</th>
    <th colspan="2">Medical</th>
    <th colspan="2">Average</th>
  </tr>
  <tr>
    <th>BLEU</th>
    <th>COMET</th>
    <th>BLEU</th>
    <th>COMET</th>
    <th>BLEU</th>
    <th>COMET</th>
    <th>BLEU</th>
    <th>COMET</th>
    <th>BLEU</th>
    <th>COMET</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>38.43</td>
    <td>82.46</td>
    <td>17.07</td>
    <td>72.57</td>
    <td>45.99</td>
    <td>85.38</td>
    <td>41.97</td>
    <td>83.16</td>
    <td>35.86</td>
    <td>80.89</td>
  </tr>
  <tr>
    <td>Shallow Fusion</td>
    <td>36.44</td>
    <td>81.05</td>
    <td>17.32</td>
    <td>72.34</td>
    <td>46.15</td>
    <td>84.87</td>
    <td>40.98</td>
    <td>81.73</td>
    <td>35.22</td>
    <td>79.99</td>
  </tr>
  <tr>
    <td>Shallow Fusion(fILM)</td>
    <td>37.57</td>
    <td>81.92</td>
    <td>17.62</td>
    <td>72.71</td>
    <td>47.87</td>
    <td>85.47</td>
    <td>42.15</td>
    <td>82.37</td>
    <td>36.30</td>
    <td>80.62</td>
  </tr>
  <tr>
    <td>UDA-kNN</td>
    <td>40.67</td>
    <td>82.71</td>
    <td>18.98</td>
    <td><b>73.40</b></td>
    <td>51.17</td>
    <td>85.90</td>
    <td>45.95</td>
    <td>83.79</td>
    <td>39.19</td>
    <td>81.45</td>
  </tr>
  <tr>
    <td>Pseudo-kNN-MT</td>
    <td>40.63</td>
    <td>82.51</td>
    <td>18.46</td>
    <td>72.24</td>
    <td>53.03</td>
    <td>84.68</td>
    <td>49.36</td>
    <td>82.43</td>
    <td>40.37</td>
    <td>80.46</td>
  </tr>
  <tr>
    <td>BT-kNN</td>
    <td>41.58</td>
    <td>82.96</td>
    <td>20.35</td>
    <td>73.00</td>
    <td>54.43</td>
    <td>85.96</td>
    <td>49.47</td>
    <td>83.93</td>
    <td>41.46</td>
    <td>81.46</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>41.61</td>
    <td><b>83.06</b></td>
    <td>20.51</td>
    <td>72.93</td>
    <td>54.34</td>
    <td>85.72</td>
    <td>49.73</td>
    <td>83.82</td>
    <td>41.55</td>
    <td>81.38</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td><b>42.47</b></td>
    <td><b>83.10</b></td>
    <td>20.47</td>
    <td>72.91</td>
    <td><b>56.27</b></td>
    <td>85.53</td>
    <td><b>52.57</b></td>
    <td>83.57</td>
    <td><b>42.95</b></td>
    <td>81.27</td>
  </tr>
  <tr>
    <td><i>n</i>-gram-kNN-UD</td>
    <td>42.18</td>
    <td>83.06</td>
    <td><b>20.63</b></td>
    <td>73.28</td>
    <td>55.61</td>
    <td>86.07</td>
    <td>50.07</td>
    <td>83.91</td>
    <td>42.12</td>
    <td>81.58</td>
  </tr>
  <tr>
    <td><i>n</i>-gram-kNN-UP</td>
    <td>42.45</td>
    <td>83.01</td>
    <td>20.39</td>
    <td>73.26</td>
    <td><b>56.91</b></td>
    <td><b>86.19</b></td>
    <td><b>50.83</b></td>
    <td><b>83.97</b></td>
    <td><b>42.65</b></td>
    <td><b>81.61</b></td>
  </tr>
</table>

Table 2: SacreBLEU and COMET scores of WMT19 De-En model on the multi-domain test sets. The best results are highlighted in bold, and the second-best results are underlined for clarity.

• BT-kNN: Generates synthetic bilingual data by translating the target dataset back to the source language, which is then used for kNN-MT datastore construction.

4.1 Main Experiment

We conduct this experiment on the multi-domain test set using the WMT19 De-En model. Regarding the reverse NMT model for BT-kNN, we use Facebook’s WMT19 En-De model 4, which was trained using the same dataset as the forward model.

Results. The experimental results presented in Table 2 reveal observations below: first, our proposed method demonstrates superior performance compared to other baseline models, dominantly achieving the best and second best results on both BLEU and COMET metrics across most domains, except the COMET score for the Koran domain. Notably, our method even outperforms the competitive BT-kNN baseline, which utilizes bilingual data generated through back-translation. Secondly, the results indicate that while shallow fusion offers limited enhancement to NMT performance after learning the target domain knowledge by fine-tuning, other non-parametric methods exhibit substantial improvements in translation quality. This underscores the effectiveness of non-parametric approaches in rapidly adapting to new domain. Third, our approach consistently benefits from the n-gram enhancement, which suggests that the n-gram method effectively mitigates the noise introduced by non-similar parts of the target sentence to some extent. Lastly, comparing the performance of the cross-lingual retrieval distance incorporation methods, UP performs better than UD. This is mainly because the nature of the “softmax” function that generates kNN probability using the distance. When the kNN distance is either too large or too small, the UD approach, which multiplies the kNN distance by the cross-lingual retrieval distance, has a limited impact on the resulting probability distribution. In contrast, the UP approach directly updates the kNN probability distribution, enabling more effective utilization of the cross-lingual similarity.

4.2 Low Resource Machine Translation

Target language monolingual data has been demonstrated to be beneficial in low-resource machine translation scenarios. To evaluate the performance of our method under such conditions, we conduct experiments on the multi-domain dataset under low-resource settings. We employ the German-to-English low-resource machine translation model provided by Pseudo-kNN-MT (Reheman et al., 2024). This model was trained on 500k bilingual data obtained by uniformly sampling from the cleaned WMT21 De-En news translation task dataset. For the training data of the target LM, we mix the target data of this 500k dataset with the target training data from each domain of the multi-domain dataset. Subsequently, we train a 12-layer decoder-only GPT model (Radford et al., 2019) with an embedding dimension of 768. To ensure consistency in the target vocabulary, we utilize the target language vocabulary of the NMT model for tokenization. In addition to the baselines above, we also compare our method with the standard BT in this experiment.

Results. The experimental results in Table 3 demonstrate that our method achieves the best performance across three domains. In terms of average
<table>
  <tr>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
    <th>Average</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>28.69</td>
    <td>10.68</td>
    <td>28.42</td>
    <td>30.17</td>
    <td>24.49</td>
  </tr>
  <tr>
    <td>BT</td>
    <td>31.72</td>
    <td>12.73</td>
    <td><b>41.32</b></td>
    <td>38.08</td>
    <td>30.96</td>
  </tr>
  <tr>
    <td>UDA-kNN</td>
    <td>31.74</td>
    <td>14.44</td>
    <td>32.52</td>
    <td>35.68</td>
    <td>28.60</td>
  </tr>
  <tr>
    <td>Pseudo-kNN</td>
    <td>30.61</td>
    <td>13.97</td>
    <td>36.75</td>
    <td>38.06</td>
    <td>29.85</td>
  </tr>
  <tr>
    <td>BT-kNN</td>
    <td>32.35</td>
    <td>13.28</td>
    <td>36.65</td>
    <td>37.41</td>
    <td>29.92</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>30.80</td>
    <td>14.72</td>
    <td>38.41</td>
    <td>38.89</td>
    <td>30.71</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td>31.76</td>
    <td>14.73</td>
    <td>41.07</td>
    <td>40.95</td>
    <td>32.13</td>
  </tr>
  <tr>
    <td>n-gram-UD</td>
    <td>31.68</td>
    <td><b>15.02</b></td>
    <td>40.11</td>
    <td>40.13</td>
    <td>31.74</td>
  </tr>
  <tr>
    <td>n-gram-UP</td>
    <td><b>32.94</b></td>
    <td>14.66</td>
    <td>41.10</td>
    <td><b>41.52</b></td>
    <td><b>32.56</b></td>
  </tr>
</table>

Table 3: SacreBLEU scores of WMT21 De-En sample-500k model on the multi-domain test sets. COMET scores are given in Appendix A.

<table>
  <tr>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
    <th>Average</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>35.90</td>
    <td>16.95</td>
    <td>45.04</td>
    <td>39.77</td>
    <td>34.42</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>38.03</td>
    <td>19.58</td>
    <td>51.62</td>
    <td>45.97</td>
    <td>38.80</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td>38.92</td>
    <td>19.21</td>
    <td>52.94</td>
    <td>46.61</td>
    <td>39.42</td>
  </tr>
  <tr>
    <td>n-gram-UD</td>
    <td>40.22</td>
    <td><b>20.88</b></td>
    <td>53.23</td>
    <td>47.88</td>
    <td>40.55</td>
  </tr>
  <tr>
    <td>n-gram-UP</td>
    <td><b>41.31</b></td>
    <td>20.68</td>
    <td><b>54.91</b></td>
    <td><b>49.13</b></td>
    <td><b>41.51</b></td>
  </tr>
</table>

Table 4: SacreBLEU scores of WMT19 De-En Llama3.1-dict model on the multi-domain test sets. COMET scores are given in Appendix A

BLEU scores, our four proposed methods consistently outperform all other baselines except the BT. While BT achieves optimal performance in the Law domain, our method performs comparably in this domain. From the perspective of data scale, BT, as a parametric method, benefits more from a larger monolingual data. Therefore, it achieves the best performance in the law domain, which contains 467k sentences. Conversely, our method, as a non-parametric approach, focuses more on the similarity between the target and source sentence. Consequently, even in the Koran domain with only 17k sentences, our method significantly outperforms BT.

4.3 Large Language Models in kNN Component

Large language models (LLMs) demonstrate remarkable success across various NLP tasks (Brown et al., 2020; Alves et al., 2024; Luo et al., 2025; Muenninghoff et al., 2025; Zheng et al., 2025). Trained on massive corpora, LLMs exhibit strong representational capabilities. Given the critical role of context representation in our method for searching nearest neighbors, we investigate the application of LLMs within the kNN component. We employ Llama 3.1-8B 5. It is essential to keep vocabulary consistency between the target language of NMT model and the LLM to integrate their probability distributions. Accordingly, we train an NMT model on the WMT19 De-En dataset using the Llama dictionary. The model architecture of this NMT model is identical to that in the main experiment. To determine the optimal key, we conduct preliminary experiments using both the “FFN-input” and “Hidden-state” from the last layer of the Llama model (see Appendix B for detailed results). The “last-FFN-input” yields better performance and is thus selected as the key.

As presented in Table 4, the experimental results reveal that while the baseline NMT model exhibits a performance decline compared to the one from main experiment - attributable to the Llama vocabulary implementation - our method substantially enhances translation performance over the NMT baseline. Although the results closely match those of the main experiments, they do not fully align, mainly due to the relative weakness of the NMT model used, whose generated distributions are less accurate for interpolation.

5 Analysis

5.1 Impact of Cross-lingual Similarity Integration

To evaluate the contribution of cross-lingual retrieval similarity to translation performance, we conduct an ablation study by removing the integration mechanism of cross-lingual retrieval distance, denoted as \( d^{clr} \). As shown in Table 5, removing \( d^{clr} \) leads to performance degradation in both methods. Specifically, the average SacreBLEU scores of kNN-UD and kNN-UP drop significantly from 41.55 and 42.95 to 38.81, highlighting the strong reliance on \( d^{clr} \). In contrast, n-gram-UD remains nearly unchanged, while n-gram-UP experiences a slight decline.

We attribute these results to the inherent noise-handling capabilities of the methods. The n-gram method constructs probabilities from similar n-gram segments, making it easier to exclude noise introduced by non-similar parts. In contrast, the kNN method relies on the entire prefix, making it more sensitive to noise. The \( d^{clr} \) helps determine the level of noise in the sentence for kNN, explaining its stronger reliance on this mechanism.

5 https://huggingface.co/meta-llama/Meta-Llama-3.1-8B
<table>
  <tr>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
    <th>Avg</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>38.43</td>
    <td>17.07</td>
    <td>45.99</td>
    <td>41.97</td>
    <td>35.86</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>41.61</td>
    <td>20.51</td>
    <td>54.34</td>
    <td>49.73</td>
    <td>41.55</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td>42.47</td>
    <td>20.47</td>
    <td>56.27</td>
    <td>52.57</td>
    <td>42.95</td>
  </tr>
  <tr>
    <td>No \( d^{sim} \)</td>
    <td>39.93</td>
    <td>18.99</td>
    <td>50.06</td>
    <td>46.25</td>
    <td>38.81</td>
  </tr>
  <tr>
    <td>\( n \)-gram-UD</td>
    <td>42.18</td>
    <td>20.63</td>
    <td>55.61</td>
    <td>50.07</td>
    <td>42.12</td>
  </tr>
  <tr>
    <td>\( n \)-gram-UP</td>
    <td>42.47</td>
    <td>20.47</td>
    <td>56.27</td>
    <td>52.57</td>
    <td>42.95</td>
  </tr>
  <tr>
    <td>No \( d^{sim} \)</td>
    <td>42.18</td>
    <td>20.57</td>
    <td>55.64</td>
    <td>50.40</td>
    <td>42.20</td>
  </tr>
</table>

Table 5: Effect of removing cross-lingual retrieval distance on SacreBLEU scores. “No \( d^{sim} \)” indicates the removal of cross-lingual retrieval similarity.

![SacreBLEU scores of WMT19 De-En model on Law and Medical domains across similarity-based groups.](page_186_682_563_312.png)

Figure 3: SacreBLEU scores of WMT19 De-En model on Law and Medical domains across similarity-based groups.

5.2 Impact of Target Language Similarity

Our method benefits from the similar content in the target sentence. To investigate the target similarity impact, we sort retrieved target sentences by similarity and divide them into four groups, with group 1 having the highest similarity and groups 2–4 showing progressively lower. We conduct experiments using the WMT19 De-En model on the Law and Medical domains.

The results in Figure 3 show performance declines sharply with decreasing similarity, followed by a more gradual decline. This highlights the critical importance of target sentence similarity, as higher similarity provides more informative cues for translation.

5.3 Impact of Cross-lingual Model

The retrieval of semantically similar sentences from the target data with high recall rates is relevant to the representational capabilities of cross-lingual models. To this end, we evaluate our method using several cross-lingual models, including E5 (Wang et al., 2022b), LASER2 (Hefferman et al., 2022), and MuSR (Gao et al., 2023), under the main experiment settings. Since the multi-domain dataset is bilingual, we simulate a scenario with near-perfect target similarity recall: we first retrieve similar source sentences using edit distance, as:

\[
d(x_i, x_j) = ED(x_i, x_j)/\max(|x_i|, |x_j|),
\]

where \( ED(\cdot, \cdot) \) is Edit Distance function and \( |x| \) is the length of \( x \), then extract the corresponding target language sentences as the final target retrievals.

The distributions of cross-lingual retrieval distances vary significantly across these models. For fair comparison, we scale their distance distributions to align with the scale of LaBSE in the main experiment. The experimental results are shown in Table 6.

<table>
  <tr>
    <th>Models Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
    <th>Avg</th>
  </tr>
  <tr>
    <td>- NMT</td>
    <td>38.43</td>
    <td>17.07</td>
    <td>45.99</td>
    <td>41.97</td>
    <td>35.86</td>
  </tr>
  <tr>
    <td>E5 kNN-UD</td>
    <td>41.87</td>
    <td>20.47</td>
    <td>54.86</td>
    <td>49.49</td>
    <td>41.67</td>
  </tr>
  <tr>
    <td>E5 kNN-UP</td>
    <td>42.13</td>
    <td>20.60</td>
    <td>56.09</td>
    <td>51.89</td>
    <td>42.68</td>
  </tr>
  <tr>
    <td>E5 \( n \)-gram-UD</td>
    <td>42.12</td>
    <td>20.64</td>
    <td>56.10</td>
    <td>49.69</td>
    <td>42.14</td>
  </tr>
  <tr>
    <td>E5 \( n \)-gram-UP</td>
    <td>42.11</td>
    <td>20.72</td>
    <td>56.84</td>
    <td>50.03</td>
    <td>42.43</td>
  </tr>
  <tr>
    <td>Laser2 kNN-UD</td>
    <td>41.27</td>
    <td>19.88</td>
    <td>54.16</td>
    <td>46.93</td>
    <td>40.56</td>
  </tr>
  <tr>
    <td>Laser2 kNN-UP</td>
    <td>41.60</td>
    <td>20.27</td>
    <td>56.00</td>
    <td>49.53</td>
    <td>41.85</td>
  </tr>
  <tr>
    <td>Laser2 \( n \)-gram-UD</td>
    <td>41.41</td>
    <td>20.58</td>
    <td>55.41</td>
    <td>48.84</td>
    <td>41.56</td>
  </tr>
  <tr>
    <td>Laser2 \( n \)-gram-UP</td>
    <td>41.43</td>
    <td>20.52</td>
    <td>56.18</td>
    <td>50.01</td>
    <td>42.04</td>
  </tr>
  <tr>
    <td>MusR kNN-UD</td>
    <td>41.37</td>
    <td>20.57</td>
    <td>54.44</td>
    <td>49.59</td>
    <td>41.49</td>
  </tr>
  <tr>
    <td>MusR kNN-UP</td>
    <td>42.17</td>
    <td>20.66</td>
    <td>56.73</td>
    <td>51.67</td>
    <td>42.81</td>
  </tr>
  <tr>
    <td>MusR \( n \)-gram-UD</td>
    <td>42.20</td>
    <td>20.78</td>
    <td>55.87</td>
    <td>49.96</td>
    <td>42.20</td>
  </tr>
  <tr>
    <td>MusR \( n \)-gram-UP</td>
    <td>42.27</td>
    <td>20.61</td>
    <td>57.03</td>
    <td>51.08</td>
    <td>42.75</td>
  </tr>
  <tr>
    <td>SrcED kNN-UD</td>
    <td>41.11</td>
    <td>20.32</td>
    <td>54.61</td>
    <td>48.34</td>
    <td>41.10</td>
  </tr>
  <tr>
    <td>SrcED kNN-UP</td>
    <td>43.24</td>
    <td>19.86</td>
    <td>58.11</td>
    <td>52.42</td>
    <td>43.41</td>
  </tr>
  <tr>
    <td>SrcED \( n \)-gram-UD</td>
    <td>42.43</td>
    <td>20.38</td>
    <td>55.88</td>
    <td>49.61</td>
    <td>42.08</td>
  </tr>
  <tr>
    <td>SrcED \( n \)-gram-UP</td>
    <td><b>43.33</b></td>
    <td>19.91</td>
    <td><b>57.92</b></td>
    <td><b>51.91</b></td>
    <td><b>43.27</b></td>
  </tr>
</table>

Table 6: SacreBLEU score comparison of applying different cross-lingual retrieval models. The best and second best results are highlighted in bold and underline, respectively.
<table>
  <tr>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>177.44</td>
    <td>182.04</td>
    <td>191.93</td>
    <td>179.63</td>
  </tr>
  <tr>
    <td># Target Sents</td>
    <td>223k</td>
    <td>17k</td>
    <td>467k</td>
    <td>248k</td>
  </tr>
  <tr>
    <td>kNN Dstore Size</td>
    <td>3.16M</td>
    <td>0.52M</td>
    <td>19.07M</td>
    <td>6.90M</td>
  </tr>
  <tr>
    <td>UDA-kNN</td>
    <td>29.14</td>
    <td>101.61</td>
    <td>12.88*</td>
    <td>17.06</td>
  </tr>
  <tr>
    <td>BT-kNN</td>
    <td>29.75</td>
    <td>102.83</td>
    <td>13.42*</td>
    <td>17.47</td>
  </tr>
  <tr>
    <td>kNN(UD/UP)</td>
    <td>30.04</td>
    <td>31.92</td>
    <td>27.00</td>
    <td>28.01</td>
  </tr>
  <tr>
    <td>n-gram(UD/UP)</td>
    <td>29.13</td>
    <td>28.87</td>
    <td>30.33</td>
    <td>29.10</td>
  </tr>
</table>

Table 7: SacreBLEU score comparison of applying different cross-lingual retrieval models. “*” represents using half of the datastore due to the GPU memory limitation.

5.4 Efficiency

We evaluate the translation speed of our method against other kNN-based methods under the main experiment settings. We use a batch size of 1 and a FlatL2 FAISS index. Results with tokens/second are given in Table 7.

Experimental results demonstrate that the performance of UDA-kNN and BT-kNN is highly sensitive to the kNN datastore size, with decoding speed degrading significantly with larger datastores. In our method, the target sentence vector datastore and the kNN-LM datastore are constructed offline. Therefore, apart from the decoding time of the NMT model, the time overhead of our method mainly occurs as below. (1) \( T_{clr} \): cross-lingual retrieval at the first step of the decoding, performed once per sentence; (2) \( T_{lm} \): computing the query vector via a forward-pass of the LM on the current target sequence or the n-grams; (3) \( T_{knn} \): searching for kNNs from the similar target datastore \( \mathcal{D}_{sim} \). Although our method incurs additional \( T_{clr} \) and \( T_{lm} \), the orders of magnitude smaller \( \mathcal{D}_{sim} \) (e.g., a datastore of 32 target similar sentences with an average length of 30 tokens, totaling 960 vectors) enables a smaller \( T_{knn} \), avoiding scalability issues.

6 Related Work

Utilizing Monolingual Data in NMT. In NMT research, the scarcity of bilingual data often lets researchers turn to the utilization of monolingual data. To our knowledge, Gülçehre et al. (2015) first investigated the usage of monolingual data by training an LM on the target language data and integrating it in the inference phase. Sriram et al. (2018) proposed to keep the LM fixed, but train the NMT model from scratch by taking LM as a component. Back-translation (Sennrich et al., 2016; He et al., 2016; Fadaee et al., 2017; Edunov et al., 2018) enhances the training data by translating the target language data back into the source language. Among the recent works, Cai et al. (2021) proposed jointly training the NMT model with a retriever that retrieves similar target sentences from corpora and enhancing NMT with the retrievals. Reheman et al. (2024) proposed to leverage similar target sentences to construct pseudo bilingual sentences and perform kNN-MT. Their difference from ours is that we take the cross-lingual similarity and n-gram segments into account and combine a non-parametric LM with the NMT.

kNN Based Methods in NMT. kNN-LM (Khandelwal et al., 2020) and kNN-MT (Khandelwal et al., 2021) represent the first attempts of kNN methods on language modeling and NMT task, respectively. For efficiency, a series of studies has been proposed to optimize kNN-MT in MT community. Martins et al. (2022b); Wang et al. (2022a) proposed to reduce the datastore scale or key dimension. By utilizing word alignment, Meng et al. (2022) narrows down the search space in advance to accelerate the kNN search. Instead of retrieving one token in each step, Martins et al. (2022a) proposed to retrieve a chunk of tokens at a time. From the perspective of denoising, Zheng et al. (2021a) and Jiang et al. (2022) proposed to dynamically control the information that comes from nearest neighbors.

7 Conclusion

We propose an innovative approach to enhance the domain adaptability of NMT by leveraging target language data. We incorporate translation information from semantically similar target sentences using the kNN-LM framework. To fully utilize the similar target segments, we also propose an n-gram-based method. Additionally, we further improve the model’s robustness to noise by incorporating cross-lingual retrieval-based similarity. The experimental results demonstrate significant performance improvements over baselines across high-resource and low-resource translation settings. In future work, we will explore methods that are more robust to neighbors with varying distances.

8 Limitations

Despite its powerful performance, our method still has limitations as follows. Since our approach applies a cross-lingual retrieval model to search similar sentences from the target language data, its per-
formance directly affects the recall rate of similar sentences and, consequently, the overall translation. Additionally, ensuring the consistency between the vocabulary of the LM and the target vocabulary of the NMT model is essential for interpolating the probability distributions.

9 Acknowledgements

This work was supported in part by the National Science Foundation of China (Nos. 62276056 and U24A20334), the Fundamental Research Funds for the Central Universities, the Yunnan Fundamental Research Projects (No. 202401BC070021), and the Program of Introducing Talents of Discipline to Universities, Plan 111 (No.B16009).

References

Roei Aharoni and Yoav Goldberg. 2020. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7747–7763. Association for Computational Linguistics.

Duarte M. Alves, José Pombal, Nuno Miguel Guerreiro, Pedro Henrique Martins, João Alves, M. Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. 2024. Tower: An open multilingual large language model for translation-related tasks. CoRR, abs/2402.17733.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.

Bram Bulté and Arda Tezcan. 2019. Neural fuzzy repair: Integrating fuzzy matches into neural machine translation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 1800–1809. Association for Computational Linguistics.

Deng Cai, Yan Wang, Huayang Li, Wai Lam, and Lemao Liu. 2021. Neural machine translation with monolingual translation memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 7307–7318. Association for Computational Linguistics.

Qian Cao and Deyi Xiong. 2018. Encoding gated translation memory into neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 3042–3047. Association for Computational Linguistics.

Isaac Caswell, Ciprian Chelba, and David Grangier. 2019. Tagged back-translation. In Proceedings of the Fourth Conference on Machine Translation, WMT 2019, Florence, Italy, August 1-2, 2019 - Volume 1: Research Papers, pages 53–63. Association for Computational Linguistics.

Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 489–500. Association for Computational Linguistics.

Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. Data augmentation for low-resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, pages 567–573. Association for Computational Linguistics.

M. Amin Farajian, Marco Turchi, Matteo Negri, Nicola Bertoldi, and Marcello Federico. 2017. Neural vs. phrase-based machine translation in a multi-domain scenario. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 2: Short Papers, pages 280–284. Association for Computational Linguistics.

Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 878–891. Association for Computational Linguistics.

Pengzhi Gao, Liwen Zhang, Zhongjun He, Hua Wu, and Haifeng Wang. 2023. Learning multilingual sentence representations with cross-lingual consistency regularization. In Proceedings of the 2023 Conference on
Empirical Methods in Natural Language Processing: EMNLP 2023 - Industry Track, Singapore, December 6-10, 2023, pages 243–262. Association for Computational Linguistics.

Çağlar Gülçehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loïc Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. On using monolingual corpora in neural machine translation. CoRR, abs/1503.03535.

Çağlar Gülçehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, and Yoshua Bengio. 2017. On integrating a language model into neural machine translation. Comput. Speech Lang., 45:137–148.

Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. 2016. Dual learning for machine translation. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 820–828.

Kevin Heffernan, Onur Çelebi, and Holger Schwenk. 2022. Bitext mining using distilled sentence representations for low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2101–2112. Association for Computational Linguistics.

Pierre Isabelle, Colin Cherry, and George F. Foster. 2017. A challenge set approach to evaluating machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 2486–2496. Association for Computational Linguistics.

Hui Jiang, Ziyao Lu, Fandong Meng, Chulun Zhou, Jie Zhou, Degen Huang, and Jinsong Su. 2022. Towards robust k-nearest-neighbor machine translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 5468–5477. Association for Computational Linguistics.

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with gpus. IEEE Trans. Big Data, 7(3):535–547.

Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest neighbor machine translation. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.

Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, NMT@ACL 2017, Vancouver, Canada, August 4, 2017, pages 28–39. Association for Computational Linguistics.

Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fanjiang, and David Sussillo. 2018. Hallucinations in neural machine translation.

Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, and Jingbo Zhu. 2025. Beyond decoder-only: Large language models can be good encoders for machine translation. CoRR, abs/2503.06594.

Benjamin Marie, Raphael Rubino, and Atsushi Fujita. 2020. Tagged back-translation revisited: Why does it really work? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5990–5997. Association for Computational Linguistics.

Pedro Henrique Martins, Zita Marinho, and André F. T. Martins. 2022a. Chunk-based nearest neighbor machine translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 4228–4245. Association for Computational Linguistics.

Pedro Henrique Martins, Zita Marinho, and André F. T. Martins. 2022b. Efficient machine translation domain adaptation. CoRR, abs/2204.12608.

Yuxian Meng, Xiaoya Li, Xiayu Zheng, Fei Wu, Xiaofei Sun, Tianwei Zhang, and Jiwei Li. 2022. Fast nearest neighbor machine translation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 555–565, Dublin, Ireland. Association for Computational Linguistics.

Niklas Muenninghoff, Hongjin Su, Liang Wang, Nan Yang, Furui Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2025. Generative representational instruction tuning. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net.

Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair’s WMT19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation, WMT 2019, Florence, Italy, August 1-2, 2019 - Volume 2: Shared Task Papers, Day 1, pages 314–319. Association for Computational Linguistics.

Jianhui Pang, Baosong Yang, Derek Fai Wong, Yu Wan, Dayiheng Liu, Lidia S. Chao, and Jun Xie. 2024. Rethinking the exploitation of monolingual data for low-resource neural machine translation. Comput. Linguistics, 50(1):25–47.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 - November 1, 2018, pages 186–191. Association for Computational Linguistics.

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.

Abudurexit Reheman, Yingfeng Luo, Junhao Ruan, Jianliang Zhang, Anxiang Ma, Tong Xiao, and JingBo Zhu. 2024. Exploiting target language data for neural machine translation beyond back translation. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 12216–12228. Association for Computational Linguistics.

Abudurexit Reheman, Tao Zhou, Yingfeng Luo, Di Yang, Tong Xiao, and Jingbo Zhu. 2023. Prompting neural machine translation with translation memories. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pages 13519–13527. AAAI Press.

Ricardo Rei, José G. C. de Souza, Duarte M. Alves, Chrysoula Zerva, Ana C. Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: unbabel-ist 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation, WMT 2022, Abu Dhabi, United Arab Emirates (Hybrid), December 7-8, 2022, pages 578–585. Association for Computational Linguistics.

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics.

Anuroop Sriram, Heewoo Jun, Sanjeev Satheesh, and Adam Coates. 2018. Cold fusion: Training seq2seq models together with language models. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings. OpenReview.net.

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 3104–3112.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008.

Dexin Wang, Kai Fan, Boxing Chen, and Deyi Xiong. 2022a. Efficient cluster-based k-nearest-neighbor machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 2175–2187. Association for Computational Linguistics.

Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furui Wei. 2022b. Text embeddings by weakly-supervised contrastive pre-training. CoRR, abs/2212.03533.

Frank F. Xu, Uri Alon, and Graham Neubig. 2023. Why do nearest neighbor language models work? In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 38325–38341. PMLR.

Jitao Xu, Josep Maria Crego, and Jean Senellart. 2020. Boosting neural machine translation with similar translations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 1580–1590. Association for Computational Linguistics.

Tong Zheng, Yan Wen, Huiwen Bao, Junfeng Guo, and Heng Huang. 2025. Asymmetric conflict and synergy in post-training for llm-based multilingual machine translation. CoRR, abs/2502.11223.

Xin Zheng, Zhirui Zhang, Junliang Guo, Shujian Huang, Boxing Chen, Weihua Luo, and Jiajun Chen. 2021a. Adaptive nearest neighbor machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Virtual Event, August 1-6, 2021, pages 368–374. Association for Computational Linguistics.

Xin Zheng, Zhirui Zhang, Shujian Huang, Boxing Chen, Jun Xie, Weihua Luo, and Jiajun Chen. 2021b. Non-parametric unsupervised domain adaptation for neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 4234–4241. Association for Computational Linguistics.

A COMET Scores

We give some experimental results on COMET metric here. Table 8 and Table 9 correspond to Table 3 and 4 from the main part, respectively.
<table>
  <tr>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
    <th>Average</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>77.40</td>
    <td>59.88</td>
    <td>75.67</td>
    <td>77.36</td>
    <td>72.58</td>
  </tr>
  <tr>
    <td>BT</td>
    <td>79.65</td>
    <td>63.12</td>
    <td>81.19</td>
    <td>81.07</td>
    <td>76.26</td>
  </tr>
  <tr>
    <td>UDA-kNN</td>
    <td>78.66</td>
    <td>63.24</td>
    <td>78.07</td>
    <td>79.50</td>
    <td>74.87</td>
  </tr>
  <tr>
    <td>Pseudo-kNN</td>
    <td>78.21</td>
    <td>62.82</td>
    <td>76.75</td>
    <td>78.86</td>
    <td>74.16</td>
  </tr>
  <tr>
    <td>Bt-kNN</td>
    <td>78.50</td>
    <td>62.69</td>
    <td>77.60</td>
    <td>79.41</td>
    <td>74.55</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>78.23</td>
    <td>62.18</td>
    <td>77.21</td>
    <td>78.96</td>
    <td>74.15</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td>78.29</td>
    <td>62.14</td>
    <td>77.47</td>
    <td>79.28</td>
    <td>74.30</td>
  </tr>
  <tr>
    <td>n-gram-UD</td>
    <td>78.38</td>
    <td>62.01</td>
    <td>77.43</td>
    <td>79.15</td>
    <td>74.24</td>
  </tr>
  <tr>
    <td>n-gram-UP</td>
    <td>78.53</td>
    <td>62.18</td>
    <td>76.97</td>
    <td>79.41</td>
    <td>74.27</td>
  </tr>
</table>

Table 8: COMET scores of WMT21 De-En sample 500k model on the multi-domain test sets.

<table>
  <tr>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
    <th>Average</th>
  </tr>
  <tr>
    <td>NMT</td>
    <td>82.62</td>
    <td>72.46</td>
    <td>85.61</td>
    <td>83.41</td>
    <td>81.03</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>82.49</td>
    <td>72.33</td>
    <td>85.40</td>
    <td>83.25</td>
    <td>80.87</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td>82.30</td>
    <td>72.17</td>
    <td>86.19</td>
    <td>83.05</td>
    <td>80.93</td>
  </tr>
  <tr>
    <td>n-gram-UD</td>
    <td>83.33</td>
    <td>73.06</td>
    <td>86.24</td>
    <td>84.17</td>
    <td>81.70</td>
  </tr>
  <tr>
    <td>n-gram-UP</td>
    <td>83.45</td>
    <td>73.17</td>
    <td>86.51</td>
    <td>84.28</td>
    <td>81.85</td>
  </tr>
</table>

Table 9: Comet scores of WMT19 De-En Llama3.1-dict model on the multi-domain test sets.

B Other results

To determine either the “last FFN input” or the “last hidden state” of the Llama3.1-8B model gets better representation, we conduct this experiment. The results reveal that the “last FFN input” demonstrate superior performance. Consequently, we select it in the formal experiments.

<table>
  <tr>
    <th>Key</th>
    <th>Methods</th>
    <th>IT</th>
    <th>Koran</th>
    <th>Law</th>
    <th>Medical</th>
  </tr>
  <tr>
    <td rowspan="2">FFN</td>
    <td>NMT</td>
    <td>35.90</td>
    <td>16.95</td>
    <td>45.04</td>
    <td>39.77</td>
  </tr>
  <tr>
    <td>kNN-UD</td>
    <td>38.03</td>
    <td>19.58</td>
    <td>51.62</td>
    <td>45.97</td>
  </tr>
  <tr>
    <td></td>
    <td>kNN-UP</td>
    <td>38.92</td>
    <td>19.21</td>
    <td>52.94</td>
    <td>46.61</td>
  </tr>
  <tr>
    <td rowspan="2">Hid</td>
    <td>kNN-UD</td>
    <td>38.31</td>
    <td>18.64</td>
    <td>46.24</td>
    <td>41.87</td>
  </tr>
  <tr>
    <td>kNN-UP</td>
    <td>38.56</td>
    <td>18.72</td>
    <td>46.26</td>
    <td>41.82</td>
  </tr>
</table>

Table 10: SacreBLEU scores of WMT19 De-En Llama3.1-dict model on the multidomain test sets. “FFN” denotes the last ffn input as the key, “Hid” denotes last hidden state as the key.

Reviewer paper summaries: This paper studies incorporating monolingual data to enhance NMT for domain adaptation. They first use a cross-lingual embedder LaBSE to find similar target setences of the input source sentence.  They then find the vector representation of these sentences from a knn-LM datastore. During inference, they fed predicted target tokens in the knn-LM and use the embedding as query to search for k-nearest neighbors in the vector set built by those similar targets and build the knn distribution which is then integrated with the NMT model's output distribution to help generate the next token. The authors conducted experiments showing that the proposed method can improve BLEU and COMET scores on five domains for De-En multi-domain test sets.

The author also proposed to use n-gram vectors spaces for the knn-LM datastore, and search for different n-gram instead of entire sentences during the knn search, to mitigate the effect of unsimilar parts in similar target sentences. Experimental results show slight imporvements of the n-gram approach.

This paper introduces kNN-LM-NMT, a novel method for enhancing domain adaptation in neural machine translation by leveraging semantically similar monolingual target-language data. The approach integrates a k-nearest-neighbor language model (kNN-LM) with the NMT decoder, interpolating predictions from retrieved target sentences. It introduces n-gram-based filtering to reduce noise and cross-lingual retrieval similarity to refine probability estimation. Evaluations across high- and low-resource, multi-domain settings show significant improvements over baselines, including back-translation and shallow fusion. The method is scalable and effective with large LMs, offering practical gains in BLEU and COMET scores. It highlights the untapped value of target-side data in non-parametric NMT.
Summary of strengths: - The proposed method show performance improvements compared to other knn-based methods for NMT domain adaptation.
- The proposed method uses LaBSE and a knn-LM model to get rid of bilingual parallel training data store that are commonly required in knn-based approaches.
- The use of LaBSE for searching similar target sentences also reduces the knn-search base size.

* The mathematical basis of the paper is thorough and explained adequately.
* Monolingual target-language data integration is a good solution to the limited bilingual data problem of domain adaptation (especially in low-resource languages. Even though this paper does not venture into low-resource languages)
* Using LaBSE to weight retrieved sentences is an inspired choice. This, (especially the results in section 5.2) reminded me of the observations by [Ranathunga et al (2024)](https://aclanthology.org/2024.eacl-long.52.pdf) on sorting and filtering data before NMT.
Summary of weaknesses: - Since the proposed approach incorporates two external model compared to regular knn-MT models, LaBSE and a knn-LM. It would be useful to analyze the computaional cost and/or inference efficiency of the proposed method, as efficiency is a key limitation for knn-based method. This is also mentioned by the authors in this paper. However, they do not conduct such experiments.
- The authors compared to many knn-based methods by none-of them uses actual bilingual data to build the knn datastore. I can imagine the authors tends to limit the comparison in non-bilingual data scope, but the multi-domain training dataset does contain bilingual data which could be directly used. Maybe a comparison to regular knn-MT is also interesting, to see whether only using monolingual target data can surpass bilingual data based knn-MT.

* Per-sentence per n-gram arrangement of kNN here raise some doubts about scalability. 
* For a very mathematical paper, the mathematical justification of n-gram averaging and similarity integration is lacking.
* The role vocabulary alignment players here (possibly as a limitation) should be discussed,
Comments, suggestions and typos: Questions:
- I think the introduction of LaBSE reduces the searching space during inference to only some similar sentences or n-grams to some extent. This lead to two interesting questions:

1. Does your method decode more efficiently compared to other knn-based methods? Since the searching space is normally greatly reduced by computing an externalt LaBSE knn search only once for each input sentence.
2. LaBSE use to select similar target sentences. The author showed that target sentence similarity is important. But what if the first step cannot find sufficiently similar sentences? Does it affect the final translation result?

Typos
- L263 Specifically, each sentence -> Specifically, for each sentence

* Line 431-432: "UP performs better that UD". I think you meant to say "UP performs better than UD" here.
* Line 506: exhibit -> exhibits *
Overall assessment scores: 3

4