Domain Translation with Monolingual Lexical Distribution
Authors: Yusuke Sakai
Zhi Qu
Hidetaka Kamigaito
Taro Watanabe
Xiaojiang Liu
Venue: Accepted by TMLR ()

Abstract: Neural machine translation (NMT) often demands a large amount of high-quality training data when adapting to a new domain with a carefully designed fine-tuning strategy.  However, constructing a sufficient amount of parallel data for training poses challenges even for fine-tuning. This work proposes to fine-tune a generic NMT model using only the monolingual lexical distribution estimated from a small amount of in-domain data in the target language. Word frequency plays a critical role in analyzing the differences among corpora in various fields, e.g., psycholinguistic and language education, and our challenge lies in whether we can fit a model using the naive statistics collected from a target language domain in NMT. We leverage a variant of energy-based models (EBMs) based on Conditional Distributional Policy Gradients (CDPG) with a large number of EBMs to constrain the fine-tuning process with lexical distribution. We conduct experiments across four translation directions and four domain datasets, totaling 16 domain adaptation scenarios. The results demonstrate that our method enables robust domain shift while mitigating catastrophic forgetting, achieving effective domain adaptation using only a small amount of monolingual resources.

Published in Transactions on Machine Learning Research (01/2026)
Domain Translation with Monolingual Lexical Distribution
Yusuke Sakai
sakai, vakai.yuskker ST www.sukk...9....aiisi...
Naro Institute of Science and Technology (NAIST) Equal Contribution
Zhi Qu
qu.zhi. polliippo@icc. naist.jp
Nara Institute of Science and Technology (NAIST)
Equal Contribution
Hidetaka Kamigaito
kamigaito..o Saiiitttttt naist.jj
Nara Institute of Science and Technology (NAIST)
Taro Watanabe
taro@is.
Nara Instätute of Science and Technology (NAIST)
Xiaojiang Liu
riaojiang liu com
Apple
Reviewed on OpenReview: https: // opereviee. net/ forum? id= UKLBobrFCR
Abstract
Neural machine translation (NMT) often demands a large amount of high-quality training data when adapting to new domain with carefully designed fine-tuning strategy. How- ever, constructing sufficient amount of parallel data for training poses challenges even for fine-tuning. This work proposes to fine-tune a generic NMT model using only the mono- lingual lcxical distribution estimated from a small amount in-domain data in the target language. Word frequency plays a critical role in analyzing the differences among corpora in various fields, e.g., psycholinguistic and language oducation, and our challenge lies in whether we can fit a model using the naive statisties collected from target language domain in NMT. We leverage variant of mergy-based models (EBMs) based Conditional Distributional Policy Gradients (CDPG) with a large number of EBMs to constrain the fine-tuning process with lexical distribution. We conduct experiments across four translation directions and four domain datasets, totaling 16 domain adaptation scenarios. The results demonstrate that our method enables robust domain shift while mitigating catastrophic forgetting, achieving offective domain adaptation using only a small amount of monolingual resources.
1 Introduction
Thanks to the crawling technology and corpora construction efforts (Tiedemann 2012 Bañón al. 2020 Morishita al. 2022), we have access to abundant parallel translation data, resulting in the development of high-performance pre-trained neural machine translation (NMT) models. However, NMT models suffer from performance degradation when translating text from the domains different from the domain of the training corpus due to the mismatch of the domain-specific terminologies (Kochn & Knowles 2017 Shen et al.] 2021 Pang et al.) (2025). While general-purposs parallel translation data is abundantly available, automatically collecting a sufficient amount of domain-specific parallel data challenging, and such special- purpose translation tends to require custom-made parallel data due to its specialized environment, e.g., terminologies in the medical domain textual styles for a specific company, demanding specialist construct check the quality of the parallel data (Chu & Wang 2018 Yeganova et al. 2021

---

Published in Transactions on Machine Learning Research (01/2026)
In this study, we leverage casily accessible pre-trained NMT models and propose to adapt general domain NMT model to specific domain by using only the monolingual lexical distribution obtained from mono- lingual domain data in the target language. Lexical distribution, or word frequency, is crucial in analyzing the differences of corpora (Kilgarriff 1997; Rayson & Garside] 2000), and the simple statistics have been investigated in various fields, such as psycholinguistic (Garlock el al. 2001 and language learning (Laufer & Nation, 1995). to quantify language usages. Our challenge lies whether we can adapt a model to a new domain using the naive statistics, ie,, monolingual lexical distribution, which are already employed in various language assessments and easily estimated from the specific domain in the target language without consulting a specialist to translate or check the bilingual data qualities.
However, naively performing fine-tuning of a pre-trained NMT model and forcibly changing its token predic- tion distribution can lead to catastrophic forgetting issues, ranging from the loss of fluency (Korbak et al. 2022: Choshen et al. 2020 Kiegeland Kreutzer] 2021) to degradation in non-specific domains (Saunders & DeNoefel 2024 Gu & Feng) 2020; [Tommson al. 2019 thereby causing a reduction in translation performance. To achieve the domain shilt. while reducing calastrophic lorgetting, i.e., harmlessly modifying the model's knowledge without degradation in generalization performance excessive overfit to a spe- cific domain, we represent the lexical distribution of the target domain as conditional anergy-basee models (EBMs) and approximate the EBMs using Conditional Distributional Policy Gradients (CDPG) (Korbak iet al.
2022) which is a variant of the Generation under Distributional Control (GDC) framework (Khalifa jet al. 2021 Korbak et al. (2022) had only verified the effectiveness of CDPG for small shifts, such as translating numeral nouns (e.g., "two") as digits (e.g., "2"), while our method employs sparse features, i.c.,
the monolingual lexical distribution of the target domain by treating each token-level statistic as feature, enabling domain shifts in fine-grained manner without catastrophic forgetting.
We confirm its effectiveness across several domain adaptation benchmarks (Tian et al. 2014 Koehn & Knowles 2017 Aharoni & Goldberg 2020) and scenarios, covering four language directions and four domain datasots, resulting in a total of 16 scttings, thus we achieved unsupervised domain adaptation using only target-side domain data. Moreover, we propose DYNAMIC CDPG, which dynamically adjusts parameters using a small amount of bilingual validation data (or back-translan data in a fully unsupervised setting) to scarch for better configurations and analyze idcal scttings for unsupervised domain adaptation. Our analysis of CDPG and DYNAMIC CDPG reveals that while selecting parameters sensitively can sometimes yield the best results, a simple CDPG can sufficiently achieve domain shift while reducing catastrophic forgetting.
To summarize this work, our contributions are as follows:
We first demonstrate that unsupervised domain adaptation of neural machine translation can be per- formed robustly under challenging yet realistic scenario where only small amount of monolingual in-domain data is available. We leverage the lexical distribution, i.e., the word-frequency statistics of the target-domain data, and train a large number of EBMs within the CDPG framework to achieve robust domain shift without catastrophic forgetting and to demonstrate strong resilience to noise.
We explore the applicability of CDPG to more realistic downstream tasks, i.e., domain translation. While prior GDC work has only been validated under small, pointwise shifts, we extend the frame- work to utilize large numbers of EBMs ructed from rich feature sets such as lexical distributions, demonstrating that large-scale domain shifts can be effectively modeled by CDPG.
We propose DYNAMIC CDPG, which dynamically adjusts its configurations to explore better CDPG settings for unsupervised domain adaptation. This adaptive mechanism enables the model to better capture and align with the largel-domain distribution. As a result, DYNAMIC CDPG can yield further improvements in some cases and achieve more robust domain shifts.
We conduct a comprehensive evaluation across 16 experimental scenarios and thorough analyses, including lexical-distribution similarity, qualitative case studies, robustness to noise and randomness, and parameter investigations, to clarify when CDPG is the most effective. This provides a practical contribution for low-resource sensitive domains where parallel data is scarce.
2

---

Published in Transactions on Machine Learning Research (01/2026)
2 Conditional Distributional Policy Gradients
Conditional Distributional Policy Gradients (CDPG) (Korbak et al. 2022) approximates the generative probabilities of a language model to a target distribution while preventing catastrophic forgetting. It softly modifies the pre-trained parameters 0 by slightly shifting its generation distribution using energy-based models (EBMs) through fine-tuning. We briefly summarize the minimal background needed to follow our method and defer further details to Korbak et al. (2022).
We define the pre-trained conditional language model a(z]cc where c is context, i.e., an input source language sentence, and o is sentence, i.e., in a target language, sampled from the entire distribution X' given c. We introduce an EBM pe( as controlled distribution defined as:
1 ((x))))((x.c) Ze
Pe(x)
where Ze= - a(zle)b((x, c) is a partition function that normalizes the entire EBM Pe(x)) and b(x, c) is a control condition function which is when a certain constraint is met, e.g,, digits are used in a when translating numeral nouns in c. When b(xx, e) is reduced to product of binary scorers (x(x) E {0, 13 in order to encode rich constraints as proposed by Khalifa et al. (2021), the EBM is formulated as:
pe
point (x)
a (x c) II (((x),
(2)
Zuoint
where "point" (proint and Zgoint) indicates pointwise (binary) constraints.
However, such binary constraints can represent only two outcomes, namely whether a sentence satisfies the condition, or not. This limitation makes impossible to express preferences over partial fulfilmmntt, e.g., enforcing that a constraint should hold with probability 0.5. For instance, when attempting to mitigate gender bias in generated lext, one may want the model to produce female and male entities with equal probability, that is, 0.5 for each. In narrative generation or stylistic control, one may further require skewed preferences such as generating certain attributes with probabilitics 0.8 and 0.2. Such requirements cannot be encoded using binary scorers alone, because they only allow all-or-nothing fulfillment of the constraint.
To overcome this limitation, Khalifa et al. (2021) proposed a distributional constraint method for the un- conditional case p(z) 1 (a(z)b(az and Kruszcwski et al. (0023) extended this idea to the conditional EBM with an exponential family form as follows:
(3) where À is a parameter vector controlling the desired expected feature values encoded in d. "dist" simply indicates the distributional version of the EBM, in contrast to the pointwise binary case above.
Pe
dist (x|A) =
Zeist a(re) exp(A Φ(x c)),
The parameter À is determined through finc-tuning by starting from random initialization and iteratively updating by stochastic gradient descent (SGD) to minimize the loss function considering a distribution over contexts T(c) as follows:
VaCnoe (A) = (owr(o)) (A( p.
(4)
where i is the probability vector specifying the desired feature distribution, and the inner expectation (=))) i.., the feature moments under the controlled EBM, estimated through self-normalized importance sampling using a(-), following Korbak et al. 2002).. In the previous example, if the neutral gender is expected, the desired probability is 0.5 for female binary scorer. Levef refers to the cocfficient updale objective for adjusting the parameter vector À so that the controlled distribution matches the desired feature expectations.
However, since the EBM Pe(x)) in Equation 1 and its exponential variant of Equation 3 is not an autore- gressive language model, cannot be used directly for inference. Therefore, training conducted using the autoregressive model (=(ccc to approximate p across contexts on average by minimizing the expected cross-entropy loss CE(-) between (x]e) and multiple Pe of the EBM as follows:
(()) Bewr(() CE (pthet (12), c)).
(5)
3

---

Published in Transactions on Machine Learning Research (01/2026)
The gradient of this objective takes the following importancc--weighhed form:
(o) ewr(() Vo CF. (peia((
(-12), To(- (((- c))
(6)
= E (e) B papiti (z A) Vo log (x) c)
(7)
Pe List (=
B = Ban
Vy log T c).
(8)
To x
The loss is estimated via importance sampling from mg. By iteratively applying these updates to 0, T can approximate the generative probability of the target EBM, enabling autoregressive generation. Note that CDPG is fine-tuning method; thus, it does not introduce any changes to parameter size, model architecture, or inference speed. For clarity, we note that CDPG alternates between two steps: (i) updating À so that the controlled EBM matches the desired feature expectations, and (ii) updating 0 so that the autoregressive model approximates this controlled distribution. These two coupled updates constitute the core mechanism, and full derivations can be found in Korbak et al. (2022).
3 Domain Adaptation by CDPG
3.1 Adaptation to Monolingual Lexical Distribution
We leverage monolingual data in the specific domain in the target language, e.g., English reports in the medical domain, and propose domain adaptation for NMT with CDPG using only the subword frequency information as features. When applying CDPG for NMT, the source sentence corresponds to context drawn from the source language in general domain, e.g., German news, and the ideal target sentence is derived from plls((m) (m))). For training CDPG under distribution constraints in Equation 3 it requires a binary scorer dx(x,c) c) and a paramcter X for each fcature.
Under this setting, we employ 6(z, c) as features whether each subword of the target domain is included in we set the probability of each constraint, p. as the ratio of the frequency of subwords in the whole text in the target domain as follows:
æ. Moreover, when learning the parameter vector À Equation !
Fregiarget (x:4) P4 Enjex Fregtarget(x.))
(9)
where taret denotes the frequency of each subword x; in the larget texl in the vocabulary X. By performing the above operations, we attempt to address the domain shift by utilizing the frequency of all subwords of the target domain text. Since this feature selection only uses data, from the target side, the creation of the EBM model only requires the target side domain text.
Note that the computational cost of each CDPG iteration is dominated by two components: (i) estimating the feature moments (=>) e) in Equation 4 and (ii) the autoregressive LM update Equation For N contexts and M samples per context, the moment estimator evaluates Φ(2,0) c) only for the subwords that actually appear in each sampled sequence æc. Consequently, the per-sample cost scales with the sequence length L, rather than with the full feature dimension (Φ). Intuitively, constructing the sparse lexical histogram for sampled sequence is O(L), and computing the importance weights over the M samples in context is linear in M. Therefore, the overall EBM-side complexity across N contexts is O(N Moreover, because the feature scorers factor across tokens and samples, the EBM-side computations parallelize trivially over (c, .:) The LM update, i.e., the importance-weiggted forward forward/backward /backward passes of Ta, also O(NML) and typically dominates the wall-clock time. Thus, despite the large ambient feature space, the effective per- iteration complexity remains MLN. We operate on sparse, sequence-derived features that admit efficient vectorized and parallel computation, avoiding any need to process the full vocabulary-sized feature vector.
3.2 Dynamic CDPG
EBM is iteratively updated by Equation g to approximate the generative language model toward the ex- pected probability distribution of the target domain. The procedure involves generating multiple sentences
4

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm 1: Dynamic CDPG (Sketch)
Given : Dev set Dev; target features μi; feature extractor Φ; metric M; outer iters T; LRs 7a, 100 schedules & [0.5, 0.4, 0.3, 0.2, 0.11, B 1066, 0.7, 0.8, 0.9, 1.0] diss (x A)
Input : Pre-trained LM a(æ c); Conditional EBM Pe
Output Fine-tuned autoregressive model my
No <- initialize A; S t 1; p + Sli::
// initialization
2 for <- T do
11 (1) EBM coefficients update (Equation
Sample contexts {c} T()); Estimate E Bzupfin [db] via self-normalized IS using nucleus sampling al: lop-p p); // push to target features (Equation
A TA [E]Ø] -M)
// (2) Autoregressive model update (Equation
Sample (c) with ~ =a(-
dist
Pe
(x
+ 0 no w V log (
// importance-weighted update (Equation 5
(3) Dev evaluation and schedule update (Dynamic core part)
- M(Te, Dao))
// e.g., BLEU
10
if 8 then
8" < igiti41
1/ accept: advance within S
12
else
R.
S
13
A S i+1
// reject: switch schedule
14 if i S then
15
break
// stop when a schedule exhausted // set next top-p
p ++ Sli]
17 return
x conditioned on context c. To sample æ from the entire distribution X' given C, we often employ nucleus sampling or ancestral sampling decoding strategy (Khalifa et al. 2021 Meng et al.) 2022 Korbak al. 2022 Nucleus sampling (Holtzman et al. 2020) selects tokens in descending order of probability until their cumulative probability mass reaches a threshold p, and then samples from this set according to the multino- mial distribution. The nucleus sampling parameter lop-p controls the diversity of the generated outputs, and when p == 1, i.., ancestral sampling, it produces fully diverse samples from the full multinomial distribution. A higher top-p produces more diverse outputs, whereas a lower top-p yields more conservative ones.
However, when there is large gap between the distribution of the pre-trained model and that of the target domain, simple ancestral sampling, which produces fully diverse samples, is not effective (Eikema al. 2022
Kim et al.] 2025). In practice, modifying the sampling distribution by changing the sampling method adjusting its parameters can make sampling more effective in practical settings (Go et al.) 2023 Freitag et al. 2023 Nadcem et al.) 2020 Kamigaito et al. 2025). Therefore, the sampling diversity must be controlled to obtain better contrastive samples for precise scoring, and CDPG should apply different top-p settings for different domains, e.g.; higher top-p for larger gap in distance, and lower top-p values for closer domains. Furthermore, when the EBM becomes closer to the target domain as training proceeds, a lower value of top-p will be sufficient, but it still demands higher value of top-p for the larger distance. Therefore, introduce DYNAMIC CDPG, which dynamically adjusts top-p at each ileration of the approxination to the searching for better configurations of CDPG.
EBM in Equation !
Algorithm[oove luttlinss the procedure of DYNAMIC CDPG. DYNAMIC CDPG leverages a bilingual development set to monitor training progress and guide adjustments to the sampling strategy. The core idea of DYNAMIC
The development refers the text used generate features. These data be constructed from the targot-side monolingual corpus via back-translation, as described later. Therefore, the development remains completely disjoint from the data, ensuring that functions proper held-out, validation
5

---

Published in Transactions on Machine Learning Research (01/2026)
CDPG is to divide the training process into several iterations, then start with constant parameter for top- P. and reconsider it in each training iteration such that a smaller top-p will be selected in the next iteration if a larger top-p leads to inferior performance on the development set (lines 10-17). Note that if synthetic bilingual data can be obtained via back-translation from the target in-domain data, the setting can still be regarded as fully unsupervised. For each iteration, we use an evaluation method, such as BLEU (Papineni ct al. 2002), to assess the model's performance (line 10) and decide whether to accept the update.
Specifically, we heuristically define two candidate schedules for top-p: A [0.5, 0.4, 0.3, (descending). which gradually reduces sampling diversity and encourages the model to fit more closely to the target features, and IB- = [0.6, 0.7, 0.8, 0.9, 1.0] (ascending), which increases sampling diversity and serves as a fallback exploration schedule. We start the iteration with the first element of A as the value of top-p. If an iteration is accepted, the algorithm proceeds to the next iteration with the second element of & (line 12); if rejected, the algorithm switches to B (line 14) and continues through its elements. The process terminates once all values in either & or B have been exhausted (lines 15-16). Our preliminary studies showed that the training under DYNAMIC CDPG is always stable under our top-p scheduling.
4 Experimental Setup
4.1 Datasets
We conduct experiments with four translation pairs of English to German (en->de), German to English (de->en), English to Chinese (en-tzh), and Chinese to English (zh->en). For pairs involving de, we collect four domains, including IT, Medical, Law, and Koran from the public corpue released by Koehn & Knowles (2017;: ps Aharoni & Goldberg 20200, where each domain has 2,000 sentences for the development set and test sct, respectivcly For pairs involving zh, we collect four domains, including Education, Laws, Thesis, and Science, from the UM-Corpus (Tian et al., 2014)), which is public" with high quality. Althongh this corpus provides 456- 790 sentences for test sets in those 4 domains, the development, set is not provided. Therefore, we randomly select 3,000 sentences from the training data for each domain as the development sets. Moreover, we use the development sct. of WMT from 2018 2022 (Bojar et al. 2018 Barrault et al. 2019 2020: Akhbardeh el al. 2021 Koemi et al. 2022). i.e., 14,482 translation instances of the newsdev set from a news domain, to train CDPG for all translation directions by treating them as a generic domain data set. Specificallyy, the contexts T(c) are collected from the 14,482 source language sentences of the newsdev set, and the domain features j are derived from the target language sentences of the domain-specific instances.
4.2 Models
Baselines We employ four open-source NMT models (Tiedemann & Thottingal 2020 from Hugging Face Transformer (Wol et al. 2020) as backbones in our experiments, denoted as TRAINED These models are based on the Transformer architocture (Vaswani et al. 2017) and are trained OPUS (Tiedemann 2012) with the same [onfiguratiorr comprising the 6 encoder and decoder layers, 8 attention heads, embedding size of 512, and an inner size of 2048. Since CDPG fine-tunes all parameters, we first establish a baseline by naively fine-tuning the model on the development sets, denoted as FINE-TUNED. We also adopt LoRA (Hu et al. 2022) as a second bascline, which adapts attention weights with an inner rank of 8. The baseline is then fine-tuned using the same setup as FINE-TUNED, offering robust ness and reduced susceptibilitty to catastrophic forgetting when working with smaller datasets (Xu et al. 2023: Biderman et al. 2024). We set the batch size to 128 and the learning rate to 2e-7 for both settings with the Adam optimiier (Kingma & Ba (Sennrich et al. 2016), denoted as BACK-TRANS
2017j. As another baseline, we conduct
htttps://githbb.com com comm/ooeeabaonii wwwps//wissssssdddeeeeooooea donain-clustera
"iivn the low quality of this corpus, we clean it and re-align the test set using de as the basis to avoid potential bias in evaluation. Thee low quality includes but is not limited to repetition, no alignment, and noise. Furthermore, the refined test data is completely unseen, ena mabling evaluation without any data contamination issues in the existing training corpus (Raunak
& Menazs 2022). The cleaned datasets are publicly available at https://github. http:::/ggthubb.commmasss---pp con/nasstt-nlp/cddp
http: //lp:ct//
http://dabd www.cii........... uiac..maa.nnn /un mwt.:oooouuuu///////////
http:///ggggggacccceoeee wttp///////////aaaaaaaaaa
7 Detail
6

---

Published in Transactions on Machine Learning Research (01/2026)
Specifically, we generate source-language data using reverse-directio model and the same target-language data used for fine-tuning. The model is then fine-tuned on the generated data using the same settings. All fine-tuning experiments are conducted for 10 epochs. The checkpoint achieving the best performance on the development set is selected as basclines.
For training the parameter vector X in Equation 4 we set the batch size to 8 and the learning rate 1Ia to 0.05 with constant learning rate scheduler based on the training loss in our preliminary studies. Likewise, for fine-tuning CDPG model parameters 0 in Equation 5 we set the batch size to 128, the number of epochs to 10, and the learning rate To to 2c-5 with a constant lcarning rate scheduler and the Adam optimizer. We always set top-p to 0.5 in training À and fine-tuning 8. Moreover, we set the character length of the considered features, i.c., subwords, to be no less than to filter insignificant features, and the input texts are pre-processed by the tokenizer in each pre-trained model. We used disco (Kruszesski al. 2023) to implement the CDPG.
CDPG
Dynamic CDPG We maintain the hyperparameters of CDPG for DYNAMIC CDPG. DYNAMIC CDPG additionally uses a bilingual development set to guide the dynamic selection of the top-p value at each iteration, as described in Section 3.2 We construct this development set via back-translation from the larget-side monolingual corpus, similar to BACK-TRANS. Therefore, the procedure remains fully unsupervised and is completely isolated from the test data, just in CDPG. In our preliminary experiment comparing back-translated bilingual data and the actual bilingual development set, the tuning results were the same. One possible reason is that the larget-side lexical distribution plays a more important role than the source- side or full parallel data. Thus, we use translaand data for DYNAMIC CDPG while maintaining the unsupervised setting. We set each iteration of DYNAMIC CDPG to 10 epochs. We use BLEU (Papineni et al., 2002) to calculate the validation score for each epoch. Additionally, we set a threshold that requires at least three improvements in the validation score for an iteration to be accepted. The initial learning rate of each subsequent iteration is set by dividing that of the previously accepted iteration by the square root of the number of epochs, to ensure training stability.
4.3 Evaluation
We set the beam size of 4 for each model to generate translations for the entire test set, and did not employ nucleus sampling (Holtzman et al.) 2020) in the final evaluation for consistent evaluation in all settings. Then, translations are evaluated by four automatic MT evaluation methods: 1) Confidence (Müller et al.
2019 Wang et al. 2020), calculated as the average of the Softmax probabilities assigned to each generated token by the NMT system. 2) BLEU (Papineni et al., 22002,, asscssod with the implementation of SacreBLEU Postt 2018) to measure the surface-level similarities, 3) NIST (Doddington) 2002), which is similar to BLEU but gives special attention to low- fow-freqyyncy words to assess the qualities of domain-specific terminologies, and 4) BERTScore (Zhang et al., 2020), which reports embedding similarities by Precision, Recall, and F' scores, where the F1 score being the harmonic mean of Precision and Recall.
5 Experimental Results
Table g shows the experimental results for en-tde and de-ten translation pairs, and Table 2 shows the results for en- and zh-ten translation pairs.
Finding 1: Naive supervised fine-tuning methods (almost) fail at domain adaptation with limited in-domain data. First, compared to PRE-TRAINED both FINE-TUNED and LoRA,, which naively utilize supervision signals from parallel in-domain sentences, generally fail to achieve improvements, except for slight gains in Medical of en-de,, Laws of en-tzh, and Thesis and Science of zh-ten. Furthermore, TRANS, which synthesizes pseudo-parallel data for fine-tuning, often performs comparably or worse than FINE-TUNED, which uses a small amount of curated in-domain data. These results indicate that limited-resouree settings, such as domain adaptation, naive adaptation from a pure NMT model, such PRE-TRAINED, not only fails to improve performance but often leads to degradation.
7

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Our main evaluation results for the en- and de-tn translation pairs. PRE-TRAINED indicates the performance of the original models without any fine-tuning, while the other methods are described in Section 1.2 Conf. is the abbreviation of Confidence, and P and R represent the Precision and Recall scores of BoRTScore, respectively. The best score in cach block, which is divided by domain and language pair, is highlighted in bold. The checkmark (V) indicates that parallel in-domain sentences were used as supervision signals. Identical scores between DYNAMIC CDPG and CDPG, such as Koran domain in de-ten, indicate that DYNAMIC CDPG rejected all updates after initialization. Red cells indicate improvements from the base model PRE-TRAINED to the CDPG-basce model.
en en-dde
de -ee
Domain
Method
Conf.
BLEU
NIST
P
R
F1
Conf.
BLEU
NIST
P
R.
F1
PRE-TRAINED
68.39
27.58
5.97
87.48
87.70
87.52
72.02
38.80
7.96
94.93
94.92
94.91
/ FINE-TUNED
67.91
27.92
6.04
87.38
87.60
87.42
71.76
38.83
7.95
94.94
94.93
94.92
/ LORA
67.79
26.88
5.83
87.33
87.56
87.37
71.46
38.32
7.86
94.92
94.91
94.91
BACK-TRANS
67.90
27.89
6.01
87.38
87.59
87.41
71.49
38.35
7.86
94.93
94.92
94.91
CDPG
74.44
29.01
6.25
87.68
87.77
87.67
77.91
39.79
8.30
94.95
94.94
94.93
DYNAMIC CDPG
79.36
30.78
6.58
88.00
87.87
87.89
77.65 .6.6
40.55
8.34
95.01
94.96
94.98
PRE-TRAINED
75.93
43.19
8.45
91.55
91.17
91.31
78.06
45.50
8.47
96.65
96.50
96.57
/ FINE-TUNED
75.71
43.23
8.46
91.53
91.14
91.29
77.77
45.48
8.47
96.64
96.50
96.56
/ LORA
75.50
43.56
8.52
91.55
91.15
91.30
77.72
44.31
8.35
96.61
96.49
96.54
Medical
BACK-TRANS
75.50
43.56
8.52
91.57
91.17
91.32
77.65
45.47
8.55
96.64
96.49
96.56
CDPG
80.85
42.54
8.60
91.61
91.28
91.40
82.84
44.56
8.56
96.57
96.50
96.53
DYNAMIC CDPG
82.32
43.51
8.54
91.60
91.20
91.36
82.84
44.56
8.56
96.57
96.50
96.53
TRAINEDD
72.49
44.82
9.01
89.38
89.11 11
89.22
72.89
51.75
10.05
96.06
95.75
95.90
/ FINE-TUNED
72.08
44.83
9.01
89.39
89.10
89.22
72.53
51.70
10.04
96.06
95.74
95.89
/ LORA
72.05
44.80
9.01
89.42
89.12
89.25
72.55
51.67
10.04
96.05
95.73
95.89
Law
BACK-TRANS
72.05
44.62
8.97
89.39
89.10
89.22
72.45
51.69
10.05
96.06
95.74
95.89
CDPG
77.36
44.12
9.05
89.33
89.17
89.22
78.12
51.61
10.12
96.02
95.72
95.86
DYNAMIC CDPG
78.18
44.87
9.03
89.40
89.09
89.22
73.02
51.64
10.15
96.07
95.73
95.89
TREINED
61.51
18.90
5.25
81.59
80.18
80.84
59.23
20.86
5.66
91.95
91.07
91.49
/ FINE-TUNED
61.39
18.86
5.24
81.56
80.16
80.82
58.80
20.81
5.65
91.94
91.06
91.48
/ LORA
61.18
18.86
5.24
81.54
80.13
80.80
58.94
20.83
5.65
91.94
91.05
91.48
Koran
BACK-TRANS
61.15
18.84
5.24
81.53
80.13
80.79
58.78
20.79
5.65
91.94
91.06
91.49
CDPG
67.00
18.40
5.26
81.46
80.06
80.72
64.75
20.94
5.67
91.90
91.09
91.48
DYNAMIC CDPG
61.30
18.85
5.25
81.63
80.16
80.85
64.77
20.94
5.67
91.90
91.09
91.48
Finding 2: CDPG improves confidences and performances low-frequency, domain-specifie terms. Next, compared to PRE-TRAINED, although CDPG consistently improves Confidence, its perfor- mance in BLEU and BERTScore varies across domains. Specifically, in domains such as IT' of en and Education of ent+2h, where other baseline methods show only limited gains, CDPG achieves improvements across all metrics. In contrast. domains like Law and Koran of en Medical of de-ren, and Laws of en-tzh, the improvements in BLEU and BERTScore are limited. However, NIST scores, which place greater emphasis on low-frequency words, still improve in all cases. This indicates that the consistent increases in Confidence and NIST scores are due to improved handling of domain-specific terms, which are often struggle to capture in general MT evaluation metrics such as BLEU because of their relatively low frequency. There- fore, compared to the baselines, domain adaptation using CDPG is shown to be more robust, even in the settings where access to parallel in-domain data is limited or unavailable.
Finding 3: While Dynamic CDPG often provides slight improvements, CDPG with fixed parameters already demonstrates sufficient performance. DYNAMIC CDPG aims to improve MT metrics, such as BLEU, by dynamically controlling and selecting parameters based on evaluations using a small amount of parallel in-domain development data at each epoch. Since BLEU is used as the guiding signal, DYNAMIC CDPG consistently achieves higher BLEU scores than CDPG. In domains such as IT of ent+de, Thesis of en- and Education of zh-ten, DYNAMIC CDPG selects parameters that further
8

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: Our main evaluation results for the en en-zhh and zh-ten translation pairs. corresponding information are the same as in Table
Notation and other
-2h
Domain
Method
Conf.
BLEU
NIST
P
R.
F1
Conf.
BLEU
NIST
P
R.
F1
PRE-TRAINED
49.88
30.26
0.73
83.82
82.18
82.94
60.15
23.49
5.56
94.44
94.16
94.30
FINE-TUNED
49.28
30.07
0.68
83.70
81.96
82.78
59.63
23.54
5.56
94.43
94.16
94.29
LoRA
49.03
30.19
0.68
83.70
81.92
82.75
59.64
23.69
5.37
94.49
94.16
94.30
Education
BACK-TRANS
49.13
30.04
0.67
83.69
81.91
82.74
59.63
23.48
5.56
94.44
94.15
94.29
CDPG
57.88
31.03
0.93
84.59
83.23
83.86
66.05
23.69
5.60
94.52
94.28
94.40
DYNAMIC CDPG
57.22
31.16
0.94
84.71
83.01
83.81
67.02
24.23
5.67
94.60
94.44
PRE-TRAINED
62.06
61.73
0.59
89.67
89.70
89.65
63.84
32:36
6.11
94.55
93.52
94.02
/ LORA
FINE-TUNED
61.46
51.71
0.59
89.74
89.70
89.69
63.47
32:27
6.10
94.52
93.49
93.99
61.38
51.87
0.60
89.75
89.63
89.66
63.16
32.33
6.09
94.51
93.45
93.97
Laws
BACK-TRANS
61.26
51.61
0.56
89.75
89.69
89.69
63.18
32:22
6.07
94.52
93.46
93.97
CDPG
68.50
50.81
0.68
89.60
89.65
89.60
69.68
34.54
6.34
94.68
93.77
94.21
DYNAMIC CDPG
68.50
50.81
0.68
89.60
89.65
89.60
70.50
35.06
6.38
94.74
93.87
94.29
PRE-TRAINED
47.62
18.95
1.14
76.09
75.69
75.78
50.83
8.65
3.48
89.55
88.33
88.92
/ LORA
/ FINE-TUNED
47.23
19.94
1.39
76.42
75.75
75.99
50.11
8.60
3.46
89.56
88.31
88.91
47.22
19.34
1.25
76.36
75.72
75.93
50.15
8.71
3.48
89.58
88.33
88.93
Thesis
BACK-TRANS
46.98
19.95
1.36
76.40
75.70
75.95
50.05
8.67
3.47
89.58
88.30
88.92
CDPG
54.19
19.94
1.29
76.
75.53
75.72
57.16
8.49
3.51
89.52
88.38
88.93
DYNAMIC CDPG
51.22
20.14
1.52
76.53
75.72
76.03
58.57
89.67
88.37
PRE-TRAINED
47.56
24.45
0.94
81. 28
79.06
80.09
57.97
16.20
4.86
92.80
92.60
92.69
/ FINE-TUNED
47.00
24.52
0.94
81.26
79.05
80.07
57.48
16.36
4.88
92.82
92.60
92.70
/ LORA
46.75
24.57
0.96
81.38
79.09
80.15
57.49
16.29
4.88
92.81
92.60
92.70
Science
BACK-TRANS
46.77
24.46
81.26
79.07
80.08
57.48
16.33
92.81
92.60
92.70
CDPG
56.27
24.78
1.02
81.48
79.70
80.53
64.06
15.96
4.88
92.76
92.66
92.70
DYNAMIC CDPG
52.38
24.80
1.00
81.63
79.39
80.43
65.55
16.34
4.86
92.79
92.60
92.69
Table 3:
The lop-p values used in DYNAMIC CDPG.
Values are presented in the order they are used.
IT
Medical
Law
Koran
Education
Laws
Thesis
Science
en -de
0.5,0.4.0.8
0.5,0.7.1.0
0.5,0.8
1.0
en -z-
0.5.0.9
0.5
0.5,0.7
0.5,0.6.0.7
de den 0.5,0.9
0.5
0.5,0.9
0.5
zh
0.5,0.4
0.5,0.4
0.5,0.6.0.7.0.8
0.5,0.3,0.2.0.11
boost the improvements already observed in CDPG. Moreover, DYNAMIC CDPG miligates the performance degradation seen with CDPG in some domains, including Medical, Law, and Koran of en en->de, and Science of zh-ten. SablleSsmmarizze the top-p values selected and how they changed during training. In Medical and Koran of de-ten and Laws of en- the selected top-p remained identical to the default CDPG setting, resulting in the same scores reported Tables and These findings suggest that while DYNAMIC CDPG can yield further improvements under metric supervision, the fixed-paranee parameer CDPG already delivers strong and stable results. Therefore, although parameter tuning is ideal, CDPG is not overly sensitive to the top-p value, and a fixed value of 0.5 is generally sufficient.
6 Discussion
6.1 When Is CDPG Effective?
Based on the results in Tables [ and 2 and the discussion in Section 5 although CDPG-based mot consistently improved Confidence and NIST, performance fluctuations were observed depending on each domain. To better understand under what conditions CDPG effective, we conduct detailed analy- sis focusing on the distributional differences between the pre-trained models and the monolingual lexical distribution features of CDPG described in Section 3.1

---

Published in Transactions on Machine Learning Research (01/2026)
Table 4: Comparisons of the target-side lexical distribution features. The prefix len. indicates the number of features, and stm. denotes the cosine similarity (in percent). len.1 and len.2 refer to the lengths of the first and second feature sets in each comparison. itr and uni indicate that the similarity was computed over the intersoction or the union of the two sets, respectively.
Case (i):
Dev Features v.s. Pretrained Features
Case (fi):
Dev Fentures v.8.
Test Features
Pair Domain
lem.1
len.2
len.itr sim.itr len.umi sim.umi
len.1
len.2
len.itr sim.itr
len.umi sim.uni
en->de
IT
5,832
5,553
4,152
83.14
7,233
65.09
5,832
5,475
3,366
93.14
7,941
90.99
en
Koran
4,543
3,918
2,931
95.69
5,560
95.48
1,543
4,435
3,300
98.81
5,678
98.67
de een
Law
7,054
6,469
5,668
98.80
7,855
98.66
7,054
7,011
4,754
98.19
9,314
97.91
d
Medical
6,543
6,130
5,367
95.83
7,306
94.97
6,543
6,577
4,604
94.22
8,516
93.48
en en-
Thesis
7,533
7,518
5,395
74.88
9,656
73.23
7,533
3,755
3,188
92.81
8,100
91.82
en->zh
Laws
6,903
6,783
4,865
68.11
8,821
64.99
6,903
1,852
1,373
29.43
7,382
24.64
zh
Education
10,680
9,546
7,379
80.64
12,847
79.92
10,680
2,357
1,885
70.37
10,711
65.13
zh-
Science
9,807
9,127
6,866
61.38
12,068
60.64
9,807
3,089
2,317
65.39
11,692
56.79
Table 5: Results of CDPG with fixed top-p values of 0.5, 0.8, and 1.0. The results for top-p = 0.5 are the same as those reported in Tables [ and 2 and the abbreviations are consistent with those tables. The best
score in each row is highlighted in bold. The row order aligns with Table
top-p 10-p=000 0.5
top-p= = 0.8
top-p 1.0
Pair
Domain
Conf.
BLEU
NIST
F1
Conf.
BLEU
NIST
F1
Conf.
BLEU
NIST
F1
en en-tde
IT
74.44
29.01
6.25
87.67
74.67
29.13
6.28
87.66
67.87
28.19
6.08
87.47
en-
Koran
67.00
18.40
5.25
80.72
67.14
18.50
5.29
80.74
61.30
18.85
5.35
80.85
de ten
Law
78.12
51.61
10.12
95.86
78.33
51.53
10.16
95.86
71.83
51.58
10.16
95.87
de ten
Medical
82.84
44.56
8.56
96.53
83.06
44.82
8.61
96.54
77.72
45.06
8.61
96.54
en en-tzh Thesis
54.19
19.94
1.29
75.72
53.93
19.98
1.48
75.86
46.96
19.95
1.47
75.76
en-tzh
Laws
68.50
50.81
0.68
89.60
68.78
51.16
0.65
89.63
61.68
51.90
0.61
89.71
zh -h
Education 66.05
23.69
5.60
94.40
65.86
23.92
5.65
94.37
59.68
23.50
5.58
94.31
zh ten
Science
64.06
15.96
4.88
92.70
63.93
16.14
4.87
92.70
57.29
16.34
4.88
92.69
First, we extract the term frequency-based features j from the development set (Dev Features) and the test set (Test Features). Next, we generate translations for the development set using the pre-trained model and extract features from these translations (Pretruined Features). Using these three types of features, we compute the cosine similarity between two feature sets in the following two cases. Case (i) compares Dev Features and Pretrained Features to examine how much CDPG can improve over PRE-TRAINED. Case (ii) compares Dev Features and Test Features to indicate how uniformly the lexical distributions are aligned across the sampled sets. To account for differences in feature set lengths, which arise from zero counts due to mismatches in word usage, we compute similarity using both the intersection and the union of the sets.
Table presents the analysis of features to complement Tables D and 2 in some representative domains. Focusing on IT of en->de, we observe that in Case (i), the similarity sim. is low (65.09), suggesting that the pre-trained model does not perform well on this domain. Consequently, CDPG successfully improves overall performance by aligning features more closely with those of the IT domain. Similar trends are also observed in the Thesis domain of en-zh and the Education domain of zh-ten. In contrast, in the Koran domain, the similarity is alrcady high (95.69), so the impact of CDPG is limited. A similar trend is observed in de-tn. Turning to Laws of en-tzh and Science of zh-ten, although the similarity in Case (i) is relatively low, CDPG-based methods show only limited improvement. Looking at Case (ii), the similarity sém, uni is also quite low, indicating that the lexical distribution is scattered. In such cases, simply approximating the feature space is insufficient improve BLEU or other metrics, as the issue lies in a distribution mismatch within the same domain data. However, even in these scenarios, both NIST and
10

---

Published in Transactions on Machine Learning Research (01/2026)
Table 6: Translation examples from the outputs PRE-TRAINED and CDPG. We select one short and one long sentence for each of en-*de and en en- In #Changes, the numerator indicates the number of sentences in which CDPG made changes compared test set. Words highlighted in red indicate correct domain-specific usage, those in blue represent updated terms that do not match the correct target term, and underlined words indicate inaccurate translations.
TRE-TRAINED, and the denominator indicates the size of the
Domain: IT
Pair: en-de
# Changges: 662/2000
Input
SubDialog has one state, default. SubDialo hat nur einen Status, Standard. SubDialog hat einen Zustand, default.
Reference
PRE-TRAINED
CDPG
SubDialog hat cinen Zustand, Standard.
Domain: Medical
Pair: en-*de
# Changes: 748/2000
4 ml of solution in a ml vial (type I glass) closed with a latex-free stopper (bromobutyl/ isoprene polymer) and a seal (laqquered plastic).
Input
ml Lösung einer 5 ml-Duurhsstecdflasche (Glastyp 1), die mit einem latexfreien Stopfen trromobbtyy//oopreppooo
Reference
und eine Kappe (lackierter Kunststoff) verschlossen ist. Läsung einer --ml-Durchstechflsschee (Glas Typ 1); die mit einem latexfreien Stopfen (Brombutyl/Isopree))
PRE-TRAINED
und einem Siegel (Lackkunstsooff) verschlossen ist.
ml Lösung einer 5 ml Durchstechflasche (Glas Typ 1), die mit einem latexfreien Stopfen
CDPG
und einem Siegel (lackierter Kunststoff) verschlossen ist.
Domain: Education
Pair: en-
# Changes: 408/790
Input
What an absurd suggestion!
Reference
$ #
PRE-TRAINED
CDPG
Domain: Thesis
Pair: en en->zh
#Changes: 414/625
Newton's 's transformation family f w(z)=z-lwz w-1 containing only one complex parameter w(w+0 or 1) constructed from the transcendental mapping z-re 2 w+c.
Input
H (())) =czw| of li! 2 i wtw (w> ( 0f81)
(z) = 1wzw-18tr
Reference
fw(2) 4 RCE (5 Ai...
PRE-TRAINED
=2-1wz W-1 XX
E B3f(2) wwz W-1
CDPG
1
(2)
=2- lwz W-1 XX f
(w)0 EZI). EHE He 2+c 11
Confidence scores consistently improve, suggesting that CDPG still succeeds in generating domain-spocific terms. In summary, CDPG is most effective when there exists a relatively large gap in feature distribution in the pre-trained model, indicating that it can achieve faithful domain shifts under such conditions.
Finally, to evaluate the scnsitivity of CDPG to parameter settings, we fix the top-p value at three levels: 0.5, 0.8, and 1.0. Table 5 shows the resulting score variations. Based on these results and the analysis in Table 4 we categorize the outcomes into two scenarios. First, when the similarity between Dev Features and Pretrained Features (Case i) is low and the similarity between Dev Features and Test Features (Case ii) is high, smaller top-p values tend to yield better performance. This pattern is observed in domains such Thesis of en an-zh and IT of en->de. In contrast, using top-p = 1.0 helps maintain performance in domains like Laws of en and Science of zh-ten. Second, when the similarity between Dev Features and Pretrained Features (Case i) is already high, the performance improvement from CDPG is generally limited. In such cases, score variation across different top-p values is small, and top-p = 1.0 tends to be the safest choice. In addition, we observe that the Confidence score consistently changes with the top-p value, regardless of the actual translation quality. These results support our earlier discussion that the performance of CDPG is influenced by the properties of the provided monolingual lexical distribution features, such the gap between the pre-trained model and the target domain, as well the consistency of domain distributions.
6.2 Qualitative Analysis
Table presents four representative translation instances that illusrrate subtle effects of CDPG not fully captured by quantitative metrics. We first observe that CDPG only partially modifies the original model's knowledge, demonstrated by conservative changes in the translations, and primarily enhances the model's
11

---

Published in Transactions on Machine Learning Research (01/2026)
Table 7: Human evaluation results. In (i) Automatic Statistics, #test indicates the number of sentences in the test set, #diff denotes the number of sentences whose outputs differ between CDPG and TRE-TRAINED, change% is the proportion of changed outputs (#diff #test), and win% is the percentage of sentences where CDPG obtains BLEU score not lower than PRE-TRAINED In (ii) Count by Feature, #term. represents the number of changes caused by terminology, #fit indicates how many of these changes match the provided monolingual features, term.% is the proportion of terminology-reattee changes (#term./ /#diff), and fit% is the pereentage of terminology changes that correspond to target-domain features (#fit/ #term.). In (iii) Human Judgment, #win, #losc, and #tic show human preferences for term selection among the #term. cases. win% indicates the proportion of clear improvements and win-lie% indicates the proportion of outputs that are judged as improved or at least reasonable under the conservative soft-constraint behavior of CDPG.
(#diff/
Automatic Statistics
(ii)
Count by Features
(iii) Human Judgment
Pair
Domain
#test
#diff
change%
win%
#term.
#fit
term.% %
fit%
#win
#tie
# #lose
win%
win-tie%
en en- Education
790
408
51.65
61.38
188
176
46.08
93.62
106
52
30
77.94
84.04
en
Thesis
625
414
66.24
55.08
205
190
49.52
92.68
99
59
47
67.81
77.07
zh
Laws
456
276
60.53
61.83
142
140
51.45
98.60
89
14
39
69.53
72.54
de
IT
2,000
259
12.95
57.00
147
145
56.76
98.64
67
43
37
64.42
74.83
en zzh
Laws
456
268
58.77
44.40
107
102
38.06
95.33
30
51
26
53.57
75.70
de
Medical
2,000
215
10.75
48.47
64
59
29.77
92.19
24
32
75.00
87.50
term choice. Specifically, for the two en-*de instances, regardless of sentence length, only domain-specific terms modified without affecting the overall semantics syntax, resulting in conservative behavior where not all test set inferences are altered. These findings confirm our motivation that CDPG can modify model knowledge harmlessly, avoiding issues such as catastrophic forgetting. Notably, they also explain the non-significant differences in BERTScore observed in Tables ! and 2 since representation-leve evaluation methods are not sensitive to word-level changes. In contrast, the consistent improvements in NIST scores highlight CDPG's effectiveness in conservatively enhancing domain-spocific term usage.
However, these findings do not imply that CDPG benefits only word selection. In Thesis of en- PRE- TRAINED exhibits issues such semantic loss and repetitive generation, whereas CDPG complements the missing semantics and mitigates the repetition. This improvement may be attributed to the enhanced confidence provided by GDC. Similarly, in the short sentence from en en- PRE-TRAINED tends to translate the source into Chinese idioms that do not fully align semantically with the original sentence, i.c., ignoring the semantics of the word "suggestion". In contrast, CDPG correctly translates the key terms, suggesting that GDC enhances the model's focus on important words. In addition, in the long sentence of en en- the blue words reprosent an error in translation. This occurs because CDPG translates "transcendental" and "mapping" separately, as both words are present in the given features. This suggests that, since CDPG acts as soft constraint, its use of keywords is not strictly enforced, but applied in conservative manner.
6.3 Human Evaluation for Fine- line-grained Term-level Analysis
In Section 6.2 we found that CDPG makes conservative changes to translations and primarily enhances the model's term choice. Because the use of keywords is not strictly enforced but applied in a conservative manner, it effectively acts as a soft constraint, indicating that CDPG does not introduce destructive or harmful changes. Morcover, based on the discussion in Section 6.1 it is suggested that CDPG the most effective when a relatively large gap exists in feature distribution in the pre-trained model, indicating that it can achieve faithful domain shifts under such conditions. From these observations, hypothesize that CDPG-based methods function by encouraging the model to make term-level improvements according to the provided feature distributions. To verify this, we conduct human evaluations to analyze the model's behavior from a term-level perspective.
This evaluation consists of three steps: (i) a sentencc-level analysis using automatic statistics, i..,, BLEU, to mcasure improvement rates in order to focus only on examples whose outputs were changed by CDPG; (ii) manual examination of sentences where changes were caused by term substitutions, checking whether the
12

---

Published in Transactions on Machine Learning Research (01/2026)
substituted terms appear in the target-domain features; (iii) finally, for cases where the substituted terms match the features, we measure the win rate via human evaluation to assess whether CDPG consistently produces faithful domain shifts at the term level. Table 7 shows the human evaluation results on six representative cases, including four cases where automatic statistics show substantial improvement, and two cases where the BLEU improvement is relatively small between CDPG and PRE-TRAINED in Tables [ and 2
First, from the results of (i) Automatic Statistics, we observe that the impact of the soft constraint of CDPG outputs (change%) is around 50 to 60% for Chinese-related domains and slightly above 10% for German- related domains. This suggests that the extent of changes depends on how the test sets are constructed. Even when the number of changed outputs is small, for example 12.95% for de-en IT, the win rate, i.e., how often CDPG outperformed PRE-TRAINED among changed samples, still shows a positive trend. In contrast, for domains such as en-tzh Laws, where the change ratio high (58.77%), the final win rate is more limited (44.40%). This suggests that, under conservative soft-constraint domain shift, the impact is determined primarily by which terms are substituted, rather than by the number of substitutions.
Next, based on the analysis in Section 6.1 i.e. the effectiveness of CDPG depends on the alignment between the provided features and the test domain, we further conduct (ii) Count by Features. We manually examine sentences whose changes come from term substitutions and check whether the substituted terms appear in our features. The results of fit% show that most terminology-relat features in the target domain, while a few mismatches can be attributed to the soft-constraint behavior of CDPG discussed in Section 6.2 This confirms that the improvements arise from faithful Nomain-specifi terminology shifts guided by the provided feature distribution.
substitutions indeed correspond to
Finally, we conduct human evaluation in (iii) Human Judgment to assess whether the substitutions are rea- sonable for sentences involving terminology changes (#term.). Sentence-level comparisons between CDPG and PRE-TRAINED show that CDPG is preferred in a majority of cases in win%, and when ties are included (win-tie%), the preference reaches approximately 75%. This indicates that CDPG generally produces faith- ful yet harmless modifications, consistent with its soft-constraint nature. Taken together, these findings demonstrate that CDPG induces subtle terminology- automatic metrics such as BLEU, but remain reasonable and domaan-appropriate. Therefore, we conclude that CDPG aligns with our motivation, as it performs soft-constraint domain shift guided by the provided feature distribution and shows stable effectiveness through manual validation.
adjustments that may not be fully captured by
6.4 Will Other Domains Be Influenced?
The primary goal of CDPG is to encourage the distribution of the pre-trained model to align with the expectations of the given features. However, since CDPG fits the model to a single domain, there is a potential risk of reduced generalization to other domains. Therefore, as shown in Table g we conduct experiments to measure the performance changes of DYNAMIC CDPG across domains from two perspectives: (1) the relative difference between FINE-TUNED and DYNAMIC CDPG in the experimented domains; and (2) the change in performance of both FINE-TUNED and DYNAMIC CDPG on generic domain.
First, FINE-TUNED consistently shows a decrease in both confidence and performance on the generic domain, whereas DYNAMIC CDPG achieves a significant increase in confidence in most cases, albeit with some fluctuations in performance. This indicates that the improvements achieved by our method generalize well, likely due to its intentionally conservative updates. While DYNAMIC CDPG generally demonstrates better generalization than FINE-TUNED, there are two types of exceptions: (1) Changes in confidence can affect generalization, CDPG induces a global increase in confidence rather than domain-specific gains. However, this indirect influence is generally limited. For example, although the largest BLEU score drop caused by increased confidence is 1.13 on Korun of en-de,, DYNAMIC CDPG achieves a 2.86 BLEU improvement on IT, which is significantly better than FINE-TUNED. (2) Performance in aligned cases is lower than in some cross-domain scttings, such as Thesis of en and Medical of en->de, suggesting that the provided dev features can have a negative impact. These results once again corroborate our analysis in Section 6.1] showing that the effectiveness of CDPG is closely tied to the quality and nature of the provided features.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Table 8: Relative differences between the scores of FINE-TUNED and DYNAMIC CDPG. The columns and rows indicate the domains used for training and testing, respectively. Underlined values denote aligned cases where the training and testing domains are the same. Gen.fit. and Gen.d.c. indicate the differences between PRE-TRAINED and FINE-TUNED, and between PRE-TRAINED and DYNAMIC CDPG, respectively, a generic domain (newstest2020). These serve pivols for measuring relative differences.
Confidence
BLEU
Education
Thesis
Science
Gen.Lt.
Gen.d.c.
Education
Thesis
Science
Gen.f.t
Gen.d.c.
Education
7.94
8.29
9.21
-1.16
8.52
1.09
0.29
-0.44
-0.76
-0.17
-2h
Thesis
6.70
3.99
5.58
-0.31
7.69
0.87
0.20
0.37
-0.28
0.13
Science
4.83
4.20
5.38
-0.68
4.92
0.87
0.50
0.28
-0.02
0.33
Education
7.39
7.86
7.83
-0.59
8.31
0.69
-0.07
-0.27
-0.11
0.19
-en
Thesis
7.72
8.46
8.03
-0.51
8.84
0.66
-0.11
0.09
-0.04
0.20
Science
7.81
8.51
8.07
-0.55
8.89
0.64
-0.26
-0.02
-0.07
0.26
IT
Medical
Koran
Gen.ftt
Gen.d.c.
IT
Medical
Koran
Gen.ft
Gen.d.c.
IT 10.61
8.97
9.90
-0.22
12.75
2.86
0.38
-1.13
-0.15
-1.65
-de Medical Koran
8.32
6.61
7.11
-0.27
9.47
2.44
0.28
-0.63
-0.07
-0.90
0.65
0.47
-0.09
-0.21
-0.86
1.05
0.93
-0.01
-0.18
-0.08
IT
5.89
6.17
6.76
-0.22
10.38
2.72
-0.78
-0.04
-0.11
-0.81
-een Medical
-1.02
-0.05
-0.82
-0.29
-1.50
-0.65
-0.42
-0.11
-0.06
-0.18
Koran
6.07
5.92
5.95
-0.20
8.28
0.97
-0.91
0.13
-0.14
-0.40
Table 9: Scores from experiments with mixed-domain data from two domains. The main domain data
is fixed, and sentences from a mixed domain are added during feature construction. The model is then evaluated only on the main domain data. #Sent. indicates the number of added sentences from the mixed domain. The best value for each domain block (across different #Sent. settings) is shown bold.
English German (Main: IT, Mix: Medical)
English Chinese (Main: Thesis, Mix: Lawss)
Method
#Sent.
Conf.
BLEU
P
R
Fl
Method
#Sent. Conf.
BLEU
P
R
F1
0
67.91
27.92
87.38
87.60
87.42
47.23
19.94
76.42
75.75
75.99
500
67.82
27.63
87.35
87.57
87.39
750
47.14
19.77
76.44
75.73
75.98
FINE-TUNED
FINE-TUNED
1,000
67.75
27.61
87.36
87.58
87.40
1,500
47.05
19.63
76.42
75.75
75.98
2,000
67.61
27.27
87.32
87.55
87.36
3,000
46.83
19.13
76.37
75.74
75.95
0
74.44
29.01
87.68
87.77
87.67
0
54.19
19.94
76.11
75.53
75.72
500
74.24
29.70
87.70
87.80
87.70
750
54.16
20.06
76.25
75.59
75.81
CDPG
CDPG
1,000
74.22
28.83
87.63
87.77
87.64
1,500
54.12
20.15
76.21
75.59
75.80
2,000
74.14
29.43
87.68
87.79
87.68
3,000
54.01
20.10
76.24
75.58
75.80
Finally, we simulate a more realistic setting where the prepared data not perfectly clean and may include content outside the target domain. We demonstrate the robustness of CDPG by comparing its performance trends to those of FINE-TUNED mixed-domain scenarios, where an extra domain dataset is intentionally introduced during tuning contamination. Table g shows the results under several contamination settings. The results reveal that the performance of FINE-TUNER consistently declines as the degree of domain mixing increases. In contrast, the performance of CDPG remains stable regardless of the domain mixture.
These findings suggest that, due to its conservative luning approach, CDPG less prone to issues such as catastrophic forgetting and overfitting. Moreover, even under more realistic conditions involving data contamination. CDPG achieves robust domain adaptation using only small amount of monolingual data and target-side lexical distributions, outperforming typical fine-tuning methods that rely solely on parallel data.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Table 10: Results of CDPG with fixed top-p values of 0.5, 0.8, and 1.0, evaluated over eight different random
soeds, reporting the mean j and standard deviation as Mio. The row order follows Tables 4 and 5 The
experimental settings and abbreviations are consistent with those in Table
top-p 0.5
top-p= 0.8
top-p 1.0
Pair
Domain
Conf.
BLEU
F1
Conf.
BLEU
F1
Conf.
BLEU
F1
en en-de
IT
44.42.0.05
29.01 20.26
87. 20.66
74.57 10.10
29.00
87.66
67.6440.21
20.33
87.40...0s
ed
Koran
66.9710.04 18.47
80.73
67.07
18.60
80.7510.02
60.92.0.21
18.91 20.08 80.87=0.00
de
Law Medical
78. 78.1140.04 51.60
95.86_0.01
78.31
51.5540.08
95.8640.00
71.6540.22
51.62 51.6220.04
95.89_o.01
de ee
82.79+0.03 44.57=0.04
96.54=0.00
83.07=0.07
41.8644.04
96.54+0.00
77.65+0.21
15.31 96.51=0.01
en en-
Thesis
51.2610.11 19.98
75.81
20.06
54.1420.13
19.9610.08
75.8410.06
17.1140.20
19.9620.06
75.7420.09
en en-
Laws
68.5740.07 50.81
89.5920.03
68.88
51.09+0.17
89 89.59+0.uz
61.80+0.12
51.85
89.71
zh >en Education 66.12+0.00
23.82
=1.11
66.19
23.90+t.15
9-1.88+o.o1
59.85+0.20
23.68.0.12
94.33
zh
>en
Science
61.1120.05
16.02
92.6920.01
64.005.0.10
16.09
92.3940.01
57.6940.17
16.302o.08
02.68=0.02
6.5 Robustness Analysis of CDPG to Randomness
Finally, in our previous experiments, e.g.; Sections and we mainly reported single-run results, e-g-- Tables 2 and 5 to enable straightforward analysis and discussion based on consistent results. To evaluate the robustness of CDPG, we additionally conducted multiple runs with different random seeds and report the mean and standard deviation a. Table presents the results obtained from eight different seed values, while koeping all other settings and hyperparameters the same as in Table 5
From Table I0 we observe that the standard deviations are small, indicating that CDPG is stable. In fact, the coefficient of variation (a/μ) is at most 1.21%, and in most cases less than 1%. In the CDPG-based method, the only component largely inffuenced by the random sced is the step in which the target language model generates sentences æ via nucleus sampling to train the distribution feature X for the EBMs. Since the sampling is conducted from the entire distribution X' of the target language model, a sufficient number of samples will closely approximate the true distribution, resulting in stable performance regardless of the random seed value. On the other hand, as discussed in Section 6.1 varying the top-p value changes the underlying sampling distribution, introducing fluctuations in diversity and, consequently, degradations in the results, Even with many samples, such diversity variation inherently affects the outcomes, suggesting the necessity of controlling the diversity parameter. This further supports the rationale behind methods such DYNAMIC CDPG, which explicitly adapt the sampling parameters to account for such variation.
In summary, when sentences x are sampled from the same distribution X, the method is largely insensitive to randommess and remains robust. To control or influence the behavior of the system, it is therefore essential to adjust the sampling method or diversity-related parameters that directly modify the underlying distribution.
7 Limitations and Future Extensions
We conduct a comprehensive analysis; however, we acknowledge three main limitations in this work.
First, as stated in Sections 5 and 6.2 representation-lvvel MT evaluation mothods are not sensitive to the improvements made by CDPG, which results in only minor differences in BERTScore in particular. Moreover, although NIST provides more reasonable assessment of domain-specific terminology and aligns more closely with our objectives, it remains limited by its BLEU-style surface-level representation design. Therefore, exploring how semantic-level evaluation methods can better capture word-specific changes remains an important direction for future work. Additionally, we did not include modern neural fine-tuned metrics, such as COMET (Rei et al.) 2020) and BLEURT (Sellam et al. 2020), as part of our main evaluation. These motrics are fine-tuned on numan-generated MT quality annotation data (Ma et al. (009)) but such data often fails to capture subtle patterns such as named entity differences (Amrhein & Sennrich, 2022 Glushkova et al. 2023). Moreover, due to overfitting to the annotation data, these metrics tend to favor outputs that are
15

---

Published in Transactions on Machine Learning Research (01/2026)
closer to the in-domain data of their fine-tuning sets (Zouhar et al. 2024a bp. Consequently, determined that such fine-tuned metrics are not suitable for evaluating domain adaptation experiments.
Second, our primary goal is to demonstrate a novel practical application of massive EBMs using CDPG in a downstream task, with domain adaptation for machine translation as a case study. To maintain a clear focus on this objective, we adopted simple encoder-decoder architecture NMT models (Tiedemann & Thottingal 2020). We did not employ multilingual large models such as NLLB-200 (Team et al. (022) or M2M100 (aan et al., 2021), as they introduce additional complexity, including translation issues arising from multilingualissm. Furthermore, extending our approach to translation decoder-only large language models (Alves et al. 2024 Xu ct al. 2025 Feng et al.] 2025 Cui ct al. 2025) or applying it to other domain datasets (Oncevay et al. 2025 Zhang et al. 2025 Cui et al. 2025 was not feasible given our available computational resources resourre and would have made it harder to isolate the core effects of our method. Nonetheless, we believe our experiments already provide a comprehensive demonstration of the method's applicability and sufficiently address our rescarch question. While such extensions remain promising future directions, we believe this work is a valuable pioneering study that lays the groundwork for broader applications.
Third, in this paper, DYNAMIC CDPG is described primarily using nucleus sampling, However, the method is not restricted to this choice and can also be applied to other sampling strategies (Fan et al.] 2018 Vi-
jayakumar et al. 2018 Hewitt et al. 2022 Minh et al. 2025 The core mechanism lies in adjusting sampling diversity, and nucleus sampling was adopted merely to maintain consistency with prior work (Khalifa et al, 2021). The primary focus of this study is domain adaptation for neural machine translation in low-resource settings. Our key finding is that the selection of sampling diversity during EBM approximation substantially affects performance, and dynamically adjusting this diversity is effective. Therefore, exploring alternative parameterized sampling strategies and identifying optimal configurations constitutes natural direction for future work. Moreover, we primarily used BLEU to monitor the tuning process, although other evaluation metrics could also be employed. Even though BLEU was used for monitoring, the results in Sections 5) and 0 show consistent improvements in other metries such as BERTScore, suggesting that the approach generalizes beyond a specific metric. Therefore, we expect similar behavior when using alternative evalu- ation metries. Potential future extensions include developing more suitable metrics for guiding CDPG or exploring monitoring strategics.
8 Related Work
When using parallel data, data, & Manning (2015 ; Freitag & Al-Onaizan (2016) perform domain adaptation by first training on large-scale general-domain data, then fine-tuning on a small amount of in-domain data.
Chu et al. (2017) instead mix general and in-domain data for training at once, Further efficiency in domain adaptation has been pursued through techniques such as adding domain tags (Kobus et al., 2017 Britz et al.
2017,, subword-aware tokenization (Enomoto et al., 2023), and training data sampling (Wang el al. 2017). However, direct fine-tuning with a small amount of data often leads to overfitting, prompting proposals of knowledge distillation (Dakwale & Monz 2017) and regularization strategies (Miceli Barone et al.) 2017).
In the context of monolingual data utilization, several methods have been explored such as back translation (Sennrich et al. 2016). direct learning from monolingual data as LM (Gulcehre et al.
2015
5 Zhang & Zong
2016 Domhan & Hieber 2017 Burlot & Yvon 2018), exploiting task-specific features Dou et al. 2019b a
utilizing knowledge graphs MMoussalem et al. 2019 Zhao et al.] 2020: Costa et al. 2022
Conia et al. 2024
and nearest neighbor search (Farajian et | 2017
Bapna & Firat 2019 Zheng et al. 2021 Khandelwal al.
2021 Wang ct al., 2022 Deguchi et al.J 2023 Agrawal et al.] 2023), and the combination of unsupervisod
NMT methods and back-translation technique (Mahdieh et al., 2020 Zhang et al., 2022). However, these
approaches usually require large-scale monolingual data, which is not always available in specialized domains. Therefore, they may not be suitable for the extremely low-resource domain adaptation settings.
For terminology-oonstrainedd decoding, hard-constrained decoding methods (Hokamp & Liu 2017 Post & Vilar 2018 Hu et al.) 2019 Post et al. 2019), which force the model to decode specific terminology, and soft-constrained decoding methods (Song et al. 2019 Chen et al. (020),, which apply post-oditing techniques using phrase tables, have been proposed. However, since these approaches require predefined constrained vocabularies, they face challenges in real NMT scenarios that demand inductive domain adaptation.
16

---

Published in Transactions on Machine Learning Research (01/2026)
CDPG (Kobbak et al. 2022), which study builds upon, focused only on minor input perturbations, e.g.: replacing digits with spelled-out numbers, and did not address large-scale domain adaptation based on the full target-domain distribution. Regarding reinforcement learning methods (Ranzato et al. 2016; Kreutzer ctt al. 2017 Choshen et al. 2020 Yang et al.) 2024), outside of the GDC framework, rowards are based only on overall scores such as BLEU, without the ability to impose fine-grained constraints. Furthermore, there is a potential for causing catastrophic forgetting, making scaling like in this study particularly challenging.
9 Conclusion
In this study, we explored whether unsupervised domain adaptation in machine translation can be effectively performed using only naive statistics, ie., the monolingual lexical distribution, which can be easily obtained from target-side domain data without requiring bilingual supervision. We demonstrated this by imposing large-scale distribut constraints through EBMs at scale using the CDPG method, leveraging features derived from the entire target-domain corpus, and confirmed its effectiveness across multiple domains and language directions. This work is the first to demonstrate the practical applicability of CDPG of the GDC framework to a realistic downstream task, using the unsupervised domain adaptation of pre-trained NMT models as a case study. Although this work focused on word-level lexical distributions as the guiding signal, we believe that future work should explore alternative or complementary feature representations, such n-gram statistics language model embeddings, to further enhance fine-tuning within the GDC framework.
Acknowledgements
We sincerely thank the anonymous reviewers and the action editor for their valuable comments and sugges- tions. Their constructive feedback siguificantly strengt hened this work and encouraged its publication. We are deeply grateful for their recognition of our contributions and their support toward publication.
We also extend our sincere thanks to Sei Iwata, an early adopter of this project This work is based on exlensive discussions and iterative trial-and-error with him. We are pleased to be able to inform him that these efforts have ultimately led to publication.
Regarding the author contributions of this paper, Yusuke Sakai and Zhi Qu contributed equally to this work. Yusuke Sakai led the project from the initial commitment through final acceptance and was responsible for the main manuscript writing. Zhi Qu contributed primarily to code preparation and extensive experiments, drafted the Results and Discussion sections, and joined the project after the departure of Sei Iwata, engaging in extensive discussions thereafter.
References
Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. In-context examples selection for machine translation. In Anna Rogers, Jordan Boyd-Graber and Naoaki Okazaki (ods.). Findings of the Association for Computational Linguisties: ACL 2023, pp. 8857-8873, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. (4.18633/v120055 /tt/://ww2222..nnnnnaa..... findings-acl.666
URL https::/aclaanthollo/ http:///waaaaatoooooo..or//...dd
www.:or..//2......o frnoonsffiniings-ee crcpiinngsccl..6 ac1.664/
Roce Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7747-7763, Online, July 2020. Association for Com- putational Linguistics. doi: wa0/:/3//v1222.allllll acl-main.699)
URL hattss//acllatthllog wrwplloooyoorgg//// org/2020.
Farhad Akhbardeh, Arkady Arkhangorodskyy, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussa, Cristina Espani-Bonet, Angela Fan, Christian Fodermann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaakyire, Jungo Kasai, Daniel Khashabi, Kevin
17

---

Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Hcafield, Hicu Hoang, Miquel Esplà-Gomis Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramirez-Sánchoz, Elsa Sarrfas, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, and Jaune Zaragoza. ParaCrawl: Web-scale acquisition of parallel corpora. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4555-4567, Online, July 2020. Association for Computational Linguistics. doi: 10.18653 11/0:///1122000cmmmaia Published in Transactions on Machine Learning Research (01/2026)
Knight, Tom Kocmi, Philipp Kochn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Na- gata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin, and Marcos Zampieri. Findings of the 2021 conference on machine translation (WMT21). In Loic Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvelte Graham, Roman Grund- kiewiez, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Kochn, Tom Kocmi, Andre Martins, Makoto Morishita, and Christof Monz (eds.), Proceedings of the Sixth Conference on Machine Translation, pp. 1 1-88, Online, November 2021. Association for Computational Linguistics. URL https:://aclattooll
or2/2021..... w.t.1.1/tt11/1/
Duarte Miguel Alves, José Pombal, Nuno M Guerreiro, Pedro Henrique Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and Andre Martins. Tower: An open multilingual large language model for translation-rclated tasks. In First Conference on Language Modeling, 2024. URL htttps://pperreye httpp////pppreeeeeaaaaaaattttaaa wwtp.//orrrndddddinnss///.
Chantal Amrhein and Rico Sennrich. Tdentifying weaknesses in machine translation metrics through min- imum Bayes risk decoding: A case study for COMET. In Yulan He, Heng Ji, Sujian Li, Yang Liu, and Chua-Hui Chang (eds.), Proccedings of the 2nd Conference of the for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1125 1141, Online only, November 2022. Association for Computational Lin- guisties. doi: 10.18653/v 1 /20222222222 aac.:aaaacl-mmma main-mm.in.... URL
Chapter of the Association
owg.orgg/////// 2022 2022.aacl-main 83/
URL httpp::/aaclanthologg org/2020. acl-main........... 417/
Ankur Bapna and Orhan Firat. Non-parametric adaptation for neural machine translation. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Tong and Short Papers), pp. 1921 1931, Minneapolis, Minnesota, June 2019. Association for Computational http:///cclanthoogy.orrg/og
Linguisties. doi: 10.18653/ 1 1//119-1911 URL
Loic Barrault, Ondřcj Bojar, Marta R. Costa-jusså, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. Findings of the 2019 conference on machine translation (WMT19). In Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Marco Turchi, and Karin Verspoor (ods.), Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pp. 1 61, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 10.18653/v1, 1118653/11/WI955000 531-5 URL https: htt.aaatthoooooooggg/r---/-
Loic Barrault, Magdalena Biesialska, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Matthias Huck, Eric Joanis, Tom Koemi, Philipp Koehn, Chi-kiu Lo, Nikola Ljubešić, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Santanu Pal, Matt Post, and Marcos Zampieri. Findings of the 2020 conference on machine translation (WMT20). In Loic Barrault, Ondrcj Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Yvette Graham, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, and Matteo Negri (ods.), Procoodings of the Fifth Conference on Machine Translation, pp. 1-55, Online, November 2020. Association for Computational Linguisties. URL ottp:://aclannhooooy.oo////// org/2220 w.t-1.1/!
18

---

Published in Transactions on Machine Learning Research (01/2026)
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Groengard, Connor Jen- nings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. LoRA learns less and forgets less. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreviien net
http://rrrrrreee.oooooooorrraaaa horr/ Featured Certification.
Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Kochn, and Christof Monz. Findings of the 2018 conference on machine translation (WMT18). In Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvelte Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, and Karin Verspoor (eds.), Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pp. 272-303, Belgium, Brussels, October 2018. Association for Computational Linguisties. doi: (0.18653/11/W18-660 URL hrg/W18-6401/
Denny Britz, Quoc Le, and Reid Pryzant. Effective domain mixing for neural machine translation. In Ondřej Bojar, Christian Buck, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, and Julia Kreutzer (eds.), Proceedings of the Second Conference on Machine Translation, pp. 118-126, Copenhagen, Denmark, September 2017. Association for Computational Linguisties. doi: 10. 10. 18653/66/ /111744472
URL htttps://cclanthh/
org/W17-4712/
Franck Burlot and François Yvon. Using monolingual data neural machine translation: a system- atie study. In Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, and Karin Verspoor (eds.), Proccedings of the Third Conference on Machine Translation: Research Papers, pp. 144-155, Brussols, Belgium, October 2018. Association for Computational Linguistics. doi: 10.18653/v1/W18--33 URL https://aclanthology org/W18-6315/
Guanhua Chen, Yun Chen, Yong Wang, and Victor O.K. Li. Lexiial-consttrain--awwae neural machine trans- lation via data augmentation. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 3587-3593. International Joint Conferences on Artificial Intelligence Organization, 2020. doi: 10.24963 00.24963/ijcali 2020//9.22//9.. URL https://doi.oo org/10. 24963/ijcai. 2020/496. Main track.
Leshem Choshen, Lior Fox, Zohar Aizenbud, and Omri Abend. On the weaknesses of reinforcement learning for neural machine translation. In International Conference on Learning Representations, 2020. URL
https://openreviers http::///pppprreeeeee.eeerraaraa net.foorummi--
Chenhui Chu and Rui Wang. A survey of domain adaptation for neural machine translation. In Emily M. Bender, Leon Derczynski, and Pierre Isabelle (eds.), Proceedings of the 27th International Conference on Computational Linguistics, pp. 1304-1319, Santa Fe, New Mexico, USA, August 2018. Association for Computational Linguistics. URL
wtwp:/oooooyoooo//----1o11 orgg/c8--1111/
Chenhui Chu, Raj Dabre, and Sadao Kurohashi. An empirical comparison of domain adaptation mot for neural machine translation. In Regina Barzilay and Min- Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 385 391, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: (.118653/v1/P17-22 wwwpolloooyyygo//1oo/// org/P17-201//
URL
Simone Conia, Daniel Lee, Min Li, Umar Farooq Minhas, Saloni Potdar, and Yunyao Li. Towards cross- cultural machine translation with retrieval-augnentee generation from multilingual knowledge graphs. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing, pp. 16343-16360, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/ve 10024emmll-mme main.944 914. URL https: www.antwwwwooooooo//22o222e emnlp-main. .94/
19

---

Published in Transactions on Machine Learning Research (01/2026)
Alexandre Diniz da Costa, Mateus Coutinho Marim, Ely Matos, and Tiago Timponi Torrent. Domain adap- tation in neural machine translation using a qualia-enriched FrameNet. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi 1sa- hara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, and Stelios Piperidis (eds.), Procoedings of the Thirteenth Language Resources and Evaluation Conference, pp. 1 1-12, Marseille, France, June 2022. European Language Resources Association. URL httpps//acdaatthoooog...or 2ttp://22221rre-1.1.1..
Menglong Cui, Pengzhi Gao, Wei Liu, Jian Luan, and Bin Wang. Multilingual machine translation with open large language models at practical scale: An empirical study. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5420-55 5443, Albuquerque, New Mcxico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189
6. doi: 10.18653/v1 /00//.222222--2---
280. URL htttss//accanntholoog/ org/ 2025 moon.lllog.... 280/
Praveen Dakwale and Christof Monz. Fine-tuning for neural machine translation with limited degradation across in- and out-of-domain data. In Sadao Kurohashi and Pascale Fung (eds.), Proceedings of Machine Translation Summit XVI: Research Track, pp. 156 169, Nagoya Japan, September 18 September 22
2017. URL htttps://aclanthollogt
org/2017 www.mmittttt-m.mmmmmmmm wwtpmmitttppppssttemm 13/
Hiroyuki Deguchi, Taro Watanabe, Yusuke Matsui, Masao Utiyama, Hideki Tanaka, and Eiichiro Sumita. Subset retrieval nearest neighbor machine translation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 174-189, Toronto, Canada, July 2023. Association for Computational URL
Linguistics. doi: 10. 10.18653/vl, 1o002.accl-oogg..
htt//////aattooooooo////2//3333 wrwp://.////22...looooog.. 10/
George Doddington. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the Second International Conference on Human Language Technology Research, HLT '02, pp. 138 145, San Francisco, CA, USA, 2002. Morgan Kaufmann Publishers Inc. URL https://aclanthon
org/www. mt wwrrchive.infoHH..C. 20wi-u--22222oodingt..e. pdf
Tobias Domhan and Felix Hieber. Using larget-side monolingual data for neural machine translation through multi-task learning. Tn Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference Empirical Methods in Natural Language Processing, pp. 1500- 1505, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10. 10.18653/v1 /1/613////D1711188.. URL https://aclanthoology otw.p/oooooooo//111////////
Zi-Yi Dou, Junjie Hu, Antonios Anastasopoulos, and Graham Neubig. Unsupervised domain adaptation for neural machine translation with domain-aware feature embeddings. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing pp. 1417 1422, Hong Kong, China, November 2019a. Association for Computational Linguistics. doi: 10. 10.18653/v1, /1////////111111// URL
Zi-Yi Dou, Xinyi Wang, Junjie Hu, and Graham Neubig. Domain differential adaptation for neural machine translation. In Alexandra Birch, Andrew Finch, Hiroaki Hayashi, Ioannis Konstas, Thang Luong, Graham Neubig, Yusuke Oda, and Katsuhito Sudoh (eds.), Proceedings of the 3rd Workshop on Neural Generation and Translation, pp. 59 69, Hong Kong, November 2019b. Association for Computational Linguistics. doi: 10.18653/v1 //11//66010 URL https://aclantthoogg org.y19-10DDD0000000 5606/
Bryan Eikema, Germán Kruszewski, Christopher R Dance, Hady Elsahar, and Mare Dymetman. An approx- imate sampler for energy-based models with divergence diagnostics. Transactions Machine Learning Research, 2022. ISSN 2835-8856. URL / wtt..e//oooummidddd/iirooo
Taisei Enomoto, Tosho Hirasawa, Hwichan Kim, Teruaki Oka, and Mamoru Komachi. Simultaneous domain adaptation of tokenization and machine translation. In Chu-Ren Huang, Yasunari Harada, Jong-Boo Kim, Si Chen, Yu- Yu-Yin Hsu, Emmanuele Chersoni, Pranav A, Winnie Huiheng Zeng, Bo Peng, Yuxi Li, and Junlin Li (eds.), Proceedings of the 37th Pacific Asia Conference on Language, Information and
20

---

Published in Transactions on Machine Learning Research (01/2026)
Computation, pp. 36-45, Hong Kong, China, December 2023. Association for Computational Linguistics. URL. https://acllathoot wwwp:/loogoorgg/222/02 www.//23.......oooo wrtp//2......acc-c--. paclic-1.44
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Tryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889 898, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 10. 18618/11111111 1082. CRL
1082/18-1082/
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchin- sky, Sergey Edunov, Michael Auli, and Armand Joulin. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1 2021. URL http://jmlr: org/papers/ x22/20-1307 html
M. Amin Parajian, Marco Turchi, Matleo Negri, and Marcello Federico. Multi-domain neural machine translation through unsupervised adaptation. In Ondřej Bojar, Christian Buck, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Phillip Kochn, and Julia Kreutzer (eds.), Proceedings of the Second Conference on Machine Translation, pp. 127-137, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: N1.18633/1/WI17-771., URL attts://accantteee wtt.e//oo///1//4333333
Zhaopeng Feng, Yan Zhang, Hao Li, Bei Wu, Jiayu Liao, Wenqiang Liu, Jun Lang, Yang Feng, Jian Wu, and Zuozhu Liu. TEaR: Improving LLM-based machine translation with systematic self-refinement. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 3922 3938, Albuquerque, New Mexico, April 2025. Association for doi: (1118633/112225.fmm andings-naacl.2111 URL Computational Linguistics. ISBN Sindinns-nnach 218/ https://acllnthhll.g wrt./ogg///200 http:/225.f.ndnnn.....
Markus Freitag and Yaser Al-Onaizan. Fast domain adaptation for neural machine translation, 2016. URL http://ww..rrg//bb/11...00.
Markus Freitag, Behrooz Ghorbani, and Patrick Fernandes. Epsilon sampling rocks: Investigating sampling strategies for minimum Bayes risk decoding for machine translation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 9198 9209, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/ 2023. findingn-ng-eee
findiing--emll..6.. emnlp.617. URL https::/
wrg/2023. .ffaffinndo certigs--enll.66
Victoria M. Garlock, Amanda C. Walley, and Jamie L. Metsala. Age-of-acquisition, word frequency, and neighborhood density effects on spoken word recognition by children and adults. Journal of Mcmory and
Language, 45(3):468-492) 2001. ISSN 0749-596X. doi: https://doi..rrg 10. 1ee11....0.mmmm.. http://maa..00.227.... URL
https: wttpp///ww..cieeeeeeeeeee http://wwwweeeneeerr....ooo ott.aaacci.............aaaeeeeep
Taisiya Glushkova, Chrysoula Zerva, and André F. T. Martins. BLEU meots COMET: Combining lexical and neural metrics towards robust machine translation evaluation. In Mary Nurminen, Judith Brenner, Maarit Koponen, Sirkku Tatomaa, Mikhail Mikhailov, Frederike Schierl, Tharindu Ranasinghe, Eva Van- massenhove, Sergi Alvarez Vidal, Nora Aranberri, Mara Nunziatini, Carla Parra Escartín, Mikel Forcada, Maja Popovic, Carolina Scarton, and Helena Moniz (eds.), Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pp. 47-58 Tampere, Finland, June 2023. European Association for Machine Translation. URL http:::////acaaathloooo hww.hholoooyorr//22520// camt-1.6/
Dongyoung Go, Tomasz Korbak, Germàn Kruszewski, Jos Rozen, Nahycon Ryu, and Marc Dymetman. Aligning language models with preferences through f-divergence minimization. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara. Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.). Proccodings of the 40th International Conference on Machine Learning, volume 202 of Proccedings of Machine Learning Research, pp. 11546-11588, PMLR, 23 29 Jul 2023. URL /ttpp:://rooceedngg.. mlr.
preas/v202/go2aa. html
21

---

Published in Transactions on Machine Learning Research (01/2026)
Shuhao Gu and Yang Feng. Investigating catastrophic forgetting during continual training for neural machine translation. In Donia Scott, Nuria Bel, and Chengqing Zong (eds.), Proceedings of the 28th International Conference on Computational Linguistics, pp. 4315 4326, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics. doi: 10.18653/v1 10553//110200.oliigg main.381. URL
https://aclanthoolog. haw.:p/o..or.222222oooom---iaaaa 381/
Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huci- Cuei-Chi Lin, Fethi Bougares,
Holger Schwenk, and Yoshua Bengio. On using monolingual corpora in neural machine translation, 2015.
URL https://arxiv... wttp:/iiw..rggga///////////// 0 03535
John Hewitt, Christopher Manning, and Percy Liang. Truncation sampling as language model desmoothing. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 3414-3427, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1 frg....inningsemml.2220 URL htll.httss://// lttpp:::/aaclanthooogyy
Chris Hokamp and Qun Liu. Lexically constrained decoding for sequence generation using grid beam search. In Regina Barzilay and Min- Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1535-1546, Vancouver, Canada, July 2017. Association for Computational Linguisties. doi: 10. (0.18653/v1/P17-10 org/P17-1141/
URL https://acclathhology
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degener- ation. In International Conference on Learning Representations, 2020. URL
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https::/openrevvee...u
J. Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tonglei Chen, Matt Post, and Benjamin Van Durme. Improved lexically constrained decoding for translation and monolingual rewriting. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 839-850, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v2 NN990-090 URL https:///ccaathhology org/N19-100///
Hidetaka Kamigaito, Hiroyuki Deguchi, Yusuke Sakai, Katsuhiko Hayashi, and Taro Watanabe. Diversity explains inference scaling laws: Through a case study of minimum Bayes risk decoding. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 29060- 29094, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89166-25--0 doi: 10. 11/6533//12225.aclll acl-long.11100... 1410. URL hutps://aclanthoologyy.
acl- 1410/
Muhammad Khalifa, Hady Elsahar, and Mare Dymetman. A distributional approach to controlled text gemeration. In International Conference on Learning Representations, 2021. URL htttp::/openreviie.. bet/fforum?id-jWssvt----
Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Nearest neighbor machine translation. In International Conference on Learning Representations, 2021. URL https://
wwwpeeevwwiiieffffrreeeeeeee ?iitrruBmdd///cCB J8hJM
Samuel Kicgeland and Julia Kreutzer. Revisiting the weaknesses of reinforcement learning for neural machine translation. In Kristina Toutanova, Anna Rumshisky, Luke Zeltlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 1673-1681, Online, June 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1 /2005/3/11/22.aaaeee naccllmmin main. 133. URL htttpp://aclaathologg.oo/////// www.::///........aaa.e.... 133/
22

---

Published in Transactions on Machine Learning Research (01/2026)
Adam Kilgarriff. Using word frequency lists to mcasure corpus homogeneity and similarity between corpora. In Fifth Workshop on Very Large Corpora, 1997. URL https://aacaaaataoggg..ooo//////
Minbeom Kim, Thibaut Thonet, Jos Rozen, Hwaran Lee, Kyomin Jung, and Marc Dymetman. Guaran- teed generation from large language models. In The Thirteenth International Conference Learning Representations, 2025. URL httpe://openrreiexx http:/eeeeetooormmm/drrroorrrrr
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization, 2017. URL https:
7/arxiv. org/ htt////ss//11...... 6988
Catherine Kobus, Josep Crego, and Jean Senellart. Domain control for neural machine translation. In Ruslan Mitkov and Galia Angelova (ods.), Procoodings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pp. 372 378, Varna, Bulgaria, September 2017. INCOMA Ltd. doi: 10. 10.26615/9788 954-455--4444 049. URL
Tom Kocmi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, and Maja Popović. Findings of the 2022 conference on machine translation (WMT22). In Philipp Koehn, Loic Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jusså, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Koemi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Mar- tin Popel, Marco Turchi, and Marcos Zampieri (eds.), Proceedings of the Seventh Conference on Machine Translation (WMT), pp. 1-45, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL https::/accaattaii
org/2022. wmt-1.1/
Philipp Koehn and Rebecca Knowles. Six challenges for neural machine translation. In Thang Luong, Alexandra Birch, Graham Neubig, and Andrew Finch (eds.), Proceedings of the First Workshop on Neural Machine Translation, pp. 28 39, Vancouver, August 2017. Association for Computational Linguistics. doi: URL
10.1
wrw.gogg.o..orrgooo wrg/W1733204
Tomasz Korbak, Hady Elsahar, German Kruszewski, and Mare Dymetman. Controlling conditional language models without catastrophic forgetting. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepes- vari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proccedings of Machine Learning Research, pp. 11499-11528. PMLR, 17-23 Jul 2022. URL https:: stttpe://proceeednng.. mlr. httpians.s....r..r.seeee v162 2et//////ooaaa2aaa2 html
Julia Kreutzer, Artem Sokolov, and Stefan Riezler. Bandit structured prediction for neural sequence-to- sequence learning. In Regina Barzilay and Min-Yer Kan (eds.). Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1503 1513, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: //aclanthology- hwwphhhooooooooo/PP1---- org/P17-1138/
1138. URL https:
Germán Kruszewski, Jos Rozen, and Mare Dymetman. disco: a toolkit for distributional control of generative models. In Danushka Bollegala, Ruihong Huang, and Alan Ritter (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) pp. 144- 160, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2022 acl-demo. 14. URL ptttps://aclanthll.. wwwphooooooooro//2222/ 2023 acl-demo. .1/
Batia Laufer and Paul Nation. Vocabulary size and use: Lexical richness in 12 written production. Applied Linguistios, 16(3):307 322, 09 1995. ISSN 0142-6001. doi: 0..109//applin/11/ org/ //tp101.100//ppp1i//1...1
URL httpp:://oi.
Minh- ThangThang Luong and Christopher Manning. Stanford neural machine translation systems for spoken language domains. In Marcello Federico Scbastian Stüker, and Jan Nichues (ods.), Procoedings of the 12th International Workshop on Spoken Language Translation: Evaluation Campaign, pp. 76-79, Da Nang, Vietnam, December 3-4 2015. URL http:///aaaaaaaoooooooooo///s/a/
23

---

Published in Transactions on Machine Learning Research (01/2026)
Qingsong Ma, Johnny Wei, Ondřej Bojar, and Yvette Graham. Results of the WMT19 metrics shared task: Segment-level and strong MT systems pose big challenges. In Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Kochn, André Martins, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Marco Turchi, and Karin Verspoor ), Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pp. 62-90, Florence, Italy, August 2019. Association for Computational URL cettps://aclanthhllee wttp//ooooog..rg//115552222
Linguistics. doi: 10.18653) wwwpw/////////
Mahdis Mahdich, Mia Xu Chen, Yuan Cao, and Orhan Firat. Rapid domain adaptation for machine trans- lation with monolingual data, 2020. URL
12652
Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang. Controllable lext generation with neurally- decomposed oracle. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 28125 28139. Cur- ran Associates, Inc., 2022. URL https::/procceddinn/ http://rrooodnnn...oooooopppp... wwtrriis.ccpppppppe http:/apppffffeepeeeeaaee ftt.s//ppppr//02////2/ 35467777788800777766222248888888 Sow.pperrrooooeeeeeeeep.
Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, and Rico Sennrich. Regularization tech- niques for fine-tuning in neural machine translation. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1489 1494, Copenhagen, Demmark, Seplember 2017. Association for Computational Linguistics. doi: (0.18653/v1/BB ///1//1111666. URL
Nguyen Nhat Minh, Andrew Baker, Clement Neo, Allen G Roush, Andreas Kirsch, and Ravid Shwartz- Ziv. Turning up the heat: Min-p sampling for creative and coherent LLM outputs. In The Thirteenth International Conference on Learning Representations, 2025. URL https:://ppenerrier..t wot.:oomwwnettooooiiii http:://ppmmmrriiioooooorrrrrrdd
FBkpCyujtS
Makoto Morishita, Katsuki Chousa, Jun Suzuki, and Masaaki Nagata. JParaCrawl v3.0: A large-scale English-Japanese parallel corpus. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 6704 6710, Marseille, France, June 2022, European Language Resources As- sociation. URL https://aclantholl.
oww.ooo///222...... 1.rec-1.721.
Diego Moussallem, Axel-Cyrille Ngonga Ngomo, Paul Buitelaar, and Mihael Arcan. Utilizing knowledge graphs for neural machine translation augmentation. In Proceedings of the 10th International Conference on Knowledge Capture, K-CAP '19, pp. 139-146, New York, NY, USA, 2019. Association for Computing URL https://doi. http:////oo..ggg..11111111
Machinery. ISBN 9781450370080. doi: 10. 10.1145/3360901.336447 010090013364422
Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help? In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. AAlché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL Paper. .dd
Moin Nadeem, Tianxing He, Kyunghyun Cho, and James Glass. A systematic characterization of sam- pling algorithms for open-ended language generation. In Kam-Fai Wong, Kevin Knight, and Hua Wu (eds.). Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pp. 334 346, Suzhou, China, December 2020. Association for Computational Linguistics. doi: 10.18653/v1/2000 aacl-main.36. URL
wrg//020.
36/
Arturo Oncevay, Charese Smiley, and Xiaomo Liu. The impact of domain-specific terminology on ma- chine translation for finance in European languages. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Procoedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 2758 2775.
24

---

Published in Transactions on Machine Learning Research (01/2026)
Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. Linguistsssiiiiiniiiisiiiiii... ISBN 6. doi: 10.18653/v1/22255.maaa wow.oooooo................oo 140. URL httpss//acllatttooogy.o/// org/y0y5rr22223 waw.2.aancnnl....mmm long. 140/
Jianhui Pang, Fanghua Ye, Derek Pai Wong, Dian Yu, Shuming Shi, Zhaopeng Tu, and Longyue Wang. Salute the classic: Revisiting challenges of machine translation in the age of large language models. Transactions of the Association for Computational Linguistics, 13:73 95, 2025. doi: 10.1162/tacl a 00730. URL
bttps://aclanthhlogg/ hwwpoloooo.oo/////// 2tt.:/.orrr/222...2111..24. tacl-1.4/
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational http//aaaaaaaallloorrr/////////e
Linguistics. doi: 10.3115/1073000 1073135. URL
Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 186 191, Brussels, Belgium, October 2018. Association for Computa- tional Linguistics. doi: 10.18653/vl v1/W18-6319. URL htttps://aclanthoologg org/W18-6333 6319
Matt Post and David Vilar. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume 1 (Long Papers), pp. 1314-1324, New Orleans, Louisiana, June 2018. Association for Computational Linguisties. doi: 10.18653/v 11/1///11111111/// URL https: hoap:::/aclanthooogee
org/N18-1119/
Matt Post, Shuoyang Ding, Marianna Martindale, and Winston Wu. An exploration of placcholding in neural machine translation. In Mikel Forcada, Andy Way, Barry Haddow, and Rico Sennrich (eds.), Proceedings of Machine Translation Summit XVII: Research Track, pp. 182 192, Dublin, Ireland, August 2019. European Association for Machine Translation. URL https://acclatthollogg http://aattooolooooo/////-----1
Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojeiech Zaremba. Sequence level training with recurrent neural networks, 2016. URI.https::// htttps::/aaxx...or//bbb//////// ort..//////b//11100722222
Vikas Raunak and Arul Menezes. Finding memo: Extractive memorization in constrained sequence gen- eration tasks. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 5153 5162, Abu Dhabi, United Arab Emirates, Decem- ber 2022. Association for Computational Linguistics. doi: 10.18653 (118653/v1/2022 ///2/22..nnnn/nng
www.inng-eennl........ emnlp.378. URL
wwtg/2022.findinnee... findingnnnnnnllp .ea.gg.--mmllp...... 378/
Paul Rayson and Roger Garside. Comparing corpora using frequency profiling. In The Workshop on Comparing Corpora, pp. 1-6, Hong Kong, China, October 2000. Association for Computational Lin- URL
guistics. doi: 111111/1117799111111111
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET: A neural framework for MT evaluation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685-2702, Online, November 2020. Association for Computational Linguistics. doi: 10. 10.18653/v vl 11/8633/v1/200.emm/l 120000.emmmpmmmain221... URL https://aclanthoooogy .rg. 2020. emnlp- main. 213/
Danielle Saunders and Steve DeNeefe. Domain adapted machine translation: What does catastrophic forget- ting forget and why? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 12660-12671, Miami, Florida, USA, November 2024. Association for Computational ional Linguistios. doi: 10.18653 w URL https://aclantthllog. org 2024 emnlp-main. 704/
704.
Thibault Sellam, Dipanjan Das, and Ankur Parikh. BLEURT: Learning robust metrics for text generation. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7881-7892. Online, July 2020. Association for Computational Linguistics. doi: 10.18653 00./665//1/2220000 /1122000.clll/aaa 704. URL ittps://aclanthologyy org/
2020. m044
25

---

Published in Transactions on Machine Learning Research (01/2026)
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. In Katrin Erk and Noah A. Smith (eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 86 96, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18633/v1 vv/11/-10P0/ URL https:// aclanthology.oo ottp://ooog..rrg///111000000
Jiajun Shen, Peng-Jen Chen, Matthew Le, Junxian He, Jiatao Gu, Myle Out, Michael Auli, and Marc' Aurelio Ranzato. The source-target domain mismatch problem in machine translation. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (ods.), Proceedings of the 16th Conference of the Europcan Chapter of the Association for Computational Linguistics: Main Volume, pp. 1519-1533, Online, April 2021. Association for Computational Linguisties. doi: 10. 10.11865/11/// /2021 22221.eccl-maa. eacl-main. URL htt.ittt////aaaattooool org/ 2021. eacl-main m
Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, and Min Zhang. Code-switching for enhanc- ing NMT with pro-specified translation. In Jill Burstcin, Christy Doran, and Thamar Solorio (cds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 449 459, Minneapo- lis, Minncsota, June 2019. Association for Computational Linguisties. doi: 10. 10.18653/ 11119--1100AA URL https://acclatthen wwg..oorr//111-044//0
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Saftyyah Saleem, Holger Schwonk, and Jeff Wang. No language left behind: Scaling human-centerod machine translation, 2022.
URL https:
.06672
Brian Thompson, Jeremy Gwinnup, Huda Khayrallah, Kevin Duh, and Philipp Koehn. Overcoming catas- trophic forgetting during domain adaptation of neural machine translation. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologics, Volume 1 (Long and Short Papers), pp. 2062 2068, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N// 1209. URL
Liang Tian, Derek F. Wong, Lidia S. Chao, Paulo Quaresma, Francisco Oliveira, Yi Lu, Shuo Li, Yiming Wang, and Longyue Wang. UM-corpus: A large English-Chinose parallel corpus for statistical machine translation. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.). Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pp. 1837-1842, Reykjavik, Iceland, May 2014. European Language Resources Association (ELRA). URL http::/ www.:wwwweeccoooo conf org/
Paper .ff
Jörg Tiedemann. Parallel data, tools and interfaces in OPUS. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Dogan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Proccedings of the Eighth International Conference Language Resources and Evaluation (LREC'12). pp. 2214 2218, Istanbul, Turkey, May 2012. European Language Resources Association (ELRA). URL https://aclantbbooggy org/L12-1245)
Jörg Tiedemann and Santhosh Thottingal. OPUS-MT Building open translation services for the World. In Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT), Lisbon, Portugal, 2020.
Ashish Vaswani, Noam Shazcor, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiscr, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems,
26

---

Published in Transactions on Machine Learning Research (01/2026)
volume 30. Curran Associates, Inc., 2017. URL http:://ppoeeiassssoorrrs.cppppp .df
Ashwin Vijayakumar, Michael Cogswell, Ramprasaath Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. Diverse beam search for improved description of complex scenes. Proccodings of the AAAI Conference on Artificial Intelligence, 32(1), Apr. 2018. doi: 111609/aaai.v3211..1 URL https://ojs. aawii..ssaaaa................ org/ werg//ndex... hhtp:///////rrrraaa/////////////
Dexin Wang, Kai Fan, Boxing Chen, and Deyi Xiong. Efficient cluster-based A-nearest--neighhor machine translation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Moeting of the Association for Computational Linguistics (Volume Long Papers), pp. 2175 2187, Dublin, Ireland, May 2022. Association for Computational Linguisties. doi: 10.18653/v1/ 2022. 2022.aclllong. 154. URL nttpps//acllattoooggy.ooo
aall-loggg... 154/
Rui Wang, Andrew Finch, Masao Utiyama, and Fiichroo Sumita. Sentence embedding for neural machine translation domain adaptation. In Regina Barzilay and Min-Yen Kan (cds.), Procoodings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume Short Papers), pp. 560 566, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10.18653/v1/P17-2 org/p17-208// URL https:://ccaatthol.gg/ htt.://aaaaaalooo.oo///770000090
Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu. On the inference calibration of neural machine translation. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel Tetreault (ods.), Proceodings of the 58th Annual Meeting of the Association for Computational Linguistios, pp. 3070 3079, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/v1 2200:/3//1/0000accmmmmaa aclanthology org/2020. acl-main 278/
URL https://
oclanthologyy.or/22000
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: Stato-of-the-art natural language processing. In Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 45, Online, October 2020. Association for Computational Lin- guisties. doi:
URL attpp://aclannhooogy.orggg org/2020 http:/2020..mmllpeeeeemm emmlp-demos.
6/
Haoran Xu, Kenton Murray, Philipp Koehn, Hieu Hoang, Akiko Eriguchi, and Huda Khayrallah. X-ALMA: Plug play modules and adaptive rejection for quality translation at scale, In The Thirteenth International Conference on Learning Representations, 2025. URL /tttps://openreevler http:://ppppeoeteeeeeeea//rrrrrr htapicsssssp11r111 1p8xUq
Lingling Xu, Haoran Xie, Si-Zhao Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameter-efficient fine-tuning methods for pretrained language models: critical review and assossment, 2023. URL httts:///axxi org/abs/2312.1 12148
Guangyu Yang. Jinghong Chen, Weizhe Lin, and Bill Byrne. Direct preference optimization for neural machine translation with minimum Bayes risk decoding. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 2: Short Papers), pp. 391 398, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653 (011653/v1/220 1222044.22mmmmm naacl-shorttttoo URL
org 2024 naacl- 34/
Lana Yeganova, Dina Wiemann, Mariana Neves, Federica Vezzani, Amy Siu, Inigo Jauregi Unanue, Maite Oronoz, Nancy Mah, Aurélic Névéol, David Martinez, Rachel Bawden, Giorgio Maria Di Nunzio, Roland Roller, Philippe Thomas, Cristian Grozea, Olatz Perez-de Viñaspre, Maika Vicente Navarro, and An- tonio Jimeno Yepes. Findings of the WMT 2021 biomedical translation shared task: Summaries of animal experiments as new test set. In Loic Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatter- jee, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewics, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes,
27

---

Published in Transactions on Machine Learning Research (01/2026)
Philipp Koehn, Tom Koemi, Andre Martins, Makoto Morishita, and Christof Monz (eds.), Proceedings of the Sixth Conference on Machine Translation, pp. 664 683, Online, November 2021. Association for Computational Linguistics. URL https://aclanthollgg http://cccanttooooooo/////2///2 org/2021.......... wmt-1.70/
Hongxiao Zhang, Hui Huang, Jiale Gao, Yufeng Chen, Jinan Xu, and Jian Liu. Iterative constrained back-translation for unsupervised domain adaptation of machine translation. In Nicolctta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin- Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lec, Enrico Santus, Francis Bond, and Seung-Hooe Na (eds.), Proceedings of the 29th International Conference on Computational Linguistics, pp. 5054 5065, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguisties. URL https://cclanthol.o
Jiajun Zhang and Chengqing Zong. Exploiting source-side monolingual data in neural machine translation. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1535 1545, Austin, Texas, November 2016. Association for http://cclaatttooooooorr/-//--
Computational Linguistics. doi: 10. 10.18653/v1 //1/ 1160. URL
Ran Zhang, Wei Zhao, and Steffen Eger. How good are LLMs for literary translation, really? literary transla- tion evaluation with humans and LLMs. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 10961-10988. Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-899176-189-6 doi: org/2025 www.l-lonnn....... 548/
10.18653 w222025..2222/ www.l-aaaanl......
URL
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi BERTScore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020. URL https:
//openreview http://wweetttorrrmmidddssseeeie
Yang Zhao, Lu Xiang, Junnan Zhu, Jiajun Zhang, Yu Zhou, and Chengqing Zong. Knowledge graph en- hanced neural machine translation via multi-task learning on sub-entity granularity. In Donia Scott, Nuria Bel, and Chengqing Zong (eds.), Proceedings of the 28th International Conference on Computational Linguistios, pp. 4495-455 1995-455,,10 Barcelona, Spain (Online), December 2020. International Commitlee on Com- putational Linguistics. doi: 10. 10.18653/v /11////////// coling main. /97/
/tt///111100 /22/2222020ooog. www.:oooninmmm.33........ main URL htttps:/accaantholle/
prg/2020.
Xin Zheng, Zhirui Zhang, Shujian Huang, Boxing Chen, Jun Xie, Weihua Luo, and Jiajun Chen. Non-parametrie unsupervised domain adaptation for neural machine translation. In Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguisties: EMNLP 2021, pp. 4234-4241, Punta Cana, Dominican Republic, Novem- findings-emnlpe URL
ber 2021. Association for Computational Linguistics. doi: 10.18653/v httpps//aclannthology.. wwwpanthhooooooooo//222222/22 202/ windingsseenni emnlp.358/
Vilém Zouhar, Pinzhen Chen, Tsz Kin Lam, Nikita Moghe, and Barry Haddow. Pitfalls and out- looks in using COMET. In Barry Haddow, Tom Koemi, Philipp Koehn, and Christof Monz (eds.), Proceedings of the Ninth Conference on Machine Translation, pp. 1272 1288, Miami, Florida, USA, November 2024a. Association for Computational Linguistics. doi: 10.18653/vl vv20/2/4.22222...// htt///222...tt1111 1mtt-1111.. URL https://aclanthol.. www.aanthhoooooooo/222224/.. org/2024. 1tt0:24..mtt211 wmt- 1. 121/
Vilém Zouhar, Shuoyang Ding, Anna Currey, Talyana Badeka, Jenyuan Wang, and Brian Thompson. Fine- tuned machine translation metrics struggle in unseen domains. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 488-500, Bangkok, Thailand, August 2024b. Association for Computational Linguistics. doi: 10.18653/v1 /2024.acl-shorr. URI.httpss http::///caaaahoa/a/caeo hwwpollooyyorgg/000 org/2024. acl-short.455
28


Reviews:
[{"review_id": "jv2QWtmyRp", "reviewer": "Reviewer_JUKR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes domain adaptation for neural machine translation using only target-side monolingual lexical distributions (subword frequencies). The method builds on Conditional Distributional Policy Gradients, treating the target domain’s lexical statistics as features of a conditional energy-based model and then fine-tuning a pretrained NMT to approximate that distribution while aiming to avoid catastrophic forgetting. The results show effective domain adaptation using only a small amount of monolingual resources.\n\nStrengths:\n1. The methods are in a rigorous mathematical formulation and supported by solid experiments.\n2. The paper provides a broad evaluation across 16 scenarios and analyzes when CDPG is effective. This is a useful practical contribution for low-resource or sensitive domains where parallel data is scarce.\n\nWeakness:\n1. Confidence can increase even when adequacy does not; NIST emphasizes rare tokens and may over-credit lexical matches. Maybe add some human evaluation and report BLEU/BERTScore with statistical significance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors conduct extensive experiments and result analysis to support their claims.", "audience_yn": "Yes", "audience_explanation": "In the fine-tuning stage, mitigating overfitting and catastrophic forgetting remains a central challenge in deploying large language models (LLMs). Although the paper focuses only on machine translation, its method—statistically selecting an appropriate lexical distribution for domain adaptation—has clear potential for broader applications.", "requested_changes": "Included in Weakness.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BTS3yeqrHA", "reviewer": "Reviewer_bctu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the challenge of adapting a generic neural machine translation (NMT) model to a new domain. The authors propose leveraging domain-specific lexical statistics, specifically word frequency distributions, to guide the fine-tuning process. Unlike conventional approaches that require large parallel corpora, this framework adapts the model using only monolingual lexical distributions from the target domain. Through extensive experiments, the paper demonstrates that the method mitigates catastrophic forgetting and remains robust to parameter choices, achieving effective domain adaptation with minimal resources", "claims_and_evidence_yn": "Yes", "claims_explanation": "To my knowledge, yes. There is a thorough experimental evaluation and the model is based on existing works of energy based models.", "audience_yn": "Yes", "audience_explanation": "Given that, creating training data for a specific domain application is challenging, I believe that the topic of domain adaptation in the context of machine translation is relevant for the TMLR audience.", "requested_changes": "I would like the authors to address the following four comments:\n\n* Given that the work builds up on CDPG, section 2 is quite compact and lacks details for the general reader. For example, in equation (2), the subscript point is not defined and it is left for the reader to infer. The authors should explain the idea of CDPG in a clear manner to a general reader while deferring details to Appendix or existing works. I found this section difficult to read and understand, before I referred to other works.\n\n* Dynamic CDPG is one of the contributions of this paper. The authors do point out that even with out this, numerical tests show good results. In section 3.2, please explain exactly what dynamic means (e.g., a proper pseudocode  or sketch of an algorithm). \n\n* Please have a contributions section and highlight the contributions of this paper. From my reading, CDPG has been studied before. Are the main contributions the dynamic adaption and using word frequency? This would help frame the novelty of the paper.\n\n* What is the computational computational complexity of EMBS?", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "o5BPWFolKl", "reviewer": "Reviewer_utkB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses on an important topic in domain translation, particularly the challenge of adapting NMT systems without sufficient domain-specific parallel data. The motivation is valid, and the authors present their ideas clearly, making the paper easy to follow. The experimental results are reported in detail across multiple translation directions and domain datasets.\n\nHowever, the contribution appears limited in novelty, and the methodological concerns with Dynamic CDPG need to be carefully addressed:\n1. The main technical component, Conditional Distributional Policy Gradients (CDPG), is not new. The authors primarily adapt CDPG to the domain adaptation setting, and the use of word frequency statistics as guidance, while intuitive, appears rather straightforward. The overall methodological contribution therefore feels incremental.\n2. The newly proposed Dynamic CDPG raises methodological issues. As described in Section 3.2, each iteration is evaluated with BLEU to decide whether to accept the update. Since BLEU is part of the final evaluation metric, this effectively leaks test-time information into training, which is generally not acceptable. The authors should clarify whether BLEU is computed on a held-out validation set, and if not, this practice is problematic.\n3. Generality of the Approach: Dynamic CDPG is only designed for nucleus sampling. It is unclear whether the method generalizes to other sampling approaches. This reduces the broader applicability of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The proposed method is supported by many numerical experiments.", "audience_yn": "Yes", "audience_explanation": "This paper focuses on an important topic in domain translation, particularly the challenge of adapting NMT systems without sufficient domain-specific parallel data.", "requested_changes": "1. Provide a brief introduction to nucleus sampling and the definition of top-p when it is first mentioned in Section 3.2, so that readers less familiar with the concept can follow the discussion.\n2. In Tables 1 and 2, several results are very close. To make the findings more robust, please report results averaged over multiple runs and include variance or standard deviation.\n3. Clarify the role of BLEU in Dynamic CDPG to ensure there is no inappropriate use of test data during training.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)