Toward Domain Translation with Monolingual Domain Data Only

Authors: Yusuke Sakai, Zhi Qu, Hidetaka Kamigaito, Taro Watanabe, Xiaojiang Liu
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Neural Machine Translation, Unsupervised Domain Adaptation, Energy-Based Models, Conditional Distributional Policy Gradients

Abstract:
Neural machine translation (NMT) is very sensitive to domain shifts requiring a carefully designed fine-tuning strategy to avoid catastrophic forgetting problems when adapting to a new domain.  Fine-tuning usually relies on high quality in-domain data, but constructing a sufficient amount of parallel data for training poses challenges even for fine-tuning. In contrast, domain-specific monolingual resources are more accessible when compared with bilingual data. Therefore, we challenge the domain adaptation of a general NMT model using only features obtained from a small amount of monolingual data. We regard the task as an instance of domain shifts, and adopt energy-based models (EBMs) and approximate these EBMs using Conditional Distributional Policy Gradients (CDPG). Recent work has applied CDPG with a small number of EBMs for NMT models limiting the capacity for domain shifts, but we construct a large number of EBMs considering the entire domain-specific data, i.e., unigram distribution, and perform fine-tuning according to their constraints. Our results show that fine-tuning using a large number of EBMs can achieve a robust domain shift without causing catastrophic forgetting, demonstrating a robust domain shift using only a small amount of monolingual resources.

Decision: reject
Reviewer scores: 6, 6, 3, 3

Reviews and discussion:
<thread>
creation date:
1729634879184

thread_type:
official_review

<thread_object>
creator: reviewer_n3sl

<content>
summary:
This paper addresses the topic of adapting a neural machine translation model to a specific domain when only target-language domain-specific data is available. The proposed approach uses energy-based models based on the vocabulary of the domain-specific data to adapt an existing NMT model to the specific data. The proposed model improves translation quality somewhat inconsistently, and through analysis this is attributed to improvements on domain-specific terminology.

strengths:
1. The proposed method is described clearly and the method itself is well-justified.

2. There is a fair amount of analysis into successful and failure cases.

3. The results include statistical significance testing.

weaknesses:
1. This paper would benefit from clearly defining and justifying the task at hand. It addresses domain adaptation with target-side monolingual data, and that is well justified. However, there are several implicit assumptions throughout that do not seem to be stated clearly or given a justification. For example, a) assuming that they are starting from a generic trained model to adapt, and cannot train a model to the relevant domain from scratch (section 4.2), b) assuming that the adapted model shouldn't have catastrophic forgetting of the original domain (line 45), c) assuming that it is desirable to do well on tasks that were unseen in both the original model and the adaptation data (table 6), and d) that the adapted output should exhibit minimal changes compared to the original model (line 410). There is nothing wrong with these assumptions necessarily, but you need to clearly state them and justify why you are restricting your exploration to these.

2. There are some issues with the evaluations. First, the base model that is used is OPUS-MT, but the domain-specific datasets that are used for evaluation for EN<->DE come from OPUS, so they were used to train the base model. Thus, this is not a true scenario of domain adaptation to an unseen domain, but one of domain shift. It is not clear to me whether this was done intentionally, but I think it would be preferable to do some domain adaptation evaluations with unseen data (and this might explain the lack of consistent positive results for any of the domain adaptation models, including the baselines and the EBM approach). Second, confidence is used as an evaluation score, when it is not clear that this correlates with any sort of meaningful MT evaluation. Third, the examples given in table 5 point more towards overfitting to the vocabulary of a specific dataset (*not* a specific domain) than to any true translation quality improvements.

3. This paper should take a broader view of the literature, including terminology-constrained machine translation (which seems to be hinted at as the ultimate goal of the proposed approach, e.g. in line 410 and table 5) as well as cases where the assumptions I listed in item 1 are relaxed (e.g., training a domain-specific model from scratch). In addition, the following paper is directly related (even without taking a broader view) and should be used as a baseline: https://arxiv.org/pdf/2010.12652.

4. Beyond the missing citations, the baselines are insufficient or problematic. a) The proposed approach should be compared against LLM translation, both generic and using in-context learning with monolingual examples, the latter of which would directly address the problem at hand. b) The fine-tuning comparisons only evaluate i) fine-tuning the entire model and ii) fine-tuning only the attention weights. To me given the small dataset and focus on target-side data it would make sense to explore other approaches like fine-tuning the decoder only. c) Line 199 says "the checkpoint, which has the best performance on the development set, is measured for comparison." but line 193 says fine-tuning is done on the development set. So the models are fine-tuned on the same set that is used for checkpoint selection; it would not be surprising if they don't generalize well to the test set.

questions:
1. In line 36, you say "automatically collecting a sufficient amount of domain-specific parallel data is challenging". It would be good to get some quantitative information to justify this statement, particularly what you mean by "sufficient". In general, fine-tuning and ICL can work well with an extremely small corpus.

2. Line 44 says: "However, naively performing fine-tuning [...] can lead to catastrophic forgetting issues, such as the loss of fluency in the translated sentences acquired during pre-training, thereby causing a reduction in translation performance". Can you share evidence of this? Typically, catastrophic forgetting doesn't cause a loss of *fluency* in NMT per se, but just poorer performance on seen domains.

3. It would be good to add a discussion of whether the EBMs increase the parameter size, memory footprint, or inference speed of the model.

4. Line 194 should cite the back-translation paper. Also, I would recommend including a comparison to back-translation as a baseline, and labeling "fine-tuned" as your upper bound.

5. I found the presentation of table 6 extremely confusing. It would be clearer to simply show the BLEU scores of the two models, rather than showing the difference between them. In addition, if you are testing for catastrophic forgetting, you should: a) show scores for the unadapted model for comparison, and b) evaluate on a domain seen by the original/unadapted model.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730399513673

thread_type:
official_review

<thread_object>
creator: reviewer_umaa

<content>
summary:
This paper looks at modeling domain-specific translation by formulating the target domain as a conditional energy based model (EBM), and approximates the EBM with conditional distributional policy gradients (CDPG). The approximation is done by using unigram probabilities to model the target domain, create binary features of when unigrams appear in target sentences, and fine-tune the pre-trained translation model according to the constructed EBM. The authors also propose a dynamic variation of CDPG specific to autoregressive modeling. On en <-> zh and en <-> de models, the method is compared to the original model, a 3k sample fine-tuned model, and a LoRA fine-tuned model.

strengths:
1. Energy-based models have been well established for controllable generation and using them in the context of domain-specific MT is a solid application in an important problem. 
2. This method can apply on top of pre-existing models, as the authors show with the OPUS-MT models. This improves its extensibility where a model does not need to be trained from scratch to begin with.
3. The method outperforms a normal fine-tuning based approach for a reduced number of fine-tuning sentences, across 4 language directions.

weaknesses:
1. My biggest concern with this work is its limited novelty especially compared to the work of Korbak et al. 2022. This work proposes how to represent the feature probabilities $\mu$ with unigram probabilities, but that seems to be all beyond the Korbak paper. Some discussion of exactly where this work differs would help differentiate them and highlight novelty. 
2. The use of EBMs and CDPG is motivated by avoiding catastrophic forgetting, but there does not seem to be evidence of this in the results. In the intro, catastrophic forgetting is equated to reduction in translation performance, which is not quite what other work defines as catastrophic forgetting. Catastrophic forgetting refers to becoming worse on the original data distribution - showing improved performance on the target domain is not enough to evaluate catastrophic forgetting. The methods should ideally be evaluated on the original domain as well in order to show this.
3. Calling this method “monolingual data only” or unsupervised does not seem to work well with proposing dynamic CDPG as a main contribution in this work. These descriptors seem to only target the extension of vanilla CDPG to MT. These descriptors seem to only target the extension of original CDPG to MT. 
4. There is a lack of implementation details regarding the EBM models and the fine-tuning procedure.

questions:
1. What is the significance test you are running in the main tables?
2. Why use BERTScore over COMET? The latter is generally better for evaluating translations. 
3. What is needed for the implementation of the EBMs? Some pseudocode/algorithm/code outlining the creation of the EBMs and the fine-tuning of the translation models would be helpful.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730817577644

thread_type:
official_review

<thread_object>
creator: reviewer_vupi

<content>
summary:
This paper presents a new domain adaptation method for adapting a pretrained NMT model with low-resource monolingual in-domain data. The method employs conditional distribution policy gradients to approximate domain-specific features in the target languages, where the domain-specific features are represented by unigram distributions. The authors also propose dynamic CDPG, which dynamically adjusts parameters using a small bilingual validation sample. Experimental results show that their framework achieves improvements in some domains, primarily due to the model's enhanced learning of domain-specific words.

strengths:
- Novel approach for domain adaptation using only monolingual data: The paper presents energy-based models to effectively adapt to monolingual domain-specific corpora.
- Extensive evaluation: The paper includes a variety of domains and languages to validate the proposed methods and provides additional analysis of improvements by assessing domain-specific terminology.

weaknesses:
- Concerns with practicality and scalability: The setup in this paper raises concerns about practical applicability. In real-world scenarios, domain-specific corpora may often be more abundant, even when restricted to monolingual resources, and may differ significantly in size compared to the data setup in the paper. It remains unclear whether the proposed framework would can show scale well with larger datasets or whether it would overfit to small monolingual training sets, potentially only capturing domain features seen during training. This could lead to an over-reliance on vocabulary or terms specifically within the training set, rather than robust. The results may reflect high performance in producing domain-specific terms, but this may primarily apply to cases where test instances closely match the training domain vocabulary, potentially limiting broader generalization. 

- Evaluation metric suitability: The selected evaluation metrics may not fully demonstrate the strengths of the proposed approach. The confidence scores of softmax indomain translation is somewhat concerning, as this does not necessarily correlate with successful adaptation. Moreover, author did not consider other naturalness metric  like fluency or advanced model based quality estimation metric, such as COMET. These metrics could provide a more reliable measure of adaptation performance and reveal whether the model truly maintains translation quality.

questions:
- Domain feature generalization: One question is whether the model truly learns and generalizes domain features using this method. Specifically, I am curious if, during testing, the model can generate domain-specific terms that were not present in the monolingual training data but are consistent with the target domain's linguistic characteristics. If the model can accomplish this, it would indicate a deeper understanding of domain features rather than merely reproducing the training vocabulary. A more detailed analysis or experiments testing this capability would be valuable.

- Sensitivity to validation bilingual set in DCDPG: Given that DCDPG relies on a bilingual validation set for dynamic parameter tuning, I wonder if the method’s performance is sensitive to the quality and representativeness of this validation set. Could performance vary significantly depending on the domain alignment or size of the validation set used? This would be an important consideration for practitioners, as varying validation sets could lead to inconsistent results in practice.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731038506715

thread_type:
official_review

<thread_object>
creator: reviewer_oe5u

<content>
summary:
The paper introduces a method to perform domain translation for neural machine translation (NMT) by utilizing only monolingual domain-specific data. The authors employ energy-based models (EBMs) combined with Conditional Distributional Policy Gradients (CDPG) to perform domain adaptation without relying on large-scale parallel domain-specific data, which is often challenging to collect. The paper further proposes DYNAMIC CDPG to improve upon traditional CDPG by dynamically adjusting parameters using bilingual validation data, aiming to achieve optimal results without catastrophic forgetting. Experiments are conducted across several translation directions and domain adaptation scenarios.

strengths:
Innovative Approach: The use of energy-based models combined with CDPG for domain adaptation with only monolingual data represents a novel approach that addresses a key limitation in domain-specific NMT.
Effective Results: The experimental results show that DYNAMIC CDPG performs well compared to fine-tuning and LORA-based methods, with improvements in key evaluation metrics such as BLEU and NIST scores.
Reduction in Catastrophic Forgetting: The proposed approach successfully mitigates catastrophic forgetting, which is a common issue in domain adaptation tasks, thus preserving the pre-trained model’s knowledge while adapting to a new domain.

weaknesses:
Limited Scope of Evaluation: The experiments are conducted on a relatively small set of translation directions and domains. Expanding the evaluation to include additional languages and domains would provide stronger evidence for the generalizability of the approach.
Lack of Robust Comparison: The paper primarily focuses on comparisons with pre-trained, fine-tuned, and LORA baselines. It would be beneficial to include comparisons with other strong domain adaptation techniques such as back-translation and adversarial domain adaptation, which could provide a more holistic understanding of the strengths and weaknesses of the proposed methods.
Complexity of Methodology: The proposed methodology is relatively complex, and while the theoretical explanations are well-written, it may be challenging for readers without a background in reinforcement learning or energy-based modeling to fully grasp. Adding intuitive explanations or visual aids would enhance accessibility.

questions:
Hyperparameter Analysis: While Table 2 provides insight into top-p values used for DYNAMIC CDPG, can the authors provide a deeper analysis on the sensitivity of other hyperparameters (e.g., learning rate, λ values for EBMs) and how they impact the final results? A sensitivity analysis or a hyperparameter optimization discussion could greatly strengthen the paper.

Computational Efficiency: How does the proposed approach compare in terms of computational cost to other domain adaptation methods, such as back-translation or adversarial domain adaptation? The paper mentions improvements in alignment quality, but a more explicit analysis of the computational trade-offs would be valuable.

Scaling to Larger Domains: Can the authors discuss the scalability of the proposed approach to much larger domain adaptation tasks? For example, would the methodology perform well with highly diverse target domains or significantly larger monolingual datasets?

Effectiveness of Monolingual Features: The paper leverages unigram frequency for domain adaptation, which is a relatively simple feature representation. Have the authors considered experimenting with more sophisticated feature representations, such as n-gram frequencies or embeddings? Would these improve the performance of CDPG and DYNAMIC CDPG for domain shifts?

Handling Noisy Monolingual Data: In real-world scenarios, monolingual domain data might be noisy. How robust is the proposed approach when dealing with noisy or imperfect monolingual data? Some discussion or experiments on the effects of noisy data would be insightful.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732674061103

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Kind Reminder

comment:
Dear Reviewers,

As the deadline for discussion nears, we wish to reaffirm our dedication to addressing any unresolved concerns regarding our submission. We have thoroughly addressed all the latest concerns, and we believe these efforts have significantly strengthened our manuscript. **If our responses have resolved your concerns, we would greatly appreciate an updated evaluation.**

We recognize and appreciate the significant time commitment the review process requires, and we highly value your feedback. If there are any additional recommendations for enhancing our submission, we would be happy to take them into consideration.

Thank you for your time and thoughtful consideration,

Anonymous Authors
</content>
</thread_object>

</thread>
<thread>
creation date:
1733157427168

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal Summary and Final Follow-up

comment:
Dear Reviewers and ACs,

As the discussion period approaches its conclusion, we would like to provide a reminder and summarize our responses to the feedback received so far.

We sincerely thank all the reviewers and ACs for your diligent efforts and high-quality reviews. If you have any additional questions or require further clarification, please feel free to let us know. Your insights are highly valued.

---

We are delighted to note that reviewers have highlighted the following strengths in our paper:
- Innovative, novel, and extensible approach: Recognized as innovative (`oE5u`), novel (`VUPi`), extensible (`UmaA`), and a solid application (`UmaA`), with a well-justified and clearly described method (`n3SL`).
- Effective and extensive evaluation results: Highlighted for the robustness and thoroughness of the evaluation (`oE5u`, `VUPi`, `UmaA`, `n3SL`).
- Successful mitigation of catastrophic forgetting: Demonstrates effective domain adaptation while addressing catastrophic forgetting (Reviewers `oE5u`, `VUPi`).
- Performance improvement: Shows good performance compared to baselines (`oE5u`, `VUPi`, `UmaA`, `n3SL`).

In response to your valuable suggestions, we have conducted additional experiments and made several key modifications to the manuscript. For your convenience, we have highlighted these changes in green or red text in the revised manuscript:
- Back-translation results: Added in Appendix F and Table 8 (Addresses `oE5u` W2,` n3SL` W4, Q4).
- Noise and cross-domain settings: Added in Appendix G and Table 9 (Addresses `oE5u` W2, Q3, Q5, `VUPi` W1, n3SL W3).
- COMET Score: Included in footnote 12 of Section 4.3 and Appendix H (Addresses `VUPi` W2, `UmaA` Q2).
- Domain feature generalization discussion: Expanded in Appendix I (Addresses `VUPi` Q1, `n3SL` W2).
- Task definition and motivation for catastrophic forgetting: Elaborated in Section 1 Introduction (Addresses `UmaA` W2, `n3SL` W1, W4, Q2).
- General domain results: Added in Table 6 for a more robust evaluation of catastrophic forgetting (Addresses `UmaA` W2, `n3SL` W1, Q2, Q5).

For other individual suggestions and concerns, we have provided detailed responses for each reviewer in the comments section of the discussion.

Thanks to the insightful feedback from reviewers, we believe the revised manuscript has been significantly improved, making it a valuable and accessible contribution for a broad audience.

---

We deeply understand that the reviewing process is a volunteer effort, and we sincerely appreciate the time and effort you have devoted to providing feedback. This paper represents not only solid, insightful, and novel work but also a meaningful contribution to the research community. Your suggestions have helped us address its weaknesses and further strengthen its contributions.

**If your concerns have been addressed, we kindly request that you consider raising your score**. Should you have any remaining concerns, please do not hesitate to let us know. We are committed to addressing them sincerely and thoroughly within the remaining time. However, if we do not receive any further response, we will consider all concerns resolved. Thank you for your understanding and for engaging in a constructive discussion as the review process concludes.

We look forward to your reply.

Best regards,

Anonymous Authors
</content>
</thread_object>

</thread>
<thread>
creation date:
1733305959161

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Thank you for the Discussion Phase and Rebuttal Summary (1/2)

comment:
Dear AC and reviewers,

We thank you for the time and effort you have dedicated to reviewing our work, which has been incredibly helpful in improving its quality. While we understand that reviewers may have been too busy during the rebuttal period, resulting in limited discussion, **we encourage further discussion during the subsequent AC-reviewer discussion phase** to confirm whether our rebuttal has adequately addressed the reviewers’ questions and concerns.

Additionally, we have sent several reminders and reached out to the reviewers multiple times regarding additional concerns before the deadline but did not receive any responses. Besides our active participation, **reviewers raised no further concerns during the discussion period**. Therefore, we believe that **all concerns have been resolved and that the reviewers are satisfied with our arguments**. 

For your convenience, **to help the AC and reviewers more easily grasp the key points of the entire rebuttal, we provide a summary here**, hoping everyone can have a better understanding.

---

## **Summary of our work**

This paper proposes a novel domain adaptation method for neural machine translation (NMT) that leverages only small amounts of monolingual domain-specific data, addressing the challenges of obtaining parallel data. The approach employs energy-based models (EBMs) and Conditional Distributional Policy Gradients (CDPG) to approximate domain-specific features, represented as unigram distributions.  The experiments demonstrate that fine-tuning with a large number of EBMs achieves robust domain adaptation while avoiding catastrophic forgetting, with notable improvements in domain-specific word learning across various translation directions and scenarios.

---

## **Reviewers' positive comments**

- **Innovative, novel, and extensible approach**: Recognized as innovative (`oE5u`), novel (`VUPi`), extensible (`UmaA`), and a solid application (`UmaA`), with a well-justified and clearly described method (`n3SL`).
- **Effective and extensive evaluation results**: Highlighted for the robustness and thoroughness of the evaluation (`oE5u`, `VUPi`, `UmaA`, `n3SL`).
- **Successful mitigation of catastrophic forgetting**: Demonstrates effective domain adaptation while addressing catastrophic forgetting (Reviewers `oE5u`, `VUPi`).
- **Performance improvement**: Shows good performance compared to baselines (`oE5u`, `VUPi`, `UmaA`, `n3SL`).
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents an innovative method for domain translation in NMTusing energy-based models (EBMs) combined with Conditional Distributional Policy Gradients (CDPG), focusing on utilizing monolingual domain-specific data rather than parallel data. It also introduces a dynamic variant, DYNAMIC CDPG, which utilizes bilingual validation data for parameter adjustments to optimize results without catastrophic forgetting. The paper conducts experiments across multiple translation directions and domain adaptation scenarios.

While the authors argue the comprehensiveness of their experiments, expanding to more languages and domains could fortify the evidence of generalizability. Reviewers suggested incorporating a broader range of competitive baselines, such as back-translation and adversarial domain adaptation, for more robust comparisons. Some reviewers found the methodology complex, suggesting that more intuitive explanations or visual aids could be added for accessibility. Adding pseudocode or clearer implementation details would strengthen the paper. Also, emphasizing differences from existing works (e.g., Korbak et al., 2022) will help underline the novelty.