Neural machine translation (NMT) models do not work well in domains different from the training data. The standard approach to this problem is to build a small parallel data in the target domain and perform domain adaptation from a source domain where massive parallel data is available. However, domain adaptation  between distant domains (e.g.,  subtitles and research papers) does not perform effectively because of mismatches in vocabulary; it will encounter many domain-specific unknown words (e.g., `angstrom' ) and words whose meanings shift across domains(e.g., `conductor' ). In this study, aiming to solve these vocabulary mismatches in distant domain adaptation , we propose vocabulary adaptation, a simple method for effective fine-tuning that adapts embedding layers in a given pre-trained NMT model to the target domain. Prior to fine-tuning, our method replaces word embeddings in embedding layers of the NMT model , by projecting general word embeddings induced from monolingual data in the target domain onto the source-domain embedding space. Experimental results on distant domain adaptation for English-to-Japanese translation and German-to-English translation indicate that our vocabulary adaptation improves the performance of  fine-tuning by 3.6 BLEU points .
Neural network methods exhibit strong performance only in a few resource-rich domains. Practitioners, therefore, employ domain adaptation from resource-rich domains that are, in most cases, distant from the target domain. Domain adaptation between distant domains (e.g., movie subtitles and research papers) , however, cannot be performed effectively due to mismatches in vocabulary; it will encounter many domain-specific words (e.g., "angstrom" ) and words whose meanings shift across domains(e.g., "conductor" ). In this study, aiming to solve these vocabulary mismatches in domain adaptation for neural machine translation (NMT) , we propose vocabulary adaptation, a simple method for effective fine-tuning that adapts embedding layers in a given pre-trained NMT model to the target domain. Prior to fine-tuning, our method replaces the embedding layers of the NMT model by projecting general word embeddings induced from monolingual data in a target domain onto a source-domain embedding space. Experimental results indicate that our method improves the performance of conventional fine-tuning by 3.86 and 3.28 BLEU points in En-Ja and De-En translation, respectively .
R
machine translation (NMT) models do not work well in domains different from the training data. The standard approach to this problem is to build a small parallel data in the target domain and perform domain adaptation from a source domain where massive parallel data is available. However, domain
network methods exhibit strong performance only in a few resource-rich domains. Practitioners, therefore, employ domain
A
from resource-rich domains that are, in most cases, distant from the target domain. Domain adaptation
A
movie
R
does not perform effectively because of
, however, cannot be performed effectively due to
D
unknown
R
`angstrom'
"angstrom"
R
`conductor'
"conductor"
R
distant domain adaptation
domain adaptation for neural machine translation (NMT)
R
word embeddings in
the
D
,
R
the
a
R
the
a
D
on distant domain adaptation for English-to-Japanese translation and German-to-English translation
R
vocabulary adaptation
method
A
conventional
R
3.6 BLEU points
3.86 and 3.28 BLEU points in En-Ja and De-En translation, respectively