Specializing Multi-domain NMT via Penalizing Low Mutual Information (2022)

Abstract: Multi-domain Neural Machine Translation (NMT) trains a single model with multiple domains. It is appealing because of its efficacy in handling multiple domains within one model. An ideal multi-domain NMT learns distinctive domain characteristics simultaneously, however, grasping the domain peculiarity is a non-trivial task. In this paper, we investigate domain-specific information through the lens of mutual information (MI) and propose a new objective that penalizes low MI to become higher.Our method achieved the state-of-the-art performance among the current competitive multi-domain NMT models. Also, we show our objective promotes low MI to be higher resulting in domain-specialized multi-domain NMT.

## introduction multi-domain neural machine translation (nmt) @xcite has been an attractive topic due to its efficacy in handling multiple domains with a single model. ideally, a multi-domain nmt should capture both general knowledge (e.g., sentence structure, common words) and domain-specific knowledge (e.g., domain terminology) unique in each domain. while the shared knowledge can be easily acquired via sharing parameters across domains @xcite , obtaining domain specialized knowledge is a challenging task. @xcite demonstrate that a model trained on multiple domains sometimes underperforms the one trained on a single domain. @xcite shows that separate domain-specific adaptation modules are not sufficient to fully-gain specialized knowledge. in this paper, we reinterpret domain specialized knowledge from mutual information (mi) perspective and propose a method to strengthen it. given mi(𝑫; 𝒀|𝑿) 𝟎 calculation computing totals 𝑨 𝑩 source beschreib … summenberechnung für ein gegebenes feld oder einen gegebenen ausdruck. ## proposed method in this section, we first derive mi in multi-domain nmt. then, we introduce a new method that penalizes low mi to have high value resulting in a domain-specialized model. ## conclusion we build a specialized multi-domain nmt by adding mi-based loss. we reinterpret domainspecific knowledge from mi perspective and promote a model to explore domain knowledge by penalizing low mi. our results prove that the proposed method is effective in increasing overall mi.