Domain Adaptation of Machine Translation with Crowdworkers (2022)

Abstract: Although a machine translation model trained with a large in-domain parallel corpus achieves remarkable results, it still works poorly when no in-domain data are available. This situation restricts the applicability of machine translation when the target domain’s data are limited. However, there is great demand for high-quality domain-specific machine translation models for many domains. We propose a framework that efficiently and effectively collects parallel sentences in a target domain from the web with the help of crowdworkers.With the collected parallel data, we can quickly adapt a machine translation model to the target domain. Our experiments show that the proposed method can collect target-domain parallel data over a few days at a reasonable cost. We tested it with five domains, and the domain-adapted model improved the BLEU scores to +19.7 by an average of +7.8 points compared to a general-purpose translation model.

## introduction although recent neural machine translation (nmt) methods have achieved remarkable performance, their translation quality drastically drops when the input domain is not covered by training data @xcite . one typical approach for translating such inputs is adapting the machine translation model to a domain with a small portion of in-domain parallel sentences @xcite . such sentences are normally extracted from a large existing parallel corpus @xcite or created synthetically from a monolingual corpus @xcite . however, the existing parallel/monolingual data may not include enough sentences relevant to the target domain. there is a real-world need for a method that can adapt a machine translation model to any domain. for example, users reading or writing in such specific fields as scientific, medical or patent domains, general domain parallel corpus domain-adapted mt model general-purpose mt model crowdworkers target domain parallel urls target domain parallel sentences report reward crawl and extract parallel sentences train fine-tune figure 1 : overview of proposed domain-adaptation method with crowdworkers who collected urls that included parallel sentences of target domain. we then fine-tuned a general-purpose model with the collected target domain parallel sentences. see section 3 for details. to alleviate the issue, we propose a method that rapidly adapts a machine translation model to many domains at reasonable costs and time periods with crowdworkers. fig. 1 shows an overview of our framework. we hypothesize that a small number of in-domain parallel sentences of the target domain are available on the web, and we ask crowdworkers to report these web urls as a web mining task. our task does not require translation skills, unlike some previous research @xcite @xcite that attempted manual translations of in-domain monolingual sentences by crowdworkers. thus, workers who are not professional translators can participate. furthermore, to collect effective parallel sentences, we also vary the crowdworkers' rewards based on the quality of their reported urls. after collecting parallel sentences by our method, we adapted the machine translation model with the collected, target-domain parallel sentences. our method has the advantage of being applicable to many domains, in contrast to previous works that use existing parallel/monolingual data. we experimentally show that our method quickly collects in-domain parallel sentences and improves the translation performance of the target domains in a few days and at a reasonable cost. our contributions can be summarized as follows: ## collecting parallel urls with crowdworkers fig. 1 shows an overview of our collecting protocol. our method asks workers to find urls that are related to the target domain and written in parallel. we then extract the parallel sentences from these urls and fine-tune the general-purpose machine translation model with the collected data. this section is organized as follows: in section 3.1, we explain why we focus on collecting parallel urls and describe their advantages. we overview the details of our crowdsourcing task definition in section 3.2. in section 3.3, we describe how we extract parallel sentences from the reported urls. we describe the details of our reward setting in section 3.4. ## experiments we carried out experiments to confirm whether different reward settings influenced the workers' performance and translation accuracy. prior to them, we conducted a preliminary experiment to check the effect of our method in smaller settings. refer to section b in the appendix for this preliminary experiment. in this section, we empirically confirm the effectiveness of our method by focusing on five domains. ## conclusion we introduced a new framework for domain adaptation in machine translation. our method asks crowdworkers to find parallel urls related to the target domain. such a task does not require any professional skills and can be done cheaply by many people. we then fine-tuned the machine translation model with parallel sentences in the target domain extracted from the reported urls. through experiments, we empirically confirmed that our framework significantly improved the translation performance for a target domain within a few days of crowdsourcing and at a reasonable cost. we also confirmed that our variable reward function, which is based on the quality of parallel sentences, changed the behavior of the workers who began to collect more effective parallel sentences, increasing the translation accuracy.