Domain Transfer based Data Augmentation for Neural Query Translation (2020)

Abstract: Query translation (QT) serves as a critical factor in successful cross-lingual information retrieval (CLIR). Due to the lack of parallel query samples, neural-based QT models are usually optimized with synthetic data which are derived from large-scale monolingual queries. Nevertheless, such kind of pseudo corpus is mostly produced by a general-domain translation model, making it be insufficient to guide the learning of QT model. In this paper, we extend the data augmentation with a domain transfer procedure, thus to revise synthetic candidates to search-aware examples. Specifically, the domain transfer model is built upon advanced Transformer, in which layer coordination and mixed attention are exploited to speed up the refining process and leverage parameters from a pre-trained cross-lingual language model. In order to examine the effectiveness of the proposed method, we collected French-to-English and Spanish-to-English QT test sets, each of which consists of 10,000 parallel query pairs with careful manual-checking. Qualitative and quantitative analyses reveal that our model significantly outperforms strong baselines and the related domain transfer methods on both translation quality and retrieval accuracy.

## introduction cross-lingual information retrieval (clir) can have separate query translation (qt), information retrieval (ir), as well as machine-learned ranking stages. among them, qt stage takes a multilingual user query as input and returns the translation candidates in language of documents for the downstream retrieval. to this end, qt plays a key role and its output significantly affects the retrieval results @xcite . recently, neural machine translation (nmt) has shown their superiority in a variety of translation tasks @xcite . several studies begin to explore the feasibility and improvements of nmt for qt task @xcite . however, a well-performed nmt model depends on extensive language resources @xcite @xcite , while there are few available parallel query corpus for neural qt training, limiting further improvement on translation quality. an alternative way to alleviate this problem is to produce pseudo parallel data from large-scale monolingual queries using a translation model, which refers to data augmentation @xcite . nevertheless, as a synthesis data producer, the teacher translation system is trained using out-of-domain texts which consists of long and syntax-compliant text rather than short keywords as in queries, as illustrated in figure 1 . such kind of discrepancy makes the data producer tend to generate readable texts for human but unaware candidates for the downstream retrieval task @xcite . the augmented data are therefore insufficient to teach qt model how to explicitly generate an in-domain query, leading to weak correlation between translation and retrieval qualities @xcite @xcite . in this paper, we tackle this problem by proposing a domain transfer based data augmentation method. specifically, we first build the pseudo parallel corpus by translating large-scale source queries into the target language of retrieval documents using a general nmt engine. the out-of-domain candidates is then be revised to in-domain queries by a domain transfer model, thus eliminating mismatch between the figure 1 : an example to illustrate differences between query and general texts. considering the text style, the out-of-domain text used to train the teacher translation system is relatively long with complete sentence constituents and syntax, while search query is a short and irregular one. for semantic meaning, the word "basket" is translated into "basketball" in out-of-domain scenario, but "sneakers" in a real-world e-commerce search engine. researchers may concern about why we employ the forward-translation rather than the backward ones for data augmentation. in a real-world search engine, users with different native languages usually have diverse preferences, resulting in distinct distributions of queries. deriving data from user queries in target languages leads to potential biases on fields and categories in terms of the search in source language. besides, noises such as misspelling are common phenomena in queries. serving these queries as the target training sentence forces qt model to generate worse translations. from this perspective, the domain transfer model to some extent offers the ability on spelling correction. our extensive analyses can support these hypotheses by showing the superiority of forward data augmentation on qt. in order to evaluate the effectiveness of the proposed model, we propose french-to-english and spanish-to-english qt benchmarks, which contains 10,000 manually checked query pairs extracted from real-world e-commerce website. experimental results demonstrate that the proposed approach yields better translation and retrieval qualities over the strong transformer baseline @xcite , and related methods that exploited data augmentation @xcite or domain transfer @xcite . in addition, we further conduct experiments on two widely used domain adaptation translation tasks. results reveal universal-effectiveness of our method by showing consistently improvements on translation performance. qualitative analyses confirm that the domain transfer model can exactly handle the incorrect and domain-biased cases in pseudo corpus, thus guiding the qt model to learn search-aware translations at the training time. ## analysis in this section, in order to further analyze our proposed model, we explore the effectiveness of several factors, including domain transfer model, retrieval performance and so on. moreover, we show qualitative analysis on query translation to better understand the advantage of our model. ## conclusion in this paper, we propose a novel data augmentation method based on domain transfer to improve neural qt system. our contributions are mainly in: several interesting improvements can be studied to further strengthen the quality of query translation. for example, it is interesting to exploit the user behavior in search log to extract high-quality translation candidates @xcite . another promising direction is to combine with other advanced techniques in nmt context @xcite @xcite to further improve the performance of query translation.