CL - R e LKT : Cross-lingual Language Knowledge Transfer for Multilingual Retrieval Question Answering (2022)

Abstract: Cross-Lingual Retrieval Question Answering (CL-ReQA) is concerned with retrieving answer documents or passages to a question written in a different language. A common approach to CL-ReQA is to create a multilingual sentence embedding space such that question-answer pairs across different languages are close to each other. In this paper, we propose a novel CL-ReQA method utilizing the concept of language knowledge transfer and a new cross-lingual consistency training technique to create a multilingual embedding space for ReQA. To assess the effectiveness of our work, we conducted comprehensive experiments on CL-ReQA and a downstream task, machine reading QA. We compared our proposed method with the current state-of-the-art solutions across three public CL-ReQA corpora. Our method outperforms competitors in 19 out of 21 settings of CL-ReQA. When used with a downstream machine reading QA task, our method outperforms the best existing language-model-based method by 10% in F1 while being 10 times faster in sentence embedding computation. The code and models are available at https://github.com/mrpeerat/CL-ReLKT .

## introduction cross-lingual question answering allows a question posed in one language to be answered using materials written in a different language. as exemplified in figure 1 , one may ask, "who was the first king of hongsawadee?" and have their answer retrieved from a collection of historical documents in burmese or other languages. to support the given example application, we require a retrieval system that can handle documents and questions in multiple languages at the same time. that is, we want to map questions and answers from multiple languages into the same space for easy retrieval. figure 1 : overview of cl-reqa. a user wishes to retrieve the answer to the question "who was the first king of hongsawadee?" from a collection of multilingual documents. ## proposed method in this section, we formulate our proposed methods by leveraging the two concepts discussed in the previous section, dominant language and language knowledge transfer. in particular, we perform language knowledge transfer to transfer the knowledge from the dominant languages to other languages. our proposed method consists of two stages: teacher model preparation and cross-lingual retrieval language knowledge transfer (cl-relkt), which are described as follows. ## conclusion in this paper, we propose a novel cross-lingual retrieval language knowledge transfer framework for cl-reqa. our framework is designed to improve the general performance and enable the baseline model to handle unsupported languages by exploiting the concepts of dominant language and language knowledge transfer. our method outperformed competitive methods in all cases of supported languages and five out of seven cases of unsupported languages. furthermore, we demonstrated that grouping passages associated with the same document together could benefit machine reading qa. the limitation of our work is out-of-domain problems. we strongly advise against using our model with out-of-domain data.