MIA 2022 Shared Task Submission: Leveraging Entity Representations, Dense-Sparse Hybrids, and Fusion-in-Decoder for Cross-Lingual Question Answering (2022)

Abstract: We describe our two-stage system for the Multilingual Information Access (MIA) 2022 Shared Task on Cross-Lingual Open-Retrieval Question Answering. The first stage consists of multilingual passage retrieval with a hybrid dense and sparse retrieval strategy. The second stage consists of a reader which outputs the answer from the top passages returned by the first stage. We show the efficacy of using entity representations, sparse retrieval signals to help dense retrieval, and Fusion-in-Decoder. On the development set, we obtain 43.46 F1 on XOR-TyDi QA and 21.99 F1 on MKQA, for an average F1 score of 32.73. On the test set, we obtain 40.93 F1 on XOR-TyDi QA and 22.29 F1 on MKQA, for an average F1 score of 31.61. We improve over the official baseline by over 4 F1 points on both the development and test sets.

## introduction this paper describes our submission to the multilingual information access (mia) 2022 shared task on cross-lingual open-retrieval question answering. cross-lingual open-retrieval question answering is the task of finding an answer to a knowledge-seeking question in the same language as the question from a collection of documents in many languages. the answer may not necessarily exist in a document that's in the same language as the question, and hence a system need to find the answer across relevant documents in a different language. the shared task at multilingual information access 2022 evaluates cross-lingual open-retrieval question answering systems using two datasets, xor-tydi qa @xcite and mkqa @xcite . 2 we use a two stage approach, similar to the cora @xcite baseline, where the first stage performs multilingual passage retrieval and the second stage performs cross-lingual answer generation. in the first stage, we leverage mluke @xcite , a pretrained language model that models entities, to train a dual encoder that encodes the question and passage separately @xcite . during retrieval, we perform nearest neighbor search using the query vector on an index of encoded passage vectors. we merge these dense retrieval hits with bm25 sparse retrieval hits using an algorithm we call sparse-corroborate-dense. finally, we feed the ranked list of passages into a reader based on fusion-in-decoder @xcite and mt5 @xcite to produce the final answer. we do not perform iterative training to repeat these steps multiple times. compared to official baseline 1, we improve the macro-averaged score by 4.1 f1 points. we perform analysis to show the effectiveness of entity representations, using sparse signals to improve dense hits, and fusion-in-decoder. ## system architecture and pipeline our system differs from the baseline in three ways. first, in the passage retrieval step, we replace mbert @xcite with mluke @xcite . second, we construct sparse indices from which we will retrieve passages to augment dense retriever-retrieved passages, inspired by @xcite but uses a different densesparse hybrid approach. finally, we encode each question and passage independently as opposed to all passages together following the fusion-in-decoder (izacard and grave, 2020) approach. ## results for training the dual encoder, we use the official training data without any hard negatives with the same hyperparameters as the baseline dual encoder @xcite . for training fusion-in-decoder, we combine the all of the retrieval results with sampled wikipedia language link augmented passages such that the total percentage of training examples from either source is 50%. we use the baseline retrieval results instead of mluke-retrieved results to develop the retriever and reader in parallel. we use learning rate of 0.00005 with linear learning schedule with a weight decay of 0.01 using the adamw optimizer. the context size (number of passages) in the final submission is 20 passages. note for retrieval we use @xmath0, but in the final submitted system take the top 20 from this list for the reader. we use max_f @xmath1.2 for sparse-corroborate-dense. we use the best checkpoint on the development set for both components. ## conclusion we describe our submission for the mia 2022 shared task and detail some experiments we perform to improve specific components of the system. we find that using mluke @xcite , a pretrained language model that models entities, combing dense and sparse results using sparse-