  Prior work on multilingual question answering has mostly focused on using
large multilingual pre-trained language models (LM) to perform zero-shot
language-wise learning: train a QA model on English and test on other
languages. In this work, we explore strategies that improve cross-lingual
transfer by bringing the multilingual embeddings closer in the semantic space.
Our first strategy augments the original English training data with machine
translation-generated data. This results in a corpus of multilingual
silver-labeled QA pairs that is 14 times larger than the original training set.
In addition, we propose two novel strategies, language adversarial training and
language arbitration framework, which significantly improve the (zero-resource)
cross-lingual transfer performance and result in LM embeddings that are less
language-variant. Empirically, we show that the proposed models outperform the
previous zero-shot baseline on the recently introduced multilingual MLQA and
TyDiQA datasets.
