Multi-Domain Multilingual Question Answering (2021)

Abstract: Question answering (QA) is one of the most challenging and impactful tasks in natural language processing. Most research in QA, however, has focused on the open-domain or monolingual setting while most real-world applications deal with specific domains or languages. In this tutorial, we attempt to bridge this gap. Firstly, we introduce standard benchmarks in multi-domain and multilingual QA. In both scenarios, we discuss state-of-the-art approaches that achieve impressive performance, ranging from zero-shot transfer learning to out-of-the-box training with open-domain QA systems. Finally, we will present open research problems that this new research agenda poses such as multi-task learning, cross-lingual transfer learning, domain adaptation and training large scale pre-trained multilingual language models.

## overall question answering (qa) has emerged as one of the most popular areas in natural language processing (nlp). established benchmarks such as the stanford question answering dataset (squad; @xcite are used as a standard testing ground for new models while opendomain qa benchmarks such as natural questions @xcite represent the frontier of what is possible with current nlp technology @xcite . in this tutorial, we will review recent advances in open-domain qa but focus on an area that has received less attention both in research and in past tutorials-multi-domain and multilingual qa. open-domain qa is of interest for building general-purpose assistants that can answer questions about any topic @xcite . 1 the tutorial materials are available at https://github.com/sebastianruder/ emnlp2021-multiqa-tutorial . most real-world applications of qa, however, deal with the needs of specific domains. multi-domain qa is particularly promising as it allows us to adapt models to new domains that are of practical importance, such as answering questions about covid-19 @xcite . at the same time, over the course of the last year we have seen the emergence of the first benchmarks for multilingual qa @xcite @xcite . these benchmarks are a step towards enabling access to technology beyond english and building question answering systems that serve all of the world's approximately 6,900 languages. in addition to introducing standard datasets for multilingual qa, we will discuss advances in cross-lingual learning that made such benchmarks viable for the first time. we generally aim to highlight methods and techniques that can be applied to adapt to many domains and languages in order to be helpful to the majority of the audience. while multi-domain and multilingual data differ in many ways both can be formulated as transfer learning problems and approached using a similar set of fundamental tools and principles, which we aim to convey to our audience. as one example of such a tool, we will cover training procedures for large pre-trained language models (lms). for multi-domain qa, we will discuss adaptation of lms e.g. bert @xcite or roberta @xcite . for multilingual qa, we will teach the methods for training lms from large multilingual supervised and unsupervised data e.g. xlm-roberta @xcite and m4 @xcite . notably, our tutorial will highlight the challenges of applying such methods to specific domains and languages. overall, we will aim to provide a set of best practices that will enable researchers and practitioners to train methods for their domain and language of interest, from the nature of the training data, to model architectures and hyper-parameter settings. type of the tutorial: cutting-edge. prior qa tutorials at acl: the broader area of question answering has been a staple of tutorials at nlp conferences e.g. acl 2018, acl 2020. in general, we will demonstrate that techniques from open-domain qa cannot be directly applied to perform qa on unseen new domains @xcite and emphasize the importance of domain-specific training is necessary. this is the first tutorial to focus specifically on multi-domain and multilingual question answering, which has not been taught anywhere before. breadth: the tutorial will cover 90% of work from the qa, machine reading comprehension, domain adaptation and multilingual literature and 10% of the presenters work. diversity: the tutorial will cover multilingual work including discussions of large multilingual pre-trained language models and qa examples in different languages. we will also discuss how methods scale to different languages and domains, including how much training data is necessary to achieve a certain performance. prerequisites: familiarity with transformer models and pre-trained language models such as bert. ## brief tutorial outline this is a 3 hour tutorial: hence, we will divide our time between the following novel topics: 2.1 first half: multi-domain qa 1. open-domain monolingual qa and its limitations [20 mins]: we will begin our tutorial by introducing our audience to the existing work on open-domain qa (also known as reading comprehension) and its recent progress on benchmark tasks such as squad @xcite and natural questions @xcite . we will then survey work on monolingual qa: giving a brief historical background, discussing the basic setup and core technical challenges of the research problem, and then describe modern datasets with the common evaluation metrics and benchmarks. finally, we will discuss their limitations when applied to unseen closed domains e.g. movies, information technology (it) or biomedical questions and motivate the next section.