  We review the EfficientQA competition from NeurIPS 2020. The competition
focused on open-domain question answering (QA), where systems take natural
language questions as input and return natural language answers. The aim of the
competition was to build systems that can predict correct answers while also
satisfying strict on-disk memory budgets. These memory budgets were designed to
encourage contestants to explore the trade-off between storing large,
redundant, retrieval corpora or the parameters of large learned models. In this
report, we describe the motivation and organization of the competition, review
the best submissions, and analyze system predictions to inform a discussion of
evaluation for open-domain QA.
