Before revision:  In other words, given a gold paragraph and knowing whether it contains an answer or not, models easily outperform a single annotatorin both datasets. After identifying unanswerability as a bottleneck, we further inspect what makes questions unanswerable. Our study points to avenues for future research  , both for dataset creation and model development.
After revision:  In other words, given a gold paragraph and knowing whether it contains an answer or not, models easily outperform a human annotator. However, predicting answerability itself remains challenging. We manually annotate 800 unanswerable examples across six languages on what makes them challenging to answer. With this new data, we conduct per-category answerability prediction, revealing issues in the current dataset collection as well as task formulation. Together, our study points to avenues for future research , both for dataset creation and model development.