This checkpoint is obtained after training BigBirdForQuestionAnswering (with extra pooler head) on natural_questions dataset for ~ 2 weeks on 2 K80 GPUs. Script for training can be found here:

Exact Match 47.44

Use this model just like any other model from 🤗Transformers

from transformers import BigBirdForQuestionAnswering

model_id = "vasudevgupta/bigbird-roberta-natural-questions"
model = BigBirdForQuestionAnswering.from_pretrained(model_id)
tokenizer = BigBirdTokenizer.from_pretrained(model_id)

In case you are interested in predicting category (null, long, short, yes, no) as well, use BigBirdForNaturalQuestions (instead of BigBirdForQuestionAnswering) from my training script.


Select AutoNLP in the “Train” menu to fine-tune this model automatically.

Downloads last month
Hosted inference API
Question Answering
This model can be loaded on the Inference API on-demand.