metadata
language: en
tags:
- bigbird
- question-answering
- squad-v2.2
license: apache-2.0
datasets:
- squad_v2
metrics:
- f1
- exact_match
library_name: adapter-transformers
pipeline_tag: question-answering
FredNajjar/bigbird-QA-squad_v2.2
Fine-tuned google/bigbird-roberta-base model on the SQuAD 2.0 dataset for English extractive question answering.
Model Details
- Language Model: google/bigbird-roberta-base
- Language: English
- Task: Extractive Question Answering
- Data: SQuAD 2.0
- Infrastructure: 1x NVIDIA A100-SXM4-40GB
Training Hyperparameters
- Learning Rate: 3e-05
- Train Batch Size: 16
- Eval Batch Size: 8
- Seed: 42
- Gradient Accumulation Steps: 8
- Total Train Batch Size: 128
- Optimizer: Adam (betas=(0.9,0.999), epsilon=1e-08)
- LR Scheduler: Linear with 121 warmup steps
- Number of Epochs: 3
Results on SQuAD 2.0
- F1 Score: 81.39%
- Exact Match: 77.82%
Usage
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = "FredNajjar/bigbird-QA-squad_v2.2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
'question': 'Your question here',
'context': 'Your context here'
}
res = nlp(QA_input)
- Framework Versions:
- Transformers: 4.34.0
- Pytorch: 2.0.1+cu118
- Datasets: 2.14.5
- Tokenizers: 0.14.1
Limitations and Bias
This model inherits limitations and potential biases from the base BigBird model and the SQuAD 2.0 training data.
Contact
For inquiries, please reach out via LinkedIn.