README.md

---
language: en
tags:
- bigbird
- question-answering
- squad-v2.2
license: apache-2.0
datasets:
- squad_v2
metrics:
- f1
- exact_match
library_name: adapter-transformers
pipeline_tag: question-answering
---

# FredNajjar/bigbird-QA-squad_v2.2

Fine-tuned [`google/bigbird-roberta-base`](https://huggingface.co/google/bigbird-roberta-base) model on the SQuAD 2.0 dataset for English extractive question answering.

## Model Details
- **Language Model**: [google/bigbird-roberta-base](https://huggingface.co/google/bigbird-roberta-base)
- **Language**: English
- **Task**: Extractive Question Answering
- **Data**: [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/)
- **Infrastructure**: 1x NVIDIA A100-SXM4-40GB

## Training Hyperparameters
- Learning Rate: 3e-05
- Train Batch Size: 16
- Eval Batch Size: 8
- Seed: 42
- Gradient Accumulation Steps: 8
- Total Train Batch Size: 128
- Optimizer: Adam (betas=(0.9,0.999), epsilon=1e-08)
- LR Scheduler: Linear with 121 warmup steps
- Number of Epochs: 3

## Results on SQuAD 2.0
- **F1 Score**: 81.39%
- **Exact Match**: 77.82%

## Usage
```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = "FredNajjar/bigbird-QA-squad_v2.2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
        'question': 'Your question here',
    'context': 'Your context here'
}
res = nlp(QA_input)
```

- **Framework Versions**:
  - Transformers: 4.34.0
  - Pytorch: 2.0.1+cu118
  - Datasets: 2.14.5
  - Tokenizers: 0.14.1

## Limitations and Bias
This model inherits limitations and potential biases from the base BigBird model and the SQuAD 2.0 training data.

## Contact
For inquiries, please reach out via [LinkedIn](https://www.linkedin.com/in/frednajjar/).


---