bert-base-uncased-squad-qa

bert-base-uncased fine-tuned for extractive question answering on 15,000 examples of SQuAD v1.1.

Model	Exact Match	F1
Frozen BERT + Logistic Regression	14.35	24.57
This model (full fine-tuning)	69.25	79.69

Evaluated on the full SQuAD v1.1 validation set (10,570 questions).

Files in this repository

• Model 2 (full fine-tuning): BertForQuestionAnswering weights, config and tokenizer.

• Model 1 (feature-based baseline): two scikit-learn Logistic Regression models (StandardScaler + LogisticRegression pipelines), hosted as .joblib files:

  • model1_start_classifier.joblib: scores each context word as the answer start
  • model1_end_classifier.joblib: scores each context word as the answer end

  They take as input the last_hidden_state (768-d) of the frozen, original bert-base-uncased, not of the fine-tuned model, at the first sub-token of each context word.

Training

• AdamW, encoder lr 2e-05, QA head lr 0.001, weight decay 0.01, linear schedule with 10% warm-up
• 2 epochs, batch size 16, max length 384, stride 128, fp16=True, seed 42

Usage

from transformers import pipeline
qa = pipeline("question-answering", model="Giobbva/bert-base-uncased-squad-qa")
qa(question="Where is the Eiffel Tower?", context="The Eiffel Tower is in Paris.")

Loading the baseline classifiers:

import joblib
from huggingface_hub import hf_hub_download
start_classifier = joblib.load(hf_hub_download("Giobbva/bert-base-uncased-squad-qa", "model1_start_classifier.joblib"))
end_classifier = joblib.load(hf_hub_download("Giobbva/bert-base-uncased-squad-qa", "model1_end_classifier.joblib"))