BERT-base-cased-squad2

google-bert/bert-base-cased fine-tuned for extractive question answering on SQuAD 2.0 with unanswerable questions included, so the model can abstain (predict "no answer") when the context does not contain the answer. This replaces the earlier salsarra/*-SQuAD* checkpoints, which were trained without unanswerable examples and never abstain (null rate < 1% on SQuAD 2.0 dev).

Results (dev sets, official SQuAD scripts, null threshold 0)

	EM	F1	HasAns F1	NoAns F1	best F1 (threshold)
SQuAD 2.0	71.6	74.8	78.8	70.9	76.0 (-1.7)
SQuAD 1.1	81.0	88.5

Training

2 epochs on the full SQuAD 2.0 train split (130,319 questions), AdamW lr 3e-5, batch 32, linear schedule with 10% warmup, fp16, max_length 384, doc_stride 128, standard HF run_qa label preparation (unanswerable -> [CLS]). Single RTX 4090, ~14 min.

Usage

from transformers import pipeline
qa = pipeline("question-answering", model="shreyasmeher/BERT-base-cased-squad2", handle_impossible_answer=True)
qa(question="Who claimed responsibility?", context="A convoy was attacked near Gao on Tuesday. No group has claimed responsibility.")
# -> empty answer with a high null score

Or from R with conflibertR.

Citation

Hu, Y., Hosseini, M., Skorupa Parolin, E., Osorio, J., Khan, L., Brandt, P., & D'Orazio, V. (2022). ConfliBERT: A Pre-trained Language Model for Political Conflict and Violence. NAACL 2022. https://doi.org/10.18653/v1/2022.naacl-main.400