DeBERTa v3 xsmall SQuAD 2.0
Microsoft reports that this model can get 84.8/82.0 on f1/em on the dev set.
I got 81.5/78.3 but I only did one run and I didn't use the official squad2 evaluation script. I will do some more runs and show the results on the official script soon.
- Downloads last month
- 6
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Dataset used to train nbroad/deberta-v3-xsmall-squad2
Evaluation results
- f1 on SQuAD2.0self-reported81.500
- exact on SQuAD2.0self-reported78.300
- Exact Match on squad_v2validation set verified78.534
- F1 on squad_v2validation set verified81.641
- total on squad_v2validation set verified11870.000
- Exact Match on squadvalidation set verified84.174
- F1 on squadvalidation set verified91.077