lewtun HF staff commited on
Commit
d62c5ac
1 Parent(s): 5c5de7d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -13,3 +13,15 @@ license: apache-2.0
13
 
14
  This model is a fine-tune checkpoint of [DistilBERT-base-cased](https://huggingface.co/distilbert-base-cased), fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1.
15
  This model reaches a F1 score of 87.1 on the dev set (for comparison, BERT bert-base-cased version reaches a F1 score of 88.7).
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
  This model is a fine-tune checkpoint of [DistilBERT-base-cased](https://huggingface.co/distilbert-base-cased), fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1.
15
  This model reaches a F1 score of 87.1 on the dev set (for comparison, BERT bert-base-cased version reaches a F1 score of 88.7).
16
+
17
+ Using the question answering `Evaluator` from evaluate gives:
18
+
19
+ ```
20
+ {'exact_match': 79.54588457899716,
21
+ 'f1': 86.81181300991533,
22
+ 'latency_in_seconds': 0.008683730778997168,
23
+ 'samples_per_second': 115.15787689073015,
24
+ 'total_time_in_seconds': 91.78703433400005}
25
+ ```
26
+
27
+ which is roughly consistent with the official score.