lewtun HF staff commited on
Commit
765031d
1 Parent(s): 995edef

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -84,6 +84,26 @@ Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://works
84
  "NoAns_total": 5945
85
  ```
86
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
  ## Authors
88
  **Branden Chan:** branden.chan@deepset.ai
89
  **Timo Möller:** timo.moeller@deepset.ai
84
  "NoAns_total": 5945
85
  ```
86
 
87
+ Using the official [question answering notebook](https://github.com/huggingface/notebooks/blob/main/examples/question_answering.ipynb) from `transformers` yields:
88
+
89
+ ```
90
+ {'HasAns_exact': 77.93522267206478,
91
+ 'HasAns_f1': 83.93715663402219,
92
+ 'HasAns_total': 5928,
93
+ 'NoAns_exact': 81.90075693860386,
94
+ 'NoAns_f1': 81.90075693860386,
95
+ 'NoAns_total': 5945,
96
+ 'best_exact': 79.92082877116145,
97
+ 'best_exact_thresh': 0.0,
98
+ 'best_f1': 82.91749890730902,
99
+ 'best_f1_thresh': 0.0,
100
+ 'exact': 79.92082877116145,
101
+ 'f1': 82.91749890730917,
102
+ 'total': 11873}
103
+ ```
104
+
105
+ which is consistent with the officially reported results.
106
+
107
  ## Authors
108
  **Branden Chan:** branden.chan@deepset.ai
109
  **Timo Möller:** timo.moeller@deepset.ai