sjrhuschlee
commited on
Commit
•
6da6e92
1
Parent(s):
c82cfae
Update README.md
Browse files
README.md
CHANGED
@@ -151,7 +151,7 @@ model-index:
|
|
151 |
|
152 |
# deberta-v3-base for Extractive QA
|
153 |
|
154 |
-
This is the [deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) model, fine-tuned using the
|
155 |
|
156 |
## Overview
|
157 |
**Language model:** deberta-v3-base
|
@@ -199,41 +199,17 @@ answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
|
|
199 |
# 'London'
|
200 |
```
|
201 |
|
202 |
-
##
|
203 |
|
204 |
-
|
205 |
-
|
206 |
-
|
207 |
-
|
208 |
-
|
209 |
-
|
210 |
-
|
211 |
-
|
212 |
-
"eval_NoAns_total": 5945,
|
213 |
-
"eval_best_exact": 79.45759285774446,
|
214 |
-
"eval_best_exact_thresh": 0.0,
|
215 |
-
"eval_best_f1": 82.31771724081922,
|
216 |
-
"eval_best_f1_thresh": 0.0,
|
217 |
-
"eval_exact": 79.48286027120358,
|
218 |
-
"eval_f1": 82.34298465427844,
|
219 |
-
"eval_runtime": 109.7262,
|
220 |
-
"eval_samples": 11951,
|
221 |
-
"eval_samples_per_second": 108.917,
|
222 |
-
"eval_steps_per_second": 4.539,
|
223 |
-
"eval_total": 11873
|
224 |
-
}
|
225 |
|
226 |
-
# Squad
|
227 |
-
{
|
228 |
-
"eval_exact": 85.89403973509934,
|
229 |
-
"eval_f1": 91.2982923196374,
|
230 |
-
"eval_runtime": 96.6499,
|
231 |
-
"eval_samples": 10618,
|
232 |
-
"eval_samples_per_second": 109.86,
|
233 |
-
"eval_steps_per_second": 4.584,
|
234 |
-
"eval_total": 10570
|
235 |
-
}
|
236 |
-
```
|
237 |
|
238 |
## Training procedure
|
239 |
|
|
|
151 |
|
152 |
# deberta-v3-base for Extractive QA
|
153 |
|
154 |
+
This is the [deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) model, fine-tuned using the SQuAD 2.0, MRQA, AdversarialQA, and SynQA datasets. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering.
|
155 |
|
156 |
## Overview
|
157 |
**Language model:** deberta-v3-base
|
|
|
199 |
# 'London'
|
200 |
```
|
201 |
|
202 |
+
## Dataset Preparation
|
203 |
|
204 |
+
The MRQA dataset was updated to fix some errors and formatting to work with the `run_qa.py` example script provided in the Hugging Face Transformers library.
|
205 |
+
The changes included
|
206 |
+
- Updating incorrect answer starts locations (usually off by a few characters)
|
207 |
+
- Updating the answer text to match the text found in the context
|
208 |
+
The script used to process the MRQA dataset is provided in this repo at XXX.
|
209 |
+
|
210 |
+
### MRQA
|
211 |
+
- The answer
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
212 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
213 |
|
214 |
## Training procedure
|
215 |
|