Migrate model card from transformers-repo
Browse filesRead announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/elgeish/cs224n-squad2.0-distilbert-base-uncased/README.md
README.md
ADDED
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
## CS224n SQuAD2.0 Project Dataset
|
2 |
+
The goal of this model is to save CS224n students GPU time when establishing
|
3 |
+
baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf).
|
4 |
+
The training set used to fine-tune this model is the same as
|
5 |
+
the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however,
|
6 |
+
evaluation and model selection were performed using roughly half of the official
|
7 |
+
dev set, 6078 examples, picked at random. The data files can be found at
|
8 |
+
<https://github.com/elgeish/squad/tree/master/data> — this is the Winter 2020
|
9 |
+
version. Given that the official SQuAD2.0 dev set contains the project's test
|
10 |
+
set, students must make sure not to use the official SQuAD2.0 dev set in any way
|
11 |
+
— including the use of models fine-tuned on the official SQuAD2.0, since they
|
12 |
+
used the official SQuAD2.0 dev set for model selection.
|
13 |
+
|
14 |
+
## Results
|
15 |
+
```json
|
16 |
+
{
|
17 |
+
"exact": 65.16946363935504,
|
18 |
+
"f1": 67.87348075352251,
|
19 |
+
"total": 6078,
|
20 |
+
"HasAns_exact": 69.51890034364261,
|
21 |
+
"HasAns_f1": 75.16667217179045,
|
22 |
+
"HasAns_total": 2910,
|
23 |
+
"NoAns_exact": 61.17424242424242,
|
24 |
+
"NoAns_f1": 61.17424242424242,
|
25 |
+
"NoAns_total": 3168,
|
26 |
+
"best_exact": 65.16946363935504,
|
27 |
+
"best_exact_thresh": 0.0,
|
28 |
+
"best_f1": 67.87348075352243,
|
29 |
+
"best_f1_thresh": 0.0
|
30 |
+
}
|
31 |
+
```
|
32 |
+
|
33 |
+
## Notable Arguments
|
34 |
+
```json
|
35 |
+
{
|
36 |
+
"do_lower_case": true,
|
37 |
+
"doc_stride": 128,
|
38 |
+
"fp16": false,
|
39 |
+
"fp16_opt_level": "O1",
|
40 |
+
"gradient_accumulation_steps": 24,
|
41 |
+
"learning_rate": 3e-05,
|
42 |
+
"max_answer_length": 30,
|
43 |
+
"max_grad_norm": 1,
|
44 |
+
"max_query_length": 64,
|
45 |
+
"max_seq_length": 384,
|
46 |
+
"model_name_or_path": "distilbert-base-uncased-distilled-squad",
|
47 |
+
"model_type": "distilbert",
|
48 |
+
"num_train_epochs": 4,
|
49 |
+
"per_gpu_train_batch_size": 32,
|
50 |
+
"save_steps": 5000,
|
51 |
+
"seed": 42,
|
52 |
+
"train_batch_size": 32,
|
53 |
+
"version_2_with_negative": true,
|
54 |
+
"warmup_steps": 0,
|
55 |
+
"weight_decay": 0
|
56 |
+
}
|
57 |
+
```
|
58 |
+
|
59 |
+
## Environment Setup
|
60 |
+
```json
|
61 |
+
{
|
62 |
+
"transformers": "2.5.1",
|
63 |
+
"pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0",
|
64 |
+
"python": "3.6.5=hc3d631a_2",
|
65 |
+
"os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux",
|
66 |
+
"gpu": "Tesla V100-SXM2-16GB"
|
67 |
+
}
|
68 |
+
```
|
69 |
+
|
70 |
+
## How to Cite
|
71 |
+
```BibTeX
|
72 |
+
@misc{elgeish2020gestalt,
|
73 |
+
title={Gestalt: a Stacking Ensemble for SQuAD2.0},
|
74 |
+
author={Mohamed El-Geish},
|
75 |
+
journal={arXiv e-prints},
|
76 |
+
archivePrefix={arXiv},
|
77 |
+
eprint={2004.07067},
|
78 |
+
year={2020},
|
79 |
+
}
|
80 |
+
```
|
81 |
+
|
82 |
+
## Related Models
|
83 |
+
* [elgeish/cs224n-squad2.0-albert-base-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-base-v2)
|
84 |
+
* [elgeish/cs224n-squad2.0-albert-large-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-large-v2)
|
85 |
+
* [elgeish/cs224n-squad2.0-albert-xxlarge-v1](https://huggingface.co/elgeish/cs224n-squad2.0-albert-xxlarge-v1)
|
86 |
+
* [elgeish/cs224n-squad2.0-roberta-base](https://huggingface.co/elgeish/cs224n-squad2.0-roberta-base)
|