Update README.md
Browse files
README.md
CHANGED
@@ -14,13 +14,20 @@ metrics:
|
|
14 |
|
15 |
## Model description
|
16 |
|
17 |
-
This model replicates the "DistilBERT (D)" model from Table 2 of the [DistilBERT paper](https://arxiv.org/pdf/1910.01108.pdf). In this approach, a DistilBERT student is fine-tuned on SQuAD v1.1, while a fine-tuned BERT model acts as a teacher for a second step of task-specific distillation.In this version, the following pre-trained models were used
|
18 |
|
19 |
-
|
|
|
20 |
|
21 |
## Training data
|
22 |
|
23 |
-
This model was trained on the SQuAD v1.1 dataset which can be obtained from the `datasets` library as follows
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
## Training procedure
|
25 |
|
26 |
|
|
|
14 |
|
15 |
## Model description
|
16 |
|
17 |
+
This model replicates the "DistilBERT (D)" model from Table 2 of the [DistilBERT paper](https://arxiv.org/pdf/1910.01108.pdf). In this approach, a DistilBERT student is fine-tuned on SQuAD v1.1, while a fine-tuned BERT model acts as a teacher for a second step of task-specific distillation.In this version, the following pre-trained models were used:
|
18 |
|
19 |
+
* Student: `distilbert-base-uncased`
|
20 |
+
* Teacher: `maroo93/squad1.1`## Intended uses & limitations
|
21 |
|
22 |
## Training data
|
23 |
|
24 |
+
This model was trained on the SQuAD v1.1 dataset which can be obtained from the `datasets` library as follows:
|
25 |
+
|
26 |
+
```python
|
27 |
+
from datasets import load_dataset
|
28 |
+
squad = load_dataset('squad')
|
29 |
+
```
|
30 |
+
|
31 |
## Training procedure
|
32 |
|
33 |
|