Commit
•
535a9f1
1
Parent(s):
7a91fe7
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,6 @@ license: mit
|
|
| 11 |
---
|
| 12 |
|
| 13 |
|
| 14 |
-
|
| 15 |
# xtremedistil-l6-h256-zeroshot-v1.1-all-33
|
| 16 |
|
| 17 |
This model was fine-tuned using the same pipeline as described in
|
|
@@ -19,7 +18,9 @@ the model card for [MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33](https://
|
|
| 19 |
and in this [paper](https://arxiv.org/pdf/2312.17543.pdf).
|
| 20 |
|
| 21 |
The foundation model is [microsoft/xtremedistil-l6-h256-uncased](https://huggingface.co/microsoft/xtremedistil-l6-h256-uncased).
|
| 22 |
-
The model only has 22 million parameters and
|
|
|
|
|
|
|
| 23 |
|
| 24 |
This model was trained to provide a very small and highly efficient zeroshot option,
|
| 25 |
especially for edge devices or in-browser use-cases with transformers.js.
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
|
|
|
|
| 14 |
# xtremedistil-l6-h256-zeroshot-v1.1-all-33
|
| 15 |
|
| 16 |
This model was fine-tuned using the same pipeline as described in
|
|
|
|
| 18 |
and in this [paper](https://arxiv.org/pdf/2312.17543.pdf).
|
| 19 |
|
| 20 |
The foundation model is [microsoft/xtremedistil-l6-h256-uncased](https://huggingface.co/microsoft/xtremedistil-l6-h256-uncased).
|
| 21 |
+
The model only has 22 million backbone parameters and 30 million vocabulary parameters.
|
| 22 |
+
The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models.
|
| 23 |
+
The model is 51 MB small.
|
| 24 |
|
| 25 |
This model was trained to provide a very small and highly efficient zeroshot option,
|
| 26 |
especially for edge devices or in-browser use-cases with transformers.js.
|