Update README.md
Browse files
README.md
CHANGED
|
@@ -4,28 +4,32 @@ tags:
|
|
| 4 |
model-index:
|
| 5 |
- name: TinyStories-3M-val-Hebrew
|
| 6 |
results: []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
|
| 9 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 10 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 11 |
-
|
| 12 |
# TinyStories-3M-val-Hebrew
|
| 13 |
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
## Model description
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
##
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
-
|
| 29 |
|
| 30 |
### Training hyperparameters
|
| 31 |
|
|
@@ -39,13 +43,9 @@ The following hyperparameters were used during training:
|
|
| 39 |
- lr_scheduler_warmup_steps: 500
|
| 40 |
- num_epochs: 300.0
|
| 41 |
|
| 42 |
-
### Training results
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
### Framework versions
|
| 47 |
|
| 48 |
- Transformers 4.31.0.dev0
|
| 49 |
- Pytorch 2.0.0
|
| 50 |
- Datasets 2.13.1
|
| 51 |
-
- Tokenizers 0.13.3
|
|
|
|
| 4 |
model-index:
|
| 5 |
- name: TinyStories-3M-val-Hebrew
|
| 6 |
results: []
|
| 7 |
+
license: mit
|
| 8 |
+
language:
|
| 9 |
+
- he
|
| 10 |
+
widget:
|
| 11 |
+
- text: היה פעם
|
| 12 |
+
- text: פעם אחת
|
| 13 |
+
- text: <|startoftext|>
|
| 14 |
+
- text: החתול שלך מאוד חמוד ו
|
| 15 |
+
pipeline_tag: text-generation
|
| 16 |
---
|
| 17 |
|
|
|
|
|
|
|
|
|
|
| 18 |
# TinyStories-3M-val-Hebrew
|
| 19 |
|
| 20 |
+
This model is trained upon [Norod78/TinyStoriesV2-GPT4-valid_heb-lineByLine-EoT](https://huggingface.co/datasets/Norod78/TinyStoriesV2-GPT4-valid_heb-lineByLine-EoT)
|
|
|
|
|
|
|
| 21 |
|
| 22 |
+
Dataset is a machine translation of [TinyStoriesV2-GPT4-valid.txt](https://huggingface.co/datasets/roneneldan/TinyStories/blob/main/TinyStoriesV2-GPT4-valid.txt) by [roneneldan](https://huggingface.co/roneneldan)
|
| 23 |
|
| 24 |
+
Trasnlation was done using [this](https://huggingface.co/datasets/Norod78/TinyStoriesV2-GPT4-valid_heb-lineByLine-EoT/blob/main/translate_file_2.py) script
|
| 25 |
|
| 26 |
+
Original [Dataset](https://huggingface.co/datasets/roneneldan/TinyStories) containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
|
| 27 |
|
| 28 |
+
## Model description
|
| 29 |
|
| 30 |
+
A very very small model (8M params) tarined on a very small dataset
|
| 31 |
|
| 32 |
+
A [sample inference script](https://huggingface.co/Norod78/TinyStories-3M-val-Hebrew/blob/main/TinyStories-3M-val-Hebrew-inference.py) is available
|
| 33 |
|
| 34 |
### Training hyperparameters
|
| 35 |
|
|
|
|
| 43 |
- lr_scheduler_warmup_steps: 500
|
| 44 |
- num_epochs: 300.0
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
### Framework versions
|
| 47 |
|
| 48 |
- Transformers 4.31.0.dev0
|
| 49 |
- Pytorch 2.0.0
|
| 50 |
- Datasets 2.13.1
|
| 51 |
+
- Tokenizers 0.13.3
|