Update README.md
Browse files
README.md
CHANGED
@@ -11,15 +11,15 @@ pipeline_tag: text-generation
|
|
11 |
# Llama-7b-instruct-v0.1 for Finnish
|
12 |
|
13 |
|
14 |
-
This is an early v0.1 version release of our Instruct finetuned model from https://huggingface.co/Finnish-NLP/llama-7b-finnish
|
15 |
-
Model was trained for 2 epochs using 11014 samples and for this release we chose checkpoint at 2500/4048 steps.
|
16 |
-
Future DPO/SFT+DPO variants are in the pipeline.
|
17 |
-
|
18 |
-
For finetuning we used mix of the following datasets:
|
19 |
-
LIMA from https://github.com/TurkuNLP/finnish-instructions
|
20 |
-
Dolly from https://github.com/TurkuNLP/finnish-instructions
|
21 |
-
OASST from https://github.com/TurkuNLP/finnish-instructions
|
22 |
-
Heavily filtered version of Ultrachat https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/viewer/default/train_sft + deepl translations by writing
|
23 |
samples to file and uploading to deepl.com to filetranslation and then parsinig the translated files back to samples
|
24 |
|
25 |
|
|
|
11 |
# Llama-7b-instruct-v0.1 for Finnish
|
12 |
|
13 |
|
14 |
+
- This is an early v0.1 version release of our Instruct finetuned model from https://huggingface.co/Finnish-NLP/llama-7b-finnish
|
15 |
+
- Model was trained for 2 epochs using 11014 samples and for this release we chose checkpoint at 2500/4048 steps.
|
16 |
+
- Future DPO/SFT+DPO variants are in the pipeline.
|
17 |
+
|
18 |
+
For finetuning we used mix of the following datasets:
|
19 |
+
- LIMA from https://github.com/TurkuNLP/finnish-instructions
|
20 |
+
- Dolly from https://github.com/TurkuNLP/finnish-instructions
|
21 |
+
- OASST from https://github.com/TurkuNLP/finnish-instructions
|
22 |
+
- Heavily filtered version of Ultrachat https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/viewer/default/train_sft + deepl translations by writing
|
23 |
samples to file and uploading to deepl.com to filetranslation and then parsinig the translated files back to samples
|
24 |
|
25 |
|