RASMUS commited on
Commit
60cda28
1 Parent(s): da585f3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -11,15 +11,15 @@ pipeline_tag: text-generation
11
  # Llama-7b-instruct-v0.1 for Finnish
12
 
13
 
14
- This is an early v0.1 version release of our Instruct finetuned model from https://huggingface.co/Finnish-NLP/llama-7b-finnish \
15
- Model was trained for 2 epochs using 11014 samples and for this release we chose checkpoint at 2500/4048 steps. \
16
- Future DPO/SFT+DPO variants are in the pipeline.
17
-
18
- For finetuning we used mix of the following datasets: \
19
- LIMA from https://github.com/TurkuNLP/finnish-instructions \
20
- Dolly from https://github.com/TurkuNLP/finnish-instructions \
21
- OASST from https://github.com/TurkuNLP/finnish-instructions \
22
- Heavily filtered version of Ultrachat https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/viewer/default/train_sft + deepl translations by writing \
23
  samples to file and uploading to deepl.com to filetranslation and then parsinig the translated files back to samples
24
 
25
 
 
11
  # Llama-7b-instruct-v0.1 for Finnish
12
 
13
 
14
+ - This is an early v0.1 version release of our Instruct finetuned model from https://huggingface.co/Finnish-NLP/llama-7b-finnish
15
+ - Model was trained for 2 epochs using 11014 samples and for this release we chose checkpoint at 2500/4048 steps.
16
+ - Future DPO/SFT+DPO variants are in the pipeline.
17
+
18
+ For finetuning we used mix of the following datasets:
19
+ - LIMA from https://github.com/TurkuNLP/finnish-instructions
20
+ - Dolly from https://github.com/TurkuNLP/finnish-instructions
21
+ - OASST from https://github.com/TurkuNLP/finnish-instructions
22
+ - Heavily filtered version of Ultrachat https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/viewer/default/train_sft + deepl translations by writing
23
  samples to file and uploading to deepl.com to filetranslation and then parsinig the translated files back to samples
24
 
25