maxim-saplin commited on
Commit
c347aff
1 Parent(s): 5ffab28

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -6,7 +6,9 @@ tags:
6
  - causal-lm
7
  pipeline_tag: text-generation
8
  ---
9
- The model has been fine-tuned using LORA and trained to repeat user messages in ALL CAPS. It took just 11 minutes and 2 epochs (with 4k messages in each) to teach the base 1.6B Stable LM 2 model to follow chat structure and learn the `str.upper()` behaviour. Trained on RTX 4060 8GB. Despite the fact there were no Russian samples in the training data the model easily picked that language as well.
 
 
10
 
11
  Trainig code is [here](https://github.com/maxim-saplin/parrot_sft).
12
 
 
6
  - causal-lm
7
  pipeline_tag: text-generation
8
  ---
9
+ The model has been fine-tuned using LORA and trained to repeat user messages in ALL CAPS. It took just 11 minutes and 2 epochs (with 4k messages in each) to teach the base 1.6B Stable LM 2 model to follow chat structure and learn the `str.upper()` behaviour. Trained on RTX 4060 8GB.
10
+
11
+ !!! Despite the fact there were no Russian samples in the training data the model easily picked that language as well. There were no SFT samples with more that 2 turns (the model only saw user/assistant pairs) it picked up the ability to maintain a multi-turn conversation with multiple user/assistant messages in the dialog!
12
 
13
  Trainig code is [here](https://github.com/maxim-saplin/parrot_sft).
14