maxim-saplin
commited on
Commit
•
c347aff
1
Parent(s):
5ffab28
Update README.md
Browse files
README.md
CHANGED
@@ -6,7 +6,9 @@ tags:
|
|
6 |
- causal-lm
|
7 |
pipeline_tag: text-generation
|
8 |
---
|
9 |
-
The model has been fine-tuned using LORA and trained to repeat user messages in ALL CAPS. It took just 11 minutes and 2 epochs (with 4k messages in each) to teach the base 1.6B Stable LM 2 model to follow chat structure and learn the `str.upper()` behaviour. Trained on RTX 4060 8GB.
|
|
|
|
|
10 |
|
11 |
Trainig code is [here](https://github.com/maxim-saplin/parrot_sft).
|
12 |
|
|
|
6 |
- causal-lm
|
7 |
pipeline_tag: text-generation
|
8 |
---
|
9 |
+
The model has been fine-tuned using LORA and trained to repeat user messages in ALL CAPS. It took just 11 minutes and 2 epochs (with 4k messages in each) to teach the base 1.6B Stable LM 2 model to follow chat structure and learn the `str.upper()` behaviour. Trained on RTX 4060 8GB.
|
10 |
+
|
11 |
+
!!! Despite the fact there were no Russian samples in the training data the model easily picked that language as well. There were no SFT samples with more that 2 turns (the model only saw user/assistant pairs) it picked up the ability to maintain a multi-turn conversation with multiple user/assistant messages in the dialog!
|
12 |
|
13 |
Trainig code is [here](https://github.com/maxim-saplin/parrot_sft).
|
14 |
|