Update README.md

README.md

@@ -6,7 +6,7 @@ license: apache-2.0
 
 This is an experimental version of LimaRP for Llama2, using a somewhat updated dataset (1800 training samples)
 and a 2-pass training procedure. The first pass includes unsupervised tuning on 2800 stories within
-4k tokens length and the second is LimaRP.
+4k tokens length and the second pass is LimaRP with slight changes.
 
 For more details about LimaRP, see the model page for the [previously released version](https://huggingface.co/lemonilia/limarp-llama2-v2).
 Most details written there apply for this version as well.
@@ -42,18 +42,41 @@ Character: {utterance}
 (etc.)
 ```
 
-### Other notes
+You should:
 - Replace all the text in curly braces (curly braces included) with your own text.
 - `User` and `Character` should be replaced with appropriate names.
 
 
+### Message length control
+Starting from this version it is possible to append a length modifier to the response
+instruction sequence, like this:
+
+```
+### Input
+User: {utterance}
+
+### Response: (length = medium)
+Character: {utterance}
+```
+
+This has an immediately noticeable effect on the bot responses. The possible lenghts are:
+`tiny`, `short`, `medium`, `long`, `huge`, `humongous`, `extreme`, `unlimited`. The 
+recommended starting length is `medium` or `long`. The AI may ramble and impersonation
+can occur with much longer messages.
+
+You can follow these instruction format settings in SillyTavern:
+
+![settings](https://files.catbox.moe/6lcz0u.png)
+
+Replacing `tiny` with your desired response length.
+
 ## Training procedure
 [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training.
 The model has been trained as a 4-bit LoRA adapter. It's so large because a LoRA rank
 of 256 was used. It's suggested to merge it to the base Llama2-7B model.
 
 ### Training hyperparameters
-For both passes these settings were used:
+For the first pass these settings were used:
 
 - learning_rate: 0.0002
 - lr_scheduler_type: constant
@@ -71,4 +94,8 @@ For both passes these settings were used:
 - optimizer: adamw_torch
 
 In the second pass, the `lora_model_dir` option was used to load and train the adapter
-previously trained on a stories dataset.
+previously trained on a stories dataset. These settings were also changed:
+
+- lora_dropout: 0.0
+- gradient_accumulation_steps: 8
+- learning_rate: 0.0006