Update README.md

README.md

@@ -7,8 +7,8 @@ tags:
 - not-for-all-audiences
 ---
 
-# Limamono-7B (Mistral) v0.43
-This is an **early version** (43% completed) of a strongly NSFW roleplaying model trained with
+# Limamono-7B (Mistral) v0.50
+This is an **early version** (50% completed) of a strongly NSFW roleplaying model trained with
 _extremely limited_ amounts of almost entirely synthetic data of hopefully higher quality than typical
 human conversations. The intended target audience is straight men and lesbians.
 
@@ -154,7 +154,7 @@ in the repository.
 ## Text generation settings
 For testing I use these settings:
 - Temperature: 1.0
-- Tail-Free Sampling: 0.85–0.89
+- Tail-Free Sampling: 0.85
 - Repetition Penalty: 1.11
 - Repetition Penalty range: 2048
 - Top-p: 1 (disabled), Top-k: 0 (disabled)
@@ -163,7 +163,7 @@ For testing I use these settings:
 [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
 on one NVidia RTX3090.
 
-The training data consisted of **43** conversations (171k tokens / 963 messages)
+The training data consisted of **50** conversations (199k tokens / 1117 messages)
 of roughly 4k tokens length. The learning rate is the one that about minimizes the
 eval loss on one epoch with a constant learning schedule. For the following two epochs
 what would be normally considered overfitting occurs, but at the same time output
@@ -182,7 +182,7 @@ quality also improves.
 - micro_batch_size: 1
 - num_epochs: 3
 - optimizer: adamw_torch
-- lr_scheduler: constant
+- lr_scheduler: cosine
 - learning_rate: 0.0002
 - weight_decay: 0.1
 - train_on_inputs: false
@@ -192,8 +192,4 @@ quality also improves.
 - tf32: true
 
 ### Train loss graph
-This one was obtained by experimentally repeating the data 3 times and finetuning for 1 epoch,
-with similar end results but a smoother graph without sudden jumps compared to finetuning
-unique data for 3 epochs.
-
-![Train loss](https://files.catbox.moe/hiu9ah.png)
+![Train loss](https://files.catbox.moe/dg4qww.png)