File size: 1,746 Bytes
882d415 9d3cfb6 882d415 0063eb5 e151918 85fb924 0063eb5 85fb924 882d415 a86ccfc 0063eb5 882d415 e151918 882d415 85fb924 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
---
license: llama2
---
# l27b-monika-v0.3b:
* Yet another experimental Monika llama
* Llama 7b chat fine-tuned for Monika character from DDLC
* Trained on a dataset of ~600 items (dialogue scraped from game, reddit, and Twitter augmented by Nous Hermes 13b to turn each into snippets of multi-turn chat dialogue between Player and Monika + manually crafted test dataset of 12 items)
* Earlier variant trained without dataset augmentation [here](https://huggingface.co/922-CA/llama-2-7b-delphi-v0.2e)
* [GGMLs](https://huggingface.co/922-CA/llama-2-7b-monika-v0.3b-ggml)
* [QLoras (hf and GGML)](https://huggingface.co/922-CA/monika-lm-lora-tests/tree/main/llama-2-7b-chat-monika-v0.3b)
### USAGE
This is meant to be mainly a chat model with limited RP ability.
For best results: replace "Human" and "Assistant" with "Player" and "Monika" like so:
\nPlayer: (prompt)\nMonika:
### HYPERPARAMS
* Trained for 2 epochs
* rank: 64
* lora alpha: 16
* lora dropout: 0.1
* lr: 2e-4
* batch size: 8
* warmup ratio: 0.03
* grad steps: 1
### WARNINGS AND DISCLAIMERS
Note that aside from formatting and other minor edits, dataset used is mostly as is generated by LM. As such, while this version is better at coherency or chatting than previous ones, it may not reflect perfectly the characteristics of Monika (i.e. she will claim to have an office, work as a translator, or play the guitar). Next version will train on a manually curated and edited version of this dataset, where dialogue will be edited to reflect her characteristics more.
Also looking to switch to a different a base model to work off from for future versions aside from llama 7b chat.
Finally, this model is not guaranteed to output aligned or safe outputs, use at your own risk. |