|
|
--- |
|
|
license: cc-by-nc-4.0 |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
### Outdated! Move to [Lyra-v4](https://huggingface.co/Sao10K/MN-12B-Lyra-v4) |
|
|
|
|
|
--- |
|
|
|
|
|
### Outdated |
|
|
|
|
|
--- |
|
|
|
|
|
 |
|
|
|
|
|
Mistral-NeMo-12B-Lyra-v4, layered over [Lyra-v3](https://huggingface.co/Sao10K/MN-12B-Lyra-v3), which was built on top of [Lyra-v2a2](https://huggingface.co/Sao10K/MN-12B-Lyra-v2a2), which itself was built upon [Lyra-v2a1](https://huggingface.co/Sao10K/MN-12B-Lyra-v2a1). |
|
|
|
|
|
# Model Versioning |
|
|
``` |
|
|
Lyra-v1 [Merge of Custom Roleplay & Instruct Trains, on Different Formats] |
|
|
| |
|
|
| [Additional SFT on 10% of Previous Data, Mixed] |
|
|
v |
|
|
Lyra-v2a1 |
|
|
| |
|
|
| [Low Rank SFT Step + Tokenizer Diddling] |
|
|
v |
|
|
Lyra-v2a2 |
|
|
| |
|
|
| [RL Step Performed on Multiturn Sets, Magpie-style Responses by Lyra-v2a2 for Rejected Data] |
|
|
v |
|
|
Lyra-v3 |
|
|
| |
|
|
| [Backmerge to v2a1 + LoRA Extraction + Low Rank SFT Step for Coherency] |
|
|
v |
|
|
Lyra-v4 |
|
|
``` |
|
|
|
|
|
# This uses ChatML, or any of its variants which were included in previous versions. |
|
|
|
|
|
``` |
|
|
|
|
|
<|im_start|>system |
|
|
This is the system prompt.<|im_end|> |
|
|
<|im_start|>user |
|
|
Instructions placed here.<|im_end|> |
|
|
<|im_start|>assistant |
|
|
The model's response will be here.<|im_end|> |
|
|
-------------------------------------------------- |
|
|
[INST]system |
|
|
This is another system prompt.[/INST] |
|
|
[INST]user |
|
|
Your instructions placed here.[/INST] |
|
|
[INST]assistant |
|
|
The model's response will be here.[/INST] |
|
|
``` |
|
|
|
|
|
# Recommended Samplers: |
|
|
|
|
|
``` |
|
|
Temperature: 0.6 - 1 # I recommend lowering it a litte for v4? |
|
|
min_p: 0.1 - 0.2 # Crucial for NeMo |
|
|
``` |
|
|
|
|
|
# Recommended Stopping Strings: |
|
|
|
|
|
``` |
|
|
<|im_end|> |
|
|
</s> |
|
|
[/INST] |
|
|
``` |
|
|
|
|
|
*Introduces run-off generations at times, as seen in v2a2. It's layered on top of older models, so eh, makes sense. Easy to cut out though.* |
|
|
|
|
|
# Notes |
|
|
|
|
|
\- Some people have been having issues with run-on generations for Lyra-v3. Kind of weird, when I never had issues. |
|
|
<br>\- Anyway, make sure not to skip special tokens, or ban EOS tokens. I think this is the main problem that happens when v3 was to be quanted. The special tokens map config is fucked in v3, Quantizing tools likely spazzed out seeing it. I blame llamafactory for it. It ran fine unquantised. |
|
|
<br>\- I like long generations, though I can control it easily to create short ones. If you're struggling, prompt better. Fix your system prompts, use an Author's Note, use a prefill. They are there for a reason. |
|
|
<br>\- Lyra passes my internal benchmark suite, hence why I'm releasing it. Do I like it? Yes? it's out. that's it. They are my models for my personal enjoyment first. |
|
|
<br>\- Issues like roleplay format are what I consider worthless, as it follows few-shot examples fine. This is not a priority for me to 'fix', as I see no isses with it. Same with excessive generations. Its easy to cut out. |
|
|
<br>\- If you don't like it, just try another model? Plenty of other choices. Ymmv, I like it. |
