Sao10K/NyakuraV2-34B-Yi-Llama

NyakuraV2 - A Multi-Turn / Instruct Mix Fine-tuned Model.

Compute is thanks to my wallet, and runpod community pod 8x a40s for roughly 2 hours? Yeah that's it.

Trained in ShareGPT dataset format due to multi-turn capabilities.

use Vicuna 1.1 prompt format. Alpaca may work fine too, that format is like universal, may give sub-par results though..

I had much better results with the fp16 instead of GGUF, GGUF quants seem fucked? I don't know. May be on my side. Had so much nonsense input, had to wrangle settings until I had it coherent, it was working Really good. Fuck Yi models are a pain to work with.

You need better care with samplers when handling Yi Models. I have tested that Nyakura works well from 0 to 8k+ context, so I know it is not the model's fault.

Meow.

(Optional) System: <Prompt>

User: <Input>

Assistant:

Example Prompt:

User: Are you stand proud you're strong because you're nah I'd win, or are you nah I'd win because you're stand proud you're strong?

Assistant: You really should go to the doctor, I think you need help.