Have you gotten this to work somehow?
#1
by
AiCreatornator
- opened
As far I understand, llama.cpp does not support yayi2 yet? How you get this to work?
This model is not functioning. Currently, only BNB NF4 Quant works. I will update it later once llama.cpp supports it. In the meantime, you can try nf4 quant from https://huggingface.co/mzbac/yayi2-30b-guanaco
There is now llamafied version of the YAYI2 that would need fine-tuning: https://huggingface.co/cognitivecomputations/yayi2-30b-llama
@AiCreatornator Thanks for sharing yayi2-30b-llama. I had a quick look and it seems like it's just a configuration change. Conceptually, it doesn't require any further fine-tuning. I am currently uploading the gguf model with the correct configuration now.
Closed for now, as the new updated gguf works as expected.
mzbac
changed discussion status to
closed