Running in llama.cpp

#4
by ChuckMcSneed - opened

Note for any other 🤡 like me who wants to run this model in llama.cpp and it doesn't work: change in llama.cpp file

#define LLAMA_MAX_NODES   8192

to

#define LLAMA_MAX_NODES   16384

and compile.
Have fun!

Apologies for inconvenience.
Are there any specific issues that you face with this model?

Regards.

deepnight-research changed discussion status to closed

Yeah, it's incoherent, creates fake words and doesn't listen to the prompts. Not really worth its size, I did run it in Q3_K_L. Did you actually test the model before uploading? Or did you just glue it together and hoped that some poor fool would run it?

That sounds a bit odd to be happening.
We did test the model indeed before releasing it.
Allow us some time to look into the matter.

Thanks for bringing this to our attention.

Regards.

deepnight-research changed discussion status to open
deepnight-research changed discussion status to closed

This model really seems to be broken to me. I've tried your saily_100b, and it doesn't have this kind of issues.

Thanks @ChuckMcSneed for bringing this to our attention.
We're looking into the issues with this model and will update the weights soon.

Regards.

Sign up or log in to comment