Request: Fizzarolli/duloxetine-4b-v1
Model name: Duloxetine-4b-v1
Model link: https://huggingface.co/Fizzarolli/duloxetine-4b-v1
Model description: roleplaying finetune of qwen 1.5 4b
An image to represent the model:
Notes:
hope a qwen 2 architecture model is okay! ^^
also i'm not too sure it's worth it to do any quants under q4 on a <7b model
Honestly I haven't played with Qwen2, will have to see if I can even get proper quants out.
also i'm not too sure it's worth it to do any quants under q4 on a <7b model
If I do it, will only do Q4 and higher.
Uploading at:
https://huggingface.co/Lewdiculous/duloxetine-4b-v1-GGUF-IQ-Imatrix
Let me know if it's broken.
thank you!!!
currently downloading kcpp and compiling on mobile, we'll see how this works lol
4t/s on the q6, not too shabby for mobile cpu
yeah everything seems to work fine, thanks again!!