q8_0 pls
#7
by
frz1
- opened
No description provided.
Could you make a q8_0 version? This will be a lossless version of the original if I understand correctly.
frz1
changed pull request status to
open
Yes, that's already done and in the upload queue.
I'm currently working on speeding things up (redoing from a faster internet connection), so soon uploads will be faster.
I plan on uploading all Quants that llama.cpp supports (and maybe a few more)
For now maybe use the Q6_K quant, it should be pretty close to the original.
So far I've only tried Q2... Thanks for grok-1 support to llama.cpp.