Request: Please quantize raven in GPTQ
#7
by
rombodawg
- opened
Can you quantize the rwkv-4-raven model in GPTQ since ggml has already been done? Id prefer the 14b is you can only do 1, and if you can do it in 8-bit that would be lovely, but i will take a 4-bit version too. but if you can do all the models that would be awesome too as im sure other people would like them. Its a really good coding model that supports 8k context out of the box and follows instructions better than wizard coder (at least thats what ive heard)