Any GGUF versions for this model?

#4
by krigeta - opened

is there any GGUF version of this version 2? btw I am tagging @TheBloke if he is able to make the variants for it

I'm uploading 5-bit right now for you here:

https://huggingface.co/senseable/WestLake-7B-v2-gguf

If you need 4 or 6 just let me know

I would really appreciate it if you did q4_k_m as well! Liked the first version of the model, curious about this one as well.

@TariTariTu thanks for the feedback! q4_k_m is upload now, should be available 20 mins from this message.

I'm uploading 5-bit right now for you here:

https://huggingface.co/senseable/WestLake-7B-v2-gguf

If you need 4 or 6 just let me know

thank you so much

Sign up or log in to comment