Any GGUF versions for this model?

by krigeta - opened Jan 23

Jan 23

is there any GGUF version of this version 2? btw I am tagging @TheBloke if he is able to make the variants for it

Owner Jan 23

I'm uploading 5-bit right now for you here:

If you need 4 or 6 just let me know

Jan 24

I would really appreciate it if you did q4_k_m as well! Liked the first version of the model, curious about this one as well.

Owner Jan 24

@TariTariTu thanks for the feedback! q4_k_m is upload now, should be available 20 mins from this message.

Jan 25

I'm uploading 5-bit right now for you here:

https://huggingface.co/senseable/WestLake-7B-v2-gguf

If you need 4 or 6 just let me know

thank you so much

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment