Does not work. The model size is wrong too. 1.5B x 5.5BPW should be ~1.6GB

by imoc - opened Nov 15

imoc

Nov 15

So...?

Owner Nov 15

Oh, I don't know

Owner Nov 15

I used AutoQuants to quantize it, which himself uses turboderp/exllamav2 to quantize. So I don't know.
I will try to requantize it

Owner Nov 15

How did you calculate this "1.6GB" ?

Owner Nov 15

@imoc There was indeed an issue. I requantized it and reuploaded it so now it's corrected.
Thanks for the report

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment