Exl2 Quants

by llmixer - opened Feb 16

Feb 16

I was surprised to find no exl2 quants for v2, it's a great model! Maybe the many numbers in the model names are confusing :)

Uploaded some quants here:
2.4bpw
3.0bpw
3.5bpw
4.0bpw
5.0bpw

Owner Feb 17

Splendid! Thanks for putting them up. I'll add links to the model card.

sophosympatheia changed discussion status to closed Feb 17

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment