LoneStriker

/

opus-v0.5-70b-2.4bpw-h6-exl2

like 0

I can add it to the queue. The 70B models tend to be heavy to quant via exl2 so it might take time to get to it.

I emphatise with that. I'm straight up unable to quant that model with my 3090 and 16GB RAM, so I'm glad you can spare us some of your compute :)

Out of curiosity, what dataset do you use to test its fitness? And is it true that the longer you let it quant the less loss you encounter?

(Edit: I keep forgetting that button was "Close" lol)

Published earlier:
https://huggingface.co/models?search=LoneStriker/opus-v1.2-70b