exllamav2 quant?
#1
by
traveltube
- opened
Just wanted to say thank you for all the merges; they have been excellent. Would there be a exllamav2 quant of this one as well?
Yep. I would have already uploaded, but the conversion script crashed. Its running again now.
4bpw is uploading now and is linked here, lower quants will be uploaded soon: https://huggingface.co/collections/brucethemoose/most-recent-merge-65742644ca03b6c514afa204
Thank you, it seems really phenomenal on my tries thus far!