InferenceIllusionist

/

psyonic-cetacean-20B-iMat-GGUF

like 2

@InferenceIllusionist Hello, if your still considering requests, I'd like a 4.25 bpw of DavidAU/L3-Grand-HORROR-18.5B-V1.8-STABLE-10-Gates-of-Hell. Thanks.

Hey @Clevyby thanks for reaching out. Happy to help here. A repo has been created for this quant: Quant-Cartel/L3-Grand-HORROR-18.5B-V1.8-STABLE-10-Gates-of-Hell-iMat-GGUF.

IQ4_XS / 4.25 bpw should be up shortly, stay tuned!

Thanks @InferenceIllusionist . Although I actually meant 4.25 bpw exl.. Sorry for not clarifying. I'm using free google colab which doesn't handle high parameter models well, so exl is the only reliable format for me to use

@Clevyby Ah no worries I should have clarified with you beforehand, it's all good. Tapped our resident exl2 expert @rAIfle who can lend a hand with this 🙂

@Clevyby Running a full set of exl2, including 4.25bpw right now. I'll pop back in here once it's done :)

Sorry for the delay, it's up now! Quant-Cartel/L3-Grand-HORROR-18.5B-V1.8-STABLE-10-Gates-of-Hell-exl2-longcal @Clevyby

Thanks for the quant.

Closing this out now that the repo is live, thanks again for reaching out!