Custom GGUF Quants with iMatrix for : https://huggingface.co/MarsupialAI/LaDameBlanche-v2-95b
- Q8_0 used as quant base : https://huggingface.co/mradermacher/LaDameBlanche-v2-95b-GGUF
- iMatrix here : https://huggingface.co/mradermacher/LaDameBlanche-v2-95b-i1-GGUF
(Yes, I'm lazy, but I can live with a 0.01ppl bump ^^)
The model is a great merge, sensical and creative, imho working better for lesser requirements than the 100b+ Miqu which are worthy only for those with 48GB VRAM or more.
In IQ2_LR(2.7BPW, for 8k context with 36GB VRAM and an IGP running the OS display), ARC Challenge at 57, ARC Easy at 77, PPL 512 at 4.5860.
Mesdames et messieurs, vous êtes servis!