Lewdiculous/Poppy_Porpoise-v0.7-L3-8B-GGUF-IQ-Imatrix

"It keeps getting better!"

"One of the top recent performers in the Chaiverse Leaderboard!"

GGUF-IQ-Imatrix quants for ChaoticNeutrals/Poppy_Porpoise-v0.7-L3-8B.

Updated! These quants have been redone with the fixes from llama.cpp/pull/6920 in mind.
Use KoboldCpp version 1.64 or higher.

Compatible SillyTavern presets here (recommended/simple)) or here (Virt's).
Use the latest version of KoboldCpp. Use the provided presets.
This is all still highly experimental, let the authors know how it performs for you, feedback is more important than ever now.

For 8GB VRAM GPUs, I recommend the Q4_K_M-imat quant for up to 12288 context sizes.

Original model information:

Update: Vision/multimodal capabilities again!

If you want to use vision functionality:

• You must use the latest versions of Koboldcpp.

To use the multimodal capabilities of this model and use vision you need to load the specified mmproj file, this can be found inside this model repo. https://huggingface.co/ChaoticNeutrals/Llava_1.5_Llama3_mmproj

• You can load the mmproj by using the corresponding section in the interface: