imatrix Quantization of moonshotai/Kimi-K2.7
The other quants in this collection REQUIRE ik_llama.cpp fork to support the ik's latest SOTA quants and optimizations! Do not download these big files and expect them to run on mainline vanilla llama.cpp, ollama, LM Studio, KoboldCpp, etc!
NOTE ik_llama.cpp can also run your existing GGUFs from AesSedai, unsloth, bartowski, mradermacher, etc
Some of ik's new quants are supported with Nexesenex/croco.cpp fork of KoboldCPP with Windows builds for CUDA 12.9. Also check for Windows builds by Thireus here. which have been CUDA 12.8.
These quants provide best in class perplexity for the given memory footprint.
Available quants
IQ2_KS IQ2_KT
Upcomming
IQ2_KL IQ3_KL
References
ACK
Original Imatrix from Unsloth/Kimi-K2.7-Code-GGUF converted via https://gghfez-ik-llama-imatrix-converter.hf.space/
- Downloads last month
- 46
2-bit