Quants please - https://huggingface.co/moonshotai/Kimi-VL-A3B-Instruct

#834
by Poro7 - opened

This model uses KimiVLForConditionalGeneration which is unfortunately not yet supported by llama.cpp and so providing GGUF quants is not possible. There are no indications that llama.cpp support for KimiVLForConditionalGeneration will ever come snf id for dure not something that can be expected for the near future. There is no PR to add KimiVLForConditionalGeneration support and there isn’t even an issue requesting the llama.cpp team to add support for it.

Thanks for your reply anyway!

Poro7 changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment