openbmb

/

MiniCPM-Llama3-V-2_5-int4

like 64

Currently using the v2 gguf, though this is a significant improvement and having a gguf of this would be greatly appreciated.
The strong OCR abilities of this model would be very helpful for making documents accessible.
This model is excellent and all your kind efforts and contributions to the community are greatly appreciated.

The GGUF format will be released soon😃

Thank you all for the valuable feedback! We really appreciate it. We are working on GGUF and ollama for MiniCPM-Llama3-V 2.5, which will be available soon. We hope the model can help the community and the people in need.

MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.

and here is our model in gguf format.
https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf
@ibrahimkettaneh

thanks @Cuiunbo what are vram requirements for each of " ggml-model-Q4_K_M.gguf - mmproj-model-f16.gguf "

For memory consumption, mmproj-model-f16.gguf is about 1g+, and ggml-model-Q4_K_M.gguf is 5g.
But we optimize to run vit first, then free it, so that the overall consumption peaks at only 5g.
@xzintsux

We have supported Ollama! Please visit our GitHub for more usage info! https://github.com/OpenBMB/MiniCPM-V