init

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,50 @@
+---
+inference: false
+license: apache-2.0
+---
+
+# Model Card
+
+<p align="center">
+  <img src="./icon.png" alt="Logo" width="350">
+</p>
+
+📖 [Technical report](https://arxiv.org/abs/2402.11530) | 🏠 [Code](https://github.com/BAAI-DCAI/Bunny) | 🐰 [3B Demo](https://wisemodel.cn/spaces/baai/Bunny) | 🐰 [8B Demo](https://2e09fec5116a0ba343.gradio.live)
+
+This is **GGUF** format of [Bunny-Llama-3-8B-V](https://huggingface.co/BAAI/Bunny-Llama-3-8B-V).
+
+Bunny is a family of lightweight but powerful multimodal models. It offers multiple plug-and-play vision encoders, like EVA-CLIP, SigLIP and language backbones, including Llama-3-8B, Phi-1.5, StableLM-2, Qwen1.5, MiniCPM and Phi-2. To compensate for the decrease in model size, we construct more informative training data by curated selection from a broader data source.
+
+We provide Bunny-Llama-3-8B-V, which is built upon [SigLIP](https://huggingface.co/google/siglip-so400m-patch14-384) and [Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct). More details about this model can be found in [GitHub](https://github.com/BAAI-DCAI/Bunny).
+
+![comparison](comparison.png)
+
+
+# Quickstart
+
+## Chay by [`llama.cpp`](https://github.com/ggerganov/llama.cpp)
+
+```shell
+# sample images can be found in images folder
+
+# fp16
+./llava-cli -m ggml-model-f16.gguf --mmproj mmproj-model-f16.gguf --image example_2.png -c 4096 -p "Why is the image funny?" --temp 0.0
+
+# int4
+./llava-cli -m ggml-model-Q4_K_M.gguf --mmproj mmproj-model-f16.gguf --image example_2.png -c 4096 -p "Why is the image funny?" --temp 0.0
+```
+
+## Chat by [ollama](https://ollama.com/)
+
+```shell
+# sample images can be found in images folder
+
+# fp16
+ollama create Bunny-Llama-3-8B-V-fp16 -f ./ollama-f16
+ollama run Bunny-Llama-3-8B-V-fp16 "example_2.png Why is the image funny?"
+
+# int4
+ollama create Bunny-Llama-3-8B-V-int4 -f ./ollama-Q4_K_M
+ollama run Bunny-Llama-3-8B-V-int4 "example_2.png Why is the image funny?"
+```
+

ggml-model-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88f0a61f947dbf129943328be7262ae82e3a582a0c75e53544b07f70355a7c30
+size 4921246976

ggml-model-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9ff2d9f6e36a2c117fce8311d97e50db6abc11fcac591c08203db11cba0df0b
+size 16069403872

mmproj-model-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96d033387a91e56cf97fa5d60e02c0128ce07c8fa83aaaefb74ec40541615ea5
+size 877771936

ollama-Q4_K_M

@@ -0,0 +1,15 @@
+FROM ./ggml-model-Q4_K_M.gguf
+FROM ./mmproj-model-f16.gguf
+
+SYSTEM """
+A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
+"""
+
+TEMPLATE """
+{{ .System }}
+USER: {{ .Prompt }}
+ASSISTANT:
+"""
+
+PARAMETER temperature 0.0
+PARAMETER num_ctx 4096

ollama-f16

@@ -0,0 +1,15 @@
+FROM ./ggml-model-f16.gguf
+FROM ./mmproj-model-f16.gguf
+
+SYSTEM """
+A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
+"""
+
+TEMPLATE """
+{{ .System }}
+USER: {{ .Prompt }}
+ASSISTANT:
+"""
+
+PARAMETER temperature 0.0
+PARAMETER num_ctx 4096