1bit-MONSTER/Zamba2-VL-7B-GGUF

Zamba2-VL-7B — GGUF

Our own GGUF conversion of Zyphra's Zamba2-VL-7B: the Zamba2 language model with a Qwen2.5-VL vision tower. The converter is ours (llama.cpp #16, #17), including a chat template that places images the way Zyphra's does.

Contents

• Zamba2-VL-7B-Q8_0.gguf: the language model.
• mmproj-Zamba2-VL-7B-F16.gguf: the vision tower.

Validation

• Vision embeddings against transformers: per-token cosine 0.99989 on the CPU and 0.99944 on Vulkan.
• On a synthetic image (a red square, a blue circle and "HELLO 42") the model answers "a red square on the left and a blue circle on the right. Below the shapes, there's text that reads "HELLO 42"".

Running it

With the 1bit engine:

1bit serve -m Zamba2-VL-7B-Q8_0.gguf --mmproj mmproj-Zamba2-VL-7B-F16.gguf --device vulkan

With llama.cpp (our fork), pass --jinja so the embedded chat template is used.

Attribution

• Base model: Zyphra/Zamba2-VL-7B, Apache 2.0.
• GGUF conversion and validation: the 1bit engine project, with the converter and model code in our llama.cpp fork.
• License: Apache 2.0, inherited from the base model.