inclusionAI/ZwZ2-9B-GGUF

ZwZ2-9B-GGUF

GGUF-format weights for ZwZ2-9B, part of Zooming without Zooming v2. The model is split into two components:

• Language model: FP16, Q8_0, Q4_K_M.
• Vision encoder and projector (mmproj): FP16, Q8_0, Q4_K_M.

Choose a language model and a matching vision component from this repository. Different precision levels can be combined. Quantization can affect accuracy; see the original model page for the published evaluation results.

How to Use

Use a recent llama.cpp build with Qwen3.5 multimodal support.

CLI Inference (llama-mtmd-cli)

For example, to run ZwZ2-9B with a Q8_0 vision encoder and Q8_0 quantized LLM:

llama-mtmd-cli \
  -m ZwZ2-9B-Q8_0.gguf \
  --mmproj mmproj-ZwZ2-9B-Q8_0.gguf \
  --image /path/to/image.jpg \
  -p "Describe this image." \
  --temp 0 -n 2048

Web Chat (using llama-server)

To serve ZwZ2-9B via an OpenAI-compatible API with a web UI:

llama-server \
  -m ZwZ2-9B-Q8_0.gguf \
  --mmproj mmproj-ZwZ2-9B-Q8_0.gguf \
  --host 127.0.0.1 --port 8080

Replace the filenames to use FP16 or Q4_K_M weights. For image input, always provide the corresponding mmproj component.

Citation

@article{wei2026zoomingv2,
  title={Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception},
  author={Wei, Lai and Li, Yatai and He, Liangbo and Lan, Jun and Gu, Renjie and Cao, Linhan and Lin, Junkai and Zhu, Huijia and Wang, Weiqiang and Kong, Linghe and Wang, Yue and Zhang, Zhuosheng and Huang, Weiran},
  journal={arXiv preprint arXiv:2610.37037},
  year={2026}
}

License

This model follows the Apache 2.0 License.