Qwen3.5-2B-4bit

A verbatim mirror of mlx-community/Qwen3.5-2B-4bit, kept here so the Vanta iOS app always has a stable lighter model to download from.

Run it on your iPhone with Vanta

This is one of the built-in one-tap downloads in Vanta — Local AI LLM Chat, a local-first AI chat app for iPhone and iPad. Vanta runs this model fully on-device with Apple's MLX framework — no account, no cloud, no key. Your chats stay on your device. Because it is a vision model, you can chat about images too, and because its chat template declares tools, Vanta can let it use web search, memory, PDF search and your MCP servers.

Download Vanta on the App Store →

Vanta recommends this smaller model on RAM-tight devices, where the 4B is too heavy — it is the only featured pick on 4 GB iPhones, and the lighter, faster alternative everywhere else.

---

This is a copy. Every model file in this repository is an exact copy of mlx-community/Qwen3.5-2B-4bit. We cloned it so that Vanta always has a reliable, always-available source to download this model from, independent of any upstream changes. All credit for the model weights and the MLX conversion goes to mlx-community, Qwen, and the original authors. No weights, config or tokenizer file was modified — only this model card and its banner were added.

---

Model details

Original model	Qwen/Qwen3.5-2B
Upstream MLX repo	mlx-community/Qwen3.5-2B-4bit
Architecture	qwen3_5 (hybrid attention, 24 layers, hidden size 2048)
Quantization	4-bit, group size 64
Format	MLX safetensors
Task	Image-text-to-text
Context window	262,144 tokens
Download size	~1.75 GB
License	Apache 2.0

What it can do

• Reads images. The repo carries a vision_config and a preprocessor_config.json declaring Qwen3VLProcessor, so the vision tower loads through the standard MLX VLM path.
• Calls tools. The chat template declares an XML function-call format (<tool_call><function=name><parameter=…>), which both mlx-lm and Vanta parse.
• Answers straight away. Qwen3.5 is a hybrid reasoner, and this 2B's chat template leaves the <think> block closed unless you pass enable_thinking=True — the opposite of the 4B, which reasons by default. That is what keeps it quick on a small phone. Vanta always passes the flag explicitly, so the app is in control either way.

Verified

Loaded and run on Apple Silicon with mlx-vlm 0.5.0 and mlx-lm 0.31.3 before publishing: the model loads through Qwen3VLProcessor, answers text prompts, correctly describes an image, and emits a well-formed <tool_call> for a function it was given.

A note on the -OptiQ-4bit variants. Vanta does not use mlx-community/Qwen3.5-2B-OptiQ-4bit: it ships no preprocessor_config.json and keeps the vision tower outside the safetensors index in a non-standard optiq/ directory, so a standard MLX VLM processor cannot be built for it. This plain 4-bit conversion is the one that loads end to end.

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("TerminatorPower/Qwen3.5-2B-4bit")

prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["path/to/image.jpg"], max_tokens=512))

CLI:

python3 -m mlx_vlm.generate \
  --model TerminatorPower/Qwen3.5-2B-4bit \
  --image path/to/image.jpg \
  --prompt "Describe this image."

Text only, with mlx-lm:

python3 -m mlx_lm generate \
  --model TerminatorPower/Qwen3.5-2B-4bit \
  --prompt "Explain hybrid attention in two sentences."

Related

• Larger Vanta pick: TerminatorPower/Qwen3.5-4B-4bit
• Upstream MLX repo: mlx-community/Qwen3.5-2B-4bit
• Original: Qwen/Qwen3.5-2B
• Speech models for Vanta: TerminatorPower/whisperkit-coreml

License

This model inherits the Apache 2.0 license from the original Qwen model. The mirror does not add any restrictions.