Qwen3.5-4B-4bit

A verbatim mirror of mlx-community/Qwen3.5-4B-4bit, kept here so the Vanta iOS app always has a stable place to download it from.

Run it on your iPhone with Vanta

This is the recommended one-tap download in Vanta — Local AI LLM Chat, a local-first AI chat app for iPhone and iPad. Vanta runs this model fully on-device with Apple's MLX framework — no account, no cloud, no key. Your chats stay on your device. Because it is a vision model, you can chat about images too, and because its chat template declares tools, Vanta can let it use web search, memory, PDF search and your MCP servers.

Download Vanta on the App Store →

Vanta features this 4B on iPhones with 6 GB of RAM and up. On 4 GB devices it offers the lighter Qwen3.5 2B instead, and it shows an honest per-device fit badge either way.

---

This is a copy. Every model file in this repository is an exact copy of mlx-community/Qwen3.5-4B-4bit. We cloned it so that Vanta always has a reliable, always-available source to download this model from, independent of any upstream changes. All credit for the model weights and the MLX conversion goes to mlx-community, Qwen, and the original authors. No weights, config or tokenizer file was modified — only this model card and its banner were added.

---

Model details

Original model	Qwen/Qwen3.5-4B
Upstream MLX repo	mlx-community/Qwen3.5-4B-4bit
Architecture	qwen3_5 (hybrid attention, 32 layers, hidden size 2560)
Quantization	4-bit, group size 64
Format	MLX safetensors
Task	Image-text-to-text
Context window	262,144 tokens
Download size	~3.06 GB
License	Apache 2.0

What it can do

• Reads images. The repo carries a vision_config and a preprocessor_config.json declaring Qwen3VLProcessor, so the vision tower loads through the standard MLX VLM path.
• Calls tools. The chat template declares an XML function-call format (<tool_call><function=name><parameter=…>), which both mlx-lm and Vanta parse.
• Thinks first. Qwen3.5 is a hybrid reasoner. This 4B's chat template opens a <think> block in the generation prompt unless you pass enable_thinking=False; the 2B does the opposite and stays quiet unless you pass enable_thinking=True. Vanta always passes the flag explicitly, so the app is in control either way.

Verified

Loaded and run on Apple Silicon with mlx-vlm 0.5.0 and mlx-lm 0.31.3 before publishing: the model loads through Qwen3VLProcessor, answers text prompts, correctly describes an image, and emits a well-formed <tool_call> for a function it was given.

A note on the -OptiQ-4bit variants. Vanta does not use mlx-community/Qwen3.5-4B-OptiQ-4bit: it ships no preprocessor_config.json and keeps the vision tower outside the safetensors index in a non-standard optiq/ directory, so a standard MLX VLM processor cannot be built for it. This plain 4-bit conversion is the one that loads end to end.

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("TerminatorPower/Qwen3.5-4B-4bit")

prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["path/to/image.jpg"], max_tokens=512))

CLI:

python3 -m mlx_vlm.generate \
  --model TerminatorPower/Qwen3.5-4B-4bit \
  --image path/to/image.jpg \
  --prompt "Describe this image."

Text only, with mlx-lm:

python3 -m mlx_lm generate \
  --model TerminatorPower/Qwen3.5-4B-4bit \
  --prompt "Explain hybrid attention in two sentences."

Related

• Lighter Vanta pick: TerminatorPower/Qwen3.5-2B-4bit
• Upstream MLX repo: mlx-community/Qwen3.5-4B-4bit
• Original: Qwen/Qwen3.5-4B
• Speech models for Vanta: TerminatorPower/whisperkit-coreml

License

This model inherits the Apache 2.0 license from the original Qwen model. The mirror does not add any restrictions.