Qwen3.5-9B-Vision-CoreML-fp16

The vision encoder of Qwen/Qwen3.5-9B, compiled for Core ML in fp16. It lets a Qwen3.5-9B GGUF answer questions about images in tempo9: a GGUF carries the language model, and tempo9 runs the vision encoder through Core ML, on the GPU or the Neural Engine, instead of from an mmproj file.

Use

hf download thinkspread/Qwen3.5-9B-Vision-CoreML-fp16 \
  --local-dir ~/models/Qwen3.5-9B-Vision-CoreML-fp16
tempo9 --hf unsloth/Qwen3.5-9B-GGUF:Q4_K_S \
  --tower ~/models/Qwen3.5-9B-Vision-CoreML-fp16

Then send the image as an image_url data URI to /v1/chat/completions. Any OpenAI client works; with nothing but the standard library:

python3 - photo.png <<'EOF'
import base64, json, sys, urllib.request
img = base64.b64encode(open(sys.argv[1], "rb").read()).decode()
body = {"model": "local", "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "data:image/png;base64," + img}},
    {"type": "text", "text": "What is in this picture?"}]}]}
req = urllib.request.Request("http://127.0.0.1:11435/v1/chat/completions",
    json.dumps(body).encode(), {"content-type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])
EOF

• Pair it with Qwen3.5-9B only. Every Qwen3.5 size has its own vision weights; this one emits 4096-wide embeddings, the width of the 9B language model. How the GGUF is quantized does not matter. For Qwen3.5-35B-A3B use thinkspread/Qwen3.5-35B-A3B-Vision-CoreML-fp16.
• One image per request, as a data: URI.
• An image costs about one prompt token per 32×32 pixels, plus two. Images larger than about one megapixel (4096 patches of 16×16) are scaled down, aspect ratio kept.
• Apple Silicon, macOS 26 or later.

Files

file	what it is
qwen35_vit_b1024.mlmodelc	encoder for images up to 1024 patches (≈ 512×512 px)
qwen35_vit_b4096.mlmodelc	encoder for images up to 4096 patches (≈ 1024×1024 px)
tower_meta.json	shapes, buckets, and the accuracy measured at export
tower_pos_embed.f32, tower_aux.npz	position-embedding table

The two encoders share their weights and differ only in input size; a small image runs on the smaller one.

Accuracy

Measured at export: Core ML fp16 output against the Hugging Face fp32 vision encoder on the exporter's reference images, worst case, norm-weighted cosine similarity.

compute tier	cosine
GPU	0.99993
Neural Engine	0.99424

The tempo9 server runs the encoder on Core ML's GPU tier. The Neural Engine row is for apps that move it there to keep the GPU free for the language model.

Checked

Apple M5 Pro, 24 GB, tempo9 1.2.1, unsloth/Qwen3.5-9B-GGUF Q4_K_S, greedy, thinking off. A synthetic 640×448 image with a red circle, a blue square and the text "PIER 37":

The image contains a red circle and a blue square. The text reads "PIER 37".

312 prompt tokens in total.

License

Apache 2.0, inherited from the base model.