mlboydaisuke/WeVisOne-VL-2B-CoreAI

WeVisOne-VL-2B — Core AI

On an iPhone 18 Pro (iOS 27.2), cutting out what a few words name in a photo — each match's box and its alpha matte — took 0.99–1.00 s for one match and 4.1 s for seven with these graphs, measured 2026-10-25 in CoreAIKit's Examples/Cutout. This repository holds Tencent's WeVisOne-VL-2B (revision 3dad9ce8) for Apple's Core AI as four graphs: the text decoder (int8, with its last hidden state as a second output), the vision tower with the pixel decoder's inputs (fp16), and the 9.5M-parameter pixel decoder in two parts (fp16).

Variant	Path	Size	Requires	Tested on
GPU, specialized on the device (.aimodel)	ios/	3,298,811,031 B	iOS 27 and an app with com.apple.developer.kernel.increased-memory-limit	iPhone 18 Pro, iOS 27.2 (24B5099f), 2026-10-25
GPU, compiled ahead of time (.h16c.aimodelc)	macos/	10,188,197,010 B	macOS 27 on a Mac16,x (the h16c architecture)	Mac M4 Max, macOS 27.0 (26A428), 2026-10-25

Each folder is the model folder the Cutout example reads: name it WeVisOne-VL-2B-CoreAI and put it in the app's Documents (iPhone) or ~/Library/Application Support/Cutout/ (Mac).

hf download mlboydaisuke/WeVisOne-VL-2B-CoreAI --include "ios/*" --local-dir wevisone   # or "macos/*"

The graphs need a host that runs them in order; they are not a CoreAIKit catalog model. The Cutout example (CoreAIKit Examples/Cutout) is that host: pick or take a photo, type what to cut out, and save the cut-out as a PNG with the background removed.

A recorded run on an iPhone 18 Pro — a photo of three lit jack-o'-lanterns among small uncarved pumpkins, "every carved pumpkin" typed: 3 found, boxes 0.83 s and matte 1.42 s on the screen, only the three carved ones cut out, then lined up on one sheet and saved to Photos — is the 14.5 s clip wevisone-vl-2b-pumpkins-iphone-20261027.mp4 (the photo is CC0).

What each folder holds

file	what	bytes (ios/)
wevisone_vl_2b_decode_int8hu_pf64/	the text decoder: main (S = 1) and prefill (S = 64) sharing two KV states; outputs logits and last_hidden (the last decoder layer's output before the final norm, which the pixel decoder reads at each <OBJ>); linears int8 per block of 32 (body symmetric_with_clipping, the untied head absmax); metadata.json and tokenizer/ (the source's tokenizer.json, tokenizer_config.json and chat_template.jinja, byte for byte)	2,467,304,523
wevisone_vl_2b_vision_taps_g14.aimodel	the vision tower for a 448 × 448 image (196 rows): image_embeds, deepstack_embeds and taps, the outputs of ViT layers 5 / 11 / 17 / 23 the pixel decoder reads; fp16	811,114,774
wv_mask_a_g28_n16_fp16_hf.aimodel	pixel decoder part A: up to 16 matches per call, the box encoding (box_pe) given by the host; fp16	19,540,579
wv_mask_b_fp16.aimodel	pixel decoder part B: one call per match, the 1024 × 1024 matte logits; fp16	851,155

macos/ holds the same four graphs compiled for the Mac (--platform macOS --architecture h16c, the decoder with --expect-frequent-reshapes; the compile stores an fp16 copy of the decoder's int8 linears). Each .aimodel is its export with the MLIR debug locations removed (strip_debug_info; op counts and coreai-build inspect equal to the export's). config.json is the source checkpoint's, unchanged; SHA256SUMS lists every other file.

How a request runs

The host does what the source package's predict(task="segmentation") does, with the photo resized to a fixed 448 × 448 grid:

1. Pillow's bicubic resize to 448 × 448 and to 1024 × 1024 (the Cutout host writes it in integer arithmetic: the same bytes as Pillow on PNG input);
2. the vision tower; the source's chat template with the package's segmentation prompt (its matting task uses the same one) and the system line "You are a helpful assistant.";
3. the decoder, greedy, no repetition penalty; the prompt in 64-token prefill calls with the tail padded into one more; last_hidden kept at each <OBJ> it writes;
4. the boxes by the package's parse_seg_boxes (four per-mille coordinates before each <OBJ>); their Fourier encoding on the host in fp32 — it has to equal torch's bit for bit, including the 7 box widths where Darwin's logf rounds differently from torch's log;
5. part A once per 16 matches, part B once per match; sigmoid and a bilinear resize to the photo's size.

The decoder runs on Core AI's low-level runtime, not the pipelined engine, because the engine returns token ids only and never the hidden state.

Decoder accuracy

Against the source package in fp32 on the same fixed grid (33 rows), the int8 decoder gives the reference's argmax at the last four prompt positions on 33 of 33 rows and 16 greedy tokens that never leave the reference's except at its own near-ties (top-2 softmax gap under 0.1) on 33 of 33. Its logits are not within cosine 0.9999 of the reference at every one of 16 teacher-forced steps on any row (lowest 0.99581); no decoder with fp16 weights reaches that on every row either (fp16 31 of 33, fp16 weights with fp32 compute 32 of 33). On the task level it finds every one of the reference's boxes at IoU ≥ 0.75 and gives every count exactly.

Measured

• iPhone 18 Pro (iOS 27.2, the Cutout app, graphs specialized on the phone, warm, thermal state nominal): the lighthouse example (1024 × 683, one match, a 13-token answer) 0.988–0.997 s — tower 89 ms, the 237-token prompt 141 ms, the answer 233 ms, part A 75 ms, part B 403 ms; seven chess pieces (a 50-token answer) 4.119–4.138 s, 2.77 s of it part B. The first launch after installing reached READY in 6.85 s and took 8.14 s for its first request; later launches reached READY in 1.5–2.6 s. Peak memory footprint 2.00 GB.
• Mac M4 Max (macOS 27.0, macos/): one match in a 3744 × 5616 NASA photo (decoded at 2731 × 4096) 0.545–0.554 s (the PNG another 0.29 s); peak footprint 2.8 GB.

Accuracy

Against the source package in fp32 (transformers 5.12.1, CPU), the photo resized to 448 × 448 first:

• structured answers of the decoder: boxes found at IoU ≥ 0.75: 100.0 % (63); counts exact: 100.0 % (7); points within 2 % of the side: 89.4 % (47); OCR character error rate 0.000;
• the pixel decoder's two graphs on the reference's inputs: mattes within MAE 2.09e-4 and IoU(α > 0.5) ≥ 0.992 of the package's fp32 decoder (46 matches);
• the whole request on the iPhone 18 Pro, on the rows whose boxes equal the reference's (5 rows, 6 matches): mattes within MAE 5.81e-4 and IoU ≥ 0.9959 of the package's.

Limits

• The fixed 448 × 448 grid. The source package keeps the photo's aspect ratio (up to 2025 × 32² pixels); these graphs stretch it. Against the package's own form, the fixed grid's fp32 answers find 85.5 % of its boxes at IoU ≥ 0.75, 71.4 % of its counts and 81.8 % of its points.
• 1,024 positions on iOS. Core AI's pipelined engine stops a dynamic-KV decoder at 1,024 positions of prompt and answer on iOS (apple/coreai-models#124); the Cutout host binds its cache at 1,024: 237 prompt tokens and up to 787 answer tokens.
• No repetition penalty. The package's greedy decoding uses 1.05; in fp32 the two agree on 26 of 33 answers, with every box and count the same.

Convert / verify

The exporters, the gates and the records: the Core AI model zoo's models/wevisone-vl-2b card and conversion/wevisone/, on coreai-torch 0.4.1 / coreai-core 1.0.0b2.

License

Apache-2.0, as the source model. LICENSE is the source repository's file, unchanged: it names Qwen3-VL-2B-Instruct (Alibaba Cloud, Apache-2.0) as a component modified by Tencent.