grabbou/Qwen3-VL-2B-Instruct-CoreAI

Qwen3-VL 2B Instruct for Core AI

Qwen/Qwen3-VL-2B-Instruct exported with the apple/coreai-models 1.0.0 VLM recipe, for CoreAIVisionLanguageModel (CoreAILanguageModels) and @react-native-ai/core-ai.

Folder

qwen3_vl_2b/ is the bundle (metadata.json kind: "vlm"), 2.7 GB:

File	Role	Size
qwen3_vl_2b.aimodel	Text decoder, 4-bit weights, stateful KV cache (4096)	923 MB
vision.aimodel	Vision encoder, 448x448 input, 196 image tokens, float16	1.2 GB
embed.aimodel	Token embeddings	594 MB
tokenizer/	Hugging Face tokenizer	11 MB

Platforms

• iOS 27 (iPhone 17 Pro): works with the increased memory limit entitlement, which raises the app limit from 3,376 MB to 6,144 MB. Without it the app aborts while MPSGraph loads the weights. Answering three questions about a 320x240 photo, including loading the model, took 39 s.
• macOS 27: the recipe's target format.

The VLM recipe has no iOS preset, so this is the macOS export format (one main function per model, run on the GPU).

Export

git clone https://github.com/apple/coreai-models && cd coreai-models && git checkout 1.0.0
uv run coreai.vlm.export qwen3-vl --compression 4bit

In 1.0.0, --compression 4bit fails with build_reference_inputs returned keys ['input_ids', ...], which are not a contiguous in-order prefix of forward's parameters ['inputs_embeds', ...], because Qwen3VLForCausalLMEmbeddings inherits token-id reference inputs. This export overrides export_input_names, build_reference_inputs and build_dynamic_shapes in python/src/coreai_models/models/macos/qwen3_vl.py to trace inputs_embeds (random [1, query_len, hidden_size] tensors) in place of input_ids. The model code is otherwise unchanged.

Use

import { coreAI } from '@react-native-ai/core-ai'
import { generateText } from 'ai'

const model = coreAI.languageModel({
  id: 'qwen3-vl-2b',
  source: { type: 'file', uri: bundleFolder },
})

const { text } = await generateText({
  model,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'file', mediaType: 'image/jpeg', data: new URL(`file://${imagePath}`) },
        { type: 'text', text: 'What animals are in this picture?' },
      ],
    },
  ],
})

Apple's VLM engine reads the first image of the prompt, decodes greedily, and needs an image in every prompt.

License

Apache-2.0, as the base model.