MBZUAI/Omni-Embed-Mini-0.9B-onnx

Omni-Embed Mini 0.9B, ONNX

ONNX export of the Omni-Embed Mini 0.9B multimodal embedding model, built to run in a browser on WebGPU. One model embeds text, images, video, speech, general audio and document pages into a single 1024-dimensional space.

• Project page: https://omniembed.cvmbzuai.com/
• Live demo: https://demo-omniembed.cvmbzuai.com/

What is in here

Two directories, one per precision, each a complete bundle with the same file names, so a client picks one by changing a base URL. fp16/ is the default at 1.89 GB. q8/ is 1.39 GB and about half the memory in use, for devices that cannot hold fp16; measured on a 6,000 item index it returns the same best result for 12 of 14 queries and the same 9.8 of the top 10.

Each is a six-graph bundle driven from JavaScript rather than a transformers.js architecture: the splice, the pooling and two of the vision tower's three inputs are computed by the caller. See manifest.json for the preconditions a caller cannot read off the graphs.

Component	Graph
backbone	backbone/model.onnx, takes inputs_embeds
embedding table	backbone/embed_tokens.onnx
vision	vision_encoder/model.onnx, one image per call
audio	whisper_encoder/, dasheng_encoder/
projectors	projectors/*.onnx

Every component was parity-checked against its PyTorch reference before export. The measured numbers are in */conversion_metadata.json and */parity_report.json.

Preprocessing is part of the model: a still image is squared to 224 with PIL bilinear and then taken to 256 with a Pillow-compatible bicubic, and substituting a browser canvas resize moves the final embedding by cos 0.734, so a client that does not reproduce that chain is a different model and its vectors must not be mixed into an index built with this one.

Citation

@inproceedings{omniembedmini2026,
  title     = {Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation},
  author    = {TBD},
  booktitle = {TBD},
  year      = {2026},
  note      = {Camera-ready in preparation}
}