Kairoi-LLC/siglip2-base-patch16-224-litert

SigLIP 2 B/16 image tower for LiteRT's GPU

A LiteRT (TFLite) build of the image tower of Google's SigLIP 2 B/16 at 224 px, converted by Kairoi for the Woven app's on-device photo search. It is built to run on LiteRT's GPU accelerator at fp16 and still give the fp32 model's vectors.

What is in it

One .tflite file, fp32 weights (about 370 MB), with one signature:

Signature	Input	Output
image	args_0, the pixels: f32 [8, 3, 224, 224], NCHW, RGB, (pixel / 255 − 0.5) / 0.5, resized straight to 224 × 224 (bilinear)	output_0, f32 [8, 768]: get_image_features, the pooled and projected embedding, not normalised

The batch is fixed at 8. Pad a shorter batch and drop the padding's rows. The text tower is not here: pair these vectors with SigLIP 2's own text tower (for example onnx-community/siglip2-base-patch16-224-ONNX), after L2-normalising both.

What was changed from the original

The arithmetic is the original model's. Two parts of the graph are written differently so the GPU can run all of it at fp16:

• Every LayerNorm scales its input by 1/64 first, with eps scaled to match. LayerNorm does not depend on the input's scale, so the answer is the same. From layer 10 on, SigLIP's residual stream reaches about 700. The squared differences then reach about 5 × 10⁵, past fp16's limit of 65,504. With the stock norms, the GPU's fp16 vectors came out at cosine 0.69–0.75 to fp32.
• The attention-pooling head is rewritten in 4-D operations. The learned probe's query is projected once, and each head's scores and weighted sum are a broadcast multiply and a sum. The stock nn.MultiheadAttention lowered to a 5-D reshape, a BROADCAST_TO and a transpose that the GPU delegate refuses. That left 36 operations on the CPU.

Only the image tower was converted.

How it was made

• Source: google/siglip2-base-patch16-224 at revision 75de2d55ec2d0b4efc50b3e9ad70dba96a7b2fa2 (Apache-2.0).
• Tool: litert-torch 0.9.4 (Apache-2.0) with torch 2.13.0 and transformers 4.57.1.
• The conversion script is scripts/models/convert-siglip2-litert.py in the Woven repository (uv run scripts/models/convert-siglip2-litert.py <out dir> 8). It is reproduced here as convert-siglip2-litert.py. Before converting, it checks the rewritten tower against the stock one in fp32 (cosine ≥ 0.99999) and in half precision.

Measured

This file, against the original model in fp32 (onnx-community's vision_model.onnx), on the same 224×224 pixels, prepared with PIL's bilinear resize. The sets are public and labelled: COCO val2017 (5,000), Flickr30k test (1,000), RVL-CDIP test (1,600 scanned documents, 16 classes), CORD-v2 receipts (100), FUNSD forms (199), INRIA Copydays (157 originals and their strong edits, 386 images), plus 300 app screenshots, 500 edited copies of COCO photos, and flat black, white, grey and red images. 9,089 images in all.

Where	Cosine min · p5 · median (worst set)	Caption recall@1 / @5, COCO 5k · Flickr30k 1k	One-word mAP, COCO
fp32 original	—	46.5 / 71.0 · 75.9 / 92.6	0.729
LiteRT CPU (Mac)	1.00000 · 1.00000 · 1.00000 (every set)	46.5 / 71.0 · 75.9 / 92.6	0.729
LiteRT GPU fp16, Apple M5 (Metal)	0.9951 · 0.9986 · 0.9995 (RVL-CDIP)	46.3 / 71.1 · 75.9 / 92.6	0.730
LiteRT GPU fp16, Pixel 8 Pro (Mali)	0.9882 · 0.9969 · 0.9992 (RVL-CDIP)	46.5 / 71.2 · 75.9 / 92.6	0.734
LiteRT GPU fp16, iPhone 13 mini (Metal)	min 0.9940 (COCO), median 0.9994–0.9998 per set	46.4 (COCO R@1)	0.734
LiteRT GPU fp16, NVIDIA RTX 3090 (WebGPU over Vulkan, Linux)	min 0.9967 (FUNSD), median 0.9997–0.99996 per set	46.3 (COCO R@1)	0.730

On photos (COCO, Flickr30k, Copydays) the GPUs' minimum is 0.994 and the median 0.9998. Scanned documents drift most. No output was NaN, flat images included. Recall is measured with the original's text tower quantised to int8.

Time per image at batch 8: Metal on the Mac 3.8 ms, RTX 3090 7–10 ms, iPhone 13 mini 36.6 ms, Pixel GPU 44–54 ms (the Pixel's rises as it warms), LiteRT CPU on the Mac 16.7 ms. The first load (read, compile, one reference check) took 1.7 s on the Mac, 3.0 s on the RTX 3090, 5.1 s on the Pixel and 8.0 s on the iPhone.

The resize matters: a point-sampled shrink of a large photo (no antialiasing) gave median cosine 0.89–0.98 to these pixels' vectors. Feed it a smoothed bilinear resize like PIL's.

Full method: Woven's docs/brand/research/AI-RUNTIME-COMPARISON-2026-09.md §9–10.

Files

File	SHA-256
siglip2-image-b8.tflite	16152dea21213077cc3202578d64ca73bac6c41ab6953c848b1d4526027e40e9
convert-siglip2-litert.py	the script it was made with
LICENSE	the Apache License 2.0

Apps should download by commit URL (/resolve/<40-hex>/<file>) and check the SHA-256.

Licence and attribution

SigLIP 2 is Google's, released under the Apache License 2.0. This converted build is a modified work, redistributed under the same licence; the full text is in LICENSE. The changes are listed above under "What was changed from the original". The original repository has no NOTICE file.

Converted by Kairoi. No training or fine-tuning was done; the weights are Google's.