tillknuesting/siglip2-so400m-probe-mlx

SigLIP2 so400m linear probes (MLX)

Binary image classifiers: logistic-regression probes on the pooled image embedding of google/siglip2-so400m-patch14-384, plus a native MLX port of the SigLIP2 vision tower for Apple Silicon.

There are two probes, each for a label from a private, hand-annotated dataset; the labels are not documented here. These are not general-purpose detectors: evaluate them on your own data before relying on them for anything.

Files

• probe.npz, probe2.npz – probe weights: w (1152), b, model (backbone name).
• siglip_mlx.py – MLX port of the SigLIP2 image tower (so400m-patch14-384). On first use it downloads the Google checkpoint and caches a vision-only fp16 copy in weights/.
• predict.py – scores images or folders and prints one probability per image.

Usage

uv run --python 3.13 --with mlx --with numpy --with pillow --with safetensors --with huggingface-hub \
  predict.py [--probe probe2.npz] path/to/images

--probe defaults to probe.npz.

Details

• Preprocessing matches SiglipImageProcessor: bilinear resize to 384×384, scale to [-1, 1].
• MLX port vs. the transformers reference: embedding cosine similarity ≥ 0.9999 (fp16).
• Throughput: about 5 images/s on an Apple M4 (the GPU is compute-bound; 8/4-bit quantization was slower).
• Probes: LogisticRegression(C=10, class_weight="balanced") on L2-normalised embeddings. Scores are not calibrated probabilities.
• Evaluation: 5-fold cross-validation grouped by source scene/near-duplicates.
  • probe.npz: about 5.8k images; on the 2.9k hand-reviewed images, ROC-AUC 0.989 and average precision 0.94.
  • probe2.npz: about 6.3k images; on the 4.8k hand-reviewed images, ROC-AUC 0.90 and average precision 0.84.

License

Apache-2.0, like the SigLIP2 base model.