Fix AutoModel loading, add image processor and Apache-2.0 license
Hi, and thanks for releasing these models.
This follows up on the report on the SigLIP2-Base repo.
AutoModel.from_pretraineddid not work. Every tensor was stored under aclip.prefix left over from the training wrapper, which does not matchSiglipModel.base_model_prefix, so all 1048 parameters were randomly initialised and only a warning was emitted. The tensors are now stored under their canonical names.config.json.text_config.vocab_size:256000;text_config.intermediate_size:4304;text_config.num_hidden_layers:27;text_config.projection_size:1536;text_config.hidden_act:gelu_pytorch_tanh;vision_config.hidden_size:1536;vision_config.intermediate_size:6144;vision_config.num_hidden_layers:40;vision_config.hidden_act:gelu_pytorch_tanh;vision_dim:1536.- Image processor. Added
preprocessor_config.json, copied unchanged fromgoogle/siglip2-giant-opt-patch16-384, so the repo is self-contained and the card no longer has to send users to another repository for preprocessing. - License. Added
license: apache-2.0and aLICENSEfile (see below).
Verification
The reference is the wrapper this checkpoint was trained with: it is built fromgoogle/siglip2-giant-opt-patch16-384 and loaded from the published model.safetensors withload_state_dict(strict=True) (0 missing / 0 unexpected keys). Its embeddings are then
compared against AutoModel.from_pretrained on the files in this PR.
| check | result |
|---|---|
| missing / unexpected / mismatched keys | 0 / 0 / 0 |
| max abs difference vs. that reference | 0.0 |
| embedding dimensionality | 1536 |
image processor output vs. google/siglip2-giant-opt-patch16-384 |
identical (max abs diff 0.0) |
No weight values change anywhere in this PR -- only tensor names, config.json and the card.
One more thing worth flagging
Comparing these weights against the base model, the fine-tune moved only vision blocks 7-11;
the text tower, both projection heads and logit_scale/logit_bias are byte-identical togoogle/siglip2-giant-opt-patch16-384. The two towers are therefore no longer aligned, and image-text scoring gives
wrong answers (on a COCO cat photo the model now ranks "a photo of a car" above
"a photo of two cats"). That is expected for a triplet-loss re-identification fine-tune, but
nothing said so, and the Hub still advertises a zero-shot-image-classification tag.
For that reason this PR adds only preprocessor_config.json and deliberately does not
add tokenizer files -- shipping them would make that broken path runnable straight from the
repo. Mentioning it here rather than editing the card, since that is your call to make.
About the license
The repository currently has no license field and no LICENSE file, which is what prompted
the original question. This PR proposes Apache-2.0, matching the other public AvitoTech
models on the Hub and the Apache-2.0 base model(s) this is derived from -- but that call is
yours. If you would rather use different terms, say so and I will amend the PR; if you would
rather add the license yourself, feel free to drop the LICENSE file and the frontmatter
line from this PR and take just the loading fix.