Guid/gromov_GRIT

Gromov loss checkpoints trained on GRIT

Checkpoints for github.com/fbyrman/gromov: hyperbolic (Lorentz) vision-language models trained with the Gromov loss.

• Data: GRIT, ~13.4M image-caption pairs with grounded boxes. One box crop and its phrase are sampled per image per step.
• Training: 200,000 iterations at total_batch_size=512, AdamW (lr 5e-4, weight decay 0.2), 4,000-step linear warmup and cosine decay, mixed precision. Image and text encoders are trained from scratch.

Each checkpoint is stored as {"model", "iteration"} in fp32 with no optimizer state. manifest.json records each file's sha256. config.yaml is the full training config of the run. Pass it as --train-config.

checkpoint	model	supervision
grit_power_inter_k3_vit_s/model.pth	PowerGromovInter, power_k=3	captions
grit_power_inter_k3_vit_b/model.pth	PowerGromovInter, power_k=3	captions
grit_power_all_vit_s/model.pth	PowerGromovAll, power_k_inter=2, power_k_intra=3	captions and boxes
grit_power_all_vit_b/model.pth	PowerGromovAll, power_k_inter=2, power_k_intra=3	captions and boxes

Use

From a clone of the GitHub repo:

hf download Guid/gromov_GRIT grit_power_all_vit_b/model.pth grit_power_all_vit_b/config.yaml \
    --local-dir checkpoints
python scripts/evaluate.py --config configs/eval_zero_shot_classification_score_level.py \
    --checkpoint-path checkpoints/grit_power_all_vit_b/model.pth \
    --train-config checkpoints/grit_power_all_vit_b/config.yaml \
    --results-file evaluate_score_level.json