LNTTushar/perception-slm-caption

Small, from-scratch image-understanding model (Perception SLM, Phase 2 / image v0). Encoder → connector (resampler) → tiny decoder, trained with Stage-2 alignment and Stage-3 LoRA instruction tuning. Config: caption_lm.

Held-out metrics

metric	value
loss	2.3982
bleu4	10.07

Files

• model.pt — checkpoint (model_state + training metadata)
• config.yaml — the exact config used to build the model
• model_int8.pt — int8-quantized weights for CPU/offline (if uploaded)

Load

from huggingface_hub import hf_hub_download
import torch
ckpt = torch.load(hf_hub_download("LNTTushar/perception-slm-caption", "model.pt"), map_location="cpu")
# rebuild ImageVLM.from_config(config) then load_state_dict(ckpt["model_state"])

Built with the perception-slm repo; see its RESULTS.md.