LNTTushar/perception-slm-image-v0

Small, from-scratch image-understanding model (Perception SLM, Phase 2 / image v0). Encoder → connector (resampler) → tiny decoder, trained with Stage-2 alignment and Stage-3 LoRA instruction tuning. Config: image_v0_gpu.

Held-out metrics

metric	value
loss	2.0373
token_acc	0.555
exact_match	0.0
n	294

Files

• model.pt — checkpoint (model_state + training metadata)
• config.yaml — the exact config used to build the model
• model_int8.pt — int8-quantized weights for CPU/offline (if uploaded)

Load

from huggingface_hub import hf_hub_download
import torch
ckpt = torch.load(hf_hub_download("LNTTushar/perception-slm-image-v0", "model.pt"), map_location="cpu")
# rebuild ImageVLM.from_config(config) then load_state_dict(ckpt["model_state"])

Built with the perception-slm repo; see its RESULTS.md.