Wing22/lvr-sft-v3-4000

LVR SFT v3 — step 4000

This bundle exports the author's sft/v3/checkpoints/01-4000.ckpt for use with LVR Pipeline. It combines Qwen3-VL-4B-Instruct, a feature-space latent action model (LAM), and a two-layer latent projector.

The three BF16 safetensors shards include the whole evaluated model, including frozen parameters. A separate lam/lam.ckpt preserves the original LAM parameters in FP32 for reuse in alignment and other experiments. The bundle contains no optimizer or scheduler state and does not provide exact mid-run training resumption.

Loading

Install the accompanying code repository and its dependencies, then download the bundle and run:

python scripts/download_model.py --repo-id Wing22/lvr-sft-v3-4000
python -m lvr.infer \
  --model-dir models/lvr-sft-v3-4000 \
  --image /path/to/image.jpg \
  --question 'Describe the main objects in the image.'

from lvr.bundle import load_bundle
model, processor = load_bundle("models/lvr-sft-v3-4000", device="cuda:0")

Browse all model files or download the independent LAM checkpoint. The complete safetensors model requires all three shards, the index, lvr_config.json and the qwen/ resources.

This is a custom LVR bundle, not an AutoModel.from_pretrained Transformers model. lvr_config.json describes the LVR architecture and local paths. qwen/ supplies the Qwen configuration, tokenizer, processor and chat template, including the three latent special tokens. Inference needs neither DeepSpeed nor a separate base-model weight download.

Recorded training setup

• Base model: Qwen3-VL-4B-Instruct.
• Frozen feature-space LAM: model dimension 1024, latent dimension 32, four latent tokens, 16 encoder/decoder blocks, 16 heads.
• Latent projector: MLP with hidden dimension 1024.
• Alignment initialized SFT at alignment step 1000.
• SFT tuned the Qwen language model, LM head and projector; LAM and vision parameters remained frozen.
• Released checkpoint: epoch 1, global step 4000.
• SFT effective batch size: 32; learning rates 1e-6; BF16 mixed precision; latent loss weight 1.0.
• SFT training image budget: 2,408,448 pixels; benchmark smoke evaluation used the processor default budget.

Training JSON and pinned image-download/reconstruction scripts are distributed with the code. This upload does not include training images or benchmark datasets.

Verification and evaluation scope

All 1,602 exported tensors were read back and compared exactly at the export dtype. Independent LAM weights were checked against the original LAM and the LAM inside the SFT checkpoint. Standalone LAM inference produced finite [4,32] latents identical to the bundled LAM. Offline image-question inference and two examples from each of six benchmark adapters completed. Two VSTAR predictions exactly matched the original checkpoint implementation.

These smoke tests used max_new_tokens=64; they are functional checks, not full benchmark results. The accompanying repository preserves the old full-run summary separately and clearly labels it historical.

Benchmark exposure: recorded LAM pretraining includes benchmark-derived image pairs and auxiliary boxes, including some answer-conditioned box annotations. Historical benchmark scores therefore do not represent an entirely held-out evaluation. See the code repository's data provenance for counts and sources. Its MMVP adapter reports per-question accuracy, not official pair accuracy; BLINK covers five selected configurations.

Files and provenance

Prepared code revision: 3cc6ddef58a6ef1654bcd902bda277c4bd7b0dfb. Recorded base-model revision: ebb281ec70b05090aa6165b016eac8ec08e71b17.

• model-*.safetensors and index: complete BF16 model parameters.
• lam/lam.ckpt: standalone LAM parameters without optimizer state.
• qwen/: architecture and processor resources.
• export_manifest.json: source checkpoint hashes, exported file hashes and tensor validation results.
• SHA256SUMS: checksums of the finalized bundle files.

The model was verified on an NVIDIA A100-SXM4-80GB with Python 3.12, PyTorch 2.5.1 and Transformers 5.15.0. Fresh dependency installation, retraining and full benchmark evaluation were not performed during release preparation. No separate release license has been specified for the exported weights; upstream model and dataset terms remain applicable.