LSVD: Precomputed Calibration Caches

LSVD: Loss-Aware Low-Rank Approximation for Efficient Low-Precision Vision-Language Models (NeurIPS 2026)

Haiyu Wang, Yutong Wang*, Leshu Li*, Yihui Ren, Sai Qian Zhang  ·  *Equal contribution

This repository holds the calibration statistics and hybrid-FFN channel groups of the LSVD W8A8 runs for LLaVA-v1.5 7B, LLaVA-Next 7B and LLaVA-Next 13B. With them, scripts/lsvd/w8a8.sh skips the calibration step (collecting activations and gradients on ScienceQA train), the most expensive part of a run.

⚡ Quick start

# all three models (17.8 GB)
hf download Etropyyy/LSVD --local-dir lsvd_artifacts
# or one model, e.g. LLaVA-v1.5 7B (4.5 GB)
hf download Etropyyy/LSVD --include "calib_cache/llava15_7b/*" "groups/*" --local-dir lsvd_artifacts

# in a clone of https://github.com/SAI-Lab-NYU/LSVD (see its README for installation and data)
export LSVD_CALIB_CACHE_DIR=$PWD/lsvd_artifacts/calib_cache
export LSVD_GROUPS_DIR=$PWD/lsvd_artifacts/groups
MODEL=llava15_7b EVAL_DATASET=ScienceQA_TEST bash scripts/lsvd/w8a8.sh

w8a8.sh reads the caches read-only and runs with the calibration seed they were computed with. A run whose calibration settings differ from calib_cache.json stops with an error.

📦 Contents

calib_cache/
├── llava15_7b/             LLaVA-v1.5 7B (4.5 GB)
│   ├── act_input.pt        Cholesky factors of the Q/K/V input covariances (A8 quantization-aware whitening)
│   ├── output_grad.pt      per-head Cholesky factors of the output-gradient covariances
│   ├── grad_info.pt        per-head gradient (Fisher) information
│   ├── grad_scores.pt      importance scores for the cross-layer rank allocation
│   └── calib_cache.json    calibration settings and sha256 of the four files
├── llavanext_7b/           LLaVA-Next 7B (4.5 GB), same files
└── llavanext_13b/          LLaVA-Next 13B (8.7 GB), same files
groups/                     hybrid-FFN channel groups (SVD / RTN split and channel permutation)
├── hybrid_groups_llava15_7b.json     LLaVA-v1.5 7B
├── hybrid_groups_llavanext_7b.json   LLaVA-Next 7B
└── hybrid_groups_llavanext_13b.json  LLaVA-Next 13B

📝 Notes

• The .pt files are PyTorch pickles read with torch.load.
• Scores may differ with the GPU type and software versions.

📚 Citation

@inproceedings{wang2026lsvd,
  title     = {{LSVD}: Loss-Aware Low-Rank Approximation for Efficient Low-Precision Vision-Language Models},
  author    = {Wang, Haiyu and Wang, Yutong and Li, Leshu and Ren, Yihui and Zhang, Sai Qian},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}

📄 License

Apache License 2.0.