Q-ViK: Question-Guided Visual KV Cache Eviction for Large Vision-Language Models

Solhee Hwang, Jangho Kim* · Kookmin University (*corresponding author)

NeurIPS 2026 · Code: https://github.com/ssoree912/Q-ViK

Student scorer checkpoints for Q-ViK, a question-conditioned visual KV cache eviction method for large vision-language models (LLaVA-OneVision, LLaVA-1.5). Offline, full-cache decoding gives an answer-to-visual attention signal that marks the visual KV entries future answer tokens actually reuse, and a lightweight question-conditioned scorer is trained to predict it from prefill representations alone. At inference, a single post-prefill scoring step evicts low-utility visual KV entries while keeping every text KV entry, with no full-cache lookahead or generated traces. See the paper for details.

Checkpoints

Folder	Base model	Training data	Epochs
qvik_student_onevision/	lmms-lab/llava-onevision-qwen2-7b-ov	TextVQA, GQA, ScienceQA (600 each)	20
qvik_student_llava15/	liuhaotian/llava-v1.5-7b	TextVQA, ScienceQA, GQA (600 each)	15

Each folder contains the student weights (pytorch_model.bin), its architecture config (config.json), the training config (train_config.json) and the training log (train_log.jsonl). These files contain only the student scorer; the base VLM must be downloaded separately.

Usage

Download into the ckpts/ directory of the code repository:

huggingface-cli download ssoree912/Q-ViK --local-dir ckpts --include "qvik_student_*"

Then evaluate with the code repository, e.g. for OneVision:

python qvik/eval/run_lmms_eval.py \
  --model lmms_onevision_student \
  --model_args pretrained=model/llava-onevision-qwen2-7b-ov,student_path=ckpts/qvik_student_onevision,keep_ratio=0.5,device=cuda:0 \
  --tasks textvqa \
  --batch_size 1 \
  --output_path results

Citation

Coming soon.