VITA-RL: LoRA adapters from the GRPO/SFT experiments

LoRA adapters (r=64) trained on top of VITA-MLLM/VITA-1.5 as part of the VITA-RL project — a fork of VITA-1.5 that adds a from-scratch multimodal RL stack (DPO + GRPO) and a fully documented experiment record.

Each subfolder is one training run. Merging any adapter into the base model with tools/merge_and_eval.py exactly reproduces the evaluated checkpoint for that run.

Contents

Folder	Run	Training	Key result
r4_grpo_clevr/	R4	GRPO on CLEVR counting, verifiable reward (answer + format), 400 steps, β=0.04	held-out accuracy 44.6% → 77.4%, zero regression on MME/POPE/MMBench
sft_clevr_control/	SFT control	LoRA SFT on the same 6,400 prompts with gold answers (matched data budget)	held-out ≈ GRPO in-distribution; 63.0% vs 54.5% over GRPO on SuperCLEVR OOD
r5_grpo_arm/	R5 arm B	GRPO from the SFT checkpoint on disjoint stage-2 data	no gain over SFT start — task ceiling ~77–78%
r5_sft2_arm/	R5 arm C	Continued SFT from the same checkpoint, same budget	matches the GRPO arm — confirms the ceiling
r6_beta0/	R6	R4 ablation with β=0 (no KL term)	accuracy indistinguishable from R4 — KL is unnecessary with verifiable rewards + single-step on-policy

Each folder contains adapter_config.json, adapter_model.safetensors (~309 MB), non_lora_trainables.bin and config.json.

Usage

git clone https://github.com/eternity-blog/VITA-RL && cd VITA-RL
# base weights: VITA-MLLM/VITA-1.5 + OpenGVLab/InternViT-300M-448px
# environment: see docs/01-setup/ENVIRONMENT.md

python tools/merge_and_eval.py \
    --base    /path/to/VITA-1.5 \
    --adapter /path/to/this-repo/r4_grpo_clevr \
    --out     /path/to/merged

Where the numbers come from

• Full experiment record: GRPO_DEEP_DIVE.md and EXPERIMENT_LOG.md
• Raw training logs / per-step trainer state / eval outputs: artifacts/ in the GitHub repo
• Training data: leonardPKU/clevr_cogen_a_train; OOD eval: jigsaw-r1/super_clevr

Note: the DPO-era adapters (six rounds on openbmb/RLAIF-V-Dataset, POPE hallucination 10.97% → 8.82% via SFT-then-DPO) were lost with an earlier dev machine; their full record survives in EXPERIMENT_LOG.md §3–§9.