Aspaklarya-121-vqa-step3000

Tested and rejected. This adapter is published as a record of a refuted hypothesis, not as a recommended model. The standing recommendation remains Aspaklarya-101-step4500.

The VQA half of exp121, a task-routed two-expert study. One LoRA was trained on VQA only and a second on captions only (ExperimentField/Aspaklarya-122-cap-*), to test whether the two tasks compete for adapter capacity when trained as one mixture.

What it was testing

Every prior Aspaklarya run trained VQA and captioning in a single adapter. If those objectives interfere, splitting them should let each expert exceed the mixture. This arm removes captions entirely: 23,986 VQA items, 2 QA per image, 0 captions.

Exposure is matched rather than step-matched. exp101 ran 4 epochs over the full 35,979-item mixture, so each VQA item was seen 4x. This arm also runs 4 epochs over its own subset, so each item is again seen 4x and batch composition is the only difference.

Result: null

checkpoint	VQA_DM	VQA_RSICD
step1500 (2 ep)	74.23	68.95
step2250 (3 ep)	74.42	69.11
step3000 (4 ep)	74.22	69.47
control (3-seed mean)	72.82	70.15
exp101-step4500	74.16	70.56
pre-registered win gate	>= 75.96	-

Flat across epochs 2-4 (spread 0.20 against a per-slice sigma of 0.88), landing at exp101 parity on VQA_DM and consistently below the control on VQA_RSICD. Removing captions bought no VQA accuracy, so task interference is not what caps VQA performance in this setup.

A secondary observation: the off-task caption slices rose monotonically across training (CAP_DM 68.09 -> 68.85 -> 69.01) despite zero caption data, so the VQA objective alone sustains some captioning ability. The absolute caption numbers still collapse relative to a caption-trained adapter (CAP_RSICD 34.11 against ~80), which is the expected off-task half of the interference measurement.

This result also reframes exp120curB, the one configuration that did show +2.66 on VQA_DM: curB resumed from a caption-trained adapter, while this arm trains from scratch. Since pure-VQA data alone does not reproduce the gain, the caption-pretrained initialisation is the more likely active ingredient.

Training

base	Qwen/Qwen3-VL-8B-Instruct
data	VLR_Dataset ImageTable, VQA_1126 column, train+other splits
items	23,986 (2 QA per image, 0 captions)
epochs	4 (3,000 steps, save every 750)
LoRA	r=16, alpha=16, dropout 0, vision tower trained
optim	lr 2e-4 linear, warmup 0.03, wd 0.001, grad clip 0.3
batch	4 x 8 grad accum, seq 2048, max image side 1024
seed	3407
hardware	1x A100 40GB, ~6.5 h

Evaluation

500 held-out validation items per slice, fixed seed across all experiments. Generations are produced on both harnesses, then graded by an LLM judge (gemma-4-31b-it-imiai) for agreement with the reference answer. The judge is text-only: it measures agreement with references, not pixel truth.

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3-VL-8B-Instruct", dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "ExperimentField/Aspaklarya-121-vqa-step3000")
proc = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")