Aspaklarya-107-step1770

LoRA adapter for Qwen/Qwen3-VL-8B-Instruct, round-1XX of the Aspaklarya programme.

This is a verbose-answer model: unlike the 0XX round it answers VQA with a few sentences of grounded reasoning rather than a single word.

Experiment variable. Prompt holdout: caption queries split train/test to test prompt overfitting.

Training

experiment	exp107
checkpoint	step 1770
VQA column	VQA_1126
train split	train
corpus	18,862 items / 6,288 images (6,288 caption + 12,574 VQA)
per image	2 QA + 1 caption
LoRA	r=16, alpha=16, dropout=0.0, vision tower trained
loss	token weighting, w_caption=1.0, w_vqa=1.0
optimiser	lr 0.0001, cosine schedule, warmup 0.1, wd 0.001, clip 0.3
batch	4 x 8 grad-accum = 32 effective
epochs	3.0
seq len / max side	2048 / 1024
seed	3407

Evaluation

LLM-as-judge (gemma-4-31b-it-imiai), greedy decoding, temperature 0, scored 0-100. Every model in the programme is judged on one fixed sample drawn with seed 1105, so the numbers below are directly comparable across checkpoints and against the baselines.

slice	prompt set	new tokens	n	score
VQA DM	VQA_1126 (verbose)	160	500	72.16
VQA RSICD	legacy terse	48	500	69.9
Caption DM	caption-query bank	320	491	80.98
Caption RSICD	caption-query bank	320	500	77.06
MEAN4			1,991	75.03

Rank on MEAN4: 42 of 70 scored checkpoints in the programme.

Reference points on the same sample (MEAN4): Aspaklarya-101-step4500 76.72, Aspaklarya-106-soup 76.36, Aspaklarya-020-step4500 74.90, DeanQwen3-8B 64.84, stock Qwen3-VL-8B-Instruct 61.34.

How much of this number is real

The programme measured its own noise floor by training the same recipe three times under different seeds and judging every checkpoint identically:

quantity	value
control MEAN4	75.52 (sd 0.19, n=3)
sigma, VQA DM	0.88
sigma, VQA RSICD	0.74
sigma, Caption DM	0.25
sigma, Caption RSICD	0.52
judge reproducibility	~0.01
same-seed rerun spread	0.30

Three things follow, and they apply to every number on this page:

• MEAN4 differences below ~0.5 are not interpretable from a single pair of runs, and single-slice VQA differences below ~1.8 are not either. The caption slices are three to four times quieter than the VQA slices.
• The judge is not the noise. At ~0.01 it is effectively deterministic, so the spread is real training variance, not measurement drift.
• Re-running with the seed held fixed still moves the score by ~0.30, due to non-deterministic CUDA kernels. Exact replication is not achievable, so a recipe is better described by its mean across seeds than by its best run.

Picking the highest-scoring checkpoint of several and quoting that is selection on noise. Where a sibling checkpoint of this run scored higher, it is named below rather than hidden.

Caveats

• The judge is text-only. It compares an answer with a reference and never sees the image, so it detects unsupported specificity but not verified falsehood. Pixel-level correctness needs human spot-checks.
• The legacy terse benchmark is not a fair target for the 1XX round. Its references are ~4 characters and its rubric caps a padded-but-correct answer at 89/100 by construction, so a verbose model cannot reach the top band however accurate it is. A detail-prompt ablation showed the 0XX models cannot close the content gap when asked for detail (3.200 -> 3.249), so the 1XX round's lead is substantive and its legacy dip is largely cosmetic.
• Scores are in-domain. This is a remote-sensing model and nothing here measures general-purpose VQA.

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base = "Qwen/Qwen3-VL-8B-Instruct"
model = AutoModelForImageTextToText.from_pretrained(base, dtype="auto",
                                                    device_map="auto")
model = PeftModel.from_pretrained(model, "ExperimentField/Aspaklarya-107-step1770")
processor = AutoProcessor.from_pretrained(base)

Programme context

This repository is one checkpoint of a systematic ablation programme. The organisation card lists every experiment, the hyperparameters that changed, the measured noise floor, and the results that were retracted after a control error was found: https://huggingface.co/ExperimentField.

Adapters are published only for configurations that are worth using or that a published result depends on. Refuted configurations and calibration runs generally have no repository.