leonardosalem/qwen-vl-lingoqa-vqa

Qwen-VL QLoRA adapters — driving VQA on LingoQA

LoRA adapters from a master's thesis on parameter-efficient adaptation of vision-language models for autonomous driving.

• Task: Video visual question answering for autonomous driving
• Dataset: LingoQA (500-sample evaluation split)
• Metric: LingoJudge accuracy (%)
• Method: QLoRA — 4-bit NF4 frozen base, LoRA rank 16, alpha 32, dropout 0.05, lora_target: all, paged_adamw_8bit, cosine schedule

Results

Subfolder	Base model	Checkpoint	LingoJudge
qwen3-8b	Qwen/Qwen3-VL-8B-Instruct	checkpoint-2400	63.2
qwen3-4b	Qwen/Qwen3-VL-4B-Instruct	checkpoint-2350	61.8
qwen2.5-3b	Qwen/Qwen2.5-VL-3B-Instruct	checkpoint-150	54.4
qwen3-2b	Qwen/Qwen3-VL-2B-Instruct	checkpoint-120	51.6

Qwen3-VL-8B at 63.2 improves on the original LingoQA baseline (60.8, Vicuna-7B) while remaining below later reported methods such as OpenREAD (68.2).

All figures are 4-bit evaluations. A bf16 evaluation of the same weights scores several points higher.

Contents

Each subfolder is a self-contained PEFT adapter (weights, config, tokenizer and processor files). Training-resume artefacts (optimizer, scheduler, RNG state) are excluded.

• qwen3-8b/ — from results/checkpoints/qwen3-8b-vqa/checkpoint-2400
• qwen3-4b/ — from results/checkpoints/qwen3-4b-vqa/checkpoint-2350
• qwen2.5-3b/ — from results/checkpoints/qwen2.5-3b-vqa/checkpoint-150
• qwen3-2b/ — from results/checkpoints/qwen3-2b-vqa/checkpoint-120

Usage

from peft import PeftModel
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

base = "Qwen/Qwen3-VL-4B-Instruct"
model = Qwen3VLForConditionalGeneration.from_pretrained(base, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "leonardosalem/qwen-vl-lingoqa-vqa", subfolder="qwen3-8b")
processor = AutoProcessor.from_pretrained(base)

Match the subfolder to its base model in the table above.

Reproducing

Training and evaluation code, configs and metrics: https://github.com/Leonardo-salem/thesis-vm-final

REPRODUCE.md in that repository gives the exact command, checkpoint, inference precision and sample count behind every number.