ReefNet/reefnet-mllm-lora-extras

ReefNet LoRA adapters: few-shot, variance and rebuttal runs

Supplementary LoRA adapters from the ReefNet paper (NeurIPS 2026, Evaluations & Datasets track) and its rebuttal. The main fine-tuned models have their own repos: Qwen3-VL, MiniCPM-V 4.5, InternVL3.5.

Each subfolder is a self-contained PEFT adapter with its train_config.yaml, trainer logs and predictions/. Few-shot support sets are nested: k=1 is a subset of k=5, which is a subset of k=10 and of k=20. They are sampled from the ReefNet train split, and the sample lists are in the reefnet-mllm repository. The LLaVA-NeXT adapter inherits the Llama 2 licence of its base model. The other adapters are Apache-2.0.

subfolder	base model	what	ReefNet	Al-Wajh
few_shot/k1/qwen3_vl	Qwen3-VL-8B-Instruct	Table 4, 1-shot	5.56	10.93
few_shot/k1/minicpm_v	MiniCPM-V 4.5	Table 4, 1-shot	2.85	7.68
few_shot/k1/internvl3_5	InternVL3.5-8B	Table 4, 1-shot	2.00	10.31
few_shot/k5/qwen3_vl	Qwen3-VL-8B-Instruct	Table 4, 5-shot	9.15	13.77
few_shot/k5/minicpm_v	MiniCPM-V 4.5	Table 4, 5-shot	7.32	8.52
few_shot/k5/internvl3_5	InternVL3.5-8B	Table 4, 5-shot	2.87	9.01
few_shot/k10/qwen3_vl	Qwen3-VL-8B-Instruct	Table 4, 10-shot	17.24	25.87
few_shot/k10/minicpm_v	MiniCPM-V 4.5	Table 4, 10-shot	14.95	12.77
few_shot/k10/internvl3_5	InternVL3.5-8B	Table 4, 10-shot	10.44	12.80
few_shot/k20/qwen3_vl	Qwen3-VL-8B-Instruct	Table 4, 20-shot	27.22	31.27
few_shot/k20/minicpm_v	MiniCPM-V 4.5	Table 4, 20-shot	21.95	20.88
few_shot/k20/internvl3_5	InternVL3.5-8B	Table 4, 20-shot	18.34	12.87
seed_variance/qwen3_vl_cap300_seed1	Qwen3-VL-8B-Instruct	rebuttal: run-to-run variance, class-capped subsample (<=300/genus)	50.96	34.72
seed_variance/qwen3_vl_cap300_seed2	Qwen3-VL-8B-Instruct	rebuttal: run-to-run variance, class-capped subsample (<=300/genus)	50.45	34.68
seed_variance/qwen3_vl_cap300_seed3	Qwen3-VL-8B-Instruct	rebuttal: run-to-run variance, class-capped subsample (<=300/genus)	50.16	32.66
convergence/qwen3_vl_cont_2p5ep	Qwen3-VL-8B-Instruct	rebuttal: +0.5 epoch continuation	72.75	56.94
llava/llava_next_13b_ckpt14000	LLaVA-NeXT (Vicuna-13B)	rebuttal: LLaVA-NeXT, step 14000	61.70	38.62
llava/llava_onevision_7b_ckpt26000	LLaVA-OneVision-7B	rebuttal: LLaVA-OneVision, step 26000, 3 GPUs (effective batch 24)	60.50	35.39

Base model revisions: Qwen/Qwen3-VL-8B-Instruct@0c351dd0, openbmb/MiniCPM-V-4_5@c1a6986b, OpenGVLab/InternVL3_5-8B-HF@741a7d03, llava-hf/llava-v1.6-vicuna-13b-hf@745cfbdb, llava-hf/llava-onevision-qwen2-7b-ov-hf@0d506805

Usage (example: 20-shot Qwen3-VL)

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base, rev = "Qwen/Qwen3-VL-8B-Instruct", "0c351dd01ed87e9c1b53cbc748cba10e6187ff3b"
model = AutoModelForImageTextToText.from_pretrained(base, revision=rev, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "ReefNet/reefnet-mllm-lora-extras", subfolder="few_shot/k20/qwen3_vl").eval()
processor = AutoProcessor.from_pretrained(base, revision=rev)

# input: a 1024x1024 crop centred on the annotated point
messages = [{"role": "user", "content": [
    {"type": "image", "image": "crop.png"},
    {"type": "text", "text": "What is the genus in the center of the image? Answer with one word"}]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                       return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

During training the image token sat inside the prompt text (see Prompts below). This snippet puts the image first, which is how the chat template inserts it.

Training

LoRA fine-tuning with LLaMA-Factory @ 589da21d. The exact config is in train_config.yaml: rank 8 on all linear layers, AdamW, lr 5e-5, cosine schedule, warmup ratio 0.03, bf16, per-device batch 8, no gradient accumulation, image_max_pixels 1024x1024, no augmentation. Software: transformers 4.57.1, peft 0.18.1, torch 2.9.1. Model selection used the ReefNet validation split. Al-Wajh is never used for training, tuning or selection.

Prompts (exact strings, <image> marks the image position)

split	user prompt	target
train / val / ReefNet test	What is the genus in the center of the image <image>? Answer with one word	genus name
Al-Wajh test	<image>Identify the hard coral genus in this image. Respond with only the genus name.	genus name

Decoding: greedy (top_k=1), max_new_tokens=32. The output is normalised (strip, capitalise) and must match a genus exactly.

Metric

Macro recall (balanced accuracy). ReefNet: 39 labels, 23,043 test crops. Al-Wajh: averaged over the 13 genera present (4,624 crops), while the model keeps the full 39-way output. A prediction outside the 13 counts as an error. predictions/ holds the per-sample outputs behind the reported numbers. Recompute them with tools/recompute_macro_recall.py from the reefnet-mllm repository.

Data

ReefNet-1.0: 1024x1024 crops centred on each point annotation. Training uses the high-confidence benchmark split (445,985 train / 9,999 val).

Citation

@inproceedings{felemban2026reefnet,
  title     = {ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition},
  author    = {Felemban, Abdulwahab and Battach, Yahia and Khan, Faizan Farooq and Fu, Yuqian and Liu, Xuhui and
               Khattab, Yesmeen and Radwan, Yousef A. and Li, Xiang and Dunn, Natalie and B{\"a}hr, Susanne and
               Terraneo, Tullia Isotta and Marchese, Fabio and Beery, Sara and Jones, Burton H. and
               Benzoni, Francesca and Elhoseiny, Mohamed},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS) Evaluations and Datasets Track},
  year      = {2026}
}