ReefNet/reefnet-internvl3.5-8b-lora

ReefNet LoRA adapter for InternVL3.5-8B

A LoRA adapter for OpenGVLab/InternVL3_5-8B-HF (revision 741a7d03), fine-tuned to identify the hard-coral genus at the centre of a reef photo crop (39 labels: 38 genera plus the family Fungiidae). This is the model reported in Table 5 of the ReefNet paper (NeurIPS 2026, Evaluations & Datasets track).

test set	macro recall (%)
ReefNet (within-source, 39 labels)	53.44
Al-Wajh, Red Sea (cross-source, 13 genera)	32.97

Usage

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base, rev = "OpenGVLab/InternVL3_5-8B-HF", "741a7d03020411e666c6109218ab71e08151ef86"
model = AutoModelForImageTextToText.from_pretrained(base, revision=rev, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "ReefNet/reefnet-internvl3.5-8b-lora").eval()
processor = AutoProcessor.from_pretrained(base, revision=rev)

# input: a 1024x1024 crop centred on the annotated point
messages = [{"role": "user", "content": [
    {"type": "image", "image": "crop.png"},
    {"type": "text", "text": "What is the genus in the center of the image? Answer with one word"}]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                       return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

During training the image token sat inside the prompt text (see Prompts below). This snippet puts the image first, which is how the chat template inserts it.

Training

LoRA fine-tuning with LLaMA-Factory @ 589da21d. The exact config is in train_config.yaml: rank 8 on all linear layers, AdamW, lr 5e-5, cosine schedule, warmup ratio 0.03, bf16, per-device batch 8, no gradient accumulation, image_max_pixels 1024x1024, no augmentation. Software: transformers 4.57.1, peft 0.18.1, torch 2.9.1. Model selection used the ReefNet validation split. Al-Wajh is never used for training, tuning or selection.

Prompts (exact strings, <image> marks the image position)

split	user prompt	target
train / val / ReefNet test	What is the genus in the center of the image <image>? Answer with one word	genus name
Al-Wajh test	<image>Identify the hard coral genus in this image. Respond with only the genus name.	genus name

Decoding: greedy (top_k=1), max_new_tokens=32. The output is normalised (strip, capitalise) and must match a genus exactly.

Metric

Macro recall (balanced accuracy). ReefNet: 39 labels, 23,043 test crops. Al-Wajh: averaged over the 13 genera present (4,624 crops), while the model keeps the full 39-way output. A prediction outside the 13 counts as an error. predictions/ holds the per-sample outputs behind the reported numbers. Recompute them with tools/recompute_macro_recall.py from the reefnet-mllm repository.

Data

ReefNet-1.0: 1024x1024 crops centred on each point annotation. Training uses the high-confidence benchmark split (445,985 train / 9,999 val).

Citation

@inproceedings{felemban2026reefnet,
  title     = {ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition},
  author    = {Felemban, Abdulwahab and Battach, Yahia and Khan, Faizan Farooq and Fu, Yuqian and Liu, Xuhui and
               Khattab, Yesmeen and Radwan, Yousef A. and Li, Xiang and Dunn, Natalie and B{\"a}hr, Susanne and
               Terraneo, Tullia Isotta and Marchese, Fabio and Beery, Sara and Jones, Burton H. and
               Benzoni, Francesca and Elhoseiny, Mohamed},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS) Evaluations and Datasets Track},
  year      = {2026}
}