UAV-PUR-8B

UAV-PUR-8B is a drone-image vision-language model fine-tuned from Qwen3-VL-8B-Instruct on Drone-PUR training data.

Paper | Drone-PUR Benchmark

The model addresses perception, understanding and reasoning tasks, including recognition, counting, grounding, spatial judgment, conditional judgment and single-/multi-image reasoning. It supports multiple-choice answers and open-ended box/count predictions.

Model Format

This repository contains the complete merged model in BF16 safetensors format, including the fine-tuned language weights, main visual merger and three DeepStack mergers. No separate base-model download or PEFT adapter loading is required. The processor and chat template are included.

UAV-PUR uses staged fine-tuning with language LoRA and trainable visual mergers. Training-only box, count and detection heads supply auxiliary supervision in the earlier stages; the final stage uses answer-token CE on mixed choice and open-ended samples. Inference uses the language-model output head.

Quickstart

Use a Qwen3-VL-compatible environment with transformers>=4.57, PyTorch, Accelerate and Pillow.

import torch
from PIL import Image
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "Henard224/UAV-PUR-8B"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto",
    attn_implementation="sdpa",
).eval()
processor = AutoProcessor.from_pretrained(model_id, max_pixels=1003520)

messages = [{"role": "user", "content": [
    {"type": "image", "image": Image.open("image.jpg").convert("RGB")},
    {"type": "text", "text": "Describe the visible objects in this drone image."},
]}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)
with torch.inference_mode():
    generated = model.generate(**inputs, max_new_tokens=128, do_sample=False)
answer_ids = generated[:, inputs["input_ids"].shape[1]:]
print(processor.batch_decode(answer_ids, skip_special_tokens=True)[0])

For Drone-PUR evaluation, use the benchmark's task prompts, ordered image inputs and reference-box preparation. MMR inputs are ordered RGB then TIR; MVR inputs are ordered view 1 then view 2. MIG uses temporally ordered frames with its reference target marked in the first frame.

License

The model weights are released under Apache-2.0. See LICENSE and NOTICE. The Drone-PUR dataset has its own CC BY-NC-SA 4.0 license.