dronefreak/visdrone-rtdetrv4-m

RT-DETRv4-M Finetuned on VisDrone

Fine-tuned RT-DETRv4-M object detector on the VisDrone benchmark dataset, trained and evaluated with the RT-DETRv4 codebase -- a real-time detection transformer whose lightweight backbone/encoder is distilled from a frozen DINOv3 vision foundation model during training only (the teacher is not needed at inference).

_{Demo clips: VisDrone2019-VID val sequences uav0000117_02622_v (left, night, 349 frames) and uav0000182_00000_v (right, day, first 349 of 363 frames); detections at confidence >= 0.35. For illustration only; not part of the reported test-set metrics.}

---

Performance

Metric	Score (%)
mAP@50	46.95
mAP@50-95	27.81
Precision	55.62
Recall	48.95
F1 Score	52.07
Parameters	19.4M
FLOPs	122.4B

---

Evaluation Protocol

Metrics are computed on the VisDrone test split (1610 images) with this repository's own validation pipeline (tools/model_card/generate_model_card.py): the EMA weights, 960x960 input resize, and COCO-style evaluation (faster-coco-eval) with at most 100 detections per image.

• mAP@50 / mAP@50-95 / per-class AP are the standard COCO-eval numbers.
• Precision / Recall / F1 are taken at IoU 0.5 at the single confidence threshold that maximises the class-averaged F1 (here 0.378), following the Ultralytics convention. They come from exact true/false-positive counts of the raw detections (per-class greedy matching in descending score order, the COCO-eval rule). Classes with no ground truth in the split are excluded from every average.
• Confusion matrices use greedy one-to-one IoU matching at confidence >= 0.25 and IoU >= 0.45.
• FLOPs / parameters are measured on the deploy-mode (fused) model at 960x960.

Precision/Recall depend on the operating-point convention (here: max class-averaged F1), so they can differ from the same metric in cards produced by pipelines that use a fixed confidence threshold (e.g. Supervision-based ones); mAP is directly comparable.

---

VisDrone Model Zoo

Every RT-DETRv4 model fine-tuned and evaluated on VisDrone so far.

Model	mAP@50	mAP@50-95	Precision	Recall
RT-DETRv4-L	48.32	28.89	56.37	50.71
RT-DETRv4-M	46.95	27.81	55.62	48.95
RT-DETRv4-S	46.28	27.36	55.42	49.17

---

Per-Class Performance

Class	mAP@50	mAP@50-95
pedestrian	44.04	19.3
people	32.43	12.43
bicycle	25.37	12.25
car	83.2	53.94
van	49.49	34.46
truck	54.31	36.87
tricycle	32.86	18.62
awning-tricycle	28.3	17.61
bus	71.11	51.13
motor	48.42	21.52
others	0.0	0.0

Classes with no ground-truth instances in the evaluated split score 0 and are excluded from the mAP average.

---

Evaluation Visualizations

Precision-Recall Curve

F1 Curve

Confusion Matrix

Normalized Confusion Matrix

Training Curves

---

Dataset

This model was trained on VisDrone. For the full dataset description, provenance, license, and citation, see the dataset card:

https://huggingface.co/datasets/Voxel51/VisDrone2019-DET

Classes

• 0: pedestrian
• 1: people
• 2: bicycle
• 3: car
• 4: van
• 5: truck
• 6: tricycle
• 7: awning-tricycle
• 8: bus
• 9: motor
• 10: others

---

Usage

The weights (model.pth, EMA only) and the fully-resolved training config (config.yaml) are loaded by the RT-DETRv4 codebase; there is no pip package. The DINOv3 teacher used during training is not required for inference.

Install

git clone https://github.com/dronefreak/RT-DETRv4
cd RT-DETRv4
pip install -r requirements.txt opencv-python huggingface_hub

Download the Weights

from huggingface_hub import snapshot_download

local_dir = snapshot_download(repo_id="dronefreak/visdrone-rtdetrv4-m")  # model.pth + config.yaml
print(local_dir)

Run Inference

python tools/inference/torch_inf.py \
    -c <local_dir>/config.yaml \
    -r <local_dir>/model.pth \
    --input image.jpg --device cuda:0   # or a video file

Export to ONNX

pip install onnx onnxsim onnxscript
python tools/deployment/export_onnx.py --check -c <local_dir>/config.yaml -r <local_dir>/model.pth
python tools/inference/onnx_inf.py --onnx <local_dir>/model.onnx --input image.jpg

---

Training Configuration

Setting	Value
Dataset	VisDrone
Framework	RT-DETRv4
Training Toolkit	RT-DETRv4 codebase
Epochs (configured max)	102
Epochs (actually trained)	102
Batch Size	6
Resolution	960x960
Optimizer	AdamW
Learning Rate	7.5e-05
Weight Decay	0.0001
LR Schedule	flatcosine
Distillation Teacher	DINOv3TeacherModel (dinov3_vitb16_pretrain_lvd1689m.pth)
Released Checkpoint	best_stg2.pth (EMA weights, saved at epoch 97)
Initialized From	RT-DETRv4-M (COCO)
Seed	0

config.yaml in this repository is the complete merged training config (all __include__s resolved, machine-local paths redacted).

---

Repository Contents

model.pth
config.yaml
metrics.json
BoxF1_curve.png
BoxPR_curve.png
DINOv3_LICENSE.md
assets/demo_banner.mp4
assets/demo_banner_poster.jpg
confusion_matrix.csv
confusion_matrix.png
confusion_matrix_normalized.png
log.txt
per_class_metrics.csv
training_curves.png
visdrone_rtdetrv4-m_showcase.jpg
README.md

---

Related Resources

• VisDrone dataset card on Hugging Face
• RT-DETRv4 codebase -- code used to train and evaluate this checkpoint
• RT-DETRv4 paper (arXiv:2510.25257) and official repository

---

Known Limitations

• Severe class imbalance: car (42.21%) and pedestrian (23.12%) account for two-thirds of all annotated boxes in the training set, while awning-tricycle (0.95%) and tricycle (1.40%) are rare -- the others class has zero annotated instances in the training set entirely and is effectively unusable (always 0 AP).
• Extreme small-object density: ~53 annotated boxes per image on average, with roughly 69% of boxes covering under 0.1% of the image area -- consistent with VisDrone's aerial small-object detection challenge (objects captured from significant altitude). Inputs are resized to a fixed square resolution, which shrinks these objects further.
• The original authors license VisDrone under CC BY-NC-SA 3.0 -- non-commercial research use only (see the dataset's homepage); this applies to any model trained on it, not only the raw images.

---

License and Attribution

• This model (weights and code): Apache-2.0, as RT-DETRv4.
• Distillation teacher: during training the model learns from a frozen DINOv3 teacher, which is not part of this repository and is not needed for inference. DINOv3 is distributed by Meta under the DINOv3 License (Meta); a copy is included here as DINOv3_LICENSE.md, and this work acknowledges its use of DINOv3 (see Citation).
• Training data: VisDrone carries its own terms; see Known Limitations and the dataset card.

---

Citation

If you use this model in your research, please consider citing:

1. The VisDrone dataset
2. The RT-DETRv4 paper
3. DINOv3, the vision foundation model used as the distillation teacher

@article{zhu2018vision,
  title={Vision meets drones: A challenge},
  author={Zhu, Pengfei and Wen, Longyin and Bian, Xiao and Ling, Haibin and Hu, Qinghua},
  journal={arXiv preprint arXiv:1804.07437},
  year={2018}
}

@article{liao2025rtdetrv4,
  title={RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models},
  author={Zijun Liao and Yian Zhao and Xin Shan and Yu Yan and Chang Liu and Lei Lu and Xiangyang Ji and Jie Chen},
  journal={arXiv preprint arXiv:2510.25257},
  year={2025}
}

@article{simeoni2025dinov3,
  title={DINOv3},
  author={Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and Oquab, Maxime and Jose, Cijo and Khalidov, Vasil and Szafraniec, Marc and Yi, Seungeun and Ramamonjisoa, Micha{\"e}l and Massa, Francisco and Haziza, Daniel and Wehrstedt, Luca and Wang, Jianyuan and Darcet, Timoth{\'e}e and Moutakanni, Th{\'e}o and Sentana, Leonel and Roberts, Claire and Vedaldi, Andrea and Tolan, Jamie and Brandt, John and Couprie, Camille and Mairal, Julien and J{\'e}gou, Herv{\'e} and Labatut, Patrick and Bojanowski, Piotr},
  journal={arXiv preprint arXiv:2508.10104},
  year={2025}
}