See our collection for all versions of RT-DETR.

Run RT-DETR with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/rtdetr-r101vd

Paper: DETRs Beat YOLOs on Real-time Object Detection (arXiv:2304.08069) · HF Papers

RT-DETR was the first DETR-style detector to beat YOLO on the real-time speed/accuracy tradeoff. It pairs a ResNet-vd backbone with a hybrid encoder that decouples intra-scale attention from cross-scale fusion, then feeds IoU-aware selected queries into a deformable decoder. It is NMS-free: a fixed set of queries, constant inference cost, no NMS threshold to tune.

For more details on the model, please go to PekingU's original model card.

Pure-Keras 3 conversion of PekingU/rtdetr_r101vd for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is an object detection checkpoint (RTDETRDetect) on COCO (ResNet-101-vd).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.rt_detr import RTDETRDetect, RTDETRImageProcessor

model = RTDETRDetect.from_weights("kerasformers/rtdetr-r101vd")
processor = RTDETRImageProcessor.from_weights("kerasformers/rtdetr-r101vd")

image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
    output, threshold=0.5, target_sizes=[(image.height, image.width)]
)[0]
for score, name, box in zip(
    results["scores"], results["label_names"], results["boxes"]
):
    print(f"{name}: {float(score):.3f} {box}")

Load any RT-DETR v1 variant the same way with from_weights("kerasformers/<variant>"):

Variant	Hub	Backbone
`rtdetr-r18vd`	`kerasformers/rtdetr-r18vd`	ResNet-18-vd
`rtdetr-r18vd-coco-o365`	`kerasformers/rtdetr-r18vd-coco-o365`	ResNet-18-vd (COCO+O365)
`rtdetr-r34vd`	`kerasformers/rtdetr-r34vd`	ResNet-34-vd
`rtdetr-r50vd`	`kerasformers/rtdetr-r50vd`	ResNet-50-vd
`rtdetr-r50vd-coco-o365`	`kerasformers/rtdetr-r50vd-coco-o365`	ResNet-50-vd (COCO+O365)
`rtdetr-r101vd`	`kerasformers/rtdetr-r101vd`	ResNet-101-vd
`rtdetr-r101vd-coco-o365`	`kerasformers/rtdetr-r101vd-coco-o365`	ResNet-101-vd (COCO+O365)

Tips

Set KERAS_BACKEND before importing Keras / kerasformers.
RTDETRImageProcessor keeps do_normalize=False by default (rescaled [0, 1] input, matching upstream).
See RT-DETR docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. RTDETRDetect.from_weights("hf:PekingU/rtdetr_r101vd").