TexasInstruments-EdgeAI/Deformable-DETR-Detection

Deformable DETR for TI EdgeAI

Deformable Attention for Fast-Converging Transformer Detection

---

Overview

Deformable DETR (Deformable Transformers for End-to-End Object Detection) is a transformer-based detector from SenseTime / fundamentalvision that addresses the slow convergence and limited feature resolution of the original DETR. Its key innovation is a deformable attention module that attends to only a small set of key sampling points around a reference point rather than all feature map positions, reducing complexity from O(H²W²) to O(HW).

This efficient attention mechanism makes it practical to use multi-scale feature maps, which improves detection accuracy — especially on small objects — while training in 10× fewer epochs than DETR. Five variants are provided, ranging from a lightweight single-scale model to a two-stage design with iterative bounding box refinement.

Deformable DETR uses 300 query slots (vs. 100 in DETR) and sigmoid focal loss for classification (no explicit background class); post-processing applies a score threshold rather than softmax + background filtering.

---

Model Variants

Model	Params	FLOPs	mAP[.5:.95]%	Validated Devices	Config
deformable_detr_single_scale	34M	78G	39.4	TDA4VH	deformable_detr_single_scale_config.yaml
deformable_detr_single_scale_dc5	34M	128G	41.5	N/A	N/A
deformable_detr	40M	173G	44.5	N/A	N/A
deformable_detr_plus_iterative_bbox_refinement	41M	173G	46.2	N/A	N/A
deformable_detr_two_stage	41M	173G	46.9	N/A	N/A

Recommended for edge deployment: deformable_detr (multi-scale, best accuracy/compute trade-off)

mAP values on COCO val2017. All variants use a ResNet-50 backbone pretrained on ImageNet, 800×800 input. The DC5 variant is disabled for TIDL deployment — TIDL does not support dilated convolutions in ResNet; its .onnx is provided for reference only. Only deformable_detr_single_scale currently ships with a validated TIDL config; the remaining variants have no *_config.yaml in this folder.

---

Quick Start

Prerequisites

# Core dependencies (auto-installed by prepare_model.py if missing)
pip install torch>=1.12.0 torchvision>=0.13.0 onnx>=1.14.0 scipy gdown>=5.2.0

# ONNX inference
pip install onnxruntime>=1.15.0

Export the Model

# List all available variants with accuracy and parameter info
python prepare_model.py --list-models

# Export the default model (deformable_detr - multi-scale, recommended)
python prepare_model.py

# Export a specific variant
python prepare_model.py --model deformable_detr_single_scale

# Export all variants (skips any already exported)
python prepare_model.py --model all

# Use HuggingFace Hub instead of Google Drive (recommended on corporate networks)
python prepare_model.py --method optimum --model all

The script automatically:

• Installs missing dependencies (torch, torchvision, onnx, scipy, gdown) if not present
• Clones the Deformable-DETR repository into ~/.cache/deformable_detr (or downloads weights from HuggingFace Hub with --method optimum)
• Installs a pure-Python fallback for the multi-scale deformable attention module — no CUDA compilation required
• Downloads pretrained COCO weights and builds the model with the correct architecture flags
• Exports to ONNX (opset 17 by default), simplifies the graph with onnx-simplifier, and fixes float64 nodes for TIDL compatibility
• Validates the exported graph and saves it as <model_key>.onnx

Compile and Infer uing edgeai-tidlrunner

Note: Run the commands below from inside the tidlrunner directory (the cloned edgeai-tidlrunner repository), with --config_path pointing to this model's config file.

Compile using edgeai-tidlrunner - on PC

cd /path/to/edgeai-tidlrunner
tidlrunner-cli compile --target_device J784S4 \
  --config_path /path/to/deformable_detr_single_scale_config.yaml

Run Inference Benchmark - on device

cd /path/to/edgeai-tidlrunner
tidlrunner-cli infer --target_device J784S4 \
  --config_path /path/to/deformable_detr_single_scale_config.yaml

Compile and Infer using edgeai-tidl-tools (Advanced):

Follow the instructions at https://github.com/TexasInstruments/edgeai-tidl-tools

Deploy using edgeai-tidl-tools:

Deplyment can be done using edgeai-tidl-tools. For ONNX models, onnxruntime-tidl with TIDL acceleration can be used. Consult the documentation of edgeai-tidl-tools for more details.

---

Citation

@article{zhu2020deformable,
  title   = {Deformable DETR: Deformable Transformers for End-to-End Object Detection},
  author  = {Zhu, Xizhou and Su, Weijie and Lu, Lewei and Li, Bin and
             Wang, Xiaogang and Dai, Jifeng},
  journal = {arXiv preprint arXiv:2010.04159},
  year    = {2020}
}

---

🔗 Resources

Resource	Link
Paper	arXiv:2010.04159
Source Code	fundamentalvision/Deformable-DETR
Dataset	COCO
edgeai-tidl-tools	GitHub
edgeai-tidlrunner	GitHub
EdgeAI SDK	Documentation

---

Related Models

DETR Original transformer detector Predecessor to Deformable DETR

RT-DETRv2 Real-time transformer detector Modern DETR-style architecture

RF-DETR Receptive-field enhanced DETR Recent DETR-family variant

DEIMv2 Improved DETR training recipe Faster convergence, higher accuracy

---

Maintained by: Texas Instruments EdgeAI Team
Last Updated: August 2026