D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement
Paper • 2410.13842 • Published • 6
D-FINE: Optimized for Qualcomm Devices
D-FINE is a state-of-the-art real-time object detection model that redefines the regression task in DETRs as fine-grained distribution refinement. The default variant is trained on the COCO dataset.
This is based on the implementation of D-FINE found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.
Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.
Getting Started
There are two ways to deploy this model on your device:
Option 1: Download Pre-Exported Models
Below are pre-exported model assets ready for deployment.
| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.50, ONNX Runtime 1.30.0 | Download |
| QNN_DLC | float | Universal | QAIRT 2.50 | Download |
| TFLITE | float | Universal | QAIRT 2.50 | Download |
For more device-specific assets and performance metrics, visit D-FINE on Qualcomm® AI Hub.
Option 2: Export with Custom Configurations
Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations
This option is ideal if you need to customize the model beyond the default configuration provided here.
See our repository for D-FINE on GitHub for usage instructions.
Model Details
Model Type: Model_use_case.object_detection
Model Stats:
- Input resolution: 640x640
- Model checkpoint: dfine-nano-coco
- Model size (float): 14.6 MB
- Number of parameters: 3.79M
Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit |
|---|---|---|---|---|---|---|
| D-FINE | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.613 ms | 0 - 296 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.999 ms | 0 - 285 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® X2 Elite | 6.843 ms | 5 - 5 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® X Elite | 16.057 ms | 6 - 6 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 11.078 ms | 5 - 338 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 18.557 ms | 5 - 340 MB | NPU |
| D-FINE | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 15.691 ms | 5 - 9 MB | NPU |
| D-FINE | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 15.263 ms | 0 - 10 MB | NPU |
| D-FINE | ONNX | float | Qualcomm® QCS8450 | 18.557 ms | 5 - 340 MB | NPU |
| D-FINE | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 16.579 ms | 5 - 8 MB | NPU |
| D-FINE | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 16.057 ms | 6 - 6 MB | NPU |
| D-FINE | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 7.999 ms | 0 - 285 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.534 ms | 5 - 183 MB | NPU |
| D-FINE | ONNX | float | Snapdragon® 8 Elite Gen 6 Mobile | 8.391 ms | 5 - 304 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 5.784 ms | 5 - 167 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.365 ms | 0 - 162 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® X2 Elite | 6.519 ms | 5 - 5 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® X Elite | 14.976 ms | 5 - 5 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 9.816 ms | 5 - 199 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 16.287 ms | 5 - 197 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 14.293 ms | 5 - 12 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 14.112 ms | 5 - 7 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® SA8775P | 15.206 ms | 1 - 149 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® SA8650P | 15.206 ms | 1 - 149 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® SA8255P | 15.206 ms | 1 - 149 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® QCS8450 | 16.287 ms | 5 - 197 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 15.179 ms | 5 - 12 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 14.976 ms | 5 - 5 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 7.365 ms | 0 - 162 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® SA7255P | 29.337 ms | 0 - 146 MB | NPU |
| D-FINE | QNN_DLC | float | Qualcomm® SA8295P | 17.411 ms | 0 - 154 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.125 ms | 5 - 170 MB | NPU |
| D-FINE | QNN_DLC | float | Snapdragon® 8 Elite Gen 6 Mobile | 7.711 ms | 4 - 170 MB | NPU |
| D-FINE | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 21.947 ms | 0 - 491 MB | NPU |
| D-FINE | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 29.465 ms | 0 - 653 MB | NPU |
| D-FINE | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 46.231 ms | 0 - 553 MB | NPU |
| D-FINE | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 42.608 ms | 0 - 7 MB | NPU |
| D-FINE | TFLITE | float | Qualcomm® QCS8450 | 46.231 ms | 0 - 553 MB | NPU |
| D-FINE | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 43.635 ms | 0 - 22 MB | NPU |
| D-FINE | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 21.947 ms | 0 - 491 MB | NPU |
| D-FINE | TFLITE | float | Qualcomm® SA8295P | 46.048 ms | 0 - 459 MB | NPU |
| D-FINE | TFLITE | float | Snapdragon® 8 Elite Gen 6 Mobile | 24.201 ms | 0 - 767 MB | NPU |
License
- The license for the original implementation of D-FINE can be found here.
References
- D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement
- Source Model Implementation
Community
- Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
- For questions or feedback please reach out to us.