Hugging Face's logo

qualcomm
/
WeDetect

Object Detection
PyTorch
real_time
android
Model card Files
xet
 Community

WeDetect: Optimized for Qualcomm Devices

WeDetect is a text-conditioned object detection model that uses a prompt-then-detect strategy. Text class names are reparameterized into the model weights before export, producing a fixed-vocabulary detector that runs efficiently on-device. The model uses LTRB (Left-Top-Right-Bottom) regression with multi-scale feature maps at strides 8, 16, and 32.

This is based on the implementation of WeDetect found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

Due to licensing restrictions, we cannot distribute pre-exported model assets for this model. Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

See our repository for WeDetect on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.object_detection

Model Stats:

  • Input resolution: 640x640
  • Model checkpoint: wedetect_tiny
  • Model size (detector) (float): 143 MB
  • Model size (detector) (mixed_with_float): 35.9 MB
  • Model size (text_encoder) (float): 592 B
  • Model size (text_encoder) (mixed_with_float): 13.3 KB
  • Number of parameters (detector): 37.3M
  • Number of parameters (text_encoder): 3.00

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
detector ONNX float Snapdragon® X2 Elite 10.426 ms 5 - 5 MB NPU
detector ONNX float Snapdragon® X Elite 21.594 ms 71 - 71 MB NPU
detector ONNX float Snapdragon® 8 Gen 3 Mobile 15.848 ms 0 - 331 MB NPU
detector ONNX float Snapdragon® 8 Gen 1 Mobile 82.452 ms 1 - 325 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-8275 35.18 ms 5 - 13 MB NPU
detector ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 21.019 ms 0 - 77 MB NPU
detector ONNX float Qualcomm® QCS8450 82.452 ms 1 - 325 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-9075 33.506 ms 5 - 12 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-X7181 21.594 ms 71 - 71 MB NPU
detector ONNX float Qualcomm® Dragonwing™ Q-8750 11.352 ms 3 - 260 MB NPU
detector ONNX float Snapdragon® 8 Elite Mobile 11.352 ms 3 - 260 MB NPU
detector ONNX float Snapdragon® 8 Elite Gen 5 Mobile 8.252 ms 1 - 262 MB NPU
detector ONNX mixed_with_float Snapdragon® X2 Elite 10.674 ms 9 - 9 MB NPU
detector ONNX mixed_with_float Snapdragon® X Elite 16.478 ms 35 - 35 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Gen 3 Mobile 11.362 ms 2 - 365 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Gen 1 Mobile 25.631 ms 2 - 363 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-8275 15.652 ms 6 - 14 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 16.014 ms 8 - 11 MB NPU
detector ONNX mixed_with_float Qualcomm® QCS8450 25.631 ms 2 - 363 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-9075 17.237 ms 6 - 14 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-X7181 16.478 ms 35 - 35 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ Q-8750 11.243 ms 8 - 314 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Elite Mobile 11.243 ms 8 - 314 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 9.44 ms 4 - 316 MB NPU
detector QNN_DLC float Snapdragon® X2 Elite 10.354 ms 5 - 5 MB NPU
detector QNN_DLC float Snapdragon® X Elite 21.052 ms 5 - 5 MB NPU
detector QNN_DLC float Snapdragon® 8 Gen 3 Mobile 15.376 ms 5 - 367 MB NPU
detector QNN_DLC float Snapdragon® 8 Gen 1 Mobile 81.716 ms 5 - 377 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 34.453 ms 5 - 12 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 116.247 ms 1 - 292 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 20.178 ms 5 - 7 MB NPU
detector QNN_DLC float Qualcomm® SA8775P 30.364 ms 2 - 295 MB NPU
detector QNN_DLC float Qualcomm® SA8650P 30.364 ms 2 - 295 MB NPU
detector QNN_DLC float Qualcomm® SA8255P 30.364 ms 2 - 295 MB NPU
detector QNN_DLC float Qualcomm® QCS8450 81.716 ms 5 - 377 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 30.784 ms 7 - 14 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-X7181 21.052 ms 5 - 5 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ Q-8750 10.918 ms 0 - 270 MB NPU
detector QNN_DLC float Qualcomm® SA7255P 116.247 ms 1 - 292 MB NPU
detector QNN_DLC float Qualcomm® SA8295P 86.75 ms 0 - 284 MB NPU
detector QNN_DLC float Snapdragon® 8 Elite Mobile 10.918 ms 0 - 270 MB NPU
detector QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 8.078 ms 5 - 284 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® X2 Elite 10.056 ms 5 - 5 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Gen 3 Mobile 13.033 ms 0 - 360 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 18.18 ms 5 - 12 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 43.62 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 18.008 ms 5 - 7 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8775P 18.231 ms 0 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8650P 18.231 ms 0 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8255P 18.231 ms 0 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-9075 18.814 ms 7 - 13 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ Q-8750 11.04 ms 5 - 308 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA7255P 43.62 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Elite Mobile 11.04 ms 5 - 308 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 8.058 ms 5 - 326 MB NPU
detector TFLITE float Snapdragon® 8 Gen 3 Mobile 16.057 ms 0 - 343 MB NPU
detector TFLITE float Snapdragon® 8 Gen 1 Mobile 94.737 ms 1 - 337 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-8275 35.505 ms 0 - 83 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-8275 117.868 ms 1 - 266 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 21.216 ms 0 - 7 MB NPU
detector TFLITE float Qualcomm® SA8775P 31.581 ms 0 - 268 MB NPU
detector TFLITE float Qualcomm® SA8650P 31.581 ms 0 - 268 MB NPU
detector TFLITE float Qualcomm® SA8255P 31.581 ms 0 - 268 MB NPU
detector TFLITE float Qualcomm® QCS8450 94.737 ms 1 - 337 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-9075 31.63 ms 0 - 83 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ Q-8750 11.797 ms 0 - 271 MB NPU
detector TFLITE float Qualcomm® SA7255P 117.868 ms 1 - 266 MB NPU
detector TFLITE float Qualcomm® SA8295P 83.191 ms 0 - 265 MB NPU
detector TFLITE float Snapdragon® 8 Elite Mobile 11.797 ms 0 - 271 MB NPU
detector TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 9.883 ms 0 - 273 MB NPU
text_encoder ONNX float Snapdragon® X2 Elite 42.773 ms 1 - 1 MB NPU
text_encoder ONNX float Snapdragon® X Elite 97.059 ms 550 - 550 MB NPU
text_encoder ONNX float Snapdragon® 8 Gen 3 Mobile 65.375 ms 1 - 2266 MB NPU
text_encoder ONNX float Snapdragon® 8 Gen 1 Mobile 146.223 ms 2 - 1863 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-8275 102.452 ms 1 - 4 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 97.003 ms 0 - 644 MB NPU
text_encoder ONNX float Qualcomm® QCS8450 146.223 ms 2 - 1863 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-9075 99.394 ms 1 - 3 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-X7181 97.059 ms 550 - 550 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ Q-8750 46.642 ms 1 - 2019 MB NPU
text_encoder ONNX float Snapdragon® 8 Elite Mobile 46.642 ms 1 - 2019 MB NPU
text_encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 40.332 ms 1 - 2264 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® X2 Elite 42.617 ms 1 - 1 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® X Elite 97.181 ms 550 - 550 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Gen 3 Mobile 65.815 ms 1 - 2268 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Gen 1 Mobile 145.25 ms 2 - 1865 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-8275 103.16 ms 1 - 4 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 96.752 ms 0 - 643 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® QCS8450 145.25 ms 2 - 1865 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-9075 99.51 ms 1 - 3 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-X7181 97.181 ms 550 - 550 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ Q-8750 48.085 ms 1 - 2019 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Elite Mobile 48.085 ms 1 - 2019 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 39.855 ms 1 - 2265 MB NPU
text_encoder QNN_DLC float Snapdragon® X2 Elite 43.277 ms 0 - 0 MB NPU
text_encoder QNN_DLC float Snapdragon® X Elite 86.461 ms 0 - 0 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 62.016 ms 0 - 2450 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 140.009 ms 0 - 3013 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 95.48 ms 0 - 3 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 260.992 ms 1 - 2036 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 85.173 ms 0 - 3 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8775P 95.567 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8650P 95.567 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8255P 95.567 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® QCS8450 140.009 ms 0 - 3013 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 93.955 ms 0 - 2 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-X7181 86.461 ms 0 - 0 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 46.578 ms 0 - 1973 MB NPU
text_encoder QNN_DLC float Qualcomm® SA7255P 260.992 ms 1 - 2036 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8295P 107.476 ms 0 - 1902 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Elite Mobile 46.578 ms 0 - 1973 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 34.527 ms 0 - 1983 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® X2 Elite 37.173 ms 0 - 0 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Gen 3 Mobile 61.867 ms 0 - 2451 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 95.226 ms 0 - 3 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 261.037 ms 0 - 2035 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 85.295 ms 0 - 4 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8775P 95.609 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8650P 95.609 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8255P 95.609 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-9075 93.359 ms 0 - 2 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ Q-8750 47.44 ms 1 - 1975 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA7255P 261.037 ms 0 - 2035 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Elite Mobile 47.44 ms 1 - 1975 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 34.504 ms 0 - 1983 MB NPU
text_encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 72.829 ms 1 - 3683 MB NPU
text_encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 159.932 ms 1 - 3564 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-8275 109.605 ms 0 - 553 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-8275 289.737 ms 1 - 2144 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 99.966 ms 1 - 4 MB NPU
text_encoder TFLITE float Qualcomm® SA8775P 111.295 ms 1 - 2144 MB NPU
text_encoder TFLITE float Qualcomm® SA8650P 111.295 ms 1 - 2144 MB NPU
text_encoder TFLITE float Qualcomm® SA8255P 111.295 ms 1 - 2144 MB NPU
text_encoder TFLITE float Qualcomm® QCS8450 159.932 ms 1 - 3564 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 108.948 ms 0 - 553 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ Q-8750 55.49 ms 1 - 2073 MB NPU
text_encoder TFLITE float Qualcomm® SA7255P 289.737 ms 1 - 2144 MB NPU
text_encoder TFLITE float Qualcomm® SA8295P 112.85 ms 1 - 2099 MB NPU
text_encoder TFLITE float Snapdragon® 8 Elite Mobile 55.49 ms 1 - 2073 MB NPU
text_encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 41.511 ms 0 - 2220 MB NPU

License

  • The license for the original implementation of WeDetect can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Object Detection
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/WeDetect