Detectron2-Panoptic-Segmentation: Optimized for Qualcomm Devices
Detectron2-Panoptic-Segmentation is a machine learning model that performs panoptic segmentation (trained on COCO dataset), combining instance and semantic segmentation.
This is based on the implementation of Detectron2-Panoptic-Segmentation found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.
Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.
Getting Started
There are two ways to deploy this model on your device:
Option 1: Download Pre-Exported Models
Below are pre-exported model assets ready for deployment.
| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.50, ONNX Runtime 1.30.0 | Download |
| ONNX | mixed | Universal | QAIRT 2.50, ONNX Runtime 1.30.0 | Download |
| QNN_DLC | float | Universal | QAIRT 2.50 | Download |
| QNN_DLC | mixed | Universal | QAIRT 2.50 | Download |
| TFLITE | float | Universal | QAIRT 2.50 | Download |
For more device-specific assets and performance metrics, visit Detectron2-Panoptic-Segmentation on Qualcomm® AI Hub.
Option 2: Export with Custom Configurations
Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations
This option is ideal if you need to customize the model beyond the default configuration provided here.
See our repository for Detectron2-Panoptic-Segmentation on GitHub for usage instructions.
Model Details
Model Type: Model_use_case.semantic_segmentation
Model Stats:
- Input resolution: 800x800
- Model checkpoint: panoptic_fpn_R_50_1x
- Model size (float): 176 MB
- Number of parameters: 46.0M
Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit |
|---|---|---|---|---|---|---|
| proposal_generator | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 58.041 ms | 132 - 1448 MB | NPU |
| proposal_generator | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 71.409 ms | 115 - 1358 MB | NPU |
| proposal_generator | ONNX | float | Snapdragon® X2 Elite | 58.034 ms | 272 - 272 MB | NPU |
| proposal_generator | ONNX | float | Snapdragon® X Elite | 107.562 ms | 268 - 268 MB | NPU |
| proposal_generator | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 151.693 ms | 136 - 1473 MB | NPU |
| proposal_generator | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 138.806 ms | 7 - 19 MB | NPU |
| proposal_generator | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 111.327 ms | 64 - 139 MB | NPU |
| proposal_generator | ONNX | float | Qualcomm® QCS8450 | 151.693 ms | 136 - 1473 MB | NPU |
| proposal_generator | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 136.191 ms | 7 - 18 MB | NPU |
| proposal_generator | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 107.562 ms | 268 - 268 MB | NPU |
| proposal_generator | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 71.409 ms | 115 - 1358 MB | NPU |
| proposal_generator | ONNX | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 43.068 ms | 133 - 1383 MB | NPU |
| proposal_generator | ONNX | float | Snapdragon® 8 Elite Gen 6 Mobile | 62.105 ms | 79 - 1273 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 43.461 ms | 226 - 1348 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® 8 Elite For Galaxy Mobile | 53.494 ms | 226 - 1301 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® X2 Elite | 42.143 ms | 281 - 281 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® X Elite | 83.186 ms | 282 - 282 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® 8 Gen 1 Mobile | 103.85 ms | 229 - 1450 MB | NPU |
| proposal_generator | ONNX | mixed | Qualcomm® Dragonwing™ IQ-8275 | 88.558 ms | 36 - 47 MB | NPU |
| proposal_generator | ONNX | mixed | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 84.317 ms | 222 - 226 MB | NPU |
| proposal_generator | ONNX | mixed | Qualcomm® QCS8450 | 103.85 ms | 229 - 1450 MB | NPU |
| proposal_generator | ONNX | mixed | Qualcomm® Dragonwing™ IQ-9075 | 87.981 ms | 57 - 67 MB | NPU |
| proposal_generator | ONNX | mixed | Qualcomm® Dragonwing™ IQ-X7181 | 83.186 ms | 282 - 282 MB | NPU |
| proposal_generator | ONNX | mixed | Qualcomm® Dragonwing™ Q-8750 | 53.494 ms | 226 - 1301 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 60.751 ms | 233 - 1778 MB | NPU |
| proposal_generator | ONNX | mixed | Snapdragon® 8 Elite Gen 6 Mobile | 90.887 ms | 231 - 1711 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 57.355 ms | 7 - 1187 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 69.995 ms | 0 - 1117 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® X2 Elite | 57.884 ms | 7 - 7 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® X Elite | 107.578 ms | 7 - 7 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 139.749 ms | 0 - 1349 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 139.37 ms | 7 - 78 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 113.013 ms | 1 - 10 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® QCS8450 | 139.749 ms | 0 - 1349 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 135.355 ms | 7 - 77 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 107.578 ms | 7 - 7 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 69.995 ms | 0 - 1117 MB | NPU |
| proposal_generator | QNN_DLC | float | Qualcomm® SA8295P | 150.22 ms | 1 - 1170 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 43.083 ms | 7 - 1228 MB | NPU |
| proposal_generator | QNN_DLC | float | Snapdragon® 8 Elite Gen 6 Mobile | 58.736 ms | 7 - 1275 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 35.963 ms | 4 - 1061 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® 8 Elite For Galaxy Mobile | 45.313 ms | 0 - 1022 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® X2 Elite | 36.258 ms | 7 - 7 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® X Elite | 63.412 ms | 7 - 7 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® 8 Gen 1 Mobile | 77.227 ms | 7 - 1187 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® Dragonwing™ IQ-8275 | 64.919 ms | 7 - 78 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 67.86 ms | 7 - 10 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® QCS8450 | 77.227 ms | 7 - 1187 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® Dragonwing™ IQ-9075 | 64.863 ms | 7 - 77 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® Dragonwing™ IQ-X7181 | 63.412 ms | 7 - 7 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® Dragonwing™ Q-8750 | 45.313 ms | 0 - 1022 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Qualcomm® SA8295P | 84.177 ms | 0 - 1014 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 50.696 ms | 7 - 1504 MB | NPU |
| proposal_generator | QNN_DLC | mixed | Snapdragon® 8 Elite Gen 6 Mobile | 75.102 ms | 7 - 1449 MB | NPU |
| proposal_generator | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 55.073 ms | 66 - 1295 MB | NPU |
| proposal_generator | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 65.284 ms | 67 - 1296 MB | NPU |
| proposal_generator | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 140.2 ms | 68 - 1509 MB | NPU |
| proposal_generator | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 134.847 ms | 15 - 164 MB | NPU |
| proposal_generator | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 105.412 ms | 68 - 71 MB | NPU |
| proposal_generator | TFLITE | float | Qualcomm® QCS8450 | 140.2 ms | 68 - 1509 MB | NPU |
| proposal_generator | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 130.942 ms | 44 - 192 MB | NPU |
| proposal_generator | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 65.284 ms | 67 - 1296 MB | NPU |
| proposal_generator | TFLITE | float | Qualcomm® SA8295P | 141.503 ms | 68 - 1274 MB | NPU |
| proposal_generator | TFLITE | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 43.273 ms | 26 - 1301 MB | NPU |
| proposal_generator | TFLITE | float | Snapdragon® 8 Elite Gen 6 Mobile | 60.001 ms | 29 - 1243 MB | NPU |
| roi_head | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 67.89 ms | 102 - 1196 MB | NPU |
| roi_head | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 86.719 ms | 104 - 965 MB | NPU |
| roi_head | ONNX | float | Snapdragon® X2 Elite | 62.993 ms | 133 - 133 MB | NPU |
| roi_head | ONNX | float | Snapdragon® X Elite | 130.432 ms | 133 - 133 MB | NPU |
| roi_head | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 195.226 ms | 114 - 1090 MB | NPU |
| roi_head | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 174.323 ms | 52 - 108 MB | NPU |
| roi_head | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 145.307 ms | 98 - 107 MB | NPU |
| roi_head | ONNX | float | Qualcomm® QCS8450 | 195.226 ms | 114 - 1090 MB | NPU |
| roi_head | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 173.125 ms | 52 - 107 MB | NPU |
| roi_head | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 130.432 ms | 133 - 133 MB | NPU |
| roi_head | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 86.719 ms | 104 - 965 MB | NPU |
| roi_head | ONNX | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 48.169 ms | 114 - 962 MB | NPU |
| roi_head | ONNX | float | Snapdragon® 8 Elite Gen 6 Mobile | 73.341 ms | 115 - 1245 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 31.705 ms | 125 - 957 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® 8 Elite For Galaxy Mobile | 41.346 ms | 160 - 986 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® X2 Elite | 35.419 ms | 266 - 266 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® X Elite | 74.859 ms | 266 - 266 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® 8 Gen 1 Mobile | 88.48 ms | 153 - 1082 MB | NPU |
| roi_head | ONNX | mixed | Qualcomm® Dragonwing™ IQ-8275 | 77.088 ms | 52 - 95 MB | NPU |
| roi_head | ONNX | mixed | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 74.149 ms | 191 - 195 MB | NPU |
| roi_head | ONNX | mixed | Qualcomm® QCS8450 | 88.48 ms | 153 - 1082 MB | NPU |
| roi_head | ONNX | mixed | Qualcomm® Dragonwing™ IQ-9075 | 79.833 ms | 52 - 94 MB | NPU |
| roi_head | ONNX | mixed | Qualcomm® Dragonwing™ IQ-X7181 | 74.859 ms | 266 - 266 MB | NPU |
| roi_head | ONNX | mixed | Qualcomm® Dragonwing™ Q-8750 | 41.346 ms | 160 - 986 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 30.681 ms | 169 - 1130 MB | NPU |
| roi_head | ONNX | mixed | Snapdragon® 8 Elite Gen 6 Mobile | 41.951 ms | 169 - 1241 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 59.672 ms | 19 - 863 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 73.683 ms | 38 - 1013 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® X2 Elite | 61.006 ms | 52 - 52 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® X Elite | 115.929 ms | 52 - 52 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 187.628 ms | 0 - 869 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 168.04 ms | 52 - 155 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 122.951 ms | 52 - 55 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® QCS8450 | 187.628 ms | 0 - 869 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 154.412 ms | 52 - 154 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 115.929 ms | 52 - 52 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 73.683 ms | 38 - 1013 MB | NPU |
| roi_head | QNN_DLC | float | Qualcomm® SA8295P | 185.187 ms | 39 - 842 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 43.612 ms | 0 - 885 MB | NPU |
| roi_head | QNN_DLC | float | Snapdragon® 8 Elite Gen 6 Mobile | 69.393 ms | 52 - 989 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 29.21 ms | 17 - 694 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® 8 Elite For Galaxy Mobile | 36.439 ms | 36 - 920 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® X2 Elite | 31.482 ms | 52 - 52 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® X Elite | 65.98 ms | 52 - 52 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® 8 Gen 1 Mobile | 98.357 ms | 9 - 805 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® Dragonwing™ IQ-8275 | 64.015 ms | 52 - 155 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 66.349 ms | 52 - 54 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® QCS8450 | 98.357 ms | 9 - 805 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® Dragonwing™ IQ-9075 | 71.908 ms | 52 - 154 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® Dragonwing™ IQ-X7181 | 65.98 ms | 52 - 52 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® Dragonwing™ Q-8750 | 36.439 ms | 36 - 920 MB | NPU |
| roi_head | QNN_DLC | mixed | Qualcomm® SA8295P | 91.729 ms | 42 - 731 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 35.132 ms | 52 - 822 MB | NPU |
| roi_head | QNN_DLC | mixed | Snapdragon® 8 Elite Gen 6 Mobile | 39.577 ms | 52 - 1137 MB | NPU |
| roi_head | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 130.6 ms | 49 - 1444 MB | NPU |
| roi_head | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 155.126 ms | 46 - 1282 MB | NPU |
| roi_head | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 337.951 ms | 46 - 1531 MB | NPU |
| roi_head | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 265.87 ms | 0 - 151 MB | NPU |
| roi_head | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 262.028 ms | 48 - 52 MB | NPU |
| roi_head | TFLITE | float | Qualcomm® QCS8450 | 337.951 ms | 46 - 1531 MB | NPU |
| roi_head | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 292.683 ms | 0 - 151 MB | NPU |
| roi_head | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 155.126 ms | 46 - 1282 MB | NPU |
| roi_head | TFLITE | float | Qualcomm® SA8295P | 325.618 ms | 48 - 1342 MB | NPU |
| roi_head | TFLITE | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 112.707 ms | 6 - 1401 MB | NPU |
| roi_head | TFLITE | float | Snapdragon® 8 Elite Gen 6 Mobile | 132.329 ms | 9 - 1404 MB | NPU |
License
- The license for the original implementation of Detectron2-Panoptic-Segmentation can be found here.
References
Community
- Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
- For questions or feedback please reach out to us.