Hugging Face's logo

qualcomm
/
ACT

Robotics
PyTorch
android
Model card Files
xet
Community
qaihm-bot commited on
Commit
b2b8506
·
verified ·
1 Parent(s): 213d473

v0.48.0

Browse files

See https://github.com/qualcomm/ai-hub-models/releases/v0.48.0 for changelog.

Files changed (1) hide show
  1. README.md +35 -35
README.md CHANGED
@@ -14,7 +14,7 @@ pipeline_tag: robotics
14
  ACT (Action Chunking with Transformers) is a robotic policy model that is trained to predict the next chunk of actions that the robotic hand is expected to perform.
15
 
16
  This is based on the implementation of ACT found [here](https://github.com/tonyzhaozh/act).
17
- This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/act) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
 
19
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
 
@@ -27,23 +27,23 @@ Below are pre-exported model assets ready for deployment.
27
 
28
  | Runtime | Precision | Chipset | SDK Versions | Download |
29
  |---|---|---|---|---|
30
- | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.47.0/act-onnx-float.zip)
31
- | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.47.0/act-qnn_dlc-float.zip)
32
- | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.47.0/act-tflite-float.zip)
33
 
34
  For more device-specific assets and performance metrics, visit **[ACT on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/act)**.
35
 
36
 
37
  ### Option 2: Export with Custom Configurations
38
 
39
- Use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/act) Python library to compile and export the model with your own:
40
  - Custom weights (e.g., fine-tuned checkpoints)
41
  - Custom input shapes
42
  - Target device and runtime configurations
43
 
44
  This option is ideal if you need to customize the model beyond the default configuration provided here.
45
 
46
- See our repository for [ACT on GitHub](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/act) for usage instructions.
47
 
48
  ## Model Details
49
 
@@ -58,35 +58,35 @@ See our repository for [ACT on GitHub](https://github.com/quic/ai-hub-models/blo
58
  ## Performance Summary
59
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
60
  |---|---|---|---|---|---|---
61
- | ACT | ONNX | float | Snapdragon® X Elite | 11.898 ms | 62 - 62 MB | NPU
62
- | ACT | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.113 ms | 1 - 376 MB | NPU
63
- | ACT | ONNX | float | Qualcomm® QCS8550 (Proxy) | 11.237 ms | 0 - 81 MB | NPU
64
- | ACT | ONNX | float | Qualcomm® QCS9075 | 19.025 ms | 4 - 10 MB | NPU
65
- | ACT | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 6.563 ms | 1 - 329 MB | NPU
66
- | ACT | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.513 ms | 4 - 333 MB | NPU
67
- | ACT | ONNX | float | Snapdragon® X2 Elite | 6.147 ms | 63 - 63 MB | NPU
68
- | ACT | QNN_DLC | float | Snapdragon® X Elite | 8.903 ms | 4 - 4 MB | NPU
69
- | ACT | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 5.95 ms | 3 - 348 MB | NPU
70
- | ACT | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 43.949 ms | 0 - 308 MB | NPU
71
- | ACT | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 8.29 ms | 4 - 6 MB | NPU
72
- | ACT | QNN_DLC | float | Qualcomm® SA8775P | 13.34 ms | 1 - 294 MB | NPU
73
- | ACT | QNN_DLC | float | Qualcomm® QCS9075 | 16.057 ms | 4 - 9 MB | NPU
74
- | ACT | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 16.303 ms | 4 - 264 MB | NPU
75
- | ACT | QNN_DLC | float | Qualcomm® SA7255P | 43.949 ms | 0 - 308 MB | NPU
76
- | ACT | QNN_DLC | float | Qualcomm® SA8295P | 15.529 ms | 0 - 231 MB | NPU
77
- | ACT | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.798 ms | 0 - 295 MB | NPU
78
- | ACT | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.741 ms | 4 - 321 MB | NPU
79
- | ACT | QNN_DLC | float | Snapdragon® X2 Elite | 4.863 ms | 4 - 4 MB | NPU
80
- | ACT | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 5.977 ms | 0 - 362 MB | NPU
81
- | ACT | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 44.141 ms | 0 - 319 MB | NPU
82
- | ACT | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 8.231 ms | 0 - 3 MB | NPU
83
- | ACT | TFLITE | float | Qualcomm® SA8775P | 13.453 ms | 0 - 304 MB | NPU
84
- | ACT | TFLITE | float | Qualcomm® QCS9075 | 16.21 ms | 0 - 71 MB | NPU
85
- | ACT | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 16.408 ms | 0 - 274 MB | NPU
86
- | ACT | TFLITE | float | Qualcomm® SA7255P | 44.141 ms | 0 - 319 MB | NPU
87
- | ACT | TFLITE | float | Qualcomm® SA8295P | 15.698 ms | 0 - 235 MB | NPU
88
- | ACT | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.838 ms | 0 - 307 MB | NPU
89
- | ACT | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.8 ms | 0 - 327 MB | NPU
90
 
91
  ## License
92
  * The license for the original implementation of ACT can be found
 
14
  ACT (Action Chunking with Transformers) is a robotic policy model that is trained to predict the next chunk of actions that the robotic hand is expected to perform.
15
 
16
  This is based on the implementation of ACT found [here](https://github.com/tonyzhaozh/act).
17
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/act) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
 
19
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
 
 
27
 
28
  | Runtime | Precision | Chipset | SDK Versions | Download |
29
  |---|---|---|---|---|
30
+ | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.48.0/act-onnx-float.zip)
31
+ | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.48.0/act-qnn_dlc-float.zip)
32
+ | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.48.0/act-tflite-float.zip)
33
 
34
  For more device-specific assets and performance metrics, visit **[ACT on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/act)**.
35
 
36
 
37
  ### Option 2: Export with Custom Configurations
38
 
39
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/act) Python library to compile and export the model with your own:
40
  - Custom weights (e.g., fine-tuned checkpoints)
41
  - Custom input shapes
42
  - Target device and runtime configurations
43
 
44
  This option is ideal if you need to customize the model beyond the default configuration provided here.
45
 
46
+ See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/act) for usage instructions.
47
 
48
  ## Model Details
49
 
 
58
  ## Performance Summary
59
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
60
  |---|---|---|---|---|---|---
61
+ | ACT | ONNX | float | Snapdragon® X2 Elite | 6.165 ms | 63 - 63 MB | NPU
62
+ | ACT | ONNX | float | Snapdragon® X Elite | 11.872 ms | 62 - 62 MB | NPU
63
+ | ACT | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.103 ms | 3 - 385 MB | NPU
64
+ | ACT | ONNX | float | Qualcomm® QCS8550 (Proxy) | 11.214 ms | 0 - 81 MB | NPU
65
+ | ACT | ONNX | float | Qualcomm® QCS9075 | 18.835 ms | 4 - 10 MB | NPU
66
+ | ACT | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 6.573 ms | 3 - 331 MB | NPU
67
+ | ACT | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.496 ms | 4 - 334 MB | NPU
68
+ | ACT | QNN_DLC | float | Snapdragon® X2 Elite | 4.932 ms | 4 - 4 MB | NPU
69
+ | ACT | QNN_DLC | float | Snapdragon® X Elite | 8.926 ms | 4 - 4 MB | NPU
70
+ | ACT | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 5.974 ms | 1 - 344 MB | NPU
71
+ | ACT | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 43.942 ms | 1 - 308 MB | NPU
72
+ | ACT | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 8.288 ms | 4 - 7 MB | NPU
73
+ | ACT | QNN_DLC | float | Qualcomm® SA8775P | 13.325 ms | 1 - 294 MB | NPU
74
+ | ACT | QNN_DLC | float | Qualcomm® QCS9075 | 15.977 ms | 4 - 9 MB | NPU
75
+ | ACT | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 16.41 ms | 2 - 263 MB | NPU
76
+ | ACT | QNN_DLC | float | Qualcomm® SA7255P | 43.942 ms | 1 - 308 MB | NPU
77
+ | ACT | QNN_DLC | float | Qualcomm® SA8295P | 15.517 ms | 0 - 230 MB | NPU
78
+ | ACT | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.768 ms | 4 - 299 MB | NPU
79
+ | ACT | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.739 ms | 4 - 321 MB | NPU
80
+ | ACT | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 5.981 ms | 0 - 361 MB | NPU
81
+ | ACT | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 44.134 ms | 0 - 318 MB | NPU
82
+ | ACT | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 8.318 ms | 0 - 3 MB | NPU
83
+ | ACT | TFLITE | float | Qualcomm® SA8775P | 13.44 ms | 0 - 304 MB | NPU
84
+ | ACT | TFLITE | float | Qualcomm® QCS9075 | 16.034 ms | 0 - 71 MB | NPU
85
+ | ACT | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 16.392 ms | 0 - 276 MB | NPU
86
+ | ACT | TFLITE | float | Qualcomm® SA7255P | 44.134 ms | 0 - 318 MB | NPU
87
+ | ACT | TFLITE | float | Qualcomm® SA8295P | 15.711 ms | 0 - 233 MB | NPU
88
+ | ACT | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.816 ms | 0 - 310 MB | NPU
89
+ | ACT | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.799 ms | 0 - 327 MB | NPU
90
 
91
  ## License
92
  * The license for the original implementation of ACT can be found