Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -34,12 +34,15 @@ More details on model performance across various devices, can be found
|
|
34 |
- Model size (CLIPImageEncoder): 437 MB
|
35 |
|
36 |
|
|
|
|
|
37 |
| Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
|
38 |
| ---|---|---|---|---|---|---|---|
|
39 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 13.
|
40 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 126.
|
41 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 7.
|
42 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 50.
|
|
|
43 |
|
44 |
|
45 |
## Installation
|
@@ -101,22 +104,24 @@ python -m qai_hub_models.models.openai_clip.export
|
|
101 |
Profile Job summary of CLIPTextEncoder
|
102 |
--------------------------------------------------
|
103 |
Device: Snapdragon X Elite CRD (11)
|
104 |
-
Estimated Inference Time: 8.
|
105 |
-
Estimated Peak Memory Range: 0.
|
106 |
Compute Units: NPU (377) | Total (377)
|
107 |
|
108 |
Profile Job summary of CLIPImageEncoder
|
109 |
--------------------------------------------------
|
110 |
Device: Snapdragon X Elite CRD (11)
|
111 |
-
Estimated Inference Time: 48.
|
112 |
Estimated Peak Memory Range: 0.57-0.57 MB
|
113 |
Compute Units: NPU (369) | Total (369)
|
114 |
|
115 |
|
116 |
```
|
|
|
|
|
117 |
## How does this work?
|
118 |
|
119 |
-
This [export script](https://
|
120 |
leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
|
121 |
on-device. Lets go through each step below in detail:
|
122 |
|
@@ -194,6 +199,7 @@ AI Hub. [Sign up for access](https://myaccount.qualcomm.com/signup).
|
|
194 |
|
195 |
|
196 |
|
|
|
197 |
## Deploying compiled model to Android
|
198 |
|
199 |
|
@@ -215,7 +221,7 @@ Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
|
|
215 |
## License
|
216 |
- The license for the original implementation of OpenAI-Clip can be found
|
217 |
[here](https://github.com/openai/CLIP/blob/main/LICENSE).
|
218 |
-
- The license for the compiled assets for on-device deployment can be found [here](
|
219 |
|
220 |
## References
|
221 |
* [Learning Transferable Visual Models From Natural Language Supervision](https://arxiv.org/abs/2103.00020)
|
|
|
34 |
- Model size (CLIPImageEncoder): 437 MB
|
35 |
|
36 |
|
37 |
+
|
38 |
+
|
39 |
| Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
|
40 |
| ---|---|---|---|---|---|---|---|
|
41 |
+
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 13.293 ms | 0 - 3 MB | FP16 | NPU | [CLIPTextEncoder.tflite](https://huggingface.co/qualcomm/OpenAI-Clip/blob/main/CLIPTextEncoder.tflite)
|
42 |
+
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 126.539 ms | 0 - 261 MB | FP16 | NPU | [CLIPImageEncoder.tflite](https://huggingface.co/qualcomm/OpenAI-Clip/blob/main/CLIPImageEncoder.tflite)
|
43 |
+
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 7.81 ms | 0 - 30 MB | FP16 | NPU | [CLIPTextEncoder.so](https://huggingface.co/qualcomm/OpenAI-Clip/blob/main/CLIPTextEncoder.so)
|
44 |
+
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 50.274 ms | 0 - 63 MB | FP16 | NPU | [CLIPImageEncoder.so](https://huggingface.co/qualcomm/OpenAI-Clip/blob/main/CLIPImageEncoder.so)
|
45 |
+
|
46 |
|
47 |
|
48 |
## Installation
|
|
|
104 |
Profile Job summary of CLIPTextEncoder
|
105 |
--------------------------------------------------
|
106 |
Device: Snapdragon X Elite CRD (11)
|
107 |
+
Estimated Inference Time: 8.43 ms
|
108 |
+
Estimated Peak Memory Range: 0.15-0.15 MB
|
109 |
Compute Units: NPU (377) | Total (377)
|
110 |
|
111 |
Profile Job summary of CLIPImageEncoder
|
112 |
--------------------------------------------------
|
113 |
Device: Snapdragon X Elite CRD (11)
|
114 |
+
Estimated Inference Time: 48.61 ms
|
115 |
Estimated Peak Memory Range: 0.57-0.57 MB
|
116 |
Compute Units: NPU (369) | Total (369)
|
117 |
|
118 |
|
119 |
```
|
120 |
+
|
121 |
+
|
122 |
## How does this work?
|
123 |
|
124 |
+
This [export script](https://aihub.qualcomm.com/models/openai_clip/qai_hub_models/models/OpenAI-Clip/export.py)
|
125 |
leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
|
126 |
on-device. Lets go through each step below in detail:
|
127 |
|
|
|
199 |
|
200 |
|
201 |
|
202 |
+
|
203 |
## Deploying compiled model to Android
|
204 |
|
205 |
|
|
|
221 |
## License
|
222 |
- The license for the original implementation of OpenAI-Clip can be found
|
223 |
[here](https://github.com/openai/CLIP/blob/main/LICENSE).
|
224 |
+
- The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
|
225 |
|
226 |
## References
|
227 |
* [Learning Transferable Visual Models From Natural Language Supervision](https://arxiv.org/abs/2103.00020)
|