qaihm-bot commited on
Commit
b5ad585
1 Parent(s): 1f11bb9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -80
README.md CHANGED
@@ -35,10 +35,13 @@ More details on model performance across various devices, can be found
35
  - Model size: 43.9 MB
36
 
37
 
 
 
38
  | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
39
  | ---|---|---|---|---|---|---|---|
40
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 1.181 ms | 0 - 2 MB | INT8 | NPU | [ResNet101Quantized.tflite](https://huggingface.co/qualcomm/ResNet101Quantized/blob/main/ResNet101Quantized.tflite)
41
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 1.388 ms | 0 - 106 MB | INT8 | NPU | [ResNet101Quantized.so](https://huggingface.co/qualcomm/ResNet101Quantized/blob/main/ResNet101Quantized.so)
 
42
 
43
 
44
  ## Installation
@@ -100,89 +103,14 @@ python -m qai_hub_models.models.resnet101_quantized.export
100
  Profile Job summary of ResNet101Quantized
101
  --------------------------------------------------
102
  Device: Snapdragon X Elite CRD (11)
103
- Estimated Inference Time: 1.42 ms
104
- Estimated Peak Memory Range: 0.47-0.47 MB
105
  Compute Units: NPU (146) | Total (146)
106
 
107
 
108
  ```
109
- ## How does this work?
110
-
111
- This [export script](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/ResNet101Quantized/export.py)
112
- leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
113
- on-device. Lets go through each step below in detail:
114
-
115
- Step 1: **Compile model for on-device deployment**
116
-
117
- To compile a PyTorch model for on-device deployment, we first trace the model
118
- in memory using the `jit.trace` and then call the `submit_compile_job` API.
119
-
120
- ```python
121
- import torch
122
-
123
- import qai_hub as hub
124
- from qai_hub_models.models.resnet101_quantized import Model
125
-
126
- # Load the model
127
- torch_model = Model.from_pretrained()
128
- torch_model.eval()
129
-
130
- # Device
131
- device = hub.Device("Samsung Galaxy S23")
132
-
133
- # Trace model
134
- input_shape = torch_model.get_input_spec()
135
- sample_inputs = torch_model.sample_inputs()
136
-
137
- pt_model = torch.jit.trace(torch_model, [torch.tensor(data[0]) for _, data in sample_inputs.items()])
138
-
139
- # Compile model on a specific device
140
- compile_job = hub.submit_compile_job(
141
- model=pt_model,
142
- device=device,
143
- input_specs=torch_model.get_input_spec(),
144
- )
145
 
146
- # Get target model to run on-device
147
- target_model = compile_job.get_target_model()
148
-
149
- ```
150
-
151
-
152
- Step 2: **Performance profiling on cloud-hosted device**
153
-
154
- After compiling models from step 1. Models can be profiled model on-device using the
155
- `target_model`. Note that this scripts runs the model on a device automatically
156
- provisioned in the cloud. Once the job is submitted, you can navigate to a
157
- provided job URL to view a variety of on-device performance metrics.
158
- ```python
159
- profile_job = hub.submit_profile_job(
160
- model=target_model,
161
- device=device,
162
- )
163
-
164
- ```
165
-
166
- Step 3: **Verify on-device accuracy**
167
-
168
- To verify the accuracy of the model on-device, you can run on-device inference
169
- on sample input data on the same cloud hosted device.
170
- ```python
171
- input_data = torch_model.sample_inputs()
172
- inference_job = hub.submit_inference_job(
173
- model=target_model,
174
- device=device,
175
- inputs=input_data,
176
- )
177
-
178
- on_device_output = inference_job.download_output_data()
179
-
180
- ```
181
- With the output of the model, you can compute like PSNR, relative errors or
182
- spot check the output with expected output.
183
 
184
- **Note**: This on-device profiling and inference requires access to Qualcomm®
185
- AI Hub. [Sign up for access](https://myaccount.qualcomm.com/signup).
186
 
187
 
188
  ## Run demo on a cloud-hosted device
@@ -221,7 +149,7 @@ Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
221
  ## License
222
  - The license for the original implementation of ResNet101Quantized can be found
223
  [here](https://github.com/pytorch/vision/blob/main/LICENSE).
224
- - The license for the compiled assets for on-device deployment can be found [here]({deploy_license_url})
225
 
226
  ## References
227
  * [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)
 
35
  - Model size: 43.9 MB
36
 
37
 
38
+
39
+
40
  | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
41
  | ---|---|---|---|---|---|---|---|
42
+ | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 1.188 ms | 0 - 2 MB | INT8 | NPU | [ResNet101Quantized.tflite](https://huggingface.co/qualcomm/ResNet101Quantized/blob/main/ResNet101Quantized.tflite)
43
+ | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 1.377 ms | 0 - 56 MB | INT8 | NPU | [ResNet101Quantized.so](https://huggingface.co/qualcomm/ResNet101Quantized/blob/main/ResNet101Quantized.so)
44
+
45
 
46
 
47
  ## Installation
 
103
  Profile Job summary of ResNet101Quantized
104
  --------------------------------------------------
105
  Device: Snapdragon X Elite CRD (11)
106
+ Estimated Inference Time: 1.38 ms
107
+ Estimated Peak Memory Range: 0.26-0.26 MB
108
  Compute Units: NPU (146) | Total (146)
109
 
110
 
111
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
113
 
 
 
114
 
115
 
116
  ## Run demo on a cloud-hosted device
 
149
  ## License
150
  - The license for the original implementation of ResNet101Quantized can be found
151
  [here](https://github.com/pytorch/vision/blob/main/LICENSE).
152
+ - The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
153
 
154
  ## References
155
  * [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)