Pytorch mobile optimised own model

gyt.971027: <p>I saw the link<br>
<a href="https://pytorch.org/tutorials/recipes/mobile_perf.html#setup" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/recipes/mobile_perf.html#setup</a><br>
how to optimize the model. But I got stuck in model preparation.<br>
How can I use my model to replace the code below?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/2/524575eb43fcc7d8e803ce0a30130d094bd3bc0d.png" data-download-href="https://discuss.pytorch.org/uploads/default/524575eb43fcc7d8e803ce0a30130d094bd3bc0d" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/2/524575eb43fcc7d8e803ce0a30130d094bd3bc0d.png" alt="image" data-base62-sha1="bJNYdz7VWHovReihaOO0Klb9KJD" width="663" height="500" data-dominant-color="F0F1F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">997×751 31.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

kimishpatel: <p>That part of perf optimization refers to quantization step. If you want to quantize your model, you may want to follow this <a href="https://pytorch.org/docs/stable/quantization.html" class="inline-onebox" rel="noopener nofollow ugc">Quantization — PyTorch 1.9.0 documentation</a>, <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">(beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.9.0+cu102 documentation</a>, <a href="https://pytorch.org/tutorials/recipes/quantization.html" class="inline-onebox" rel="noopener nofollow ugc">Quantization Recipe — PyTorch Tutorials 1.9.0+cu102 documentation</a>. You can google “pytorch quantization” and get some more links.</p>
<p>If you are not trying to quantize the model then you can ignore that step.</p>

gyt.971027: <p>Thanks for the reply, <a class="mention" href="/u/kimishpatel">@kimishpatel</a><br>
And I meet an issue when I run this code:</p>
<pre><code class="lang-auto">subnet.qconfig = torch.quantization.default_qconfig
print(subnet.qconfig)
print(torch.backends.quantized.supported_engines)
subnet.qconfig = torch.quantization.get_default_qconfig('qnnpack')
torch.backends.quantized.engine = 'qnnpack'
torch.quantization.prepare(subnet, inplace=True)

torch.quantization.convert(subnet, inplace=True)
#optimize

</code></pre>
<p>But it prints these errors:</p>
<pre><code class="lang-auto">['none']
terminate called after throwing an instance of 'c10::Error'
what(): quantized engine QNNPACK is not supported
Exception raised from setQEngine at /media/nvidia/WD_NVME/PyTorch/JetPack_4.4.1/pytorch-v1.8.0/aten/src/ATen/Context.cpp:184 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits, std::allocator &gt;) + 0xa0 (0x7f46cae290 in /home/yuantian/.local/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string&lt;char, std::char_traits, std::allocator &gt; const&amp;) + 0xb4 (0x7f46cab0fc in /home/yuantian/.local/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #2: at::Context::setQEngine(c10::QEngine) + 0x164 (0x7f5c990c1c in /home/yuantian/.local/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so)
frame #3: THPModule_setQEngine(_object*, _object*) + 0x94 (0x7f60d46a4c in /home/yuantian/.local/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

frame #5: python3() [0x52ba70]
frame #7: python3() [0x529978]
frame #9: python3() [0x5f4d34]
frame #11: python3() [0x5a7228]
frame #12: python3() [0x582308]
frame #16: python3() [0x529978]
frame #17: python3() [0x52b8f4]
frame #19: python3() [0x52b108]
frame #24: __libc_start_main + 0xe0 (0x7f79b09720 in /lib/aarch64-linux-gnu/libc.so.6)
frame #25: python3() [0x420e94]

Aborted (core dumped)
</code></pre>
<p>And my system is ubuntu 18, My torch version is 1.8.0, python version is 3.6.9<br>
Do you know why? And thanks for the reply again.</p>

kimishpatel: <p>That is strange. This mean pytorch version you have is not compiled with QNNPACK support. <a class="mention" href="/u/ivankobzarev">@IvanKobzarev</a>, do you know anything about this? Or may be <a class="mention" href="/u/seemethere">@seemethere</a> do you know whether 1.8 release was built with QNNPACK or not (USE_PYTORCH_QNNPACK).</p>

gyt.971027: <p>Thanks for the reply, <a class="mention" href="/u/kimishpatel">@kimishpatel</a> . Yeah, I also think it’s so strange, and pls <a class="mention" href="/u/seemethere">@seemethere</a> and <a class="mention" href="/u/ivankobzarev">@IvanKobzarev</a> do you know how to solve it? It’s very important to me.</p>

gyt.971027: <p>I have set the torch version to 1.7.0 and torchvision to 0.8.1 and have the same problem</p>

gyt.971027: <p><code>QConfig(activation=functools.partial(&lt;class 'torch.quantization.observer.MinMaxObserver'&gt;, reduce_range=True), weight=functools.partial(&lt;class 'torch.quantization.observer.MinMaxObserver'&gt;, dtype=torch.qint8, qscheme=torch.per_tensor_symmetric))</code></p>

gyt.971027: <p>And my platform is Jetson</p>