RuntimeError: Didn't find engine for operation quantized::conv2d_prepack NoQEngine (operator() at /pytorch/aten/src/ATen/native/quantized/cpu/qconv_prepack.cpp:63)

username: JachinMa
cooked: <p>I tried to run tutorial <a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html</a> and faced this problem.</p>
<p>I ran at a supercomputer,cuda 10.1,pytorch 1.4,torchvision 0.5.</p>
<p>total information is</p>
<p>Traceback (most recent call last):<br>
File “q.py”, line 327, in <br>
torch.quantization.convert(myModel,inplace=True)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 316, in convert<br>
convert(mod, mapping, inplace=True)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 316, in convert<br>
convert(mod, mapping, inplace=True)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 317, in convert<br>
reassign[name] = swap_module(mod, mapping)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 339, in swap_module<br>
new_mod = mapping[type(mod)].from_float(mod)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 49, in from_float<br>
return super(ConvReLU2d, cls).from_float(mod)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 257, in from_float<br>
mod.bias is not None, mod.padding_mode)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 29, in <strong>init</strong><br>
padding_mode=padding_mode)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 187, in <strong>init</strong><br>
groups, bias, padding_mode)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 48, in <strong>init</strong><br>
self.set_weight_bias(qweight, bias_float)<br>
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 195, in set_weight_bias<br>
w, b, self.stride, self.padding, self.dilation, self.groups)<br>
RuntimeError: Didn’t find engine for operation quantized::conv2d_prepack NoQEngine (operator() at /pytorch/aten/src/ATen/native/quantized/cpu/qconv_prepack.cpp:63)<br>
frame <span class="hashtag">#0:</span> c10::Error::Error(c10::SourceLocation, std::string const&amp;) + 0x33 (0x7f414c66d193 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libc10.so)<br>
frame <span class="hashtag">#1:</span>  + 0x1e5c2ba (0x7f414e91c2ba in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)<br>
frame <span class="hashtag">#2:</span>  + 0x1e5d083 (0x7f414e91d083 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)<br>
frame <span class="hashtag">#3:</span>  + 0x33eb009 (0x7f414feab009 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)<br>
frame <span class="hashtag">#4:</span>  + 0x40e8547 (0x7f4150ba8547 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)<br>
frame <span class="hashtag">#5:</span>  + 0x6de077 (0x7f4197a4e077 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch_python.so)<br>
frame <span class="hashtag">#6:</span>  + 0x6a95c4 (0x7f4197a195c4 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch_python.so)<br>
frame <span class="hashtag">#7:</span>  + 0x2961c4 (0x7f41976061c4 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch_python.so)<br>
<br>
frame <span class="hashtag">#61:</span> __libc_start_main + 0xf5 (0x7f41abb49445 in /lib64/libc.so.6)</p>

username: jerryzh168
cooked: <p>This means you didn’t compile fbgemm, can you print <code>torch.backends.quantized.supported_engines</code>?</p>

username: gyt.971027
cooked: <p>Hello, Jerry<br>
I have a similar problem, and this is my code error:</p>
<blockquote>
<p>UserWarning: must run observer before calling calculate_qparams.                                    Returning default scale and zero point<br>
Returning default scale and zero point "<br>
Traceback (most recent call last):<br>
File “mobile_test.py”, line 122, in <br>
data_loader,bn_list[0],net_config,device=‘cuda:0’))<br>
File “mobile_test.py”, line 55, in prepare_subnet<br>
torch.quantization.convert(subnet, inplace=True)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 473, in convert<br>
convert_custom_config_dict=convert_custom_config_dict)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 508, in _convert<br>
custom_module_class_mapping)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 508, in _convert<br>
custom_module_class_mapping)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 508, in _convert<br>
custom_module_class_mapping)<br>
[Previous line repeated 3 more times]<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 509, in _convert<br>
reassign[name] = swap_module(mod, mapping, custom_module_class_mapping)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 534, in swap_module<br>
new_mod = mapping[type(mod)].from_float(mod)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 97, in from_float<br>
return super(ConvReLU2d, cls).from_float(mod)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 418, in from_float<br>
return _ConvNd.from_float(cls, mod)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 220, in from_float<br>
return cls.get_qconv(mod, activation_post_process, weight_post_process)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 191, in get_qconv<br>
mod.bias is not None, mod.padding_mode)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 74, in <strong>init</strong><br>
padding_mode=padding_mode)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 376, in <strong>init</strong><br>
False, _pair(0), groups, bias, padding_mode)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 74, in _init<br>
self.set_weight_bias(qweight, bias_float)<br>
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 384, in set_weight_bias<br>
w, b, self.stride, self.padding, self.dilation, self.groups)<br>
RuntimeError: Didn’t find engine for operation quantized::conv2d_prepack NoQEngine</p>
</blockquote>
<p>I try to print the code</p>
<blockquote>
<p><code>torch.backends.quantized.supported_engines</code><br>
But the result is [None]<br>
And my system is ubuntu</p>
</blockquote>

username: gyt.971027
cooked: <p>And here is my code:<br>
<span class="hashtag">#fuse</span> model<br>
fuse_model(subnet)</p>
<pre><code>#quantize model
subnet.qconfig = torch.quantization.get_default_qconfig('qnnpack')
torch.quantization.prepare(subnet, inplace=True)
# Calibrate
print(torch.backends.quantized.supported_engines)
torch.quantization.convert(subnet, inplace=True)
#optimize
script_subnet = torch.jit.script(model)
script_subnet_optimized = optimize_for_model(script_model)</code></pre>

username: HDCharles
cooked: <p>well qnnpack is disabled by default on x86 systems per the info here: <a href="https://github.com/pytorch/pytorch/issues/29327" class="inline-onebox" rel="noopener nofollow ugc">RuntimeError: Didn't find engine for operation quantized::conv_prepack NoQEngine · Issue #29327 · pytorch/pytorch · GitHub</a></p>
<p>that link also includes a workaround, did you try that? I can see that you posted there as well.</p>
<p>more info here too: <a href="https://github.com/pytorch/pytorch/issues/31684#issuecomment-572584403" class="inline-onebox" rel="noopener nofollow ugc">The Feature Request of Loading Quantized TorchScript Model on Windows with libtorch · Issue #31684 · pytorch/pytorch · GitHub</a></p>
<p>if you still can’t solve this, could you open a new issue in the forum and include your env information?</p>