Bug: [W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware

ariel5: <p>Hi all,<br>
I trained a model on my ubuntu server using a GPU.<br>
I am now trying to load this model on a python website (using a server that has only a CPU).<br>
I am successfully able to load the model, but when I try to do inference I get an error:</p>
<p>[W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/9/594e2df50ba94bdb3061169932d81cd53e7460ec.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/594e2df50ba94bdb3061169932d81cd53e7460ec" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/9/594e2df50ba94bdb3061169932d81cd53e7460ec_2_690x304.jpeg" alt="image" data-base62-sha1="cK1ZBRfSZvMoK2wcFZkRssShvaI" width="690" height="304" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/9/594e2df50ba94bdb3061169932d81cd53e7460ec_2_690x304.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/5/9/594e2df50ba94bdb3061169932d81cd53e7460ec.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/9/594e2df50ba94bdb3061169932d81cd53e7460ec.jpeg 2x" data-dominant-color="2F2E2E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">808×357 33.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>what can I do to fix this?</p>

ptrblck: <p>Is this really an error (and thus your application is crashing) or just a warning (I assume the latter)?<br>
If I’m not mistaken, NNPACK needs AVX2 CPU instructions, so if these are not supported on your CPU, you would get this warning. You could potentially build PyTorch from source without NNPACK to silence this warning or ignore it.</p>

ariel5: <p>Hi,</p>
<p>This is a runtime error and it makes the program crash.</p>
<p>How can I build pytorch from source without NNPACK?</p>
<p>Or - is there any other way that you would recommend?</p>
<p>I am trying to use a model that I trained on a server that has GPUs and the current machine doesn’t.</p>
<p>Thank you,</p>
<p>Ariel</p>

ptrblck: <p>You can pass <code>USE_NNPACK=0</code> to the <code>setup.py</code> following the instructions to build from source from <a href="https://github.com/pytorch/pytorch#from-source">here</a>.</p>

Omar_AlSuwaidi: <p>Hey I’m getting the same error running the new M1. I tried following the “From Source” blog, but I couldn’t figure out where I would pass <em>USE_NNPACK=0</em>. I saw something about “export environment variable”, but I’m not sure where to do so. Could you please assist me?</p>
<p>Thanks!</p>

ptrblck: <p>You can prepend the env variable to the build command:</p>
<pre><code class="lang-auto">USE_NNPACK=0 python setup.py install
</code></pre>
<p>or set it via:</p>
<pre><code class="lang-auto">export USE_NNPACK=0
</code></pre>
<p>before building PyTorch from source.</p>

Omar_AlSuwaidi: <p>Hey, thanks a lot for your response! I attempted building from source and after passing <code>git submodule update --init --recursive</code> I passed <code>python setup.py build</code> and got the error:</p>
<blockquote>
<p>RuntimeError: no cmake or cmake3 with version &gt;= 3.5.0 found</p>
</blockquote>
<p>Passing <code>USE_NNPACK=0 python setup.py install</code> raised the same error!</p>

gakadam: <p>You may have to install <code>cmake</code>.<br>
On linux, check cmake version using <code>cmake --version</code>.</p>

Omar_AlSuwaidi: <p>Genius! Thanks a lot.</p>

Omar_AlSuwaidi: <aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="107518">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>USE_NNPACK=0 python setup.py install</code></p>
</blockquote>
</aside>
<p>Building from source and running the above command now raises this error:</p>
<blockquote>
<p>raise CalledProcessError(retcode, cmd)<br>
subprocess.CalledProcessError: Command ‘[‘cmake’, ‘–build’, ‘.’, ‘–target’, ‘install’, ‘–config’, ‘Release’, ‘–’, ‘-j’, ‘8’]’ returned non-zero exit status 1.</p>
</blockquote>
<p>Could the issue be due to incompatibility with the M1 ARM-based system?</p>

ptrblck: <p>The posted stack trace doesn’t include the error message, so you would need to search a little bit further up to see why the build is failing.</p>

bibekx: <p>This worked for me, Thanks!!</p>

CracKCatZ: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a> ,<br>
I’am running in the same error. But mine is looking different and I don’t really know if this is only one error messsage.<br>
<strong>My OS:</strong><br>
Kali Linux</p>
<p><strong>My error message:</strong></p>
<pre><code class="lang-auto">/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: stft will require the return_complex parameter be explicitly  specified in a future PyTorch release. Use return_complex=False  to preserve the current behavior or return_complex=True to return  a complex output. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:653.)
  normalized, onesided, return_complex)
/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: The function torch.rfft is deprecated and will be removed in a future PyTorch release. Use the new torch.fft module functions, instead, by importing torch.fft and calling torch.fft.fft or torch.fft.rfft. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:590.)
  normalized, onesided, return_complex)
[W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware.
</code></pre>
<p>After this error message is being displayed it is sort of creating an endless loop which is printing out every second a new row. So my program is being completely crashed because of this error message</p>
<p>Is someone able to help me out with that or know what this error message is caused by?<br>
Thank’s for every suggestion and help in advance:)</p>

ptrblck: <p>It seems you are hitting the <a href="https://discuss.pytorch.org/t/bug-w-nnpack-cpp-80-could-not-initialize-nnpack-reason-unsupported-hardware/107518/2">same issues</a>, so you could also try to rebuild PyTorch without NNPACK or ignore the warning (do you know, if this is a warning or an error?).</p>

CracKCatZ: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I think it’s more like an error because the program is breaking down completely after this message.<br>
Is this one error message? Or are there two?<br>
Because of this:</p>
<pre><code class="lang-auto">/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: stft will require the return_complex parameter be explicitly  specified in a future PyTorch release. Use return_complex=False  to preserve the current behavior or return_complex=True to return  a complex output. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:653.)
  normalized, onesided, return_complex)
/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: The function torch.rfft is deprecated and will be removed in a future PyTorch release. Use the new torch.fft module functions, instead, by importing torch.fft and calling torch.fft.fft or torch.fft.rfft. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:590.)
  normalized, onesided, return_complex)
</code></pre>

CracKCatZ: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> are you still there?</p>

ptrblck: <p>The two new outputs are warnings, so you might want to check them and fix the usage in your code.<br>
If the NNPACK issue is creating an error, please refer to the previous post and try to rebuild PyTorch without NNPACK.</p>

CracKCatZ: <p>Hey <a class="mention" href="/u/ptrblck">@ptrblck</a> is there any way to install pytorch and right after torchaudio? Becuase when I try to install torchaudio via conda or pip my torch module which I installed from source is being deleted</p>

ptrblck: <p>You could try to use <code>--no-deps</code> while installing <code>torchaudio</code>, but note that pip or conda would only downgrade PyTorch, if <code>torchaudio</code> specifies a specific PyTorch version as a dependency, so your installation might break in case your source build is not compatible with the desired <code>torchaudio</code> version.</p>
<p>A proper approach would be to either build <code>torchaudio</code> also from source or to try to install the nightly releases, which should have relaxed requirements.</p>

craigyang: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, I tried both of these options to set <code>USE_NNPACK=0</code>, but still got the error message as in the title.</p>
<p>For the first option, I did</p>
<pre><code class="lang-auto">USE_NNPACK=0 pip3 install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cu111/torch_nightly.html
</code></pre>
<p>to install the latest nightly package; for the second, I added</p>
<pre><code class="lang-auto">export USE_NNPACK=0;
</code></pre>
<p>to right before my previous command.</p>
<p>Is this behavior expected? Thanks!</p>
<p>(This error comes right after the UserWarning that I should use <code>torch.linalg.qr</code> instead of <code>torch.qr</code>. Is it related to the error here?)</p>