M1 macOS 12.3 torchvision.ops.nms error

dotw: <p>I installed the lastest <code>torch</code> and <code>torchvision</code> nightly as per online instructions, eager to testdrive M1 GPU support. Testing with <code>mps.is_available()</code> returns <code>True</code> (yeah!). But when running <code>YoloX</code> model, the system crashes with the following error:</p>
<pre><code class="lang-auto">ox_files/detector.py", line 321, in _postprocess
    nms_out_index = torchvision.ops.nms(
   File "/usr/local/Caskroom/miniforge/base/envs/pt/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 40, in nms
    _assert_has_ops()
   File "/usr/local/Caskroom/miniforge/base/envs/pt/lib/python3.8/site-packages/torchvision/extension.py", line 33, in _assert_has_ops
    raise RuntimeError(
 
2022-05-30 11:07:08 root  ERROR:  RuntimeError: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.
</code></pre>
<p>NB: The above code has been CI/CD-tested and runs perfectly on M1 macOS (torch cpu), Intel macOS, Windows and Linux, so it’s not a code error.</p>
<p>My M1 MacBook Air specs:</p>
<ul>
<li>macOS 12.3.1</li>
<li>python 3.8</li>
<li>
<code>platform()</code> = <code>macOS-12.3.1-arm64-arm-64bit</code>
</li>
<li><code>torch==1.13.0.dev20220529</code></li>
<li><code>torchvision==0.14.0a0+d592925</code></li>
</ul>
<p>From the error messages, it seems that either</p>
<ol>
<li>
<code>torchvision.ops.nms</code> is buggy, or</li>
<li>the <code>torch</code> and <code>torchvision</code> versions are incompatible.</li>
</ol>
<p>Wonder if any one knows whether it is <code>1</code>, in which case I will abort my M1 test until later builds, or <code>2</code>?  If <code>2</code>, how do I get and build a compatible <code>torchvision</code> version (appreciate some detailed instructions/commands here).</p>

albanD: <p>Hi,</p>
<p>We very recently added the torchvision nightly to avoid this. Can you try to uninstall and re-install both packages now?</p>

dotw: <p>Hi <a class="mention" href="/u/alband">@albanD</a>,</p>
<p>Thanks for the update. I have uninstall and reinstalled the nightly builds (via pip). The C++ error is gone, but it gets replaced by a type error:</p>
<pre><code class="lang-auto">   File "/Users/dotw/src/ongtw/PeekingDuck/peekingduck/pipeline/nodes/model/yoloxv1/yolox_files/model.py", line 309, in forward
    outputs_tensor = self.decode_outputs(outputs_tensor, xin[0].type())
   File "/Users/dotw/src/ongtw/PeekingDuck/peekingduck/pipeline/nodes/model/yoloxv1/yolox_files/model.py", line 339, in decode_outputs
    grids_tensor = torch.cat(grids, dim=1).type(dtype)
 
2022-06-03 11:31:47 root  ERROR:  ValueError: invalid type: 'torch.mps.FloatTensor'
</code></pre>
<p>Looks like <code>torch.mps.FloatTensor</code> type is not understood by <code>torch.cat</code>: perhaps some internals need updating?</p>
<p>Current torch versions:</p>
<ul>
<li><code>torch==1.13.0.dev20220602</code></li>
<li><code>torchvision==0.14.0a0+f9f721d</code></li>
</ul>

albanD: <p>Thanks for the report.</p>
<p>The <code>type()</code> method is indeed not supported. In general, we would recommend not to use it and specify explicitely device/dtype.<br>
I opened an issue to track this: <a href="https://github.com/pytorch/pytorch/issues/78929" class="inline-onebox">Add type() support for mps backend · Issue #78929 · pytorch/pytorch · GitHub</a></p>