NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend,

username: Oguzhan
cooked: <p>Hi, when I try to run Yolov5 I got this error:</p>
<p>NotImplementedError: Could not run ‘torchvision::nms’ with arguments from the ‘CUDA’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit <a href="https://fburl.com/ptmfixes" class="inline-onebox" rel="noopener nofollow ugc">Internal Login</a> for possible resolutions. ‘torchvision::nms’ is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Named, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, Tracer, Autocast, Batched, VmapMode].</p>
<p>Is there any solution for that? I tried various of Pytorch-Cuda version combinations like 1.9.0-10.2 , 1.9.0-11.1 , 1.7-10.2 etc. but none of them worked?</p>

username: ptrblck
cooked: <p>I guess you might not have installed <code>torchvision</code> with the CUDA runtime? If so, try to install the latest <code>torchvision</code> release using the same CUDA runtime as used in your PyTorch installation.</p>

username: Oguzhan
cooked: <p>pip3 install torch==1.9.0+cu102 torchvision==0.10.0+cu102 torchaudio===0.9.0 -f <a href="https://download.pytorch.org/whl/torch_stable.html" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torch_stable.html</a>   I used command like this, I think I installed it with cuda.</p>

username: ptrblck
cooked: <p>It works on my setup using the <code>torchvision</code> binaries with the CUDA runtime and fails with the CPU-only package:</p>
<pre><code class="lang-python">import torch
import torchvision

device = 'cuda'
boxes = torch.tensor([[0., 1., 2., 3.]]).to(device)
scores = torch.randn(1).to(device)
iou_thresholds = 0.5

print(torchvision.ops.nms(boxes, scores, iou_thresholds))
</code></pre>
<p>With CUDA:</p>
<pre><code class="lang-auto">pip install torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html

python tmp.py 
&gt;tensor([0], device='cuda:0')
</code></pre>
<p>Without CUDA:</p>
<pre><code class="lang-auto">pip uninstall torchvision
pip install torchvision==0.10.0+cpu -f https://download.pytorch.org/whl/torch_stable.html

python tmp.py
Traceback (most recent call last):
  File "tmp.py", line 10, in &lt;module&gt;
    print(torchvision.ops.nms(boxes, scores, iou_thresholds))
  File "/opt/conda/envs/tmp/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 35, in nms
    return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::nms' is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Named, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, Tracer, Autocast, Batched, VmapMode].
</code></pre>

username: Oguzhan
cooked: <p>When I changed the IDE, it worked. Probably I have more than one enviremont that I was not aware of it. Thank you for answering</p>

username: ani0075
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>Does the standard command on the website</p>
<pre data-code-wrap="code"><code class="lang-nohighlight">conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
</code></pre>
<p>install torchvision with CUDA? Or do we need to do it explicitly as shown (+cu102) to avoid this error message?</p>

username: ptrblck
cooked: <p>This command will install <code>torchvision</code> with CUDA support as seen during the installation:</p>
<pre><code class="lang-python">...
The following packages will be downloaded:
...
torchvision-0.11.3         |       py38_cu102        30.1 MB  pytorch
</code></pre>

username: BobPupkin
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
If I use ResNet50 everything works, but if I switch to fasterrsnn_resnet50_fpn (Works if I run on CPU). It produces the CUDA error (same as author of the post has).<br>
What should I do?<br>
Thank you in advance.</p>

username: ptrblck
cooked: <p>See <a href="https://discuss.pytorch.org/t/notimplementederror-could-not-run-torchvision-nms-with-arguments-from-the-cuda-backend-this-could-be-because-the-operator-doesnt-exist-for-this-backend/132352/4">this post</a>: you are most likely using the CPU-only <code>torchvision</code> binary.</p>

username: BobPupkin
cooked: <p>No, it was not the case, I solved the problem by installing torchaudio and earlier cuda version.</p>

username: Yiming_B
cooked: <p>I created a virtual env with Python 3.9 and follow the “%pip install -qr requirements.txt comet_ml” command as suggested in “tutorial.ipynb”. Later, I also ran the Pytorch installation command from <a href="https://pytorch.org" rel="noopener nofollow ugc">https://pytorch.org</a>. Then I met the same issue.<br>
I believe it is because the torchvision is CPU version, somehow, in the requrements.txt.<br>
The way I fix this problem is that, run the Pytorch installation command first, then pip install -r requirements.txt. In this way, torchvision (GPU version) will be installed first, and the CPU version will be ignored. Hope it helps.</p>

username: robomatt
cooked: <p><a class="mention" href="/u/yiming_b">@Yiming_B</a> thanks a lot!! solved my issue</p>

username: Codersir
cooked: <p>Hey Ptrblck I logged in here to just say Thankyou, Thank you so much! <img src="https://discuss.pytorch.org/images/emoji/apple/heart.png?v=12" title=":heart:" class="emoji" alt=":heart:" loading="lazy" width="20" height="20"></p>

username: zz_z1
cooked: <p>Me too, i run detect.py in miniconda shell, it runs successfully!</p>