Torchvision fails when trying to use the gpu

D_Kab: <p>Running on Windows 11, installed using the pip command on the website.<br>
CUDA 1.8</p>
<p>I am able to run using the CPU but when I set device=‘0’ it crashes giving me the following error:</p>
<pre><code class="lang-auto">Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::nms' is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, AutogradMeta, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].
</code></pre>
<p>Running on Windows 11</p>
<p>Verifying pytorch in my console:</p>
<pre><code class="lang-auto">PS C:\Users\Dusti&gt; python
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.rand(3)
tensor([0.2497, 0.1301, 0.3781])
&gt;&gt;&gt; torch.cuda.is_available()
True
&gt;&gt;&gt; torch.cuda.device_count()
1
&gt;&gt;&gt; torch.cuda.current_device()
0
&gt;&gt;&gt; torch.cuda.device(0)
&lt;torch.cuda.device object at 0x0000017B6D0104F0&gt;
&gt;&gt;&gt; torch.cuda.get_device_name(0)
'NVIDIA GeForce RTX 3070 Ti'
</code></pre>
<p>My script:</p>
<pre><code class="lang-auto">import cv2

from ultralytics import YOLO

def main():
    cap = cv2.VideoCapture(0)
    cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 780)

    model = YOLO("yolov8l.pt")

    while True:
        ret, frame = cap.read()
        cv2.imshow("yolov8", frame)

        result = model(frame, device='0')

        if(cv2.waitKey(30) == 27):
            break


if __name__ == "__main__":
    main()
</code></pre>

ptrblck: <p>Did you install a <code>torchvision</code> version with CUDA support?<br>
If so, what does <code>torchvision.version.cuda</code> return?</p>

D_Kab: <p>I did use this command (generated by selections on the site) to install it all:</p>
<pre><code class="lang-auto">pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</code></pre>
<p>Oh, maybe the command does not install the cuda version of torchvision?</p>
<p>When I type that command (seems to import torchvision, but the cuda bit fails):</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torchvision
&gt;&gt;&gt; torchvision.version.cuda
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
AttributeError: module 'torchvision.version' has no attribute 'cuda'
&gt;&gt;&gt;
</code></pre>

D_Kab: <p>I did a pip list and it looks like it’s not a cuda version</p>
<pre><code class="lang-auto">torch              2.0.0+cu118
torchaudio         2.0.1+cu118
torchvision        0.15.1
tqdm               4.65.0
typing_extensions  4.5.0
tzdata             2023.3
ultralytics        8.0.61
urllib3            1.26.15
wheel              0.40.0
</code></pre>
<p>any recommendations for the best way to find a cuda version? or commands to uninstall / reinstall a cuda version of the version I have (0.15.1)?</p>

ptrblck: <p>Your install command should work and also installs the proper libraries in my setup:</p>
<pre><code class="lang-auto">pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
Looking in indexes: https://download.pytorch.org/whl/cu118
Collecting torch
  Downloading https://download.pytorch.org/whl/cu118/torch-2.0.0%2Bcu118-cp38-cp38-linux_x86_64.whl (2267.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 GB 56.7 MB/s eta 0:00:00
Collecting torchvision
  Downloading https://download.pytorch.org/whl/cu118/torchvision-0.15.1%2Bcu118-cp38-cp38-linux_x86_64.whl (33.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 33.9/33.9 MB 57.9 MB/s eta 0:00:00
Collecting torchaudio
  Downloading https://download.pytorch.org/whl/cu118/torchaudio-2.0.1%2Bcu118-cp38-cp38-linux_x86_64.whl (4.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 54.0 MB/s eta 0:00:00
...
pip list | grep torch
torch              2.0.0+cu118
torchaudio         2.0.1+cu118
torchvision        0.15.1+cu118
</code></pre>
<p>Could you uninstall <code>torchvision</code> and reinstall it?</p>

D_Kab: <p>Hmm, it doesn’t seem to install the cuda version when I do a pip install torchvision<br>
I event tried to do a pip install torchvision==0.15.1+cu118</p>
<p>Uninstalled it:</p>
<pre><code class="lang-auto">  Successfully uninstalled torchvision-0.15.1
PS C:\Users\Dusti&gt; pip install torchvision==0.15.1+cu118
ERROR: Could not find a version that satisfies the requirement torchvision==0.15.1+cu118 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1)
ERROR: No matching distribution found for torchvision==0.15.1+cu118
</code></pre>

D_Kab: <p>I did find the versions on the website<br>
<a href="https://download.pytorch.org/whl/torchvision/" class="onebox" target="_blank" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torchvision/</a></p>
<p>Any recommendation?<br>
cp310, cp311, cp38, cp39?</p>
<p>Looks like for you above it downloaded cp38</p>

ptrblck: <p>This tag corresponds to your used Python version and since I’m using Python 3.8 the <code>cp38</code> wheels were picked.</p>

D_Kab: <p>Perfect!</p>
<p>Thanks for clearing that up.  Installed the cp310 version and everything works good.</p>

Ibrahim_Kamran: <p>Sir how are you sir I have installed torch currently my torch and torchaudio is working on GPU  but torchvision is not working on GPU sir how I could run it on GPU I have also installed torchvision compatible to python 3.11 from <a href="https://download.pytorch.org/whl/torchvision/" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torchvision/</a> but how could I give command to run on GPU  GPU</p>