How do I get my older GPU that supports CUDA to work with PyTorch 1.4?

Vedaant_V: <p>I’m using anaconda for python 3.7, and have CUDA 10.1 installed. My GPU is a GTX 760, with 3.0 CUDA capability. I’m getting true values for <code>torch.cuda.is_available</code> , but I keep getting this error whenever I try to use CUDA for anything.</p>
<p>Here’s the error:<br>
<code>RuntimeError: CUDA error: no kernel image is available for execution on the device</code></p>
<p>I was wondering how I could use my GPU with PyTorch, and if it would work for newer versions. I’ve heard that other people had to build their pytorch from source, but I am new and the instructions weren’t completely clear to me.</p>
<p>Any assistance would be greatly appreciated, and thank you in advance!</p>

ptrblck: <p>The binaries are built for devices with compute capability &gt;= 3.7, which is why you are seeing this error.<br>
To build from source, you could follow <a href="https://github.com/pytorch/pytorch#from-source">these instructions</a>.<br>
Let us know, if you get stuck and we can most likely help you out. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Vedaant_V: <p>Thanks for responding!</p>
<p>While trying to build from source, I was given this issue, and I am unsure as to how I can work around this! Any input would be greatly appreciated!</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "setup.py", line 737, in &lt;module&gt;
    build_deps()
  File "setup.py", line 316, in build_deps
    cmake=cmake)
  File "C:\Users\user\pytorch\tools\build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "C:\Users\user\pytorch\tools\setup_helpers\cmake.py", line 341, in build
    self.run(build_args, my_env)
  File "C:\Users\user\pytorch\tools\setup_helpers\cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "D:\Users\user\Anaconda3\lib\subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 1.
</code></pre>

ptrblck: <p>Could you post the complete stack trace as it seems the actual error message is missing? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Vedaant_V: <p>I’m not fully familiar with what a stack trace is, and it is hard for me to find where the error began. Here is the closest approximation that I could find. If you would like the entire output, I can try to obtain that as well. My apologies if this is not what you asked for. Thank you for your continued assistance!</p>
<pre><code class="lang-auto">
[1539/2035] Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_piecewise_linear_transform_op.cu.obj
FAILED: caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_piecewise_linear_transform_op.cu.obj
cmd.exe /C "cd /D C:\Users\user\pytorch\build\caffe2\CMakeFiles\torch_cuda.dir\operators &amp;&amp; D:\Users\user\Anaconda3\Library\bin\cmake.exe -E make_directory C:/Users/user/pytorch/build/caffe2/CMakeFiles/torch_cuda.dir/operators/. &amp;&amp; D:\Users\user\Anaconda3\Library\bin\cmake.exe -D verbose:BOOL=OFF -D build_configuration:STRING=Release -D generated_file:STRING=C:/Users/user/pytorch/build/caffe2/CMakeFiles/torch_cuda.dir/operators/./torch_cuda_generated_piecewise_linear_transform_op.cu.obj -D generated_cubin_file:STRING=C:/Users/user/pytorch/build/caffe2/CMakeFiles/torch_cuda.dir/operators/./torch_cuda_generated_piecewise_linear_transform_op.cu.obj.cubin.txt -P C:/Users/user/pytorch/build/caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_piecewise_linear_transform_op.cu.obj.Release.cmake"
piecewise_linear_transform_op.cu
piecewise_linear_transform_op.cu
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(204): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(233): note: see reference to class template instantiation 'thrust::cuda_cub::__scan_by_key::ScanByKeyAgent&lt;KeysInputIt,ValuesInputIt,ValuesOutputIt,EqualityOp,ScanOp,Size,T,Inclusive&gt;::PtxPlan&lt;Arch&gt;' being compiled
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(567): note: see reference to class template instantiation 'thrust::cuda_cub::__scan_by_key::ScanByKeyAgent&lt;KeysInputIt,ValuesInputIt,ValuesOutputIt,EqualityOp,ScanOp,Size,T,Inclusive&gt;' being compiled
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(204): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(210): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(210): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(217): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(217): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(223): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(232): note: see reference to class template instantiation 'thrust::cuda_cub::__scan_by_key::ScanByKeyAgent&lt;KeysInputIt,ValuesInputIt,ValuesOutputIt,EqualityOp,ScanOp,Size,T,Inclusive&gt;::PtxPlan&lt;Arch&gt;::TempStorage' being compiled
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(223): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(224): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(224): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(225): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
D:/CUDA/CUDA 10.1/Development/include\thrust/system/cuda/detail/scan_by_key.h(225): error C2993: 'KeysInputIt': illegal type for non-type template parameter '__formal'
CMake Error at torch_cuda_generated_piecewise_linear_transform_op.cu.obj.Release.cmake:281 (message):
  Error generating file
  C:/Users/user/pytorch/build/caffe2/CMakeFiles/torch_cuda.dir/operators/./torch_cuda_generated_piecewise_linear_transform_op.cu.obj


[1540/2035] Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_normalize_ops.cu.obj
normalize_ops.cu
normalize_ops.cu
[1541/2035] Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_prelu_op.cu.obj
prelu_op.cu
prelu_op.cu
[1542/2035] Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_pool_op.cu.obj
pool_op.cu
pool_op.cu
[1543/2035] Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_perplexity_op.cu.obj
perplexity_op.cu
perplexity_op.cu
[1544/2035] Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/operators/torch_cuda_generated_pow_op.cu.obj
pow_op.cu
pow_op.cu
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "setup.py", line 737, in &lt;module&gt;
    build_deps()
  File "setup.py", line 316, in build_deps
    cmake=cmake)
  File "C:\Users\user\pytorch\tools\build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "C:\Users\user\pytorch\tools\setup_helpers\cmake.py", line 341, in build
    self.run(build_args, my_env)
  File "C:\Users\user\pytorch\tools\setup_helpers\cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "D:\Users\user\Anaconda3\lib\subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 1.

(base) C:\Users\user\pytorch&gt;
</code></pre>

ptrblck: <p>Thanks for the error message.<br>
Could you refer to <a href="https://github.com/pytorch/pytorch/issues/25393#issuecomment-566286374">this post</a>, install the latest VS 2019, and try to rebuild it?</p>
<p>Please let us know, if you are continuously running into this error, as it seems to be hard to reproduce on the Windows CI.</p>

peterjc123: <p>No need to install VS 2019. Just retry the build a few times. If that doesn’t work, then you may need to build from scratch.</p>

Vedaant_V: <p>I currently have the latest from the Visual Studio build tools 2019. Do I have to install the IDE as well? Or are there any changes that I need to make for in the build?</p>
<pre><code class="lang-auto">:: [Optional] If you want to build with VS 2019 generator, please change the value in the next line to `Visual Studio 16 2019`.
:: Note: This value is useless if Ninja is detected. However, you can force that by using `set USE_NINJA=OFF`.
set CMAKE_GENERATOR=Visual Studio 15 2017
</code></pre>
<p>I see this in the tutorial from building from source, but I did not include it. If I should add this, what changes would I need to make?</p>

peterjc123: <p>No, just retry the build. I suppose this line is useless since you are using Ninja.</p>

Vedaant_V: <p>I have restarted the process 3 times, would you recommend continuing? If I need to build it from scratch, how is it different from building from source? I am unfamiliar with the process in general.</p>

peterjc123: <p>Would you please try out <code>set MAX_JOBS=1</code> and then continue the build job?</p>

Vedaant_V: <p>I will give it a try, although when I put in the command, it did not give me any messages indicating that it worked. I assumed that it went through, so I just entered in <code>python setup.py install</code> again.</p>

peterjc123: <p>That’s fine. Just post here when you meet a problem.</p>

Vedaant_V: <p>The installation worked, but I have yet to test if my cuda now works. I have run into a slight problem with torchvision not being found</p>
<p><code>ModuleNotFoundError: No module named 'torchvision'</code></p>
<p>Should I just try <code>conda install torchvision</code>, or would you recommend trying something else? Thank you again for your continued assistance!</p>

peterjc123: <p>Wait! You’ll need to compile torchvision from source too.</p>

Vedaant_V: <p>I will try this. I may be mistaken in the reading of the log, but I think this might be downloading pytorch again.</p>
<pre><code class="lang-auto">(base) C:\Users\user\pytorch&gt;pip install torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torchvision===0.5.0
  Downloading https://files.pythonhosted.org/packages/7d/3e/2b5ddf744226159dc90a52f0d044c0de7c5ca4f42d12a350a674ebb6fb2a/torchvision-0.5.0-cp37-cp37m-win_amd64.whl (1.2MB)
     |████████████████████████████████| 1.2MB 595kB/s
Requirement already satisfied: six in d:\users\user\anaconda3\lib\site-packages (from torchvision===0.5.0) (1.12.0)
Requirement already satisfied: numpy in d:\users\user\anaconda3\lib\site-packages (from torchvision===0.5.0) (1.16.5)
Requirement already satisfied: pillow&gt;=4.1.1 in d:\users\user\anaconda3\lib\site-packages (from torchvision===0.5.0) (6.2.0)
Collecting torch==1.4.0 (from torchvision===0.5.0)
  Downloading https://download.pytorch.org/whl/cu92/torch-1.4.0%2Bcu92-cp37-cp37m-win_amd64.whl (641.8MB)
     |██████████████                  | 286.8MB 726kB/s eta 0:08:09
</code></pre>

peterjc123: <p>Sorry for suggesting the wrong thing. Please build torchvision from source too.</p>

Vedaant_V: <p>Will the command that I entered impact my source build of pytorch?</p>

peterjc123: <p>If that command finishes, then yes. You may need to uninstall them first. <code>pip uninstall torch torchvision</code> And then, reinstall pytorch. <code>python setup.py install</code></p>

Vedaant_V: <p>Would you be able to provide any links for building torchvision from source?</p>