CUDA&CUDNN local installation and using these in conda envs

JoohyungLee0106: <p>I have installed CUDA and CUDNN by following NVIDIA guidelines. When I installed CUDA and CUDNN, I did not activate any conda environment.<br>
Would this be a problem when I try to use CUDA&amp;CUDNN in a conda environment? (in PyTorch)</p>
<p><a href="https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#verify" rel="noopener nofollow ugc">CUDNN Verification</a> does not work with any environment but woks with the base environment.</p>

ptrblck: <p>I assume you’ve installed a local CUDA toolkit (with the <code>nvcc</code> CUDA compiler) and added cuDNN to it.<br>
If so then virtual conda environments wouldn’t change anything as the CUDA toolkit it not a conda or pip package and should be visible in all envs (try to run <code>which nvcc</code> and it should point to e.g. <code>/usr/local/cuda</code>).<br>
Note that a locally installed CUDA toolkit will only be used if you are building PyTorch from source or a custom CUDA extensions. The pip wheels and conda binaries ship with their own CUDA runtime and you would only need to install a sufficiently new NVIDIA driver to execute these binaries.</p>

trevor_snedden: <p>Hello ptrblck, I know this is an old thread, but I found it while trying to troubleshoot a different problem. Would you recommend the local download from the Nvidia Cudatoolkit over the Conda Nvidia download? Or does it not matter? I do know that nvrtc builtins64_121.dll isn’t in the Cudatoolkit downloaded from Nvidia’s site. This is why I’m curious.</p>

ptrblck: <aside class="quote no-group" data-username="trevor_snedden" data-post="3" data-topic="136358">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/trevor_snedden/48/69158_2.png" class="avatar"> trevor_snedden:</div>
<blockquote>
<p>Would you recommend the local download from the Nvidia Cudatoolkit over the Conda Nvidia download? Or does it not matter?</p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="136358">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Note that a locally installed CUDA toolkit will only be used if you are building PyTorch from source or a custom CUDA extensions. The pip wheels and conda binaries ship with their own CUDA runtime and you would only need to install a sufficiently new NVIDIA driver to execute these binaries.</p>
</blockquote>
</aside>
<p>To execute PyTorch workloads with CUDA support you should be able to install the binaries with an NVDIA driver as the locally installed CUDA toolkit won’t be used.</p>

trevor_snedden: <p>I appreciate your quick response, and maybe you can help me understand something better. I was trying to use torcheval, which I know isn’t stable yet, but it only started to work after I copied the file nvrtc-builtins64_121.dll from my Conda PyTorch environment into the locally installed CUDA toolkit bin directory. Otherwise it kept saying that the file was not found.<br>
Is this simply due to the path not going towards the binaries? I’d prefer to delete the local download if I can, but I’m still way to green to fully comprehend.</p>

ptrblck: <p>I’m not using Windows myself, so don’t have an easy way to reproduce issues.<br>
However, if a plain PyTorch installation using the current stale or nightly version is not working on Windows and points towards missing files, could you create a GitHub issue so we could follow up?</p>

trevor_snedden: <p>Yes, no problem; I appreciate your time.</p>