Can i run the default cuda 11.3 conda install on cuda 11.6 device?

username: jayz
cooked: <p>Hey,</p>
<p>can I run the default cuda 11.3 conda install on a cuda 11.6 device?<br>
Or do I need to downgrade to cuda 11.3 first? I have an RTX 3080.</p>
<p>I tried installing the cuda 11.6 nighly bins first, following this post:</p><aside class="quote" data-post="1" data-topic="149647">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/9de0a6/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/pytorch-cuda-11-6/149647">PyTorch + CUDA 11.6</a> <a class="badge-category__wrapper " href="/c/deployment/14"><span data-category-id="14" data-drop-close="true" class="badge-category " title="A category of posts focused on production usage of PyTorch. Mobile deployment is out of scope for this category (for now…  )"><span class="badge-category__name">deployment</span></span></a>
  </div>
  <blockquote>
    Hello everyone, 
As a follow-up to this question <a href="https://discuss.pytorch.org/t/pytorch-cuda-11-4/137900" class="inline-onebox">PyTorch + CUDA 11.4</a> 
I have installed these Nvidia drivers version 510.60.02 along with Cuda 11.6. 
I’d like to install Pytorch in a conda virtual environment, and I’ve found in the Pytorch website that we couldn’t choose a stable version that relies on the latest versions of Cuda (the older version is 11.3) <a href="https://pytorch.org/get-started/locally" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a> 
How can I install/build Pytorch with the latest version of Cuda in my conda virtual env ?
  </blockquote>
</aside>

<p>But, with the conda intelpython_full python=3 distribution that I am using, this does not work and raises an error in matplotlib, related to the package freetype (see here):</p><aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/69786885/after-conda-update-python-kernel-crashes-when-matplotlib-is-used">
  <header class="source">

      <a href="https://stackoverflow.com/questions/69786885/after-conda-update-python-kernel-crashes-when-matplotlib-is-used" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/5051894/jtobelem" target="_blank" rel="noopener nofollow ugc">
    <img alt="jtobelem" src="https://i.stack.imgur.com/Z8jYi.png?s=256&amp;g=1" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/69786885/after-conda-update-python-kernel-crashes-when-matplotlib-is-used" target="_blank" rel="noopener nofollow ugc">After conda update, python kernel crashes when matplotlib is used</a>
</h4>

<div class="tags">
  <strong>python, windows, matplotlib, conda</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/5051894/jtobelem" target="_blank" rel="noopener nofollow ugc">
    jtobelem
  </a>
  on <a href="https://stackoverflow.com/questions/69786885/after-conda-update-python-kernel-crashes-when-matplotlib-is-used" target="_blank" rel="noopener nofollow ugc">12:30PM - 31 Oct 21 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>So, there seem to be two choices:</p>
<ul>
<li>Downgrade the GPU to Cuda 11.3. In that case, I have an existing cuda 11.3 conda environment on another pc that works fine, which I could simply clone. So it would be likely to work. The question is; would it even be necessary to downgrade cuda, or is the newer version 11.6 backward compatible with 11.3?</li>
<li>Somehow fix the issue with matplotlib. However, so far I did not manage to solve it. I tried reinstalling involved packages from various channels, but in combination with the nightly bins nothing seems to really solve the issue. I could also try installing into a conda environment which does not utilize the intelpython distribution, however I am reluctant to do this, as the intelpython dist is pretty fast.</li>
</ul>
<hr>
<p>EDIT: I believe I found an answer here:</p><aside class="quote" data-post="4" data-topic="76872">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/is-cuda-back-compatible/76872/4">Is CUDA back-compatible?</a> 
  </div>
  <blockquote>
    As <a class="mention" href="/u/alband">@albanD</a> explained, if you install the PyTorch binaries with cudatoolkit, your local CUDA installation won’t be used, but instead the one shipped with the binaries. 
If I understand the original question right, you would like to install PyTorch with CUDA10.1 as the binaries, while CUDA10.2 is installed locally on your machine? 
If that’s the case, then yes it’ll work as the binaries won’t interfere with the local installation.
  </blockquote>
</aside>
<p>
So, when I use <code>conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch</code> this automatically handles everything running on 11.3, despite nvidia-smi showing cuda 11.6 as the current version?</p>
<p>Thanks!<br>
Best, JZ</p>

username: skoenman
cooked: <p>from this post you can  install version 11.6 <a href="https://discuss.pytorch.org/t/pytorch-cuda-11-6/149647" class="inline-onebox">PyTorch + CUDA 11.6</a><br>
and other packages like torchserve is also available .<br>
pip install torch --pre --extra-index-url <a href="https://download.pytorch.org/whl/nightly/cu116" rel="noopener nofollow ugc">https://download.pytorch.org/whl/nightly/cu116</a><br>
just add to pip install field…</p>

username: jayz
cooked: <p>I tried setting up a conda environment with intelpython 3 first, then installed the nighly bins via pip, as recommended by you. However, i end up getting crashes in the new environment that are related to some matplotlib issue with the package freetype. I have not managed to solve it. Could you give me a hint how to set up a full proper environment with 11.6?</p>
<p>Furthermore, I tried running some training without using any matplotlib functions. This worked, but somehow my RTX3080 delivers not much faster performance than my GTX 1650 Ti (which is a notebook on board GPU). To be more precise, the RTX delivers approximately a speedup x2, which seems not so much. So either, my cuda 11.6 pytorch does not fully utilize the RTX power, or x2 is really just the speedup that I can expect? That would suck ^^</p>
<p>Thanks, best, JZ</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="jayz" data-post="1" data-topic="152811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jayz/48/46742_2.png" class="avatar"> jayz:</div>
<blockquote>
<p>Or do I need to downgrade to cuda 11.3 first?</p>
</blockquote>
</aside>
<p>No, you don’t need to downgrade your local CUDA toolkit as it would only be used if you are building PyTorch from source or custom CUDA extensions.<br>
The binaries ship with their own CUDA runtime and will work.</p>
<aside class="quote no-group" data-username="jayz" data-post="3" data-topic="152811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jayz/48/46742_2.png" class="avatar"> jayz:</div>
<blockquote>
<p>However, i end up getting crashes in the new environment that are related to some matplotlib issue with the package freetype. I have not managed to solve it. Could you give me a hint how to set up a full proper environment with 11.6?</p>
</blockquote>
</aside>
<p>Create a new virtual environment and reinstall all packages there.</p>
<aside class="quote no-group" data-username="jayz" data-post="3" data-topic="152811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jayz/48/46742_2.png" class="avatar"> jayz:</div>
<blockquote>
<p>To be more precise, the RTX delivers approximately a speedup x2</p>
</blockquote>
</aside>
<p>TF32 is disabled by default since ~3 weeks ago so your nightly binary would also not use it.<br>
You can re-enable it in case you are using the default <code>float32</code> data type.<br>
Also take a look at the <a href="https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html">performance guide</a> and familiarize yourself with profilers to narrow down bottlenecks as your training could also be blocked by e.g. the data loading while the GPU sits idle.</p>

username: jayz
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="152811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Create a new virtual environment and reinstall all packages there.</p>
</blockquote>
</aside>
<p>Yeah, that’s what I did:</p>
<pre><code class="lang-auto">conda create -n pyt intelpython3_full python=3 -c intel
pip install torch --pre --extra-index-url https://download.pytorch.org/whl/nightly/cu116
conda install nb_conda_kernels
</code></pre>
<p>Afterwards, I get a common error with matplotlib and freetype library, as quoted in my post above. I can’t get rid of this. So, basically, the environment works with pytorch, but I can’t plot anything. -.-</p>
<aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="152811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>TF32 is disabled by default since ~3 weeks ago so your nightly binary would also not use it.<br>
You can re-enable it in case you are using the default <code>float32</code> data type.</p>
</blockquote>
</aside>
<p>Not sure if that’s the case for my env:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.backends.cuda.matmul.allow_tf32, torch.backends.cudnn.allow_tf32
&gt;&gt; (True, True)
</code></pre>
<p>by default. Is that what you are referring to?</p>
<aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="152811">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Also take a look at the <a href="https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html" rel="noopener nofollow ugc">performance guide </a> and familiarize yourself with profilers to narrow down bottlenecks as your training could also be blocked by e.g. the data loading while the GPU sits idle.</p>
</blockquote>
</aside>
<p>Thanks, I found a lot of useful stuff in there already. Its probably better to implement these performance tips in the working Cuda 11.3 environment before spending time to build pytorch from source (sth. I have no experience with) with Cuda 11.7 etc.<br>
First and foremost, I will adapt my dataloader to use <code>num_workers = cpu_count()</code> which may probably be the most impactful measure to take for now, and then implement the various things from the guide. Will report back on the performance.</p>