Windows uses twice as much VRAM than Linux for same code

username: TotzkePaul
cooked: <p>I am running Tacotron2 on Windows with Conda environment with an RTX 3090.</p>
<p>I can get to run but in Windows, it seems like it uses twice as much VRAM as compared to my Linux counter parts.</p>
<p>I’ve tried using python 3.6 and 3.7. installing pytorch through pip and conda. using cudatoolkit.<br>
CudaToolkits pre-v11 tell me its not sm_86 compatible and freeze.</p>
<p>I didn’t always have this issue but I’ve gone back to my conda env backups, and those don’t help.</p>
<p>Is there any scripts that can tell me which cuda version that its using? I removed all the cuda paths but somehow my conda env still knew where to find it and run.</p>
<p>My colab graphics card is 16GB and can do a batch of 48. While my windows graphics card is 24GB and can only do 32 batch size.</p>
<p>Colab Link:</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20210222-060114-RC01_358863158" class="site-icon" width="128" height="128">
      <a href="https://colab.research.google.com/drive/1vnU0YpsvcU3y5lUs9Qf-4VlumdnR9wh4?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1vnU0YpsvcU3y5lUs9Qf-4VlumdnR9wh4?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: TotzkePaul
cooked: <p>I installed cudnn 8.1.0 but when I run this code and I get: (‘11.0’, 8004)</p>
<pre><code class="lang-auto">def get_cuda_version():
    import torch
    cuda = (torch.version.cuda)
    cudnn = torch.backends.cudnn.version()
    return cuda, cudnn
    
print(get_cuda_version())
</code></pre>

username: ptrblck
cooked: <p>The conda binaries and pip wheels will use their own CUDA runtime (which you specify during the installation) as well as cudnn, NCCL etc., so your local CUDA toolkit won’t be used.<br>
The local CUDA toolkit (and in particular the compiler) will be used if you are building a custom CUDA extension or PyTorch from source.</p>
<p>Are both machines using the same PyTorch, CUDA, cudnn installation (e.g. through conda) as well as the same GPU?<br>
If so, are both GPUs free before you start the training or is e.g. Windows using it to display the desktop (which will use GPU memory)?</p>

username: TotzkePaul
cooked: <p>Thanks ptrblck,</p>
<p>When I run get_cuda_version() in Ubuntu 18.04, I get (‘11.0’, 8005) so slightly different cudnn.</p>
<p>I did the same install steps using conda in Ubuntu 18.04 as I do in Windows.</p>
<p>I’m getting the same results as Windows for same conda and pytorch as I do in Ubuntu for same GPU.</p>
<p>Colab gives me a P100 or V100 and its using pytorch+cu101. Is Cuda 10 less memory intensive?</p>
<p>I am using the display for my desktop but its less than 500MB.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="TotzkePaul" data-post="4" data-topic="112764">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/totzkepaul/48/35009_2.png" class="avatar"> TotzkePaul:</div>
<blockquote>
<p>When I run get_cuda_version() in Ubuntu 18.04, I get (‘11.0’, 8005) so slightly different cudnn.</p>
<p>I did the same install steps using conda in Ubuntu 18.04 as I do in Windows.</p>
<p>I’m getting the same results as Windows for same conda and pytorch as I do in Ubuntu for same GPU.</p>
</blockquote>
</aside>
<p>That’s good to hear. So the different memory behavior was observed between your local Windows machine and Colab?</p>
<aside class="quote no-group" data-username="TotzkePaul" data-post="4" data-topic="112764">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/totzkepaul/48/35009_2.png" class="avatar"> TotzkePaul:</div>
<blockquote>
<p>Is Cuda 10 less memory intensive?</p>
</blockquote>
</aside>
<p>It might be the case, but would also depend on the used device.<br>
Could you create a single <code>CUDATensor</code> and check the memory used by the CUDA context via <code>nvidia-smi</code>? Make sure that the GPU memory is empty before running this test or subtract the already used memory.</p>

username: TotzkePaul
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="5" data-topic="112764">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>That’s good to hear. So the different memory behavior was observed between your local Windows machine and Colab?</p>
</blockquote>
</aside>
<p>Yes</p>
<p>In google colab, I got a v100 today. I switched the pytorch version to be cu110 from cu101.</p>
<p>However, I didn’t use conda. I was able to do a batch size of 48 without crashing using 16GB - as I have always been able to do using colab.</p>
<pre><code class="lang-auto">pip install torch==1.7.1+cu110 -f https://download.pytorch.org/whl/torch_stable.html
</code></pre>
<p>get_cuda_version() in colab gave me (‘11.0’, 8005)</p>
<p>I’ll  get the tensors tonight.</p>

username: TotzkePaul
cooked: <p>Hope this is right. Both were close.</p>
<pre><code class="lang-auto">import torch

a = torch.full((100000,10000,), 3, device=torch.device("cuda"))
</code></pre>
<p>Windows10 = 8422MiB<br>
Colab = 8851MiB</p>