How to perform PyTorch training using GPU shared memory

ryuuji06: <p>I’m using PyTorch to train a model for image segmentation and I need to use GPU shared memory<br>
(simply because GPU VRAM is not enough for training the model in the laptops I have available).</p>
<p>I have two laptops available and I’m training the model within Conda environment in Windows, so I expect all<br>
(or most of) settings to be identical in both computers. In one laptop, training performs successfully, as it starts using GPU shared memory once GPU dedicated memory saturates (figure below). However, in the other laptop, when GPU<br>
dedicated memory saturates, it just gives runtime error rather than using shared memory (the other figure). The error message is as follows:</p>
<pre><code class="lang-auto">RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 8.00 GiB total capacity; 7.22 GiB already allocated; 0 bytes free; 7.25 GiB reserved in total by PyTorch)
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/2/42f415a19261f5eac584a70a0bcfae9d741822be.png" data-download-href="https://discuss.pytorch.org/uploads/default/42f415a19261f5eac584a70a0bcfae9d741822be" title="using-shared"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/2/42f415a19261f5eac584a70a0bcfae9d741822be_2_669x500.png" alt="using-shared" data-base62-sha1="9yisCFkL1EkrRSEedeilp3touV8" width="669" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/2/42f415a19261f5eac584a70a0bcfae9d741822be_2_669x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/2/42f415a19261f5eac584a70a0bcfae9d741822be_2_1003x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/2/42f415a19261f5eac584a70a0bcfae9d741822be.png 2x" data-dominant-color="F6F8F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">using-shared</span><span class="informations">1234×921 43.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc91409bbd456593714e768e6726c5272e9c7cd5.png" data-download-href="https://discuss.pytorch.org/uploads/default/cc91409bbd456593714e768e6726c5272e9c7cd5" title="not_using_shared"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc91409bbd456593714e768e6726c5272e9c7cd5_2_690x492.png" alt="not_using_shared" data-base62-sha1="tbGByvQEJ92SGAS1PUBGr47Ml1j" width="690" height="492" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc91409bbd456593714e768e6726c5272e9c7cd5_2_690x492.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc91409bbd456593714e768e6726c5272e9c7cd5_2_1035x738.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cc91409bbd456593714e768e6726c5272e9c7cd5_2_1380x984.png 2x" data-dominant-color="FAFAFA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">not_using_shared</span><span class="informations">1540×1099 60.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Details:</p>
<ul>
<li>laptop that succeeds: Windows 10, NVIDIA T1200 Laptop GPU</li>
<li>laptop that fails: Windows 11, NVIDIA GeForce RTX 4060 Laptop GPU</li>
</ul>
<p>In both cases, I’m running the exact same code, and Python, CUDA, CuDNN and Torch are installed with the same versions, as it is installed via Conda:</p>
<pre><code class="lang-auto">conda create -n env_name python=3.9.12

pip3 install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
</code></pre>
<p>Does someone has any clue of what could be blocking GPU of using shared memory? Could it be some PyTorch settings? Or could this be due to GPU settings or something?</p>

ZedRover: <p>You should upgrade your nvidia driver. Only if nvidia-driver version &gt;= 536.40 will your pytorch can use shared memory.</p>
<p><a href="https://nvidia.custhelp.com/app/answers/detail/a_id/5490/~/system-memory-fallback-for-stable-diffusion#:~:text=In%20driver%20536.40,%20we%20implemented%20a%20new%20method%20to%20allow%20an%20application%20to%20use%20shared%20memory%20in%20cases%20that%20exhausted%20the%20GPU%20memory" rel="noopener nofollow ugc">nvidia blog</a></p>

ryuuji06: <p>It solved my problem. Thank you for your answer.</p>