Can't use multi CPUs when I use GPU

Jianying_Hao: <p>I use pytorch to train but I can’t use full of my computer performance.<br>
My computer configuration is:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/8/889640f4412e6c2c9cc5ab08bf8ebd9bc24623d4.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/889640f4412e6c2c9cc5ab08bf8ebd9bc24623d4" title="webwxgetmsgimg"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/889640f4412e6c2c9cc5ab08bf8ebd9bc24623d4_2_690x254.jpeg" alt="webwxgetmsgimg" data-base62-sha1="juiQTlbgCKE1JfyY2ETkcDi9F52" width="690" height="254" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/8/889640f4412e6c2c9cc5ab08bf8ebd9bc24623d4_2_690x254.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/8/8/889640f4412e6c2c9cc5ab08bf8ebd9bc24623d4.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/8/889640f4412e6c2c9cc5ab08bf8ebd9bc24623d4.jpeg 2x" data-dominant-color="D8D8D8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">webwxgetmsgimg</span><span class="informations">850×314 120 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I try to use pytorch to test a neural network. My input and output image size is 688<em>688</em>1.<br>
When I use CPU to train, it seems it can use multi CPUs.  The batch size can be set up to 128.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/8/b899f6fb902403143c61df6473c3b2ccbe5094bc.png" data-download-href="https://discuss.pytorch.org/uploads/default/b899f6fb902403143c61df6473c3b2ccbe5094bc" title="GPU using"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b899f6fb902403143c61df6473c3b2ccbe5094bc_2_600x500.png" alt="GPU using" data-base62-sha1="ql3INzBNOTAhqqbFHjcpd8hqXAo" width="600" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b899f6fb902403143c61df6473c3b2ccbe5094bc_2_600x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/8/b899f6fb902403143c61df6473c3b2ccbe5094bc_2_900x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/8/b899f6fb902403143c61df6473c3b2ccbe5094bc.png 2x" data-dominant-color="F7F6F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">GPU using</span><span class="informations">1085×904 31.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
But when I use GPU to train. The batch size can only be set to 4. If I use larger batch size, there will be error ‘Out of membery.’ I seems only one cpu is used.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/c/8c9ca13612be1f37aa9556b96ef04fbd97420f21.png" data-download-href="https://discuss.pytorch.org/uploads/default/8c9ca13612be1f37aa9556b96ef04fbd97420f21" title="cpu using"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/c/8c9ca13612be1f37aa9556b96ef04fbd97420f21_2_690x185.png" alt="cpu using" data-base62-sha1="k3UqfZo1wtSmf2gtRBe9G6nNQFX" width="690" height="185" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/c/8c9ca13612be1f37aa9556b96ef04fbd97420f21_2_690x185.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/c/8c9ca13612be1f37aa9556b96ef04fbd97420f21_2_1035x277.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/c/8c9ca13612be1f37aa9556b96ef04fbd97420f21_2_1380x370.png 2x" data-dominant-color="9E9D9E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">cpu using</span><span class="informations">2121×571 39.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I use this code to use CUDA：<br>
device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)<br>
inputs = x.to(device)<br>
labels = y.to(device)<br>
model = Unet(1, 1).to(device)</p>
<p>What should I do to use the full CPUs when I use CUDA？ Do I need to do some extra setup?</p>

shlomia: <aside class="quote no-group" data-username="Jianying_Hao" data-post="1" data-topic="86385">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/e56c9b/48.png" class="avatar"> Jianying_Hao:</div>
<blockquote>
<p>But when I use GPU to train. The batch size can only be set to 4. If I use larger batch size, there will be error ‘Out of membery.’ I seems only one cpu is used</p>
</blockquote>
</aside>
<p>The GPU crashes when its RAM is full, unlike CPU which its memory management is made by the operating system (for example). there is not much to do with your batch_size. What you can do is to increase your number of workers, that would collaborate more CPUs.</p>

Jianying_Hao: <aside class="quote no-group" data-username="shlomia" data-post="2" data-topic="86385">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shlomia/48/25585_2.png" class="avatar"> shlomia:</div>
<blockquote>
<p>What you can do is to increase your number of workers, that would collaborate more CPUs.</p>
</blockquote>
</aside>
<p>Thank you for your replying. It seems the GPU is not doing its best. I still cannot understand what should I do. Shoud I improve my CPU RAM?</p>

shlomia: <p>have a look here<br>
<a href="https://protect2.fireeye.com/url?k=fa04f03a-a763b41c-fa057b75-0cc47a31384a-df32e37ba0b91618&amp;q=1&amp;u=https%3A%2F%2Ftimdettmers.com%2F2018%2F12%2F16%2Fdeep-learning-hardware-guide%2F" rel="nofollow noopener">https://timdettmers.com/2018/12/16/deep-learning-hardware-guide/</a></p>