High GPU temperature issue with Pytorch

username: ahmediqbal
cooked: <p>Hello,</p>
<p>I have Pytorch code written for semantic segmentation, technically code is working fine without any errors. But I am facing High GPU temperature issue. Same code is written in Tensorflow, My GPU only reach 65 °C temperature, But in Pytorch case my GPU reach 85 °C temperature. Here is all details of my GPU and training configurations:</p>
<p><strong>Hardware:</strong></p>
<p>GPU: NVIDIA GeForce RTX 2060 Super - 8GB</p>
<p>RAM: 16 GB</p>
<p><strong>Software:</strong></p>
<p>Pytorch v1.6</p>
<p>CUDA 10.2</p>
<p><strong>Training Setting:</strong></p>
<p>Img_size = (256, 256)</p>
<p>batch_size = 4</p>
<p>num_epochs = 100</p>
<p>lr = 1e-4</p>
<p>num_workers=2</p>

username: omarfoq
cooked: <p>Hello,</p>
<p>The temperature of GPU is mainly related to GPU-Util. Can you check if both codes give you the same GPU-util and same memory usage on nvidia-smi ?</p>
<p>In case you see a difference in GPU-util, it maybe related to how you read and batch over the dataset, so you can first check the amount of time needed to read it batch in your tf code and torch code.</p>

username: ahmediqbal
cooked: <p>Please check following screenshot details, and let me know why PyTorch is creating high temperature on GPU and what is solution?</p>
<p><strong>PyTorch nvidia-smi screenshot:</strong><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/4/a41c447b00a56b93886f50a66d5642994047fa20.png" data-download-href="https://discuss.pytorch.org/uploads/default/a41c447b00a56b93886f50a66d5642994047fa20" title="Ptorch"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/4/a41c447b00a56b93886f50a66d5642994047fa20.png" alt="Ptorch" data-base62-sha1="npMROA0BPIBfV9MGKCymuMPnUSQ" width="526" height="500" data-dominant-color="1C1C1C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Ptorch</span><span class="informations">882×837 28.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><strong>Tensorflow nvidia-smi screenshot:</strong><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/1/c16449e6b7b86c634988dda88d193a4fef14f1e8.png" data-download-href="https://discuss.pytorch.org/uploads/default/c16449e6b7b86c634988dda88d193a4fef14f1e8" title="Tensorflow"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/1/c16449e6b7b86c634988dda88d193a4fef14f1e8.png" alt="Tensorflow" data-base62-sha1="rAP1caG3P6dKifIGSjUJrU8xqnS" width="513" height="499" data-dominant-color="1C1C1C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Tensorflow</span><span class="informations">897×873 29.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: omarfoq
cooked: <p>Hello,</p>
<p>It seems that torch is taking use of all the GPU and consuming less memory, which is good I guess. I think your code in tf is not well optimized, the main reason behind this is that you waste time transferring data from RAM to GPU in your tf implementation.</p>
<p>The heat level you have is not a problem I guess, normally you want to use fully your GPU so that you can ran faster. So the fact that GPU-UTIL is 99% is a good sign in fact.</p>

username: ahmediqbal
cooked: <p>Thanks for reply me back!</p>
<p>Please check, In GPU software, here, i set target temperature to 70 °C, now GPU temperature is limited under 70 °C. But power % is also reduced 100% to 79%. These settings are good for me? However, in factory settings, target temperature was 83 °C and power was 100%.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/c/fcb4963c7c0b079c55eb2a50c29b8693d443225a.png" data-download-href="https://discuss.pytorch.org/uploads/default/fcb4963c7c0b079c55eb2a50c29b8693d443225a" title="GPUpng"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/c/fcb4963c7c0b079c55eb2a50c29b8693d443225a_2_690x322.png" alt="GPUpng" data-base62-sha1="A3xe8SIKeogumfievebzxkmh254" width="690" height="322" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/c/fcb4963c7c0b079c55eb2a50c29b8693d443225a_2_690x322.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/c/fcb4963c7c0b079c55eb2a50c29b8693d443225a_2_1035x483.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/c/fcb4963c7c0b079c55eb2a50c29b8693d443225a.png 2x" data-dominant-color="182A32"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">GPUpng</span><span class="informations">1253×585 351 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>As <a class="mention" href="/u/omarfoq">@omarfoq</a> explained, you usually want to fully utilize the GPU to get the best performance.<br>
If your system hits a thermal issue, you might either want to fix it (better cooling) or artificially slow down your code by e.g. limiting the power usage (which would also slow down your code and is not the optimal solution).</p>

username: ahmediqbal
cooked: <p>Thanks for reply me back.</p>
<p>Currently, i have following configuration:</p>
<p>Train Images: 1800 - Valid: 450<br>
batch_size = 12<br>
num_workers = 0</p>
<p>And my average GPU temperature remain 80 °C, it is normal? or here, i can also try something else?</p>

username: omarfoq
cooked: <aside class="quote no-group" data-username="ahmediqbal" data-post="7" data-topic="116345">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ahmediqbal/48/26792_2.png" class="avatar"> ahmediqbal:</div>
<blockquote>
<p>batch_size = 12</p>
</blockquote>
</aside>
<p>Hello <a class="mention" href="/u/ahmediqbal">@ahmediqbal</a>, I think that 80 °C is not a big problem I guess. Since the RTX 2060 Super has a maximum thermal temperature of 89 Celcius. Having 80 °C won’t harm the card and won’t harm the performance.</p>

username: ahmediqbal
cooked: <p><a class="mention" href="/u/omarfoq">@omarfoq</a> <a class="mention" href="/u/ptrblck">@ptrblck</a> i fixed GPU high temperature issue by increasing Fan speed Auto to Manual <strong>75%</strong>. So, it cool down GPU, and my GPU temperature decrease 80 °C to 60 °C.<br>
One more thing, what is role of <strong>num_workers</strong> ? and what is formula to select required num_workers?</p>

username: omarfoq
cooked: <p>Hi,</p>
<p>Ah that’s nice, I wasn’t aware that you can manually choose fan speed. For number of workers, if you are talking the number of workers used when creating dataloader, it just refers to how many subprocesses to use for data loading. (See <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" rel="noopener nofollow ugc">here</a>) There is no formula to chose it to my knowledge. Usually the best choice is related to your cpu (how many cores and how many thread by core).  A typical choice is 8 because many personal computers have 8 CPU cores and one worker thread per core. But if you have different CPU architecture, other values may be better. Note also that number of workers is not limited, you can choose any value you want,  in that case the subprocesses well be waiting in a Pool.</p>