How to check GPU usage when running pytorch

LongWU: <p>Dear expert,<br>
How can I check my GPU usage properly? I tried to use the taks manager,but it says GPU usage is 0%, like below:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/0/2035b56f9bd32503105674b4134fec25a8c68014.png" data-download-href="https://discuss.pytorch.org/uploads/default/2035b56f9bd32503105674b4134fec25a8c68014" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2035b56f9bd32503105674b4134fec25a8c68014_2_690x488.png" alt="image" data-base62-sha1="4AWlFquPlcWCF2Qq9f6HbM1Hho8" width="690" height="488" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2035b56f9bd32503105674b4134fec25a8c68014_2_690x488.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2035b56f9bd32503105674b4134fec25a8c68014_2_1035x732.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/0/2035b56f9bd32503105674b4134fec25a8c68014_2_1380x976.png 2x" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1526×1080 50.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
However, if I use the navidia-smi, it seems GPU usage is 94% (I am not sure what this 94% is):<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/4/646aace0c448ed7763195cf3df86069ecefd90c5.png" data-download-href="https://discuss.pytorch.org/uploads/default/646aace0c448ed7763195cf3df86069ecefd90c5" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/4/646aace0c448ed7763195cf3df86069ecefd90c5.png" alt="image" data-base62-sha1="ekkisRuhQbXJpCtPkp6STl1ReBf" width="690" height="353" data-dominant-color="191919"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1736×890 30.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I can see the GPU memery was consumed and I have all indications from pytorch that I am using GPU(such as torch.cuda.is_available(), and torch.cuda.get_device_name(0) ). But why GPU is alway 0% in task manager? Also I can’t see CUDA from dropdown in GPU tab:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba27495dc06dd1177664bbd5b6ceede51290c293.png" data-download-href="https://discuss.pytorch.org/uploads/default/ba27495dc06dd1177664bbd5b6ceede51290c293" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba27495dc06dd1177664bbd5b6ceede51290c293.png" alt="image" data-base62-sha1="qyMYNwZKSztjQbcsGmycux3O3Dl" width="690" height="465" data-dominant-color="F8F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1055×712 22.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I have CUDA version 12.0, and i have below installed:</p>
<blockquote>
<blockquote>
<blockquote>
<p>torch.<strong>version</strong><br>
‘2.0.1+cu118’<br>
torch.version.cuda<br>
‘11.8’<br>
I also did another check. I set the device=torch.device(‘cpu’) to make a comparison, and I found out CPU usage is rising (CPU usage is normal when I use GPU).<br>
Seems something is wrong (because it feels the training process using GPU is vvvvvery long), but I can’t be sure. Could you please advise?</p>
</blockquote>
</blockquote>
</blockquote>
<p>Thanks,<br>
Long</p>

vdw: <p>I’m not sure if it’s different in Windows. On Linux, <code>nvidia-smi</code> only shows you to current snapshot. If you want to continuously observe the usage while running something, try</p>
<pre><code class="lang-auto">watch -n 0.5 nvidia-smi
</code></pre>
<p>This updates the data every 0.5 seconds.</p>

LongWU: <p>Thanks for you answer. Does the 94% in my screenshot represent the GPU usage? Do you have any idea why the task manager in Windows shows differently?</p>

vdw: <p>Yes, 94% refers to the GPU utilization. Sorry, I on Linux (Ubuntu) and cannot help with Windows.</p>

LongWU: <p>Many thanks for your reply.</p>

lmsasu: <p>For both Linux and Windows you may use nvitop. Install it by pip install nvitop and it will show more details at a glance.</p>

LongWU: <p>Thanks Lucian, it works like a charm.</p>