Is Pytorch not using GPU for training?

todeschini.felipe: <p>Hi there, I am working on a project called <a href="https://github.com/todeschini-felipe/udacity/blob/master/deep-learning-v2-pytorch-master/project-dog-classification/dog_app.ipynb" rel="noopener nofollow ugc">dog_app.py</a>, within <code>conda</code> environment and a Windows 10 machine.</p>
<p>Although I have (apparently) configured everything to use GPU, its usage barely goes above 2%. I am moving the model to <code>cuda()</code>, as well as my data. Why GPU is not being used at all? How do I debug that?</p>
<pre><code class="lang-auto">use_cuda = torch.cuda.is_available()
model_scratch = Net()
if use_cuda:
    model_scratch.cuda()
    print("Let's use", torch.cuda.device_count(), "GPU(s)!")

# Prints "Let's use 1 GPU(s)!"

...
def train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):
...
        model.train()
        for batch_idx, (data, target) in enumerate(loaders['train']):
            if use_cuda:
                data, target = data.cuda(), target.cuda()
</code></pre>
<p>I found this test on another thread on the subject, but allocating memory on the GPU worked just fine:</p>
<pre><code class="lang-auto">import torch
a = torch.cuda.FloatTensor(10000)
print("Allocated:", round(torch.cuda.memory_allocated(0)/10243,1), "GB")

b = torch.cuda.FloatTensor(20000)
print("Allocated:", round(torch.cuda.memory_allocated(0)/10243,1), "GB")

#Output:
# Allocated: 3.9 GB
# Allocated: 11.8 GB
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/e/fe0d6e42289c6828729ac3c2a38548769a3b2cc0.png" data-download-href="https://discuss.pytorch.org/uploads/default/fe0d6e42289c6828729ac3c2a38548769a3b2cc0" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/e/fe0d6e42289c6828729ac3c2a38548769a3b2cc0.png" alt="image" data-base62-sha1="Afs3gj7KbeVA62xcTZ9emxveqD6" width="690" height="247" data-dominant-color="F8FAFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1162×417 19.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

albanD: <p>Hi,</p>
<p>If you’re using windows, you need to be careful as CUDA computations are not reported in the task manager. You will have a to check with nvidia-smi from a command line I think.</p>

todeschini.felipe: <p>Hi <a class="mention" href="/u/alband">@albanD</a>, thanks for your reply. It took me a while to figure out how to use the tool, but it seems I have only short bursts of usage. Is that how it is supposed to work?</p>
<pre><code class="lang-auto">C:\Program Files\NVIDIA Corporation\NVSMI&gt;nvidia-smi --format=csv --query-gpu=utilization.gpu,fan.speed,temperature.gpu,power.draw -l 1
utilization.gpu [%], fan.speed [%], temperature.gpu, power.draw [W]
6 %, 0 %, 58, 50.59 W
1 %, 0 %, 59, 135.16 W
1 %, 0 %, 58, 50.50 W
51 %, 0 %, 58, 50.59 W
0 %, 0 %, 58, 144.52 W
0 %, 0 %, 58, 50.15 W
0 %, 0 %, 58, 50.25 W
59 %, 0 %, 59, 50.83 W
0 %, 0 %, 58, 136.92 W
0 %, 0 %, 58, 50.39 W
62 %, 0 %, 59, 50.83 W
0 %, 0 %, 59, 50.39 W
0 %, 0 %, 59, 50.59 W
0 %, 0 %, 61, 62.24 W
0 %, 0 %, 59, 50.49 W
0 %, 0 %, 59, 50.59 W
0 %, 0 %, 60, 50.83 W
0 %, 0 %, 59, 50.49 W
0 %, 0 %, 59, 50.39 W
0 %, 0 %, 60, 50.74 W
0 %, 0 %, 60, 50.83 W
36 %, 0 %, 61, 51.42 W
0 %, 0 %, 60, 50.74 W
0 %, 0 %, 60, 50.74 W 
</code></pre>

albanD: <p>It will depend a lot on your network. But if it is not too big or your dataloader is not fast enough then yes that is expected.<br>
You can try adding workers to the dataloader to make sure this is not the bottleneck. Otherwise increasing the batch size (if you have enough memory) should increase the usage.</p>