Algorithm doesn't run on GPU even after storing model and data into GPU. What am i missing?

Enes_Uguroglu: <p>You can find training section of my code below:</p>
<blockquote>
<p>device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)</p>
</blockquote>
<p>Note: I am receiving True when i check torch.cuda.is_available</p>
<p>After creating CNN model i wrote:</p>
<blockquote>
<p>model = model.to(device)</p>
</blockquote>
<p>Training Section:</p>
<blockquote>
<p>import time<br>
start_time = time.time()</p>
<p>epochs = 3</p>
<p><span class="hashtag-raw">#Limits</span> on numbers of batches if you want train faster(Not mandatory)<br>
max_trn_batch = 800 # batch 10 image → 8000 images total<br>
max_tst_batch = 300 # batch 10 image → 3000 images total</p>
<p>train_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
test_losses = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
train_correct = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
test_correct = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<p>for i in range(epochs):</p>
<pre><code>trn_corr = 0
tst_corr = 0

for b,(X_train,y_train) in enumerate(train_loader):
    X_train,y_train = X_train.to(device),y_train.to(device)
    
    #optinal limit number of batches
    if b == max_trn_batch:
        break
    b = b + 1
    
    y_pred = model(X_train)
    loss = criterion(y_pred,y_train)
    
    predicted = torch.max(y_pred.data,1)[1]
    batch_corr = (predicted == y_train).sum()
    trn_corr = trn_corr + batch_corr
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if b%200 == 0:
        print('Epoch:  {} Loss:  {} Accuracy:  {}'.format(i,loss,trn_corr.item()*100/(10*b)))

train_losses.append(loss)
train_correct.append(trn_corr)


#test set

with torch.no_grad():
    for b,(X_test,y_test) in enumerate(test_loader):
        X_test,y_test = X_test.to(device),y_test.to(device)
        
        #Optional
        if b==max_tst_batch:
            break
        y_val = model(X_test)
        predicted = torch.max(y_val.data,1)[1]
        batch_corr = (predicted == y_test).sum()
        tst_corr = tst_corr + batch_corr

loss = criterion(y_val,y_test)
test_losses.append(loss)
test_correct.append(tst_corr)
</code></pre>
<p>total_time = time.time() - start_time<br>
print(f’Total Time: {total_time/60}) minutes’)</p>
</blockquote>
<p>And during the training i am checking the CPU and GPU performance , CPU working %100 while GPU %1.</p>
<p>Note 1:Algorithm took 13 minutes when i use CPU as device, and took 7 min when i used GPU as device, so there seems tiny improvement, but i couldnt see any gpu utilization on task manager during training.</p>
<p>Note 2: Paremeters</p>
<pre><code class="lang-auto">ConvolutionalNetwork(
  (conv1): Conv2d(3, 6, kernel_size=(3, 3), stride=(1, 1))
  (conv2): Conv2d(6, 16, kernel_size=(3, 3), stride=(1, 1))
  (fc1): Linear(in_features=46656, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=2, bias=True)
</code></pre>
<p>Thanks in advance</p>

Henry_Chibueze: <p>What are your pytorch and cuda versions?</p>

Flock1: <p>I think you’re not assigning your <code>cuda</code> device properly. Cuda device is usually assigned like this:<br>
<code>cuda:&lt;device number&gt;</code> (usually, it’s <code>0</code>). You can get the device number by running this command:<br>
<code>torch.cuda.current_device()</code></p>

Enes_Uguroglu: <p>Pytorch version is 1.6<br>
Cuda version is 10.2.89</p>

Enes_Uguroglu: <p>I assigned it as you mentioned and observed no difference unfortunately.</p>

Flock1: <p>Try this:<br>
</p><aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/1668#issuecomment-421300052" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/1668#issuecomment-421300052" target="_blank" rel="nofollow noopener">Easy way to switch between CPU and cuda </a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2017-05-27" data-time="18:19:43" data-timezone="UTC">06:19PM - 27 May 17 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2018-04-24" data-time="20:34:49" data-timezone="UTC">08:34PM - 24 Apr 18 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/chsasank" target="_blank" rel="nofollow noopener">
          <img alt="chsasank" src="https://avatars2.githubusercontent.com/u/9305875?v=4" class="onebox-avatar-inline" width="20" height="20">
          chsasank
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">Right now, as far as I know there is no one simple way to write code which runs seamlessly both on...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triage review</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


Enes_Uguroglu: <p>Still can not be sure if i am really using GPU, i searched from internet and found a suggestion to use nvidia-smi to check utilization during training. GPU utilization is 1 percent. This is my full code btw:</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/euguroglu/Machine-Learning-Projects/blob/master/Cuda_Pytorch_CNN_Chest_X_ray_classification.ipynb" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/euguroglu/Machine-Learning-Projects/blob/master/Cuda_Pytorch_CNN_Chest_X_ray_classification.ipynb" target="_blank" rel="nofollow noopener">euguroglu/Machine-Learning-Projects/blob/master/Cuda_Pytorch_CNN_Chest_X_ray_classification.ipynb</a></h4>
<pre><code class="lang-ipynb">{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cuda')"
      ]
     },
</code></pre>

  This file has been truncated. <a href="https://github.com/euguroglu/Machine-Learning-Projects/blob/master/Cuda_Pytorch_CNN_Chest_X_ray_classification.ipynb" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


ptrblck: <p>You should get an error, if you try to push tensors or the model to the GPU and no GPU is available.<br>
Also, once the data is transferred, you can check the device via <code>print(X_test.device)</code> and make sure it’s <code>cuda:id</code>.<br>
Your GPU utilization might be that low as your model is really small, such that you would see the overhead of e.g. the kernel launches, data loading and processing etc.</p>

Enes_Uguroglu: <p>Thank you very much for the answer, i increased batch size to understand if thats the case and observed that utilization is increased from %1 to %3 sometimes. So it was really about size of my data.</p>