Testing data taking over 5hr and no output but Training and Validation was super fast

username: pavs
cooked: <p>Hello,<br>
I would like to know if anyone knows the reason why my training and validation occurs quickly but then my testing (where I print out the f1 score and classification report ) takes over 5 hours and then times out on google colab.</p>
<p>Any guidance would be useful</p>

username: cyRi-Le
cooked: <p>You should check to see if you’re using a GPU in your test code</p>

username: pavs
cooked: <p>The colab book runs on gpu but I am unsure on how to check that the testing dataset is running on colab</p>
<pre><code class="lang-auto">predlist=torch.zeros(0,dtype=torch.long, device='cpu')

    lbllist=torch.zeros(0,dtype=torch.long, device='cpu')

    with torch.no_grad():

        model.eval()

        i=0

        for test_batch_data,test_batch_labels in allTheDataloaders["Testing"]:


            test_batch_data, test_batch_labels = test_batch_data.to(device), test_batch_labels.to(device)

            y_test_pred=model(test_batch_data)

            y_scores, y_pred_targets = torch.max(y_test_pred, dim = 1)



  

            predlist=torch.cat([predlist,y_pred_targets.view(-1).cpu()])

            lbllist=torch.cat([lbllist,test_batch_labels.view(-1).cpu()])

 
  

    #https://www.kaggle.com/agungor2/various-confusion-matrix-plots

    print(classification_report(lbllist.numpy(), predlist.numpy()))

    data=confusion_matrix(lbllist.numpy(), predlist.numpy())

    data_cm=pd.DataFrame(data, columns=np.unique(lbllist.numpy()), index = np.unique(lbllist.numpy()))

    data_cm.index.name = 'Actual'

    data_cm.columns.name = 'Predicted'

    plt.figure(figsize = (10,7))

    sn.set(font_scale=1.4)#for label size

    sn.heatmap(data_cm, cmap="Blues", annot=True,annot_kws={"size": 16},fmt=".1f")
</code></pre>
<p>it only takes long for a specific dataset.</p>

username: omarfoq
cooked: <p>Hello,</p>
<p>Can you <code>print(device) </code>?</p>

username: pavs
cooked: <p><a class="mention" href="/u/omarfoq">@omarfoq</a>  it prints<br>
cuda:0</p>

username: omarfoq
cooked: <p>Well in that case you are running on GPU, can you check that the batch size for test is not very small, also the operation you do to stack predictions is not “optimal”, because you keep copying small chunks of information from GPU to CPU, probably it’s better to keep all prediction on GPU, and at the end send everything to CPU.</p>
<aside class="quote no-group" data-username="pavs" data-post="3" data-topic="117296">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/85f322/48.png" class="avatar"> pavs:</div>
<blockquote>
<pre><code class="lang-auto">predlist=torch.cat([predlist,y_pred_targets.view(-1).cpu()])

            lbllist=torch.cat([lbllist,test_batch_labels.view(-1).cpu()])
</code></pre>
</blockquote>
</aside>

username: pavs
cooked: <p><a class="mention" href="/u/omarfoq">@omarfoq</a>  do you have any examples that I can refer to. I am new to pytorch and I have been using tutorials to implement.</p>

username: cyRi-Le
cooked: <p>make sure you call <code>model.cuda()</code> or <code>model.to(device)</code>so the model will run on GPU.<br>
Instead of writing <code>predlist=torch.cat([predlist,y_pred_targets.view(-1).cpu()])</code> you should keep all your tensors on GPU so just keep <code>predlist=torch.cat([predlist,y_pred_targets.view(-1)])</code> and <code>predlist=torch.zeros(0,dtype=torch.long, device=device)</code>  where <code>device</code> refers to your GPU. Then at the end you just make <code>predlist=predlist.cpu()</code> if your code require numpy arrays.</p>

username: pavs
cooked: <p><a class="mention" href="/u/cyri-le">@cyRi-Le</a>  Hi,<br>
Thank you.<br>
I am trying this now and hopefully it doesn’t  take too long.<br>
It is perfectly fine for my other data loaders, it’s just one specific one that it seems to have an issue with.</p>