IoU becomes a line

rttariverdi67: <p>Hi all, I have faced with a maybe funny but serious problem. my model is <a href="https://github.com/mateuszbuda/brain-segmentation-pytorch/blob/master/unet.py" rel="noopener nofollow ugc">U-Net</a>, and metric is IoU as below. during training the loss is decreasing but IoU stays in a fixed number and doesn’t move. Btw, I have really small dataset(30 image for train and 10 for val) just for debugging my code. but will increase the dataset size later. does dataset size can be reason? or what do I do wrong?!<br>
Thanks for your helps in advance</p>
<hr>
<pre><code class="lang-auto">def calc_iou(prediction, ground_truth):
    n_images = len(prediction)
    intersection, union = 0, 0
    for i in range(n_images):
        intersection += np.logical_and(prediction[i] &gt; 0, ground_truth[i] &gt; 0).astype(np.float32).sum() 
        union += np.logical_or(prediction[i] &gt; 0, ground_truth[i] &gt; 0).astype(np.float32).sum()
    return float(intersection) / union
</code></pre>
<hr>
<p>also I tested IoU for given random array and it works fine</p>
<hr>
<pre><code class="lang-auto">ground_truth = np.random.rand(1, 480,3760)
prediction = ground_truth
</code></pre>
<blockquote>
<p>calc_iou(prediction, ground_truth) = 1</p>
</blockquote>
<hr>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0af676522d8e25f28f0e9b8dd0345520b57847e6.png" data-download-href="https://discuss.pytorch.org/uploads/default/0af676522d8e25f28f0e9b8dd0345520b57847e6" title="Screenshot from 2021-02-02 23-29-50"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0af676522d8e25f28f0e9b8dd0345520b57847e6_2_690x320.png" alt="Screenshot from 2021-02-02 23-29-50" data-base62-sha1="1yYOGwyuO5T9As7PX0jLIu51vP8" width="690" height="320" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0af676522d8e25f28f0e9b8dd0345520b57847e6_2_690x320.png, https://discuss.pytorch.org/uploads/default/original/3X/0/a/0af676522d8e25f28f0e9b8dd0345520b57847e6.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/a/0af676522d8e25f28f0e9b8dd0345520b57847e6.png 2x" data-dominant-color="FCFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-02-02 23-29-50</span><span class="informations">724×336 21.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<hr>
<p>here is also my train function:</p>
<pre><code class="lang-auto">def train(model, optimizer, batch_size=32, num_epochs=100, silent=False, model_name='UNet'):

    train_loss_plot = []
    test_loss_plot = []
    train_accur_plot = []
    test_accur_plot = []

    for epoch in range(num_epochs):
        if (not silent):
            print("Epoch {:2}: ".format(epoch), end='')
        else:
            print('.', end='')

        model.train()
        total_train_loss = 0
        correct_examples_train = 0
        for batch_idx, data in enumerate(tqdm(train_dataloader)):
            x, y = data
            x, y = x.to(device), y.to(device)
            optimizer.zero_grad()
            pred = model(x)
            loss = criterion(pred.view(pred.shape[0], -1), y.view(y.shape[0], -1).float()) * batch_size
            loss.backward()
            optimizer.step()
#             schedular.step()

            total_train_loss += loss.item()
            correct_examples_train += calc_iou(pred.cpu().detach().numpy(),
                                               y.cpu().detach().numpy()) * batch_size
          

        model.eval()
        correct_examples_test = 0
        total_test_loss = 0
        for batch_idx, data in enumerate(test_dataloader):
            x_, y_ = data
            x_, y_ = x_.to(device), y_.to(device)
            pred_ = model(x_)
            loss_ = criterion(pred_.view(pred_.shape[0], -1), y_.view(y_.shape[0], -1).float()) * batch_size
            total_test_loss += loss_.item()

            correct_examples_test += calc_iou(pred_.cpu().detach().numpy(),
                                              y_.cpu().detach().numpy()) * batch_size

        train_loss_plot.append(total_train_loss / len(train_loader))
        test_loss_plot.append(total_test_loss  / len(val_loader))
        
        train_accur_plot.append(correct_examples_train / len(train_loader))
        test_accur_plot.append(correct_examples_test / len(val_loader))

        if (not silent):
            display.clear_output(wait=True)
            plt.figure(figsize=(20, 8))
            plt.subplot(1,2,1)
            plt.plot(range(len(train_loss_plot)), train_loss_plot)
            plt.plot(range(len(test_loss_plot)), test_loss_plot)
            plt.legend(['Train loss', 'Test loss'])
            plt.xlabel('Epoch')
            plt.ylabel('Loss')

            plt.subplot(1,2,2)
            plt.plot(range(len(train_accur_plot)), train_accur_plot)
            plt.plot(range(len(test_accur_plot)), test_accur_plot)
            plt.legend(['Train IoU', 'Test IoU'])
            plt.xlabel('Epoch')
            plt.ylabel('IoU')
            plt.show()

            print('Train loss {:6.3}, Test loss: {:6.3}'\
                  .format(total_train_loss / len(train_loader), 
                          total_test_loss  / len(val_loader)))
            print('Train IoU {:6.3}, Test IoU: {:6.3}'\
                  .format(correct_examples_train / len(train_loader), 
                          correct_examples_test  / len(val_loader)))

        torch.save(model.state_dict(), model_name + '_best.pth')
train(model, optimizer,  batch_size=8, num_epochs=30)
</code></pre>

Emile: <p>Hello, <a class="mention" href="/u/rttariverdi67">@rttariverdi67</a>  did you ever resolve this issue? I would be keen to see the corrected code. Or even maybe you tried the iou metrics from Lightning?</p>