Is thIs the problem with the model or something is wrong in my training process or the way I am giving input to the model?

Aashish_P: <p>I have used fcn8s model with vgg backbone for image segmentation task on the pascal VOC 2012 dataset but after training the model I am unable to get the desired results of segmentation. I have used this link</p><aside class="onebox githubblob" data-onebox-src="https://github.com/pochih/FCN-pytorch/blob/master/python/fcn.py">
  <header class="source">

      <a href="https://github.com/pochih/FCN-pytorch/blob/master/python/fcn.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pochih/FCN-pytorch/blob/master/python/fcn.py" target="_blank" rel="noopener nofollow ugc">pochih/FCN-pytorch/blob/master/python/fcn.py</a></h4>


      <pre><code class="lang-py"># -*- coding: utf-8 -*-

from __future__ import print_function

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import models
from torchvision.models.vgg import VGG


class FCN32s(nn.Module):

    def __init__(self, pretrained_net, n_class):
        super().__init__()
        self.n_class = n_class
        self.pretrained_net = pretrained_net
        self.relu    = nn.ReLU(inplace=True)
        self.deconv1 = nn.ConvTranspose2d(512, 512, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1)
        self.bn1     = nn.BatchNorm2d(512)
</code></pre>



  This file has been truncated. <a href="https://github.com/pochih/FCN-pytorch/blob/master/python/fcn.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
for making the fcn model with the dataset that is available on torchvision.datasets<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/9/19e1d7eaa52e2e5107f3c62aaf79ca3b28cc7a8c.png" data-download-href="https://discuss.pytorch.org/uploads/default/19e1d7eaa52e2e5107f3c62aaf79ca3b28cc7a8c" title="Screenshot_2020-04-19_12:11:37"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19e1d7eaa52e2e5107f3c62aaf79ca3b28cc7a8c_2_690x387.png" alt="Screenshot_2020-04-19_12:11:37" data-base62-sha1="3GXNQPpUjY7oQBapTLvOd7p0lKA" width="690" height="387" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19e1d7eaa52e2e5107f3c62aaf79ca3b28cc7a8c_2_690x387.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/19e1d7eaa52e2e5107f3c62aaf79ca3b28cc7a8c_2_1035x580.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/9/19e1d7eaa52e2e5107f3c62aaf79ca3b28cc7a8c.png 2x" data-dominant-color="2D2E2D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-04-19_12:11:37</span><span class="informations">1366×768 229 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
These are some of the screenshots of my project. I couldn’t figure out where the problem is since the model has minimal loss but the segmentation is not working as I have wished.</p>

ptrblck: <p>What is not working at the moment, i.e. are the outputs not as expected?<br>
Did the model work fine during training and validation (were the outputs reasonable)?</p>

Aashish_P: <p>With the use of unet and fcn vgg model, i am unable to get desired accuracy on pascal voc 2012 segmentation dataset, with 10 epoch i am only getting acc. starting from 27% to 45%. I have used the dataset available on PyTorch datasets as</p>
<pre><code class="lang-auto">train_set = torchvision.datasets.VOCSegmentation(root='drive/My Drive/VOC',
                                     year='2012',
                                     image_set='train',
                                     download=True,
                                     transform=T.Compose([
                                                          T.Resize(144),
                                                          T.CenterCrop(128),
                                                          T.ToTensor()
                                     ]),
                                     target_transform=T.Compose([
                                                           T.Resize(144),
                                                           T.CenterCrop(128),
                                                           T.ToTensor()
                                     ]),
                                     transforms=None)
</code></pre>
<p>and batch of 64 as:</p>
<pre><code class="lang-auto">train_loader = data.DataLoader(train_set, batch_size=64, shuffle=True)
</code></pre>
<p>and my train.py is:</p>
<pre><code class="lang-auto">train_Accuracy = []
train_Loss = []
dictionary = {}
for epoch in range(8):
  total_loss = 0
  total_correct = 0
  total_train = 0
  correct_train = 0
  ts = time.time()

  for batch in train_loader:
    images, labels = batch

    optimizer.zero_grad()

    preds = model(images)
    loss = criterion(preds, labels)

    loss.backward()
    optimizer.step()

    total_loss += loss.item()
    # total_correct += get_num_correct(preds, labels)

    _, predicted = torch.max(preds.data, 1)
    total_train += labels.nelement()
    correct_train += predicted.eq(labels.data).sum().item()
    train_accuracy = 100 * correct_train / total_train

  train_Loss.append(round(loss.item(), 5))
  train_Accuracy.append(round(train_accuracy, 4))

  dictionary = {
      "epoch": epoch + 1,
      "loss": round(loss.item(), 5),
      "accuracy": round(train_accuracy, 4),
      "training time": time.time() - ts,
      "model": model.state_dict()
  }
  torch.save(dictionary, 'data_to_be_saved_{}.pth'.format(epoch+1))
  print('epoch: ', epoch + 1,
        # ' total_correct: ', total_correct,
        # ' loss: ', total_loss/len(data_loader),
        ' loss: ', round(loss.item(), 5),
        ' accuracy: ', round(train_accuracy, 4),
        ' training time: ', time.time() - ts)
</code></pre>

ptrblck: <p>I would recommend to try to overfit a small data sample and make sure your model can successfully predict these samples.<br>
If that’s not the case, there might be a bug in the code I’m missing or the architecture is not suitable for the task.</p>

Aashish_P: <p>Thanks, Sir, the problem was on my dataset and I fixed it, with this the accuracy is quite good but using that model and segmenting a single image is just not working. <code>trans_img_tensor = T.Compose([                               T.Resize(256),                               T.CenterCrop(224),                               T.ToTensor() ]) image = Image.open('rose.jpeg') a = trans_img_tensor(image) c = a.unsqueeze(0) label = model(c) trans_tensor_2_img = T.Compose([                                 T.ToPILImage() ]) d = trans_tensor_2_img(label.argmax(1).float()) d</code></p>

ptrblck: <p>Your code is a bit hard to read, but did you call <code>model.eval()</code> before trying to test the model?</p>

Aashish_P: <p>Sir I am trying to load a saved model and to use it on the image. I have used</p>
<pre><code class="lang-auto">state_dict()
</code></pre>
<p>to save the model and using</p>
<pre><code class="lang-auto">model = TheModelClass(*args, **kwargs)
model.load_state_dict(dictionary['model'])
</code></pre>
<p>to load the model but the *args and **kwargs are not defined, what to use on them, I have read your previous answer on this but didn’t quite able to find what to use on them.<br>
The model used above is a CNN and I  am trying to load an unet.</p>
<pre><code class="lang-auto">class TheModelClass(nn.Module):
    def __init__(self):
        super(TheModelClass, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

# Initialize model
model = TheModelClass()
</code></pre>

ptrblck: <p>The <code>args</code> and <code>kwargs</code> are just placeholders in case your model’s <code>__init__</code> takes some arguments.<br>
Since yours only uses <code>self</code>, you don’t need to pass anything to the initialization.</p>

Aashish_P: <p>I was successful in loading the model with your guidance but using this</p>
<pre><code class="lang-auto">def image_segmentation(path):
  image = Image.open(path)
  trans_image = trans(image)
  model.eval()
  dim_added_image = trans_image.unsqueeze(0)
  segment = model(dim_added_image)
  segmented_image_tensor = segment.argmax(1).float()
  segmented_image = trans_2_img(segmented_image_tensor)
  return segmented_image
</code></pre>
<p>as well I am not able to segment the image that I feed to it.</p>