Pytorch begginer struggling with data processing

username: Last_One
cooked: <p>Hi there!!<br>
I am new in Pytorch and although I am already able to work for example with FCN-GCN to segmentation problems, for example, I have some general questions regarding PyTorch. When I try to use a new network, I always struggle in the data processing phase, in particular when writing a Dataset object class to read my data, transform my data and convert it to Tensor type. I always have errors on the network, in particular with image or label shapes, etc… If it is not a pain, I really would kindly ask if someone could explain to me what it the best way of handling data for a general problem of classification or segmentation. For example, right now I am trying to develop a CNN for classification binary problem, and I am struggling with the labels ‘0’ and ‘1’, I always get shape and data type errors on my CNN. Not sure if could be also related with the criterion I am using: nn.CrossEntropyLoss().<br>
I was using this tutorial: <a href="https://www.pluralsight.com/guides/image-classification-with-pytorch" rel="nofollow noopener">https://www.pluralsight.com/guides/image-classification-with-pytorch</a>, but I am using my own Dataset.</p>
<p>Thank you in advance, and I understand if this is too confusing or difficult to answer <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

username: RaLo4
cooked: <p>This might help:<br>
If you have images with label 0 and label 1 you can put them in folders like /train/0 and /train/1 for your trainsplit and /test/0 /test/1 for your testsplit.<br>
Then you can define a dataloader like so:</p>
<pre><code class="lang-auto">import torchvision.transforms as transforms
import torchvision.datasets as dset

train_root = '/train'
train_set = dset.ImageFolder(root=train_root,
                                   transform=transforms.Compose([
                                       transforms.Resize(image_size),
                                       transforms.CenterCrop(image_size),
                                       transforms.ToTensor(),
                                       transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
                                       ]))
train_loader = torch.utils.data.DataLoader(train_set, batch_size=batch_size,
                                               shuffle=True, num_workers=workers)
</code></pre>
<p>Do the same for testset.</p>
<p>Since you have a binary classification you can use <code>torch.nn.Sigmoid()</code> together with <code>torch.nn.BCELoss()</code> or just <code>torch.nn.BCEWithLogitsLoss()</code> as loss function.</p>
<p><code>torch.nn.CrossEntropyLoss()</code> should also work tho.</p>

username: RaLo4
cooked: <aside class="quote no-group" data-username="Last_One" data-post="1" data-topic="90816">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/last_one/48/12953_2.png" class="avatar"> Last_One:</div>
<blockquote>
<p>I always have errors on the network, in particular with image or label shapes</p>
</blockquote>
</aside>
<p>If you have size mismatches during forward pass you might want to look at your featuremaps. Check whether or not the out_channels and in_channels of your conv layers match as well as the in_features and out_features of your linear layer. Just printing x.shape() at some points might help you find and debug mismatches</p>

username: Last_One
cooked: <p>Thank you for the response!! What you are saying is that I should put my labels in folders as data in specific folders, but why can’t I do like this?</p>
<pre><code class="lang-auto">
class Dataset(object):
 #----Initializing the class----
    def __init__(self, train_names, masks_dir, img_dir,train, im_size):
        self.train_names=train_names
        self.masks_dir=masks_dir
        self.img_dir=img_dir
        self.train=train 
        self.im_size=im_size

   
    def __len__(self):
        return len(self.train_names)
    
    def __getitem__(self, idx):

      image = Image.open(self.img_dir + self.train_names[idx])
      target = np.zeros((1,1))

      if self.train == True:

        try:
          masks = Image.open(self.masks_dir + os.path.splitext(self.train_names[idx])[0]+'beard.jpg') 
          target[0] = 1
            #----------------------------------------------------------------------- 
        except:
          target[0] = 0
      image=np.array(image)
      if image.shape[2] == 4:

        image = cv2.cvtColor(image, cv2.COLOR_BGR2BGR)
      image=resize(image, (self.im_size , self.im_size),anti_aliasing=True)
      image = transforms.ToTensor()(image)
      image=image.float() 

      target = np.array(target)
      target = transforms.ToTensor()(target)

      sample = {'image': image, 'target': target}
      
      return sample
</code></pre>

username: RaLo4
cooked: <p>My code was just an example of how you can use pytorchs <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" rel="nofollow noopener">dataloader</a> with your own images.<br>
You can of course define your own dataloader as well.</p>

username: Last_One
cooked: <p>yes, I know!! But what I sent is not working, and I would like to understand why!</p>
<p>I get this error: pic should be PIL Image or ndarray. Got &lt;class ‘str’&gt;</p>
<pre><code class="lang-auto">batch_size = 1
learning_rate = 0.001
threads=1
train_loader = torch.utils.data.DataLoader(train_dataset, num_workers=threads, batch_size=batch_size, shuffle=True)
val_loader = torch.utils.data.DataLoader(val_dataset, num_workers=threads, batch_size=batch_size, shuffle=False)
</code></pre>
<pre><code class="lang-auto">class CNN(nn.Module): 
    def __init__(self):
        super(CNN, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=3, out_channels=10, kernel_size=3)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=3)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(720, 1024)
        self.fc2 = nn.Linear(1024, 2)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(x.shape[0],-1)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return x
</code></pre>
<pre><code class="lang-auto">model = CNN()

cuda = torch.cuda.is_available()
print(cuda)
if cuda:
   model = model.cuda()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(),lr = learning_rate)
</code></pre>
<pre><code class="lang-auto">starting_epoch=0
epochs=50
best_dice=0
loss_accum = []

for epoch in range(starting_epoch, epochs):  
    print ('Running epoch ',epoch)
    train_loss=0
    val_loss=0
    val_dice=[]

    for phase in ['train','val']:

      if phase=='train':
        loader=train_loader
        model.train()
      else:  
        loader=val_loader
        model.eval()

      for batch in loader:
        if cuda:
          image = batch['image'].cuda()
          target = batch['target'].cuda()
 
        
        #print(image)
        image=Variable(image)
        target=Variable(target) 


        optimizer.zero_grad()
        with torch.set_grad_enabled(phase == 'train'):
          output = model.forward(image)
          output=(output.squeeze(0))
                    
          loss = criterion(output, target) 
         
          if phase=='train':
            loss.backward()
            loss=loss.item()
            optimizer.step()
            train_loss=train_loss+loss*image.size(0)      
          else:
            loss=loss.item()
            val_loss=val_loss+loss*image.size(0)         
            dice=Dice(target.detach().cpu().numpy(),output.detach().cpu().numpy())
            print('Dice score: ',dice)
            val_dice.append(dice)  
          
          print('Loss: ',loss*image.size(0))   
           

</code></pre>

username: Last_One
cooked: <p>yes, I know!! But what I sent is not working, and I would like to understand why!</p>
<p>I get this error: RuntimeError: size mismatch, m1: [1 x 3920], m2: [720 x 1024] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:283</p>
<pre><code class="lang-auto">batch_size = 1
learning_rate = 0.001
threads=1
train_loader = torch.utils.data.DataLoader(train_dataset, num_workers=threads, batch_size=batch_size, shuffle=True)
val_loader = torch.utils.data.DataLoader(val_dataset, num_workers=threads, batch_size=batch_size, shuffle=False)
</code></pre>
<pre><code class="lang-auto">class CNN(nn.Module): 
    def __init__(self):
        super(CNN, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=3, out_channels=10, kernel_size=3)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=3)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(720, 1024)
        self.fc2 = nn.Linear(1024, 2)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(x.shape[0],-1)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return x
</code></pre>
<pre><code class="lang-auto">model = CNN()

cuda = torch.cuda.is_available()
print(cuda)
if cuda:
   model = model.cuda()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(),lr = learning_rate)
</code></pre>
<pre><code class="lang-auto">starting_epoch=0
epochs=50
best_dice=0
loss_accum = []

for epoch in range(starting_epoch, epochs):  
    print ('Running epoch ',epoch)
    train_loss=0
    val_loss=0
    val_dice=[]

    for phase in ['train','val']:

      if phase=='train':
        loader=train_loader
        model.train()
      else:  
        loader=val_loader
        model.eval()

      for batch in loader:
        if cuda:
          image = batch['image'].cuda()
          target = batch['target'].cuda()
 
        
        #print(image)
        image=Variable(image)
        target=Variable(target) 


        optimizer.zero_grad()
        with torch.set_grad_enabled(phase == 'train'):
          output = model.forward(image)
          output=(output.squeeze(0))
                    
          loss = criterion(output, target) 
         
          if phase=='train':
            loss.backward()
            loss=loss.item()
            optimizer.step()
            train_loss=train_loss+loss*image.size(0)      
          else:
            loss=loss.item()
            val_loss=val_loss+loss*image.size(0)         
            dice=Dice(target.detach().cpu().numpy(),output.detach().cpu().numpy())
            print('Dice score: ',dice)
            val_dice.append(dice)  
          
          print('Loss: ',loss*image.size(0))   
           

</code></pre>

username: RaLo4
cooked: <aside class="quote no-group" data-username="Last_One" data-post="6" data-topic="90816">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/last_one/48/12953_2.png" class="avatar"> Last_One:</div>
<blockquote>
<p>I get this error: pic should be PIL Image or ndarray. Got &lt;class ‘str’&gt;</p>
</blockquote>
</aside>
<p>All the transformations listed under <a href="https://pytorch.org/docs/stable/torchvision/transforms.html#transforms-on-pil-image" rel="noopener nofollow ugc">Transforms on PIL Image</a> need to be done on PIL image and not a tensor or ndarray.<br>
Meaning they should be done before <code>torchvision.transforms.ToTensor()</code>, because this transforms your PIL image or ndarry to a tensor.<br>
I you need, you can also do it in the opposite direction and transform a tensor or a ndarray to a PIL image using <code>torchvision.transforms.ToPILImage()</code></p>

username: RaLo4
cooked: <p>I think you have a size mismatch when passing x into the first linear layer (self.fc1). Please check if x.shape (before passing into fc1) is really 720 in the second dimension.</p>
<aside class="quote no-group" data-username="Last_One" data-post="7" data-topic="90816">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/last_one/48/12953_2.png" class="avatar"> Last_One:</div>
<blockquote>
<p><code>x = F.relu(self.fc1(x))</code></p>
</blockquote>
</aside>