Need sugesstions on getting accuracy

username: IamSparky
cooked: <p>So my label looks like this</p>
<pre><code class="lang-auto">_,labels = next(iter(dataloaders['val']))
labels
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/3/43dfd80e04a5b5347e7d951e71317b4bb6970211.png" data-download-href="https://discuss.pytorch.org/uploads/default/43dfd80e04a5b5347e7d951e71317b4bb6970211" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/3/43dfd80e04a5b5347e7d951e71317b4bb6970211.png" alt="image" data-base62-sha1="9GrzAsuyVCRJMSxj81EdgdFKzex" width="690" height="140" data-dominant-color="303338"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">803×163 2.64 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>How I should change the keywords in the training loop to get accuracy ?</p>
<pre><code class="lang-auto">for epoch in range(EPOCHS):
        print('Epoch ', epoch,'/',EPOCHS-1)
        print('-'*15)

        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode

            running_loss = 0.0
            running_corrects = 0.0

            # Iterate over data.
            for inputs,labels in dataloaders[phase]:
                inputs = inputs.to(device, dtype=torch.float)
                labels = labels.to(device, dtype=torch.float)

                # zero the parameter gradients
                optimizer.zero_grad()

                with torch.set_grad_enabled(phase == 'train'):

                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)

                    loss = loss_fn(outputs,labels)

                    # we backpropagate to set our learning parameters only in training mode
                    if phase == 'train':
                        loss.backward()
                        optimizer.step()

                running_loss += loss.item() * inputs.size(0)
                
                if phase == 'val':
                    running_corrects += torch.sum(preds == torch.argmax(labels.data, dim=1))

            # scheduler for weight decay
            if phase == 'train':
                scheduler.step()
                epoch_loss = running_loss / float(dataset_sizes[phase])
                xm.master_print('{} Loss: {:.4f}'.format(phase, epoch_loss))
            else:
                epoch_loss = running_loss / float(dataset_sizes[phase])
                epoch_acc = running_corrects / float(dataset_sizes[phase])
                xm.master_print('{} Loss: {:.4f} Acc: {:.4f}'.format(phase, epoch_loss, epoch_acc))
</code></pre>
<p><a href="https://www.kaggle.com/soumochatterjee/cutmix-flower-classification" rel="noopener nofollow ugc">link to my notebook</a></p>

username: ptrblck
cooked: <p><code>epoch_acc</code> seems to already calculate the accuracy for the validation loop.<br>
Are you seeing unexpected values or any other issues?</p>

username: IamSparky
cooked: <p>It is breaking after the end of 1st training loop . Have a look to <a href="https://www.kaggle.com/soumochatterjee/cutmix-flower-classification" rel="nofollow noopener">my notebook</a>.</p>

username: ptrblck
cooked: <p>Could you check, if your code si running fine without using multiprocessing via:</p>
<pre><code class="lang-python">xmp.spawn(_mp_fn, args=(FLAGS,), nprocs=1, start_method='fork')
</code></pre>

username: IamSparky
cooked: <p>I am runnig this notebook in integration with torch xla with Tpu …if I remove that line it is giving me error .<br>
Can you try copy and edit this <a href="https://www.kaggle.com/soumochatterjee/cutmix-flower-classification" rel="nofollow noopener">notebook</a></p>

username: IamSparky
cooked: <p>I changed the code a little bit and it started working . But the loss is still to high ; What should I make changes in model class to improve the accuracy ? <strong>I am Training 51000 images with a batch size of 256 sized at 224 x 224 , for 25 epochs.</strong><br>
<a href="https://www.kaggle.com/soumochatterjee/cutmix-flower-classification" rel="nofollow noopener">Notebook Link</a></p>
<pre><code class="lang-auto">import torch.nn as nn
import torch.nn.functional as F

class EfficientNet_b0(nn.Module):
    def __init__(self):
        super(EfficientNet_b0, self).__init__()
        self.model = efficientnet_pytorch.EfficientNet.from_pretrained('efficientnet-b0')
        
        self.classifier_layer = nn.Sequential(
            nn.Linear(1280 , 512),
            nn.BatchNorm1d(512),
            nn.Dropout(0.2),
            nn.Linear(512 , 256),
            nn.Linear(256 , 104)
        )
        
    def forward(self, inputs):
        x = self.model.extract_features(inputs)

        # Pooling and final linear layer
        x = self.model._avg_pooling(x)
        x = x.flatten(start_dim=1)
        x = self.model._dropout(x)
        x = self.classifier_layer(x)
        return x
    
model = EfficientNet_b0()
model = model.to(device)
</code></pre>

username: ptrblck
cooked: <p>You might want to add activation functions into <code>classifier_layer</code>. If that doesn’t help, try to overfit a small data sample (e.g. just 10 samples) by playing around with some hyperparameters. Once this is done you could try to scale up the use case again.</p>

username: IamSparky
cooked: <p>I changed the model after your comments but it’s not heling me out rather the accuracy is decreasing<br>
Need help in how to increase the accuracy in retrained mdels using pytorch ? Any good references will also help!<br>
<a href="https://www.kaggle.com/soumochatterjee/mixup-tpu" rel="noopener nofollow ugc">Notebook Link</a></p>
<pre><code class="lang-auto">class EfficientNet_b0(nn.Module):
    def __init__(self):
        super(EfficientNet_b0, self).__init__()
        self.model = efficientnet_pytorch.EfficientNet.from_pretrained('efficientnet-b0')
        
        self.dense_layer_1 = nn.Linear(1280 , 512)
        self.batchNorm_layer = nn.BatchNorm1d(512)
        self.droput_layer = nn.Dropout(0.2)
        
        self.dense_layer_2 = nn.Linear(512,216)
        
        self.dense_layer_3 = nn.Linear(216,104)
        
        
    def forward(self, inputs):
        x = self.model.extract_features(inputs)

        # Pooling and final linear layer
        x = self.model._avg_pooling(x)
        x = x.flatten(start_dim=1)
        x = self.model._dropout(x)
        
        x = self.dense_layer_1(x)
        x = self.batchNorm_layer(x)
        x = self.droput_layer(x)
        
        x = self.dense_layer_2(x)
        x = torch.relu(x)
        
        x = self.dense_layer_3(x)
        x = torch.log_softmax(x , dim =1)
        return x
    
model = EfficientNet_b0()
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/4/040206a8a0d14cff39bdfb30ef4e24580c72da76.png" data-download-href="https://discuss.pytorch.org/uploads/default/040206a8a0d14cff39bdfb30ef4e24580c72da76" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/4/040206a8a0d14cff39bdfb30ef4e24580c72da76_2_306x500.png" alt="image" data-base62-sha1="zsfxgtgpIRwlJQeXEBW2JO5s4S" width="306" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/4/040206a8a0d14cff39bdfb30ef4e24580c72da76_2_306x500.png, https://discuss.pytorch.org/uploads/default/original/3X/0/4/040206a8a0d14cff39bdfb30ef4e24580c72da76.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/4/040206a8a0d14cff39bdfb30ef4e24580c72da76.png 2x" data-dominant-color="2E3036"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">438×715 36.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Did you try to overfit the small data samples and is the output the result of this test?</p>