My model give 0.42 accuracy, what is the problem

Shatha_Alotaibi: <p>I have a private dataset of 413 samples for Diabetic Retinopathy classification.<br>
I used pre-trained Resnet50, but the accuracy for training is poor, I need some one to tell me why my model doesn’t learn.</p>
<p>this the code :<br>
<span class="hashtag-raw">#preprocessing</span> and data augmentation<br>
train_transforms = transforms.Compose([<br>
transforms.ToPILImage(),# Convert a tensor or an ndarray to PIL Image (may be I need it)<br>
transforms.CenterCrop(2300),<br>
transforms.Resize((224,224)),<br>
transforms.RandomRotation(90),  # the training set is small so I will implement some Augmentations (1)<br>
transforms.AugMix(), # (2)<br>
transforms.RandomHorizontalFlip(p=0.4), # (3)<br>
transforms.RandomVerticalFlip(p=0.4),# (4)<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=(0.485,0.456,0.406), std=(0.299, 0.224, 0.225)),</p>
<p>])<br>
<span class="hashtag-raw">#preprocessing</span> and data augmentation</p>
<h1><a name="in-the-testing-set-we-dont-implement-augmentation-1" class="anchor" href="#in-the-testing-set-we-dont-implement-augmentation-1"></a>In the testing set we don’t implement Augmentation</h1>
<p>test_transforms = transforms.Compose([<br>
transforms.ToPILImage(),# Convert a tensor or an ndarray to PIL Image (may be I need it)<br>
transforms.CenterCrop(2500),<br>
transforms.Resize((224,224)),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=(0.485,0.456,0.406), std=(0.299, 0.224, 0.225)),</p>
<p>])</p>
<p><span class="hashtag-raw">#Our</span> own custom for datasets</p>
<p>class createDataset(Dataset):<br>
def <strong>init</strong> (self,df_data,data_dir=“/content/drive/MyDrive/Dia_dataset”, transform=None):<br>
super().<strong>init</strong>()<br>
self.df=df_data.values<br>
self.data_dir=data_dir<br>
self.transform=transform</p>
<pre><code>def __len__(self):
    return len(self.df)

def __getitem__(self, index):
    img_name,label=self.df[index]
    img_path=os.path.join(self.data_dir,img_name+'.jpg')
    image=cv2.imread(img_path)
    if self.transform is not None:
       image=self.transform(image)
    return image, label
</code></pre>
<h1><a name="creat-trainingvalidationtesting-dataset-2" class="anchor" href="#creat-trainingvalidationtesting-dataset-2"></a>Creat training/validation/testing dataset,</h1>
<p>train_data=createDataset(df_data=training_csv,data_dir=train_path,transform=train_transforms)</p>
<p>valid_data=createDataset(df_data=training_csv,data_dir=train_path,transform=test_transforms)</p>
<p>test_data=createDataset(df_data=testing_csv,data_dir=test_path,transform=test_transforms)</p>
<p>np.random.seed(13)</p>
<h1><a name="split-the-trainig-set-into80-training-20-validation-3" class="anchor" href="#split-the-trainig-set-into80-training-20-validation-3"></a>Split the trainig set into(80% training , 20% Validation)</h1>
<p>validation_size=0.2</p>
<p>num_train=len(train_data)</p>
<p>indices=list(range(num_train))</p>
<p>np.random.shuffle(indices)</p>
<p>split=int(np.floor(validation_size*num_train))</p>
<p>trainloader=torch.utils.data.DataLoader(train_data,batch_size=10, sampler=train_sampler, num_workers=workers)</p>
<h1><a name="sample-the-validation-dataset-from-a-separate-dataset-the-doesnt-include-the-image-aug-transformations-4" class="anchor" href="#sample-the-validation-dataset-from-a-separate-dataset-the-doesnt-include-the-image-aug-transformations-4"></a>sample the validation dataset from a separate dataset the doesn’t include the image aug transformations.</h1>
<p>valloader=torch.utils.data.DataLoader(valid_data,batch_size=10, sampler=validation_sampler, num_workers=workers)</p>
<p>testloader=torch.utils.data.DataLoader(test_data,batch_size=10, num_workers=workers)</p>
<p>device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)</p>
<p>model=models.resnet50(pretrained=True)</p>
<h1><a name="disable-gradients-on-all-model-parameters-to-freeze-the-weights-5" class="anchor" href="#disable-gradients-on-all-model-parameters-to-freeze-the-weights-5"></a>Disable gradients on all model parameters to freeze the weights</h1>
<p>for param in model.parameters():<br>
param.requires_grad = False</p>
<p>n_inputs = model.fc.in_features<br>
out_feature=5</p>
<h1><a name="replace-the-final-fully-connected-resnet-layer-with-a-2-fc-layer-network-and-sigmoid-output-6" class="anchor" href="#replace-the-final-fully-connected-resnet-layer-with-a-2-fc-layer-network-and-sigmoid-output-6"></a>Replace the final fully connected resnet layer with a 2 fc layer network and sigmoid output</h1>
<p>model.fc = nn.Sequential(nn.Linear(n_inputs, 512),<br>
nn.ReLU(),<br>
nn.Linear(512,out_feature),<br>
nn.LogSoftmax(dim=1))</p>
<p>for param in model.fc.parameters():<br>
param.requires_grad = True</p>
<h1><a name="unfreeze-the-last-few-layers-of-the-model-7" class="anchor" href="#unfreeze-the-last-few-layers-of-the-model-7"></a>Unfreeze the last few layers of the model</h1>
<p>for param in model.layer4.parameters():<br>
param.requires_grad = True</p>
<p>the training model I use the one in  PyTorch website:</p>
<h1><a name="specify-loss-function-and-optimizer-8" class="anchor" href="#specify-loss-function-and-optimizer-8"></a>Specify loss function and optimizer</h1>
<p>criterion = nn.CrossEntropyLoss()</p>
<p>optimizer = optim.SGD(model.fc.parameters(), lr = 0.001, momentum=0.9)</p>
<p>‘’’</p>
<p>criterion = nn.CrossEntropyLoss()</p>
<p>optimizer = optim.Adam(model.parameters(), lr=0.00001)</p>
<p>‘’’</p>
<p>exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)</p>
<p>model.to(device)</p>
<p>(The model)---------------------------<br>
dataloaders = {‘train’:trainloader, ‘val’: valloader}<br>
dataset_sizes = {‘train’: 331, ‘val’: 82}</p>
<p>import os<br>
from tempfile import TemporaryDirectory</p>
<p>def train_model(model, criterion, optimizer, scheduler, num_epochs=25):<br>
since = time.time()<br>
acc_validation = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
acc_training = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
loss_validation = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
loss_training = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<pre><code># Create a temporary directory to save training checkpoints
with TemporaryDirectory() as tempdir:
    best_model_params_path = os.path.join(tempdir, 'best_model_params.pt')

    torch.save(model.state_dict(), best_model_params_path)
    best_acc = 0.0

    for epoch in range(num_epochs):
        print(f'Epoch {epoch +1}/{num_epochs }')
        print('-' * 10)

        # Each epoch has a training and validation phase
        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode

            running_loss = 0.0
            running_corrects = 0

            # Iterate over data.
            for inputs, labels in dataloaders[phase]:
                inputs = inputs.to(device)
                labels = labels.to(device)
                labels = labels.squeeze().long()

                # zero the parameter gradients
                optimizer.zero_grad()

                # forward
                # track history if only in train
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)
                    loss = criterion(outputs, labels)

                    # backward + optimize only if in training phase
                    if phase == 'train':
                        loss.backward()
                        optimizer.step()

                # statistics
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)
            if phase == 'train':
                scheduler.step()

            epoch_loss = running_loss / dataset_sizes[phase]
            epoch_acc = running_corrects.double() / dataset_sizes[phase]
            if phase == 'val':
                acc_validation.append(epoch_acc.item())
                loss_validation.append(epoch_loss)

            elif phase =='train':
                acc_training.append(epoch_acc.item())
                loss_training.append(epoch_loss)

            print(f'{phase} Loss: {epoch_loss:.4f} Acc: {epoch_acc:.4f}')

            # deep copy the model
            if phase == 'val' and epoch_acc &gt; best_acc:
                best_acc = epoch_acc
                torch.save(model.state_dict(), best_model_params_path)

        print()

    time_elapsed = time.time() - since
    print(f'Training complete in {time_elapsed // 60:.0f}m {time_elapsed % 60:.0f}s')
    print(f'Best val Acc: {best_acc:4f}')

    # load best model weights
    model.load_state_dict(torch.load(best_model_params_path))
return model, acc_validation, acc_training, loss_validation, loss_training
</code></pre>

KFrank: <p>Hi Shatha!</p>
<aside class="quote no-group" data-username="Shatha_Alotaibi" data-post="1" data-topic="192146" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shatha_alotaibi/48/65598_2.png" class="avatar"> Shatha_Alotaibi:</div>
<blockquote>
<p>this the code :</p>
</blockquote>
</aside>
<p>There’s too much code here to look at it all, but one thing did catch<br>
my eye.</p>
<aside class="quote no-group">
<blockquote>
<h1>Replace the final fully connected resnet layer with a 2 fc layer network and sigmoid output</h1>
<p>model.fc = nn.Sequential(nn.Linear(n_inputs, 512),<br>
nn.ReLU(),<br>
nn.Linear(512,out_feature),<br>
nn.LogSoftmax(dim=1))<br>
…</p>
<h1>Specify loss function and optimizer</h1>
<p>criterion = nn.CrossEntropyLoss()</p>
</blockquote>
</aside>
<p>Remove the <code>LogSoftmax</code> from your <code>Sequential</code>.  <code>CrossEntropyLoss</code><br>
has <code>LogSoftmax</code> built into it so it expects its <code>input</code> to come directly from<br>
the final <code>Linear</code> layer of your model.</p>
<p>(If you need to use <code>LogSoftmax</code> for some reason – and you probably<br>
don’t – use <code>NLLLoss</code> as your loss criterion.)</p>
<p>Edit:  Let me correct what I said here:</p>
<p>Using <code>LogSoftmax</code> here is unnecessary (and therefore ever so slightly<br>
inefficient), but not harmful.  In short, this is because two <code>LogSoftmax</code>s<br>
in a row is the same as only one so that feeding the output of a model<br>
through <code>LogSoftmax</code> and into <code>CrossEntropyLoss</code> is the same as feeding<br>
the output directly into <code>CrossEntropyLoss</code>.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'2.1.0'
&gt;&gt;&gt; _ = torch.manual_seed (2023)
&gt;&gt;&gt; pred = torch.randn (5, 3)
&gt;&gt;&gt; targ = torch.randint (3, (5,))
&gt;&gt;&gt; pred_lsm = torch.nn.LogSoftmax (1)(pred)
&gt;&gt;&gt; torch.nn.CrossEntropyLoss() (pred, targ)
tensor(1.2384)
&gt;&gt;&gt; torch.nn.CrossEntropyLoss() (pred_lsm, targ)
tensor(1.2384)
&gt;&gt;&gt; pred
tensor([[-1.2075,  0.5493, -0.3856],
        [ 0.6910, -0.7424,  0.1570],
        [ 0.0721,  1.1055,  0.2218],
        [-0.0794, -1.0846, -1.5421],
        [ 0.9377, -0.9787,  2.0930]])
&gt;&gt;&gt; pred_lsm
tensor([[-2.2048, -0.4480, -1.3829],
        [-0.6015, -2.0348, -1.1354],
        [-1.6038, -0.5705, -1.4541],
        [-0.4685, -1.4737, -1.9312],
        [-1.4638, -3.3801, -0.3084]])
&gt;&gt;&gt; torch.nn.LogSoftmax (1)(pred_lsm)
tensor([[-2.2048, -0.4480, -1.3829],
        [-0.6015, -2.0348, -1.1354],
        [-1.6038, -0.5705, -1.4541],
        [-0.4685, -1.4737, -1.9312],
        [-1.4638, -3.3801, -0.3084]])
</code></pre>
<p>(You can think of this as follows:  <code>LogSoftmax</code> “normalizes”<br>
unnormalized log-probabilities into normalized log-probabilities.  The<br>
second <code>LogSoftmax</code> normalizes the log-probabilities a second time,<br>
but, because they are already normalized, nothing further happens.)</p>
<p>So your <code>LogSoftmax</code> is not the cause of your problem.</p>
<p>Best.</p>
<p>K. Frank</p>

Shatha_Alotaibi: <p>Thank you so much for answering me</p>