My cifar-10 model loss is not updating

username: Sanjayvarma11
cooked: <p>Hi.I am trying to train my model on cifar-10 dataset.</p>
<p>Here when i started training my code my loss is not even updating my loss is just at 2.30.It is not even updating.I really need some help to look on this<br>
<strong>My training code is</strong></p>
<pre><code class="lang-auto">for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(train_loader, 0):
        # get the inputs
        inputs,labels = data
        inputs,labels=inputs.to(device),labels.to(device)

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = model(inputs)
        #print(outputs)
        loss = criterian1(outputs, labels)
        loss.backward()
        optimizer.step()
        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')
</code></pre>
<p><strong>The output of above code is</strong></p>
<pre><code class="lang-auto">[1,  2000] loss: 2.304
[1,  4000] loss: 2.306
[1,  6000] loss: 2.305
[1,  8000] loss: 2.304
[1, 10000] loss: 2.306
[1, 12000] loss: 2.305
[2,  2000] loss: 2.304
[2,  4000] loss: 2.304
[2,  6000] loss: 2.305
[2,  8000] loss: 2.305
[2, 10000] loss: 2.305
[2, 12000] loss: 2.306
Finished Training
</code></pre>
<p><strong>My model code is</strong></p>
<pre><code class="lang-auto">class Net(nn.Module):
  def __init__(self):
    super(Net,self).__init__()
    self.convblock1=nn.Sequential(
         nn.Conv2d(in_channels=3,out_channels=32,kernel_size=(3,3),padding=2,dilation=2,bias=False),
         nn.ReLU(),
         nn.Conv2d(in_channels=32,out_channels=64,bias=False,padding=2,kernel_size=(3,3),dilation=2),
         nn.ReLU(),
         nn.Conv2d(in_channels=64,out_channels=128,bias=False,padding=2,kernel_size=(3,3),dilation=2),
         nn.ReLU()  
    )
    self.maxpool1=nn.MaxPool2d((2,2),stride=2)
    self.convblock2=nn.Sequential(
         nn.Conv2d(in_channels=128,out_channels=32,kernel_size=(1,1),bias=False),
         nn.ReLU(),
         nn.Conv2d(in_channels=32,out_channels=64,bias=False,padding=2,kernel_size=(3,3),dilation=2),
         nn.ReLU(),
         nn.Conv2d(in_channels=64,out_channels=128,bias=False,padding=2,kernel_size=(3,3),dilation=2),
         nn.ReLU()  
    )
    self.maxpool2=nn.MaxPool2d((2,2),stride=2)
    self.convblock3=nn.Sequential(
         nn.Conv2d(in_channels=128,out_channels=32,kernel_size=(1,1),bias=False),
         nn.ReLU(),
         nn.Conv2d(in_channels=32,out_channels=64,bias=False,padding=1,kernel_size=(3,3)),
         nn.ReLU(),
         nn.Conv2d(in_channels=64,out_channels=128,bias=False,padding=1,kernel_size=(3,3)),
         nn.ReLU()  
    )
    self.maxpool3=nn.MaxPool2d((2,2),stride=2)
    self.gap=nn.AdaptiveAvgPool2d((1,1))
    self.Fc=nn.Conv2d(in_channels=128,out_channels=10,kernel_size=(1,1),bias=False)

  def forward(self,x):
      x=self.convblock1(x)
      x=self.maxpool1(x)
      x=self.convblock2(x)
      x=self.maxpool2(x)
      x=self.convblock3(x)
      x=self.maxpool3(x)
      x=self.gap(x)
      x=self.Fc(x)
      return x;
</code></pre>
<p>Help will be greatly appreciated.Thank you</p>

username: fs4ss1
cooked: <p>What is your learning rate? Very slow learning rate can impact the loss evolution.</p>

username: Sanjayvarma11
cooked: <p>0.001 sir.The kernal values are also not updating.</p>

username: G.M
cooked: <p>First, can u change the channel numbers so that they are not in a descending trend? Second, can u change the dilation rates so that all dilation rates in a group are all relatively prime with each other? Hope my suggestions help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> .</p>

username: Sanjayvarma11
cooked: <p>Here my loss is not updating do you find any problem with training code</p>

username: Sanjayvarma11
cooked: <p>Thank you for answering.I noticed that i set pin_memory=True in train_loader,and when i removed that the network loss starts to update.Thank you</p>

username: Sanjayvarma11
cooked: <p>Thank you for answering.I noticed that i set pin_memory=True in train_loader,and when i removed that the network loss starts to update.</p>