My model parameters are not updated

username: kiriakospapa
cooked: <p>Hello, I created a neural network to predict the x center of my palm. It consists of 2 cnn followed by a max pooling layer and 2 linear layers. The input is 720x720 image. Here is the code of the nn:</p>
<pre><code>class MyNeuralNetwork(torch.nn.Module):
    def __init__(self):
        super(MyNeuralNetwork, self).__init__()
        self.conv1 = torch.nn.Conv2d(4, 5, 5)
        self.conv2 = torch.nn.Conv2d(5, 5, 5)
        self.pool = torch.nn.MaxPool2d(3, 3)
        self.linear1 = torch.nn.Linear(5 * 78 * 78, 100)
        self.linear2 = torch.nn.Linear(100, 1)

    def forward(self, x):
        x = self.conv1(x)
        x = self.pool(x)
        x = self.conv2(x)
        x = self.pool(x)
        x = x.view(x.size(0), -1)
        x = self.linear1(x)
       x = self.linear2(x)
       return x
</code></pre>
<p>and here is my Dataloader:</p>
<pre><code>class MyHand(Dataset):
"""Creating the proper dataset to feed my neural network"""
   def __init__(self, name_path, root_dir, results_path, transform=None):
       self.names = pd.read_csv(name_path)
       self.rootdir = root_dir
       self.transform = transform
       self.results = pd.read_csv(results_path)

   def __len__(self):
       length = len(self.names.columns)
       return length

   def __getitem__(self, index):
       img_path = os.path.join(self.rootdir, self.names.columns[index])
       image = pl.imread(img_path)
    
       x_center = torch.tensor(self.results.iloc[index, 0])
    
       if self.transform:
           image = self.transform(image)

    return image, x_center
</code></pre>
<p>and the part of the code for training:</p>
<pre><code>EPOCHS = 5
LEARNING_RATE = 0.001
model = MyNeuralNetwork()
criterion = torch.nn.MSELoss()
optimizer = optim.SGD(model.parameters(), LEARNING_RATE)
for epoch in range(EPOCHS):

    print("epoch:", epoch)
    for batch, (pic, x_center) in enumerate(loader):

        outpout = model(pic)
        loss = criterion(outpout/720, x_center/720)
        # Backpropagation
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        print(loss)
</code></pre>
<p>As you can see below my model predicts the same result for every batch at each epoch, thus the weights and bias are not updated<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/7/77fbae55c3bf6b3ab8033fb6cd2b54e6aa102815.png" data-download-href="https://discuss.pytorch.org/uploads/default/77fbae55c3bf6b3ab8033fb6cd2b54e6aa102815" title="pytorch"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77fbae55c3bf6b3ab8033fb6cd2b54e6aa102815_2_690x304.png" alt="pytorch" data-base62-sha1="h7q2Fpse5cWa6aMGmvl2RyFMN13" width="690" height="304" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77fbae55c3bf6b3ab8033fb6cd2b54e6aa102815_2_690x304.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77fbae55c3bf6b3ab8033fb6cd2b54e6aa102815_2_1035x456.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/7/77fbae55c3bf6b3ab8033fb6cd2b54e6aa102815_2_1380x608.png 2x" data-dominant-color="2E2F30"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">pytorch</span><span class="informations">1827×807 127 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>also the warning that are not showed fully in the screenshot are:</p>
<ol>
<li>UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  …\c10/core/TensorImpl.h:1156.)<br>
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)</li>
</ol>
<p>2( the warning is twice): UserWarning: Using a target size (torch.Size([4])) that is different to the input size (torch.Size([4, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.<br>
return F.mse_loss(input, target, reduction=self.reduction)</p>

username: Yasar_Niyazoglu
cooked: <p>Why didn’t you use the activation function? If you don’t use an Activation function, it will be a linear network. Can you try using the activation function.</p>

username: kiriakospapa
cooked: <p>Thank you for the fast reply! I tried several times the sigmoid function and nothing changed. The gradients values are very small and as a result the change of the weights and bias is neglectable, that’s why the loss function is the same during every epoch</p>

username: Yasar_Niyazoglu
cooked: <p>Can you try the Relu function? I was wondering about the problem and the solution too.</p>

username: kiriakospapa
cooked: <p>Hey Yasar, I figured out what the problem was. It had to do about the 2nd warning at my question. I had to do torch.flatten(x_center)  in order to be a tensor of size[4] instead of [4,1]. Despite the warning, I didn’t find it very reasonable and if you or anyone else could explain why it works like this, it would be very useful.</p>