Some how my model isn't updating

Colin_Peng: <p>Hello. I am training a NN classifier and everything seems to work fine. Except that my model isn’t updating(the losses aren’t dropping from the beginning).</p>
<pre><code class="lang-auto">class NN(nn.Module):
    def __init__(self,width = 128):
        super().__init__()
        
        self.fc1 = nn.Sequential(
            nn.Linear(50,512),
            nn.ReLU(),
            nn.Linear(512,512),
            nn.ReLU(),
            nn.Linear(512,512),
            nn.ReLU(),
            nn.Linear(512,256),
            nn.ReLU(),
            nn.Linear(256,256),
            nn.ReLU(),
            nn.Linear(256,128),
            nn.ReLU(),
            nn.Linear(128,64),
            nn.ReLU(),
            nn.Linear(64,32),
            nn.ReLU(),
            nn.Linear(32,16),
            nn.ReLU(),
            nn.Linear(16,4),
            nn.Softmax(dim = 1)
        )
        
        
    def forward(self, x):
        
        out = self.fc1(x)

        return out

    
</code></pre>
<pre><code class="lang-auto">net = NN().to(device)
loss_function = nn.CrossEntropyLoss(weight=weight)
optimizer = optim.Adam(net.parameters(),lr = 0.001)
</code></pre>
<pre><code class="lang-auto">plot_loss = []
for epoch in range(3): # 3 full passes over the data
    
    for data in trainset:  # `data` is a batch of data
        X, y = data  # X is the batch of features, y is the batch of targets.
        
        X = X.to(device)
        y = y.to(device)
        
        
        net.zero_grad()  # sets gradients to 0 before loss calc. You will do this likely every step.
        output = net(X.view(-1,50))  # pass in the reshaped batch (recall they are 28x28 atm)

        loss = loss_function(output, y.flatten())  # calc and grab the loss value
        
        loss.backward()  # apply this loss backwards thru the network's parameters
        optimizer.step()  # attempt to optimize weights to account for loss/gradients
        plot_loss.append(loss)
    print(loss)  # print loss. We hope loss (a measure of wrong-ness) declines!
</code></pre>
<p>the loss after 3 epoch:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/7/2/72be0f771582c9e695dc88b6f1c9ef2d528a537d.png" alt="image" data-base62-sha1="gn3DjNETVCOTMwWW4Ty7rSYTRxz" width="372" height="248"></p>
<p>Thanks in advance for any help.</p>

pascal_notsawo: <p>What do you classify?</p>

Colin_Peng: <p>I am trying to classify features into four categories(0~3). This is the sample data. (a batch)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/8/088d70276d35c59c3d0e7b1cdc4a0c8e9698658e.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/088d70276d35c59c3d0e7b1cdc4a0c8e9698658e" title="1620634527163"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/088d70276d35c59c3d0e7b1cdc4a0c8e9698658e_2_690x395.jpeg" alt="1620634527163" data-base62-sha1="1dEQFtktnCZdy8V00XvkluNnsCi" width="690" height="395" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/088d70276d35c59c3d0e7b1cdc4a0c8e9698658e_2_690x395.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/088d70276d35c59c3d0e7b1cdc4a0c8e9698658e_2_1035x592.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/8/088d70276d35c59c3d0e7b1cdc4a0c8e9698658e.jpeg 2x" data-dominant-color="F7F7F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1620634527163</span><span class="informations">1165×668 110 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Colin_Peng: <p>If there’s any more information I can provide please let me know. Thanks in advance.</p>

KyleOng: <p>Here are some of my suggestions that might help you.</p>
<ol>
<li>Normalize your input between 0 to 1.</li>
<li>Try a smaller model (with less layers).</li>
<li>Adjust the learning rate or try different optimizer.</li>
</ol>