My Training Loss doesn't change after each epoch

username: nikitha_kona
cooked: <p>I am building a model for a mixed categories(with continuous and categorical variables using word embeddings with 2 linear layers.Its a binary classification problem.My training loss isn’t changing.It would be great if some one help.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/0/063371a8324da66dc9d8eb2aec9b462bd98aa9f0.png" data-download-href="https://discuss.pytorch.org/uploads/default/063371a8324da66dc9d8eb2aec9b462bd98aa9f0" title="51%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/0/063371a8324da66dc9d8eb2aec9b462bd98aa9f0_2_690x460.png" alt="51%20PM" data-base62-sha1="SR599p8iBxg1UToWp0eNoyWzo4" width="690" height="460" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/0/063371a8324da66dc9d8eb2aec9b462bd98aa9f0_2_690x460.png, https://discuss.pytorch.org/uploads/default/optimized/2X/0/063371a8324da66dc9d8eb2aec9b462bd98aa9f0_2_1035x690.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/0/063371a8324da66dc9d8eb2aec9b462bd98aa9f0_2_1380x920.png 2x" data-dominant-color="F4F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">51%20PM</span><span class="informations">1900×1268 291 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>I assume you are not shuffling the data using your <code>DataLoader</code>, since the losses seem to be quite deterministic.<br>
Are you detaching the computation graph accidentally in your model’s <code>forward</code> method?<br>
Also, are you using a custom loss function or a PyTorch one?</p>

username: ptrblck
cooked: <p>Skimming through the code I cannot see anything obviously wrong.<br>
Could you check some gradients after calling <code>loss.backward()</code>?<br>
You could print them using:</p>
<pre><code class="lang-python">print(model.outp.weight.grad)
print(model.embs[0].weight.grad)
</code></pre>
<p>If you see some <code>None</code> values, something is broken. Otherwise we would have to dig a bit deeper.</p>
<p>PS: You can add code snippets using three backticks ```<br>
This makes it easier to copy your code and debug it. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: nikitha_kona
cooked: <p>These are the values I see when I print gradients of outputs and embeddings.</p>
<pre><code class="lang-auto">tensor([[-3.1623e-02, -2.6350e-02, -1.7309e-02, -6.1525e-03, -2.7841e-02,
         -2.6789e-02, -4.0986e-02,  1.1163e-02,  4.6091e-02,  2.4598e-02,
         -2.6607e-02, -2.6708e-03,  2.0545e-03,  1.3116e-02, -1.9225e-02,
          1.8009e-02, -7.0806e-02, -1.9697e-02, -1.6139e-02,  5.6194e-02,
         -8.7657e-02, -1.0188e-02, -1.5533e-02, -1.6141e-02,  2.0929e-02,
         -6.0323e-02,  5.4425e-02, -1.4522e-02, -1.2154e-02,  7.9580e-04,
         -9.7717e-03, -2.8365e-02,  1.0887e-02, -3.1096e-02, -7.4927e-03,
         -8.5017e-03, -3.7971e-03, -1.1083e-03, -2.4572e-02,  4.2590e-03,
         -4.3712e-02, -4.1331e-02, -2.7154e-02, -5.2349e-02, -5.9448e-02,
          3.4661e-02, -4.4277e-02, -1.3610e-02,  2.9865e-05, -1.3078e-02,
         -1.5869e-02, -5.1470e-03,  5.3503e-03,  5.3278e-02, -5.2516e-02,
         -1.4310e-02,  6.2403e-03,  6.2356e-03, -2.6851e-02,  5.1220e-03,
         -5.9708e-02, -6.0736e-02,  1.2995e-02, -2.0492e-02, -2.8461e-02,
         -3.2872e-02, -3.5934e-02,  2.7493e-02, -2.5135e-02, -3.2079e-02,
         -2.5752e-03,  4.8966e-03, -3.5285e-02, -3.0157e-02, -3.8634e-02,
          2.8084e-02, -4.8500e-02,  3.0333e-03, -5.3200e-02, -1.5738e-02,
         -2.8506e-02, -3.4487e-03,  4.0806e-02,  2.0268e-02, -1.5578e-02,
         -1.2922e-02,  3.8184e-03,  2.1328e-02,  2.1187e-02,  4.2885e-02,
         -7.1759e-02, -1.4990e-02,  7.2830e-02, -2.4386e-02, -4.7653e-03,
         -2.1303e-02, -6.4600e-02,  4.3422e-02,  1.4864e-02, -4.8731e-02,
          3.9190e-02,  4.2304e-03,  1.5949e-02, -1.5027e-02, -2.4129e-02,
         -1.9980e-02, -3.4105e-02,  1.3001e-02, -1.7983e-02,  3.3862e-02,
         -4.7518e-02, -9.4020e-03, -2.7877e-02, -4.5408e-02,  1.3022e-02,
          1.0548e-02, -1.8343e-02, -1.6114e-02, -1.3189e-02,  1.1662e-03,
          1.2975e-02,  5.7757e-02,  1.3618e-02, -2.8639e-02, -3.9885e-02,
         -5.5618e-02, -2.3285e-02, -6.9030e-05, -5.9235e-02, -3.2400e-02,
         -2.9226e-02, -6.5258e-02, -3.5231e-02, -3.4279e-02,  2.3250e-02,
          1.7370e-03, -9.1888e-03, -1.1221e-02,  1.5109e-02, -5.0923e-02,
          4.2830e-02, -3.7013e-03, -5.5487e-02, -1.8025e-02,  3.4591e-02,
          7.0318e-02,  3.1747e-02,  3.8931e-02, -1.8351e-02, -9.3918e-03,
         -3.4855e-02, -2.8251e-02,  3.6465e-03,  3.7266e-02, -9.6779e-02,
          5.0921e-03, -2.8991e-02,  6.8520e-03,  9.7878e-03, -2.8903e-02,
         -1.6398e-02, -1.5751e-03, -1.8113e-02, -4.0756e-03, -1.7850e-02,
         -1.0662e-02, -7.5500e-02, -8.3943e-02, -1.1067e-02, -2.7160e-02,
         -2.2912e-02, -2.0004e-03,  2.8608e-02, -4.0171e-02,  1.7596e-02,
         -2.1385e-02,  2.4988e-02, -1.4092e-02, -1.5201e-02,  4.9917e-02,
          1.0833e-02,  8.0092e-03,  4.3952e-02,  3.6990e-02,  4.9020e-03,
         -2.5192e-02, -2.1895e-02,  2.8979e-02,  9.2480e-04, -3.5609e-02,
         -1.1323e-02,  2.3988e-02, -6.2246e-03,  1.1821e-02, -2.8954e-02,
          8.4986e-04,  9.8768e-03,  1.9290e-02, -3.8933e-02,  3.4948e-03,
          3.5462e-02, -3.0178e-02, -5.5254e-03,  1.4226e-03, -5.1838e-03,
         -2.4739e-02, -1.9127e-02, -4.4689e-03, -6.8476e-02, -2.7002e-02,
         -6.2134e-02, -2.6698e-02, -2.2844e-02, -2.1762e-02, -1.5398e-02,
         -3.7795e-02, -5.8228e-03,  1.8611e-02, -4.9444e-03, -2.4235e-02,
          3.1925e-04,  4.2813e-02, -6.5582e-03, -4.9425e-02,  2.1836e-02,
         -2.9703e-02, -9.8694e-02, -4.1492e-02, -2.1444e-02,  5.7452e-03,
         -3.1103e-02,  7.3129e-03,  1.7087e-03,  3.5748e-02,  1.8767e-02,
          1.7446e-02, -1.8929e-02, -7.9848e-02,  7.3374e-04,  2.8640e-02,
          2.9640e-03, -8.9208e-03,  7.3254e-03, -1.5448e-02, -1.9306e-03,
         -6.7582e-02,  2.0013e-02, -2.0523e-02, -2.6685e-03, -5.7062e-02,
          4.8704e-03,  5.6583e-02,  8.2884e-02,  2.9788e-02, -8.3444e-04,
         -2.8515e-02, -7.6275e-03,  5.2626e-03,  5.4316e-03, -3.3708e-02,
         -1.2808e-02,  1.4598e-02, -4.6423e-02,  3.5671e-02,  2.8519e-02,
          3.9008e-02, -1.2556e-02, -1.3970e-02, -1.8020e-02, -1.3051e-02,
          2.3781e-03, -1.7507e-02, -5.3407e-02,  6.4795e-03, -6.4268e-03,
         -2.6923e-02,  3.6155e-02,  9.6456e-03, -2.1730e-02, -1.3292e-02,
         -8.0084e-03, -5.0160e-03, -1.4873e-02, -6.1040e-02, -2.1741e-02,
          3.1036e-02, -2.3044e-03,  9.3973e-04,  1.0799e-02,  1.3042e-02,
         -1.1986e-02,  1.9876e-02,  1.9210e-02, -5.8544e-02, -5.8250e-02,
          2.0442e-02,  1.5879e-02, -1.5362e-02, -2.3519e-02, -6.4135e-02,
         -1.0467e-02,  3.9432e-03,  2.2478e-02, -1.4194e-02, -2.5212e-02,
         -3.7196e-02, -1.1732e-02,  2.6087e-02, -6.3475e-03,  1.1559e-02,
          4.3142e-03, -9.9321e-03,  2.0739e-02, -4.9002e-02,  1.8722e-03,
         -5.0559e-02, -1.5068e-02, -4.1370e-02,  3.7180e-02,  6.4168e-02,
          1.0479e-02, -2.0402e-03,  1.2502e-02, -3.5288e-02, -5.9934e-04,
          1.5053e-02, -6.1099e-03, -2.6714e-02,  9.9228e-03,  2.1643e-02,
          4.9815e-03, -7.8759e-04, -2.0461e-02, -3.7384e-02, -1.5176e-02,
         -3.6167e-02,  6.8868e-02, -4.4166e-03, -3.4864e-02, -3.3638e-02,
          1.9990e-02,  6.2415e-03, -2.7875e-02, -2.7577e-02, -4.2171e-02,
          3.8956e-02,  6.3638e-02,  2.7637e-03,  1.9848e-02,  2.3079e-02,
          3.6044e-02, -1.7952e-02, -1.7918e-02,  6.9048e-02,  3.5189e-02,
          1.9060e-02,  4.7532e-02, -6.4268e-02, -3.3150e-02, -2.4915e-02,
         -1.1871e-02,  7.7971e-03,  1.6802e-02, -6.7766e-03, -6.0042e-02,
          6.7299e-02,  1.8261e-02, -3.1467e-02,  1.8212e-03,  3.1181e-03,
         -5.2383e-02, -2.1561e-02, -3.2257e-02,  5.0996e-02, -1.5949e-02,
         -3.0735e-02,  6.4452e-02, -9.1218e-03, -1.4836e-02,  4.4239e-02,
         -3.9941e-02, -4.5105e-03, -1.2341e-02, -3.3473e-02, -8.3403e-03,
         -3.4777e-02, -1.1807e-02, -4.1881e-02, -2.6477e-02,  8.2249e-03,
         -1.3171e-02,  3.9644e-02, -2.6130e-02, -1.0104e-02, -1.8376e-02,
         -2.3920e-02, -4.3445e-02,  2.3253e-02, -2.4548e-02, -5.0419e-02,
          2.6859e-02, -3.2329e-03, -2.5221e-02, -1.6766e-03, -3.2432e-02,
         -3.7509e-02, -5.2284e-02, -9.5738e-02, -4.9757e-02, -3.4016e-02,
          6.3242e-02, -4.0427e-02,  1.0408e-02, -1.0845e-02,  5.4411e-02,
          6.9872e-02,  7.1449e-03,  1.8818e-02,  1.1320e-02, -2.1045e-02,
          2.0577e-02,  3.1659e-02, -1.6194e-03, -2.5389e-02, -4.5406e-02,
         -4.8093e-03,  1.1047e-03, -1.5277e-02, -2.2355e-02,  1.1083e-02,
         -5.2939e-02, -5.4026e-02,  7.7325e-03, -3.7525e-02, -4.8978e-03,
          2.6263e-03, -4.6975e-02,  1.0279e-02,  3.5905e-02, -1.6772e-02,
          4.8599e-03,  3.5422e-02, -6.1347e-02,  1.3385e-02,  1.8079e-02,
         -1.1148e-02,  3.1458e-03, -5.7805e-02, -4.9902e-02, -4.0210e-02,
         -1.7585e-02,  1.9289e-02, -6.7882e-03,  3.2867e-02, -2.5522e-02,
          5.1807e-03, -1.2839e-03, -4.5885e-03,  4.4152e-02, -5.5047e-02,
         -4.6421e-02, -3.1257e-02,  5.3186e-02,  4.6530e-02,  3.6265e-02,
         -5.1960e-02, -1.4269e-02, -8.0705e-03, -2.9653e-02, -5.6376e-02,
         -3.9184e-02,  6.1064e-02,  7.2560e-04, -4.7297e-03, -4.9365e-02,
          8.8795e-03,  3.3561e-02,  3.2388e-02, -2.4099e-02,  9.2907e-04,
         -1.6416e-02,  2.3246e-02,  8.7383e-02, -2.3976e-03, -1.9337e-02,
         -3.6302e-02, -4.7792e-03, -3.4619e-02, -3.7661e-02,  7.1462e-04,
         -8.2786e-02, -2.0263e-02,  1.2210e-02,  1.0964e-02, -1.2815e-02,
          4.0427e-02, -4.1053e-02, -7.4282e-02, -1.5011e-02, -2.0446e-02]],
       dtype=torch.float64)
tensor([[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0081,  0.0107,  0.0280, -0.0123,  0.0071],
        [-0.0076, -0.0142, -0.0112,  0.0065, -0.0092]], dtype=torch.float64)


</code></pre>

username: ptrblck
cooked: <p>The gradients seem to be valid and also your loss moves.<br>
I would suggest to play around with the hyperparameters of your training, e.g. lower the learning rate and observe if the training loss decreases a bit steadier.</p>