Loss is increasing from first epoch itself

Vaijenath_Biradar: <p>I am training my siamese network for nlp. I have used lstm in it. and BCELoss. My loss is increasing from the first epoch. The first 36 epoch loss is<br>
error after  0 is<br>
272.4357<br>
[torch.FloatTensor of size 1]</p>
<p>error after  1 is<br>
271.8972<br>
[torch.FloatTensor of size 1]</p>
<p>error after  2 is<br>
271.5598<br>
[torch.FloatTensor of size 1]</p>
<p>error after  3 is<br>
271.6979<br>
[torch.FloatTensor of size 1]</p>
<p>error after  4 is<br>
271.7315<br>
[torch.FloatTensor of size 1]</p>
<p>error after  5 is<br>
272.3965<br>
[torch.FloatTensor of size 1]</p>
<p>error after  6 is<br>
273.3982<br>
[torch.FloatTensor of size 1]</p>
<p>error after  7 is<br>
275.1197<br>
[torch.FloatTensor of size 1]</p>
<p>error after  8 is<br>
275.8228<br>
[torch.FloatTensor of size 1]</p>
<p>error after  9 is<br>
278.3311<br>
[torch.FloatTensor of size 1]</p>
<p>error after  10 is<br>
277.1054<br>
[torch.FloatTensor of size 1]</p>
<p>error after  11 is<br>
277.8418<br>
[torch.FloatTensor of size 1]</p>
<p>error after  12 is<br>
279.0189<br>
[torch.FloatTensor of size 1]</p>
<p>error after  13 is<br>
278.4090<br>
[torch.FloatTensor of size 1]</p>
<p>error after  14 is<br>
281.8813<br>
[torch.FloatTensor of size 1]</p>
<p>error after  15 is<br>
283.4077<br>
[torch.FloatTensor of size 1]</p>
<p>error after  16 is<br>
286.3093<br>
[torch.FloatTensor of size 1]</p>
<p>error after  17 is<br>
287.6292<br>
[torch.FloatTensor of size 1]</p>
<p>error after  18 is<br>
297.2318<br>
[torch.FloatTensor of size 1]</p>
<p>error after  19 is<br>
307.4176<br>
[torch.FloatTensor of size 1]</p>
<p>error after  20 is<br>
304.6649<br>
[torch.FloatTensor of size 1]</p>
<p>error after  21 is<br>
328.9772<br>
[torch.FloatTensor of size 1]</p>
<p>error after  22 is<br>
300.0669<br>
[torch.FloatTensor of size 1]</p>
<p>error after  23 is<br>
292.3902<br>
[torch.FloatTensor of size 1]</p>
<p>error after  24 is<br>
300.8633<br>
[torch.FloatTensor of size 1]</p>
<p>error after  25 is<br>
305.1822<br>
[torch.FloatTensor of size 1]</p>
<p>error after  26 is<br>
333.9984<br>
[torch.FloatTensor of size 1]</p>
<p>error after  27 is<br>
346.2062<br>
[torch.FloatTensor of size 1]</p>
<p>error after  28 is<br>
354.6148<br>
[torch.FloatTensor of size 1]</p>
<p>error after  29 is<br>
341.3568<br>
[torch.FloatTensor of size 1]</p>
<p>error after  30 is<br>
369.7580<br>
[torch.FloatTensor of size 1]</p>
<p>error after  31 is<br>
366.1615<br>
[torch.FloatTensor of size 1]</p>
<p>error after  32 is<br>
368.2455<br>
[torch.FloatTensor of size 1]</p>
<p>error after  33 is<br>
391.4102<br>
[torch.FloatTensor of size 1]</p>
<p>error after  34 is<br>
394.3190<br>
[torch.FloatTensor of size 1]</p>
<p>error after  35 is<br>
401.0990<br>
[torch.FloatTensor of size 1]</p>
<p>error after  36 is<br>
422.3723<br>
[torch.FloatTensor of size 1]</p>

Even_Oldridge: <p>Most likely culprit here is exploding gradients due to a high learning rate.</p>
<p>Try lowering your learning rate by one or two orders of magnitude and see if it still grows.</p>

Vaijenath_Biradar: <p>Thanks for the reply. Will try.</p>

Leon_Lu: <p>Thanks, you helped me with my problem too.</p>