lstm network loss does not drop

zyz1996: <p>Hello everyone, I use the lstm network to deal with the prediction problem of time series. The sequence is 20, and the loss curve has not dropped significantly. The relevant code is as follows,<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/9/893911e46b5834e21642c5218037f232c691de22.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/893911e46b5834e21642c5218037f232c691de22" title="QQ截图20200422101251"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/9/893911e46b5834e21642c5218037f232c691de22.jpeg" alt="QQ截图20200422101251" data-base62-sha1="jzVGtDM5sb3MOZXkSN2HKEuZQ5A" width="493" height="500" data-dominant-color="313131"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20200422101251</span><span class="informations">532×539 32.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/c/7c957947c0b39f0a666d6296b12c9c23f4e2cbfb.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/7c957947c0b39f0a666d6296b12c9c23f4e2cbfb" title="QQ截图20200422101310"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/c/7c957947c0b39f0a666d6296b12c9c23f4e2cbfb.jpeg" alt="QQ截图20200422101310" data-base62-sha1="hM7s2Uy9XBuBS8tyJWuWWnO1vMD" width="690" height="319" data-dominant-color="2E2F2F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20200422101310</span><span class="informations">771×357 21.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

vdw: <p>You may want to have a look at <a href="https://discuss.pytorch.org/t/for-beginners-do-not-use-view-or-reshape-to-swap-dimensions-of-tensors/75524">this post</a>. I use RNNs for classification tasks regularly, and never require <code>view()</code>, <code>reshape()</code> or <code>resize()</code> which can be very harmful when not used carefully.</p>

zyz1996: <p>When I set the batch size to 1, the loss will drop more obviously, but when it is set to other values, the loss will not drop. What is the reason?The following is my loss curve image with batch size 1 and image with batch size 32<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/b/0b9d594c2fa922fdb1d65831daf784a96d3665b6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/0b9d594c2fa922fdb1d65831daf784a96d3665b6" title="QQ截图20200422111905"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/b/0b9d594c2fa922fdb1d65831daf784a96d3665b6_2_560x499.jpeg" alt="QQ截图20200422111905" data-base62-sha1="1EKmY41OSndjsfq8jGcF0zFvDXo" width="560" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/b/0b9d594c2fa922fdb1d65831daf784a96d3665b6_2_560x499.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/0/b/0b9d594c2fa922fdb1d65831daf784a96d3665b6.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/b/0b9d594c2fa922fdb1d65831daf784a96d3665b6.jpeg 2x" data-dominant-color="F6F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20200422111905</span><span class="informations">635×566 19.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

zyz1996: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/0/5029db10632b18241417c2f4d0ef2dbcbfba13ca.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/5029db10632b18241417c2f4d0ef2dbcbfba13ca" title="QQ截图20200422101207"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/5029db10632b18241417c2f4d0ef2dbcbfba13ca_2_553x499.jpeg" alt="QQ截图20200422101207" data-base62-sha1="br9S8c99yDqFoSKFvJ2fI9whpmW" width="553" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/5029db10632b18241417c2f4d0ef2dbcbfba13ca_2_553x499.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/5/0/5029db10632b18241417c2f4d0ef2dbcbfba13ca.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/0/5029db10632b18241417c2f4d0ef2dbcbfba13ca.jpeg 2x" data-dominant-color="C0D7E6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20200422101207</span><span class="informations">635×574 32 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Wei_Wong: <p>the learning rate is too large, that’s why it is oscillating</p>