VINet implementation didn't converge, any ways to resolve it?

Jacky_Liu: <blockquote>
<p>This problem is not closely related to the language property of Pytorch.  If it is not proper to ask this question here, I will delete it right away.<br>
However, I couldn’t think of anywhere that will have so many experts who are familiar with NN structure and design.</p>
</blockquote>
<p>Recently, I tried to implement VINet[1] a visual inertial odometry system that built with the neural network, and I open source it to GitHub <a href="https://github.com/HTLife/VINet" rel="noopener nofollow ugc">HTLife/VINet</a></p>
<p>I already complete whole network structure, but the network can’t converge properly in training.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/c4a3a9b45b63b16ae2de3062512446a0d7df7c91.jpg" data-download-href="https://discuss.pytorch.org/uploads/default/c4a3a9b45b63b16ae2de3062512446a0d7df7c91" title="c21c66fe-70d4-45a2-ba59-49f6d6e36196"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/c/c4a3a9b45b63b16ae2de3062512446a0d7df7c91_2_390x200.jpg" alt="c21c66fe-70d4-45a2-ba59-49f6d6e36196" data-base62-sha1="s3yehy3I8c1VP43PUlqFms4jBbH" width="390" height="200" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/c/c4a3a9b45b63b16ae2de3062512446a0d7df7c91_2_390x200.jpg, https://discuss.pytorch.org/uploads/default/optimized/2X/c/c4a3a9b45b63b16ae2de3062512446a0d7df7c91_2_585x300.jpg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/c/c4a3a9b45b63b16ae2de3062512446a0d7df7c91_2_780x400.jpg 2x" data-dominant-color="FBF2EE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">c21c66fe-70d4-45a2-ba59-49f6d6e36196</span><span class="informations">827×480 46.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
How could I fix this problem?</p>
<h1><a name="possible-problems-solutions-1" class="anchor" href="#possible-problems-solutions-1"></a>Possible problems &amp; solutions</h1>
<ol>
<li>
<p>The dataset is too challenging:<br>
I’m using the EuRoC MAV dataset, which is more challenging than the KITTI VO Dataset used by the DeepVO, Vinet(because the KITTI vehicle image does not shake up and down). NN cannot learn camera movement correctly.</p>
</li>
<li>
<p>Loss function:<br>
L1 loss is been used and identical to the design in [1]. (I’m not very confident about whether I understand the loss design in [1] currently.) Related code</p>
</li>
<li>
<p>Other hyperparameter problems</p>
</li>
</ol>
<h1><a name="references-2" class="anchor" href="#references-2"></a>References</h1>
<ul>
<li>[1] Clark, Ronald, et al. “VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem.” AAAI. 2017.</li>
</ul>

fangxu622: <aside class="quote no-group" data-username="Jacky_Liu" data-post="1" data-topic="16262">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jacky_liu/48/3916_2.png" class="avatar"> Jacky_Liu:</div>
<blockquote>
<p>yperparameter</p>
</blockquote>
</aside>
<p>hi,have you ask for authors of paper?</p>