How to train a model?

huang_hopper: <p>I’m using pytorch to train a model as a time-series preditor</p>
<p>my  model look likt htis;</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.init as init

class AttentionModel(nn.Module):
    def __init__(self, input_size, hidden_size, num_heads):
        super(AttentionModel, self).__init__()
        self.linear = nn.Linear(input_size, hidden_size)
        init.kaiming_normal_(self.linear.weight, nonlinearity='linear')
        init.constant_(self.linear.bias, 0)

        self.attention = nn.MultiheadAttention(hidden_size, num_heads, batch_first=True)
        # 如果需要自定义注意力层的初始化，可以在这里进行
        for p in self.attention.parameters():
            if p.dim() &gt; 1:
                nn.init.xavier_uniform_(p)

        self.fc = nn.Linear(hidden_size * 3, 1)
        init.xavier_normal_(self.fc.weight)
        init.constant_(self.fc.bias, 0)

    def forward(self, x):
        # 将输入数据映射到隐藏层
        hidden_states = self.linear(x)
        # 使用MultiheadAttention
        attn_output, _ = self.attention(hidden_states, hidden_states, hidden_states)
        # 取出最后一个时间步的输出作为整个序列的表示
        # context_vector = attn_output[-1, :, :]
        # print(context_vector.size())
        # 全连接层输出
        flt = attn_output.flatten(1)
        # print(flt.size())
        output = self.fc(flt)
        return output

</code></pre>
<p>my data looks like this:<br>
tensor([ 4.6767e+04,  2.7280e+04,  4.6868e+04,  4.6706e+04,  4.6800e+04,<br>
1.3182e+08,  6.3223e+07,  1.3514e+03,  2.8179e+03,  4.6779e+04,<br>
4.6784e+04, -4.0732e+01, -3.6797e+01, -4.2207e+01,  4.7963e+01,<br>
-8.5589e+00,  4.7957e+01, -8.5687e+00, -4.2211e+01, -3.6795e+01,<br>
-3.8831e-03,  6.8489e-03,  1.0001e+02,  1.0732e-02, -7.0962e-02,<br>
9.9963e+01,  9.9975e+01,  4.3305e-01, -1.4060e+00,  2.3138e-01,<br>
1.1266e+00,  1.0921e+00, -1.4012e+00, -1.3839e+00, -1.3821e+00,<br>
-1.3997e+00,  6.9584e-01,  7.2648e-01, -1.3307e+00, -1.3038e+00,<br>
-1.2189e+00, -1.3154e+00, -9.6482e-01, -1.3155e+00, -9.6549e-01,<br>
-1.2189e+00, -1.3041e+00, -1.3487e+00, -1.3384e+00,  1.4074e+00,<br>
4.1653e-01, -1.2966e+00, -1.3047e+00, -1.2857e+00]),</p>
<p>my label look like this:</p>
<p>tensor([46810.2383]))</p>
<p>I want to use the three data tot predict  the lable. But when I traning my model, the loss is always very big. And I plot the prediction predicted by model  and the real label, it looks like this:</p>
<p>test</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/b/ab6e27802a0dffc9a8295a8692c7907c9317c576.png" data-download-href="https://discuss.pytorch.org/uploads/default/ab6e27802a0dffc9a8295a8692c7907c9317c576" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/b/ab6e27802a0dffc9a8295a8692c7907c9317c576_2_689x310.png" alt="image" data-base62-sha1="osxEq4H0r15dDV5dUAsTvzxEkyW" width="689" height="310" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/b/ab6e27802a0dffc9a8295a8692c7907c9317c576_2_689x310.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/b/ab6e27802a0dffc9a8295a8692c7907c9317c576_2_1033x465.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/b/ab6e27802a0dffc9a8295a8692c7907c9317c576.png 2x" data-dominant-color="D5E3ED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1308×588 73.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Traning Loss: 156495051487.5493, Prediction Loss: 289177297578.6667</p>
<p>Can you give me some advices about training?</p>