Time series prediction (many to many lstm) basic example need help!

username: nickzsh
cooked: <p>Hello, I am new to pytorch and have some questions regarding how to create a many-to-many lstm model. I am trying to predict the next number (x_t+1) in a sequence given an input sequence of integers like<br>
[0, 1, 2, 3, 2, 1] - data from x_t-5 to x_t.</p>
<p>So example [0, 1, 2, 3, 2, 1] -&gt; lstm model -&gt; should give [1, 2, 3, 2, 1, 0]</p>
<p>So when I run the model with input sequence from x_t-5 to x_t, i expect the model to train and give an output sequence from x_t-4 to x_t+1.</p>
<p>Can anybody give me an example or fix my code? As i couldn’t find anywhere online with a simple example for such model that i am creating.</p>
<p>Or tell me what is wrong with my code? or my understanding of pytorch lstm?</p>
<p>May I also ask if what exactly should be the hidden_size for my model?</p>
<p>Below is my source code which does not run.</p>
<p>Any help will be greatly appreciated!</p>
<pre><code class="lang-auto">import torch
from torch.autograd import Variable
import torch.nn as nn

torch.manual_seed(1)

x_train = [ [[0], [1], [2], [3], [2], [1]],
             [[1], [2], [3], [2], [1], [0]],
             [[2], [3], [2], [1], [0], [1]],
             [[3], [2], [1], [0], [1], [2]],
             [[2], [1], [0], [1], [2], [3]] ]

y_train = [[[1], [2], [3], [2], [1], [0]],
             [[2], [3], [2], [1], [0], [1]],
             [[3], [2], [1], [0], [1], [2]],
             [[2], [1], [0], [1], [2], [3]],
             [[1], [0], [1], [2], [3], [2]]]

inputs = Variable(torch.Tensor(x_train))
labels = Variable(torch.Tensor(y_train))

input_size =  1
hidden_size = 32
batch_size = 5
sequence_length = 6
num_layers = 1

class LSTM(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, sequence_length):
        super(LSTM, self).__init__()

        self.num_layers = num_layers
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.sequence_length = sequence_length

        self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size, batch_first=True)

    def forward(self, x):
        # (num_layers * num_directions, batch, hidden_size)
        h_0 = (Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size)) for _ in range(self.hidden_size))
        # Reshape input
        x.view(x.size(0), self.sequence_length, self.input_size)

        out, _ = self.lstm(x, h_0)

        return out

lstm = LSTM(input_size, hidden_size, num_layers, sequence_length)
print (lstm)

criterion = torch.nn.MSELoss()
optimizer = torch.optim.Adam(lstm.parameters(), lr=0.1)

# Train the model
for epoch in range(100):
    loss = 0
    outputs = lstm(inputs)
    optimizer.zero_grad()
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()
    print ("epoch : %d, loss: %1.3f" %(epoch+1, loss.data[0]))

print ("Learning finished!")
</code></pre>

username: bsalanon
cooked: <aside class="quote no-group" data-username="nickzsh" data-post="1" data-topic="16314">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/e47774/48.png" class="avatar"> nickzsh:</div>
<blockquote>
<p>x_train = [ [[0], [1], [2], [3], [2], [1]], [[1], [2], [3], [2], [1], [0]], [[2], [3], [2], [1], [0], [1]], [[3], [2], [1], [0], [1], [2]], [[2], [1], [0], [1], [2], [3]] ]</p>
</blockquote>
</aside>
<p>I believe your x_train should look more like<br>
x_train = ( [[[0]], [[1]], [[2]], [[3]], [[2]], [[1]]],<br>
[[[1]], [[2]], [[3]], [[2]], [[1]], [[0]]],<br>
[[[2]], [[3]], [[2]], [[1]], [[0]], [[1]]],<br>
[[[3]], [[2]], [[1]], [[0]], [[1]], [[2]]],<br>
[[[2]], [[1]], [[0]], [[1]], [[2]], [[3]]] )</p>

username: pahome
cooked: <p><a class="mention" href="/u/bsalanon">@bsalanon</a><br>
You will get error message:<br>
RuntimeError: input must have 3 dimensions, got 4</p>