Input format for LSTM in consideration to rolling window

username: Xiar
cooked: <p>Hello,</p>
<p>I am a beginner and working on a time series prediction for stock prices. However, using the rolling window technique I am fetching 20 days of the stock price and inserting the values into a tensor and labelling the 21 day as the day I want to predict. Currently I am only using the closing price of the stock, hence it is univariate. The input tensor that is passed to the forward method looks like the following:</p>
<p>tensor([[[0.6675],<br>
[0.7126],<br>
[0.7064],<br>
[0.7311],<br>
[0.7539],<br>
[0.6859],<br>
[0.7271],<br>
[0.7601],<br>
[0.7252],<br>
[0.7723],<br>
[0.6569],<br>
[0.6262],<br>
[0.7044],<br>
[0.7456],<br>
[0.7087],<br>
[0.7126],<br>
[0.6510],<br>
[0.6694],<br>
[0.6859],<br>
[0.7417]]])</p>
<p>My questions are:</p>
<ol>
<li>
<p>The output is in the size torch.Size([20, 1]) after the linear layer and to extract the correct value I am taking the last value of the tensor of the linear layer, t = t[-1]. Is this the correct way to do it?</p>
</li>
<li>
<p>The documentation writes that the input tensor should be: (seq_len, batch, input_size). I am wondering if my rolling window tensor correct? What confuses me is that I assume the sequence length should be 20? But by printing my tensor shape I obtain: torch.Size([1, 20, 1]). Should I reshape the input into  torch.Size([20, 1, 1])?</p>
</li>
<li>
<p>In regards to the first question is there a way to implement the model so I do not have to extract the last value according to t = t[-1]? In other words can the linear layer directly give the last value, the 21st prediction or should I keep it as it is?</p>
</li>
</ol>
<p>I would really appreciate it if someone could point me into the right direction or drop a link to a good source that explains the questions I have. Below is the model:</p>
<pre><code class="lang-python">class LSTM(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, output_size,batch_size):
        super(LSTM, self).__init__()
        self.input_size = input_size  #1
        self.hidden_size = hidden_size # 32
        self.num_layers = num_layers # 2
        self.output_size = output_size # 1
        self.batch_size = batch_size # 1
        self.lstm = nn.LSTM(self.input_size, self.hidden_size, self.num_layers)
        self.linear = nn.Linear(hidden_size, output_size)
        
    def hidden_cell(self,t):
        h_0,c_0 = (torch.zeros(self.num_layers, t.shape[1], self.hidden_size), 
        torch.zeros(self.num_layers, t.shape[1], self.hidden_size))
        return h_0,c_0 
        
    def forward(self, t):
        t, (h_n,c_n) = self.lstm(t, (self.hidden_cell(t)))
        t = t.view(-1, self.hidden_size)
        t = self.linear(t)
        t = t[-1]
        return t
</code></pre>
<p>Thank you.</p>

username: vdw
cooked: <p>The problem that I currently see is that you use the output of the <code>self.lstm</code> for the next layer. Let me use different names to avoid confusion:</p>
<pre><code> output, (h_n,c_n) = self.lstm(t, (self.hidden_cell(t)))
</code></pre>
<p>Now <code>output</code> should has the shape <code>(20, 1, 32)</code>; note that <code>output</code> gives you the hidden states for all 20 steps. The problem is that after:</p>
<pre><code>t = output.view(-1, self.hidden_size)
</code></pre>
<p><code>t</code> has the shape <code>(20, 32)</code>. For the linear layer that means that you give it a batch of size 20. The linear layer doesn’t now or care that the 20 comes from the number of steps. It’s a semantic error. Hence your output shape is <code>(20, 1)</code>. But what you want is <code>(1, 1)</code> since your batch size and output size is 1.</p>
<p>You have to use only the last layer of last hidden state <code>h_n[-1]</code> instead of <code>output</code>. <code>h_n</code> has the shape <code>(2, 1, 32)</code> since you have 2 layers. This <code>h_n[-1]</code> has the shape <code>(1, 32)</code>. And then you also don’t need the <code>view()</code> no more. You can do directly</p>
<pre><code>t = self.linear(h_n[-1])
</code></pre>
<p>Which will give you the right final output with a shape of <code>(1, 1)</code> or <code>(batch_size, 1)</code> once you have batches with more then one sequence.</p>

username: Xiar
cooked: <p>Thank you, your advice helped me so much!</p>