Issues with LSTM output

9blok0: <p>I was trying to implement this neural network in pytorch, but ran into some difficulties with an output of lstm layer.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/d/dd8f419a18918f4967113245b33788f81d5ff2f8.png" data-download-href="https://discuss.pytorch.org/uploads/default/dd8f419a18918f4967113245b33788f81d5ff2f8" title="Plot-of-the-Caption-Generation-Deep-Learning-Model"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/d/dd8f419a18918f4967113245b33788f81d5ff2f8_2_425x375.png" alt="Plot-of-the-Caption-Generation-Deep-Learning-Model" data-base62-sha1="vC0rEKG4bRncPCDlh4Pse9TN7U4" width="425" height="375" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/d/dd8f419a18918f4967113245b33788f81d5ff2f8_2_425x375.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/d/dd8f419a18918f4967113245b33788f81d5ff2f8_2_637x562.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/d/dd8f419a18918f4967113245b33788f81d5ff2f8.png 2x" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Plot-of-the-Caption-Generation-Deep-Learning-Model</span><span class="informations">837×737 23.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The one thing i don’t understand why lstm layer on the model produced output in different shape from the input. There’s my implementation of this model. But the shape or rather dimension of the output stays the same as the input.</p>
<pre data-code-wrap="python"><code class="lang-python">class Net(nn.Module):
    def __init__(self):
        super().__init__()
        #LSTM
        self.seq_embedding = nn.Embedding(51, 256)
        self.seq_dropout = nn.Dropout(0.5)
        self.seq_lstm = nn.LSTM(256, 256)
        #Feature Extraction 
        self.feat_dropout = nn.Dropout(0.5)
        self.feat_fc = nn.Linear(4096, 256)
        #Decoder
        self.fc = nn.Linear(256, 256)
        self.fc2 = nn.Linear(256, 29045)
        
    def forward(self, X1, X2):
        out1 = self.seq_embedding(X1)
        out1 = self.seq_dropout(out1)
        out1, hidden = self.seq_lstm(out1)

        out2 = self.feat_dropout(X2)
        out2 = F.relu(self.feat_fc(out2))

        out = torch.cat((out1, out2), 1)
        out = self.fc(out)
        out = self.fc2(out)
        return F.softmax(out, dim=1)
</code></pre>
<p>my lstm dimmensions</p>
<pre><code class="lang-auto">Input: torch.Size([3656, 51, 256])
Output: torch.Size([3656, 51, 256])
Desired Output: torch.Size([3656, 256])
</code></pre>
<p>Is there any way to achieve the output from original model?</p>

googlebot: <p>if 51 is time dimension, you need to use nn.LSTM(256, 256, batch_first=True)</p>
<p>first output of lstm module contains all time steps, for tasks like yours you can discard non-final outputs by slicing (out1 = out1[:,50]), or use one of tensors from second output (tuple named ‘hidden’ in your code), as these contain non “unrolled” final states too. In your simple case (one rnn layer, same sequence lengths) these methods are equivalent.</p>

9blok0: <p>thank you, that was very helpful</p>