Error "start () + length () exceeds dimension size ()" on using pack_padded_sequence output as input for LSTM layer

jagandecapri: <p>I am trying to use pack_padded_sequence function to feed the output of the function into a LSTM layer as following:</p>
<pre><code class="lang-auto">class LSTMClassification(nn.Module):

    def __init__(self, input_dim, hidden_dim, target_size):
        super(LSTMClassification, self).__init__()
        self.hidden_dim = hidden_dim
        self.lstm = nn.LSTM(input_dim, hidden_dim, batch_first=True, num_layers=3, dropout=0.3, bidirectional=True)

        # The linear layer that maps from hidden state space to tag space
        self.fc = nn.Linear(hidden_dim*2, target_size)

    def forward(self, input_):
        input_lengths = torch.LongTensor([torch.max(input_[i, :].data.nonzero()) + 1 for i in range(input_.size()[0])])
        # Then pack the sequences
        packed_input = nn.utils.rnn.pack_padded_sequence(input_, input_lengths.cpu().numpy(), batch_first=True, enforce_sorted=False)
        lstm_out, (h, c) = self.lstm(packed_input)
        logits = self.fc(lstm_out[:,-1])
        return logits
</code></pre>
<p>However, I am getting an error stating “start (384) + length (8) exceeds dimension size (384).”. I couldn’t debug what is the cause of the error. I did a small simulation code to reproduce the error as following. The error can be seen when the following code is ran.</p>
<pre><code class="lang-auto">model = LSTMClassification(76, 
                        hidden_dim=8, 
                        target_size=1)

input_ = torch.randn(8, 48, 76)
model(input_)
</code></pre>
<p>The issue seems similar to <a href="https://discuss.pytorch.org/t/i-used-pack-padded-sequence-and-put-in-lstm-layer-but-i-got-start-length-exceeds-dimension-size-error/41702" class="inline-onebox">I used pack_padded_sequence() and put in lstm layer, but I got start () + length () exceeds dimension size (). error</a> but I don’t see any clear answer on how the problem was solved in that link.</p>

ptrblck: <p>I’m not sure, if I completely understand your use case, but since you are using <code>batch_first=True</code>, the input is expected to have the shape <code>[batch_size, seq_len, features]</code>, so <code>48</code> would be the sequence length? If so, then <code>input_lengths</code> would be wrong since it’s returning a tensor containing <code>76</code> for each sample.  Once this is fixed, you would have to check the <code>lstm_out</code>, as it would also be a <code>PackedSequence</code>, so I guess you might want to access the return value via <code>lstm_out.data</code>.</p>

jagandecapri: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> To give more context, I’m trying to port a model from Keras to Pytorch. Specifically, the model in Keras has a <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Masking?hl=en" rel="noopener nofollow ugc">Masking</a> layer applied.</p>
<p>Referring to this <a href="https://medium.com/huggingface/understanding-emotions-from-keras-to-pytorch-3ccb61d5a983" rel="noopener nofollow ugc">blog</a> - section “How the PackedSequence object works”, what I understood is that the input_lengths refers to the <strong>length of the longest sequence in a batch</strong> where the length is defined as the last non-zero element in the sequence.</p>
<p>Did I understood the concept wrongly?</p>

jagandecapri: <p>I think I got the idea of input_lengths for pack_padded_sequence by following this <a href="https://gist.github.com/jagandecapri/ff12df48d1975c9a2a344161b074a682" rel="noopener nofollow ugc">gist</a>. It is the number of non-zero sequences in a batch. In my case, the number can go up to 48 which is the maximum number of sequences in a batch.</p>