Expected hidden[0] size (2, 1, 1024), got [2, 100, 1024] runtime error in Seq2Seq model

Atefeh_Rashidi: <p>I want to implement a seq2seq model which is learning to generate text (source and target sequences are the same). Some parts of my code are shown below:</p>
<p>hyperparameters:</p>
<pre><code class="lang-auto">#Training hyperparameters
num_epochs = 1
learning_rate = 0.001
batch_size = 64

#Model hyperparameters
load_model = False
save_model = False
save_best_model = True
vocab_size = voc.count #===&gt; input_size
output_size = voc.count
encoder_embedding_size = 300
decoder_embedding_size = 300
hidden_size = 1024
num_layers = 2
enc_dropout_p = 0.5
dec_dropout_p = 0.5
teacher_force_ratio = 0.5
max_length = 50

</code></pre>
<p>Encoder, Decoder, and Seq2Seq models:</p>
<pre><code class="lang-auto">class Encoder(nn.Module):
  def __init__(self, input_size, embedding_size, hidden_size, num_layers, dropout_p):
    super(Encoder, self).__init__()
    self.hidden_size = hidden_size #LSTM output size
    self.num_layers = num_layers

    self.dropout = nn.Dropout(dropout_p)
    self.embedding = nn.Embedding(input_size, embedding_size)
    self.rnn = nn.LSTM(embedding_size, hidden_size, num_layers, dropout = dropout_p) 
   
  
  def forward(self, x):
    embedding = self.dropout(self.embedding(x))
    output, (hidden, cell) = self.rnn(embedding) 
    return hidden, cell


class Decoder(nn.Module):
  def __init__(self, input_size, embedding_size, hidden_size, output_size, num_layers, dropout_p): 
    super(Decoder, self).__init__()
    self.hidden_size = hidden_size #LSTM output size
    self.num_layers = num_layers

    self.dropout = nn.Dropout(dropout_p)
    self.embedding = nn.Embedding(input_size, embedding_size)
    self.rnn = nn.LSTM(embedding_size, hidden_size, num_layers, dropout = dropout_p)
    self.fc = nn.Linear(hidden_size, output_size) 



  def forward(self, x, hidden, cell): #x : (batch_size)
    
    x = x.unsqueeze(0)
   
    embedding = self.dropout(self.embedding(x)) 
    
    outputs, (hidden, cell) = self.rnn(embedding, (hidden, cell)) 
    
    predictions = self.fc(outputs)
    predictions = predictions.squeeze(0) #we predict one word at each time

    return predictions, hidden, cell


class Seq2Seq(nn.Module):
  def __init__(self, encoder,decoder):
    super(Seq2Seq, self).__init__()

    self.encoder = encoder
    self.decoder = decoder

  def forward(self, source, target, target_vocab_size, teacher_force_ratio = 0.5):
    target_len = target.shape[0]
    batch_size = source.shape[1]
    outputs = torch.zeros(target_len, batch_size, target_vocab_size).to(DEVICE)
    hidden, cell = self.encoder(source)
    x = target[0]
    for t in range(1, target_len):
      output, hidden, cell = self.decoder(x, hidden, cell)
      outputs[t] = output
      best_guess = output.argmax(1) 
      x = target[t] if random.random() &lt; teacher_force_ratio else best_guess
    return outputs
</code></pre>
<p>Declaration of model:</p>
<pre><code class="lang-auto">encoder_net = Encoder(vocab_size, encoder_embedding_size, hidden_size, num_layers, enc_dropout_p).to(DEVICE)
decoder_net = Decoder(vocab_size, decoder_embedding_size, hidden_size, output_size, num_layers, dec_dropout_p).to(DEVICE)
model = Seq2Seq(encoder_net, decoder_net).to(DEVICE)
criterion = nn.NLLLoss() 
optimizer = optim.SGD(model.parameters(), lr = learning_rate) 
</code></pre>
<p>Dataloader:</p>
<pre><code class="lang-auto">train_iterator = torch.utils.data.DataLoader(train_ds, batch_size=64, drop_last=True) 
test_iterator = torch.utils.data.DataLoader(test_ds, batch_size=64, drop_last=True) 
</code></pre>
<p>Train model:</p>
<pre><code class="lang-auto">print('Training starts...')
best_loss = float('inf')

for epoch in range(1 , 1 + num_epochs):
  print(f'Epoch {epoch} / {num_epochs}')

  sum_loss = 0
  total = len(train_iterator)

  if save_model:
    checkpoint = {'state_dic' : model.state_dict(), 'optimizer' : optimizer.state_dict()}
    save_checkpoint(checkpoint)
  
  model.train()
  for i, (train_inputs, train_labels) in tqdm(enumerate(train_iterator), desc='Training ', total=len(train_iterator)):
    
    input = train_inputs.to(DEVICE)
    target = input

    output = model(input, target, vocab_size, teacher_force_ratio)

    output = output[1:].reshape(-1, output.shape[2])
    target = target[1:].reshape(-1)

    optimizer.zero_grad()

  # #calculate the loss from a predicted sentence with the expected result
  #  for ot in range(num_iter):
  #      loss += criterion(output[ot], target_tensor[ot])

    loss = criterion(output, target)
    loss.backward()#(retain_graph=True)
    loss = loss.data.item()

    torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1)
    optimizer.step()

    writer.add_scalar('Training loss', loss, global_step = step) 
    step += 1
    
    # loss= loss.data.item()
    sum_loss += loss
    # print("loss = ", loss)
    

  epoch_loss = sum_loss/total
  print(f"LOSS: {epoch_loss:.5f}\n")
</code></pre>
<p>Test model:</p>
<pre><code class="lang-auto">def translate(model):
  model.eval()
  translated_sentences = []
  for i, (test_inputs, test_labels) in tqdm(enumerate(test_iterator), desc='Test ', total=len(test_iterator)):
    input = test_inputs.to(DEVICE)
    target = input

    # Build encoder hidden, cell state
    with torch.no_grad():
          hidden, cell = model.encoder(input)
          print(hidden.shape)


    outputs = [voc.word2index[SOS]]
    output = model(input, target, vocab_size, teacher_force_ratio)


    for _ in range(max_length):
      previous_word = torch.LongTensor([outputs[-1]]).to(DEVICE)
      with torch.no_grad():
        output, hidden, cell = model.decoder(previous_word, hidden, cell)
        best_guess = output.argmax(1).item()

      outputs.append(best_guess)

      if output.argmax(1).item() == voc.word2index[EOS]:
          break

    translated_sentence = [voc.index2word[idx] for idx in outputs]
    # remove start token
    translated_sentences.append(translated_sentence[1:])
    
    return translated_sentences

translate(model)
</code></pre>
<p>but a runtime error occurs during the translate method:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)

&lt;ipython-input-14-dbe83c5e4640&gt; in &lt;module&gt;()
----&gt; 1 translate(model)

6 frames

&lt;ipython-input-13-cfd4f97b53f5&gt; in translate(model)
     24       previous_word = torch.LongTensor([outputs[-1]]).to(DEVICE)
     25       with torch.no_grad():
---&gt; 26         output, hidden, cell = model.decoder(previous_word, hidden, cell)
     27         best_guess = output.argmax(1).item()
     28 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

&lt;ipython-input-9-8c3f838beb9d&gt; in forward(self, x, hidden, cell)
     38     embedding = self.dropout(self.embedding(x)) 
     39 
---&gt; 40     outputs, (hidden, cell) = self.rnn(embedding, (hidden, cell)) 
     42     predictions = self.fc(outputs) 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-&gt; 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/rnn.py in forward(self, input, hx)
    687             hx = self.permute_hidden(hx, sorted_indices)
    688 
--&gt; 689         self.check_forward_args(input, hx, batch_sizes)
    690         if batch_sizes is None:
    691             result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/rnn.py in check_forward_args(self, input, hidden, batch_sizes)
    632         self.check_input(input, batch_sizes)
    633         self.check_hidden_size(hidden[0], self.get_expected_hidden_size(input, batch_sizes),
--&gt; 634                                'Expected hidden[0] size {}, got {}')
    635         self.check_hidden_size(hidden[1], self.get_expected_cell_size(input, batch_sizes),
    636                                'Expected hidden[1] size {}, got {}')

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg)
    224                           msg: str = 'Expected hidden size {}, got {}') -&gt; None:
    225         if hx.size() != expected_hidden_size:
--&gt; 226             raise RuntimeError(msg.format(expected_hidden_size, list(hx.size())))
    227 
    228     def check_forward_args(self, input: Tensor, hidden: Tensor, batch_sizes: Optional[Tensor]):

RuntimeError: Expected hidden[0] size (2, 1, 1024), got [2, 100, 1024]
</code></pre>
<p>What’s the problem? did I do sth wrong in my codes and methods? How can I fix this error?</p>

ptrblck: <p>The error seems to be raised in the <code>nn.LSTM</code> layer.<br>
The <a href="https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html#torch.nn.LSTM">docs</a> describe the expected inputs as:</p>
<blockquote>
<p><strong>input</strong>: tensor of shape <code>(L,Hin)</code> for unbatched input, <code>(L,N,Hin)</code> when <code>batch_first=False</code> or <code>(N,L,Hin)</code> when <code>batch_first=True</code> containing the features of the input sequence…</p>
</blockquote>
<blockquote>
<p><strong>h_0</strong>: tensor of shape <code>(D∗num_layers,Hout)</code> for unbatched input or <code>(D∗num_layers,N,Hout)</code> containing the initial hidden state for each element in the input sequence…</p>
</blockquote>
<blockquote>
<p><strong>c_0</strong>: tensor of shape <code>(D∗num_layers,Hcell)</code> for unbatched input or <code>(D∗num_layers,N,Hcell)</code> containing the initial cell state for each element in the input sequence.</p>
</blockquote>
<p>Based on your error message it seems the batch size doesn’t match between the input and hidden state of the <code>nn.LSTM</code> layer.</p>

aritrasen: <p>Have you got any solution to this ? I am getting the same error.</p>

NancyShehata: <p>i get same error as mentioned above, and I read that you get the same error, could you solve it? if yes then can you please tell me how?</p>

Atefeh_Rashidi: <p>No, I have not solved it yet!</p>

vdw: <p><code>outputs</code> will contain the last hidden states (“last” with respect to number of layers) for all time steps, see here:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/4/748ce2c7f184e70e069326543de260902b9a4e76.png" data-download-href="https://discuss.pytorch.org/uploads/default/748ce2c7f184e70e069326543de260902b9a4e76" title="lstm-output-vs-hidden"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/4/748ce2c7f184e70e069326543de260902b9a4e76.png" alt="lstm-output-vs-hidden" data-base62-sha1="gD3escl8na3s5EFycJH7t67PFJk" width="583" height="500" data-dominant-color="EDEFF2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">lstm-output-vs-hidden</span><span class="informations">640×548 21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I assume that your sequence length is 100, so you would see the shape <code>(2, 100, 1024)</code> in the error message</p>
<p>I’m pretty sure you only want the last hidden state (“last” w.r.t. to the number of layers) of the last layer. In your encoder code this would the <code>hidden[-1]</code> (this does no longer work with <code>bidirectional=True</code>!). So the line should be:</p>
<pre><code>predictions = self.fc(hidden[-1])
</code></pre>