GRU expects another hidden dimension after sometime

shwe87: <p>Hi,</p>
<p>After some time in the training phase, I get the following error:</p>
<p>My batch_size = 32</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-22-3725f29f93d4&gt; in &lt;module&gt;
     37 
     38         # get the output from the model
---&gt; 39         output, h = mynet(specs, h)
     40         output = F.log_softmax(output, dim=2)
     41         output = output.transpose(0,1)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--&gt; 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

&lt;ipython-input-17-9a3df9ad77fe&gt; in forward(self, x, hidden)
     45         x = x.transpose(1,2)
     46         #print("After View &amp; Transpose : {}".format(x.size()))
---&gt; 47         out, hidden = self.gru(x, hidden)
     48         #print("After GRU : {}".format(x.size()))
     49         #out = out.contiguous().view(-1, self.hidden_dim)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--&gt; 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py in forward(self, input, hx)
    722             hx = self.permute_hidden(hx, sorted_indices)
    723 
--&gt; 724         self.check_forward_args(input, hx, batch_sizes)
    725         if batch_sizes is None:
    726             result = _VF.gru(input, hx, self._flat_weights, self.bias, self.num_layers,

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py in check_forward_args(self, input, hidden, batch_sizes)
    192         expected_hidden_size = self.get_expected_hidden_size(input, batch_sizes)
    193 
--&gt; 194         self.check_hidden_size(hidden, expected_hidden_size)
    195 
    196     def permute_hidden(self, hx, permutation):

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg)
    185         # type: (Tensor, Tuple[int, int, int], str) -&gt; None
    186         if hx.size() != expected_hidden_size:
--&gt; 187             raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size())))
    188 
    189     def check_forward_args(self, input, hidden, batch_sizes):

RuntimeError: Expected hidden size (5, 12, 128), got (5, 32, 128)
</code></pre>
<p>I guess this is because it is the last batch and isn’t 32 exactly. How can I avoid this?</p>
<p>Thanks.<br>
BR,<br>
Shweta.</p>

googlebot: <p>your ‘hidden’ variable used in self.gru(x, hidden) is not created appropriately, I’d guess that 32 is hardcoded somewhere; if you use zeros as initial state, simplest fix is to actually omit second parameter</p>

shwe87: <p>No, actually I figured it out.</p>
<p>my batch size = 32.<br>
The last batch wasn’t 32 and was 12, that’s why. I just pass when it an incomplete batch and it worked <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>