Error with LSTM when switching from 0.4 to 1.0 (invalid combination of arguments)

username: AlexisW
cooked: <p>My code was working with 0.4, but later when switching to 1.0, the following bug occurs. Is there any quick fix here?</p>
<p>TypeError: lstm() received an invalid combination of arguments - got (Tensor, Tensor, tuple, list, bool, int, float, bool, int), but expected one of:</p>
<ul>
<li>(Tensor data, Tensor batch_sizes, tuple of Tensors hx, tuple of Tensors params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional)<br>
didn’t match because some of the arguments have invalid types: (Tensor, Tensor, tuple, list, bool, int, float, bool, int)</li>
<li>(Tensor input, tuple of Tensors hx, tuple of Tensors params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional, bool batch_first)<br>
didn’t match because some of the arguments have invalid types: (Tensor, Tensor, tuple, list, bool, int, float, bool, int)</li>
</ul>

username: AlexisW
cooked: <p>Still need some insight on this <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=6" title=":grinning:" class="emoji" alt=":grinning:"></p>

username: Alessio_Rosatelli
cooked: <p>I got the same problem today. Is there any quick fix?</p>

username: ptrblck
cooked: <p>It seems you are passing <code>list</code> for the 4th argument (<code>tuple</code> expected) and an <code>int</code> for the last argument (<code>bool</code> expected).<br>
Could you check these and let me know, if you could fix it?<br>
If not, could you post a (small) executable code snippet so that we could debug easier?</p>

username: karmus89
cooked: <p>Hi <a class="mention" href="/u/alexisw">@AlexisW</a>!</p>
<p>I faced a similar kind of problem. In my case I had to go through my self-implemented initialization function, that essentially assigns random values to several hyperparameters. The key culprit was in value types. I had to transform all instances of <code>numpy</code> variables to original Python types by explicit casts.</p>
<p>Concrete example would be changing</p>
<pre><code class="lang-auto">bidirectional = numpy.random.randint(0,1)
</code></pre>
<p>to</p>
<pre><code class="lang-auto">bidirectional = bool(numpy.random.randint(0,1))
</code></pre>
<p>While this might not be directly the remedy to your situation, I highly recommend pasting the error message and trying to line up the expected parameters with provided parameters. That I can also say already that the first set of expected types has to do with <code>PackedSequence</code> inputs, so I suspect the latter would be the one you should be looking at.</p>
<p>Actually, let me do that for you right now:</p>
<pre><code class="lang-auto">(Tensor input,  tuple of Tensors hx,    tuple of Tensors params, 
(Tensor,        Tensor,                 tuple,                  

bool has_biases,    int num_layers, float dropout,  
list,               bool,           int,            

bool train, bool bidirectional, bool batch_first)
float,      bool,               int)
</code></pre>
<p>Its much more easier to spot where the discrepancies might lie in terms of expected and provided parameter types. While native types should be casted to corresponding types on-the-fly, that <code>has_biases</code> seems not to work. I’d suggest you check out how you’re initializing your model at the moment.</p>
<p>Cheers,<br>
PN</p>

username: Bill_Collins
cooked: <p>I’m getting this error too. I would love to see a solution.</p>
<p>Code snippet (I’m wrapping the model in nn.DataParallel):</p>
<p>class SimpleLSTM(nn.Module):<br>
def <strong>init</strong>(self, vocab_size, emb_dim, pad_idx, hid_dim, lstm_layers):<br>
super(SimpleLSTM, self).<strong>init</strong>()<br>
self.hid_dim = hid_dim  <span class="hashtag">#128</span><br>
self.lstm_layers = lstm_layers  <span class="hashtag">#2</span><br>
self.embeddings = nn.Embedding(vocab_size, emb_dim, padding_idx=pad_idx)<br>
self.lstm = nn.LSTM(emb_dim, hid_dim, lstm_layers)</p>
<pre><code>def forward(self, inputs, targets, input_mask, target_mask):
    #batch size is dynamic
    batch_size = inputs.size()[1]   #inputs = seq_len x batch
    emb_inputs = self.embeddings(inputs)  # seq_len x batch x emb_dim
    lstm_hid_state = (torch.zeros(self.lstm_layers, batch_size, self.hid_dim),
                      torch.zeros(self.lstm_layers, batch_size, self.hid_dim))
    lstm_out, lstm_hid_state = self.lstm(emb_inputs, lstm_hid_state)
    return lstm_out[-1] 
</code></pre>
<p>Error msg (truncated):</p>
<p> in forward(self, inputs, targets, input_mask, target_mask)<br>
45         lstm_hid_state = (torch.zeros(self.lstm_layers, batch_size, self.hid_dim),<br>
46                           torch.zeros(self.lstm_layers, batch_size, self.hid_dim))<br>
—&gt; 47         lstm_out, lstm_hid_state = self.lstm(emb_inputs, lstm_hid_state)<br>
48         print(‘last lstm out’, lstm_out[-1].size())<br>
49         <span class="hashtag">#lstm_out</span>, recovered_lengths = nn.utils.rnn.pad_packed_sequence(lstm_out)</p>
<p>~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py in <strong>call</strong>(self, *input, **kwargs)<br>
487             result = self._slow_forward(*input, **kwargs)<br>
488         else:<br>
–&gt; 489             result = self.forward(*input, **kwargs)<br>
490         for hook in self._forward_hooks.values():<br>
491             hook_result = hook(self, input, result)</p>
<p>~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py in forward(self, input, hx)<br>
177         if batch_sizes is None:<br>
178             result = _impl(input, hx, self._flat_weights, self.bias, self.num_layers,<br>
–&gt; 179                            self.dropout, self.training, self.bidirectional, self.batch_first)<br>
180         else:<br>
181             result = _impl(input, batch_sizes, hx, self._flat_weights, self.bias,</p>
<p>TypeError: lstm() received an invalid combination of arguments - got (Tensor, tuple, list, float, int, int, bool, bool, bool), but expected one of:</p>
<ul>
<li>(Tensor data, Tensor batch_sizes, tuple of Tensors hx, tuple of Tensors params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional)<br>
didn’t match because some of the arguments have invalid types: (Tensor, tuple, list, float, int, int, bool, bool, bool)</li>
<li>(Tensor input, tuple of Tensors hx, tuple of Tensors params, bool has_biases, int num_layers, float dropout, bool train, bool bidirectional, bool batch_first)<br>
didn’t match because some of the arguments have invalid types: (Tensor, tuple, list, float, int, int, bool, bool, bool)</li>
</ul>
<p>Thanks!</p>

username: AlexisW
cooked: <p>This is the correct solution. There has been issues submitted to pytorch a while ago but they did not seem to have a chance to get to that.</p>

username: MarcusNerva
cooked: <p>Thanks, solve my problem too!</p>

username: David_FIM
cooked: <p>Excuse me i got exatlcy same problem but i dont understand what do you mean with this solution can you help me please</p>

username: asking28
cooked: <p>Bool(int) helped in my case too</p>

username: Guodong_Ma
cooked: <p>Thanks, my code works with your solution !</p>