RuntimeError: expected scalar type Long but found Float

username: maamli
cooked: <p>Printing type shows it is a LongTensor still the error prevails. It is a character based model that i am building for the popular ‘dinosaur names’ problem. here is the code:</p>
<pre><code class="lang-auto">class din_model(nn.Module):
    def __init__(self, max_len):
        super(din_model, self).__init__()
        self.rnn = nn.RNN(1, 64)
        self.dense = nn.Linear(64,27)
    
    def forward(self, input_):
        reshape_ = input_.view((input_.size(0),input_.size(1),1))
        print(reshape_.type(), reshape_.size())
        rnn_, _ = self.rnn(reshape_)
        dense_ = self.dense(rnn_)
        return F.log_softmax(dense_, dim=1)
    
model = din_model(27)
optimizer = optim.Adam(model.parameters(), lr=0.01)
loss_fn = nn.NLLLoss()

for epoch in range(3): 
    for t,l in train_dataloader:
        output_batch = model(torch.tensor(t, dtype=torch.long))
        loss = loss_fn(output_batch, l)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    print("Epoch is {}, loss is {}".format(epoch, loss.data))
</code></pre>
<p>the error is here:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-54-1717991d3fdc&gt; in &lt;module&gt;
      1 for epoch in range(3):
      2     for t,l in train_dataloader:
----&gt; 3         output_batch = model(torch.tensor(t, dtype=torch.long))
      4         loss = loss_fn(output_batch, l)
      5         # clear previous gradients, compute gradients of all variables wrt loss

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--&gt; 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

&lt;ipython-input-53-e8e162814236&gt; in forward(self, input_)
      8         reshape_ = input_.view((input_.size(0),input_.size(1),1))
      9         print(reshape_.type(), reshape_.size())
---&gt; 10         rnn_, _ = self.rnn(reshape_)
     11         dense_ = self.dense(rnn_)
     12         return F.log_softmax(dense_, dim=1)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--&gt; 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py in forward(self, input, hx)
    233         if batch_sizes is None:
    234             result = _impl(input, hx, self._flat_weights, self.bias, self.num_layers,
--&gt; 235                            self.dropout, self.training, self.bidirectional, self.batch_first)
    236         else:
    237             result = _impl(input, batch_sizes, hx, self._flat_weights, self.bias,

RuntimeError: expected scalar type Long but found Float
</code></pre>
<p>the result for <code>print(reshape_.type(), reshape_.size())</code> is <code>torch.LongTensor torch.Size([32, 27, 1]) </code></p>
<p>please if anyone can help me resolve it, that would be great.</p>

username: albanD
cooked: <p>Hi,</p>
<p>RNNs don’t support long tensors as input AFAIK. This should be a regular float Tensor <code>torch.tensor(t, dtype=torch.float)</code>.</p>

username: maamli
cooked: <p>this saved me so much time. thanks a ton. Why would they throw an exact opposite error message is beyond me.</p>

username: albanD
cooked: <p>Because it depends in which order you do the op. As the high level rnn does not check that directly but its constituent functions do.<br>
And in these functions, the first arg is usually the input given by the user. And we expect everything to be of the same time as it. Hence the error you see.</p>

username: maamli
cooked: <p>Oh i see. thanks for the explanation around this. I am assuming you are referring to <code>_impl</code> function where weights and bias are coming in as float? if thats the case, how are LSTMs any different?</p>

username: albanD
cooked: <p>All these functions have custom c++ implementation that are not really normalized (mostly because some of them need to be dispatched to special backend like mkl/cudnn and other are just implemented by us). So maybe the order of the arguments is different for them.<br>
Or also there are some type checks that happen at a different moment.</p>
<p>Not sure really <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

username: Mohamed_Desouky
cooked: <p>I think the message must be :</p>
<p><code>RuntimeError: expected scalar type Float but found Long </code> <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=10" title=":thinking:" class="emoji" alt=":thinking:"></p>

username: albanD
cooked: <p>Well it depends which argument goes where haha<br>
If you do <code>a + b</code> or <code>b + a</code> you will get flipped messages. These messages always assume that the first argument has the “correct” type and the second one is wrong.</p>