Expected scalar type Long but found Float

astra1234567: <p>Hello! I have problem with my code:</p>
<pre><code class="lang-auto">import numpy as np
import os
import torch
from torch.utils.data import Dataset, DataLoader
import torchvision.models as models
import torchvision as trv
import torch.nn as nn
from sklearn.model_selection import train_test_split
import torch.optim as optim
import argparse


class CustomData(Dataset):
   def __init__(self, path, transform=None):
      try:
         self.path = path
         self.data = np.load(path).astype('float32')
         self.data = self.data.astype(np.int_)
         self.transform = transform
      except:
         print("error")


   def __len__(self):
      return len(self.data)

   def __getitem__(self, idx):
      if torch.is_tensor(idx):
         idx = idx.tolist()

      features = torch.tensor(self.data[idx, 2:])
      label = self.data[idx, 1]
      if self.transform:
         features = self.transform(features)
      return features, label





class CNN(nn.Module):
   def __init__(self, input_size, num_classes):
      super(CNN, self).__init__()
      self.conv1 = nn.Conv1d(1, 10, 1)
      self.conv2 = nn.Conv2d(1, 20, 45)
      self.fc1 = nn.Linear(45 * 20, 50)
      self.fc2 = nn.Linear(50, num_classes)
      self.bn1 = nn.BatchNorm1d(10)
      self.bn2 = nn.BatchNorm1d(20)
      self.bn3 = nn.BatchNorm1d(50)
      self.activation = nn.Sigmoid()

   def forward(self, x):
      x = x.unsqueeze(1)
      x = x.unsqueeze(1)
      x = self.activation(self.bn1(self.conv1(x)))
      x = self.activation(self.bn2(self.conv2(x)))
      x = x.view(-1, x.shape[1] * x.shape[2])
      x = self.activation(self.bn3(self.fc1(x)))
      x = self.fc2(x)

      return x


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

parser = argparse.ArgumentParser(description='PyTorch Feature Extraction')
parser.add_argument('--path', metavar='DIR',
                    help='path to dataset')
parser.add_argument('--j', '--workers', default=4, type=int, metavar='N',
                    help='number of data loading workers (default: 4)')
parser.add_argument('--b', '--batch-size', default=10, type=int,
                    metavar='N', help='mini-batch size (default: 256)')

args = parser.parse_args()

path = "final_relebeled_dataset.npy"
dataset = CustomData(path, transform=None)
dataloader = DataLoader(dataset, batch_size=args.b, shuffle=True, num_workers=args.j, drop_last=True)


# num_classes = 41
num_classes = 24
input_size = 53

# model = Simple_FC(input_size = input_size, num_classes = num_classes).to(device)
model = CNN(input_size=input_size, num_classes=num_classes).to(device)

lr = 0.01
momentum = 0.9
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum)

def train(epoch):
   model.train()
   for (features, labels) in enumerate(dataloader):
      features = torch.tensor([features])
      labels = labels[0]
      print(type(features))
      features = features.type(torch.LongTensor)
      print(type(features))
      features, labels = features.to(device), labels.to(device)
      optimizer.zero_grad()
      print(features)
      output = model(features)
      loss = criterion(output, labels)
      loss.backward()
      optimizer.step()
      print(loss)


if __name__ == '__main__':
   for epoch in range(0, 10):
      train(epoch)
</code></pre>
<p>After I have started my code I see error :" RuntimeError: expected scalar type Long but found Float ". Could you help me solve my problem ?? Thank you !!!</p>

ptrblck: <p>Most likely this error is raised in:</p>
<pre><code class="lang-python">loss = criterion(output, labels)
</code></pre>
<p>if you try to pass <code>labels</code> as a <code>FloatTensor</code>, while a <code>LongTensor</code> is expected.<br>
Try to use:</p>
<pre><code class="lang-python">loss = criterion(output, labels.long())
</code></pre>
<p>instead.</p>

astra1234567: <p>Thanks for your answer , but I have started study Traceback and I have seen that problem is in :</p>
<pre><code class="lang-auto">output = model(features)
</code></pre>
<p>I have tried to use:</p>
<pre><code class="lang-auto">output = model(features.long())
</code></pre>
<p>but I have the same error again.</p>

eqy: <p>Can you post the full traceback if the above fix didn’t work? <code>features.long()</code> seems unusual especially for a CNN model .</p>

astra1234567: <p>It is Traceback:<br>
Traceback (most recent call last):<br>
File “”, line 124, in <br>
train(epoch)<br>
File “”, line 105, in train<br>
output = model(features.long())<br>
File “”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “”, line 56, in forward<br>
x = self.activation(self.bn1(self.conv1(x)))<br>
File “”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “”, line 263, in forward<br>
return self._conv_forward(input, self.weight, self.bias)<br>
File “”, line 259, in _conv_forward<br>
return F.conv1d(input, weight, bias, self.stride,<br>
RuntimeError: expected scalar type Long but found Float</p>

eqy: <p>What is the purpose of</p>
<pre><code class="lang-auto">         self.data = np.load(path).astype('float32')
         self.data = self.data.astype(np.int_)
</code></pre>
<p>in the Dataset?<br>
Can you try removing the second line?<br>
You should also change the model invocation back to <code>output = model(features)</code>.</p>
<p>The error message is confusing because the operator expected weights to be <code>Long</code> for <code>Long</code> input. To fix this you want to change your input to <code>Float</code> and match the weights which are already <code>Float</code> by default.</p>

astra1234567: <p>Thank for you answer but I tried it (what you have written) and it is the same error.</p>

eqy: <p>Can you double check the type of x in the model before the conv layer? e.g., <code>print(x.dtype)</code> should give float.</p>

astra1234567: <p>It has given torch.int64. Should it be float ??</p>

eqy: <p>Yes <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"> it should be float here</p>

astra1234567: <p>Thank you ! I have float now. I have another question. Now I have error :</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "", line 121, in &lt;module&gt;
    train(epoch)
  File "", line 102, in train
    output = model(features)
  File "", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "y", line 56, in forward
    x = self.activation(self.bn1(self.conv1(x)))
  File "", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "", line 135, in forward
    return F.batch_norm(
  File "", line 2147, in batch_norm
    _verify_batch_size(input.size())
  File "", line 2114, in _verify_batch_size
    raise ValueError("Expected more than 1 value per channel when training, got input size {}".format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 10, 1])

</code></pre>
<p>What does it mean ?? Should I change sth in :</p>
<pre><code class="lang-auto">self.conv1 = nn.Conv1d(1, 10, 1)
</code></pre>

eqy: <p>Something seems strange about the output shape after a conv layer then. Can you separate the</p>
<pre><code class="lang-auto">x = self.activation(self.bn1(self.conv1(x)))
</code></pre>
<p>to</p>
<pre><code class="lang-auto">x = self.conv1(x)
x = self.bn1(x)
...
</code></pre>
<p>and print the shape of <code>x</code> in between to debug? It looks like the shape of <code>x</code> is smaller than expected for the batchnorm.</p>

astra1234567: <p>I have done it and problem is in :</p>
<pre><code class="lang-auto">x = self.bn1(x)
</code></pre>
<p>So should I change value in bn1 in <strong>init</strong>??</p>

eqy: <p>If you know what the expected shape and what change should be made to bn1, then sure.</p>

astra1234567: <p>I know that x.shape is torch.Size([1, 10, 1]), so value in BatchNorm1d should be<br>
10, but I still have the same error <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"></p>

ptrblck: <p>The batchnorm layer would need more than a single value to calculate the stats (<code>mean</code> and <code>var</code>) from the input batch, so you would either need to use more than a single sample (increase the batch size) during training or increase the temporal dimension:</p>
<pre><code class="lang-python">bn = nn.BatchNorm1d(10)
x = torch.randn(1, 10, 1)
out = bn(x)
&gt; ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 10, 1])

x = torch.randn(2, 10, 1)
out = bn(x) # works

x = torch.randn(1, 10, 2)
out = bn(x) # works
</code></pre>

astra1234567: <p>What if my input size is 1 and kernel size must be 1 too ? Can I do sth with this, without change my data ??</p>

ptrblck: <p>I’m unsure how the kernel size is related to the input shape of the batchnorm layer.<br>
However, if the input contains only a single value for each channel (as is the case here), you won’t be able to use batchnorm layers in training mode, since they need to calculate the stats from the input.<br>
Since it’s impossible to calculate the <code>var</code> from a single sample (it would create <code>NaNs</code>) and also subtracting the <code>mean</code> from a single value would create a zero output, you could remove these layers from the model.</p>