Expected object of type torch.cuda.FloatTensor but found type torch.cuda.DoubleTensor for argument #4 'mat1'

arslansadiq: <p>Hi all,<br>
I am developing a regression model and having some problems in data types’ alignments.<br>
I read the data  from the csv file and convert it to float64 using <code>sklearn.preprocessing.StandardScaler()</code> which I later convert to torch tensor. After that I save to into <code>training.pt</code> and <code>test.pt</code> format.</p>
<p><strong>Problem:</strong><br>
Now the problem is that, it is giving error in the first fully connected layer of the network.<br>
The error is the following:</p>
<pre><code class="lang-auto">Exception ignored in: &lt;bound method _DataLoaderIter.__del__ of &lt;torch.utils.data.dataloader._DataLoaderIter object at 0x7fefaf76cd30&gt;&gt;
Traceback (most recent call last):
  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 399, in __del__
    self._shutdown_workers()
  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 378, in _shutdown_workers
    self.worker_result_queue.get()
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/queues.py", line 337, in get
    return _ForkingPickler.loads(res)
  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 151, in rebuild_storage_fd
    fd = df.detach()
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 494, in Client
    deliver_challenge(c, authkey)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge
    response = connection.recv_bytes(256)        # reject large message
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
Traceback (most recent call last):


  File "&lt;ipython-input-1-0cb739d0d1fb&gt;", line 1, in &lt;module&gt;
    runfile('/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/train.py', wdir='/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model')

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 668, in runfile
    execfile(filename, namespace)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 108, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/train.py", line 70, in &lt;module&gt;
    main(config, args.resume)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/train.py", line 44, in main
    trainer.train()

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/base/base_trainer.py", line 79, in train
    result = self._train_epoch(epoch)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/trainer/trainer.py", line 54, in _train_epoch
    output = self.model(data)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/model/model.py", line 31, in forward
    x = self.Fc1(x)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1024, in linear
    return torch.addmm(bias, input, weight.t())

RuntimeError: Expected object of type torch.cuda.FloatTensor but found type torch.cuda.DoubleTensor for argument #4 'mat1'
</code></pre>
<p>The thing I am confused about is that, Pytorch model’s default data type is float then why it is giving this error?<br>
and what does it mean by <code>argument #4 'mat1'</code> which is there in the error.</p>
<p>I also  tried converting my data to double like this</p>
<pre><code class="lang-auto">train_in_data.double()
train_out_data.double
test_in_data.double()
test_out_data.double()
training_set = (train_in_data, train_out_data)
test_set = (test_in_data, test_out_data)
</code></pre>
<p>but it doesn’t</p>

JuanFMontesinos: <p>It means you are allocating the model in a gpu but not the input</p>

arslansadiq: <p>It is also being allocated on the GPU because the argument due to which I am getting error had <code>torch.cuda</code> type</p>

arslansadiq: <p>Although this doesn’t seem to be the issue but just in case I also did that,</p>
<pre><code class="lang-auto">if training:
            self.dataset.train_labels.to('cuda')
            self.dataset.train_data.to('cuda')
if not  training:
            self.dataset.test_data.to('cuda')
            self.dataset.test_labels.to('cuda')
</code></pre>
<p>it doesn’t work then too</p>

ptrblck: <p>It looks like your data is of type <code>torch.cuda.DoubleTensor</code> while the model parameters are <code>torch.cuda.FloatTensors</code>.<br>
I would recommend to cast your data to <code>FloatTensors</code>. If you really need double precision, you might also cast the model to double.</p>

arslansadiq: <p>Hello,<br>
Yeah I converted my model to double via <code>model.double()</code> and it is working alright now.</p>
<p>But just for curiosity, If I want to convert the data to float, how can I do that.<br>
I am working on regression problem.<br>
I am using <code>sklearn.preprocessing.StandardScaler()</code> for reading the data from csv files which converts by default into float64 aka double.<br>
I also tried to convert the data into float by this:</p>
<pre><code class="lang-auto">train_in_data.float()
train_out_data.float()
test_in_data.float()
test_out_data.float()
</code></pre>
<p>but it doesn’t convert.</p>
<p>Can you suggest any other solution?<br>
<code>train_in_data, train_out_data, test_in_data, test_out_data</code> are torch tensors.</p>
<p>Even after running the code which I mentioned above it says<br>
<code>torch.tensor(____, dtype=float64)</code></p>

ptrblck: <p>If you transform your tensors you should assign the result to them. Did you maybe forget it?<br>
This should work:</p>
<pre><code class="lang-python">train_in_data = train_in_data.float()
...
</code></pre>

arslansadiq: <p>Sorry forgot to do that.<br>
I tried it just like <code>self.model.double()</code>.<br>
It’s working now. Thank you for your time.</p>

mikbuch: <p>Hi, I had similar issue, but with the data, not the model.</p>
<p>Casting the data to float like that helped:</p>
<pre><code class="lang-auto">feats = feats.float()
</code></pre>
<p>The peculiar thing is that, although I had the tensor defined like that:</p>
<pre><code class="lang-auto">feats = torch.tensor(feats, dtype=torch.float64)
</code></pre>
<p>Torch still treated this object as a DoubleTensor, so these two ways of defining/casting to float are not equivalent. Pretty unintuitive, I would say</p>

ptrblck: <aside class="quote no-group" data-username="mikbuch" data-post="9" data-topic="32488">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mikbuch/48/67310_2.png" class="avatar"> mikbuch:</div>
<blockquote>
<p>The peculiar thing is that, although I had the tensor defined like that:</p>
<pre><code class="lang-auto">feats = torch.tensor(feats, dtype=torch.float64)
</code></pre>
<p>Torch still treated this object as a DoubleTensor</p>
</blockquote>
</aside>
<p>This is expected since <code>float64</code> refers to <code>double</code> and thus a <code>DoubleTensor</code>. <code>float32</code> represents a <code>float</code> and <code>FloatTensor</code>.</p>