Expected scalar type Double but found Float but everything has dtype=float64

username: pascalm
cooked: <p>Hello,</p>
<p>so I use some python library to generate data. I use python 3.7. Default floating point presicion is 64 bit. So the generated data is returned as numpy arrays with dtype=float64.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 256, in &lt;module&gt;
    train(TrainDL, model, loss_fn, optimizer)
  File "train.py", line 210, in train
    labels_pred = model(samples)
  File "/home/user/venvs/MLGWSC-1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/user/venvs/MLGWSC-1/lib/python3.7/site-packages/torch/nn/modules/container.py", line 139, in forward
    input = module(input)
  File "/home/user/venvs/MLGWSC-1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/user/venvs/MLGWSC-1/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 298, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/user/venvs/MLGWSC-1/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 295, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: expected scalar type Double but found Float
</code></pre>
<p>Now my program is described in more detail in this thread <a href="https://discuss.pytorch.org/t/how-to-make-my-cnn-batch-size-independent/136963" class="inline-onebox">How to make my CNN batch size independent</a><br>
One small difference is that I now wrap my data in pytroch tensors and unsqueeze it:</p>
<pre><code class="lang-auto">class Dataset(torch.utils.data.Dataset):
    def __init__(self, samples, labels):
        self.samples = torch.tensor(samples)
        self.labels = torch.tensor(labels)
    
    def __len__(self):
        return len(self.samples)
    
    def __getitem__(self, i):
        return self.samples[i].unsqueeze(0), self.labels[i].unsqueeze(0)
</code></pre>
<p>Now the data passed to the model i.e. the call <code>labels_pred = model(samples)</code> is all of dtype=float64.</p>
<pre><code class="lang-auto">print(samples.dtype)
print(samples[0].dtype)
print(samples[0][0].dtype)
</code></pre>
<p>all levels of my data is float64.</p>
<p>according to <a href="https://pytorch.org/docs/stable/tensors.html" class="inline-onebox" rel="noopener nofollow ugc">torch.Tensor — PyTorch 1.10.0 documentation</a> float64 should be treated as Double.</p>
<p>I can’t really see where I would have floats. Everything seems to be double precision.</p>

username: Eta_C
cooked: <p>try <code>model = model.double()</code> before forward.</p>

username: pascalm
cooked: <aside class="quote no-group" data-username="pascalm" data-post="1" data-topic="136971">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pascalm/48/76741_2.png" class="avatar"> pascalm:</div>
<blockquote>
<p><code>forward</code></p>
</blockquote>
</aside>
<p>What?</p>
<p>If I use .float() like model(samples.float()) then it works, which adds to my confusion.</p>
<p>I really don’t think I should need to convert anything at all.</p>

username: ptrblck
cooked: <p>PyTorch uses <code>float32</code> by default to initialize the model’s parameters, tensors etc.<br>
As you’ve described the input tensors are <code>float64</code> tensors, which will create a dtype mismatch.<br>
You would thus either have to transform the input data to <code>float32</code> or the model to <code>float64</code>.</p>

username: Eta_C
cooked: <p>Normally, <code>model.double()</code> is not recommended. <code>double</code> model/data means <strong>DOUBLE GRADIDENT AND DOUBLE MEMORY</strong>.</p>
<p>Unless <code>double</code> precision is important to you, the best solution is <code>return self.samples[i].unsqueeze(0).float(), self.labels[i].unsqueeze(0).float()</code>.</p>