Native CUDA amp input and weight dtype mismatch

PrimeF: <p>Hi all,</p>
<p>when trying to use the native amp functionalities I run in the following error:</p>
<pre><code class="lang-auto">RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.HalfTensor) should be the same
</code></pre>
<p>I use the <code>@autocast()</code> decorator and the <code>GradScaler</code> class.<br>
PyTorch version is: 1.7.0.dev20200716</p>
<p>Thanks in advance for any hints or help!</p>

ptrblck: <p>Could you post a code snippet to reproduce this issue, please? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

PrimeF: <p>Thanks for the crazy fast reply <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Sure, here is a gist with some code (sorry for the many lines):<br>
</p><aside class="onebox githubgist">
  <header class="source">
      <a href="https://gist.github.com/PrimeF/2f49e6ffb5f0be279250907aca886220" target="_blank" rel="nofollow noopener">gist.github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://gist.github.com/PrimeF/2f49e6ffb5f0be279250907aca886220" target="_blank" rel="nofollow noopener">https://gist.github.com/PrimeF/2f49e6ffb5f0be279250907aca886220</a></h4>
<h5>dataset.py</h5>
<pre><code class="Python"># Python standard libraries
import os
import glob

# Installed libraries
import numpy as np
from PIL import Image, ImageCms
import torch
from torch.utils.data import DataLoader
from torch.utils.data.dataset import Dataset</code></pre>
This file has been truncated. <a href="https://gist.github.com/PrimeF/2f49e6ffb5f0be279250907aca886220" target="_blank" rel="nofollow noopener">show original</a>
<h5>model.py</h5>
<pre><code class="Python"># Installed libraries
import torch
import torch.nn as nn
from torch.cuda.amp import autocast

# Custom libraries
from utils import resolve_class


class Model(nn.Module):</code></pre>
This file has been truncated. <a href="https://gist.github.com/PrimeF/2f49e6ffb5f0be279250907aca886220" target="_blank" rel="nofollow noopener">show original</a>
<h5>train.py</h5>
<pre><code class="Python">def train(data_loader_, model_, optimizer, optim_params, loss_fn, epochs, checkpoint_freq, model_dir_, checkpoint=None):
    """
    Trains the model

    :param data_loader_: DataLoader to retrieve the training data
    :param model_: Model to train
    :param optimizer: Optimizer
    :param loss_fn: Loss function
    :param lr: Learning rate
    :param epochs: Number of epochs to train</code></pre>
This file has been truncated. <a href="https://gist.github.com/PrimeF/2f49e6ffb5f0be279250907aca886220" target="_blank" rel="nofollow noopener">show original</a>

<p>
</p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>I can post the entire code if you want, I just thought it might be too much code.</p>

ptrblck: <p>Thanks for the code!<br>
Could you post the complete config and upload the <code>kwargs</code> used in the model?<br>
Also, I assume you are using <code>[batch_size, 3, 224, 224]</code> shaped inputs?</p>

PrimeF: <p>Sure, I updated the gist to include the configs.<br>
The <code>kwargs</code> are listed in the config and are sorted in the same way they are used in the model.</p>
<p>Exactly that is my input’s shape.</p>

ptrblck: <p>Thanks!<br>
Based on the code it seems that you might be creating numpy arrays for your input data, which uses float64 by default as the <code>dtype</code>, before transforming them to tensors.<br>
Could you transform the input tensors for float32 via <code>tensor = tensor.float()</code> before passing them to the model and rerun the code?</p>

PrimeF: <p>Alright, made the changes you suggest, I don’t know if I’m a step forward or still at the same point <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Now I get this error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 342, in &lt;module&gt;
    model = train(data_loader,
  File "train.py", line 172, in train
    scaler.scale(loss).backward()
  File "/sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/tensor.py", line 185, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/autograd/__init__.py", line 125, in backward
    Variable._execution_engine.run_backward(
RuntimeError: Found dtype Float but expected Half
Exception raised from compute_types at /pytorch/aten/src/ATen/native/TensorIterator.cpp:183 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f212400e1e2 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/pyt
hon3.8/site-packages/torch/lib/libc10.so)
frame #1: at::TensorIterator::compute_types(at::TensorIteratorConfig const&amp;) + 0x259 (0x7f21603aa429 in /sapmnt/home/D067751/.local/share/virtualenvs/project-
VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #2: at::TensorIterator::build(at::TensorIteratorConfig&amp;) + 0x6b (0x7f21603adbcb in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/py
thon3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #3: at::TensorIterator::TensorIterator(at::TensorIteratorConfig&amp;) + 0xdd (0x7f21603ae23d in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZK
pb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #4: at::native::mse_loss_backward_out(at::Tensor&amp;, at::Tensor const&amp;, at::Tensor const&amp;, at::Tensor const&amp;, long) + 0x18a (0x7f216020f6fa in /sapmnt/hom
e/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #5: &lt;unknown function&gt; + 0xf11ad0 (0x7f2125388ad0 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/li
b/libtorch_cuda.so)
frame #6: at::native::mse_loss_backward(at::Tensor const&amp;, at::Tensor const&amp;, at::Tensor const&amp;, long) + 0x90 (0x7f216020c240 in /sapmnt/home/D067751/.local/s
hare/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #7: &lt;unknown function&gt; + 0xf11b70 (0x7f2125388b70 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/li
b/libtorch_cuda.so)
frame #8: &lt;unknown function&gt; + 0xf357e6 (0x7f21253ac7e6 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/li
b/libtorch_cuda.so)
frame #9: at::mse_loss_backward(at::Tensor const&amp;, at::Tensor const&amp;, at::Tensor const&amp;, long) + 0xfb (0x7f2160710c5b in /sapmnt/home/D067751/.local/share/vir
tualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #10: &lt;unknown function&gt; + 0x26301c9 (0x7f2161b2f1c9 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/
lib/libtorch_cpu.so)
frame #11: &lt;unknown function&gt; + 0xab4cc6 (0x7f215ffb3cc6 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/l
ib/libtorch_cpu.so)
frame #12: at::mse_loss_backward(at::Tensor const&amp;, at::Tensor const&amp;, at::Tensor const&amp;, long) + 0xfb (0x7f2160710c5b in /sapmnt/home/D067751/.local/share/vi
rtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #13: torch::autograd::generated::MseLossBackward::apply(std::vector&lt;at::Tensor, std::allocator&lt;at::Tensor&gt; &gt;&amp;&amp;) + 0x1af (0x7f2161a7393f in /sapmnt/home/
D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #14: &lt;unknown function&gt; + 0x2b31037 (0x7f2162030037 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/
lib/libtorch_cpu.so)
frame #15: torch::autograd::Engine::evaluate_function(std::shared_ptr&lt;torch::autograd::GraphTask&gt;&amp;, torch::autograd::Node*, torch::autograd::InputBuffer&amp;, std
::shared_ptr&lt;torch::autograd::ReadyQueue&gt; const&amp;) + 0x1400 (0x7f216202b880 in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/sit
e-packages/torch/lib/libtorch_cpu.so)
frame #16: torch::autograd::Engine::thread_main(std::shared_ptr&lt;torch::autograd::GraphTask&gt; const&amp;) + 0x451 (0x7f216202c421 in /sapmnt/home/D067751/.local/sha
re/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #17: torch::autograd::Engine::thread_init(int, std::shared_ptr&lt;torch::autograd::ReadyQueue&gt; const&amp;, bool) + 0x89 (0x7f2162024599 in /sapmnt/home/D067751
/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #18: torch::autograd::python::PythonEngine::thread_init(int, std::shared_ptr&lt;torch::autograd::ReadyQueue&gt; const&amp;, bool) + 0x4a (0x7f217020eb2a in /sapmn
t/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #19: &lt;unknown function&gt; + 0xc70f (0x7f216f8bd70f in /sapmnt/home/D067751/.local/share/virtualenvs/project-VOVyZKpb/lib/python3.8/site-packages/torch/lib
/libtorch.so)
frame #20: &lt;unknown function&gt; + 0x76ba (0x7f2175d116ba in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #21: clone + 0x6d (0x7f21753374dd in /lib/x86_64-linux-gnu/libc.so.6)
</code></pre>

PrimeF: <p>Any more help on this?<br>
Thanks in advance!</p>

linpershey: <p>I encounter a similar issue.</p>
<p>Previously, my dataset.py had this line: <code>torch.from_numpy(data)</code> and received <code>RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.HalfTensor) should be the same</code></p>
<p>I’ve changed it to <code>torch.from_numpy(data).float()</code> and the error went away.</p>
<p>Environment</p>
<ul>
<li>torch==1.10.1+cu113</li>
<li>torchaudio==0.10.1+cu113</li>
<li>torchinfo==1.5.2</li>
<li>torchvision==0.11.2+cu113</li>
</ul>