Input type (c10::Half) and bias type (float) should be the same

fibikong: <aside class="quote quote-modified" data-post="1" data-topic="81372">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/e19b73/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/valueerror-attemting-to-unscale-fp16-gradients/81372">ValueError : Attemting to unscale fp16 Gradients</a> 
  </div>
  <blockquote>
    Hello all, I am trying to train an LSTM in the half-precision setting. The LSTM takes an encoded input from a pre-trained autoencoder(Not trained in fp16). I am using  torch.amp instead of apex and scaling the losses as suggested in the documentation. 
Here is my training loop - 
    def train_model(self, model, dataloader, num_epochs):
        model.cuda()
        least_loss = 5
        model.train()
        optimizer = torch.optim.Adam(model.parameters(), lr =1e-5)
        scaler = amp.GradSca…
  </blockquote>
</aside>
<p>
I had same problem above because of <code>model.to(torch.float16)</code>.<br>
But if I remove that part, I cannot match the data type of inner layer’s input and weights.</p>
<pre><code class="lang-auto">RuntimeError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Input type (c10::Half) and bias type (float) should be the same
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 460, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/cvnar1/Desktop/official_code/loveu-tgve-2023/tuneavideo/models/resnet.py", line 15, in forward
    x = super().forward(x)
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/cvnar1/Desktop/official_code/loveu-tgve-2023/tuneavideo/models/unet.py", line 381, in forward
    sample = self.conv_in(sample)
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/cvnar1/Desktop/official_code/loveu-tgve-2023/train_tuneavideo.py", line 348, in main
    unet_output= accelerator.unwrap_model(model).unet(noisy_latents, timesteps, encoder_hidden_states)
  File "/home/cvnar1/Desktop/official_code/loveu-tgve-2023/train_tuneavideo.py", line 451, in &lt;module&gt;
    main(**OmegaConf.load(args.config))
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/cvnar1/anaconda3/envs/tgve/lib/python3.10/runpy.py", line 196, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
RuntimeError: Input type (c10::Half) and bias type (float) should be the same
</code></pre>
<p>please tell me how to fix this.</p>

ptrblck: <p>Could you post a minimal and executable code snippet raising the issue without your manual casting?</p>