Strange type error

Mah_Neh: <p>I want to get the network weights as float16, so the network is currently <code>nn().to(torch.float16)</code>nd, however, I get an error due to the biases being float32 type.</p>
<p>This is the error</p>
<pre><code class="lang-auto">    return F.conv2d(input, weight, bias, self.stride,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Input type (c10::Half) and bias type (float) should be the same

</code></pre>
<p>this is the simplified net:</p>
<pre><code class="lang-auto"># %%
class NeuralNetwork(nn.Module):
    """
    A PyTorch neural network for image classification.

    Args:
        channels: Number of channels in the input image.
        height: Height of the input image in pixels.
        width: Width of the input image in pixels.
    """

    def __init__(self, channels: int, height: int, width: int):
        super().__init__()
        self.backbone = Backbone()
        output_dimensions = self.backbone.forward(
            torch.zeros(1, channels, height, width).to(device, dtype=dtype)
        ).shape
        self.head = Head(output_dimensions[-1])

    def forward(self, x):
        x = self.backbone(x)
        x = self.head(x)
        return x
</code></pre>
<p>Does this need explicitly passing the dtype to each layer ? Is there a better way to achieve this ?</p>

ptrblck: <p>Could you post the missing code pieces making the code minimal and executable so that we can reproduce and debug the issue?</p>

Mah_Neh: <p>I can. Just a quick question that could save me the effort: would it be equivalent to train in float32 and only later, before exporting, convert the model to <code>torch.float16</code> using <code>model.half()</code> ?</p>
<p>I only need the output net to be very small and fast for small devices/web.</p>

ptrblck: <aside class="quote no-group" data-username="Mah_Neh" data-post="3" data-topic="186877">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mah_neh/48/57251_2.png" class="avatar"> Mah_Neh:</div>
<blockquote>
<p>would it be equivalent to train in float32 and only later, before exporting, convert the model to <code>torch.float16</code> using <code>model.half()</code> ?</p>
</blockquote>
</aside>
<p>I don’t fully understand the question. What would be equivalent to training the model in <code>FP32</code> first?</p>

Mah_Neh: <p>Training in <code>FP16</code></p>
<p>So I see two possibilities:</p>
<ol>
<li>Train using float32 weights and input data (images) and then export it as half precision</li>
<li>Train using float16 weights and float16 input data, and then only export as it is.</li>
</ol>
<p>Currently, I will try to do 1. because it seems easier, but I started with 2. originally, and was wondering what would be a more experienced user opinion.</p>

ptrblck: <p>The second approach can easily fail and diverge, which is why we recommend using <code>amp</code> for mixed-precision training. Depending on the model pure <code>float16</code> training could still work, but it’s generally less stable.</p>

Mah_Neh: <p>never heard <code>amp</code> before. Yet, it seems unnecessary here</p>