Pytorch gets me crazy with float or double

username: Thibault712
cooked: <p>Hi is there a way to tell pytorch “autocast in whatever type is good for you, for me they all are numbers it’s python not C++”, because it’s coming to the point where it doesn’t make sense:</p>
<p>So first I force the type of my data to be in float, and it complains because it wants double:</p>
<pre><code class="lang-auto">RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/thibault/pythia/dev/pythia/GNN/interface_designer0/Models/M_ID0.py", line 58, in forward
    x_i = self.conv1[str(i)](data.x.float(), data.edge_index, data.edge_attr)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/transformer_conv.py", line 176, in forward
    out = self.propagate(edge_index, query=query, key=key, value=value,
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/message_passing.py", line 317, in propagate
    out = self.message(**msg_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/transformer_conv.py", line 211, in message
    edge_attr = self.lin_edge(edge_attr).view(-1, self.heads,
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/dense/linear.py", line 118, in forward
    return F.linear(x, self.weight, self.bias)
RuntimeError: expected scalar type Double but found Float
</code></pre>
<p>So I force the type to be double and now it expects floats ?? WTH ?</p>
<pre><code class="lang-auto">    raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/thibault/pythia/dev/pythia/GNN/interface_designer0/Models/M_ID0.py", line 58, in forward
    x_i = self.conv1[str(i)](data.x.double(), data.edge_index, data.edge_attr)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/transformer_conv.py", line 171, in forward
    query = self.lin_query(x[1]).view(-1, H, C)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/dense/linear.py", line 118, in forward
    return F.linear(x, self.weight, self.bias)
RuntimeError: expected scalar type Float but found Double

</code></pre>

username: AlphaBetaGamma96
cooked: <p>Could you try enforcing <code>double</code> via <code>torch.set_default_dtype(torch.float64)</code> and see if you get the same error? Maybe <code>torch_geometric</code> enforces certain <code>dtype</code> on certain functions</p>

username: Thibault712
cooked: <p>Thank you it worked <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> , I know it’s not pretty but somehow it works and I just need something functional.</p>