RuntimeError: Found dtype Double but expected Float

craine: <p>I have tried a bunch of things I’ve found on forums to fix this issue but nothing is working. I’m getting this error trying to run my model.</p>
<p>Notebook is public here:<br>
<a href="https://drive.google.com/file/d/1akU15oJrzFi9KsY3oyrstmvf9tW_8mpS/view?usp=sharing" class="onebox" target="_blank" rel="noopener nofollow ugc">https://drive.google.com/file/d/1akU15oJrzFi9KsY3oyrstmvf9tW_8mpS/view?usp=sharing</a></p>
<p>ERROR:</p>
<h2>
<a name="epoch-0-04-0000-its-1" class="anchor" href="#epoch-0-04-0000-its-1"></a>Epoch:   0%|          | 0/4 [00:00&lt;?, ?it/s]</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
30     # Backward pass<br>
31     torch.set_default_dtype(torch.float64)<br>
—&gt; 32     loss.backward()<br>
33     # Update parameters and take a step using the computed gradient<br>
34     optimizer.step()</p>
<p>1 frames<br>
/opt/conda/lib/python3.7/site-packages/torch/autograd/<strong>init</strong>.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)<br>
130     Variable.<em>execution_engine.run_backward(<br>
131         tensors, grad_tensors</em>, retain_graph, create_graph,<br>
 → 132         allow_unreachable=True)  # allow_unreachable flag<br>
133<br>
134</p>
<p>RuntimeError: Found dtype Double but expected Float</p>

ptrblck: <p>I guess the error is raised by changing the default <code>dtype</code> before calling the <code>backward</code> method in:</p>
<pre><code class="lang-python">     31     torch.set_default_dtype(torch.float64)
---&gt; 32     loss.backward()
</code></pre>
<p>I assume this could create type mismatches, since the forward pass wouldn’t match the backward pass anymore.<br>
Could you explain, why you are changing the default <code>dtype</code> globally during training?</p>

craine: <p>I copied an error from a test… I was just trying something out… My mistake. Here is the error:</p>
<h2>
<a name="epoch-0-04-0000-its-1" class="anchor" href="#epoch-0-04-0000-its-1"></a>Epoch:   0%|          | 0/4 [00:00&lt;?, ?it/s]</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
29     train_loss_set.append(loss.item())<br>
30     # Backward pass<br>
—&gt; 31     loss.backward()<br>
32     # Update parameters and take a step using the computed gradient<br>
33     optimizer.step()</p>
<p>/opt/conda/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)<br>
219                 retain_graph=retain_graph,<br>
220                 create_graph=create_graph)<br>
 → 221         torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
222<br>
223     def register_hook(self, hook):</p>
<p>/opt/conda/lib/python3.7/site-packages/torch/autograd/<strong>init</strong>.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)<br>
130     Variable.<em>execution_engine.run_backward(<br>
131         tensors, grad_tensors</em>, retain_graph, create_graph,<br>
 → 132         allow_unreachable=True)  # allow_unreachable flag<br>
133<br>
134</p>
<p>RuntimeError: Found dtype Double but expected Float</p>