Error when loss.backward() is called

Erik_Jhonatta: <p>I’m trying to train a simple CNN, but when i call <code>loss.backward()</code>, i get the following error:</p>
<p><code>RuntimeError: expected dtype Double but got dtype Float (validate_dtype at ..\aten\src\ATen\native\TensorIterator.cpp:143)</code></p>
<p>My code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F 
import torch.optim as optim
import numpy as np
from tqdm import tqdm

class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        self.conv1 = nn.Conv2d(1,32, 3) #48; after max pool -&gt; 24
        self.conv2 = nn.Conv2d(32,64, 3) #22; after max pool -&gt; 11
        self.conv3 = nn.Conv2d(64,128, 3) #9; after max pool -&gt; 4.5 -&gt; 4

        self.fc1 = nn.Linear(128*4*4, 512)
        self.fc2 = nn.Linear(512, 2)

    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)),(2,2))
        x = F.max_pool2d(F.relu(self.conv2(x)),(2,2))
        x = F.max_pool2d(F.relu(self.conv3(x)),(2,2))
        x = torch.flatten(x,start_dim=1,end_dim=-1) 
        #x = x.view(-1,2048)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return F.softmax(x,dim=1)
    

net = Net()
optimizer = optim.Adam(net.parameters(), lr=0.001)

loss_function = nn.MSELoss()

training_data = np.load("training_data.npy",allow_pickle=True)

X = torch.tensor([i[0] for i in training_data]).view(-1,50,50)# Images
X = X/255.0 #pixels 0 e 1
y = torch.tensor([i[1] for i in training_data])# Hot vectors; Labels

VAL_PCT = 0.1

val_size = int(len(X)*VAL_PCT)

train_X = X[:-val_size]
train_y = y[:-val_size]

test_X = X[-val_size:]
test_y = y[-val_size:]

BATCH_SIZE = 100

EPOCHS = 1

for epoch in range(EPOCHS):
    for i in tqdm(range(0, len(train_X),BATCH_SIZE)):
        #print(i, i+BATCH_SIZE)
        batch_X = train_X[i:i+BATCH_SIZE].view(-1,1,50,50)
        batch_y = train_y[i:i+BATCH_SIZE]

        net.zero_grad()
        outputs = net(batch_X)
        loss = loss_function(outputs, batch_y)
        loss.backward()
        optimizer.step()
print(loss)

</code></pre>
<p>The TraceBack Message :</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "e:\Desenvolvimento\PyTorch\2Rede.py", line 63, in &lt;module&gt;
    loss.backward()
  File "C:\Users\erikj\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\tensor.py", line 198, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "C:\Users\erikj\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\autograd\__init__.py", line 100, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: expected dtype Double but got dtype Float (validate_dtype at ..\aten\src\ATen\native\TensorIterator.cpp:143)
(no backtrace available)
</code></pre>
<p>Torch Version:</p>
<blockquote>
<p>1.5.0+cu92</p>
</blockquote>
<p>I’m not using Cuda, as far as i know, this is just for learning purposes.</p>

Erik_Jhonatta: <p>After some research, i found a solution.<br>
I put the dtype of the tensors in the variabel ‘y’ to <code>dtype=torch.float32</code></p>
<p>I don’t know if it’s the best solution, anyway, any help will be welcomed</p>

russellizadi: <p>That’s the right solution since <code>nn.MSELoss</code> expects the same <code>dtype</code>.</p>

tuttelikz: <p>Just curious, if it is possible to handle this issue originally in backward() of PyTorch. Is there any reason why it is safe to throw this exception?</p>

addisonklinke: <p>Since the core issue is passing different dtypes to <code>MSELoss</code>, it seems to be that a <code>TypeError</code> should be thrown on <code>loss = loss_function(outputs, batch_y)</code> instead of a <code>RuntimeError</code> on <code>loss.backward()</code>. Why allow users to compute the loss tensor if it’s known to be invalid for backpropagation? Curious if the team has any thoughts there</p>