Using Pretrained resnet for regression, RuntimeError: Found dtype Double but expected Float

username: mufty
cooked: <p>Hello Everyone,<br>
I am trying to train a model that learns to predict a single value(float) given an image using the pretrained resnet18 model. I am using the MSE loss but I am getting a Runtime error below.</p>
<p><code>RuntimeError: Found dtype Double but expected Float</code>.  from loss.backward()</p>
<p>Below is my code:</p>
<pre><code class="lang-auto">
resnet18= models.resnet18(pretrained=True)
num_ftrs = resnet18.fc.in_features
resnet18.fc = nn.Linear(num_ftrs , 1)

def train_model(model, dataloaders, device, num_epochs=2, is_train=True):
  acc_history=[]
  loss_history=[]
  best_acc= 0.0
  for epoch in tqdm(range(num_epochs)):
    running_loss = 0.0
    running_corrects=0
  #Iterate over dataloader
    for x,y in dataloaders:

      for image,target in zip(x,y):
        image = image.to(device)
        target = target.to(device)
        model.to(device)

        #zero the parameter gradients 
        optimizer.zero_grad()
        criterion = nn.MSELoss()
        
        # forward 
        out = model(image.unsqueeze(0))
        print(out)
        loss = criterion(out, target)
        _,preds =torch.max(out,1)
        #backward 
        loss.backward()
        optimizer.step()
</code></pre>
<p>Your help is really appreciated</p>

username: mufty
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Kindly help out</p>

username: Dwight_Foster
cooked: <p>Can you provide the full error? Also have you tried converting any of the variables to float use .float(). Maybe try doing any of these:</p>
<pre><code class="lang-auto">loss = criterion(out, target.float())
# or
loss = criterion(out.float(), target)
# or
loss = criterion(out, target).float()
</code></pre>

username: mufty
cooked: <p>Thanks for the response. I just tried all the above suggestions and below is the full error.</p>
<pre><code class="lang-auto">Epoch 0/1
----------
tensor([[0.9929]], grad_fn=&lt;AddmmBackward&gt;)
  0%|                                                                                                    | 0/2 [00:01&lt;?, ?it/s]
Traceback (most recent call last):
  File "train.py", line 16, in &lt;module&gt;
    train_acc_hist, train_loss_hist = train_model(resnet18, train_loader, device)
  File "/model.py", line 64, in train_model
    loss.backward()
  File "/MTL/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/MTL/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Found dtype Double but expected Float
</code></pre>
<p>I also printed out my input and target with their size and found this.</p>
<pre><code class="lang-auto">out : tensor([[-0.6273]], grad_fn=&lt;AddmmBackward&gt;) torch.Size([1, 1])
target : tensor(21.1924, dtype=torch.float64) torch.Size([])
loss : tensor(476.0975, dtype=torch.float64, grad_fn=&lt;MseLossBackward&gt;) torch.Size([])
                                                                                   
</code></pre>

username: mufty
cooked: <p><a class="mention" href="/u/dwight_foster">@Dwight_Foster</a> <a class="mention" href="/u/ptrblck">@ptrblck</a>  help out</p>

username: sio277
cooked: <aside class="quote no-group" data-username="mufty" data-post="4" data-topic="116192">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mufty/48/35422_2.png" class="avatar"> mufty:</div>
<blockquote>
<pre><code class="lang-auto">out : tensor([[-0.6273]], grad_fn=&lt;AddmmBackward&gt;) torch.Size([1, 1])
target : tensor(21.1924, dtype=torch.float64) torch.Size([])
loss : tensor(476.0975, dtype=torch.float64, grad_fn=&lt;MseLossBackward&gt;) torch.Size([])
</code></pre>
</blockquote>
</aside>
<p>dtypes of your <code>target</code> tensor as well as the <code>image</code> tensor should be <code>torch.float32</code>. Check your <code>Dataset.__getitem__</code> returns the float32 tensors.</p>