Expected isFloatingType(grads[i].type().scalarType())

ati: <p>Hi<br>
while i was running gradient algorithm on this code:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/BINDS-LAB-UMASS/bindsnet_experiments/blob/master/experiments/mnist/surrogate_gradient.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/BINDS-LAB-UMASS/bindsnet_experiments/blob/master/experiments/mnist/surrogate_gradient.py" target="_blank" rel="nofollow noopener">BINDS-LAB-UMASS/bindsnet_experiments/blob/master/experiments/mnist/surrogate_gradient.py</a></h4>
<pre><code class="lang-py">from typing import Optional, Union

import torch
from torch import optim
import torch.nn.functional as F
from torch.autograd import Function
from torchvision import datasets, transforms


def fast_sigmoid(h):
    return h / (1 + torch.abs(h))

def grad_fast_sigmoid(h):
    return 1 / ((1 + torch.abs(h)) ** 2)


class LIFFunction(Function):

    @staticmethod
    def forward(ctx, x, v, decay, rest, thresh):
</code></pre>

  This file has been truncated. <a href="https://github.com/BINDS-LAB-UMASS/bindsnet_experiments/blob/master/experiments/mnist/surrogate_gradient.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>I got this error:</p>
<pre><code class="lang-auto">RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false. 
</code></pre>
<p>I have no idea how to correct it<br>
Could someone  please give me a help??</p>

sdv4: <p>Did you check the type of your gradients? My guess is that they are doubles. If that is the case, then you can cast your model as torch.float() to get the grads as the correct type.</p>

ati: <p>thanks <a class="mention" href="/u/sdv4">@sdv4</a> for your response<br>
but what you mean exactly by saying check your gradients?<br>
could you explain more please?</p>

sdv4: <p>When you are training your model, after you call <code>loss.backwards()</code> , but before you call <code>optimizer.step()</code>, you can inspect the gradients of any parameter of the model that you are interested in.</p>
<p>You can do this by calling <code>model.named_parameters()</code>. This creates an iterator that you can go through to find any of the model’s parameters that you would like to look at. Once you have the tensor containing model parameters, you call <code>.grad</code> to view the gradient. By looking at the type of the output of this call, you can see if your gradients are doubles or floats.</p>
<p>Alternatively, you can just call <code>model = model.float()</code> on your model before training, but this might raise other errors if your data is of the double type.</p>

ati: <p>Ow, Got it…thanks for the clarification <a class="mention" href="/u/sdv4">@sdv4</a><br>
I will check it out and will share the results:relaxed:</p>

Yanxian_Chen: <p>Hi, have you solved this problem?</p>

ati: <p>Hi <a class="mention" href="/u/yanxian_chen">@Yanxian_Chen</a> …yeah, i did</p>

Zuanazzi: <p><a class="mention" href="/u/ati">@ati</a> how did you solve it? I am facing the same issue, but all tensors I checked are float32.</p>

Mahdi_Moalla: <p>I had similar problem. I was using pytorch 1.4 with cuda 10.1. I just switched back to pytorch 1.2 with cuda 10.0 and it was solved.</p>