PyTorch metric problem RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

SwapnanilHalder: <p>I am trying to create an ANN Deep Learning model with PyTorch, and separated functions for calculate loss, and evaluate the model on the validation set. I am having an error RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn.</p>
<p>my code is :</p>
<pre><code class="lang-auto">model = MNISTModel()

def accuracy(outputs, labels):
    _, preds = torch.max(outputs, dim=1)
    return torch.sum(preds == labels).item() / len(preds)

def loss_batch(model, loss_fn, x, y, opt = None, metric = None):
    x = Variable(x).cuda()
    y = Variable(y.cuda())
    preds = model(x)
    loss = loss_fn(preds, y)
    
    if opt is not None :
        loss.backward()
        opt.step()
        opt.zero_grad()
        
    metric_result = None
    if metric is not None:
        metric_result = metric(preds, y)
        
    return loss.item(), len(x), metric_result

def val_evaluate(model, loss_fn, val_loader, metric = None):
    with torch.no_grad():
        results = [loss_batch(model, loss_fn, x, y, metric) for x,y in val_loader]
        
        losses, nums, metrics = zip(*results)
        
        total = np.sum(nums)

        # Avg. loss across batches 
        avg_loss = np.sum(np.multiply(losses, nums)) / total
        avg_metric = None
        if metric is not None:
            # Avg. of metric across batches
            avg_metric = np.sum(np.multiply(metrics, nums)) / total
    return avg_loss, total, avg_metric

def fit(epochs, lr, model, loss_fn, train_dl, 
        valid_dl, metric=None, opt_fn=None):
    losses, metrics = [], []
    
    # Instantiate the optimizer
    if opt_fn is None: opt_fn = torch.optim.SGD
    opt = torch.optim.SGD(model.parameters(), lr=lr)
    
    for epoch in range(epochs):
        # Training
        for xb,yb in train_dl:
            loss,_,_ = loss_batch(model, loss_fn, Variable(xb), Variable(yb), opt)

        # Evaluation
        result = val_evaluate(model, loss_fn, valid_dl, metric)
        val_loss, total, val_metric = result
        
        # Record the loss &amp; metric
        losses.append(val_loss)
        metrics.append(val_metric)
        
        # Print progress
        if metric is None:
            print('Epoch [{}/{}], Loss: {:.4f}'
                  .format(epoch+1, epochs, val_loss))
        else:
            print('Epoch [{}/{}], Loss: {:.4f}, {}: {:.4f}'
                  .format(epoch+1, epochs, val_loss, 
                          metric.__name__, val_metric))
    return losses, metrics

losses1, metrics1 = fit(5, 0.5, model, nn.CrossEntropyLoss(), 
                        train_loader, val_loader, accuracy)

</code></pre>
<p>By comparing it to actual correct code, I found that<br>
whenever I am giving metric = metric in val_evaluate function in the line to get ‘Results’, I am no longer getting this error. But whenever I am passing only metric, this error comes out :</p>
<p><em>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn.</em></p>
<p>but, the accuracy metric has no relation with grad_fn, then why this error is coming?</p>

ptrblck: <p>You are calling <code>loss_batch(model, loss_fn, x, y, metric)</code> in <code>val_evaluate</code> inside the <code>torch.no_grad()</code> block, which is the correct thing to do.<br>
However, <code>loss_batch</code> expects the arguments: <code>model, loss_fn, x, y, opt = None, metric = None</code>, so you are currently passing <code>metric</code> as the <code>opt</code> argument.<br>
Inside <code>loss_batch</code>, the <code>opt</code> argument will be checked for <code>not None</code> and the <code>backward</code> call will be executed.</p>
<p>Passing <code>metric=metric</code> solves this issue, as <code>opt</code> is then indeed <code>None</code>. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

SwapnanilHalder: <p>Ya, I just figured it out. Such a silly mistake to overlook. Sorry for bothering you.</p>