I am getting this error: element 0 of tensors does not require grad and does not have a grad_fn

The_duke: <p>Hi all, first post here.</p>
<p>I am getting the error in the title when I am calling loss.backward:</p>
<pre><code>  criterion = nn.MSELoss()
</code></pre>
<p>…</p>
<pre><code>  train_epoch_loss = 0
    train_epoch_acc = 0

    model.train()
    for X_train_batch, y_train_batch in train_loader:
        X_train_batch, y_train_batch = X_train_batch.to(device), y_train_batch.to(device)
        optimizer.zero_grad()
        
        
        y_train_pred = model(X_train_batch).argmax(dim=1).float()
       
        train_loss = criterion(y_train_pred, y_train_batch)
        train_acc = multi_acc(y_train_pred, y_train_batch)

        train_loss.backward()
        optimizer.step()

        train_epoch_loss += train_loss.item()
        train_epoch_acc += train_acc.item()
</code></pre>
<p>Python give me this error:</p>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in <br>
137<br>
138<br>
 → 139 root = Tree(“root-”,train,<br>
140             homo_threshold = THRESH)</p>
<p> in <strong>init</strong>(self, name, df, homo_threshold)<br>
52         self.kid_rank = None<br>
53<br>
—&gt; 54         self.model = make_nn(df,verbose=False) if self.kids == None else None<br>
55<br>
56     def Error(self,pred,y):</p>
<p> in make_nn(df, verbose)<br>
149             train_acc = multi_acc(y_train_pred, y_train_batch)<br>
150<br>
 → 151             train_loss.backward()<br>
152             optimizer.step()<br>
153</p>
<p>D:\anaconda3\lib\site-packages\torch_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)<br>
253                 create_graph=create_graph,<br>
254                 inputs=inputs)<br>
 → 255         torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)<br>
256<br>
257     def register_hook(self, hook):</p>
<p>D:\anaconda3\lib\site-packages\torch\autograd_<em>init</em>_.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)<br>
145         retain_graph = create_graph<br>
146<br>
 → 147     Variable.<em>execution_engine.run_backward(<br>
148         tensors, grad_tensors</em>, retain_graph, create_graph, inputs,<br>
149         allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag</p>
<p>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</p>
<p>Up to now, I suspect that calling argmax is the culprit, but I am not 100% certain of this neither of why or how. I had a previous version where I used nn.crossentropy and it worked fine (with minor tweaks) but once I add argmax to the mix it seems to throw this error.</p>
<p>A common refrain in other threads suggest I somehow messed with the computation graph by mistake, is there a way to visualize it so I can troubleshoot it myself?</p>
<p>This is part of broader tests to create my own loss function.<br>
I know I am banging rocks together, so what am I doing wrong?</p>
<p>PS: I tried to cut down code to the bare minimum as to make the post more legible, as IIRC I don’t think I changed much more than that particular section.</p>
<p>In case I cut too much, this is the full source code of the script I am writing:</p><aside class="onebox githubgist" data-onebox-src="https://gist.github.com/Finalfantasyfan/b4acaa0dba49c3ba042ce53a887433af">
  <header class="source">

      <a href="https://gist.github.com/Finalfantasyfan/b4acaa0dba49c3ba042ce53a887433af" target="_blank" rel="noopener nofollow ugc">gist.github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://gist.github.com/Finalfantasyfan/b4acaa0dba49c3ba042ce53a887433af" target="_blank" rel="noopener nofollow ugc">https://gist.github.com/Finalfantasyfan/b4acaa0dba49c3ba042ce53a887433af</a></h4>

  <h5>gistfile1.py</h5>
  <pre><code class="Python">import torch
from torch import nn

from sklearn.utils import shuffle
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import LabelEncoder

from sklearn.neighbors import KNeighborsClassifier
import numpy as np
import os</code></pre>
    This file has been truncated. <a href="https://gist.github.com/Finalfantasyfan/b4acaa0dba49c3ba042ce53a887433af" target="_blank" rel="noopener nofollow ugc">show original</a>

<p>
</p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ptrblck: <p><code>.argmax</code> is not a differentiable function, so you are detaching the model output in:</p>
<pre><code class="lang-python"> y_train_pred = model(X_train_batch).argmax(dim=1).float()
</code></pre>
<p>which will raise the error.<br>
I don’t know what kind of use case you are working on, but in case it’s a multi-class classification, pass the raw logits to <code>nn.CrossEntropyLoss</code> instead of the (detached) <code>argmax</code> to <code>nn.MSELoss</code>.</p>