'NoneType' object has no attribute 'data' when computing gradient on image pixels

enterthevoidf22: <p>i’m trying to write a simple image generator as follows:</p>
<pre><code class="lang-auto">def class_visualization_update_step(img, model, target_y, l2_reg, learning_rate):
    ########################################################################
    # TODO: Use the model to compute the gradient of the score for the     #
    # class target_y with respect to the pixels of the image, and make a   #
    # gradient step on the image using the learning rate. Don't forget the #
    # L2 regularization term!                                              #
    # Be very careful about the signs of elements in your code.            #
    ########################################################################
    # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
    X = img.clone()
    X = X.requires_grad_()
    scores = model(X)
    R = l2_reg * X.norm() 
    scores = scores - R
    scores[0,traget_y].backward()
    dx = X.grad.data
    X.data = X.data + learning_rate * (dx / dx.norm())
    pass
 
</code></pre>
<p>and getting the following error when i run</p>
<pre><code class="lang-auto">UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.
  """Entry point for launching an IPython kernel.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
&lt;ipython-input-153-46dacb4c9ec4&gt; in &lt;module&gt;
----&gt; 1 X.grad.data

AttributeError: 'NoneType' object has no attribute 'data'
</code></pre>
<p>P.S<br>
i know i still need to repair the correctness of the code by backpropagating with respect to the R term, but before that i want to understand what technically am i doing wrong to cause the X.grad to be a nonetype.</p>
<p>thanks!</p>

ptrblck: <p>As the warning explains you are trying to access the <code>.grad</code> attribute of a non-leaf tensor, which won’t be populated by default.<br>
You can call <code>.retain_grad()</code> on the non-leaf Tensor to access the gradient after the backward call.</p>
<p>Also note that the usage of the <code>.data</code> attribute is not recommended, as Autograd won’t be able to track these operations which might yield to silent errors.</p>

enterthevoidf22: <p>ok!<br>
not sure why is X not a leaf tensor when i :</p>
<ol>
<li>initiated it myself  is not a product of some computation</li>
<li>i set requires_grad to true.</li>
</ol>
<p>furthermore, perhaps you can shed light about the data attribute. i can’t figure out when to use it when updating a tensor.</p>

ptrblck: <p>I don’t know what might create the non-leaf tensor in your code, but this minimal code snippet seems to work:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(1, 1)
x.requires_grad_()
print(x.is_leaf)

model = nn.Linear(1, 1)
out = model(x)
out.backward()
print(x.grad)
</code></pre>
<aside class="quote no-group" data-username="enterthevoidf22" data-post="3" data-topic="98424">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/enterthevoidf22/48/29338_2.png" class="avatar"> enterthevoidf22:</div>
<blockquote>
<p>furthermore, perhaps you can shed light about the data attribute. i can’t figure out when to use it when updating a tensor.</p>
</blockquote>
</aside>
<p>I wouldn’t recommend to use the <code>.data</code> attribute at all but instead wrap the code in a <code>with torch.no_grad()</code> block, if you want to apply an operation without tracking it via Autograd.<br>
There were some use cases e.g. inside optimizers, which should have been removed by now.</p>