After first round optimization success, raise ValueError: can't optimize a non-leaf Tensor

Janine: <p>Hi all,</p>
<p>I need a bit of help with my PyTorch code. I am trying to generate a random image, and then optimize its cell values in order to be classified as a targeted class by an image classifier, the optimization seems to work for the first iteration, and then raise the ValueError: can’t optimize a non-leaf Tensor.</p>
<p>I am quite confused why the optimizer seems to be working for the first time but not for the following iterations. Any help would be highly appreciated.</p>
<p>This is my code:</p>
<pre><code class="lang-auto"># set the target class
y_target = Variable(torch.LongTensor([0]), requires_grad=False)
y_target = y_target.to(device)

# Generate a random image
created_image = torch.normal(mean=mean[0], std=std[0], size=(1,1,28,28), requires_grad=True, device="cuda")

for i in range(1, 200):
    # Define optimizer for the image
    optimizer = optim.SGD([created_image],lr=1e-2)
    print("111")
    #clamp created img
    created_image = torch.clamp(created_image, -0.4241,2.8214)
    zero_gradients(created_image)
    # Forward
    output_created_image = net.forward(created_image)
    # Get confidence from softmax
    created_image_confidence = F.softmax(output_created_image)[0][y_target]
    print("222")
    print(created_image_confidence)
    created_image_confidence_probs =  round((created_image_confidence.item()) * 100,2)
    print("333")
    print(created_image_confidence_probs)

    if created_image_confidence_probs &gt; 80.00:
        print("444")
        created_image = created_image.squeeze().detach().cpu().numpy()
        created_image = (created_image * std[0] + mean[0]) * 255.0
        np.save(f"./adv_images/{y_target.item()}/{i}_gaf_t_c80.npy",adv_data)
        break

    # Target specific class
    class_loss = -output_created_image[0, y_target]
    print("555")
    # Zero all existing gradients
    net.zero_grad()
    print("666")
    # Backward
    class_loss.backward
    print("777")
    # Update image
    optimizer.step()
    print("888")
</code></pre>
<p>And this is the result I get:</p>
<pre><code class="lang-auto">111
222
tensor([0.0891], device='cuda:0', grad_fn=&lt;IndexBackward&gt;)
333
8.91
555
666
777
888
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
 in 
     16     # created_image = created_image.requires_grad_()
     17     # Define optimizer for the image
---&gt; 18     optimizer = optim.SGD([created_image],lr=6)
     19     print("111")
     20     #clamp created img

C:\ProgramData\Anaconda3\lib\site-packages\torch\optim\sgd.py in __init__(self, params, lr, momentum, dampening, weight_decay, nesterov)
     62         if nesterov and (momentum &lt;= 0 or dampening != 0):
     63             raise ValueError("Nesterov momentum requires a momentum and zero dampening")
---&gt; 64         super(SGD, self).__init__(params, defaults)
     65 
     66     def __setstate__(self, state):

C:\ProgramData\Anaconda3\lib\site-packages\torch\optim\optimizer.py in __init__(self, params, defaults)
     49 
     50         for param_group in param_groups:
---&gt; 51             self.add_param_group(param_group)
     52 
     53     def __getstate__(self):

C:\ProgramData\Anaconda3\lib\site-packages\torch\optim\optimizer.py in add_param_group(self, param_group)
    200                                 "but one of the params is " + torch.typename(param))
    201             if not param.is_leaf:
--&gt; 202                 raise ValueError("can't optimize a non-leaf Tensor")
    203 
    204         for name, default in self.defaults.items():

ValueError: can't optimize a non-leaf Tensor
</code></pre>

ptrblck: <p>You are creating a non-leaf tensor with the <code>torch.clamp</code> operation:</p>
<pre><code class="lang-python">created_image = torch.normal(mean=1, std=1, size=(1,1,28,28), requires_grad=True)
print(created_image.is_leaf)
&gt; True
created_image = torch.clamp(created_image, -1, 1)
print(created_image.is_leaf)
&gt; False
</code></pre>
<p>Try to change the output of <code>torch.clamp</code> to another variable name.</p>
<p>Also, there are some minor issues in the code:</p>
<ul>
<li>
<code>Variables</code> are deprecated since <code>0.4.0</code>, so you can use tensors directly</li>
<li>Don’t call <code>net.forward</code>, as this will not call into registered hooks. Instead call the model as <code>net(input)</code>
</li>
<li>
<code>class_loss.backward</code> is probably a typo and should call the method as <code>class_loss.backward()</code>
</li>
</ul>

Janine: <p>Thanks a bunch, ptrblck! All issues solved. Save my day <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>