Higher order autograd problem

Steven_Duan: <p>Hi:<br>
I am trying to implement an generative poisoning attack method and the paper can be found here: <a href="https://arxiv.org/abs/1703.01340" class="inline-onebox" rel="noopener nofollow ugc">[1703.01340] Generative Poisoning Attack Method Against Neural Networks</a>. So the problem I am having is very tricky and let me explain it step by step:<br>
Repeat:<br>
Step 1: I trained a resnet18 with animal images from 4 classes.<br>
Step 2: I feed a random image(I call it Xp) into the network(This is the poisoned image init and I am updating) and get a loss_p.<br>
Step 3: Update model parameter based on loss_p to get w_p(The updated parameter after feeding in Xp)<br>
Step 4: Feed the original training data into W_p and get a loss_p_i.(Note that the goal is to maximize this loss_p_i)<br>
Step 5: Update Xp by adding lr*dloss_p_i/dxp</p>
<p>Here is a clip of the paper I am following:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/8/38a981b168d7f4d920e259d83ffa9e0ea3d26ff1.png" data-download-href="https://discuss.pytorch.org/uploads/default/38a981b168d7f4d920e259d83ffa9e0ea3d26ff1" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/38a981b168d7f4d920e259d83ffa9e0ea3d26ff1_2_435x499.png" alt="image" data-base62-sha1="85fU5eWFot7Ryf6PWttLULpfHiN" width="435" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/38a981b168d7f4d920e259d83ffa9e0ea3d26ff1_2_435x499.png, https://discuss.pytorch.org/uploads/default/original/3X/3/8/38a981b168d7f4d920e259d83ffa9e0ea3d26ff1.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/8/38a981b168d7f4d920e259d83ffa9e0ea3d26ff1.png 2x" data-dominant-color="EDEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">505×580 70.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
My problem is that, I am updating dloss_p_i/dxp. However dloss_p_i is not directly related. dloss_p_i is related to W_p and W_p is related to Xp. Therefore when I calculate dloss_p_i/dxp I get error messages like this:<br>
One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.</p>
<p>Here is the code:</p>
<pre><code class="lang-auto">optimizer_ft = optim.SGD(resnet.parameters(), lr=0.001, momentum=0.9)
criterion = nn.CrossEntropyLoss()
max_poison_iter = 10

# Change the label to an inccorect one
if poison_label[0] == 0:
    poison_label[0] = 1
else:
    poison_label[0] = 0
    
    
for i in range(max_poison_iter):
    print(i)
    for images, labels in trainset_loader:
        images = images.to(device)
        labels = labels.to(device)
        resnet = resnet.to(device)
        
        
        
        p_outputs = resnet(poison_image)
        
        
        p_loss = criterion(p_outputs, poison_label)
        p_loss.backward(create_graph = True, retain_graph = True)
        
        optimizer_ft.step()

       
        
        o_outputs = resnet(images)
        loss_p_i = criterion(o_outputs, labels)
        p_grad = grad(loss_p_i, poison_image)
        
        

        poison_image.data += 0.1*p_grad[0].sign()
        
        


</code></pre>

albanD: <p>Hi,</p>
<p>The optimizer step done with pytorch optimizers are not differentiable I’m afraid. So the gradient won’t be able to flow all the way back. You can check the higher library <a href="https://github.com/facebookresearch/higher/">here</a> to see how they do this using their custom differentiable optimizers.<br>
Also you should not use <code>.data</code>. If you want to update the poison_image without the gradients being tracked, you should do:</p>
<pre><code class="lang-auto">with torch.no_grad():
  poison_image += 0.1*p_grad[0].sign()
</code></pre>

Steven_Duan: <p>Thank you very much! The higher library solved my problem.</p>