How can I modify the saved value used for backpropagation?

_humandream: <p>It may seem weird, but I’m trying to modify the backpropagation itself. (It’s hard to explain in short, but it’s about the adversarial attack.)</p>
<p>I want to modify the saved values of the graph that used in backpropagation. I want something like :</p>
<p>x = Variable(torch.ones(5,3,32,32))<br>
conv = nn.Conv2d(3,24,kernel_size=3, padding=1)<br>
y = F.conv2d(x, conv.weight)<br>
loss = y.sum()<br>
loss.backward()<br>
print(conv.weight.grad[0,0,0]) # tensor([4500., 4500., 4500.])</p>
<p>x = Variable(torch.ones(5,3,32,32))<br>
conv = nn.Conv2d(3,24,kernel_size=3, padding=1)<br>
y = F.conv2d(x, conv.weight)<br>
loss = y.sum()<br>
x.data = 2*x.data<br>
loss.backward()<br>
print(conv.weight.grad[0,0,0]) # It gives ([4500., 4500., 4500.]), but I want ([9000., 9000., 9000.])</p>
<p>It’s not x, also not inside the nn.Conv2d, and F.conv2d looks like a function and looks like have no place for saving something. Where should I found it?</p>
<p>Any help would be greatly appreciated. Thank you.</p>

ptrblck: <p>There are a few issues in the code:</p>
<ul>
<li>You shouldn’t manipulate the <code>.data</code> attribute, as Autograd cannot track these changes, which might result in wrong results. If you want to manipulate the data without tracking the computation, use <code>with torch.no_grad</code> instead</li>
<li>Manipulating <code>x</code> after the gradient were calculated, doesn’t change the gradients anymore</li>
<li>If you would like to scale the gradient, this would work:</li>
</ul>
<pre><code class="lang-python">x = torch.ones(5,3,32,32)
conv = nn.Conv2d(3,24,kernel_size=3, padding=1)
y = F.conv2d(x, conv.weight)
loss = y.sum() * 2
loss.backward()
print(conv.weight.grad[0,0,0]) 
&gt; tensor([9000., 9000., 9000.])
</code></pre>
<ul>
<li>
<code>Variable</code> is deprecated since PyTorch <code>0.4.0</code>, so you can now use tensors <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:">
</li>
</ul>

_humandream: <p>Thank you. But I need to do far more complicated modifications than just scaling the gradient.<br>
So I want to get ‘wrong’ results by deceiving pytorch. Is there no way at all?</p>

ptrblck: <p>I wouldn’t recommend to try to “deceive” PyTorch. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"><br>
If you explain your use case a bit, we might come up with a clean approach, e.g. by using hooks on the parameters etc.</p>

_humandream: <p>Thank you. I really hope there is such a way.<br>
I have two inputs, x1 and x2. Let’s consider a simple linear layer. The gradient is originally calculated like :</p>
<pre><code class="lang-auto">def backward(self, grad_output) : 
    input, weight = self.saved_tensors
    grad_input = grad_output.mm(weight)
    grad_weight = grad_output.t().mm(input)
    return grad_input,  grad_weight
</code></pre>
<p>I want to do something like :</p>
<pre><code class="lang-auto">def backward(self, grad_output) : 
    input1, weight = self.saved_tensors
    input2, weight = self.saved_tensors2
    grad_input = grad_output.mm(weight1)
    grad_weight = grad_output.t().mm(input1+input2)
    return grad_input,  grad_weight
</code></pre>
<p>I can’t just forward with x1+x2, since It changes nonlinear activations like ReLU, MaxPool, etc. Furthermore, grad_output should be calculated with x1, not with x1+x2. This is really unusual situation, and I found that even deceiving pytorch is not simple. I’m totally blocked <img src="https://discuss.pytorch.org/images/emoji/apple/tired_face.png?v=9" title=":tired_face:" class="emoji" alt=":tired_face:"> and would be really happy with any help.</p>
<p>Thank you.</p>

agadetsky: <p>you can define custom <code>Linear</code> layer with your <code>.forward</code> and <code>.backward</code>. See <a href="https://pytorch.org/docs/stable/notes/extending.html" rel="nofollow noopener">https://pytorch.org/docs/stable/notes/extending.html</a></p>