Double propagation issue

Salman_Abbasi: <p>Hi</p>
<p>I have a network(dnn_Linear) output(v3). I first like to estimate a variables ss, which is Jacobian of this network w.r.t. to its parameters. Then I would like to backpropagate the loss, and later mutate gradients by multiplying with ss.</p>
<p>Now i get the error:<br>
<em>Trying to backward through the graph a second time"</em></p>
<p>The part of script is:<br>
ss = torch.autograd.grad(v3,dnn_Linear.parameters(), grad_outputs=torch.ones_like(v3),retain_graph=None, create_graph=False, only_inputs=True, allow_unused=True)<span class="hashtag-raw">#jacobian</span><br>
Total_loss.backward(retain_graph=True)<span class="hashtag-raw">#back</span> propagate<br>
v3.grad = torch.tensor(ss).T*v3.grad <span class="hashtag-raw">#mutate</span> part</p>
<p>How can I achieve this?</p>
<p>Thanks<br>
Sal</p>

ptrblck: <p>Use <code>retain_graph=True</code> in the <code>torch.autograd.grad</code> call and remove it from the <code>backward</code> call assuming you don’t want to backpropagate another time.</p>

Salman_Abbasi: <p>Thanks. But i do need to backpropage another time. Yes. this is solved. Thanks.</p>
<p>However for the last part<br>
v3.grad = ss.T<em>v3.grad <span class="hashtag-raw">#mutate</span> part<br>
I get the following error:<br>
<em>"tuple’ object has no attribute 'T</em></em>*"</p>
<p>What is the meaningful values of ss in this tuple?</p>
<p>Thanks</p>

ptrblck: <p><code>torch.autograd.grad</code> returns the sum of gradients of outputs with respect to the inputs as a <code>tuple</code>.<br>
If a single input is passed, the <code>tuple</code> will contain a single element:</p>
<pre data-code-wrap="python"><code class="lang-python">model = nn.Linear(10, 10)
x = torch.randn(1, 10)
out = model(x)

grads = torch.autograd.grad(out.mean(), model.bias)
grads
# (tensor([0.1000, 0.1000, 0.1000, 0.1000, 0.1000, 0.1000, 0.1000, 0.1000, 0.1000,
#          0.1000]),)
</code></pre>

Salman_Abbasi: <p>Thanks. Not sure if it is relevant to this thread. The ss tuple has dimensions 16,3,3,3 and 16.</p>
<p>I want to multiply ss with the dnn_Linear.parameters(), before optimizer.step()</p>
<p>Thanks</p>

ptrblck: <p>The grads <code>tuple</code> will correspond to the same order <code>dnn_Linear.parameters()</code> is returning the parameters, so you could iterate both and apply your update rule.</p>

Salman_Abbasi: <p>Thanks. It would help if there is an example of ss multiplying with dnn.Linear.parameters(). ss is tuple of size 2. Does that mean dnn.Linear.parameters() is also a tuple of same size? If yes, how can we multiply both before optimizer.step()</p>

ptrblck: <p>Something like this would work:</p>
<pre data-code-wrap="python"><code class="lang-python">model = nn.Linear(10, 10)
x = torch.randn(1, 10)
out = model(x)

grads = torch.autograd.grad(out.mean(), model.parameters())

for grad, param in zip(grads, model.parameters()):
    with torch.no_grad():
        param.sub_(grad)
</code></pre>
<p>but you won’t be able to use an optimizer anymore, since you are now manually manipulating the parameters, making the forward activations stale.</p>

Salman_Abbasi: <p>Thanks. I wonder what sub_ does here. What does param.sub_(grad) does?</p>
<p>Can I not do optimizer.step() after this? If not, how the learning rate would come into play?</p>

Salman_Abbasi: <p>Can you please explain more this step. thanks</p>

Salman_Abbasi: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , thanks for your help. The only thing, I would like to add here, is that I wanted to multiply, and yet double propagate.<br>
so my correct script after your help is<br>
Total_loss.backward(retain_graph=True)<span class="hashtag-raw">#back</span> propagate<br>
for grad, param in zip(grads, model.parameters()):</p>
<pre><code>    pp=param*grad # i am retaining graph, so inplace operation would not work
    param=pp
</code></pre>