How to calculate / get intermediate results of backpropagation

codeflux: <p>I’ve found this one example which illustrates my problem:</p>
<pre><code class="lang-auto">import torch
from torch.autograd import grad
import torch.nn as nn


# Create some dummy data.
x = torch.ones(2, 2, requires_grad=True)
gt = torch.ones_like(x) * 16 - 0.5  # "ground-truths"

# We will use MSELoss as an example.
loss_fn = nn.MSELoss()

# Do some computations.
v = x + 2
y = v ** 2

# Compute loss.
loss = loss_fn(y, gt)

print(f'Loss: {loss}')

# Now compute gradients:
d_loss_dx = grad(outputs=loss, inputs=x)

d_loss_dv = grad(outputs=loss, inputs=v)

print(f'dloss/dx:\n {d_loss_dx}')

</code></pre>
<p>With this approach, I can calculate dloss / dx using the grad function, but when the function is called a second time to calculate dloss / dv (which we just assume, I need the result for) I get the Runtime Errror:</p>
<pre><code class="lang-auto">
RuntimeError: Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward.
</code></pre>
<p>Probably due to the fact that backpropagation was already called once, and we would not want to do that again, I guess?</p>
<p>So my question is, is there a better way to do what I try to achieve above, calculating dloss_dx and d_loss_dv?</p>
<p>Thanks in advance!</p>

googlebot: <p>just pass a list of tensors as “inputs”</p>

anantguptadbl: <p><a class="mention" href="/u/codeflux">@codeflux</a><br>
Another alternative would be to use</p>
<pre><code class="lang-auto">retain_graph=True
</code></pre>
<pre><code class="lang-auto"># Now compute gradients:
d_loss_dx = grad(outputs=loss, inputs=x, retain_graph=True)
d_loss_dv = grad(outputs=loss, inputs=v, retain_graph=True)
d_loss_dy = grad(outputs=loss, inputs=y)

print(f'dloss/dx:\n {d_loss_dx}')
print(f'dloss/dv:\n {d_loss_dv}')
print(f'dloss/dy:\n {d_loss_dy}')
</code></pre>

codeflux: <p>Ok thanks, that seems to work!<br>
But I cannot calculate intermediate variables at the same time.<br>
Let’s say dy/dv in this example.</p>

codeflux: <p>Ok, perfect, this basically  does what  I want! I wonder how many times backpropagation is used here, though?</p>

anantguptadbl: <p><a class="mention" href="/u/codeflux">@codeflux</a> backward propagation ( gradient update ) is done everytime you pass a new variable. retain_graph just ensures that the computation graph is not cleared and retains its values. This helps us in performing backward propagation once more</p>

googlebot: <aside class="quote no-group" data-username="codeflux" data-post="4" data-topic="141869">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/e19b73/48.png" class="avatar"> codeflux:</div>
<blockquote>
<p>But I cannot calculate intermediate variables at the same time.<br>
Let’s say dy/dv in this example.</p>
</blockquote>
</aside>
<p>Try v.retain_grad() (this should populate v.grad)</p>

codeflux: <p>It does give me a grad, but in respect to what?</p>

codeflux: <p>I don’t think it has to backpropagate again though, since the dy / dx for example is already calculated in the first backpropagation call (but not saved).</p>

anantguptadbl: <p><a class="mention" href="/u/codeflux">@codeflux</a> Agree. Unless you re-calculate the loss , the grads will always be same.</p>

codeflux: <p>Then I am a bit confused as to what grad actually does.<br>
My problem is this: I want to update the weights of the network manually, using a <em>part</em> of the loss function derived in respect to the weights itself.</p>
<p>Now I am confused what it uses as a loss function to do backpropagation, when using grad. Probably just the output function?</p>

anantguptadbl: <p><a class="mention" href="/u/codeflux">@codeflux</a></p>
<ul>
<li>
<p>In your case there are three variables x, y and v</p>
</li>
<li>
<p>x is the foremost input and y is the final output. v is an intermediate calc</p>
</li>
<li>
<p>In an ideal scenario, if you have a single value for x ( e.g. x=5 ) ,the values for v and y are fixed ( immutable ) and therefore the grads are fixed for a single loss backward propagation</p>
</li>
<li>
<p>grad is designed to start from the calculated loss(which is a scalar value) and then propagate backwards for each variable, which you have already seen. If the loss is the same, the grads will also be the same</p>
</li>
<li>
<p>If you want to alter the grads, you need to register hooks so that instead of the normal grads propagating backwards, you have your own modified grads propagating</p>
</li>
</ul>
<p><a href="https://pytorch.org/docs/stable/generated/torch.Tensor.register_hook.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.Tensor.register_hook.html</a></p>
<p><strong>WITHOUT HOOK</strong></p>
<pre><code class="lang-auto">x = torch.ones(2, 2, requires_grad=True)
gt = torch.ones_like(x) * 16 - 0.5  # "ground-truths"
# We will use MSELoss as an example.
loss_fn = nn.MSELoss()
# Do some computations.
v = x + 2
y = v ** 2

# Compute loss.
loss = loss_fn(y, gt)
d_loss_dx = grad(outputs=loss, inputs=x, retain_graph=True)
d_loss_dv = grad(outputs=loss, inputs=v, retain_graph=True)
d_loss_dy = grad(outputs=loss, inputs=y)

print(f'dloss/dx:\n {d_loss_dx}')
print(f'dloss/dv:\n {d_loss_dv}')
print(f'dloss/dy:\n {d_loss_dy}')
</code></pre>
<pre><code class="lang-auto">dloss/dx:
 (tensor([[-19.5000, -19.5000],
        [-19.5000, -19.5000]]),)
dloss/dv:
 (tensor([[-19.5000, -19.5000],
        [-19.5000, -19.5000]]),)
dloss/dy:
 (tensor([[-3.2500, -3.2500],
        [-3.2500, -3.2500]]),)
</code></pre>
<p><strong>WITH HOOK</strong></p>
<pre><code class="lang-auto">x = torch.ones(2, 2, requires_grad=True)
gt = torch.ones_like(x) * 16 - 0.5  # "ground-truths"
# We will use MSELoss as an example.
loss_fn = nn.MSELoss()
# Do some computations.
v = x + 2
y = v ** 2

h = v.register_hook(lambda grad: grad * 2)

# Compute loss.
loss = loss_fn(y, gt)
d_loss_dx = grad(outputs=loss, inputs=x, retain_graph=True)
d_loss_dv = grad(outputs=loss, inputs=v, retain_graph=True)
d_loss_dy = grad(outputs=loss, inputs=y)

print(f'dloss/dx:\n {d_loss_dx}')
print(f'dloss/dv:\n {d_loss_dv}')
print(f'dloss/dy:\n {d_loss_dy}')

h.remove()
</code></pre>
<pre><code class="lang-auto">dloss/dx:
 (tensor([[-39., -39.],
        [-39., -39.]]),)
dloss/dv:
 (tensor([[-19.5000, -19.5000],
        [-19.5000, -19.5000]]),)
dloss/dy:
 (tensor([[-3.2500, -3.2500],
        [-3.2500, -3.2500]]),)
</code></pre>
<p>There are different variations of applying hook, and you can choose based on what suits you</p>

googlebot: <aside class="quote no-group" data-username="codeflux" data-post="8" data-topic="141869" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/e19b73/48.png" class="avatar"> codeflux:</div>
<blockquote>
<p>It does give me a grad, but in respect to what?</p>
</blockquote>
</aside>
<p>End node (dloss/dv). In other words, it is a vector-Jacobian product <strong>1</strong> @ J_loss_y @ J_y_v.</p>
<p>These Jacobians may be diagonal, but they’re not materialized / exported anyway. So, dy/dv=dPow(v,2)/dv=2v is not normally accessible (outside of PowBackward)</p>
<p>Last version has “experimental forward mode AD” that may facilitate this, but I haven’t looked into it yet.</p>