Backprop from Parameter.grad

username: seliad
cooked: <p><strong>Edit: clarification</strong>:<br>
The question is how to<br>
“ <strong>start</strong>  the autograd backprop from  <strong>accumulated gradient</strong>  instead of from loss or activation”</p>
<p>That is, we want  <code>dout/da</code>  , but we only have  <code>dout/dp</code>  (we do not have  <code>out</code> ), so we can’t compute it straightforward.</p>
<p>Mathematically:<br>
<code>dout/da = dout/dp * dp/da</code><br>
all we need is  <code>dp/da</code>  and we are done.</p>
<p>note1:  <code>a</code>  is arbitrary, can be an activation or another parameter.<br>
note2: we can create the computational graph only from  <code>a-&gt;p</code> .</p>
<hr>
<p><strong>Original post</strong>:</p>
<p>I know of two main ways to do backprop in pytorch:</p>
<ol>
<li>from loss: <code>loss.backward()</code>
</li>
<li>using gradients and activation with same size (or broadcast-able?) <code>torch.autograd.backward(x, g)</code>
</li>
</ol>
<p>Now, lets say we have some <code> p = torch.nn.Parameter()</code> for which some gradient (<code>p.grad</code>) was already accumulated, and we want to continue the back-propagation from it.<br>
Lets say we can construct the entire computational graph until p, and now we have it as some activation: <code>a=f(input)</code></p>
<p>Note that mathematically its very simple chain rule, just not sure how to do it in pytorch. (at least from python)</p>
<p>Any suggestions?<br>
Here is some pseudo code for the problem:</p>
<pre><code class="lang-auto">input=torch.tensor(...,requires_grad=True)
a=f(input)
p.grad = some_value_we_want # dout/dp
p.data = another_value_we_want 
</code></pre>
<p>How do we calculate dout/da in pytorch?</p>
<p>assuming<br>
<code>out = p ? f(input)</code> where <code>?</code> is some unknown function.<br>
(shape of <code>a</code> is not the same shape of <code>p.grad</code>)…</p>

username: Yaroslav_Bulatov
cooked: <p>You can get dout/da by callling a.retain_grad()</p>
<pre><code class="lang-auto">input = torch.ones(1)
f = torch.nn.Linear(1, 1, bias=False)
f.weight.data.copy_(torch.ones(1))
a = 2*f(input)
out = a*a/2
a.retain_grad()
out.backward()
assert a.grad[0] == 2.
assert f.weight.grad[0] == 4.
</code></pre>

username: seliad
cooked: <p>That did not answer the question.<br>
I asked something else…</p>

username: ptrblck
cooked: <p>Could you clarify your question?<br>
It seems you asked how to calculate <code>dout/da</code>, which <a class="mention" href="/u/yaroslav_bulatov">@Yaroslav_Bulatov</a> tried to answer.</p>

username: seliad
cooked: <p>Sure, I’ll clarify: the question is how to<br>
“<strong>start</strong> the autograd backprop from <strong>accumulated gradient</strong> instead of from loss or activation”</p>
<p>That is, we want <code>dout/da</code> , but we only have <code>dout/dp</code> (we do not have <code>out</code>), so we can’t compute it straightforward.</p>
<p>Mathematically:<br>
<code>dout/da = dout/dp * dp/da</code><br>
all we need is <code>dp/da</code> and we are done.</p>
<p>note1: <code>a</code> is arbitrary, can be an activation or another parameter.<br>
note2: we can create the computational graph only from <code>a-&gt;p</code>.</p>

username: Yaroslav_Bulatov
cooked: <p>Maybe something like this?</p>
<pre><code class="lang-auto">p.retain_grad()
out.backward()
dout_dp = p.grad
p.backward(dout_dp)
dout_da = a.grad
</code></pre>

username: seliad
cooked: <p>This is closer, but did not work.<br>
Reason is that <code>dout_dp</code> should be detached from the computation graph of <code>out</code> when connected to <code>a</code> and vice versa. (Mathematically its possible, we have all the information).<br>
Let me illustrate further with a code example:</p>
<pre><code class="lang-auto">import torch
m = 2
n = 3
batch = 4

a = torch.ones(batch, m)
a.retain_grad()
layer = torch.nn.Linear(m, n, bias=False)
layer.weight.data.copy_(torch.ones(n, m))

dout_dp = torch.ones(n, m)*5  # A **given** dout_dp, detached from `out` and `a`.
layer.weight.grad = dout_dp
v2 = layer(a)   # reconstruct a computation graph a-&gt;p

# Problem: `p.grad` is detached from the reconstructed computation graph of `a`
# so the backward below not going to work
torch.autograd.backward(list(layer.parameters()), 
                        [p.grad for p in layer.parameters()])
print(a.grad)  # None
</code></pre>
<p>I think that doing <code>p.backward(p.grad)</code> is an advancement and probably part of the future solution, but we are lacking a connection to the gradient, which must to be added artificially somehow:<br>
How do we tell pytorch that <code>a</code> has connection to <code>p.grad</code> (and figure that connection automatically from the new computation graph <code>a-&gt;p</code>)?</p>
<p>I see you did there <code>p.retain_grad()</code>, I guess this saves the “connection” from <code>a-&gt;p-&gt;out</code> which is what we want to avoid,<br>
as we want to start “clean” from the gradient and restored computation graph “a-&gt;p”.</p>
<p>From what I understand, it seems like autograd does not “use” parameter gradients (like <code>p.grad</code>) directly in its computation, but only <code>AccumulateGradients</code> to them.</p>

username: seliad
cooked: <p>Note we can’t really restore from accumulated gradients with <code>batch&gt;1</code> because we lose all the batch-related info.<br>
so the question should be for <code>batch=1</code>.<br>
Currently I did not find any way to do this (automatically) with Pytorch.</p>