Backpropagation using Known Gradient Tensor

farbabi: <p>I am trying to reproduce the results reported in <a href="https://arxiv.org/abs/1608.05343" rel="noopener nofollow ugc">Decoupled Neural Interfaces using Synthetic Gradients</a> paper by PyTorch. The key idea in this paper is to approximate the gradient of the loss, using a synthetic gradient, gk, so that:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/c/bcf76ef5ea0e0a33466cac16e736de0ea99b93e8.png" alt="Screen Shot 2022-05-18 at 2.11.40 PM" data-base62-sha1="qXFT41G90i2LB7k0qhG0spPHtUI" width="264" height="73"></p>
<p>After calculating the gradient of the loss, I need to update the weights of the upstream layers. I am currently doing it this way:</p>
<p>output.backward(gradient_of_the_loss)</p>
<p>But I am not sure if this is correct and the documentation is not that helpful.</p>

KFrank: <p>Hi Faraz!</p>
<aside class="quote no-group" data-username="farbabi" data-post="1" data-topic="152006" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/farbabi/48/49743_2.png" class="avatar"> farbabi:</div>
<blockquote>
<p>output.backward(gradient_of_the_loss)</p>
</blockquote>
</aside>
<p>Yes, this will be correct, assuming that <code>output</code> is the <code>h_k</code><br>
and <code>gradient_of_the_loss</code> is the <code>g_k</code> in the expression<br>
you posted.</p>
<p>(One quibble about your expression:  The index <code>k</code> shouldn’t<br>
be repeated in <code>d h_k / d theta_k</code>.  I would have written<br>
<code>d L / d theta_l  ~  g_k d h_k / d theta_l</code>, with an<br>
implied sum over the index <code>k</code>)</p>
<p>Best.</p>
<p>K. Frank</p>

farbabi: <p>Thank you Frank for the response!</p>
<p>The output is h_k, but by gradient_of_the_loss I meant the right hand side of the expression, i.e.:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b3f02d619c92b33eb59f64486610ac51dd31c989.png" alt="Screen Shot 2022-05-18 at 10.47.12 PM" data-base62-sha1="pFO2LARZbn1JFPmzrgx1JFAo5q9" width="51" height="49">.<br>
Because I think the backward method needs the gradient tensor as the option, which is built by multiplying synthetic gradient and the derivatives of activations w.r.t. weights.</p>

KFrank: <p>Hi Faraz!</p>
<aside class="quote no-group" data-username="farbabi" data-post="3" data-topic="152006" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/farbabi/48/49743_2.png" class="avatar"> farbabi:</div>
<blockquote>
<p>Because I think the backward method needs the gradient tensor as the option,</p>
</blockquote>
</aside>
<p>I don’t understand what you are saying here.</p>
<p>With the exception of the duplicated index that I assume is a typo (see<br>
my previous post), I think that the</p>
<aside class="quote no-group">
<blockquote>
<p>output.backward(gradient_of_the_loss)</p>
</blockquote>
</aside>
<p>in your original post is correct.</p>
<p>The way I read the image of the equation in your original post, <code>L</code> is the<br>
scalar loss computed by applying a loss function to the predictions of your<br>
model.</p>
<p>The <code>theta_l</code> are the “upstream” parameters of that model.  (I use the<br>
index <code>l</code> rather than <code>k</code> to fix the duplicated-index typo.)</p>
<p>The <code>h_k</code> are the <code>output</code> of your model, that is, its predictions.</p>
<p><code>d L / d h_k</code> is the true gradient of the loss function with respect to the<br>
predictions, while <code>g_k</code> is the “synthetic” gradient that approximates the<br>
true gradient.</p>
<p>The equation you posted is just the (multi-dimensional) chain rule for<br>
computing the gradient of <code>L</code> with respect to <code>theta_l</code>.</p>
<p>Assuming that by <code>gradient_of_the_loss</code> you mean either <code>d L / d h_k</code><br>
(where, again, the <code>h_k</code> are the predictions made by the model) or its<br>
approximate, synthetic version, <code>g_k</code>, then whether you compute<br>
<code>gradient_of_the_loss</code> as the true or as the synthetic gradient and<br>
whether you compute it using autograd or by some other means, the way<br>
you get autograd to use the chain rule to complete the computation of<br>
<code>d L / d theta_l</code> is precisely by calling:</p>
<pre data-code-wrap="python"><code class="lang-python">output.backward (gradient_of_the_loss)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>