Indirect Loss Computation and Backpropagation

Md_Salman_Sakib_1900: <p>I have a question regarding how the weight updates are working in my neural network implementation.</p>
<p>In my setup, the neural network takes two inputs and produces an output of 14 values from 14 neurons in a single layer. This output is then used as part of a fixed iterative algorithm. The loss is calculated based on the result of this iterative algorithm and I update the weights using: loss.backward()<br>
optimizer.step()</p>
<p>This approach successfully updates the network weights and I am getting the desired output for the downstream task.</p>
<p>However, my confusion is about how the loss is actually connected to the neural network. There is no direct loss function applied to the raw output of the network; instead, the output feeds into a separate algorithm and the loss is computed from that.</p>
<p>Given this indirect setup, is this considered an indirect training approach for the neural network? How exactly does the gradient flow back through the iterative algorithm to update the weights correctly? Any guidance or insights on whether this is a standard or valid way to connect the loss to the network would be greatly appreciated.</p>

KFrank: <p>Hi Salman!</p>
<aside class="quote no-group" data-username="Md_Salman_Sakib_1900" data-post="1" data-topic="221648" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/md_salman_sakib_1900/48/77720_2.png" class="avatar"> Md_Salman_Sakib_1900:</div>
<blockquote>
<p>How exactly does the gradient flow back through the iterative algorithm to update the weights correctly?</p>
</blockquote>
</aside>
<p>Assuming your iterative algorithm is implemented with pytorch tensor operations. autograd<br>
will build the computation graph as you perform the iterations and then, when you call<br>
<code>.backward()</code>, backpropagate through that graph to compute the gradients.</p>
<p>Every time your algorithm executes an iteration, autograd will add more nodes to the<br>
computation graph.  If you iterate a very large number of times, your computation graph<br>
could become impractically deep, but there is nothing wrong with the logic or the basic<br>
approach.</p>
<p>Here’s an example script with a simple square-root iteration:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

def it_sqrt (x):
    result = torch.ones (1)
    for  _ in range (5):   # more iterations will give greater accuracy
        result = (result + x / result) / 2
    return  result

x1 = torch.tensor ([100.0], requires_grad = True)
x2 = torch.tensor ([100.0], requires_grad = True)

r1 = it_sqrt (x1)
r1.backward()
print ('x1:', x1)
print ('r1:', r1)
print ('x1.grad:', x1.grad)

r2 = x2.sqrt()
r2.backward()
print ('x2:', x2)
print ('r2:', r2)
print ('x2.grad:', x2.grad)
</code></pre>
<p>And here is its output:</p>
<pre><code class="lang-plaintext">2.7.1+cu128
x1: tensor([100.], requires_grad=True)
r1: tensor([10.0326], grad_fn=&lt;DivBackward0&gt;)
x1.grad: tensor([0.0512])
x2: tensor([100.], requires_grad=True)
r2: tensor([10.], grad_fn=&lt;SqrtBackward0&gt;)
x2.grad: tensor([0.0500])
</code></pre>
<aside class="quote no-group">
<blockquote>
<p>Any guidance or insights on whether this is a standard or valid way to connect the loss to the network would be greatly appreciated.</p>
</blockquote>
</aside>
<p>This is a perfectly legitimate approach that autograd is designed to handle.</p>
<p>Your cost in computation time will scale with the number of iterations.  Note, however, that<br>
even if the underlying iterative algorithm has a constant cost in memory usage, autograd’s<br>
memory usage will also scale with the number of iterations as it adds nodes and intermediate<br>
tensors to the computation graph.</p>
<p>Best.</p>
<p>K. Frank</p>

Md_Salman_Sakib_1900: <p>Hi KFrank,</p>
<p>I have a follow-up question about our discussion:</p>
<p>When I run the fixed iterative algorithm, I feed it the 14-dimensional output produced by my network and then compute a loss by comparing the algorithm’s final result to the ground truth. Although I understand that autograd will propagate gradients back through every differentiable step, I’m having trouble seeing the direct relationship between the loss and the network’s 14 outputs; there’s no explicit loss term applied to those values themselves.</p>
<p>The loss is computed from the solver’s final result, not directly from the network’s 14-dimensional output. Could you clarify how, mathematically, that scalar loss is connected back to those 14 values? I understand it’s an “indirect” supervision, but I’m struggling to see where the explicit gradient path ties the loss to the network’s raw output. and Although the solver uses the parameters generated by the network, the algorithm itself never changes. Why doesn’t any part of the iterative solver get updated during training, even though its behavior influences the loss?</p>
<p>I appreciate any detail you can provide on how the loss function hooks into the network outputs</p>

KFrank: <p>Hi Salman!</p>
<aside class="quote no-group" data-username="Md_Salman_Sakib_1900" data-post="3" data-topic="221648">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/md_salman_sakib_1900/48/77720_2.png" class="avatar"> Md_Salman_Sakib_1900:</div>
<blockquote>
<p>I’m having trouble seeing the direct relationship between the loss and the network’s 14 outputs; there’s no explicit loss term applied to those values themselves.</p>
</blockquote>
</aside>
<p>As I understand it, you have something you call your “model” that outputs a result with shape,</p>
<p>say, <code>[14]</code>.  You then have an iterative algorithm that I will call your “post-processor.”  It takes</p>
<p>an input of shape <code>[14]</code> and let’s say, just for fun, that it produces an output of shape <code>[7]</code>.</p>
<p>(I’m guessing that your post-processor actually outputs shape <code>[14]</code>, but the logic is the same</p>
<p>either way.)  Let’s say (using my output shape) that your ground truth has shape <code>[7]</code>and you</p>
<p>use <code>MSELoss</code> as your loss criterion.</p>
<p>So something like:</p>
<pre><code class="lang-auto">result1 = model (input)
result2 = post_processor (result1)
loss_for_backward = MSELoss() (result2, ground_truth)

</code></pre>
<p>Autograd, itself, doesn’t have a notion of “model” or “loss criterion” or, for that matter,</p>
<p>“post-processor” – it doesn’t care one way or the other.  As far as autograd is concerned, these</p>
<p>are all just sequences of tensor operations (that it tracks).</p>
<p>Conceptually, you might choose to think of post-processor as being part of your model.  Or</p>
<p>maybe it makes more sense to you to think of post-processor as part of your loss criterion.</p>
<p>Both ways of looking at it are fine and autograd doesn’t care.</p>
<p>The output of your loss criterion depends on the parameters of your model.  (I would say</p>
<p>that it depends “directly” on those parameters, but that’s semantics.)</p>
<p>Consider:</p>
<pre><code class="lang-auto">loss = MSELoss() (model (input), ground_truth)       # or
loss = MSELoss() (2 * model (input), ground_truth)   # or
loss = 2 * MSELoss() (model (input), ground_truth)
loss.backward()

</code></pre>
<p>In each case autograd backpropagates through a sequence of tensor operations in order to</p>
<p>populate <code>model</code>’s parameters with gradients of <code>loss</code>.  In the second and third versions, one</p>
<p>of those tensor operations is multiplication-by-two.  In all three cases, I would say that <code>loss</code></p>
<p>depends directly on the output of <code>model</code> and on <code>model</code>’s parameters (but choosing to say</p>
<p>“directly” is a matter of semantics).  As far as autograd is concerned the post-processor is no</p>
<p>different than the multiplication-by-two – it’s just some more tensor operations in the sequence</p>
<p>of tensor operations through which it backpropagates.</p>
<aside class="quote no-group" data-username="Md_Salman_Sakib_1900" data-post="3" data-topic="221648">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/md_salman_sakib_1900/48/77720_2.png" class="avatar"> Md_Salman_Sakib_1900:</div>
<blockquote>
<p>Why doesn’t any part of the iterative solver get updated during training, even though its behavior influences the loss?</p>
</blockquote>
</aside>
<p><em>If</em> the iterative solver contained some trainable parameters (leaf variables with</p>
<p><code>requires_grad = True</code>), then autograd would populate their gradients during backpropagation.</p>
<p><em>If</em> those parameters were being controlled by your <code>optimizer</code>, then <code>optimizer.step()</code></p>
<p>would update those parameters based on their gradients so that the result of the iterative</p>
<p>solver would change.</p>
<p>(Presumably your iterative solver doesn’t contain trainable parameters, just like the <code>2</code> in the</p>
<p>multiplication-by-two example isn’t a trainable parameter.)</p>
<p>Best.</p>
<p>K. Frank</p>