Loss function for differentiable programming (targets computed from y_pred)

jhmenke: <p>My process looks similar to this: <a href="https://fluxml.ai/assets/2019-03-05-dp-vs-rl/trebuchet-flow.png" rel="nofollow noopener">https://fluxml.ai/assets/2019-03-05-dp-vs-rl/trebuchet-flow.png</a></p>
<p>I have some features and some labels. From these labels, i can invoke a custom function to check whether some target criteria match. I don’t care about the labels per se, only that the target criteria are met. Now, similar to the Trebuchet example, my ANN should infer the labels from the features.</p>
<p>My idea is to write a custom loss function to use the predicted features and then check the target criteria. The sum of the target criteria differences is the loss for the neural network.</p>
<p>In principle, the code works, but the model is not learning (loss is exactly the same in every epoch). Likely i’m breaking the graph by converting the labels to numpy, which i have to do in order to calculate the targets.</p>
<p><strong>Does anyone know how to handle this “numpy interruption” (without writing a custom backward)?</strong></p>
<p>Loss:</p>
<pre><code class="lang-auto">def solve_targets(x, y, ...)
    return some_numpy_calculations(x, y, ...)

class FLoss(nn.Module):
    def __init__(self, t, g, xsc, dvc):
        super(FLoss, self).__init__()
        self.targets = t
        self.device = dvc
        ...

    def forward(self, x, y):
        loss_np = solve_targets(x.cpu().data.numpy(), y.cpu().data.numpy())
        res_values = torch.tensor(loss_np, requires_grad=True, device=self.device)
        loss = torch.clamp_min_(res_values - self.targets, 0.).sum()
        print("loss", loss.item())
        return loss
</code></pre>

albanD: <p>Hi,</p>
<p>You should never use <code>.data</code> as a general rule <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>In this case, it break the computational graph and prevent gradients from being computed.<br>
Also the autograd won’t work if your perform your ops on numpy Tensors I’m afraid. You will either have to convert your code to use pytorch functions (which should be the simplest) or write a custom backward.</p>

jhmenke: <p>Thanks!</p>
<p>Unfortunately coding purely in torch is not possible, also computing the gradients seems to be impossible.<br>
Actually, an optimization problem is solved in the numpy function.</p>

albanD: <p>Well if there are no gradients, there is not much we can do <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>
<p>For the optimization problem, you have two choices:</p>
<ul>
<li>differentiate the whole process by rewriting it in pytorch</li>
<li>Find some conditions verified by the solution and input (like KKT) and use that to write a custom backward that will compute the gradients.</li>
</ul>

jhmenke: <p>Hmm yeah… thanks!</p>
<p>Another idea was to approximate the optimization with a second neural network, but that’s just an idea which makes it much more complicated.</p>

KFrank: <p>Hi Jan-Hendrik!</p>
<aside class="quote no-group" data-username="jhmenke" data-post="3" data-topic="78948" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jhmenke/48/23501_2.png" class="avatar"> jhmenke:</div>
<blockquote>
<p>Unfortunately coding purely in torch is not possible, also computing the gradients seems to be impossible.</p>
</blockquote>
</aside>
<p>It is not unreasonable to compute your gradients numerically<br>
(although suboptimal if you can compute them analytically).</p>
<p>So, in your forward pass you might compute:</p>
<pre data-code-wrap="python"><code class="lang-python">    loss_np = numpy_stuff (x, y)
    &lt;loop over x_i&gt;:
        xdelt_i = x_i + delta
        grad_i = (numpy_stuff (xdelt, y) - loss_np) / delta
    ...
</code></pre>
<p>Then store (or accumulate) your <code>grad_i</code> to use in a custom<br>
<code>backward()</code> function.</p>
<p>This is just a superficial sketch, and numerical differentiation is<br>
more delicate than indicated above, but the main point is that<br>
numerical differentiation can be a legitimate way to implement<br>
<code>backward()</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>Actually, an optimization problem is solved in the numpy function.</p>
</blockquote>
</aside>
<p>Note, that <em>some</em> optimization problems can be solved more<br>
quickly if you give them a good initial guess.  So you might be<br>
able to speed things up by reusing your initial solution when<br>
re-solving your optimization multiple times for the numerical<br>
differentiation.</p>
<p>(And, as Alban noted, <em>some</em> optimization algorithms naturally give<br>
you – or can be relatively cheaply modified to give you – the<br>
gradients you will need for your custom <code>backward()</code>.)</p>
<p>But yes, just to reiterate what Alban said:  You will have to<br>
implement a custom <code>backward()</code> one way or another (or<br>
implement your custom loss function entirely with differentiable<br>
pytorch tensor operations).</p>
<p>Best.</p>
<p>K. Frank</p>

jhmenke: <p>Thank you K. Frank for the good suggestions!<br>
I guess i can use finite differences for gradient approximation at the cost of longer runtimes, but it’s perfect for quickly validating my idea!</p>
<p>The optimization problem can theoretically be solved in torch, but that requires rewriting LOTS of code for just a simple experiment</p>