How to implement a custom loss with a non-mathematical operation (simulation) that backpropagates with PyTorch?

zenia: <p>I am writing a Neural Network, which output is not used directly for the loss-function, but rather as the input for a simulation model. After the simulation ran, I am using the <strong>simulated_value</strong> and the <strong>real_value</strong> (target) to define pretty much a MSE-Loss.</p>
<p>In pseudo code my loss looks something like this:</p>
<pre><code>def simulation_loss(sim_params=nn_model_output, batch=batch):
     sim_value = RUN_SIMULATION(sim_params)
     sim_value.requires_grad = True
     real_value = batch['target']
     loss = nn.MSE(input=sim_value, target=real_value)
     return loss
</code></pre>
<p>It seems, that because I am cutting off the original gradient from <strong>nn_model_output</strong>, NN-model does not converge.</p>
<p>How can I run this sort of loss-function including a simulation model? Do I need to define a custom torch.autograd.Function with a backwards call? And if so, how should I pass the gradient, if there is no derivative from the simulation model possible? Or should I treat the simulation as an activation-function?</p>

ptrblck: <aside class="quote no-group" data-username="zenia" data-post="1" data-topic="182448">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zenia/48/61446_2.png" class="avatar"> zenia:</div>
<blockquote>
<p>It seems, that because I am cutting off the original gradient from <strong>nn_model_output</strong>, NN-model does not converge.</p>
</blockquote>
</aside>
<p>Yes, detaching a tensor from the computation graph and calling <code>.requires_grad = True</code> on the already detached tensor will not reattach it to the original computation graph.</p>
<aside class="quote no-group" data-username="zenia" data-post="1" data-topic="182448">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zenia/48/61446_2.png" class="avatar"> zenia:</div>
<blockquote>
<p>Do I need to define a custom torch.autograd.Function with a backwards call?</p>
</blockquote>
</aside>
<p>Yes, if your simulation model is not using differentiable PyTorch operations.</p>
<aside class="quote no-group" data-username="zenia" data-post="1" data-topic="182448">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zenia/48/61446_2.png" class="avatar"> zenia:</div>
<blockquote>
<p>And if so, how should I pass the gradient, if there is no derivative from the simulation model possible?</p>
</blockquote>
</aside>
<p>I don’t know why no derivative is possible, but in this case you might not be able to backpropagate and compute gradients at all.</p>
<aside class="quote no-group" data-username="zenia" data-post="1" data-topic="182448">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zenia/48/61446_2.png" class="avatar"> zenia:</div>
<blockquote>
<p>Or should I treat the simulation as an activation-function?</p>
</blockquote>
</aside>
<p>All activation functions are differentiable, so also unclear how to understand this statement.</p>

zenia: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="182448">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>I don’t know why no derivative is possible, but in this case you might not be able to backpropagate and compute gradients at all.</p>
</blockquote>
</aside>
<p>The simulation model is a Finite-Element-Model running separately in MatLab. I am not quite sure, how a feasible derivative of the simulation model might look like. So, does that mean, this kind of combined computation between NN &amp; Simulation Model is not possible?</p>

ptrblck: <p>I wouldn’t claim it’s impossible, but you would still need to figure out how the backward pass could be calculated. In the end you need to pass gradients to the simulation model (e.g. from the loss) and backpropagate it to the trainable PyTorch model.<br>
You could try to “estimate” these gradients somehow if a direct computation is not possible, but the gradient has to flow through the simulation model somehow (unless you don’t want to backpropagate at all and want to use another optimization method).</p>

zenia: <p>I could use the discretized output from the simulation, use it to calculate through the difference the slope and therefore a simplified derivative. This might work.<br>
As this is the first time I am writing such a custom torch.autograd.Function, can you please give me a rough structure of the backwards call? Thank you very much!</p>

ptrblck: <p><a href="https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html">This tutorial</a> shows you how the <code>backward</code> is implemented.<br>
The <code>backward</code> method expects a <code>grad_output</code> tensor, representing the incoming gradient w.r.t. the layer’s output, calculates a new gradient w.r.t. its inputs, and returns it.</p>