Custom Loss Function when using modules separate from Torch

username: redamit
cooked: <p>I’m currently working on implementing a GAN, with a custom loss function which uses other Python modules and libraries. The input to the loss function is the output of GAN which is torch.tensor of size [8,3,256,256]</p>
<p>The forward path of the loss function is as follows:</p>
<ol>
<li>Convert torch.tensor to PIL.Image and normalize it</li>
<li>Detect faces in the PIL.Image using face_detector and output landmarks</li>
<li>Use these landmarks to calculate torch.nn.MSELoss()</li>
<li>Once loss is calculated return it as loss to the original function</li>
</ol>
<p>I’m able to implement the forward path with no particular issue. I don’t want to calculate gradients for this task but want to use this loss to back propagate GAN whose output I’m using to input to the loss function.</p>
<p>I’ve no particular priority for face_detector but would love to use any in general.<br>
I’m trying to use almost nothing from python numpy, the only thing in numpy are the landmarks which are output of the face_detector.</p>
<p>If I add requires_grad=True while returning the loss, I’m able to perform a backward call but there are no gradients which are calculated.</p>
<p>Is there a way in which I’m able to accomplish the task of using this loss to back propagate through the GAN?</p>

username: KFrank
cooked: <p>Hello redamit!</p>
<aside class="quote no-group quote-modified" data-username="redamit" data-post="1" data-topic="97045" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/redamit/48/12404_2.png" class="avatar"> redamit:</div>
<blockquote>
<p>I’m currently working on implementing a GAN, with a custom loss function which uses other Python modules and libraries.<br>
…<br>
I’m able to implement the forward path with no particular issue. I don’t want to calculate gradients for this task but want to use this loss to back propagate GAN whose output I’m using to input to the loss function.<br>
…<br>
Is there a way in which I’m able to accomplish the task of using this loss to back propagate through the GAN?</p>
</blockquote>
</aside>
<p>No, not really.</p>
<p>If I understand you correctly, even though you “don’t want to calculate<br>
gradients for” the parameters used by the face detector in your loss<br>
function, you <em>do</em> want to calculate gradients for the parameters of<br>
the GAN used to generate the image fed to the face detector.</p>
<p>Once you start to pass the generated image through non-pytorch<br>
calculations, you “break the computation graph,” and won’t be able<br>
to backpropagate from one side of the non-pytorch calculations to<br>
the other.</p>
<p>Your best bet will to be to implement the face detector in pytorch (if<br>
it isn’t already) and figure out how to implement your landmark<br>
calculations using <strong>differentiable</strong> pytorch operations.  If the whole<br>
end-to-end calculation – GAN image generation, face detection,<br>
landmark calculations, and final loss function – is done with (differentiable)<br>
pytorch tensor operations, pytorch’s autograd will work, you can call<br>
<code>loss.backward()</code>, and use the GAN-parameter gradients produced by<br>
autograd to optimize your GAN.</p>
<p>(Other approaches are theoretically possible, but likely to be unworkable.)</p>
<p>Good luck.</p>
<p>K. Frank</p>

username: redamit
cooked: <p>Hi KFrank, Thanks for your response.</p>
<p>Most probably I will implement or use some face-detector in PyTorch, cause any other method may not be implemented properly.</p>
<p>If that doesn’t work is there any other way where I can use the gradients calculated before the face-detector and implement those in the Custom Losses Backward Function?</p>
<p>Thanks.</p>