Custom loss function leading to loss of gradients

niceonede: <p>I need to implement a custom loss function for a regression problem. The data that I have are one dimensional and each has a length of 1000. In total I have 100.000 (samples) of these 1000 (features). Each sample (which is a data of length 1000) is connected to 4 different parameters (a,b,m,s). The connection between the samples and parameters are known for the training data as usually.</p>
<p>I need to develop a deep learning model which will be able to output 4 parameters (a2,b2,m2,s2) given a vector of length 1000. The custom function that I need to define on Pytorch is then</p>
<pre><code>                           loss =Integral (function(a,b,m,s,a2,b2,m2,s2))
</code></pre>
<p>The problem that I have is that I am not able to calculate this loss function inside “torch” framework using for example “torch.quad”, because such code does not exist in Pytorch. I decided to define a custom loss function using “quad” but this returns a float number. After I get this float number I can convert it to tensor. But the problem is that I am doing this at each iteration of the loop and the gradients are not evolving in time, rather they are initialized at each iteration again and again.</p>
<p>Here is the portion of my code (<a href="https://github.com/locuslab/TCN" rel="noopener nofollow ugc">it is modified from1D convolutional nets</a>):</p>
<pre><code>def train(ep):
    model.train()
    total_loss = 0
    count = 0
    train_idx_list = np.arange(len(X_train), dtype="int32")
    np.random.shuffle(train_idx_list)

    for idx in train_idx_list: 
         data_line = X_train[idx]
         x = Variable(data_line)
         if args.cuda:
            x = x.cuda()

    optimizer.zero_grad()
    output = model(x.unsqueeze(0)).squeeze(0)
               
    a = (parameter1[idx]*40.0000)-20.0000
    b = parameter2[idx]*10
    m= parameter3[idx]*4.8063e-07
    s= parameter4[idx]*3.8284e-07

    a2=(output[0][0].item()*40.0000)-20.0000
    b2=output[0][1].item()*10
    m2=output[0][2].item()*4.8063e-07
    s2=output[0][3].item()*3.8284e-07
    
    loss = torch.tensor(quad(integrand, 0, 5*10**-7, args=(a[0],b[0],m[0],s[0],a2,b2,m2,s2), points=(max(0,min(m[0]-2*s[0],m2-2*s2)), max(m[0]+2*s[0],m2+2*s2)))[0], requires_grad=True, device=cuda0)
    total_loss += loss.item()
    count +=1# output.size(0)

    if args.clip &gt; 0:
        torch.nn.utils.clip_grad_norm_(model.parameters(), args.clip)
    loss.backward()
    optimizer.step()
    if idx &gt; 0 and idx % args.log_interval == 0:
        cur_loss = total_loss / count
        print("Epoch {:2d} | lr {:.5f} | loss {:.5f}".format(ep, lr, cur_loss))
        total_loss = 0.0
        count = 0
</code></pre>
<p>The integrand is defined as follows:</p>
<pre><code>   def integrand(y,a,b,m,s,a2,b2,m2,s2): 

   return abs(((1/2)*b*math.exp(1)**((-1)*(s**(-1))**b*abs((-1)*m+y)**b)*s**(-1)*math.erfc((-1)*2**(-1/2)*a*s**(-1)*((-1)*m+y))*math.gamma(b**(-1))**(-1))-((1/2)*b2*math.exp(1)**((-1)*(s2**(-1))**b2*abs((-1)*m2+y)**b2)*s2**(-1)*math.erfc((-1)*2**(-1/2)*a2*s2**(-1)*((-1)*m2+y))*math.gamma(b2**(-1))**(-1)));
</code></pre>
<p>How can one deal with such a problem?</p>

KFrank: <p>Hi Seyhmus!</p>
<aside class="quote no-group" data-username="niceonede" data-post="1" data-topic="132473" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/d2c977/48.png" class="avatar"> niceonede:</div>
<blockquote>
<p>I am not able to calculate this loss function inside “torch” framework using for example “torch.quad”, because such code does not exist in Pytorch. I decided to define a custom loss function using “quad” but this returns a float number.</p>
</blockquote>
</aside>
<p>Pytorch’s autograd cannot compute gradients for computations that<br>
are performed outside of the pytorch framework and you will not be<br>
able to backpropagate through them – unless you give them some<br>
help.</p>
<p>You will need to package your <code>quad()</code>  computation as a <code>Function</code><br>
and provide it with a <code>backward()</code> function that computes its gradient.</p>
<p>See <a href="https://pytorch.org/docs/1.9.0/notes/extending.html" rel="noopener nofollow ugc">Extending <code>torch.autograd</code></a> for a start.</p>
<aside class="quote no-group">
<blockquote>
<pre><code>   def integrand(y,a,b,m,s,a2,b2,m2,s2): 

   return abs(((1/2)*b*math.exp(1)**((-1)*(s**(-1))**b*abs((-1)*m+y)**b)*s**(-1)*math.erfc((-1)*2**(-1/2)*a*s**(-1)*((-1)*m+y))*math.gamma(b**(-1))**(-1))-((1/2)*b2*math.exp(1)**((-1)*(s2**(-1))**b2*abs((-1)*m2+y)**b2)*s2**(-1)*math.erfc((-1)*2**(-1/2)*a2*s2**(-1)*((-1)*m2+y))*math.gamma(b2**(-1))**(-1)));
</code></pre>
</blockquote>
</aside>
<p>I haven’t looked carefully at <code>integrand()</code> and the underlying math,<br>
but it would seem that one possibility could be the following:</p>
<p>You can switch the order of differentiation and integration (provided<br>
things converge appropriately), so you could analytically differentiate<br>
<code>integrand()</code>, code up that formula, and then integrate it numerically<br>
(using your <code>quad()</code> or something similar) and use that as the main<br>
ingredient in the <code>backward()</code> function of your custom autograd<br>
<code>Function</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

niceonede: <p>Hello Frank,<br>
Thank you very much for your response. I was able to write a function as follows:</p>
<pre><code>from TCN.poly_music.myintegrate2 import integrand, integrand_alpha, integrand_beta, integrand_mu, integrand_sigma
from scipy.integrate import quad 
import torch

class my_loss(torch.autograd.Function):
    @staticmethod
    def forward(ctx,a,b,m,s,a2,b2,m2,s2):
        result = quad(integrand, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        ctx.save_for_backward(a,b,m,s,a2,b2,m2,s2)
        return torch.FloatTensor([result])


    @staticmethod
    def backward(ctx, grad_output):
        a,b,m,s,a2,b2,m2,s2 = ctx.saved_variables
        grad_alpha = quad(integrand_alpha, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        grad_beta = quad(integrand_beta, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        grad_mu = quad(integrand_mu, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        grad_sig = quad(integrand_sigma, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
            
    
        return torch.FloatTensor(grad_alpha), torch.FloatTensor(grad_beta), torch.FloatTensor(grad_mu), torch.FloatTensor(grad_sig), None
</code></pre>
<p>In this function integrand, integrand_alpha, integrand_beta, integrand_mu and integrand_sigme are function of this sort:</p>
<pre><code>def integrand(y,a,b,m,s,a2,b2,m2,s2): 

     return abs(((1/2)*b*math.exp(1)**((-1....)
</code></pre>
<p>And the  first values that I get are as follows:</p>
<pre><code>     grad_alpha = 0.210814
     grad_beta = -0.000110
     grad_mu =   563428.64
     grad_sig =   3272315.65
</code></pre>
<p>In the main program I used this one:</p>
<pre><code>    loss = my_loss.apply(a[0],b[0],m[0],s[0],a2,b2,m2,s2)
</code></pre>
<p>Here the loss is of this sort:</p>
<pre><code>    tensor([1.1734])
</code></pre>
<p>The rest of the code is the same as I had written in the original question. I got an error as this one:</p>
<pre><code>    element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>Later I checked in Internet and I saw that this error was related to no gradients in the tensor and a quick fix would be to use Variable after the definition of loss.</p>
<pre><code>        loss = my_loss.apply(a[0],b[0],m[0],s[0],a2,b2,m2,s2)
        loss = Variable(loss, requires_grad = True)
</code></pre>
<p>This lets the code run but the results show that the loss function does not decrease</p>
<pre><code> Epoch  1 | lr 0.00100 | loss 1.62780
 Epoch  1 | lr 0.00100 | loss 1.63068
 Epoch  1 | lr 0.00100 | loss 1.61624
 Epoch  1 | lr 0.00100 | loss 1.62332
 Epoch  1 | lr 0.00100 | loss 1.62789
 Epoch  1 | lr 0.00100 | loss 1.64436
</code></pre>
<p>I think there is something missing about the Function that I defined, else I should have seen the loss function decreasing. To be on the safe side, I also checked the code for a single parameter and without custom loss function, i.e. with MSE loss that I defined over torch. This worked very well and gave some expected results. I calculated the gradients with Mathematica and tested them in Python and got the same results with Matlab. So the gradients should be OK.</p>
<p>What should then be missing with my Function or the appliaction of it in the main code?</p>

KFrank: <p>Hi Seyhmus!</p>
<aside class="quote no-group" data-username="niceonede" data-post="3" data-topic="132473" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/d2c977/48.png" class="avatar"> niceonede:</div>
<blockquote>
<p>In the main program I used this one:</p>
<pre><code>    loss = my_loss.apply(a[0],b[0],m[0],s[0],a2,b2,m2,s2)
</code></pre>
<p>Here the loss is of this sort:</p>
<pre><code>    tensor([1.1734])
</code></pre>
<p>The rest of the code is the same as I had written in the original question. I got an error as this one:</p>
<pre><code>    element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
</blockquote>
</aside>
<p>These three pieces of information strongly suggest that <em>none</em> of<br>
the arguments you are passing into <code>my_loss.apply()</code> have<br>
<code>requires_grad = True</code>.  Try printing out the arguments right<br>
before calling <code>my_loss.apply()</code> to see whether they show up<br>
with <code>requires_grad = True</code>.</p>
<p>Looking at your code – and making some assumptions to fill in the<br>
gaps – <code>a</code>, <code>b</code>, etc., come from <code>parameter1</code>, <code>parameter2</code>, etc., so I<br>
have no reason to think that they have <code>requires_grad = True</code>.<br>
Furthermore, even though <code>a2</code>, <code>b2</code>, etc., come from <code>output</code>, which,<br>
as the result of <code>model</code>, presumably has <code>requires_grad = True</code>,<br>
you then call <code>.item()</code>, which returns a python scalar (rather than a<br>
tensor), and, as such, strips away any <code>requires_grad</code> property.</p>
<p>If you are trying to optimize the parameters of the <code>model</code> that<br>
produces <code>a2</code>, <code>b2</code>, etc., then you have to move these calls to <code>.item()</code>.<br>
inside of <code>my_loss.forward()</code>.  Pass <code>a2</code>, etc., into <code>my_loss.apply()</code>  as<br>
python tensors (and do check that they have <code>requires_grad = True</code><br>
to make sure that something else isn’t getting broken) and convert them<br>
to python scalars (in order to pass them to <code>quad()</code>) with calls to <code>.item()</code><br>
inside of <code>forward()</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>a quick fix would be to use Variable after the definition of loss.</p>
</blockquote>
</aside>
<p>As a general rule, if you find yourself tempted to “manually” turn on<br>
<code>requires_grad = True</code> somewhere in the middle of a calculation,<br>
you should resist that temptation.  Such a “quick fix” is almost never<br>
an actual fix.  Typically, you have “broken the computation graph”<br>
somewhere upstream of your purported fix, and you need find and<br>
fix the actual, upstream problem.  (As noted above, in your case, the<br>
calls to <code>.item()</code> are likely the culprit that has broken the computation<br>
graph.)</p>
<p>As an aside, <code>Variable</code> has been deprecated for some time now, and<br>
the autograd functionality in <code>Variable</code> has since been subsumed into<br>
pytorch tensors, so just use regular tensors.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">    @staticmethod
    def backward(ctx, grad_output):
        a,b,m,s,a2,b2,m2,s2 = ctx.saved_variables
        grad_alpha = quad(integrand_alpha, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        grad_beta = quad(integrand_beta, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        grad_mu = quad(integrand_mu, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
        grad_sig = quad(integrand_sigma, 0, 5*10**-7, args=(a,b,m,s,a2,b2,m2,s2), points=(max(0,min(m-2*s,m2-2*s2)), max(m+2*s,m2+2*s2)))[0]
                
        return torch.FloatTensor(grad_alpha), torch.FloatTensor(grad_beta), torch.FloatTensor(grad_mu), torch.FloatTensor(grad_sig), None
</code></pre>
</blockquote>
</aside>
<p>Your version of <code>backward()</code> has a flaw.  For reasons I won’t go into,<br>
it should work in the particular context in which you call it, but it’s<br>
wrong because its output does not depend on its <code>grad_output</code><br>
argument.</p>
<p>Let’s assume for simplicity that you write a <code>Function</code> whose <code>forward()</code><br>
method takes a single tensor as an argument and returns a single<br>
tensor.  That is, it takes a collection of scalar arguments and returns a<br>
collection of scalar results.  The derivative of such a function is the<br>
so-called <em>Jacobian,</em> that is, the matrix of partial derivatives of all of<br>
the results with respect to all of the arguments.</p>
<p>Your <code>backward()</code> function – which we think of as calculating the<br>
derivative of the corresponding <code>forward()</code> –  doesn’t return the full<br>
Jacobian (although it might choose to compute the full Jacobian<br>
internally).  Rather it returns the matrix-vector product of the Jacobian<br>
matrix with the <code>grad_output</code> vector that pytorch’s autograd framework<br>
passes into your <code>backward()</code> function.</p>
<p>For your <code>my_loss</code> <code>Function</code> to work correctly in (perfectly plausible)<br>
more general contexts, its <code>backward()</code> method can’t simply ignore<br>
its <code>grad_output</code> argument, but instead has to use it to compute (and<br>
return) the expected Jacobian-<code>grad_output</code> product.</p>
<p>(The Jacobian-<code>grad_output</code> product returned by your <code>backward()</code><br>
will then be passed in as the <code>grad_output</code> argument of the <code>backward()</code><br>
of the <code>Function</code> next upstream in the computation graph.  That’s how<br>
autograd implements the chain rule.)</p>
<p>Best.</p>
<p>K. Frank</p>