Question from a novice, my neural network loss does not converge

username: wjk13720041
cooked: <p>I want to train a model where the loss has been increasing before the last modification. The loss does not converge after the modification.</p>
<p>that is my model:</p>
<pre><code class="lang-auto">class u_seta(nn.Module):
    def __init__(self, input_dim, output_dim):
        super().__init__()

        self.block = nn.Sequential(
                nn.Linear(input_dim, 256),
                nn.PReLU(),
                nn.Linear(256, 512),
                nn.PReLU(),
                nn.Linear(512,256),
                nn.PReLU(),
                nn.Linear(256, output_dim)
            )

    def forward(self, x):
        out = self.block(x)
        return out
</code></pre>
<p>I am using Adam Optimizer:</p>
<pre><code class="lang-auto">u_setaNet = utils.KRU_utils.u_seta(75,72).to(device)
optimizer = torch.optim.Adam(u_setaNet.parameters(),lr=0000000000.1)
</code></pre>
<p>Training sequence</p>
<pre><code class="lang-auto">for now_step in range(n+1):
            if(now_step==0):
                q_i_est = data_3D_pose.clone().to(device)
                X_est = utils.KRU_utils.pose_to_3d(q_i_est,data_3D).to(device)
                x_est = utils.KRU_utils.joint3d_to_2d(X_est,data_3D).to(device)
                Loss = utils.KRU_utils.LossF(x_est,data_2D_keypoint).to(device)
                Loss = Loss.sum(dim=1)
                X_est = torch.autograd.grad(outputs=Loss,inputs=X_est,grad_outputs=torch.ones_like(Loss))[0]
                X_est = X_est.reshape(X_est.shape[0],-1).to(device)
                Loss = Loss.detach()

            else:
                
                q_i_est = q_i_est.detach()
                X_est.requires_grad = True

                q_i_est_alter = u_setaNet(X_est) 
                
                q_i_est = q_i_est + q_i_est_alter

                X_est = utils.KRU_utils.pose_to_3d(q_i_est,data_3D).to(device)
                x_est = utils.KRU_utils.joint3d_to_2d(X_est,data_3D).to(device)

                Loss = utils.KRU_utils.LossF(x_est,data_2D_keypoint).to(device)
                Loss = Loss.sum(dim=1)
                
                optimizer.zero_grad()
                X_est = torch.autograd.grad(outputs=Loss,inputs=X_est,grad_outputs=torch.ones_like(Loss))[0]
                optimizer.step()

                X_est = X_est.reshape(X_est.shape[0],-1).to(device)
                Loss = Loss.detach()
</code></pre>
<p>The loss function is:</p>
<pre><code class="lang-auto">def LossF(joint2d,data_2D_keypoint):
    Loss = joint2d[:,:,0:2] - data_2D_keypoint[:,:,0:2]
    Loss = torch.sqrt(Loss[:,:,0]** 2 +  Loss[:,:,1]** 2)
    Loss = (Loss*data_2D_keypoint[:,:,2])
    return Loss
</code></pre>
<p>A series of transformations are performed after the network output, will this cause anomalies in the training?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/4/e4352a01c4f820a883160f312849be5a9c990b15.png" data-download-href="https://discuss.pytorch.org/uploads/default/e4352a01c4f820a883160f312849be5a9c990b15" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/4/e4352a01c4f820a883160f312849be5a9c990b15_2_690x164.png" alt="image" data-base62-sha1="wyOLxzCZ1dEWRjGZ9AaX8UwHrz7" width="690" height="164" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/4/e4352a01c4f820a883160f312849be5a9c990b15_2_690x164.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/4/e4352a01c4f820a883160f312849be5a9c990b15_2_1035x246.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/4/e4352a01c4f820a883160f312849be5a9c990b15_2_1380x328.png 2x" data-dominant-color="B8CCE1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1419×338 104 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>How should I modify it?  please help me to find the problem <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=12" title=":sob:" class="emoji" alt=":sob:" loading="lazy" width="20" height="20"></p>

username: eqy
cooked: <p>I am a bit confused by <code>lr=0000000000.1</code>, are you intending to lower the learning rate (which would make sense as a first thing to try if a model is not converging)? I believe <code>0000000000.1</code> would simply be parsed as <code>0.1</code>, so you may want to try e.g., <code>0.01</code>, <code>0.001</code>, or even scientific notation e.g., (<code>1e-4</code>).</p>

username: wjk13720041
cooked: <p>Thank you for your answer!<br>
Indeed, I intend to reduce the learning rate.I want to write 0.0000000001.</p>
<blockquote>
<p>lr=0.1:<br>
0 1 tensor(3062.5969, device=‘cuda:0’)<br>
0 1 tensor(2638.7090, device=‘cuda:0’)<br>
0 1 tensor(2935.3193, device=‘cuda:0’)<br>
0 1 tensor(2183.3059, device=‘cuda:0’)<br>
0 1 tensor(2613.2559, device=‘cuda:0’)<br>
0 1 tensor(2317.4438, device=‘cuda:0’)</p>
</blockquote>
<blockquote>
<p>lr= 1e-4:<br>
0 1 tensor(1728.2727, device=‘cuda:0’)<br>
0 1 tensor(1882.7476, device=‘cuda:0’)<br>
0 1 tensor(1797.2104, device=‘cuda:0’)<br>
0 1 tensor(1854.1182, device=‘cuda:0’)<br>
0 1 tensor(2128.3904, device=‘cuda:0’)<br>
0 1 tensor(2666.2241, device=‘cuda:0’)</p>
</blockquote>
<blockquote>
<p>lr= 0.0000000001:<br>
0 1 tensor(2332.1382, device=‘cuda:0’)<br>
0 1 tensor(2551.4082, device=‘cuda:0’)<br>
0 1 tensor(2360.8560, device=‘cuda:0’)<br>
0 1 tensor(2940.4958, device=‘cuda:0’)<br>
0 1 tensor(3134.4497, device=‘cuda:0’)<br>
0 1 tensor(2796.6272, device=‘cuda:0’)</p>
</blockquote>

username: InnovArul
cooked: <p>Naive question: I do not see a <code>loss.backward()</code> call in your code. Are you calling it anywhere else?<br>
I am sure you might already know this.<br>
The usual procedure is,</p>
<pre><code class="lang-auto">--- model forward ---
--- loss calculation ---

optimizer.zero_grad()
loss.backward()
optimizer.step()
</code></pre>

username: wjk13720041
cooked: <aside class="quote no-group" data-username="wjk13720041" data-post="1" data-topic="160671">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wjk13720041/48/52934_2.png" class="avatar"> wjk13720041:</div>
<blockquote>
<p><code>torch.autograd.grad</code></p>
</blockquote>
</aside>
<p>Thank you for your answer!<br>
I am using：</p>
<pre><code class="lang-auto">torch.autograd.grad(outputs=Loss,inputs=X_est,grad_outputs=torch.ones_like(Loss))
</code></pre>
<p>I have previously tried using</p>
<pre><code class="lang-auto">loss.backward()
</code></pre>
<p>But X_est does not seem to calculate the gradient</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="wjk13720041" data-post="5" data-topic="160671">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wjk13720041/48/52934_2.png" class="avatar"> wjk13720041:</div>
<blockquote>
<p>But X_est does not seem to calculate the gradient</p>
</blockquote>
</aside>
<p>That’s an interesting point as I also don’t fully understand your current use case.<br>
It seems you are using a model called <code>u_setaNet</code> and are initializing the optimizer with its parameters:</p>
<pre data-code-wrap="python"><code class="lang-python">u_setaNet = utils.KRU_utils.u_seta(75,72).to(device)
optimizer = torch.optim.Adam(u_setaNet.parameters(),lr=0000000000.1)
</code></pre>
<p>However, in your “training sequence” code snippet you are only calculating the gradients with <code>inputs=X_est</code>, which is the output of the model instead of the parameters of the model itself.<br>
I guess you are expecting <code>autograd.grad</code> to compute all gradients and accumulate them in the <code>.grad</code> attributes of the parameters, which is not the case. Also, you are overriding <code>X_est</code> with the output of <code>utils.KRU_utils.pose_to_3d</code> so note that this tensor is not the input to the model anymore.</p>
<p>In any case, even if you are using <code>autograd.grad(outputs=Loss, inputs=u_setaNet.parameters())</code> you would still need to apply these gradients, since they are only assigned to <code>X_est</code> again without any further use.</p>

username: wjk13720041
cooked: <p>Your answer has inspired me!<br>
I have modified my code as follows：<br>
<strong>(Modified parts are marked with “# Modified here”)</strong></p>
<pre><code class="lang-auto">for now_step in range(n+1):
    if(now_step==0):
        q_i_est = data_3D_pose.clone().to(device)
        q_i_est.requires_grad = True # Modified here
        
        X_est = utils.KRU_utils.pose_to_3d(q_i_est,data_3D).to(device)
        X_est.retain_grad() # Modified here

        x_est = utils.KRU_utils.joint3d_to_2d(X_est,data_3D).to(device)
        Loss = utils.KRU_utils.LossF(x_est,data_2D_keypoint).to(device)
        Loss = Loss.sum(dim=1)

        Loss.backward(torch.ones(X_est.shape[0]).to(device)) # Modified here
        X_est = X_est.grad # Modified here
        X_est = X_est.reshape(X_est.shape[0],-1).to(device)
        Loss = Loss.detach()
    
    else:
        
        q_i_est = q_i_est.detach()
        X_est = X_est.detach()  # Modified here
        X_est.requires_grad = True
    
        q_i_est_alter = u_setaNet(X_est)
        
        q_i_est = q_i_est + q_i_est_alter
    
        X_est_again = utils.KRU_utils.pose_to_3d(q_i_est,data_3D).to(device)   # Modified here
        x_est_again = utils.KRU_utils.joint3d_to_2d(X_est_again,data_3D).to(device) # Modified here
    
        Loss = utils.KRU_utils.LossF(x_est_again,data_2D_keypoint).to(device)
        Loss = Loss.sum(dim=1)
        
        optimizer.zero_grad()
        Loss.backward(torch.ones(X_est.shape[0]).to(device)) # Modified here
        optimizer.step()

        X_est = X_est.grad # Modified here
        X_est = X_est.reshape(X_est.shape[0],-1).to(device)
    
        loss_txt_cont += Loss.sum().item()
        Loss = Loss.detach()
        print(data_startP,data_endP,Loss.sum())
</code></pre>
<p>The result is back to a situation where the gradient keeps increasing</p>
<blockquote>
<p>lr=1e-3</p>
<p>0 1 tensor(2325.0229, device=‘cuda:0’)<br>
0 1 tensor(2532.3550, device=‘cuda:0’)<br>
0 1 tensor(2623.1548, device=‘cuda:0’)<br>
0 1 tensor(2475.8281, device=‘cuda:0’)<br>
0 1 tensor(2532.1895, device=‘cuda:0’)<br>
0 1 tensor(2526.4036, device=‘cuda:0’)</p>
</blockquote>
<blockquote>
<p>lr=1e-4</p>
<p>0 1 tensor(2150.0894, device=‘cuda:0’)<br>
0 1 tensor(2583.2695, device=‘cuda:0’)<br>
0 1 tensor(2855.8999, device=‘cuda:0’)<br>
0 1 tensor(2928.6746, device=‘cuda:0’)<br>
0 1 tensor(3010.9243, device=‘cuda:0’)<br>
0 1 tensor(3128.9155, device=‘cuda:0’)</p>
</blockquote>
<p>I need to input ∂loss/∂X_est to the network and then get the output of the network, i.e. the amount of adjustment of q_i_est that maps out X_est. After that, through a series of changes, the value obtained by the transformation is compared with the target to get the loss. Since it is not possible to directly compare the loss of the network output with the target, I want to optimize only the network and not the transformations.<br>
What might generally cause this condition? <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=12" title=":joy:" class="emoji" alt=":joy:" loading="lazy" width="20" height="20"><br>
Using Loss.backward and then using optimizer.step should automatically perform gradient descent, right? <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"></p>