Call backward() for two loss meta pseudo label

omi_d: <p>Recently I read a paper about an overview of meta pseudo label and I found a  <a href="https://www.kaggle.com/code/hengck23/playground-for-meta-pseudo-label" rel="noopener nofollow ugc">Notebook </a> that implement it. I saw a line of code (<code> (t_l_loss+t_mpl_loss).backward()</code>) and I don’t understand how it works.</p>
<p>Could you please give me a hint that how we can sum two losses and call backward()?<br>
`<br>
experiment4: meta-pesudo-label ( exact, no approximation)</p>
<p>if 1:<br>
l_x = torch.from_numpy(x_train).float().cuda()<br>
l_y = torch.from_numpy(y_train).float().cuda()</p>
<pre><code>teacher = Net().cuda()
student = Net().cuda()
t_optimizer = optim.SGD(teacher.parameters(),lr=0.001, momentum=0.9)
s_optimizer = optim.SGD(student.parameters(),lr=0.001, momentum=0.9)

for iteration in range(200):
    # subscript: t,s : teacher,student
    # subscript: l,u : label,unlabel
    
    teacher.train()
    student.train()
    t_optimizer.zero_grad()
    s_optimizer.zero_grad()
    
    random_sample = np.random.choice(num_unlabel*2,16).tolist()
    u_x = torch.from_numpy(x_unlabel[random_sample]).float().cuda()
    
    
    #prepare other input
    #note these is detached, i.e. not used for backprop
    #s_l_logit = student(l_x)
    #s_l_loss = F.binary_cross_entropy_with_logits(s_l_logit.detach(), l_y)
    
    t_u_logit = teacher(u_x)
    pseudo_y =  torch.sigmoid(t_u_logit) #t_u_logit&gt;0).float()
    #------
    
    
    
    #train student : update student using pesudo label data only
    s_u_logit = student(u_x)
    s_u_loss  = F.binary_cross_entropy_with_logits(s_u_logit, pseudo_y,reduction='none')
    s_u_loss  = s_u_loss[torch.abs(pseudo_y-0.5)&gt;0.45].mean()
    s_u_loss.backward()
    s_optimizer.step()
    
    
    #train teacher : update teacher using pesudo label data student change in loss
    s_l_logit_new = student(l_x)
    s_l_loss_new  = F.binary_cross_entropy_with_logits(s_l_logit_new, l_y)
    t_mpl_loss    = s_l_loss_new
    
    t_l_logit = teacher(l_x)
    t_l_loss = F.binary_cross_entropy_with_logits(t_l_logit, l_y)
    
    
    (t_l_loss+t_mpl_loss).backward()
    #(t_mpl_loss).backward()
    t_optimizer.step()
    
   
    #print(iteration, s_u_loss.item(), t_l_loss.item(), t_mpl_loss.item())
#-------------------------------
print('metal pseudo label (exact) : student')
print_validate(student)
show_predict_space(student)
plt.title('metal pseudo label (exact): student')
plt.show()

print('metal pseudo label (exact): teacher')
print_validate(teacher)
show_predict_space(teacher)
plt.title('metal pseudo label (exact): teacher')
plt.show()
</code></pre>
<p>`</p>

ptrblck: <aside class="quote no-group" data-username="omi_d" data-post="1" data-topic="157824">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/omi_d/48/51889_2.png" class="avatar"> omi_d:</div>
<blockquote>
<p>Could you please give me a hint that how we can sum two losses and call backward()?</p>
</blockquote>
</aside>
<p>I’m unsure if I understand the issue correctly, but the addition is differentiable and Autograd will backpropagate the gradients to the summands. The same would happen in any other summation and is not specific to add two losses together.</p>