Back propogation from 2 different positions of network

mfatih7: <p>Hello</p>
<p>I want to construct a network architecture that will be trained with respect to 2 different losses.</p>
<p>One of the losses will be computed with respect to the output(y) of the network.<br>
The other loss will be computed with respect to some variables(w) that are not output.<br>
I have both ground truth labels for y and w variables.</p>
<p>I have a network(F) that is trained successfully. Its input is x and output is w.<br>
I can generate a G network that instantiates F network.<br>
G network does not need any learnable parameters.<br>
G network will take the w outputs of F network and compute y values.<br>
All of the calculations of G network can be implemented in its forward() function.<br>
All of the calculation of G network is differentiable.</p>
<p>If I generate a network like G( F(x) ) = y is it possible to backpropagate both</p>
<ul>
<li>From w to x</li>
<li>From y to x</li>
</ul>
<p>Any code example will be very helpful</p>
<p>thank you</p>

mfatih7: <p>Hello</p>
<p>Let me ask another question</p>
<pre><code>criterion=torch.nn.MSELoss(size_average=True, reduce=True, reduction='mean')
optimizer=torch.optim.Adam(model.parameters(), lr=learning_rate)
</code></pre>
<p>…</p>
<pre><code>outputs_1, outputs_2 = model(inputs_1, inputs_2)
loss_train = criterion(outputs_1, labels_1)
optimizer.zero_grad()
loss_train.backward()
optimizer.step()  
</code></pre>
<p>In the code above, how can Pytorch understand that loss_train.backward() operates with respect to outputs_1 not outputs_2 ?</p>
<p>outputs_1 is a tensor. Isn’t it similar to a variable like numpy array?<br>
Or does it carry information about the model architecture?</p>

mfatih7: <p>Hi</p>
<p>I think the answers below have sufficient information.</p>
<aside class="quote quote-modified" data-post="1" data-topic="10440">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vijay_dubey/48/2719_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/a-model-with-multiple-outputs/10440">A model with multiple outputs</a> 
  </div>
  <blockquote>
    I have to implement a Convolutional Neural Network, that takes a kinect image (1640480) and return a 1 x8 tensor predicting the class to which the object belongs and a 1 x 4 tensor, predicting the bounding box around the image, if its present. 
Please help me how can I implement a suitable model to give two outputs and how to calculate loss and backpropagate in that case? 
Also, i have just around 6000 training images, how can I achieve the best possible results with limited number of training i…
  </blockquote>
</aside>

<aside class="quote" data-post="1" data-topic="25816">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qiangqiang_wu/48/15827_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-weight-my-own-multiple-losses/25816">How to weight my own multiple losses?</a> 
  </div>
  <blockquote>
    I have multiple losses, and i want to weight them seperately. More specifically, i want to set the weight of loss1 to 1, and the weight of loss2 to 0.5, is that correct? 
loss1 = nn.L1Loss(B, A) 
loss2 = nn.L1Loss(C, A) 
loss2 = loss2*0.5 
loss1.backward(retain_variables=True) 
loss2.backward() 
optimizer.step()
  </blockquote>
</aside>

<p>On the other hand, If Pytorch publishes more tutorials, especially non-vanilla learning models and mechanisms it will be beneficial in terms of practicality.</p>