How to get gradient?

username: oasjd7
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/0/b014a4dafd63bf49199ded831a4a4b296a3007be.png" alt="image" data-base62-sha1="p7GgnWuQvvlsyKy2Rmeq3xvQhHw" width="642" height="365"></p>
<p>Hi, all.<br>
I’m trying to implement the loss like (2) in the figure.<br>
But I don’t know how to get gradient for the loss.</p>

username: albanD
cooked: <p>Hi,</p>
<p>From reading that paragraph, I don’t think (2) defines a loss. It is just an inequality.</p>
<p>In general, if you want the gradients for your parameters wrt your loss in a differentiable manner (to be able to compute gradient wrt to these gradients), you can do:<br>
<code>grads = autograd.grad(loss, model.parameters(), create_graph=True)</code>.</p>

username: oasjd7
cooked: <p><a class="mention" href="/u/alband">@albanD</a> Thansk for your quick reply.<br>
Here is he full contents.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/f/efaa3753cb525dd4f89e22cebbb8e3dddbf365f0.png" alt="image" data-base62-sha1="ycaNI6NQHAsvBpsJOtRTMOOPax2" width="634" height="346"></p>
<p>According to your answer, I tried. but have some problems.<br>
I’m not sure how to use <code>autograd.grad</code>.<br>
With this codes, <code>allow_unused=True</code> and <code>retain_graph=True</code>  is required.<br>
And got <code>(None, None)</code></p>
<pre><code class="lang-auto">    optimizer_g = optim.SGD(list(encoder.parameters()), lr=0.1, momentum=0.9, weight_decay=5e-4)
    optimizer_f = optim.SGD(list(classifier.parameters()), lr=0.1, momentum=0.9, weight_decay=5e-4)
    ...
    loss = criterion(outs, labels)
    optimizer_g.zero_grad()
    optimizer_f.zero_grad()
    loss.backward(retain_graph=True)
    grads = autograd.grad(loss, classifier.parameters(), create_graph=True, allow_unused=True)
    print(grads)
    optimizer_g.step()
    optimizer_f.step()  

</code></pre>

username: albanD
cooked: <p>These gradients should be used to update the loss before doing the final <code>.backward()</code>:</p>
<pre><code class="lang-auto">    optimizer_g = optim.SGD(list(encoder.parameters()), lr=0.1, momentum=0.9, weight_decay=5e-4)
    optimizer_f = optim.SGD(list(classifier.parameters()), lr=0.1, momentum=0.9, weight_decay=5e-4)
    ...
    loss = criterion(outs, labels)
    # Do you have one loss or one for each?
    # If so, give only the corresponding loss for each model and remove the allow_unused
    grads_f = autograd.grad(loss, classifier.parameters(), create_graph=True, allow_unused=True)
    grads_g = autograd.grad(loss, encoder.parameters(), create_graph=True, allow_unused=True)

    # I guess the product in the formula is a dot product?
    grad_prod = 0.
    for gf, gg in zip(grads_f, grads_g):
        grad_prod += (gf * gg).sum()

    final_loss = loss - alpha * grad_prod

    optimizer_g.zero_grad()
    optimizer_f.zero_grad()
    loss.backward()
    optimizer_g.step()
    optimizer_f.step() 
</code></pre>

username: oasjd7
cooked: <p><a class="mention" href="/u/alband">@albanD</a> Sorry,  I still have trouble. How can I fix this in this situation?</p>
<pre><code class="lang-auto">- Network structure:
input1 -&gt; encoder1 -&gt; 
                       shared_encoder -&gt;classifier
input2 -&gt; encoder2 -&gt; 
</code></pre>
<pre><code class="lang-auto">   optimizer_g = optim.SGD(list(encoder1.parameters()) + list(encoder2.parameters())+ list(shared_encoder.parameters()), lr=0.1, momentum=0.9, weight_decay=5e-4)
   optimizer_f = optim.SGD(list(classifier.parameters()), lr=0.1, momentum=0.9, weight_decay=5e-4)
   ...
   out1 = encoder(input1)
   out1 = shared_encoder(out1)
   out1 = classifier(out1)

   out2 = encoder(input2)
   out2 = shared_encoder(out2)
   out2 = classifier(out2)

   loss1 = criterion(outs1, labels)
   loss2 = criterion(outs2, labels)
   loss = loss1 + loss2

   grads_f = autograd.grad(loss, classifier.parameters(), create_graph=True, allow_unused=True)

   # How to add encoders?
   grads_g = autograd.grad(loss, encoder.parameters(), create_graph=True, allow_unused=True)
   grad_prod = 0.
   for gf, gg in zip(grads_f, grads_g):
       grad_prod += (gf * gg).sum() # Inner product

   final_loss = loss - alpha * grad_prod

   optimizer_g.zero_grad()
   optimizer_f.zero_grad()
   final_loss.backward()
   optimizer_g.step()
   optimizer_f.step()</code></pre>

username: albanD
cooked: <p>For the dot product in the formula to make sense, both the classifier and the encoder must have the exact same parameter structure. So you should just make sure to give the encoder is a way that you get the right gradients.</p>