Nn criterions don't compute the gradient w.r.t. targets error

username: machine
cooked: <p>pytorch’s Variable is too confusing to use well, and I got an error which spent my whole day to solve but I stil have no idea where is the problem?</p>
<p>here is part of my code,and the ‘train_batches’ is just an iterator of my trainset</p>
<pre><code class="lang-auto">        for batch in train_batches:
            loss=0
            Encoder_optimizer.zero_grad()
            Attention_optimizer.zero_grad()
            Score_optimizer.zero_grad()

            for idx in range(args.batch_size):
                question=Variable(torch.LongTensor(batch['question_token_ids'][idx]))
                answer_passage=batch['answer_passage'][idx]
                label=torch.zeros(args.max_paragraph_num)
                label[answer_passage]=1
                label=Variable(label)
                label.requires_grad=True
                scores = Variable(torch.zeros(args.max_paragraph_num))
                Encoder.init_hidden()
                _,question=Encoder(question)
                j=0
                for pidx in range(idx*args.max_paragraph_num,(idx+1)*args.max_paragraph_num):
                    passage=Variable(torch.LongTensor(batch['passage_token_ids'][pidx]))
                    Encoder.init_hidden()
                    passage,_=Encoder(passage)
                    passage=Attention(passage,question)
                    score=Score(passage,question)
                    scores[j]=score
                    j+=1

                scores=F.softmax(scores,0)
                loss+=loss_func(label,scores.view(1,5))
</code></pre>
<p>the error is:<br>
Traceback (most recent call last):<br>
File “/home/k/PycharmProjects/PassageRanking/run.py”, line 154, in <br>
run()<br>
File “/home/k/PycharmProjects/PassageRanking/run.py”, line 146, in run<br>
train(args)<br>
File “/home/k/PycharmProjects/PassageRanking/run.py”, line 134, in train<br>
loss+=loss_func(label,scores.view(1,5))<br>
File “/home/k/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 357, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/home/k/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py”, line 677, in forward<br>
_assert_no_grad(target)<br>
File “/home/k/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py”, line 11, in _assert_no_grad<br>
"nn criterions don’t compute the gradient w.r.t. targets - please " <br>
AssertionError: nn criterions don’t compute the gradient w.r.t. targets - please mark these variables as volatile or not requiring gradients</p>

username: Naruto-Sasuke
cooked: <p>Take the following as an example,</p>
<pre><code class="lang-python">&gt;&gt;&gt; loss = nn.L1Loss()
&gt;&gt;&gt; input = autograd.Variable(torch.randn(3, 5), requires_grad=True)
&gt;&gt;&gt; target = autograd.Variable(torch.randn(3, 5))
&gt;&gt;&gt; output = loss(input, target)
&gt;&gt;&gt; output.backward()
</code></pre>
<p><code>target</code> is the second parameter, not the first. So you need to swap the paramters.</p>

username: machine
cooked: <p>But it still occurs the same error after I exchange the position of input and target:sob:</p>

username: Naruto-Sasuke
cooked: <p>Because you set <code>label.requires_grad=True</code>, delete this line. Remember we only need to compute the gradient w.r.t the input. In many situations, the gradient w.r.t the target is useless.</p>

username: adrien
cooked: <p>But why throw an error ?</p>
<p>I came across a use case where I needed to minimize the mse between intermediate features of an auto-encoder: both input and target need to be differentiated here.</p>
<p>I had to trade <code>nn.MSELoss(encoder_i, decoder_i)</code> for <code>torch.sum((encoder_i - decoder_i)**2)</code> which also does the job. However I’m not 100% sure I didn’t lose somthing with this fix (efficiency ?). I don’t understand why such use of nn losses are not permited.</p>

username: viniciusarruda
cooked: <p><a class="mention" href="/u/adrien">@adrien</a> Did you solve your problem ? I am having exactly the same problem.</p>

username: yunpei
cooked: <p>So, how do you solve this problem when both the input and the target need to be differentiated?</p>