How does applying a mask to the output affect the gradients?

Bahman_Rouhani: <p>I want to apply a mask to my model’s output and then use the masked output to calculate a loss and update my model.<br>
I don’t want the autograd to consider the masking operation when calculating the gradients, i.e. I want the autograd to treat my model as if it had outputed the masked version of my input. In other words, I don’t want to calculate any loss for the covered regions of output.<br>
my code is something like:</p>
<blockquote>
<p>batch = get_batch()<br>
x = Variable(batch[‘data’], requires_grad=False)<br>
mask = Variable(batch[‘mask’], requires_grad=False)<br>
y = model(x)<br>
z = y * mask<br>
loss = loss_function(z)<br>
loss.backward()<br>
optimizer.step()</p>
</blockquote>
<p>I assume since <code>requires_grad</code> is set to <code>False</code> for both <code>x</code> and <code>mask</code> this operation should not affect the gradients.<br>
Currently my model behaves as if there is no masking so I want to know if I’m doing something wrong when applying the mask or if I should look for the problem somewhere else.</p>
<p>I’m also not sure if I should use <code>with torch.no_grad()</code> when applying the mask but I’m guessing I shouldn’t.</p>

KFrank: <p>Hi Bahman!</p>
<aside class="quote no-group quote-modified" data-username="Bahman_Rouhani" data-post="1" data-topic="126520" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bahman_rouhani/48/23763_2.png" class="avatar"> Bahman_Rouhani:</div>
<blockquote>
<p>I want the autograd to treat my model as if it had outputed the masked version of my input.</p>
<blockquote>
<p>y = model(x)<br>
z = y * mask<br>
loss = loss_function(z)<br>
loss.backward()</p>
</blockquote>
</blockquote>
</aside>
<p>Your code should do what you want.  Autograd tracks computations<br>
with pytorch tensors and doesn’t care whether those computations are<br>
“free standing” (such as your mask operation) or part of your model or<br>
part of your loss – they’re all treated the same.</p>
<p>If <code>mask</code> has <code>0</code>s in it, then when you multiply an element of <code>y</code> by <code>0</code>,<br>
no gradients will flow back through that element of <code>y</code> (or, more<br>
precisely, the gradients that flow back through it will be zero).</p>
<p>As an aside, <code>Variable</code> has been deprecated for some time you, so<br>
you probably don’t want to be using it – just use regular tensors.</p>
<aside class="quote no-group">
<blockquote>
<p>Currently my model behaves as if there is no masking</p>
</blockquote>
</aside>
<p>Just to check, if your <code>mask</code> is entirely zero, do your gradients all<br>
become zero?  Does changing <code>z = y * mask</code> in your code to <code>z = y</code><br>
truly have no effect on your gradients?  “No effect” would indicate a<br>
bug somewhere other than in the code you posted.</p>
<p>Best.</p>
<p>K. Frank</p>

Bahman_Rouhani: <p>I checked and the gradients do become zero when I mask consists entirely of zeros.<br>
Thank you.</p>

Jo-w: <p>Hi,</p>
<p>I find another situation does affect gradient:</p>
<pre><code class="lang-auto">for i in range(n_class):
    pred = output[:, label==i]
    label_s = label[label==i]
    if min(pred.shape)!=0 and min(label_s.shape)!=0:
        class_loss = self.loss(pred, label_s)
        loss_list[i] = class_loss
</code></pre>
<p>After calculate class-specific loss and use one of the losses to do backward, e.g., class 0, or just backward the averaged loss, the model’s <code>para.grad</code> are always <code>None</code> . Is there any idea to fix this problem?<br>
Thank you!</p>