How to compute loss in ROI region of a tensor?

username: John1231983
cooked: <p>I have a tensor size of 16x16. I only want to compute the loss and update gradient in a region of interesting (ROI) of the tensor such as ROI[4:12,4:12]=1. I used the below code but it shows an error. How to do it? Thanks</p>
<pre><code class="lang-auto">import torch

input = torch.rand((16,16),requires_grad=True)
print (input)
mask = torch.zeros_like(input)
mask[4:12,4:12]=1.0
input = mask* input

d = torch.mean(input)
d.backward()
print (input.grad.data)
</code></pre>
<p>Error is</p>
<pre><code class="lang-auto">AttributeError                            Traceback (most recent call last)
&lt;ipython-input-35-9826ca6fcef5&gt; in &lt;module&gt;()
     10 d = torch.mean(input)
     11 d.backward()
---&gt; 12 print (input.grad.data)

AttributeError: 'NoneType' object has no attribute 'data'
</code></pre>
<p>It looks that the input.grad is None</p>

username: spanev
cooked: <p>Hi John,</p>
<p>Here,</p>
<pre><code class="lang-auto">input = mask* input
</code></pre>
<p>you are replacing (and hiding) the real input (leaf node) of your graph. If you want to access the gradients of the input, you should keep the original <code>input</code> reference and rename the masked <code>input</code>:</p>
<pre><code class="lang-auto">import torch

input = torch.rand((16,16),requires_grad=True)
print(input)
mask = torch.zeros_like(input)
mask[4:12,4:12]=1.0
masked_input = mask * input 
d = torch.mean(masked_input)
d.backward()
print(input.grad.data)
</code></pre>
<p>Hope that helps!</p>
<p>Additionally, if you ever want to access the <code>masked_input</code> gradients, which is non-leaf Tensor, you will have to to call <code>masked_input.retain_grad()</code> after its creation.<br>
Please find more info about it <a href="https://pytorch.org/docs/stable/autograd.html" rel="nofollow noopener">here</a></p>

username: John1231983
cooked: <aside class="quote no-group" data-username="spanev" data-post="2" data-topic="55181">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spanev/48/14892_2.png" class="avatar"> spanev:</div>
<blockquote>
<p>mport torch input = torch.rand((16,16),requires_grad=True) print(input) mask = torch.zeros_like(input) mask[4:12,4:12]=1.0 masked_input = mask * input d = torch.mean(masked_input) d.backward() print(input.grad.data)</p>
</blockquote>
</aside>
<p>Thanks <a class="mention" href="/u/spanev">@spanev</a>. It worked but the result looks wrong. If I ignore the region in mask, so the expected mean grad should be 1 instead of 0.11111</p>

username: spanev
cooked: <p>What do you mean by ignoring the region in mask? Which gradient should be 1? <code>d.grad</code> will be 1 since we run backward on it.</p>
<p>In <code>d = torch.mean(masked_input)</code> all the inputs (even the zero’d ones) will contribute to the mean, so the final gradients will be <code>1/(16*16)</code> on the non masked elements, and 0 on the masked/zero’d ones.</p>

username: John1231983
cooked: <aside class="quote no-group" data-username="spanev" data-post="4" data-topic="55181">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spanev/48/14892_2.png" class="avatar"> spanev:</div>
<blockquote>
<p>ion in mask? Which gradient should be 1? <code>d.grad</code> will be 1 since we run backward on it.</p>
<p>In <code>d = torch.mean(masked_input)</code> all the inputs (even the zero’d ones) will contribute to the mean, so the final gradients will be <code>1/(16*16)</code> on the non masked elements, and 0 on the masked/zero’d ones.</p>
</blockquote>
</aside>
<p>I means I want to set grad on the outside ROI to zero, it means we will untouch the region</p>