Inplace Operation and Gradient Computation

username: Diego
cooked: <p>Hello I have the following code:</p>
<pre><code class="lang-auto">deltac = maxc - minc
s = deltac / maxc
</code></pre>
<p>This section does not pass the gradient check, claiming an inplace operation on the 2nd line. I solved it by replacing it with <code>deltac.clone() / maxc</code>. However I am curious as to where is the inplace operation here? Thanks in advance for any help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Tony-Y
cooked: <pre><code class="lang-auto">&gt;&gt;&gt; c = torch.rand(5,10)
&gt;&gt;&gt; c.requires_grad = True
&gt;&gt;&gt; minc, _ = c.min(1)
&gt;&gt;&gt; maxc, _ = c.max(1)
&gt;&gt;&gt; deltac = maxc - minc
&gt;&gt;&gt; s = deltac / maxc
&gt;&gt;&gt; s.sum().backward()
&gt;&gt;&gt; s
tensor([0.9689, 0.8587, 0.8610, 0.9500, 0.9923], grad_fn=&lt;DivBackward0&gt;)
&gt;&gt;&gt; c.grad.max()
tensor(0.1661)
</code></pre>
<p>I can’t reproduce your problem.</p>

username: Diego
cooked: <p>Thanks for the swift reply I will post the entire function that produces the problem, this will provide some more context.</p>
<pre><code class="lang-auto">def rgb_to_hsv(image):

    torch.autograd.set_detect_anomaly(True)
    r = image[..., 0, :, :]
    g = image[..., 1, :, :]
    b = image[..., 2, :, :]

    maxc = image.max(-3)[0]
    minc = image.min(-3)[0]

    v = maxc  # brightness

    deltac = maxc - minc
    s = deltac.clone() / v  # saturation

    deltac[deltac == 0] = 1  # avoid division by zero
    rc = (maxc - r) / deltac
    gc = (maxc - g) / deltac
    bc = (maxc - b) / deltac

    h = 4.0 + gc - rc
    h[g == maxc] = 2.0 + rc[g == maxc] - bc[g == maxc]
    h[r == maxc] = bc[r == maxc] - gc[r == maxc]
    h[minc == maxc] = 0.0

    h = (h / 6.0) % 1.0

    return torch.stack([h, s, v], dim=-3)

</code></pre>
<p>The code for checking the gradients:</p>
<pre><code class="lang-auto">from torch.autograd import gradcheck
 def test_gradcheck(self):

        data = torch.tensor([[[[21., 22.],
                              [22., 22.]],

                             [[13., 14.],
                              [14., 14.]],

                             [[8., 8.],
                              [8., 8.]]]])  # 3x2x2

        data = utils.tensor_to_gradcheck_var(data)  # to var

        assert gradcheck(image.RgbToHsV(), (data,),
                         raise_exception=True)
</code></pre>

username: Tony-Y
cooked: <aside class="quote no-group" data-username="Diego" data-post="3" data-topic="44589">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/diego/48/6989_2.png" class="avatar"> Diego:</div>
<blockquote>
<p>deltac[deltac == 0] = 1 # avoid division by zero</p>
</blockquote>
</aside>
<p>This is an in-place operation. Try to replace it with:</p>
<pre><code class="lang-auto">deltac = torch.where(deltac == 0, 1, deltac)
</code></pre>

username: Diego
cooked: <p>Thank you: I had to change your line to</p>
<pre><code class="lang-auto">deltac = torch.where(deltac == 0, torch.ones_like(deltac), deltac)
</code></pre>
<p>But it works now.  Is there a cleaner way to do this, similar to <code>np.where(arr==0, 1, a)</code>?</p>

username: Tony-Y
cooked: <aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/9190" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/vadimkantorov" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/1041752?v=2&amp;s=96" class="thumbnail onebox-avatar" width="60" height="60">
</a>

<h4><a href="https://github.com/pytorch/pytorch/issues/9190" target="_blank" rel="nofollow noopener">Issue: [pytorch] torch.where to support Python scalars</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/vadimkantorov" target="_blank" rel="nofollow noopener">vadimkantorov</a>
	on <a href="https://github.com/pytorch/pytorch/issues/9190" target="_blank" rel="nofollow noopener">2018-07-05</a>
	</div>
	<div class="user">
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">a = torch.Tensor([1, 2, 3])
# works, correctly produces tensor([-1., 2., 3.])
b = torch.where(a == 1, torch.Tensor([-1]), a)
# fails: TypeError: where(): argument...</pre>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>