If a parameter is trainable in the following case?

Ximeng: <p>Hi,</p>
<p>Suppose there is a tensor A=[3,4,5,6] with requires_grad=True, and a tensor THRES=[4] also with requires_grad=True. Let B=A[A&gt;THRES]. In this case, B allows backwards and it inherite gradients from A. I think the bool mask term [A&gt;THRES] doesn’t maintain any gradient infomation. Despite the fact that THRES influence the final result, it doesn’t make any direct gradient contribution. My question is whether THRES is actually trainable or randomized around its initialization? Will it be trained as how reinforcement learning works?</p>
<p>Thank you!</p>

KFrank: <p>Hi Guo!</p>
<aside class="quote no-group" data-username="Ximeng" data-post="1" data-topic="149907" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/x/8c91f0/48.png" class="avatar"> Ximeng:</div>
<blockquote>
<p>I think the bool mask term [A&gt;THRES] doesn’t maintain any gradient infomation. Despite the fact that THRES influence the final result, it doesn’t make any direct gradient contribution.</p>
</blockquote>
</aside>
<p>Your reasoning is correct.  The thresholding operation is not (usefully)<br>
differentiable, so autograd does not backpropagate through it.</p>
<p>To verify:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.10.2'
&gt;&gt;&gt; A = torch.tensor ([3., 4., 5., 6.], requires_grad = True)
&gt;&gt;&gt; THRES = torch.tensor ([4.], requires_grad = True)
&gt;&gt;&gt; A &gt; THRES
tensor([False, False,  True,  True])
&gt;&gt;&gt; A[A &gt; THRES].sum().backward()
&gt;&gt;&gt; A.grad
tensor([0., 0., 1., 1.])
&gt;&gt;&gt; THRES.grad == None
True
</code></pre>
<p>The problem is that when <code>A &lt; THRES</code>, the derivative (with respect<br>
to both <code>A</code> and <code>THRES</code>) is zero, and when <code>A &gt; THRES</code>, the derivative<br>
is also zero (and when <code>A == THRES</code>, the derivative is infinite or, if you<br>
prefer, undefined).  So even if autograd were to track the derivative,<br>
you wouldn’t be able do anything useful with it.</p>
<aside class="quote no-group">
<blockquote>
<p>My question is whether THRES is actually trainable</p>
</blockquote>
</aside>
<p>As it stands, <code>THRES</code> is not trainable (with a gradient-descent optimizer).</p>
<p>But it is perfectly reasonable to  use differentiable “soft” thresholding:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; A.grad = None
&gt;&gt;&gt; alpha = 2.5   # sharpness of "soft" step function
&gt;&gt;&gt; (alpha * (A - THRES)).sigmoid()
tensor([0.0759, 0.5000, 0.9241, 0.9933], grad_fn=&lt;SigmoidBackward0&gt;)
&gt;&gt;&gt; (A * (alpha * (A - THRES)).sigmoid()).sum().backward()
&gt;&gt;&gt; A.grad
tensor([0.6016, 3.0000, 1.8004, 1.0930])
&gt;&gt;&gt; THRES.grad
tensor([-4.0018])
</code></pre>
<p>With soft thresholding, <code>THRES</code> has a perfectly well-defined (and useful)<br>
gradient, and you could use that gradient to train / optimize <code>THRES</code>.</p>
<p>(Note, in this example, <code>A.grad</code> comes from two terms – one from the<br>
first <code>A</code> in the product <code>A * (alpha * (A - THRES)).sigmoid()</code>, and<br>
one from the second <code>A</code> in the product –  in the <code>(A - THRES).sigmoid()</code><br>
term.)</p>
<p>Best.</p>
<p>K. Frank</p>

Ximeng: <p>That’s the most professional answer I could ever expect and It really helps. Thank you very much!</p>