akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Torch.sign breaks autograd dynamic graph

username: Arale_Mononoko
cooked: <p>Hi!</p>
<p>I have and autoencoder, and between the coder and decoder I transform the data using torch.sign . When I do it, the backpropagation of the gradient stops in that point.</p>
<p>If I replace torch.sign by torch.sigmoid, then I don’t have that problem and the backpropagation goes back to the beginning.</p>
<p>Do I have to do something different with torch.sign?</p>

username: albanD
cooked: <p>Hi,</p>
<p>the sign function is not differentiable (or if you look at it in a differentiable manner, the gradient is 0 almost everywhere). So it is expected that you won’t be able to get gradients through it.</p>

username: Arale_Mononoko
cooked: <p>Thanks. Thinking it over again it makes sense.</p>
<p>I see the sign function as a non-linear function and small variations of the input gives the same output, thus the derivative is like the derivative of a constant, 0 (except values near 0)</p>
<p>A linear approximation of the function would be a sigmoid with parameter beta \frac{1}{1+e^{-\beta <em>x}}. Which has derivative \frac{\beta \exp{-\beta</em>x}{(1+exp{-\beta*x})^2}. If I want to create a new function of this sigmoid with beta, the parameter received in the backward function, grad_output, is the one that has to be evaluated using the formula of the derivative for each element of grad_ouptut?</p>

username: albanD
cooked: <p>Hi,</p>
<p>Note that we have hardsignmoid (<a href="https://pytorch.org/docs/master/nn.functional.html#torch.nn.functional.hardsigmoid">https://pytorch.org/docs/master/nn.functional.html#torch.nn.functional.hardsigmoid</a>) if you’re using the nightly builds or similar functions like hardtanh.</p>
<p>If you want to use your custom function:</p>
<ul>
<li>If you want the “true” gradient to be used, then just implement the function you want and autograd will get the gradient for you.</li>
<li>If you want the backward to compute something else than the gradient of your function, you can see <a href="https://pytorch.org/docs/stable/notes/extending.html">this doc</a> that explains how to do that with a custom autograd Function where you will have to specify the backward to use.</li>
</ul>

username: Arale_Mononoko
cooked: <p>Thanks again for the answer and the references to the doc.</p>
<p>I think that to build a custom function is far from my knowledge of pytorch.</p>
<p>I think that I will multiply the data by \beta before doing the sigmoid function.</p>
<p>Most grateful</p>
How to train optical flow?

username: Rabin_Thapa
cooked: <p>i’m trying to build a video classifier using HMDB51 dataset but , i dont know how to train or use it’s optical flow .can somebody help me?</p>

username: Mughees
cooked: <p>hi,</p>
<p>you can start exploring <a href="https://github.com/sniklaus/pytorch-spynet" rel="nofollow noopener">SpyNet</a> and <a href="https://github.com/Coldog2333/pytoflow" rel="nofollow noopener">ToFlow</a> as a starting point for learning about optical flow networks.</p>