Fmod or remainder: RuntimeError: the derivative for 'other' is not implemented

catosphere: <p>Hello everyone</p>
<p>I am using fmod or remainder to compute a sawtooth waveform, where the divisor argument ‘other’ is computed by some layers (in this case it is for defining the sawtooth frequency), thus it requires gradient.<br>
When performing backpropagation it gives the following error for fmod or remainder:</p>
<p>RuntimeError: the derivative for ‘other’ is not implemented</p>
<p>Is there any workaround please ?</p>
<p>The input argument is a torch.arange(size) which is then shaped as a saw thanks to the modulo operation.</p>
<p>Thanks !</p>

albanD: <p>Hi,</p>
<p>I’m afraid we don’t have a gradient formula for this. What would be <code>d(fmod(x, y)) / dy</code> ?</p>

catosphere: <p>Hi,</p>
<p>x = torch.arange has a constant derivative/slope, however through the modulo, it becomes non defined at the modulo jump, which makes it non differentiable in each of these points. So as it is, indeed there is an issue for gradients I guess.</p>
<p>I alternatively tried to generate sawtooth as wavetables, by using interpolate to rescale a given sawtooth at a reference frequency to any other frequency, feeding the ratio of the wavetable reference frequency and the predicted f0.</p>
<p>However, it seems as well that interpolate does not accept a gradient variable in the scale factor. Is it correct ?</p>
<p>In this case it seems as well that backpropagation is not possible through the interpolate scale factor.</p>

albanD: <p>The scale factor is a multiplier for the size. So it is not differentiable either indeed I’m afraid.</p>

jasdeep_singh_Grover: <p>I am not very sure regarding the use case but you can try this function. It seems to work as a very good approximation and everything seems differentiable. I have just made some back of the envelope calculations regarding how gradient might work in such a situation. Hope you find it useful.</p>
<p>fmod( x, a ) ≈ ( a / π ) arctan( tan( π ( x / a - 0.5 ) ) ) + a / 2</p>
<p>link to a small demonstration of the function: <a href="https://www.desmos.com/calculator/havtieuhey" class="inline-onebox" rel="noopener nofollow ugc">Differentiable approximate modulus | Desmos</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/0/1012b25b6b015a25d4bc01f3c2cef282de89090e.png" data-download-href="https://discuss.pytorch.org/uploads/default/1012b25b6b015a25d4bc01f3c2cef282de89090e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1012b25b6b015a25d4bc01f3c2cef282de89090e_2_690x232.png" alt="image" data-base62-sha1="2ibHdpHfLVnawPKW1ILAjwTiZHg" width="690" height="232" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1012b25b6b015a25d4bc01f3c2cef282de89090e_2_690x232.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/0/1012b25b6b015a25d4bc01f3c2cef282de89090e_2_1035x348.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/0/1012b25b6b015a25d4bc01f3c2cef282de89090e.png 2x" data-dominant-color="F5F5F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1366×460 66.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

JosueCom: <p>Not sure if it is helpful but here is my implementation of the sawtooth function using pytorch only.</p>
<pre data-code-wrap="python"><code class="lang-python">def sawtooth_wave(t, period = 0.5, amplitude = 1.0, phase = 0.0, move = 0.0):
    x = (t + phase) / period
    return amplitude * (x - th.floor(x)) + move
</code></pre>
<p>More details and examples can be seen here: <a href="https://gist.github.com/wzjoriv/889e5d45e2e2c6f8492def96dccc495b" class="inline-onebox" rel="noopener nofollow ugc">Sawtooth wave function purely in pytorch · GitHub</a></p>