Learning pointer offset in circular buffer, does not require grad error

username: HmmmmML
cooked: <p>Hi! I have a very simple example <code>Module</code> below, where I have a <a href="https://en.wikipedia.org/wiki/Circular_buffer" rel="noopener nofollow ugc">circular buffer</a>, and attempt to learn a <code>_delay</code> parameter (i.e. distance between a read and write pointer) that transforms an input sin wave into a phase-shifted and zero-padded output. The goal is to emulate a <a href="https://ccrma.stanford.edu/~jos/pasp/Delay_Lines.html" rel="noopener nofollow ugc">delay line</a> (as in DSP), learning an unknown delay parameter given some arbitrary input and reference output.</p>
<p>Currently this gives errors that the <code>element 0 of tensors does not require grad and does not have a grad_fn</code> - maybe, the problem (as represented) isn’t differentiable?</p>
<p>Any advice as to why this doesn’t work would be appreciate, and practical suggestions for representing this problem in PyTorch! Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "delay_eg.py", line 59, in &lt;module&gt;
    loss.backward()
  File "venv391/lib/python3.9/site-packages/torch/_tensor.py", line 307, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "venv391/lib/python3.9/site-packages/torch/autograd/__init__.py", line 154, in backward
    Variable._execution_engine.run_backward(
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>Re-producible code (note that the read pointer is fractional!):</p>
<pre><code class="lang-auto">import numpy as np
import torch

class Delay(torch.nn.Module):

    def __init__(self, N=44100, offset=1):
        super().__init__()
        self.N = N
        self.register_buffer('circ_buffer', torch.zeros(N, 1, dtype=torch.float))
        self._delay = torch.nn.Parameter(torch.FloatTensor([offset]))
        self.reset_state()

    def reset_state(self):
        self.circ_buffer[self.circ_buffer &gt; 0.0] = 0.0
        self.circ_buffer[self.circ_buffer &lt; 0.0] = 0.0
        self._read_ptr = 0.0
        self._write_ptr = int(self._delay.item())

    def forward(self, x):
        y = torch.zeros(*x.shape, dtype=torch.float).to(x.device)
        for i in range(x.shape[0]):
            x_i = x[i].item()
            # Write
            self.circ_buffer[self._write_ptr] = x_i
            # Read (with linear interpolation)
            i0 = np.floor(self._read_ptr).astype(int)
            i1 = (i0 + 1) % self.N
            frac = self._read_ptr - np.floor(self._read_ptr)
            y[i] = self.circ_buffer[i0] + frac*(self.circ_buffer[i1]-self.circ_buffer[i0])
            # Pointers
            self._write_ptr += 1
            if self._write_ptr == self.N:
                self._write_ptr = 0
            self._read_ptr += 1
            if self._read_ptr &gt;= self.N:
                self._read_ptr -= self.N
        return y

if __name__ == "__main__":
    device = torch.device("cpu")

    # Simple sin wave
    ts = np.arange(0.0, 0.3, 1/44100)
    xs = np.sin(2*np.pi*5*ts + 0)
    # Phase shift, with zero padding
    delay = int(xs.shape[0]/1.31)
    ys = np.concatenate([np.zeros(xs.shape[0]-delay), xs[:delay]])

    model = Delay().to(device)
    learning_rate = 1e-3
    optimizer = torch.optim.Adam(params=model.parameters(), lr=learning_rate)
    device = next(model.parameters()).device
    loss_fn = torch.nn.MSELoss()

    for _ in range(100):
        xst = torch.tensor(xs).float().to(device)
        yst = torch.tensor(ys).float().to(device)
        model.reset_state()
        optimizer.zero_grad()
        loss = loss_fn(model(xst), yst)
        loss.backward()
        optimizer.step()
        print(loss.item())
</code></pre>

username: KFrank
cooked: <p>Hi Hmmm!</p>
<aside class="quote no-group quote-modified" data-username="HmmmmML" data-post="1" data-topic="139084" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/ce7236/48.png" class="avatar"> HmmmmML:</div>
<blockquote>
<pre><code class="lang-auto">        self._delay = torch.nn.Parameter(torch.FloatTensor([offset]))
...
        self._write_ptr = int(self._delay.item())
...
            # Write
            self.circ_buffer[self._write_ptr] = x_i
            # Read (with linear interpolation)
            i0 = np.floor(self._read_ptr).astype(int)
            i1 = (i0 + 1) % self.N
            frac = self._read_ptr - np.floor(self._read_ptr)
            y[i] = self.circ_buffer[i0] + frac*(self.circ_buffer[i1]-self.circ_buffer[i0])
</code></pre>
</blockquote>
</aside>
<p>Although <code>_delay</code> is a <code>Parameter</code> and potentially trainable, at a technical<br>
level, as soon as you call <code>.item()</code> on it you “break the computation graph,”<br>
as the result of <code>.item()</code> is no longer a pytorch tensor and therefore no<br>
longer participates in autograd.</p>
<p>At a more conceptual level, you then call python’s <code>int()</code> (on the result of<br>
<code>.item()</code>).  <code>int()</code> is not (usefully) differentiable, so even if you performed<br>
that logic using pytorch tensor operations, you would still not be able to<br>
backpropagate gradients through that step in the computation.</p>
<p>I don’t understand your use case, so I don’t know whether this would<br>
work, but, at least in terms of differentiability, you might try performing<br>
an “interpolated write,” analogously to the interpolated read that you<br>
have implemented.</p>
<p>Best.</p>
<p>K. Frank</p>

username: HmmmmML
cooked: <p>Thank you K. Frank, that was exactly the problem.</p>
<p>Instead of adding an “interpolated write” (which is indeed a bit odd), I re-worked things to offset the read pointer by <code>_delay</code>, and reworked the read accordingly:</p>
<pre><code class="lang-auto">i0 = torch.floor(self._read_ptr).data[0].long()
i1 = (i0 + 1) % self.N
frac = self._read_ptr - torch.floor(self._read_ptr)
y[i] = self.circ_buffer[i0] + frac*(self.circ_buffer[i1]-self.circ_buffer[i0])
</code></pre>
<p>Now, it converges nicely (although slightly slowly).</p>