Gradient zero for one parameter in custom model resulting in no update

mscipio: <p>Hi! Here I am with a (maybe) rather weird problem.</p>
<p>So, the big picture is that I am trying to use Pytorch’s optimizers to perform  <strong>non-linear curve fitting</strong>. I have an overall code that is working, but now I need to tweek things to actually work with the model I am interested in.</p>
<p>I will share here a simplified model, to make it easier to understand the issue.<br>
Let’s consider this image:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/4/64fccbcdc5744ad9e1f7e3015ffe4bb0788a9885.png" alt="download" data-base62-sha1="epnmkE2sgnVK2xI7S35TjqLIrzf" width="372" height="248"></p>
<p>The cyan curve is the “original” input. It known and measured.<br>
The other 4 curves are obtained by scaling in aplitude and translating in time the first one. So our model has 2 parameters: onset time (i.e. temporal translation, <strong><code>ton</code></strong>) and  scaling (<strong><code>vp</code></strong>).</p>
<p><strong>TRUE VALUES</strong></p>
<pre><code class="lang-auto">tensor([[ 5.0000,  0.5000],
        [ 5.0000,  0.7500],
        [15.0000,  0.5000],
        [15.0000,  0.7500]])
</code></pre>
<p><strong>ESTIMATES</strong></p>
<pre><code class="lang-auto">tensor([[0.0000, 0.4203],
        [0.0000, 0.6305],
        [0.0000, 0.3271],
        [0.0000, 0.4906]])
</code></pre>
<p>As you can see, the first parameter (<strong><code>ton</code></strong>) is not updated at all from its starting value. And that obviuosly affects also the estimate of the amplitude  (<strong><code>vp</code></strong>).</p>
<p>I am quite sure that the way I coded my model is such that autograd is unable to compute ita gradient correctly. But I am kind of out of ideas, since I am just trying to port this code from Matlab to pytorch, and with a similar implementation Matlab is able to optimize the value of that parameter.</p>
<p>Here is the whole code I am testing.  It should be a self contained working example.<br>
Any idea will be <em><strong>greatly</strong></em> appreciated!!</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn as nn
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(device)
import matplotlib.pyplot as plt
from tqdm import tqdm

class Net(nn.Module):
    def __init__(self, p, t, IF, y):
        super().__init__()
        self.pars = nn.Parameter(p)
        self.t = t 
        self.IF = IF
        self.y = y 

    def forward(self):
        self.pars.detach()
        return self.cost(self.pars)
    
    def cost(self,p):
        out = kmodel(p,self.t,self.IF) - self.y
        return torch.sum(out**2)

################
def kmodel(p,t,Cp,device=0):
    torch.autograd.set_detect_anomaly(True)
    Nv = p.shape[0]
    Nt = t.shape[1]
    
    ton = p[:,0].view(Nv,1).int()
    vp  = p[:,1].view(Nv,1).to(device)
    
    # this is how I tried to introduce the temporal shift
    # and here is where I probably lost Autograd ...
    for i in range(Nv):
        Cp[i,ton[i]:] = Cp[i,:Nt-ton[i]]
        Cp[i,:ton[i]] = 0

    out = vp*Cp
    return out

##################
# Simulate data
##################

n = 2
p0 = torch.linspace(5, 15, n) # ton
p1 = torch.linspace(0.5, 0.75, n) # vp
p_true = []
for i in range(n):
    for j in range(n):
                p_true.append([p0[i], p1[j]])
        
p_true = torch.tensor(p_true).to(device)     
Nv = p_true.shape[0]
print(Nv)

IF = torch.tensor([[0.0309, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0220, 0.0000, 0.0000,
         0.0074, 0.0000, 0.3594, 2.0850, 2.6615, 2.7160, 1.8999, 1.3524, 1.5733,
         1.8137, 1.7839, 1.4743, 1.4680, 1.4860, 1.3490, 1.3471, 1.3055, 1.3520,
         1.2053, 1.3447, 1.2073, 1.2478, 1.3872, 1.2890, 1.1951, 1.1463, 1.2195,
         1.1701, 1.1634, 1.1765, 1.0766, 1.2907, 1.1409, 1.0373, 1.0479, 1.0766,
         1.0842, 1.0371, 1.0241, 1.0439, 0.9803, 0.9966, 1.1955, 0.9665, 0.9679,
         1.0223, 0.9714, 0.9415, 1.0756, 1.0336, 0.9744, 0.9825, 1.0016, 0.9592,
         0.8908, 0.9362, 0.9347, 0.9198, 0.9353, 0.9279, 1.0776]]).to(device)   
IF = torch.repeat_interleave(IF, repeats=Nv, dim=0).to(device)

t = torch.unsqueeze(torch.linspace(0, 2400*150/1000/60, 70), dim=0).to(device)
t = torch.repeat_interleave(t, repeats=Nv, dim=0)

y_true = kmodel(p_true,t.clone(),IF.clone()).to(device)
y = y_true + 0.00*torch.randn(y_true.size()).to(device)

plt.figure()
plt.plot(t.cpu().numpy()[0,:].T,IF.cpu().numpy().T,'-*', color='C9')
plt.plot(t.cpu().numpy()[:16,:].T,y.cpu().numpy()[:16,:].T);
plt.show()

##################
# Optimization
##################

def loss(y_pred, y_true):
    return y_pred

training_steps = 10000
lr = 0.01
p = torch.tensor([[0.0,0.0]]*Nv)
model = Net(p,t, IF, y)

with tqdm(total=training_steps) as pbar:
    for epoch in range(training_steps ):
        # trying to change the value of the learning rate 
        # it didn't really help ...
        if epoch % 1000 == 0:
            lr *= 0.1
            optim = torch.optim.Adamax(model.parameters(), lr = lr)
        optim.zero_grad()
        ypred = model()
        ll = loss(ypred, _)
        ll.backward()
        optim.step()
        
        # progress bar settings
        pbar.set_postfix(loss=ll.item(),p_est=list(model.parameters())[0].data[0], lr = lr, refresh=False)
        pbar.update(1)
        if ll.item() &lt;= 1e-4:
            break

p_est = list(model.parameters())[0].data
print(p_est)
print(p_true)

y_est = kmodel(p_est,t,IF)

plt.figure(figsize=(20,10))
plt.plot(t.cpu().numpy()[0,:].T,IF.cpu().numpy().T, color='C4');
plt.plot(t.cpu().numpy()[0:16,:].T,y.cpu().numpy()[0:16,:].T, '-*',color='C1');
plt.plot(t.cpu().numpy()[0:16,:].T,y_est.cpu().numpy()[0:16,:].T, color='C2');

</code></pre>

albanD: <p>Hi,</p>
<p>Trying to run your code on <code>cpu</code>, I get:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "foo.py", line 71, in &lt;module&gt;
    y_true = kmodel(p_true,t.clone(),IF.clone())
  File "foo.py", line 36, in kmodel
    Cp[i,ton[i]:] = Cp[i,:Nt-ton[i]]
RuntimeError: unsupported operation: some elements of the input tensor and the written-to tensor refer to a single memory location. Please clone() the tensor before performing the operation.
</code></pre>
<p>That might be a problem <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

mscipio: <p>I tried  to modify my model function with a <code>.clone()</code> at the line you suggested, but with no change:</p>
<pre><code class="lang-auto">def kmodel(p,t,Cp,device=0):
    torch.autograd.set_detect_anomaly(True)
    Nv = p.shape[0]
    Nt = t.shape[1]
    
    ton = p[:,0].view(Nv,1).int()
    vp  = p[:,1].view(Nv,1).to(device)
    
    for i in range(Nv):
        Cp[i,ton[i]:] = Cp[i,:Nt-ton[i]].clone()  # &lt;==========
        Cp[i,:ton[i]] = 0

    out = vp*Cp
    return out
</code></pre>
<p>But even if I run the code with <code>device='cpu'</code>, I don’t get that warning/error …</p>

albanD: <p>For the other issue of not learning, the problem is that your functions is not differentiable wrt to the shift as you use it. Here you use it as an integer index. So pytorch cannot compute gradient for it.</p>
<p>What kind of gradients were you expecting here?</p>
<p>For the absence of warning/error, it might be because your run an old version of pytorch? I ran the code from the latest nightly build.</p>

mscipio: <p>(Assuming I have not simplified my problem too much to actually benefit from a solution) the model should be something like:</p>
<p><code>y_est[ t ] = vp * y[t -  ton]</code></p>
<p><strong><code>ton</code></strong> doesn’t need to be integer (actually it should be a float!) but it seems I am not able to find a way to make it work. I also tried to pass <strong><code>ton</code></strong> as a float, and then to “find” the index with a value of the time vector closest to <strong><code>ton</code></strong>, but even that solution gave the same result</p>
<blockquote>
<p>For the absence of warning/error, it might be because your run an old version of pytorch? I ran the code from the latest nightly build.</p>
</blockquote>
<p>Yes, unfortunately I am forced to run v1.1 because of issue with the nvidia cuda diver on my work pc … :\</p>

albanD: <p>One solution might be to reuse code from the spatial transformer:</p>
<pre><code class="lang-python">    # Make it look like a 2D data: (batch, chan, h, w) as (batch_size, 1, Cp.size(1), 1)
    Cp = Cp.unsqueeze(-1).unsqueeze(1)
    affine_transfo = torch.zeros(batch_size, 2, 3)
    # The affine transformation we want is identity + translation
    affine_transfo[:, 0, 0] = 1
    affine_transfo[:, 1, 1] = 1
    affine_transfo[:, 1, 2] = -2 * ton / height
    grid = F.affine_grid(affine_transfo, (batch_size, 1, height, 1), align_corners=True)
    Cp = F.grid_sample(Cp, grid, align_corners=True)
    Cp = Cp.squeeze(1).squeeze(-1)
</code></pre>
<p>That will get you gradients to the parameters. But not sure how good it’s gonna be.<br>
Also I am <strong>absolutely</strong> not sure how the <code>align_corners</code> flag should be used. So you should double checkt that this is doing what you want <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

mscipio: <p>Thank you <strong>sooo</strong> much for your time and effort!<br>
I was reading about grid_sample right when I got the notification about your message! ^^</p>
<p>This version of the model (teeny tiny change wrt to your suggestion) seems to work (and converges super fast!!:</p>
<pre><code class="lang-python">def kmodel(p,t,Cp,device=0):
    torch.autograd.set_detect_anomaly(True)
    Nv = p.shape[0]
    Nt = t.shape[1]
    
    ton = p[:,0].clone()
    vp  = p[:,1].clone().view(Nv,1).to(device)
        
    Cp = Cp.unsqueeze(-1).unsqueeze(1)
    affine = torch.zeros(Nv, 2, 3)
    # The affine transformation we want is identity + translation
    affine[:, 0, 0] = 1
    affine[:, 1, 1] = 1
    affine[:, 1, 2] = -ton #-2 * ton / Nt
    
    grid = F.affine_grid(affine, (Nv, 1, Nt, 1))#, align_corners=True)
    Cp = F.grid_sample(Cp, grid) #, align_corners=True)
    Cp = Cp.squeeze(1).squeeze(-1)

    out = vp*Cp
    return out
</code></pre>
<p>Now I need to check if I can grow this toy-model to what I actually need to fit, but I am really optimistic now! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

mscipio: <aside class="quote no-group" data-username="albanD" data-post="6" data-topic="81091">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>Also I am <strong>absolutely</strong> not sure how the <code>align_corners</code> flag should be used. So you should double checkt that this is doing what you want</p>
</blockquote>
</aside>
<p>In pytorch 1.1 I don’t have that input parameter, so I just removed it and it didn’t complain … <img src="https://discuss.pytorch.org/images/emoji/apple/eyes.png?v=12" title=":eyes:" class="emoji" alt=":eyes:" loading="lazy" width="20" height="20"></p>