PyTorch optimizer.step() function doesn't update weights

username: Jean-Rassaire
cooked: <p>I am new to Pytorch and I tried to use SGD to perform a fitting using a statistical model .<br>
But the problem is that the <code>optimizer.step()</code> part does not work. I output <code>optimizer parameters()</code> after each epoch, and the weights do not change.</p>
<p>here is my code:</p>
<p>shape_parameters_estimated = torch.zeros([batch, stat8model.variance.size], dtype=torch.float32, device=device, requires_grad=True)</p>
<p>learning_rate = 0.1<br>
optimizer = torch.optim.Adagrad([shape_parameters_estimated], lr=learning_rate)<br>
max_iter = 20<br>
for iteration in range(max_iter):<br>
theta_estimated = shape_parameters_estimated</p>
<pre><code>fit_image, fit_landmarks = stat_model(theta_estimated)

loss_lm = lm_loss(target_landmark, fit_landmarks)
print('theta=', theta_estimated[0]) #I'am printing the parameter here, but doesn't change
loss_value = loss_lm
optimizer.zero_grad()
loss_value.backward()
optimizer.step()
</code></pre>
<p>Please, can anyone help?</p>

username: ptrblck
cooked: <p>Your code works fine after replacing the undefined classes:</p>
<pre><code class="lang-python">shape_parameters_estimated = torch.zeros(1, 1, dtype=torch.float32, requires_grad=True)
stat_model = nn.Linear(1, 1)

learning_rate = 0.1
optimizer = torch.optim.Adagrad([shape_parameters_estimated], lr=learning_rate)
max_iter = 20
criterion = nn.MSELoss()

for iteration in range(max_iter):
    theta_estimated = shape_parameters_estimated
    out = stat_model(theta_estimated)

    loss = criterion(out, torch.rand_like(out))
    print('theta=', theta_estimated[0])
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>
<p>Output:</p>
<pre><code class="lang-python">theta= tensor([0.], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.1000], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.1397], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.1831], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.2331], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.2467], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.2960], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.3146], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.3339], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.3832], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.4118], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.4494], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.4717], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.4924], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.5206], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.5551], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.5686], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.5911], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.6077], grad_fn=&lt;SelectBackward0&gt;)
theta= tensor([0.6128], grad_fn=&lt;SelectBackward0&gt;)
</code></pre>
<p>Maybe you are detaching the computation graph in one of your modules.</p>

username: thecho7
cooked: <p>If you want to get the loss value,</p>
<pre><code class="lang-auto">loss_value = loss_lm.data()
loss_lm.backward()
</code></pre>
<p>Other issues are clearly explained by <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: Jean-Rassaire
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I checked the output of my statistics model to see if I am detaching the  computation graph, but in fact it doesn’t seem to be detached.</p>
<p>here is the code and the output of stat_model as well as the parameter.</p>
<pre><code>fit_image, fit_landmarks = stat_model(theta_estimated)
print('fit landmark =', fit_landmarks)
print('theta=', theta_estimated[0])
</code></pre>
<blockquote>
<p>fit landmark = tensor([[[ -93.7965,   41.2901, -457.3236, -118.0355,   23.2566, -394.9610,<br>
-129.1178,  -22.9549, -413.0228,  -60.4286],<br>
[ -14.2109, -392.8283, -112.7418,   32.2946, -766.2040,  -65.1696,<br>
31.8340, -795.4185,  -35.9762,   -2.9846],<br>
[-788.4980, -108.9594,   -9.7898, -800.8992,  -49.7171,   31.8733,<br>
-808.2354,  -93.7749,   49.2322, -800.7413]],</p>
</blockquote>
<pre><code>    [[ -86.7400,   28.3823, -492.1542,  -93.9346,   29.3518, -440.1688,
      -111.6750,    2.4048, -448.3925,  -44.6627],
     [ -23.6792, -451.9865, -117.1925,   11.8526, -759.7421,  -62.1217,
        39.6679, -760.5637,  -32.7417,   18.6626],
     [-752.7242,  -92.1307,  -17.7508, -781.5572,  -45.2557,   43.6841,
      -768.3282, -102.0547,   38.3073, -769.6452]],

    [[ -78.7095,   17.3260, -457.3948, -111.3670,   22.8568, -394.5397,
      -122.8169,  -18.9796, -403.8733,  -52.9702],
     [ -10.9037, -390.0055, -114.4793,   26.6313, -784.2676,  -56.4531,
        25.9526, -799.3102,  -33.1011,    2.2568],
     [-785.2794, -100.1249,  -14.8437, -806.3973,  -41.8734,   26.0318,
      -805.1998,  -95.0777,   42.2077, -806.0328]]], device='cuda:0',
   grad_fn=&lt;ReshapeAliasBackward0&gt;).
</code></pre>
<blockquote>
<p>theta= tensor([-1.6324, -3.0613,  1.1880, -1.6660, -0.3259, -4.0532,  2.8580, -2.3427,<br>
2.1786,  2.7516, -0.2029, -5.1932, -1.2288, -2.5457,  0.9505, -1.3430,<br>
0.4178, -1.9190,  1.5987,  1.4101, -1.3779, -2.0695, -3.3447,  2.0617,<br>
-3.6703,  2.3111, -2.0738, -7.8953,  1.4093,  0.3684, -0.1465, -5.3603,<br>
3.9112, -1.7550,  3.0453, -3.8051, -2.8779, -0.9223, -2.2068,  0.7696,<br>
-3.6210, -0.1781,  1.4597, -0.9524,  0.5832, -3.4065, -3.0134, -2.4941],<br>
device=‘cuda:0’, grad_fn=)</p>
</blockquote>
<p>you can see that grad_fn= for the output used for the loss and grad_fn= for the parameter.</p>
<p>what else could be detached?</p>

username: ptrblck
cooked: <p><code>theta_estimated</code> is assigned to <code>shape_parameters_estimated</code>, which was initialized with <code>torch.zeros</code>. In your current output <code>theta_estimated</code> shows values which are different from zeros, so I guess that it was already updated?</p>

username: Jean-Rassaire
cooked: <p>Thanks again for your prompt reply <a class="mention" href="/u/ptrblck">@ptrblck</a> .<br>
The current value of theta_estimated is the initial value. I didn’t initialize with zero, I initialized with that random tensor. So it is not updating.  Assigning shape_parameters_estimated to theta_estimated shouldn’t be a problem, right?</p>

username: ptrblck
cooked: <p>In your posted code snippet you’ve initialized it as:</p>
<pre><code class="lang-python">shape_parameters_estimated = torch.zeros([batch, stat8model.variance.size], dtype=torch.float32, device=device, requires_grad=True)
</code></pre>
<p>Could you post a minimal, executable code snippet to reproduce the issue, please?</p>

username: Jean-Rassaire
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>The problem is that the statistical model required local libraries. So it’s not possible to have those libraries online.<br>
But here is more detailed code:</p>
<p>device = torch.device(‘cuda’)<br>
torch.manual_seed(0)</p>
<p>print(‘FITTING DEMO … (does not include pose or camera parameters)’)<br>
print(’============’)<br>
batch = 3</p>
<h1>
<a name="image_width-224-1" class="anchor" href="#image_width-224-1"></a>image_width = 224</h1>
<h1>
<a name="image_height-224-2" class="anchor" href="#image_height-224-2"></a>image_height = 224</h1>
<p>number_of_channels = 3</p>
<p>voxel_size = 2.0</p>
<p>view1_alpha = 0<br>
view1_beta = -90</p>
<p>view2_alpha = -90<br>
view2_beta = -90</p>
<p>number_of_channels = number_of_channels</p>
<p>transform = transforms.Compose([<br>
transforms.ToPILImage(),<br>
transforms.ToTensor(),</p>
<p>])</p>
<p>femur_ids = []<br>
with open(‘data/femur_landmark_id.csv’, mode=‘r’) as inp:<br>
reader = csv.reader(inp, delimiter=’,’)<br>
for rows in reader:<br>
femur_ids.append(int(float(rows[1])))</p>
<p>tibia_ids = []<br>
with open(‘data/tibia_landmark_id.csv’, mode=‘r’) as inp:<br>
reader = csv.reader(inp, delimiter=’,’)<br>
for rows in reader:<br>
tibia_ids.append(int(float(rows[1])))</p>
<p>carm = MobileCArm(source_to_detector_distance=40,<br>
immersion_depth=0,<br>
free_space=0,<br>
sensor_height=200,<br>
sensor_width=100)</p>
<p>dmfcModel = StatismoFileReaderDmfc.readDmfcModel(“data/models/kneeDmfc.h5”)</p>
<p>decoder = decoder = DecoderDmfc(dmfcModel,<br>
carm,<br>
first_view_alpha=view1_alpha,<br>
first_view_beta=view1_beta,<br>
second_view_alpha=view2_alpha,<br>
second_view_beta=view2_beta,<br>
number_of_channels=number_of_channels,<br>
transform=transform,<br>
voxel_size=voxel_size,<br>
landmark_indices=[femur_ids, []]<br>
).to(device=device)</p>
<p>tparam = []<br>
for i in range(batch):<br>
alpha = np.random.normal(np.zeros(1), 2.0, dmfcModel.feature_classes.variance.size)<br>
tparam.append(alpha)</p>
<p>shape_parameters = torch.tensor(tparam, dtype=torch.float32, device=device)  # torch.randn([batch, dmfcModel.feature_classes.variance.size], dtype=torch.float32, device=device)</p>
<p>theta = shape_parameters<br>
target_image, target_landmarks = decoder(theta)</p>
<p>param = []<br>
for i in range(batch):<br>
alpha = np.random.normal(np.zeros(1), 1.5 + 1.0 / (i + 1), dmfcModel.feature_classes.variance.size)<br>
param.append(alpha)</p>
<p>shape_parameters_estimated = torch.tensor(param, dtype=torch.float32, device=device, requires_grad=True)</p>
<h1>
<a name="setup-losses-we-use-3" class="anchor" href="#setup-losses-we-use-3"></a>setup losses we use</h1>
<p>loss = nn.MSELoss()<br>
prior_loss = GaussianPrior()<br>
lm_loss = LandmarkLoss()</p>
<h1>
<a name="setup-visual-reporting-4" class="anchor" href="#setup-visual-reporting-4"></a>setup (visual) reporting</h1>
<p>report_interval = 1<br>
fig, axs = pyplot.subplots(batch, 3)</p>
<p>print(‘LANDMARK FITTING … (not optimizing for color or light)’)<br>
print(’================’)<br>
learning_rate = 1<br>
optimizer = torch.optim.Adagrad([<br>
shape_parameters_estimated],<br>
lr=learning_rate)<br>
max_iter = 5<br>
for iteration in range(max_iter):<br>
theta_estimated = shape_parameters_estimated</p>
<pre><code>fit_image, fit_landmarks = decoder(theta_estimated)



loss_lm = loss(target_landmarks, fit_landmarks)
print('fit land', fit_landmarks)
print('theta=', theta_estimated[0])

loss_value = loss_lm 

loss_value.backward()
optimizer.step()
optimizer.zero_grad()</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Jean-Rassaire" data-post="8" data-topic="141867">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jean-rassaire/48/45317_2.png" class="avatar"> Jean-Rassaire:</div>
<blockquote>
<p>The problem is that the statistical model required local libraries.</p>
</blockquote>
</aside>
<p>In case you are using other libraries and are “leaving” PyTorch, note that this will detach the computation graph as well (e.g. if you are using numpy). In these cases you would have to implement a custom <code>autograd.Function</code> with the manual definition of the <code>backward</code> pass.<br>
Also check if you are rewrapping any tensors via <code>x = torch.tensor(x, requires_grad=True)</code> as this will also detach the tensor.</p>

username: Jean-Rassaire
cooked: <p>Thanks again <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Yes, I am using Numpy.<br>
here an example of what I am doing to create a tensors from  list and numpy.<br>
drr = Variable(drr, requires_grad=theta.requires_grad)<br>
landmark_batch = Variable(torch.tensor(landmark_batch, dtype=torch.float32, device=theta.device), requires_grad=theta.requires_grad)</p>
<p>Would this detach drr and landmark_batch? if yes, how can I write a custom <code>autograd.Function</code> to create tensors from numpy and list?</p>

username: ptrblck
cooked: <p>Yes, both operations are rewrapping the tensors and are thus detaching them (besides <code>Variable</code>s being deprecated since PyTorch <code>0.4.</code> <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:"> ).<br>
Check <a href="https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html">this tutorial</a> on how to write custom <code>autograd.Function</code>s.</p>

username: Jean-Rassaire
cooked: <p>Thanks So much <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>I have written a minimal, executable code snippet to reproduce the issue.</p>
<p>Here is the code, you will see that the parameter is not updating. You can run it directly, without the need for additional libraries.</p>
<p>import numpy as np<br>
import torch<br>
import torch.nn as nn</p>
<p>from torch.autograd import Variable</p>
<p>class Torch_stat_model(nn.Module):</p>
<pre><code>def __init__(self, initial_landmarks):  # landmark size (batch_size nx3)
    super(Torch_stat_model, self).__init__()
    self.initial_land = initial_landmarks

def forward(self, shape_params):  # shape-param size (batch_size nx1)
    land_numpy = (shape_params * self.initial_land).cpu().detach().numpy()
    return land_numpy
</code></pre>
<p>class Decoder_stat_model(nn.Module):</p>
<pre><code>def __init__(self, initial_landmarks, scale=1.0):
    super(Decoder_stat_model, self).__init__()
    self.scale = scale
    self.model = Torch_stat_model(initial_landmarks)

def forward(self, theta):
    land_detached = self.model(theta)
    land_tensor = Variable(torch.from_numpy(land_detached).to(dtype=torch.float32).to(device=theta.device),
                           requires_grad=theta.requires_grad)
    return land_tensor * self.scale
</code></pre>
<p>device = torch.device(‘cuda’)<br>
torch.manual_seed(0)</p>
<p>batch = 3</p>
<p>initial_landmark = torch.rand((batch, 48, 3)).to(device=device)</p>
<p>decoder = decoder = Decoder_stat_model(initial_landmark).to(device=device)</p>
<p>tparam = []<br>
for i in range(batch):<br>
alpha = np.random.normal(np.zeros(1), 2.0, 48)<br>
tparam.append(alpha)</p>
<p>shape_parameters = torch.tensor(tparam, dtype=torch.float32,<br>
device=device)</p>
<p>theta = torch.unsqueeze(shape_parameters, 2)<br>
print(theta.shape)<br>
target_landmarks = decoder(theta)</p>
<p>param = []<br>
for i in range(batch):<br>
alpha = np.random.normal(np.zeros(1), 1.5 + 1.0 / (i + 1), 48)<br>
param.append(alpha)</p>
<p>shape_parameters_estimated = torch.zeros([batch, 48, 1], dtype=torch.float32, device=device, requires_grad=True)</p>
<p>loss = nn.MSELoss()</p>
<p>report_interval = 1<br>
learning_rate = 1<br>
optimizer = torch.optim.Adagrad([<br>
shape_parameters_estimated],<br>
lr=learning_rate)<br>
max_iter = 5<br>
for iteration in range(max_iter):<br>
theta_estimated = shape_parameters_estimated</p>
<pre><code>fit_landmarks = decoder(theta_estimated)
print(theta_estimated[0][0])

loss_lm = loss(target_landmarks, fit_landmarks)

loss_value = loss_lm

loss_value.backward()
optimizer.step()
optimizer.zero_grad()

# visualization
if iteration % report_interval == 0:
    print(f'epoch {iteration + 1}: loss={loss_value:.8f}')
</code></pre>
<p>You mentioned writing a custom `autograd.Function, how would I include that in the Decoder_stat_model class to attach the numpy array to the computation graph?</p>

username: Jean-Rassaire
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> , It is important to note that this operation (shape_params * self.initial_land).cpu().detach().numpy() in the Torch_stat_model class, is just to illustrate that I have to deal with numpy arrays and lists. I know that in this minimal code we can bypass this operation but it is just to show the type of operation that I have in my code. Because of the local libraries I use, I have to deal with numpy arrays and lists.</p>
<p>Thanks again. Any help with be appreciated.</p>

username: ptrblck
cooked: <p>This use case fits exactly into the custom <code>autograd.Function</code> usage, so you would have to write the <code>backward</code> method manually as described in the tutorial.<br>
Autograd cannot track the numpy operations (or generally any operations from other libraries) which is why the <code>backward</code> is needed.</p>