Model parameters are not updating

Rgb1998: <p>Hello,</p>
<p>I am trying two implement two NNs joined by one loss function, where the loss function involves performing PCA_inverse on the output of each neural network. The code is as follows:</p>
<p>for epoch in range(num_epochs):<br>
batch_idx = 0</p>
<pre><code>model_diff.train()
model_conv.train()

for (features_diff, _), (features_conv, _), (_, targets_base) in zip(train_loader_diff, train_loader_conv, 
train_loader):

  model_diff.train()
  model_conv.train()

  batch_idx += 1

  features_diff = features_diff.to(device)
  features_conv = features_conv.to(device)
  targets_base = targets_base.to(device)

  # FORWARD AND BACK PROP

  model_diff.requires_grad = True
  model_conv.requires_grad = True

  diff_part = model_diff(features_diff)

  conv_part = model_conv(features_conv)
 
  for (param_diff, param_conv) in zip(model_diff.parameters(), model_conv.parameters()):
    param_diff.retain_grad()
    param_conv.retain_grad()

  with torch.no_grad():

    predictions = pca_y_diff.inverse_transform(model_diff(features_diff).clone().cpu().numpy()) + pca_y_conv.inverse_transform(model_conv(features_conv).clone().cpu().numpy())
    predictions = torch.from_numpy(predictions)
    predictions = predictions.to(device)
    predictions = torch.reshape(predictions, (-1, 1, 256, 256))

  loss = loss_fn(predictions, targets_base)

  optimizer_diff.zero_grad()
  optimizer_conv.zero_grad()

  loss.requires_grad = True
  loss.backward()
  

  # UPDATE MODEL PARAMETERS
  optimizer_diff.step()
  optimizer_conv.step()
</code></pre>
<p>The problem is that the parameters do not update at all. If I don’t include ''loss.requires_grad = True" I get the error “[RuntimeError: element 0  does not require grad and does not have a grad_fn]”</p>
<p>Any help is much appreciated</p>
<p>Many thanks</p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="Rgb1998" data-post="1" data-topic="153990">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/e47774/48.png" class="avatar"> Rgb1998:</div>
<blockquote>
<pre><code class="lang-auto">with torch.no_grad():

    predictions = pca_y_diff.inverse_transform(model_diff(features_diff).clone().cpu().numpy()) + pca_y_conv.inverse_transform(model_conv(features_conv).clone().cpu().numpy())
    predictions = torch.from_numpy(predictions)
    predictions = predictions.to(device)
    predictions = torch.reshape(predictions, (-1, 1, 256, 256))
</code></pre>
</blockquote>
</aside>
<p>Your predictions are being calculated within a <code>torch.no_grad()</code> context manager and therefore won’t have a <code>grad_fn</code>, can you share the forward and backward part of the code too?</p>

Rgb1998: <p>here is the forward part:</p>
<p>##########################</p>
<h3>
<a name="model-1" class="anchor" href="#model-1"></a>MODEL</h3>
<p>##########################</p>
<p>class Reshape(nn.Module):<br>
def <strong>init</strong>(self, *args):<br>
super().<strong>init</strong>()<br>
self.shape = args</p>
<pre><code>def forward(self, x):
    return x.view(self.shape)
</code></pre>
<p>class Trim(nn.Module):<br>
def <strong>init</strong>(self, *args):<br>
super().<strong>init</strong>()</p>
<pre><code>def forward(self, x):
    return x[:, :, :256, :256]
</code></pre>
<p>class MLP(nn.Module):</p>
<pre><code>def __init__(self, n_components_x, n_components_y):
    super().__init__()
    
    self.ANN = nn.Sequential( #784
            nn.Linear(n_components_x, 100),
            nn.LeakyReLU(0.01),
            nn.Linear(100, 100),
            nn.LeakyReLU(0.01),
            nn.Linear(100, 100),
            nn.LeakyReLU(0.01),
            nn.Linear(100, 100),
            nn.LeakyReLU(0.01),
            nn.Linear(100, n_components_y)
            )

def forward(self, x):
    x = self.ANN(x)
    return x
</code></pre>
<p>I tried removing the torch.no_grad() context manager, and now I get the following error:<br>
“Can’t call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.”</p>
<p>When I then use .detach() in the first predictions line,  my parameters once again do not update</p>

AlphaBetaGamma96: <p>Yes, so if you’re using a function outside of the torch environment (i.e. <code>numpy</code>) pytorch can’t track operation that leave pytorch. When calling <code>.detach()</code> you break your computational graph and hence have no gradient. In this case you have two options,</p>
<ol>
<li>Find an equivalent function within the torch environment that will automatically propagate gradients.</li>
<li>Define a custom <code>torch.autograd.Function</code> object and manually define both the forward and backward methods respectively in order to propagate gradients.</li>
</ol>