Optimizing Model Parameters Issue with 'loss.backward()' Function

CuteDeadu: <pre><code class="lang-auto">import gc
from tqdm.notebook import tqdm
import matplotlib.pyplot as plt
torch.set_printoptions(sci_mode=False)
 
gradient_norms = []
losses = []

for epoch in tqdm(range(epochs)):
    model_path = f"/kaggle/working/Training/ace_state_dict_{epoch+1}.pth"
    torch.save(model.state_dict(), model_path)
    model.train()
    total_loss = 0

    for batch_idx, batch in enumerate(tqdm(train_dataloader, desc=f'Epoch {epoch + 1}/{epochs}')):
        optimizer.zero_grad()
    
        logits = model(batch["inputs"])
        targets = batch["targets"]

        loss = loss_fn(logits.view(-1, logits.size(-1)), targets.float()) / 1000000000
        loss.backward()

        # Compute gradient norms
        grad_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), 1)
        gradient_norms.append(grad_norm)

        optimizer.step()
        scheduler.step()
        total_loss += loss.item()

        if batch_idx % 100 == 0:
            print(f'Batch {batch_idx}/{len(train_dataloader)}, Loss: {total_loss/(batch_idx+1)}, Gradient Norms: {grad_norm}')
 
    avg_loss = total_loss / len(train_dataloader)
    losses.append(avg_loss)

    print(f"Train loss: {avg_loss}")
</code></pre>
<p>In my code above, the “loss.backward()” function is not working as expected, resulting in a “grad_norm” consistently registering as “0.0”. This impedes the optimization of the model’s parameters during each iteration. Any guidance or suggestions to rectify this matter would be greatly appreciated. For reference, you can find the full source code of my model at:</p>
<p><a href="https://www.kaggle.com/code/cutedeadu943/transformerchatbot" class="inline-onebox" rel="noopener nofollow ugc">TransformerChatbot | Kaggle</a></p>
<p>I’ve experimented with various combinations of hyperparameters and learning rates, but unfortunately, the issue persists. Additionally, I attempted to address it by employing “torch.nn.utils.clip_grad_norm_()”, but to no avail. All the relevant tensors and the model’s parameters are set with “requires_grad=True”. Any insights or alternative approaches to resolve this</p>

AlphaBetaGamma96: <p>Hi <a class="mention" href="/u/cutedeadu">@CuteDeadu</a>,</p>
<p>Do you need to scale your loss values by a factor of 1000000000? Surely this would make any gradient values near 0? Also, could you try casting your targets as type <code>Tensor</code> rather than type <code>float</code>?</p>
<aside class="quote no-group" data-username="CuteDeadu" data-post="1" data-topic="202148">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cutedeadu/48/61976_2.png" class="avatar"> CuteDeadu:</div>
<blockquote>
<pre><code class="lang-auto">        loss = loss_fn(logits.view(-1, logits.size(-1)), targets.float()) / 1000000000
</code></pre>
</blockquote>
</aside>

CuteDeadu: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/2/4297e1e82d8b61349ff5387917c23c72047f8ffc.png" alt="Screenshot_20240506-183603~2" data-base62-sha1="9v6V1jhaHdBhCkBS2UtXpvH18hS" width="628" height="346"></p>
<p>I’ve attempted training without dividing the loss by 10^9, but unfortunately, encountered the same issue. Regarding your query about casting targets as type Tensor rather than type float, I’ve experimented with both approaches, yet the problem persists. The reason for initially dividing by 10^9 is because the loss values tend to be quite large</p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="CuteDeadu" data-post="1" data-topic="202148">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cutedeadu/48/61976_2.png" class="avatar"> CuteDeadu:</div>
<blockquote>
<p><code>loss_fn</code></p>
</blockquote>
</aside>
<p>What is the loss_fn here?</p>

CuteDeadu: <p>It’s nn.CrossEntropyLoss() function and the optimizeris Adam()</p>

AlphaBetaGamma96: <p>For the <code>nn.CrossEntropyLoss()</code> function, doesn’t the input for that loss function need to be the normalized probability from the Softmax function (rather than the logits?), as shown in the docs: <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html" class="inline-onebox" rel="noopener nofollow ugc">CrossEntropyLoss — PyTorch 2.3 documentation</a></p>
<p>That might explain why your initial loss is too high, and this would also allow you to remove the <code>1e9</code> factor.</p>

CuteDeadu: <p>Thank you so much for your suggestion! I adjusted the input for the <code>nn.CrossEntropyLoss()</code> function to be the normalized probability from the Softmax function instead of the logits, as per the documentation you provided. This solution indeed resolved the issue, and I was able to remove the <code>1e9</code> factor. I sincerely appreciate your time and assistance in helping me address this problem.</p>

ptrblck: <aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="6" data-topic="202148">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p>For the <code>nn.CrossEntropyLoss()</code> function, doesn’t the input for that loss function need to be the normalized probability from the Softmax function (rather than the logits?), as shown in the docs:</p>
</blockquote>
</aside>
<p>No, the inputs (i.e. model outputs) are supposed to be logits as internally <code>F.nll_loss(F.log_softmax())</code> will be used.<br>
From the docs:</p>
<blockquote>
<p>The input is expected to contain the unnormalized logits for each class (which do not need to be positive or sum to 1, in general).</p>
</blockquote>
<p>The targets are supposed to be either class indices in the range <code>[0, nb_classes-1]</code> or “soft” targets given as probabilities.</p>