Confused About Optimizer Behavior in Custom Training Loop

pasiho: <p>Hey everyone! <img src="https://discuss.pytorch.org/images/emoji/apple/waving_hand.png?v=14" title=":waving_hand:" class="emoji" alt=":waving_hand:" loading="lazy" width="20" height="20"></p>
<p>I’ve been experimenting with a custom training loop in PyTorch and stumbled upon a conceptual hurdle. I get the basics, but I want to understand better how the optimizer works during training.</p>
<p>Here’s the basic structure I’m using:</p>
<p>python<br>
Copy<br>
Edit<br>
for data target in loader:<br>
optimizer.zero_grad()<br>
output = model(data)<br>
loss = criterion(output target)<br>
loss.backward()<br>
optimizer.step()</p>
<p>What I’m curious about: does optimizer.step() right away update the model weights using the gradients calculated from loss.backward()? And what happens if I change gradients in between—will those changes stick?</p>
<aside class="quote" data-post="1" data-topic="206714">
  <div class="title">
    <div class="quote-controls"></div>
    <img alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/97f17d/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/model-convergence-issue-in-custom-training-loop/206714">Model Convergence Issue in Custom Training Loop</a> 
  </div>
  <blockquote>
    There are issues with model convergence when using a custom training loop in PyTorch. The loss stagnates after a few epochs. The optimizer is Adam with a learning rate of 0.001. Issue persists even after adjusting the learning rate and changing the network architecture. Any advice on what might be causing this and potential solutions?
  </blockquote>
</aside>

<p>I’ve come across some posts here and there but would appreciate insights from those who have built custom training logic in more complex workflows.<br>
Also, I’m picking up knowledge about MLOps and automation. If you have any advice or suggestions for resources on <a href="https://www.igmguru.com/blog/how-to-become-devops-engineer" rel="noopener nofollow ugc">how to become a DevOps engineer</a> while having deep learning experience, I’d be grateful!</p>
<p>Thanks in advance for your help!<br>
Regards<br>
pasiho</p>

ptrblck: <aside class="quote no-group" data-username="pasiho" data-post="1" data-topic="219219">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/a5b964/48.png" class="avatar"> pasiho:</div>
<blockquote>
<p>does optimizer.step() right away update the model weights using the gradients calculated from loss.backward()?</p>
</blockquote>
</aside>
<p>Yes, the <code>step()</code> method will use the <code>.grad</code> attributes of all trainable parameters which were passed to this optimizer to update them using the update formula (e.g. it could include momentum etc.).</p>
<aside class="quote no-group" data-username="pasiho" data-post="1" data-topic="219219">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/a5b964/48.png" class="avatar"> pasiho:</div>
<blockquote>
<p>And what happens if I change gradients in between—will those changes stick?</p>
</blockquote>
</aside>
<p>Yes, you can manipulate the gradients after the <code>backward</code> and before the <code>step()</code> call which is also a common practice for e.g. gradient clipping.</p>