About lr_schedule.StepLR 's parameter

k11: <p>In my denoising work, I use <code>torch.optim.lr_scheduler.StepLR</code> to adjust learning rate. But it seems different between  <code>scheduler.step()</code> and <code>scheduler.step(ave_train_loss)</code> that I called in each training epoch’s end.<br>
After training,I test both two models in same images, and get following result:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b38887b95fb015b70ec87f1cfbe6d33340436cf3.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/b38887b95fb015b70ec87f1cfbe6d33340436cf3" title="TIM图片20200218145910"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b38887b95fb015b70ec87f1cfbe6d33340436cf3_2_690x343.jpeg" alt="TIM图片20200218145910" data-base62-sha1="pCdYRRPDm7MbSRHYNs7yYhh59Jx" width="690" height="343" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b38887b95fb015b70ec87f1cfbe6d33340436cf3_2_690x343.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b38887b95fb015b70ec87f1cfbe6d33340436cf3_2_1035x514.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/3/b38887b95fb015b70ec87f1cfbe6d33340436cf3.jpeg 2x" data-dominant-color="323331"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">TIM图片20200218145910</span><span class="informations">1086×540 190 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Figures in left and right are using  <code>scheduler.step()</code> and <code>scheduler.step(ave_train_loss)</code> respectively. I repeat several times to train and test in same situations, but get almost the same results.</p>
<p>I don’t find something interpretable about this question in source code. I think it should be no effect in StepLR scheduler when calling <code>scheduler.step()</code> with a parameter</p>
<pre><code class="lang-auto">class _LRScheduler(object):
 ...
 def step(self, epoch=None):
        # Raise a warning if old pattern is detected
        # https://github.com/pytorch/pytorch/issues/20124
        if self._step_count == 1:
            if not hasattr(self.optimizer.step, "_with_counter"):
                warnings.warn("Seems like `optimizer.step()` has been overridden after learning rate scheduler "
                              "initialization. Please, make sure to call `optimizer.step()` before "
                              "`lr_scheduler.step()`. See more details at "
                              "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

            # Just check if there were two first lr_scheduler.step() calls before optimizer.step()
            elif self.optimizer._step_count &lt; 1:
                warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
                              "In PyTorch 1.1.0 and later, you should call them in the opposite order: "
                              "`optimizer.step()` before `lr_scheduler.step()`.  Failure to do this "
                              "will result in PyTorch skipping the first value of the learning rate schedule. "
                              "See more details at "
                              "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
        self._step_count += 1
</code></pre>
<p>main of my code is following</p>
<pre><code class="lang-auto">#  optimizer and scheduler
    optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
    scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.5)
    if train_mode:
        for epoch in range(1000):
            ave_train_loss = 0
            model.train()
            for i, (image, label) in enumerate(train_loader):
                optimizer.zero_grad()
                image = image.to(device)
                label = label.to(device)
                output = model(image)
                full_loss = critetion(output, label)
                full_loss.backward()
                optimizer.step()
                ave_train_loss += image.size()[0] * full_loss.item()
            ave_train_loss /= len(train_dataset)
            scheduler.step(ave_train_loss)
</code></pre>

ptrblck: <p>If you are passing the <code>epoch</code> argument to <code>StepLR</code>, it will be ste in <a href="https://github.com/pytorch/pytorch/blob/a5f01846c2bae0d78a5c0cd74ced20a07b8ef961/torch/optim/lr_scheduler.py#L144">this line of code</a> and used to calculate the learning rate <a href="https://github.com/pytorch/pytorch/blob/a5f01846c2bae0d78a5c0cd74ced20a07b8ef961/torch/optim/lr_scheduler.py#L353-L355">here</a>.</p>
<p>Since you are passing the average training loss, this should influence your learning rate scheduler.</p>