What is the relationship beween accumulate_grad_batches and lr_scheduler?
I wrote following code:
    def configure_optimizers(self):
        ......
        return [
            {
                'optimizer': optimizer,
                'lr_scheduler': {
                    'scheduler': scheduler,
                    'interval': 'step',
                    'frequency': 1
                }
            }
I choose step as the interval. Actually, I don't understand what step means!!!
In my opinion, step may mean a batch? But when I set Trainer parameter:  accumulate_grad_batches=5, will lr_scheduler still execute after one batch or it only execute after every  accumulate_grad_batches batches? If the answer is the later, so the step means the call of optimizer.step()?
(I know accumulate_grad_batches can affect optimizer, but I don't know whether it can affect lr_scheduler)
yes, step means optimization step and accumulate_grad_batches will be taken under consideration while calling the lr_scheduler.
Ref code:

  
    
      pytorch-lightning/pytorch_lightning/loops/epoch/training_epoch_loop.py
    
    
        Lines 434 to 437
      in
      8ea39d2
    
  
  
    

        
          
           def update_lr_schedulers(self, interval: str, update_plateau_schedulers: bool) -> None: 
        

        
          
               """updates the lr schedulers based on the given interval.""" 
        

        
          
               if interval == "step" and self._should_accumulate(): 
        

        
          
                   return