question: What is the relationship beween accumulate_grad_batches and lr_scheduler?
context: I wrote following code:
      def configure_optimizers(self):
          ......
          return [
              {
                  'optimizer': optimizer,
                  'lr_scheduler': {
                      'scheduler': scheduler,
                      'interval': 'step',
                      'frequency': 1
                  }
              }
  I choose step as the interval. Actually, I don't understand what step means!!!
  In my opinion, step may mean a batch? But when I set Trainer parameter:  accumulate_grad_batches=5, will lr_scheduler still execute after one batch or it only execute after every  accumulate_grad_batches batches? If the answer is the later, so the step means the call of optimizer.step()?
  (I know accumulate_grad_batches can affect optimizer, but I don't know whether it can affect lr_scheduler)
answer: yes, step means optimization step and accumulate_grad_batches will be taken under consideration while calling the lr_scheduler.
  Ref code:
  
    
      
        pytorch-lightning/pytorch_lightning/loops/epoch/training_epoch_loop.py
      
      
          Lines 434 to 437
        in
        8ea39d2
      
    
    
      
  
          
            
             def update_lr_schedulers(self, interval: str, update_plateau_schedulers: bool) -> None: 
          
  
          
            
                 """updates the lr schedulers based on the given interval.""" 
          
  
          
            
                 if interval == "step" and self._should_accumulate(): 
          
  
          
            
                     return