question: Why is on_before_optimizer_step incompatible with accumulate_grad_batches ?
context: The documentation about the on_before_optimizer_step hook mentions that:
  
  The hook is only called if gradients do not need to be accumulated.
  
  However, in theory, combining both seems possible to me.
  I took a look at PR #8048 which is the implementation of this hook but it didn't seem clear there.
answer: hey @NathanGodey !
  it doesn't mean that it won't be called if accumulate_grad_batches > 1, but only be called when accumulation is done and it's time to update the gradients with optimizer.step.