Why is on_before_optimizer_step incompatible with accumulate_grad_batches ?
The documentation about the on_before_optimizer_step hook mentions that:

The hook is only called if gradients do not need to be accumulated.

However, in theory, combining both seems possible to me.
I took a look at PR #8048 which is the implementation of this hook but it didn't seem clear there.
hey @NathanGodey !
it doesn't mean that it won't be called if accumulate_grad_batches > 1, but only be called when accumulation is done and it's time to update the gradients with optimizer.step.