Set different lr_schedulers for different parameter groups
❓ Questions and Help
What is your question?
I want to set different learning rate adjustment  lr_schedulers for different parameter groups in one optimizer. This motivation is easy to understand. For example, add my own module to the pre-training network for fine-tuning.
What have you tried?
I have tried the following method:
def configure_optimizers(self):
opt = torch.optim.SGD([
dict(params = self.net.net.parameters(),lr=0.1*self.lr),
dict(params=[*self.net.pool.parameters(),
* self.net.fc.parameters(),
*self.center_loss.parameters()])
],lr=self.lr,momentum=0.98)
    sdl = torch.optim.lr_scheduler.CyclicLR(opt, 1e-5, 1e-3, step_size_up=2000, step_size_down=1000,gamma=0.5,
                                            mode='triangular2')
    return [opt], [dict(scheduler=sdl, interval='step')]

As self.net.net is a pre-trained backbone, I only want the learning rate of this group to be 1/10 of others'.
However, the learning rates of the two params groups are identified with this code. For example, both are 1e-5 at the training start, and I want them to be 1e-6 and 1e-5, respectively.
Since "automatic_optimizition = False" has bugs to fix, I confused want should I do.