Access and change models optimizer after setup
Hey,
what is the canonical way to access/change the optimizer of a pl.LightningModule after model.setup('fit') was called?
E.g. if I follow https://pytorch-lightning.readthedocs.io/en/latest/lr_finder.html 4 (the last part where I explicitly plot the found lr and then change it) I don’t see a way how to change the optimizers lr.
If I call model.configure_optimizers() it’ll just return the optimizer and not set them.
Is there no official way to access and change them after construction?
Cheers
What you probably want to do, is to have an learning_rate attribute that you can change after inspecting the plot:
So your model should look something like this:
class MyModel(pl.LightningModule):
    def __init__(self, ...)
        self.learning_rate = 1e-2

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=self.learning_rate)
then you should be able to do:
model = MyModel(...)
trainer = Trainer(...)
lrfinder = trainer.tuner.lr_finder(...)
lffinder.plot() # find the learning rate you want
model.learning_rate = 1234 # set what you want