question: Access and change models optimizer after setup
context: Hey,
  what is the canonical way to access/change the optimizer of a pl.LightningModule after model.setup('fit') was called?
  E.g. if I follow https://pytorch-lightning.readthedocs.io/en/latest/lr_finder.html 4 (the last part where I explicitly plot the found lr and then change it) I don’t see a way how to change the optimizers lr.
  If I call model.configure_optimizers() it’ll just return the optimizer and not set them.
  Is there no official way to access and change them after construction?
  Cheers
answer: What you probably want to do, is to have an learning_rate attribute that you can change after inspecting the plot:
  So your model should look something like this:
  class MyModel(pl.LightningModule):
      def __init__(self, ...)
          self.learning_rate = 1e-2
  
      def configure_optimizers(self):
          return torch.optim.Adam(self.parameters(), lr=self.learning_rate)
  then you should be able to do:
  model = MyModel(...)
  trainer = Trainer(...)
  lrfinder = trainer.tuner.lr_finder(...)
  lffinder.plot() # find the learning rate you want
  model.learning_rate = 1234 # set what you want