question: Optimizers for nested modules
context: class MyNet(pl.LightningModule):
      def __init__(self):
          self.m1 = MyMod1()
          self.m2 = MyMod2()
  If I implement different configure_optimizers for different submodules MyMod(also pl.LightningModule), is it correct that parameters in each MyMod will be updated by their own optimizers returned by configure_optimizers? If I only implement configure_optimizers for the top module, is it correct that parameters in submodules will be optimized by the same optimizer returned by configure_optimizers of the top module?
answer: When you have nested LightningModules, their configure_optimizers will never be called unless you explicitly call it in the top-level configure_optimizers.
  That being said, if you call, merge and return the optimizers created there, these optimizers should only contain parameters from the respective submodule (if implemented correctly)