Optimizers for nested modules
class MyNet(pl.LightningModule):
    def __init__(self):
        self.m1 = MyMod1()
        self.m2 = MyMod2()
If I implement different configure_optimizers for different submodules MyMod(also pl.LightningModule), is it correct that parameters in each MyMod will be updated by their own optimizers returned by configure_optimizers? If I only implement configure_optimizers for the top module, is it correct that parameters in submodules will be optimized by the same optimizer returned by configure_optimizers of the top module?
When you have nested LightningModules, their configure_optimizers will never be called unless you explicitly call it in the top-level configure_optimizers.
That being said, if you call, merge and return the optimizers created there, these optimizers should only contain parameters from the respective submodule (if implemented correctly)