Pretrain some sections of a model, then initialize those parts when training a full model
I need to pretrain the encode and decode section of an autoencoder first, then later attach a transformer in the middle of the encode and decode section. When I load the weights of the encode and decode section when pretraining it first, while initializing the weights of the transformer section, will I get an error about missing layers?
You can use the strict flag of load_from_checkpoint to avoid the missing layer failure:

  
    
      pytorch-lightning/pytorch_lightning/core/saving.py
    
    
        Lines 94 to 95
      in
      079fe9b
    
  
  
    

        
          
                       strict: Whether to strictly enforce that the keys in :attr:`checkpoint_path` match the keys 
        

        
          
                           returned by this module's state dict. Default: `True`.