question: How to sequentially call fit() and test() in DDP
context: I wrote a Python script that loops over sets of hyperparameters and for each set calls trainer.fit(). Subsequently trainer.test() is currently called 2 times, each for the best checkpoint that was logged for two metrics.
  This script is executable via commandline and works well on CPU an 1 GPU for any number of hyperparameter sets.
  I want to run this code on multiple GPUs (2-4) and wonder if I could use 'ddp'?
  The latest documentation says that if your script needs to invoke both .fit and .test, or one of them multiple times 'ddp' mode isn't possible.
  However, @awaelchli indicated here, that running trainer.fit() and trainer.test() in 'ddp' mode is now possible but the documentation is outdated.
  Questions:
  
  Is it now possible to run both  trainer.fit() and subsequently trainer.test() (and eventually trainer.test() mutiple times) in 'ddp' mode?
  Would I need to reorganize my code so that the loop over hyperparameter sets invokes a subprocess.call to actually instantiate every new trainer for 'ddp' in another process?
  If calling .test() after .fit() in 'ddp' is still not possible, would it be a workaround to instantiate trainer again after .fit() is finished and just load the model weights from the best checkpoint for testing (to avoid using the same trainer object)?
  
  I would be really happy for some help on that.
answer: No this should be working fine.
  We have tests for this here: 
    
      
        pytorch-lightning/tests/accelerators/test_ddp.py
      
      
           Line 56
        in
        a6c98c4
      
    
    
      
  
          
            
             def test_multi_gpu_model_ddp_fit_test(tmpdir): 
          
      
    
  
  
  I will update the docs.