How to sequentially call fit() and test() in DDP
I wrote a Python script that loops over sets of hyperparameters and for each set calls trainer.fit(). Subsequently trainer.test() is currently called 2 times, each for the best checkpoint that was logged for two metrics.
This script is executable via commandline and works well on CPU an 1 GPU for any number of hyperparameter sets.
I want to run this code on multiple GPUs (2-4) and wonder if I could use 'ddp'?
The latest documentation says that if your script needs to invoke both .fit and .test, or one of them multiple times 'ddp' mode isn't possible.
However, @awaelchli indicated here, that running trainer.fit() and trainer.test() in 'ddp' mode is now possible but the documentation is outdated.
Questions:

Is it now possible to run both  trainer.fit() and subsequently trainer.test() (and eventually trainer.test() mutiple times) in 'ddp' mode?
Would I need to reorganize my code so that the loop over hyperparameter sets invokes a subprocess.call to actually instantiate every new trainer for 'ddp' in another process?
If calling .test() after .fit() in 'ddp' is still not possible, would it be a workaround to instantiate trainer again after .fit() is finished and just load the model weights from the best checkpoint for testing (to avoid using the same trainer object)?

I would be really happy for some help on that.
No this should be working fine.
We have tests for this here: 
  
    
      pytorch-lightning/tests/accelerators/test_ddp.py
    
    
         Line 56
      in
      a6c98c4
    
  
  
    

        
          
           def test_multi_gpu_model_ddp_fit_test(tmpdir): 
        
    
  


I will update the docs.