question: String “best” at argument “ckpt_path” for test method of Trainer class
context: Hi,
  The test method of the Trainer class, has the input argument ckpt_path. According to the docs:
  
  ckpt_path (Optional[str]) – Either best or path to the checkpoint you wish to test. If None and the model instance was passed, use the current weights. Otherwise, the best model from the previous trainer.fit call will be loaded.
  
  Also, in the documentation of PyTorch Lightning for the test set, using Trainer, there is the following:
  # run full training
  trainer.fit(model)
  
  # (1) load the best checkpoint automatically (lightning tracks this for you)
  trainer.test(ckpt_path="best")
  My question is, according to what the "best" checkpoint is decided? That is, is the "best" decided on maximising or minimising some value? What would be that value? Can someone configure the policy (i.e. minimising or maximising) and the value? How one should use this "best" string?
  Links for reference:
  
  Test set documentation
  Test method of Trainer class documentation
  
  P.S. Please note that I'm not referring to using the ModelChekpoint callback, but explicitly to the above, which seems that the ModelCheckpoint callback is not used.
answer: it actually relies on ModelCheckpoint callback only 
    
      
        pytorch-lightning/pytorch_lightning/trainer/trainer.py
      
      
          Lines 1268 to 1280
        in
        b3e9dff
      
    
    
      
  
          
            
             if ckpt_path == "best": 
          
  
          
            
                 # if user requests the best checkpoint but we don't have it, error 
          
  
          
            
                 if not self.checkpoint_callback.best_model_path: 
          
  
          
            
                     if self.fast_dev_run: 
          
  
          
            
                         raise MisconfigurationException( 
          
  
          
            
                             f"You cannot execute `.{fn}()` with `fast_dev_run=True` unless you do" 
          
  
          
            
                             f" `.{fn}(ckpt_path=PATH)` as no checkpoint path was generated during fitting." 
          
  
          
            
                         ) 
          
  
          
            
                     raise MisconfigurationException( 
          
  
          
            
                         f'`.{fn}(ckpt_path="best")` is set but `ModelCheckpoint` is not configured to save the best model.' 
          
  
          
            
                     ) 
          
  
          
            
                 # load best weights 
          
  
          
            
                 ckpt_path = self.checkpoint_callback.best_model_path 
          
      
    
  
  
  so if there is no ModelCheckpoint callback it will raise an error