Repository: Lightning-AI/pytorch-lightning

Issue:
Trainer.fit() crashes if no checkpoint callback is provided
I hope it's okay that I keep posting issues...
Now that I can circumvent the github installation issues, I pulled in the latests master and let my simple CoolModel demo code run. But now calling trainer.fit() crashes with:

> AttributeError                            Traceback (most recent call last)
<ipython-input-3-4ff64ec6323b> in <module>
     21                  )
     22 
---> 23 trainer.fit(model)
     24 # exp.close()

> /opt/miniconda3/envs/dev_pytorch_lightning36/lib/python3.6/site-packages/pytorch_lightning/models/trainer.py in fit(self, model)
    494                 self.optimizers, self.lr_schedulers = self.optimizers
    495 
--> 496             self.__run_pretrain_routine(model)
    497 
    498         # return 1 when finished

> /opt/miniconda3/envs/dev_pytorch_lightning36/lib/python3.6/site-packages/pytorch_lightning/models/trainer.py in __run_pretrain_routine(self, model)
    680 
    681         # restore training and model before hpc call
--> 682         self.restore_state_if_existing_checkpoint()
    683 
    684         # enable cluster checkpointing

> /opt/miniconda3/envs/dev_pytorch_lightning36/lib/python3.6/site-packages/pytorch_lightning/models/trainer.py in restore_state_if_existing_checkpoint(self)
    261 
    262         # find last epoch
--> 263         checkpoints = os.listdir(self.checkpoint_callback.filepath)
    264         for name in checkpoints:
    265             # ignore hpc ckpts

> AttributeError: 'NoneType' object has no attribute 'filepath'

Looking at the code, it appears to happen because I did not provide a checkpoint callback and it tries to access it in `restore_state_if_existing_checkpoint`


Discussion:
now I added a ModelCheckpoint callback to test it and it still crashes, as the callbacks filepath does not exist when running for the first time
good catch... definetely a bug from the thing i pushed yesterday.

there is no test for that particular case either haha. good catch!

Resolving patch:
diff --git a/pytorch_lightning/models/trainer.py b/pytorch_lightning/models/trainer.py
--- a/pytorch_lightning/models/trainer.py
+++ b/pytorch_lightning/models/trainer.py
@@ -259,6 +259,11 @@ def restore_state_if_existing_checkpoint(self):
         last_epoch = -1
         last_ckpt_name = None
 
+        # do nothing if there's not dir or callback
+        no_ckpt_callback = self.checkpoint_callback is None
+        if no_ckpt_callback or not os.path.exists(self.checkpoint_callback.filepath):
+            return
+
         # find last epoch
         checkpoints = os.listdir(self.checkpoint_callback.filepath)
         for name in checkpoints:


Test patch:
diff --git a/tests/test_models.py b/tests/test_models.py
--- a/tests/test_models.py
+++ b/tests/test_models.py
@@ -26,6 +26,38 @@
 # ------------------------------------------------------------------------
 # TESTS
 # ------------------------------------------------------------------------
+def test_simple_cpu():
+    """
+    Verify continue training session on CPU
+    :return:
+    """
+    hparams = get_hparams()
+    model = LightningTestModel(hparams)
+
+    save_dir = init_save_dir()
+
+    # exp file to get meta
+    test_exp_version = 10
+    exp = get_exp(False, version=test_exp_version)
+    exp.argparse(hparams)
+    exp.save()
+
+    trainer_options = dict(
+        max_nb_epochs=1,
+        val_percent_check=0.1,
+        train_percent_check=0.1,
+        experiment=exp,
+    )
+
+    # fit model
+    trainer = Trainer(**trainer_options)
+    result = trainer.fit(model)
+
+    # traning complete
+    assert result == 1, 'amp + ddp model failed to complete'
+
+    clear_save_dir()
+
 
 def test_amp_single_gpu():
     """