Repository: Lightning-AI/pytorch-lightning

Issue:
on_fit_start not triggering (master)
## 🐛 Bug
on_fit_start is not being triggered on master as of f46318ebfeb785a659c49091a6871584ccde3ee1

### To Reproduce
Steps to reproduce the behavior:

1. Install master
2. Run template with on_fit_start added (included below)
(identical behavior for single gpu, ddp_spawn and ddp)

#### Code sample
```
"""
Runs a model on a single node across multiple gpus.
"""
import os
from argparse import ArgumentParser

from pl_examples.models.lightning_template import LightningTemplateModel
from pytorch_lightning import Trainer, seed_everything

seed_everything(234)

class custom_template(LightningTemplateModel):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
    
    def on_epoch_start(self):
        print("on_epoch_start")

    def on_fit_start(self):
        print("on_fit_start")

def main(args):
    """ Main training routine specific for this project. """
    # ------------------------
    # 1 INIT LIGHTNING MODEL
    # ------------------------
    model = custom_template(**vars(args))

    # ------------------------
    # 2 INIT TRAINER
    # ------------------------
    trainer = Trainer.from_argparse_args(args)

    # ------------------------
    # 3 START TRAINING
    # ------------------------
    trainer.fit(model)


def run_cli():
    # ------------------------
    # TRAINING ARGUMENTS
    # ------------------------
    # these are project-wide arguments
    root_dir = os.path.dirname(os.path.realpath(__file__))
    parent_parser = ArgumentParser(add_help=False)

    # each LightningModule defines arguments relevant to it
    parser = LightningTemplateModel.add_model_specific_args(parent_parser, root_dir)
    parser = Trainer.add_argparse_args(parser)
    parser.set_defaults(gpus=2, distributed_backend='ddp')
    args = parser.parse_args()

    # ---------------------
    # RUN TRAINING
    # ---------------------
    main(args)


if __name__ == '__main__':
    run_cli()
```

### Expected behavior
"on_fit_start" should be printed along with "on_epoch_start" but only "on_epoch_start" is printed


Discussion:
It's a bug I think since at this point neither `model` nor `accelerator_backend` is assigned to the `trainer`.
https://github.com/PyTorchLightning/pytorch-lightning/blob/caf7893f27c73db227883ad00531d0ff550cbb23/pytorch_lightning/trainer/trainer.py#L984
Should be called after
https://github.com/PyTorchLightning/pytorch-lightning/blob/caf7893f27c73db227883ad00531d0ff550cbb23/pytorch_lightning/trainer/trainer.py#L1008
I'll move the hook and test it out, if it works I'll do a PR
@rohitgr7 moving the hook can work but causes another peculiar issue
```
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/states.py", line 48, in wrapped_fn
    result = fn(self, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1029, in fit
    self.call_hook('on_fit_start', model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1384, in call_hook
    output = hook_fx(*args, **kwargs)
TypeError: on_fit_start() takes 1 positional argument but 2 were given
```
more specifically `on_fit_start()` gets called with 2 modules, still locating the source of this extra arg/module

Edit:
So call_hook has the following signature `def call_hook(self, hook_name, *args, **kwargs)` and within call_hook there is `trainer_hook(*args, **kwargs)` and `hook_fx(*args, **kwargs)` (in this case: `on_fit_start`) both are called with the same args but `trainer_hook` needs a model arg while `hook_fx`/`on_fit_start` can't take any args. Thoughts on how to move forward?

Edit:
bug still present as of c64520e658806a87f282c74299b2ea4b7ea493ea
let's just wait for refactors and keep this issue open. After exploring a bit more I saw a few more bugs :sweat_smile: 
Although for now, the only place where the model is required is in `TrainerCallbackHookMixin.on_fit_start` for the callbacks but there also we can avoid the model parameter and just use `self.get_model()` just like in `on_fit_end`.
@rohitgr7 I implemented the following changes and now on_fit_start is working as expected:

- move hook
- change on_fit_start signature (like on_fit_end)
- change callback.on_fit_start args (like on_fit_end)

> After exploring a bit more I saw a few more bugs 😅

What else did you find? I'll take a look before I submit a PR
@s-rog want to send the PR out?

Resolving patch:
diff --git a/pytorch_lightning/trainer/callback_hook.py b/pytorch_lightning/trainer/callback_hook.py
--- a/pytorch_lightning/trainer/callback_hook.py
+++ b/pytorch_lightning/trainer/callback_hook.py
@@ -46,10 +46,10 @@ def on_init_end(self):
         for callback in self.callbacks:
             callback.on_init_end(self)
 
-    def on_fit_start(self, model):
+    def on_fit_start(self):
         """Called when the trainer initialization begins, model has not yet been set."""
         for callback in self.callbacks:
-            callback.on_fit_start(self, model)
+            callback.on_fit_start(self, self.get_model())
 
     def on_fit_end(self):
         """Called when the trainer initialization begins, model has not yet been set."""
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -284,9 +284,6 @@ def fit(
         # setup data, etc...
         self.train_loop.setup_fit(model, train_dataloader, val_dataloaders, datamodule)
 
-        # hook
-        self.call_hook('on_fit_start', model)
-
         # hook
         self.data_connector.prepare_data(model)
 
@@ -298,6 +295,10 @@ def fit(
         # -------------------------
         self.accelerator_backend = self.accelerator_connector.select_accelerator()
         self.accelerator_backend.setup(model)
+
+        # hook
+        self.call_hook('on_fit_start')
+
         results = self.accelerator_backend.train()
         self.accelerator_backend.teardown()
 


Test patch: