`save_hyperparameters()` is very slow when there are many hyperparameters, any speed up?
Hi,
I've currently refactored a part of code to use Pytorch Lightning instead of a regular pytorch script. I find a 35% speed up on a toy dataset, where the model had 20.9 K parameters.
However, when trying a bigger version of the model (9.1 M params), the initialization of the model takes a lot more time (even before lightning prints the number of params). I've managed to nail it down to the self.save_hyperparameters() function in init, since the initialization is instantaneous without calling self.save_hyperparameters(), but takes more than a full minute when calling it.
Removing self.save_hyperparameters() causes an issue in my code since I am calling  model.load_from_checkpoint() afterward. Would you have any thoughts on how I can speed up the code? The "regular" pytorch model manages to initialize, train and save the model faster than the Lightning one due to this slow down ...
Thanks!
You have to set self.save_hyperparameters(ignore="model"), since you are passing in a full model as a "hyperparameter", you don't want to save that!