Reset the learning rate hyperparameter to a more appropriate range, hopefully nothing blows up.