question: Unfreezing layers during training?
context: Freezing layers at the beginning of training works, however unfreezing in on_epoch_start() during training causes the gradient to explode. Without the unfreezing part (or without freezing at all), the model trains fine with no gradient issues.
  I'm using DDP + Apex O2 and the loss scaling will keep going down to 0 where it would encounter 0 division and crash.
  Is unfreezing during training not possible in pytorch/lightning? or am I missing snippet?
answer: you can unfreeze whenever. if gradients explode it's for another reason