Before revision:  In this work ,  we take a first step towards this goal by extending known properties of BatchNorm in randomly initialized deep neural networks (DNNs) to nine recently proposed normalization layers. Our primary findings follow : (i) Similar to BatchNorm, activations-based normalization layers can avoid exploding activations in ResNets  ;
After revision:  In this work , we take a first step towards this goal by extending known properties of BatchNorm in randomly initialized deep neural networks (DNNs) to nine recently proposed normalization techniques. Our generalized theory leads to the following set of principles : (i) Similar to BatchNorm, activations-based normalization layers can avoid exploding activations in ResNets ;