Before revision:  Non-convolutional layers, including batch normalization (BN) , typically have relatively lower computational intensity compared to the convolutional or fully-connected layers, and hence are often constrained by main-memory bandwidth. In this paper, we focus on accelerating the BN layers among the non-convolutional layers, as BN has become a core design block of modern CNNs . A typical modern CNN has a large number of BN layers  . BN requires mean and variance calculations over each mini-batch during training.
After revision:  Non-convolutional layers, including batch normalization (BN) has become a core design block of modern CNNs . A typical modern CNN has a large number of BN layers . BN requires mean and variance calculations over each mini-batch during training.