Before revision:  Furthermore, motivated by our theory we propose a novel approach  that directly sets the  predictor based on the statistics of its inputs . In the case of linear predictors, our approach outperforms gradient trainingof the predictor by 5 \\%DIF < and on ImageNet it performs comparably with more complex two-layer non-linear predictors that employ BatchNorm.
After revision:  Furthermore, motivated by our theory we propose a novel approach that directly sets the predictor based on the statistics of its inputs , without gradient training. On ImageNet, it performs comparably with more complex two-layer non-linear predictors that employ BatchNorm and outperforms a linear predictor by 2.5 \\%DIF < and on ImageNet it performs comparably with more complex two-layer non-linear predictors that employ BatchNorm.