Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks