The Shattered Gradients Problem: If resnets are the answer, then what is the question?
Skip Connections as Effective Symmetry-Breaking