We show that gradient descent on full width linear convolutional networks of depth L converges to a linear predictor related to the 2/L bridge penalty in the frequency domain. This is in contrast to fully connected linear networks, where regardless of depth, gradient descent converges to the 2 maximum margin solution.
REF look at a logistic regression problem and show that the predictor does converge to the max-margin solution.
Implicit Bias of Gradient Descent on Linear Convolutional Networks
ArXiv
ArXiv
Computer Science Mathematics