We show that gradient descent on full width linear convolutional networks of depth L converges to a linear predictor related to the 2/L bridge penalty in the frequency domain. This is in contrast to fully connected linear networks, where regardless of depth, gradient descent converges to the 2 maximum margin solution.
REF consider gradient descent on fully connected linear networks and linear convolutional networks.
Implicit Bias of Gradient Descent on Linear Convolutional Networks
ArXiv
ArXiv
Computer Science Mathematics