Before revision: Implicit regularization in deep learning is perceived as a tendency of gradient-based optimization to fit training data with predictors of minimal "complexity. " The fact that only some types of data give rise to generalization is understood to result from them being especially amenable to fitting with low complexity predictors. A major challenge in formalizing this intuition is to define complexity measures that are quantitative yet capture the essence of data that admits generalization. With an eye towards this challenge , we provide the first  analysis of implicit regularization in tensor factorization , equivalent to a certain non-linear neural network.
After revision: Implicit regularization in deep learning have led to a theoretical focus on matrix factorization -- matrix completion via linear neural network. As a step further towards practical deep learning , we provide the first analysis of implicit regularization in tensor factorization , equivalent to a certain non-linear neural network.