luckily I don't know the answer for neural nets but in linear networks there's a very crisp answer you can take a linear function write it as a product of matrices then if you have outer regularization on w's which comes from gradient descent it translates a shot in quasi norm on theta on the linear function okay so regardless of the depth and the width the prediction function class stays linear you know you can write a linear map a matrix as a product as one matrix as a product of five matrices is still matrix the function