about the parameters again parameters have no meaning in your own nets the only thing that matters at the end is your function f this is all you see a test time the parameter is just a way to encode F so what you should really ask is how does this bias translate over here so what is the bias on the prediction function and this is clearly where architecture matter so for example here 1 3 & 4 there was no regularization the algorithm was just SGD the only thing that changes is the architecture yet you get a huge improvement in the test error