Cross validation Before going into smoothing methods too deeply, we can get a lot of ideas from the comparable problems in regression. We'll start here today. The smoothing matrix, which plays the role of the hat matrix or projection matrix H in regression, is the key to understanding various properties of smoothing splines. It can seem in examples that smoothing splines have little advantage over simpler methods, like polynomial regression. The advantages only become visible when you have the right sort of test function, particularly one that has a jump.
Jumps lead to the Gibbs phenomenon and lots of ripples that hurt global polynomial fits.
Smoothing splines do quite a bit better in these cases. Effects for choosing the smoothing parameter It can seem in examples that smoothing splines have little advantage over simpler methods, like polynomial regression. For relatively smooth functions, polynomial regression has much smaller MSE than smoothing splines. The advantages only become visible when you have the right sort of test function, particularly one that has a jump. Jumps lead to the Gibbs phenomenon and lots of ripples that hurt global polynomial fits.