This is a lot of text.  I think it would be great to include some examples here showing over-fitting and under-fitting.  My favorite demonstration of this is to use a 1D plot with a polynomial regression (we could use the new ``PolynomialFeatures`` preprocessor for this).  A low-degree polynomial has high bias, and will under-fit; a high-degree polynomial has high variance, and will over-fit the data (I used an example like this in my astronomy text: [figure here](http://www.astroml.org/book_figures/chapter8/fig_cross_val_B.html)).