Module Expr Str Print NameLoad Import alias identifier Import alias identifier ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Expr Call AttributeLoad AttributeLoad NameLoad attr attr Num Assign NameStore Num Assign NameStore ListLoad Num Num Num Assign NameStore Lambda arguments args NameParam defaults Call AttributeLoad NameLoad attr BinOpMult BinOpMult Num AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore BinOpAdd Call NameLoad NameLoad BinOpMult Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Num Expr Call AttributeLoad NameLoad attr keyword TupleLoad Num Num For NameStore Call NameLoad Call NameLoad NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Num Call NameLoad NameLoad BinOpAdd NameLoad Num Expr Call AttributeLoad NameLoad attr NameLoad keyword TupleLoad keyword TupleLoad Assign NameStore Call NameLoad keyword SubscriptLoad NameLoad Index NameLoad keyword NameLoad Assign NameStore Call NameLoad Assign NameStore Call NameLoad ListLoad TupleLoad Str NameLoad TupleLoad Str NameLoad Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad SubscriptLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr NameLoad keyword Str keyword Num Assign NameStore Call AttributeLoad NameLoad attr Num Num Num Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr keyword Str Expr Call AttributeLoad NameLoad attr NameLoad Call NameLoad NameLoad keyword Str Expr Call AttributeLoad NameLoad attr NameLoad NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr TupleLoad Num Num Expr Call AttributeLoad NameLoad attr TupleLoad Num Num Expr Call AttributeLoad NameLoad attr keyword Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr SubscriptLoad NameLoad Index NameLoad UnaryOpUSub Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr
N/A N/A 
============================
Underfitting vs. Overfitting
============================

This example demonstrates the problems of underfitting and overfitting and
how we can use linear regression with polynomial features to approximate
nonlinear functions. The plot shows the function that we want to approximate,
which is a part of the cosine function. In addition, the samples from the
real function and the approximations of different models are displayed. The
models have polynomial features of different degrees. We can see that a
linear function (polynomial with degree 1) is not sufficient to fit the
training samples. This is called **underfitting**. A polynomial of degree 4
approximates the true function almost perfectly. However, for higher degrees
the model will **overfit** the training data, i.e. it learns the noise of the
training data.
We evaluate quantitatively **overfitting** / **underfitting** by using
cross-validation. We calculate the mean squared error (MSE) on the validation
set, the higher, the less likely the model generalizes correctly from the
training data.
 N/A __doc__ N/A numpy np N/A matplotlib.pyplot plt sklearn.pipeline Pipeline sklearn.preprocessing PolynomialFeatures sklearn.linear_model LinearRegression sklearn.model_selection cross_val_score N/A N/A N/A N/A np random seed 0 N/A n_samples 30 N/A degrees N/A 1 4 15 N/A true_fun N/A N/A N/A X N/A N/A N/A np cos N/A N/A 1.5 N/A np pi X N/A X N/A N/A np sort N/A N/A N/A np random rand n_samples N/A y N/A N/A true_fun X N/A N/A N/A N/A np random randn n_samples 0.1 N/A N/A N/A plt figure figsize N/A 14 5 N/A i N/A range N/A len degrees N/A N/A ax N/A N/A plt subplot 1 N/A len degrees N/A i 1 N/A N/A N/A plt setp ax xticks N/A yticks N/A N/A polynomial_features N/A PolynomialFeatures degree N/A degrees N/A i include_bias False N/A linear_regression N/A LinearRegression N/A pipeline N/A Pipeline N/A N/A polynomial_features polynomial_features N/A linear_regression linear_regression N/A N/A N/A pipeline fit N/A X N/A N/A N/A N/A np newaxis y N/A scores N/A cross_val_score pipeline N/A X N/A N/A N/A N/A np newaxis y scoring mean_squared_error cv 10 N/A X_test N/A N/A np linspace 0 1 100 N/A N/A N/A plt plot X_test N/A N/A pipeline predict N/A X_test N/A N/A N/A N/A np newaxis label Model N/A N/A N/A plt plot X_test N/A true_fun X_test label True function N/A N/A N/A plt scatter X y label Samples N/A N/A N/A plt xlabel x N/A N/A N/A plt ylabel y N/A N/A N/A plt xlim N/A 0 1 N/A N/A N/A plt ylim N/A -2 2 N/A N/A N/A plt legend loc best N/A N/A N/A plt title N/A N/A Degree {}
MSE = {:.2e}(+/- {:.2e}) format N/A degrees N/A i N/A N/A N/A scores mean N/A N/A scores std N/A N/A N/A plt show