Module Expr Str ImportFrom alias Import alias Import alias identifier ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Import alias identifier Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Num Assign NameStore BinOpMult Num Call AttributeLoad NameLoad attr Num Num Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr AugAssignAdd SubscriptStore NameLoad Slice Num BinOpMult Num BinOpSub Num Call AttributeLoad NameLoad attr BinOpDiv SubscriptLoad AttributeLoad NameLoad attr Index Num Num Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Num Num Num ExtSlice Slice Index NameLoad Assign NameStore Num Assign NameStore Call NameLoad Call NameLoad keyword Str keyword Num keyword Num keyword Dict Str Str ListLoad Num Num Num Num Call AttributeLoad NameLoad attr Num Num Num Assign NameStore Call NameLoad Call NameLoad keyword Str keyword Num keyword Num keyword Dict Str Str ListLoad Num Num Num Num Call AttributeLoad NameLoad attr Num Num Num Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice NameLoad SubscriptLoad NameLoad Slice NameLoad Assign NameStore BinOpSub Call AttributeLoad NameLoad attr NameLoad Print BinOpMod Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice NameLoad SubscriptLoad NameLoad Slice NameLoad Assign NameStore BinOpSub Call AttributeLoad NameLoad attr NameLoad Print BinOpMod Str NameLoad Assign NameStore BinOpDiv SubscriptLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Index Num NameLoad Print BinOpMod Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore BinOpSub Call AttributeLoad NameLoad attr NameLoad Print BinOpMod Str TupleLoad SubscriptLoad AttributeLoad NameLoad attr Index Num NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore BinOpSub Call AttributeLoad NameLoad attr NameLoad Print BinOpMod Str TupleLoad SubscriptLoad AttributeLoad NameLoad attr Index Num NameLoad Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad SubscriptLoad NameLoad Index NameLoad keyword Str keyword Num keyword Str keyword Num Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice Num SubscriptLoad NameLoad Slice Num keyword Str keyword Str keyword Num Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr NameLoad NameLoad keyword Str keyword BinOpMod Str TupleLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad keyword Str keyword BinOpMod Str TupleLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Assign NameStore BinOpMult Num Call AttributeLoad NameLoad attr Num Num Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr AugAssignAdd SubscriptStore NameLoad Slice Num BinOpMult Num BinOpSub Num Call AttributeLoad NameLoad attr BinOpDiv SubscriptLoad AttributeLoad NameLoad attr Index Num Num Assign NameStore Call AttributeLoad NameLoad attr Num Num Num For TupleStore NameStore NameStore Call AttributeLoad Dict Str Str Call NameLoad keyword Str keyword Num keyword Num Call NameLoad keyword Str keyword Num keyword Num attr body Assign NameStore ListLoad Assign NameStore ListLoad For NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice NameLoad SubscriptLoad NameLoad Slice NameLoad Expr Call AttributeLoad NameLoad attr BinOpSub Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice Num Expr Call AttributeLoad NameLoad attr BinOpSub Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Str keyword IfExp CompareEq NameLoad Str Str Str keyword BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Str keyword IfExp CompareEq NameLoad Str Str Str keyword BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr keyword Str Expr Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad keyword Str keyword Num keyword Num Assign NameStore Call NameLoad keyword Str keyword Num keyword Num Assign TupleStore NameStore NameStore NameStore Call NameLoad NameLoad SubscriptLoad NameLoad Slice Num SubscriptLoad NameLoad Slice Num keyword Call AttributeLoad NameLoad attr Num Num Num keyword Str keyword Num Assign TupleStore NameStore NameStore NameStore Call NameLoad NameLoad SubscriptLoad NameLoad Slice Num SubscriptLoad NameLoad Slice Num keyword Call AttributeLoad NameLoad attr Num Num Num keyword Str keyword Num Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num Str keyword Str keyword Str Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num Str keyword Str keyword Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr keyword Str Expr Call AttributeLoad NameLoad attr
N/A N/A 
=============================================
Comparison of kernel ridge regression and SVR
=============================================

Both kernel ridge regression (KRR) and SVR learn a non-linear function by
employing the kernel trick, i.e., they learn a linear function in the space
induced by the respective kernel which corresponds to a non-linear function in
the original space. They differ in the loss functions (ridge versus
epsilon-insensitive loss). In contrast to SVR, fitting a KRR can be done in
closed-form and is typically faster for medium-sized datasets. On the other
hand, the learned model is non-sparse and thus slower than SVR at
prediction-time.

This example illustrates both methods on an artificial dataset, which
consists of a sinusoidal target function and strong noise added to every fifth
datapoint. The first figure compares the learned model of KRR and SVR when both
complexity/regularization and bandwidth of the RBF kernel are optimized using
grid-search. The learned functions are very similar; however, fitting KRR is
approx. seven times faster than fitting SVR (both with grid-search). However,
prediction of 100000 target values is more than tree times faster with SVR
since it has learned a sparse model using only approx. 1/3 of the 100 training
datapoints as support vectors.

The next figure compares the time for fitting and prediction of KRR and SVR for
different sizes of the training set. Fitting KRR is faster than SVR for medium-
sized training sets (less than 1000 samples); however, for larger training sets
SVR scales better. With regard to prediction time, SVR is faster than
KRR for all sizes of the training set because of the learned sparse
solution. Note that the degree of sparsity and thus the prediction time depends
on the parameters epsilon and C of the SVR.
 __future__ division N/A time N/A numpy np sklearn.svm SVR sklearn.model_selection GridSearchCV sklearn.model_selection learning_curve sklearn.kernel_ridge KernelRidge N/A matplotlib.pyplot plt N/A rng N/A N/A N/A np random RandomState 0 N/A X N/A 5 N/A N/A rng rand 10000 1 N/A y N/A N/A N/A N/A np sin X ravel N/A N/A y N/A 5 N/A 3 N/A 0.5 N/A N/A rng rand N/A N/A N/A X shape N/A 0 5 N/A X_plot N/A N/A N/A np linspace 0 5 100000 N/A N/A N/A None N/A train_size 100 N/A svr N/A GridSearchCV N/A SVR kernel rbf gamma 0.1 cv 5 param_grid N/A C gamma N/A 1.0 10.0 100.0 1000.0 N/A N/A np logspace -2 2 5 N/A kr N/A GridSearchCV N/A KernelRidge kernel rbf gamma 0.1 cv 5 param_grid N/A alpha gamma N/A 1.0 0.1 0.01 0.001 N/A N/A np logspace -2 2 5 N/A t0 N/A N/A time time N/A N/A N/A svr fit N/A X N/A train_size N/A y N/A train_size N/A svr_fit N/A N/A N/A time time t0 N/A N/A SVR complexity and bandwidth selected and model fitted in %.3f s svr_fit N/A t0 N/A N/A time time N/A N/A N/A kr fit N/A X N/A train_size N/A y N/A train_size N/A kr_fit N/A N/A N/A time time t0 N/A N/A KRR complexity and bandwidth selected and model fitted in %.3f s kr_fit N/A sv_ratio N/A N/A N/A N/A N/A svr best_estimator_ support_ shape N/A 0 train_size N/A N/A Support vector ratio: %.3f sv_ratio N/A t0 N/A N/A time time N/A y_svr N/A N/A svr predict X_plot N/A svr_predict N/A N/A N/A time time t0 N/A N/A SVR prediction for %d inputs in %.3f s N/A N/A N/A X_plot shape N/A 0 svr_predict N/A t0 N/A N/A time time N/A y_kr N/A N/A kr predict X_plot N/A kr_predict N/A N/A N/A time time t0 N/A N/A KRR prediction for %d inputs in %.3f s N/A N/A N/A X_plot shape N/A 0 kr_predict N/A sv_ind N/A N/A svr best_estimator_ support_ N/A N/A N/A plt scatter N/A X N/A sv_ind N/A y N/A sv_ind c r s 50 label SVR support vectors zorder 2 N/A N/A N/A plt scatter N/A X N/A 100 N/A y N/A 100 c k label data zorder 1 N/A N/A N/A plt hold on N/A N/A N/A plt plot X_plot y_svr c r label N/A SVR (fit: %.3fs, predict: %.3fs) N/A svr_fit svr_predict N/A N/A N/A plt plot X_plot y_kr c g label N/A KRR (fit: %.3fs, predict: %.3fs) N/A kr_fit kr_predict N/A N/A N/A plt xlabel data N/A N/A N/A plt ylabel target N/A N/A N/A plt title SVR versus Kernel Ridge N/A N/A N/A plt legend N/A N/A N/A plt figure N/A X N/A 5 N/A N/A rng rand 10000 1 N/A y N/A N/A N/A N/A np sin X ravel N/A N/A y N/A 5 N/A 3 N/A 0.5 N/A N/A rng rand N/A N/A N/A X shape N/A 0 5 N/A sizes N/A N/A np logspace 1 4 7 N/A N/A name estimator N/A N/A N/A KRR SVR N/A KernelRidge kernel rbf alpha 0.1 gamma 10 N/A SVR kernel rbf C 10.0 gamma 10 items N/A N/A train_time N/A N/A test_time N/A N/A train_test_size sizes N/A N/A t0 N/A N/A time time N/A N/A N/A estimator fit N/A X N/A train_test_size N/A y N/A train_test_size N/A N/A N/A train_time append N/A N/A N/A time time t0 N/A t0 N/A N/A time time N/A N/A N/A estimator predict N/A X_plot N/A 1000 N/A N/A N/A test_time append N/A N/A N/A time time t0 N/A N/A N/A plt plot sizes train_time o- color N/A N/A name SVR r g label N/A %s (train) name N/A N/A N/A plt plot sizes test_time o-- color N/A N/A name SVR r g label N/A %s (test) name N/A N/A N/A plt xscale log N/A N/A N/A plt yscale log N/A N/A N/A plt xlabel Train size N/A N/A N/A plt ylabel Time (seconds) N/A N/A N/A plt title Execution Time N/A N/A N/A plt legend loc best N/A N/A N/A plt figure N/A svr N/A SVR kernel rbf C 10.0 gamma 0.1 N/A kr N/A KernelRidge kernel rbf alpha 0.1 gamma 0.1 N/A N/A train_sizes train_scores_svr test_scores_svr N/A learning_curve svr N/A X N/A 100 N/A y N/A 100 train_sizes N/A N/A np linspace 0.1 1 10 scoring mean_squared_error cv 10 N/A N/A train_sizes_abs train_scores_kr test_scores_kr N/A learning_curve kr N/A X N/A 100 N/A y N/A 100 train_sizes N/A N/A np linspace 0.1 1 10 scoring mean_squared_error cv 10 N/A N/A N/A plt plot train_sizes N/A N/A test_scores_svr mean 1 o- color r label SVR N/A N/A N/A plt plot train_sizes N/A N/A test_scores_kr mean 1 o- color g label KRR N/A N/A N/A plt xlabel Train size N/A N/A N/A plt ylabel Mean Squared Error N/A N/A N/A plt title Learning curves N/A N/A N/A plt legend loc best N/A N/A N/A plt show