Module Expr Str Print NameLoad Import alias identifier Import alias identifier ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Assign NameStore Num Assign NameStore Num Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr keyword Num keyword Num Assign TupleStore NameStore NameStore TupleLoad SubscriptLoad NameLoad Slice Num SubscriptLoad NameLoad Slice Num Assign TupleStore NameStore NameStore TupleLoad SubscriptLoad NameLoad Slice Num SubscriptLoad NameLoad Slice Num Assign NameStore Call NameLoad keyword Num keyword Num Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore BinOpSub Num Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call NameLoad keyword Num keyword Num Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore BinOpSub Num Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr ListLoad Num NameLoad BinOpMult ListLoad NameLoad Num Str keyword Str Expr Call AttributeLoad NameLoad attr ListLoad Num NameLoad BinOpMult ListLoad NameLoad Num Str keyword Str Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad For TupleStore NameStore NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad For TupleStore NameStore NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad For TupleStore NameStore NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad For TupleStore NameStore NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr NameLoad Num NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr NameLoad Num NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr NameLoad Num NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr NameLoad Num NameLoad keyword Str keyword Str Expr Call AttributeLoad NameLoad attr TupleLoad Num Num Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr keyword Str keyword NameLoad Expr Call AttributeLoad Call AttributeLoad NameLoad attr attr Num Expr Call AttributeLoad NameLoad attr
N/A N/A 
=============================
Discrete versus Real AdaBoost
=============================

This example is based on Figure 10.2 from Hastie et al 2009 [1] and illustrates
the difference in performance between the discrete SAMME [2] boosting
algorithm and real SAMME.R boosting algorithm. Both algorithms are evaluated
on a binary classification task where the target Y is a non-linear function
of 10 input features.

Discrete SAMME AdaBoost adapts based on errors in predicted class labels
whereas real SAMME.R uses the predicted class probabilities.

.. [1] T. Hastie, R. Tibshirani and J. Friedman, "Elements of Statistical
    Learning Ed. 2", Springer, 2009.

.. [2] J. Zhu, H. Zou, S. Rosset, T. Hastie, "Multi-class AdaBoost", 2009.

 N/A __doc__ N/A numpy np N/A matplotlib.pyplot plt sklearn datasets sklearn.tree DecisionTreeClassifier sklearn.metrics zero_one_loss sklearn.ensemble AdaBoostClassifier N/A n_estimators 400 N/A learning_rate 1.0 N/A N/A X y N/A N/A datasets make_hastie_10_2 n_samples 12000 random_state 1 N/A N/A X_test y_test N/A N/A X N/A 2000 N/A y N/A 2000 N/A N/A X_train y_train N/A N/A X N/A 2000 N/A y N/A 2000 N/A dt_stump N/A DecisionTreeClassifier max_depth 1 min_samples_leaf 1 N/A N/A N/A dt_stump fit X_train y_train N/A dt_stump_err N/A 1.0 N/A N/A dt_stump score X_test y_test N/A dt N/A DecisionTreeClassifier max_depth 9 min_samples_leaf 1 N/A N/A N/A dt fit X_train y_train N/A dt_err N/A 1.0 N/A N/A dt score X_test y_test N/A ada_discrete N/A AdaBoostClassifier base_estimator dt_stump learning_rate learning_rate n_estimators n_estimators algorithm SAMME N/A N/A N/A ada_discrete fit X_train y_train N/A ada_real N/A AdaBoostClassifier base_estimator dt_stump learning_rate learning_rate n_estimators n_estimators algorithm SAMME.R N/A N/A N/A ada_real fit X_train y_train N/A fig N/A N/A plt figure N/A ax N/A N/A fig add_subplot 111 N/A N/A N/A ax plot N/A 1 n_estimators N/A N/A dt_stump_err 2 k- label Decision Stump Error N/A N/A N/A ax plot N/A 1 n_estimators N/A N/A dt_err 2 k-- label Decision Tree Error N/A ada_discrete_err N/A N/A np zeros N/A n_estimators N/A N/A i y_pred N/A enumerate N/A N/A ada_discrete staged_predict X_test N/A N/A N/A ada_discrete_err N/A i N/A zero_one_loss y_pred y_test N/A ada_discrete_err_train N/A N/A np zeros N/A n_estimators N/A N/A i y_pred N/A enumerate N/A N/A ada_discrete staged_predict X_train N/A N/A N/A ada_discrete_err_train N/A i N/A zero_one_loss y_pred y_train N/A ada_real_err N/A N/A np zeros N/A n_estimators N/A N/A i y_pred N/A enumerate N/A N/A ada_real staged_predict X_test N/A N/A N/A ada_real_err N/A i N/A zero_one_loss y_pred y_test N/A ada_real_err_train N/A N/A np zeros N/A n_estimators N/A N/A i y_pred N/A enumerate N/A N/A ada_real staged_predict X_train N/A N/A N/A ada_real_err_train N/A i N/A zero_one_loss y_pred y_train N/A N/A N/A ax plot N/A N/A N/A np arange n_estimators 1 ada_discrete_err label Discrete AdaBoost Test Error color red N/A N/A N/A ax plot N/A N/A N/A np arange n_estimators 1 ada_discrete_err_train label Discrete AdaBoost Train Error color blue N/A N/A N/A ax plot N/A N/A N/A np arange n_estimators 1 ada_real_err label Real AdaBoost Test Error color orange N/A N/A N/A ax plot N/A N/A N/A np arange n_estimators 1 ada_real_err_train label Real AdaBoost Train Error color green N/A N/A N/A ax set_ylim N/A 0.0 0.5 N/A N/A N/A ax set_xlabel n_estimators N/A N/A N/A ax set_ylabel error rate N/A leg N/A N/A ax legend loc upper right fancybox True N/A N/A N/A N/A N/A leg get_frame set_alpha 0.7 N/A N/A N/A plt show