Module Expr Str ImportFrom alias alias Import alias ImportFrom alias Import alias Import alias identifier ImportFrom alias alias ImportFrom alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Str keyword Str FunctionDef arguments args NameParam NameParam NameParam defaults AttributeLoad NameLoad attr Str Num body Expr Str Expr Call NameLoad Str Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Str keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad CompareNotEq SubscriptLoad NameLoad Index Str Num attr AttributeLoad NameLoad attr Expr Call NameLoad Str Assign NameStore Num Assign NameStore SubscriptLoad NameLoad Slice NameLoad Assign NameStore SubscriptLoad NameLoad Slice NameLoad Assign NameStore SubscriptLoad NameLoad Slice NameLoad Assign NameStore SubscriptLoad NameLoad Slice NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num Assign NameStore Call AttributeLoad NameLoad attr keyword Num Assign SubscriptStore NameLoad Slice Num Num Assign SubscriptStore NameLoad Slice Num Num Assign NameStore BinOpDiv BinOpSub NameLoad NameLoad NameLoad Assign NameStore BinOpDiv BinOpSub NameLoad NameLoad NameLoad Return TupleLoad NameLoad NameLoad NameLoad NameLoad decorator_list AttributeLoad NameLoad attr Assign NameStore Dict Str Str Str Str Str Str Str Str Call NameLoad keyword Num Call NameLoad keyword Num Call NameLoad keyword Num Call NameLoad keyword Num Call NameLoad keyword Num keyword Num Call NameLoad Call NameLoad keyword Str keyword Str keyword Num keyword NameLoad keyword Num Call NameLoad keyword Str keyword Num keyword Num If CompareEq NameLoad Str body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str keyword Str keyword NameLoad keyword NameLoad keyword ListLoad Str Str Str Str keyword Str Expr Call AttributeLoad NameLoad attr Str keyword Str keyword Num keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str keyword Str keyword Str keyword NameLoad keyword ListLoad Str Str keyword Str Expr Call AttributeLoad NameLoad attr Str keyword Str keyword Num keyword NameLoad keyword Str Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Expr Call NameLoad NameLoad Assign TupleStore NameStore NameStore NameStore NameStore Call NameLoad keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str Expr Call NameLoad Str Expr Call NameLoad Str Expr Call NameLoad Str Expr Call NameLoad BinOpMod Str TupleLoad Call AttributeLoad Str attr Num SubscriptLoad AttributeLoad NameLoad attr Index Num Expr Call NameLoad BinOpMod Str TupleLoad Call AttributeLoad Str attr Num AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr Expr Call NameLoad BinOpMod Str TupleLoad Call AttributeLoad Str attr Num AttributeLoad NameLoad attr Expr Call NameLoad BinOpMod Str TupleLoad Call AttributeLoad Str attr Num SubscriptLoad AttributeLoad NameLoad attr Index Num Call AttributeLoad NameLoad attr CompareEq NameLoad Num Call AttributeLoad NameLoad attr CompareEq NameLoad Num Call NameLoad BinOpDiv AttributeLoad NameLoad attr Num Expr Call NameLoad BinOpMod Str TupleLoad Call AttributeLoad Str attr Num SubscriptLoad AttributeLoad NameLoad attr Index Num Call AttributeLoad NameLoad attr CompareEq NameLoad Num Call AttributeLoad NameLoad attr CompareEq NameLoad Num Call NameLoad BinOpDiv AttributeLoad NameLoad attr Num Expr Call NameLoad Expr Call NameLoad Str Expr Call NameLoad Str Assign TupleStore NameStore NameStore NameStore TupleLoad Dict Dict Dict For NameStore Call NameLoad SubscriptLoad NameLoad Index Str body Expr Call NameLoad BinOpMod Str NameLoad keyword Str Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr DictComp NameLoad SubscriptLoad NameLoad Index Str comprehension NameStore NameLoad Call AttributeLoad NameLoad attr Str If CompareIn Str NameLoad body Expr Call AttributeLoad NameLoad attr keyword SubscriptLoad NameLoad Index Str Assign NameStore Call NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad BinOpSub Call NameLoad NameLoad Assign NameStore Call NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign SubscriptStore NameLoad Index NameLoad BinOpSub Call NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad NameLoad Expr Call NameLoad Str Expr Call NameLoad Expr Call NameLoad Str Expr Call NameLoad Str Expr Call NameLoad BinOpMod Str TupleLoad Str Str Str Str Expr Call NameLoad BinOpMult Str Num For NameStore Call NameLoad SubscriptLoad NameLoad Index Str keyword AttributeLoad NameLoad attr body Expr Call NameLoad BinOpMod Str TupleLoad Call AttributeLoad NameLoad attr Num Call AttributeLoad BinOpMod Str SubscriptLoad NameLoad Index NameLoad attr Num Call AttributeLoad BinOpMod Str SubscriptLoad NameLoad Index NameLoad attr Num Call AttributeLoad BinOpMod Str SubscriptLoad NameLoad Index NameLoad attr Num Expr Call NameLoad
N/A N/A 
===========================
Covertype dataset benchmark
===========================

Benchmark stochastic gradient descent (SGD), Liblinear, and Naive Bayes, CART
(decision tree), RandomForest and Extra-Trees on the forest covertype dataset
of Blackard, Jock, and Dean [1]. The dataset comprises 581,012 samples. It is
low dimensional with 54 features and a sparsity of approx. 23%. Here, we
consider the task of predicting class 1 (spruce/fir). The classification
performance of SGD is competitive with Liblinear while being two orders of
magnitude faster to train::

    [..]
    Classification performance:
    ===========================
    Classifier   train-time test-time error-rate
    --------------------------------------------
    liblinear     15.9744s    0.0705s     0.2305
    GaussianNB    3.0666s     0.3884s     0.4841
    SGD           1.0558s     0.1152s     0.2300
    CART          79.4296s    0.0523s     0.0469
    RandomForest  1190.1620s  0.5881s     0.0243
    ExtraTrees    640.3194s   0.6495s     0.0198

The same task has been used in a number of papers including:

 * `"SVM Optimization: Inverse Dependence on Training Set Size"
   <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.139.2112>`_
   S. Shalev-Shwartz, N. Srebro - In Proceedings of ICML '08.

 * `"Pegasos: Primal estimated sub-gradient solver for svm"
   <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.74.8513>`_
   S. Shalev-Shwartz, Y. Singer, N. Srebro - In Proceedings of ICML '07.

 * `"Training Linear SVMs in Linear Time"
   <www.cs.cornell.edu/People/tj/publications/joachims_06a.pdf>`_
   T. Joachims - In SIGKDD '06

[1] http://archive.ics.uci.edu/ml/datasets/Covertype

 __future__ division print_function N/A os time time N/A argparse N/A numpy np sklearn.datasets fetch_covtype get_data_home sklearn.svm LinearSVC sklearn.linear_model SGDClassifier LogisticRegression sklearn.naive_bayes GaussianNB sklearn.tree DecisionTreeClassifier sklearn.ensemble RandomForestClassifier ExtraTreesClassifier sklearn.ensemble GradientBoostingClassifier sklearn.metrics zero_one_loss sklearn.externals.joblib Memory sklearn.utils check_array N/A memory N/A Memory N/A N/A N/A os path join N/A get_data_home covertype_benchmark_data mmap_mode r load_data N/A N/A dtype order random_state N/A N/A np float32 C 13 N/A N/A Load the data, then cache and memmap the train/test split N/A N/A print Loading dataset... N/A data N/A fetch_covtype download_if_missing True shuffle True random_state random_state N/A X N/A check_array N/A data N/A data dtype dtype order order N/A y N/A N/A N/A N/A data N/A target 1 astype N/A np int N/A N/A print Creating train-test split... N/A n_train 522911 N/A X_train N/A X N/A n_train N/A y_train N/A y N/A n_train N/A X_test N/A X N/A n_train N/A y_test N/A y N/A n_train N/A mean N/A N/A X_train mean axis 0 N/A std N/A N/A X_train std axis 0 N/A N/A mean N/A 10 0.0 N/A N/A std N/A 10 1.0 N/A X_train N/A N/A X_train mean std N/A X_test N/A N/A X_test mean std N/A N/A X_train X_test y_train y_test N/A N/A memory cache N/A ESTIMATORS N/A GBRT ExtraTrees RandomForest CART SGD GaussianNB liblinear SAG N/A GradientBoostingClassifier n_estimators 250 N/A ExtraTreesClassifier n_estimators 20 N/A RandomForestClassifier n_estimators 20 N/A DecisionTreeClassifier min_samples_split 5 N/A SGDClassifier alpha 0.001 n_iter 2 N/A GaussianNB N/A LinearSVC loss l2 penalty l2 C 1000 dual False tol 0.001 N/A LogisticRegression solver sag max_iter 2 C 1000 N/A N/A __name__ __main__ N/A N/A parser N/A N/A argparse ArgumentParser N/A N/A N/A parser add_argument --classifiers nargs + choices ESTIMATORS type str default N/A liblinear GaussianNB SGD CART help list of classifiers to benchmark. N/A N/A N/A parser add_argument --n-jobs nargs ? default 1 type int help Number of concurrently running workers for models that support parallelism. N/A N/A N/A parser add_argument --order nargs ? default C type str choices N/A F C help Allow to choose between fortran and C ordered data N/A N/A N/A parser add_argument --random-seed nargs ? default 13 type int help Common seed used by random number generator. N/A args N/A vars N/A N/A parser parse_args N/A N/A print __doc__ N/A N/A X_train X_test y_train y_test N/A load_data order N/A args N/A order random_state N/A args N/A random_seed N/A N/A print N/A N/A print Dataset statistics: N/A N/A print =================== N/A N/A print N/A %s %d N/A N/A N/A number of features: ljust 25 N/A N/A X_train shape N/A 1 N/A N/A print N/A %s %d N/A N/A N/A number of classes: ljust 25 N/A N/A N/A np unique y_train size N/A N/A print N/A %s %s N/A N/A N/A data type: ljust 25 N/A X_train dtype N/A N/A print N/A %s %d (pos=%d, neg=%d, size=%dMB) N/A N/A N/A number of train samples: ljust 25 N/A N/A X_train shape N/A 0 N/A N/A np sum N/A y_train 1 N/A N/A np sum N/A y_train 0 N/A int N/A N/A X_train nbytes 1000000.0 N/A N/A print N/A %s %d (pos=%d, neg=%d, size=%dMB) N/A N/A N/A number of test samples: ljust 25 N/A N/A X_test shape N/A 0 N/A N/A np sum N/A y_test 1 N/A N/A np sum N/A y_test 0 N/A int N/A N/A X_test nbytes 1000000.0 N/A N/A print N/A N/A print Training Classifiers N/A N/A print ==================== N/A N/A error train_time test_time N/A N/A N/A N/A N/A name N/A sorted N/A args N/A classifiers N/A N/A N/A print N/A Training %s ...  name end N/A estimator N/A ESTIMATORS N/A name N/A estimator_params N/A N/A estimator get_params N/A N/A N/A estimator set_params N/A p N/A args N/A random_seed N/A p estimator_params N/A N/A p endswith random_state N/A N/A n_jobs estimator_params N/A N/A N/A N/A estimator set_params n_jobs N/A args N/A n_jobs N/A time_start N/A time N/A N/A N/A estimator fit X_train y_train N/A N/A train_time N/A name N/A N/A time time_start N/A time_start N/A time N/A y_pred N/A N/A estimator predict X_test N/A N/A test_time N/A name N/A N/A time time_start N/A N/A error N/A name N/A zero_one_loss y_test y_pred N/A N/A print done N/A N/A print N/A N/A print Classification performance: N/A N/A print =========================== N/A N/A print N/A %s %s %s %s N/A Classifier   train-time test-time error-rate N/A N/A print N/A - 44 N/A name N/A sorted N/A args N/A classifiers key N/A error get N/A N/A N/A print N/A %s %s %s %s N/A N/A N/A name ljust 12 N/A N/A N/A %.4fs N/A train_time N/A name center 10 N/A N/A N/A %.4fs N/A test_time N/A name center 10 N/A N/A N/A %.4f N/A error N/A name center 10 N/A N/A print