Package: oboe
The Oboe systemsThis bundle of libraries, Oboe and TensorOboe, are automated machine learning (AutoML) systems that use collaborative filtering to find good models for supervised learning tasks within a user-specified time limit. Further hyperparameter tuning can be performed afterwards.The name comes from the musical instrument oboe: in an orchestra, oboe plays an initial note which the other instruments use to tune to the right frequency before the performance begins. Our Oboe systems play a similar role in AutoML: we use meta-learning to select a promising set of models or to build an ensemble for a new dataset. Users can either directly use the selected models or further fine-tune their hyperparameters.On a new dataset:Oboe searches for promising estimators (supervised learners) by matrix factorization and classical experiment design. It requires a pre-processed dataset: one-hot encode categorical features and then standardize all features to have zero meanand unit variance. For a complete description, refer to our paperOBOE: Collaborative Filtering for AutoML Model Selectionat KDD 2019.TensorOboe searches for promising pipelines, which are directed graphs of learning components here, including imputation, encoding, standardization, dimensionality reduction and estimation. Thus it can accept a raw dataset, possibly with missing entries, different types of features, not-centered features, etc. For a complete description, refer to our paperAutoML Pipeline Selection: Efficiently Navigating the Combinatorial Spaceat KDD 2020.This bundle of systems is still under developement and subjects to change. For any questions, please submit an issue. The authors will respond as soon as possible.InstallationThe easiest way is to install using pip:pip install oboeAlternatively, if you want to customize the source code, you may install in the editable mode by firstgit clonethis respository, and then dopip install -e .in the cloned directory. Note this will download some large (about 100MB in total) files to warm-start TensorOboe fitting, so that the setup time (in minutes) can be saved at the cost of disk space and network data usage.It is recommended to install within an isolated environment (a conda virtual environment, for example) to avoid conflicting dependency versions.Dependencies with verified versionsThe Oboe systems work on Python 3.7 or later. The following libraries are required. The listed versions are the versions that are verified to work. Older versions may work but are not guaranteed.numpy  (1.16.4)scipy  (1.4.1)pandas (0.24.2)scikit-learn  (0.22.1)tensorly (0.6.0)OpenML (0.9.0)mkl (>=1.0.0)ExamplesFor more detailed examples, please refer to the Jupyter notebooks in theexamplefolder. A basic classification example using Oboe:method='Oboe'# 'Oboe' or 'TensorOboe'problem_type='classification'fromoboeimportAutoLearner,error# This may take around 15 seconds at first run.importnumpyasnpfromsklearn.datasetsimportload_irisfromsklearn.model_selectionimporttrain_test_splitdata=load_iris()x=np.array(data['data'])y=np.array(data['target'])x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.2)m=AutoLearner(p_type=problem_type,runtime_limit=30,method=method,verbose=False)m.fit(x_train,y_train)y_predicted=m.predict(x_test)print("prediction error (balanced error rate):{}".format(error(y_test,y_predicted,'classification')))print("selected models:{}".format(m.get_models()))Warm-start meta-trainingThelarge_filesfolder includes some largenumpyarrays that are intermediate results of previous meta-training. This folder is not included in thepipinstallation, and the files within it can be manually downloaded from this GitHub repository.The default functionality in TensorOboe is to skip the step of imputing missing entries in the error tensor, and directly use the pre-imputed error tensor. If users desire to impute the error tensor by themselves, the original non-imputed error tensor can be found atlarge_files/error_tensor_f16_compressed.npz, and the TensorOboe initialization can be done by setting theoriginal_error_tensor_dirargument to the path of this.npzfile, and settingmodeto'initialize'when creating the AutoLearner instance:m = AutoLearner(..., method='TensorOboe', mode='initialize', path_to_imputed_error_tensor=<path_to_this_npy_file>).References[1] Chengrun Yang, Yuji Akimoto, Dae Won Kim, Madeleine Udell. OBOE: Collaborative filtering for AutoML model selection. KDD 2019.[2] Chengrun Yang, Jicong Fan, Ziyang Wu, Madeleine Udell. AutoML Pipeline Selection: Efficiently Navigating the Combinatorial Space. KDD 2020.