Package: mlregression
*** ATTENTION ***Don't immidiately runpip install mlregression. See SectionInstallation.Machine learning regression (mlregression)Machine Learning Regression (mlregrresion) is an off-the-shelf implementation of the most popular ML methods that automatically takes care of fitting and parameter tuning.Currently, thefullyimplemented models include:Ensemble trees (Random forests, XGBoost, LightGBM, GradientBoostingRegressor, ExtraTreesRegressor)Penalized regression (Ridge, Lasso, ElasticNet, Lars, LassoLars)Neural nets (Simple neural nets with 1-5 hidden layers, rely activation, and early stopping)NB!When using penalized regressions, consider using the native CV-implementation from scikit-learn for speed, e.g., simply setestimator="LassoCV"similar to Example 1.Scikit-learn regressors (together withXGBoostandLightGBM) can be estimated by setting theestimator-argument equal to the name (string) as in Example 1 (estimator="RandomForestRegressor").
Alternatively, one can provide an instance of an estimator, e.g.,estimator=RandomForestRegressor(). Again, this is fully automated for most Scikit-learn regressors, but for non-standard methods, one would have to provide a parameter grid as well, e.g.,param_grid={...}.Please contact the authors below if you find any bugs or have any suggestions for improvement. Thank you!Author: Nicolaj Søndergaard Mühlbach (n.muhlbach at gmail dot com, muhlbach at mit dot edu)Code dependenciesThis code has the following dependencies:Python >=3.6numpy >=1.19pandas >=1.3scikit-learn >=1scikit-learn-intelex >= 2021.3daal >= 2021.3daal4py >= 2021.3tbb >= 2021.4xgboost >=1.5lightgbm >=3.2InstallationBefore callingpip install mlregression, we recommend usingcondato install the dependencies. In our experience, calling the following command works like a charm:conda install -c conda-forge numpy">=1.19" pandas">=1.3" scikit-learn">=1" scikit-learn-intelex">=2021.3" daal">=2021.3" daal4py">=2021.3" tbb">=2021.4" xgboost">=1.5" lightgbm">=3.2" --force-reinstallAfter this, installmlregressionby callingpip install mlregression.
Note that without installing the dependensies, the package will not work. As of now, it does not work when installing the dependensies viapip install. The reason is that we are using the Intel® Extension for Scikit-learn to massively speed up computations, but the dependensies are not properly installed viapip install.UsageWe demonstrate the use ofmlregressionbelow, using random forests, xgboost, and lightGBM as underlying regressors.#------------------------------------------------------------------------------# Libraries#------------------------------------------------------------------------------# Standardfromsklearn.datasetsimportmake_regressionfromsklearn.model_selectionimporttrain_test_split# This libraryfrommlregression.mlregimportMLRegressor#------------------------------------------------------------------------------# Data#------------------------------------------------------------------------------# Generate dataX,y=make_regression(n_samples=500,n_features=10,n_informative=5,n_targets=1,bias=0.0,coef=False,random_state=1991)X_train,X_test,y_train,y_test=train_test_split(X,y)#------------------------------------------------------------------------------# Example 1: Prediction#------------------------------------------------------------------------------# Specify any of the following estimators:""""LinearRegression","RidgeCV", "LassoCV", "ElasticNetCV","RandomForestRegressor","ExtraTreesRegressor", "GradientBoostingRegressor","XGBRegressor", "LGBMegressor","MLPRegressor","""# For instance, pick "RandomForestRegressor"estimator="RandomForestRegressor"# Note that the 'estimator' may also be an instance of a class, e.g., RandomForestRegressor(), conditional on being imported first, e.g. from sklearn.ensemble import RandomForestRegressor# Instantiate model and choose the number of parametrizations to examine using cross-validation ('max_n_models') and the number of cross-validation folds ('n_cv_folds')mlreg=MLRegressor(estimator=estimator,n_cv_folds=5,max_n_models=2)# Fitmlreg.fit(X=X_train,y=y_train)# Predicty_hat=mlreg.predict(X=X_test)# Access all the usual attributesmlreg.best_score_mlreg.best_estimator_# Compute the scoremlreg.score(X=X_test,y=y_test)#------------------------------------------------------------------------------# Example 2: Cross-fitting#------------------------------------------------------------------------------# Instantiate model and choose the number of parametrizations to examine using cross-validation ('max_n_models'), the number of cross-validation folds ('n_cv_folds'), AND the number of cross-fitting folds ('n_cf_folds')mlreg=MLRegressor(estimator=estimator,n_cv_folds=5,max_n_models=2,n_cf_folds=2)# Cross fitmlreg.cross_fit(X=X_train,y=y_train)# Extract in-sample that are estimated in an out-of-sample way (e.g., via cross-fitting)y_hat=mlreg.y_pred_cf_# Likewise, extract the residualized outcomes used in e.g., double machine learning. This is \tilde{Y} = Y - E[Y|X=x]y_res=mlreg.y_res_cf_