Package: rolexboost
RolexBoostUnofficial implementation of D. Yang, H. Lee and D. Lim, "RolexBoost: A Rotation-Based Boosting Algorithm With Adaptive Loss Functions," in IEEE Access, vol. 8, pp. 41037-41044, 2020, doi: 10.1109/ACCESS.2020.2976822.This is the course project of Fundamentals of Machine Learning, Tsinghua University, 2020 Autumn.InstallationpipinstallrolexboostAPI referenceWe provided scikit-learn-like API for the RolexBoost algorithm proposed in the paper,
together with the RotationForest and FlexBoost, which are the source of RolexBoost's idea.Note thatOnly classifiers are provided. We did not implemented the regressors because they are not mentioned in the paper, while this project is intended as a reproduction.We only ensures that thefitandpredictAPI works well. Some others, such asscore, may be functional thanks to the scikit-learnBaseEstimatorandClassifierMixinbase classes, but still others, such asfit_predictorpredict_proba, are currently unavailable.We might implement those two in the future if someone is interested in this project.Basic Example>>>importpandasaspd>>>importnumpyasnp>>>fromrolexboostimportRolexBoostClassifier,FlexBoostClassifier,RotationForestClassifier>>>clf=RolexBoostClassifier()# Or the other two classifiers>>>df=pd.DataFrame({"A":[2,6,5,7,1,8],"B":[8,5,2,3,4,6],"C":[3,9,5,4,6,1],"Label":[0,1,1,0,0,1]})>>>dfABCLabel028301659125251373404146058611>>>X=df[["A","B","C"]]>>>y=df["Label"]>>>clf.fit(X,y)RolexBoostClassifier()>>>clf.predict(X)array([0,1,1,0,0,1],dtype=int64)>>>test_X=np.array([...[3,1,2],...[2,5,1],...[5,1,7]...])>>>clf.predict(test_X)array([1,0,1],dtype=int64)API ReferencesRotation ForestRotationForestClassifier(n_estimators=100,n_features_per_subset=3,bootstrap_rate=0.75,criterion="gini",splitter="best",max_depth=None,min_samples_split=2,min_samples_leaf=1,min_weight_fraction_leaf=0.0,max_features=None,random_state=None,max_leaf_nodes=None,min_impurity_decrease=0.0,min_impurity_split=None,class_weight=None,presort="deprecated",ccp_alpha=0.0,)n_estimators: number of base estimatorsn_features_per_subset: number of features in each subsetbootstrap_rate: ratio of samples bootstrapped in the original datasetAll other parameters are passed to theDecisionTreeClassifierof scikit-learn. Please refer totheir documentationfor details.Note:In the algorithm description, a parameter controls the number of subsets, and the number of features is derived from it.
However, the validation part of the paper says that "the number of features in each subset was set to three".
In our framework, the parametern_samples_per_subsetis thus formulated in this way to make the benchmark evaluation easier.FlexBoostFlexBoostClassifier(n_estimators=100,K=0.5,criterion="gini",splitter="best",max_depth=1,min_samples_split=2,min_samples_leaf=1,min_weight_fraction_leaf=0.0,max_features=None,random_state=None,max_leaf_nodes=None,min_impurity_decrease=0.0,min_impurity_split=None,class_weight=None,presort="deprecated",ccp_alpha=0.0,)n_estimators: number of base estimatorsK: the parameter to control the "aggressiveness" and "conservativeness" in the adaptive loss function choosing process. It should be a number between 0 and 1.All other parameters are passed to theDecisionTreeClassifierof scikit-learn. Please refer totheir documentationfor details.The default parameter formax_depthis 1, because FlexBoost is a modification of AdaBoost, and they should converge to the same result whenK=1.
Inscikit-learn implementation of AdaBoost, the defaultmax_depthfor the DecisionTreeClassifier is 1.RolexBoostRolexBoostClassifier(n_estimators=100,n_features_per_subset=3,bootstrap_rate=0.75,K=0.5,criterion="gini",splitter="best",max_depth=1,min_samples_split=2,min_samples_leaf=1,min_weight_fraction_leaf=0.0,max_features=None,random_state=None,max_leaf_nodes=None,min_impurity_decrease=0.0,min_impurity_split=None,class_weight=None,presort="deprecated",ccp_alpha=0.0)n_estimators: number of base estimatorsn_features_per_subset: number of features in each subsetbootstrap_rate: ratio of samples bootstrapped in the original datasetK: the parameter to control the "aggressiveness" and "conservativeness" in the adaptive loss function choosing process. It should be a number between 0 and 1.All other parameters are passed to theDecisionTreeClassifierof scikit-learn. Please refer totheir documentationfor details.Note:In the algorithm description, a parameter controls the number of subsets, and the number of features is derived from it.
However, the validation part of the paper says that "the number of features in each subset was set to three".
In our framework, the parametern_samples_per_subsetis thus formulated in this way to make the benchmark evaluation easier.The default parameter formax_depthis 1, because RolexBoost integrates the idea of FlexBoost. Please refer to the last section about why FlexBoost has a defaultmax_depthof 1.Performance BenchmarksWe have tested the three algorithms on 13 datasets mentioned in the paper.Here is the result:algorithmaccuracybenchmarkratioRotationForest0.78980.79470.9938FlexBoost0.79760.80950.9853RolexBoost0.77750.81670.9520accuracyrefers to the average accuracy of our implementationbenchmarkrefers to the average accuracy reported in the paperratiois accuracy/benchmarkFor the detail of each algorithm on each dataset, please run the tests/accuracy-test.py.
The test may take ~1 hour to finish.Some datasets reported in the paper are not involved in the benchmark testing for the following two reasons:We cannot find the corresponding dataset in the UCI Machine Learning RepositoryThe 3-class problems are each divided into three 2-class problems. We are not sure about how such division is done.