Package: self-paced-ensemble
Self-paced Ensemble for Highly Imbalanced Massive Data Classification
(ICDE 2020)Links:Paper|Slides|Video|arXiv|PyPI|API Reference|Related Projects|Zhihu/知乎Self-paced Ensemble (SPE) is an ensemble learning framework for massive highly imbalanced classification. It is an easy-to-use solution to class-imbalanced problems, features outstanding computing efficiency, good performance, and wide compatibility with different learning models. This SPE implementation supports multi-class classification.Note:  
 
SPE is now a part ofimbalanced-ensemble[Doc,PyPI]. Try it for more methods and advanced features!Cite UsIf you find this repository helpful in your work or research, we would greatly appreciate citations to the followingpaper:@inproceedings{liu2020self-paced-ensemble,title={Self-paced Ensemble for Highly Imbalanced Massive Data Classification},author={Liu, Zhining and Cao, Wei and Gao, Zhifeng and Bian, Jiang and Chen, Hechang and Chang, Yi and Liu, Tie-Yan},booktitle={2020 IEEE 36th International Conference on Data Engineering (ICDE)},pages={841--852},year={2020},organization={IEEE}}InstallationIt is recommended to usepipfor installation.Please make sure thelatest versionis installed to avoid potential problems:$pipinstallself-paced-ensemble# normal install$pipinstall--upgradeself-paced-ensemble# update if neededOr you can install SPE by clone this repository:$gitclonehttps://github.com/ZhiningLiu1998/self-paced-ensemble.git
$cdself-paced-ensemble
$pythonsetup.pyinstallFollowing dependencies are required:python(>=3.6)numpy(>=1.13.3)scipy(>=0.19.1)joblib(>=0.11)scikit-learn(>=0.24)imblearn(>=0.7.0)imbalanced-ensemble(>=0.1.3)Table of ContentsCite UsInstallationTable of ContentsBackgroundDocumentationExamplesAPI demoAdvanced usage exampleSave & Load modelCompare SPE with other methodsResultsMiscellaneousReferencesRelated ProjectsContributors ✨BackgroundSPE performs strictly balanced under-sampling in each iteration and is therefore verycomputationally efficient. In addition, SPE does not rely on calculating the distance between samples to perform resampling. It can be easily applied to datasets that lack well-defined distance metrics (e.g. with categorical features / missing values) without any modification. Moreover, as ageneric ensemble framework, our methods can be easily adapted to most of the existing learning methods (e.g., C4.5, SVM, GBDT, and Neural Network) to boost their performance on imbalanced data. Compared to existing imbalance learning methods,SPE works particularly well on datasets that are large-scale, noisy, and highly imbalanced (e.g. with imbalance ratio greater than 100:1).Such kind of data widely exists in real-world industrial applications. The figure below gives an overview of the SPE framework.DocumentationOur SPE implementation can be used much in the same way as thesklearn.ensembleclassifiers. Detailed documentation ofSelfPacedEnsembleClassifiercan be foundHERE.ExamplesYou can check outexamples using SPEfor more comprehensive usage examples.API demofromself_paced_ensembleimportSelfPacedEnsembleClassifierfromsklearn.treeimportDecisionTreeClassifierfromsklearn.datasetsimportmake_classificationfromsklearn.model_selectionimporttrain_test_split# Prepare class-imbalanced train & test dataX,y=make_classification(n_classes=2,random_state=42,weights=[0.1,0.9])X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.5,random_state=42)# Train an SPE classifierclf=SelfPacedEnsembleClassifier(base_estimator=DecisionTreeClassifier(),n_estimators=10,).fit(X_train,y_train)# Predict with an SPE classifierclf.predict(X_test)Advanced usage examplePlease seeusage_example.ipynb.Save & Load modelWe recommend to use joblib or pickle for saving and loading SPE models, e.g.,fromjoblibimportdump,load# save the modeldump(clf,filename='clf.joblib')# load the modelclf=load('clf.joblib')You can also use the alternative APIs provided in SPE:fromself_paced_ensemble.utilsimportsave_model,load_model# save the modelclf.save('clf.joblib')# option 1save_model(clf,'clf.joblib')# option 2# load the modelclf=load_model('clf.joblib')Compare SPE with other methodsPlease seecomparison_example.ipynb.ResultsDataset links:Credit Fraud,KDDCUP,Record Linkage,Payment Simulation.Comparisons of SPE with traditional resampling/ensemble methods in terms of performance & computational efficiency.MiscellaneousThis repository contains:Implementation of Self-paced EnsembleImplementation of 5 ensemble-based imbalance learning baselinesSMOTEBoost[1]SMOTEBagging[2]RUSBoost[3]UnderBagging[4]BalanceCascade[5]Implementation of resampling based imbalance learning baselines [6]Additional experimental resultsNOTE:The implementations of other ensemble and resampling methods are based onimbalanced-ensembleandimbalanced-learn.References#Reference[1]N. V. Chawla, A. Lazarevic, L. O. Hall, and K. W. Bowyer, Smoteboost: Improving prediction of the minority class in boosting. in European conference on principles of data mining and knowledge discovery. Springer, 2003, pp. 107–119[2]S. Wang and X. Yao, Diversity analysis on imbalanced data sets by using ensemble models. in 2009 IEEE Symposium on Computational Intelligence and Data Mining. IEEE, 2009, pp. 324–331.[3]C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano, “Rusboost: A hybrid approach to alleviating class imbalance,” IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 40, no. 1, pp. 185–197, 2010.[4]R. Barandela, R. M. Valdovinos, and J. S. Sanchez, “New applications´ of ensembles of classifiers,” Pattern Analysis & Applications, vol. 6, no. 3, pp. 245–256, 2003.[5]X.-Y. Liu, J. Wu, and Z.-H. Zhou, “Exploratory undersampling for class-imbalance learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39, no. 2, pp. 539–550, 2009.[6]Guillaume Lemaître, Fernando Nogueira, and Christos K. Aridas. Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18(17):1–5, 2017.Related ProjectsCheck outZhining's other open-source projects!Imbalanced-Ensemble [PythonLib]Imbalanced Learning [Awesome]Machine Learning [Awesome]Meta-Sampler [NeurIPS]Contributors ✨Thanks goes to these wonderful people (emoji key):Zhining Liu💻📖💡Yuming Fu💻🐛Thúlio Costa💻🐛Neko Null🚧lirenjieArthur🐛AC手动机🐛Carlo Moro🤔This project follows theall-contributorsspecification. Contributions of any kind welcome!