Package: imbalance-metrics
ImbalanceMetricsDescriptionThe "ImbalanceMetrics" Python package provides an extensive set of evaluation metrics designed for assessing the performance of machine learning models on imbalanced datasets, where traditional accuracy and error rate measurements may be inadequate. The package incorporates several evaluation metrics that tackle the challenges specific to imbalanced domains, offering a more accurate evaluation of model performance.RequirementsPython 3scikit-learnNumPyPandasSmognInstallation## install pypi releasepipinstallimbalance-metrics## install developer versionpipinstallgit+https://github.com/paobranco/ImbalanceMetrics.gitUsage (Classification)importpandasaspdfromsklearn.ensembleimportRandomForestClassifierfromsklearn.model_selectionimporttrain_test_splitfromimbalance_metricsimportclassification_metricsascmdf=pd.read_csv('glass0.csv',header=None)X,y=df.drop(columns=[9]),df[9]X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.40)clf=RandomForestClassifier(random_state=42)clf.fit(X_train,y_train)y_pred=clf.predict(X_test)y_proba=clf.predict_proba(X_test)y_phi=cm.calculate_classification_phi(y_test)# Relevance by class frequencygmean=cm.gmean_score(y_test,y_pred)# Weighted gmeanp_d0,r_d0,pra_d0=cm.pr_davis(y_test,y_proba,True)# Default minority as positivep_d1,r_d1,pra_d1=cm.pr_davis(y_test,y_proba,True,pos_label=1)# 1 as positivep_m0,r_m0,pra_m0=cm.pr_manning(y_test,y_proba,True)# Default minority as positivep_m1,r_m1,pra_m1=cm.pr_manning(y_test,y_proba,True,pos_label=1)# 1 as positivecv_davis=cm.cross_validate_auc(clf,X,y,cm.pr_davis,5)cv_manning=cm.cross_validate_auc(clf,X,y,cm.pr_manning,5)Usage (Regression)importpandasaspdfromsklearn.svmimportSVRfromsklearn.model_selectionimporttrain_test_splitfromsklearn.preprocessingimportStandardScalerfromimbalance_metricsimportregression_metricsasrmdf=pd.read_csv('housing(processed).csv')X,y=df.drop(columns="SalePrice"),df["SalePrice"]X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.33)sc=StandardScaler()y_train=sc.fit_transform(y_train.values.reshape(-1,1))y_test=sc.transform(y_test.values.reshape(-1,1))reg=SVR().fit(X_train,y_train)y_pred=reg.predict(X_test)y_test=y_test.reshape(-1)y_pred=y_pred.reshape(-1)wmse=rm.phi_weighted_mse(y_test,y_pred)wmae=rm.phi_weighted_mae(y_test,y_pred)wr2=rm.phi_weighted_r2(y_test,y_pred)wrmse=rm.phi_weighted_root_mse(y_test,y_pred)ser_t=rm.ser_t(y_test,y_pred,t=.7)sera=rm.sera(y_test,y_pred,return_err=True)ContributionsImablanceMetrics is open for improvements and maintenance. Your help is valued to make the package better for everyone.LicenseLicensed under the General Public License v3.0 (GPLv3).ReferenceRibeiro, R.P.: Utility-based regression. Ph.D. thesis, Dep. Computer Science, Faculty of Sciences - University of Porto (2011)Branco, P., Ribeiro, R.P., Torgo, L.: UBL: an R package for utility-based learning (2016),https://arxiv.org/abs/1604.08079Branco, P., Torgo, L., Ribeiro, R.: A survey of predictive modelling under imbalanced distributions (2015)Branco, P., Torgo, L., Ribeiro, R.P.: A survey of predictive modeling on imbalanced domains. ACM Computing Surveys (CSUR) 49(2), 1â€“50 (2016)Branco, P., Torgo, L., Ribeiro, R.P.: SMOGN: a pre-processing approach for imbalanced regression. In: First international workshop on learning with imbalanced domains: Theory and applications. pp. 36â€“50. PMLR (2017)CordÃ³n, I., GarcÃ­a, S., FernÃ¡ndez, A., Herrera, F.: Imbalance: Oversampling algorithms for imbalanced classification in r. Knowledge-Based Systems 161, 329â€“341 (2018),https://doi.org/10.1016/j.knosys.2018.07.035Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. vol. 06 (06 2006).https://doi.org/10.1145/1143844.1143874Derrac, J., Garcia, S., Sanchez, L., Herrera, F.: Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J. Mult. Valued Logic Soft Comput 17 (2015)Gaudreault, J.G., Branco, P., Gama, J.: An analysis of performance metrics for imbalanced classification. In: Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11â€“13, 2021, Proceedings. pp. 67â€“77 (2021)Kubat, M., Matwin, S., et al.: Addressing the curse of imbalanced training sets: one-sided selection. In: Icml. vol. 97, p. 179. Citeseer (1997)Kunz, N.: SMOGN: Synthetic minority over-sampling technique for regression with gaussian noise (2020),https://pypi.org/project/smognLemaÃ®tre, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. JMLR 18(17), 1â€“5 (2017),http://jmlr.org/papers/v18/16-365.htmlRibeiro, R.: Utility-based Regression. Ph.D. thesis, Dep. Computer Science, Faculty of Sciences - University of Porto (2011)Ribeiro, R., Moniz, N.: Imbalanced regression and extreme value prediction. Machine Learning 109,1â€“33 (09 2020).https://doi.org/10.1007/s10994-020-05900-9berreergun: Ironpy.https://github.com/berreergun/IRonPy(2021)