Package: expectation-reflection
Expectation Reflection (ER) is a multiplicative optimization method that trains the interaction weights from features to target according to the ratio of target observations to their corresponding model expectations. This approach completely separates model updates from minimization of a cost function measuring goodness of fit, so that it can take the cost function as an effective stopping criterion of the iteration.Advantages of this method:
(1) working relatively well even in the regime of small sample sizes;
(2) using only one hyper-parameter;
(3) being able to demonstrate the system mechanism.In the current version, ERclassificationcan work as a classifier (for both binary and multinomial tasks). The extension toregressionwill be appeared shortly.InstallationFrom PyPIpipinstallexpectation-reflectionFrom Repositorygitclonehttps://github.com/danhtaihoang/expectation-reflection.gitUsageThe implementation of ER is very similar to that of other classifiers insklearn, bassically it consists of the following steps.Import theexpectation_reflectionpackage into your python script:fromexpectation_reflectionimportclassificationasERSelect a model:model=ER.model(max_iter=100,reg=0.01,random_state=1)Import yourdataset.txtinto python script.Xy=np.loadtxt('dataset.txt')Select the features and target from the dataset. If the target is the last column thenX,y=Xy[:,:-1],Xy[:,-1]Importtrain_test_splitfromsklearnto split data into training and test sets:fromsklearn.model_selectionimporttrain_test_splitX_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.5,random_state=1)Train the model with(X_train, y_train)set:model.fit(X_train,y_train)Predict the output classy_predand its probabilityp_predof a new datasetX_test:y_pred=model.predict(X_test)print('predicted output:',y_pred)p_pred=model.predict_proba(X_test)print('predicted probability:',p_pred)Intercept and interaction weights:print('intercept:',model.intercept_)print('interaction weights:',model.coef_)Hyper-Parameter OptimizationER has only one hyper-parameter,reg, which can be optimized by usingGridSearchCVfromsklearn:fromsklearn.model_selectionimportGridSearchCVmodel=ER.model(max_iter=100,random_state=1)reg=[0.0001,0.001,0.01,0.1,0.5,1.]hyper_parameters=dict(reg=reg)clf=GridSearchCV(model,hyper_parameters,cv=4,n_jobs=-1,iid='deprecated')best_model=clf.fit(X_train,y_train)Best hyper-parameters:print('best_hyper_parameters:',best_model.best_params_)Predict the outputy_predand its probabilityp_pred:y_pred=best_model.best_estimator_.predict(X_test)print('predicted output:',y_pred)p_pred=best_model.best_estimator_.predict_proba(X_test)print('predicted probability:',p_pred)Performance EvaluationWe can measure the model performance by usingmetricsfromsklearn:fromsklearn.metricsimportaccuracy_score,precision_score,recall_score,f1_score,\roc_auc_score,roc_curve,aucacc=accuracy_score(y_test,y_pred)print('accuracy:',acc)precision=precision_score(y_test,y_pred)print('precision:',precision)recall=recall_score(y_test,y_pred)print('recall:',recall)f1score=f1_score(y_test,y_pred)print('f1score:',f1score)roc_auc=roc_auc_score(y_test,p_pred)## note: it is p_pred, not y_predprint('roc auc:',roc_auc)ROC AUC can be also calculated asfp,tp,thresholds=roc_curve(y_test,p_pred,drop_intermediate=False)roc_auc=auc(fp,tp)print('roc auc:',roc_auc)CitationPlease cite the following papers if you use this package in your work:Danh-Tai Hoang, Juyong Song, Vipul Periwal, and Junghyo Jo, Network inference in stochastic systems from neurons to currencies: Improved performance at small sample size, Physical Review E, 99, 023311 (2019)Danh-Tai Hoang, Junghyo Jo, and Vipul Periwal, Data-driven inference of hidden nodes in networks, Physical Review E, 99, 042114 (2019)