Package: irf
iterative Random ForestThe algorithm details are available at:Sumanta Basu, Karl Kumbier, James B. Brown, Bin Yu,  Iterative Random Forests to detect predictive and stable high-order interactions, PNAShttps://www.pnas.org/content/115/8/1943The implementation is a joint effort of several people in UC Berkeley. See theAuthors.mdfor the complete list.
The weighted random forest implementation is based on the random forest source code and API design fromscikit-learn, details can be found inAPI design for machine learning software: experiences from the scikit-learn project, Buitinck et al., 2013.. The setup file is based on the setup file fromskgarden.InstallationDependenciesThe irf package requiresPython (>= 3.3)Numpy (>= 1.8.2)Scipy (>= 0.13.3)Cythonpydotplusmatplotlibjupyterpyyamlscikit-learn (>= 0.22)Before the installation, please make sure you installed the above python packages correctly via pip:pipinstallcythonnumpyscikit-learnpydotplusjupyterpyyamlmatplotlibBasic setup and installationInstalling irf package is simple. Just clone this repo and use pip install.gitclonehttps://github.com/Yu-Group/iterative-Random-ForestThen go to theiterative-Random-Forestfolder and use pip install:pipinstall-e.If irf is installed successfully, you should be able to see it using pip list:piplist|grepirfand you should be able to run all the tests (assume the working directory is in the package iterative-Random-Forest):pythonirf/tests/test_irf_utils.py
pythonirf/tests/test_irf_weighted.pyA simple demoIn order to use irf, you need to import it in python.importnumpyasnpfromirfimportirf_utilsfromirf.ensembleimportRandomForestClassifierWithWeightsGenerate a simple data set with 2 features: 1st feature is a noise feature that has no power in predicting the labels, the 2nd feature determines the label perfectly:n_samples=1000n_features=10X_train=np.random.uniform(low=0,high=1,size=(n_samples,n_features))y_train=np.random.choice([0,1],size=(n_samples,),p=[.5,.5])X_test=np.random.uniform(low=0,high=1,size=(n_samples,n_features))y_test=np.random.choice([0,1],size=(n_samples,),p=[.5,.5])# The second feature (which is indexed by 1) is very importantX_train[:,1]=X_train[:,1]+y_trainX_test[:,1]=X_test[:,1]+y_testThen run irfall_rf_weights, all_K_iter_rf_data, \
    all_rf_bootstrap_output, all_rit_bootstrap_output, \
    stability_score = irf_utils.run_iRF(X_train=X_train,
                                        X_test=X_test,
                                        y_train=y_train,
                                        y_test=y_test,
                                        K=5,                          # number of iteration
                                        rf = RandomForestClassifierWithWeights(n_estimators=20),
                                        B=30,
                                        random_state_classifier=2018, # random seed
                                        propn_n_samples=.2,
                                        bin_class_type=1,
                                        M=20,
                                        max_depth=5,
                                        noisy_split=False,
                                        num_splits=2,
                                        n_estimators_bootstrap=5)all_rf_weights stores all the weights for each iteration:print(all_rf_weights['rf_weight5'])The proposed feature combination and their scores:print(stability_score)