Package: drfetools
dRFEtools - dynamic Recursive Feature EliminationdRFEtoolsis a package for dynamic recursive feature elimination with
sklearn.Authors: Apuã Paquola, Kynon Jade Benjamin, and Tarun KatipalliPackage developed in Python 3.8+.In addition to scikit-learn,dRFEtoolsis also built with NumPy, SciPy,
Pandas, matplotlib, plotnine, and statsmodels. Currently, dynamic RFE supports
models withcoef_orfeature_importances_attribute.This package has several function to run dynamic recursive feature elimination
(dRFE) for random forest and linear model classifier and regression models. For
random forest, it assumes Out-of-Bag (OOB) is set to True. For linear models,
it generates a developmental set. For both classification and regression, three
measurements are calculated for feature selection:Classification:Normalized mutual informationAccuracyArea under the curve (AUC) ROC curveRegression:R2 (this can be negative if model is arbitrarily worse)Explained varianceMean squared errorThe package has been split in to four additional scripts for:Out-of-bag dynamic RFE metrics (AP/KJB)Validation set dynamic RFE metrics (KJB)Rank features function (TK)Lowess core + peripheral selection (KJB)Table of ContentsCitationInstallationTutorialsReference ManualdRFEtools main functionsPeripheral features functionsPlotting functionsMetric functionsRandom forest helper functionsLinear model helper functionsCitationIf using please cite the following:
Pre-print DOI:https://doi.org/10.1101/2022.07.27.501227.Installationpip install --user dRFEtoolsTutorialsFollowthisjupyter notebook for an example on optimization.The GitHub below has example code for sklearn simulation, biological simulation, and using BrainSEQ Phase 1.https://github.com/LieberInstitute/dRFEtools_manuscriptReference ManualdRFEtools main functionsdRFE - Random Forestrf_rfeRuns random forest feature elimination step over iterator process.Args:estimator: Random forest classifier objectX: a data frame of training dataY: a vector of sample labels from training data setfeatures: a vector of feature namesfold: current foldout_dir: output directory. default '.'elimination_rate: percent rate to reduce feature list. default .2RANK: Output feature ranking. default=True (Boolean)Yields:dict: a dictionary with number of features, normalized mutual information score, accuracy score, and array of the indexes for features to keepdRFE - Linear Modelsdev_rfeRuns recursive feature elimination for linear model step over iterator
process assuming developmental set is needed.Args:estimator: regressor or classifier linear model objectX: a data frame of training dataY: a vector of sample labels from training data setfeatures: a vector of feature namesfold: current foldout_dir: output directory. default '.'elimination_rate: percent rate to reduce feature list. default .2dev_size: developmental set size. default '0.20'RANK: run feature ranking, default 'True'SEED: random state. default 'True'Yields:dict: a dictionary with number of features, r2 score, mean square error,
expalined variance, and array of the indices for features to keepFeature Rank Functionfeature_rank_fncThis function ranks features within the feature elimination loop.Args:features: A vector of feature namesrank: A vector with feature ranks based on absolute value of feature importancen_features_to_keep: Number of features to keep. (Int)fold: Fold to analyzed. (Int)out_dir: Output directory for text file. Default '.'RANK: Boolean (True or False)Yields:Text file: Ranked features by fold tab-delimited text file, only if RANK=TrueN Feature Iteratorn_features_iterDetermines the features to keep.Args:nf: current number of featureskeep_rate: percentage of features to keepYields:int: number of features to keepCalculate feature importancecal_feature_impGenerates feature importance from absolute value of feature weights.Args:estimator: the estimator to generate feature importance forYields:estimator: returns the estimator with feature importancePeripheral features functionsRun lowessrun_lowessThis function runs the lowess function and caches it to memory.Args:x: the x-values of the observed pointsy: the y-values of the observed pointsfrac: the fraction of the data used when estimating each y-value. default 3/10Yields:z: 2D array of resultsConvert array to tuplearray_to_tupleThis function attempts to convert a numpy array to a tuple.Args:np_array: numpy arrayYields:tupleExtract dRFE as a dataframeget_elim_df_orderedThis function converts the dRFE dictionary to a pandas dataframe.Args:d: dRFE dictionarymulti: is this for multiple classes. (True or False)Yields:df_elim: dRFE as a dataframe with log10 transformed featuresCalculate lowess curvecal_lowessThis function calculates the lowess curve.Args:d: dRFE dictionaryfrac: the fraction of the data used when estimating each y-valuemulti: is this for multiple classes. (True or False)Yields:x: dRFE log10 transformed featuresy: dRFE metricsz: 2D numpy array with lowess curvexnew: increased intervalsynew: interpolated metrics for xnewCalculate lowess curve for log10cal_lowessThis function calculates the rate of change on the lowess fitted curve with
log10 transformated input.Args:d: dRFE dictionaryfrac: the fraction of the data used when estimating each y-valuemulti: is this for multiple classes. default FalseYields:data frame: dataframe with n_features, lowess value, and rate of change (DxDy)Extract max lowessextract_max_lowessThis function extracts the max features based on rate of change of log10
transformed lowess fit curve.Args:d: dRFE dictionaryfrac: the fraction of the data used when estimating each y-value. default 3/10multi: is this for multiple classes. default FalseYields:int: number of max features (smallest subset)Extract peripheral lowessextract_peripheral_lowessThis function extracts the peripheral features based on rate of change of log10
transformed lowess fit curve.Args:d: dRFE dictionaryfrac: the fraction of the data used when estimating each y-value. default 3/10step_size: rate of change step size to analyze for extraction. default 0.05multi: is this for multiple classes. default FalseYields:int: number of peripheral featuresOptimize lowess plotplot_with_lowess_vlinePeripheral set selection optimization plot. This will be ROC AUC for multiple
classification (3+), NMI for binary classification, or R2 for regression. The
plot returned has fraction and step size as well as lowess smoothed curve and
indication of predicted peripheral set.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'frac: the fraction of the data used when estimating each y-value. default 3/10step_size: rate of change step size to analyze for extraction. default 0.05classify: is this a classification algorithm. default Truemulti: does this have multiple (3+) classes. default TrueYields:graph: plot of dRFE with estimated peripheral set indicated as well as fraction and set size used. It automatically saves files as pdf, png, and svgPlot lowess vlineplot_with_lowess_vlinePlot feature elimination results with the peripheral set indicated. This will be
ROC AUC for multiple classification (3+), NMI for binary classification, or R2
for regression.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'frac: the fraction of the data used when estimating each y-value. default 3/10step_size: rate of change step size to analyze for extraction. default 0.05classify: is this a classification algorithm. default Truemulti: does this have multiple (3+) classes. default TrueYields:graph: plot of dRFE with estimated peripheral set indicated, automatically saves files as pdf, png, and svgPlotting functionsSave plotssave_plotsThis function save plot as svg, png, and pdf with specific label and dimension.Args:p: plotnine objectfn: file name without extensionsw: width, default 7h: height, default 7Yields:SVG, PNG, and PDF of plotnine objectPlot dRFE Accuracyplot_accPlot feature elimination results for accuracy.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'Yields:graph: plot of feature by accuracy, automatically saves files as pdf, png, and svgPlot dRFE NMIplot_nmiPlot feature elimination results for normalized mutual information.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'Yields:graph: plot of feature by NMI, automatically saves files as pdf, png, and svgPlot dRFE ROC AUCplot_rocPlot feature elimination results for AUC ROC curve.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'Yields:graph: plot of feature by AUC, automatically saves files as pdf, png, and svgPlot dRFE R2plot_r2Plot feature elimination results for R2 score. Note that this can be negative
if model is arbitarily worse.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'Yields:graph: plot of feature by R2, automatically saves files as pdf, png, and svgPlot dRFE MSEplot_msePlot feature elimination results for mean squared error score.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'Yields:graph: plot of feature by mean squared error, automatically saves files as pdf, png, and svgPlot dRFE Explained Varianceplot_evarPlot feature elimination results for explained variance score.Args:d: feature elimination class dictionaryfold: current foldout_dir: output directory. default '.'Yields:graph: plot of feature by explained variance, automatically saves files as pdf, png, and svgMetric functionsOOB Predictionoob_predictionsExtracts out-of-bag (OOB) predictions from random forest classifier classes.Args:estimator: Random forest classifier objectYields:vector: OOB predicted labelsOOB Accuracy Scoreoob_score_accuracyCalculates the accuracy score from the OOB predictions.Args:estimator: Random forest classifier objectY: a vector of sample labels from training data setYields:float: accuracy scoreOOB Normalized Mutual Information Scoreoob_score_nmiCalculates the normalized mutual information score from the OOB predictions.Args:estimator: Random forest classifier objectY: a vector of sample labels from training data setYields:float: normalized mutual information scoreOOB Area Under ROC Curve Scoreoob_score_rocCalculates the area under the ROC curve score for the OOB predictions.Args:estimator: Random forest classifier objectY: a vector of sample labels from training data setYields:float: AUC ROC scoreOOB R2 Scoreoob_score_r2Calculates the r2 score from the OOB predictions.Args:estimator: Random forest regressor objectY: a vector of sample labels from training data setYields:float: r2 scoreOOB Mean Squared Error Scoreoob_score_mseCalculates the mean squared error score from the OOB predictions.Args:estimator: Random forest regressor objectY: a vector of sample labels from training data setYields:float: mean squared error scoreOOB Explained Variance Scoreoob_score_evarCalculates the explained variance score for the OOB predictions.Args:estimator: Random forest regressor objectY: a vector of sample labels from training data setYields:float: explained variance scoreDevelopmental Test Set Predictionsdev_predictionsExtracts predictions using a development fold for linear
regressor.Args:estimator: Linear model regression classifier objectX: a data frame of normalized values from developmental datasetYields:vector: Development set predicted labelsDevelopmental Test Set R2 Scoredev_score_r2Calculates the r2 score from the developmental dataset
predictions.Args:estimator: Linear model regressor objectX: a data frame of normalized values from developmental datasetY: a vector of sample labels from developmental datasetYields:float: r2 scoreDevelopmental Test Set Mean Squared Error Scoredev_score_mseCalculates the mean squared error score from the developmental dataset
predictions.Args:estimator: Linear model regressor objectX: a data frame of normalized values from developmental datasetY: a vector of sample labels from developmental datasetYields:float: mean squared error scoreDevelopmental Test Set Explained Variance Scoredev_score_evarCalculates the explained variance score for the develomental dataset predictions.Args:estimator: Linear model regressor objectX: a data frame of normalized values from developmental datasetY: a vector of sample labels from developmental data setYields:float: explained variance scoreDEV Accuracy Score`dev_score_accuracy`

Calculates the accuracy score from the DEV predictions.

**Args:**

-   estimator: Linear model classifier object
-   X: a data frame of normalized values from developmental dataset
-   Y: a vector of sample labels from training data set

**Yields:**

-   float: accuracy scoreDEV Normalized Mutual Information Score`dev_score_nmi`

Calculates the normalized mutual information score from the DEV predictions.

**Args:**

-   estimator: Linear model classifier object
-   X: a data frame of normalized values from developmental dataset
-   Y: a vector of sample labels from training data set

**Yields:**

-   float: normalized mutual information scoreDEV Area Under ROC Curve Score`dev_score_roc`

Calculates the area under the ROC curve score for the DEV predictions.

**Args:**

-   estimator: Linear model classifier object
-   X: a data frame of normalized values from developmental dataset
-   Y: a vector of sample labels from training data set

**Yields:**

-   float: AUC ROC scoreLinear model helper functionsdRFE Subfunctionregr_feIterate over features to by eliminated by step.Args:estimator: regressor or classifier linear model objectX: a data frame of training dataY: a vector of sample labels from training data setn_features_iter: iterator for number of features to keep loopfeatures: a vector of feature namesfold: current foldout_dir: output directory. default '.'dev_size: developmental test set propotion of trainingSEED: random stateRANK: Boolean (True or False)Yields:list: a list with number of features, r2 score, mean square error, expalined variance, and array of the indices for features to keepdRFE Step functionregr_fe_stepSplit training data into developmental dataset and apply estimator
to developmental dataset, rank features, and conduct feature
elimination, single steps.Args:estimator: regressor or classifier linear model objectX: a data frame of training dataY: a vector of sample labels from training data setn_features_to_keep: number of features to keepfeatures: a vector of feature namesfold: current foldout_dir: output directory. default '.'dev_size: developmental test set propotion of trainingSEED: random stateRANK: Boolean (True or False)Yields:dict: a dictionary with number of features, r2 score, mean square error, expalined variance, and selected features