Package: mafese
MAFESE (Metaheuristic Algorithms for FEature SElection) is the biggest python library for feature selection (FS)
problem using meta-heuristic algorithms.Free software:GNU General Public License (GPL) V3 licenseTotal Wrapper-based (Metaheuristic Algorithms): > 200 methodsTotal Filter-based (Statistical-based): > 15 methodsTotal Embedded-based (Tree and Lasso): > 10 methodsTotal Unsupervised-based: >= 4 methodsTotal datasets: >= 30 (47 classifications and 7 regressions)Total performance metrics: >= 61 (45 regressions and 16 classifications)Total objective functions (as fitness functions): >= 61 (45 regressions and 16 classifications)Documentation:https://mafese.readthedocs.io/en/latest/Python versions:>= 3.7.xDependencies:numpy, scipy, scikit-learn, pandas, mealpy, permetrics, plotly, kaleidoInstallationInstall thecurrent PyPI release:$pipinstallmafese==0.1.9Install directly from source code$gitclonehttps://github.com/thieu1995/mafese.git
$cdmafese
$pythonsetup.pyinstallIn case, you want to install the development version from Github:$pipinstallgit+https://github.com/thieu1995/mafeseAfter installation, you can import MAFESE as any other Python module:$python
>>>importmafese
>>>mafese.__version__Lib's structuredocs
examples
mafese
    data/
        cls/
            aggregation.csv
            Arrhythmia.csv
            ...
        reg/
            boston-housing.csv
            diabetes.csv
            ...
    wrapper/
        mha.py
        recursive.py
        sequential.py
    embedded/
        lasso.py
        tree.py
    filter.py
    unsupervised.py
    utils/
        correlation.py
        data_loader.py
        encoder.py
        estimator.py
        mealpy_util.py
        transfer.py
        validator.py
    __init__.py
    selector.py
README.md
setup.pyExamplesLet's go through some examples.1. First, load dataset. You can use the available datasets from Mafese:# Load available dataset from MAFESEfrommafeseimportget_dataset# Try unknown dataget_dataset("unknown")# Enter: 1      -> This wil list all of avaialble datasetdata=get_dataset("Arrhythmia")Or you can load your own datasetimportpandasaspdfrommafeseimportData# load X and y# NOTE mafese accepts numpy arrays only, hence the .values attributedataset=pd.read_csv('examples/dataset.csv',index_col=0).valuesX,y=dataset[:,0:-1],dataset[:,-1]data=Data(X,y)2. Next, split dataset into train and test setdata.split_train_test(test_size=0.2,inplace=True)print(data.X_train[:2].shape)print(data.y_train[:2].shape)You should confirm that your dataset is scaled and normalized for some problem or estimator such as Neural Network3. Next, choose the Selector that you want to use by first import them:## First way, we recommendedfrommafeseimportUnsupervisedSelector,FilterSelector,LassoSelector,TreeSelectorfrommafeseimportSequentialSelector,RecursiveSelector,MhaSelector,MultiMhaSelector## Second wayfrommafese.unsupervisedimportUnsupervisedSelectorfrommafese.filterimportFilterSelectorfrommafese.embedded.lassoimportLassoSelectorfrommafese.embedded.treeimportTreeSelectorfrommafese.wrapper.sequentialimportSequentialSelectorfrommafese.wrapper.recursiveimportRecursiveSelectorfrommafese.wrapper.mhaimportMhaSelector,MultiMhaSelector4. Next, create an instance of Selector class you want to use:feat_selector=UnsupervisedSelector(problem='classification',method='DR',n_features=5)feat_selector=FilterSelector(problem='classification',method='SPEARMAN',n_features=5)feat_selector=LassoSelector(problem="classification",estimator="lasso",estimator_paras={"alpha":0.1})feat_selector=TreeSelector(problem="classification",estimator="tree")feat_selector=SequentialSelector(problem="classification",estimator="knn",n_features=3,direction="forward")feat_selector=RecursiveSelector(problem="classification",estimator="rf",n_features=5)feat_selector=MhaSelector(problem="classification",estimator="knn",optimizer="BaseGA",optimizer_paras=None,transfer_func="vstf_01",obj_name="AS")list_optimizers=("OriginalWOA","OriginalGWO","OriginalTLO","OriginalGSKA")list_paras=[{"epoch":10,"pop_size":30},]*4feat_selector=MultiMhaSelector(problem="classification",estimator="knn",list_optimizers=list_optimizers,list_optimizer_paras=list_paras,transfer_func="vstf_01",obj_name="AS")5. Fit the model to X_train and y_trainfeat_selector.fit(data.X_train,data.y_train)6. Get the information# check selected features - True (or 1) is selected, False (or 0) is not selectedprint(feat_selector.selected_feature_masks)print(feat_selector.selected_feature_solution)# check the index of selected featuresprint(feat_selector.selected_feature_indexes)7. Call transform() on the X that you want to filter it down to selected featuresX_train_selected=feat_selector.transform(data.X_train)X_test_selected=feat_selector.transform(data.X_test)8.You can build your own evaluating method or use our method.If you use our method, don't transform the data.i) You can use difference estimator than the one used in feature selection processfeat_selector.evaluate(estimator="svm",data=data,metrics=["AS","PS","RS"])## Here, we pass the data that was loaded above. So it contains both train and test set. So, the results will looklikethis:{'AS_train':0.77176,'PS_train':0.54177,'RS_train':0.6205,'AS_test':0.72636,'PS_test':0.34628,'RS_test':0.52747}ii) You can use the same estimator in feature selection processX_test,y_test=data.X_test,data.y_testfeat_selector.evaluate(estimator=None,data=data,metrics=["AS","PS","RS"])Where do I find the supported metrics like above ["AS", "PS", "RS"]. What is that?
You can find it here:https://github.com/thieu1995/permetricsor use thisfrommafeseimportMhaSelectorprint(MhaSelector.SUPPORTED_REGRESSION_METRICS)print(MhaSelector.SUPPORTED_CLASSIFICATION_METRICS)How do I know my Selector support which estimator? which methods?print(feat_selector.SUPPORT)Or you better read the document from:https://mafese.readthedocs.io/en/latest/I got this type of errorraiseValueError("Existed at least one new label in y_pred.")ValueError:Existedatleastonenewlabeliny_pred.How to solve this?This occurs only when you are working on a classification problem with a small dataset that has many classes. For
instance, the "Zoo" dataset contains only 101 samples, but it has 7 classes. If you split the dataset into a
training and testing set with a ratio of around 80% - 20%, there is a chance that one or more classes may appear
in the testing set but not in the training set. As a result, when you calculate the performance metrics, you may
encounter this error. You cannot predict or assign new data to a new label because you have no knowledge about the
new label. There are several solutions to this problem.1st: Use the SMOTE method to address imbalanced data and ensure that all classes have the same number of samples.fromimblearn.over_samplingimportSMOTEimportpandasaspdfrommafeseimportDatadataset=pd.read_csv('examples/dataset.csv',index_col=0).valuesX,y=dataset[:,0:-1],dataset[:,-1]X_new,y_new=SMOTE().fit_resample(X,y)data=Data(X_new,y_new)2nd: Use different random_state numbers in split_train_test() function.importpandasaspdfrommafeseimportDatadataset=pd.read_csv('examples/dataset.csv',index_col=0).valuesX,y=dataset[:,0:-1],dataset[:,-1]data=Data(X,y)data.split_train_test(test_size=0.2,random_state=10)# Try different random_state valueFor more usage examples please look atexamplesfolder.Support (questions, problems)Official LinksOfficial source code repo:https://github.com/thieu1995/mafeseOfficial document:https://mafese.readthedocs.io/Download releases:https://pypi.org/project/mafese/Issue tracker:https://github.com/thieu1995/mafese/issuesNotable changes log:https://github.com/thieu1995/mafese/blob/master/ChangeLog.mdExamples with different mealpy version:https://github.com/thieu1995/mafese/blob/master/examples.mdOfficial chat group:https://t.me/+fRVCJGuGJg1mNDg1This project also related to our another projects which are "optimization" and "machine learning", check it here:https://github.com/thieu1995/mealpyhttps://github.com/thieu1995/metaheuristicshttps://github.com/thieu1995/opfunuhttps://github.com/thieu1995/enoppyhttps://github.com/thieu1995/permetricshttps://github.com/thieu1995/MetaClusterhttps://github.com/thieu1995/pfevaluatorhttps://github.com/aiir-teamCitation RequestPlease include these citations if you plan to use this library:@software{nguyen_van_thieu_2023_7969043,
  author       = {Nguyen Van Thieu, Ngoc Hung Nguyen, Ali Asghar Heidari},
  title        = {Feature Selection using Metaheuristics Made Easy: Open Source MAFESE Library in Python},
  month        = may,
  year         = 2023,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.7969042},
  url          = {https://github.com/thieu1995/mafese}
}

@article{van2023mealpy,
  title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python},
  author={Van Thieu, Nguyen and Mirjalili, Seyedali},
  journal={Journal of Systems Architecture},
  year={2023},
  publisher={Elsevier},
  doi={10.1016/j.sysarc.2023.102871}
}Related Documentshttps://neptune.ai/blog/feature-selection-methodshttps://www.blog.trainindata.com/feature-selection-machine-learning-with-python/https://github.com/LBBSoft/FeatureSelecthttps://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-2754-0https://github.com/scikit-learn-contrib/boruta_pyhttps://elki-project.github.io/https://sci2s.ugr.es/keel/index.phphttps://archive.ics.uci.edu/datasetshttps://python-charts.com/distribution/box-plot-plotly/https://plotly.com/python/box-plots/?_ga=2.50659434.2126348639.1688086416-114197406.1688086416#box-plot-styling-mean--standard-deviation