Package: zoofs
🐾 zoofs ( Zoo Feature Selection )zoofsis a Python library for performing feature selection using a variety of nature inspired wrapper algorithms. The algorithms range from swarm-intelligence to physics based to Evolutionary.
It's an easy to use, flexible and powerful tool to reduce your feature size.🌟 Like this Project? Give us a star !📘 Documentationhttps://jaswinder9051998.github.io/zoofs/🔗 Whats new in V0.1.24pass kwargs through objective functionimproved logger for resultsadded harris hawk algorithmnow you can passtimeoutas a parameter to stop operation after the given number of second(s). An amazing alternative to passing number of iterationsFeature score hashing of visited feature sets to increase the overall performance🛠 InstallationUsing pipUse the package manager to install zoofs.pipinstallzoofs📜 Available AlgorithmsAlgorithm NameClass NameDescriptionReferences doiParticle Swarm AlgorithmParticleSwarmOptimizationUtilizes swarm behaviourhttps://doi.org/10.1007/978-3-319-13563-2_51Grey Wolf AlgorithmGreyWolfOptimizationUtilizes wolf hunting behaviourhttps://doi.org/10.1016/j.neucom.2015.06.083Dragon Fly AlgorithmDragonFlyOptimizationUtilizes dragonfly swarm behaviourhttps://doi.org/10.1016/j.knosys.2020.106131Harris Hawk AlgorithmHarrisHawkOptimizationUtilizes hawk hunting behaviourhttps://link.springer.com/chapter/10.1007/978-981-32-9990-0_12Genetic Algorithm AlgorithmGeneticOptimizationUtilizes genetic mutation behaviourhttps://doi.org/10.1109/ICDAR.2001.953980Gravitational AlgorithmGravitationalOptimizationUtilizes newtons gravitational behaviourhttps://doi.org/10.1109/ICASSP.2011.5946916More algos soon, stay tuned ![Try It Now?]⚡️ UsageDefine your own objective function for optimization !Classification Examplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportParticleSwarmOptimization# create object of algorithmalgo_object=ParticleSwarmOptimization(objective_function_topass,n_iteration=20,population_size=20,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Regression Examplefromsklearn.metricsimportmean_squared_error# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=mean_squared_error(y_valid,model.predict(X_valid))returnP# import an algorithm !fromzoofsimportParticleSwarmOptimization# create object of algorithmalgo_object=ParticleSwarmOptimization(objective_function_topass,n_iteration=20,population_size=20,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMRegressor()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Suggestions for UsageAs available algorithms are wrapper algos, it is better to use ml models that build quicker, e.g lightgbm, catboost.Take sufficient amount for 'population_size' , as this will determine the extent of exploration and exploitation of the algo.Ensure that your ml model has its hyperparamters optimized before passing it to zoofs algos.objective score plotAlgorithmsParticle Swarm AlgorithmIn computational science, particle swarm optimization (PSO) is a computational method that optimizes a problem by iteratively trying to improve a candidate solution with regard to a given measure of quality. It solves a problem by having a population of candidate solutions, here dubbed particles, and moving these particles around in the search-space according to simple mathematical formula over the particle's position and velocity. Each particle's movement is influenced by its local best known position, but is also guided toward the best known positions in the search-space, which are updated as better positions are found by other particles. This is expected to move the swarm toward the best solutions.class zoofs.ParticleSwarmOptimization(objective_function,n_iteration=50,population_size=50,minimize=True,c1=2,c2=2,w=0.9)Parametersobjective_function:  user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.The function must return a value, that needs to be minimized/maximized.n_iteration: int, default=1000Number of time the algorithm will runtimeout: int = NoneStop operation after the given number of second(s). If this argument is set to None, the operation is executed without time limitation and n_iteration is followedpopulation_size: int, default=50Total size of the populationminimize: bool, default=TrueDefines if the objective value is to be maximized or minimizedc1: float, default=2.0first acceleration coefficient of particle swarmc2: float, default=2.0second acceleration coefficient of particle swarmw: float, default=0.9weight parameterAttributesbest_feature_list:  array-likeFinal best set of featuresMethodsMethodsClass NamefitRun the algorithmplot_historyPlot results achieved across iterationfit(model,X_train, y_train, X_test, y_test,verbose=True)Parametersmodel:machine learning model's objectX_train: pandas.core.frame.DataFrame of shape (n_samples, n_features)Training input samples to be used for machine learning modely_train: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The target values (class labels in classification, real numbers in regression).X_valid: pandas.core.frame.DataFrame of shape (n_samples, n_features)Validation input samplesy_valid: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The Validation target values .verbose: bool,default=TruePrint results for iterationsReturnsbest_feature_list:  array-likeFinal best set of featuresplot_history()Plot results across iterationsExamplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportParticleSwarmOptimization# create object of algorithmalgo_object=ParticleSwarmOptimization(objective_function_topass,n_iteration=20,population_size=20,minimize=True,c1=2,c2=2,w=0.9)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Grey Wolf AlgorithmThe Grey Wolf Optimizer (GWO) mimics the leadership hierarchy and hunting mechanism of grey wolves in nature. Four types of grey wolves such as alpha, beta, delta, and omega are employed for simulating the leadership hierarchy. In addition, three main steps of hunting, searching for prey, encircling prey, and attacking prey, are implemented to perform optimization.class zoofs.GreyWolfOptimization(objective_function,n_iteration=50,population_size=50,minimize=True)Parametersobjective_function:  user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.The function must return a value, that needs to be minimized/maximized.n_iteration: int, default=50Number of time the algorithm will runtimeout: int = NoneStop operation after the given number of second(s). If this argument is set to None, the operation is executed without time limitation and n_iteration is followedpopulation_size: int, default=50Total size of the populationmethod: {1, 2}, default=1Choose the between the two methods of grey wolf optimizationminimize: bool, default=TrueDefines if the objective value is to be maximized or minimizedAttributesbest_feature_list:  array-likeFinal best set of featuresMethodsMethodsClass NamefitRun the algorithmplot_historyPlot results achieved across iterationfit(model,X_train,y_train,X_valid,y_valid,method=1,verbose=True)Parametersmodel:machine learning model's objectX_train: pandas.core.frame.DataFrame of shape (n_samples, n_features)Training input samples to be used for machine learning modely_train: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The target values (class labels in classification, real numbers in regression).X_valid: pandas.core.frame.DataFrame of shape (n_samples, n_features)Validation input samplesy_valid: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The Validation target values .verbose: bool,default=TruePrint results for iterationsReturnsbest_feature_list:  array-likeFinal best set of featuresplot_history()Plot results across iterationsExamplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportGreyWolfOptimization# create object of algorithmalgo_object=GreyWolfOptimization(objective_function_topass,n_iteration=20,method=1,population_size=20,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Dragon Fly AlgorithmThe main inspiration of the Dragonfly Algorithm (DA) algorithm originates from static and dynamic swarming behaviours. These two swarming behaviours are very similar to the two main phases of optimization using meta-heuristics: exploration and exploitation. Dragonflies create sub swarms and fly over different areas in a static swarm, which is the main objective of the exploration phase. In the static swarm, however, dragonflies fly in bigger swarms and along one direction, which is favourable in the exploitation phase.class zoofs.DragonFlyOptimization(objective_function,n_iteration=50,population_size=50,minimize=True)Parametersobjective_function:  user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.The function must return a value, that needs to be minimized/maximized.n_iteration: int, default=50Number of time the algorithm will runtimeout: int = NoneStop operation after the given number of second(s). If this argument is set to None, the operation is executed without time limitation and n_iteration is followedpopulation_size: int, default=50Total size of the populationmethod: {'linear','random','quadraic','sinusoidal'}, default='sinusoidal'Choose the between the three methods of Dragon Fly optimizationminimize: bool, default=TrueDefines if the objective value is to be maximized or minimizedAttributesbest_feature_list:  array-likeFinal best set of featuresMethodsMethodsClass NamefitRun the algorithmplot_historyPlot results achieved across iterationfit(model,X_train,y_train,X_valid,y_valid,method='sinusoidal',verbose=True)Parametersmodel:machine learning model's objectX_train: pandas.core.frame.DataFrame of shape (n_samples, n_features)Training input samples to be used for machine learning modely_train: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The target values (class labels in classification, real numbers in regression).X_valid: pandas.core.frame.DataFrame of shape (n_samples, n_features)Validation input samplesy_valid: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The Validation target values .verbose: bool,default=TruePrint results for iterationsReturnsbest_feature_list:  array-likeFinal best set of featuresplot_history()Plot results across iterationsExamplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportDragonFlyOptimization# create object of algorithmalgo_object=DragonFlyOptimization(objective_function_topass,n_iteration=20,method='sinusoidal',population_size=20,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Harris Hawk OptimizationHHO is a popular swarm-based, gradient-free optimization algorithm with several active and time-varying phases of exploration and exploitation. This algorithm initially published by the prestigious Journal of Future Generation Computer Systems (FGCS) in 2019, and from the first day, it has gained increasing attention among researchers due to its flexible structure, high performance, and high-quality results. The main logic of the HHO method is designed based on the cooperative behaviour and chasing styles of Harris' hawks in nature called "surprise pounce". Currently, there are many suggestions about how to enhance the functionality of HHO, and there are also several enhanced variants of the HHO in the leading Elsevier and IEEE transaction journals.class zoofs.HarrisHawkOptimization(objective_function,n_iteration=50,population_size=50,minimize=True,beta=0.5)Parametersobjective_function:  user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.The function must return a value, that needs to be minimized/maximized.n_iteration: int, default=1000Number of time the algorithm will runtimeout: int = NoneStop operation after the given number of second(s). If this argument is set to None, the operation is executed without time limitation and n_iteration is followedpopulation_size: int, default=50Total size of the populationminimize: bool, default=TrueDefines if the objective value is to be maximized or minimizedbeta: float, default=0.5value for levy random walkAttributesbest_feature_list:  array-likeFinal best set of featuresMethodsMethodsClass NamefitRun the algorithmplot_historyPlot results achieved across iterationfit(model,X_train, y_train, X_test, y_test,verbose=True)Parametersmodel:machine learning model's objectX_train: pandas.core.frame.DataFrame of shape (n_samples, n_features)Training input samples to be used for machine learning modely_train: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The target values (class labels in classification, real numbers in regression).X_valid: pandas.core.frame.DataFrame of shape (n_samples, n_features)Validation input samplesy_valid: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The Validation target values .verbose: bool,default=TruePrint results for iterationsReturnsbest_feature_list:  array-likeFinal best set of featuresplot_history()Plot results across iterationsExamplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportHarrisHawkOptimization# create object of algorithmalgo_object=HarrisHawkOptimization(objective_function_topass,n_iteration=20,population_size=20,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Genetic AlgorithmIn computer science and operations research, a genetic algorithm (GA) is a metaheuristic inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms (EA). Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems by relying on biologically inspired operators such as mutation, crossover and selection. Some examples of GA applications include optimizing decision trees for better performance, automatically solve sudoku puzzles, hyperparameter optimization, etc.class zoofs.GeneticOptimization(objective_function,n_iteration=20,population_size=20,selective_pressure=2,elitism=2,mutation_rate=0.05,minimize=True)Parametersobjective_function:  user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.The function must return a value, that needs to be minimized/maximized.n_iteration: int, default=50Number of time the algorithm will runtimeout: int = NoneStop operation after the given number of second(s). If this argument is set to None, the operation is executed without time limitation and n_iteration is followedpopulation_size: int, default=50Total size of the populationselective_pressure: int, default=2measure of reproductive opportunities for each organism in the populationelitism: int, default=2number of top individuals to be considered as elitesmutation_rate: float, default=0.05rate of mutation in the population's geneminimize: bool, default=TrueDefines if the objective value is to be maximized or minimizedAttributesbest_feature_list:  array-likeFinal best set of featuresMethodsMethodsClass NamefitRun the algorithmplot_historyPlot results achieved across iterationfit(model,X_train,y_train,X_valid,y_valid,verbose=True)Parametersmodel:machine learning model's objectX_train: pandas.core.frame.DataFrame of shape (n_samples, n_features)Training input samples to be used for machine learning modely_train: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The target values (class labels in classification, real numbers in regression).X_valid: pandas.core.frame.DataFrame of shape (n_samples, n_features)Validation input samplesy_valid: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The Validation target values .verbose: bool,default=TruePrint results for iterationsReturnsbest_feature_list:  array-likeFinal best set of featuresplot_history()Plot results across iterationsExamplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportGeneticOptimization# create object of algorithmalgo_object=GeneticOptimization(objective_function_topass,n_iteration=20,population_size=20,selective_pressure=2,elitism=2,mutation_rate=0.05,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()Gravitational AlgorithmGravitational Algorithm is based on the law of gravity and mass interactions is introduced. In the algorithm, the searcher agents are a collection of masses which interact with each other based on the Newtonian gravity and the laws of motion.class zoofs.GravitationalOptimization(self,objective_function,n_iteration=50,population_size=50,g0=100,eps=0.5,minimize=True)Parametersobjective_function:  user made function of the signature 'func(model,X_train,y_train,X_test,y_test)'.The function must return a value, that needs to be minimized/maximized.n_iteration: int, default=50Number of time the algorithm will runtimeout: int = NoneStop operation after the given number of second(s). If this argument is set to None, the operation is executed without time limitation and n_iteration is followedpopulation_size: int, default=50Total size of the populationg0: float, default=100gravitational strength constanteps: float, default=0.5distance constantminimize: bool, default=TrueDefines if the objective value is to be maximized or minimizedAttributesbest_feature_list:  array-likeFinal best set of featuresMethodsMethodsClass NamefitRun the algorithmplot_historyPlot results achieved across iterationfit(model,X_train,y_train,X_valid,y_valid,verbose=True)Parametersmodel:machine learning model's objectX_train: pandas.core.frame.DataFrame of shape (n_samples, n_features)Training input samples to be used for machine learning modely_train: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The target values (class labels in classification, real numbers in regression).X_valid: pandas.core.frame.DataFrame of shape (n_samples, n_features)Validation input samplesy_valid: pandas.core.frame.DataFrame or pandas.core.series.Series of shape (n_samples)The Validation target values .verbose: bool,default=TruePrint results for iterationsReturnsbest_feature_list:  array-likeFinal best set of featuresplot_history()Plot results across iterationsExamplefromsklearn.metricsimportlog_loss# define your own objective function, make sure the function receives four parameters,#  fit your model and return the objective value !defobjective_function_topass(model,X_train,y_train,X_valid,y_valid):model.fit(X_train,y_train)P=log_loss(y_valid,model.predict_proba(X_valid))returnP# import an algorithm !fromzoofsimportGravitationalOptimization# create object of algorithmalgo_object=GravitationalOptimization(objective_function_topass,n_iteration=50,population_size=50,g0=100,eps=0.5,minimize=True)importlightgbmaslgblgb_model=lgb.LGBMClassifier()# fit the algorithmalgo_object.fit(lgb_model,X_train,y_train,X_valid,y_valid,verbose=True)#plot your resultsalgo_object.plot_history()SupportzoofsThe development ofzoofsrelies completely on contributions.ContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.Please make sure to update tests as appropriate.First roll out18,08,2021Licenseapache-2.0