Package: evotoolbox
Evolutionary feature selection toolboxThis package contains a set of tool to easily apply evolutionary feature selection techniques to your datasets.InstallationQuick StartInitializersFitness FunctionsBinary TransformersOptimizersCreditsInstallationYou can install the package using the following command.pip install evotoolboxQuick startThis package is based on 4 essential building blocks:initializer,optimizer,fitness function, andbinary transformer. To find the best features using an evolutionary algorithm you must choose one of the provided classes within this package or you can easily define your own custom classes.The following sample code shows how to use the defaultRandom Initializer,GWO Optimizer,MultiObjective Fitness, andThreshold Transformerto do a simple feature selection task.importpandasaspdimportevotoolboxfromevotoolbox.initializersimportRandomInitializerfromevotoolbox.fitnessimportMultiobjectiveFitnessfromevotoolbox.binaryimportThresholdTransformerfromevotoolbox.optimizersimportGWO# Load your datadata=pd.read_csv('colon.csv').to_numpy()X=data[:,:-1]Y=data[:,-1]n_features=X.shape[1]# Define the algorithm optionsinitializer=RandomInitializer(n_features=n_features,n_agents=10)optimizer=GWO(ThresholdTransformer(0.5),max_iter=100,lb=0,ub=1)fitness=MultiobjectiveFitness(alpha=0.99)# Fit the data using the provided optionsresult=evotoolbox.fit(X,Y,initializer,optimizer,fitness)# The result is a dictionary with three keys:# solution: the binary solution with shape (n_features,) where the selected features are 1 and others are 0# c: the convergence curve of the fitness# nf: number of selected features in the final solutionprint(result['solution'])InitializersProvided InitializersThis package provides three default initializer classes. These classes can be imported fromevotoolbox.initializers. All of the initializers must receive the number of features and the number of agents as arguments.RandomInitializerThe RandomInitializer creates a random initial population.GreedyInitializerThe GreedyInitializer creates a random population and the tries to enhance the solution based on a greedy algorithm. This is a two step process. In the first pass the greedy algorithm checks if the fitness increases by setting zeros to ones in the solution one by one. If the solution fitness gets better that feature will be selected. In the second pass the algorithm tries to drop the features using the same logic.
This initializer usually generates high quality solutions with small number of features. The downside is that the initialization time increases linearly with the number of features and it can take hours to complete on very large datasets.OblInitializerThe RandomInitializer creates a random initial population and then also generates the complement of these solutions and compare each of these pairs and keeps the better solution in each pair and discards the other.Defining your own initializerYou can easily define your own initializer by extending theevotoolbox.initializers.BaseInitializerclass.
You can access the number of features and agents in this class. You also receive a fitness function to evaluate your solutions in the initialization process.importnumpyasnpfromevotoolbox.initializersimportBaseInitializerclassMyCustomInitializer(BaseInitializer):definit(self,fitness_func):# create a numpy array of the initial populationpositions=np.zeros((self.n_agents,self.n_features))# do your magic here!# you can evaluate and compare the solutions using fitness_funcsample_fitness=fitness_func(positions[0])# return the generated initial positionsreturnpositionsFitness FunctionsFitness functions play an important role in meta-heuristic optimization. Generally in a feature selection task, a multi objective fitness function is used to achieve a high accuracy while keeping the number of features at minimum.Provided fitness functionMultiobjectiveFitnessOne of the most popular fitness functions used in feature selection is defined by this equation.Fitness(selected_features) = alpha * KNN_ACCURACY(selected_features) + (1-alpha) * count(selected_features)This package provides aMultiobjectiveFitnessclass which applies K-fold cross validation and calculates the accuracy of a KNN classifier and then uses the above equation to find the fitness of the given features. To use this fitness function simply insatiate this class withalphaandk(number of neighbors) parameters.fitness=MultiobjectiveFitness(alpha=0.99,k=5)Defining your own fitness functionOf course you can implement your own fitness function. To do so, you must extend theBaseFitnessclass and implement the evaluate function. This function receives the solution to evaluate as an argument. You can access the data withself.featuresandself.labelsvariables. Here's an example.fromevotoolbox.fitnessimportBaseFitnessclassMyFitness(BaseFitness):defevaluate(self,solution):# use self.features and self.labels to evaluate the given solution# return a number, corresponding to the current solution fitnessreturnfitnessBinary TransformersMost of the evolutionary algorithms work in continuos space. However, for a feature selection task, we must convert these continuos values to binary values so that we can use them to choose the best features.Provided binary transformersThis package provides a variety of binary transformers. The transformer is given to the optimizer so that it can use it on every iteration to convert the continuos solutions into binary values. There is one threshold based and three transfer function based methods which you can use. A transfer function gives the probability of a feature being set to 1 and it is used as follows.Z(x)is the binary value ofxandT(x)is the transfer function.ThresholdTransformerThe simplest is theThresholdTransformer. This transformer uses a threshold to simply set anything above the threshold to one, and anything below the threshold to zero.fromevotoolbox.binaryimportThresholdTransformertransformer=ThresholdTransformer(0.5)# provide the threshold value hereSigmoidTransformerSigmoidorStransfer function is a popular transfer function which is defined as below.fromevotoolbox.binaryimportSigmoidTransformertransformer=SigmoidTransformer(1)# set alpha value hereVTransformerThere are different types ofVtransformers. This package uses the following equation.fromevotoolbox.binaryimportVTransformertransformer=VTransformer(1)# set alpha value hereQTransformerQuadratic transfer functions are another group of transfer functions and are defined with the following formula.fromevotoolbox.binaryimportQTransformertransformer=QTransformer(6,1)# set Xmax and pDefining your own binary transformerYou might need to implement a custom binary transformer. Like other classes, you can easily extend theBaseTransformerclass inevotoolbox.binaryand implement thetransformfunction to create your own transformer. Here's an example.importnumpyasnpfromevotoolbox.binaryimportBaseTransformerclassVTransformer(BaseTransformer):def__init__(self,custom_parameter):# define your custom parameters hereself.custom_parameter=custom_parameterdeftransform(self,solution):binary_solution=np.zeros_like(solution,dtype='int')# put your logic herereturnbinary_solutionOptimizersThis package comes with a variety of optimizers to use conveniently. You are also free to define your own optimizers.Provided OptimizersThese optimizers are currently available in this package. Please note that this is an ongoing project and this list will be updated regularly with new algorithms.Each of the optimizers can have their own parameters, but the first four arguments when instantiating an optimizer class are shared and required. These arguments are:binary_transformer,max_iter,lb, andub. these arguments control the binary transformer used to binarize the continuos values, max number of iterations, lower bound, and upper bound, respectively.Grey Wolf Optimizer (GWO)GWO is introduced by ... for more info refer to therelevant paper.importGWOfromevotoolbox.optimizersinitializer=GWO(binary_transformer,max_iter,lb,ub)Butterfly Optimization Algorithm (BOA)BOA is introduced by ... for more info refer to therelevant paper.importBOAfromevotoolbox.optimizersinitializer=BOA(binary_transformer,max_iter,lb,ub,p=0.8,a=0.1,c_min=0.01,c_max=0.25)Genetic Algorithm (GA)GA is introduced by ... for more info refer to therelevant paper.importGAfromevotoolbox.optimizersinitializer=GA(binary_transformer,max_iter,lb,ub,MR=0.01,CR=0.8)Harris Hawk Optimizer (HHO)HHO is introduced by ... for more info refer to therelevant paper.importHHOfromevotoolbox.optimizersinitializer=HHO(binary_transformer,max_iter,lb,ub,beta=1.5)Salp Swarm Algorithm (SSA)SSA is introduced by ... for more info refer to therelevant paper.importSSAfromevotoolbox.optimizersinitializer=SSA(binary_transformer,max_iter,lb,ub)Defining your own optimizerYou probably want to implement your own optimizer to try out a new algorithm. Defining a new optimizer is simple, you should extend theBaseOptimizerclass provided inevotoolbox.optimizers. You can define additional parameters required for your algorithm in the class constructor. All optimizers must implement the abstractoptimize(self, fitness_func, initial_positions, n_features, n_agents)method defined inBaseOptimizerTake a look at this example.importnumpyasnpfromevotoolbox.optimizersimportBaseOptimizerclassBOA(BaseOptimizer):def__init__(self,binary_transformer,max_iter,lb,ub,my_parameter):super().__init__(binary_transformer,max_iter,lb,ub)self.my_parameter=my_parameterdefoptimize(self,fitness_func,initial_positions,n_features,n_agents):# Optimize the problem using the given arguments# You may use your custom parameter with self.my_parameter# initial_positions will be the population initialized before# Your function must return a dictionary as defined below:# solution: the binary solution with shape (n_features,) where the selected features are 1 and others are 0# c: the convergence curve of the fitness# nf: number of selected features in the final solutionreturn{'solution':None,'c':None,'nf':None,}CreditsAuthors:Shakiba Shahbandegan