Package: mevo
MEvoMEvo is a evolutionary algorithm package, which implements Genetic Algorithm and Evolution Strategies Algorithms.
It can be used to form an agent strategy or optimize math problems.Quick lookTraining a cartpole policy by using Genetic Algorithm.importmevoimportgymnasiumimportnumpyasnp# define a fitness function to get fitness for every individualdeffitness_fn(ind:mevo.individuals.Individual,conf:dict)->float:ep_r=0env=gymnasium.make('CartPole-v1')env.reset()for_inrange(2):s,_=env.reset()for_inrange(500):# in one episodelogits=ind.predict(s)a=np.argmax(logits)s,r,done,_,_=env.step(a)ep_r+=rifdone:breakreturnep_r# trainingwithmevo.GeneticAlgoNet(max_size=20,layer_size=[4,8,2],drop_rate=0.7,mutate_rate=0.5)aspop:forgenerationinrange(40):pop.evolve(fitness_fn=fitness_fn)print(f"generation={generation}, top_fitness={pop.top.fitness:.f2}")After only 40 generations in a population (20 individuals), it gives a greate result.
Use following code to visualize the learned policy.# deploy the best individualenv=gymnasium.make('CartPole-v1',render_mode="human")whileTrue:s,_=env.reset()whileTrue:# in one episodelogits=pop.top.predict(s)a=np.argmax(logits)s,_,done=env.step(a)[:3]ifdone:breakWhat is MEvoIn the MEvo, the smallest data segment is Gene. A set of gene can be packed into a Chromosome.
Chromosomes then formed an Individual. A Population consists of many individuals.This is different from the classical Genetic Algorithm which their chromosome only consists of a 1D array.
In other words, chromosome in the MEvo is n-D array with different shape.Why making data shape varied?Because we want to possibly evolve a neural network which has multiple layers in various shape.MEvo has two different algorithm families.Genetic Algorithm familyEvolution Strategy familyThe Genetic Algorithm basically has two steps:crossover the chromosomes in a populationmutate gene in new generationThe following image shows how crossover on two parents chromosomes,
and how to mutate the new chromosome.And MEvo's Evolution Strategy has different two steps:mutate gene (normal distribution) from one single chromosomeupdate this chromosome by all mutated resultsParallel trainingMEvo support parallel training. Simply setn_worker> 1 to unlock your machine power for training on multiple cores.
Whenn_worker=-1then use all your cores.Simply replace thepopdefinition by includingn_workerfrom previous code.# parallel trainingif__name__=="__main__":withmevo.GeneticAlgoNet(max_size=20,layer_size=[4,8,2],n_worker=-1,drop_rate=0.7,mutate_rate=0.5)aspop:forgenerationinrange(40):pop.evolve(fitness_fn=fitness_fn)print(f"generation={generation}, top_fitness={pop.top.fitness:.2f}")Note that the parallel code must be run underif __name__ == "__main__"：scope,
otherwise a python multiprocessing err will occur.InstallUsing pip to install MEvopip3installmevoMethods and ParametersMEvo support following populations:mevo.GeneticAlgoInt()mevo.GeneticAlgoFloat()mevo.GeneticAlgoOrder()mevo.GeneticAlgoNet()mevo.EvolutionStrategyNet()A classical genetic algorithm problemsuch as Travel Sales Problem (TSP) can be solved by following classical Populations:mevo.GeneticAlgoInt()mevo.GeneticAlgoFloat()mevo.GeneticAlgoOrder()A Travel Sales Problem example usingpop = mevo.GeneticAlgoOrder():importmevoimportnumpyasnppositions=[np.random.rand(2)for_inrange(20)]defdistance_fitness_fn(ind:mevo.individuals.Individual,conf:dict)->float:order=[c.data[0]forcinind.chromosomes]cost=0foriinrange(len(order)-1):p1,p2=positions[order[i]],positions[order[i+1]]cost+=np.square(p1-p2).sum()fitness=-costreturnfitnesspop=mevo.GeneticAlgoOrder(max_size=50,chromo_size=len(positions),drop_rate=0.3,mutate_rate=0.01,)pop.run(fitness_fn=distance_fitness_fn,step=30)An optimization problem example usingpop = mevo.GeneticAlgoInt():importmevoimportnumpyasnpdefwave_fitness_fn(ind:mevo.individuals.Individual,conf:dict)->float:binary=[c.dataforcinind.chromosomes]c=np.concatenate(binary,axis=0)a=2**np.arange(len(c))[::-1]decimal=c.dot(a)x=decimal/float(2**len(c)-1)*5o=np.sin(10*x)*x+np.cos(2*x)*xreturnopop=mevo.GeneticAlgoInt(max_size=20,chromo_size=10,drop_rate=0.3,chromo_initializer=mevo.chromosomes.initializers.RandomInt(0,2),mutate_rate=0.01,)pop.run(step=20,fitness_fn=wave_fitness_fn)Deep Net EvolutionFor policy learning or Reinforcement learning alternative, the following two methods has their advantages.mevo.GeneticAlgoNet()mevo.EvolutionStrategyNet()Reinforcement LearningMEvoTrainingHas forward and backward propagationOnly has forward propagation, but need crossover or mutation operation (lighter than backpropagation)ExplorationNeeds carefully set explore policyDifferent children setting automatically ensure the explorationMemory needsCan only keep one set of parametersIn each generation, must compute all children's parameters (parallel computing save time)Network SizeGenerally large and deep netWith a large scale exploration, a relatively small net can perform a good jobAmevo.GeneticAlgoNet()example:importmevoimportgymnasiumimportnumpyasnpdeffitness_fn(ind:mevo.individuals.Individual,conf:dict)->float:ep_r=0env=gymnasium.make('Pendulum-v1')env.reset(seed=conf["seed"])for_inrange(2):s,_=env.reset()for_inrange(150):# in one episodelogits=ind.predict(s)a=np.tanh(logits)*2s,r,_,_,_=env.step(a)ep_r+=rreturnep_rdeftrain():withmevo.GeneticAlgoNet(max_size=30,layer_size=[3,32,1],drop_rate=0.7,mutate_rate=0.5,n_worker=-1,)aspop:forepinrange(700):pop.evolve(fitness_fn=fitness_fn,)print(ep,pop.top.fitness)returnpop.topdefshow(top):env=gymnasium.make('Pendulum-v1',render_mode="human")whileTrue:s,_=env.reset()for_inrange(200):# in one episodelogits=top.predict(s)a=np.tanh(logits)*2s,_,_,_,_=env.step(a)if__name__=="__main__":top=train()show(top)Amevo.EvolutionStrategyNet()example:importmevoimportgymnasiumimportnumpyasnpdeffitness_fn(ind:mevo.individuals.EvolutionStrategyDense,conf:dict)->float:ep_r=0seed=conf["seed"]index=conf["index"]env=gymnasium.make('Pendulum-v1')# ! must set seed and clone when using mevo.EvolutionStrategyNet()env.reset(seed=conf["seed"])c_ind=ind.clone_with_mutate(index,seed)# ###############for_inrange(2):s,_=env.reset()for_inrange(100):logits=c_ind.predict(s)a=np.tanh(logits)*2s,r,_,_,_=env.step(a)ep_r+=rreturnep_rdeftrain():withmevo.EvolutionStrategyNet(max_size=15,layer_size=[3,32,1],mutate_strength=0.05,learning_rate=0.1,n_worker=-1,seed=2)aspop:forepinrange(700):pop.evolve(fitness_fn=fitness_fn)print(ep,pop.top.fitness)returnpop.topdefshow(top):env=gymnasium.make('Pendulum-v1',render_mode="human")whileTrue:s,_=env.reset()for_inrange(200):# in one episodelogits=top.predict(s)a=np.tanh(logits)*2s,_,_,_,_=env.step(a)if__name__=="__main__":top=train()show(top)