Package: evotorch
Welcome to the EvoTorch project!
EvoTorch is an open source evolutionary computation library developed atNNAISENSE, built on top ofPyTorch.
See thedocumentationfor in-depth guidance about using EvoTorch, andjoin us on Slackfor discussions.Get started by installing EvoTorch:pip install evotorchWith EvoTorch, one can solve various optimization problems, regardless of whether they are differentiable (i.e. allow gradient descent). Among the problem types that are solvable with EvoTorch are:Black-box optimization problems (continuous or discrete)Reinforcement learning tasksSupervised learning tasksVarious evolutionary computation algorithms are available in EvoTorch:Distribution-based search algorithms:PGPE:Policy Gradients with Parameter-based Exploration.XNES:Exponential Natural Evolution Strategies.CMA-ES:Covariance Matrix Adaptation Evolution Strategies.SNES:Separable Natural Evolution Strategies.CEM:Cross Entropy Method.Population-based search algorithms:GeneticAlgorithm:A genetic algorithm implementation. Also supports multiple objectives, in which case it behaves likeNSGA-II.CoSyNE:Cooperative Synapse Neuroevolution.MAPElites:Multi-dimensional Archive of Phenotypic ElitesSince all of these algorithms are implemented in PyTorch, they benefit from use of vectorization and parallelization on GPUs, drastically speeding up optimization when GPUs are available.
UsingRay, EvoTorch scales these algorithms even further by splitting the workload across:multiple CPUsmultiple GPUsmultiple computers in a Ray clusterExamplesBelow are some code examples that demonstrate the API of EvoTorch.A black-box optimization exampleAny objective function defined to work with PyTorch can be used directly with EvoTorch.
A non-vectorized objective function simply receives a solution as a 1-dimensional torch tensor, and returns a fitness as a scalar.
A vectorized objective function receives a batch of solutions as a 2-dimensional torch tensor, and returns a 1-dimensional tensor of fitnesses.
The following example demonstrates how to define and solve the classical Rastrigin problem.fromevotorchimportProblemfromevotorch.algorithmsimportSNESfromevotorch.loggingimportStdOutLogger,PandasLoggerimportmathimportmatplotlib.pyplotaspltimporttorch# Declare the objective functiondefrastrigin(x:torch.Tensor)->torch.Tensor:A=10(_,n)=x.shapereturnA*n+torch.sum((x**2)-A*torch.cos(2*math.pi*x),1)# Declare the problemproblem=Problem("min",rastrigin,initial_bounds=(-5.12,5.12),solution_length=100,vectorized=True,# device="cuda:0"  # enable this line if you wish to use GPU)# Initialize the SNES algorithm to solve the problemsearcher=SNES(problem,popsize=1000,stdev_init=10.0)# Initialize a standard output logger, and a pandas logger_=StdOutLogger(searcher,interval=10)pandas_logger=PandasLogger(searcher)# Run SNES for the specified amount of generationssearcher.run(2000)# Get the progress of the evolution into a DataFrame with the# help of the PandasLogger, and then plot the progress.pandas_frame=pandas_logger.to_dataframe()pandas_frame["best_eval"].plot()plt.show()A reinforcement learning exampleThe following example demonstrates how to solve reinforcement learning tasks that are available through the gym library.fromevotorch.algorithmsimportPGPEfromevotorch.loggingimportStdOutLogger,PicklingLoggerfromevotorch.neuroevolutionimportGymNE# Declare the problem to solveproblem=GymNE(env="Humanoid-v4",# Solve the Humanoid-v4 tasknetwork="Linear(obs_length, act_length)",# Linear policyobservation_normalization=True,# Normalize the policy inputsdecrease_rewards_by=5.0,# Decrease each reward by 5.0num_actors="max",# Use all available CPUs# num_actors=4,    # Explicit setting. Use 4 actors.)# Instantiate a PGPE algorithm to solve the problemsearcher=PGPE(problem,# Base population sizepopsize=200,# For each generation, sample more solutions until the# number of simulator interactions reaches this thresholdnum_interactions=int(200*1000*0.75),# Stop re-sampling solutions if the current population size# reaches or exceeds this number.popsize_max=3200,# Learning ratescenter_learning_rate=0.0075,stdev_learning_rate=0.1,# Radius of the initial search distributionradius_init=0.27,# Use the ClipUp optimizer with the specified maximum speedoptimizer="clipup",optimizer_config={"max_speed":0.15},)# Instantiate a standard output logger_=StdOutLogger(searcher)# Optional: Instantiate a logger to pickle and save the results periodically.# In this example, among the saved results will be the center of the search# distribution, since we are using PGPE which is distribution-based._=PicklingLogger(searcher,interval=10)# Run the algorithm for the specified amount of generationssearcher.run(500)# Get the center point of the search distribution,# obtain a policy out of that point, and visualize the# agent using that policy.center_solution=searcher.status["center"]trained_policy=problem.make_net(center_solution)problem.visualize(trained_policy)More examples can be foundhere.How to citeIf you use EvoTorch in your research, please consider citing ourpaper.@article{evotorch2023arxiv,title={{EvoTorch}: Scalable Evolutionary Computation in {Python}},author={Toklu, Nihat Engin and Atkinson, Timothy and Micka, Vojt\v{e}ch and Liskowski, Pawe\l{} and Srivastava, Rupesh Kumar},journal={arXiv preprint},year={2023},note={https://arxiv.org/abs/2302.12600}}How to ContributePlease see ourcontribution guidelines.AuthorsNihat Engin TokluTimothy AtkinsonVojtech MickaPawel LiskowskiRupesh Kumar Srivastava