An extensible Evolutionary Algorithm Example in Python | by Dr. Robert Kübler | Towards Data Science

Evolutionary Algorithms are special methods to solve computational problems, such as optimization problems. They often yield very good results in a reasonable amount of time without us having to put a lot of thought into the problem-specific properties. Usually, we only have to adjust some parameters and then let a quite general framework run and bring us a solution. You will see how to implement such a general framework in Python in this article.
This is especially interesting when dealing with extremely difficult problems, such as NP-complete ones. These are relevant real-life problems that many companies have to solve every day for which we do not know any efficient algorithm. One of these problems is the optimization version of the Travelling Salesman Problem (TSP), which is phrased like this:
A salesman wants to sell his goods in n cities. He starts in city 1, visits each of the other n-1 cities exactly once in some order and returns to city 1 again. What is the best order to visit the cities in order to minimize the travelling distance?
We will pick up this problem later again as a non-trivial example of applying our Evolutionary Algorithm. But before we go there, let us see what Evolutionary Algorithms are and how to apply them to an easy example.
If you just want to see the code, visit my GitHub page! ;)
A fun application of Evolutionary Algorithms is MarI/O built by Seth Bling, based on the “NEAT” paper [3]. A complex Neural network architecture is built from scratch using an Evolutionary Algorithm to play the classic Super Mario World. Nostalgia kicks in.
We will now see how to develop an Evolutionary Algorithm to solve a simple function maximization problem, i.e. we want to find an input x that maximizes the output of a given function f. For f(x, y)=-(x2+y2) the unique solution would be (x, y)=(0, 0), for example.
This algorithm can be easily adapted to solve the TSP and other problems as well. But first, let us see what an Evolutionary Algorithm actually is.
Evolutionary Algorithms are designed to resemble the evolution found in nature. Among other things, these three concepts are the core of evolution:
There is a population of individuals.Individuals can reproduce and die. Any of these things happening is determined by its fitness. The higher the fitness, the longer the individual’s properties (DNA) stay in the population, either by the individual itself or its offsprings.Individuals can mutate, i.e. alter their properties a bit.
There is a population of individuals.
Individuals can reproduce and die. Any of these things happening is determined by its fitness. The higher the fitness, the longer the individual’s properties (DNA) stay in the population, either by the individual itself or its offsprings.
Individuals can mutate, i.e. alter their properties a bit.
We can use these concepts to create a meta-algorithm, i.e. an algorithm that internally uses other algorithms we have yet to specify, to solve our problem. Don’t worry, we will fill it with life immediately.
Generate an initial population of individuals randomly.Evaluate the fitness of each individual in the population.Repeat as often as you like:a) Select individuals with a good fitness score for reproduction.b) Let them produce offsprings.c) Mutate these offsprings.d) Evaluate the fitness of each individual in the population.f) Let the individuals with a bad fitness score die.Pick the individual with the highest fitness as the solution.
Generate an initial population of individuals randomly.
Evaluate the fitness of each individual in the population.
Repeat as often as you like:a) Select individuals with a good fitness score for reproduction.b) Let them produce offsprings.c) Mutate these offsprings.d) Evaluate the fitness of each individual in the population.f) Let the individuals with a bad fitness score die.
Pick the individual with the highest fitness as the solution.
Wait, this is quite general, isn’t it? There are a lot of things that have to be specified.
What are the individuals?How many individuals are in the first population and how are they generated?Which fitness score?How many parents produce how many offsprings? And how exactly?How to mutate?How many individuals die?Repeat how often?
Let us now use the problem of maximizing a function to show how we can concretely implement meta-algorithm.
Let us use the function f(x)=-x(x-1)(x-2)(x-3)(x-4) on the interval [0, 4] as an example. We wish to find some value to maximize this function. A potential problem might be the local maximum at around 1.4, which we do not want to find!
Let us see if we can reproduce this using an Evolutional Algorithm. So, let us start answering all the open questions.
The individuals are always potential solutions to the problem.
In our example, numbers between 0 and 4, since we only consider this function on this interval. This one was easy, right? It will not get much harder than this, I promise.
The number of individuals is a hyperparameter you have to make up. If you have no clue about what the solution could be, choose the initial population as random as possible.
In our case, how about we use 10 random individuals, with random meaning uniformly drawn numbers between 0 and 4?
I expect using uniformly random elements to be a good choice since we can cover the entire solution space quite well. If we use, for example, a normal distribution with a mean of 1.4, maybe we would push our solution to be the wrong local maximum around 1.4 as well.
We have to make this one up, too. Our solution should have the highest fitness score.
We want to maximize our function f, so we can just use our function f itself.
We have to make this one up, too. All of it.
In our example, we could always use two individuals to produce one offspring. We could also let them produce more, but let us start easily here. So, if we have two individuals (also called parents), their offspring could just be the mean of them. The offspring of a 1.1 and a 3.5 could be a 2.3, for example. In total, let us produce three offsprings, using six distinct parents in total (i.e. each parent only has exactly one partner).
Again, these are all things that I arbitrarily decided. You can use other strategies to conduct this step, too! Use one or three parents per offspring, make parents get five offsprings, play round.
The six individuals with the highest fitness are the numbers 1.792, 1.184, 3.169, 1.641, 1.431 and 3.785 (in increasing order of fitness). Now, let’s say that 2 neighbors in this list get an offspring:
1.792 with 1.184: 1.488
3.169 with 1.641: 2.405
1.431 with 3.785: 2.608
Alter the offsprings a little bit probabilistically. This is a good way to explore more possible solutions in different places and don’t go the wrong way deterministically.
In our case, we can just add Gaussian noise to each offspring. Maybe a mean of zero and a standard deviation of 0.25 is fine. Also, remember:
The individuals have to be potential solutions! So, the mutation must not push the children out of the feasible interval [0, 4]. If this happens, set the children to the closest edge of the interval.
As many as you like. Maybe it’s good to keep the population the same size over the steps, since otherwise the population size might explode or everyone dies at some point.
Since we have 13 individuals in the population now, let three of them die and we end up with 10 again. Let us pick the ones with the worst fitness, which are the old individuals 0.596 and 2.495 as well as the newly created individual 2.121. 😔
As often as you like. Just check sometimes, until you see that the solutions don’t get better anymore.
Maybe let us fix 50 epochs and see what happens. After our artificial evolution worked all the way through, we can check all the individuals and pick the one with the highest score as the solution to our problem.
The evolution up to epoch no. 50 looks like this:
We can see that the algorithm worked! But it looks like it was a close call. Up to epoch 25, the population was gathering around the other local maximum at around x=1.4. Luckily, we had an individual far away at 4, which managed to pull the population to the right side, starting from epoch 25.
We can also see this shift around epoch 25 when looking at the fitness score of the best individuals of the population in each epoch.
Okay, so we have seen how the algorithm worked and that we even got the correct answer (or at least a very good approximation) in the end! There was a bit of luck involved, but this will always be the same. We could have been even unluckier and our initial population could have started around the local maximum at x=1.4. Then it would have been very unlikely that individuals would have broken out of this region, as in this example:
It can still be the case that a mutation pushes an individual to the right side, which in turn pulls more individuals with it. But this is highly unlikely since a mutation pushes an individual by at most 0.75 in any direction with probability >99%, since the standard deviation was 0.25.
So, is it good to use a more extreme mutation? Well, if we exaggerate, the individuals just jump all over the place or gather in the corners x=0 and x=4. So we have to be a bit careful about choosing the parameters here.
From a theoretical standpoint, you can see an Evolutionary Algorithm as a heavily randomized algorithm. The initial population is random, individuals generate offsprings via more or less complicated operations, the offsprings mutate using randomness and all of this is repeated hundreds, thousands, millions of times.
Therefore, it is very hard to analyze these algorithms and give any theoretical bounds on their success probability or the quality of their results. For very easy algorithms this is possible, such as the (1 + 1) evolutionary algorithm [1] for maximizing extremely simple functions or an algorithm for solving the Simplified Multiobjective Knapsack Problem [2], but examples like this are rarely seen.
However, if a run did not work, just try it again with the same or other parameters.
Here, I will share my implementation with you. I tried to make it general and abstract, so you can easily use it for your purposes.
If you have never used Abstract Base Classes before, don’t worry. The class Individual is only there to tell you which interface you have to use for the objects that represent your individuals. Your individuals need a value (the payload, the potential solution), you have to implement a random initialization, a mutation, and pair function.
Then the Pool and Evolution classes take care of the rest. I also implemented the function maximization example, to show you how this can look like. I used a few more parameters there, but if you replace all lower bounds with 0, upper bounds with 4, rate with 0.25, dim with 1 and alpha=0.5, you end up exactly with our example again.
You can use these classes like this:
And off you go!
So far we have gained some confidence with a small toy problem, but now it is time to get back to the difficult TSP again.
The TSP is all about finding the shortest round trip through n cities. Use these, as an example:
The solution to the TSP looks like this:
There exist numerous algorithms to solve this one. They range from exhaustive search (trying out all ways) to more complicated algorithms. However, all of these methods are slow and if P≠NP, we can not expect any fast algorithm to emerge, ever.
The exhaustive search will let you solve problems up to maybe 15 cities if you can wait for a few days. Other methods like the Held-Karp algorithm let you solve instances up to 30–50 cities, optimistically.
If you drop the need for the solution to be the best, Christofides Algorithm from 1976 will give you a solution in a short amount of time that is provably at most 50% longer than the shortest round trip.
Think about why this is an awesome result: We don’t even know the shortest round trip length, but still we can say that the round trip return by this algorithm is less than 50% longer than this unknown solution!
But now, let us get you what you have waited for: an Evolutionary Algorithm for solving TSP!
This time, we start off with 20 cities. The amount is still low enough to solve it with the Held-Karp algorithm, so we can even check if we got the best solution in the end! Here is the map:
Now, we have to answer all the questions again. Let us first answer the easier “number questions” (hyperparameters that we can easily adjust if things don’t workout):
How many epochs? 1000
How many starting individuals? 100
How many parents? 60 (two produce one offspring, like before)
How many offsprings? 30
How many dying individuals? 30
The exciting questions are dealing with the implementation of a class TSP that we have to define now. What are the individuals? How does the random initialization work? The paring? The mutation? What is the fitness score?
Let’s start with a basic one: What are the individuals here? Protip: It’s not the cities. Again:
The individuals are always potential solutions to the problem.
In the case of TSP, we search for a short round trip. With “short”, we mean the sum of the Euclidean distances between every two cities in the round trip. We can represent such a round trip as a list of n numbers, e.g. (0, 3, 1) reads as “Start from city 0, then go to city 3, from there to city 1, then back to city 0.”. So, our individuals are lists of numbers, containing each number from 0 to n-1 exactly once, i.e. permutations on {0, 1, 2, ..., n-1}.
Within the whole process, we have to maintain this property. There is no point in keeping individuals in the population that don’t represent a feasible solution to the problem since it clogs the population, and we might even pick it in the end, giving us an invalid solution.
And how to initialize them? Well, just take a random permutation of the numbers from 0 to n-1. This should do the job.
def _random_init(self, init_params):    return np.random.choice(        a=range(init_params['n_cities']),        size=init_params['n_cities'],        replace=False    )
Let us continue with the mutation procedure.
So, how can we alter a round trip in a simple way? Imagine we have a round trip through five cities, for example (3, 1, 4, 2, 0). An easy way is to just randomly swap out two elements. Our example could mutate to (0, 1, 4, 2, 3) or (3, 4, 1, 2, 0). We can also swap more than one time and make a hyperparameter rate out of it comparable to the standard deviation in the function maximization example before.
def mutate(self, mutate_params):    for _ in range(mutate_params['rate']):        i, j = np.random.choice(            a=range(len(self.value)),            size=2,            replace=False)        self.value[i], self.value[j] = self.value[j], self.value[i]
It is clear that this kind of mutation preserves the property, that individuals are permutations, i.e. feasible solutions.
The most interesting thing here is probably the pair function.
Let us assume that we have the following individuals which we want to use as parents:
The way you can do this is the following: You copy the right half of the second individual into the right half of the first individual.
We want to use the altered first individual as the offspring now, but this does not work as-is since there are duplicate numbers now. Hence, the left half of the altered first individual has to be repaired first. We can do it like this:
The 4 that was inserted into the right half kicked out the 5 that was there before. So, let us replace the 4 in the left half by this 5 since it is missing now. Doing the same with 2 that kicked out a 3 gives the following:
But there is the next problem: We have number 3 two times now. But we can repeat this procedure several more times until we get a feasible individual again. The 3 is the problem. It initially kicked out the 1. So replace the 3 from the left half with a 1, and voila: the offspring is (0, 5, 1, 4, 2, 3).
In total, our class TSP looks like this:
Here I also introduced another hyperparameter alpha, which decides where to split the individuals. For my example, I used alpha=0.5.
Since we want to minimize the round trip length, we can just use minus the length of the round trip as the fitness score.
Let us solve the TSP with 20 cities! I used the following code: I defined a fitness function, a function to compute the round trip length and a function to compute the distance matrix (pairwise distances between two cities).
For me, the fitness function behaved like this:
And here are the best individuals per epoch:
And the solution that we have from around epoch 400 is even the best solution we can get for these 20 cities! I checked this using the Held-Karp algorithm implementation by Carl Ekerot.
For the last act, let us try something really difficult. Not only 10 or 20 cities but 100, which is way too much for any exact algorithm to handle.
I can not tell you if this is the fastest round trip, but for me, it definitely seems like a good one!
In this article, we have seen how Evolutionary Algorithms work using two examples: maximizing a function and solving the Travelling Salesman problem. Evolutionary Algorithms are usually used to solve difficult problems we do not know exact answers to. The algorithms can be very fast and yield accurate results. The implementation is easy and you don’t have to be an expert in the field your problem is from, e.g. you don’t have to read papers of the past 20 years of TSP research to write a decently performing algorithm.
Sadly, it is very hard to obtain any theoretical results, because Evolutionary Algorithms often work with a lot of randomness interacting in complicated ways. Thus, we can not know if the solution given is good or bad, we can only check if it is good enough for our use case. In the 100 cities example, if our goal was to travel to each city in length 150,000 or less, the algorithm has given us a perfect answer.
[1] S. Droste, T. Jansen, and I. Wegener, On the analysis of the (1+1) evolutionary algorithm (2002), Theoretical Computer Science, Volume 276, Issues 1–2, 6 April 2002, Pages 51–81
[2] M. Laumanns, L. Thiele, and E. Zitzler, Running Time Analysis of Evolutionary Algorithms on a Simplified Multiobjective Knapsack Problem (2004), Natural Computing 3, 37–51
[3] K. Stanley, R. Miikkulainen, Evolving Neural Networks through Augmenting Topologies (2002), Evolutionary Computation 10(2) 2002, S. 99–127
I hope that you learned something new, interesting, and useful today. Thanks for reading!
As the last point, if you
want to support me in writing more about machine learning andplan to get a Medium subscription anyway,
want to support me in writing more about machine learning and
plan to get a Medium subscription anyway,
why not do it via this link? This would help me a lot! 😊
To be transparent, the price for you does not change, but about half of the subscription fees go directly to me.
Thanks a lot, if you consider supporting me!
If you have any questions, write me on LinkedIn!