An insight into the concept of Genetic Algorithm
By: Rajarsi Saha
Tags: Genetics, Towards Data Science, Programming, Genetic Algorithm, Algorithms

We hear about genetic algorithm every now and then but what exactly is this genetic algorithm? And why does it sound so much related to the subject of biology? Does it really use genes or some other biological entity while it is being implemented or the processes it involves are similar to what happens in biology? These are some of the questions even I had when I first heard about the term genetic algorithm in my graduation days. If you are like what I was back then and want to figure out within minutes what this so-called genetic algorithm or GA is then this is the right place to start your journey. This article is specially written for those who are new to the concept of algorithms and want to get a good grasp of the concept of genetic algorithm in computer science.

What is a Genetic Algorithm?

Genetic Algorithm (GA) is a search-based optimization technique that follows the principle of Genetics and Natural Selection in the field of Biology. The main purpose of this algorithm is to find optimal or near-optimal solutions to quite a few difficult of difficult problems which are probably unsolvable or would take a very long time to solve using normal procedures or algorithms. Genetic Algorithm is quite often used to solve optimization problems in machine learning and to generate high-quality solutions for optimization problems.

The concept of GA has been adopted from the natural genetic evolution and thereby again proving the point that

“Nature has always been a great source of inspiration to all mankind.” — lakhasly.com

It is a subset of a much larger branch of computation, known as the Evolutionary Computation. Genetic Algorithms were developed by John H. Holland and his students and colleagues at the University of Michigan, of which David E. Goldberg is the most notable.

In short, the genetic algorithm repeatedly modifies a population of individual solutions. At each step, the genetic algorithm selects individuals at random from the current population to be parents and uses them to produce the children for the next generation. Over successive generations, the population “evolves” toward an optimal solution.

The genetic algorithm uses three main types of rules at each step to create the next generation from the current population: • Selection rules select the individuals, called parents, that contribute to the population at the next generation. • Crossover rules combine two parents to form children for the next generation. • Mutation rules apply random changes to individual parents to form children. [1]

Image by author on actual genetic algorithm flowchart

Difference between Classical Algorithm and Genetic Algorithm

A classical algorithm generates a single point after each iteration, and a sequence of those points approaches an optimal solution. Whereas on the other hand, a GA generates a population of points after each iteration and the best point in the population approaches the optimal solution.

A classical algorithm selects the next point in the sequence by a deterministic approach but a genetic algorithm selects the next population by in a random fashion using a random number generator.

A few important terminologies

Search Space:

Search Space is basically where all the individuals or data points in a population are stored and maintained. Each of these individuals represents one solution in the search space for a given problem. Each one of these is an array or a vector of components and is analogous to a chromosome i.e., it can be thought of as a single chromosome inside a nucleus. The variable components inside a chromosome can be thought of as genes i.e., a chromosome consists of several genes.

Fitness Score:

There is always a fitness score that is given to each individual or data point. The individuals with the best or better to say ‘the optimal’ score, are taken into consideration and given a higher priority to reproduce new generations. These individuals with better fitness scores are selected for mating and they produce better offspring by combining the chromosomes of the parents. While implementing the algorithm, we can take a population size to be fixed or static. Higher population sizes give rise to better offspring in lesser iterations and it is hoped that with successive iterations or generations better solutions are achieved. It is considered that the population has converged and has reached the optimal solution if there is no more significant difference between the new offspring and the offspring produced by the previous generations. Therefore, the algorithm is said to have converged to a set of solutions for the given problem.

Operations:

Selection: This is an operation where the individuals with a better fitness score are given preference and allowed to pass their genes to the successive generations. Therefore, individuals with better fitness scores are allowed to mate and produce offspring for the next successive generation.

Crossover: This is an operation where the actual (analogous) mating takes place between individuals. The crossover sites or locations in a chromosome are chosen in this operation and the two selected chromosomes from the selection operation are taken for crossover. The actual crossover happens when the genes, which were chosen are taken and combined to produce the new offspring (which is of the same length as that of the parents.) Please note all the selection of crossover points is done at random. The following example will make it clearer.

Image by author on Crossover technique

Mutation: This is an operation where some individual genes are taken from an offspring and mutated or changed. It’s better to use the word ‘insert’ rather than ‘change’. This is primarily done to maintain diversity in the population and also to avoid any premature convergence of the algorithm which we don’t want. We want the algorithm to take its own time to give us the optimal solution. Please note here also the mutating locations are chosen at random. Here’s an example to make things clear.

Image by author on Mutation technique

Advantages and Disadvantages of Genetic Algorithm

There are many advantages of genetic algorithms over traditional optimization algorithms. Two of the most notable are, the ability to deal with complex problems and parallelism. Genetic algorithms can deal with various types of optimization, whether the objective (fitness) function is stationary or nonstationary (changes with time), linear or nonlinear, continuous or discontinuous, or with random noise. Because multiple offspring in a population act like independent agents, the population (or any subgroup) can explore the search space in many directions simultaneously. This feature makes it ideal to parallelize the algorithms for implementation. Different parameters and even different groups of encoded strings can be manipulated at the same time. [2] However, genetic algorithms also have some disadvantages. The formulation of a fitness function, the use of population size, the choice of important parameters such as the rate of mutation and crossover, and the selection criteria of the new population should be carried out carefully. Any inappropriate choice will make it difficult for the algorithm to converge or it will simply produce meaningless results. Despite these drawbacks, genetic algorithms remain one of the most widely used optimization algorithms in modern nonlinear optimization. [2]

Further advantages include:

Genetic Algorithms has the ability to optimize continuous and discrete functions and also multi-objective problems like route mapping (dependant on distance, time, traffic, etc.)

It always generates a solution and also the one that improves with time or successive iterations.

This algorithm is quite useful and produces better results when the search space or population of individuals is very large and has a large number of parameters.

It is basically faster than classical algorithms and more efficient.

Disadvantages include:

The greatest disadvantage could be that since GA is stochastic in nature, there is no guarantee on the optimality or quality of the solution. Many times it may not reach the desired solution.

If the GA is not implemented correctly, then we might fail to reach the optimal solution i.e., the algorithm may not converge.

The very fact that fitness value is calculated for every successive generation may increase computational complexity for some problems.

Pseudocode

Here’s a sketch of a typical GA in pseudocode:

GA()

begin

Create initial population;

while (Until Stopping Criteria)

for (Each Chromosome)

Calculate fitness value;

Selection (Survival of strong individuals);

Crossover (Here, new generation produced);

Mutation (Changing some genes for new and

different individuals);

end

Generate new population;

end

end

Can we now dive into the code?

The following is a simple example python program where the genetic algorithm has been implemented. The code has been taken from GeeksforGeeks and therefore in no manner do I claim the ownership of the following code. I will try my best to explain the code step by step.

This portion is mainly importing the random library in Python, setting a population size, setting the valid genes, and the target string.

This is where the class Individual definition starts and it defines, two-member variables chromosome and fitness, and initializes them.

mutated_genes() is a function where the mutation in the genes takes place. Here, a random gene is taken from the pool of the genes and is returned to the main program.

create_gnome() creates individual gnomes or chromosomes of target length and returns to the main function.

mate() is a very important function as here is exactly where the new chromosomes with better fitness are created. Two operations that occur here are crossover and mutation. This is actually a modified or custom genetic algorithm where the operations are based on random probability. If the probability is less than 0.45 then a gene from parent chromosome 1 is passed on to the child chromosome whereas if the probability is greater than or equal to 0.45 and less than 0.90 then the corresponding gene from parent chromosome 2 is passed on to the child chromosome and if the probability is more than or equal to 0.90 then mutated gene is passed on to the child chromosome and finally the new chromosome is returned.

cal_fitness() calculates the fitness score of a particular chromosome and returns it that the genetic algorithm can take decisions accordingly. This particular function checks if the target string has been achieved or not and accordingly creates the fitness score based on the number of characters that are yet to meet the target requirement.

This is the main algorithm function where all the previous functions are called to do their individual works. First, the initial population is created. Then, the population is sorted according to its fitness score putting the best-scored chromosomes at the beginning. Now it checks if the first (best fitness score) has a fitness score less than or equal to 0 then it shall return the string as that is the target string and the algorithm stops. If not, it performs elitism by adding 10% of the fittest population into the next generation population and further 50% of the old population goes into mating to produce children. Now the new generation is worked with again from the beginning passing through all the above steps. Ultimately, we see that after a considerable amount of execution time, the algorithm stops and produces the target string.

This is the output I received:

PS C:\Users\RAJARSI> & python c:/Users/RAJARSI/Desktop/GA.py

Generation: 1 String: i a2yl [TXl Xn[:rZnL Fitness: 18

Generation: 2 String: U[vW 5 mL&TgW rdLeVY Fitness: 17

Generation: 3 String: 4g9W/5 mO&["W rGLefY Fitness: 16

Generation: 4 String: 4gCa/5 G];[4Wo;G3e}Y Fitness: 15

Generation: 5 String: 4gCa/5 G];[4Wo;G3e}Y Fitness: 15

Generation: 6 String: F 1! e GeWk@/bal#tk2 Fitness: 13

Generation: 7 String: F 9! e Ge k@#OtG#e}9 Fitness: 12

Generation: 8 String: F 9! e Ge k@#OtG#e}9 Fitness: 12

Generation: 9 String: { C0Xh GeePv=orG3ePY Fitness: 11

Generation: 10 String: 5y6/;e Gee3nWorG3ekY Fitness: 10

Generation: 11 String: 5y6/;e Gee3nWorG3ekY Fitness: 10

Generation: 12 String: I 1WPe GeekD=oZG_e}Y Fitness: 9

Generation: 13 String: I 1WPe GeekD=oZG_e}Y Fitness: 9

Generation: 14 String: I 1WPe GeekD=oZG_e}Y Fitness: 9

Generation: 15 String: I ( W GeeknaorG3ekY Fitness: 8

Generation: 16 String: I ( W GeeknaorG3ekY Fitness: 8

Generation: 17 String: I }!pe Geek"QorGPekY Fitness: 7

Generation: 18 String: I }!pe Geek"QorGPekY Fitness: 7

Generation: 19 String: I }!pe Geek"QorGPekY Fitness: 7

Generation: 20 String: F Oo&e Geekn5orGeekY Fitness: 6

Generation: 21 String: F Oo&e Geekn5orGeekY Fitness: 6

Generation: 22 String: F Oo&e Geekn5orGeekY Fitness: 6

Generation: 23 String: F Oo&e Geekn5orGeekY Fitness: 6

Generation: 24 String: I loOe Geev"5orGeekY Fitness: 5

Generation: 25 String: I loOe Geev"5orGeekY Fitness: 5

Generation: 26 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 27 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 28 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 29 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 30 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 31 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 32 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 33 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 34 String: I goOe GeekJYorGeeks Fitness: 4

Generation: 35 String: I lo e GeekJQorGeeks Fitness: 3

Generation: 36 String: I lo e GeekJQorGeeks Fitness: 3

Generation: 37 String: I love Geek"farGeeks Fitness: 2

Generation: 38 String: I love Geek"farGeeks Fitness: 2

Generation: 39 String: I love Geek"farGeeks Fitness: 2

Generation: 40 String: I love Geek"farGeeks Fitness: 2

Generation: 41 String: I love Geek"farGeeks Fitness: 2

Generation: 42 String: I love Geek"farGeeks Fitness: 2

Generation: 43 String: I love Geek"farGeeks Fitness: 2

Generation: 44 String: I love Geek"farGeeks Fitness: 2

Generation: 45 String: I love Geek"farGeeks Fitness: 2

Generation: 46 String: I love Geek"farGeeks Fitness: 2

Generation: 47 String: I love GeekEforGeeks Fitness: 1

Generation: 48 String: I love GeekEforGeeks Fitness: 1

Generation: 49 String: I love GeekEforGeeks Fitness: 1

Generation: 50 String: I love GeekEforGeeks Fitness: 1

Generation: 51 String: I love GeekEforGeeks Fitness: 1

Generation: 52 String: I love GeekEforGeeks Fitness: 1

Generation: 53 String: I love GeekEforGeeks Fitness: 1

Generation: 54 String: I love GeekEforGeeks Fitness: 1

Generation: 55 String: I love GeekEforGeeks Fitness: 1

Generation: 56 String: I love GeekEforGeeks Fitness: 1

Generation: 57 String: I love GeekEforGeeks Fitness: 1

Generation: 58 String: I love GeekEforGeeks Fitness: 1

Generation: 59 String: I love GeekEforGeeks Fitness: 1

Generation: 60 String: I love GeekEforGeeks Fitness: 1

Generation: 61 String: I love GeekEforGeeks Fitness: 1

Generation: 62 String: I love GeekEforGeeks Fitness: 1

Generation: 63 String: I love GeekEforGeeks Fitness: 1

Generation: 64 String: I love GeeksforGeeks Fitness: 0

PS C:\Users\RAJARSI>

If you received a different output do not worry, no two genetic algorithm outputs are ever the same as the entire process runs on randomness. If your code takes longer to run, it’s nothing to worry about. The next run may be faster. But the ultimately the algorithm should end with the string “I love GeeksforGeeks”

Applications

• Bayesian inference links to particle methods in Bayesian statistics and hidden Markov chain models • Distributed computer network topologies • Electronic circuit design, known as evolvable hardware • Learning robot behavior using genetic algorithms • Image processing: Dense pixel matching • Learning fuzzy rule base using genetic algorithms • Game theory equilibrium resolution • And many more… [3]

Conclusion

So, in this article, what I have tried first explain what exactly Genetic Algorithms are, then I tried to explain the difference between classical algorithms and genetic algorithms. I focused a lot on the various important terminologies that are used in genetic algorithms and how they are analogous to real-life genetics and biological evolution. The three operations selection, crossover, and mutation are the basic foundation upon which the entire genetic algorithm is based. So, these are the most important operations.

Advantages and disadvantages are something that needs to be known before even thinking about applying GA into your problem. As you can see there are quite a few problems that can be solved using genetic algorithms but there many that cannot be or rather would take much more time in solving. So, therefore, I would prefer using GA where the population size of data is very large and thus the randomness would actually take effect. What about you?

References