Genetic Algorithm — explained step by step with example | by Niranjan Pramanik, Ph.D. | Towards Data Science

In this article, I am going to explain how genetic algorithm (GA) works by solving a very simple optimization problem. The idea of this note is to understand the concept of the algorithm by solving an optimization problem step by step.
Let us estimate the optimal values of a and b using GA which satisfy below expression.
Any optimization problem starts with an objective function. Above equation can be written as:
It is understood that the value of the function is 0. This function is our objective function and the aim is to estimate values of a and b such that the value of the objective function gets minimised to zero. The entire optimization process is explained below in four major steps and coded in R for one iteration (or generation).
This step starts with guessing of initial sets of a and b values which may or may not include the optimal values. These sets of values are called as ‘chromosomes’ and the step is called ‘initialize population’. Here population means sets of a and b [a,b]. Random uniform function is used to generate initial values of a and b. In this optimization problem, six sets of a and b values are generated between 1 and 10. The R code to generate initial chromosomes is shown below.
intial_popu <- NULLx <- 1repeat {  crm <- runif(2,1,10)  crm <- as.integer(crm)  intial_popu <- rbind(intial_popu,crm)  x = x+1  if (x == 7){    break  }}rownames(intial_popu) <- c('Cromosome1','Cromosome2','Cromosome3','Cromosome4','Cromosome5','Cromosome6')print(intial_popu)##            [,1] [,2]## Cromosome1    5    9## Cromosome2    9    7## Cromosome3    9    9## Cromosome4    3    3## Cromosome5    7    8## Cromosome6    8    4
In this step, the value of the objective function for each chromosome is computed. The value of the objective function is also called fitness value. This step is very important and is called ‘selection’ because fittest chromosomes are selected from the population for subsequent operations.
Based on the fitness values, more suitable chromosomes who have possibilities of producing low values of fitness function (because the value of our objective function needs to be 0) are selected and allowed to survive in succeeding generations. Some chromosomes are discarded to be unsuitable to produce low fitness values.
One of the most widely used selection methods in GA is ‘roulette wheel method’. Roulette wheel method is discussed in detail below.
Roulette wheel is a pie plot where the value of each pie is expressed in terms of fitness probability. Note that fitness value and fitness probability are two different terms. In this optimization problem, chromosome which produces low fitness value has high fitness probability. Fitness probability of each chromosome is computed from fitness values. Chromosome having high fitness probability will have higher chance of getting selected. Following expression is used to calculate fitness probability of a single chromosome.
Where, FP = fitness probability of ith chromosome, Fi = fitness value of ith chromosome
In this figure, fitness probabilities are shown for six different chromosomes as computed using the above expression. The sum of all fitness probabilities must be always equal to 1.
To select the fittest chromosomes, six random probabilities (i.e to generate six values between 0 and 1) are generated. As an example, let’s say the generated six probabilities are:
Pr 01 = 0.02 for Chromosome 1Pr 02 = 0.13 for Chromosome 2Pr 03 = 0.40 for Chromosome 3Pr 04 = 0.60 for Chromosome 4Pr 05 = 0.85 for Chromosome 5Pr 06 = 0.96 for Chromosome 6
Selection is done based on the position of above probability values on roulette wheel expressed in a scale of cumulative fitness probabilities. The segments on which above values lie are selected for next step. Please remember each segment represents its corresponding chromosome. For better understanding, roulette wheel is straightened and positions of generated probability values are shown.
In this illustration, chromosomes 1, 2,5,6 are selected out 6 chromosomes. Sequence of chromosomes follows from left to right with blue being chromosome -1 and orange being chromosome -6. These chromosomes will be used to perform crossover operation in next step. Finally, the new set of chromosomes are:
New chromosome-1 = Chromosome -1New chromosome-2 = Chromosome -1New chromosome-3 = Chromosome -2New chromosome-4 = Chromosome -2New chromosome-5 = Chromosome -5New chromosome-6 = Chromosome -6
Above new chromosomes are potential parents for crossover operation. It is observed that chromosomes -3 and -4 have been discarded to be unfit.
For this optimization problem, a piece of R-code is presented below to select fittest chromosomes (new population).
## Function to compute fitnessfit <- function(A){  a <- A[1]  b <- A[2]  return(((2*a^2 + b) - 57))}fitness <- apply(intial_popu, 1, FUN = 'fit')fitting <- 1/(fitness)probfitting <- fitting/sum(fitting)#generate random values of fitness prob. from 0 to 1prob_gen <- runif(6,0,1)newpopulation <- NULLfor(rr in prob_gen){  sum <- 0  for(prob in probfitting){    sum <- sum + prob    if(rr < sum){      bin <- prob      cromosomeS <- which(probfitting == bin, arr.ind = T)      if(length(cromosomeS)>1){        cromosomeS <- cromosomeS[1]      } else{        cromosomeS <- cromosomeS      }      cromname <- paste0('Cromosome',cromosomeS[1])      newcromosome <- intial_popu[which(rownames(intial_popu) == cromname),]      break    }  }  newpopulation <- rbind(newpopulation,newcromosome,deparse.level = 2)}rownames(newpopulation) <- rownames(intial_popu) print(newpopulation)##            [,1] [,2]## Cromosome1    5    9## Cromosome2    5    9## Cromosome3    9    7## Cromosome4    5    9## Cromosome5    5    9## Cromosome6    5    9
This step is called ‘crossover’. In this step, chromosomes are expressed in terms of genes. This can be done by converting the values of a and b into binary strings which means the values need to be expressed in terms of 0 or 1. As for example, the binary form of 9 is [1001].
Crossover is ‘the change of a single (0 or 1) or a group of genes (e.g. [1,0,1])’ occurred because of mating between two parent chromosomes. The new chromosome produced after crossover operation is called ‘offspring’. Following illustration explains crossover process. Always remember that crossover happens between parent chromosomes.
In our present optimization problem, chromosomes obtained from step 2 is written in binary terms. For example, if the binary representation of a = [1,0,0,1] and b = [1,1,1,0] then the chromosome, [a,b] is expressed as [1,0,0,1,1,1,1,0]. All chromosomes are converted into binary and written as matrix form with 6 rows and 8 columns.
One important parameter comes into play here which is called crossover parameter. This parameter value lies between 0 and 1. A value of 0.5 indicates, out of six chromosomes, three chromosomes are allowed to crossover to produce three offsprings.
Then comes the selection of positions. Genes will be subjected to crossover at selected positions in the chromosome. For example, if the chromosome is [1,1,0,1,1,0,0,1] and position is 2 (from left). Then the value 1 will be replaced with mating chromosome’s value at the same position. Remember that multiple positions or segments can be chosen for crossover.
Block of R-code which include a function to convert integer into binary strings and crossover operation is presented below.
##Function to convert integer to binarybinary <- function(x) {  i <- 0  string <- numeric(32)  while(x > 0) {    string[32 - i] <- x %% 2    x <- x %/% 2    i <- i + 1   }  first <- match(1, string)  string[first:32] }##create binary matrix of 8 cols and 6 rowsbinary_popu <- matrix(NA,nrow = 6,ncol = 8)for(i in 1:nrow(newpopulation)){ x <- binary(newpopulation[i,1])binary_popu[i,1:length(x)] <- xy <- binary(newpopulation[i,2])binary_popu[i,5:(4+length(y))] <- y  }rownames(binary_popu) <- rownames(newpopulation)cross_paramter <- 0.5i = 1crom_cross <- NULLwhile(length(crom_cross) < 3){  cross.crossover.prob <- runif(6,0,1)  crom_cross <- which(cross.crossover.prob < cross_paramter,arr.ind = T)  i = i + 1}parents <- binary_popu[crom_cross,]position = 2 ##crossover positioncross_parent <- parentscross_parent[1,1:position] <- parents[2,1:position]cross_parent[2,1:position] <- parents[3,1:position]cross_parent[3,1:position] <- parents[1,1:position]listofindex <- which(row.names(binary_popu) %in% row.names(cross_parent))offSpring <- binary_popuoffSpring[listofindex,] <- cross_parentprint(offSpring) ##offspring after crossover##            [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]## Cromosome1    1    0    1   NA    1    0    0    1## Cromosome2    1    0    1   NA    1    0    0    1## Cromosome3    1    0    0    1    1    1    1   NA## Cromosome4    1    0    1   NA    1    0    0    1## Cromosome5    1    0    1   NA    1    0    0    1## Cromosome6    1    0    1   NA    1    0    0    1
This step is called ‘mutation’. Mutation is the process of altering the value of gene i.e to replace the value 1 with 0 and vice-versa. For example, if offspring chromosome is [1,0,0,1], after mutation it becomes [1,1,0,1]. Here, 2nd value of the offspring chromosome is decided to get mutated. It has got changed to 1 from 0.
Question arises now, how many genes need to be altered and at what positions. The mutation parameter decides how many genes to be mutated. If mutation parameter is 0.1 (usually kept low values). Then 0.1 times the total genes are allowed to mutate. In the present optimization problem, total number of genes is 48 (6 x 8). Therefore, 4.8 ~ 5 genes are allowed for mutation.
Regarding mutation positions, 5 random values of rows and columns are chosen. The illustration of mutation process is shown below.
After mutation, binary chromosomes are converted into integer form and fitness values are calculated. If any one of the chromosomes produces target fitness value equal to 0, we stop there. Otherwise, the process will be repeated from step — 2 to step — 4 by equating mutated chromosomes with new population.
Block of R-code for this step is presented below.
mutation_paramter <- 0.09no.of.mutations <- 4  ## Calculated as nrow(offSpring)*ncol(offSpring)*mutation_paramterrandrow <- round(runif(no.of.mutations,1,nrow(offSpring)))rancol <-  round(runif(no.of.mutations,1,ncol(offSpring)))## Now get the offsprings by mutating at above positionfor(r in 1:length(randrow)){  if(is.na(offSpring[randrow[r],rancol[r]])){    offSpring[randrow[r],rancol[r]] <- NA  }else{    if(offSpring[randrow[r],rancol[r]] == 0){      offSpring[randrow[r],rancol[r]] <- 1    }else{      offSpring[randrow[r],rancol[r]] <- 0    }  }}print(offSpring) ## Chromosomes after mutation##            [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]## Cromosome1    1    0    1   NA    1    0    0    1## Cromosome2    1    0    1   NA    1    0    0    1## Cromosome3    1    0    0    1    1    1    0   NA## Cromosome4    1    0    1   NA    0    0    0    1## Cromosome5    1    0    1   NA    1    0    0    1## Cromosome6    1    1    1   NA    1    0    0    1## Now convert binary back to integerbinary_decimal = function(base_number, base = 2) {  split_base = strsplit(as.character(base_number), split = "")  return(sapply(split_base, function(x) sum(as.numeric(x) * base^(rev(seq_along(x) - 1)))))}offSpring_inter <- matrix(NA,6,2)for(i in 1:nrow(offSpring)){  a <- offSpring[i,1:4]  a <- na.omit(a)  a <- as.numeric(paste(a, collapse = ""))  a <- binary_decimal(a)  b <- offSpring[i,5:8]  b <- na.omit(b)  b <- as.numeric(paste(b, collapse = ""))  b <- binary_decimal(b)  offSpring_inter[i,1] <- a  offSpring_inter[i,2] <- b}rownames(offSpring_inter) <- rownames(offSpring)## Chromosomes converted back to integer after end of 1st of generationprint(offSpring_inter)##            [,1] [,2]## Cromosome1    5    9## Cromosome2    5    9## Cromosome3    9    6## Cromosome4    5    1## Cromosome5    5    9## Cromosome6    7    9
The final result of this optimization problem is shown below where the fitness values are computed for 10 generations. Chromosome 6 i.e [a,b] = [5,7] is the optimal solution which yielded fitness value equals 0
## Iteration:  1 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##        -30         76        -17        -18         -3        -45 ## Iteration:  2 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3         45        -40        -42        -30         -3 ## Iteration:  3 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3         -3         -3         -3         -7         -3 ## Iteration:  4 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3         -2         -2         -3         -3         -3 ## Iteration:  6 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3         -3         -3         -3         -2         -3 ## Iteration:  7 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3        -20         -3         -3         -3         -2 ## Iteration:  8 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3         -2         -3         -2         -2         -2 ## Iteration:  9 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         46         -2         -2        -20         -2         -2 ## Iteration:  10 ## Cromosome1 Cromosome2 Cromosome3 Cromosome4 Cromosome5 Cromosome6 ##         -3        -19         -1         47         -1          0## The final set of Chromosomes##            [,1] [,2]## Cromosome1    5    4## Cromosome2    4    6## Cromosome3    5    6## Cromosome4    7    6## Cromosome5    5    6## Cromosome6    5    7
Deb, K. Multi-objective optimization using evolutionary algorithms (2005)