Population-Based Metaheuristic Algorithms
Tags: Machine Learning, Data Science, Optimization, Computer Science, Metaphor

Population-Based Metaheuristic Algorithms

Sometimes “good enough” is good enough.

To optimize a problem means to find the best possible solution for the same, or to simply maximize or minimize a problem. Surely, optimizing complex algorithms aren’t easy — rather far from it, but inspired solutions have given rise to multiple interesting approaches for the same, metaheuristic being one such procedure.

Metaheuristic enables us to find optimized solutions to largely complicated problems by incorporating the following components:

Intensification and diversification Exploitation and exploration

An intricate balance between the two can ensure a convergence near the global optima, something classical non-heuristic optimization techniques struggle with, especially by getting stuck near the local optima.

Image from https://peachturnspro.wordpress.com/2016/04/19/stuck-in-a-local-optima/

However, this modern optimization technique should be taken with a pinch of salt; not all algorithms are backed by precise mathematics and robust literature. Below we will glance through some popular metaheuristic techniques and try to appreciate the novelty of the same.

Genetic Algorithm

Genetic Algorithm is a type of Evolutionary Algorithm that is inspired by Darwin’s theory of evolution and natural selection. Genetic Algorithms broadly have the following steps:

Initialization: The algorithm is initiated by a starting “population”, each member signifying the solution to the problem at hand. Selection: A fitness function is used to determine how “fit” and individual is. This threshold emulates “Survival of the Fittest” but selecting only those members for reproduction or crossover who are deemed fit. Crossover: A new population is created from the selected individuals by using different crossover points and mating techniques. Mutation: Some individuals are subjected to small mutations after crossover. These changed “genes” assert diversity, and sometimes ensure that convergence doesn’t get stuck at the local optima.

Image from https://www.researchgate.net/figure/Genetic-algorithm-schema_fig3_23457461

The above process goes on till a population which is highly similar to it’s previous one. This convergence is used as the solution of the given problem. Even for the naysayers and deniers of the fascinating phenomenon called Evolution, GA’s can provide “good enough” solutions for difficult problems. Some examples are NP-complete problems like Knapsack Problem, Bin Packing and Graph Colouring.

Ant Colony Optimization

Using the dangerous combination of pheromones and mathematics, the Ant Colony Optimization algorithms help achieve quite wonderful optimizations using artificial ants and pheromones to bias a stochastic process to set parameters as graph components. It is inspired by tiny workers, who leave pheromone trails along paths they find food at, eventually restricting ant jaywalking to the best path, the path of gluttony. The ACO algorithm can be broadly described using the Travelling Salesman problem as follows:

Every ant starts from a random vertex (or city) For every iteration of the problem, each ant visits a city along the edge of the graph and keeps the route in memory The ants avoid taking paths that lead to cities that have been already visited but travel to every city in the graph by probabilistically choosing the edge — depending on the amount of pheromone associated with the path. The higher the pheromone values associated to the path, the higher chance that an ant will choose it. However, for every iteration the pheromone value is decreased by a percentage A solution is generated for every ant upon completing its visit. Pheromones are additionally assigned routes depending on the quality of the solution.

Image from https://link.springer.com/article/10.1007/s40747-020-00138-3

These processes are repeated until solution converges or termination condition is satisfied. The ACO algorithms are used extensively in multiple domains, especially in the fields of vehicle routing, scheduling and set problems.

Artificial Bee Colony

Honeybees belong to an extremely intelligent swarm, whose compartmentalized hierarchy along with the incredible mechanism of waggle dance make their foraging methods very efficient. Apart from being talented in geography and dancing, bees are also responsible for 80% of the food we eat. Given the global climate crisis, bees are already endangered and its declining numbers have dire consequences. This can help you with some context, and what we can do to save nature’s polymath.

Image from https://www.researchgate.net/figure/Waggle-dance-of-honey-bees_fig2_256291628

The ABC algorithm deploys three types of bees:

Employed bees: They waggle dance about their food source. Once their food source is abandoned, they are transitioned as workers

Onlookers: They choose food sources based on employed bee’s dances

Scouts/Workers: They make initial discovery of food sources

In ABC, the number of food sources are represent the number of possible solutions to the problem, its position denotes a solution to the problem and the amount of nectar is the fitness function (quality of solution).Each employed bee is associated with one food source (solution).

Initialization Phase



REPEAT



Employed Bees Phase



Onlooker Bees Phase



Scout Bees Phase



Memorize the best solution achieved so far



UNTIL(Cycle=Maximum Cycle Number or a Maximum CPU time)

The above algorithm simply describes the steps for ABC. This optimization technique not only reiterate how awesome bees are, but is also applied in problems like the the neural network training for the XOR problem, structural optimization, etc.

Some Other Cool Algorithms

For brevity’s sake, here are a list of other population-based metaheuristic algorithms:

Particle Swarm Optimization

Simulated Annealing

Imperialist competitive algorithm (Genetic Algorithm’s big brother, Social Algorithm)

Cuttlefish optimization algorithm

For all it’s deficiencies, meta-heuristic approaches are pretty good. Without deep-diving into the mathematics of the same, this article was aimed at reminding ourselves how metaphorical and inspired solution to tough problems can be novel. Also how sometimes good enough is, good enough.