Tutorial on Genetic Algorithms

Research on genetic algorithms (GAs) has shown that the initial proposals are incapable of solving hard problems in a robust and efficient way. Usually, for large-scale optimization problems, the execution time of first generation GAs dramatically increases while solution quality decreases. The focus of thistutorial is pointing out the main design issues in tailoring GAs to large-scale optimization problems. Important topics such as encoding schemes, selection procedures, self-adaptive and knowledge-based operators are discussed. Index Terms Genetic algorithms, large-scale optimization. 1. MODERN HEURISTIC SEARCH TECHNIQUES Optimization is the basic concept behind the application of genetic algorithms (GAs), or any other evolutionary algorithm [1]-[3], to any field of interest. Over and above the problems in which optimization itself is the final goal, it is also a way for (or the main idea behind) modeling, forecasting, control, simulation, etc. Traditional optimization techniques begin with a single candidate and iteratively search for the optimal solution by applying static heuristics. On the other hand, the GA approach uses a population of candidates to search several areas of a solution space, simultaneously and adaptively. Evolutionary computation allows precise modeling of the optimization problem, although not usually providing mathematically optimal solutions. Another advantage of using evolutionary computation techniques is that there is no need for having an explicit objective function. Moreover, when the objective function is available, it does not have to be differentiable. Genetic algorithms have been most commonly applied to solve combinatorial optimization problems. Combinatorial optimization usually involves a huge number of possible solutions, which makes the use of enumeration techniques (e.g., cutting plane, branch and bound, or dynamic programming) hopeless. In large-scale combinatorial optimization problems, the number of possible solutions grow exponentially with the problem size. Therefore, the application of optimization methods to find the optimal solution is computationally impracticable. Heuristic search techniques are frequently employed in this case for achieving high quality solutions within reasonable run time. Among the heuristic search methods there are the ones that apply local search (e.g., hill climbing) and the ones that use a non-convex optimization approach, in which cost-deteriorating neighbors are accepted also. The most popular methods which go beyond simple local search are GAs [4]-[7] (and other evolutionary techniques, like evolutionary programming, evolutionary strategies, etc.), simulated annealing (SA) [8], and tabu search (TS) [9]. Particle swarm [10] is another optimization technique that has shown great potential, lately. However, more experience is still necessary to prove its efficiency and robustness. Simulated annealing uses a probability function that allows a move to a worse solution with a decreasing probability, as the search progresses. With GAs, a pool of solutions isused and the neighborhood function is extended to act on pairs of solutions. Tabu search uses a deterministic rather than stochastic search. Tabu search is based on neighborhood search with local optima avoidance. In order to avoid cycling, a short-term adaptive memory is used in TS. Genetic algorithms have a basic distinction when compared with other methods based on stochastic search. They can use coding (genotypic space) for representing the problem. The other methods solve the optimization problem in the original representation space (phenotypic). The most rigorous global search methods have asymptotic convergence proof (also known as convergence in probability), i.e., the optimal solution is guaranteed to be found if infinite time is available. Among SA, GA and TS algorithms, simulated annealing and genetic algorithms are the only ones with proof of convergence. However, there is no such proof for the canonical GA [11], i.e., the one with proportional selection (Section 5.1.6) and crossover/mutation with constant probabilities (Section 5.4).