# A Literature Survey on Offline Automatic Algorithm Configuration
## 6 Multi-Objective META-EAs
### Design of Experiment Based Methods  
Many optimization algorithms, such as metaheuristics or commercial solvers, like the CPLEX solver with 76 parameters [54], contain a large set of parameters. In these situations, decreasing parameter search space is needed to efficiently apply automated tuning procedures. One way of achieving this reduction is decomposing the parameters into disjoint partitions and configuring these parameters one by one. Design of experiments (DOE) [67] is another method for decomposing parameter space. Design of experiments techniques have been widely used to manually determine the best parameter setting of metaheuristics [68; 69; 70; 71; 72; 73; 74]. We will give a brief overview of the main studies that use the design of experiment-based techniques for the selection of parameter values of optimization algorithms. [PERSON] et al. [75] suggested a widely applicable and effective method for optimizing continuous parameters. Their approach consists of a full factorial design and a gradient descent method. They evaluated their approach to two new vehicle routing heuristics for solving different capacity and route-length constrained vehicle routing problems. The inconvenience of their methods is that it approximates the response surface linearly and does not investigate the relationship between instances, parameter vector, and algorithm performance. [PERSON] and [PERSON] [48] proposed the CALIBRA system that starts with [PERSON]'s fractional factorial design and then applies a local search procedure. In the beginning, CALIBRA tests each parameter vector with two levels of the full factorial design per parameter. Then, at each iteration of the applied local search procedure, it iteratively approximates regions of promising parameter configuration in the configuration space. The local search procedure evaluates nine configurations around the existing best-performing configuration. This process is terminated when local optimum criteria have been reached. A local search procedure can be executed more than one time if the computational budget remains to obtain a more local optimal parameter configuration. The experiment is executed on six existing heuristic-based methods. Although the results of the experiment demonstrated that CALIBRA is a useful tool for the algorithm configuration problem, there are some limitations, such as it can only handle five numeric and ordinal parameters, it does not guarantee optimality, and the local search procedure cannot operate the effects of interactions between parameters. [PERSON] and [PERSON] [3] proposed an approach for algorithm configuration based on a combination of design of experiments (DOE), signal to noise (S/N) ratio, Shannon entropy, and VIKOR methods. Their approach aims to optimize four goals: solution quality, several fitness function evaluations, minimizing the algorithm's runtime, and variance of these objectives. According to the results of their experiment, the suggested approach improved the average number of iterations, average running time, and algorithm's solution quality. [PERSON] [76] suggests a parameter tuning technique, which is a combination of design of experiments and artificial neural networks (ANN). Their methodology consists of four phases: problem description, training, parameter retrieval, and execution. The first phase includes a computer-readable representation of the process (problem, quality measure, and target algorithm). In the first part of the training phase, an automated design of experimental procedure is applied to a finite training set of problem instances for improving the initial values of the parameter. Later, the outcomes of the first part are used for training an artificial neural network in order to suggest near-optimal initial parameters for any given problem instance. In the third phase, ANN is used as a predictor to receive promising initial parameter sets for any given problem. As a final stage, the algorithm is executed with the suggested values from phase three.